diff --git a/CHANGELOG.md b/CHANGELOG.md index d0d25908cd..f0a9a9651d 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -1,8 +1,76 @@ # Changelog -## [3.11.0-nightly.2](https://github.com/pypeclub/OpenPype/tree/HEAD) +## [3.11.2-nightly.1](https://github.com/pypeclub/OpenPype/tree/HEAD) -[Full Changelog](https://github.com/pypeclub/OpenPype/compare/3.10.0...HEAD) +[Full Changelog](https://github.com/pypeclub/OpenPype/compare/3.11.1...HEAD) + +### 📖 Documentation + +- General: Add ability to change user value for templates [\#3366](https://github.com/pypeclub/OpenPype/pull/3366) +- Feature/multiverse [\#3350](https://github.com/pypeclub/OpenPype/pull/3350) + +**🚀 Enhancements** + +- Hosts: More options for in-host callbacks [\#3357](https://github.com/pypeclub/OpenPype/pull/3357) +- TVPaint: Extractor use mark in/out range to render [\#3308](https://github.com/pypeclub/OpenPype/pull/3308) +- Maya: Allow more data to be published along camera 🎥 [\#3304](https://github.com/pypeclub/OpenPype/pull/3304) + +**🐛 Bug fixes** + +- TVPaint: Make sure exit code is set to not None [\#3382](https://github.com/pypeclub/OpenPype/pull/3382) +- Maya: vray device aspect ratio fix [\#3381](https://github.com/pypeclub/OpenPype/pull/3381) +- Harmony: added unc path to zifile command in Harmony [\#3372](https://github.com/pypeclub/OpenPype/pull/3372) + +**🔀 Refactored code** + +- Harmony: Use client query functions [\#3378](https://github.com/pypeclub/OpenPype/pull/3378) +- Photoshop: Use client query functions [\#3375](https://github.com/pypeclub/OpenPype/pull/3375) +- AfterEffects: Use client query functions [\#3374](https://github.com/pypeclub/OpenPype/pull/3374) +- TVPaint: Use client query functions [\#3340](https://github.com/pypeclub/OpenPype/pull/3340) +- Ftrack: Use client query functions [\#3339](https://github.com/pypeclub/OpenPype/pull/3339) +- Standalone Publisher: Use client query functions [\#3330](https://github.com/pypeclub/OpenPype/pull/3330) + +**Merged pull requests:** + +- Maya - added support for single frame playblast review [\#3369](https://github.com/pypeclub/OpenPype/pull/3369) + +## [3.11.1](https://github.com/pypeclub/OpenPype/tree/3.11.1) (2022-06-20) + +[Full Changelog](https://github.com/pypeclub/OpenPype/compare/CI/3.11.1-nightly.1...3.11.1) + +**🆕 New features** + +- Flame: custom export temp folder [\#3346](https://github.com/pypeclub/OpenPype/pull/3346) +- Nuke: removing third-party plugins [\#3344](https://github.com/pypeclub/OpenPype/pull/3344) + +**🚀 Enhancements** + +- Pyblish Pype: Hiding/Close issues [\#3367](https://github.com/pypeclub/OpenPype/pull/3367) +- Ftrack: Removed requirement of pypeclub role from default settings [\#3354](https://github.com/pypeclub/OpenPype/pull/3354) +- Kitsu: Prevent crash on missing frames information [\#3352](https://github.com/pypeclub/OpenPype/pull/3352) +- Ftrack: Open browser from tray [\#3320](https://github.com/pypeclub/OpenPype/pull/3320) +- Enhancement: More control over thumbnail processing. [\#3259](https://github.com/pypeclub/OpenPype/pull/3259) + +**🐛 Bug fixes** + +- Nuke: bake streams with slate on farm [\#3368](https://github.com/pypeclub/OpenPype/pull/3368) +- Harmony: audio validator has wrong logic [\#3364](https://github.com/pypeclub/OpenPype/pull/3364) +- Nuke: Fix missing variable in extract thumbnail [\#3363](https://github.com/pypeclub/OpenPype/pull/3363) +- Nuke: Fix precollect writes [\#3361](https://github.com/pypeclub/OpenPype/pull/3361) +- AE- fix validate\_scene\_settings and renderLocal [\#3358](https://github.com/pypeclub/OpenPype/pull/3358) +- deadline: fixing misidentification of revieables [\#3356](https://github.com/pypeclub/OpenPype/pull/3356) +- General: Create only one thumbnail per instance [\#3351](https://github.com/pypeclub/OpenPype/pull/3351) +- nuke: adding extract thumbnail settings 3.10 [\#3347](https://github.com/pypeclub/OpenPype/pull/3347) +- General: Fix last version function [\#3345](https://github.com/pypeclub/OpenPype/pull/3345) +- Deadline: added OPENPYPE\_MONGO to filter [\#3336](https://github.com/pypeclub/OpenPype/pull/3336) + +**🔀 Refactored code** + +- Webpublisher: Use client query functions [\#3333](https://github.com/pypeclub/OpenPype/pull/3333) + +## [3.11.0](https://github.com/pypeclub/OpenPype/tree/3.11.0) (2022-06-17) + +[Full Changelog](https://github.com/pypeclub/OpenPype/compare/CI/3.11.0-nightly.4...3.11.0) ### 📖 Documentation @@ -11,7 +79,9 @@ **🚀 Enhancements** +- Settings: Settings can be extracted from UI [\#3323](https://github.com/pypeclub/OpenPype/pull/3323) - updated poetry installation source [\#3316](https://github.com/pypeclub/OpenPype/pull/3316) +- Ftrack: Action to easily create daily review session [\#3310](https://github.com/pypeclub/OpenPype/pull/3310) - TVPaint: Extractor use mark in/out range to render [\#3309](https://github.com/pypeclub/OpenPype/pull/3309) - Ftrack: Delivery action can work on ReviewSessions [\#3307](https://github.com/pypeclub/OpenPype/pull/3307) - Maya: Look assigner UI improvements [\#3298](https://github.com/pypeclub/OpenPype/pull/3298) @@ -22,13 +92,13 @@ - Maya: reference loaders could store placeholder in referenced url [\#3264](https://github.com/pypeclub/OpenPype/pull/3264) - TVPaint: Init file for TVPaint worker also handle guideline images [\#3250](https://github.com/pypeclub/OpenPype/pull/3250) - Nuke: Change default icon path in settings [\#3247](https://github.com/pypeclub/OpenPype/pull/3247) -- Maya: publishing of animation and pointcache on a farm [\#3225](https://github.com/pypeclub/OpenPype/pull/3225) -- Maya: Look assigner UI improvements [\#3208](https://github.com/pypeclub/OpenPype/pull/3208) -- Nuke: add pointcache and animation to loader [\#3186](https://github.com/pypeclub/OpenPype/pull/3186) **🐛 Bug fixes** - Houdini: Fix Houdini VDB manage update wrong file attribute name [\#3322](https://github.com/pypeclub/OpenPype/pull/3322) +- Nuke: anatomy compatibility issue hacks [\#3321](https://github.com/pypeclub/OpenPype/pull/3321) +- hiero: otio p3 compatibility issue - metadata on effect use update 3.11 [\#3314](https://github.com/pypeclub/OpenPype/pull/3314) +- Nuke: fixing farm publishing if review is disabled [\#3306](https://github.com/pypeclub/OpenPype/pull/3306) - General: Vendorized modules for Python 2 and update poetry lock [\#3305](https://github.com/pypeclub/OpenPype/pull/3305) - Fix - added local targets to install host [\#3303](https://github.com/pypeclub/OpenPype/pull/3303) - Settings: Add missing default settings for nuke gizmo [\#3301](https://github.com/pypeclub/OpenPype/pull/3301) @@ -42,14 +112,16 @@ - Nuke: bake reformat was failing on string type [\#3261](https://github.com/pypeclub/OpenPype/pull/3261) - Maya: hotfix Pxr multitexture in looks [\#3260](https://github.com/pypeclub/OpenPype/pull/3260) - Unreal: Fix Camera Loading if Layout is missing [\#3255](https://github.com/pypeclub/OpenPype/pull/3255) -- Unreal: Fixed Animation loading in UE5 [\#3240](https://github.com/pypeclub/OpenPype/pull/3240) -- Unreal: Fixed Render creation in UE5 [\#3239](https://github.com/pypeclub/OpenPype/pull/3239) -- Unreal: Fixed Camera loading in UE5 [\#3238](https://github.com/pypeclub/OpenPype/pull/3238) -- Flame: debugging [\#3224](https://github.com/pypeclub/OpenPype/pull/3224) + +**🔀 Refactored code** + +- Blender: Use client query functions [\#3331](https://github.com/pypeclub/OpenPype/pull/3331) +- General: Define query functions [\#3288](https://github.com/pypeclub/OpenPype/pull/3288) **Merged pull requests:** - Maya: add pointcache family to gpu cache loader [\#3318](https://github.com/pypeclub/OpenPype/pull/3318) +- Maya look: skip empty file attributes [\#3274](https://github.com/pypeclub/OpenPype/pull/3274) ## [3.10.0](https://github.com/pypeclub/OpenPype/tree/3.10.0) (2022-05-26) @@ -59,58 +131,21 @@ - Maya: FBX camera export [\#3253](https://github.com/pypeclub/OpenPype/pull/3253) - General: updating common vendor `scriptmenu` to 1.5.2 [\#3246](https://github.com/pypeclub/OpenPype/pull/3246) -- Project Manager: Allow to paste Tasks into multiple assets at the same time [\#3226](https://github.com/pypeclub/OpenPype/pull/3226) -- Project manager: Sped up project load [\#3216](https://github.com/pypeclub/OpenPype/pull/3216) -- Loader UI: Speed issues of loader with sync server [\#3199](https://github.com/pypeclub/OpenPype/pull/3199) **🐛 Bug fixes** - nuke: use framerange issue [\#3254](https://github.com/pypeclub/OpenPype/pull/3254) - Ftrack: Chunk sizes for queries has minimal condition [\#3244](https://github.com/pypeclub/OpenPype/pull/3244) -- Maya: renderman displays needs to be filtered [\#3242](https://github.com/pypeclub/OpenPype/pull/3242) -- Ftrack: Validate that the user exists on ftrack [\#3237](https://github.com/pypeclub/OpenPype/pull/3237) -- Maya: Fix support for multiple resolutions [\#3236](https://github.com/pypeclub/OpenPype/pull/3236) -- TVPaint: Look for more groups than 12 [\#3228](https://github.com/pypeclub/OpenPype/pull/3228) -- Hiero: debugging frame range and other 3.10 [\#3222](https://github.com/pypeclub/OpenPype/pull/3222) -- Project Manager: Fix persistent editors on project change [\#3218](https://github.com/pypeclub/OpenPype/pull/3218) -- Deadline: instance data overwrite fix [\#3214](https://github.com/pypeclub/OpenPype/pull/3214) -- Ftrack: Push hierarchical attributes action works [\#3210](https://github.com/pypeclub/OpenPype/pull/3210) -- Standalone Publisher: Always create new representation for thumbnail [\#3203](https://github.com/pypeclub/OpenPype/pull/3203) -- Photoshop: skip collector when automatic testing [\#3202](https://github.com/pypeclub/OpenPype/pull/3202) -- Nuke: render/workfile version sync doesn't work on farm [\#3185](https://github.com/pypeclub/OpenPype/pull/3185) -- Ftrack: Review image only if there are no mp4 reviews [\#3183](https://github.com/pypeclub/OpenPype/pull/3183) - -**🔀 Refactored code** - -- Avalon repo removed from Jobs workflow [\#3193](https://github.com/pypeclub/OpenPype/pull/3193) **Merged pull requests:** - Harmony: message length in 21.1 [\#3257](https://github.com/pypeclub/OpenPype/pull/3257) - Harmony: 21.1 fix [\#3249](https://github.com/pypeclub/OpenPype/pull/3249) -- Maya: added jpg to filter for Image Plane Loader [\#3223](https://github.com/pypeclub/OpenPype/pull/3223) ## [3.9.8](https://github.com/pypeclub/OpenPype/tree/3.9.8) (2022-05-19) [Full Changelog](https://github.com/pypeclub/OpenPype/compare/3.9.7...3.9.8) -**🚀 Enhancements** - -- nuke: generate publishing nodes inside render group node [\#3206](https://github.com/pypeclub/OpenPype/pull/3206) -- Loader UI: Speed issues of loader with sync server [\#3200](https://github.com/pypeclub/OpenPype/pull/3200) -- Backport of fix for attaching renders to subsets [\#3195](https://github.com/pypeclub/OpenPype/pull/3195) -- Looks: add basic support for Renderman [\#3190](https://github.com/pypeclub/OpenPype/pull/3190) - -**🐛 Bug fixes** - -- Standalone Publisher: Always create new representation for thumbnail [\#3204](https://github.com/pypeclub/OpenPype/pull/3204) -- Nuke: render/workfile version sync doesn't work on farm [\#3184](https://github.com/pypeclub/OpenPype/pull/3184) -- Ftrack: Review image only if there are no mp4 reviews [\#3182](https://github.com/pypeclub/OpenPype/pull/3182) - -**Merged pull requests:** - -- hiero: otio p3 compatibility issue - metadata on effect use update [\#3194](https://github.com/pypeclub/OpenPype/pull/3194) - ## [3.9.7](https://github.com/pypeclub/OpenPype/tree/3.9.7) (2022-05-11) [Full Changelog](https://github.com/pypeclub/OpenPype/compare/3.9.6...3.9.7) diff --git a/openpype/api.py b/openpype/api.py index e049a683c6..9ce745b653 100644 --- a/openpype/api.py +++ b/openpype/api.py @@ -44,7 +44,6 @@ from . import resources from .plugin import ( Extractor, - Integrator, ValidatePipelineOrder, ValidateContentsOrder, @@ -87,7 +86,6 @@ __all__ = [ # plugin classes "Extractor", - "Integrator", # ordering "ValidatePipelineOrder", "ValidateContentsOrder", diff --git a/openpype/client/__init__.py b/openpype/client/__init__.py new file mode 100644 index 0000000000..e3b4ef5132 --- /dev/null +++ b/openpype/client/__init__.py @@ -0,0 +1,73 @@ +from .entities import ( + get_projects, + get_project, + + get_asset_by_id, + get_asset_by_name, + get_assets, + get_archived_assets, + get_asset_ids_with_subsets, + + get_subset_by_id, + get_subset_by_name, + get_subsets, + get_subset_families, + + get_version_by_id, + get_version_by_name, + get_versions, + get_hero_version_by_id, + get_hero_version_by_subset_id, + get_hero_versions, + get_last_versions, + get_last_version_by_subset_id, + get_last_version_by_subset_name, + get_output_link_versions, + + get_representation_by_id, + get_representation_by_name, + get_representations, + get_representation_parents, + get_representations_parents, + + get_thumbnail, + get_thumbnails, + get_thumbnail_id_from_source, +) + +__all__ = ( + "get_projects", + "get_project", + + "get_asset_by_id", + "get_asset_by_name", + "get_assets", + "get_archived_assets", + "get_asset_ids_with_subsets", + + "get_subset_by_id", + "get_subset_by_name", + "get_subsets", + "get_subset_families", + + "get_version_by_id", + "get_version_by_name", + "get_versions", + "get_hero_version_by_id", + "get_hero_version_by_subset_id", + "get_hero_versions", + "get_last_versions", + "get_last_version_by_subset_id", + "get_last_version_by_subset_name", + "get_output_link_versions", + + "get_representation_by_id", + "get_representation_by_name", + "get_representations", + "get_representation_parents", + "get_representations_parents", + + "get_thumbnail", + "get_thumbnails", + "get_thumbnail_id_from_source", +) diff --git a/openpype/client/entities.py b/openpype/client/entities.py new file mode 100644 index 0000000000..4b4a3729fe --- /dev/null +++ b/openpype/client/entities.py @@ -0,0 +1,1758 @@ +"""Unclear if these will have public functions like these. + +Goal is that most of functions here are called on (or with) an object +that has project name as a context (e.g. on 'ProjectEntity'?). + ++ We will need more specific functions doing wery specific queires really fast. +""" + +import os +import collections + +import six +from bson.objectid import ObjectId + +from openpype.lib.mongo import OpenPypeMongoConnection + + +def _get_project_connection(project_name=None): + db_name = os.environ.get("AVALON_DB") or "avalon" + mongodb = OpenPypeMongoConnection.get_mongo_client()[db_name] + if project_name: + return mongodb[project_name] + return mongodb + + +def _prepare_fields(fields, required_fields=None): + if not fields: + return None + + output = { + field: True + for field in fields + } + if "_id" not in output: + output["_id"] = True + + if required_fields: + for key in required_fields: + output[key] = True + return output + + +def _convert_id(in_id): + if isinstance(in_id, six.string_types): + return ObjectId(in_id) + return in_id + + +def _convert_ids(in_ids): + _output = set() + for in_id in in_ids: + if in_id is not None: + _output.add(_convert_id(in_id)) + return list(_output) + + +def get_projects(active=True, inactive=False, fields=None): + mongodb = _get_project_connection() + for project_name in mongodb.collection_names(): + if project_name in ("system.indexes",): + continue + project_doc = get_project( + project_name, active=active, inactive=inactive, fields=fields + ) + if project_doc is not None: + yield project_doc + + +def get_project(project_name, active=True, inactive=False, fields=None): + # Skip if both are disabled + if not active and not inactive: + return None + + query_filter = {"type": "project"} + # Keep query untouched if both should be available + if active and inactive: + pass + + # Add filter to keep only active + elif active: + query_filter["$or"] = [ + {"data.active": {"$exists": False}}, + {"data.active": True}, + ] + + # Add filter to keep only inactive + elif inactive: + query_filter["$or"] = [ + {"data.active": {"$exists": False}}, + {"data.active": False}, + ] + + conn = _get_project_connection(project_name) + return conn.find_one(query_filter, _prepare_fields(fields)) + + +def get_asset_by_id(project_name, asset_id, fields=None): + """Receive asset data by it's id. + + Args: + project_name (str): Name of project where to look for queried entities. + asset_id (str|ObjectId): Asset's id. + fields (list[str]): Fields that should be returned. All fields are + returned if 'None' is passed. + + Returns: + dict: Asset entity data. + None: Asset was not found by id. + """ + + asset_id = _convert_id(asset_id) + if not asset_id: + return None + + query_filter = {"type": "asset", "_id": asset_id} + conn = _get_project_connection(project_name) + return conn.find_one(query_filter, _prepare_fields(fields)) + + +def get_asset_by_name(project_name, asset_name, fields=None): + """Receive asset data by it's name. + + Args: + project_name (str): Name of project where to look for queried entities. + asset_name (str): Asset's name. + fields (list[str]): Fields that should be returned. All fields are + returned if 'None' is passed. + + Returns: + dict: Asset entity data. + None: Asset was not found by name. + """ + + if not asset_name: + return None + + query_filter = {"type": "asset", "name": asset_name} + conn = _get_project_connection(project_name) + return conn.find_one(query_filter, _prepare_fields(fields)) + + +# NOTE this could be just public function? +# - any better variable name instead of 'standard'? +# - same approach can be used for rest of types +def _get_assets( + project_name, + asset_ids=None, + asset_names=None, + standard=True, + archived=False, + fields=None +): + """Assets for specified project by passed filters. + + Passed filters (ids and names) are always combined so all conditions must + match. + + To receive all assets from project just keep filters empty. + + Args: + project_name (str): Name of project where to look for queried entities. + asset_ids (list[str|ObjectId]): Asset ids that should be found. + asset_names (list[str]): Name assets that should be found. + standard (bool): Query standart assets (type 'asset'). + archived (bool): Query archived assets (type 'archived_asset'). + fields (list[str]): Fields that should be returned. All fields are + returned if 'None' is passed. + + Returns: + Cursor: Query cursor as iterable which returns asset documents matching + passed filters. + """ + + asset_types = [] + if standard: + asset_types.append("asset") + if archived: + asset_types.append("archived_asset") + + if not asset_types: + return [] + + if len(asset_types) == 1: + query_filter = {"type": asset_types[0]} + else: + query_filter = {"type": {"$in": asset_types}} + + if asset_ids is not None: + asset_ids = _convert_ids(asset_ids) + if not asset_ids: + return [] + query_filter["_id"] = {"$in": asset_ids} + + if asset_names is not None: + if not asset_names: + return [] + query_filter["name"] = {"$in": list(asset_names)} + + conn = _get_project_connection(project_name) + + return conn.find(query_filter, _prepare_fields(fields)) + + +def get_assets( + project_name, + asset_ids=None, + asset_names=None, + archived=False, + fields=None +): + """Assets for specified project by passed filters. + + Passed filters (ids and names) are always combined so all conditions must + match. + + To receive all assets from project just keep filters empty. + + Args: + project_name (str): Name of project where to look for queried entities. + asset_ids (list[str|ObjectId]): Asset ids that should be found. + asset_names (list[str]): Name assets that should be found. + archived (bool): Add also archived assets. + fields (list[str]): Fields that should be returned. All fields are + returned if 'None' is passed. + + Returns: + Cursor: Query cursor as iterable which returns asset documents matching + passed filters. + """ + + return _get_assets( + project_name, asset_ids, asset_names, True, archived, fields + ) + + +def get_archived_assets( + project_name, + asset_ids=None, + asset_names=None, + fields=None +): + """Archived assets for specified project by passed filters. + + Passed filters (ids and names) are always combined so all conditions must + match. + + To receive all archived assets from project just keep filters empty. + + Args: + project_name (str): Name of project where to look for queried entities. + asset_ids (list[str|ObjectId]): Asset ids that should be found. + asset_names (list[str]): Name assets that should be found. + fields (list[str]): Fields that should be returned. All fields are + returned if 'None' is passed. + + Returns: + Cursor: Query cursor as iterable which returns asset documents matching + passed filters. + """ + + return _get_assets( + project_name, asset_ids, asset_names, False, True, fields + ) + + +def get_asset_ids_with_subsets(project_name, asset_ids=None): + """Find out which assets have existing subsets. + + Args: + project_name (str): Name of project where to look for queried entities. + asset_ids (list[str|ObjectId]): Look only for entered asset ids. + + Returns: + List[ObjectId]: Asset ids that have existing subsets. + """ + + subset_query = { + "type": "subset" + } + if asset_ids is not None: + asset_ids = _convert_ids(asset_ids) + if not asset_ids: + return [] + subset_query["parent"] = {"$in": asset_ids} + + conn = _get_project_connection(project_name) + result = conn.aggregate([ + { + "$match": subset_query + }, + { + "$group": { + "_id": "$parent", + "count": {"$sum": 1} + } + } + ]) + asset_ids_with_subsets = [] + for item in result: + asset_id = item["_id"] + count = item["count"] + if count > 0: + asset_ids_with_subsets.append(asset_id) + return asset_ids_with_subsets + + +def get_subset_by_id(project_name, subset_id, fields=None): + """Single subset entity data by it's id. + + Args: + project_name (str): Name of project where to look for queried entities. + subset_id (str|ObjectId): Id of subset which should be found. + fields (list[str]): Fields that should be returned. All fields are + returned if 'None' is passed. + + Returns: + None: If subset with specified filters was not found. + Dict: Subset document which can be reduced to specified 'fields'. + """ + + subset_id = _convert_id(subset_id) + if not subset_id: + return None + + query_filters = {"type": "subset", "_id": subset_id} + conn = _get_project_connection(project_name) + return conn.find_one(query_filters, _prepare_fields(fields)) + + +def get_subset_by_name(project_name, subset_name, asset_id, fields=None): + """Single subset entity data by it's name and it's version id. + + Args: + project_name (str): Name of project where to look for queried entities. + subset_name (str): Name of subset. + asset_id (str|ObjectId): Id of parent asset. + fields (list[str]): Fields that should be returned. All fields are + returned if 'None' is passed. + + Returns: + None: If subset with specified filters was not found. + Dict: Subset document which can be reduced to specified 'fields'. + """ + + if not subset_name: + return None + + asset_id = _convert_id(asset_id) + if not asset_id: + return None + + query_filters = { + "type": "subset", + "name": subset_name, + "parent": asset_id + } + conn = _get_project_connection(project_name) + return conn.find_one(query_filters, _prepare_fields(fields)) + + +def get_subsets( + project_name, + subset_ids=None, + subset_names=None, + asset_ids=None, + archived=False, + fields=None +): + """Subset entities data from one project filtered by entered filters. + + Filters are additive (all conditions must pass to return subset). + + Args: + project_name (str): Name of project where to look for queried entities. + subset_ids (list[str|ObjectId]): Subset ids that should be queried. + Filter ignored if 'None' is passed. + subset_names (list[str]): Subset names that should be queried. + Filter ignored if 'None' is passed. + asset_ids (list[str|ObjectId]): Asset ids under which should look for + the subsets. Filter ignored if 'None' is passed. + fields (list[str]): Fields that should be returned. All fields are + returned if 'None' is passed. + + Returns: + Cursor: Iterable cursor yielding all matching subsets. + """ + + subset_types = ["subset"] + if archived: + subset_types.append("archived_subset") + + if len(subset_types) == 1: + query_filter = {"type": subset_types[0]} + else: + query_filter = {"type": {"$in": subset_types}} + + if asset_ids is not None: + asset_ids = _convert_ids(asset_ids) + if not asset_ids: + return [] + query_filter["parent"] = {"$in": asset_ids} + + if subset_ids is not None: + subset_ids = _convert_ids(subset_ids) + if not subset_ids: + return [] + query_filter["_id"] = {"$in": subset_ids} + + if subset_names is not None: + if not subset_names: + return [] + query_filter["name"] = {"$in": list(subset_names)} + + conn = _get_project_connection(project_name) + return conn.find(query_filter, _prepare_fields(fields)) + + +def get_subset_families(project_name, subset_ids=None): + """Set of main families of subsets. + + Args: + project_name (str): Name of project where to look for queried entities. + subset_ids (list[str|ObjectId]): Subset ids that should be queried. + All subsets from project are used if 'None' is passed. + + Returns: + set[str]: Main families of matching subsets. + """ + + subset_filter = { + "type": "subset" + } + if subset_ids is not None: + if not subset_ids: + return set() + subset_filter["_id"] = {"$in": list(subset_ids)} + + conn = _get_project_connection(project_name) + result = list(conn.aggregate([ + {"$match": subset_filter}, + {"$project": { + "family": {"$arrayElemAt": ["$data.families", 0]} + }}, + {"$group": { + "_id": "family_group", + "families": {"$addToSet": "$family"} + }} + ])) + if result: + return set(result[0]["families"]) + return set() + + +def get_version_by_id(project_name, version_id, fields=None): + """Single version entity data by it's id. + + Args: + project_name (str): Name of project where to look for queried entities. + version_id (str|ObjectId): Id of version which should be found. + fields (list[str]): Fields that should be returned. All fields are + returned if 'None' is passed. + + Returns: + None: If version with specified filters was not found. + Dict: Version document which can be reduced to specified 'fields'. + """ + + version_id = _convert_id(version_id) + if not version_id: + return None + + query_filter = { + "type": {"$in": ["version", "hero_version"]}, + "_id": version_id + } + conn = _get_project_connection(project_name) + return conn.find_one(query_filter, _prepare_fields(fields)) + + +def get_version_by_name(project_name, version, subset_id, fields=None): + """Single version entity data by it's name and subset id. + + Args: + project_name (str): Name of project where to look for queried entities. + version (int): name of version entity (it's version). + subset_id (str|ObjectId): Id of version which should be found. + fields (list[str]): Fields that should be returned. All fields are + returned if 'None' is passed. + + Returns: + None: If version with specified filters was not found. + Dict: Version document which can be reduced to specified 'fields'. + """ + + subset_id = _convert_id(subset_id) + if not subset_id: + return None + + conn = _get_project_connection(project_name) + query_filter = { + "type": "version", + "parent": subset_id, + "name": version + } + return conn.find_one(query_filter, _prepare_fields(fields)) + + +def _get_versions( + project_name, + subset_ids=None, + version_ids=None, + versions=None, + standard=True, + hero=False, + fields=None +): + version_types = [] + if standard: + version_types.append("version") + + if hero: + version_types.append("hero_version") + + if not version_types: + return [] + elif len(version_types) == 1: + query_filter = {"type": version_types[0]} + else: + query_filter = {"type": {"$in": version_types}} + + if subset_ids is not None: + subset_ids = _convert_ids(subset_ids) + if not subset_ids: + return [] + query_filter["parent"] = {"$in": subset_ids} + + if version_ids is not None: + version_ids = _convert_ids(version_ids) + if not version_ids: + return [] + query_filter["_id"] = {"$in": version_ids} + + if versions is not None: + versions = list(versions) + if not versions: + return [] + + if len(versions) == 1: + query_filter["name"] = versions[0] + else: + query_filter["name"] = {"$in": versions} + + conn = _get_project_connection(project_name) + + return conn.find(query_filter, _prepare_fields(fields)) + + +def get_versions( + project_name, + version_ids=None, + subset_ids=None, + versions=None, + hero=False, + fields=None +): + """Version entities data from one project filtered by entered filters. + + Filters are additive (all conditions must pass to return subset). + + Args: + project_name (str): Name of project where to look for queried entities. + version_ids (list[str|ObjectId]): Version ids that will be queried. + Filter ignored if 'None' is passed. + subset_ids (list[str]): Subset ids that will be queried. + Filter ignored if 'None' is passed. + versions (list[int]): Version names (as integers). + Filter ignored if 'None' is passed. + hero (bool): Look also for hero versions. + fields (list[str]): Fields that should be returned. All fields are + returned if 'None' is passed. + + Returns: + Cursor: Iterable cursor yielding all matching versions. + """ + + return _get_versions( + project_name, + subset_ids, + version_ids, + versions, + standard=True, + hero=hero, + fields=fields + ) + + +def get_hero_version_by_subset_id(project_name, subset_id, fields=None): + """Hero version by subset id. + + Args: + project_name (str): Name of project where to look for queried entities. + subset_id (str|ObjectId): Subset id under which is hero version. + fields (list[str]): Fields that should be returned. All fields are + returned if 'None' is passed. + + Returns: + None: If hero version for passed subset id does not exists. + Dict: Hero version entity data. + """ + + subset_id = _convert_id(subset_id) + if not subset_id: + return None + + versions = list(_get_versions( + project_name, + subset_ids=[subset_id], + standard=False, + hero=True, + fields=fields + )) + if versions: + return versions[0] + return None + + +def get_hero_version_by_id(project_name, version_id, fields=None): + """Hero version by it's id. + + Args: + project_name (str): Name of project where to look for queried entities. + version_id (str|ObjectId): Hero version id. + fields (list[str]): Fields that should be returned. All fields are + returned if 'None' is passed. + + Returns: + None: If hero version with passed id was not found. + Dict: Hero version entity data. + """ + + version_id = _convert_id(version_id) + if not version_id: + return None + + versions = list(_get_versions( + project_name, + version_ids=[version_id], + standard=False, + hero=True, + fields=fields + )) + if versions: + return versions[0] + return None + + +def get_hero_versions( + project_name, + subset_ids=None, + version_ids=None, + fields=None +): + """Hero version entities data from one project filtered by entered filters. + + Args: + project_name (str): Name of project where to look for queried entities. + subset_ids (list[str|ObjectId]): Subset ids for which should look for + hero versions. Filter ignored if 'None' is passed. + version_ids (list[str|ObjectId]): Hero version ids. Filter ignored if + 'None' is passed. + fields (list[str]): Fields that should be returned. All fields are + returned if 'None' is passed. + + Returns: + Cursor|list: Iterable yielding hero versions matching passed filters. + """ + + return _get_versions( + project_name, + subset_ids, + version_ids, + standard=False, + hero=True, + fields=fields + ) + + +def get_output_link_versions(project_name, version_id, fields=None): + """Versions where passed version was used as input. + + Question: + Not 100% sure about the usage of the function so the name and docstring + maybe does not match what it does? + + Args: + project_name (str): Name of project where to look for queried entities. + version_id (str|ObjectId): Version id which can be used as input link + for other versions. + fields (list[str]): Fields that should be returned. All fields are + returned if 'None' is passed. + + Returns: + Cursor|list: Iterable cursor yielding versions that are used as input + links for passed version. + """ + + version_id = _convert_id(version_id) + if not version_id: + return [] + + conn = _get_project_connection(project_name) + # Does make sense to look for hero versions? + query_filter = { + "type": "version", + "data.inputLinks.input": version_id + } + return conn.find(query_filter, _prepare_fields(fields)) + + +def get_last_versions(project_name, subset_ids, fields=None): + """Latest versions for entered subset_ids. + + Args: + subset_ids (list): List of subset ids. + + Returns: + dict[ObjectId, int]: Key is subset id and value is last version name. + """ + + subset_ids = _convert_ids(subset_ids) + if not subset_ids: + return {} + + _pipeline = [ + # Find all versions of those subsets + {"$match": { + "type": "version", + "parent": {"$in": subset_ids} + }}, + # Sorting versions all together + {"$sort": {"name": 1}}, + # Group them by "parent", but only take the last + {"$group": { + "_id": "$parent", + "_version_id": {"$last": "$_id"} + }} + ] + + conn = _get_project_connection(project_name) + version_ids = [ + doc["_version_id"] + for doc in conn.aggregate(_pipeline) + ] + + fields = _prepare_fields(fields, ["parent"]) + + version_docs = get_versions( + project_name, version_ids=version_ids, fields=fields + ) + + return { + version_doc["parent"]: version_doc + for version_doc in version_docs + } + + +def get_last_version_by_subset_id(project_name, subset_id, fields=None): + """Last version for passed subset id. + + Args: + project_name (str): Name of project where to look for queried entities. + subset_id (str|ObjectId): Id of version which should be found. + fields (list[str]): Fields that should be returned. All fields are + returned if 'None' is passed. + + Returns: + None: If version with specified filters was not found. + Dict: Version document which can be reduced to specified 'fields'. + """ + + subset_id = _convert_id(subset_id) + if not subset_id: + return None + + last_versions = get_last_versions( + project_name, subset_ids=[subset_id], fields=fields + ) + return last_versions.get(subset_id) + + +def get_last_version_by_subset_name( + project_name, subset_name, asset_id, fields=None +): + """Last version for passed subset name under asset id. + + Args: + project_name (str): Name of project where to look for queried entities. + subset_name (str): Name of subset. + asset_id (str|ObjectId): Asset id which is parnt of passed subset name. + fields (list[str]): Fields that should be returned. All fields are + returned if 'None' is passed. + + Returns: + None: If version with specified filters was not found. + Dict: Version document which can be reduced to specified 'fields'. + """ + + subset_doc = get_subset_by_name( + project_name, subset_name, asset_id, fields=["_id"] + ) + if not subset_doc: + return None + return get_last_version_by_subset_id( + project_name, subset_doc["_id"], fields=fields + ) + + +def get_representation_by_id(project_name, representation_id, fields=None): + """Representation entity data by it's id. + + Args: + project_name (str): Name of project where to look for queried entities. + representation_id (str|ObjectId): Representation id. + fields (list[str]): Fields that should be returned. All fields are + returned if 'None' is passed. + + Returns: + None: If representation with specified filters was not found. + Dict: Representation entity data which can be reduced + to specified 'fields'. + """ + + if not representation_id: + return None + + repre_types = ["representation", "archived_representations"] + query_filter = { + "type": {"$in": repre_types} + } + if representation_id is not None: + query_filter["_id"] = _convert_id(representation_id) + + conn = _get_project_connection(project_name) + + return conn.find_one(query_filter, _prepare_fields(fields)) + + +def get_representation_by_name( + project_name, representation_name, version_id, fields=None +): + """Representation entity data by it's name and it's version id. + + Args: + project_name (str): Name of project where to look for queried entities. + representation_name (str): Representation name. + version_id (str|ObjectId): Id of parent version entity. + fields (list[str]): Fields that should be returned. All fields are + returned if 'None' is passed. + + Returns: + None: If representation with specified filters was not found. + Dict: Representation entity data which can be reduced + to specified 'fields'. + """ + + version_id = _convert_id(version_id) + if not version_id or not representation_name: + return None + repre_types = ["representation", "archived_representations"] + query_filter = { + "type": {"$in": repre_types}, + "name": representation_name, + "parent": version_id + } + + conn = _get_project_connection(project_name) + return conn.find_one(query_filter, _prepare_fields(fields)) + + +def get_representations( + project_name, + representation_ids=None, + representation_names=None, + version_ids=None, + extensions=None, + names_by_version_ids=None, + archived=False, + fields=None +): + """Representaion entities data from one project filtered by filters. + + Filters are additive (all conditions must pass to return subset). + + Args: + project_name (str): Name of project where to look for queried entities. + representation_ids (list[str|ObjectId]): Representation ids used as + filter. Filter ignored if 'None' is passed. + representation_names (list[str]): Representations names used as filter. + Filter ignored if 'None' is passed. + version_ids (list[str]): Subset ids used as parent filter. Filter + ignored if 'None' is passed. + extensions (list[str]): Filter by extension of main representation + file (without dot). + names_by_version_ids (dict[ObjectId, list[str]]): Complex filtering + using version ids and list of names under the version. + archived (bool): Output will also contain archived representations. + fields (list[str]): Fields that should be returned. All fields are + returned if 'None' is passed. + + Returns: + Cursor: Iterable cursor yielding all matching representations. + """ + + repre_types = ["representation"] + if archived: + repre_types.append("archived_representations") + if len(repre_types) == 1: + query_filter = {"type": repre_types[0]} + else: + query_filter = {"type": {"$in": repre_types}} + + if representation_ids is not None: + representation_ids = _convert_ids(representation_ids) + if not representation_ids: + return [] + query_filter["_id"] = {"$in": representation_ids} + + if representation_names is not None: + if not representation_names: + return [] + query_filter["name"] = {"$in": list(representation_names)} + + if version_ids is not None: + version_ids = _convert_ids(version_ids) + if not version_ids: + return [] + query_filter["parent"] = {"$in": version_ids} + + if extensions is not None: + if not extensions: + return [] + query_filter["context.ext"] = {"$in": list(extensions)} + + if names_by_version_ids is not None: + or_query = [] + for version_id, names in names_by_version_ids.items(): + if version_id and names: + or_query.append({ + "parent": _convert_id(version_id), + "name": {"$in": list(names)} + }) + if not or_query: + return [] + query_filter["$or"] = or_query + + conn = _get_project_connection(project_name) + + return conn.find(query_filter, _prepare_fields(fields)) + + +def get_representations_parents(project_name, representations): + """Prepare parents of representation entities. + + Each item of returned dictionary contains version, subset, asset + and project in that order. + + Args: + project_name (str): Name of project where to look for queried entities. + representations (list[dict]): Representation entities with at least + '_id' and 'parent' keys. + + Returns: + dict[ObjectId, tuple]: Parents by representation id. + """ + + repres_by_version_id = collections.defaultdict(list) + versions_by_version_id = {} + versions_by_subset_id = collections.defaultdict(list) + subsets_by_subset_id = {} + subsets_by_asset_id = collections.defaultdict(list) + for representation in representations: + repre_id = representation["_id"] + version_id = representation["parent"] + repres_by_version_id[version_id].append(representation) + + versions = get_versions( + project_name, version_ids=repres_by_version_id.keys() + ) + for version in versions: + version_id = version["_id"] + subset_id = version["parent"] + versions_by_version_id[version_id] = version + versions_by_subset_id[subset_id].append(version) + + subsets = get_subsets( + project_name, subset_ids=versions_by_subset_id.keys() + ) + for subset in subsets: + subset_id = subset["_id"] + asset_id = subset["parent"] + subsets_by_subset_id[subset_id] = subset + subsets_by_asset_id[asset_id].append(subset) + + assets = get_assets(project_name, asset_ids=subsets_by_asset_id.keys()) + assets_by_id = { + asset["_id"]: asset + for asset in assets + } + + project = get_project(project_name) + + output = {} + for version_id, representations in repres_by_version_id.items(): + asset = None + subset = None + version = versions_by_version_id.get(version_id) + if version: + subset_id = version["parent"] + subset = subsets_by_subset_id.get(subset_id) + if subset: + asset_id = subset["parent"] + asset = assets_by_id.get(asset_id) + + for representation in representations: + repre_id = representation["_id"] + output[repre_id] = (version, subset, asset, project) + return output + + +def get_representation_parents(project_name, representation): + """Prepare parents of representation entity. + + Each item of returned dictionary contains version, subset, asset + and project in that order. + + Args: + project_name (str): Name of project where to look for queried entities. + representation (dict): Representation entities with at least + '_id' and 'parent' keys. + + Returns: + dict[ObjectId, tuple]: Parents by representation id. + """ + + if not representation: + return None + + repre_id = representation["_id"] + parents_by_repre_id = get_representations_parents( + project_name, [representation] + ) + return parents_by_repre_id.get(repre_id) + + +def get_thumbnail_id_from_source(project_name, src_type, src_id): + """Receive thumbnail id from source entity. + + Args: + project_name (str): Name of project where to look for queried entities. + src_type (str): Type of source entity ('asset', 'version'). + src_id (str|objectId): Id of source entity. + + Returns: + ObjectId: Thumbnail id assigned to entity. + None: If Source entity does not have any thumbnail id assigned. + """ + + if not src_type or not src_id: + return None + + query_filter = {"_id": _convert_id(src_id)} + + conn = _get_project_connection(project_name) + src_doc = conn.find_one(query_filter, {"data.thumbnail_id"}) + if src_doc: + return src_doc.get("data", {}).get("thumbnail_id") + return None + + +def get_thumbnails(project_name, thumbnail_ids, fields=None): + """Receive thumbnails entity data. + + Thumbnail entity can be used to receive binary content of thumbnail based + on it's content and ThumbnailResolvers. + + Args: + project_name (str): Name of project where to look for queried entities. + thumbnail_ids (list[str|ObjectId]): Ids of thumbnail entities. + fields (list[str]): Fields that should be returned. All fields are + returned if 'None' is passed. + + Returns: + cursor: Cursor of queried documents. + """ + + if thumbnail_ids: + thumbnail_ids = _convert_ids(thumbnail_ids) + + if not thumbnail_ids: + return [] + query_filter = { + "type": "thumbnail", + "_id": {"$in": thumbnail_ids} + } + conn = _get_project_connection(project_name) + return conn.find(query_filter, _prepare_fields(fields)) + + +def get_thumbnail(project_name, thumbnail_id, fields=None): + """Receive thumbnail entity data. + + Args: + project_name (str): Name of project where to look for queried entities. + thumbnail_id (str|ObjectId): Id of thumbnail entity. + fields (list[str]): Fields that should be returned. All fields are + returned if 'None' is passed. + + Returns: + None: If thumbnail with specified id was not found. + Dict: Thumbnail entity data which can be reduced to specified 'fields'. + """ + + if not thumbnail_id: + return None + query_filter = {"type": "thumbnail", "_id": _convert_id(thumbnail_id)} + conn = _get_project_connection(project_name) + return conn.find_one(query_filter, _prepare_fields(fields)) + + +""" +## Custom data storage: +- Settings - OP settings overrides and local settings +- Logging - logs from PypeLogger +- Webpublisher - jobs +- Ftrack - events +- Maya - Shaders + - openpype/hosts/maya/api/shader_definition_editor.py + - openpype/hosts/maya/plugins/publish/validate_model_name.py + +## Global launch hooks +- openpype/hooks/pre_global_host_data.py + Query: + - project + - asset + +## Global load plugins +- openpype/plugins/load/delete_old_versions.py + Query: + - versions + - representations +- openpype/plugins/load/delivery.py + Query: + - representations + +## Global publish plugins +- openpype/plugins/publish/collect_avalon_entities.py + Query: + - asset + - project +- openpype/plugins/publish/collect_anatomy_instance_data.py + Query: + - assets + - subsets + - last version +- openpype/plugins/publish/collect_scene_loaded_versions.py + Query: + - representations +- openpype/plugins/publish/extract_hierarchy_avalon.py + Query: + - asset + - assets + - project + Create: + - asset + Update: + - asset +- openpype/plugins/publish/integrate_hero_version.py + Query: + - version + - hero version + - representations +- openpype/plugins/publish/integrate_new.py + Query: + - asset + - subset + - version + - representations +- openpype/plugins/publish/integrate_thumbnail.py + Query: + - version +- openpype/plugins/publish/validate_editorial_asset_name.py + Query: + - assets + +## Lib +- openpype/lib/applications.py + Query: + - project + - asset +- openpype/lib/avalon_context.py + Query: + - project + - asset + - linked assets (new function get_linked_assets?) + - subset + - subsets + - version + - versions + - last version + - representations + - linked representations (new function get_linked_ids_for_representations) + Update: + - workfile data +- openpype/lib/plugin_tools.py + Query: + - asset +- openpype/lib/project_backpack.py + Query: + - project + - everything from mongo + Update: + - project +- openpype/lib/usdlib.py + Query: + - project + - asset + +## Pipeline +- openpype/pipeline/load/utils.py + Query: + - project + - assets + - subsets + - version + - versions + - representation + - representations +- openpype/pipeline/mongodb.py + Query: + - project +- openpype/pipeline/thumbnail.py + Query: + - project + +## Hosts +### Aftereffects +- openpype/hosts/aftereffects/plugins/create/workfile_creator.py + Query: + - asset + +### Blender +- openpype/hosts/blender/api/pipeline.py + Query: + - asset +- openpype/hosts/blender/plugins/publish/extract_layout.py + Query: + - representation + +### Celaction +- openpype/hosts/celaction/plugins/publish/collect_audio.py + Query: + - subsets + - last versions + - representations + +### Fusion +- openpype/hosts/fusion/api/lib.py + Query: + - asset + - subset + - version + - representation +- openpype/hosts/fusion/plugins/load/load_sequence.py + Query: + - version +- openpype/hosts/fusion/scripts/fusion_switch_shot.py + Query: + - project + - asset + - versions +- openpype/hosts/fusion/utility_scripts/switch_ui.py + Query: + - assets + +### Harmony +- openpype/hosts/harmony/api/pipeline.py + Query: + - representation + +### Hiero +- openpype/hosts/hiero/api/lib.py + Query: + - project + - version + - versions + - representation +- openpype/hosts/hiero/api/tags.py + Query: + - task types + - assets +- openpype/hosts/hiero/plugins/load/load_clip.py + Query: + - version + - versions +- openpype/hosts/hiero/plugins/publish_old_workflow/collect_assetbuilds.py + Query: + - assets + +### Houdini +- openpype/hosts/houdini/api/lib.py + Query: + - asset +- openpype/hosts/houdini/api/usd.py + Query: + - asset +- openpype/hosts/houdini/plugins/create/create_hda.py + Query: + - asset + - subsets +- openpype/hosts/houdini/plugins/publish/collect_usd_bootstrap.py + Query: + - asset + - subset +- openpype/hosts/houdini/plugins/publish/extract_usd_layered.py + Query: + - asset + - subset + - version + - representation +- openpype/hosts/houdini/plugins/publish/validate_usd_shade_model_exists.py + Query: + - asset + - subset +- openpype/hosts/houdini/vendor/husdoutputprocessors/avalon_uri_processor.py + Query: + - project + - asset + +### Maya +- openpype/hosts/maya/api/action.py + Query: + - asset +- openpype/hosts/maya/api/commands.py + Query: + - asset + - project +- openpype/hosts/maya/api/lib.py + Query: + - project + - asset + - subset + - subsets + - version + - representation +- openpype/hosts/maya/api/setdress.py + Query: + - version + - representation +- openpype/hosts/maya/plugins/inventory/import_modelrender.py + Query: + - representation +- openpype/hosts/maya/plugins/load/load_audio.py + Query: + - asset + - subset + - version +- openpype/hosts/maya/plugins/load/load_image_plane.py + Query: + - asset + - subset + - version +- openpype/hosts/maya/plugins/load/load_look.py + Query: + - representation +- openpype/hosts/maya/plugins/load/load_vrayproxy.py + Query: + - representation +- openpype/hosts/maya/plugins/load/load_yeti_cache.py + Query: + - representation +- openpype/hosts/maya/plugins/publish/collect_review.py + Query: + - subsets +- openpype/hosts/maya/plugins/publish/validate_node_ids_in_database.py + Query: + - assets +- openpype/hosts/maya/plugins/publish/validate_node_ids_related.py + Query: + - asset +- openpype/hosts/maya/plugins/publish/validate_renderlayer_aovs.py + Query: + - asset + - subset + +### Nuke +- openpype/hosts/nuke/api/command.py + Query: + - project + - asset +- openpype/hosts/nuke/api/lib.py + Query: + - project + - asset + - version + - versions + - representation +- openpype/hosts/nuke/plugins/load/load_backdrop.py + Query: + - version + - versions +- openpype/hosts/nuke/plugins/load/load_camera_abc.py + Query: + - version + - versions +- openpype/hosts/nuke/plugins/load/load_clip.py + Query: + - version + - versions +- openpype/hosts/nuke/plugins/load/load_effects_ip.py + Query: + - version + - versions +- openpype/hosts/nuke/plugins/load/load_effects.py + Query: + - version + - versions +- openpype/hosts/nuke/plugins/load/load_gizmo_ip.py + Query: + - version + - versions +- openpype/hosts/nuke/plugins/load/load_gizmo.py + Query: + - version + - versions +- openpype/hosts/nuke/plugins/load/load_image.py + Query: + - version + - versions +- openpype/hosts/nuke/plugins/load/load_model.py + Query: + - version + - versions +- openpype/hosts/nuke/plugins/load/load_script_precomp.py + Query: + - version + - versions +- openpype/hosts/nuke/plugins/publish/collect_reads.py + Query: + - asset +- openpype/hosts/nuke/plugins/publish/precollect_instances.py + Query: + - asset +- openpype/hosts/nuke/plugins/publish/precollect_writes.py + Query: + - representation +- openpype/hosts/nuke/plugins/publish/validate_script.py + Query: + - asset + - project + +### Photoshop +- openpype/hosts/photoshop/plugins/create/workfile_creator.py + Query: + - asset + +### Resolve +- openpype/hosts/resolve/plugins/load/load_clip.py + Query: + - version + - versions + +### Standalone publisher +- openpype/hosts/standalonepublisher/plugins/publish/collect_bulk_mov_instances.py + Query: + - asset +- openpype/hosts/standalonepublisher/plugins/publish/collect_matching_asset.py + Query: + - assets +- openpype/hosts/standalonepublisher/plugins/publish/collect_hierarchy.py + Query: + - project + - asset +- openpype/hosts/standalonepublisher/plugins/publish/validate_task_existence.py + Query: + - assets + +### TVPaint +- openpype/hosts/tvpaint/api/pipeline.py + Query: + - project + - asset +- openpype/hosts/tvpaint/plugins/load/load_workfile.py + Query: + - project + - asset +- openpype/hosts/tvpaint/plugins/publish/collect_instances.py + Query: + - asset +- openpype/hosts/tvpaint/plugins/publish/collect_scene_render.py + Query: + - asset +- openpype/hosts/tvpaint/plugins/publish/collect_workfile.py + Query: + - asset + +### Unreal +- openpype/hosts/unreal/plugins/load/load_camera.py + Query: + - asset + - assets +- openpype/hosts/unreal/plugins/load/load_layout.py + Query: + - asset + - assets +- openpype/hosts/unreal/plugins/publish/extract_layout.py + Query: + - representation + +### Webpublisher +- openpype/hosts/webpublisher/webserver_service/webpublish_routes.py + Query: + - assets +- openpype/hosts/webpublisher/plugins/publish/collect_published_files.py + Query: + - last versions + +## Tools +openpype/tools/assetlinks/widgets.py +- SimpleLinkView + Query: + - get_versions + - get_subsets + - get_assets + - get_output_link_versions + +openpype/tools/creator/window.py +- CreatorWindow + Query: + - get_asset_by_name + - get_subsets + +openpype/tools/launcher/models.py +- LauncherModel + Query: + - get_project + - get_assets + +openpype/tools/libraryloader/app.py +- LibraryLoaderWindow + Query: + - get_project + +openpype/tools/loader/app.py +- LoaderWindow + Query: + - get_project +- show + Query: + - get_projects + +openpype/tools/loader/model.py +- SubsetsModel + Query: + - get_assets + - get_subsets + - get_last_versions + - get_versions + - get_hero_versions + - get_version_by_name +- RepresentationModel + Query: + - get_representations + - sync server specific queries (separated into multiple functions?) + - NOT REPLACED + +openpype/tools/loader/widgets.py +- FamilyModel + Query: + - get_subset_families +- VersionTextEdit + Query: + - get_subset_by_id + - get_version_by_id +- SubsetWidget + Query: + - get_subsets + - get_representations + Update: + - Subset groups (combination of asset id and subset names) +- RepresentationWidget + Query: + - get_subsets + - get_versions + - get_representations +- ThumbnailWidget + Query: + - get_thumbnail_id_from_source + - get_thumbnail + +openpype/tools/mayalookassigner/app.py +- MayaLookAssignerWindow + Query: + - get_last_version_by_subset_id + +openpype/tools/mayalookassigner/commands.py +- create_items_from_nodes + Query: + - get_asset_by_id + +openpype/tools/mayalookassigner/vray_proxies.py +- get_look_relationships + Query: + - get_representation_by_name +- load_look + Query: + - get_representation_by_name +- vrayproxy_assign_look + Query: + - get_last_version_by_subset_name + +openpype/tools/project_manager/project_manager/model.py +- HierarchyModel + Query: + - get_asset_ids_with_subsets + - get_project + - get_assets + +openpype/tools/project_manager/project_manager/view.py +- ProjectDocCache + Query: + - get_project + +openpype/tools/project_manager/project_manager/widgets.py +- CreateProjectDialog + Query: + - get_projects + +openpype/tools/publisher/widgets/create_dialog.py +- CreateDialog + Query: + - get_asset_by_name + - get_subsets + +openpype/tools/publisher/control.py +- AssetDocsCache + Query: + - get_assets + +openpype/tools/sceneinventory/model.py +- InventoryModel + Query: + - get_asset_by_id + - get_subset_by_id + - get_version_by_id + - get_last_version_by_subset_id + - get_representation + +openpype/tools/sceneinventory/switch_dialog.py +- SwitchAssetDialog + Query: + - get_asset_by_name + - get_assets + - get_subset_by_name + - get_subsets + - get_versions + - get_hero_versions + - get_last_versions + - get_representations + +openpype/tools/sceneinventory/view.py +- SceneInventoryView + Query: + - get_version_by_id + - get_versions + - get_hero_versions + - get_representation_by_id + - get_representations + +openpype/tools/standalonepublish/widgets/model_asset.py +- AssetModel + Query: + - get_assets + +openpype/tools/standalonepublish/widgets/widget_asset.py +- AssetWidget + Query: + - get_project + - get_asset_by_id + +openpype/tools/standalonepublish/widgets/widget_family.py +- FamilyWidget + Query: + - get_asset_by_name + - get_subset_by_name + - get_subsets + - get_last_version_by_subset_id + +openpype/tools/standalonepublish/app.py +- Window + Query: + - get_asset_by_id + +openpype/tools/texture_copy/app.py +- TextureCopy + Query: + - get_project + - get_asset_by_name + +openpype/tools/workfiles/files_widget.py +- FilesWidget + Query: + - get_asset_by_id + +openpype/tools/workfiles/model.py +- PublishFilesModel + Query: + - get_subsets + - get_versions + - get_representations + +openpype/tools/workfiles/save_as_dialog.py +- build_workfile_data + Query: + - get_project + - get_asset_by_name + +openpype/tools/workfiles/window.py +- Window + Query: + - get_asset_by_id + - get_asset_by_name + +openpype/tools/utils/assets_widget.py +- AssetModel + Query: + - get_project + - get_assets + +openpype/tools/utils/delegates.py +- VersionDelegate + Query: + - get_versions + - get_hero_versions + +openpype/tools/utils/lib.py +- GroupsConfig + Query: + - get_project +- FamilyConfigCache + Query: + - get_asset_by_name + +openpype/tools/utils/tasks_widget.py +- TasksModel + Query: + - get_project + - get_asset_by_id +""" diff --git a/openpype/hosts/aftereffects/api/pipeline.py b/openpype/hosts/aftereffects/api/pipeline.py index a428a1470d..0bc47665b0 100644 --- a/openpype/hosts/aftereffects/api/pipeline.py +++ b/openpype/hosts/aftereffects/api/pipeline.py @@ -65,14 +65,14 @@ def on_pyblish_instance_toggled(instance, old_value, new_value): instance[0].Visible = new_value -def get_asset_settings(): +def get_asset_settings(asset_doc): """Get settings on current asset from database. Returns: dict: Scene data. """ - asset_data = lib.get_asset()["data"] + asset_data = asset_doc["data"] fps = asset_data.get("fps") frame_start = asset_data.get("frameStart") frame_end = asset_data.get("frameEnd") diff --git a/openpype/hosts/aftereffects/plugins/create/workfile_creator.py b/openpype/hosts/aftereffects/plugins/create/workfile_creator.py index 88e55e21b5..badb3675fd 100644 --- a/openpype/hosts/aftereffects/plugins/create/workfile_creator.py +++ b/openpype/hosts/aftereffects/plugins/create/workfile_creator.py @@ -1,4 +1,5 @@ import openpype.hosts.aftereffects.api as api +from openpype.client import get_asset_by_name from openpype.pipeline import ( AutoCreator, CreatedInstance, @@ -41,10 +42,7 @@ class AEWorkfileCreator(AutoCreator): host_name = legacy_io.Session["AVALON_APP"] if existing_instance is None: - asset_doc = legacy_io.find_one({ - "type": "asset", - "name": asset_name - }) + asset_doc = get_asset_by_name(project_name, asset_name) subset_name = self.get_subset_name( variant, task_name, asset_doc, project_name, host_name ) @@ -69,10 +67,7 @@ class AEWorkfileCreator(AutoCreator): existing_instance["asset"] != asset_name or existing_instance["task"] != task_name ): - asset_doc = legacy_io.find_one({ - "type": "asset", - "name": asset_name - }) + asset_doc = get_asset_by_name(project_name, asset_name) subset_name = self.get_subset_name( variant, task_name, asset_doc, project_name, host_name ) diff --git a/openpype/hosts/aftereffects/plugins/publish/collect_render.py b/openpype/hosts/aftereffects/plugins/publish/collect_render.py index fa23bf92b0..97b3175c57 100644 --- a/openpype/hosts/aftereffects/plugins/publish/collect_render.py +++ b/openpype/hosts/aftereffects/plugins/publish/collect_render.py @@ -21,7 +21,7 @@ class AERenderInstance(RenderInstance): projectEntity = attr.ib(default=None) stagingDir = attr.ib(default=None) app_version = attr.ib(default=None) - publish_attributes = attr.ib(default=None) + publish_attributes = attr.ib(default={}) file_name = attr.ib(default=None) @@ -90,7 +90,7 @@ class CollectAERender(abstract_collect_render.AbstractCollectRender): subset_name = inst.data["subset"] instance = AERenderInstance( - family=family, + family="render", families=inst.data.get("families", []), version=version, time="", @@ -116,7 +116,7 @@ class CollectAERender(abstract_collect_render.AbstractCollectRender): toBeRenderedOn='deadline', fps=fps, app_version=app_version, - publish_attributes=inst.data.get("publish_attributes"), + publish_attributes=inst.data.get("publish_attributes", {}), file_name=render_q.file_name ) diff --git a/openpype/hosts/aftereffects/plugins/publish/validate_scene_settings.py b/openpype/hosts/aftereffects/plugins/publish/validate_scene_settings.py index 14e224fdc2..78f98d7445 100644 --- a/openpype/hosts/aftereffects/plugins/publish/validate_scene_settings.py +++ b/openpype/hosts/aftereffects/plugins/publish/validate_scene_settings.py @@ -1,5 +1,9 @@ # -*- coding: utf-8 -*- -"""Validate scene settings.""" +"""Validate scene settings. +Requires: + instance -> assetEntity + instance -> anatomyData +""" import os import re @@ -54,7 +58,7 @@ class ValidateSceneSettings(OptionalPyblishPluginMixin, order = pyblish.api.ValidatorOrder label = "Validate Scene Settings" - families = ["render.farm", "render"] + families = ["render.farm", "render.local", "render"] hosts = ["aftereffects"] optional = True @@ -67,7 +71,8 @@ class ValidateSceneSettings(OptionalPyblishPluginMixin, if not self.is_active(instance.data): return - expected_settings = get_asset_settings() + asset_doc = instance.data["assetEntity"] + expected_settings = get_asset_settings(asset_doc) self.log.info("config from DB::{}".format(expected_settings)) task_name = instance.data["anatomyData"]["task"]["name"] diff --git a/openpype/hosts/blender/api/pipeline.py b/openpype/hosts/blender/api/pipeline.py index 5b81764644..93d81145bc 100644 --- a/openpype/hosts/blender/api/pipeline.py +++ b/openpype/hosts/blender/api/pipeline.py @@ -10,6 +10,7 @@ from . import ops import pyblish.api +from openpype.client import get_asset_by_name from openpype.pipeline import ( schema, legacy_io, @@ -83,11 +84,9 @@ def uninstall(): def set_start_end_frames(): + project_name = legacy_io.active_project() asset_name = legacy_io.Session["AVALON_ASSET"] - asset_doc = legacy_io.find_one({ - "type": "asset", - "name": asset_name - }) + asset_doc = get_asset_by_name(project_name, asset_name) scene = bpy.context.scene diff --git a/openpype/hosts/blender/plugins/publish/extract_layout.py b/openpype/hosts/blender/plugins/publish/extract_layout.py index 8ecc78a2c6..75d9cf440d 100644 --- a/openpype/hosts/blender/plugins/publish/extract_layout.py +++ b/openpype/hosts/blender/plugins/publish/extract_layout.py @@ -1,13 +1,11 @@ import os import json -from bson.objectid import ObjectId - import bpy import bpy_extras import bpy_extras.anim_utils -from openpype.pipeline import legacy_io +from openpype.client import get_representation_by_name from openpype.hosts.blender.api import plugin from openpype.hosts.blender.api.pipeline import AVALON_PROPERTY import openpype.api @@ -131,43 +129,32 @@ class ExtractLayout(openpype.api.Extractor): fbx_count = 0 + project_name = instance.context.data["projectEntity"]["name"] for asset in asset_group.children: metadata = asset.get(AVALON_PROPERTY) - parent = metadata["parent"] + version_id = metadata["parent"] family = metadata["family"] - self.log.debug("Parent: {}".format(parent)) + self.log.debug("Parent: {}".format(version_id)) # Get blend reference - blend = legacy_io.find_one( - { - "type": "representation", - "parent": ObjectId(parent), - "name": "blend" - }, - projection={"_id": True}) + blend = get_representation_by_name( + project_name, "blend", version_id, fields=["_id"] + ) blend_id = None if blend: blend_id = blend["_id"] # Get fbx reference - fbx = legacy_io.find_one( - { - "type": "representation", - "parent": ObjectId(parent), - "name": "fbx" - }, - projection={"_id": True}) + fbx = get_representation_by_name( + project_name, "fbx", version_id, fields=["_id"] + ) fbx_id = None if fbx: fbx_id = fbx["_id"] # Get abc reference - abc = legacy_io.find_one( - { - "type": "representation", - "parent": ObjectId(parent), - "name": "abc" - }, - projection={"_id": True}) + abc = get_representation_by_name( + project_name, "abc", version_id, fields=["_id"] + ) abc_id = None if abc: abc_id = abc["_id"] diff --git a/openpype/hosts/flame/plugins/publish/extract_subset_resources.py b/openpype/hosts/flame/plugins/publish/extract_subset_resources.py index 0bad3f7cfc..6ad8f8cf96 100644 --- a/openpype/hosts/flame/plugins/publish/extract_subset_resources.py +++ b/openpype/hosts/flame/plugins/publish/extract_subset_resources.py @@ -1,5 +1,6 @@ import os import re +import tempfile from pprint import pformat from copy import deepcopy @@ -420,3 +421,30 @@ class ExtractSubsetResources(openpype.api.Extractor): "Path `{}` is containing more that one clip".format(path) ) return clips[0] + + def staging_dir(self, instance): + """Provide a temporary directory in which to store extracted files + + Upon calling this method the staging directory is stored inside + the instance.data['stagingDir'] + """ + staging_dir = instance.data.get('stagingDir', None) + openpype_temp_dir = os.getenv("OPENPYPE_TEMP_DIR") + + if not staging_dir: + if openpype_temp_dir and os.path.exists(openpype_temp_dir): + staging_dir = os.path.normpath( + tempfile.mkdtemp( + prefix="pyblish_tmp_", + dir=openpype_temp_dir + ) + ) + else: + staging_dir = os.path.normpath( + tempfile.mkdtemp(prefix="pyblish_tmp_") + ) + instance.data['stagingDir'] = staging_dir + + instance.context.data["cleanupFullPaths"].append(staging_dir) + + return staging_dir diff --git a/openpype/hosts/harmony/api/README.md b/openpype/hosts/harmony/api/README.md index dd45eb14dd..b39f900886 100644 --- a/openpype/hosts/harmony/api/README.md +++ b/openpype/hosts/harmony/api/README.md @@ -610,7 +610,8 @@ class ImageSequenceLoader(load.LoaderPlugin): def update(self, container, representation): node = container.pop("node") - version = legacy_io.find_one({"_id": representation["parent"]}) + project_name = legacy_io.active_project() + version = get_version_by_id(project_name, representation["parent"]) files = [] for f in version["data"]["files"]: files.append( diff --git a/openpype/hosts/harmony/api/pipeline.py b/openpype/hosts/harmony/api/pipeline.py index b953d0e984..86b5753f7e 100644 --- a/openpype/hosts/harmony/api/pipeline.py +++ b/openpype/hosts/harmony/api/pipeline.py @@ -2,10 +2,10 @@ import os from pathlib import Path import logging -from bson.objectid import ObjectId import pyblish.api from openpype import lib +from openpype.client import get_representation_by_id from openpype.lib import register_event_callback from openpype.pipeline import ( legacy_io, @@ -104,22 +104,20 @@ def check_inventory(): If it does it will colorize outdated nodes and display warning message in Harmony. """ - if not lib.any_outdated(): - return + project_name = legacy_io.active_project() outdated_containers = [] for container in ls(): - representation = container['representation'] - representation_doc = legacy_io.find_one( - { - "_id": ObjectId(representation), - "type": "representation" - }, - projection={"parent": True} + representation_id = container['representation'] + representation_doc = get_representation_by_id( + project_name, representation_id, fields=["parent"] ) if representation_doc and not lib.is_latest(representation_doc): outdated_containers.append(container) + if not outdated_containers: + return + # Colour nodes. outdated_nodes = [] for container in outdated_containers: diff --git a/openpype/hosts/harmony/plugins/publish/validate_audio.py b/openpype/hosts/harmony/plugins/publish/validate_audio.py index cb6b2307cd..e9b8609803 100644 --- a/openpype/hosts/harmony/plugins/publish/validate_audio.py +++ b/openpype/hosts/harmony/plugins/publish/validate_audio.py @@ -47,6 +47,6 @@ class ValidateAudio(pyblish.api.InstancePlugin): formatting_data = { "audio_url": audio_path } - if os.path.isfile(audio_path): + if not os.path.isfile(audio_path): raise PublishXmlValidationError(self, msg, formatting_data=formatting_data) diff --git a/openpype/hosts/hiero/api/lib.py b/openpype/hosts/hiero/api/lib.py index 06dfd2f2ee..8c8c31bc4c 100644 --- a/openpype/hosts/hiero/api/lib.py +++ b/openpype/hosts/hiero/api/lib.py @@ -12,8 +12,13 @@ import shutil import hiero from Qt import QtWidgets -from bson.objectid import ObjectId +from openpype.client import ( + get_project, + get_versions, + get_last_versions, + get_representations, +) from openpype.pipeline import legacy_io from openpype.api import (Logger, Anatomy, get_anatomy_settings) from . import tags @@ -477,7 +482,7 @@ def sync_avalon_data_to_workfile(): project.setProjectRoot(active_project_root) # get project data from avalon db - project_doc = legacy_io.find_one({"type": "project"}) + project_doc = get_project(project_name) project_data = project_doc["data"] log.debug("project_data: {}".format(project_data)) @@ -1065,35 +1070,63 @@ def check_inventory_versions(track_items=None): clip_color_last = "green" clip_color = "red" - # get all track items from current timeline + item_with_repre_id = [] + repre_ids = set() + # Find all containers and collect it's node and representation ids for track_item in track_item: container = parse_container(track_item) if container: - # get representation from io - representation = legacy_io.find_one({ - "type": "representation", - "_id": ObjectId(container["representation"]) - }) + repre_id = container["representation"] + repre_ids.add(repre_id) + item_with_repre_id.append((track_item, repre_id)) - # Get start frame from version data - version = legacy_io.find_one({ - "type": "version", - "_id": representation["parent"] - }) + # Skip if nothing was found + if not repre_ids: + return - # get all versions in list - versions = legacy_io.find({ - "type": "version", - "parent": version["parent"] - }).distinct('name') + project_name = legacy_io.active_project() + # Find representations based on found containers + repre_docs = get_representations( + project_name, + repre_ids=repre_ids, + fields=["_id", "parent"] + ) + # Store representations by id and collect version ids + repre_docs_by_id = {} + version_ids = set() + for repre_doc in repre_docs: + # Use stringed representation id to match value in containers + repre_id = str(repre_doc["_id"]) + repre_docs_by_id[repre_id] = repre_doc + version_ids.add(repre_doc["parent"]) - max_version = max(versions) + version_docs = get_versions( + project_name, version_ids, fields=["_id", "name", "parent"] + ) + # Store versions by id and collect subset ids + version_docs_by_id = {} + subset_ids = set() + for version_doc in version_docs: + version_docs_by_id[version_doc["_id"]] = version_doc + subset_ids.add(version_doc["parent"]) - # set clip colour - if version.get("name") == max_version: - track_item.source().binItem().setColor(clip_color_last) - else: - track_item.source().binItem().setColor(clip_color) + # Query last versions based on subset ids + last_versions_by_subset_id = get_last_versions( + project_name, subset_ids=subset_ids, fields=["_id", "parent"] + ) + + for item in item_with_repre_id: + # Some python versions of nuke can't unfold tuple in for loop + track_item, repre_id = item + + repre_doc = repre_docs_by_id[repre_id] + version_doc = version_docs_by_id[repre_doc["parent"]] + last_version_doc = last_versions_by_subset_id[version_doc["parent"]] + # Check if last version is same as current version + if version_doc["_id"] == last_version_doc["_id"]: + track_item.source().binItem().setColor(clip_color_last) + else: + track_item.source().binItem().setColor(clip_color) def selection_changed_timeline(event): diff --git a/openpype/hosts/hiero/api/tags.py b/openpype/hosts/hiero/api/tags.py index 8c6ff2a77b..10df96fa53 100644 --- a/openpype/hosts/hiero/api/tags.py +++ b/openpype/hosts/hiero/api/tags.py @@ -2,6 +2,7 @@ import re import os import hiero +from openpype.client import get_project, get_assets from openpype.api import Logger from openpype.pipeline import legacy_io @@ -141,7 +142,9 @@ def add_tags_to_workfile(): nks_pres_tags = tag_data() # Get project task types. - tasks = legacy_io.find_one({"type": "project"})["config"]["tasks"] + project_name = legacy_io.active_project() + project_doc = get_project(project_name) + tasks = project_doc["config"]["tasks"] nks_pres_tags["[Tasks]"] = {} log.debug("__ tasks: {}".format(tasks)) for task_type in tasks.keys(): @@ -159,7 +162,9 @@ def add_tags_to_workfile(): # asset builds and shots. if int(os.getenv("TAG_ASSETBUILD_STARTUP", 0)) == 1: nks_pres_tags["[AssetBuilds]"] = {} - for asset in legacy_io.find({"type": "asset"}): + for asset in get_assets( + project_name, fields=["name", "data.entityType"] + ): if asset["data"]["entityType"] == "AssetBuild": nks_pres_tags["[AssetBuilds]"][asset["name"]] = { "editable": "1", diff --git a/openpype/hosts/hiero/plugins/load/load_clip.py b/openpype/hosts/hiero/plugins/load/load_clip.py index a3365253b3..2a7d1af41e 100644 --- a/openpype/hosts/hiero/plugins/load/load_clip.py +++ b/openpype/hosts/hiero/plugins/load/load_clip.py @@ -1,3 +1,7 @@ +from openpype.client import ( + get_version_by_id, + get_last_version_by_subset_id +) from openpype.pipeline import ( legacy_io, get_representation_path, @@ -103,12 +107,12 @@ class LoadClip(phiero.SequenceLoader): namespace = container['namespace'] track_item = phiero.get_track_items( track_item_name=namespace).pop() - version = legacy_io.find_one({ - "type": "version", - "_id": representation["parent"] - }) - version_data = version.get("data", {}) - version_name = version.get("name", None) + + project_name = legacy_io.active_project() + version_doc = get_version_by_id(project_name, representation["parent"]) + + version_data = version_doc.get("data", {}) + version_name = version_doc.get("name", None) colorspace = version_data.get("colorspace", None) object_name = "{}_{}".format(name, namespace) file = get_representation_path(representation).replace("\\", "/") @@ -143,7 +147,7 @@ class LoadClip(phiero.SequenceLoader): }) # update color of clip regarding the version order - self.set_item_color(track_item, version) + self.set_item_color(track_item, version_doc) return phiero.update_container(track_item, data_imprint) @@ -166,21 +170,14 @@ class LoadClip(phiero.SequenceLoader): cls.sequence = cls.track.parent() @classmethod - def set_item_color(cls, track_item, version): - + def set_item_color(cls, track_item, version_doc): + project_name = legacy_io.active_project() + last_version_doc = get_last_version_by_subset_id( + project_name, version_doc["parent"], fields=["_id"] + ) clip = track_item.source() - # define version name - version_name = version.get("name", None) - # get all versions in list - versions = legacy_io.find({ - "type": "version", - "parent": version["parent"] - }).distinct('name') - - max_version = max(versions) - # set clip colour - if version_name == max_version: + if version_doc["_id"] == last_version_doc["_id"]: clip.binItem().setColor(cls.clip_color_last) else: clip.binItem().setColor(cls.clip_color) diff --git a/openpype/hosts/hiero/plugins/publish_old_workflow/collect_assetbuilds.py b/openpype/hosts/hiero/plugins/publish_old_workflow/collect_assetbuilds.py index 10baf25803..5f96533052 100644 --- a/openpype/hosts/hiero/plugins/publish_old_workflow/collect_assetbuilds.py +++ b/openpype/hosts/hiero/plugins/publish_old_workflow/collect_assetbuilds.py @@ -1,4 +1,5 @@ from pyblish import api +from openpype.client import get_assets from openpype.pipeline import legacy_io @@ -17,8 +18,9 @@ class CollectAssetBuilds(api.ContextPlugin): hosts = ["hiero"] def process(self, context): + project_name = legacy_io.active_project() asset_builds = {} - for asset in legacy_io.find({"type": "asset"}): + for asset in get_assets(project_name): if asset["data"]["entityType"] == "AssetBuild": self.log.debug("Found \"{}\" in database.".format(asset)) asset_builds[asset["name"]] = asset diff --git a/openpype/hosts/houdini/api/lib.py b/openpype/hosts/houdini/api/lib.py index 603519069a..96ca019f8f 100644 --- a/openpype/hosts/houdini/api/lib.py +++ b/openpype/hosts/houdini/api/lib.py @@ -130,6 +130,8 @@ def get_output_parameter(node): elif node_type == "arnold": if node.evalParm("ar_ass_export_enable"): return node.parm("ar_ass_file") + elif node_type == "Redshift_Proxy_Output": + return node.parm("RS_archive_file") raise TypeError("Node type '%s' not supported" % node_type) diff --git a/openpype/hosts/houdini/plugins/create/create_redshift_proxy.py b/openpype/hosts/houdini/plugins/create/create_redshift_proxy.py new file mode 100644 index 0000000000..da4d80bf2b --- /dev/null +++ b/openpype/hosts/houdini/plugins/create/create_redshift_proxy.py @@ -0,0 +1,48 @@ +from openpype.hosts.houdini.api import plugin + + +class CreateRedshiftProxy(plugin.Creator): + """Redshift Proxy""" + + label = "Redshift Proxy" + family = "redshiftproxy" + icon = "magic" + + def __init__(self, *args, **kwargs): + super(CreateRedshiftProxy, self).__init__(*args, **kwargs) + + # Remove the active, we are checking the bypass flag of the nodes + self.data.pop("active", None) + + # Redshift provides a `Redshift_Proxy_Output` node type which shows + # a limited set of parameters by default and is set to extract a + # Redshift Proxy. However when "imprinting" extra parameters needed + # for OpenPype it starts showing all its parameters again. It's unclear + # why this happens. + # TODO: Somehow enforce so that it only shows the original limited + # attributes of the Redshift_Proxy_Output node type + self.data.update({"node_type": "Redshift_Proxy_Output"}) + + def _process(self, instance): + """Creator main entry point. + + Args: + instance (hou.Node): Created Houdini instance. + + """ + parms = { + "RS_archive_file": '$HIP/pyblish/`chs("subset")`.$F4.rs', + } + + if self.nodes: + node = self.nodes[0] + path = node.path() + parms["RS_archive_sopPath"] = path + + instance.setParms(parms) + + # Lock some Avalon attributes + to_lock = ["family", "id"] + for name in to_lock: + parm = instance.parm(name) + parm.lock(True) diff --git a/openpype/hosts/houdini/plugins/publish/collect_frames.py b/openpype/hosts/houdini/plugins/publish/collect_frames.py index fac40b4d2b..9bd43d8a09 100644 --- a/openpype/hosts/houdini/plugins/publish/collect_frames.py +++ b/openpype/hosts/houdini/plugins/publish/collect_frames.py @@ -20,7 +20,7 @@ class CollectFrames(pyblish.api.InstancePlugin): order = pyblish.api.CollectorOrder label = "Collect Frames" - families = ["vdbcache", "imagesequence", "ass"] + families = ["vdbcache", "imagesequence", "ass", "redshiftproxy"] def process(self, instance): diff --git a/openpype/hosts/houdini/plugins/publish/collect_output_node.py b/openpype/hosts/houdini/plugins/publish/collect_output_node.py index 938ee81cc3..0130c0a8da 100644 --- a/openpype/hosts/houdini/plugins/publish/collect_output_node.py +++ b/openpype/hosts/houdini/plugins/publish/collect_output_node.py @@ -12,6 +12,7 @@ class CollectOutputSOPPath(pyblish.api.InstancePlugin): "imagesequence", "usd", "usdrender", + "redshiftproxy" ] hosts = ["houdini"] @@ -54,6 +55,8 @@ class CollectOutputSOPPath(pyblish.api.InstancePlugin): else: out_node = node.parm("loppath").evalAsNode() + elif node_type == "Redshift_Proxy_Output": + out_node = node.parm("RS_archive_sopPath").evalAsNode() else: raise ValueError( "ROP node type '%s' is" " not supported." % node_type diff --git a/openpype/hosts/houdini/plugins/publish/extract_redshift_proxy.py b/openpype/hosts/houdini/plugins/publish/extract_redshift_proxy.py new file mode 100644 index 0000000000..c754d60c59 --- /dev/null +++ b/openpype/hosts/houdini/plugins/publish/extract_redshift_proxy.py @@ -0,0 +1,48 @@ +import os + +import pyblish.api +import openpype.api +from openpype.hosts.houdini.api.lib import render_rop + + +class ExtractRedshiftProxy(openpype.api.Extractor): + + order = pyblish.api.ExtractorOrder + 0.1 + label = "Extract Redshift Proxy" + families = ["redshiftproxy"] + hosts = ["houdini"] + + def process(self, instance): + + ropnode = instance[0] + + # Get the filename from the filename parameter + # `.evalParm(parameter)` will make sure all tokens are resolved + output = ropnode.evalParm("RS_archive_file") + staging_dir = os.path.normpath(os.path.dirname(output)) + instance.data["stagingDir"] = staging_dir + file_name = os.path.basename(output) + + self.log.info("Writing Redshift Proxy '%s' to '%s'" % (file_name, + staging_dir)) + + render_rop(ropnode) + + output = instance.data["frames"] + + if "representations" not in instance.data: + instance.data["representations"] = [] + + representation = { + "name": "rs", + "ext": "rs", + "files": output, + "stagingDir": staging_dir, + } + + # A single frame may also be rendered without start/end frame. + if "frameStart" in instance.data and "frameEnd" in instance.data: + representation["frameStart"] = instance.data["frameStart"] + representation["frameEnd"] = instance.data["frameEnd"] + + instance.data["representations"].append(representation) diff --git a/openpype/hosts/maya/api/action.py b/openpype/hosts/maya/api/action.py index ca1006b6aa..90605734e7 100644 --- a/openpype/hosts/maya/api/action.py +++ b/openpype/hosts/maya/api/action.py @@ -3,6 +3,7 @@ from __future__ import absolute_import import pyblish.api +from openpype.client import get_asset_by_name from openpype.pipeline import legacy_io from openpype.api import get_errored_instances_from_context @@ -74,12 +75,21 @@ class GenerateUUIDsOnInvalidAction(pyblish.api.Action): from . import lib - asset = instance.data['asset'] - asset_id = legacy_io.find_one( - {"name": asset, "type": "asset"}, - projection={"_id": True} - )['_id'] - for node, _id in lib.generate_ids(nodes, asset_id=asset_id): + # Expecting this is called on validators in which case 'assetEntity' + # should be always available, but kept a way to query it by name. + asset_doc = instance.data.get("assetEntity") + if not asset_doc: + asset_name = instance.data["asset"] + project_name = legacy_io.active_project() + self.log.info(( + "Asset is not stored on instance." + " Querying by name \"{}\" from project \"{}\"" + ).format(asset_name, project_name)) + asset_doc = get_asset_by_name( + project_name, asset_name, fields=["_id"] + ) + + for node, _id in lib.generate_ids(nodes, asset_id=asset_doc["_id"]): lib.set_id(node, _id, overwrite=True) diff --git a/openpype/hosts/maya/api/commands.py b/openpype/hosts/maya/api/commands.py index dd616b6dd6..355edf3ae4 100644 --- a/openpype/hosts/maya/api/commands.py +++ b/openpype/hosts/maya/api/commands.py @@ -2,6 +2,7 @@ """OpenPype script commands to be used directly in Maya.""" from maya import cmds +from openpype.client import get_asset_by_name, get_project from openpype.pipeline import legacy_io @@ -79,8 +80,9 @@ def reset_frame_range(): cmds.currentUnit(time=fps) # Set frame start/end + project_name = legacy_io.active_project() asset_name = legacy_io.Session["AVALON_ASSET"] - asset = legacy_io.find_one({"name": asset_name, "type": "asset"}) + asset = get_asset_by_name(project_name, asset_name) frame_start = asset["data"].get("frameStart") frame_end = asset["data"].get("frameEnd") @@ -145,8 +147,9 @@ def reset_resolution(): resolution_height = 1080 # Get resolution from asset + project_name = legacy_io.active_project() asset_name = legacy_io.Session["AVALON_ASSET"] - asset_doc = legacy_io.find_one({"name": asset_name, "type": "asset"}) + asset_doc = get_asset_by_name(project_name, asset_name) resolution = _resolution_from_document(asset_doc) # Try get resolution from project if resolution is None: @@ -155,7 +158,7 @@ def reset_resolution(): "Asset \"{}\" does not have set resolution." " Trying to get resolution from project" ).format(asset_name)) - project_doc = legacy_io.find_one({"type": "project"}) + project_doc = get_project(project_name) resolution = _resolution_from_document(project_doc) if resolution is None: diff --git a/openpype/hosts/maya/api/lib.py b/openpype/hosts/maya/api/lib.py index bce03a648b..de9a9da911 100644 --- a/openpype/hosts/maya/api/lib.py +++ b/openpype/hosts/maya/api/lib.py @@ -12,11 +12,17 @@ import contextlib from collections import OrderedDict, defaultdict from math import ceil from six import string_types -import bson from maya import cmds, mel import maya.api.OpenMaya as om +from openpype.client import ( + get_project, + get_asset_by_name, + get_subsets, + get_last_versions, + get_representation_by_name +) from openpype import lib from openpype.api import get_anatomy_settings from openpype.pipeline import ( @@ -1387,15 +1393,11 @@ def generate_ids(nodes, asset_id=None): if asset_id is None: # Get the asset ID from the database for the asset of current context - asset_data = legacy_io.find_one( - { - "type": "asset", - "name": legacy_io.Session["AVALON_ASSET"] - }, - projection={"_id": True} - ) - assert asset_data, "No current asset found in Session" - asset_id = asset_data['_id'] + project_name = legacy_io.active_project() + asset_name = legacy_io.Session["AVALON_ASSET"] + asset_doc = get_asset_by_name(project_name, asset_name, fields=["_id"]) + assert asset_doc, "No current asset found in Session" + asset_id = asset_doc['_id'] node_ids = [] for node in nodes: @@ -1548,13 +1550,15 @@ def list_looks(asset_id): # # get all subsets with look leading in # the name associated with the asset - subset = legacy_io.find({ - "parent": bson.ObjectId(asset_id), - "type": "subset", - "name": {"$regex": "look*"} - }) - - return list(subset) + # TODO this should probably look for family 'look' instead of checking + # subset name that can not start with family + project_name = legacy_io.active_project() + subset_docs = get_subsets(project_name, asset_ids=[asset_id]) + return [ + subset_doc + for subset_doc in subset_docs + if subset_doc["name"].startswith("look") + ] def assign_look_by_version(nodes, version_id): @@ -1570,18 +1574,15 @@ def assign_look_by_version(nodes, version_id): None """ - # Get representations of shader file and relationships - look_representation = legacy_io.find_one({ - "type": "representation", - "parent": version_id, - "name": "ma" - }) + project_name = legacy_io.active_project() - json_representation = legacy_io.find_one({ - "type": "representation", - "parent": version_id, - "name": "json" - }) + # Get representations of shader file and relationships + look_representation = get_representation_by_name( + project_name, "ma", version_id + ) + json_representation = get_representation_by_name( + project_name, "json", version_id + ) # See if representation is already loaded, if so reuse it. host = registered_host() @@ -1639,42 +1640,54 @@ def assign_look(nodes, subset="lookDefault"): parts = pype_id.split(":", 1) grouped[parts[0]].append(node) + project_name = legacy_io.active_project() + subset_docs = get_subsets( + project_name, subset_names=[subset], asset_ids=grouped.keys() + ) + subset_docs_by_asset_id = { + str(subset_doc["parent"]): subset_doc + for subset_doc in subset_docs + } + subset_ids = { + subset_doc["_id"] + for subset_doc in subset_docs_by_asset_id.values() + } + last_version_docs = get_last_versions( + project_name, + subset_ids=subset_ids, + fields=["_id", "name", "data.families"] + ) + last_version_docs_by_subset_id = { + last_version_doc["parent"]: last_version_doc + for last_version_doc in last_version_docs + } + for asset_id, asset_nodes in grouped.items(): # create objectId for database - try: - asset_id = bson.ObjectId(asset_id) - except bson.errors.InvalidId: - log.warning("Asset ID is not compatible with bson") - continue - subset_data = legacy_io.find_one({ - "type": "subset", - "name": subset, - "parent": asset_id - }) - - if not subset_data: + subset_doc = subset_docs_by_asset_id.get(asset_id) + if not subset_doc: log.warning("No subset '{}' found for {}".format(subset, asset_id)) continue - # get last version - # with backwards compatibility - version = legacy_io.find_one( - { - "parent": subset_data['_id'], - "type": "version", - "data.families": {"$in": ["look"]} - }, - sort=[("name", -1)], - projection={ - "_id": True, - "name": True - } - ) + last_version = last_version_docs_by_subset_id.get(subset_doc["_id"]) + if not last_version: + log.warning(( + "Not found last version for subset '{}' on asset with id {}" + ).format(subset, asset_id)) + continue - log.debug("Assigning look '{}' ".format(subset, - version["name"])) + families = last_version.get("data", {}).get("families") or [] + if "look" not in families: + log.warning(( + "Last version for subset '{}' on asset with id {}" + " does not have look family" + ).format(subset, asset_id)) + continue - assign_look_by_version(asset_nodes, version['_id']) + log.debug("Assigning look '{}' ".format( + subset, last_version["name"])) + + assign_look_by_version(asset_nodes, last_version["_id"]) def apply_shaders(relationships, shadernodes, nodes): @@ -2126,9 +2139,11 @@ def set_scene_resolution(width, height, pixelAspect): control_node = "defaultResolution" current_renderer = cmds.getAttr("defaultRenderGlobals.currentRenderer") + aspect_ratio_attr = "deviceAspectRatio" # Give VRay a helping hand as it is slightly different from the rest if current_renderer == "vray": + aspect_ratio_attr = "aspectRatio" vray_node = "vraySettings" if cmds.objExists(vray_node): control_node = vray_node @@ -2141,7 +2156,8 @@ def set_scene_resolution(width, height, pixelAspect): cmds.setAttr("%s.height" % control_node, height) deviceAspectRatio = ((float(width) / float(height)) * float(pixelAspect)) - cmds.setAttr("%s.deviceAspectRatio" % control_node, deviceAspectRatio) + cmds.setAttr( + "{}.{}".format(control_node, aspect_ratio_attr), deviceAspectRatio) cmds.setAttr("%s.pixelAspect" % control_node, pixelAspect) @@ -2155,7 +2171,8 @@ def reset_scene_resolution(): None """ - project_doc = legacy_io.find_one({"type": "project"}) + project_name = legacy_io.active_project() + project_doc = get_project(project_name) project_data = project_doc["data"] asset_data = lib.get_asset()["data"] @@ -2188,7 +2205,8 @@ def set_context_settings(): """ # Todo (Wijnand): apply renderer and resolution of project - project_doc = legacy_io.find_one({"type": "project"}) + project_name = legacy_io.active_project() + project_doc = get_project(project_name) project_data = project_doc["data"] asset_data = lib.get_asset()["data"] diff --git a/openpype/hosts/maya/api/setdress.py b/openpype/hosts/maya/api/setdress.py index f8d3ed79b8..bea8f154b1 100644 --- a/openpype/hosts/maya/api/setdress.py +++ b/openpype/hosts/maya/api/setdress.py @@ -6,10 +6,16 @@ import contextlib import copy import six -from bson.objectid import ObjectId from maya import cmds +from openpype.client import ( + get_version_by_name, + get_last_version_by_subset_id, + get_representation_by_id, + get_representation_by_name, + get_representation_parents, +) from openpype.pipeline import ( schema, legacy_io, @@ -283,36 +289,35 @@ def update_package_version(container, version): """ # Versioning (from `core.maya.pipeline`) - current_representation = legacy_io.find_one({ - "_id": ObjectId(container["representation"]) - }) + project_name = legacy_io.active_project() + current_representation = get_representation_by_id( + project_name, container["representation"] + ) assert current_representation is not None, "This is a bug" - version_, subset, asset, project = legacy_io.parenthood( - current_representation + repre_parents = get_representation_parents( + project_name, current_representation ) + version_doc = subset_doc = asset_doc = project_doc = None + if repre_parents: + version_doc, subset_doc, asset_doc, project_doc = repre_parents if version == -1: - new_version = legacy_io.find_one({ - "type": "version", - "parent": subset["_id"] - }, sort=[("name", -1)]) + new_version = get_last_version_by_subset_id( + project_name, subset_doc["_id"] + ) else: - new_version = legacy_io.find_one({ - "type": "version", - "parent": subset["_id"], - "name": version, - }) + new_version = get_version_by_name( + project_name, version, subset_doc["_id"] + ) assert new_version is not None, "This is a bug" # Get the new representation (new file) - new_representation = legacy_io.find_one({ - "type": "representation", - "parent": new_version["_id"], - "name": current_representation["name"] - }) + new_representation = get_representation_by_name( + project_name, current_representation["name"], new_version["_id"] + ) update_package(container, new_representation) @@ -330,10 +335,10 @@ def update_package(set_container, representation): """ # Load the original package data - current_representation = legacy_io.find_one({ - "_id": ObjectId(set_container['representation']), - "type": "representation" - }) + project_name = legacy_io.active_project() + current_representation = get_representation_by_id( + project_name, set_container["representation"] + ) current_file = get_representation_path(current_representation) assert current_file.endswith(".json") @@ -380,6 +385,7 @@ def update_scene(set_container, containers, current_data, new_data, new_file): from openpype.hosts.maya.lib import DEFAULT_MATRIX, get_container_transforms set_namespace = set_container['namespace'] + project_name = legacy_io.active_project() # Update the setdress hierarchy alembic set_root = get_container_transforms(set_container, root=True) @@ -481,12 +487,12 @@ def update_scene(set_container, containers, current_data, new_data, new_file): # Check whether the conversion can be done by the Loader. # They *must* use the same asset, subset and Loader for # `update_container` to make sense. - old = legacy_io.find_one({ - "_id": ObjectId(representation_current) - }) - new = legacy_io.find_one({ - "_id": ObjectId(representation_new) - }) + old = get_representation_by_id( + project_name, representation_current + ) + new = get_representation_by_id( + project_name, representation_new + ) is_valid = compare_representations(old=old, new=new) if not is_valid: log.error("Skipping: %s. See log for details.", diff --git a/openpype/hosts/maya/plugins/create/create_multiverse_usd.py b/openpype/hosts/maya/plugins/create/create_multiverse_usd.py index 034714d51b..5290d5143f 100644 --- a/openpype/hosts/maya/plugins/create/create_multiverse_usd.py +++ b/openpype/hosts/maya/plugins/create/create_multiverse_usd.py @@ -16,7 +16,7 @@ class CreateMultiverseUsd(plugin.Creator): self.data.update(lib.collect_animation_data(True)) self.data["fileFormat"] = ["usd", "usda", "usdz"] - self.data["stripNamespaces"] = False + self.data["stripNamespaces"] = True self.data["mergeTransformAndShape"] = False self.data["writeAncestors"] = True self.data["flattenParentXforms"] = False @@ -37,15 +37,15 @@ class CreateMultiverseUsd(plugin.Creator): self.data["writeUVs"] = True self.data["writeColorSets"] = False self.data["writeTangents"] = False - self.data["writeRefPositions"] = False + self.data["writeRefPositions"] = True self.data["writeBlendShapes"] = False - self.data["writeDisplayColor"] = False + self.data["writeDisplayColor"] = True self.data["writeSkinWeights"] = False self.data["writeMaterialAssignment"] = False self.data["writeHardwareShader"] = False self.data["writeShadingNetworks"] = False self.data["writeTransformMatrix"] = True - self.data["writeUsdAttributes"] = False + self.data["writeUsdAttributes"] = True self.data["writeInstancesAsReferences"] = False self.data["timeVaryingTopology"] = False self.data["customMaterialNamespace"] = '' diff --git a/openpype/hosts/maya/plugins/create/create_yeti_cache.py b/openpype/hosts/maya/plugins/create/create_yeti_cache.py index 86e13b95b2..e8c3203f21 100644 --- a/openpype/hosts/maya/plugins/create/create_yeti_cache.py +++ b/openpype/hosts/maya/plugins/create/create_yeti_cache.py @@ -22,7 +22,8 @@ class CreateYetiCache(plugin.Creator): # Add animation data without step and handles anim_data = lib.collect_animation_data() anim_data.pop("step") - anim_data.pop("handles") + anim_data.pop("handleStart") + anim_data.pop("handleEnd") self.data.update(anim_data) # Add samples diff --git a/openpype/hosts/maya/plugins/inventory/import_modelrender.py b/openpype/hosts/maya/plugins/inventory/import_modelrender.py index a5367f16e5..8a7390bc8d 100644 --- a/openpype/hosts/maya/plugins/inventory/import_modelrender.py +++ b/openpype/hosts/maya/plugins/inventory/import_modelrender.py @@ -1,6 +1,10 @@ +import re import json -from bson.objectid import ObjectId +from openpype.client import ( + get_representation_by_id, + get_representations +) from openpype.pipeline import ( InventoryAction, get_representation_context, @@ -31,6 +35,7 @@ class ImportModelRender(InventoryAction): def process(self, containers): from maya import cmds + project_name = legacy_io.active_project() for container in containers: con_name = container["objectName"] nodes = [] @@ -40,9 +45,9 @@ class ImportModelRender(InventoryAction): else: nodes.append(n) - repr_doc = legacy_io.find_one({ - "_id": ObjectId(container["representation"]), - }) + repr_doc = get_representation_by_id( + project_name, container["representation"], fields=["parent"] + ) version_id = repr_doc["parent"] print("Importing render sets for model %r" % con_name) @@ -63,26 +68,38 @@ class ImportModelRender(InventoryAction): from maya import cmds + project_name = legacy_io.active_project() + repre_docs = get_representations( + project_name, version_ids=[version_id], fields=["_id", "name"] + ) # Get representations of shader file and relationships - look_repr = legacy_io.find_one({ - "type": "representation", - "parent": version_id, - "name": {"$regex": self.scene_type_regex}, - }) - if not look_repr: + json_repre = None + look_repres = [] + scene_type_regex = re.compile(self.scene_type_regex) + for repre_doc in repre_docs: + repre_name = repre_doc["name"] + if repre_name == self.look_data_type: + json_repre = repre_doc + continue + + if scene_type_regex.fullmatch(repre_name): + look_repres.append(repre_doc) + + # QUESTION should we care if there is more then one look + # representation? (since it's based on regex match) + look_repre = None + if look_repres: + look_repre = look_repres[0] + + # QUESTION shouldn't be json representation validated too? + if not look_repre: print("No model render sets for this model version..") return - json_repr = legacy_io.find_one({ - "type": "representation", - "parent": version_id, - "name": self.look_data_type, - }) - - context = get_representation_context(look_repr["_id"]) + context = get_representation_context(look_repre["_id"]) maya_file = self.filepath_from_context(context) - context = get_representation_context(json_repr["_id"]) + context = get_representation_context(json_repre["_id"]) json_file = self.filepath_from_context(context) # Import the look file diff --git a/openpype/hosts/maya/plugins/load/load_audio.py b/openpype/hosts/maya/plugins/load/load_audio.py index ce814e1299..6f60cb5726 100644 --- a/openpype/hosts/maya/plugins/load/load_audio.py +++ b/openpype/hosts/maya/plugins/load/load_audio.py @@ -1,5 +1,10 @@ from maya import cmds, mel +from openpype.client import ( + get_asset_by_id, + get_subset_by_id, + get_version_by_id, +) from openpype.pipeline import ( legacy_io, load, @@ -65,9 +70,16 @@ class AudioLoader(load.LoaderPlugin): ) # Set frame range. - version = legacy_io.find_one({"_id": representation["parent"]}) - subset = legacy_io.find_one({"_id": version["parent"]}) - asset = legacy_io.find_one({"_id": subset["parent"]}) + project_name = legacy_io.active_project() + version = get_version_by_id( + project_name, representation["parent"], fields=["parent"] + ) + subset = get_subset_by_id( + project_name, version["parent"], fields=["parent"] + ) + asset = get_asset_by_id( + project_name, subset["parent"], fields=["parent"] + ) audio_node.sourceStart.set(1 - asset["data"]["frameStart"]) audio_node.sourceEnd.set(asset["data"]["frameEnd"]) diff --git a/openpype/hosts/maya/plugins/load/load_image_plane.py b/openpype/hosts/maya/plugins/load/load_image_plane.py index 5e44917f28..b267921bdc 100644 --- a/openpype/hosts/maya/plugins/load/load_image_plane.py +++ b/openpype/hosts/maya/plugins/load/load_image_plane.py @@ -1,5 +1,10 @@ from Qt import QtWidgets, QtCore +from openpype.client import ( + get_asset_by_id, + get_subset_by_id, + get_version_by_id, +) from openpype.pipeline import ( legacy_io, load, @@ -216,9 +221,16 @@ class ImagePlaneLoader(load.LoaderPlugin): ) # Set frame range. - version = legacy_io.find_one({"_id": representation["parent"]}) - subset = legacy_io.find_one({"_id": version["parent"]}) - asset = legacy_io.find_one({"_id": subset["parent"]}) + project_name = legacy_io.active_project() + version = get_version_by_id( + project_name, representation["parent"], fields=["parent"] + ) + subset = get_subset_by_id( + project_name, version["parent"], fields=["parent"] + ) + asset = get_asset_by_id( + project_name, subset["parent"], fields=["parent"] + ) start_frame = asset["data"]["frameStart"] end_frame = asset["data"]["frameEnd"] image_plane_shape.frameOffset.set(1 - start_frame) diff --git a/openpype/hosts/maya/plugins/load/load_look.py b/openpype/hosts/maya/plugins/load/load_look.py index ae3a683241..7392adc4dd 100644 --- a/openpype/hosts/maya/plugins/load/load_look.py +++ b/openpype/hosts/maya/plugins/load/load_look.py @@ -5,6 +5,7 @@ from collections import defaultdict from Qt import QtWidgets +from openpype.client import get_representation_by_name from openpype.pipeline import ( legacy_io, get_representation_path, @@ -75,11 +76,10 @@ class LookLoader(openpype.hosts.maya.api.plugin.ReferenceLoader): shader_nodes = cmds.ls(members, type='shadingEngine') nodes = set(self._get_nodes_with_shader(shader_nodes)) - json_representation = legacy_io.find_one({ - "type": "representation", - "parent": representation['parent'], - "name": "json" - }) + project_name = legacy_io.active_project() + json_representation = get_representation_by_name( + project_name, "json", representation["parent"] + ) # Load relationships shader_relation = get_representation_path(json_representation) diff --git a/openpype/hosts/maya/plugins/load/load_vrayproxy.py b/openpype/hosts/maya/plugins/load/load_vrayproxy.py index 22d56139f6..e3d6166d3a 100644 --- a/openpype/hosts/maya/plugins/load/load_vrayproxy.py +++ b/openpype/hosts/maya/plugins/load/load_vrayproxy.py @@ -7,10 +7,9 @@ loader will use them instead of native vray vrmesh format. """ import os -from bson.objectid import ObjectId - import maya.cmds as cmds +from openpype.client import get_representation_by_name from openpype.api import get_project_settings from openpype.pipeline import ( legacy_io, @@ -185,12 +184,8 @@ class VRayProxyLoader(load.LoaderPlugin): """ self.log.debug( "Looking for abc in published representations of this version.") - abc_rep = legacy_io.find_one({ - "type": "representation", - "parent": ObjectId(version_id), - "name": "abc" - }) - + project_name = legacy_io.active_project() + abc_rep = get_representation_by_name(project_name, "abc", version_id) if abc_rep: self.log.debug("Found, we'll link alembic to vray proxy.") file_name = get_representation_path(abc_rep) diff --git a/openpype/hosts/maya/plugins/load/load_yeti_cache.py b/openpype/hosts/maya/plugins/load/load_yeti_cache.py index fb903785ae..8435ba2493 100644 --- a/openpype/hosts/maya/plugins/load/load_yeti_cache.py +++ b/openpype/hosts/maya/plugins/load/load_yeti_cache.py @@ -1,15 +1,13 @@ import os import json import re -import glob from collections import defaultdict -from pprint import pprint +import clique from maya import cmds from openpype.api import get_project_settings from openpype.pipeline import ( - legacy_io, load, get_representation_path ) @@ -17,7 +15,15 @@ from openpype.hosts.maya.api import lib from openpype.hosts.maya.api.pipeline import containerise +def set_attribute(node, attr, value): + """Wrapper of set attribute which ignores None values""" + if value is None: + return + lib.set_attribute(node, attr, value) + + class YetiCacheLoader(load.LoaderPlugin): + """Load Yeti Cache with one or more Yeti nodes""" families = ["yeticache", "yetiRig"] representations = ["fur"] @@ -28,6 +34,16 @@ class YetiCacheLoader(load.LoaderPlugin): color = "orange" def load(self, context, name=None, namespace=None, data=None): + """Loads a .fursettings file defining how to load .fur sequences + + A single yeticache or yetiRig can have more than a single pgYetiMaya + nodes and thus load more than a single yeti.fur sequence. + + The .fursettings file defines what the node names should be and also + what "cbId" attribute they should receive to match the original source + and allow published looks to also work for Yeti rigs and its caches. + + """ try: family = context["representation"]["context"]["family"] @@ -43,22 +59,11 @@ class YetiCacheLoader(load.LoaderPlugin): if not cmds.pluginInfo("pgYetiMaya", query=True, loaded=True): cmds.loadPlugin("pgYetiMaya", quiet=True) - # Get JSON - fbase = re.search(r'^(.+)\.(\d+|#+)\.fur', self.fname) - if not fbase: - raise RuntimeError('Cannot determine fursettings file path') - settings_fname = "{}.fursettings".format(fbase.group(1)) - with open(settings_fname, "r") as fp: - fursettings = json.load(fp) - - # Check if resources map exists - # Get node name from JSON - if "nodes" not in fursettings: - raise RuntimeError("Encountered invalid data, expect 'nodes' in " - "fursettings.") - - node_data = fursettings["nodes"] - nodes = self.create_nodes(namespace, node_data) + # Create Yeti cache nodes according to settings + settings = self.read_settings(self.fname) + nodes = [] + for node in settings["nodes"]: + nodes.extend(self.create_node(namespace, node)) group_name = "{}:{}".format(namespace, name) group_node = cmds.group(nodes, name=group_name) @@ -111,28 +116,14 @@ class YetiCacheLoader(load.LoaderPlugin): def update(self, container, representation): - legacy_io.install() namespace = container["namespace"] container_node = container["objectName"] - fur_settings = legacy_io.find_one( - {"parent": representation["parent"], "name": "fursettings"} - ) - - pprint({"parent": representation["parent"], "name": "fursettings"}) - pprint(fur_settings) - assert fur_settings is not None, ( - "cannot find fursettings representation" - ) - - settings_fname = get_representation_path(fur_settings) path = get_representation_path(representation) - # Get all node data - with open(settings_fname, "r") as fp: - settings = json.load(fp) + settings = self.read_settings(path) # Collect scene information of asset - set_members = cmds.sets(container["objectName"], query=True) + set_members = lib.get_container_members(container) container_root = lib.get_container_transforms(container, members=set_members, root=True) @@ -147,7 +138,7 @@ class YetiCacheLoader(load.LoaderPlugin): # Re-assemble metadata with cbId as keys meta_data_lookup = {n["cbId"]: n for n in settings["nodes"]} - # Compare look ups and get the nodes which ar not relevant any more + # Delete nodes by "cbId" that are not in the updated version to_delete_lookup = {cb_id for cb_id in scene_lookup.keys() if cb_id not in meta_data_lookup} if to_delete_lookup: @@ -163,25 +154,18 @@ class YetiCacheLoader(load.LoaderPlugin): fullPath=True) or [] to_remove.extend(shapes + transforms) - # Remove id from look uop + # Remove id from lookup scene_lookup.pop(_id, None) cmds.delete(to_remove) - # replace frame in filename with %04d - RE_frame = re.compile(r"(\d+)(\.fur)$") - file_name = re.sub(RE_frame, r"%04d\g<2>", os.path.basename(path)) - for cb_id, data in meta_data_lookup.items(): - - # Update cache file name - data["attrs"]["cacheFileName"] = os.path.join( - os.path.dirname(path), file_name) + for cb_id, node_settings in meta_data_lookup.items(): if cb_id not in scene_lookup: - + # Create new nodes self.log.info("Creating new nodes ..") - new_nodes = self.create_nodes(namespace, [data]) + new_nodes = self.create_node(namespace, node_settings) cmds.sets(new_nodes, addElement=container_node) cmds.parent(new_nodes, container_root) @@ -218,14 +202,8 @@ class YetiCacheLoader(load.LoaderPlugin): children=True) yeti_node = yeti_nodes[0] - for attr, value in data["attrs"].items(): - # handle empty attribute strings. Those are reported - # as None, so their type is NoneType and this is not - # supported on attributes in Maya. We change it to - # empty string. - if value is None: - value = "" - lib.set_attribute(attr, value, yeti_node) + for attr, value in node_settings["attrs"].items(): + set_attribute(attr, value, yeti_node) cmds.setAttr("{}.representation".format(container_node), str(representation["_id"]), @@ -235,7 +213,6 @@ class YetiCacheLoader(load.LoaderPlugin): self.update(container, representation) # helper functions - def create_namespace(self, asset): """Create a unique namespace Args: @@ -253,100 +230,122 @@ class YetiCacheLoader(load.LoaderPlugin): return namespace - def validate_cache(self, filename, pattern="%04d"): - """Check if the cache has more than 1 frame + def get_cache_node_filepath(self, root, node_name): + """Get the cache file path for one of the yeti nodes. - All caches with more than 1 frame need to be called with `%04d` - If the cache has only one frame we return that file name as we assume + All caches with more than 1 frame need cache file name set with `%04d` + If the cache has only one frame we return the file name as we assume it is a snapshot. + This expects the files to be named after the "node name" through + exports with in Yeti. + Args: - filename(str) - pattern(str) + root(str): Folder containing cache files to search in. + node_name(str): Node name to search cache files for Returns: - str + str: Cache file path value needed for cacheFileName attribute """ - glob_pattern = filename.replace(pattern, "*") + name = node_name.replace(":", "_") + pattern = r"^({name})(\.[0-4]+)?(\.fur)$".format(name=re.escape(name)) - escaped = re.escape(filename) - re_pattern = escaped.replace(pattern, "-?[0-9]+") - - files = glob.glob(glob_pattern) - files = [str(f) for f in files if re.match(re_pattern, f)] + files = [fname for fname in os.listdir(root) if re.match(pattern, + fname)] + if not files: + self.log.error("Could not find cache files for '{}' " + "with pattern {}".format(node_name, pattern)) + return if len(files) == 1: - return files[0] - elif len(files) == 0: - self.log.error("Could not find cache files for '%s'" % filename) + # Single file + return os.path.join(root, files[0]) - return filename + # Get filename for the sequence with padding + collections, remainder = clique.assemble(files) + assert not remainder, "This is a bug" + assert len(collections) == 1, "This is a bug" + collection = collections[0] - def create_nodes(self, namespace, settings): + # Formats name as {head}%d{tail} like cache.%04d.fur + fname = collection.format("{head}{padding}{tail}") + return os.path.join(root, fname) + + def create_node(self, namespace, node_settings): """Create nodes with the correct namespace and settings Args: namespace(str): namespace - settings(list): list of dictionaries + node_settings(dict): Single "nodes" entry from .fursettings file. Returns: - list + list: Created nodes """ - nodes = [] - for node_settings in settings: - # Create pgYetiMaya node - original_node = node_settings["name"] - node_name = "{}:{}".format(namespace, original_node) - yeti_node = cmds.createNode("pgYetiMaya", name=node_name) + # Get original names and ids + orig_transform_name = node_settings["transform"]["name"] + orig_shape_name = node_settings["name"] - # Create transform node - transform_node = node_name.rstrip("Shape") + # Add namespace + transform_name = "{}:{}".format(namespace, orig_transform_name) + shape_name = "{}:{}".format(namespace, orig_shape_name) - lib.set_id(transform_node, node_settings["transform"]["cbId"]) - lib.set_id(yeti_node, node_settings["cbId"]) + # Create pgYetiMaya node + transform_node = cmds.createNode("transform", + name=transform_name) + yeti_node = cmds.createNode("pgYetiMaya", + name=shape_name, + parent=transform_node) - nodes.extend([transform_node, yeti_node]) + lib.set_id(transform_node, node_settings["transform"]["cbId"]) + lib.set_id(yeti_node, node_settings["cbId"]) - # Ensure the node has no namespace identifiers - attributes = node_settings["attrs"] + nodes.extend([transform_node, yeti_node]) - # Check if cache file name is stored + # Update attributes with defaults + attributes = node_settings["attrs"] + attributes.update({ + "viewportDensity": 0.1, + "verbosity": 2, + "fileMode": 1, - # get number of # in path and convert it to C prinf format - # like %04d expected by Yeti - fbase = re.search(r'^(.+)\.(\d+|#+)\.fur', self.fname) - if not fbase: - raise RuntimeError('Cannot determine file path') - padding = len(fbase.group(2)) - if "cacheFileName" not in attributes: - cache = "{}.%0{}d.fur".format(fbase.group(1), padding) + # Fix render stats, like Yeti's own + # ../scripts/pgYetiNode.mel script + "visibleInReflections": True, + "visibleInRefractions": True + }) - self.validate_cache(cache) - attributes["cacheFileName"] = cache + # Apply attributes to pgYetiMaya node + for attr, value in attributes.items(): + set_attribute(attr, value, yeti_node) - # Update attributes with requirements - attributes.update({"viewportDensity": 0.1, - "verbosity": 2, - "fileMode": 1}) - - # Apply attributes to pgYetiMaya node - for attr, value in attributes.items(): - if value is None: - continue - lib.set_attribute(attr, value, yeti_node) - - # Fix for : YETI-6 - # Fixes the render stats (this is literally taken from Perigrene's - # ../scripts/pgYetiNode.mel script) - cmds.setAttr("{}.visibleInReflections".format(yeti_node), True) - cmds.setAttr("{}.visibleInRefractions".format(yeti_node), True) - - # Connect to the time node - cmds.connectAttr("time1.outTime", "%s.currentTime" % yeti_node) + # Connect to the time node + cmds.connectAttr("time1.outTime", "%s.currentTime" % yeti_node) return nodes + + def read_settings(self, path): + """Read .fursettings file and compute some additional attributes""" + + with open(path, "r") as fp: + fur_settings = json.load(fp) + + if "nodes" not in fur_settings: + raise RuntimeError("Encountered invalid data, " + "expected 'nodes' in fursettings.") + + # Compute the cache file name values we want to set for the nodes + root = os.path.dirname(path) + for node in fur_settings["nodes"]: + cache_filename = self.get_cache_node_filepath( + root=root, node_name=node["name"]) + + attrs = node.get("attrs", {}) # allow 'attrs' to not exist + attrs["cacheFileName"] = cache_filename + node["attrs"] = attrs + + return fur_settings diff --git a/openpype/hosts/maya/plugins/publish/collect_instances.py b/openpype/hosts/maya/plugins/publish/collect_instances.py index 433fa9886d..ad1f794680 100644 --- a/openpype/hosts/maya/plugins/publish/collect_instances.py +++ b/openpype/hosts/maya/plugins/publish/collect_instances.py @@ -28,18 +28,19 @@ def get_all_children(nodes): dag = sel.getDagPath(0) iterator.reset(dag) - next(iterator) # ignore self + # ignore self + iterator.next() # noqa: B305 while not iterator.isDone(): path = iterator.fullPathName() if path in traversed: iterator.prune() - next(iterator) + iterator.next() # noqa: B305 continue traversed.add(path) - next(iterator) + iterator.next() # noqa: B305 return list(traversed) diff --git a/openpype/hosts/maya/plugins/publish/collect_look.py b/openpype/hosts/maya/plugins/publish/collect_look.py index dc17ddc605..ec583bcce7 100644 --- a/openpype/hosts/maya/plugins/publish/collect_look.py +++ b/openpype/hosts/maya/plugins/publish/collect_look.py @@ -40,7 +40,7 @@ FILE_NODES = { "aiImage": "filename", - "RedshiftNormalMap": "text0", + "RedshiftNormalMap": "tex0", "PxrBump": "filename", "PxrNormalMap": "filename", @@ -603,6 +603,18 @@ class CollectLook(pyblish.api.InstancePlugin): source, computed_source)) + # renderman allows nodes to have filename attribute empty while + # you can have another incoming connection from different node. + pxr_nodes = set() + if cmds.pluginInfo("RenderMan_for_Maya", query=True, loaded=True): + pxr_nodes = set( + cmds.pluginInfo("RenderMan_for_Maya", + query=True, + dependNode=True) + ) + if not source and cmds.nodeType(node) in pxr_nodes: + self.log.info("Renderman: source is empty, skipping...") + continue # We replace backslashes with forward slashes because V-Ray # can't handle the UDIM files with the backslashes in the # paths as the computed patterns diff --git a/openpype/hosts/maya/plugins/publish/collect_review.py b/openpype/hosts/maya/plugins/publish/collect_review.py index e9e0d74c03..15b89ad53c 100644 --- a/openpype/hosts/maya/plugins/publish/collect_review.py +++ b/openpype/hosts/maya/plugins/publish/collect_review.py @@ -3,6 +3,7 @@ import pymel.core as pm import pyblish.api +from openpype.client import get_subset_by_name from openpype.pipeline import legacy_io @@ -78,11 +79,15 @@ class CollectReview(pyblish.api.InstancePlugin): self.log.debug('isntance data {}'.format(instance.data)) else: legacy_subset_name = task + 'Review' - asset_doc_id = instance.context.data['assetEntity']["_id"] - subsets = legacy_io.find({"type": "subset", - "name": legacy_subset_name, - "parent": asset_doc_id}).distinct("_id") - if len(list(subsets)) > 0: + asset_doc = instance.context.data['assetEntity'] + project_name = legacy_io.active_project() + subset_doc = get_subset_by_name( + project_name, + legacy_subset_name, + asset_doc["_id"], + fields=["_id"] + ) + if subset_doc: self.log.debug("Existing subsets found, keep legacy name.") instance.data['subset'] = legacy_subset_name diff --git a/openpype/hosts/maya/plugins/publish/collect_yeti_rig.py b/openpype/hosts/maya/plugins/publish/collect_yeti_rig.py index 029432223b..bc15edd9e0 100644 --- a/openpype/hosts/maya/plugins/publish/collect_yeti_rig.py +++ b/openpype/hosts/maya/plugins/publish/collect_yeti_rig.py @@ -43,11 +43,12 @@ class CollectYetiRig(pyblish.api.InstancePlugin): instance.data["resources"] = yeti_resources - # Force frame range for export - instance.data["frameStart"] = cmds.playbackOptions( - query=True, animationStartTime=True) - instance.data["frameEnd"] = cmds.playbackOptions( - query=True, animationStartTime=True) + # Force frame range for yeti cache export for the rig + start = cmds.playbackOptions(query=True, animationStartTime=True) + for key in ["frameStart", "frameEnd", + "frameStartHandle", "frameEndHandle"]: + instance.data[key] = start + instance.data["preroll"] = 0 def collect_input_connections(self, instance): """Collect the inputs for all nodes in the input_SET""" diff --git a/openpype/hosts/maya/plugins/publish/extract_camera_alembic.py b/openpype/hosts/maya/plugins/publish/extract_camera_alembic.py index 5ad6b79d5c..4110ad474d 100644 --- a/openpype/hosts/maya/plugins/publish/extract_camera_alembic.py +++ b/openpype/hosts/maya/plugins/publish/extract_camera_alembic.py @@ -34,7 +34,6 @@ class ExtractCameraAlembic(openpype.api.Extractor): dag=True, type="camera") # validate required settings - assert len(cameras) == 1, "Not a single camera found in extraction" assert isinstance(step, float), "Step must be a float value" camera = cameras[0] @@ -44,8 +43,12 @@ class ExtractCameraAlembic(openpype.api.Extractor): path = os.path.join(dir_path, filename) # Perform alembic extraction + member_shapes = cmds.ls( + members, leaf=True, shapes=True, long=True, dag=True) with lib.maintained_selection(): - cmds.select(camera, replace=True, noExpand=True) + cmds.select( + member_shapes, + replace=True, noExpand=True) # Enforce forward slashes for AbcExport because we're # embedding it into a job string @@ -57,10 +60,12 @@ class ExtractCameraAlembic(openpype.api.Extractor): job_str += ' -step {0} '.format(step) if bake_to_worldspace: - transform = cmds.listRelatives(camera, - parent=True, - fullPath=True)[0] - job_str += ' -worldSpace -root {0}'.format(transform) + job_str += ' -worldSpace' + for member in member_shapes: + self.log.info(f"processing {member}") + transform = cmds.listRelatives( + member, parent=True, fullPath=True)[0] + job_str += ' -root {0}'.format(transform) job_str += ' -file "{0}"'.format(path) diff --git a/openpype/hosts/maya/plugins/publish/extract_camera_mayaScene.py b/openpype/hosts/maya/plugins/publish/extract_camera_mayaScene.py index 49c156f9cd..1cb30e65ea 100644 --- a/openpype/hosts/maya/plugins/publish/extract_camera_mayaScene.py +++ b/openpype/hosts/maya/plugins/publish/extract_camera_mayaScene.py @@ -131,12 +131,12 @@ class ExtractCameraMayaScene(openpype.api.Extractor): "bake to world space is ignored...") # get cameras - members = instance.data['setMembers'] + members = cmds.ls(instance.data['setMembers'], leaf=True, shapes=True, + long=True, dag=True) cameras = cmds.ls(members, leaf=True, shapes=True, long=True, dag=True, type="camera") # validate required settings - assert len(cameras) == 1, "Single camera must be found in extraction" assert isinstance(step, float), "Step must be a float value" camera = cameras[0] transform = cmds.listRelatives(camera, parent=True, fullPath=True) @@ -158,15 +158,24 @@ class ExtractCameraMayaScene(openpype.api.Extractor): frame_range=[start, end], step=step ) - baked_shapes = cmds.ls(baked, + baked_camera_shapes = cmds.ls(baked, type="camera", dag=True, shapes=True, long=True) + + members = members + baked_camera_shapes + members.remove(camera) else: - baked_shapes = cameras + baked_camera_shapes = cmds.ls(cameras, + type="camera", + dag=True, + shapes=True, + long=True) # Fix PLN-178: Don't allow background color to be non-black - for cam in baked_shapes: + for cam in cmds.ls( + baked_camera_shapes, type="camera", dag=True, + shapes=True, long=True): attrs = {"backgroundColorR": 0.0, "backgroundColorG": 0.0, "backgroundColorB": 0.0, @@ -177,7 +186,8 @@ class ExtractCameraMayaScene(openpype.api.Extractor): cmds.setAttr(plug, value) self.log.info("Performing extraction..") - cmds.select(baked_shapes, noExpand=True) + cmds.select(cmds.ls(members, dag=True, + shapes=True, long=True), noExpand=True) cmds.file(path, force=True, typ="mayaAscii" if self.scene_type == "ma" else "mayaBinary", # noqa: E501 diff --git a/openpype/hosts/maya/plugins/publish/extract_playblast.py b/openpype/hosts/maya/plugins/publish/extract_playblast.py index bb1ecf279d..ba939d5428 100644 --- a/openpype/hosts/maya/plugins/publish/extract_playblast.py +++ b/openpype/hosts/maya/plugins/publish/extract_playblast.py @@ -111,7 +111,8 @@ class ExtractPlayblast(openpype.api.Extractor): self.log.debug("playblast path {}".format(path)) collected_files = os.listdir(stagingdir) - collections, remainder = clique.assemble(collected_files) + collections, remainder = clique.assemble(collected_files, + minimum_items=1) self.log.debug("filename {}".format(filename)) frame_collection = None @@ -134,10 +135,15 @@ class ExtractPlayblast(openpype.api.Extractor): # Add camera node name to representation data camera_node_name = pm.ls(camera)[0].getTransform().name() + collected_files = list(frame_collection) + # single frame file shouldn't be in list, only as a string + if len(collected_files) == 1: + collected_files = collected_files[0] + representation = { 'name': 'png', 'ext': 'png', - 'files': list(frame_collection), + 'files': collected_files, "stagingDir": stagingdir, "frameStart": start, "frameEnd": end, diff --git a/openpype/hosts/maya/plugins/publish/extract_yeti_cache.py b/openpype/hosts/maya/plugins/publish/extract_yeti_cache.py index 0d85708789..cf6db00e9a 100644 --- a/openpype/hosts/maya/plugins/publish/extract_yeti_cache.py +++ b/openpype/hosts/maya/plugins/publish/extract_yeti_cache.py @@ -25,13 +25,10 @@ class ExtractYetiCache(openpype.api.Extractor): # Define extract output file path dirname = self.staging_dir(instance) - # Yeti related staging dirs - data_file = os.path.join(dirname, "yeti.fursettings") - # Collect information for writing cache - start_frame = instance.data.get("frameStartHandle") - end_frame = instance.data.get("frameEndHandle") - preroll = instance.data.get("preroll") + start_frame = instance.data["frameStartHandle"] + end_frame = instance.data["frameEndHandle"] + preroll = instance.data["preroll"] if preroll > 0: start_frame -= preroll @@ -57,32 +54,35 @@ class ExtractYetiCache(openpype.api.Extractor): cache_files = [x for x in os.listdir(dirname) if x.endswith(".fur")] self.log.info("Writing metadata file") - settings = instance.data.get("fursettings", None) - if settings is not None: - with open(data_file, "w") as fp: - json.dump(settings, fp, ensure_ascii=False) + settings = instance.data["fursettings"] + fursettings_path = os.path.join(dirname, "yeti.fursettings") + with open(fursettings_path, "w") as fp: + json.dump(settings, fp, ensure_ascii=False) # build representations if "representations" not in instance.data: instance.data["representations"] = [] self.log.info("cache files: {}".format(cache_files[0])) - instance.data["representations"].append( - { - 'name': 'fur', - 'ext': 'fur', - 'files': cache_files[0] if len(cache_files) == 1 else cache_files, - 'stagingDir': dirname, - 'frameStart': int(start_frame), - 'frameEnd': int(end_frame) - } - ) + + # Workaround: We do not explicitly register these files with the + # representation solely so that we can write multiple sequences + # a single Subset without renaming - it's a bit of a hack + # TODO: Implement better way to manage this sort of integration + if 'transfers' not in instance.data: + instance.data['transfers'] = [] + + publish_dir = instance.data["publishDir"] + for cache_filename in cache_files: + src = os.path.join(dirname, cache_filename) + dst = os.path.join(publish_dir, os.path.basename(cache_filename)) + instance.data['transfers'].append([src, dst]) instance.data["representations"].append( { - 'name': 'fursettings', + 'name': 'fur', 'ext': 'fursettings', - 'files': os.path.basename(data_file), + 'files': os.path.basename(fursettings_path), 'stagingDir': dirname } ) diff --git a/openpype/hosts/maya/plugins/publish/extract_yeti_rig.py b/openpype/hosts/maya/plugins/publish/extract_yeti_rig.py index d12567a55a..6e21bffa4e 100644 --- a/openpype/hosts/maya/plugins/publish/extract_yeti_rig.py +++ b/openpype/hosts/maya/plugins/publish/extract_yeti_rig.py @@ -124,8 +124,8 @@ class ExtractYetiRig(openpype.api.Extractor): settings_path = os.path.join(dirname, "yeti.rigsettings") # Yeti related staging dirs - maya_path = os.path.join( - dirname, "yeti_rig.{}".format(self.scene_type)) + maya_path = os.path.join(dirname, + "yeti_rig.{}".format(self.scene_type)) self.log.info("Writing metadata file") @@ -157,7 +157,7 @@ class ExtractYetiRig(openpype.api.Extractor): input_set = next(i for i in instance if i == "input_SET") # Get all items - set_members = cmds.sets(input_set, query=True) + set_members = cmds.sets(input_set, query=True) or [] set_members += cmds.listRelatives(set_members, allDescendents=True, fullPath=True) or [] @@ -167,7 +167,7 @@ class ExtractYetiRig(openpype.api.Extractor): resources = instance.data.get("resources", {}) with disconnect_plugs(settings, members): with yetigraph_attribute_values(resources_dir, resources): - with maya.attribute_values(attr_value): + with lib.attribute_values(attr_value): cmds.select(nodes, noExpand=True) cmds.file(maya_path, force=True, diff --git a/openpype/hosts/maya/plugins/publish/validate_camera_contents.py b/openpype/hosts/maya/plugins/publish/validate_camera_contents.py index 20af8d2315..87712a4cea 100644 --- a/openpype/hosts/maya/plugins/publish/validate_camera_contents.py +++ b/openpype/hosts/maya/plugins/publish/validate_camera_contents.py @@ -20,6 +20,7 @@ class ValidateCameraContents(pyblish.api.InstancePlugin): hosts = ['maya'] label = 'Camera Contents' actions = [openpype.hosts.maya.api.action.SelectInvalidAction] + validate_shapes = True @classmethod def get_invalid(cls, instance): @@ -32,7 +33,7 @@ class ValidateCameraContents(pyblish.api.InstancePlugin): invalid = [] cameras = cmds.ls(shapes, type='camera', long=True) if len(cameras) != 1: - cls.log.warning("Camera instance must have a single camera. " + cls.log.error("Camera instance must have a single camera. " "Found {0}: {1}".format(len(cameras), cameras)) invalid.extend(cameras) @@ -49,15 +50,32 @@ class ValidateCameraContents(pyblish.api.InstancePlugin): raise RuntimeError("No cameras found in empty instance.") + if not cls.validate_shapes: + cls.log.info("Not validating shapes in the content.") + + for member in members: + parents = cmds.ls(member, long=True)[0].split("|")[1:-1] + parents_long_named = [ + "|".join(parents[:i]) for i in range(1, 1 + len(parents)) + ] + if cameras[0] in parents_long_named: + cls.log.error( + "{} is parented under camera {}".format( + member, cameras[0])) + invalid.extend(member) + return invalid + # non-camera shapes valid_shapes = cmds.ls(shapes, type=('camera', 'locator'), long=True) shapes = set(shapes) - set(valid_shapes) if shapes: shapes = list(shapes) - cls.log.warning("Camera instance should only contain camera " + cls.log.error("Camera instance should only contain camera " "shapes. Found: {0}".format(shapes)) invalid.extend(shapes) + + invalid = list(set(invalid)) return invalid diff --git a/openpype/hosts/maya/plugins/publish/validate_mesh_shader_connections.py b/openpype/hosts/maya/plugins/publish/validate_mesh_shader_connections.py index 0969573a90..e0835000f0 100644 --- a/openpype/hosts/maya/plugins/publish/validate_mesh_shader_connections.py +++ b/openpype/hosts/maya/plugins/publish/validate_mesh_shader_connections.py @@ -12,28 +12,41 @@ def pairs(iterable): yield i, y -def get_invalid_sets(shape): - """Get sets that are considered related but do not contain the shape. +def get_invalid_sets(shapes): + """Return invalid sets for the given shapes. - In some scenarios Maya keeps connections to multiple shaders - even if just a single one is assigned on the full object. + This takes a list of shape nodes to cache the set members for overlapping + sets in the queries. This avoids many Maya set member queries. - These are related sets returned by `maya.cmds.listSets` that don't - actually have the shape as member. + Returns: + dict: Dictionary of shapes and their invalid sets, e.g. + {"pCubeShape": ["set1", "set2"]} """ - invalid = [] - sets = cmds.listSets(object=shape, t=1, extendToShape=False) or [] - for s in sets: - members = cmds.sets(s, query=True, nodesOnly=True) - if not members: - invalid.append(s) - continue + cache = dict() + invalid = dict() - members = set(cmds.ls(members, long=True)) - if shape not in members: - invalid.append(s) + # Collect the sets from the shape + for shape in shapes: + invalid_sets = [] + sets = cmds.listSets(object=shape, t=1, extendToShape=False) or [] + for set_ in sets: + + members = cache.get(set_, None) + if members is None: + members = set(cmds.ls(cmds.sets(set_, + query=True, + nodesOnly=True), long=True)) + cache[set_] = members + + # If the shape is not actually present as a member of the set + # consider it invalid + if shape not in members: + invalid_sets.append(set_) + + if invalid_sets: + invalid[shape] = invalid_sets return invalid @@ -92,15 +105,9 @@ class ValidateMeshShaderConnections(pyblish.api.InstancePlugin): @staticmethod def get_invalid(instance): - shapes = cmds.ls(instance[:], dag=1, leaf=1, shapes=1, long=True) - - # todo: allow to check anything that can have a shader - shapes = cmds.ls(shapes, noIntermediate=True, long=True, type="mesh") - - invalid = [] - for shape in shapes: - if get_invalid_sets(shape): - invalid.append(shape) + nodes = instance[:] + shapes = cmds.ls(nodes, noIntermediate=True, long=True, type="mesh") + invalid = get_invalid_sets(shapes).keys() return invalid @@ -108,7 +115,7 @@ class ValidateMeshShaderConnections(pyblish.api.InstancePlugin): def repair(cls, instance): shapes = cls.get_invalid(instance) - for shape in shapes: - invalid_sets = get_invalid_sets(shape) + invalid = get_invalid_sets(shapes) + for shape, invalid_sets in invalid.items(): for set_node in invalid_sets: disconnect(shape, set_node) diff --git a/openpype/hosts/maya/plugins/publish/validate_node_ids_in_database.py b/openpype/hosts/maya/plugins/publish/validate_node_ids_in_database.py index 068d6b38a1..632b531668 100644 --- a/openpype/hosts/maya/plugins/publish/validate_node_ids_in_database.py +++ b/openpype/hosts/maya/plugins/publish/validate_node_ids_in_database.py @@ -1,6 +1,7 @@ import pyblish.api import openpype.api +from openpype.client import get_assets from openpype.pipeline import legacy_io import openpype.hosts.maya.api.action from openpype.hosts.maya.api import lib @@ -42,8 +43,12 @@ class ValidateNodeIdsInDatabase(pyblish.api.InstancePlugin): nodes=instance[:]) # check ids against database ids - db_asset_ids = legacy_io.find({"type": "asset"}).distinct("_id") - db_asset_ids = set(str(i) for i in db_asset_ids) + project_name = legacy_io.active_project() + asset_docs = get_assets(project_name, fields=["_id"]) + db_asset_ids = { + str(asset_doc["_id"]) + for asset_doc in asset_docs + } # Get all asset IDs for node in id_required_nodes: diff --git a/openpype/hosts/maya/plugins/publish/validate_node_ids_related.py b/openpype/hosts/maya/plugins/publish/validate_node_ids_related.py index 38407e4176..c8bac6e569 100644 --- a/openpype/hosts/maya/plugins/publish/validate_node_ids_related.py +++ b/openpype/hosts/maya/plugins/publish/validate_node_ids_related.py @@ -1,7 +1,6 @@ import pyblish.api import openpype.api -from openpype.pipeline import legacy_io import openpype.hosts.maya.api.action from openpype.hosts.maya.api import lib @@ -36,15 +35,7 @@ class ValidateNodeIDsRelated(pyblish.api.InstancePlugin): """Return the member nodes that are invalid""" invalid = list() - asset = instance.data['asset'] - asset_data = legacy_io.find_one( - { - "name": asset, - "type": "asset" - }, - projection={"_id": True} - ) - asset_id = str(asset_data['_id']) + asset_id = str(instance.data['assetEntity']["_id"]) # We do want to check the referenced nodes as we it might be # part of the end product diff --git a/openpype/hosts/maya/plugins/publish/validate_renderlayer_aovs.py b/openpype/hosts/maya/plugins/publish/validate_renderlayer_aovs.py index e65150eb0f..6b6fb03eec 100644 --- a/openpype/hosts/maya/plugins/publish/validate_renderlayer_aovs.py +++ b/openpype/hosts/maya/plugins/publish/validate_renderlayer_aovs.py @@ -1,8 +1,8 @@ import pyblish.api +from openpype.client import get_subset_by_name import openpype.hosts.maya.api.action from openpype.pipeline import legacy_io -import openpype.api class ValidateRenderLayerAOVs(pyblish.api.InstancePlugin): @@ -33,26 +33,23 @@ class ValidateRenderLayerAOVs(pyblish.api.InstancePlugin): raise RuntimeError("Found unregistered subsets: {}".format(invalid)) def get_invalid(self, instance): - invalid = [] - asset_name = instance.data["asset"] + project_name = legacy_io.active_project() + asset_doc = instance.data["assetEntity"] render_passses = instance.data.get("renderPasses", []) for render_pass in render_passses: - is_valid = self.validate_subset_registered(asset_name, render_pass) + is_valid = self.validate_subset_registered( + project_name, asset_doc, render_pass + ) if not is_valid: invalid.append(render_pass) return invalid - def validate_subset_registered(self, asset_name, subset_name): + def validate_subset_registered(self, project_name, asset_doc, subset_name): """Check if subset is registered in the database under the asset""" - asset = legacy_io.find_one({"type": "asset", "name": asset_name}) - is_valid = legacy_io.find_one({ - "type": "subset", - "name": subset_name, - "parent": asset["_id"] - }) - - return is_valid + return get_subset_by_name( + project_name, subset_name, asset_doc["_id"], fields=["_id"] + ) diff --git a/openpype/hosts/maya/plugins/publish/validate_rendersettings.py b/openpype/hosts/maya/plugins/publish/validate_rendersettings.py index ba6c1397ab..1dab3274a0 100644 --- a/openpype/hosts/maya/plugins/publish/validate_rendersettings.py +++ b/openpype/hosts/maya/plugins/publish/validate_rendersettings.py @@ -94,6 +94,7 @@ class ValidateRenderSettings(pyblish.api.InstancePlugin): def get_invalid(cls, instance): invalid = False + multipart = False renderer = instance.data['renderer'] layer = instance.data['setMembers'] @@ -113,6 +114,7 @@ class ValidateRenderSettings(pyblish.api.InstancePlugin): "{aov_separator}", instance.data.get("aovSeparator", "_")) required_prefix = "maya/" + default_prefix = cls.ImagePrefixTokens[renderer] if not anim_override: invalid = True @@ -213,14 +215,16 @@ class ValidateRenderSettings(pyblish.api.InstancePlugin): cls.log.error("Wrong image prefix [ {} ] - " "You can't use '' token " "with merge AOVs turned on".format(prefix)) + default_prefix = re.sub( + cls.R_AOV_TOKEN, "", default_prefix) + # remove aov token from prefix to pass validation + default_prefix = default_prefix.split("{aov_separator}")[0] elif not re.search(cls.R_AOV_TOKEN, prefix): invalid = True cls.log.error("Wrong image prefix [ {} ] - " "doesn't have: '' or " "token".format(prefix)) - # prefix check - default_prefix = cls.ImagePrefixTokens[renderer] default_prefix = default_prefix.replace( "{aov_separator}", instance.data.get("aovSeparator", "_")) if prefix.lower() != default_prefix.lower(): diff --git a/openpype/hosts/nuke/api/__init__.py b/openpype/hosts/nuke/api/__init__.py index b571c4098c..77fe4503d3 100644 --- a/openpype/hosts/nuke/api/__init__.py +++ b/openpype/hosts/nuke/api/__init__.py @@ -26,7 +26,11 @@ from .pipeline import ( update_container, ) from .lib import ( - maintained_selection + maintained_selection, + reset_selection, + get_view_process_node, + duplicate_node + ) from .utils import ( @@ -58,6 +62,9 @@ __all__ = ( "update_container", "maintained_selection", + "reset_selection", + "get_view_process_node", + "duplicate_node", "colorspace_exists_on_node", "get_colorspace_list" diff --git a/openpype/hosts/nuke/api/lib.py b/openpype/hosts/nuke/api/lib.py index 505eb19419..a8e01d0a36 100644 --- a/openpype/hosts/nuke/api/lib.py +++ b/openpype/hosts/nuke/api/lib.py @@ -3,6 +3,7 @@ from pprint import pformat import re import six import platform +import tempfile import contextlib from collections import OrderedDict @@ -711,6 +712,20 @@ def get_imageio_input_colorspace(filename): return preset_clrsp +def get_view_process_node(): + reset_selection() + + ipn_orig = None + for v in nuke.allNodes(filter="Viewer"): + ipn = v['input_process_node'].getValue() + if "VIEWER_INPUT" not in ipn: + ipn_orig = nuke.toNode(ipn) + ipn_orig.setSelected(True) + + if ipn_orig: + return duplicate_node(ipn_orig) + + def on_script_load(): ''' Callback for ffmpeg support ''' @@ -2374,6 +2389,8 @@ def process_workfile_builder(): env_value_to_bool, get_custom_workfile_template ) + # to avoid looping of the callback, remove it! + nuke.removeOnCreate(process_workfile_builder, nodeClass="Root") # get state from settings workfile_builder = get_current_project_settings()["nuke"].get( @@ -2429,9 +2446,6 @@ def process_workfile_builder(): if not openlv_on or not os.path.exists(last_workfile_path): return - # to avoid looping of the callback, remove it! - nuke.removeOnCreate(process_workfile_builder, nodeClass="Root") - log.info("Opening last workfile...") # open workfile open_file(last_workfile_path) @@ -2617,6 +2631,57 @@ class DirmapCache: return cls._sync_module +@contextlib.contextmanager +def _duplicate_node_temp(): + """Create a temp file where node is pasted during duplication. + + This is to avoid using clipboard for node duplication. + """ + + duplicate_node_temp_path = os.path.join( + tempfile.gettempdir(), + "openpype_nuke_duplicate_temp_{}".format(os.getpid()) + ) + + # This can happen only if 'duplicate_node' would be + if os.path.exists(duplicate_node_temp_path): + log.warning(( + "Temp file for node duplication already exists." + " Trying to remove {}" + ).format(duplicate_node_temp_path)) + os.remove(duplicate_node_temp_path) + + try: + # Yield the path where node can be copied + yield duplicate_node_temp_path + + finally: + # Remove the file at the end + os.remove(duplicate_node_temp_path) + + +def duplicate_node(node): + reset_selection() + + # select required node for duplication + node.setSelected(True) + + with _duplicate_node_temp() as filepath: + # copy selected to temp filepath + nuke.nodeCopy(filepath) + + # reset selection + reset_selection() + + # paste node and selection is on it only + dupli_node = nuke.nodePaste(filepath) + + # reset selection + reset_selection() + + return dupli_node + + def dirmap_file_name_filter(file_name): """Nuke callback function with single full path argument. diff --git a/openpype/hosts/nuke/api/plugin.py b/openpype/hosts/nuke/api/plugin.py index b8b56ef2b8..925cab0bef 100644 --- a/openpype/hosts/nuke/api/plugin.py +++ b/openpype/hosts/nuke/api/plugin.py @@ -14,12 +14,12 @@ from openpype.pipeline import ( from .lib import ( Knobby, check_subsetname_exists, - reset_selection, maintained_selection, set_avalon_knob_data, add_publish_knob, get_nuke_imageio_settings, - set_node_knobs_from_settings + set_node_knobs_from_settings, + get_view_process_node ) @@ -216,37 +216,6 @@ class ExporterReview(object): self.data["representations"].append(repre) - def get_view_input_process_node(self): - """ - Will get any active view process. - - Arguments: - self (class): in object definition - - Returns: - nuke.Node: copy node of Input Process node - """ - reset_selection() - ipn_orig = None - for v in nuke.allNodes(filter="Viewer"): - ip = v["input_process"].getValue() - ipn = v["input_process_node"].getValue() - if "VIEWER_INPUT" not in ipn and ip: - ipn_orig = nuke.toNode(ipn) - ipn_orig.setSelected(True) - - if ipn_orig: - # copy selected to clipboard - nuke.nodeCopy("%clipboard%") - # reset selection - reset_selection() - # paste node and selection is on it only - nuke.nodePaste("%clipboard%") - # assign to variable - ipn = nuke.selectedNode() - - return ipn - def get_imageio_baking_profile(self): from . import lib as opnlib nuke_imageio = opnlib.get_nuke_imageio_settings() @@ -311,7 +280,7 @@ class ExporterReviewLut(ExporterReview): self._temp_nodes = [] self.log.info("Deleted nodes...") - def generate_lut(self): + def generate_lut(self, **kwargs): bake_viewer_process = kwargs["bake_viewer_process"] bake_viewer_input_process_node = kwargs[ "bake_viewer_input_process"] @@ -329,7 +298,7 @@ class ExporterReviewLut(ExporterReview): if bake_viewer_process: # Node View Process if bake_viewer_input_process_node: - ipn = self.get_view_input_process_node() + ipn = get_view_process_node() if ipn is not None: # connect ipn.setInput(0, self.previous_node) @@ -511,7 +480,7 @@ class ExporterReviewMov(ExporterReview): if bake_viewer_process: if bake_viewer_input_process_node: # View Process node - ipn = self.get_view_input_process_node() + ipn = get_view_process_node() if ipn is not None: # connect ipn.setInput(0, self.previous_node) diff --git a/openpype/hosts/nuke/plugins/publish/extract_render_local.py b/openpype/hosts/nuke/plugins/publish/extract_render_local.py index 50a5d01483..057bca11ac 100644 --- a/openpype/hosts/nuke/plugins/publish/extract_render_local.py +++ b/openpype/hosts/nuke/plugins/publish/extract_render_local.py @@ -7,7 +7,7 @@ import clique class NukeRenderLocal(openpype.api.Extractor): # TODO: rewrite docstring to nuke - """Render the current Fusion composition locally. + """Render the current Nuke composition locally. Extract the result of savers by starting a comp render This will run the local render of Fusion. diff --git a/openpype/hosts/nuke/plugins/publish/extract_review_data_mov.py b/openpype/hosts/nuke/plugins/publish/extract_review_data_mov.py index 2a79d600ba..5ea7c352b9 100644 --- a/openpype/hosts/nuke/plugins/publish/extract_review_data_mov.py +++ b/openpype/hosts/nuke/plugins/publish/extract_review_data_mov.py @@ -1,4 +1,5 @@ import os +from pprint import pformat import re import pyblish.api import openpype @@ -50,6 +51,8 @@ class ExtractReviewDataMov(openpype.api.Extractor): with maintained_selection(): generated_repres = [] for o_name, o_data in self.outputs.items(): + self.log.debug( + "o_name: {}, o_data: {}".format(o_name, pformat(o_data))) f_families = o_data["filter"]["families"] f_task_types = o_data["filter"]["task_types"] f_subsets = o_data["filter"]["subsets"] @@ -88,7 +91,13 @@ class ExtractReviewDataMov(openpype.api.Extractor): # check if settings have more then one preset # so we dont need to add outputName to representation # in case there is only one preset - multiple_presets = bool(len(self.outputs.keys()) > 1) + multiple_presets = len(self.outputs.keys()) > 1 + + # adding bake presets to instance data for other plugins + if not instance.data.get("bakePresets"): + instance.data["bakePresets"] = {} + # add preset to bakePresets + instance.data["bakePresets"][o_name] = o_data # create exporter instance exporter = plugin.ExporterReviewMov( diff --git a/openpype/hosts/nuke/plugins/publish/extract_slate_frame.py b/openpype/hosts/nuke/plugins/publish/extract_slate_frame.py index fb52fc18b4..e0c4bdb953 100644 --- a/openpype/hosts/nuke/plugins/publish/extract_slate_frame.py +++ b/openpype/hosts/nuke/plugins/publish/extract_slate_frame.py @@ -1,11 +1,16 @@ import os +from pprint import pformat import nuke import copy import pyblish.api import openpype -from openpype.hosts.nuke.api.lib import maintained_selection +from openpype.hosts.nuke.api import ( + maintained_selection, + duplicate_node, + get_view_process_node +) class ExtractSlateFrame(openpype.api.Extractor): @@ -15,14 +20,13 @@ class ExtractSlateFrame(openpype.api.Extractor): """ - order = pyblish.api.ExtractorOrder - 0.001 + order = pyblish.api.ExtractorOrder + 0.011 label = "Extract Slate Frame" families = ["slate"] hosts = ["nuke"] # Settings values - # - can be extended by other attributes from node in the future key_value_mapping = { "f_submission_note": [True, "{comment}"], "f_submitting_for": [True, "{intent[value]}"], @@ -30,44 +34,107 @@ class ExtractSlateFrame(openpype.api.Extractor): } def process(self, instance): - if hasattr(self, "viewer_lut_raw"): - self.viewer_lut_raw = self.viewer_lut_raw - else: - self.viewer_lut_raw = False + + if "representations" not in instance.data: + instance.data["representations"] = [] + + self._create_staging_dir(instance) with maintained_selection(): self.log.debug("instance: {}".format(instance)) self.log.debug("instance.data[families]: {}".format( instance.data["families"])) - self.render_slate(instance) + if instance.data.get("bakePresets"): + for o_name, o_data in instance.data["bakePresets"].items(): + self.log.info("_ o_name: {}, o_data: {}".format( + o_name, pformat(o_data))) + self.render_slate( + instance, + o_name, + o_data["bake_viewer_process"], + o_data["bake_viewer_input_process"] + ) + else: + # backward compatibility + self.render_slate(instance) + + # also render image to sequence + self._render_slate_to_sequence(instance) + + def _create_staging_dir(self, instance): - def render_slate(self, instance): - node_subset_name = instance.data.get("name", None) - node = instance[0] # group node self.log.info("Creating staging dir...") - if "representations" not in instance.data: - instance.data["representations"] = list() - staging_dir = os.path.normpath( - os.path.dirname(instance.data['path'])) + os.path.dirname(instance.data["path"])) instance.data["stagingDir"] = staging_dir self.log.info( "StagingDir `{0}`...".format(instance.data["stagingDir"])) - frame_start = instance.data["frameStart"] - frame_end = instance.data["frameEnd"] - handle_start = instance.data["handleStart"] - handle_end = instance.data["handleEnd"] + def _check_frames_exists(self, instance): + # rendering path from group write node + fpath = instance.data["path"] - frame_length = int( - (frame_end - frame_start + 1) + (handle_start + handle_end) - ) + # instance frame range with handles + first = instance.data["frameStartHandle"] + last = instance.data["frameEndHandle"] + + padding = fpath.count('#') + + test_path_template = fpath + if padding: + repl_string = "#" * padding + test_path_template = fpath.replace( + repl_string, "%0{}d".format(padding)) + + for frame in range(first, last + 1): + test_file = test_path_template % frame + if not os.path.exists(test_file): + self.log.debug("__ test_file: `{}`".format(test_file)) + return None + + return True + + def render_slate( + self, + instance, + output_name=None, + bake_viewer_process=True, + bake_viewer_input_process=True + ): + """Slate frame renderer + + Args: + instance (PyblishInstance): Pyblish instance with subset data + output_name (str, optional): + Slate variation name. Defaults to None. + bake_viewer_process (bool, optional): + Switch for viewer profile baking. Defaults to True. + bake_viewer_input_process (bool, optional): + Switch for input process node baking. Defaults to True. + """ + slate_node = instance.data["slateNode"] + + # rendering path from group write node + fpath = instance.data["path"] + + # instance frame range with handles + first_frame = instance.data["frameStartHandle"] + last_frame = instance.data["frameEndHandle"] + + # fill slate node with comments + self.add_comment_slate_node(instance, slate_node) + + # solve output name if any is set + _output_name = output_name or "" + if _output_name: + _output_name = "_" + _output_name + + slate_first_frame = first_frame - 1 - temporary_nodes = [] collection = instance.data.get("collection", None) if collection: @@ -75,99 +142,101 @@ class ExtractSlateFrame(openpype.api.Extractor): fname = os.path.basename(collection.format( "{head}{padding}{tail}")) fhead = collection.format("{head}") - - collected_frames_len = int(len(collection.indexes)) - - # get first and last frame - first_frame = min(collection.indexes) - 1 - self.log.info('frame_length: {}'.format(frame_length)) - self.log.info( - 'len(collection.indexes): {}'.format(collected_frames_len) - ) - if ("slate" in instance.data["families"]) \ - and (frame_length != collected_frames_len): - first_frame += 1 - - last_frame = first_frame else: - fname = os.path.basename(instance.data.get("path", None)) + fname = os.path.basename(fpath) fhead = os.path.splitext(fname)[0] + "." - first_frame = instance.data.get("frameStartHandle", None) - 1 - last_frame = first_frame if "#" in fhead: fhead = fhead.replace("#", "")[:-1] - previous_node = node + self.log.debug("__ first_frame: {}".format(first_frame)) + self.log.debug("__ slate_first_frame: {}".format(slate_first_frame)) - # get input process and connect it to baking - ipn = self.get_view_process_node() - if ipn is not None: - ipn.setInput(0, previous_node) - previous_node = ipn - temporary_nodes.append(ipn) + # fallback if files does not exists + if self._check_frames_exists(instance): + # Read node + r_node = nuke.createNode("Read") + r_node["file"].setValue(fpath) + r_node["first"].setValue(first_frame) + r_node["origfirst"].setValue(first_frame) + r_node["last"].setValue(last_frame) + r_node["origlast"].setValue(last_frame) + r_node["colorspace"].setValue(instance.data["colorspace"]) + previous_node = r_node + temporary_nodes = [previous_node] + else: + previous_node = slate_node.dependencies().pop() + temporary_nodes = [] - if not self.viewer_lut_raw: + # only create colorspace baking if toggled on + if bake_viewer_process: + if bake_viewer_input_process: + # get input process and connect it to baking + ipn = get_view_process_node() + if ipn is not None: + ipn.setInput(0, previous_node) + previous_node = ipn + temporary_nodes.append(ipn) + + # add duplicate slate node and connect to previous + duply_slate_node = duplicate_node(slate_node) + duply_slate_node.setInput(0, previous_node) + previous_node = duply_slate_node + temporary_nodes.append(duply_slate_node) + + # add viewer display transformation node dag_node = nuke.createNode("OCIODisplay") dag_node.setInput(0, previous_node) previous_node = dag_node temporary_nodes.append(dag_node) + else: + # add duplicate slate node and connect to previous + duply_slate_node = duplicate_node(slate_node) + duply_slate_node.setInput(0, previous_node) + previous_node = duply_slate_node + temporary_nodes.append(duply_slate_node) + # create write node write_node = nuke.createNode("Write") - file = fhead + "slate.png" - path = os.path.join(staging_dir, file).replace("\\", "/") - instance.data["slateFrame"] = path + file = fhead[:-1] + _output_name + "_slate.png" + path = os.path.join( + instance.data["stagingDir"], file).replace("\\", "/") + + # add slate path to `slateFrames` instance data attr + if not instance.data.get("slateFrames"): + instance.data["slateFrames"] = {} + + instance.data["slateFrames"][output_name or "*"] = path + + # create write node write_node["file"].setValue(path) write_node["file_type"].setValue("png") write_node["raw"].setValue(1) write_node.setInput(0, previous_node) temporary_nodes.append(write_node) - # fill slate node with comments - self.add_comment_slate_node(instance) - # Render frames - nuke.execute(write_node.name(), int(first_frame), int(last_frame)) - # also render slate as sequence frame - nuke.execute(node_subset_name, int(first_frame), int(last_frame)) - - self.log.debug( - "slate frame path: {}".format(instance.data["slateFrame"])) + nuke.execute( + write_node.name(), int(slate_first_frame), int(slate_first_frame)) # Clean up for node in temporary_nodes: nuke.delete(node) - def get_view_process_node(self): - # Select only the target node - if nuke.selectedNodes(): - [n.setSelected(False) for n in nuke.selectedNodes()] + def _render_slate_to_sequence(self, instance): + # set slate frame + first_frame = instance.data["frameStartHandle"] + slate_first_frame = first_frame - 1 - ipn_orig = None - for v in [n for n in nuke.allNodes() - if "Viewer" in n.Class()]: - ip = v['input_process'].getValue() - ipn = v['input_process_node'].getValue() - if "VIEWER_INPUT" not in ipn and ip: - ipn_orig = nuke.toNode(ipn) - ipn_orig.setSelected(True) + # render slate as sequence frame + nuke.execute( + instance.data["name"], + int(slate_first_frame), + int(slate_first_frame) + ) - if ipn_orig: - nuke.nodeCopy('%clipboard%') - - [n.setSelected(False) for n in nuke.selectedNodes()] # Deselect all - - nuke.nodePaste('%clipboard%') - - ipn = nuke.selectedNode() - - return ipn - - def add_comment_slate_node(self, instance): - node = instance.data.get("slateNode") - if not node: - return + def add_comment_slate_node(self, instance, node): comment = instance.context.data.get("comment") intent = instance.context.data.get("intent") @@ -186,8 +255,8 @@ class ExtractSlateFrame(openpype.api.Extractor): "intent": intent }) - for key, value in self.key_value_mapping.items(): - enabled, template = value + for key, _values in self.key_value_mapping.items(): + enabled, template = _values if not enabled: self.log.debug("Key \"{}\" is disabled".format(key)) continue @@ -221,5 +290,5 @@ class ExtractSlateFrame(openpype.api.Extractor): )) except NameError: self.log.warning(( - "Failed to set value \"{}\" on node attribute \"{}\"" + "Failed to set value \"{0}\" on node attribute \"{0}\"" ).format(value)) diff --git a/openpype/hosts/nuke/plugins/publish/extract_thumbnail.py b/openpype/hosts/nuke/plugins/publish/extract_thumbnail.py index ef6d486ca2..2a919051d2 100644 --- a/openpype/hosts/nuke/plugins/publish/extract_thumbnail.py +++ b/openpype/hosts/nuke/plugins/publish/extract_thumbnail.py @@ -3,7 +3,10 @@ import os import nuke import pyblish.api import openpype -from openpype.hosts.nuke.api.lib import maintained_selection +from openpype.hosts.nuke.api import ( + maintained_selection, + get_view_process_node +) if sys.version_info[0] >= 3: @@ -17,15 +20,19 @@ class ExtractThumbnail(openpype.api.Extractor): """ - order = pyblish.api.ExtractorOrder + 0.01 + order = pyblish.api.ExtractorOrder + 0.011 label = "Extract Thumbnail" families = ["review"] hosts = ["nuke"] - # presets + # settings + use_rendered = False + bake_viewer_process = True + bake_viewer_input_process = True nodes = {} + def process(self, instance): if "render.farm" in instance.data["families"]: return @@ -35,14 +42,37 @@ class ExtractThumbnail(openpype.api.Extractor): self.log.debug("instance.data[families]: {}".format( instance.data["families"])) - self.render_thumbnail(instance) + if instance.data.get("bakePresets"): + for o_name, o_data in instance.data["bakePresets"].items(): + self.render_thumbnail(instance, o_name, **o_data) + else: + viewer_process_swithes = { + "bake_viewer_process": True, + "bake_viewer_input_process": True + } + self.render_thumbnail(instance, None, **viewer_process_swithes) + + def render_thumbnail(self, instance, output_name=None, **kwargs): + first_frame = instance.data["frameStartHandle"] + last_frame = instance.data["frameEndHandle"] + + # find frame range and define middle thumb frame + mid_frame = int((last_frame - first_frame) / 2) + + # solve output name if any is set + output_name = output_name or "" + if output_name: + output_name = "_" + output_name + + bake_viewer_process = kwargs["bake_viewer_process"] + bake_viewer_input_process_node = kwargs[ + "bake_viewer_input_process"] - def render_thumbnail(self, instance): node = instance[0] # group node self.log.info("Creating staging dir...") if "representations" not in instance.data: - instance.data["representations"] = list() + instance.data["representations"] = [] staging_dir = os.path.normpath( os.path.dirname(instance.data['path'])) @@ -53,7 +83,11 @@ class ExtractThumbnail(openpype.api.Extractor): "StagingDir `{0}`...".format(instance.data["stagingDir"])) temporary_nodes = [] + + # try to connect already rendered images + previous_node = node collection = instance.data.get("collection", None) + self.log.debug("__ collection: `{}`".format(collection)) if collection: # get path @@ -61,43 +95,38 @@ class ExtractThumbnail(openpype.api.Extractor): "{head}{padding}{tail}")) fhead = collection.format("{head}") - # get first and last frame - first_frame = min(collection.indexes) - last_frame = max(collection.indexes) + thumb_fname = list(collection)[mid_frame] else: - fname = os.path.basename(instance.data.get("path", None)) + fname = thumb_fname = os.path.basename( + instance.data.get("path", None)) fhead = os.path.splitext(fname)[0] + "." - first_frame = instance.data.get("frameStart", None) - last_frame = instance.data.get("frameEnd", None) + + self.log.debug("__ fhead: `{}`".format(fhead)) if "#" in fhead: fhead = fhead.replace("#", "")[:-1] - path_render = os.path.join(staging_dir, fname).replace("\\", "/") - # check if file exist otherwise connect to write node - if os.path.isfile(path_render): + path_render = os.path.join( + staging_dir, thumb_fname).replace("\\", "/") + self.log.debug("__ path_render: `{}`".format(path_render)) + + if self.use_rendered and os.path.isfile(path_render): + # check if file exist otherwise connect to write node rnode = nuke.createNode("Read") rnode["file"].setValue(path_render) - rnode["first"].setValue(first_frame) - rnode["origfirst"].setValue(first_frame) - rnode["last"].setValue(last_frame) - rnode["origlast"].setValue(last_frame) + # turn it raw if none of baking is ON + if all([ + not self.bake_viewer_input_process, + not self.bake_viewer_process + ]): + rnode["raw"].setValue(True) + temporary_nodes.append(rnode) previous_node = rnode - else: - previous_node = node - - # get input process and connect it to baking - ipn = self.get_view_process_node() - if ipn is not None: - ipn.setInput(0, previous_node) - previous_node = ipn - temporary_nodes.append(ipn) reformat_node = nuke.createNode("Reformat") - ref_node = self.nodes.get("Reformat", None) if ref_node: for k, v in ref_node: @@ -110,14 +139,24 @@ class ExtractThumbnail(openpype.api.Extractor): previous_node = reformat_node temporary_nodes.append(reformat_node) - dag_node = nuke.createNode("OCIODisplay") - dag_node.setInput(0, previous_node) - previous_node = dag_node - temporary_nodes.append(dag_node) + # only create colorspace baking if toggled on + if bake_viewer_process: + if bake_viewer_input_process_node: + # get input process and connect it to baking + ipn = get_view_process_node() + if ipn is not None: + ipn.setInput(0, previous_node) + previous_node = ipn + temporary_nodes.append(ipn) + + dag_node = nuke.createNode("OCIODisplay") + dag_node.setInput(0, previous_node) + previous_node = dag_node + temporary_nodes.append(dag_node) # create write node write_node = nuke.createNode("Write") - file = fhead + "jpg" + file = fhead[:-1] + output_name + ".jpg" name = "thumbnail" path = os.path.join(staging_dir, file).replace("\\", "/") instance.data["thumbnail"] = path @@ -128,26 +167,18 @@ class ExtractThumbnail(openpype.api.Extractor): temporary_nodes.append(write_node) tags = ["thumbnail", "publish_on_farm"] - # retime for - mid_frame = int((int(last_frame) - int(first_frame)) / 2) \ - + int(first_frame) - first_frame = int(last_frame) / 2 - last_frame = int(last_frame) / 2 - repre = { 'name': name, 'ext': "jpg", "outputName": "thumb", 'files': file, "stagingDir": staging_dir, - "frameStart": first_frame, - "frameEnd": last_frame, "tags": tags } instance.data["representations"].append(repre) # Render frames - nuke.execute(write_node.name(), int(mid_frame), int(mid_frame)) + nuke.execute(write_node.name(), mid_frame, mid_frame) self.log.debug( "representations: {}".format(instance.data["representations"])) @@ -155,30 +186,3 @@ class ExtractThumbnail(openpype.api.Extractor): # Clean up for node in temporary_nodes: nuke.delete(node) - - def get_view_process_node(self): - - # Select only the target node - if nuke.selectedNodes(): - [n.setSelected(False) for n in nuke.selectedNodes()] - - ipn_orig = None - for v in [n for n in nuke.allNodes() - if "Viewer" == n.Class()]: - ip = v['input_process'].getValue() - ipn = v['input_process_node'].getValue() - if "VIEWER_INPUT" not in ipn and ip: - ipn_orig = nuke.toNode(ipn) - ipn_orig.setSelected(True) - - if ipn_orig: - nuke.nodeCopy('%clipboard%') - - # Deselect all - [n.setSelected(False) for n in nuke.selectedNodes()] - - nuke.nodePaste('%clipboard%') - - ipn = nuke.selectedNode() - - return ipn diff --git a/openpype/hosts/nuke/plugins/publish/precollect_instances.py b/openpype/hosts/nuke/plugins/publish/precollect_instances.py index 1a8fa3e6ad..8bf7280cea 100644 --- a/openpype/hosts/nuke/plugins/publish/precollect_instances.py +++ b/openpype/hosts/nuke/plugins/publish/precollect_instances.py @@ -151,15 +151,11 @@ class PreCollectNukeInstances(pyblish.api.ContextPlugin): "resolutionWidth": resolution_width, "resolutionHeight": resolution_height, "pixelAspect": pixel_aspect, - "review": review + "review": review, + "representations": [] }) self.log.info("collected instance: {}".format(instance.data)) instances.append(instance) - # create instances in context data if not are created yet - if not context.data.get("instances"): - context.data["instances"] = list() - - context.data["instances"].extend(instances) self.log.debug("context: {}".format(context)) diff --git a/openpype/hosts/nuke/plugins/publish/precollect_workfile.py b/openpype/hosts/nuke/plugins/publish/precollect_workfile.py index a2d1c80628..7349a8f424 100644 --- a/openpype/hosts/nuke/plugins/publish/precollect_workfile.py +++ b/openpype/hosts/nuke/plugins/publish/precollect_workfile.py @@ -17,7 +17,7 @@ class CollectWorkfile(pyblish.api.ContextPlugin): label = "Pre-collect Workfile" hosts = ['nuke'] - def process(self, context): + def process(self, context): # sourcery skip: avoid-builtin-shadow root = nuke.root() current_file = os.path.normpath(nuke.root().name()) @@ -74,20 +74,6 @@ class CollectWorkfile(pyblish.api.ContextPlugin): } context.data.update(script_data) - # creating instance data - instance.data.update({ - "subset": subset, - "label": base_name, - "name": base_name, - "publish": root.knob('publish').value(), - "family": family, - "families": [family], - "representations": list() - }) - - # adding basic script data - instance.data.update(script_data) - # creating representation representation = { 'name': 'nk', @@ -96,12 +82,18 @@ class CollectWorkfile(pyblish.api.ContextPlugin): "stagingDir": staging_dir, } - instance.data["representations"].append(representation) + # creating instance data + instance.data.update({ + "subset": subset, + "label": base_name, + "name": base_name, + "publish": root.knob('publish').value(), + "family": family, + "families": [family], + "representations": [representation] + }) + + # adding basic script data + instance.data.update(script_data) self.log.info('Publishing script version') - - # create instances in context data if not are created yet - if not context.data.get("instances"): - context.data["instances"] = list() - - context.data["instances"].append(instance) diff --git a/openpype/hosts/nuke/plugins/publish/precollect_writes.py b/openpype/hosts/nuke/plugins/publish/precollect_writes.py index 8669f4f485..7e50679ed5 100644 --- a/openpype/hosts/nuke/plugins/publish/precollect_writes.py +++ b/openpype/hosts/nuke/plugins/publish/precollect_writes.py @@ -72,12 +72,12 @@ class CollectNukeWrites(pyblish.api.InstancePlugin): if "representations" not in instance.data: instance.data["representations"] = list() - representation = { - 'name': ext, - 'ext': ext, - "stagingDir": output_dir, - "tags": list() - } + representation = { + 'name': ext, + 'ext': ext, + "stagingDir": output_dir, + "tags": list() + } try: collected_frames = [f for f in os.listdir(output_dir) @@ -175,6 +175,11 @@ class CollectNukeWrites(pyblish.api.InstancePlugin): "frameEndHandle": last_frame, }) + # make sure rendered sequence on farm will + # be used for exctract review + if not instance.data["review"]: + instance.data["useSequenceForReview"] = False + # * Add audio to instance if exists. # Find latest versions document version_doc = pype.get_latest_version( diff --git a/openpype/hosts/nuke/startup/KnobScripter/__init__.py b/openpype/hosts/nuke/startup/KnobScripter/__init__.py deleted file mode 100644 index 8fe91d63f5..0000000000 --- a/openpype/hosts/nuke/startup/KnobScripter/__init__.py +++ /dev/null @@ -1 +0,0 @@ -import knob_scripter \ No newline at end of file diff --git a/openpype/hosts/nuke/startup/KnobScripter/icons/icon_clearConsole.png b/openpype/hosts/nuke/startup/KnobScripter/icons/icon_clearConsole.png deleted file mode 100644 index 75ac04ef84..0000000000 Binary files a/openpype/hosts/nuke/startup/KnobScripter/icons/icon_clearConsole.png and /dev/null differ diff --git a/openpype/hosts/nuke/startup/KnobScripter/icons/icon_download.png b/openpype/hosts/nuke/startup/KnobScripter/icons/icon_download.png deleted file mode 100644 index 1e3e9b7631..0000000000 Binary files a/openpype/hosts/nuke/startup/KnobScripter/icons/icon_download.png and /dev/null differ diff --git a/openpype/hosts/nuke/startup/KnobScripter/icons/icon_exitnode.png b/openpype/hosts/nuke/startup/KnobScripter/icons/icon_exitnode.png deleted file mode 100644 index 7714cd2b92..0000000000 Binary files a/openpype/hosts/nuke/startup/KnobScripter/icons/icon_exitnode.png and /dev/null differ diff --git a/openpype/hosts/nuke/startup/KnobScripter/icons/icon_pick.png b/openpype/hosts/nuke/startup/KnobScripter/icons/icon_pick.png deleted file mode 100644 index 2395537550..0000000000 Binary files a/openpype/hosts/nuke/startup/KnobScripter/icons/icon_pick.png and /dev/null differ diff --git a/openpype/hosts/nuke/startup/KnobScripter/icons/icon_prefs.png b/openpype/hosts/nuke/startup/KnobScripter/icons/icon_prefs.png deleted file mode 100644 index efef5ffc92..0000000000 Binary files a/openpype/hosts/nuke/startup/KnobScripter/icons/icon_prefs.png and /dev/null differ diff --git a/openpype/hosts/nuke/startup/KnobScripter/icons/icon_prefs2.png b/openpype/hosts/nuke/startup/KnobScripter/icons/icon_prefs2.png deleted file mode 100644 index 5c3c941d59..0000000000 Binary files a/openpype/hosts/nuke/startup/KnobScripter/icons/icon_prefs2.png and /dev/null differ diff --git a/openpype/hosts/nuke/startup/KnobScripter/icons/icon_refresh.png b/openpype/hosts/nuke/startup/KnobScripter/icons/icon_refresh.png deleted file mode 100644 index 559bfd74ab..0000000000 Binary files a/openpype/hosts/nuke/startup/KnobScripter/icons/icon_refresh.png and /dev/null differ diff --git a/openpype/hosts/nuke/startup/KnobScripter/icons/icon_run.png b/openpype/hosts/nuke/startup/KnobScripter/icons/icon_run.png deleted file mode 100644 index 6b2e4ddc23..0000000000 Binary files a/openpype/hosts/nuke/startup/KnobScripter/icons/icon_run.png and /dev/null differ diff --git a/openpype/hosts/nuke/startup/KnobScripter/icons/icon_save.png b/openpype/hosts/nuke/startup/KnobScripter/icons/icon_save.png deleted file mode 100644 index e29c667f34..0000000000 Binary files a/openpype/hosts/nuke/startup/KnobScripter/icons/icon_save.png and /dev/null differ diff --git a/openpype/hosts/nuke/startup/KnobScripter/icons/icon_search.png b/openpype/hosts/nuke/startup/KnobScripter/icons/icon_search.png deleted file mode 100644 index d4ed2e1a2b..0000000000 Binary files a/openpype/hosts/nuke/startup/KnobScripter/icons/icon_search.png and /dev/null differ diff --git a/openpype/hosts/nuke/startup/KnobScripter/icons/icon_snippets.png b/openpype/hosts/nuke/startup/KnobScripter/icons/icon_snippets.png deleted file mode 100644 index 479c44f19e..0000000000 Binary files a/openpype/hosts/nuke/startup/KnobScripter/icons/icon_snippets.png and /dev/null differ diff --git a/openpype/hosts/nuke/startup/KnobScripter/knob_scripter.py b/openpype/hosts/nuke/startup/KnobScripter/knob_scripter.py deleted file mode 100644 index 368ee64e32..0000000000 --- a/openpype/hosts/nuke/startup/KnobScripter/knob_scripter.py +++ /dev/null @@ -1,4196 +0,0 @@ -# ------------------------------------------------- -# KnobScripter by Adrian Pueyo -# Complete python script editor for Nuke -# adrianpueyo.com, 2016-2019 -import string -import traceback -from webbrowser import open as openUrl -from threading import Event, Thread -import platform -import subprocess -from functools import partial -import re -import sys -from nukescripts import panels -import json -import os -import nuke -version = "2.3 wip" -date = "Aug 12 2019" -# ------------------------------------------------- - - -# Symlinks on windows... -if os.name == "nt": - def symlink_ms(source, link_name): - import ctypes - csl = ctypes.windll.kernel32.CreateSymbolicLinkW - csl.argtypes = (ctypes.c_wchar_p, ctypes.c_wchar_p, ctypes.c_uint32) - csl.restype = ctypes.c_ubyte - flags = 1 if os.path.isdir(source) else 0 - try: - if csl(link_name, source.replace('/', '\\'), flags) == 0: - raise ctypes.WinError() - except: - pass - os.symlink = symlink_ms - -try: - if nuke.NUKE_VERSION_MAJOR < 11: - from PySide import QtCore, QtGui, QtGui as QtWidgets - from PySide.QtCore import Qt - else: - from PySide2 import QtWidgets, QtGui, QtCore - from PySide2.QtCore import Qt -except ImportError: - from Qt import QtCore, QtGui, QtWidgets - -KS_DIR = os.path.dirname(__file__) -icons_path = KS_DIR + "/icons/" -DebugMode = False -AllKnobScripters = [] # All open instances at a given time - -PrefsPanel = "" -SnippetEditPanel = "" - -nuke.tprint('KnobScripter v{}, built {}.\nCopyright (c) 2016-2019 Adrian Pueyo. All Rights Reserved.'.format(version, date)) - - -class KnobScripter(QtWidgets.QWidget): - - def __init__(self, node="", knob="knobChanged"): - super(KnobScripter, self).__init__() - - # Autosave the other knobscripters and add this one - for ks in AllKnobScripters: - try: - ks.autosave() - except: - pass - if self not in AllKnobScripters: - AllKnobScripters.append(self) - - self.nodeMode = (node != "") - if node == "": - self.node = nuke.toNode("root") - else: - self.node = node - - self.isPane = False - self.knob = knob - # For the option to also display the knob labels on the knob dropdown - self.show_labels = False - self.unsavedKnobs = {} - self.modifiedKnobs = set() - self.scrollPos = {} - self.cursorPos = {} - self.fontSize = 10 - self.font = "Monospace" - self.tabSpaces = 4 - self.windowDefaultSize = [500, 300] - self.color_scheme = "sublime" # Can be nuke or sublime - self.pinned = 1 - self.toLoadKnob = True - self.frw_open = False # Find replace widget closed by default - self.icon_size = 17 - self.btn_size = 24 - self.qt_icon_size = QtCore.QSize(self.icon_size, self.icon_size) - self.qt_btn_size = QtCore.QSize(self.btn_size, self.btn_size) - self.origConsoleText = "" - self.nukeSE = self.findSE() - self.nukeSEOutput = self.findSEOutput(self.nukeSE) - self.nukeSEInput = self.findSEInput(self.nukeSE) - self.nukeSERunBtn = self.findSERunBtn(self.nukeSE) - - self.scripts_dir = os.path.expandvars( - os.path.expanduser("~/.nuke/KnobScripter_Scripts")) - self.current_folder = "scripts" - self.folder_index = 0 - self.current_script = "Untitled.py" - self.current_script_modified = False - self.script_index = 0 - self.toAutosave = False - - # Load prefs - self.prefs_txt = os.path.expandvars( - os.path.expanduser("~/.nuke/KnobScripter_Prefs.txt")) - self.loadedPrefs = self.loadPrefs() - if self.loadedPrefs != []: - try: - if "font_size" in self.loadedPrefs: - self.fontSize = self.loadedPrefs['font_size'] - self.windowDefaultSize = [ - self.loadedPrefs['window_default_w'], self.loadedPrefs['window_default_h']] - self.tabSpaces = self.loadedPrefs['tab_spaces'] - self.pinned = self.loadedPrefs['pin_default'] - if "font" in self.loadedPrefs: - self.font = self.loadedPrefs['font'] - if "color_scheme" in self.loadedPrefs: - self.color_scheme = self.loadedPrefs['color_scheme'] - if "show_labels" in self.loadedPrefs: - self.show_labels = self.loadedPrefs['show_labels'] - except TypeError: - log("KnobScripter: Failed to load preferences.") - - # Load snippets - self.snippets_txt_path = os.path.expandvars( - os.path.expanduser("~/.nuke/KnobScripter_Snippets.txt")) - self.snippets = self.loadSnippets(maxDepth=5) - - # Current state of script (loaded when exiting node mode) - self.state_txt_path = os.path.expandvars( - os.path.expanduser("~/.nuke/KnobScripter_State.txt")) - - # Init UI - self.initUI() - - # Talk to Nuke's Script Editor - self.setSEOutputEvent() # Make the output windowS listen! - self.clearConsole() - - def initUI(self): - ''' Initializes the tool UI''' - # ------------------- - # 1. MAIN WINDOW - # ------------------- - self.resize(self.windowDefaultSize[0], self.windowDefaultSize[1]) - self.setWindowTitle("KnobScripter - %s %s" % - (self.node.fullName(), self.knob)) - self.setObjectName("com.adrianpueyo.knobscripter") - self.move(QtGui.QCursor().pos() - QtCore.QPoint(32, 74)) - - # --------------------- - # 2. TOP BAR - # --------------------- - # --- - # 2.1. Left buttons - self.change_btn = QtWidgets.QToolButton() - # self.exit_node_btn.setIcon(QtGui.QIcon(KS_DIR+"/KnobScripter/icons/icons8-delete-26.png")) - self.change_btn.setIcon(QtGui.QIcon(icons_path + "icon_pick.png")) - self.change_btn.setIconSize(self.qt_icon_size) - self.change_btn.setFixedSize(self.qt_btn_size) - self.change_btn.setToolTip( - "Change to node if selected. Otherwise, change to Script Mode.") - self.change_btn.clicked.connect(self.changeClicked) - - # --- - # 2.2.A. Node mode UI - self.exit_node_btn = QtWidgets.QToolButton() - self.exit_node_btn.setIcon(QtGui.QIcon( - icons_path + "icon_exitnode.png")) - self.exit_node_btn.setIconSize(self.qt_icon_size) - self.exit_node_btn.setFixedSize(self.qt_btn_size) - self.exit_node_btn.setToolTip( - "Exit the node, and change to Script Mode.") - self.exit_node_btn.clicked.connect(self.exitNodeMode) - self.current_node_label_node = QtWidgets.QLabel(" Node:") - self.current_node_label_name = QtWidgets.QLabel(self.node.fullName()) - self.current_node_label_name.setStyleSheet("font-weight:bold;") - self.current_knob_label = QtWidgets.QLabel("Knob: ") - self.current_knob_dropdown = QtWidgets.QComboBox() - self.current_knob_dropdown.setSizeAdjustPolicy( - QtWidgets.QComboBox.AdjustToContents) - self.updateKnobDropdown() - self.current_knob_dropdown.currentIndexChanged.connect( - lambda: self.loadKnobValue(False, updateDict=True)) - - # Layout - self.node_mode_bar_layout = QtWidgets.QHBoxLayout() - self.node_mode_bar_layout.addWidget(self.exit_node_btn) - self.node_mode_bar_layout.addSpacing(2) - self.node_mode_bar_layout.addWidget(self.current_node_label_node) - self.node_mode_bar_layout.addWidget(self.current_node_label_name) - self.node_mode_bar_layout.addSpacing(2) - self.node_mode_bar_layout.addWidget(self.current_knob_dropdown) - self.node_mode_bar = QtWidgets.QWidget() - self.node_mode_bar.setLayout(self.node_mode_bar_layout) - - self.node_mode_bar_layout.setContentsMargins(0, 0, 0, 0) - - # --- - # 2.2.B. Script mode UI - self.script_label = QtWidgets.QLabel("Script: ") - - self.current_folder_dropdown = QtWidgets.QComboBox() - self.current_folder_dropdown.setSizeAdjustPolicy( - QtWidgets.QComboBox.AdjustToContents) - self.current_folder_dropdown.currentIndexChanged.connect( - self.folderDropdownChanged) - # self.current_folder_dropdown.setEditable(True) - # self.current_folder_dropdown.lineEdit().setReadOnly(True) - # self.current_folder_dropdown.lineEdit().setAlignment(Qt.AlignRight) - - self.current_script_dropdown = QtWidgets.QComboBox() - self.current_script_dropdown.setSizeAdjustPolicy( - QtWidgets.QComboBox.AdjustToContents) - self.updateFoldersDropdown() - self.updateScriptsDropdown() - self.current_script_dropdown.currentIndexChanged.connect( - self.scriptDropdownChanged) - - # Layout - self.script_mode_bar_layout = QtWidgets.QHBoxLayout() - self.script_mode_bar_layout.addWidget(self.script_label) - self.script_mode_bar_layout.addSpacing(2) - self.script_mode_bar_layout.addWidget(self.current_folder_dropdown) - self.script_mode_bar_layout.addWidget(self.current_script_dropdown) - self.script_mode_bar = QtWidgets.QWidget() - self.script_mode_bar.setLayout(self.script_mode_bar_layout) - - self.script_mode_bar_layout.setContentsMargins(0, 0, 0, 0) - - # --- - # 2.3. File-system buttons - # Refresh dropdowns - self.refresh_btn = QtWidgets.QToolButton() - self.refresh_btn.setIcon(QtGui.QIcon(icons_path + "icon_refresh.png")) - self.refresh_btn.setIconSize(QtCore.QSize(50, 50)) - self.refresh_btn.setIconSize(self.qt_icon_size) - self.refresh_btn.setFixedSize(self.qt_btn_size) - self.refresh_btn.setToolTip("Refresh the dropdowns.\nShortcut: F5") - self.refresh_btn.setShortcut('F5') - self.refresh_btn.clicked.connect(self.refreshClicked) - - # Reload script - self.reload_btn = QtWidgets.QToolButton() - self.reload_btn.setIcon(QtGui.QIcon(icons_path + "icon_download.png")) - self.reload_btn.setIconSize(QtCore.QSize(50, 50)) - self.reload_btn.setIconSize(self.qt_icon_size) - self.reload_btn.setFixedSize(self.qt_btn_size) - self.reload_btn.setToolTip( - "Reload the current script. Will overwrite any changes made to it.\nShortcut: Ctrl+R") - self.reload_btn.setShortcut('Ctrl+R') - self.reload_btn.clicked.connect(self.reloadClicked) - - # Save script - self.save_btn = QtWidgets.QToolButton() - self.save_btn.setIcon(QtGui.QIcon(icons_path + "icon_save.png")) - self.save_btn.setIconSize(QtCore.QSize(50, 50)) - self.save_btn.setIconSize(self.qt_icon_size) - self.save_btn.setFixedSize(self.qt_btn_size) - self.save_btn.setToolTip( - "Save the script into the selected knob or python file.\nShortcut: Ctrl+S") - self.save_btn.setShortcut('Ctrl+S') - self.save_btn.clicked.connect(self.saveClicked) - - # Layout - self.top_file_bar_layout = QtWidgets.QHBoxLayout() - self.top_file_bar_layout.addWidget(self.refresh_btn) - self.top_file_bar_layout.addWidget(self.reload_btn) - self.top_file_bar_layout.addWidget(self.save_btn) - - # --- - # 2.4. Right Side buttons - - # Run script - self.run_script_button = QtWidgets.QToolButton() - self.run_script_button.setIcon( - QtGui.QIcon(icons_path + "icon_run.png")) - self.run_script_button.setIconSize(self.qt_icon_size) - # self.run_script_button.setIconSize(self.qt_icon_size) - self.run_script_button.setFixedSize(self.qt_btn_size) - self.run_script_button.setToolTip( - "Execute the current selection on the KnobScripter, or the whole script if no selection.\nShortcut: Ctrl+Enter") - self.run_script_button.clicked.connect(self.runScript) - - # Clear console - self.clear_console_button = QtWidgets.QToolButton() - self.clear_console_button.setIcon( - QtGui.QIcon(icons_path + "icon_clearConsole.png")) - self.clear_console_button.setIconSize(QtCore.QSize(50, 50)) - self.clear_console_button.setIconSize(self.qt_icon_size) - self.clear_console_button.setFixedSize(self.qt_btn_size) - self.clear_console_button.setToolTip( - "Clear the text in the console window.\nShortcut: Click Backspace on the console.") - self.clear_console_button.clicked.connect(self.clearConsole) - - # FindReplace button - self.find_button = QtWidgets.QToolButton() - self.find_button.setIcon(QtGui.QIcon(icons_path + "icon_search.png")) - self.find_button.setIconSize(self.qt_icon_size) - self.find_button.setFixedSize(self.qt_btn_size) - self.find_button.setToolTip( - "Call the snippets by writing the shortcut and pressing Tab.\nShortcut: Ctrl+F") - self.find_button.setShortcut('Ctrl+F') - #self.find_button.setMaximumWidth(self.find_button.fontMetrics().boundingRect("Find").width() + 20) - self.find_button.setCheckable(True) - self.find_button.setFocusPolicy(QtCore.Qt.NoFocus) - self.find_button.clicked[bool].connect(self.toggleFRW) - if self.frw_open: - self.find_button.toggle() - - # Snippets - self.snippets_button = QtWidgets.QToolButton() - self.snippets_button.setIcon( - QtGui.QIcon(icons_path + "icon_snippets.png")) - self.snippets_button.setIconSize(QtCore.QSize(50, 50)) - self.snippets_button.setIconSize(self.qt_icon_size) - self.snippets_button.setFixedSize(self.qt_btn_size) - self.snippets_button.setToolTip( - "Call the snippets by writing the shortcut and pressing Tab.") - self.snippets_button.clicked.connect(self.openSnippets) - - # PIN - ''' - self.pin_button = QtWidgets.QPushButton("P") - self.pin_button.setCheckable(True) - if self.pinned: - self.setWindowFlags(self.windowFlags() | QtCore.Qt.WindowStaysOnTopHint) - self.pin_button.toggle() - self.pin_button.setToolTip("Toggle 'Always On Top'. Keeps the KnobScripter on top of all other windows.") - self.pin_button.setFocusPolicy(QtCore.Qt.NoFocus) - self.pin_button.setFixedSize(self.qt_btn_size) - self.pin_button.clicked[bool].connect(self.pin) - ''' - - # Prefs - self.createPrefsMenu() - self.prefs_button = QtWidgets.QPushButton() - self.prefs_button.setIcon(QtGui.QIcon(icons_path + "icon_prefs.png")) - self.prefs_button.setIconSize(self.qt_icon_size) - self.prefs_button.setFixedSize( - QtCore.QSize(self.btn_size + 10, self.btn_size)) - # self.prefs_button.clicked.connect(self.openPrefs) - self.prefs_button.setMenu(self.prefsMenu) - self.prefs_button.setStyleSheet("text-align:left;padding-left:2px;") - #self.prefs_button.setMaximumWidth(self.prefs_button.fontMetrics().boundingRect("Prefs").width() + 12) - - # Layout - self.top_right_bar_layout = QtWidgets.QHBoxLayout() - self.top_right_bar_layout.addWidget(self.run_script_button) - self.top_right_bar_layout.addWidget(self.clear_console_button) - self.top_right_bar_layout.addWidget(self.find_button) - # self.top_right_bar_layout.addWidget(self.snippets_button) - # self.top_right_bar_layout.addWidget(self.pin_button) - # self.top_right_bar_layout.addSpacing(10) - self.top_right_bar_layout.addWidget(self.prefs_button) - - # --- - # Layout - self.top_layout = QtWidgets.QHBoxLayout() - self.top_layout.setContentsMargins(0, 0, 0, 0) - # self.top_layout.setSpacing(10) - self.top_layout.addWidget(self.change_btn) - self.top_layout.addWidget(self.node_mode_bar) - self.top_layout.addWidget(self.script_mode_bar) - self.node_mode_bar.setVisible(False) - # self.top_layout.addSpacing(10) - self.top_layout.addLayout(self.top_file_bar_layout) - self.top_layout.addStretch() - self.top_layout.addLayout(self.top_right_bar_layout) - - # ---------------------- - # 3. SCRIPTING SECTION - # ---------------------- - # Splitter - self.splitter = QtWidgets.QSplitter(Qt.Vertical) - - # Output widget - self.script_output = ScriptOutputWidget(parent=self) - self.script_output.setReadOnly(1) - self.script_output.setAcceptRichText(0) - self.script_output.setTabStopWidth( - self.script_output.tabStopWidth() / 4) - self.script_output.setFocusPolicy(Qt.ClickFocus) - self.script_output.setAutoFillBackground(0) - self.script_output.installEventFilter(self) - - # Script Editor - self.script_editor = KnobScripterTextEditMain(self, self.script_output) - self.script_editor.setMinimumHeight(30) - self.script_editor.setStyleSheet( - 'background:#282828;color:#EEE;') # Main Colors - self.script_editor.textChanged.connect(self.setModified) - self.highlighter = KSScriptEditorHighlighter( - self.script_editor.document(), self) - self.script_editor.cursorPositionChanged.connect(self.setTextSelection) - self.script_editor_font = QtGui.QFont() - self.script_editor_font.setFamily(self.font) - self.script_editor_font.setStyleHint(QtGui.QFont.Monospace) - self.script_editor_font.setFixedPitch(True) - self.script_editor_font.setPointSize(self.fontSize) - self.script_editor.setFont(self.script_editor_font) - self.script_editor.setTabStopWidth( - self.tabSpaces * QtGui.QFontMetrics(self.script_editor_font).width(' ')) - - # Add input and output to splitter - self.splitter.addWidget(self.script_output) - self.splitter.addWidget(self.script_editor) - self.splitter.setStretchFactor(0, 0) - - # FindReplace widget - self.frw = FindReplaceWidget(self) - self.frw.setVisible(self.frw_open) - - # --- - # Layout - self.scripting_layout = QtWidgets.QVBoxLayout() - self.scripting_layout.setContentsMargins(0, 0, 0, 0) - self.scripting_layout.setSpacing(0) - self.scripting_layout.addWidget(self.splitter) - self.scripting_layout.addWidget(self.frw) - - # --------------- - # MASTER LAYOUT - # --------------- - self.master_layout = QtWidgets.QVBoxLayout() - self.master_layout.setSpacing(5) - self.master_layout.setContentsMargins(8, 8, 8, 8) - self.master_layout.addLayout(self.top_layout) - self.master_layout.addLayout(self.scripting_layout) - # self.master_layout.addLayout(self.bottom_layout) - self.setLayout(self.master_layout) - - # ---------------- - # MAIN WINDOW UI - # ---------------- - size_policy = QtWidgets.QSizePolicy( - QtWidgets.QSizePolicy.Minimum, QtWidgets.QSizePolicy.Minimum) - self.setSizePolicy(size_policy) - self.setMinimumWidth(160) - - if self.pinned: - self.setWindowFlags(self.windowFlags() | - QtCore.Qt.WindowStaysOnTopHint) - - # Set default values based on mode - if self.nodeMode: - self.current_knob_dropdown.blockSignals(True) - self.node_mode_bar.setVisible(True) - self.script_mode_bar.setVisible(False) - self.setCurrentKnob(self.knob) - self.loadKnobValue(check=False) - self.setKnobModified(False) - self.current_knob_dropdown.blockSignals(False) - self.splitter.setSizes([0, 1]) - else: - self.exitNodeMode() - self.script_editor.setFocus() - - # Preferences submenus - def createPrefsMenu(self): - - # Actions - self.echoAct = QtWidgets.QAction("Echo python commands", self, checkable=True, - statusTip="Toggle nuke's 'Echo all python commands to ScriptEditor'", triggered=self.toggleEcho) - if nuke.toNode("preferences").knob("echoAllCommands").value(): - self.echoAct.toggle() - self.pinAct = QtWidgets.QAction("Always on top", self, checkable=True, - statusTip="Keeps the KnobScripter window always on top or not.", triggered=self.togglePin) - if self.pinned: - self.setWindowFlags(self.windowFlags() | - QtCore.Qt.WindowStaysOnTopHint) - self.pinAct.toggle() - self.helpAct = QtWidgets.QAction( - "&Help", self, statusTip="Open the KnobScripter help in your browser.", shortcut="F1", triggered=self.showHelp) - self.nukepediaAct = QtWidgets.QAction( - "Show in Nukepedia", self, statusTip="Open the KnobScripter download page on Nukepedia.", triggered=self.showInNukepedia) - self.githubAct = QtWidgets.QAction( - "Show in GitHub", self, statusTip="Open the KnobScripter repo on GitHub.", triggered=self.showInGithub) - self.snippetsAct = QtWidgets.QAction( - "Snippets", self, statusTip="Open the Snippets editor.", triggered=self.openSnippets) - self.snippetsAct.setIcon(QtGui.QIcon(icons_path + "icon_snippets.png")) - # self.snippetsAct = QtWidgets.QAction("Keywords", self, statusTip="Add custom keywords.", triggered=self.openSnippets) #TODO THIS - self.prefsAct = QtWidgets.QAction( - "Preferences", self, statusTip="Open the Preferences panel.", triggered=self.openPrefs) - self.prefsAct.setIcon(QtGui.QIcon(icons_path + "icon_prefs.png")) - - # Menus - self.prefsMenu = QtWidgets.QMenu("Preferences") - self.prefsMenu.addAction(self.echoAct) - self.prefsMenu.addAction(self.pinAct) - self.prefsMenu.addSeparator() - self.prefsMenu.addAction(self.nukepediaAct) - self.prefsMenu.addAction(self.githubAct) - self.prefsMenu.addSeparator() - self.prefsMenu.addAction(self.helpAct) - self.prefsMenu.addSeparator() - self.prefsMenu.addAction(self.snippetsAct) - self.prefsMenu.addAction(self.prefsAct) - - def initEcho(self): - ''' Initializes the echo chechable QAction based on nuke's state ''' - echo_knob = nuke.toNode("preferences").knob("echoAllCommands") - self.echoAct.setChecked(echo_knob.value()) - - def toggleEcho(self): - ''' Toggle the "Echo python commands" from Nuke ''' - echo_knob = nuke.toNode("preferences").knob("echoAllCommands") - echo_knob.setValue(self.echoAct.isChecked()) - - def togglePin(self): - ''' Toggle "always on top" based on the submenu button ''' - self.pin(self.pinAct.isChecked()) - - def showInNukepedia(self): - openUrl("http://www.nukepedia.com/python/ui/knobscripter") - - def showInGithub(self): - openUrl("https://github.com/adrianpueyo/KnobScripter") - - def showHelp(self): - openUrl("https://vimeo.com/adrianpueyo/knobscripter2") - - # Node Mode - - def updateKnobDropdown(self): - ''' Populate knob dropdown list ''' - self.current_knob_dropdown.clear() # First remove all items - defaultKnobs = ["knobChanged", "onCreate", "onScriptLoad", "onScriptSave", "onScriptClose", "onDestroy", - "updateUI", "autolabel", "beforeRender", "beforeFrameRender", "afterFrameRender", "afterRender"] - permittedKnobClasses = ["PyScript_Knob", "PythonCustomKnob"] - counter = 0 - for i in self.node.knobs(): - if i not in defaultKnobs and self.node.knob(i).Class() in permittedKnobClasses: - if self.show_labels: - i_full = "{} ({})".format(self.node.knob(i).label(), i) - else: - i_full = i - - if i in self.unsavedKnobs.keys(): - self.current_knob_dropdown.addItem(i_full + "(*)", i) - else: - self.current_knob_dropdown.addItem(i_full, i) - - counter += 1 - if counter > 0: - self.current_knob_dropdown.insertSeparator(counter) - counter += 1 - self.current_knob_dropdown.insertSeparator(counter) - counter += 1 - for i in self.node.knobs(): - if i in defaultKnobs: - if i in self.unsavedKnobs.keys(): - self.current_knob_dropdown.addItem(i + "(*)", i) - else: - self.current_knob_dropdown.addItem(i, i) - counter += 1 - return - - def loadKnobValue(self, check=True, updateDict=False): - ''' Get the content of the knob value and populate the editor ''' - if self.toLoadKnob == False: - return - dropdown_value = self.current_knob_dropdown.itemData( - self.current_knob_dropdown.currentIndex()) # knobChanged... - try: - obtained_knobValue = str(self.node[dropdown_value].value()) - obtained_scrollValue = 0 - edited_knobValue = self.script_editor.toPlainText() - except: - error_message = QtWidgets.QMessageBox.information( - None, "", "Unable to find %s.%s" % (self.node.name(), dropdown_value)) - error_message.setWindowFlags(QtCore.Qt.WindowStaysOnTopHint) - error_message.exec_() - return - # If there were changes to the previous knob, update the dictionary - if updateDict == True: - self.unsavedKnobs[self.knob] = edited_knobValue - self.scrollPos[self.knob] = self.script_editor.verticalScrollBar( - ).value() - prev_knob = self.knob # knobChanged... - - self.knob = self.current_knob_dropdown.itemData( - self.current_knob_dropdown.currentIndex()) # knobChanged... - - if check and obtained_knobValue != edited_knobValue: - msgBox = QtWidgets.QMessageBox() - msgBox.setText("The Script Editor has been modified.") - msgBox.setInformativeText( - "Do you want to overwrite the current code on this editor?") - msgBox.setStandardButtons( - QtWidgets.QMessageBox.Yes | QtWidgets.QMessageBox.No) - msgBox.setIcon(QtWidgets.QMessageBox.Question) - msgBox.setWindowFlags(QtCore.Qt.WindowStaysOnTopHint) - msgBox.setDefaultButton(QtWidgets.QMessageBox.Yes) - reply = msgBox.exec_() - if reply == QtWidgets.QMessageBox.No: - self.setCurrentKnob(prev_knob) - return - # If order comes from a dropdown update, update value from dictionary if possible, otherwise update normally - self.setWindowTitle("KnobScripter - %s %s" % - (self.node.name(), self.knob)) - if updateDict: - if self.knob in self.unsavedKnobs: - if self.unsavedKnobs[self.knob] == obtained_knobValue: - self.script_editor.setPlainText(obtained_knobValue) - self.setKnobModified(False) - else: - obtained_knobValue = self.unsavedKnobs[self.knob] - self.script_editor.setPlainText(obtained_knobValue) - self.setKnobModified(True) - else: - self.script_editor.setPlainText(obtained_knobValue) - self.setKnobModified(False) - - if self.knob in self.scrollPos: - obtained_scrollValue = self.scrollPos[self.knob] - else: - self.script_editor.setPlainText(obtained_knobValue) - - cursor = self.script_editor.textCursor() - self.script_editor.setTextCursor(cursor) - self.script_editor.verticalScrollBar().setValue(obtained_scrollValue) - return - - def loadAllKnobValues(self): - ''' Load all knobs button's function ''' - if len(self.unsavedKnobs) >= 1: - msgBox = QtWidgets.QMessageBox() - msgBox.setText( - "Do you want to reload all python and callback knobs?") - msgBox.setInformativeText( - "Unsaved changes on this editor will be lost.") - msgBox.setStandardButtons( - QtWidgets.QMessageBox.Yes | QtWidgets.QMessageBox.No) - msgBox.setIcon(QtWidgets.QMessageBox.Question) - msgBox.setWindowFlags(QtCore.Qt.WindowStaysOnTopHint) - msgBox.setDefaultButton(QtWidgets.QMessageBox.Yes) - reply = msgBox.exec_() - if reply == QtWidgets.QMessageBox.No: - return - self.unsavedKnobs = {} - return - - def saveKnobValue(self, check=True): - ''' Save the text from the editor to the node's knobChanged knob ''' - dropdown_value = self.current_knob_dropdown.itemData( - self.current_knob_dropdown.currentIndex()) - try: - obtained_knobValue = str(self.node[dropdown_value].value()) - self.knob = dropdown_value - except: - error_message = QtWidgets.QMessageBox.information( - None, "", "Unable to find %s.%s" % (self.node.name(), dropdown_value)) - error_message.setWindowFlags(QtCore.Qt.WindowStaysOnTopHint) - error_message.exec_() - return - edited_knobValue = self.script_editor.toPlainText() - if check and obtained_knobValue != edited_knobValue: - msgBox = QtWidgets.QMessageBox() - msgBox.setText("Do you want to overwrite %s.%s?" % - (self.node.name(), dropdown_value)) - msgBox.setStandardButtons( - QtWidgets.QMessageBox.Yes | QtWidgets.QMessageBox.No) - msgBox.setIcon(QtWidgets.QMessageBox.Question) - msgBox.setWindowFlags(QtCore.Qt.WindowStaysOnTopHint) - msgBox.setDefaultButton(QtWidgets.QMessageBox.Yes) - reply = msgBox.exec_() - if reply == QtWidgets.QMessageBox.No: - return - self.node[dropdown_value].setValue(edited_knobValue) - self.setKnobModified( - modified=False, knob=dropdown_value, changeTitle=True) - nuke.tcl("modified 1") - if self.knob in self.unsavedKnobs: - del self.unsavedKnobs[self.knob] - return - - def saveAllKnobValues(self, check=True): - ''' Save all knobs button's function ''' - if self.updateUnsavedKnobs() > 0 and check: - msgBox = QtWidgets.QMessageBox() - msgBox.setText( - "Do you want to save all modified python and callback knobs?") - msgBox.setStandardButtons( - QtWidgets.QMessageBox.Yes | QtWidgets.QMessageBox.No) - msgBox.setIcon(QtWidgets.QMessageBox.Question) - msgBox.setWindowFlags(QtCore.Qt.WindowStaysOnTopHint) - msgBox.setDefaultButton(QtWidgets.QMessageBox.Yes) - reply = msgBox.exec_() - if reply == QtWidgets.QMessageBox.No: - return - saveErrors = 0 - savedCount = 0 - for k in self.unsavedKnobs.copy(): - try: - self.node.knob(k).setValue(self.unsavedKnobs[k]) - del self.unsavedKnobs[k] - savedCount += 1 - nuke.tcl("modified 1") - except: - saveErrors += 1 - if saveErrors > 0: - errorBox = QtWidgets.QMessageBox() - errorBox.setText("Error saving %s knob%s." % - (str(saveErrors), int(saveErrors > 1) * "s")) - errorBox.setIcon(QtWidgets.QMessageBox.Warning) - errorBox.setWindowFlags(QtCore.Qt.WindowStaysOnTopHint) - errorBox.setDefaultButton(QtWidgets.QMessageBox.Yes) - reply = errorBox.exec_() - else: - log("KnobScripter: %s knobs saved" % str(savedCount)) - return - - def setCurrentKnob(self, knobToSet): - ''' Set current knob ''' - KnobDropdownItems = [] - for i in range(self.current_knob_dropdown.count()): - if self.current_knob_dropdown.itemData(i) is not None: - KnobDropdownItems.append( - self.current_knob_dropdown.itemData(i)) - else: - KnobDropdownItems.append("---") - if knobToSet in KnobDropdownItems: - index = KnobDropdownItems.index(knobToSet) - self.current_knob_dropdown.setCurrentIndex(index) - return - - def updateUnsavedKnobs(self, first_time=False): - ''' Clear unchanged knobs from the dict and return the number of unsaved knobs ''' - if not self.node: - # Node has been deleted, so simply return 0. Who cares. - return 0 - edited_knobValue = self.script_editor.toPlainText() - self.unsavedKnobs[self.knob] = edited_knobValue - if len(self.unsavedKnobs) > 0: - for k in self.unsavedKnobs.copy(): - if self.node.knob(k): - if str(self.node.knob(k).value()) == str(self.unsavedKnobs[k]): - del self.unsavedKnobs[k] - else: - del self.unsavedKnobs[k] - # Set appropriate knobs modified... - knobs_dropdown = self.current_knob_dropdown - all_knobs = [knobs_dropdown.itemData(i) - for i in range(knobs_dropdown.count())] - for key in all_knobs: - if key in self.unsavedKnobs.keys(): - self.setKnobModified( - modified=True, knob=key, changeTitle=False) - else: - self.setKnobModified( - modified=False, knob=key, changeTitle=False) - - return len(self.unsavedKnobs) - - def setKnobModified(self, modified=True, knob="", changeTitle=True): - ''' Sets the current knob modified, title and whatever else we need ''' - if knob == "": - knob = self.knob - if modified: - self.modifiedKnobs.add(knob) - else: - self.modifiedKnobs.discard(knob) - - if changeTitle: - title_modified_string = " [modified]" - windowTitle = self.windowTitle().split(title_modified_string)[0] - if modified == True: - windowTitle += title_modified_string - self.setWindowTitle(windowTitle) - - try: - knobs_dropdown = self.current_knob_dropdown - kd_index = knobs_dropdown.currentIndex() - kd_data = knobs_dropdown.itemData(kd_index) - if self.show_labels and i not in defaultKnobs: - kd_data = "{} ({})".format( - self.node.knob(kd_data).label(), kd_data) - if modified == False: - knobs_dropdown.setItemText(kd_index, kd_data) - else: - knobs_dropdown.setItemText(kd_index, kd_data + "(*)") - except: - pass - - # Script Mode - def updateFoldersDropdown(self): - ''' Populate folders dropdown list ''' - self.current_folder_dropdown.blockSignals(True) - self.current_folder_dropdown.clear() # First remove all items - defaultFolders = ["scripts"] - scriptFolders = [] - counter = 0 - for f in defaultFolders: - self.makeScriptFolder(f) - self.current_folder_dropdown.addItem(f + "/", f) - counter += 1 - - try: - scriptFolders = sorted([f for f in os.listdir(self.scripts_dir) if os.path.isdir( - os.path.join(self.scripts_dir, f))]) # Accepts symlinks!!! - except: - log("Couldn't read any script folders.") - - for f in scriptFolders: - fname = f.split("/")[-1] - if fname in defaultFolders: - continue - self.current_folder_dropdown.addItem(fname + "/", fname) - counter += 1 - - # print scriptFolders - if counter > 0: - self.current_folder_dropdown.insertSeparator(counter) - counter += 1 - # self.current_folder_dropdown.insertSeparator(counter) - #counter += 1 - self.current_folder_dropdown.addItem("New", "create new") - self.current_folder_dropdown.addItem("Open...", "open in browser") - self.current_folder_dropdown.addItem("Add custom", "add custom path") - self.folder_index = self.current_folder_dropdown.currentIndex() - self.current_folder = self.current_folder_dropdown.itemData( - self.folder_index) - self.current_folder_dropdown.blockSignals(False) - return - - def updateScriptsDropdown(self): - ''' Populate py scripts dropdown list ''' - self.current_script_dropdown.blockSignals(True) - self.current_script_dropdown.clear() # First remove all items - QtWidgets.QApplication.processEvents() - log("# Updating scripts dropdown...") - log("scripts dir:" + self.scripts_dir) - log("current folder:" + self.current_folder) - log("previous current script:" + self.current_script) - #current_folder = self.current_folder_dropdown.itemData(self.current_folder_dropdown.currentIndex()) - current_folder_path = os.path.join( - self.scripts_dir, self.current_folder) - defaultScripts = ["Untitled.py"] - found_scripts = [] - counter = 0 - # All files and folders inside of the folder - dir_list = os.listdir(current_folder_path) - try: - found_scripts = sorted([f for f in dir_list if f.endswith(".py")]) - found_temp_scripts = [ - f for f in dir_list if f.endswith(".py.autosave")] - except: - log("Couldn't find any scripts in the selected folder.") - if not len(found_scripts): - for s in defaultScripts: - if s + ".autosave" in found_temp_scripts: - self.current_script_dropdown.addItem(s + "(*)", s) - else: - self.current_script_dropdown.addItem(s, s) - counter += 1 - else: - for s in defaultScripts: - if s + ".autosave" in found_temp_scripts: - self.current_script_dropdown.addItem(s + "(*)", s) - elif s in found_scripts: - self.current_script_dropdown.addItem(s, s) - for s in found_scripts: - if s in defaultScripts: - continue - sname = s.split("/")[-1] - if s + ".autosave" in found_temp_scripts: - self.current_script_dropdown.addItem(sname + "(*)", sname) - else: - self.current_script_dropdown.addItem(sname, sname) - counter += 1 - # else: #Add the found scripts to the dropdown - if counter > 0: - counter += 1 - self.current_script_dropdown.insertSeparator(counter) - counter += 1 - self.current_script_dropdown.insertSeparator(counter) - self.current_script_dropdown.addItem("New", "create new") - self.current_script_dropdown.addItem("Duplicate", "create duplicate") - self.current_script_dropdown.addItem("Delete", "delete script") - self.current_script_dropdown.addItem("Open", "open in browser") - #self.script_index = self.current_script_dropdown.currentIndex() - self.script_index = 0 - self.current_script = self.current_script_dropdown.itemData( - self.script_index) - log("Finished updating scripts dropdown.") - log("current_script:" + self.current_script) - self.current_script_dropdown.blockSignals(False) - return - - def makeScriptFolder(self, name="scripts"): - folder_path = os.path.join(self.scripts_dir, name) - if not os.path.exists(folder_path): - try: - os.makedirs(folder_path) - return True - except: - print "Couldn't create the scripting folders.\nPlease check your OS write permissions." - return False - - def makeScriptFile(self, name="Untitled.py", folder="scripts", empty=True): - script_path = os.path.join(self.scripts_dir, self.current_folder, name) - if not os.path.isfile(script_path): - try: - self.current_script_file = open(script_path, 'w') - return True - except: - print "Couldn't create the scripting folders.\nPlease check your OS write permissions." - return False - - def setCurrentFolder(self, folderName): - ''' Set current folder ON THE DROPDOWN ONLY''' - folderList = [self.current_folder_dropdown.itemData( - i) for i in range(self.current_folder_dropdown.count())] - if folderName in folderList: - index = folderList.index(folderName) - self.current_folder_dropdown.setCurrentIndex(index) - self.current_folder = folderName - self.folder_index = self.current_folder_dropdown.currentIndex() - self.current_folder = self.current_folder_dropdown.itemData( - self.folder_index) - return - - def setCurrentScript(self, scriptName): - ''' Set current script ON THE DROPDOWN ONLY ''' - scriptList = [self.current_script_dropdown.itemData( - i) for i in range(self.current_script_dropdown.count())] - if scriptName in scriptList: - index = scriptList.index(scriptName) - self.current_script_dropdown.setCurrentIndex(index) - self.current_script = scriptName - self.script_index = self.current_script_dropdown.currentIndex() - self.current_script = self.current_script_dropdown.itemData( - self.script_index) - return - - def loadScriptContents(self, check=False, pyOnly=False, folder=""): - ''' Get the contents of the selected script and populate the editor ''' - log("# About to load script contents now.") - obtained_scrollValue = 0 - obtained_cursorPosValue = [0, 0] # Position, anchor - if folder == "": - folder = self.current_folder - script_path = os.path.join( - self.scripts_dir, folder, self.current_script) - script_path_temp = script_path + ".autosave" - if (self.current_folder + "/" + self.current_script) in self.scrollPos: - obtained_scrollValue = self.scrollPos[self.current_folder + - "/" + self.current_script] - if (self.current_folder + "/" + self.current_script) in self.cursorPos: - obtained_cursorPosValue = self.cursorPos[self.current_folder + - "/" + self.current_script] - - # 1: If autosave exists and pyOnly is false, load it - if os.path.isfile(script_path_temp) and not pyOnly: - log("Loading .py.autosave file\n---") - with open(script_path_temp, 'r') as script: - content = script.read() - self.script_editor.setPlainText(content) - self.setScriptModified(True) - self.script_editor.verticalScrollBar().setValue(obtained_scrollValue) - - # 2: Try to load the .py as first priority, if it exists - elif os.path.isfile(script_path): - log("Loading .py file\n---") - with open(script_path, 'r') as script: - content = script.read() - current_text = self.script_editor.toPlainText().encode("utf8") - if check and current_text != content and current_text.strip() != "": - msgBox = QtWidgets.QMessageBox() - msgBox.setText("The script has been modified.") - msgBox.setInformativeText( - "Do you want to overwrite the current code on this editor?") - msgBox.setStandardButtons( - QtWidgets.QMessageBox.Yes | QtWidgets.QMessageBox.No) - msgBox.setIcon(QtWidgets.QMessageBox.Question) - msgBox.setWindowFlags(QtCore.Qt.WindowStaysOnTopHint) - msgBox.setDefaultButton(QtWidgets.QMessageBox.Yes) - reply = msgBox.exec_() - if reply == QtWidgets.QMessageBox.No: - return - # Clear trash - if os.path.isfile(script_path_temp): - os.remove(script_path_temp) - log("Removed " + script_path_temp) - self.setScriptModified(False) - self.script_editor.setPlainText(content) - self.script_editor.verticalScrollBar().setValue(obtained_scrollValue) - self.setScriptModified(False) - self.loadScriptState() - self.setScriptState() - - # 3: If .py doesn't exist... only then stick to the autosave - elif os.path.isfile(script_path_temp): - with open(script_path_temp, 'r') as script: - content = script.read() - - msgBox = QtWidgets.QMessageBox() - msgBox.setText("The .py file hasn't been found.") - msgBox.setInformativeText( - "Do you want to clear the current code on this editor?") - msgBox.setStandardButtons( - QtWidgets.QMessageBox.Yes | QtWidgets.QMessageBox.No) - msgBox.setIcon(QtWidgets.QMessageBox.Question) - msgBox.setWindowFlags(QtCore.Qt.WindowStaysOnTopHint) - msgBox.setDefaultButton(QtWidgets.QMessageBox.Yes) - reply = msgBox.exec_() - if reply == QtWidgets.QMessageBox.No: - return - - # Clear trash - os.remove(script_path_temp) - log("Removed " + script_path_temp) - self.script_editor.setPlainText("") - self.updateScriptsDropdown() - self.loadScriptContents(check=False) - self.loadScriptState() - self.setScriptState() - - else: - content = "" - self.script_editor.setPlainText(content) - self.setScriptModified(False) - if self.current_folder + "/" + self.current_script in self.scrollPos: - del self.scrollPos[self.current_folder + - "/" + self.current_script] - if self.current_folder + "/" + self.current_script in self.cursorPos: - del self.cursorPos[self.current_folder + - "/" + self.current_script] - - self.setWindowTitle("KnobScripter - %s/%s" % - (self.current_folder, self.current_script)) - return - - def saveScriptContents(self, temp=True): - ''' Save the current contents of the editor into the python file. If temp == True, saves a .py.autosave file ''' - log("\n# About to save script contents now.") - log("Temp mode is: " + str(temp)) - log("self.current_folder: " + self.current_folder) - log("self.current_script: " + self.current_script) - script_path = os.path.join( - self.scripts_dir, self.current_folder, self.current_script) - script_path_temp = script_path + ".autosave" - orig_content = "" - content = self.script_editor.toPlainText().encode('utf8') - - if temp == True: - if os.path.isfile(script_path): - with open(script_path, 'r') as script: - orig_content = script.read() - # If script path doesn't exist and autosave does but the script is empty... - elif content == "" and os.path.isfile(script_path_temp): - os.remove(script_path_temp) - return - if content != orig_content: - with open(script_path_temp, 'w') as script: - script.write(content) - else: - if os.path.isfile(script_path_temp): - os.remove(script_path_temp) - log("Nothing to save") - return - else: - with open(script_path, 'w') as script: - script.write(self.script_editor.toPlainText().encode('utf8')) - # Clear trash - if os.path.isfile(script_path_temp): - os.remove(script_path_temp) - log("Removed " + script_path_temp) - self.setScriptModified(False) - self.saveScrollValue() - self.saveCursorPosValue() - log("Saved " + script_path + "\n---") - return - - def deleteScript(self, check=True, folder=""): - ''' Get the contents of the selected script and populate the editor ''' - log("# About to delete the .py and/or autosave script now.") - if folder == "": - folder = self.current_folder - script_path = os.path.join( - self.scripts_dir, folder, self.current_script) - script_path_temp = script_path + ".autosave" - if check: - msgBox = QtWidgets.QMessageBox() - msgBox.setText("You're about to delete this script.") - msgBox.setInformativeText( - "Are you sure you want to delete {}?".format(self.current_script)) - msgBox.setStandardButtons( - QtWidgets.QMessageBox.Yes | QtWidgets.QMessageBox.No) - msgBox.setIcon(QtWidgets.QMessageBox.Question) - msgBox.setWindowFlags(QtCore.Qt.WindowStaysOnTopHint) - msgBox.setDefaultButton(QtWidgets.QMessageBox.No) - reply = msgBox.exec_() - if reply == QtWidgets.QMessageBox.No: - return False - - if os.path.isfile(script_path_temp): - os.remove(script_path_temp) - log("Removed " + script_path_temp) - - if os.path.isfile(script_path): - os.remove(script_path) - log("Removed " + script_path) - - return True - - def folderDropdownChanged(self): - '''Executed when the current folder dropdown is changed''' - self.saveScriptState() - log("# folder dropdown changed") - folders_dropdown = self.current_folder_dropdown - fd_value = folders_dropdown.currentText() - fd_index = folders_dropdown.currentIndex() - fd_data = folders_dropdown.itemData(fd_index) - if fd_data == "create new": - panel = FileNameDialog(self, mode="folder") - # panel.setWidth(260) - # panel.addSingleLineInput("Name:","") - if panel.exec_(): - # Accepted - folder_name = panel.text - if os.path.isdir(os.path.join(self.scripts_dir, folder_name)): - self.messageBox("Folder already exists.") - self.setCurrentFolder(self.current_folder) - if self.makeScriptFolder(name=folder_name): - self.saveScriptContents(temp=True) - # Success creating the folder - self.current_folder = folder_name - self.updateFoldersDropdown() - self.setCurrentFolder(folder_name) - self.updateScriptsDropdown() - self.loadScriptContents(check=False) - else: - self.messageBox("There was a problem creating the folder.") - self.current_folder_dropdown.blockSignals(True) - self.current_folder_dropdown.setCurrentIndex( - self.folder_index) - self.current_folder_dropdown.blockSignals(False) - else: - # Canceled/rejected - self.current_folder_dropdown.blockSignals(True) - self.current_folder_dropdown.setCurrentIndex(self.folder_index) - self.current_folder_dropdown.blockSignals(False) - return - - elif fd_data == "open in browser": - current_folder_path = os.path.join( - self.scripts_dir, self.current_folder) - self.openInFileBrowser(current_folder_path) - self.current_folder_dropdown.blockSignals(True) - self.current_folder_dropdown.setCurrentIndex(self.folder_index) - self.current_folder_dropdown.blockSignals(False) - return - - elif fd_data == "add custom path": - folder_path = nuke.getFilename('Select custom folder.') - if folder_path is not None: - if folder_path.endswith("/"): - aliasName = folder_path.split("/")[-2] - else: - aliasName = folder_path.split("/")[-1] - if not os.path.isdir(folder_path): - self.messageBox( - "Folder not found. Please try again with the full path to a folder.") - elif not len(aliasName): - self.messageBox( - "Folder with the same name already exists. Please delete or rename it first.") - else: - # All good - os.symlink(folder_path, os.path.join( - self.scripts_dir, aliasName)) - self.saveScriptContents(temp=True) - self.current_folder = aliasName - self.updateFoldersDropdown() - self.setCurrentFolder(aliasName) - self.updateScriptsDropdown() - self.loadScriptContents(check=False) - self.script_editor.setFocus() - return - self.current_folder_dropdown.blockSignals(True) - self.current_folder_dropdown.setCurrentIndex(self.folder_index) - self.current_folder_dropdown.blockSignals(False) - else: - # 1: Save current script as temp if needed - self.saveScriptContents(temp=True) - # 2: Set the new folder in the variables - self.current_folder = fd_data - self.folder_index = fd_index - # 3: Update the scripts dropdown - self.updateScriptsDropdown() - # 4: Load the current script! - self.loadScriptContents() - self.script_editor.setFocus() - - self.loadScriptState() - self.setScriptState() - - return - - def scriptDropdownChanged(self): - '''Executed when the current script dropdown is changed. Should only be called by the manual dropdown change. Not by other functions.''' - self.saveScriptState() - scripts_dropdown = self.current_script_dropdown - sd_value = scripts_dropdown.currentText() - sd_index = scripts_dropdown.currentIndex() - sd_data = scripts_dropdown.itemData(sd_index) - if sd_data == "create new": - self.current_script_dropdown.blockSignals(True) - panel = FileNameDialog(self, mode="script") - if panel.exec_(): - # Accepted - script_name = panel.text + ".py" - script_path = os.path.join( - self.scripts_dir, self.current_folder, script_name) - log(script_name) - log(script_path) - if os.path.isfile(script_path): - self.messageBox("Script already exists.") - self.current_script_dropdown.setCurrentIndex( - self.script_index) - if self.makeScriptFile(name=script_name, folder=self.current_folder): - # Success creating the folder - self.saveScriptContents(temp=True) - self.updateScriptsDropdown() - if self.current_script != "Untitled.py": - self.script_editor.setPlainText("") - self.current_script = script_name - self.setCurrentScript(script_name) - self.saveScriptContents(temp=False) - # self.loadScriptContents() - else: - self.messageBox("There was a problem creating the script.") - self.current_script_dropdown.setCurrentIndex( - self.script_index) - else: - # Canceled/rejected - self.current_script_dropdown.setCurrentIndex(self.script_index) - return - self.current_script_dropdown.blockSignals(False) - - elif sd_data == "create duplicate": - self.current_script_dropdown.blockSignals(True) - current_folder_path = os.path.join( - self.scripts_dir, self.current_folder, self.current_script) - current_script_path = os.path.join( - self.scripts_dir, self.current_folder, self.current_script) - - current_name = self.current_script - if self.current_script.endswith(".py"): - current_name = current_name[:-3] - - test_name = current_name - while True: - test_name += "_copy" - new_script_path = os.path.join( - self.scripts_dir, self.current_folder, test_name + ".py") - if not os.path.isfile(new_script_path): - break - - script_name = test_name + ".py" - - if self.makeScriptFile(name=script_name, folder=self.current_folder): - # Success creating the folder - self.saveScriptContents(temp=True) - self.updateScriptsDropdown() - # self.script_editor.setPlainText("") - self.current_script = script_name - self.setCurrentScript(script_name) - self.script_editor.setFocus() - else: - self.messageBox("There was a problem duplicating the script.") - self.current_script_dropdown.setCurrentIndex(self.script_index) - - self.current_script_dropdown.blockSignals(False) - - elif sd_data == "open in browser": - current_script_path = os.path.join( - self.scripts_dir, self.current_folder, self.current_script) - self.openInFileBrowser(current_script_path) - self.current_script_dropdown.blockSignals(True) - self.current_script_dropdown.setCurrentIndex(self.script_index) - self.current_script_dropdown.blockSignals(False) - return - - elif sd_data == "delete script": - if self.deleteScript(): - self.updateScriptsDropdown() - self.loadScriptContents() - else: - self.current_script_dropdown.blockSignals(True) - self.current_script_dropdown.setCurrentIndex(self.script_index) - self.current_script_dropdown.blockSignals(False) - - else: - self.saveScriptContents() - self.current_script = sd_data - self.script_index = sd_index - self.setCurrentScript(self.current_script) - self.loadScriptContents() - self.script_editor.setFocus() - self.loadScriptState() - self.setScriptState() - return - - def setScriptModified(self, modified=True): - ''' Sets self.current_script_modified, title and whatever else we need ''' - self.current_script_modified = modified - title_modified_string = " [modified]" - windowTitle = self.windowTitle().split(title_modified_string)[0] - if modified == True: - windowTitle += title_modified_string - self.setWindowTitle(windowTitle) - try: - scripts_dropdown = self.current_script_dropdown - sd_index = scripts_dropdown.currentIndex() - sd_data = scripts_dropdown.itemData(sd_index) - if modified == False: - scripts_dropdown.setItemText(sd_index, sd_data) - else: - scripts_dropdown.setItemText(sd_index, sd_data + "(*)") - except: - pass - - def openInFileBrowser(self, path=""): - OS = platform.system() - if not os.path.exists(path): - path = KS_DIR - if OS == "Windows": - os.startfile(path) - elif OS == "Darwin": - subprocess.Popen(["open", path]) - else: - subprocess.Popen(["xdg-open", path]) - - def loadScriptState(self): - ''' - Loads the last state of the script from a file inside the SE directory's root. - SAVES self.scroll_pos, self.cursor_pos, self.last_open_script - ''' - self.state_dict = {} - if not os.path.isfile(self.state_txt_path): - return False - else: - with open(self.state_txt_path, "r") as f: - self.state_dict = json.load(f) - - log("Loading script state into self.state_dict, self.scrollPos, self.cursorPos") - log(self.state_dict) - - if "scroll_pos" in self.state_dict: - self.scrollPos = self.state_dict["scroll_pos"] - if "cursor_pos" in self.state_dict: - self.cursorPos = self.state_dict["cursor_pos"] - - def setScriptState(self): - ''' - Sets the already script state from self.state_dict into the current script if applicable - ''' - script_fullname = self.current_folder + "/" + self.current_script - - if "scroll_pos" in self.state_dict: - if script_fullname in self.state_dict["scroll_pos"]: - self.script_editor.verticalScrollBar().setValue( - int(self.state_dict["scroll_pos"][script_fullname])) - - if "cursor_pos" in self.state_dict: - if script_fullname in self.state_dict["cursor_pos"]: - cursor = self.script_editor.textCursor() - cursor.setPosition(int( - self.state_dict["cursor_pos"][script_fullname][1]), QtGui.QTextCursor.MoveAnchor) - cursor.setPosition(int( - self.state_dict["cursor_pos"][script_fullname][0]), QtGui.QTextCursor.KeepAnchor) - self.script_editor.setTextCursor(cursor) - - if 'splitter_sizes' in self.state_dict: - self.splitter.setSizes(self.state_dict['splitter_sizes']) - - def setLastScript(self): - if 'last_folder' in self.state_dict and 'last_script' in self.state_dict: - self.updateFoldersDropdown() - self.setCurrentFolder(self.state_dict['last_folder']) - self.updateScriptsDropdown() - self.setCurrentScript(self.state_dict['last_script']) - self.loadScriptContents() - self.script_editor.setFocus() - - def saveScriptState(self): - ''' Stores the current state of the script into a file inside the SE directory's root ''' - log("About to save script state...") - ''' - # self.state_dict = {} - if os.path.isfile(self.state_txt_path): - with open(self.state_txt_path, "r") as f: - self.state_dict = json.load(f) - - if "scroll_pos" in self.state_dict: - self.scrollPos = self.state_dict["scroll_pos"] - if "cursor_pos" in self.state_dict: - self.cursorPos = self.state_dict["cursor_pos"] - - ''' - self.loadScriptState() - - # Overwrite current values into the scriptState - self.saveScrollValue() - self.saveCursorPosValue() - - self.state_dict['scroll_pos'] = self.scrollPos - self.state_dict['cursor_pos'] = self.cursorPos - self.state_dict['last_folder'] = self.current_folder - self.state_dict['last_script'] = self.current_script - self.state_dict['splitter_sizes'] = self.splitter.sizes() - - with open(self.state_txt_path, "w") as f: - state = json.dump(self.state_dict, f, sort_keys=True, indent=4) - return state - - # Autosave background loop - def autosave(self): - if self.toAutosave: - # Save the script... - self.saveScriptContents() - self.toAutosave = False - self.saveScriptState() - log("autosaving...") - return - - # Global stuff - def setTextSelection(self): - self.highlighter.selected_text = self.script_editor.textCursor().selection().toPlainText() - return - - def eventFilter(self, object, event): - if event.type() == QtCore.QEvent.KeyPress: - return QtWidgets.QWidget.eventFilter(self, object, event) - else: - return QtWidgets.QWidget.eventFilter(self, object, event) - - def resizeEvent(self, res_event): - w = self.frameGeometry().width() - self.current_node_label_node.setVisible(w > 460) - self.script_label.setVisible(w > 460) - return super(KnobScripter, self).resizeEvent(res_event) - - def changeClicked(self, newNode=""): - ''' Change node ''' - try: - print "Changing from " + self.node.name() - except: - self.node = None - if not len(nuke.selectedNodes()): - self.exitNodeMode() - return - nuke.menu("Nuke").findItem( - "Edit/Node/Update KnobScripter Context").invoke() - selection = knobScripterSelectedNodes - if self.nodeMode: # Only update the number of unsaved knobs if we were already in node mode - if self.node is not None: - updatedCount = self.updateUnsavedKnobs() - else: - updatedCount = 0 - else: - updatedCount = 0 - self.autosave() - if newNode != "" and nuke.exists(newNode): - selection = [newNode] - elif not len(selection): - node_dialog = ChooseNodeDialog(self) - if node_dialog.exec_(): - # Accepted - selection = [nuke.toNode(node_dialog.name)] - else: - return - - # Change to node mode... - self.node_mode_bar.setVisible(True) - self.script_mode_bar.setVisible(False) - if not self.nodeMode: - self.saveScriptContents() - self.toAutosave = False - self.saveScriptState() - self.splitter.setSizes([0, 1]) - self.nodeMode = True - - # If already selected, pass - if self.node is not None and selection[0].fullName() == self.node.fullName(): - self.messageBox("Please select a different node first!") - return - elif updatedCount > 0: - msgBox = QtWidgets.QMessageBox() - msgBox.setText( - "Save changes to %s knob%s before changing the node?" % (str(updatedCount), int(updatedCount > 1) * "s")) - msgBox.setStandardButtons( - QtWidgets.QMessageBox.Yes | QtWidgets.QMessageBox.No | QtWidgets.QMessageBox.Cancel) - msgBox.setWindowFlags(QtCore.Qt.WindowStaysOnTopHint) - msgBox.setDefaultButton(QtWidgets.QMessageBox.Yes) - reply = msgBox.exec_() - if reply == QtWidgets.QMessageBox.Yes: - self.saveAllKnobValues(check=False) - elif reply == QtWidgets.QMessageBox.Cancel: - return - if len(selection) > 1: - self.messageBox( - "More than one node selected.\nChanging knobChanged editor to %s" % selection[0].fullName()) - # Reinitialise everything, wooo! - self.current_knob_dropdown.blockSignals(True) - self.node = selection[0] - - self.script_editor.setPlainText("") - self.unsavedKnobs = {} - self.scrollPos = {} - self.setWindowTitle("KnobScripter - %s %s" % - (self.node.fullName(), self.knob)) - self.current_node_label_name.setText(self.node.fullName()) - - self.toLoadKnob = False - self.updateKnobDropdown() # onee - # self.current_knob_dropdown.repaint() - # self.current_knob_dropdown.setMinimumWidth(self.current_knob_dropdown.minimumSizeHint().width()) - self.toLoadKnob = True - self.setCurrentKnob(self.knob) - self.loadKnobValue(False) - self.script_editor.setFocus() - self.setKnobModified(False) - self.current_knob_dropdown.blockSignals(False) - # self.current_knob_dropdown.setMinimumContentsLength(80) - return - - def exitNodeMode(self): - self.nodeMode = False - self.setWindowTitle("KnobScripter - Script Mode") - self.node_mode_bar.setVisible(False) - self.script_mode_bar.setVisible(True) - self.node = nuke.toNode("root") - # self.updateFoldersDropdown() - # self.updateScriptsDropdown() - self.splitter.setSizes([1, 1]) - self.loadScriptState() - self.setLastScript() - - self.loadScriptContents(check=False) - self.setScriptState() - - def clearConsole(self): - self.origConsoleText = self.nukeSEOutput.document().toPlainText().encode("utf8") - self.script_output.setPlainText("") - - def toggleFRW(self, frw_pressed): - self.frw_open = frw_pressed - self.frw.setVisible(self.frw_open) - if self.frw_open: - self.frw.find_lineEdit.setFocus() - self.frw.find_lineEdit.selectAll() - else: - self.script_editor.setFocus() - return - - def openSnippets(self): - ''' Whenever the 'snippets' button is pressed... open the panel ''' - global SnippetEditPanel - if SnippetEditPanel == "": - SnippetEditPanel = SnippetsPanel(self) - - if not SnippetEditPanel.isVisible(): - SnippetEditPanel.reload() - - if SnippetEditPanel.show(): - self.snippets = self.loadSnippets(maxDepth=5) - SnippetEditPanel = "" - - def loadSnippets(self, path="", maxDepth=5, depth=0): - ''' - Load prefs recursive. When maximum recursion depth, ignores paths. - ''' - max_depth = maxDepth - cur_depth = depth - if path == "": - path = self.snippets_txt_path - if not os.path.isfile(path): - return {} - else: - loaded_snippets = {} - with open(path, "r") as f: - file = json.load(f) - for i, (key, val) in enumerate(file.items()): - if re.match(r"\[custom-path-[0-9]+\]$", key): - if cur_depth < max_depth: - new_dict = self.loadSnippets( - path=val, maxDepth=max_depth, depth=cur_depth + 1) - loaded_snippets.update(new_dict) - else: - loaded_snippets[key] = val - return loaded_snippets - - def messageBox(self, the_text=""): - ''' Just a simple message box ''' - if self.isPane: - msgBox = QtWidgets.QMessageBox() - else: - msgBox = QtWidgets.QMessageBox(self) - msgBox.setText(the_text) - msgBox.setWindowFlags(QtCore.Qt.WindowStaysOnTopHint) - msgBox.exec_() - - def openPrefs(self): - ''' Open the preferences panel ''' - global PrefsPanel - if PrefsPanel == "": - PrefsPanel = KnobScripterPrefs(self) - - if PrefsPanel.show(): - PrefsPanel = "" - - def loadPrefs(self): - ''' Load prefs ''' - if not os.path.isfile(self.prefs_txt): - return [] - else: - with open(self.prefs_txt, "r") as f: - prefs = json.load(f) - return prefs - - def runScript(self): - ''' Run the current script... ''' - self.script_editor.runScript() - - def saveScrollValue(self): - ''' Save scroll values ''' - if self.nodeMode: - self.scrollPos[self.knob] = self.script_editor.verticalScrollBar( - ).value() - else: - self.scrollPos[self.current_folder + "/" + - self.current_script] = self.script_editor.verticalScrollBar().value() - - def saveCursorPosValue(self): - ''' Save cursor pos and anchor values ''' - self.cursorPos[self.current_folder + "/" + self.current_script] = [ - self.script_editor.textCursor().position(), self.script_editor.textCursor().anchor()] - - def closeEvent(self, close_event): - if self.nodeMode: - updatedCount = self.updateUnsavedKnobs() - if updatedCount > 0: - msgBox = QtWidgets.QMessageBox() - msgBox.setText("Save changes to %s knob%s before closing?" % ( - str(updatedCount), int(updatedCount > 1) * "s")) - msgBox.setStandardButtons( - QtWidgets.QMessageBox.Yes | QtWidgets.QMessageBox.No | QtWidgets.QMessageBox.Cancel) - msgBox.setWindowFlags(QtCore.Qt.WindowStaysOnTopHint) - msgBox.setDefaultButton(QtWidgets.QMessageBox.Yes) - reply = msgBox.exec_() - if reply == QtWidgets.QMessageBox.Yes: - self.saveAllKnobValues(check=False) - close_event.accept() - return - elif reply == QtWidgets.QMessageBox.Cancel: - close_event.ignore() - return - else: - close_event.accept() - else: - self.autosave() - if self in AllKnobScripters: - AllKnobScripters.remove(self) - close_event.accept() - - # Landing functions - - def refreshClicked(self): - ''' Function to refresh the dropdowns ''' - if self.nodeMode: - knob = self.current_knob_dropdown.itemData( - self.current_knob_dropdown.currentIndex()).encode('UTF8') - self.current_knob_dropdown.blockSignals(True) - self.current_knob_dropdown.clear() # First remove all items - self.updateKnobDropdown() - availableKnobs = [] - for i in range(self.current_knob_dropdown.count()): - if self.current_knob_dropdown.itemData(i) is not None: - availableKnobs.append( - self.current_knob_dropdown.itemData(i).encode('UTF8')) - if knob in availableKnobs: - self.setCurrentKnob(knob) - self.current_knob_dropdown.blockSignals(False) - else: - folder = self.current_folder - script = self.current_script - self.autosave() - self.updateFoldersDropdown() - self.setCurrentFolder(folder) - self.updateScriptsDropdown() - self.setCurrentScript(script) - self.script_editor.setFocus() - - def reloadClicked(self): - if self.nodeMode: - self.loadKnobValue() - else: - log("Node mode is off") - self.loadScriptContents(check=True, pyOnly=True) - - def saveClicked(self): - if self.nodeMode: - self.saveKnobValue(False) - else: - self.saveScriptContents(temp=False) - - def setModified(self): - if self.nodeMode: - self.setKnobModified(True) - elif not self.current_script_modified: - self.setScriptModified(True) - if not self.nodeMode: - self.toAutosave = True - - def pin(self, pressed): - if pressed: - self.setWindowFlags(self.windowFlags() | - QtCore.Qt.WindowStaysOnTopHint) - self.pinned = True - self.show() - else: - self.setWindowFlags(self.windowFlags() & ~ - QtCore.Qt.WindowStaysOnTopHint) - self.pinned = False - self.show() - - def findSE(self): - for widget in QtWidgets.QApplication.allWidgets(): - if "Script Editor" in widget.windowTitle(): - return widget - - # FunctiosaveScrollValuens for Nuke's Script Editor - def findScriptEditors(self): - script_editors = [] - for widget in QtWidgets.QApplication.allWidgets(): - if "Script Editor" in widget.windowTitle() and len(widget.children()) > 5: - script_editors.append(widget) - return script_editors - - def findSEInput(self, se): - return se.children()[-1].children()[0] - - def findSEOutput(self, se): - return se.children()[-1].children()[1] - - def findSERunBtn(self, se): - for btn in se.children(): - try: - if "Run the current script" in btn.toolTip(): - return btn - except: - pass - return False - - def setSEOutputEvent(self): - nukeScriptEditors = self.findScriptEditors() - # Take the console from the first script editor found... - self.origConsoleText = self.nukeSEOutput.document().toPlainText().encode("utf8") - for se in nukeScriptEditors: - se_output = self.findSEOutput(se) - se_output.textChanged.connect( - partial(consoleChanged, se_output, self)) - consoleChanged(se_output, self) # Initialise. - - -class KnobScripterPane(KnobScripter): - def __init__(self, node="", knob="knobChanged"): - super(KnobScripterPane, self).__init__() - self.isPane = True - - def showEvent(self, the_event): - try: - killPaneMargins(self) - except: - pass - return KnobScripter.showEvent(self, the_event) - - def hideEvent(self, the_event): - self.autosave() - return KnobScripter.hideEvent(self, the_event) - - -def consoleChanged(self, ks): - ''' This will be called every time the ScriptEditor Output text is changed ''' - try: - if ks: # KS exists - ksOutput = ks.script_output # The console TextEdit widget - ksText = self.document().toPlainText().encode("utf8") - # The text from the console that will be omitted - origConsoleText = ks.origConsoleText - if ksText.startswith(origConsoleText): - ksText = ksText[len(origConsoleText):] - else: - ks.origConsoleText = "" - ksOutput.setPlainText(ksText) - ksOutput.verticalScrollBar().setValue(ksOutput.verticalScrollBar().maximum()) - except: - pass - - -def killPaneMargins(widget_object): - if widget_object: - target_widgets = set() - target_widgets.add(widget_object.parentWidget().parentWidget()) - target_widgets.add(widget_object.parentWidget( - ).parentWidget().parentWidget().parentWidget()) - - for widget_layout in target_widgets: - try: - widget_layout.layout().setContentsMargins(0, 0, 0, 0) - except: - pass - - -def debug(lev=0): - ''' Convenience function to set the KnobScripter on debug mode''' - # levels = [logging.DEBUG, logging.INFO, logging.WARNING, logging.ERROR, logging.CRITICAL] - # for handler in logging.root.handlers[:]: - # logging.root.removeHandler(handler) - # logging.basicConfig(level=levels[lev]) - # Changed to a shitty way for now - global DebugMode - DebugMode = True - - -def log(text): - ''' Display a debug info message. Yes, in a stupid way. I know.''' - global DebugMode - if DebugMode: - print(text) - - -# --------------------------------------------------------------------- -# Dialogs -# --------------------------------------------------------------------- -class FileNameDialog(QtWidgets.QDialog): - ''' - Dialog for creating new... (mode = "folder", "script" or "knob"). - ''' - - def __init__(self, parent=None, mode="folder", text=""): - if parent.isPane: - super(FileNameDialog, self).__init__() - else: - super(FileNameDialog, self).__init__(parent) - #self.setWindowFlags(self.windowFlags() | QtCore.Qt.WindowStaysOnTopHint) - self.mode = mode - self.text = text - - title = "Create new {}.".format(self.mode) - self.setWindowTitle(title) - - self.initUI() - - def initUI(self): - # Widgets - self.name_label = QtWidgets.QLabel("Name: ") - self.name_label.setAlignment( - QtCore.Qt.AlignRight | QtCore.Qt.AlignVCenter) - self.name_lineEdit = QtWidgets.QLineEdit() - self.name_lineEdit.setText(self.text) - self.name_lineEdit.textChanged.connect(self.nameChanged) - - # Buttons - self.button_box = QtWidgets.QDialogButtonBox( - QtWidgets.QDialogButtonBox.Ok | QtWidgets.QDialogButtonBox.Cancel) - self.button_box.button( - QtWidgets.QDialogButtonBox.Ok).setEnabled(self.text != "") - self.button_box.accepted.connect(self.clickedOk) - self.button_box.rejected.connect(self.clickedCancel) - - # Layout - self.master_layout = QtWidgets.QVBoxLayout() - self.name_layout = QtWidgets.QHBoxLayout() - self.name_layout.addWidget(self.name_label) - self.name_layout.addWidget(self.name_lineEdit) - self.master_layout.addLayout(self.name_layout) - self.master_layout.addWidget(self.button_box) - self.setLayout(self.master_layout) - - self.name_lineEdit.setFocus() - self.setMinimumWidth(250) - - def nameChanged(self): - txt = self.name_lineEdit.text() - m = r"[\w]*$" - if self.mode == "knob": # Knobs can't start with a number... - m = r"[a-zA-Z_]+" + m - - if re.match(m, txt) or txt == "": - self.text = txt - else: - self.name_lineEdit.setText(self.text) - - self.button_box.button( - QtWidgets.QDialogButtonBox.Ok).setEnabled(self.text != "") - return - - def clickedOk(self): - self.accept() - return - - def clickedCancel(self): - self.reject() - return - - -class TextInputDialog(QtWidgets.QDialog): - ''' - Simple dialog for a text input. - ''' - - def __init__(self, parent=None, name="", text="", title=""): - if parent.isPane: - super(TextInputDialog, self).__init__() - else: - super(TextInputDialog, self).__init__(parent) - #self.setWindowFlags(self.windowFlags() | QtCore.Qt.WindowStaysOnTopHint) - - self.name = name # title of textinput - self.text = text # default content of textinput - - self.setWindowTitle(title) - - self.initUI() - - def initUI(self): - # Widgets - self.name_label = QtWidgets.QLabel(self.name + ": ") - self.name_label.setAlignment( - QtCore.Qt.AlignRight | QtCore.Qt.AlignVCenter) - self.name_lineEdit = QtWidgets.QLineEdit() - self.name_lineEdit.setText(self.text) - self.name_lineEdit.textChanged.connect(self.nameChanged) - - # Buttons - self.button_box = QtWidgets.QDialogButtonBox( - QtWidgets.QDialogButtonBox.Ok | QtWidgets.QDialogButtonBox.Cancel) - #self.button_box.button(QtWidgets.QDialogButtonBox.Ok).setEnabled(self.text != "") - self.button_box.accepted.connect(self.clickedOk) - self.button_box.rejected.connect(self.clickedCancel) - - # Layout - self.master_layout = QtWidgets.QVBoxLayout() - self.name_layout = QtWidgets.QHBoxLayout() - self.name_layout.addWidget(self.name_label) - self.name_layout.addWidget(self.name_lineEdit) - self.master_layout.addLayout(self.name_layout) - self.master_layout.addWidget(self.button_box) - self.setLayout(self.master_layout) - - self.name_lineEdit.setFocus() - self.setMinimumWidth(250) - - def nameChanged(self): - self.text = self.name_lineEdit.text() - - def clickedOk(self): - self.accept() - return - - def clickedCancel(self): - self.reject() - return - - -class ChooseNodeDialog(QtWidgets.QDialog): - ''' - Dialog for selecting a node by its name. Only admits nodes that exist (including root, preferences...) - ''' - - def __init__(self, parent=None, name=""): - if parent.isPane: - super(ChooseNodeDialog, self).__init__() - else: - super(ChooseNodeDialog, self).__init__(parent) - - self.name = name # Name of node (will be "" by default) - self.allNodes = [] - - self.setWindowTitle("Enter the node's name...") - - self.initUI() - - def initUI(self): - # Widgets - self.name_label = QtWidgets.QLabel("Name: ") - self.name_label.setAlignment( - QtCore.Qt.AlignRight | QtCore.Qt.AlignVCenter) - self.name_lineEdit = QtWidgets.QLineEdit() - self.name_lineEdit.setText(self.name) - self.name_lineEdit.textChanged.connect(self.nameChanged) - - self.allNodes = self.getAllNodes() - completer = QtWidgets.QCompleter(self.allNodes, self) - completer.setCaseSensitivity(QtCore.Qt.CaseInsensitive) - self.name_lineEdit.setCompleter(completer) - - # Buttons - self.button_box = QtWidgets.QDialogButtonBox( - QtWidgets.QDialogButtonBox.Ok | QtWidgets.QDialogButtonBox.Cancel) - self.button_box.button(QtWidgets.QDialogButtonBox.Ok).setEnabled( - nuke.exists(self.name)) - self.button_box.accepted.connect(self.clickedOk) - self.button_box.rejected.connect(self.clickedCancel) - - # Layout - self.master_layout = QtWidgets.QVBoxLayout() - self.name_layout = QtWidgets.QHBoxLayout() - self.name_layout.addWidget(self.name_label) - self.name_layout.addWidget(self.name_lineEdit) - self.master_layout.addLayout(self.name_layout) - self.master_layout.addWidget(self.button_box) - self.setLayout(self.master_layout) - - self.name_lineEdit.setFocus() - self.setMinimumWidth(250) - - def getAllNodes(self): - self.allNodes = [n.fullName() for n in nuke.allNodes( - recurseGroups=True)] # if parent is in current context?? - self.allNodes.extend(["root", "preferences"]) - return self.allNodes - - def nameChanged(self): - self.name = self.name_lineEdit.text() - self.button_box.button(QtWidgets.QDialogButtonBox.Ok).setEnabled( - self.name in self.allNodes) - - def clickedOk(self): - self.accept() - return - - def clickedCancel(self): - self.reject() - return - - -# ------------------------------------------------------------------------------------------------------ -# Script Editor Widget -# Wouter Gilsing built an incredibly useful python script editor for his Hotbox Manager, so I had it -# really easy for this part! -# Starting from his script editor, I changed the style and added the sublime-like functionality. -# I think this bit of code has the potential to get used in many nuke tools. -# Credit to him: http://www.woutergilsing.com/ -# Originally used on W_Hotbox v1.5: http://www.nukepedia.com/python/ui/w_hotbox -# ------------------------------------------------------------------------------------------------------ -class KnobScripterTextEdit(QtWidgets.QPlainTextEdit): - # Signal that will be emitted when the user has changed the text - userChangedEvent = QtCore.Signal() - - def __init__(self, knobScripter=""): - super(KnobScripterTextEdit, self).__init__() - - self.knobScripter = knobScripter - self.selected_text = "" - - # Setup line numbers - if self.knobScripter != "": - self.tabSpaces = self.knobScripter.tabSpaces - else: - self.tabSpaces = 4 - self.lineNumberArea = KSLineNumberArea(self) - self.blockCountChanged.connect(self.updateLineNumberAreaWidth) - self.updateRequest.connect(self.updateLineNumberArea) - self.updateLineNumberAreaWidth() - - # Highlight line - self.cursorPositionChanged.connect(self.highlightCurrentLine) - - # -------------------------------------------------------------------------------------------------- - # This is adapted from an original version by Wouter Gilsing. - # Extract from his original comments: - # While researching the implementation of line number, I had a look at Nuke's Blinkscript node. [..] - # thefoundry.co.uk/products/nuke/developers/100/pythonreference/nukescripts.blinkscripteditor-pysrc.html - # I stripped and modified the useful bits of the line number related parts of the code [..] - # Credits to theFoundry for writing the blinkscripteditor, best example code I could wish for. - # -------------------------------------------------------------------------------------------------- - - def lineNumberAreaWidth(self): - digits = 1 - maxNum = max(1, self.blockCount()) - while (maxNum >= 10): - maxNum /= 10 - digits += 1 - - space = 7 + self.fontMetrics().width('9') * digits - return space - - def updateLineNumberAreaWidth(self): - self.setViewportMargins(self.lineNumberAreaWidth(), 0, 0, 0) - - def updateLineNumberArea(self, rect, dy): - - if (dy): - self.lineNumberArea.scroll(0, dy) - else: - self.lineNumberArea.update( - 0, rect.y(), self.lineNumberArea.width(), rect.height()) - - if (rect.contains(self.viewport().rect())): - self.updateLineNumberAreaWidth() - - def resizeEvent(self, event): - QtWidgets.QPlainTextEdit.resizeEvent(self, event) - - cr = self.contentsRect() - self.lineNumberArea.setGeometry(QtCore.QRect( - cr.left(), cr.top(), self.lineNumberAreaWidth(), cr.height())) - - def lineNumberAreaPaintEvent(self, event): - - if self.isReadOnly(): - return - - painter = QtGui.QPainter(self.lineNumberArea) - painter.fillRect(event.rect(), QtGui.QColor(36, 36, 36)) # Number bg - - block = self.firstVisibleBlock() - blockNumber = block.blockNumber() - top = int(self.blockBoundingGeometry( - block).translated(self.contentOffset()).top()) - bottom = top + int(self.blockBoundingRect(block).height()) - currentLine = self.document().findBlock( - self.textCursor().position()).blockNumber() - - painter.setPen(self.palette().color(QtGui.QPalette.Text)) - - painterFont = QtGui.QFont() - painterFont.setFamily("Courier") - painterFont.setStyleHint(QtGui.QFont.Monospace) - painterFont.setFixedPitch(True) - if self.knobScripter != "": - painterFont.setPointSize(self.knobScripter.fontSize) - painter.setFont(self.knobScripter.script_editor_font) - - while (block.isValid() and top <= event.rect().bottom()): - - textColor = QtGui.QColor(110, 110, 110) # Numbers - - if blockNumber == currentLine and self.hasFocus(): - textColor = QtGui.QColor(255, 170, 0) # Number highlighted - - painter.setPen(textColor) - - number = "%s" % str(blockNumber + 1) - painter.drawText(-3, top, self.lineNumberArea.width(), - self.fontMetrics().height(), QtCore.Qt.AlignRight, number) - - # Move to the next block - block = block.next() - top = bottom - bottom = top + int(self.blockBoundingRect(block).height()) - blockNumber += 1 - - def keyPressEvent(self, event): - ''' - Custom actions for specific keystrokes - ''' - key = event.key() - ctrl = bool(event.modifiers() & Qt.ControlModifier) - alt = bool(event.modifiers() & Qt.AltModifier) - shift = bool(event.modifiers() & Qt.ShiftModifier) - pre_scroll = self.verticalScrollBar().value() - #modifiers = QtWidgets.QApplication.keyboardModifiers() - #ctrl = (modifiers == Qt.ControlModifier) - #shift = (modifiers == Qt.ShiftModifier) - - up_arrow = 16777235 - down_arrow = 16777237 - - # if Tab convert to Space - if key == 16777217: - self.indentation('indent') - - # if Shift+Tab remove indent - elif key == 16777218: - self.indentation('unindent') - - # if BackSpace try to snap to previous indent level - elif key == 16777219: - if not self.unindentBackspace(): - QtWidgets.QPlainTextEdit.keyPressEvent(self, event) - else: - # COOL BEHAVIORS SIMILAR TO SUBLIME GO NEXT! - cursor = self.textCursor() - cpos = cursor.position() - apos = cursor.anchor() - text_before_cursor = self.toPlainText()[:min(cpos, apos)] - text_after_cursor = self.toPlainText()[max(cpos, apos):] - text_all = self.toPlainText() - to_line_start = text_before_cursor[::-1].find("\n") - if to_line_start == -1: - # Position of the start of the line that includes the cursor selection start - linestart_pos = 0 - else: - linestart_pos = len(text_before_cursor) - to_line_start - - to_line_end = text_after_cursor.find("\n") - if to_line_end == -1: - # Position of the end of the line that includes the cursor selection end - lineend_pos = len(text_all) - else: - lineend_pos = max(cpos, apos) + to_line_end - - text_before_lines = text_all[:linestart_pos] - text_after_lines = text_all[lineend_pos:] - if len(text_after_lines) and text_after_lines.startswith("\n"): - text_after_lines = text_after_lines[1:] - text_lines = text_all[linestart_pos:lineend_pos] - - if cursor.hasSelection(): - selection = cursor.selection().toPlainText() - else: - selection = "" - if key == Qt.Key_ParenLeft and (len(selection) > 0 or re.match(r"[\s)}\];]+", text_after_cursor) or not len(text_after_cursor)): # ( - cursor.insertText("(" + selection + ")") - cursor.setPosition(apos + 1, QtGui.QTextCursor.MoveAnchor) - cursor.setPosition(cpos + 1, QtGui.QTextCursor.KeepAnchor) - self.setTextCursor(cursor) - # ) - elif key == Qt.Key_ParenRight and text_after_cursor.startswith(")"): - cursor.movePosition(QtGui.QTextCursor.NextCharacter) - self.setTextCursor(cursor) - elif key == Qt.Key_BracketLeft and (len(selection) > 0 or re.match(r"[\s)}\];]+", text_after_cursor) or not len(text_after_cursor)): # [ - cursor.insertText("[" + selection + "]") - cursor.setPosition(apos + 1, QtGui.QTextCursor.MoveAnchor) - cursor.setPosition(cpos + 1, QtGui.QTextCursor.KeepAnchor) - self.setTextCursor(cursor) - # ] - elif key in [Qt.Key_BracketRight, 43] and text_after_cursor.startswith("]"): - cursor.movePosition(QtGui.QTextCursor.NextCharacter) - self.setTextCursor(cursor) - elif key == Qt.Key_BraceLeft and (len(selection) > 0 or re.match(r"[\s)}\];]+", text_after_cursor) or not len(text_after_cursor)): # { - cursor.insertText("{" + selection + "}") - cursor.setPosition(apos + 1, QtGui.QTextCursor.MoveAnchor) - cursor.setPosition(cpos + 1, QtGui.QTextCursor.KeepAnchor) - self.setTextCursor(cursor) - # } - elif key in [199, Qt.Key_BraceRight] and text_after_cursor.startswith("}"): - cursor.movePosition(QtGui.QTextCursor.NextCharacter) - self.setTextCursor(cursor) - elif key == 34: # " - if len(selection) > 0: - cursor.insertText('"' + selection + '"') - cursor.setPosition(apos + 1, QtGui.QTextCursor.MoveAnchor) - cursor.setPosition(cpos + 1, QtGui.QTextCursor.KeepAnchor) - # and not re.search(r"(?:[\s)\]]+|$)",text_before_cursor): - elif text_after_cursor.startswith('"') and '"' in text_before_cursor.split("\n")[-1]: - cursor.movePosition(QtGui.QTextCursor.NextCharacter) - # If chars after cursor, act normal - elif not re.match(r"(?:[\s)\]]+|$)", text_after_cursor): - QtWidgets.QPlainTextEdit.keyPressEvent(self, event) - # If chars before cursor, act normal - elif not re.search(r"[\s.({\[,]$", text_before_cursor) and text_before_cursor != "": - QtWidgets.QPlainTextEdit.keyPressEvent(self, event) - else: - cursor.insertText('"' + selection + '"') - cursor.setPosition(apos + 1, QtGui.QTextCursor.MoveAnchor) - cursor.setPosition(cpos + 1, QtGui.QTextCursor.KeepAnchor) - self.setTextCursor(cursor) - elif key == 39: # ' - if len(selection) > 0: - cursor.insertText("'" + selection + "'") - cursor.setPosition(apos + 1, QtGui.QTextCursor.MoveAnchor) - cursor.setPosition(cpos + 1, QtGui.QTextCursor.KeepAnchor) - # and not re.search(r"(?:[\s)\]]+|$)",text_before_cursor): - elif text_after_cursor.startswith("'") and "'" in text_before_cursor.split("\n")[-1]: - cursor.movePosition(QtGui.QTextCursor.NextCharacter) - # If chars after cursor, act normal - elif not re.match(r"(?:[\s)\]]+|$)", text_after_cursor): - QtWidgets.QPlainTextEdit.keyPressEvent(self, event) - # If chars before cursor, act normal - elif not re.search(r"[\s.({\[,]$", text_before_cursor) and text_before_cursor != "": - QtWidgets.QPlainTextEdit.keyPressEvent(self, event) - else: - cursor.insertText("'" + selection + "'") - cursor.setPosition(apos + 1, QtGui.QTextCursor.MoveAnchor) - cursor.setPosition(cpos + 1, QtGui.QTextCursor.KeepAnchor) - self.setTextCursor(cursor) - elif key == 35 and len(selection): # (yes, a hash) - # If there's a selection, insert a hash at the start of each line.. how the fuck? - if selection != "": - selection_split = selection.split("\n") - if all(i.startswith("#") for i in selection_split): - selection_commented = "\n".join( - [s[1:] for s in selection_split]) # Uncommented - else: - selection_commented = "#" + "\n#".join(selection_split) - cursor.insertText(selection_commented) - if apos > cpos: - cursor.setPosition( - apos + len(selection_commented) - len(selection), QtGui.QTextCursor.MoveAnchor) - cursor.setPosition(cpos, QtGui.QTextCursor.KeepAnchor) - else: - cursor.setPosition(apos, QtGui.QTextCursor.MoveAnchor) - cursor.setPosition( - cpos + len(selection_commented) - len(selection), QtGui.QTextCursor.KeepAnchor) - self.setTextCursor(cursor) - - elif key == 68 and ctrl and shift: # Ctrl+Shift+D, to duplicate text or line/s - - if not len(selection): - self.setPlainText( - text_before_lines + text_lines + "\n" + text_lines + "\n" + text_after_lines) - cursor.setPosition( - apos + len(text_lines) + 1, QtGui.QTextCursor.MoveAnchor) - cursor.setPosition( - cpos + len(text_lines) + 1, QtGui.QTextCursor.KeepAnchor) - self.setTextCursor(cursor) - self.verticalScrollBar().setValue(pre_scroll) - self.scrollToCursor() - else: - if text_before_cursor.endswith("\n") and not selection.startswith("\n"): - cursor.insertText(selection + "\n" + selection) - cursor.setPosition( - apos + len(selection) + 1, QtGui.QTextCursor.MoveAnchor) - cursor.setPosition( - cpos + len(selection) + 1, QtGui.QTextCursor.KeepAnchor) - else: - cursor.insertText(selection + selection) - cursor.setPosition( - apos + len(selection), QtGui.QTextCursor.MoveAnchor) - cursor.setPosition( - cpos + len(selection), QtGui.QTextCursor.KeepAnchor) - self.setTextCursor(cursor) - - # Ctrl+Shift+Up, to move the selected line/s up - elif key == up_arrow and ctrl and shift and len(text_before_lines): - prev_line_start_distance = text_before_lines[:-1][::-1].find( - "\n") - if prev_line_start_distance == -1: - prev_line_start_pos = 0 # Position of the start of the previous line - else: - prev_line_start_pos = len( - text_before_lines) - 1 - prev_line_start_distance - prev_line = text_before_lines[prev_line_start_pos:] - - text_before_prev_line = text_before_lines[:prev_line_start_pos] - - if prev_line.endswith("\n"): - prev_line = prev_line[:-1] - - if len(text_after_lines): - text_after_lines = "\n" + text_after_lines - - self.setPlainText( - text_before_prev_line + text_lines + "\n" + prev_line + text_after_lines) - cursor.setPosition(apos - len(prev_line) - 1, - QtGui.QTextCursor.MoveAnchor) - cursor.setPosition(cpos - len(prev_line) - 1, - QtGui.QTextCursor.KeepAnchor) - self.setTextCursor(cursor) - self.verticalScrollBar().setValue(pre_scroll) - self.scrollToCursor() - return - - elif key == down_arrow and ctrl and shift: # Ctrl+Shift+Up, to move the selected line/s up - if not len(text_after_lines): - text_after_lines = "" - next_line_end_distance = text_after_lines.find("\n") - if next_line_end_distance == -1: - next_line_end_pos = len(text_all) - else: - next_line_end_pos = next_line_end_distance - next_line = text_after_lines[:next_line_end_pos] - text_after_next_line = text_after_lines[next_line_end_pos:] - - self.setPlainText(text_before_lines + next_line + - "\n" + text_lines + text_after_next_line) - cursor.setPosition(apos + len(next_line) + 1, - QtGui.QTextCursor.MoveAnchor) - cursor.setPosition(cpos + len(next_line) + 1, - QtGui.QTextCursor.KeepAnchor) - self.setTextCursor(cursor) - self.verticalScrollBar().setValue(pre_scroll) - self.scrollToCursor() - return - - # If up key and nothing happens, go to start - elif key == up_arrow and not len(text_before_lines): - if not shift: - cursor.setPosition(0, QtGui.QTextCursor.MoveAnchor) - self.setTextCursor(cursor) - else: - cursor.setPosition(0, QtGui.QTextCursor.KeepAnchor) - self.setTextCursor(cursor) - - # If up key and nothing happens, go to start - elif key == down_arrow and not len(text_after_lines): - if not shift: - cursor.setPosition( - len(text_all), QtGui.QTextCursor.MoveAnchor) - self.setTextCursor(cursor) - else: - cursor.setPosition( - len(text_all), QtGui.QTextCursor.KeepAnchor) - self.setTextCursor(cursor) - - # if enter or return, match indent level - elif key in [16777220, 16777221]: - self.indentNewLine() - else: - QtWidgets.QPlainTextEdit.keyPressEvent(self, event) - - self.scrollToCursor() - - def scrollToCursor(self): - self.cursor = self.textCursor() - # Does nothing, but makes the scroll go to the right place... - self.cursor.movePosition(QtGui.QTextCursor.NoMove) - self.setTextCursor(self.cursor) - - def getCursorInfo(self): - - self.cursor = self.textCursor() - - self.firstChar = self.cursor.selectionStart() - self.lastChar = self.cursor.selectionEnd() - - self.noSelection = False - if self.firstChar == self.lastChar: - self.noSelection = True - - self.originalPosition = self.cursor.position() - self.cursorBlockPos = self.cursor.positionInBlock() - - def unindentBackspace(self): - ''' - #snap to previous indent level - ''' - self.getCursorInfo() - - if not self.noSelection or self.cursorBlockPos == 0: - return False - - # check text in front of cursor - textInFront = self.document().findBlock( - self.firstChar).text()[:self.cursorBlockPos] - - # check whether solely spaces - if textInFront != ' ' * self.cursorBlockPos: - return False - - # snap to previous indent level - spaces = len(textInFront) - for space in range(spaces - ((spaces - 1) / self.tabSpaces) * self.tabSpaces - 1): - self.cursor.deletePreviousChar() - - def indentNewLine(self): - - # in case selection covers multiple line, make it one line first - self.insertPlainText('') - - self.getCursorInfo() - - # check how many spaces after cursor - text = self.document().findBlock(self.firstChar).text() - - textInFront = text[:self.cursorBlockPos] - - if len(textInFront) == 0: - self.insertPlainText('\n') - return - - indentLevel = 0 - for i in textInFront: - if i == ' ': - indentLevel += 1 - else: - break - - indentLevel /= self.tabSpaces - - # find out whether textInFront's last character was a ':' - # if that's the case add another indent. - # ignore any spaces at the end, however also - # make sure textInFront is not just an indent - if textInFront.count(' ') != len(textInFront): - while textInFront[-1] == ' ': - textInFront = textInFront[:-1] - - if textInFront[-1] == ':': - indentLevel += 1 - - # new line - self.insertPlainText('\n') - # match indent - self.insertPlainText(' ' * (self.tabSpaces * indentLevel)) - - def indentation(self, mode): - - pre_scroll = self.verticalScrollBar().value() - self.getCursorInfo() - - # if nothing is selected and mode is set to indent, simply insert as many - # space as needed to reach the next indentation level. - if self.noSelection and mode == 'indent': - - remainingSpaces = self.tabSpaces - \ - (self.cursorBlockPos % self.tabSpaces) - self.insertPlainText(' ' * remainingSpaces) - return - - selectedBlocks = self.findBlocks(self.firstChar, self.lastChar) - beforeBlocks = self.findBlocks( - last=self.firstChar - 1, exclude=selectedBlocks) - afterBlocks = self.findBlocks( - first=self.lastChar + 1, exclude=selectedBlocks) - - beforeBlocksText = self.blocks2list(beforeBlocks) - selectedBlocksText = self.blocks2list(selectedBlocks, mode) - afterBlocksText = self.blocks2list(afterBlocks) - - combinedText = '\n'.join( - beforeBlocksText + selectedBlocksText + afterBlocksText) - - # make sure the line count stays the same - originalBlockCount = len(self.toPlainText().split('\n')) - combinedText = '\n'.join(combinedText.split('\n')[:originalBlockCount]) - - self.clear() - self.setPlainText(combinedText) - - if self.noSelection: - self.cursor.setPosition(self.lastChar) - - # check whether the the original selection was from top to bottom or vice versa - else: - if self.originalPosition == self.firstChar: - first = self.lastChar - last = self.firstChar - firstBlockSnap = QtGui.QTextCursor.EndOfBlock - lastBlockSnap = QtGui.QTextCursor.StartOfBlock - else: - first = self.firstChar - last = self.lastChar - firstBlockSnap = QtGui.QTextCursor.StartOfBlock - lastBlockSnap = QtGui.QTextCursor.EndOfBlock - - self.cursor.setPosition(first) - self.cursor.movePosition( - firstBlockSnap, QtGui.QTextCursor.MoveAnchor) - self.cursor.setPosition(last, QtGui.QTextCursor.KeepAnchor) - self.cursor.movePosition( - lastBlockSnap, QtGui.QTextCursor.KeepAnchor) - - self.setTextCursor(self.cursor) - self.verticalScrollBar().setValue(pre_scroll) - - def findBlocks(self, first=0, last=None, exclude=[]): - blocks = [] - if last == None: - last = self.document().characterCount() - for pos in range(first, last + 1): - block = self.document().findBlock(pos) - if block not in blocks and block not in exclude: - blocks.append(block) - return blocks - - def blocks2list(self, blocks, mode=None): - text = [] - for block in blocks: - blockText = block.text() - if mode == 'unindent': - if blockText.startswith(' ' * self.tabSpaces): - blockText = blockText[self.tabSpaces:] - self.lastChar -= self.tabSpaces - elif blockText.startswith('\t'): - blockText = blockText[1:] - self.lastChar -= 1 - - elif mode == 'indent': - blockText = ' ' * self.tabSpaces + blockText - self.lastChar += self.tabSpaces - - text.append(blockText) - - return text - - def highlightCurrentLine(self): - ''' - Highlight currently selected line - ''' - extraSelections = [] - - selection = QtWidgets.QTextEdit.ExtraSelection() - - lineColor = QtGui.QColor(62, 62, 62, 255) - - selection.format.setBackground(lineColor) - selection.format.setProperty( - QtGui.QTextFormat.FullWidthSelection, True) - selection.cursor = self.textCursor() - selection.cursor.clearSelection() - - extraSelections.append(selection) - - self.setExtraSelections(extraSelections) - self.scrollToCursor() - - def format(self, rgb, style=''): - ''' - Return a QtWidgets.QTextCharFormat with the given attributes. - ''' - color = QtGui.QColor(*rgb) - textFormat = QtGui.QTextCharFormat() - textFormat.setForeground(color) - - if 'bold' in style: - textFormat.setFontWeight(QtGui.QFont.Bold) - if 'italic' in style: - textFormat.setFontItalic(True) - if 'underline' in style: - textFormat.setUnderlineStyle(QtGui.QTextCharFormat.SingleUnderline) - - return textFormat - - -class KSLineNumberArea(QtWidgets.QWidget): - def __init__(self, scriptEditor): - super(KSLineNumberArea, self).__init__(scriptEditor) - - self.scriptEditor = scriptEditor - self.setStyleSheet("text-align: center;") - - def paintEvent(self, event): - self.scriptEditor.lineNumberAreaPaintEvent(event) - return - - -class KSScriptEditorHighlighter(QtGui.QSyntaxHighlighter): - ''' - This is also adapted from an original version by Wouter Gilsing. His comments: - - Modified, simplified version of some code found I found when researching: - wiki.python.org/moin/PyQt/Python%20syntax%20highlighting - They did an awesome job, so credits to them. I only needed to make some - modifications to make it fit my needs. - ''' - - def __init__(self, document, parent=None): - - super(KSScriptEditorHighlighter, self).__init__(document) - self.knobScripter = parent - self.script_editor = self.knobScripter.script_editor - self.selected_text = "" - self.selected_text_prev = "" - self.rules_sublime = "" - - self.styles = { - 'keyword': self.format([238, 117, 181], 'bold'), - 'string': self.format([242, 136, 135]), - 'comment': self.format([143, 221, 144]), - 'numbers': self.format([174, 129, 255]), - 'custom': self.format([255, 170, 0], 'italic'), - 'selected': self.format([255, 255, 255], 'bold underline'), - 'underline': self.format([240, 240, 240], 'underline'), - } - - self.keywords = [ - 'and', 'assert', 'break', 'class', 'continue', 'def', - 'del', 'elif', 'else', 'except', 'exec', 'finally', - 'for', 'from', 'global', 'if', 'import', 'in', - 'is', 'lambda', 'not', 'or', 'pass', 'print', - 'raise', 'return', 'try', 'while', 'yield', 'with', 'as' - ] - - self.operatorKeywords = [ - '=', '==', '!=', '<', '<=', '>', '>=', - '\+', '-', '\*', '/', '//', '\%', '\*\*', - '\+=', '-=', '\*=', '/=', '\%=', - '\^', '\|', '\&', '\~', '>>', '<<' - ] - - self.variableKeywords = ['int', 'str', - 'float', 'bool', 'list', 'dict', 'set'] - - self.numbers = ['True', 'False', 'None'] - self.loadAltStyles() - - self.tri_single = (QtCore.QRegExp("'''"), 1, self.styles['comment']) - self.tri_double = (QtCore.QRegExp('"""'), 2, self.styles['comment']) - - # rules - rules = [] - - rules += [(r'\b%s\b' % i, 0, self.styles['keyword']) - for i in self.keywords] - rules += [(i, 0, self.styles['keyword']) - for i in self.operatorKeywords] - rules += [(r'\b%s\b' % i, 0, self.styles['numbers']) - for i in self.numbers] - - rules += [ - - # integers - (r'\b[0-9]+\b', 0, self.styles['numbers']), - # Double-quoted string, possibly containing escape sequences - (r'"[^"\\]*(\\.[^"\\]*)*"', 0, self.styles['string']), - # Single-quoted string, possibly containing escape sequences - (r"'[^'\\]*(\\.[^'\\]*)*'", 0, self.styles['string']), - # From '#' until a newline - (r'#[^\n]*', 0, self.styles['comment']), - ] - - # Build a QRegExp for each pattern - self.rules_nuke = [(QtCore.QRegExp(pat), index, fmt) - for (pat, index, fmt) in rules] - self.rules = self.rules_nuke - - def loadAltStyles(self): - ''' Loads other color styles apart from Nuke's default. ''' - self.styles_sublime = { - 'base': self.format([255, 255, 255]), - 'keyword': self.format([237, 36, 110]), - 'string': self.format([237, 229, 122]), - 'comment': self.format([125, 125, 125]), - 'numbers': self.format([165, 120, 255]), - 'functions': self.format([184, 237, 54]), - 'blue': self.format([130, 226, 255], 'italic'), - 'arguments': self.format([255, 170, 10], 'italic'), - 'custom': self.format([200, 200, 200], 'italic'), - 'underline': self.format([240, 240, 240], 'underline'), - 'selected': self.format([255, 255, 255], 'bold underline'), - } - - self.keywords_sublime = [ - 'and', 'assert', 'break', 'continue', - 'del', 'elif', 'else', 'except', 'exec', 'finally', - 'for', 'from', 'global', 'if', 'import', 'in', - 'is', 'lambda', 'not', 'or', 'pass', 'print', - 'raise', 'return', 'try', 'while', 'yield', 'with', 'as' - ] - self.operatorKeywords_sublime = [ - '=', '==', '!=', '<', '<=', '>', '>=', - '\+', '-', '\*', '/', '//', '\%', '\*\*', - '\+=', '-=', '\*=', '/=', '\%=', - '\^', '\|', '\&', '\~', '>>', '<<' - ] - - self.baseKeywords_sublime = [ - ',', - ] - - self.customKeywords_sublime = [ - 'nuke', - ] - - self.blueKeywords_sublime = [ - 'def', 'class', 'int', 'str', 'float', 'bool', 'list', 'dict', 'set' - ] - - self.argKeywords_sublime = [ - 'self', - ] - - self.tri_single_sublime = (QtCore.QRegExp( - "'''"), 1, self.styles_sublime['comment']) - self.tri_double_sublime = (QtCore.QRegExp( - '"""'), 2, self.styles_sublime['comment']) - self.numbers_sublime = ['True', 'False', 'None'] - - # rules - - rules = [] - # First turn everything inside parentheses orange - rules += [(r"def [\w]+[\s]*\((.*)\)", 1, - self.styles_sublime['arguments'])] - # Now restore unwanted stuff... - rules += [(i, 0, self.styles_sublime['base']) - for i in self.baseKeywords_sublime] - rules += [(r"[^\(\w),.][\s]*[\w]+", 0, self.styles_sublime['base'])] - - # Everything else - rules += [(r'\b%s\b' % i, 0, self.styles_sublime['keyword']) - for i in self.keywords_sublime] - rules += [(i, 0, self.styles_sublime['keyword']) - for i in self.operatorKeywords_sublime] - rules += [(i, 0, self.styles_sublime['custom']) - for i in self.customKeywords_sublime] - rules += [(r'\b%s\b' % i, 0, self.styles_sublime['blue']) - for i in self.blueKeywords_sublime] - rules += [(i, 0, self.styles_sublime['arguments']) - for i in self.argKeywords_sublime] - rules += [(r'\b%s\b' % i, 0, self.styles_sublime['numbers']) - for i in self.numbers_sublime] - - rules += [ - - # integers - (r'\b[0-9]+\b', 0, self.styles_sublime['numbers']), - # Double-quoted string, possibly containing escape sequences - (r'"[^"\\]*(\\.[^"\\]*)*"', 0, self.styles_sublime['string']), - # Single-quoted string, possibly containing escape sequences - (r"'[^'\\]*(\\.[^'\\]*)*'", 0, self.styles_sublime['string']), - # From '#' until a newline - (r'#[^\n]*', 0, self.styles_sublime['comment']), - # Function definitions - (r"def[\s]+([\w\.]+)", 1, self.styles_sublime['functions']), - # Class definitions - (r"class[\s]+([\w\.]+)", 1, self.styles_sublime['functions']), - # Class argument (which is also a class so must be green) - (r"class[\s]+[\w\.]+[\s]*\((.*)\)", - 1, self.styles_sublime['functions']), - # Function arguments also pick their style... - (r"def[\s]+[\w]+[\s]*\(([\w]+)", 1, - self.styles_sublime['arguments']), - ] - - # Build a QRegExp for each pattern - self.rules_sublime = [(QtCore.QRegExp(pat), index, fmt) - for (pat, index, fmt) in rules] - - def format(self, rgb, style=''): - ''' - Return a QtWidgets.QTextCharFormat with the given attributes. - ''' - - color = QtGui.QColor(*rgb) - textFormat = QtGui.QTextCharFormat() - textFormat.setForeground(color) - - if 'bold' in style: - textFormat.setFontWeight(QtGui.QFont.Bold) - if 'italic' in style: - textFormat.setFontItalic(True) - if 'underline' in style: - textFormat.setUnderlineStyle(QtGui.QTextCharFormat.SingleUnderline) - - return textFormat - - def highlightBlock(self, text): - ''' - Apply syntax highlighting to the given block of text. - ''' - # Do other syntax formatting - - if self.knobScripter.color_scheme: - self.color_scheme = self.knobScripter.color_scheme - else: - self.color_scheme = "nuke" - - if self.color_scheme == "nuke": - self.rules = self.rules_nuke - elif self.color_scheme == "sublime": - self.rules = self.rules_sublime - - for expression, nth, format in self.rules: - index = expression.indexIn(text, 0) - - while index >= 0: - # We actually want the index of the nth match - index = expression.pos(nth) - length = len(expression.cap(nth)) - self.setFormat(index, length, format) - index = expression.indexIn(text, index + length) - - self.setCurrentBlockState(0) - - # Multi-line strings etc. based on selected scheme - if self.color_scheme == "nuke": - in_multiline = self.match_multiline(text, *self.tri_single) - if not in_multiline: - in_multiline = self.match_multiline(text, *self.tri_double) - elif self.color_scheme == "sublime": - in_multiline = self.match_multiline(text, *self.tri_single_sublime) - if not in_multiline: - in_multiline = self.match_multiline( - text, *self.tri_double_sublime) - - # TODO if there's a selection, highlight same occurrences in the full document. If no selection but something highlighted, unhighlight full document. (do it thru regex or sth) - - def match_multiline(self, text, delimiter, in_state, style): - ''' - Check whether highlighting requires multiple lines. - ''' - # If inside triple-single quotes, start at 0 - if self.previousBlockState() == in_state: - start = 0 - add = 0 - # Otherwise, look for the delimiter on this line - else: - start = delimiter.indexIn(text) - # Move past this match - add = delimiter.matchedLength() - - # As long as there's a delimiter match on this line... - while start >= 0: - # Look for the ending delimiter - end = delimiter.indexIn(text, start + add) - # Ending delimiter on this line? - if end >= add: - length = end - start + add + delimiter.matchedLength() - self.setCurrentBlockState(0) - # No; multi-line string - else: - self.setCurrentBlockState(in_state) - length = len(text) - start + add - # Apply formatting - self.setFormat(start, length, style) - # Look for the next match - start = delimiter.indexIn(text, start + length) - - # Return True if still inside a multi-line string, False otherwise - if self.currentBlockState() == in_state: - return True - else: - return False - -# -------------------------------------------------------------------------------------- -# Script Output Widget -# The output logger works the same way as Nuke's python script editor output window -# -------------------------------------------------------------------------------------- - - -class ScriptOutputWidget(QtWidgets.QTextEdit): - def __init__(self, parent=None): - super(ScriptOutputWidget, self).__init__(parent) - self.knobScripter = parent - self.setSizePolicy(QtWidgets.QSizePolicy.Expanding, - QtWidgets.QSizePolicy.Expanding) - self.setMinimumHeight(20) - - def keyPressEvent(self, event): - ctrl = ((event.modifiers() and (Qt.ControlModifier)) != 0) - alt = ((event.modifiers() and (Qt.AltModifier)) != 0) - shift = ((event.modifiers() and (Qt.ShiftModifier)) != 0) - key = event.key() - if type(event) == QtGui.QKeyEvent: - # print event.key() - if key in [32]: # Space - return KnobScripter.keyPressEvent(self.knobScripter, event) - elif key in [Qt.Key_Backspace, Qt.Key_Delete]: - self.knobScripter.clearConsole() - return QtWidgets.QTextEdit.keyPressEvent(self, event) - - # def mousePressEvent(self, QMouseEvent): - # if QMouseEvent.button() == Qt.RightButton: - # self.knobScripter.clearConsole() - # QtWidgets.QTextEdit.mousePressEvent(self, QMouseEvent) - -# --------------------------------------------------------------------- -# Modified KnobScripterTextEdit to include snippets etc. -# --------------------------------------------------------------------- - - -class KnobScripterTextEditMain(KnobScripterTextEdit): - def __init__(self, knobScripter, output=None, parent=None): - super(KnobScripterTextEditMain, self).__init__(knobScripter) - self.knobScripter = knobScripter - self.script_output = output - self.nukeCompleter = None - self.currentNukeCompletion = None - - ######## - # FROM NUKE's SCRIPT EDITOR START - ######## - self.setSizePolicy(QtWidgets.QSizePolicy.Expanding, - QtWidgets.QSizePolicy.Expanding) - - # Setup completer - self.nukeCompleter = QtWidgets.QCompleter(self) - self.nukeCompleter.setWidget(self) - self.nukeCompleter.setCompletionMode( - QtWidgets.QCompleter.UnfilteredPopupCompletion) - self.nukeCompleter.setCaseSensitivity(Qt.CaseSensitive) - try: - self.nukeCompleter.setModel(QtGui.QStringListModel()) - except: - self.nukeCompleter.setModel(QtCore.QStringListModel()) - - self.nukeCompleter.activated.connect(self.insertNukeCompletion) - self.nukeCompleter.highlighted.connect(self.completerHighlightChanged) - ######## - # FROM NUKE's SCRIPT EDITOR END - ######## - - def findLongestEndingMatch(self, text, dic): - ''' - If the text ends with a key in the dictionary, it returns the key and value. - If there are several matches, returns the longest one. - False if no matches. - ''' - longest = 0 # len of longest match - match_key = None - match_snippet = "" - for key, val in dic.items(): - #match = re.search(r"[\s\.({\[,;=+-]"+key+r"(?:[\s)\]\"]+|$)",text) - match = re.search(r"[\s\.({\[,;=+-]" + key + r"$", text) - if match or text == key: - if len(key) > longest: - longest = len(key) - match_key = key - match_snippet = val - if match_key is None: - return False - return match_key, match_snippet - - def placeholderToEnd(self, text, placeholder): - '''Returns distance (int) from the first occurrence of the placeholder, to the end of the string with placeholders removed''' - search = re.search(placeholder, text) - if not search: - return -1 - from_start = search.start() - total = len(re.sub(placeholder, "", text)) - to_end = total - from_start - return to_end - - def addSnippetText(self, snippet_text): - ''' Adds the selected text as a snippet (taking care of $$, $name$ etc) to the script editor ''' - cursor_placeholder_find = r"(? 1: - cursor_len = positions[1] - positions[0] - 2 - - text = re.sub(cursor_placeholder_find, "", text) - self.cursor.insertText(text) - if placeholder_to_end >= 0: - for i in range(placeholder_to_end): - self.cursor.movePosition(QtGui.QTextCursor.PreviousCharacter) - for i in range(cursor_len): - self.cursor.movePosition( - QtGui.QTextCursor.NextCharacter, QtGui.QTextCursor.KeepAnchor) - self.setTextCursor(self.cursor) - - def keyPressEvent(self, event): - - ctrl = bool(event.modifiers() & Qt.ControlModifier) - alt = bool(event.modifiers() & Qt.AltModifier) - shift = bool(event.modifiers() & Qt.ShiftModifier) - key = event.key() - - # ADAPTED FROM NUKE's SCRIPT EDITOR: - # Get completer state - self.nukeCompleterShowing = self.nukeCompleter.popup().isVisible() - - # BEFORE ANYTHING ELSE, IF SPECIAL MODIFIERS SIMPLY IGNORE THE REST - if not self.nukeCompleterShowing and (ctrl or shift or alt): - # Bypassed! - if key not in [Qt.Key_Return, Qt.Key_Enter, Qt.Key_Tab]: - KnobScripterTextEdit.keyPressEvent(self, event) - return - - # If the completer is showing - if self.nukeCompleterShowing: - tc = self.textCursor() - # If we're hitting enter, do completion - if key in [Qt.Key_Return, Qt.Key_Enter, Qt.Key_Tab]: - if not self.currentNukeCompletion: - self.nukeCompleter.setCurrentRow(0) - self.currentNukeCompletion = self.nukeCompleter.currentCompletion() - # print str(self.nukeCompleter.completionModel[0]) - self.insertNukeCompletion(self.currentNukeCompletion) - self.nukeCompleter.popup().hide() - self.nukeCompleterShowing = False - # If you're hitting right or escape, hide the popup - elif key == Qt.Key_Right or key == Qt.Key_Escape: - self.nukeCompleter.popup().hide() - self.nukeCompleterShowing = False - # If you hit tab, escape or ctrl-space, hide the completer - elif key == Qt.Key_Tab or key == Qt.Key_Escape or (ctrl and key == Qt.Key_Space): - self.currentNukeCompletion = "" - self.nukeCompleter.popup().hide() - self.nukeCompleterShowing = False - # If none of the above, update the completion model - else: - QtWidgets.QPlainTextEdit.keyPressEvent(self, event) - # Edit completion model - colNum = tc.columnNumber() - posNum = tc.position() - inputText = self.toPlainText() - inputTextSplit = inputText.splitlines() - runningLength = 0 - currentLine = None - for line in inputTextSplit: - length = len(line) - runningLength += length - if runningLength >= posNum: - currentLine = line - break - runningLength += 1 - if currentLine: - completionPart = currentLine.split(" ")[-1] - if "(" in completionPart: - completionPart = completionPart.split("(")[-1] - self.completeNukePartUnderCursor(completionPart) - return - - if type(event) == QtGui.QKeyEvent: - if key == Qt.Key_Escape: # Close the knobscripter... - self.knobScripter.close() - elif not ctrl and not alt and not shift and event.key() == Qt.Key_Tab: - self.placeholder = "$$" - # 1. Set the cursor - self.cursor = self.textCursor() - - # 2. Save text before and after - cpos = self.cursor.position() - text_before_cursor = self.toPlainText()[:cpos] - line_before_cursor = text_before_cursor.split('\n')[-1] - text_after_cursor = self.toPlainText()[cpos:] - - # 3. Check coincidences in snippets dicts - try: # Meaning snippet found - match_key, match_snippet = self.findLongestEndingMatch( - line_before_cursor, self.knobScripter.snippets) - for i in range(len(match_key)): - self.cursor.deletePreviousChar() - # This function takes care of adding the appropriate snippet and moving the cursor... - self.addSnippetText(match_snippet) - except: # Meaning snippet not found... - # ADAPTED FROM NUKE's SCRIPT EDITOR: - tc = self.textCursor() - allCode = self.toPlainText() - colNum = tc.columnNumber() - posNum = tc.position() - - # ...and if there's text in the editor - if len(allCode.split()) > 0: - # There is text in the editor - currentLine = tc.block().text() - - # If you're not at the end of the line just add a tab - if colNum < len(currentLine): - # If there isn't a ')' directly to the right of the cursor add a tab - if currentLine[colNum:colNum + 1] != ')': - KnobScripterTextEdit.keyPressEvent(self, event) - return - # Else show the completer - else: - completionPart = currentLine[:colNum].split( - " ")[-1] - if "(" in completionPart: - completionPart = completionPart.split( - "(")[-1] - - self.completeNukePartUnderCursor( - completionPart) - - return - - # If you are at the end of the line, - else: - # If there's nothing to the right of you add a tab - if currentLine[colNum - 1:] == "" or currentLine.endswith(" "): - KnobScripterTextEdit.keyPressEvent(self, event) - return - # Else update completionPart and show the completer - completionPart = currentLine.split(" ")[-1] - if "(" in completionPart: - completionPart = completionPart.split("(")[-1] - - self.completeNukePartUnderCursor(completionPart) - return - - KnobScripterTextEdit.keyPressEvent(self, event) - elif event.key() in [Qt.Key_Enter, Qt.Key_Return]: - modifiers = QtWidgets.QApplication.keyboardModifiers() - if modifiers == QtCore.Qt.ControlModifier: - self.runScript() - else: - KnobScripterTextEdit.keyPressEvent(self, event) - else: - KnobScripterTextEdit.keyPressEvent(self, event) - - def getPyObjects(self, text): - ''' Returns a list containing all the functions, classes and variables found within the selected python text (code) ''' - matches = [] - # 1: Remove text inside triple quotes (leaving the quotes) - text_clean = '""'.join(text.split('"""')[::2]) - text_clean = '""'.join(text_clean.split("'''")[::2]) - - # 2: Remove text inside of quotes (leaving the quotes) except if \" - lines = text_clean.split("\n") - text_clean = "" - for line in lines: - line_clean = '""'.join(line.split('"')[::2]) - line_clean = '""'.join(line_clean.split("'")[::2]) - line_clean = line_clean.split("#")[0] - text_clean += line_clean + "\n" - - # 3. Split into segments (lines plus ";") - segments = re.findall(r"[^\n;]+", text_clean) - - # 4. Go case by case. - for s in segments: - # Declared vars - matches += re.findall(r"([\w\.]+)(?=[,\s\w]*=[^=]+$)", s) - # Def functions and arguments - function = re.findall(r"[\s]*def[\s]+([\w\.]+)[\s]*\([\s]*", s) - if len(function): - matches += function - args = re.split(r"[\s]*def[\s]+([\w\.]+)[\s]*\([\s]*", s) - if len(args) > 1: - args = args[-1] - matches += re.findall( - r"(?adrianpueyo.com, 2016-2019') - kspSignature.setOpenExternalLinks(True) - kspSignature.setStyleSheet('''color:#555;font-size:9px;''') - kspSignature.setAlignment(QtCore.Qt.AlignRight) - - fontLabel = QtWidgets.QLabel("Font:") - self.fontBox = QtWidgets.QFontComboBox() - self.fontBox.setCurrentFont(QtGui.QFont(self.font)) - self.fontBox.currentFontChanged.connect(self.fontChanged) - - fontSizeLabel = QtWidgets.QLabel("Font size:") - self.fontSizeBox = QtWidgets.QSpinBox() - self.fontSizeBox.setValue(self.oldFontSize) - self.fontSizeBox.setMinimum(6) - self.fontSizeBox.setMaximum(100) - self.fontSizeBox.valueChanged.connect(self.fontSizeChanged) - - windowWLabel = QtWidgets.QLabel("Width (px):") - windowWLabel.setToolTip("Default window width in pixels") - self.windowWBox = QtWidgets.QSpinBox() - self.windowWBox.setValue(self.knobScripter.windowDefaultSize[0]) - self.windowWBox.setMinimum(200) - self.windowWBox.setMaximum(4000) - self.windowWBox.setToolTip("Default window width in pixels") - - windowHLabel = QtWidgets.QLabel("Height (px):") - windowHLabel.setToolTip("Default window height in pixels") - self.windowHBox = QtWidgets.QSpinBox() - self.windowHBox.setValue(self.knobScripter.windowDefaultSize[1]) - self.windowHBox.setMinimum(100) - self.windowHBox.setMaximum(2000) - self.windowHBox.setToolTip("Default window height in pixels") - - # TODO: "Grab current dimensions" button - - tabSpaceLabel = QtWidgets.QLabel("Tab spaces:") - tabSpaceLabel.setToolTip("Number of spaces to add with the tab key.") - self.tabSpace2 = QtWidgets.QRadioButton("2") - self.tabSpace4 = QtWidgets.QRadioButton("4") - tabSpaceButtonGroup = QtWidgets.QButtonGroup(self) - tabSpaceButtonGroup.addButton(self.tabSpace2) - tabSpaceButtonGroup.addButton(self.tabSpace4) - self.tabSpace2.setChecked(self.knobScripter.tabSpaces == 2) - self.tabSpace4.setChecked(self.knobScripter.tabSpaces == 4) - - pinDefaultLabel = QtWidgets.QLabel("Always on top:") - pinDefaultLabel.setToolTip("Default mode of the PIN toggle.") - self.pinDefaultOn = QtWidgets.QRadioButton("On") - self.pinDefaultOff = QtWidgets.QRadioButton("Off") - pinDefaultButtonGroup = QtWidgets.QButtonGroup(self) - pinDefaultButtonGroup.addButton(self.pinDefaultOn) - pinDefaultButtonGroup.addButton(self.pinDefaultOff) - self.pinDefaultOn.setChecked(self.knobScripter.pinned == True) - self.pinDefaultOff.setChecked(self.knobScripter.pinned == False) - self.pinDefaultOn.clicked.connect(lambda: self.knobScripter.pin(True)) - self.pinDefaultOff.clicked.connect( - lambda: self.knobScripter.pin(False)) - - colorSchemeLabel = QtWidgets.QLabel("Color scheme:") - colorSchemeLabel.setToolTip("Syntax highlighting text style.") - self.colorSchemeSublime = QtWidgets.QRadioButton("subl") - self.colorSchemeNuke = QtWidgets.QRadioButton("nuke") - colorSchemeButtonGroup = QtWidgets.QButtonGroup(self) - colorSchemeButtonGroup.addButton(self.colorSchemeSublime) - colorSchemeButtonGroup.addButton(self.colorSchemeNuke) - colorSchemeButtonGroup.buttonClicked.connect(self.colorSchemeChanged) - self.colorSchemeSublime.setChecked( - self.knobScripter.color_scheme == "sublime") - self.colorSchemeNuke.setChecked( - self.knobScripter.color_scheme == "nuke") - - showLabelsLabel = QtWidgets.QLabel("Show labels:") - showLabelsLabel.setToolTip( - "Display knob labels on the knob dropdown\nOtherwise, shows the internal name only.") - self.showLabelsOn = QtWidgets.QRadioButton("On") - self.showLabelsOff = QtWidgets.QRadioButton("Off") - showLabelsButtonGroup = QtWidgets.QButtonGroup(self) - showLabelsButtonGroup.addButton(self.showLabelsOn) - showLabelsButtonGroup.addButton(self.showLabelsOff) - self.showLabelsOn.setChecked(self.knobScripter.pinned == True) - self.showLabelsOff.setChecked(self.knobScripter.pinned == False) - self.showLabelsOn.clicked.connect(lambda: self.knobScripter.pin(True)) - self.showLabelsOff.clicked.connect( - lambda: self.knobScripter.pin(False)) - - self.buttonBox = QtWidgets.QDialogButtonBox( - QtWidgets.QDialogButtonBox.Ok | QtWidgets.QDialogButtonBox.Cancel) - self.buttonBox.accepted.connect(self.savePrefs) - self.buttonBox.rejected.connect(self.cancelPrefs) - - # Loaded custom values - self.ksPrefs = self.knobScripter.loadPrefs() - if self.ksPrefs != []: - try: - self.fontSizeBox.setValue(self.ksPrefs['font_size']) - self.windowWBox.setValue(self.ksPrefs['window_default_w']) - self.windowHBox.setValue(self.ksPrefs['window_default_h']) - self.tabSpace2.setChecked(self.ksPrefs['tab_spaces'] == 2) - self.tabSpace4.setChecked(self.ksPrefs['tab_spaces'] == 4) - self.pinDefaultOn.setChecked(self.ksPrefs['pin_default'] == 1) - self.pinDefaultOff.setChecked(self.ksPrefs['pin_default'] == 0) - self.showLabelsOn.setChecked(self.ksPrefs['show_labels'] == 1) - self.showLabelsOff.setChecked(self.ksPrefs['show_labels'] == 0) - self.colorSchemeSublime.setChecked( - self.ksPrefs['color_scheme'] == "sublime") - self.colorSchemeNuke.setChecked( - self.ksPrefs['color_scheme'] == "nuke") - except: - pass - - # Layouts - font_layout = QtWidgets.QHBoxLayout() - font_layout.addWidget(fontLabel) - font_layout.addWidget(self.fontBox) - - fontSize_layout = QtWidgets.QHBoxLayout() - fontSize_layout.addWidget(fontSizeLabel) - fontSize_layout.addWidget(self.fontSizeBox) - - windowW_layout = QtWidgets.QHBoxLayout() - windowW_layout.addWidget(windowWLabel) - windowW_layout.addWidget(self.windowWBox) - - windowH_layout = QtWidgets.QHBoxLayout() - windowH_layout.addWidget(windowHLabel) - windowH_layout.addWidget(self.windowHBox) - - tabSpacesButtons_layout = QtWidgets.QHBoxLayout() - tabSpacesButtons_layout.addWidget(self.tabSpace2) - tabSpacesButtons_layout.addWidget(self.tabSpace4) - tabSpaces_layout = QtWidgets.QHBoxLayout() - tabSpaces_layout.addWidget(tabSpaceLabel) - tabSpaces_layout.addLayout(tabSpacesButtons_layout) - - pinDefaultButtons_layout = QtWidgets.QHBoxLayout() - pinDefaultButtons_layout.addWidget(self.pinDefaultOn) - pinDefaultButtons_layout.addWidget(self.pinDefaultOff) - pinDefault_layout = QtWidgets.QHBoxLayout() - pinDefault_layout.addWidget(pinDefaultLabel) - pinDefault_layout.addLayout(pinDefaultButtons_layout) - - showLabelsButtons_layout = QtWidgets.QHBoxLayout() - showLabelsButtons_layout.addWidget(self.showLabelsOn) - showLabelsButtons_layout.addWidget(self.showLabelsOff) - showLabels_layout = QtWidgets.QHBoxLayout() - showLabels_layout.addWidget(showLabelsLabel) - showLabels_layout.addLayout(showLabelsButtons_layout) - - colorSchemeButtons_layout = QtWidgets.QHBoxLayout() - colorSchemeButtons_layout.addWidget(self.colorSchemeSublime) - colorSchemeButtons_layout.addWidget(self.colorSchemeNuke) - colorScheme_layout = QtWidgets.QHBoxLayout() - colorScheme_layout.addWidget(colorSchemeLabel) - colorScheme_layout.addLayout(colorSchemeButtons_layout) - - self.master_layout = QtWidgets.QVBoxLayout() - self.master_layout.addWidget(kspTitle) - self.master_layout.addWidget(kspSignature) - self.master_layout.addWidget(kspLine) - self.master_layout.addLayout(font_layout) - self.master_layout.addLayout(fontSize_layout) - self.master_layout.addLayout(windowW_layout) - self.master_layout.addLayout(windowH_layout) - self.master_layout.addLayout(tabSpaces_layout) - self.master_layout.addLayout(pinDefault_layout) - self.master_layout.addLayout(showLabels_layout) - self.master_layout.addLayout(colorScheme_layout) - self.master_layout.addWidget(self.buttonBox) - self.setLayout(self.master_layout) - self.setFixedSize(self.minimumSize()) - - def savePrefs(self): - self.font = self.fontBox.currentFont().family() - ks_prefs = { - 'font_size': self.fontSizeBox.value(), - 'window_default_w': self.windowWBox.value(), - 'window_default_h': self.windowHBox.value(), - 'tab_spaces': self.tabSpaceValue(), - 'pin_default': self.pinDefaultValue(), - 'show_labels': self.showLabelsValue(), - 'font': self.font, - 'color_scheme': self.colorSchemeValue(), - } - self.knobScripter.script_editor_font.setFamily(self.font) - self.knobScripter.script_editor.setFont( - self.knobScripter.script_editor_font) - self.knobScripter.font = self.font - self.knobScripter.color_scheme = self.colorSchemeValue() - self.knobScripter.tabSpaces = self.tabSpaceValue() - self.knobScripter.script_editor.tabSpaces = self.tabSpaceValue() - with open(self.prefs_txt, "w") as f: - prefs = json.dump(ks_prefs, f, sort_keys=True, indent=4) - self.accept() - self.knobScripter.highlighter.rehighlight() - self.knobScripter.show_labels = self.showLabelsValue() - if self.knobScripter.nodeMode: - self.knobScripter.refreshClicked() - return prefs - - def cancelPrefs(self): - self.knobScripter.script_editor_font.setPointSize(self.oldFontSize) - self.knobScripter.script_editor.setFont( - self.knobScripter.script_editor_font) - self.knobScripter.color_scheme = self.oldScheme - self.knobScripter.highlighter.rehighlight() - self.reject() - - def fontSizeChanged(self): - self.knobScripter.script_editor_font.setPointSize( - self.fontSizeBox.value()) - self.knobScripter.script_editor.setFont( - self.knobScripter.script_editor_font) - return - - def fontChanged(self): - self.font = self.fontBox.currentFont().family() - self.knobScripter.script_editor_font.setFamily(self.font) - self.knobScripter.script_editor.setFont( - self.knobScripter.script_editor_font) - return - - def colorSchemeChanged(self): - self.knobScripter.color_scheme = self.colorSchemeValue() - self.knobScripter.highlighter.rehighlight() - return - - def tabSpaceValue(self): - return 2 if self.tabSpace2.isChecked() else 4 - - def pinDefaultValue(self): - return 1 if self.pinDefaultOn.isChecked() else 0 - - def showLabelsValue(self): - return 1 if self.showLabelsOn.isChecked() else 0 - - def colorSchemeValue(self): - return "nuke" if self.colorSchemeNuke.isChecked() else "sublime" - - def closeEvent(self, event): - self.cancelPrefs() - self.close() - - -def updateContext(): - ''' - Get the current selection of nodes with their appropriate context - Doing this outside the KnobScripter -> forces context update inside groups when needed - ''' - global knobScripterSelectedNodes - knobScripterSelectedNodes = nuke.selectedNodes() - return - -# -------------------------------- -# FindReplace -# -------------------------------- - - -class FindReplaceWidget(QtWidgets.QWidget): - ''' SearchReplace Widget for the knobscripter. FindReplaceWidget(editor = QPlainTextEdit) ''' - - def __init__(self, parent): - super(FindReplaceWidget, self).__init__(parent) - - self.editor = parent.script_editor - - self.initUI() - - def initUI(self): - - # -------------- - # Find Row - # -------------- - - # Widgets - self.find_label = QtWidgets.QLabel("Find:") - # self.find_label.setSizePolicy(QtWidgets.QSizePolicy.Fixed,QtWidgets.QSizePolicy.Fixed) - self.find_label.setFixedWidth(50) - self.find_label.setAlignment( - QtCore.Qt.AlignRight | QtCore.Qt.AlignVCenter) - self.find_lineEdit = QtWidgets.QLineEdit() - self.find_next_button = QtWidgets.QPushButton("Next") - self.find_next_button.clicked.connect(self.find) - self.find_prev_button = QtWidgets.QPushButton("Previous") - self.find_prev_button.clicked.connect(self.findBack) - self.find_lineEdit.returnPressed.connect(self.find_next_button.click) - - # Layout - self.find_layout = QtWidgets.QHBoxLayout() - self.find_layout.addWidget(self.find_label) - self.find_layout.addWidget(self.find_lineEdit, stretch=1) - self.find_layout.addWidget(self.find_next_button) - self.find_layout.addWidget(self.find_prev_button) - - # -------------- - # Replace Row - # -------------- - - # Widgets - self.replace_label = QtWidgets.QLabel("Replace:") - # self.replace_label.setSizePolicy(QtWidgets.QSizePolicy.Fixed,QtWidgets.QSizePolicy.Fixed) - self.replace_label.setFixedWidth(50) - self.replace_label.setAlignment( - QtCore.Qt.AlignRight | QtCore.Qt.AlignVCenter) - self.replace_lineEdit = QtWidgets.QLineEdit() - self.replace_button = QtWidgets.QPushButton("Replace") - self.replace_button.clicked.connect(self.replace) - self.replace_all_button = QtWidgets.QPushButton("Replace All") - self.replace_all_button.clicked.connect( - lambda: self.replace(rep_all=True)) - self.replace_lineEdit.returnPressed.connect(self.replace_button.click) - - # Layout - self.replace_layout = QtWidgets.QHBoxLayout() - self.replace_layout.addWidget(self.replace_label) - self.replace_layout.addWidget(self.replace_lineEdit, stretch=1) - self.replace_layout.addWidget(self.replace_button) - self.replace_layout.addWidget(self.replace_all_button) - - # Info text - self.info_text = QtWidgets.QLabel("") - self.info_text.setVisible(False) - self.info_text.mousePressEvent = lambda x: self.info_text.setVisible( - False) - #f = self.info_text.font() - # f.setItalic(True) - # self.info_text.setFont(f) - # self.info_text.clicked.connect(lambda:self.info_text.setVisible(False)) - - # Divider line - line = QtWidgets.QFrame() - line.setFrameShape(QtWidgets.QFrame.HLine) - line.setFrameShadow(QtWidgets.QFrame.Sunken) - line.setLineWidth(0) - line.setMidLineWidth(1) - line.setFrameShadow(QtWidgets.QFrame.Sunken) - - # -------------- - # Main Layout - # -------------- - - self.layout = QtWidgets.QVBoxLayout() - self.layout.addSpacing(4) - self.layout.addWidget(self.info_text) - self.layout.addLayout(self.find_layout) - self.layout.addLayout(self.replace_layout) - self.layout.setSpacing(4) - try: # >n11 - self.layout.setMargin(2) - except: # 0: # If not found but there are matches, start over - cursor.movePosition(QtGui.QTextCursor.Start) - self.editor.setTextCursor(cursor) - self.editor.find(find_str, flags) - else: - cursor.insertText(rep_str) - self.editor.find( - rep_str, flags | QtGui.QTextDocument.FindBackward) - - cursor.endEditBlock() - self.replace_lineEdit.setFocus() - return - - -# -------------------------------- -# Snippets -# -------------------------------- -class SnippetsPanel(QtWidgets.QDialog): - def __init__(self, parent): - super(SnippetsPanel, self).__init__(parent) - - self.knobScripter = parent - - self.setWindowFlags(self.windowFlags() | - QtCore.Qt.WindowStaysOnTopHint) - self.setWindowTitle("Snippet editor") - - self.snippets_txt_path = self.knobScripter.snippets_txt_path - self.snippets_dict = self.loadSnippetsDict(path=self.snippets_txt_path) - #self.snippets_dict = snippets_dic - - # self.saveSnippets(snippets_dic) - - self.initUI() - self.resize(500, 300) - - def initUI(self): - self.layout = QtWidgets.QVBoxLayout() - - # First Area (Titles) - title_layout = QtWidgets.QHBoxLayout() - shortcuts_label = QtWidgets.QLabel("Shortcut") - code_label = QtWidgets.QLabel("Code snippet") - title_layout.addWidget(shortcuts_label, stretch=1) - title_layout.addWidget(code_label, stretch=2) - self.layout.addLayout(title_layout) - - # Main Scroll area - self.scroll_content = QtWidgets.QWidget() - self.scroll_layout = QtWidgets.QVBoxLayout() - - self.buildSnippetWidgets() - - self.scroll_content.setLayout(self.scroll_layout) - - # Scroll Area Properties - self.scroll = QtWidgets.QScrollArea() - self.scroll.setVerticalScrollBarPolicy(Qt.ScrollBarAlwaysOn) - self.scroll.setHorizontalScrollBarPolicy(Qt.ScrollBarAlwaysOff) - self.scroll.setWidgetResizable(True) - self.scroll.setWidget(self.scroll_content) - - self.layout.addWidget(self.scroll) - - # File knob test - #self.filePath_lineEdit = SnippetFilePath(self) - # self.filePath_lineEdit - # self.layout.addWidget(self.filePath_lineEdit) - - # Lower buttons - self.bottom_layout = QtWidgets.QHBoxLayout() - - self.add_btn = QtWidgets.QPushButton("Add snippet") - self.add_btn.setToolTip("Create empty fields for an extra snippet.") - self.add_btn.clicked.connect(self.addSnippet) - self.bottom_layout.addWidget(self.add_btn) - - self.addPath_btn = QtWidgets.QPushButton("Add custom path") - self.addPath_btn.setToolTip( - "Add a custom path to an external snippets .txt file.") - self.addPath_btn.clicked.connect(self.addCustomPath) - self.bottom_layout.addWidget(self.addPath_btn) - - self.bottom_layout.addStretch() - - self.save_btn = QtWidgets.QPushButton('OK') - self.save_btn.setToolTip( - "Save the snippets into a json file and close the panel.") - self.save_btn.clicked.connect(self.okPressed) - self.bottom_layout.addWidget(self.save_btn) - - self.cancel_btn = QtWidgets.QPushButton("Cancel") - self.cancel_btn.setToolTip("Cancel any new snippets or modifications.") - self.cancel_btn.clicked.connect(self.close) - self.bottom_layout.addWidget(self.cancel_btn) - - self.apply_btn = QtWidgets.QPushButton('Apply') - self.apply_btn.setToolTip("Save the snippets into a json file.") - self.apply_btn.setShortcut('Ctrl+S') - self.apply_btn.clicked.connect(self.applySnippets) - self.bottom_layout.addWidget(self.apply_btn) - - self.help_btn = QtWidgets.QPushButton('Help') - self.help_btn.setShortcut('F1') - self.help_btn.clicked.connect(self.showHelp) - self.bottom_layout.addWidget(self.help_btn) - - self.layout.addLayout(self.bottom_layout) - - self.setLayout(self.layout) - - def reload(self): - ''' - Clears everything without saving and redoes the widgets etc. - Only to be called if the panel isn't shown meaning it's closed. - ''' - for i in reversed(range(self.scroll_layout.count())): - self.scroll_layout.itemAt(i).widget().deleteLater() - - self.snippets_dict = self.loadSnippetsDict(path=self.snippets_txt_path) - - self.buildSnippetWidgets() - - def buildSnippetWidgets(self): - for i, (key, val) in enumerate(self.snippets_dict.items()): - if re.match(r"\[custom-path-[0-9]+\]$", key): - file_edit = SnippetFilePath(val) - self.scroll_layout.insertWidget(-1, file_edit) - else: - snippet_edit = SnippetEdit(key, val, parent=self) - self.scroll_layout.insertWidget(-1, snippet_edit) - - def loadSnippetsDict(self, path=""): - ''' Load prefs. TO REMOVE ''' - if path == "": - path = self.knobScripter.snippets_txt_path - if not os.path.isfile(self.snippets_txt_path): - return {} - else: - with open(self.snippets_txt_path, "r") as f: - self.snippets = json.load(f) - return self.snippets - - def getSnippetsAsDict(self): - dic = {} - num_snippets = self.scroll_layout.count() - path_i = 1 - for s in range(num_snippets): - se = self.scroll_layout.itemAt(s).widget() - if se.__class__.__name__ == "SnippetEdit": - key = se.shortcut_editor.text() - val = se.script_editor.toPlainText() - if key != "": - dic[key] = val - else: - path = se.filepath_lineEdit.text() - if path != "": - dic["[custom-path-{}]".format(str(path_i))] = path - path_i += 1 - return dic - - def saveSnippets(self, snippets=""): - if snippets == "": - snippets = self.getSnippetsAsDict() - with open(self.snippets_txt_path, "w") as f: - prefs = json.dump(snippets, f, sort_keys=True, indent=4) - return prefs - - def applySnippets(self): - self.saveSnippets() - self.knobScripter.snippets = self.knobScripter.loadSnippets(maxDepth=5) - self.knobScripter.loadSnippets() - - def okPressed(self): - self.applySnippets() - self.accept() - - def addSnippet(self, key="", val=""): - se = SnippetEdit(key, val, parent=self) - self.scroll_layout.insertWidget(0, se) - self.show() - return se - - def addCustomPath(self, path=""): - cpe = SnippetFilePath(path) - self.scroll_layout.insertWidget(0, cpe) - self.show() - cpe.browseSnippets() - return cpe - - def showHelp(self): - ''' Create a new snippet, auto-completed with the help ''' - help_key = "help" - help_val = """Snippets are a convenient way to have code blocks that you can call through a shortcut.\n\n1. Simply write a shortcut on the text input field on the left. You can see this one is set to "test".\n\n2. Then, write a code or whatever in this script editor. You can include $$ as the placeholder for where you'll want the mouse cursor to appear.\n\n3. Finally, click OK or Apply to save the snippets. On the main script editor, you'll be able to call any snippet by writing the shortcut (in this example: help) and pressing the Tab key.\n\nIn order to remove a snippet, simply leave the shortcut and contents blank, and save the snippets.""" - help_se = self.addSnippet(help_key, help_val) - help_se.script_editor.resize(160, 160) - - -class SnippetEdit(QtWidgets.QWidget): - ''' Simple widget containing two fields, for the snippet shortcut and content ''' - - def __init__(self, key="", val="", parent=None): - super(SnippetEdit, self).__init__(parent) - - self.knobScripter = parent.knobScripter - self.color_scheme = self.knobScripter.color_scheme - self.layout = QtWidgets.QHBoxLayout() - - self.shortcut_editor = QtWidgets.QLineEdit(self) - f = self.shortcut_editor.font() - f.setWeight(QtGui.QFont.Bold) - self.shortcut_editor.setFont(f) - self.shortcut_editor.setText(str(key)) - #self.script_editor = QtWidgets.QTextEdit(self) - self.script_editor = KnobScripterTextEdit() - self.script_editor.setMinimumHeight(100) - self.script_editor.setStyleSheet( - 'background:#282828;color:#EEE;') # Main Colors - self.highlighter = KSScriptEditorHighlighter( - self.script_editor.document(), self) - self.script_editor_font = self.knobScripter.script_editor_font - self.script_editor.setFont(self.script_editor_font) - self.script_editor.resize(90, 90) - self.script_editor.setPlainText(str(val)) - self.layout.addWidget(self.shortcut_editor, - stretch=1, alignment=Qt.AlignTop) - self.layout.addWidget(self.script_editor, stretch=2) - self.layout.setContentsMargins(0, 0, 0, 0) - - self.setLayout(self.layout) - - -class SnippetFilePath(QtWidgets.QWidget): - ''' Simple widget containing a filepath lineEdit and a button to open the file browser ''' - - def __init__(self, path="", parent=None): - super(SnippetFilePath, self).__init__(parent) - - self.layout = QtWidgets.QHBoxLayout() - - self.custompath_label = QtWidgets.QLabel(self) - self.custompath_label.setText("Custom path: ") - - self.filepath_lineEdit = QtWidgets.QLineEdit(self) - self.filepath_lineEdit.setText(str(path)) - #self.script_editor = QtWidgets.QTextEdit(self) - self.filepath_lineEdit.setStyleSheet( - 'background:#282828;color:#EEE;') # Main Colors - self.script_editor_font = QtGui.QFont() - self.script_editor_font.setFamily("Courier") - self.script_editor_font.setStyleHint(QtGui.QFont.Monospace) - self.script_editor_font.setFixedPitch(True) - self.script_editor_font.setPointSize(11) - self.filepath_lineEdit.setFont(self.script_editor_font) - - self.file_button = QtWidgets.QPushButton(self) - self.file_button.setText("Browse...") - self.file_button.clicked.connect(self.browseSnippets) - - self.layout.addWidget(self.custompath_label) - self.layout.addWidget(self.filepath_lineEdit) - self.layout.addWidget(self.file_button) - self.layout.setContentsMargins(0, 10, 0, 10) - - self.setLayout(self.layout) - - def browseSnippets(self): - ''' Opens file panel for ...snippets.txt ''' - browseLocation = nuke.getFilename('Select snippets file', '*.txt') - - if not browseLocation: - return - - self.filepath_lineEdit.setText(browseLocation) - return - - -# -------------------------------- -# Implementation -# -------------------------------- - -def showKnobScripter(knob="knobChanged"): - selection = nuke.selectedNodes() - if not len(selection): - pan = KnobScripter() - else: - pan = KnobScripter(selection[0], knob) - pan.show() - - -def addKnobScripterPanel(): - global knobScripterPanel - try: - knobScripterPanel = panels.registerWidgetAsPanel('nuke.KnobScripterPane', 'Knob Scripter', - 'com.adrianpueyo.KnobScripterPane') - knobScripterPanel.addToPane(nuke.getPaneFor('Properties.1')) - - except: - knobScripterPanel = panels.registerWidgetAsPanel( - 'nuke.KnobScripterPane', 'Knob Scripter', 'com.adrianpueyo.KnobScripterPane') - - -nuke.KnobScripterPane = KnobScripterPane -log("KS LOADED") -ksShortcut = "alt+z" -addKnobScripterPanel() -nuke.menu('Nuke').addCommand( - 'Edit/Node/Open Floating Knob Scripter', showKnobScripter, ksShortcut) -nuke.menu('Nuke').addCommand('Edit/Node/Update KnobScripter Context', - updateContext).setVisible(False) diff --git a/openpype/hosts/nuke/startup/init.py b/openpype/hosts/nuke/startup/init.py deleted file mode 100644 index d7560814bf..0000000000 --- a/openpype/hosts/nuke/startup/init.py +++ /dev/null @@ -1,4 +0,0 @@ -import nuke - -# default write mov -nuke.knobDefault('Write.mov.colorspace', 'sRGB') diff --git a/openpype/hosts/photoshop/plugins/create/workfile_creator.py b/openpype/hosts/photoshop/plugins/create/workfile_creator.py index 875a9b8a94..43302329f1 100644 --- a/openpype/hosts/photoshop/plugins/create/workfile_creator.py +++ b/openpype/hosts/photoshop/plugins/create/workfile_creator.py @@ -1,4 +1,5 @@ import openpype.hosts.photoshop.api as api +from openpype.client import get_asset_by_name from openpype.pipeline import ( AutoCreator, CreatedInstance, @@ -40,10 +41,7 @@ class PSWorkfileCreator(AutoCreator): task_name = legacy_io.Session["AVALON_TASK"] host_name = legacy_io.Session["AVALON_APP"] if existing_instance is None: - asset_doc = legacy_io.find_one({ - "type": "asset", - "name": asset_name - }) + asset_doc = get_asset_by_name(project_name, asset_name) subset_name = self.get_subset_name( variant, task_name, asset_doc, project_name, host_name ) @@ -67,10 +65,7 @@ class PSWorkfileCreator(AutoCreator): existing_instance["asset"] != asset_name or existing_instance["task"] != task_name ): - asset_doc = legacy_io.find_one({ - "type": "asset", - "name": asset_name - }) + asset_doc = get_asset_by_name(project_name, asset_name) subset_name = self.get_subset_name( variant, task_name, asset_doc, project_name, host_name ) diff --git a/openpype/hosts/standalonepublisher/plugins/publish/collect_bulk_mov_instances.py b/openpype/hosts/standalonepublisher/plugins/publish/collect_bulk_mov_instances.py index 3e7fb19c00..052a97af7d 100644 --- a/openpype/hosts/standalonepublisher/plugins/publish/collect_bulk_mov_instances.py +++ b/openpype/hosts/standalonepublisher/plugins/publish/collect_bulk_mov_instances.py @@ -3,7 +3,7 @@ import json import pyblish.api from openpype.lib import get_subset_name_with_asset_doc -from openpype.pipeline import legacy_io +from openpype.client import get_asset_by_name class CollectBulkMovInstances(pyblish.api.InstancePlugin): @@ -24,12 +24,9 @@ class CollectBulkMovInstances(pyblish.api.InstancePlugin): def process(self, instance): context = instance.context + project_name = context.data["projectEntity"]["name"] asset_name = instance.data["asset"] - - asset_doc = legacy_io.find_one({ - "type": "asset", - "name": asset_name - }) + asset_doc = get_asset_by_name(project_name, asset_name) if not asset_doc: raise AssertionError(( "Couldn't find Asset document with name \"{}\"" @@ -52,7 +49,7 @@ class CollectBulkMovInstances(pyblish.api.InstancePlugin): self.subset_name_variant, task_name, asset_doc, - legacy_io.Session["AVALON_PROJECT"] + project_name ) instance_name = f"{asset_name}_{subset_name}" diff --git a/openpype/hosts/standalonepublisher/plugins/publish/collect_hierarchy.py b/openpype/hosts/standalonepublisher/plugins/publish/collect_hierarchy.py index 77163651c4..7922ca7f31 100644 --- a/openpype/hosts/standalonepublisher/plugins/publish/collect_hierarchy.py +++ b/openpype/hosts/standalonepublisher/plugins/publish/collect_hierarchy.py @@ -3,7 +3,7 @@ import re from copy import deepcopy import pyblish.api -from openpype.pipeline import legacy_io +from openpype.client import get_asset_by_id class CollectHierarchyInstance(pyblish.api.ContextPlugin): @@ -61,27 +61,32 @@ class CollectHierarchyInstance(pyblish.api.ContextPlugin): **instance.data["anatomyData"]) def create_hierarchy(self, instance): - parents = list() - hierarchy = list() - visual_hierarchy = [instance.context.data["assetEntity"]] + asset_doc = instance.context.data["assetEntity"] + project_doc = instance.context.data["projectEntity"] + project_name = project_doc["name"] + visual_hierarchy = [asset_doc] + current_doc = asset_doc while True: - visual_parent = legacy_io.find_one( - {"_id": visual_hierarchy[-1]["data"]["visualParent"]} - ) - if visual_parent: - visual_hierarchy.append(visual_parent) - else: - visual_hierarchy.append( - instance.context.data["projectEntity"]) + visual_parent_id = current_doc["data"]["visualParent"] + visual_parent = None + if visual_parent_id: + visual_parent = get_asset_by_id(project_name, visual_parent_id) + + if not visual_parent: + visual_hierarchy.append(project_doc) break + visual_hierarchy.append(visual_parent) + current_doc = visual_parent # add current selection context hierarchy from standalonepublisher + parents = list() for entity in reversed(visual_hierarchy): parents.append({ "entity_type": entity["data"]["entityType"], "entity_name": entity["name"] }) + hierarchy = list() if self.shot_add_hierarchy: parent_template_patern = re.compile(r"\{([a-z]*?)\}") # fill the parents parts from presets @@ -129,9 +134,8 @@ class CollectHierarchyInstance(pyblish.api.ContextPlugin): self.log.debug(f"Hierarchy: {hierarchy}") self.log.debug(f"parents: {parents}") + tasks_to_add = dict() if self.shot_add_tasks: - tasks_to_add = dict() - project_doc = legacy_io.find_one({"type": "project"}) project_tasks = project_doc["config"]["tasks"] for task_name, task_data in self.shot_add_tasks.items(): _task_data = deepcopy(task_data) @@ -150,9 +154,7 @@ class CollectHierarchyInstance(pyblish.api.ContextPlugin): task_name, list(project_tasks.keys()))) - instance.data["tasks"] = tasks_to_add - else: - instance.data["tasks"] = dict() + instance.data["tasks"] = tasks_to_add # updating hierarchy data instance.data["anatomyData"].update({ diff --git a/openpype/hosts/standalonepublisher/plugins/publish/collect_matching_asset.py b/openpype/hosts/standalonepublisher/plugins/publish/collect_matching_asset.py index 9d94bfdc91..82d7247b2b 100644 --- a/openpype/hosts/standalonepublisher/plugins/publish/collect_matching_asset.py +++ b/openpype/hosts/standalonepublisher/plugins/publish/collect_matching_asset.py @@ -4,7 +4,7 @@ import collections import pyblish.api from pprint import pformat -from openpype.pipeline import legacy_io +from openpype.client import get_assets class CollectMatchingAssetToInstance(pyblish.api.InstancePlugin): @@ -119,8 +119,9 @@ class CollectMatchingAssetToInstance(pyblish.api.InstancePlugin): def _asset_docs_by_parent_id(self, instance): # Query all assets for project and store them by parent's id to list + project_name = instance.context.data["projectEntity"]["name"] asset_docs_by_parent_id = collections.defaultdict(list) - for asset_doc in legacy_io.find({"type": "asset"}): + for asset_doc in get_assets(project_name): parent_id = asset_doc["data"]["visualParent"] asset_docs_by_parent_id[parent_id].append(asset_doc) return asset_docs_by_parent_id diff --git a/openpype/hosts/standalonepublisher/plugins/publish/validate_task_existence.py b/openpype/hosts/standalonepublisher/plugins/publish/validate_task_existence.py index 4c761c7a4c..19ea1a4778 100644 --- a/openpype/hosts/standalonepublisher/plugins/publish/validate_task_existence.py +++ b/openpype/hosts/standalonepublisher/plugins/publish/validate_task_existence.py @@ -1,9 +1,7 @@ import pyblish.api -from openpype.pipeline import ( - PublishXmlValidationError, - legacy_io, -) +from openpype.client import get_assets +from openpype.pipeline import PublishXmlValidationError class ValidateTaskExistence(pyblish.api.ContextPlugin): @@ -20,15 +18,11 @@ class ValidateTaskExistence(pyblish.api.ContextPlugin): for instance in context: asset_names.add(instance.data["asset"]) - asset_docs = legacy_io.find( - { - "type": "asset", - "name": {"$in": list(asset_names)} - }, - { - "name": 1, - "data.tasks": 1 - } + project_name = context.data["projectEntity"]["name"] + asset_docs = get_assets( + project_name, + asset_names=asset_names, + fields=["name", "data.tasks"] ) tasks_by_asset_names = {} for asset_doc in asset_docs: diff --git a/openpype/hosts/tvpaint/api/communication_server.py b/openpype/hosts/tvpaint/api/communication_server.py index 65cb9aa2f3..6ac3e6324c 100644 --- a/openpype/hosts/tvpaint/api/communication_server.py +++ b/openpype/hosts/tvpaint/api/communication_server.py @@ -707,6 +707,9 @@ class BaseCommunicator: if exit_code is not None: self.exit_code = exit_code + if self.exit_code is None: + self.exit_code = 0 + def stop(self): """Stop communication and currently running python process.""" log.info("Stopping communication") diff --git a/openpype/hosts/tvpaint/api/pipeline.py b/openpype/hosts/tvpaint/api/pipeline.py index 60c61a8cbf..0118c0104b 100644 --- a/openpype/hosts/tvpaint/api/pipeline.py +++ b/openpype/hosts/tvpaint/api/pipeline.py @@ -8,6 +8,7 @@ import requests import pyblish.api +from openpype.client import get_project, get_asset_by_name from openpype.hosts import tvpaint from openpype.api import get_current_project_settings from openpype.lib import register_event_callback @@ -442,14 +443,14 @@ def set_context_settings(asset_doc=None): Change fps, resolution and frame start/end. """ - if asset_doc is None: - # Use current session asset if not passed - asset_doc = legacy_io.find_one({ - "type": "asset", - "name": legacy_io.Session["AVALON_ASSET"] - }) - project_doc = legacy_io.find_one({"type": "project"}) + project_name = legacy_io.active_project() + if asset_doc is None: + asset_name = legacy_io.Session["AVALON_ASSET"] + # Use current session asset if not passed + asset_doc = get_asset_by_name(project_name, asset_name) + + project_doc = get_project(project_name) framerate = asset_doc["data"].get("fps") if framerate is None: diff --git a/openpype/hosts/tvpaint/plugins/load/load_workfile.py b/openpype/hosts/tvpaint/plugins/load/load_workfile.py index 0eab083c22..462f12abf0 100644 --- a/openpype/hosts/tvpaint/plugins/load/load_workfile.py +++ b/openpype/hosts/tvpaint/plugins/load/load_workfile.py @@ -1,5 +1,6 @@ import os +from openpype.client import get_project, get_asset_by_name from openpype.lib import ( StringTemplate, get_workfile_template_key_from_context, @@ -44,21 +45,17 @@ class LoadWorkfile(plugin.Loader): # Save workfile. host_name = "tvpaint" + project_name = context.get("project") asset_name = context.get("asset") task_name = context.get("task") # Far cases when there is workfile without context if not asset_name: + project_name = legacy_io.active_project() asset_name = legacy_io.Session["AVALON_ASSET"] task_name = legacy_io.Session["AVALON_TASK"] - project_doc = legacy_io.find_one({ - "type": "project" - }) - asset_doc = legacy_io.find_one({ - "type": "asset", - "name": asset_name - }) - project_name = project_doc["name"] + project_doc = get_project(project_name) + asset_doc = get_asset_by_name(project_name, asset_name) template_key = get_workfile_template_key_from_context( asset_name, diff --git a/openpype/hosts/tvpaint/plugins/publish/collect_instances.py b/openpype/hosts/tvpaint/plugins/publish/collect_instances.py index 782907b65d..9b6d5c4879 100644 --- a/openpype/hosts/tvpaint/plugins/publish/collect_instances.py +++ b/openpype/hosts/tvpaint/plugins/publish/collect_instances.py @@ -2,6 +2,7 @@ import json import copy import pyblish.api +from openpype.client import get_asset_by_name from openpype.lib import get_subset_name_with_asset_doc from openpype.pipeline import legacy_io @@ -92,17 +93,15 @@ class CollectInstances(pyblish.api.ContextPlugin): if family == "review": # Change subset name of review instance + # Project name from workfile context + project_name = context.data["workfile_context"]["project"] + # Collect asset doc to get asset id # - not sure if it's good idea to require asset id in # get_subset_name? asset_name = context.data["workfile_context"]["asset"] - asset_doc = legacy_io.find_one({ - "type": "asset", - "name": asset_name - }) + asset_doc = get_asset_by_name(project_name, asset_name) - # Project name from workfile context - project_name = context.data["workfile_context"]["project"] # Host name from environment variable host_name = context.data["hostName"] # Use empty variant value diff --git a/openpype/hosts/tvpaint/plugins/publish/collect_scene_render.py b/openpype/hosts/tvpaint/plugins/publish/collect_scene_render.py index 2b8dbdc5b4..20c5bb586a 100644 --- a/openpype/hosts/tvpaint/plugins/publish/collect_scene_render.py +++ b/openpype/hosts/tvpaint/plugins/publish/collect_scene_render.py @@ -2,8 +2,8 @@ import json import copy import pyblish.api +from openpype.client import get_asset_by_name from openpype.lib import get_subset_name_with_asset_doc -from openpype.pipeline import legacy_io class CollectRenderScene(pyblish.api.ContextPlugin): @@ -56,14 +56,11 @@ class CollectRenderScene(pyblish.api.ContextPlugin): # - not sure if it's good idea to require asset id in # get_subset_name? workfile_context = context.data["workfile_context"] - asset_name = workfile_context["asset"] - asset_doc = legacy_io.find_one({ - "type": "asset", - "name": asset_name - }) - # Project name from workfile context project_name = context.data["workfile_context"]["project"] + asset_name = workfile_context["asset"] + asset_doc = get_asset_by_name(project_name, asset_name) + # Host name from environment variable host_name = context.data["hostName"] # Variant is using render pass name diff --git a/openpype/hosts/tvpaint/plugins/publish/collect_workfile.py b/openpype/hosts/tvpaint/plugins/publish/collect_workfile.py index 70d92f82e9..88c5f4dbc7 100644 --- a/openpype/hosts/tvpaint/plugins/publish/collect_workfile.py +++ b/openpype/hosts/tvpaint/plugins/publish/collect_workfile.py @@ -2,6 +2,7 @@ import os import json import pyblish.api +from openpype.client import get_asset_by_name from openpype.lib import get_subset_name_with_asset_doc from openpype.pipeline import legacy_io @@ -22,19 +23,17 @@ class CollectWorkfile(pyblish.api.ContextPlugin): basename, ext = os.path.splitext(filename) instance = context.create_instance(name=basename) + # Project name from workfile context + project_name = context.data["workfile_context"]["project"] + # Get subset name of workfile instance # Collect asset doc to get asset id # - not sure if it's good idea to require asset id in # get_subset_name? family = "workfile" asset_name = context.data["workfile_context"]["asset"] - asset_doc = legacy_io.find_one({ - "type": "asset", - "name": asset_name - }) + asset_doc = get_asset_by_name(project_name, asset_name) - # Project name from workfile context - project_name = context.data["workfile_context"]["project"] # Host name from environment variable host_name = os.environ["AVALON_APP"] # Use empty variant value diff --git a/openpype/hosts/webpublisher/plugins/publish/collect_published_files.py b/openpype/hosts/webpublisher/plugins/publish/collect_published_files.py index bdd3caccfd..20e277d794 100644 --- a/openpype/hosts/webpublisher/plugins/publish/collect_published_files.py +++ b/openpype/hosts/webpublisher/plugins/publish/collect_published_files.py @@ -13,9 +13,13 @@ import tempfile import math import pyblish.api + +from openpype.client import ( + get_asset_by_name, + get_last_version_by_subset_name +) from openpype.lib import ( prepare_template_data, - get_asset, get_ffprobe_streams, convert_ffprobe_fps_value, ) @@ -23,7 +27,6 @@ from openpype.lib.plugin_tools import ( parse_json, get_subset_name_with_asset_doc ) -from openpype.pipeline import legacy_io class CollectPublishedFiles(pyblish.api.ContextPlugin): @@ -56,8 +59,9 @@ class CollectPublishedFiles(pyblish.api.ContextPlugin): self.log.info("task_sub:: {}".format(task_subfolders)) + project_name = context.data["project_name"] asset_name = context.data["asset"] - asset_doc = get_asset() + asset_doc = get_asset_by_name(project_name, asset_name) task_name = context.data["task"] task_type = context.data["taskType"] project_name = context.data["project_name"] @@ -80,7 +84,9 @@ class CollectPublishedFiles(pyblish.api.ContextPlugin): family, variant, task_name, asset_doc, project_name=project_name, host_name="webpublisher" ) - version = self._get_last_version(asset_name, subset_name) + 1 + version = self._get_next_version( + project_name, asset_doc, subset_name + ) instance = context.create_instance(subset_name) instance.data["asset"] = asset_name @@ -219,55 +225,19 @@ class CollectPublishedFiles(pyblish.api.ContextPlugin): config["families"], config["tags"]) - def _get_last_version(self, asset_name, subset_name): - """Returns version number or 0 for 'asset' and 'subset'""" - query = [ - { - "$match": {"type": "asset", "name": asset_name} - }, - { - "$lookup": - { - "from": os.environ["AVALON_PROJECT"], - "localField": "_id", - "foreignField": "parent", - "as": "subsets" - } - }, - { - "$unwind": "$subsets" - }, - { - "$match": {"subsets.type": "subset", - "subsets.name": subset_name}}, - { - "$lookup": - { - "from": os.environ["AVALON_PROJECT"], - "localField": "subsets._id", - "foreignField": "parent", - "as": "versions" - } - }, - { - "$unwind": "$versions" - }, - { - "$group": { - "_id": { - "asset_name": "$name", - "subset_name": "$subsets.name" - }, - 'version': {'$max': "$versions.name"} - } - } - ] - version = list(legacy_io.aggregate(query)) + def _get_next_version(self, project_name, asset_doc, subset_name): + """Returns version number or 1 for 'asset' and 'subset'""" - if version: - return version[0].get("version") or 0 - else: - return 0 + version_doc = get_last_version_by_subset_name( + project_name, + subset_name, + asset_doc["_id"], + fields=["name"] + ) + version = 1 + if version_doc: + version += int(version_doc["name"]) + return version def _get_number_of_frames(self, file_url): """Return duration in frames""" diff --git a/openpype/hosts/webpublisher/webserver_service/webpublish_routes.py b/openpype/hosts/webpublisher/webserver_service/webpublish_routes.py index 70324fc39c..4cb3cee8e1 100644 --- a/openpype/hosts/webpublisher/webserver_service/webpublish_routes.py +++ b/openpype/hosts/webpublisher/webserver_service/webpublish_routes.py @@ -2,11 +2,15 @@ import os import json import datetime -from bson.objectid import ObjectId import collections -from aiohttp.web_response import Response import subprocess +from bson.objectid import ObjectId +from aiohttp.web_response import Response +from openpype.client import ( + get_projects, + get_assets, +) from openpype.lib import ( OpenPypeMongoConnection, PypeLogger, @@ -16,30 +20,29 @@ from openpype.lib.remote_publish import ( ERROR_STATUS, REPROCESS_STATUS ) -from openpype.pipeline import AvalonMongoDB -from openpype_modules.avalon_apps.rest_api import _RestApiEndpoint from openpype.settings import get_project_settings +from openpype_modules.webserver.base_routes import RestApiEndpoint + +log = PypeLogger.get_logger("WebpublishRoutes") - -log = PypeLogger.get_logger("WebServer") +class ResourceRestApiEndpoint(RestApiEndpoint): + def __init__(self, resource): + self.resource = resource + super(ResourceRestApiEndpoint, self).__init__() -class RestApiResource: - """Resource carrying needed info and Avalon DB connection for publish.""" - def __init__(self, server_manager, executable, upload_dir, - studio_task_queue=None): - self.server_manager = server_manager - self.upload_dir = upload_dir - self.executable = executable +class WebpublishApiEndpoint(ResourceRestApiEndpoint): + @property + def dbcon(self): + return self.resource.dbcon - if studio_task_queue is None: - studio_task_queue = collections.deque().dequeu - self.studio_task_queue = studio_task_queue - self.dbcon = AvalonMongoDB() - self.dbcon.install() +class JsonApiResource: + """Resource for json manipulation. + All resources handling sending output to REST should inherit from + """ @staticmethod def json_dump_handler(value): if isinstance(value, datetime.datetime): @@ -59,19 +62,33 @@ class RestApiResource: ).encode("utf-8") -class OpenPypeRestApiResource(RestApiResource): +class RestApiResource(JsonApiResource): + """Resource carrying needed info and Avalon DB connection for publish.""" + def __init__(self, server_manager, executable, upload_dir, + studio_task_queue=None): + self.server_manager = server_manager + self.upload_dir = upload_dir + self.executable = executable + + if studio_task_queue is None: + studio_task_queue = collections.deque().dequeu + self.studio_task_queue = studio_task_queue + + +class WebpublishRestApiResource(JsonApiResource): """Resource carrying OP DB connection for storing batch info into DB.""" - def __init__(self, ): + + def __init__(self): mongo_client = OpenPypeMongoConnection.get_mongo_client() database_name = os.environ["OPENPYPE_DATABASE_NAME"] self.dbcon = mongo_client[database_name]["webpublishes"] -class ProjectsEndpoint(_RestApiEndpoint): +class ProjectsEndpoint(ResourceRestApiEndpoint): """Returns list of dict with project info (id, name).""" async def get(self) -> Response: output = [] - for project_doc in self.dbcon.projects(): + for project_doc in get_projects(): ret_val = { "id": project_doc["_id"], "name": project_doc["name"] @@ -84,7 +101,7 @@ class ProjectsEndpoint(_RestApiEndpoint): ) -class HiearchyEndpoint(_RestApiEndpoint): +class HiearchyEndpoint(ResourceRestApiEndpoint): """Returns dictionary with context tree from assets.""" async def get(self, project_name) -> Response: query_projection = { @@ -96,10 +113,7 @@ class HiearchyEndpoint(_RestApiEndpoint): "type": 1, } - asset_docs = self.dbcon.database[project_name].find( - {"type": "asset"}, - query_projection - ) + asset_docs = get_assets(project_name, fields=query_projection.keys()) asset_docs_by_id = { asset_doc["_id"]: asset_doc for asset_doc in asset_docs @@ -183,7 +197,7 @@ class TaskNode(Node): self["attributes"] = {} -class BatchPublishEndpoint(_RestApiEndpoint): +class BatchPublishEndpoint(WebpublishApiEndpoint): """Triggers headless publishing of batch.""" async def post(self, request) -> Response: # Validate existence of openpype executable @@ -288,7 +302,7 @@ class BatchPublishEndpoint(_RestApiEndpoint): ) -class TaskPublishEndpoint(_RestApiEndpoint): +class TaskPublishEndpoint(WebpublishApiEndpoint): """Prepared endpoint triggered after each task - for future development.""" async def post(self, request) -> Response: return Response( @@ -298,8 +312,12 @@ class TaskPublishEndpoint(_RestApiEndpoint): ) -class BatchStatusEndpoint(_RestApiEndpoint): - """Returns dict with info for batch_id.""" +class BatchStatusEndpoint(WebpublishApiEndpoint): + """Returns dict with info for batch_id. + + Uses 'WebpublishRestApiResource'. + """ + async def get(self, batch_id) -> Response: output = self.dbcon.find_one({"batch_id": batch_id}) @@ -318,8 +336,12 @@ class BatchStatusEndpoint(_RestApiEndpoint): ) -class UserReportEndpoint(_RestApiEndpoint): - """Returns list of dict with batch info for user (email address).""" +class UserReportEndpoint(WebpublishApiEndpoint): + """Returns list of dict with batch info for user (email address). + + Uses 'WebpublishRestApiResource'. + """ + async def get(self, user) -> Response: output = list(self.dbcon.find({"user": user}, projection={"log": False})) @@ -338,7 +360,7 @@ class UserReportEndpoint(_RestApiEndpoint): ) -class ConfiguredExtensionsEndpoint(_RestApiEndpoint): +class ConfiguredExtensionsEndpoint(WebpublishApiEndpoint): """Returns dict of extensions which have mapping to family. Returns: @@ -378,8 +400,12 @@ class ConfiguredExtensionsEndpoint(_RestApiEndpoint): ) -class BatchReprocessEndpoint(_RestApiEndpoint): - """Marks latest 'batch_id' for reprocessing, returns 404 if not found.""" +class BatchReprocessEndpoint(WebpublishApiEndpoint): + """Marks latest 'batch_id' for reprocessing, returns 404 if not found. + + Uses 'WebpublishRestApiResource'. + """ + async def post(self, batch_id) -> Response: batches = self.dbcon.find({"batch_id": batch_id, "status": ERROR_STATUS}).sort("_id", -1) diff --git a/openpype/hosts/webpublisher/webserver_service/webserver_cli.py b/openpype/hosts/webpublisher/webserver_service/webserver_cli.py index 909ea38bc6..1ed8f22b2c 100644 --- a/openpype/hosts/webpublisher/webserver_service/webserver_cli.py +++ b/openpype/hosts/webpublisher/webserver_service/webserver_cli.py @@ -10,7 +10,7 @@ from openpype.lib import PypeLogger from .webpublish_routes import ( RestApiResource, - OpenPypeRestApiResource, + WebpublishRestApiResource, HiearchyEndpoint, ProjectsEndpoint, ConfiguredExtensionsEndpoint, @@ -27,7 +27,7 @@ from openpype.lib.remote_publish import ( ) -log = PypeLogger().get_logger("webserver_gui") +log = PypeLogger.get_logger("webserver_gui") def run_webserver(*args, **kwargs): @@ -69,16 +69,14 @@ def run_webserver(*args, **kwargs): ) # triggers publish - webpublisher_task_publish_endpoint = \ - BatchPublishEndpoint(resource) + webpublisher_task_publish_endpoint = BatchPublishEndpoint(resource) server_manager.add_route( "POST", "/api/webpublish/batch", webpublisher_task_publish_endpoint.dispatch ) - webpublisher_batch_publish_endpoint = \ - TaskPublishEndpoint(resource) + webpublisher_batch_publish_endpoint = TaskPublishEndpoint(resource) server_manager.add_route( "POST", "/api/webpublish/task", @@ -86,27 +84,26 @@ def run_webserver(*args, **kwargs): ) # reporting - openpype_resource = OpenPypeRestApiResource() - batch_status_endpoint = BatchStatusEndpoint(openpype_resource) + webpublish_resource = WebpublishRestApiResource() + batch_status_endpoint = BatchStatusEndpoint(webpublish_resource) server_manager.add_route( "GET", "/api/batch_status/{batch_id}", batch_status_endpoint.dispatch ) - user_status_endpoint = UserReportEndpoint(openpype_resource) + user_status_endpoint = UserReportEndpoint(webpublish_resource) server_manager.add_route( "GET", "/api/publishes/{user}", user_status_endpoint.dispatch ) - webpublisher_batch_reprocess_endpoint = \ - BatchReprocessEndpoint(openpype_resource) + batch_reprocess_endpoint = BatchReprocessEndpoint(webpublish_resource) server_manager.add_route( "POST", "/api/webpublish/reprocess/{batch_id}", - webpublisher_batch_reprocess_endpoint.dispatch + batch_reprocess_endpoint.dispatch ) server_manager.start_server() diff --git a/openpype/lib/avalon_context.py b/openpype/lib/avalon_context.py index 9d8a92cfe9..a03f066300 100644 --- a/openpype/lib/avalon_context.py +++ b/openpype/lib/avalon_context.py @@ -7,7 +7,6 @@ import platform import logging import collections import functools -import getpass from bson.objectid import ObjectId @@ -19,6 +18,7 @@ from .anatomy import Anatomy from .profiles_filtering import filter_profiles from .events import emit_event from .path_templates import StringTemplate +from .local_settings import get_openpype_username legacy_io = None @@ -550,7 +550,7 @@ def get_workdir_data(project_doc, asset_doc, task_name, host_name): "asset": asset_doc["name"], "parent": parent_name, "app": host_name, - "user": getpass.getuser(), + "user": get_openpype_username(), "hierarchy": hierarchy, } @@ -797,8 +797,14 @@ def update_current_task(task=None, asset=None, app=None, template_key=None): else: os.environ[key] = value + data = changes.copy() + # Convert env keys to human readable keys + data["project_name"] = legacy_io.Session["AVALON_PROJECT"] + data["asset_name"] = legacy_io.Session["AVALON_ASSET"] + data["task_name"] = legacy_io.Session["AVALON_TASK"] + # Emit session change - emit_event("taskChanged", changes.copy()) + emit_event("taskChanged", data) return changes diff --git a/openpype/lib/transcoding.py b/openpype/lib/transcoding.py index adb9bb2c3a..ee9a0f08de 100644 --- a/openpype/lib/transcoding.py +++ b/openpype/lib/transcoding.py @@ -533,7 +533,7 @@ def convert_input_paths_for_ffmpeg( output_dir, logger=None ): - """Contert source file to format supported in ffmpeg. + """Convert source file to format supported in ffmpeg. Currently can convert only exrs. The input filepaths should be files with same type. Information about input is loaded only from first found diff --git a/openpype/modules/base.py b/openpype/modules/base.py index bca64b19f8..b9ccec13cc 100644 --- a/openpype/modules/base.py +++ b/openpype/modules/base.py @@ -463,6 +463,25 @@ class OpenPypeModule: pass + def on_host_install(self, host, host_name, project_name): + """Host was installed which gives option to handle in-host logic. + + It is a good option to register in-host event callbacks which are + specific for the module. The module is kept in memory for rest of + the process. + + Arguments may change in future. E.g. 'host_name' should be possible + to receive from 'host' object. + + Args: + host (ModuleType): Access to installed/registered host object. + host_name (str): Name of host. + project_name (str): Project name which is main part of host + context. + """ + + pass + def cli(self, module_click_group): """Add commands to click group. diff --git a/openpype/modules/deadline/plugins/publish/submit_aftereffects_deadline.py b/openpype/modules/deadline/plugins/publish/submit_aftereffects_deadline.py index b6584f239e..de8df3dd9e 100644 --- a/openpype/modules/deadline/plugins/publish/submit_aftereffects_deadline.py +++ b/openpype/modules/deadline/plugins/publish/submit_aftereffects_deadline.py @@ -33,6 +33,7 @@ class AfterEffectsSubmitDeadline( hosts = ["aftereffects"] families = ["render.farm"] # cannot be "render' as that is integrated use_published = True + targets = ["local"] priority = 50 chunk_size = 1000000 diff --git a/openpype/modules/deadline/plugins/publish/submit_harmony_deadline.py b/openpype/modules/deadline/plugins/publish/submit_harmony_deadline.py index 912f0f4026..a1ee5e0957 100644 --- a/openpype/modules/deadline/plugins/publish/submit_harmony_deadline.py +++ b/openpype/modules/deadline/plugins/publish/submit_harmony_deadline.py @@ -238,6 +238,7 @@ class HarmonySubmitDeadline( order = pyblish.api.IntegratorOrder + 0.1 hosts = ["harmony"] families = ["render.farm"] + targets = ["local"] optional = True use_published = False @@ -321,7 +322,9 @@ class HarmonySubmitDeadline( ) unzip_dir = (published_scene.parent / published_scene.stem) with _ZipFile(published_scene, "r") as zip_ref: - zip_ref.extractall(unzip_dir.as_posix()) + # UNC path (//?/) added to minimalize risk with extracting + # to large file paths + zip_ref.extractall("//?/" + str(unzip_dir.as_posix())) # find any xstage files in directory, prefer the one with the same name # as directory (plus extension) diff --git a/openpype/modules/deadline/plugins/publish/submit_maya_deadline.py b/openpype/modules/deadline/plugins/publish/submit_maya_deadline.py index 8562c85f7d..9964e3c646 100644 --- a/openpype/modules/deadline/plugins/publish/submit_maya_deadline.py +++ b/openpype/modules/deadline/plugins/publish/submit_maya_deadline.py @@ -287,6 +287,7 @@ class MayaSubmitDeadline(pyblish.api.InstancePlugin): order = pyblish.api.IntegratorOrder + 0.1 hosts = ["maya"] families = ["renderlayer"] + targets = ["local"] use_published = True tile_assembler_plugin = "OpenPypeTileAssembler" diff --git a/openpype/modules/deadline/plugins/publish/submit_maya_remote_publish_deadline.py b/openpype/modules/deadline/plugins/publish/submit_maya_remote_publish_deadline.py index 4f82818d6d..57572fcb24 100644 --- a/openpype/modules/deadline/plugins/publish/submit_maya_remote_publish_deadline.py +++ b/openpype/modules/deadline/plugins/publish/submit_maya_remote_publish_deadline.py @@ -10,7 +10,7 @@ import openpype.api import pyblish.api -class MayaSubmitRemotePublishDeadline(openpype.api.Integrator): +class MayaSubmitRemotePublishDeadline(pyblish.api.InstancePlugin): """Submit Maya scene to perform a local publish in Deadline. Publishing in Deadline can be helpful for scenes that publish very slow. @@ -31,6 +31,7 @@ class MayaSubmitRemotePublishDeadline(openpype.api.Integrator): order = pyblish.api.IntegratorOrder hosts = ["maya"] families = ["publish.farm"] + targets = ["local"] def process(self, instance): settings = get_project_settings(os.getenv("AVALON_PROJECT")) diff --git a/openpype/modules/deadline/plugins/publish/submit_nuke_deadline.py b/openpype/modules/deadline/plugins/publish/submit_nuke_deadline.py index 94c703d66d..93fb511a34 100644 --- a/openpype/modules/deadline/plugins/publish/submit_nuke_deadline.py +++ b/openpype/modules/deadline/plugins/publish/submit_nuke_deadline.py @@ -23,6 +23,7 @@ class NukeSubmitDeadline(pyblish.api.InstancePlugin): hosts = ["nuke", "nukestudio"] families = ["render.farm", "prerender.farm"] optional = True + targets = ["local"] # presets priority = 50 @@ -54,8 +55,8 @@ class NukeSubmitDeadline(pyblish.api.InstancePlugin): self._ver = re.search(r"\d+\.\d+", context.data.get("hostVersion")) self._deadline_user = context.data.get( "deadlineUser", getpass.getuser()) - self._frame_start = int(instance.data["frameStartHandle"]) - self._frame_end = int(instance.data["frameEndHandle"]) + submit_frame_start = int(instance.data["frameStartHandle"]) + submit_frame_end = int(instance.data["frameEndHandle"]) # get output path render_path = instance.data['path'] @@ -81,13 +82,16 @@ class NukeSubmitDeadline(pyblish.api.InstancePlugin): # exception for slate workflow if "slate" in instance.data["families"]: - self._frame_start -= 1 + submit_frame_start -= 1 - response = self.payload_submit(instance, - script_path, - render_path, - node.name() - ) + response = self.payload_submit( + instance, + script_path, + render_path, + node.name(), + submit_frame_start, + submit_frame_end + ) # Store output dir for unified publisher (filesequence) instance.data["deadlineSubmissionJob"] = response.json() instance.data["outputDir"] = os.path.dirname( @@ -95,20 +99,22 @@ class NukeSubmitDeadline(pyblish.api.InstancePlugin): instance.data["publishJobState"] = "Suspended" if instance.data.get("bakingNukeScripts"): + # exception for slate workflow + if "slate" in instance.data["families"]: + submit_frame_start += 1 + for baking_script in instance.data["bakingNukeScripts"]: render_path = baking_script["bakeRenderPath"] script_path = baking_script["bakeScriptPath"] exe_node_name = baking_script["bakeWriteNodeName"] - # exception for slate workflow - if "slate" in instance.data["families"]: - self._frame_start += 1 - resp = self.payload_submit( instance, script_path, render_path, exe_node_name, + submit_frame_start, + submit_frame_end, response.json() ) @@ -125,13 +131,16 @@ class NukeSubmitDeadline(pyblish.api.InstancePlugin): families.insert(0, "prerender") instance.data["families"] = families - def payload_submit(self, - instance, - script_path, - render_path, - exe_node_name, - responce_data=None - ): + def payload_submit( + self, + instance, + script_path, + render_path, + exe_node_name, + start_frame, + end_frame, + responce_data=None + ): render_dir = os.path.normpath(os.path.dirname(render_path)) script_name = os.path.basename(script_path) jobname = "%s - %s" % (script_name, instance.name) @@ -191,8 +200,8 @@ class NukeSubmitDeadline(pyblish.api.InstancePlugin): "Plugin": "Nuke", "Frames": "{start}-{end}".format( - start=self._frame_start, - end=self._frame_end + start=start_frame, + end=end_frame ), "Comment": self._comment, @@ -292,7 +301,13 @@ class NukeSubmitDeadline(pyblish.api.InstancePlugin): self.log.info(json.dumps(payload, indent=4, sort_keys=True)) # adding expectied files to instance.data - self.expected_files(instance, render_path) + self.expected_files( + instance, + render_path, + start_frame, + end_frame + ) + self.log.debug("__ expectedFiles: `{}`".format( instance.data["expectedFiles"])) response = requests.post(self.deadline_url, json=payload, timeout=10) @@ -338,9 +353,13 @@ class NukeSubmitDeadline(pyblish.api.InstancePlugin): self.log.debug("_ path: `{}`".format(path)) return path - def expected_files(self, - instance, - path): + def expected_files( + self, + instance, + path, + start_frame, + end_frame + ): """ Create expected files in instance data """ if not instance.data.get("expectedFiles"): @@ -358,7 +377,7 @@ class NukeSubmitDeadline(pyblish.api.InstancePlugin): instance.data["expectedFiles"].append(path) return - for i in range(self._frame_start, (self._frame_end + 1)): + for i in range(start_frame, (end_frame + 1)): instance.data["expectedFiles"].append( os.path.join(dir, (file % i)).replace("\\", "/")) diff --git a/openpype/modules/deadline/plugins/publish/submit_publish_job.py b/openpype/modules/deadline/plugins/publish/submit_publish_job.py index 78ab935e42..b54b00d099 100644 --- a/openpype/modules/deadline/plugins/publish/submit_publish_job.py +++ b/openpype/modules/deadline/plugins/publish/submit_publish_job.py @@ -103,6 +103,7 @@ class ProcessSubmittedJobOnFarm(pyblish.api.InstancePlugin): order = pyblish.api.IntegratorOrder + 0.2 icon = "tractor" deadline_plugin = "OpenPype" + targets = ["local"] hosts = ["fusion", "maya", "nuke", "celaction", "aftereffects", "harmony"] @@ -128,7 +129,8 @@ class ProcessSubmittedJobOnFarm(pyblish.api.InstancePlugin): "OPENPYPE_LOG_NO_COLORS", "OPENPYPE_USERNAME", "OPENPYPE_RENDER_JOB", - "OPENPYPE_PUBLISH_JOB" + "OPENPYPE_PUBLISH_JOB", + "OPENPYPE_MONGO" ] # custom deadline attributes @@ -145,7 +147,7 @@ class ProcessSubmittedJobOnFarm(pyblish.api.InstancePlugin): # mapping of instance properties to be transfered to new instance for every # specified family instance_transfer = { - "slate": ["slateFrame"], + "slate": ["slateFrames"], "review": ["lutPath"], "render2d": ["bakingNukeScripts", "version"], "renderlayer": ["convertToScanline"] @@ -640,6 +642,7 @@ class ProcessSubmittedJobOnFarm(pyblish.api.InstancePlugin): def _solve_families(self, instance, preview=False): families = instance.get("families") + # if we have one representation with preview tag # flag whole instance for review and for ftrack if preview: @@ -719,10 +722,17 @@ class ProcessSubmittedJobOnFarm(pyblish.api.InstancePlugin): " This may cause issues." ).format(source)) - families = ["render"] + family = "render" + if "prerender" in instance.data["families"]: + family = "prerender" + families = [family] + + # pass review to families if marked as review + if data.get("review"): + families.append("review") instance_skeleton_data = { - "family": "render", + "family": family, "subset": subset, "families": families, "asset": asset, @@ -744,11 +754,6 @@ class ProcessSubmittedJobOnFarm(pyblish.api.InstancePlugin): "useSequenceForReview": data.get("useSequenceForReview", True) } - if "prerender" in instance.data["families"]: - instance_skeleton_data.update({ - "family": "prerender", - "families": []}) - # skip locking version if we are creating v01 instance_version = instance.data.get("version") # take this if exists if instance_version != 1: diff --git a/openpype/modules/ftrack/event_handlers_server/action_prepare_project.py b/openpype/modules/ftrack/event_handlers_server/action_prepare_project.py index 975e49cb28..361aa98d16 100644 --- a/openpype/modules/ftrack/event_handlers_server/action_prepare_project.py +++ b/openpype/modules/ftrack/event_handlers_server/action_prepare_project.py @@ -1,8 +1,8 @@ import json +from openpype.client import get_project from openpype.api import ProjectSettings from openpype.lib import create_project -from openpype.pipeline import AvalonMongoDB from openpype.settings import SaveWarningExc from openpype_modules.ftrack.lib import ( @@ -363,12 +363,8 @@ class PrepareProjectServer(ServerAction): project_name = project_entity["full_name"] # Try to find project document - dbcon = AvalonMongoDB() - dbcon.install() - dbcon.Session["AVALON_PROJECT"] = project_name - project_doc = dbcon.find_one({ - "type": "project" - }) + project_doc = get_project(project_name) + # Create project if is not available # - creation is required to be able set project anatomy and attributes if not project_doc: @@ -376,9 +372,7 @@ class PrepareProjectServer(ServerAction): self.log.info("Creating project \"{} [{}]\"".format( project_name, project_code )) - create_project(project_name, project_code, dbcon=dbcon) - - dbcon.uninstall() + create_project(project_name, project_code) project_settings = ProjectSettings(project_name) project_anatomy_settings = project_settings["project_anatomy"] diff --git a/openpype/modules/ftrack/event_handlers_server/event_sync_to_avalon.py b/openpype/modules/ftrack/event_handlers_server/event_sync_to_avalon.py index b5f199b3e4..a4e791aaf0 100644 --- a/openpype/modules/ftrack/event_handlers_server/event_sync_to_avalon.py +++ b/openpype/modules/ftrack/event_handlers_server/event_sync_to_avalon.py @@ -12,6 +12,12 @@ from pymongo import UpdateOne import arrow import ftrack_api +from openpype.client import ( + get_project, + get_assets, + get_archived_assets, + get_asset_ids_with_subsets +) from openpype.pipeline import AvalonMongoDB, schema from openpype_modules.ftrack.lib import ( @@ -149,12 +155,11 @@ class SyncToAvalonEvent(BaseEvent): @property def avalon_entities(self): if self._avalon_ents is None: + project_name = self.cur_project["full_name"] self.dbcon.install() - self.dbcon.Session["AVALON_PROJECT"] = ( - self.cur_project["full_name"] - ) - avalon_project = self.dbcon.find_one({"type": "project"}) - avalon_entities = list(self.dbcon.find({"type": "asset"})) + self.dbcon.Session["AVALON_PROJECT"] = project_name + avalon_project = get_project(project_name) + avalon_entities = list(get_assets(project_name)) self._avalon_ents = (avalon_project, avalon_entities) return self._avalon_ents @@ -284,28 +289,21 @@ class SyncToAvalonEvent(BaseEvent): self._avalon_ents_by_ftrack_id[ftrack_id] = doc @property - def avalon_subsets_by_parents(self): - if self._avalon_subsets_by_parents is None: - self._avalon_subsets_by_parents = collections.defaultdict(list) - self.dbcon.install() - self.dbcon.Session["AVALON_PROJECT"] = ( - self.cur_project["full_name"] + def avalon_asset_ids_with_subsets(self): + if self._avalon_asset_ids_with_subsets is None: + project_name = self.cur_project["full_name"] + self._avalon_asset_ids_with_subsets = get_asset_ids_with_subsets( + project_name ) - for subset in self.dbcon.find({"type": "subset"}): - self._avalon_subsets_by_parents[subset["parent"]].append( - subset - ) - return self._avalon_subsets_by_parents + + return self._avalon_asset_ids_with_subsets @property def avalon_archived_by_id(self): if self._avalon_archived_by_id is None: self._avalon_archived_by_id = {} - self.dbcon.install() - self.dbcon.Session["AVALON_PROJECT"] = ( - self.cur_project["full_name"] - ) - for asset in self.dbcon.find({"type": "archived_asset"}): + project_name = self.cur_project["full_name"] + for asset in get_archived_assets(project_name): self._avalon_archived_by_id[asset["_id"]] = asset return self._avalon_archived_by_id @@ -327,7 +325,7 @@ class SyncToAvalonEvent(BaseEvent): avalon_project, avalon_entities = self.avalon_entities self._changeability_by_mongo_id[avalon_project["_id"]] = False self._bubble_changeability( - list(self.avalon_subsets_by_parents.keys()) + list(self.avalon_asset_ids_with_subsets) ) return self._changeability_by_mongo_id @@ -449,14 +447,9 @@ class SyncToAvalonEvent(BaseEvent): if not entity: # if entity is not found then it is subset without parent if entity_id in unchangeable_ids: - _subset_ids = [ - str(sub["_id"]) for sub in - self.avalon_subsets_by_parents[entity_id] - ] - joined_subset_ids = "| ".join(_subset_ids) self.log.warning(( - "Parent <{}> for subsets <{}> does not exist" - ).format(str(entity_id), joined_subset_ids)) + "Parent <{}> with subsets does not exist" + ).format(str(entity_id))) else: self.log.warning(( "In avalon are entities without valid parents that" @@ -483,7 +476,7 @@ class SyncToAvalonEvent(BaseEvent): self._avalon_ents_by_parent_id = None self._avalon_ents_by_ftrack_id = None self._avalon_ents_by_name = None - self._avalon_subsets_by_parents = None + self._avalon_asset_ids_with_subsets = None self._changeability_by_mongo_id = None self._avalon_archived_by_id = None self._avalon_archived_by_name = None diff --git a/openpype/modules/ftrack/event_handlers_server/event_user_assigment.py b/openpype/modules/ftrack/event_handlers_server/event_user_assigment.py index 593fc5e596..82b79e986b 100644 --- a/openpype/modules/ftrack/event_handlers_server/event_user_assigment.py +++ b/openpype/modules/ftrack/event_handlers_server/event_user_assigment.py @@ -1,11 +1,9 @@ import re import subprocess +from openpype.client import get_asset_by_id, get_asset_by_name from openpype_modules.ftrack.lib import BaseEvent from openpype_modules.ftrack.lib.avalon_sync import CUST_ATTR_ID_KEY -from openpype.pipeline import AvalonMongoDB - -from bson.objectid import ObjectId from openpype.api import Anatomy, get_project_settings @@ -36,8 +34,6 @@ class UserAssigmentEvent(BaseEvent): 3) path to publish files of task user was (de)assigned to """ - db_con = AvalonMongoDB() - def error(self, *err): for e in err: self.log.error(e) @@ -101,26 +97,16 @@ class UserAssigmentEvent(BaseEvent): :rtype: dict """ parent = task['parent'] - self.db_con.install() - self.db_con.Session['AVALON_PROJECT'] = task['project']['full_name'] - + project_name = task["project"]["full_name"] avalon_entity = None parent_id = parent['custom_attributes'].get(CUST_ATTR_ID_KEY) if parent_id: - parent_id = ObjectId(parent_id) - avalon_entity = self.db_con.find_one({ - '_id': parent_id, - 'type': 'asset' - }) + avalon_entity = get_asset_by_id(project_name, parent_id) if not avalon_entity: - avalon_entity = self.db_con.find_one({ - 'type': 'asset', - 'name': parent['name'] - }) + avalon_entity = get_asset_by_name(project_name, parent["name"]) if not avalon_entity: - self.db_con.uninstall() msg = 'Entity "{}" not found in avalon database'.format( parent['name'] ) @@ -129,7 +115,6 @@ class UserAssigmentEvent(BaseEvent): 'success': False, 'message': msg } - self.db_con.uninstall() return avalon_entity def _get_hierarchy(self, asset): diff --git a/openpype/modules/ftrack/event_handlers_user/action_applications.py b/openpype/modules/ftrack/event_handlers_user/action_applications.py index b25bc1b5cb..102f04c956 100644 --- a/openpype/modules/ftrack/event_handlers_user/action_applications.py +++ b/openpype/modules/ftrack/event_handlers_user/action_applications.py @@ -1,5 +1,6 @@ import os +from openpype.client import get_project from openpype_modules.ftrack.lib import BaseAction from openpype.lib.applications import ( ApplicationManager, @@ -7,7 +8,6 @@ from openpype.lib.applications import ( ApplictionExecutableNotFound, CUSTOM_LAUNCH_APP_GROUPS ) -from openpype.pipeline import AvalonMongoDB class AppplicationsAction(BaseAction): @@ -25,7 +25,6 @@ class AppplicationsAction(BaseAction): super(AppplicationsAction, self).__init__(*args, **kwargs) self.application_manager = ApplicationManager() - self.dbcon = AvalonMongoDB() @property def discover_identifier(self): @@ -110,12 +109,7 @@ class AppplicationsAction(BaseAction): if avalon_project_doc is None: ft_project = self.get_project_from_entity(entity) project_name = ft_project["full_name"] - if not self.dbcon.is_installed(): - self.dbcon.install() - self.dbcon.Session["AVALON_PROJECT"] = project_name - avalon_project_doc = self.dbcon.find_one({ - "type": "project" - }) or False + avalon_project_doc = get_project(project_name) or False event["data"]["avalon_project_doc"] = avalon_project_doc if not avalon_project_doc: diff --git a/openpype/modules/ftrack/event_handlers_user/action_create_cust_attrs.py b/openpype/modules/ftrack/event_handlers_user/action_create_cust_attrs.py index 88dc8213bd..d04440a564 100644 --- a/openpype/modules/ftrack/event_handlers_user/action_create_cust_attrs.py +++ b/openpype/modules/ftrack/event_handlers_user/action_create_cust_attrs.py @@ -140,9 +140,9 @@ class CustomAttributes(BaseAction): identifier = 'create.update.attributes' #: Action label. label = "OpenPype Admin" - variant = '- Create/Update Avalon Attributes' + variant = '- Create/Update Custom Attributes' #: Action description. - description = 'Creates Avalon/Mongo ID for double check' + description = 'Creates required custom attributes in ftrack' icon = statics_icon("ftrack", "action_icons", "OpenPypeAdmin.svg") settings_key = "create_update_attributes" diff --git a/openpype/modules/ftrack/event_handlers_user/action_delete_asset.py b/openpype/modules/ftrack/event_handlers_user/action_delete_asset.py index ee5c3d0d97..03d029b0c1 100644 --- a/openpype/modules/ftrack/event_handlers_user/action_delete_asset.py +++ b/openpype/modules/ftrack/event_handlers_user/action_delete_asset.py @@ -4,6 +4,7 @@ from datetime import datetime from bson.objectid import ObjectId +from openpype.client import get_assets, get_subsets from openpype.pipeline import AvalonMongoDB from openpype_modules.ftrack.lib import BaseAction, statics_icon from openpype_modules.ftrack.lib.avalon_sync import create_chunks @@ -91,10 +92,8 @@ class DeleteAssetSubset(BaseAction): continue ftrack_id = entity.get("entityId") - if not ftrack_id: - continue - - ftrack_ids.append(ftrack_id) + if ftrack_id: + ftrack_ids.append(ftrack_id) if project_in_selection: msg = "It is not possible to use this action on project entity." @@ -120,48 +119,51 @@ class DeleteAssetSubset(BaseAction): "message": "Invalid selection for this action (Bug)" } - if entities[0].entity_type.lower() == "project": - project = entities[0] - else: - project = entities[0]["project"] - + project = self.get_project_from_entity(entities[0], session) project_name = project["full_name"] self.dbcon.Session["AVALON_PROJECT"] = project_name - selected_av_entities = list(self.dbcon.find({ - "type": "asset", - "data.ftrackId": {"$in": ftrack_ids} - })) + asset_docs = list(get_assets( + project_name, + fields=["_id", "name", "data.ftrackId", "data.parents"] + )) + selected_av_entities = [] + found_ftrack_ids = set() + asset_docs_by_name = collections.defaultdict(list) + for asset_doc in asset_docs: + ftrack_id = asset_doc["data"].get("ftrackId") + if ftrack_id: + found_ftrack_ids.add(ftrack_id) + if ftrack_id in entity_mapping: + selected_av_entities.append(asset_doc) + + asset_name = asset_doc["name"] + asset_docs_by_name[asset_name].append(asset_doc) + found_without_ftrack_id = {} - if len(selected_av_entities) != len(ftrack_ids): - found_ftrack_ids = [ - ent["data"]["ftrackId"] for ent in selected_av_entities - ] - for ftrack_id, entity in entity_mapping.items(): - if ftrack_id in found_ftrack_ids: + for ftrack_id, entity in entity_mapping.items(): + if ftrack_id in found_ftrack_ids: + continue + + av_ents_by_name = asset_docs_by_name[entity["name"]] + if not av_ents_by_name: + continue + + ent_path_items = [ent["name"] for ent in entity["link"]] + end_index = len(ent_path_items) - 1 + parents = ent_path_items[1:end_index:] + # TODO we should say to user that + # few of them are missing in avalon + for av_ent in av_ents_by_name: + if av_ent["data"]["parents"] != parents: continue - av_ents_by_name = list(self.dbcon.find({ - "type": "asset", - "name": entity["name"] - })) - if not av_ents_by_name: - continue - - ent_path_items = [ent["name"] for ent in entity["link"]] - parents = ent_path_items[1:len(ent_path_items)-1:] - # TODO we should say to user that - # few of them are missing in avalon - for av_ent in av_ents_by_name: - if av_ent["data"]["parents"] != parents: - continue - - # TODO we should say to user that found entity - # with same name does not match same ftrack id? - if "ftrackId" not in av_ent["data"]: - selected_av_entities.append(av_ent) - found_without_ftrack_id[str(av_ent["_id"])] = ftrack_id - break + # TODO we should say to user that found entity + # with same name does not match same ftrack id? + if "ftrackId" not in av_ent["data"]: + selected_av_entities.append(av_ent) + found_without_ftrack_id[str(av_ent["_id"])] = ftrack_id + break if not selected_av_entities: return { @@ -206,10 +208,7 @@ class DeleteAssetSubset(BaseAction): items.append(id_item) asset_ids = [ent["_id"] for ent in selected_av_entities] - subsets_for_selection = self.dbcon.find({ - "type": "subset", - "parent": {"$in": asset_ids} - }) + subsets_for_selection = get_subsets(project_name, asset_ids=asset_ids) asset_ending = "" if len(selected_av_entities) > 1: @@ -459,13 +458,9 @@ class DeleteAssetSubset(BaseAction): if len(assets_to_delete) > 0: map_av_ftrack_id = spec_data["without_ftrack_id"] # Prepare data when deleting whole avalon asset - avalon_assets = self.dbcon.find( - {"type": "asset"}, - { - "_id": 1, - "data.visualParent": 1, - "data.ftrackId": 1 - } + avalon_assets = get_assets( + project_name, + fields=["_id", "data.visualParent", "data.ftrackId"] ) avalon_assets_by_parent = collections.defaultdict(list) for asset in avalon_assets: diff --git a/openpype/modules/ftrack/event_handlers_user/action_delete_old_versions.py b/openpype/modules/ftrack/event_handlers_user/action_delete_old_versions.py index a0bf6622e9..3400c509ab 100644 --- a/openpype/modules/ftrack/event_handlers_user/action_delete_old_versions.py +++ b/openpype/modules/ftrack/event_handlers_user/action_delete_old_versions.py @@ -5,7 +5,12 @@ import uuid import clique from pymongo import UpdateOne - +from openpype.client import ( + get_assets, + get_subsets, + get_versions, + get_representations +) from openpype.api import Anatomy from openpype.lib import StringTemplate, TemplateUnsolved from openpype.pipeline import AvalonMongoDB @@ -198,10 +203,9 @@ class DeleteOldVersions(BaseAction): self.log.debug("Project is set to {}".format(project_name)) # Get Assets from avalon database - assets = list(self.dbcon.find({ - "type": "asset", - "name": {"$in": avalon_asset_names} - })) + assets = list( + get_assets(project_name, asset_names=avalon_asset_names) + ) asset_id_to_name_map = { asset["_id"]: asset["name"] for asset in assets } @@ -210,10 +214,9 @@ class DeleteOldVersions(BaseAction): self.log.debug("Collected assets ({})".format(len(asset_ids))) # Get Subsets - subsets = list(self.dbcon.find({ - "type": "subset", - "parent": {"$in": asset_ids} - })) + subsets = list( + get_subsets(project_name, asset_ids=asset_ids) + ) subsets_by_id = {} subset_ids = [] for subset in subsets: @@ -230,10 +233,9 @@ class DeleteOldVersions(BaseAction): self.log.debug("Collected subsets ({})".format(len(subset_ids))) # Get Versions - versions = list(self.dbcon.find({ - "type": "version", - "parent": {"$in": subset_ids} - })) + versions = list( + get_versions(project_name, subset_ids=subset_ids) + ) versions_by_parent = collections.defaultdict(list) for ent in versions: @@ -295,10 +297,9 @@ class DeleteOldVersions(BaseAction): "message": msg } - repres = list(self.dbcon.find({ - "type": "representation", - "parent": {"$in": version_ids} - })) + repres = list( + get_representations(project_name, version_ids=version_ids) + ) self.log.debug( "Collected representations to remove ({})".format(len(repres)) diff --git a/openpype/modules/ftrack/event_handlers_user/action_delivery.py b/openpype/modules/ftrack/event_handlers_user/action_delivery.py index 86d88ef7cc..4b799b092b 100644 --- a/openpype/modules/ftrack/event_handlers_user/action_delivery.py +++ b/openpype/modules/ftrack/event_handlers_user/action_delivery.py @@ -3,8 +3,13 @@ import copy import json import collections -from bson.objectid import ObjectId - +from openpype.client import ( + get_project, + get_assets, + get_subsets, + get_versions, + get_representations +) from openpype.api import Anatomy, config from openpype_modules.ftrack.lib import BaseAction, statics_icon from openpype_modules.ftrack.lib.avalon_sync import CUST_ATTR_ID_KEY @@ -18,11 +23,9 @@ from openpype.lib.delivery import ( process_single_file, process_sequence ) -from openpype.pipeline import AvalonMongoDB class Delivery(BaseAction): - identifier = "delivery.action" label = "Delivery" description = "Deliver data to client" @@ -30,11 +33,6 @@ class Delivery(BaseAction): icon = statics_icon("ftrack", "action_icons", "Delivery.svg") settings_key = "delivery_action" - def __init__(self, *args, **kwargs): - self.dbcon = AvalonMongoDB() - - super(Delivery, self).__init__(*args, **kwargs) - def discover(self, session, entities, event): is_valid = False for entity in entities: @@ -57,9 +55,7 @@ class Delivery(BaseAction): project_entity = self.get_project_from_entity(entities[0]) project_name = project_entity["full_name"] - self.dbcon.install() - self.dbcon.Session["AVALON_PROJECT"] = project_name - project_doc = self.dbcon.find_one({"type": "project"}, {"name": True}) + project_doc = get_project(project_name, fields=["name"]) if not project_doc: return { "success": False, @@ -68,8 +64,7 @@ class Delivery(BaseAction): ).format(project_name) } - repre_names = self._get_repre_names(session, entities) - self.dbcon.uninstall() + repre_names = self._get_repre_names(project_name, session, entities) items.append({ "type": "hidden", @@ -198,17 +193,21 @@ class Delivery(BaseAction): "title": title } - def _get_repre_names(self, session, entities): - version_ids = self._get_interest_version_ids(session, entities) + def _get_repre_names(self, project_name, session, entities): + version_ids = self._get_interest_version_ids( + project_name, session, entities + ) if not version_ids: return [] - repre_docs = self.dbcon.find({ - "type": "representation", - "parent": {"$in": version_ids} - }) - return list(sorted(repre_docs.distinct("name"))) + repre_docs = get_representations( + project_name, + version_ids=version_ids, + fields=["name"] + ) + repre_names = {repre_doc["name"] for repre_doc in repre_docs} + return list(sorted(repre_names)) - def _get_interest_version_ids(self, session, entities): + def _get_interest_version_ids(self, project_name, session, entities): # Extract AssetVersion entities asset_versions = self._extract_asset_versions(session, entities) # Prepare Asset ids @@ -235,14 +234,18 @@ class Delivery(BaseAction): subset_names.add(asset["name"]) version_nums.add(asset_version["version"]) - asset_docs_by_ftrack_id = self._get_asset_docs(session, parent_ids) + asset_docs_by_ftrack_id = self._get_asset_docs( + project_name, session, parent_ids + ) subset_docs = self._get_subset_docs( + project_name, asset_docs_by_ftrack_id, subset_names, asset_versions, assets_by_id ) version_docs = self._get_version_docs( + project_name, asset_docs_by_ftrack_id, subset_docs, version_nums, @@ -290,6 +293,7 @@ class Delivery(BaseAction): def _get_version_docs( self, + project_name, asset_docs_by_ftrack_id, subset_docs, version_nums, @@ -300,11 +304,11 @@ class Delivery(BaseAction): subset_doc["_id"]: subset_doc for subset_doc in subset_docs } - version_docs = list(self.dbcon.find({ - "type": "version", - "parent": {"$in": list(subset_docs_by_id.keys())}, - "name": {"$in": list(version_nums)} - })) + version_docs = list(get_versions( + project_name, + subset_ids=subset_docs_by_id.keys(), + versions=version_nums + )) version_docs_by_parent_id = collections.defaultdict(dict) for version_doc in version_docs: subset_doc = subset_docs_by_id[version_doc["parent"]] @@ -345,6 +349,7 @@ class Delivery(BaseAction): def _get_subset_docs( self, + project_name, asset_docs_by_ftrack_id, subset_names, asset_versions, @@ -354,11 +359,11 @@ class Delivery(BaseAction): asset_doc["_id"] for asset_doc in asset_docs_by_ftrack_id.values() ] - subset_docs = list(self.dbcon.find({ - "type": "subset", - "parent": {"$in": asset_doc_ids}, - "name": {"$in": list(subset_names)} - })) + subset_docs = list(get_subsets( + project_name, + asset_ids=asset_doc_ids, + subset_names=subset_names + )) subset_docs_by_parent_id = collections.defaultdict(dict) for subset_doc in subset_docs: asset_id = subset_doc["parent"] @@ -385,15 +390,21 @@ class Delivery(BaseAction): filtered_subsets.append(subset_doc) return filtered_subsets - def _get_asset_docs(self, session, parent_ids): - asset_docs = list(self.dbcon.find({ - "type": "asset", - "data.ftrackId": {"$in": list(parent_ids)} - })) + def _get_asset_docs(self, project_name, session, parent_ids): + asset_docs = list(get_assets( + project_name, fields=["_id", "name", "data.ftrackId"] + )) + asset_docs_by_id = {} + asset_docs_by_name = {} asset_docs_by_ftrack_id = {} for asset_doc in asset_docs: + asset_id = str(asset_doc["_id"]) + asset_name = asset_doc["name"] ftrack_id = asset_doc["data"].get("ftrackId") + + asset_docs_by_id[asset_id] = asset_doc + asset_docs_by_name[asset_name] = asset_doc if ftrack_id: asset_docs_by_ftrack_id[ftrack_id] = asset_doc @@ -406,15 +417,15 @@ class Delivery(BaseAction): avalon_mongo_id_values = query_custom_attributes( session, [attr_def["id"]], parent_ids, True ) - entity_ids_by_mongo_id = { - ObjectId(item["value"]): item["entity_id"] - for item in avalon_mongo_id_values - if item["value"] - } - missing_ids = set(parent_ids) - for entity_id in set(entity_ids_by_mongo_id.values()): - if entity_id in missing_ids: + for item in avalon_mongo_id_values: + if not item["value"]: + continue + asset_id = item["value"] + entity_id = item["entity_id"] + asset_doc = asset_docs_by_id.get(asset_id) + if asset_doc: + asset_docs_by_ftrack_id[entity_id] = asset_doc missing_ids.remove(entity_id) entity_ids_by_name = {} @@ -427,36 +438,10 @@ class Delivery(BaseAction): for entity in not_found_entities } - expressions = [] - if entity_ids_by_mongo_id: - expression = { - "type": "asset", - "_id": {"$in": list(entity_ids_by_mongo_id.keys())} - } - expressions.append(expression) - - if entity_ids_by_name: - expression = { - "type": "asset", - "name": {"$in": list(entity_ids_by_name.keys())} - } - expressions.append(expression) - - if expressions: - if len(expressions) == 1: - filter = expressions[0] - else: - filter = {"$or": expressions} - - asset_docs = self.dbcon.find(filter) - for asset_doc in asset_docs: - if asset_doc["_id"] in entity_ids_by_mongo_id: - entity_id = entity_ids_by_mongo_id[asset_doc["_id"]] - asset_docs_by_ftrack_id[entity_id] = asset_doc - - elif asset_doc["name"] in entity_ids_by_name: - entity_id = entity_ids_by_name[asset_doc["name"]] - asset_docs_by_ftrack_id[entity_id] = asset_doc + for asset_name, entity_id in entity_ids_by_name.items(): + asset_doc = asset_docs_by_name.get(asset_name) + if asset_doc: + asset_docs_by_ftrack_id[entity_id] = asset_doc return asset_docs_by_ftrack_id @@ -490,7 +475,6 @@ class Delivery(BaseAction): session.commit() try: - self.dbcon.install() report = self.real_launch(session, entities, event) except Exception as exc: @@ -516,7 +500,6 @@ class Delivery(BaseAction): else: job["status"] = "failed" session.commit() - self.dbcon.uninstall() if not report["success"]: self.show_interface( @@ -558,16 +541,15 @@ class Delivery(BaseAction): if not os.path.exists(location_path): os.makedirs(location_path) - self.dbcon.Session["AVALON_PROJECT"] = project_name - self.log.debug("Collecting representations to process.") - version_ids = self._get_interest_version_ids(session, entities) - repres_to_deliver = list(self.dbcon.find({ - "type": "representation", - "parent": {"$in": version_ids}, - "name": {"$in": repre_names} - })) - + version_ids = self._get_interest_version_ids( + project_name, session, entities + ) + repres_to_deliver = list(get_representations( + project_name, + representation_names=repre_names, + version_ids=version_ids + )) anatomy = Anatomy(project_name) format_dict = get_format_dict(anatomy, location_path) diff --git a/openpype/modules/ftrack/event_handlers_user/action_fill_workfile_attr.py b/openpype/modules/ftrack/event_handlers_user/action_fill_workfile_attr.py index c7237a1150..d30c41a749 100644 --- a/openpype/modules/ftrack/event_handlers_user/action_fill_workfile_attr.py +++ b/openpype/modules/ftrack/event_handlers_user/action_fill_workfile_attr.py @@ -7,6 +7,10 @@ import datetime import ftrack_api +from openpype.client import ( + get_project, + get_assets, +) from openpype.api import get_project_settings from openpype.lib import ( get_workfile_template_key, @@ -14,7 +18,6 @@ from openpype.lib import ( Anatomy, StringTemplate, ) -from openpype.pipeline import AvalonMongoDB from openpype_modules.ftrack.lib import BaseAction, statics_icon from openpype_modules.ftrack.lib.avalon_sync import create_chunks @@ -248,10 +251,8 @@ class FillWorkfileAttributeAction(BaseAction): # Find matchin asset documents and map them by ftrack task entities # - result stored to 'asset_docs_with_task_entities' is list with # tuple `(asset document, [task entitis, ...])` - dbcon = AvalonMongoDB() - dbcon.Session["AVALON_PROJECT"] = project_name # Quety all asset documents - asset_docs = list(dbcon.find({"type": "asset"})) + asset_docs = list(get_assets(project_name)) job_entity["data"] = json.dumps({ "description": "(1/3) Asset documents queried." }) @@ -276,7 +277,7 @@ class FillWorkfileAttributeAction(BaseAction): # Keep placeholders in the template unfilled host_name = "{app}" extension = "{ext}" - project_doc = dbcon.find_one({"type": "project"}) + project_doc = get_project(project_name) project_settings = get_project_settings(project_name) anatomy = Anatomy(project_name) templates_by_key = {} diff --git a/openpype/modules/ftrack/event_handlers_user/action_prepare_project.py b/openpype/modules/ftrack/event_handlers_user/action_prepare_project.py index 0b14e7aa2b..e9dc11de9f 100644 --- a/openpype/modules/ftrack/event_handlers_user/action_prepare_project.py +++ b/openpype/modules/ftrack/event_handlers_user/action_prepare_project.py @@ -1,8 +1,8 @@ import json +from openpype.client import get_project from openpype.api import ProjectSettings from openpype.lib import create_project -from openpype.pipeline import AvalonMongoDB from openpype.settings import SaveWarningExc from openpype_modules.ftrack.lib import ( @@ -389,12 +389,8 @@ class PrepareProjectLocal(BaseAction): project_name = project_entity["full_name"] # Try to find project document - dbcon = AvalonMongoDB() - dbcon.install() - dbcon.Session["AVALON_PROJECT"] = project_name - project_doc = dbcon.find_one({ - "type": "project" - }) + project_doc = get_project(project_name) + # Create project if is not available # - creation is required to be able set project anatomy and attributes if not project_doc: @@ -402,9 +398,7 @@ class PrepareProjectLocal(BaseAction): self.log.info("Creating project \"{} [{}]\"".format( project_name, project_code )) - create_project(project_name, project_code, dbcon=dbcon) - - dbcon.uninstall() + create_project(project_name, project_code) project_settings = ProjectSettings(project_name) project_anatomy_settings = project_settings["project_anatomy"] diff --git a/openpype/modules/ftrack/event_handlers_user/action_rv.py b/openpype/modules/ftrack/event_handlers_user/action_rv.py index 040ca75582..2480ea7f95 100644 --- a/openpype/modules/ftrack/event_handlers_user/action_rv.py +++ b/openpype/modules/ftrack/event_handlers_user/action_rv.py @@ -5,9 +5,16 @@ import json import ftrack_api +from openpype.client import ( + get_asset_by_name, + get_subset_by_name, + get_version_by_name, + get_representation_by_name +) +from openpype.api import Anatomy from openpype.pipeline import ( get_representation_path, - legacy_io, + AvalonMongoDB, ) from openpype_modules.ftrack.lib import BaseAction, statics_icon @@ -255,9 +262,10 @@ class RVAction(BaseAction): "Component", list(event["data"]["values"].values())[0] )["version"]["asset"]["parent"]["link"][0] project = session.get(link["type"], link["id"]) - os.environ["AVALON_PROJECT"] = project["name"] - legacy_io.Session["AVALON_PROJECT"] = project["name"] - legacy_io.install() + project_name = project["full_name"] + dbcon = AvalonMongoDB() + dbcon.Session["AVALON_PROJECT"] = project_name + anatomy = Anatomy(project_name) location = ftrack_api.Session().pick_location() @@ -281,37 +289,38 @@ class RVAction(BaseAction): if online_source: continue - asset = legacy_io.find_one({"type": "asset", "name": parent_name}) - subset = legacy_io.find_one( - { - "type": "subset", - "name": component["version"]["asset"]["name"], - "parent": asset["_id"] - } + subset_name = component["version"]["asset"]["name"] + version_name = component["version"]["version"] + representation_name = component["file_type"][1:] + + asset_doc = get_asset_by_name( + project_name, parent_name, fields=["_id"] ) - version = legacy_io.find_one( - { - "type": "version", - "name": component["version"]["version"], - "parent": subset["_id"] - } + subset_doc = get_subset_by_name( + project_name, + subset_name=subset_name, + asset_id=asset_doc["_id"] ) - representation = legacy_io.find_one( - { - "type": "representation", - "parent": version["_id"], - "name": component["file_type"][1:] - } + version_doc = get_version_by_name( + project_name, + version=version_name, + subset_id=subset_doc["_id"] ) - if representation is None: - representation = legacy_io.find_one( - { - "type": "representation", - "parent": version["_id"], - "name": "preview" - } + repre_doc = get_representation_by_name( + project_name, + version_id=version_doc["_id"], + representation_name=representation_name + ) + if not repre_doc: + repre_doc = get_representation_by_name( + project_name, + version_id=version_doc["_id"], + representation_name="preview" ) - paths.append(get_representation_path(representation)) + + paths.append(get_representation_path( + repre_doc, root=anatomy.roots, dbcon=dbcon + )) return paths diff --git a/openpype/modules/ftrack/event_handlers_user/action_store_thumbnails_to_avalon.py b/openpype/modules/ftrack/event_handlers_user/action_store_thumbnails_to_avalon.py index 62fdfa2bdd..d655dddcaf 100644 --- a/openpype/modules/ftrack/event_handlers_user/action_store_thumbnails_to_avalon.py +++ b/openpype/modules/ftrack/event_handlers_user/action_store_thumbnails_to_avalon.py @@ -5,6 +5,14 @@ import requests from bson.objectid import ObjectId +from openpype.client import ( + get_project, + get_asset_by_id, + get_assets, + get_subset_by_name, + get_version_by_name, + get_representations +) from openpype_modules.ftrack.lib import BaseAction, statics_icon from openpype.api import Anatomy from openpype.pipeline import AvalonMongoDB @@ -385,7 +393,7 @@ class StoreThumbnailsToAvalon(BaseAction): db_con.Session["AVALON_PROJECT"] = project_name - avalon_project = db_con.find_one({"type": "project"}) + avalon_project = get_project(project_name) output["project"] = avalon_project if not avalon_project: @@ -399,19 +407,17 @@ class StoreThumbnailsToAvalon(BaseAction): asset_mongo_id = parent["custom_attributes"].get(CUST_ATTR_ID_KEY) if asset_mongo_id: try: - asset_mongo_id = ObjectId(asset_mongo_id) - asset_ent = db_con.find_one({ - "type": "asset", - "_id": asset_mongo_id - }) + asset_ent = get_asset_by_id(project_name, asset_mongo_id) except Exception: pass if not asset_ent: - asset_ent = db_con.find_one({ - "type": "asset", - "data.ftrackId": parent["id"] - }) + asset_docs = get_assets(project_name, asset_names=[parent["name"]]) + for asset_doc in asset_docs: + ftrack_id = asset_doc.get("data", {}).get("ftrackId") + if ftrack_id == parent["id"]: + asset_ent = asset_doc + break output["asset"] = asset_ent @@ -422,13 +428,11 @@ class StoreThumbnailsToAvalon(BaseAction): ) return output - asset_mongo_id = asset_ent["_id"] - - subset_ent = db_con.find_one({ - "type": "subset", - "parent": asset_mongo_id, - "name": subset_name - }) + subset_ent = get_subset_by_name( + project_name, + subset_name=subset_name, + asset_id=asset_ent["_id"] + ) output["subset"] = subset_ent @@ -439,11 +443,11 @@ class StoreThumbnailsToAvalon(BaseAction): ).format(subset_name, ent_path) return output - version_ent = db_con.find_one({ - "type": "version", - "name": version, - "parent": subset_ent["_id"] - }) + version_ent = get_version_by_name( + project_name, + version, + subset_ent["_id"] + ) output["version"] = version_ent @@ -454,10 +458,10 @@ class StoreThumbnailsToAvalon(BaseAction): ).format(version, subset_name, ent_path) return output - repre_ents = list(db_con.find({ - "type": "representation", - "parent": version_ent["_id"] - })) + repre_ents = list(get_representations( + project_name, + version_ids=[version_ent["_id"]] + )) output["representations"] = repre_ents return output diff --git a/openpype/modules/ftrack/lib/avalon_sync.py b/openpype/modules/ftrack/lib/avalon_sync.py index e4ba651bfd..68b5c62c53 100644 --- a/openpype/modules/ftrack/lib/avalon_sync.py +++ b/openpype/modules/ftrack/lib/avalon_sync.py @@ -6,6 +6,14 @@ import numbers import six +from openpype.client import ( + get_project, + get_assets, + get_archived_assets, + get_subsets, + get_versions, + get_representations +) from openpype.api import ( Logger, get_anatomy_settings @@ -576,6 +584,10 @@ class SyncEntitiesFactory: self.ft_project_id = ft_project_id self.entities_dict = entities_dict + @property + def project_name(self): + return self.entities_dict[self.ft_project_id]["name"] + @property def avalon_ents_by_id(self): """ @@ -660,9 +672,9 @@ class SyncEntitiesFactory: (list) of assets """ if self._avalon_archived_ents is None: - self._avalon_archived_ents = [ - ent for ent in self.dbcon.find({"type": "archived_asset"}) - ] + self._avalon_archived_ents = list( + get_archived_assets(self.project_name) + ) return self._avalon_archived_ents @property @@ -730,7 +742,7 @@ class SyncEntitiesFactory: """ if self._subsets_by_parent_id is None: self._subsets_by_parent_id = collections.defaultdict(list) - for subset in self.dbcon.find({"type": "subset"}): + for subset in get_subsets(self.project_name): self._subsets_by_parent_id[str(subset["parent"])].append( subset ) @@ -1421,8 +1433,8 @@ class SyncEntitiesFactory: # Avalon entities self.dbcon.install() self.dbcon.Session["AVALON_PROJECT"] = ft_project_name - avalon_project = self.dbcon.find_one({"type": "project"}) - avalon_entities = self.dbcon.find({"type": "asset"}) + avalon_project = get_project(ft_project_name) + avalon_entities = get_assets(ft_project_name) self.avalon_project = avalon_project self.avalon_entities = avalon_entities @@ -2258,46 +2270,37 @@ class SyncEntitiesFactory: self._delete_subsets_without_asset(subsets_to_remove) def _delete_subsets_without_asset(self, not_existing_parents): - subset_ids = [] - version_ids = [] repre_ids = [] to_delete = [] + subset_ids = [] for parent_id in not_existing_parents: subsets = self.subsets_by_parent_id.get(parent_id) if not subsets: continue for subset in subsets: - if subset.get("type") != "subset": - continue - subset_ids.append(subset["_id"]) + if subset.get("type") == "subset": + subset_ids.append(subset["_id"]) - db_subsets = self.dbcon.find({ - "_id": {"$in": subset_ids}, - "type": "subset" - }) - if not db_subsets: - return - - db_versions = self.dbcon.find({ - "parent": {"$in": subset_ids}, - "type": "version" - }) - if db_versions: - version_ids = [ver["_id"] for ver in db_versions] - - db_repres = self.dbcon.find({ - "parent": {"$in": version_ids}, - "type": "representation" - }) - if db_repres: - repre_ids = [repre["_id"] for repre in db_repres] + db_versions = get_versions( + self.project_name, + subset_ids=subset_ids, + fields=["_id"] + ) + version_ids = [ver["_id"] for ver in db_versions] + db_repres = get_representations( + self.project_name, + version_ids=version_ids, + fields=["_id"] + ) + repre_ids = [repre["_id"] for repre in db_repres] to_delete.extend(subset_ids) to_delete.extend(version_ids) to_delete.extend(repre_ids) - self.dbcon.delete_many({"_id": {"$in": to_delete}}) + if to_delete: + self.dbcon.delete_many({"_id": {"$in": to_delete}}) # Probably deprecated def _check_changeability(self, parent_id=None): @@ -2779,8 +2782,7 @@ class SyncEntitiesFactory: def report(self): items = [] - project_name = self.entities_dict[self.ft_project_id]["name"] - title = "Synchronization report ({}):".format(project_name) + title = "Synchronization report ({}):".format(self.project_name) keys = ["error", "warning", "info"] for key in keys: diff --git a/openpype/modules/ftrack/plugins/publish/integrate_hierarchy_ftrack.py b/openpype/modules/ftrack/plugins/publish/integrate_hierarchy_ftrack.py index 73398941eb..1a5d74bf26 100644 --- a/openpype/modules/ftrack/plugins/publish/integrate_hierarchy_ftrack.py +++ b/openpype/modules/ftrack/plugins/publish/integrate_hierarchy_ftrack.py @@ -3,7 +3,8 @@ import collections import six import pyblish.api from copy import deepcopy -from openpype.pipeline import legacy_io +from openpype.client import get_asset_by_id + # Copy of constant `openpype_modules.ftrack.lib.avalon_sync.CUST_ATTR_AUTO_SYNC` CUST_ATTR_AUTO_SYNC = "avalon_auto_sync" @@ -82,9 +83,6 @@ class IntegrateHierarchyToFtrack(pyblish.api.ContextPlugin): auto_sync_state = project[ "custom_attributes"][CUST_ATTR_AUTO_SYNC] - if not legacy_io.Session: - legacy_io.install() - self.ft_project = None # disable termporarily ftrack project's autosyncing @@ -93,14 +91,14 @@ class IntegrateHierarchyToFtrack(pyblish.api.ContextPlugin): try: # import ftrack hierarchy - self.import_to_ftrack(hierarchy_context) + self.import_to_ftrack(project_name, hierarchy_context) except Exception: raise finally: if auto_sync_state: self.auto_sync_on(project) - def import_to_ftrack(self, input_data, parent=None): + def import_to_ftrack(self, project_name, input_data, parent=None): # Prequery hiearchical custom attributes hier_custom_attributes = get_pype_attr(self.session)[1] hier_attr_by_key = { @@ -222,7 +220,7 @@ class IntegrateHierarchyToFtrack(pyblish.api.ContextPlugin): six.reraise(tp, value, tb) # Incoming links. - self.create_links(entity_data, entity) + self.create_links(project_name, entity_data, entity) try: self.session.commit() except Exception: @@ -255,9 +253,9 @@ class IntegrateHierarchyToFtrack(pyblish.api.ContextPlugin): # Import children. if 'childs' in entity_data: self.import_to_ftrack( - entity_data['childs'], entity) + project_name, entity_data['childs'], entity) - def create_links(self, entity_data, entity): + def create_links(self, project_name, entity_data, entity): # Clear existing links. for link in entity.get("incoming_links", []): self.session.delete(link) @@ -270,9 +268,15 @@ class IntegrateHierarchyToFtrack(pyblish.api.ContextPlugin): six.reraise(tp, value, tb) # Create new links. - for input in entity_data.get("inputs", []): - input_id = legacy_io.find_one({"_id": input})["data"]["ftrackId"] - assetbuild = self.session.get("AssetBuild", input_id) + for asset_id in entity_data.get("inputs", []): + asset_doc = get_asset_by_id(project_name, asset_id) + ftrack_id = None + if asset_doc: + ftrack_id = asset_doc["data"].get("ftrackId") + if not ftrack_id: + continue + + assetbuild = self.session.get("AssetBuild", ftrack_id) self.log.debug( "Creating link from {0} to {1}".format( assetbuild["name"], entity["name"] diff --git a/openpype/modules/ftrack/tray/ftrack_tray.py b/openpype/modules/ftrack/tray/ftrack_tray.py index c6201a94f6..2919ae22fb 100644 --- a/openpype/modules/ftrack/tray/ftrack_tray.py +++ b/openpype/modules/ftrack/tray/ftrack_tray.py @@ -2,6 +2,7 @@ import os import time import datetime import threading + from Qt import QtCore, QtWidgets, QtGui import ftrack_api @@ -48,6 +49,9 @@ class FtrackTrayWrapper: self.widget_login.activateWindow() self.widget_login.raise_() + def show_ftrack_browser(self): + QtGui.QDesktopServices.openUrl(self.module.ftrack_url) + def validate(self): validation = False cred = credentials.get_credentials() @@ -284,6 +288,13 @@ class FtrackTrayWrapper: tray_server_menu.addAction(self.action_server_stop) self.tray_server_menu = tray_server_menu + + # Ftrack Browser + browser_open = QtWidgets.QAction("Open Ftrack...", tray_menu) + browser_open.triggered.connect(self.show_ftrack_browser) + tray_menu.addAction(browser_open) + self.browser_open = browser_open + self.bool_logged = False self.set_menu_visibility() diff --git a/openpype/modules/kitsu/utils/update_op_with_zou.py b/openpype/modules/kitsu/utils/update_op_with_zou.py index 673a195747..08e50d959b 100644 --- a/openpype/modules/kitsu/utils/update_op_with_zou.py +++ b/openpype/modules/kitsu/utils/update_op_with_zou.py @@ -85,7 +85,7 @@ def update_op_assets( # Frame in, fallback on 0 frame_in = int(item_data.get("frame_in") or 0) item_data["frameStart"] = frame_in - item_data.pop("frame_in") + item_data.pop("frame_in", None) # Frame out, fallback on frame_in + duration frames_duration = int(item.get("nb_frames") or 1) frame_out = ( @@ -94,7 +94,7 @@ def update_op_assets( else frame_in + frames_duration ) item_data["frameEnd"] = int(frame_out) - item_data.pop("frame_out") + item_data.pop("frame_out", None) # Fps, fallback to project's value when entity fps is deleted if not item_data.get("fps") and item_doc["data"].get("fps"): item_data["fps"] = project_doc["data"]["fps"] diff --git a/openpype/modules/sync_server/resources/disabled.png b/openpype/modules/sync_server/resources/disabled.png new file mode 100644 index 0000000000..e036d7ef6a Binary files /dev/null and b/openpype/modules/sync_server/resources/disabled.png differ diff --git a/openpype/modules/sync_server/sync_server.py b/openpype/modules/sync_server/sync_server.py index 22eed01ef3..356a75f99d 100644 --- a/openpype/modules/sync_server/sync_server.py +++ b/openpype/modules/sync_server/sync_server.py @@ -280,14 +280,13 @@ class SyncServerThread(threading.Thread): while self.is_running and not self.module.is_paused(): try: import time - start_time = None + start_time = time.time() self.module.set_sync_project_settings() # clean cache - for collection, preset in self.module.sync_project_settings.\ - items(): - if collection not in self.module.get_enabled_projects(): - continue + collection = None + enabled_projects = self.module.get_enabled_projects() + for collection in enabled_projects: + preset = self.module.sync_project_settings[collection] - start_time = time.time() local_site, remote_site = self._working_sites(collection) if not all([local_site, remote_site]): continue diff --git a/openpype/modules/sync_server/sync_server_module.py b/openpype/modules/sync_server/sync_server_module.py index 5a1d8467ec..698b296a52 100644 --- a/openpype/modules/sync_server/sync_server_module.py +++ b/openpype/modules/sync_server/sync_server_module.py @@ -926,9 +926,22 @@ class SyncServerModule(OpenPypeModule, ITrayModule): return enabled_projects - def is_project_enabled(self, project_name): + def is_project_enabled(self, project_name, single=False): + """Checks if 'project_name' is enabled for syncing. + 'get_sync_project_setting' is potentially expensive operation (pulls + settings for all projects if cached version is not available), using + project_settings for specific project should be faster. + Args: + project_name (str) + single (bool): use 'get_project_settings' method + """ if self.enabled: - project_settings = self.get_sync_project_setting(project_name) + if single: + project_settings = get_project_settings(project_name) + project_settings = \ + self._parse_sync_settings_from_settings(project_settings) + else: + project_settings = self.get_sync_project_setting(project_name) if project_settings and project_settings.get("enabled"): return True return False @@ -1026,21 +1039,13 @@ class SyncServerModule(OpenPypeModule, ITrayModule): """ self.server_init() - from .tray.app import SyncServerWindow - self.widget = SyncServerWindow(self) - def server_init(self): """Actual initialization of Sync Server.""" # import only in tray or Python3, because of Python2 hosts - from .sync_server import SyncServerThread - if not self.enabled: return - enabled_projects = self.get_enabled_projects() - if not enabled_projects: - self.enabled = False - return + from .sync_server import SyncServerThread self.lock = threading.Lock() @@ -1060,7 +1065,7 @@ class SyncServerModule(OpenPypeModule, ITrayModule): self.server_start() def server_start(self): - if self.sync_project_settings and self.enabled: + if self.enabled: self.sync_server_thread.start() else: log.info("No presets or active providers. " + @@ -1851,6 +1856,9 @@ class SyncServerModule(OpenPypeModule, ITrayModule): Returns: (int): in seconds """ + if not project_name: + return 60 + ld = self.sync_project_settings[project_name]["config"]["loop_delay"] return int(ld) diff --git a/openpype/modules/sync_server/tray/app.py b/openpype/modules/sync_server/tray/app.py index fc8558bdbc..96fad6a247 100644 --- a/openpype/modules/sync_server/tray/app.py +++ b/openpype/modules/sync_server/tray/app.py @@ -46,6 +46,14 @@ class SyncServerWindow(QtWidgets.QDialog): left_column_layout.addWidget(self.pause_btn) + checkbox = QtWidgets.QCheckBox("Show only enabled", self) + checkbox.setStyleSheet("QCheckBox{spacing: 5px;" + "padding:5px 5px 5px 5px;}") + checkbox.setChecked(True) + self.show_only_enabled_chk = checkbox + + left_column_layout.addWidget(self.show_only_enabled_chk) + repres = SyncRepresentationSummaryWidget( sync_server, project=self.projects.current_project, @@ -86,15 +94,27 @@ class SyncServerWindow(QtWidgets.QDialog): repres.message_generated.connect(self._update_message) self.projects.message_generated.connect(self._update_message) + self.show_only_enabled_chk.stateChanged.connect( + self._on_enabled_change + ) + self.representationWidget = repres + def showEvent(self, event): + self.representationWidget.set_project(self.projects.current_project) + self.projects.refresh() + self._set_running(True) + super().showEvent(event) + + def closeEvent(self, event): + self._set_running(False) + super().closeEvent(event) + def _on_project_change(self): if self.projects.current_project is None: return - self.representationWidget.table_view.model().set_project( - self.projects.current_project - ) + self.representationWidget.set_project(self.projects.current_project) project_name = self.projects.current_project if not self.sync_server.get_sync_project_setting(project_name): @@ -103,16 +123,12 @@ class SyncServerWindow(QtWidgets.QDialog): self.projects.refresh() return - def showEvent(self, event): - self.representationWidget.model.set_project( - self.projects.current_project) + def _on_enabled_change(self): + """Called when enabled projects only checkbox is toggled.""" + self.projects.show_only_enabled = \ + self.show_only_enabled_chk.isChecked() self.projects.refresh() - self._set_running(True) - super().showEvent(event) - - def closeEvent(self, event): - self._set_running(False) - super().closeEvent(event) + self.representationWidget.set_project(None) def _set_running(self, running): self.representationWidget.model.is_running = running diff --git a/openpype/modules/sync_server/tray/models.py b/openpype/modules/sync_server/tray/models.py index 7241cc3472..c49edeafb9 100644 --- a/openpype/modules/sync_server/tray/models.py +++ b/openpype/modules/sync_server/tray/models.py @@ -52,7 +52,8 @@ class _SyncRepresentationModel(QtCore.QAbstractTableModel): All queries should go through this (because of collection). """ - return self.sync_server.connection.database[self.project] + if self.project: + return self.sync_server.connection.database[self.project] @property def project(self): @@ -150,6 +151,9 @@ class _SyncRepresentationModel(QtCore.QAbstractTableModel): @property def can_edit(self): """Returns true if some site is user local site, eg. could edit""" + if not self.project: + return False + return get_local_site_id() in (self.active_site, self.remote_site) def get_column(self, index): @@ -190,7 +194,7 @@ class _SyncRepresentationModel(QtCore.QAbstractTableModel): actually queried (scrolled a couple of times to list more than single page of records) """ - if self.is_editing or not self.is_running: + if self.is_editing or not self.is_running or not self.project: return self.refresh_started.emit() self.beginResetModel() @@ -232,6 +236,9 @@ class _SyncRepresentationModel(QtCore.QAbstractTableModel): more records in DB than loaded. """ log.debug("fetchMore") + if not self.dbcon: + return + items_to_fetch = min(self._total_records - self._rec_loaded, self.PAGE_SIZE) self.query = self.get_query(self._rec_loaded) @@ -286,9 +293,10 @@ class _SyncRepresentationModel(QtCore.QAbstractTableModel): # replace('False', 'false').\ # replace('True', 'true').replace('None', 'null')) - representations = self.dbcon.aggregate(pipeline=self.query, - allowDiskUse=True) - self.refresh(representations) + if self.dbcon: + representations = self.dbcon.aggregate(pipeline=self.query, + allowDiskUse=True) + self.refresh(representations) def set_word_filter(self, word_filter): """ @@ -378,9 +386,9 @@ class _SyncRepresentationModel(QtCore.QAbstractTableModel): project (str): name of project """ self._project = project - self.sync_server.set_sync_project_settings() # project might have been deactivated in the meantime if not self.sync_server.get_sync_project_setting(project): + self._data = {} return self.active_site = self.sync_server.get_active_site(self.project) @@ -509,25 +517,23 @@ class SyncRepresentationSummaryModel(_SyncRepresentationModel): self._word_filter = None - if not self._project or self._project == lib.DUMMY_PROJECT: - return - self.sync_server = sync_server # TODO think about admin mode + self.sort_criteria = self.DEFAULT_SORT + + self.timer = QtCore.QTimer() + if not self._project or self._project == lib.DUMMY_PROJECT: + self.active_site = sync_server.DEFAULT_SITE + self.remote_site = sync_server.DEFAULT_SITE + return + # this is for regular user, always only single local and single remote self.active_site = self.sync_server.get_active_site(self.project) self.remote_site = self.sync_server.get_remote_site(self.project) - self.sort_criteria = self.DEFAULT_SORT - self.query = self.get_query() self.default_query = list(self.get_query()) - representations = self.dbcon.aggregate(pipeline=self.query, - allowDiskUse=True) - self.refresh(representations) - - self.timer = QtCore.QTimer() self.timer.timeout.connect(self.tick) self.timer.start(self.REFRESH_SEC) @@ -1003,9 +1009,6 @@ class SyncRepresentationDetailModel(_SyncRepresentationModel): self.sort_criteria = self.DEFAULT_SORT self.query = self.get_query() - representations = self.dbcon.aggregate(pipeline=self.query, - allowDiskUse=True) - self.refresh(representations) self.timer = QtCore.QTimer() self.timer.timeout.connect(self.tick) diff --git a/openpype/modules/sync_server/tray/widgets.py b/openpype/modules/sync_server/tray/widgets.py index 6aae9562cf..b4ee447ac4 100644 --- a/openpype/modules/sync_server/tray/widgets.py +++ b/openpype/modules/sync_server/tray/widgets.py @@ -47,6 +47,7 @@ class SyncProjectListWidget(QtWidgets.QWidget): message_generated = QtCore.Signal(str) refresh_msec = 10000 + show_only_enabled = True def __init__(self, sync_server, parent): super(SyncProjectListWidget, self).__init__(parent) @@ -122,11 +123,15 @@ class SyncProjectListWidget(QtWidgets.QWidget): self._model_reset = False selected_item = None - for project_name in self.sync_server.sync_project_settings.\ - keys(): + sync_settings = self.sync_server.sync_project_settings + for project_name in sync_settings.keys(): if self.sync_server.is_paused() or \ self.sync_server.is_project_paused(project_name): icon = self._get_icon("paused") + elif not sync_settings[project_name]["enabled"]: + if self.show_only_enabled: + continue + icon = self._get_icon("disabled") else: icon = self._get_icon("synced") @@ -139,12 +144,12 @@ class SyncProjectListWidget(QtWidgets.QWidget): if self.current_project == project_name: selected_item = item + if model.item(0) is None: + return + if selected_item: selected_index = model.indexFromItem(selected_item) - if len(self.sync_server.sync_project_settings.keys()) == 0: - model.appendRow(QtGui.QStandardItem(lib.DUMMY_PROJECT)) - if not self.current_project: self.current_project = model.item(0).data(QtCore.Qt.DisplayRole) @@ -248,6 +253,9 @@ class _SyncRepresentationWidget(QtWidgets.QWidget): active_changed = QtCore.Signal() # active index changed message_generated = QtCore.Signal(str) + def set_project(self, project): + self.model.set_project(project) + def _selection_changed(self, _new_selected, _all_selected): idxs = self.selection_model.selectedRows() self._selected_ids = set() @@ -581,7 +589,6 @@ class SyncRepresentationSummaryWidget(_SyncRepresentationWidget): super(SyncRepresentationSummaryWidget, self).__init__(parent) self.sync_server = sync_server - self._selected_ids = set() # keep last selected _id txt_filter = QtWidgets.QLineEdit() @@ -625,7 +632,6 @@ class SyncRepresentationSummaryWidget(_SyncRepresentationWidget): column = table_view.model().get_header_index("priority") priority_delegate = delegates.PriorityDelegate(self) table_view.setItemDelegateForColumn(column, priority_delegate) - layout = QtWidgets.QVBoxLayout(self) layout.setContentsMargins(0, 0, 0, 0) layout.addLayout(top_bar_layout) @@ -633,21 +639,16 @@ class SyncRepresentationSummaryWidget(_SyncRepresentationWidget): self.table_view = table_view self.model = model - horizontal_header = HorizontalHeader(self) - table_view.setHorizontalHeader(horizontal_header) table_view.setSortingEnabled(True) - for column_name, width in self.default_widths: idx = model.get_header_index(column_name) table_view.setColumnWidth(idx, width) - table_view.doubleClicked.connect(self._double_clicked) self.txt_filter.textChanged.connect(lambda: model.set_word_filter( self.txt_filter.text())) table_view.customContextMenuRequested.connect(self._on_context_menu) - model.refresh_started.connect(self._save_scrollbar) model.refresh_finished.connect(self._set_scrollbar) model.modelReset.connect(self._set_selection) @@ -963,7 +964,6 @@ class HorizontalHeader(QtWidgets.QHeaderView): super(HorizontalHeader, self).__init__(QtCore.Qt.Horizontal, parent) self._parent = parent self.checked_values = {} - self.setModel(self._parent.model) self.setSectionsClickable(True) diff --git a/openpype/modules/timers_manager/timers_manager.py b/openpype/modules/timers_manager/timers_manager.py index 3f77a2b7dc..3cf1614316 100644 --- a/openpype/modules/timers_manager/timers_manager.py +++ b/openpype/modules/timers_manager/timers_manager.py @@ -7,6 +7,7 @@ from openpype_interfaces import ( ITrayService, ILaunchHookPaths ) +from openpype.lib.events import register_event_callback from openpype.pipeline import AvalonMongoDB from .exceptions import InvalidContextError @@ -422,3 +423,20 @@ class TimersManager(OpenPypeModule, ITrayService, ILaunchHookPaths): } return requests.post(rest_api_url, json=data) + + def on_host_install(self, host, host_name, project_name): + self.log.debug("Installing task changed callback") + register_event_callback("taskChanged", self._on_host_task_change) + + def _on_host_task_change(self, event): + project_name = event["project_name"] + asset_name = event["asset_name"] + task_name = event["task_name"] + self.log.debug(( + "Sending message that timer should change to" + " Project: {} Asset: {} Task: {}" + ).format(project_name, asset_name, task_name)) + + self.start_timer_with_webserver( + project_name, asset_name, task_name, self.log + ) diff --git a/openpype/pipeline/context_tools.py b/openpype/pipeline/context_tools.py index c6e09cfba1..4a147c230b 100644 --- a/openpype/pipeline/context_tools.py +++ b/openpype/pipeline/context_tools.py @@ -16,9 +16,7 @@ from openpype.modules import load_modules, ModulesManager from openpype.settings import get_project_settings from openpype.lib import ( Anatomy, - register_event_callback, filter_pyblish_plugins, - change_timer_to_current_context, ) from . import ( @@ -33,6 +31,9 @@ from . import ( _is_installed = False _registered_root = {"_": ""} _registered_host = {"_": None} +# Keep modules manager (and it's modules) in memory +# - that gives option to register modules' callbacks +_modules_manager = None log = logging.getLogger(__name__) @@ -44,6 +45,23 @@ PUBLISH_PATH = os.path.join(PLUGINS_DIR, "publish") LOAD_PATH = os.path.join(PLUGINS_DIR, "load") +def _get_modules_manager(): + """Get or create modules manager for host installation. + + This is not meant for public usage. Reason is to keep modules + in memory of process to be able trigger their event callbacks if they + need any. + + Returns: + ModulesManager: Manager wrapping discovered modules. + """ + + global _modules_manager + if _modules_manager is None: + _modules_manager = ModulesManager() + return _modules_manager + + def register_root(path): """Register currently active root""" log.info("Registering root: %s" % path) @@ -74,6 +92,7 @@ def install_host(host): _is_installed = True legacy_io.install() + modules_manager = _get_modules_manager() missing = list() for key in ("AVALON_PROJECT", "AVALON_ASSET"): @@ -95,8 +114,6 @@ def install_host(host): register_host(host) - register_event_callback("taskChanged", _on_task_change) - def modified_emit(obj, record): """Method replacing `emit` in Pyblish's MessageHandler.""" record.msg = record.getMessage() @@ -112,7 +129,14 @@ def install_host(host): else: pyblish.api.register_target("local") - install_openpype_plugins() + project_name = os.environ.get("AVALON_PROJECT") + host_name = os.environ.get("AVALON_APP") + + # Give option to handle host installation + for module in modules_manager.get_enabled_modules(): + module.on_host_install(host, host_name, project_name) + + install_openpype_plugins(project_name, host_name) def install_openpype_plugins(project_name=None, host_name=None): @@ -124,7 +148,7 @@ def install_openpype_plugins(project_name=None, host_name=None): pyblish.api.register_discovery_filter(filter_pyblish_plugins) register_loader_plugin_path(LOAD_PATH) - modules_manager = ModulesManager() + modules_manager = _get_modules_manager() publish_plugin_dirs = modules_manager.collect_plugin_paths()["publish"] for path in publish_plugin_dirs: pyblish.api.register_plugin_path(path) @@ -168,10 +192,6 @@ def install_openpype_plugins(project_name=None, host_name=None): register_inventory_action(path) -def _on_task_change(): - change_timer_to_current_context() - - def uninstall_host(): """Undo all of what `install()` did""" host = registered_host() diff --git a/openpype/pipeline/create/context.py b/openpype/pipeline/create/context.py index 4bfa68c9d4..12cd9bbc68 100644 --- a/openpype/pipeline/create/context.py +++ b/openpype/pipeline/create/context.py @@ -824,9 +824,10 @@ class CreateContext: discover_result = publish_plugins_discover() publish_plugins = discover_result.plugins - targets = pyblish.logic.registered_targets() or ["default"] + targets = set(pyblish.logic.registered_targets()) + targets.add("default") plugins_by_targets = pyblish.logic.plugins_by_targets( - publish_plugins, targets + publish_plugins, list(targets) ) # Collect plugins that can have attribute definitions for plugin in publish_plugins: diff --git a/openpype/pipeline/legacy_io.py b/openpype/pipeline/legacy_io.py index d586756671..bde2b24c2a 100644 --- a/openpype/pipeline/legacy_io.py +++ b/openpype/pipeline/legacy_io.py @@ -148,3 +148,12 @@ def parenthood(*args, **kwargs): @requires_install def bulk_write(*args, **kwargs): return _connection_object.bulk_write(*args, **kwargs) + + +@requires_install +def active_project(*args, **kwargs): + return _connection_object.active_project(*args, **kwargs) + + +def current_project(*args, **kwargs): + return Session.get("AVALON_PROJECT") diff --git a/openpype/pipeline/mongodb.py b/openpype/pipeline/mongodb.py index 565e26b966..dab5bb9e13 100644 --- a/openpype/pipeline/mongodb.py +++ b/openpype/pipeline/mongodb.py @@ -199,6 +199,10 @@ class AvalonMongoDB: """Return the name of the active project""" return self.Session["AVALON_PROJECT"] + def current_project(self): + """Currently set project in Session without triggering installation.""" + return self.Session.get("AVALON_PROJECT") + @requires_install @auto_reconnect def projects(self, projection=None, only_active=True): diff --git a/openpype/plugin.py b/openpype/plugin.py index 6637ad1d8b..bb9bc2ff85 100644 --- a/openpype/plugin.py +++ b/openpype/plugin.py @@ -18,16 +18,6 @@ class InstancePlugin(pyblish.api.InstancePlugin): super(InstancePlugin, cls).process(cls, *args, **kwargs) -class Integrator(InstancePlugin): - """Integrator base class. - - Wraps pyblish instance plugin. Targets set to "local" which means all - integrators should run on "local" publishes, by default. - "remote" targets could be used for integrators that should run externally. - """ - targets = ["local"] - - class Extractor(InstancePlugin): """Extractor base class. @@ -38,8 +28,6 @@ class Extractor(InstancePlugin): """ - targets = ["local"] - order = 2.0 def staging_dir(self, instance): diff --git a/openpype/plugins/publish/collect_current_pype_user.py b/openpype/plugins/publish/collect_current_pype_user.py index 1a52a59012..2d507ba292 100644 --- a/openpype/plugins/publish/collect_current_pype_user.py +++ b/openpype/plugins/publish/collect_current_pype_user.py @@ -1,5 +1,3 @@ -import os -import getpass import pyblish.api from openpype.lib import get_openpype_username diff --git a/openpype/plugins/publish/extract_jpeg_exr.py b/openpype/plugins/publish/extract_jpeg_exr.py index ae29f8b95b..a467728e77 100644 --- a/openpype/plugins/publish/extract_jpeg_exr.py +++ b/openpype/plugins/publish/extract_jpeg_exr.py @@ -3,22 +3,20 @@ import os import pyblish.api from openpype.lib import ( get_ffmpeg_tool_path, + get_oiio_tools_path, + is_oiio_supported, run_subprocess, path_to_subprocess_arg, - get_transcode_temp_directory, - convert_input_paths_for_ffmpeg, - should_convert_for_ffmpeg + execute, ) -import shutil - -class ExtractJpegEXR(pyblish.api.InstancePlugin): +class ExtractThumbnail(pyblish.api.InstancePlugin): """Create jpg thumbnail from sequence using ffmpeg""" - label = "Extract Jpeg EXR" + label = "Extract Thumbnail" order = pyblish.api.ExtractorOrder families = [ "imagesequence", "render", "render2d", @@ -49,7 +47,6 @@ class ExtractJpegEXR(pyblish.api.InstancePlugin): return filtered_repres = self._get_filtered_repres(instance) - for repre in filtered_repres: repre_files = repre["files"] if not isinstance(repre_files, (list, tuple)): @@ -62,78 +59,37 @@ class ExtractJpegEXR(pyblish.api.InstancePlugin): full_input_path = os.path.join(stagingdir, input_file) self.log.info("input {}".format(full_input_path)) - - do_convert = should_convert_for_ffmpeg(full_input_path) - # If result is None the requirement of conversion can't be - # determined - if do_convert is None: - self.log.info(( - "Can't determine if representation requires conversion." - " Skipped." - )) - continue - - # Do conversion if needed - # - change staging dir of source representation - # - must be set back after output definitions processing - convert_dir = None - if do_convert: - convert_dir = get_transcode_temp_directory() - filename = os.path.basename(full_input_path) - convert_input_paths_for_ffmpeg( - [full_input_path], - convert_dir, - self.log - ) - full_input_path = os.path.join(convert_dir, filename) - filename = os.path.splitext(input_file)[0] if not filename.endswith('.'): filename += "." jpeg_file = filename + "jpg" full_output_path = os.path.join(stagingdir, jpeg_file) - self.log.info("output {}".format(full_output_path)) + thumbnail_created = False + # Try to use FFMPEG if OIIO is not supported (for cases when + # oiiotool isn't available) + if not is_oiio_supported(): + thumbnail_created = self.create_thumbnail_ffmpeg(full_input_path, full_output_path) # noqa + else: + # Check if the file can be read by OIIO + oiio_tool_path = get_oiio_tools_path() + args = [ + oiio_tool_path, "--info", "-i", full_output_path + ] + returncode = execute(args, silent=True) + # If the input can read by OIIO then use OIIO method for + # conversion otherwise use ffmpeg + if returncode == 0: + self.log.info("Input can be read by OIIO, converting with oiiotool now.") # noqa + thumbnail_created = self.create_thumbnail_oiio(full_input_path, full_output_path) # noqa + else: + self.log.info("Converting with FFMPEG because input can't be read by OIIO.") # noqa + thumbnail_created = self.create_thumbnail_ffmpeg(full_input_path, full_output_path) # noqa - ffmpeg_path = get_ffmpeg_tool_path("ffmpeg") - ffmpeg_args = self.ffmpeg_args or {} - - jpeg_items = [] - jpeg_items.append(path_to_subprocess_arg(ffmpeg_path)) - # override file if already exists - jpeg_items.append("-y") - # use same input args like with mov - jpeg_items.extend(ffmpeg_args.get("input") or []) - # input file - jpeg_items.append("-i {}".format( - path_to_subprocess_arg(full_input_path) - )) - # output arguments from presets - jpeg_items.extend(ffmpeg_args.get("output") or []) - - # If its a movie file, we just want one frame. - if repre["ext"] == "mov": - jpeg_items.append("-vframes 1") - - # output file - jpeg_items.append(path_to_subprocess_arg(full_output_path)) - - subprocess_command = " ".join(jpeg_items) - - # run subprocess - self.log.debug("{}".format(subprocess_command)) - try: # temporary until oiiotool is supported cross platform - run_subprocess( - subprocess_command, shell=True, logger=self.log - ) - except RuntimeError as exp: - if "Compression" in str(exp): - self.log.debug( - "Unsupported compression on input files. Skipping!!!" - ) - return - self.log.warning("Conversion crashed", exc_info=True) - raise + # Skip the rest of the process if the thumbnail wasn't created + if not thumbnail_created: + self.log.warning("Thumbanil has not been created.") + return new_repre = { "name": "thumbnail", @@ -145,16 +101,11 @@ class ExtractJpegEXR(pyblish.api.InstancePlugin): } # adding representation - self.log.debug("Adding: {}".format(new_repre)) + self.log.debug( + "Adding thumbnail representation: {}".format(new_repre) + ) instance.data["representations"].append(new_repre) - - # Cleanup temp folder - if convert_dir is not None and os.path.exists(convert_dir): - shutil.rmtree(convert_dir) - - # Create only one representation with name 'thumbnail' - # TODO maybe handle way how to decide from which representation - # will be thumbnail created + # There is no need to create more then one thumbnail break def _get_filtered_repres(self, instance): @@ -175,3 +126,61 @@ class ExtractJpegEXR(pyblish.api.InstancePlugin): filtered_repres.append(repre) return filtered_repres + + def create_thumbnail_oiio(self, src_path, dst_path): + self.log.info("outputting {}".format(dst_path)) + oiio_tool_path = get_oiio_tools_path() + oiio_cmd = [oiio_tool_path, "-a", + src_path, "-o", + dst_path + ] + subprocess_exr = " ".join(oiio_cmd) + self.log.info(f"running: {subprocess_exr}") + try: + run_subprocess(oiio_cmd, logger=self.log) + return True + except Exception: + self.log.warning( + "Failed to create thubmnail using oiiotool", + exc_info=True + ) + return False + + def create_thumbnail_ffmpeg(self, src_path, dst_path): + self.log.info("outputting {}".format(dst_path)) + + ffmpeg_path = get_ffmpeg_tool_path("ffmpeg") + ffmpeg_args = self.ffmpeg_args or {} + + jpeg_items = [] + jpeg_items.append(path_to_subprocess_arg(ffmpeg_path)) + # override file if already exists + jpeg_items.append("-y") + # flag for large file sizes + max_int = 2147483647 + jpeg_items.append("-analyzeduration {}".format(max_int)) + jpeg_items.append("-probesize {}".format(max_int)) + # use same input args like with mov + jpeg_items.extend(ffmpeg_args.get("input") or []) + # input file + jpeg_items.append("-i {}".format( + path_to_subprocess_arg(src_path) + )) + # output arguments from presets + jpeg_items.extend(ffmpeg_args.get("output") or []) + # we just want one frame from movie files + jpeg_items.append("-vframes 1") + # output file + jpeg_items.append(path_to_subprocess_arg(dst_path)) + subprocess_command = " ".join(jpeg_items) + try: + run_subprocess( + subprocess_command, shell=True, logger=self.log + ) + return True + except Exception: + self.log.warning( + "Failed to create thubmnail using ffmpeg", + exc_info=True + ) + return False diff --git a/openpype/plugins/publish/extract_review.py b/openpype/plugins/publish/extract_review.py index 879125dac3..b6e5fee1fe 100644 --- a/openpype/plugins/publish/extract_review.py +++ b/openpype/plugins/publish/extract_review.py @@ -763,7 +763,8 @@ class ExtractReview(pyblish.api.InstancePlugin): start_frame = int(start_frame) end_frame = int(end_frame) collections = clique.assemble(files)[0] - assert len(collections) == 1, "Multiple collections found." + msg = "Multiple collections {} found.".format(collections) + assert len(collections) == 1, msg col = collections[0] # do nothing if no gap is found in input range diff --git a/openpype/plugins/publish/extract_review_slate.py b/openpype/plugins/publish/extract_review_slate.py index cff71f67ac..28685c2e90 100644 --- a/openpype/plugins/publish/extract_review_slate.py +++ b/openpype/plugins/publish/extract_review_slate.py @@ -1,4 +1,6 @@ import os +from pprint import pformat +import re import openpype.api import pyblish from openpype.lib import ( @@ -21,6 +23,8 @@ class ExtractReviewSlate(openpype.api.Extractor): families = ["slate", "review"] match = pyblish.api.Subset + SUFFIX = "_slate" + hosts = ["nuke", "shell"] optional = True @@ -29,28 +33,19 @@ class ExtractReviewSlate(openpype.api.Extractor): if "representations" not in inst_data: raise RuntimeError("Burnin needs already created mov to work on.") - suffix = "_slate" - slate_path = inst_data.get("slateFrame") + # get slates frame from upstream + slates_data = inst_data.get("slateFrames") + if not slates_data: + # make it backward compatible and open for slates generator + # premium plugin + slates_data = { + "*": inst_data["slateFrame"] + } + + self.log.info("_ slates_data: {}".format(pformat(slates_data))) + ffmpeg_path = get_ffmpeg_tool_path("ffmpeg") - slate_streams = get_ffprobe_streams(slate_path, self.log) - # Try to find first stream with defined 'width' and 'height' - # - this is to avoid order of streams where audio can be as first - # - there may be a better way (checking `codec_type`?)+ - slate_width = None - slate_height = None - for slate_stream in slate_streams: - if "width" in slate_stream and "height" in slate_stream: - slate_width = int(slate_stream["width"]) - slate_height = int(slate_stream["height"]) - break - - # Raise exception of any stream didn't define input resolution - if slate_width is None: - raise AssertionError(( - "FFprobe couldn't read resolution from input file: \"{}\"" - ).format(slate_path)) - if "reviewToWidth" in inst_data: use_legacy_code = True else: @@ -77,6 +72,12 @@ class ExtractReviewSlate(openpype.api.Extractor): streams = get_ffprobe_streams( input_path, self.log ) + # get slate data + slate_path = self._get_slate_path(input_file, slates_data) + self.log.info("_ slate_path: {}".format(slate_path)) + + slate_width, slate_height = self._get_slates_resolution(slate_path) + # Get video metadata ( input_width, @@ -138,7 +139,7 @@ class ExtractReviewSlate(openpype.api.Extractor): _remove_at_end = [] ext = os.path.splitext(input_file)[1] - output_file = input_file.replace(ext, "") + suffix + ext + output_file = input_file.replace(ext, "") + self.SUFFIX + ext _remove_at_end.append(input_path) @@ -369,6 +370,43 @@ class ExtractReviewSlate(openpype.api.Extractor): self.log.debug(inst_data["representations"]) + def _get_slate_path(self, input_file, slates_data): + slate_path = None + for sl_n, _slate_path in slates_data.items(): + if "*" in sl_n: + slate_path = _slate_path + break + elif re.search(sl_n, input_file): + slate_path = _slate_path + break + + if not slate_path: + raise AttributeError( + "Missing slates paths: {}".format(slates_data)) + + return slate_path + + def _get_slates_resolution(self, slate_path): + slate_streams = get_ffprobe_streams(slate_path, self.log) + # Try to find first stream with defined 'width' and 'height' + # - this is to avoid order of streams where audio can be as first + # - there may be a better way (checking `codec_type`?)+ + slate_width = None + slate_height = None + for slate_stream in slate_streams: + if "width" in slate_stream and "height" in slate_stream: + slate_width = int(slate_stream["width"]) + slate_height = int(slate_stream["height"]) + break + + # Raise exception of any stream didn't define input resolution + if slate_width is None: + raise AssertionError(( + "FFprobe couldn't read resolution from input file: \"{}\"" + ).format(slate_path)) + + return (slate_width, slate_height) + def _get_video_metadata(self, streams): input_timecode = "" input_width = None diff --git a/openpype/plugins/publish/integrate_new.py b/openpype/plugins/publish/integrate_new.py index 91f6102501..2471105250 100644 --- a/openpype/plugins/publish/integrate_new.py +++ b/openpype/plugins/publish/integrate_new.py @@ -940,9 +940,8 @@ class IntegrateAssetNew(pyblish.api.InstancePlugin): families += current_families # create relative source path for DB - if "source" in instance.data: - source = instance.data["source"] - else: + source = instance.data.get("source") + if not source: source = context.data["currentFile"] anatomy = instance.context.data["anatomy"] source = self.get_rootless_path(anatomy, source) diff --git a/openpype/settings/defaults/project_settings/deadline.json b/openpype/settings/defaults/project_settings/deadline.json index 5c5a14bf21..a6e7b4a94a 100644 --- a/openpype/settings/defaults/project_settings/deadline.json +++ b/openpype/settings/defaults/project_settings/deadline.json @@ -83,9 +83,6 @@ "maya": [ ".*([Bb]eauty).*" ], - "nuke": [ - ".*" - ], "aftereffects": [ ".*" ], diff --git a/openpype/settings/defaults/project_settings/global.json b/openpype/settings/defaults/project_settings/global.json index 7b223798f1..cedd0eed99 100644 --- a/openpype/settings/defaults/project_settings/global.json +++ b/openpype/settings/defaults/project_settings/global.json @@ -33,7 +33,7 @@ "enabled": false, "profiles": [] }, - "ExtractJpegEXR": { + "ExtractThumbnail": { "enabled": true, "ffmpeg_args": { "input": [ diff --git a/openpype/settings/defaults/project_settings/maya.json b/openpype/settings/defaults/project_settings/maya.json index efd22e13c8..cdd3a62d00 100644 --- a/openpype/settings/defaults/project_settings/maya.json +++ b/openpype/settings/defaults/project_settings/maya.json @@ -66,6 +66,28 @@ "defaults": [], "joint_hints": "jnt_org" }, + "CreateMultiverseLook": { + "enabled": true, + "publish_mip_map": true + }, + "CreateMultiverseUsd": { + "enabled": true, + "defaults": [ + "Main" + ] + }, + "CreateMultiverseUsdComp": { + "enabled": true, + "defaults": [ + "Main" + ] + }, + "CreateMultiverseUsdOver": { + "enabled": true, + "defaults": [ + "Main" + ] + }, "CreateAnimation": { "enabled": true, "defaults": [ @@ -379,6 +401,14 @@ "optional": true, "active": true }, + "ExtractAlembic": { + "enabled": true, + "families": [ + "pointcache", + "model", + "vrayproxy" + ] + }, "ValidateRigContents": { "enabled": false, "optional": true, @@ -413,6 +443,11 @@ "optional": true, "active": true }, + "ValidateCameraContents": { + "enabled": true, + "optional": true, + "validate_shapes": true + }, "ExtractPlayblast": { "capture_preset": { "Codec": { diff --git a/openpype/settings/defaults/project_settings/nuke.json b/openpype/settings/defaults/project_settings/nuke.json index 16348bec85..6c45e2a9c1 100644 --- a/openpype/settings/defaults/project_settings/nuke.json +++ b/openpype/settings/defaults/project_settings/nuke.json @@ -166,6 +166,9 @@ }, "ExtractThumbnail": { "enabled": true, + "use_rendered": true, + "bake_viewer_process": true, + "bake_viewer_input_process": true, "nodes": { "Reformat": [ [ diff --git a/openpype/settings/defaults/system_settings/modules.json b/openpype/settings/defaults/system_settings/modules.json index 537e287366..8cd4114cb0 100644 --- a/openpype/settings/defaults/system_settings/modules.json +++ b/openpype/settings/defaults/system_settings/modules.json @@ -59,13 +59,11 @@ "applications": { "write_security_roles": [ "API", - "Administrator", - "Pypeclub" + "Administrator" ], "read_security_roles": [ "API", - "Administrator", - "Pypeclub" + "Administrator" ] } }, @@ -73,25 +71,21 @@ "tools_env": { "write_security_roles": [ "API", - "Administrator", - "Pypeclub" + "Administrator" ], "read_security_roles": [ "API", - "Administrator", - "Pypeclub" + "Administrator" ] }, "avalon_mongo_id": { "write_security_roles": [ "API", - "Administrator", - "Pypeclub" + "Administrator" ], "read_security_roles": [ "API", - "Administrator", - "Pypeclub" + "Administrator" ] }, "fps": { diff --git a/openpype/settings/entities/schemas/projects_schema/schemas/schema_global_publish.json b/openpype/settings/entities/schemas/projects_schema/schemas/schema_global_publish.json index 061874e31c..a3cbf0cfcd 100644 --- a/openpype/settings/entities/schemas/projects_schema/schemas/schema_global_publish.json +++ b/openpype/settings/entities/schemas/projects_schema/schemas/schema_global_publish.json @@ -126,8 +126,8 @@ "type": "dict", "collapsible": true, "checkbox_key": "enabled", - "key": "ExtractJpegEXR", - "label": "ExtractJpegEXR", + "key": "ExtractThumbnail", + "label": "ExtractThumbnail", "is_group": true, "children": [ { diff --git a/openpype/settings/entities/schemas/projects_schema/schemas/schema_maya_create.json b/openpype/settings/entities/schemas/projects_schema/schemas/schema_maya_create.json index 6dc10ed2a5..09287a8b50 100644 --- a/openpype/settings/entities/schemas/projects_schema/schemas/schema_maya_create.json +++ b/openpype/settings/entities/schemas/projects_schema/schemas/schema_maya_create.json @@ -124,10 +124,41 @@ ] }, + { + "type": "dict", + "collapsible": true, + "key": "CreateMultiverseLook", + "label": "Create Multiverse Look", + "checkbox_key": "enabled", + "children": [ + { + "type": "boolean", + "key": "enabled", + "label": "Enabled" + }, + { + "type": "boolean", + "key": "publish_mip_map", + "label": "Publish Mip Maps" + } + ] + }, { "type": "schema_template", "name": "template_create_plugin", "template_data": [ + { + "key": "CreateMultiverseUsd", + "label": "Create Multiverse USD" + }, + { + "key": "CreateMultiverseUsdComp", + "label": "Create Multiverse USD Composition" + }, + { + "key": "CreateMultiverseUsdOver", + "label": "Create Multiverse USD Override" + }, { "key": "CreateAnimation", "label": "Create Animation" diff --git a/openpype/settings/entities/schemas/projects_schema/schemas/schema_maya_publish.json b/openpype/settings/entities/schemas/projects_schema/schemas/schema_maya_publish.json index 9877b5ff0d..41b681d893 100644 --- a/openpype/settings/entities/schemas/projects_schema/schemas/schema_maya_publish.json +++ b/openpype/settings/entities/schemas/projects_schema/schemas/schema_maya_publish.json @@ -504,6 +504,30 @@ "label": "ValidateUniqueNames" } ] + }, + { + "type": "label", + "label": "Extractors" + }, + { + "type": "dict", + "collapsible": true, + "key": "ExtractAlembic", + "label": "Extract Alembic", + "checkbox_key": "enabled", + "children": [ + { + "type": "boolean", + "key": "enabled", + "label": "Enabled" + }, + { + "key": "families", + "label": "Families", + "type": "list", + "object_type": "text" + } + ] } ] }, @@ -570,6 +594,30 @@ } ] }, + { + "type": "dict", + "collapsible": true, + "key": "ValidateCameraContents", + "label": "Validate Camera Content", + "checkbox_key": "enabled", + "children": [ + { + "type": "boolean", + "key": "enabled", + "label": "Enabled" + }, + { + "type": "boolean", + "key": "optional", + "label": "Optional" + }, + { + "type": "boolean", + "key": "validate_shapes", + "label": "Validate presence of shapes" + } + ] + }, { "type": "splitter" }, diff --git a/openpype/settings/entities/schemas/projects_schema/schemas/schema_nuke_publish.json b/openpype/settings/entities/schemas/projects_schema/schemas/schema_nuke_publish.json index 04df957d67..575bfe79e7 100644 --- a/openpype/settings/entities/schemas/projects_schema/schemas/schema_nuke_publish.json +++ b/openpype/settings/entities/schemas/projects_schema/schemas/schema_nuke_publish.json @@ -135,9 +135,31 @@ "label": "Enabled" }, { - "type": "raw-json", - "key": "nodes", - "label": "Nodes" + "type": "boolean", + "key": "use_rendered", + "label": "Use rendered images" + }, + { + "type": "boolean", + "key": "bake_viewer_process", + "label": "Bake viewer process" + }, + { + "type": "boolean", + "key": "bake_viewer_input_process", + "label": "Bake viewer input process" + }, + { + "type": "collapsible-wrap", + "label": "Nodes", + "collapsible": true, + "children": [ + { + "type": "raw-json", + "key": "nodes", + "label": "Nodes" + } + ] } ] }, diff --git a/openpype/tools/assetlinks/widgets.py b/openpype/tools/assetlinks/widgets.py index 22e8848a60..1b168e542c 100644 --- a/openpype/tools/assetlinks/widgets.py +++ b/openpype/tools/assetlinks/widgets.py @@ -1,10 +1,16 @@ +import collections +from openpype.client import ( + get_versions, + get_subsets, + get_assets, + get_output_link_versions, +) from Qt import QtWidgets class SimpleLinkView(QtWidgets.QWidget): - - def __init__(self, dbcon, parent=None): + def __init__(self, dbcon, parent): super(SimpleLinkView, self).__init__(parent=parent) self.dbcon = dbcon @@ -24,6 +30,11 @@ class SimpleLinkView(QtWidgets.QWidget): self._in_view = in_view self._out_view = out_view + self._version_doc_to_process = None + + @property + def project_name(self): + return self.dbcon.current_project() def clear(self): self._in_view.clear() @@ -31,60 +42,114 @@ class SimpleLinkView(QtWidgets.QWidget): def set_version(self, version_doc): self.clear() - if not version_doc or not self.isVisible(): - return + self._version_doc_to_process = version_doc + if version_doc and self.isVisible(): + self._fill_values() - # inputs - # + def showEvent(self, event): + super(SimpleLinkView, self).showEvent(event) + self._fill_values() + + def _fill_values(self): + if self._version_doc_to_process is None: + return + version_doc = self._version_doc_to_process + self._version_doc_to_process = None + self._fill_inputs(version_doc) + self._fill_outputs(version_doc) + + def _fill_inputs(self, version_doc): + version_ids = set() for link in version_doc["data"].get("inputLinks", []): # Backwards compatibility for "input" key used as "id" if "id" not in link: link_id = link["input"] else: link_id = link["id"] - version = self.dbcon.find_one( - {"_id": link_id, "type": "version"}, - projection={"name": 1, "parent": 1} - ) - if not version: - continue - subset = self.dbcon.find_one( - {"_id": version["parent"], "type": "subset"}, - projection={"name": 1, "parent": 1} - ) - if not subset: - continue - asset = self.dbcon.find_one( - {"_id": subset["parent"], "type": "asset"}, - projection={"name": 1} - ) + version_ids.add(link_id) - self._in_view.addItem("{asset} {subset} v{version:0>3}".format( - asset=asset["name"], - subset=subset["name"], - version=version["name"], + version_docs = list(get_versions( + self.project_name, + version_ids=version_ids, + fields=["name", "parent"] + )) + + versions_by_subset_id = collections.defaultdict(list) + for version_doc in version_docs: + subset_id = version_doc["parent"] + versions_by_subset_id[subset_id].append(version_doc) + + subset_docs = [] + if versions_by_subset_id: + subset_docs = list(get_subsets( + self.project_name, + subset_ids=versions_by_subset_id.keys(), + fields=["_id", "name", "parent"] )) - # outputs - # - outputs = self.dbcon.find( - {"type": "version", "data.inputLinks.input": version_doc["_id"]}, - projection={"name": 1, "parent": 1} - ) - for version in outputs or []: - subset = self.dbcon.find_one( - {"_id": version["parent"], "type": "subset"}, - projection={"name": 1, "parent": 1} - ) - if not subset: - continue - asset = self.dbcon.find_one( - {"_id": subset["parent"], "type": "asset"}, - projection={"name": 1} - ) + asset_docs = [] + subsets_by_asset_id = collections.defaultdict(list) + if subset_docs: + for subset_doc in subset_docs: + asset_id = subset_doc["parent"] + subsets_by_asset_id[asset_id].append(subset_doc) - self._out_view.addItem("{asset} {subset} v{version:0>3}".format( - asset=asset["name"], - subset=subset["name"], - version=version["name"], + asset_docs = list(get_assets( + self.project_name, + asset_ids=subsets_by_asset_id.keys(), + fields=["_id", "name"] )) + + for asset_doc in asset_docs: + asset_id = asset_doc["_id"] + for subset_doc in subsets_by_asset_id[asset_id]: + subset_id = subset_doc["_id"] + for version_doc in versions_by_subset_id[subset_id]: + self._in_view.addItem("{} {} v{:0>3}".format( + asset_doc["name"], + subset_doc["name"], + version_doc["name"], + )) + + def _fill_outputs(self, version_doc): + version_docs = list(get_output_link_versions( + self.project_name, + version_doc["_id"], + fields=["name", "parent"] + )) + versions_by_subset_id = collections.defaultdict(list) + for version_doc in version_docs: + subset_id = version_doc["parent"] + versions_by_subset_id[subset_id].append(version_doc) + + subset_docs = [] + if versions_by_subset_id: + subset_docs = list(get_subsets( + self.project_name, + subset_ids=versions_by_subset_id.keys(), + fields=["_id", "name", "parent"] + )) + + asset_docs = [] + subsets_by_asset_id = collections.defaultdict(list) + if subset_docs: + for subset_doc in subset_docs: + asset_id = subset_doc["parent"] + subsets_by_asset_id[asset_id].append(subset_doc) + + asset_docs = list(get_assets( + self.project_name, + asset_ids=subsets_by_asset_id.keys(), + fields=["_id", "name"] + )) + + for asset_doc in asset_docs: + asset_id = asset_doc["_id"] + for subset_doc in subsets_by_asset_id[asset_id]: + subset_id = subset_doc["_id"] + for version_doc in versions_by_subset_id[subset_id]: + self._out_view.addItem("{} {} v{:0>3}".format( + asset_doc["name"], + subset_doc["name"], + version_doc["name"], + )) diff --git a/openpype/tools/creator/window.py b/openpype/tools/creator/window.py index e0c329fb78..a3937d6a40 100644 --- a/openpype/tools/creator/window.py +++ b/openpype/tools/creator/window.py @@ -4,6 +4,7 @@ import re from Qt import QtWidgets, QtCore +from openpype.client import get_asset_by_name, get_subsets from openpype import style from openpype.api import get_current_project_settings from openpype.tools.utils.lib import qt_app_context @@ -215,12 +216,12 @@ class CreatorWindow(QtWidgets.QDialog): self._set_valid_state(False) return + project_name = legacy_io.active_project() asset_doc = None if creator_plugin: # Get the asset from the database which match with the name - asset_doc = legacy_io.find_one( - {"name": asset_name, "type": "asset"}, - projection={"_id": 1} + asset_doc = get_asset_by_name( + project_name, asset_name, fields=["_id"] ) # Get plugin @@ -235,7 +236,6 @@ class CreatorWindow(QtWidgets.QDialog): self._set_valid_state(False) return - project_name = legacy_io.Session["AVALON_PROJECT"] asset_id = asset_doc["_id"] task_name = legacy_io.Session["AVALON_TASK"] @@ -269,14 +269,13 @@ class CreatorWindow(QtWidgets.QDialog): self._subset_name_input.setText(subset_name) # Get all subsets of the current asset - subset_docs = legacy_io.find( - { - "type": "subset", - "parent": asset_id - }, - {"name": 1} + subset_docs = get_subsets( + project_name, asset_ids=[asset_id], fields=["name"] ) - existing_subset_names = set(subset_docs.distinct("name")) + existing_subset_names = { + subset_doc["name"] + for subset_doc in subset_docs + } existing_subset_names_low = set( _name.lower() for _name in existing_subset_names diff --git a/openpype/tools/launcher/models.py b/openpype/tools/launcher/models.py index 13567e7916..3f899cc05e 100644 --- a/openpype/tools/launcher/models.py +++ b/openpype/tools/launcher/models.py @@ -9,6 +9,10 @@ import appdirs from Qt import QtCore, QtGui import qtawesome +from openpype.client import ( + get_project, + get_assets, +) from openpype.lib import JSONSettingRegistry from openpype.lib.applications import ( CUSTOM_LAUNCH_APP_GROUPS, @@ -81,13 +85,11 @@ class ActionModel(QtGui.QStandardItemModel): def get_application_actions(self): actions = [] - if not self.dbcon.Session.get("AVALON_PROJECT"): + if not self.dbcon.current_project(): return actions - project_doc = self.dbcon.find_one( - {"type": "project"}, - {"config.apps": True} - ) + project_name = self.dbcon.active_project() + project_doc = get_project(project_name, fields=["config.apps"]) if not project_doc: return actions @@ -448,7 +450,7 @@ class LauncherModel(QtCore.QObject): @property def project_name(self): """Current project name.""" - return self._dbcon.Session.get("AVALON_PROJECT") + return self._dbcon.current_project() @property def refreshing_assets(self): @@ -649,9 +651,8 @@ class LauncherModel(QtCore.QObject): self._asset_refresh_thread = None def _refresh_assets(self): - asset_docs = list(self._dbcon.find( - {"type": "asset"}, - self._asset_projection + asset_docs = list(get_assets( + self._last_project_name, fields=self._asset_projection.keys() )) if not self._refreshing_assets: return diff --git a/openpype/tools/libraryloader/app.py b/openpype/tools/libraryloader/app.py index 7fda6bd6f9..5f4d10d796 100644 --- a/openpype/tools/libraryloader/app.py +++ b/openpype/tools/libraryloader/app.py @@ -3,6 +3,7 @@ import sys from Qt import QtWidgets, QtCore, QtGui from openpype import style +from openpype.client import get_project from openpype.pipeline import AvalonMongoDB from openpype.tools.utils import lib as tools_lib from openpype.tools.loader.widgets import ( @@ -303,14 +304,26 @@ class LibraryLoaderWindow(QtWidgets.QDialog): families = self._subsets_widget.get_subsets_families() self._families_filter_view.set_enabled_families(families) - def set_context(self, context, refresh=True): - self.echo("Setting context: {}".format(context)) - lib.schedule( - lambda: self._set_context(context, refresh=refresh), - 50, channel="mongo" - ) - # ------------------------------ + def set_context(self, context, refresh=True): + """Set the selection in the interface using a context. + The context must contain `asset` data by name. + + Args: + context (dict): The context to apply. + Returns: + None + """ + + asset_name = context.get("asset", None) + if asset_name is None: + return + + if refresh: + self._refresh_assets() + + self._assets_widget.select_asset_by_name(asset_name) + def _on_family_filter_change(self, families): self._subsets_widget.set_family_filters(families) @@ -323,10 +336,7 @@ class LibraryLoaderWindow(QtWidgets.QDialog): """Load assets from database""" if self.current_project is not None: # Ensure a project is loaded - project_doc = self.dbcon.find_one( - {"type": "project"}, - {"type": 1} - ) + project_doc = get_project(self.current_project, fields=["_id"]) assert project_doc, "This is a bug" self._families_filter_view.set_enabled_families(set()) @@ -371,7 +381,7 @@ class LibraryLoaderWindow(QtWidgets.QDialog): # Clear the version information on asset change self._version_info_widget.set_version(None) - self._thumbnail_widget.set_thumbnail(asset_ids) + self._thumbnail_widget.set_thumbnail("asset", asset_ids) self.data["state"]["assetIds"] = asset_ids @@ -426,34 +436,17 @@ class LibraryLoaderWindow(QtWidgets.QDialog): version_doc["_id"] for version_doc in version_docs ] + src_type = "version" if not thumbnail_src_ids: + src_type = "asset" thumbnail_src_ids = self._assets_widget.get_selected_asset_ids() - self._thumbnail_widget.set_thumbnail(thumbnail_src_ids) + self._thumbnail_widget.set_thumbnail(src_type, thumbnail_src_ids) version_ids = [doc["_id"] for doc in version_docs or []] if self._repres_widget: self._repres_widget.set_version_ids(version_ids) - def _set_context(self, context, refresh=True): - """Set the selection in the interface using a context. - The context must contain `asset` data by name. - - Args: - context (dict): The context to apply. - Returns: - None - """ - - asset_name = context.get("asset", None) - if asset_name is None: - return - - if refresh: - self._refresh_assets() - - self._assets_widget.select_asset_by_name(asset_name) - def _on_message_timeout(self): self._message_label.setText("") diff --git a/openpype/tools/loader/app.py b/openpype/tools/loader/app.py index bb589c199d..1917f23c60 100644 --- a/openpype/tools/loader/app.py +++ b/openpype/tools/loader/app.py @@ -3,6 +3,7 @@ import traceback from Qt import QtWidgets, QtCore +from openpype.client import get_projects, get_project from openpype import style from openpype.lib import register_event_callback from openpype.pipeline import ( @@ -39,7 +40,7 @@ class LoaderWindow(QtWidgets.QDialog): def __init__(self, parent=None): super(LoaderWindow, self).__init__(parent) title = "Asset Loader 2.1" - project_name = legacy_io.Session.get("AVALON_PROJECT") + project_name = legacy_io.active_project() if project_name: title += " - {}".format(project_name) self.setWindowTitle(title) @@ -274,8 +275,9 @@ class LoaderWindow(QtWidgets.QDialog): """Load assets from database""" # Ensure a project is loaded - project = legacy_io.find_one({"type": "project"}, {"type": 1}) - assert project, "Project was not found! This is a bug" + project_name = legacy_io.active_project() + project_doc = get_project(project_name, fields=["_id"]) + assert project_doc, "Project was not found! This is a bug" self._assets_widget.refresh() self._assets_widget.setFocus() @@ -314,7 +316,7 @@ class LoaderWindow(QtWidgets.QDialog): ) # Clear the version information on asset change - self._thumbnail_widget.set_thumbnail(asset_ids) + self._thumbnail_widget.set_thumbnail("asset", asset_ids) self._version_info_widget.set_version(None) self.data["state"]["assetIds"] = asset_ids @@ -371,10 +373,12 @@ class LoaderWindow(QtWidgets.QDialog): version_doc["_id"] for version_doc in version_docs ] + source_type = "version" if not thumbnail_src_ids: + source_type = "asset" thumbnail_src_ids = self._assets_widget.get_selected_asset_ids() - self._thumbnail_widget.set_thumbnail(thumbnail_src_ids) + self._thumbnail_widget.set_thumbnail(source_type, thumbnail_src_ids) if self._repres_widget is not None: version_ids = [doc["_id"] for doc in version_docs] @@ -576,8 +580,7 @@ def show(debug=False, parent=None, use_context=False): legacy_io.install() any_project = next( - project for project in legacy_io.projects() - if project.get("active", True) is not False + project for project in get_projects(fields=["name"]) ) legacy_io.Session["AVALON_PROJECT"] = any_project["name"] diff --git a/openpype/tools/loader/model.py b/openpype/tools/loader/model.py index 6f39c428be..a5174bd804 100644 --- a/openpype/tools/loader/model.py +++ b/openpype/tools/loader/model.py @@ -7,6 +7,15 @@ from uuid import uuid4 from Qt import QtCore, QtGui import qtawesome +from openpype.client import ( + get_assets, + get_subsets, + get_last_versions, + get_versions, + get_hero_versions, + get_version_by_name, + get_representations +) from openpype.pipeline import ( HeroVersionType, schema, @@ -56,7 +65,7 @@ class BaseRepresentationModel(object): remote_site = remote_provider = None if not project_name: - project_name = self.dbcon.Session.get("AVALON_PROJECT") + project_name = self.dbcon.active_project() else: self.dbcon.Session["AVALON_PROJECT"] = project_name @@ -89,7 +98,7 @@ class BaseRepresentationModel(object): self._last_manager_cache = now_time sync_server = self._modules_manager.modules_by_name["sync_server"] - if sync_server.is_project_enabled(project_name): + if sync_server.is_project_enabled(project_name, single=True): active_site = sync_server.get_active_site(project_name) active_provider = sync_server.get_provider_for_site( project_name, active_site) @@ -197,9 +206,6 @@ class SubsetsModel(TreeModel, BaseRepresentationModel): if subset_doc_projection: self.subset_doc_projection = subset_doc_projection - self.asset_doc_projection = asset_doc_projection - self.subset_doc_projection = subset_doc_projection - self.repre_icons = {} self.sync_server = None self.active_site = self.active_provider = None @@ -225,7 +231,7 @@ class SubsetsModel(TreeModel, BaseRepresentationModel): self._doc_fetching_stop = False self._doc_payload = {} - self.doc_fetched.connect(self.on_doc_fetched) + self.doc_fetched.connect(self._on_doc_fetched) self.refresh() @@ -244,7 +250,7 @@ class SubsetsModel(TreeModel, BaseRepresentationModel): def set_grouping(self, state): self._grouping = state - self.on_doc_fetched() + self._on_doc_fetched() def get_subsets_families(self): return self._doc_payload.get("subset_families") or set() @@ -254,57 +260,61 @@ class SubsetsModel(TreeModel, BaseRepresentationModel): # because it also updates the information in other columns if index.column() == self.columns_index["version"]: item = index.internalPointer() - parent = item["_id"] + subset_id = item["_id"] if isinstance(value, HeroVersionType): - versions = list(self.dbcon.find({ - "type": {"$in": ["version", "hero_version"]}, - "parent": parent - }, sort=[("name", -1)])) - - version = None - last_version = None - for __version in versions: - if __version["type"] == "hero_version": - version = __version - elif last_version is None: - last_version = __version - - if version is not None and last_version is not None: - break - - _version = None - for __version in versions: - if __version["_id"] == version["version_id"]: - _version = __version - break - - version["data"] = _version["data"] - version["name"] = _version["name"] - version["is_from_latest"] = ( - last_version["_id"] == _version["_id"] - ) + version_doc = self._get_hero_version(subset_id) else: - version = self.dbcon.find_one({ - "name": value, - "type": "version", - "parent": parent - }) + project_name = self.dbcon.active_project() + version_doc = get_version_by_name( + project_name, value, subset_id + ) # update availability on active site when version changes - if self.sync_server.enabled and version: - query = self._repre_per_version_pipeline([version["_id"]], - self.active_site, - self.remote_site) + if self.sync_server.enabled and version_doc: + query = self._repre_per_version_pipeline( + [version_doc["_id"]], + self.active_site, + self.remote_site + ) docs = list(self.dbcon.aggregate(query)) if docs: repre = docs.pop() - version["data"].update(self._get_repre_dict(repre)) + version_doc["data"].update(self._get_repre_dict(repre)) - self.set_version(index, version) + self.set_version(index, version_doc) return super(SubsetsModel, self).setData(index, value, role) + def _get_hero_version(self, subset_id): + project_name = self.dbcon.active_project() + version_docs = get_versions( + project_name, subset_ids=[subset_id], hero=True + ) + standard_versions = [] + hero_version_doc = None + for version_doc in version_docs: + if version_doc["type"] == "hero_version": + hero_version_doc = version_doc + continue + standard_versions.append(version_doc) + + src_version_id = hero_version_doc["version_id"] + src_version = None + is_from_latest = True + for version_doc in reversed(sorted( + standard_versions, key=lambda item: item["name"] + )): + if version_doc["_id"] == src_version_id: + src_version = version_doc + break + is_from_latest = False + + hero_version_doc["data"] = src_version["data"] + hero_version_doc["name"] = src_version["name"] + hero_version_doc["is_from_latest"] = is_from_latest + return hero_version_doc + def set_version(self, index, version): """Update the version data of the given index. @@ -391,26 +401,25 @@ class SubsetsModel(TreeModel, BaseRepresentationModel): item["repre_info"] = repre_info def _fetch(self): - asset_docs = self.dbcon.find( - { - "type": "asset", - "_id": {"$in": self._asset_ids} - }, - self.asset_doc_projection + project_name = self.dbcon.active_project() + asset_docs = get_assets( + project_name, + asset_ids=self._asset_ids, + fields=self.asset_doc_projection.keys() ) + asset_docs_by_id = { asset_doc["_id"]: asset_doc for asset_doc in asset_docs } subset_docs_by_id = {} - subset_docs = self.dbcon.find( - { - "type": "subset", - "parent": {"$in": self._asset_ids} - }, - self.subset_doc_projection + subset_docs = get_subsets( + project_name, + asset_ids=self._asset_ids, + fields=self.subset_doc_projection.keys() ) + subset_families = set() for subset_doc in subset_docs: if self._doc_fetching_stop: @@ -423,37 +432,13 @@ class SubsetsModel(TreeModel, BaseRepresentationModel): subset_docs_by_id[subset_doc["_id"]] = subset_doc subset_ids = list(subset_docs_by_id.keys()) - _pipeline = [ - # Find all versions of those subsets - {"$match": { - "type": "version", - "parent": {"$in": subset_ids} - }}, - # Sorting versions all together - {"$sort": {"name": 1}}, - # Group them by "parent", but only take the last - {"$group": { - "_id": "$parent", - "_version_id": {"$last": "$_id"}, - "name": {"$last": "$name"}, - "type": {"$last": "$type"}, - "data": {"$last": "$data"}, - "locations": {"$last": "$locations"}, - "schema": {"$last": "$schema"} - }} - ] - last_versions_by_subset_id = dict() - for doc in self.dbcon.aggregate(_pipeline): - if self._doc_fetching_stop: - return - doc["parent"] = doc["_id"] - doc["_id"] = doc.pop("_version_id") - last_versions_by_subset_id[doc["parent"]] = doc + last_versions_by_subset_id = get_last_versions( + project_name, + subset_ids, + fields=["_id", "parent", "name", "type", "data", "schema"] + ) - hero_versions = self.dbcon.find({ - "type": "hero_version", - "parent": {"$in": subset_ids} - }) + hero_versions = get_hero_versions(project_name, subset_ids=subset_ids) missing_versions = [] for hero_version in hero_versions: version_id = hero_version["version_id"] @@ -462,10 +447,9 @@ class SubsetsModel(TreeModel, BaseRepresentationModel): missing_versions_by_id = {} if missing_versions: - missing_version_docs = self.dbcon.find({ - "type": "version", - "_id": {"$in": missing_versions} - }) + missing_version_docs = get_versions( + project_name, version_ids=missing_versions + ) missing_versions_by_id = { missing_version_doc["_id"]: missing_version_doc for missing_version_doc in missing_version_docs @@ -488,23 +472,16 @@ class SubsetsModel(TreeModel, BaseRepresentationModel): last_versions_by_subset_id[subset_id] = hero_version - self._doc_payload = { - "asset_docs_by_id": asset_docs_by_id, - "subset_docs_by_id": subset_docs_by_id, - "subset_families": subset_families, - "last_versions_by_subset_id": last_versions_by_subset_id - } - + repre_info = {} if self.sync_server.enabled: version_ids = set() for _subset_id, doc in last_versions_by_subset_id.items(): version_ids.add(doc["_id"]) - query = self._repre_per_version_pipeline(list(version_ids), - self.active_site, - self.remote_site) + query = self._repre_per_version_pipeline( + list(version_ids), self.active_site, self.remote_site + ) - repre_info = {} for doc in self.dbcon.aggregate(query): if self._doc_fetching_stop: return @@ -512,7 +489,13 @@ class SubsetsModel(TreeModel, BaseRepresentationModel): doc["remote_provider"] = self.remote_provider repre_info[doc["_id"]] = doc - self._doc_payload["repre_info_by_version_id"] = repre_info + self._doc_payload = { + "asset_docs_by_id": asset_docs_by_id, + "subset_docs_by_id": subset_docs_by_id, + "subset_families": subset_families, + "last_versions_by_subset_id": last_versions_by_subset_id, + "repre_info_by_version_id": repre_info + } self.doc_fetched.emit() @@ -545,7 +528,7 @@ class SubsetsModel(TreeModel, BaseRepresentationModel): self.fetch_subset_and_version() - def on_doc_fetched(self): + def _on_doc_fetched(self): self.clear() self._items_by_id = {} self.beginResetModel() @@ -1036,7 +1019,6 @@ class RepresentationSortProxyModel(GroupMemberFilterProxyModel): class RepresentationModel(TreeModel, BaseRepresentationModel): - doc_fetched = QtCore.Signal() refreshed = QtCore.Signal(bool) @@ -1062,33 +1044,43 @@ class RepresentationModel(TreeModel, BaseRepresentationModel): "remote_site": "Remote" } - def __init__(self, dbcon, header, version_ids): + repre_projection = { + "_id": 1, + "name": 1, + "context.subset": 1, + "context.asset": 1, + "context.version": 1, + "context.representation": 1, + 'files.sites': 1 + } + + def __init__(self, dbcon, header): super(RepresentationModel, self).__init__() self.dbcon = dbcon self._data = [] self._header = header - self.version_ids = version_ids + self._version_ids = [] manager = ModulesManager() sync_server = active_site = remote_site = None active_provider = remote_provider = None - project = dbcon.Session["AVALON_PROJECT"] - if project: + project_name = dbcon.current_project() + if project_name: sync_server = manager.modules_by_name["sync_server"] - active_site = sync_server.get_active_site(project) - remote_site = sync_server.get_remote_site(project) + active_site = sync_server.get_active_site(project_name) + remote_site = sync_server.get_remote_site(project_name) # TODO refactor - active_provider = \ - sync_server.get_provider_for_site(project, - active_site) + active_provider = sync_server.get_provider_for_site( + project_name, active_site + ) if active_site == 'studio': active_provider = 'studio' - remote_provider = \ - sync_server.get_provider_for_site(project, - remote_site) + remote_provider = sync_server.get_provider_for_site( + project_name, remote_site + ) if remote_site == 'studio': remote_provider = 'studio' @@ -1099,7 +1091,7 @@ class RepresentationModel(TreeModel, BaseRepresentationModel): self.remote_site = remote_site self.remote_provider = remote_provider - self.doc_fetched.connect(self.on_doc_fetched) + self.doc_fetched.connect(self._on_doc_fetched) self._docs = {} self._icons = lib.get_repre_icons() @@ -1110,7 +1102,7 @@ class RepresentationModel(TreeModel, BaseRepresentationModel): self._items_by_id = {} def set_version_ids(self, version_ids): - self.version_ids = version_ids + self._version_ids = version_ids self.refresh() def data(self, index, role): @@ -1127,8 +1119,7 @@ class RepresentationModel(TreeModel, BaseRepresentationModel): if index.column() == self.Columns.index("name"): if item.get("isMerged"): return item["icon"] - else: - return self._icons["repre"] + return self._icons["repre"] active_index = self.Columns.index("active_site") remote_index = self.Columns.index("remote_site") @@ -1144,12 +1135,12 @@ class RepresentationModel(TreeModel, BaseRepresentationModel): # site added, sync in progress progress_str = "not avail." if progress >= 0: - # progress == 0 for isMerged is unavailable if progress == 0 and item.get("isMerged"): progress_str = "not avail." else: - progress_str = "{}% {}".format(int(progress * 100), - label) + progress_str = "{}% {}".format( + int(progress * 100), label + ) return progress_str @@ -1179,7 +1170,7 @@ class RepresentationModel(TreeModel, BaseRepresentationModel): return super(RepresentationModel, self).data(index, role) - def on_doc_fetched(self): + def _on_doc_fetched(self): self.clear() self.beginResetModel() subsets = set() @@ -1189,10 +1180,9 @@ class RepresentationModel(TreeModel, BaseRepresentationModel): group = None self._items_by_id = {} for doc in self._docs: - if len(self.version_ids) > 1: + if len(self._version_ids) > 1: group = repre_groups.get(doc["name"]) if not group: - group_item = Item() item_id = str(uuid4()) group_item.update({ @@ -1213,9 +1203,9 @@ class RepresentationModel(TreeModel, BaseRepresentationModel): repre_groups_items[doc["name"]] = 0 group = group_item - progress = lib.get_progress_for_repre(doc, - self.active_site, - self.remote_site) + progress = lib.get_progress_for_repre( + doc, self.active_site, self.remote_site + ) active_site_icon = self._icons.get(self.active_provider) remote_site_icon = self._icons.get(self.remote_provider) @@ -1248,9 +1238,9 @@ class RepresentationModel(TreeModel, BaseRepresentationModel): 'remote_site_progress': progress[self.remote_site] } if group: - group = self._sum_group_progress(doc["name"], group, - current_progress, - repre_groups_items) + group = self._sum_group_progress( + doc["name"], group, current_progress, repre_groups_items + ) self.add_child(item, group) @@ -1269,47 +1259,39 @@ class RepresentationModel(TreeModel, BaseRepresentationModel): return self._items_by_id.get(item_id) def refresh(self): - docs = [] - session_project = self.dbcon.Session['AVALON_PROJECT'] - if not session_project: + project_name = self.dbcon.current_project() + if not project_name: return - if self.version_ids: + repre_docs = [] + if self._version_ids: # Simple find here for now, expected to receive lower number of # representations and logic could be in Python - docs = list(self.dbcon.find( - {"type": "representation", "parent": {"$in": self.version_ids}, - "files.sites.name": {"$exists": 1}}, self.projection())) - self._docs = docs + repre_docs = list(get_representations( + project_name, + version_ids=self._version_ids, + fields=self.repre_projection.keys() + )) + + self._docs = repre_docs self.doc_fetched.emit() - @classmethod - def projection(cls): - return { - "_id": 1, - "name": 1, - "context.subset": 1, - "context.asset": 1, - "context.version": 1, - "context.representation": 1, - 'files.sites': 1 - } + def _sum_group_progress( + self, repre_name, group, current_item_progress, repre_groups_items + ): + """Update final group progress - def _sum_group_progress(self, repre_name, group, current_item_progress, - repre_groups_items): - """ - Update final group progress - Called after every item in group is added + Called after every item in group is added - Args: - repre_name(string) - group(dict): info about group of selected items - current_item_progress(dict): {'active_site_progress': XX, - 'remote_site_progress': YY} - repre_groups_items(dict) - Returns: - (dict): updated group info + Args: + repre_name(string) + group(dict): info about group of selected items + current_item_progress(dict): {'active_site_progress': XX, + 'remote_site_progress': YY} + repre_groups_items(dict) + Returns: + (dict): updated group info """ repre_groups_items[repre_name] += 1 diff --git a/openpype/tools/loader/widgets.py b/openpype/tools/loader/widgets.py index 42fb62b632..1f6d8b9fa2 100644 --- a/openpype/tools/loader/widgets.py +++ b/openpype/tools/loader/widgets.py @@ -7,6 +7,16 @@ import collections from Qt import QtWidgets, QtCore, QtGui +from openpype.client import ( + get_subset_families, + get_subset_by_id, + get_subsets, + get_version_by_id, + get_versions, + get_representations, + get_thumbnail_id_from_source, + get_thumbnail, +) from openpype.api import Anatomy from openpype.pipeline import HeroVersionType from openpype.pipeline.thumbnail import get_thumbnail_binary @@ -237,8 +247,7 @@ class SubsetWidget(QtWidgets.QWidget): self.model = model self.view = view - actual_project = dbcon.Session["AVALON_PROJECT"] - self.on_project_change(actual_project) + self.on_project_change(dbcon.current_project()) view.customContextMenuRequested.connect(self.on_context_menu) @@ -302,33 +311,23 @@ class SubsetWidget(QtWidgets.QWidget): item["version_document"] ) - subset_docs = list(self.dbcon.find( - { - "_id": {"$in": list(version_docs_by_subset_id.keys())}, - "type": "subset" - }, - { - "schema": 1, - "data.families": 1 - } + project_name = self.dbcon.active_project() + subset_docs = list(get_subsets( + project_name, + subset_ids=version_docs_by_subset_id.keys(), + fields=["schema", "data.families"] )) subset_docs_by_id = { subset_doc["_id"]: subset_doc for subset_doc in subset_docs } version_ids = list(version_docs_by_id.keys()) - repre_docs = self.dbcon.find( - # Query all representations for selected versions at once - { - "type": "representation", - "parent": {"$in": version_ids} - }, - # Query only name and parent from representation - { - "name": 1, - "parent": 1 - } + repre_docs = get_representations( + project_name, + version_ids=version_ids, + fields=["name", "parent"] ) + repre_docs_by_version_id = { version_id: [] for version_id in version_ids @@ -356,9 +355,10 @@ class SubsetWidget(QtWidgets.QWidget): enabled = False if project_name: self.model.reset_sync_server(project_name) - if self.model.sync_server: - enabled_proj = self.model.sync_server.get_enabled_projects() - enabled = project_name in enabled_proj + sync_server = self.model.sync_server + if sync_server: + enabled = sync_server.is_project_enabled(project_name, + single=True) lib.change_visibility(self.model, self.view, "repre_info", enabled) @@ -566,28 +566,42 @@ class SubsetWidget(QtWidgets.QWidget): # same representation available # Trigger - repre_ids = [] + project_name = self.dbcon.active_project() + subset_names_by_version_id = collections.defaultdict(set) for item in items: - representation = self.dbcon.find_one( - { - "type": "representation", - "name": representation_name, - "parent": item["version_document"]["_id"] - }, - {"_id": 1} - ) - if not representation: - self.echo("Subset '{}' has no representation '{}'".format( - item["subset"], representation_name - )) - continue - repre_ids.append(representation["_id"]) + version_id = item["version_document"]["_id"] + subset_names_by_version_id[version_id].add(item["subset"]) + + version_ids = set(subset_names_by_version_id.keys()) + repre_docs = get_representations( + project_name, + representation_names=[representation_name], + version_ids=version_ids, + fields=["_id", "parent"] + ) + + repre_ids = [] + for repre_doc in repre_docs: + repre_ids.append(repre_doc["_id"]) + + version_id = repre_doc["parent"] + if version_id not in version_ids: + version_ids.remove(version_id) + + for version_id in version_ids: + joined_subset_names = ", ".join([ + '"{}"'.format(subset) + for subset in subset_names_by_version_id[version_id] + ]) + self.echo("Subsets {} don't have representation '{}'".format( + joined_subset_names, representation_name + )) # get contexts only for selected menu option repre_contexts = get_repres_contexts(repre_ids, self.dbcon) - options = lib.get_options(action, loader, self, - list(repre_contexts.values())) - + options = lib.get_options( + action, loader, self, list(repre_contexts.values()) + ) error_info = _load_representations_by_loader( loader, repre_contexts, options=options ) @@ -661,27 +675,21 @@ class VersionTextEdit(QtWidgets.QTextEdit): print("Querying..") + project_name = self.dbcon.active_project() if not version_doc: - version_doc = self.dbcon.find_one({ - "_id": version_id, - "type": {"$in": ["version", "hero_version"]} - }) + version_doc = get_version_by_id(project_name, version_id) assert version_doc, "Not a valid version id" if version_doc["type"] == "hero_version": - _version_doc = self.dbcon.find_one({ - "_id": version_doc["version_id"], - "type": "version" - }) + _version_doc = get_version_by_id( + project_name, version_doc["version_id"] + ) version_doc["data"] = _version_doc["data"] version_doc["name"] = HeroVersionType( _version_doc["name"] ) - subset = self.dbcon.find_one({ - "_id": version_doc["parent"], - "type": "subset" - }) + subset = get_subset_by_id(project_name, version_doc["parent"]) assert subset, "No valid subset parent for version" # Define readable creation timestamp @@ -752,7 +760,7 @@ class VersionTextEdit(QtWidgets.QTextEdit): if not source: return - project_name = self.dbcon.Session["AVALON_PROJECT"] + project_name = self.dbcon.current_project() if self._anatomy is None or self._anatomy.project_name != project_name: self._anatomy = Anatomy(project_name) @@ -833,24 +841,19 @@ class ThumbnailWidget(QtWidgets.QLabel): QtCore.Qt.SmoothTransformation ) - def set_thumbnail(self, doc_id=None): - if not doc_id: + def set_thumbnail(self, src_type, doc_ids): + if not doc_ids: self.set_pixmap() return - if isinstance(doc_id, (list, tuple)): - if len(doc_id) < 1: - self.set_pixmap() - return - doc_id = doc_id[0] + src_id = doc_ids[0] - doc = self.dbcon.find_one( - {"_id": doc_id}, - {"data.thumbnail_id"} + project_name = self.dbcon.active_project() + thumbnail_id = get_thumbnail_id_from_source( + project_name, + src_type, + src_id, ) - thumbnail_id = None - if doc: - thumbnail_id = doc.get("data", {}).get("thumbnail_id") if thumbnail_id == self.current_thumb_id: if self.current_thumbnail is None: self.set_pixmap() @@ -861,9 +864,7 @@ class ThumbnailWidget(QtWidgets.QLabel): self.set_pixmap() return - thumbnail_ent = self.dbcon.find_one( - {"type": "thumbnail", "_id": thumbnail_id} - ) + thumbnail_ent = get_thumbnail(project_name, thumbnail_id) if not thumbnail_ent: return @@ -917,21 +918,9 @@ class FamilyModel(QtGui.QStandardItemModel): def refresh(self): families = set() - if self.dbcon.Session.get("AVALON_PROJECT"): - result = list(self.dbcon.aggregate([ - {"$match": { - "type": "subset" - }}, - {"$project": { - "family": {"$arrayElemAt": ["$data.families", 0]} - }}, - {"$group": { - "_id": "family_group", - "families": {"$addToSet": "$family"} - }} - ])) - if result: - families = set(result[0]["families"]) + project_name = self.dbcon.current_project() + if project_name: + families = get_subset_families(project_name) root_item = self.invisibleRootItem() @@ -1176,7 +1165,7 @@ class RepresentationWidget(QtWidgets.QWidget): headers = [item[0] for item in self.default_widths] - model = RepresentationModel(self.dbcon, headers, []) + model = RepresentationModel(self.dbcon, headers) proxy_model = RepresentationSortProxyModel(self) proxy_model.setSourceModel(model) @@ -1213,8 +1202,8 @@ class RepresentationWidget(QtWidgets.QWidget): self.proxy_model = proxy_model self.sync_server_enabled = False - actual_project = dbcon.Session["AVALON_PROJECT"] - self.on_project_change(actual_project) + + self.on_project_change(dbcon.current_project()) self.model.refresh() @@ -1228,9 +1217,10 @@ class RepresentationWidget(QtWidgets.QWidget): enabled = False if project_name: self.model.reset_sync_server(project_name) - if self.model.sync_server: - enabled_proj = self.model.sync_server.get_enabled_projects() - enabled = project_name in enabled_proj + sync_server = self.model.sync_server + if sync_server: + enabled = sync_server.is_project_enabled(project_name, + single=True) self.sync_server_enabled = enabled lib.change_visibility(self.model, self.tree_view, @@ -1243,23 +1233,18 @@ class RepresentationWidget(QtWidgets.QWidget): for item in items: repre_ids.append(item["_id"]) - repre_docs = list(self.dbcon.find( - { - "type": "representation", - "_id": {"$in": repre_ids} - }, - { - "name": 1, - "parent": 1 - } + project_name = self.dbcon.active_project() + repre_docs = list(get_representations( + project_name, + representation_ids=repre_ids, + fields=["name", "parent"] )) + version_ids = [ repre_doc["parent"] for repre_doc in repre_docs ] - version_docs = self.dbcon.find({ - "_id": {"$in": version_ids} - }) + version_docs = get_versions(project_name, version_ids=version_ids) version_docs_by_id = {} version_docs_by_subset_id = collections.defaultdict(list) @@ -1269,15 +1254,10 @@ class RepresentationWidget(QtWidgets.QWidget): version_docs_by_id[version_id] = version_doc version_docs_by_subset_id[subset_id].append(version_doc) - subset_docs = list(self.dbcon.find( - { - "_id": {"$in": list(version_docs_by_subset_id.keys())}, - "type": "subset" - }, - { - "schema": 1, - "data.families": 1 - } + subset_docs = list(get_subsets( + project_name, + subset_ids=version_docs_by_subset_id.keys(), + fields=["schema", "data.families"] )) subset_docs_by_id = { subset_doc["_id"]: subset_doc @@ -1446,13 +1426,12 @@ class RepresentationWidget(QtWidgets.QWidget): self._process_action(items, menu, point) def _process_action(self, items, menu, point): - """ - Show the context action menu and process selected + """Show the context action menu and process selected - Args: - items(dict): menu items - menu(OptionalMenu) - point(PointIndex) + Args: + items(dict): menu items + menu(OptionalMenu) + point(PointIndex) """ global_point = self.tree_view.mapToGlobal(point) action = menu.exec_(global_point) @@ -1468,21 +1447,23 @@ class RepresentationWidget(QtWidgets.QWidget): data_by_repre_id = {} selected_side = action_representation.get("selected_side") + is_sync_loader = tools_lib.is_sync_loader(loader) for item in items: - if tools_lib.is_sync_loader(loader): - site_name = "{}_site_name".format(selected_side) - data = { - "_id": item.get("_id"), - "site_name": item.get(site_name), - "project_name": self.dbcon.Session["AVALON_PROJECT"] - } + item_id = item.get("_id") + repre_ids.append(item_id) + if not is_sync_loader: + continue - if not data["site_name"]: - continue + site_name = "{}_site_name".format(selected_side) + data_site_name = item.get(site_name) + if not data_site_name: + continue - data_by_repre_id[data["_id"]] = data - - repre_ids.append(item.get("_id")) + data_by_repre_id[item_id] = { + "_id": item_id, + "site_name": data_site_name, + "project_name": self.dbcon.active_project() + } repre_contexts = get_repres_contexts(repre_ids, self.dbcon) options = lib.get_options(action, loader, self, diff --git a/openpype/tools/mayalookassigner/app.py b/openpype/tools/mayalookassigner/app.py index 1b6cad77a8..5665acea42 100644 --- a/openpype/tools/mayalookassigner/app.py +++ b/openpype/tools/mayalookassigner/app.py @@ -4,6 +4,7 @@ import logging from Qt import QtWidgets, QtCore +from openpype.client import get_last_version_by_subset_id from openpype import style from openpype.pipeline import legacy_io from openpype.tools.utils.lib import qt_app_context @@ -211,6 +212,7 @@ class MayaLookAssignerWindow(QtWidgets.QWidget): selection = self.assign_selected.isChecked() asset_nodes = self.asset_outliner.get_nodes(selection=selection) + project_name = legacy_io.active_project() start = time.time() for i, (asset, item) in enumerate(asset_nodes.items()): @@ -222,23 +224,20 @@ class MayaLookAssignerWindow(QtWidgets.QWidget): assign_look = next((subset for subset in item["looks"] if subset["name"] in looks), None) if not assign_look: - self.echo("{} No matching selected " - "look for {}".format(prefix, asset)) + self.echo( + "{} No matching selected look for {}".format(prefix, asset) + ) continue # Get the latest version of this asset's look subset - version = legacy_io.find_one( - { - "type": "version", - "parent": assign_look["_id"] - }, - sort=[("name", -1)] + version = get_last_version_by_subset_id( + project_name, assign_look["_id"], fields=["_id"] ) subset_name = assign_look["name"] - self.echo("{} Assigning {} to {}\t".format(prefix, - subset_name, - asset)) + self.echo("{} Assigning {} to {}\t".format( + prefix, subset_name, asset + )) nodes = item["nodes"] if cmds.pluginInfo('vrayformaya', query=True, loaded=True): diff --git a/openpype/tools/mayalookassigner/commands.py b/openpype/tools/mayalookassigner/commands.py index d41d8ca5a2..2e7a51efde 100644 --- a/openpype/tools/mayalookassigner/commands.py +++ b/openpype/tools/mayalookassigner/commands.py @@ -2,9 +2,9 @@ from collections import defaultdict import logging import os -from bson.objectid import ObjectId import maya.cmds as cmds +from openpype.client import get_asset_by_id from openpype.pipeline import ( legacy_io, remove_container, @@ -159,11 +159,9 @@ def create_items_from_nodes(nodes): log.warning("No id hashes") return asset_view_items + project_name = legacy_io.active_project() for _id, id_nodes in id_hashes.items(): - asset = legacy_io.find_one( - {"_id": ObjectId(_id)}, - projection={"name": True} - ) + asset = get_asset_by_id(project_name, _id, fields=["name"]) # Skip if asset id is not found if not asset: @@ -180,10 +178,12 @@ def create_items_from_nodes(nodes): namespace = get_namespace_from_node(node) namespaces.add(namespace) - asset_view_items.append({"label": asset["name"], - "asset": asset, - "looks": looks, - "namespaces": namespaces}) + asset_view_items.append({ + "label": asset["name"], + "asset": asset, + "looks": looks, + "namespaces": namespaces + }) return asset_view_items diff --git a/openpype/tools/mayalookassigner/vray_proxies.py b/openpype/tools/mayalookassigner/vray_proxies.py index 3523b24bf3..889396e555 100644 --- a/openpype/tools/mayalookassigner/vray_proxies.py +++ b/openpype/tools/mayalookassigner/vray_proxies.py @@ -6,11 +6,14 @@ import logging import json import six -from bson.objectid import ObjectId import alembic.Abc from maya import cmds +from openpype.client import ( + get_representation_by_name, + get_last_version_by_subset_name, +) from openpype.pipeline import ( legacy_io, load_container, @@ -155,13 +158,12 @@ def get_look_relationships(version_id): Returns: dict: Dictionary of relations. - """ - json_representation = legacy_io.find_one({ - "type": "representation", - "parent": version_id, - "name": "json" - }) + + project_name = legacy_io.active_project() + json_representation = get_representation_by_name( + project_name, representation_name="json", version_id=version_id + ) # Load relationships shader_relation = get_representation_path(json_representation) @@ -184,12 +186,12 @@ def load_look(version_id): list of shader nodes. """ + + project_name = legacy_io.active_project() # Get representations of shader file and relationships - look_representation = legacy_io.find_one({ - "type": "representation", - "parent": version_id, - "name": "ma" - }) + look_representation = get_representation_by_name( + project_name, representation_name="ma", version_id=version_id + ) # See if representation is already loaded, if so reuse it. host = registered_host() @@ -220,42 +222,6 @@ def load_look(version_id): return shader_nodes -def get_latest_version(asset_id, subset): - # type: (str, str) -> dict - """Get latest version of subset. - - Args: - asset_id (str): Asset ID - subset (str): Subset name. - - Returns: - Latest version - - Throws: - RuntimeError: When subset or version doesn't exist. - - """ - subset = legacy_io.find_one({ - "name": subset, - "parent": ObjectId(asset_id), - "type": "subset" - }) - if not subset: - raise RuntimeError("Subset does not exist: %s" % subset) - - version = legacy_io.find_one( - { - "type": "version", - "parent": subset["_id"] - }, - sort=[("name", -1)] - ) - if not version: - raise RuntimeError("Version does not exist.") - - return version - - def vrayproxy_assign_look(vrayproxy, subset="lookDefault"): # type: (str, str) -> None """Assign look to vray proxy. @@ -281,13 +247,20 @@ def vrayproxy_assign_look(vrayproxy, subset="lookDefault"): asset_id = node_id.split(":", 1)[0] node_ids_by_asset_id[asset_id].add(node_id) + project_name = legacy_io.active_project() for asset_id, node_ids in node_ids_by_asset_id.items(): # Get latest look version - try: - version = get_latest_version(asset_id, subset=subset) - except RuntimeError as exc: - print(exc) + version = get_last_version_by_subset_name( + project_name, + subset_name=subset, + asset_id=asset_id, + fields=["_id"] + ) + if not version: + print("Didn't find last version for subset name {}".format( + subset + )) continue relationships = get_look_relationships(version["_id"]) diff --git a/openpype/tools/project_manager/project_manager/model.py b/openpype/tools/project_manager/project_manager/model.py index 223cfa629d..c5bde5aaec 100644 --- a/openpype/tools/project_manager/project_manager/model.py +++ b/openpype/tools/project_manager/project_manager/model.py @@ -7,6 +7,11 @@ from pymongo import UpdateOne, DeleteOne from Qt import QtCore, QtGui +from openpype.client import ( + get_project, + get_assets, + get_asset_ids_with_subsets, +) from openpype.lib import ( CURRENT_DOC_SCHEMAS, PypeLogger, @@ -255,10 +260,11 @@ class HierarchyModel(QtCore.QAbstractItemModel): return # Find project'd document - project_doc = self.dbcon.database[project_name].find_one( - {"type": "project"}, - ProjectItem.query_projection + project_doc = get_project( + project_name, + fields=list(ProjectItem.query_projection.keys()) ) + # Skip if project document does not exist # - this shouldn't happen using only UI elements if not project_doc: @@ -269,9 +275,8 @@ class HierarchyModel(QtCore.QAbstractItemModel): self.add_item(project_item) # Query all assets of the project - asset_docs = self.dbcon.database[project_name].find( - {"type": "asset"}, - AssetItem.query_projection + asset_docs = get_assets( + project_name, fields=AssetItem.query_projection.keys() ) asset_docs_by_id = { asset_doc["_id"]: asset_doc @@ -282,31 +287,16 @@ class HierarchyModel(QtCore.QAbstractItemModel): # if asset item can be modified (name and hierarchy change) # - the same must be applied to all it's parents asset_ids = list(asset_docs_by_id.keys()) - result = [] + asset_ids_with_subsets = [] if asset_ids: - result = self.dbcon.database[project_name].aggregate([ - { - "$match": { - "type": "subset", - "parent": {"$in": asset_ids} - } - }, - { - "$group": { - "_id": "$parent", - "count": {"$sum": 1} - } - } - ]) + asset_ids_with_subsets = get_asset_ids_with_subsets( + project_name, asset_ids=asset_ids + ) asset_modifiable = { - asset_id: True + asset_id: asset_id not in asset_ids_with_subsets for asset_id in asset_docs_by_id.keys() } - for item in result: - asset_id = item["_id"] - count = item["count"] - asset_modifiable[asset_id] = count < 1 # Store assets by their visual parent to be able create their hierarchy asset_docs_by_parent_id = collections.defaultdict(list) diff --git a/openpype/tools/project_manager/project_manager/view.py b/openpype/tools/project_manager/project_manager/view.py index a16def276d..cca892ef72 100644 --- a/openpype/tools/project_manager/project_manager/view.py +++ b/openpype/tools/project_manager/project_manager/view.py @@ -3,6 +3,7 @@ from queue import Queue from Qt import QtWidgets, QtCore, QtGui +from openpype.client import get_project from .delegates import ( NumberDelegate, NameDelegate, @@ -47,12 +48,8 @@ class ProjectDocCache: def set_project(self, project_name): self.project_doc = None - if not project_name: - return - - self.project_doc = self.dbcon.database[project_name].find_one( - {"type": "project"} - ) + if project_name: + self.project_doc = get_project(project_name) class ToolsCache: diff --git a/openpype/tools/project_manager/project_manager/widgets.py b/openpype/tools/project_manager/project_manager/widgets.py index dc75b30bd7..371d1ba2ef 100644 --- a/openpype/tools/project_manager/project_manager/widgets.py +++ b/openpype/tools/project_manager/project_manager/widgets.py @@ -1,5 +1,6 @@ import re +from openpype.client import get_projects from .constants import ( NAME_ALLOWED_SYMBOLS, NAME_REGEX @@ -272,15 +273,9 @@ class CreateProjectDialog(QtWidgets.QDialog): def _get_existing_projects(self): project_names = set() project_codes = set() - for project_name in self.dbcon.database.collection_names(): - # Each collection will have exactly one project document - project_doc = self.dbcon.database[project_name].find_one( - {"type": "project"}, - {"name": 1, "data.code": 1} - ) - if not project_doc: - continue - + for project_doc in get_projects( + inactive=True, fields=["name", "data.code"] + ): project_name = project_doc.get("name") if not project_name: continue diff --git a/openpype/tools/publisher/control.py b/openpype/tools/publisher/control.py index 2973d6a5bb..915fb7f32e 100644 --- a/openpype/tools/publisher/control.py +++ b/openpype/tools/publisher/control.py @@ -13,6 +13,7 @@ except Exception: import pyblish.api +from openpype.client import get_assets from openpype.pipeline import ( PublishValidationError, registered_host, @@ -116,10 +117,10 @@ class AssetDocsCache: def _query(self): if self._asset_docs is None: - asset_docs = list(self.dbcon.find( - {"type": "asset"}, - self.projection - )) + project_name = self.dbcon.active_project() + asset_docs = get_assets( + project_name, fields=self.projection.keys() + ) task_names_by_asset_name = {} for asset_doc in asset_docs: asset_name = asset_doc["name"] diff --git a/openpype/tools/publisher/widgets/create_dialog.py b/openpype/tools/publisher/widgets/create_dialog.py index 9e357f3a56..53bbef8b75 100644 --- a/openpype/tools/publisher/widgets/create_dialog.py +++ b/openpype/tools/publisher/widgets/create_dialog.py @@ -9,6 +9,8 @@ try: except Exception: commonmark = None from Qt import QtWidgets, QtCore, QtGui + +from openpype.client import get_asset_by_name, get_subsets from openpype.lib import TaskNotSetError from openpype.pipeline.create import ( CreatorError, @@ -647,21 +649,19 @@ class CreateDialog(QtWidgets.QDialog): if asset_name is None: return - asset_doc = self.dbcon.find_one({ - "type": "asset", - "name": asset_name - }) + project_name = self.dbcon.active_project() + asset_doc = get_asset_by_name(project_name, asset_name) self._asset_doc = asset_doc if asset_doc: - subset_docs = self.dbcon.find( - { - "type": "subset", - "parent": asset_doc["_id"] - }, - {"name": 1} + asset_id = asset_doc["_id"] + subset_docs = get_subsets( + project_name, asset_ids=[asset_id], fields=["name"] ) - self._subset_names = set(subset_docs.distinct("name")) + self._subset_names = { + subset_doc["name"] + for subset_doc in subset_docs + } if not asset_doc: self.subset_name_input.setText("< Asset is not set >") diff --git a/openpype/tools/pyblish_pype/window.py b/openpype/tools/pyblish_pype/window.py index d27ec34345..78590259bc 100644 --- a/openpype/tools/pyblish_pype/window.py +++ b/openpype/tools/pyblish_pype/window.py @@ -468,10 +468,8 @@ class Window(QtWidgets.QDialog): current_page == "terminal" ) - self.state = { - "is_closing": False, - "current_page": current_page - } + self._current_page = current_page + self._hidden_for_plugin_process = False self.tabs[current_page].setChecked(True) @@ -590,14 +588,14 @@ class Window(QtWidgets.QDialog): target_page = page if direction is None: direction = -1 - elif name == self.state["current_page"]: + elif name == self._current_page: previous_page = page if direction is None: direction = 1 else: page.setVisible(False) - self.state["current_page"] = target + self._current_page = target self.slide_page(previous_page, target_page, direction) def slide_page(self, previous_page, target_page, direction): @@ -684,7 +682,7 @@ class Window(QtWidgets.QDialog): comment_visible=None, terminal_filters_visibile=None ): - target = self.state["current_page"] + target = self._current_page comment_visibility = ( not self.perspective_widget.isVisible() and not target == "terminal" @@ -845,7 +843,7 @@ class Window(QtWidgets.QDialog): def apply_log_suspend_value(self, value): self._suspend_logs = value - if self.state["current_page"] == "terminal": + if self._current_page == "terminal": self.tabs["overview"].setChecked(True) self.tabs["terminal"].setVisible(not self._suspend_logs) @@ -882,9 +880,21 @@ class Window(QtWidgets.QDialog): visibility = True if hasattr(plugin, "hide_ui_on_process") and plugin.hide_ui_on_process: visibility = False + self._hidden_for_plugin_process = not visibility - if self.isVisible() != visibility: - self.setVisible(visibility) + self._ensure_visible(visibility) + + def _ensure_visible(self, visible): + if self.isVisible() == visible: + return + + if not visible: + self.setVisible(visible) + else: + self.show() + self.raise_() + self.activateWindow() + self.showNormal() def on_plugin_action_menu_requested(self, pos): """The user right-clicked on a plug-in @@ -955,7 +965,7 @@ class Window(QtWidgets.QDialog): self.intent_box.setEnabled(True) # Refresh tab - self.on_tab_changed(self.state["current_page"]) + self.on_tab_changed(self._current_page) self.update_compatibility() self.button_suspend_logs.setEnabled(False) @@ -1027,8 +1037,9 @@ class Window(QtWidgets.QDialog): self._update_state() - if not self.isVisible(): - self.setVisible(True) + if self._hidden_for_plugin_process: + self._hidden_for_plugin_process = False + self._ensure_visible(True) def on_was_skipped(self, plugin): plugin_item = self.plugin_model.plugin_items[plugin.id] @@ -1103,8 +1114,9 @@ class Window(QtWidgets.QDialog): plugin_item, instance_item ) - if not self.isVisible(): - self.setVisible(True) + if self._hidden_for_plugin_process: + self._hidden_for_plugin_process = False + self._ensure_visible(True) # ------------------------------------------------------------------------- # @@ -1223,53 +1235,20 @@ class Window(QtWidgets.QDialog): """ - # Make it snappy, but take care to clean it all up. - # TODO(marcus): Enable GUI to return on problem, such - # as asking whether or not the user really wants to quit - # given there are things currently running. - self.hide() + self.info(self.tr("Closing..")) - if self.state["is_closing"]: + if self.controller.is_running: + self.info(self.tr("..as soon as processing is finished..")) + self.controller.stop() - # Explicitly clear potentially referenced data - self.info(self.tr("Cleaning up models..")) - self.intent_model.deleteLater() - self.plugin_model.deleteLater() - self.terminal_model.deleteLater() - self.terminal_proxy.deleteLater() - self.plugin_proxy.deleteLater() + self.info(self.tr("Cleaning up controller..")) + self.controller.cleanup() self.overview_instance_view.setModel(None) self.overview_plugin_view.setModel(None) self.terminal_view.setModel(None) - self.info(self.tr("Cleaning up controller..")) - self.controller.cleanup() - - self.info(self.tr("All clean!")) - self.info(self.tr("Good bye")) - return super(Window, self).closeEvent(event) - - self.info(self.tr("Closing..")) - - def on_problem(): - self.heads_up( - "Warning", "Had trouble closing down. " - "Please tell someone and try again." - ) - self.show() - - if self.controller.is_running: - self.info(self.tr("..as soon as processing is finished..")) - self.controller.stop() - self.finished.connect(self.close) - util.defer(200, on_problem) - return event.ignore() - - self.state["is_closing"] = True - - util.defer(200, self.close) - return event.ignore() + event.accept() def reject(self): """Handle ESC key""" diff --git a/openpype/tools/sceneinventory/model.py b/openpype/tools/sceneinventory/model.py index 8d72020c98..0bb9c4a658 100644 --- a/openpype/tools/sceneinventory/model.py +++ b/openpype/tools/sceneinventory/model.py @@ -5,8 +5,14 @@ from collections import defaultdict from Qt import QtCore, QtGui import qtawesome -from bson.objectid import ObjectId +from openpype.client import ( + get_asset_by_id, + get_subset_by_id, + get_version_by_id, + get_last_version_by_subset_id, + get_representation_by_id, +) from openpype.pipeline import ( legacy_io, schema, @@ -55,7 +61,7 @@ class InventoryModel(TreeModel): if not self.sync_enabled: return - project_name = legacy_io.Session["AVALON_PROJECT"] + project_name = legacy_io.current_project() active_site = sync_server.get_active_site(project_name) remote_site = sync_server.get_remote_site(project_name) @@ -192,12 +198,12 @@ class InventoryModel(TreeModel): self.clear() if self._hierarchy_view and selected: - if not hasattr(host.pipeline, "update_hierarchy"): # If host doesn't support hierarchical containers, then # cherry-pick only. self.add_items((item for item in items if item["objectName"] in selected)) + return # Update hierarchy info for all containers items_by_name = {item["objectName"]: item @@ -291,6 +297,9 @@ class InventoryModel(TreeModel): node.Item: root node which has children added based on the data """ + # NOTE: @iLLiCiTiT this need refactor + project_name = legacy_io.active_project() + self.beginResetModel() # Group by representation @@ -304,32 +313,36 @@ class InventoryModel(TreeModel): for repre_id, group_dict in sorted(grouped.items()): group_items = group_dict["items"] # Get parenthood per group - representation = legacy_io.find_one({"_id": ObjectId(repre_id)}) + representation = get_representation_by_id( + project_name, repre_id + ) if not representation: not_found["representation"].append(group_items) not_found_ids.append(repre_id) continue - version = legacy_io.find_one({"_id": representation["parent"]}) + version = get_version_by_id( + project_name, representation["parent"] + ) if not version: not_found["version"].append(group_items) not_found_ids.append(repre_id) continue elif version["type"] == "hero_version": - _version = legacy_io.find_one({ - "_id": version["version_id"] - }) + _version = get_version_by_id( + project_name, version["version_id"] + ) version["name"] = HeroVersionType(_version["name"]) version["data"] = _version["data"] - subset = legacy_io.find_one({"_id": version["parent"]}) + subset = get_subset_by_id(project_name, version["parent"]) if not subset: not_found["subset"].append(group_items) not_found_ids.append(repre_id) continue - asset = legacy_io.find_one({"_id": subset["parent"]}) + asset = get_asset_by_id(project_name, subset["parent"]) if not asset: not_found["asset"].append(group_items) not_found_ids.append(repre_id) @@ -390,10 +403,9 @@ class InventoryModel(TreeModel): # Store the highest available version so the model can know # whether current version is currently up-to-date. - highest_version = legacy_io.find_one({ - "type": "version", - "parent": version["parent"] - }, sort=[("name", -1)]) + highest_version = get_last_version_by_subset_id( + project_name, version["parent"] + ) # create the group header group_node = Item() diff --git a/openpype/tools/sceneinventory/switch_dialog.py b/openpype/tools/sceneinventory/switch_dialog.py index b2d770330f..1d1d5cbb91 100644 --- a/openpype/tools/sceneinventory/switch_dialog.py +++ b/openpype/tools/sceneinventory/switch_dialog.py @@ -4,6 +4,16 @@ from Qt import QtWidgets, QtCore import qtawesome from bson.objectid import ObjectId +from openpype.client import ( + get_asset_by_name, + get_assets, + get_subset_by_name, + get_subsets, + get_versions, + get_hero_versions, + get_last_versions, + get_representations, +) from openpype.pipeline import legacy_io from openpype.pipeline.load import ( discover_loader_plugins, @@ -144,6 +154,9 @@ class SwitchAssetDialog(QtWidgets.QDialog): self._prepare_content_data() self.refresh(True) + def active_project(self): + return legacy_io.active_project() + def _prepare_content_data(self): repre_ids = set() content_loaders = set() @@ -151,10 +164,12 @@ class SwitchAssetDialog(QtWidgets.QDialog): repre_ids.add(ObjectId(item["representation"])) content_loaders.add(item["loader"]) - repres = list(legacy_io.find({ - "type": {"$in": ["representation", "archived_representation"]}, - "_id": {"$in": list(repre_ids)} - })) + project_name = self.active_project() + repres = list(get_representations( + project_name, + representation_ids=repre_ids, + archived=True + )) repres_by_id = {repre["_id"]: repre for repre in repres} # stash context values, works only for single representation @@ -179,10 +194,11 @@ class SwitchAssetDialog(QtWidgets.QDialog): content_repres[repre_id] = repres_by_id[repre_id] version_ids.append(repre["parent"]) - versions = legacy_io.find({ - "type": {"$in": ["version", "hero_version"]}, - "_id": {"$in": list(set(version_ids))} - }) + versions = get_versions( + project_name, + version_ids=set(version_ids), + hero=True + ) content_versions = {} hero_version_ids = set() for version in versions: @@ -198,10 +214,9 @@ class SwitchAssetDialog(QtWidgets.QDialog): else: subset_ids.append(content_versions[version_id]["parent"]) - subsets = legacy_io.find({ - "type": {"$in": ["subset", "archived_subset"]}, - "_id": {"$in": subset_ids} - }) + subsets = get_subsets( + project_name, subset_ids=subset_ids, archived=True + ) subsets_by_id = {sub["_id"]: sub for sub in subsets} asset_ids = [] @@ -220,10 +235,7 @@ class SwitchAssetDialog(QtWidgets.QDialog): asset_ids.append(subset["parent"]) content_subsets[subset_id] = subset - assets = legacy_io.find({ - "type": {"$in": ["asset", "archived_asset"]}, - "_id": {"$in": list(asset_ids)} - }) + assets = get_assets(project_name, asset_ids=asset_ids, archived=True) assets_by_id = {asset["_id"]: asset for asset in assets} missing_assets = [] @@ -472,9 +484,10 @@ class SwitchAssetDialog(QtWidgets.QDialog): # Prepare asset document if asset is selected asset_doc = None if selected_asset: - asset_doc = legacy_io.find_one( - {"type": "asset", "name": selected_asset}, - {"_id": True} + asset_doc = get_asset_by_name( + self.active_project(), + selected_asset, + fields=["_id"] ) if not asset_doc: return [] @@ -523,38 +536,35 @@ class SwitchAssetDialog(QtWidgets.QDialog): def _get_current_output_repre_ids_xxx( self, asset_doc, selected_subset, selected_repre ): - subset_doc = legacy_io.find_one( - { - "type": "subset", - "name": selected_subset, - "parent": asset_doc["_id"] - }, - {"_id": True} + project_name = self.active_project() + subset_doc = get_subset_by_name( + project_name, + selected_subset, + asset_doc["_id"], + fields=["_id"] ) + subset_id = subset_doc["_id"] last_versions_by_subset_id = self.find_last_versions([subset_id]) version_doc = last_versions_by_subset_id.get(subset_id) if not version_doc: return [] - repre_docs = legacy_io.find( - { - "type": "representation", - "parent": version_doc["_id"], - "name": selected_repre - }, - {"_id": True} + repre_docs = get_representations( + project_name, + version_ids=[version_doc["_id"]], + representation_names=[selected_repre], + fields=["_id"] ) return [repre_doc["_id"] for repre_doc in repre_docs] def _get_current_output_repre_ids_xxo(self, asset_doc, selected_subset): - subset_doc = legacy_io.find_one( - { - "type": "subset", - "parent": asset_doc["_id"], - "name": selected_subset - }, - {"_id": True} + project_name = self.active_project() + subset_doc = get_subset_by_name( + project_name, + selected_subset, + asset_doc["_id"], + fields=["_id"] ) if not subset_doc: return [] @@ -563,41 +573,51 @@ class SwitchAssetDialog(QtWidgets.QDialog): for repre_doc in self.content_repres.values(): repre_names.add(repre_doc["name"]) - repre_docs = legacy_io.find( - { - "type": "representation", - "parent": subset_doc["_id"], - "name": {"$in": list(repre_names)} - }, - {"_id": True} + # TODO where to take version ids? + version_ids = [] + repre_docs = get_representations( + project_name, + representation_names=repre_names, + version_ids=version_ids, + fields=["_id"] ) return [repre_doc["_id"] for repre_doc in repre_docs] def _get_current_output_repre_ids_xox(self, asset_doc, selected_repre): - susbet_names = set() + subset_names = set() for subset_doc in self.content_subsets.values(): - susbet_names.add(subset_doc["name"]) + subset_names.add(subset_doc["name"]) - subset_docs = legacy_io.find( - { - "type": "subset", - "name": {"$in": list(susbet_names)}, - "parent": asset_doc["_id"] - }, - {"_id": True} + project_name = self.active_project() + subset_docs = get_subsets( + project_name, + asset_ids=[asset_doc["_id"]], + subset_names=subset_names, + fields=["_id", "name"] ) - subset_ids = [subset_doc["_id"] for subset_doc in subset_docs] - repre_docs = legacy_io.find( - { - "type": "representation", - "parent": {"$in": subset_ids}, - "name": selected_repre - }, - {"_id": True} + subset_name_by_id = { + subset_doc["_id"]: subset_doc["name"] + for subset_doc in subset_docs + } + subset_ids = list(subset_name_by_id.keys()) + last_versions_by_subset_id = self.find_last_versions(subset_ids) + last_version_id_by_subset_name = {} + for subset_id, last_version in last_versions_by_subset_id.items(): + subset_name = subset_name_by_id[subset_id] + last_version_id_by_subset_name[subset_name] = ( + last_version["_id"] + ) + + repre_docs = get_representations( + project_name, + version_ids=last_version_id_by_subset_name.values(), + representation_names=[selected_repre], + fields=["_id"] ) return [repre_doc["_id"] for repre_doc in repre_docs] def _get_current_output_repre_ids_xoo(self, asset_doc): + project_name = self.active_project() repres_by_subset_name = collections.defaultdict(set) for repre_doc in self.content_repres.values(): repre_name = repre_doc["name"] @@ -606,13 +626,11 @@ class SwitchAssetDialog(QtWidgets.QDialog): subset_name = subset_doc["name"] repres_by_subset_name[subset_name].add(repre_name) - subset_docs = list(legacy_io.find( - { - "type": "subset", - "parent": asset_doc["_id"], - "name": {"$in": list(repres_by_subset_name.keys())} - }, - {"_id": True, "name": True} + subset_docs = list(get_subsets( + project_name, + asset_ids=[asset_doc["_id"]], + subset_names=repres_by_subset_name.keys(), + fields=["_id", "name"] )) subset_name_by_id = { subset_doc["_id"]: subset_doc["name"] @@ -627,60 +645,59 @@ class SwitchAssetDialog(QtWidgets.QDialog): last_version["_id"] ) - repre_or_query = [] + repre_names_by_version_id = {} for subset_name, repre_names in repres_by_subset_name.items(): version_id = last_version_id_by_subset_name.get(subset_name) # This should not happen but why to crash? - if version_id is None: - continue - repre_or_query.append({ - "parent": version_id, - "name": {"$in": list(repre_names)} - }) - repre_docs = legacy_io.find( - {"$or": repre_or_query}, - {"_id": True} + if version_id is not None: + repre_names_by_version_id[version_id] = list(repre_names) + + repre_docs = get_representations( + project_name, + names_by_version_ids=repre_names_by_version_id, + fields=["_id"] ) return [repre_doc["_id"] for repre_doc in repre_docs] def _get_current_output_repre_ids_oxx( self, selected_subset, selected_repre ): - subset_docs = list(legacy_io.find({ - "type": "subset", - "parent": {"$in": list(self.content_assets.keys())}, - "name": selected_subset - })) + project_name = self.active_project() + subset_docs = get_subsets( + project_name, + asset_ids=self.content_assets.keys(), + subset_names=[selected_subset], + fields=["_id"] + ) subset_ids = [subset_doc["_id"] for subset_doc in subset_docs] last_versions_by_subset_id = self.find_last_versions(subset_ids) last_version_ids = [ last_version["_id"] for last_version in last_versions_by_subset_id.values() ] - repre_docs = legacy_io.find({ - "type": "representation", - "parent": {"$in": last_version_ids}, - "name": selected_repre - }) - + repre_docs = get_representations( + project_name, + version_ids=last_version_ids, + representation_names=[selected_repre], + fields=["_id"] + ) return [repre_doc["_id"] for repre_doc in repre_docs] def _get_current_output_repre_ids_oxo(self, selected_subset): - subset_docs = list(legacy_io.find( - { - "type": "subset", - "parent": {"$in": list(self.content_assets.keys())}, - "name": selected_subset - }, - {"_id": True, "parent": True} - )) - if not subset_docs: - return list() - + project_name = self.active_project() + subset_docs = get_subsets( + project_name, + asset_ids=self.content_assets.keys(), + subset_names=[selected_subset], + fields=["_id", "parent"] + ) subset_docs_by_id = { subset_doc["_id"]: subset_doc for subset_doc in subset_docs } + if not subset_docs: + return list() + last_versions_by_subset_id = self.find_last_versions( subset_docs_by_id.keys() ) @@ -702,56 +719,44 @@ class SwitchAssetDialog(QtWidgets.QDialog): asset_id = asset_doc["_id"] repre_names_by_asset_id[asset_id].add(repre_name) - repre_or_query = [] + repre_names_by_version_id = {} for last_version_id, subset_id in subset_id_by_version_id.items(): subset_doc = subset_docs_by_id[subset_id] asset_id = subset_doc["parent"] repre_names = repre_names_by_asset_id.get(asset_id) if not repre_names: continue - repre_or_query.append({ - "parent": last_version_id, - "name": {"$in": list(repre_names)} - }) - repre_docs = legacy_io.find( - { - "type": "representation", - "$or": repre_or_query - }, - {"_id": True} - ) + repre_names_by_version_id[last_version_id] = repre_names + repre_docs = get_representations( + project_name, + names_by_version_ids=repre_names_by_version_id, + fields=["_id"] + ) return [repre_doc["_id"] for repre_doc in repre_docs] def _get_current_output_repre_ids_oox(self, selected_repre): - repre_docs = legacy_io.find( - { - "name": selected_repre, - "parent": {"$in": list(self.content_versions.keys())} - }, - {"_id": True} + project_name = self.active_project() + repre_docs = get_representations( + project_name, + representation_names=[selected_repre], + version_ids=self.content_versions.keys(), + fields=["_id"] ) return [repre_doc["_id"] for repre_doc in repre_docs] def _get_asset_box_values(self): - asset_docs = legacy_io.find( - {"type": "asset"}, - {"_id": 1, "name": 1} - ) + project_name = self.active_project() + asset_docs = get_assets(project_name, fields=["_id", "name"]) asset_names_by_id = { asset_doc["_id"]: asset_doc["name"] for asset_doc in asset_docs } - subsets = legacy_io.find( - { - "type": "subset", - "parent": {"$in": list(asset_names_by_id.keys())} - }, - { - "parent": 1 - } + subsets = get_subsets( + project_name, + asset_ids=asset_names_by_id.keys(), + fields=["parent"] ) - filtered_assets = [] for subset in subsets: asset_name = asset_names_by_id[subset["parent"]] @@ -760,25 +765,20 @@ class SwitchAssetDialog(QtWidgets.QDialog): return sorted(filtered_assets) def _get_subset_box_values(self): + project_name = self.active_project() selected_asset = self._assets_box.get_valid_value() if selected_asset: - asset_doc = legacy_io.find_one({ - "type": "asset", - "name": selected_asset - }) + asset_doc = get_asset_by_name( + project_name, selected_asset, fields=["_id"] + ) asset_ids = [asset_doc["_id"]] else: asset_ids = list(self.content_assets.keys()) - subsets = legacy_io.find( - { - "type": "subset", - "parent": {"$in": asset_ids} - }, - { - "parent": 1, - "name": 1 - } + subsets = get_subsets( + project_name, + asset_ids=asset_ids, + fields=["parent", "name"] ) subset_names_by_parent_id = collections.defaultdict(set) @@ -800,6 +800,7 @@ class SwitchAssetDialog(QtWidgets.QDialog): def _representations_box_values(self): # NOTE hero versions are not used because it is expected that # hero version has same representations as latests + project_name = self.active_project() selected_asset = self._assets_box.currentText() selected_subset = self._subsets_box.currentText() @@ -807,16 +808,11 @@ class SwitchAssetDialog(QtWidgets.QDialog): # [ ] [ ] [?] if not selected_asset and not selected_subset: # Find all representations of selection's subsets - possible_repres = list(legacy_io.find( - { - "type": "representation", - "parent": {"$in": list(self.content_versions.keys())} - }, - { - "parent": 1, - "name": 1 - } - )) + possible_repres = get_representations( + project_name, + version_ids=self.content_versions.keys(), + fields=["parent", "name"] + ) possible_repres_by_parent = collections.defaultdict(set) for repre in possible_repres: @@ -836,29 +832,23 @@ class SwitchAssetDialog(QtWidgets.QDialog): # [x] [x] [?] if selected_asset and selected_subset: - asset_doc = legacy_io.find_one( - {"type": "asset", "name": selected_asset}, - {"_id": 1} + asset_doc = get_asset_by_name( + project_name, selected_asset, fields=["_id"] ) - subset_doc = legacy_io.find_one( - { - "type": "subset", - "name": selected_subset, - "parent": asset_doc["_id"] - }, - {"_id": 1} + subset_doc = get_subset_by_name( + project_name, + selected_subset, + asset_doc["_id"], + fields=["_id"] ) + subset_id = subset_doc["_id"] last_versions_by_subset_id = self.find_last_versions([subset_id]) version_doc = last_versions_by_subset_id.get(subset_id) - repre_docs = legacy_io.find( - { - "type": "representation", - "parent": version_doc["_id"] - }, - { - "name": 1 - } + repre_docs = get_representations( + project_name, + version_ids=[version_doc["_id"]], + fields=["name"] ) return [ repre_doc["name"] @@ -868,9 +858,8 @@ class SwitchAssetDialog(QtWidgets.QDialog): # [x] [ ] [?] # If asset only is selected if selected_asset: - asset_doc = legacy_io.find_one( - {"type": "asset", "name": selected_asset}, - {"_id": 1} + asset_doc = get_asset_by_name( + project_name, selected_asset, fields=["_id"] ) if not asset_doc: return list() @@ -879,13 +868,12 @@ class SwitchAssetDialog(QtWidgets.QDialog): subset_names = set() for subset_doc in self.content_subsets.values(): subset_names.add(subset_doc["name"]) - subset_docs = legacy_io.find( - { - "type": "subset", - "parent": asset_doc["_id"], - "name": {"$in": list(subset_names)} - }, - {"_id": 1} + + subset_docs = get_subsets( + project_name, + asset_ids=[asset_doc["_id"]], + subset_names=subset_names, + fields=["_id"] ) subset_ids = [ subset_doc["_id"] @@ -903,15 +891,10 @@ class SwitchAssetDialog(QtWidgets.QDialog): if not subset_id_by_version_id: return list() - repre_docs = list(legacy_io.find( - { - "type": "representation", - "parent": {"$in": list(subset_id_by_version_id.keys())} - }, - { - "name": 1, - "parent": 1 - } + repre_docs = list(get_representations( + project_name, + version_ids=subset_id_by_version_id.keys(), + fields=["name", "parent"] )) if not repre_docs: return list() @@ -933,13 +916,11 @@ class SwitchAssetDialog(QtWidgets.QDialog): return list(available_repres) # [ ] [x] [?] - subset_docs = list(legacy_io.find( - { - "type": "subset", - "parent": {"$in": list(self.content_assets.keys())}, - "name": selected_subset - }, - {"_id": 1, "parent": 1} + subset_docs = list(get_subsets( + project_name, + asset_ids=self.content_assets.keys(), + subset_names=[selected_subset], + fields=["_id", "parent"] )) if not subset_docs: return list() @@ -960,16 +941,13 @@ class SwitchAssetDialog(QtWidgets.QDialog): if not subset_id_by_version_id: return list() - repre_docs = list(legacy_io.find( - { - "type": "representation", - "parent": {"$in": list(subset_id_by_version_id.keys())} - }, - { - "name": 1, - "parent": 1 - } - )) + repre_docs = list( + get_representations( + project_name, + version_ids=subset_id_by_version_id.keys(), + fields=["name", "parent"] + ) + ) if not repre_docs: return list() @@ -1016,14 +994,14 @@ class SwitchAssetDialog(QtWidgets.QDialog): return # [x] [ ] [?] - asset_doc = legacy_io.find_one( - {"type": "asset", "name": selected_asset}, - {"_id": 1} + project_name = self.active_project() + asset_doc = get_asset_by_name( + project_name, selected_asset, fields=["_id"] ) - subset_docs = legacy_io.find( - {"type": "subset", "parent": asset_doc["_id"]}, - {"name": 1} + subset_docs = get_subsets( + project_name, asset_ids=[asset_doc["_id"]], fields=["name"] ) + subset_names = set( subset_doc["name"] for subset_doc in subset_docs @@ -1035,27 +1013,12 @@ class SwitchAssetDialog(QtWidgets.QDialog): break def find_last_versions(self, subset_ids): - _pipeline = [ - # Find all versions of those subsets - {"$match": { - "type": "version", - "parent": {"$in": list(subset_ids)} - }}, - # Sorting versions all together - {"$sort": {"name": 1}}, - # Group them by "parent", but only take the last - {"$group": { - "_id": "$parent", - "_version_id": {"$last": "$_id"}, - "type": {"$last": "$type"} - }} - ] - last_versions_by_subset_id = dict() - for doc in legacy_io.aggregate(_pipeline): - doc["parent"] = doc["_id"] - doc["_id"] = doc.pop("_version_id") - last_versions_by_subset_id[doc["parent"]] = doc - return last_versions_by_subset_id + project_name = self.active_project() + return get_last_versions( + project_name, + subset_ids=subset_ids, + fields=["_id", "parent", "type"] + ) def _is_repre_ok(self, validation_state): selected_asset = self._assets_box.get_valid_value() @@ -1078,33 +1041,28 @@ class SwitchAssetDialog(QtWidgets.QDialog): return # [x] [x] [ ] + project_name = self.active_project() if selected_asset is not None and selected_subset is not None: - asset_doc = legacy_io.find_one( - {"type": "asset", "name": selected_asset}, - {"_id": 1} + asset_doc = get_asset_by_name( + project_name, selected_asset, fields=["_id"] ) - subset_doc = legacy_io.find_one( - { - "type": "subset", - "parent": asset_doc["_id"], - "name": selected_subset - }, - {"_id": 1} + subset_doc = get_subset_by_name( + project_name, + selected_subset, + asset_doc["_id"], + fields=["_id"] ) - last_versions_by_subset_id = self.find_last_versions( - [subset_doc["_id"]] - ) - last_version = last_versions_by_subset_id.get(subset_doc["_id"]) + subset_id = subset_doc["_id"] + last_versions_by_subset_id = self.find_last_versions([subset_id]) + last_version = last_versions_by_subset_id.get(subset_id) if not last_version: validation_state.repre_ok = False return - repre_docs = legacy_io.find( - { - "type": "representation", - "parent": last_version["_id"] - }, - {"name": 1} + repre_docs = get_representations( + project_name, + version_ids=[last_version["_id"]], + fields=["name"] ) repre_names = set( @@ -1119,16 +1077,13 @@ class SwitchAssetDialog(QtWidgets.QDialog): # [x] [ ] [ ] if selected_asset is not None: - asset_doc = legacy_io.find_one( - {"type": "asset", "name": selected_asset}, - {"_id": 1} + asset_doc = get_asset_by_name( + project_name, selected_asset, fields=["_id"] ) - subset_docs = list(legacy_io.find( - { - "type": "subset", - "parent": asset_doc["_id"] - }, - {"_id": 1, "name": 1} + subset_docs = list(get_subsets( + project_name, + asset_ids=[asset_doc["_id"]], + fields=["_id", "name"] )) subset_name_by_id = {} @@ -1145,15 +1100,10 @@ class SwitchAssetDialog(QtWidgets.QDialog): version_id = last_version["_id"] subset_id_by_version_id[version_id] = subset_id - repre_docs = legacy_io.find( - { - "type": "representation", - "parent": {"$in": list(subset_id_by_version_id.keys())} - }, - { - "name": 1, - "parent": 1 - } + repre_docs = get_representations( + project_name, + version_ids=subset_id_by_version_id.keys(), + fields=["name", "parent"] ) repres_by_subset_name = {} for repre_doc in repre_docs: @@ -1176,15 +1126,12 @@ class SwitchAssetDialog(QtWidgets.QDialog): # [ ] [x] [ ] # Subset documents - subset_docs = legacy_io.find( - { - "type": "subset", - "parent": {"$in": list(self.content_assets.keys())}, - "name": selected_subset - }, - {"_id": 1, "name": 1, "parent": 1} + subset_docs = get_subsets( + project_name, + asset_ids=self.content_assets.keys(), + subset_names=[selected_subset], + fields=["_id", "name", "parent"] ) - subset_docs_by_id = {} for subset_doc in subset_docs: subset_docs_by_id[subset_doc["_id"]] = subset_doc @@ -1197,15 +1144,10 @@ class SwitchAssetDialog(QtWidgets.QDialog): version_id = last_version["_id"] subset_id_by_version_id[version_id] = subset_id - repre_docs = legacy_io.find( - { - "type": "representation", - "parent": {"$in": list(subset_id_by_version_id.keys())} - }, - { - "name": 1, - "parent": 1 - } + repre_docs = get_representations( + project_name, + version_ids=subset_id_by_version_id.keys(), + fields=["name", "parent"] ) repres_by_asset_id = {} for repre_doc in repre_docs: @@ -1245,11 +1187,9 @@ class SwitchAssetDialog(QtWidgets.QDialog): selected_subset = self._subsets_box.get_valid_value() selected_representation = self._representations_box.get_valid_value() + project_name = self.active_project() if selected_asset: - asset_doc = legacy_io.find_one({ - "type": "asset", - "name": selected_asset - }) + asset_doc = get_asset_by_name(project_name, selected_asset) asset_docs_by_id = {asset_doc["_id"]: asset_doc} else: asset_docs_by_id = self.content_assets @@ -1259,16 +1199,15 @@ class SwitchAssetDialog(QtWidgets.QDialog): for asset_doc in asset_docs_by_id.values() } - asset_ids = list(asset_docs_by_id.keys()) - - subset_query = { - "type": "subset", - "parent": {"$in": asset_ids} - } + subset_names = None if selected_subset: - subset_query["name"] = selected_subset + subset_names = [selected_subset] - subset_docs = list(legacy_io.find(subset_query)) + subset_docs = list(get_subsets( + project_name, + subset_names=subset_names, + asset_ids=asset_docs_by_id.keys() + )) subset_ids = [] subset_docs_by_parent_and_name = collections.defaultdict(dict) for subset in subset_docs: @@ -1278,15 +1217,14 @@ class SwitchAssetDialog(QtWidgets.QDialog): subset_docs_by_parent_and_name[parent_id][name] = subset # versions - version_docs = list(legacy_io.find({ - "type": "version", - "parent": {"$in": subset_ids} - }, sort=[("name", -1)])) + _version_docs = get_versions(project_name, subset_ids=subset_ids) + version_docs = list(reversed( + sorted(_version_docs, key=lambda item: item["name"]) + )) - hero_version_docs = list(legacy_io.find({ - "type": "hero_version", - "parent": {"$in": subset_ids} - })) + hero_version_docs = list(get_hero_versions( + project_name, subset_ids=subset_ids + )) version_ids = list() @@ -1303,10 +1241,7 @@ class SwitchAssetDialog(QtWidgets.QDialog): parent_id = hero_version_doc["parent"] hero_version_docs_by_parent_id[parent_id] = hero_version_doc - repre_docs = legacy_io.find({ - "type": "representation", - "parent": {"$in": version_ids} - }) + repre_docs = get_representations(project_name, version_ids=version_ids) repre_docs_by_parent_id_by_name = collections.defaultdict(dict) for repre_doc in repre_docs: parent_id = repre_doc["parent"] diff --git a/openpype/tools/sceneinventory/view.py b/openpype/tools/sceneinventory/view.py index 448e3f4e6f..63d181b2d6 100644 --- a/openpype/tools/sceneinventory/view.py +++ b/openpype/tools/sceneinventory/view.py @@ -6,6 +6,13 @@ from Qt import QtWidgets, QtCore import qtawesome from bson.objectid import ObjectId +from openpype.client import ( + get_version_by_id, + get_versions, + get_hero_versions, + get_representation_by_id, + get_representations, +) from openpype import style from openpype.pipeline import ( legacy_io, @@ -83,12 +90,9 @@ class SceneInventoryView(QtWidgets.QTreeView): if item_id not in repre_ids: repre_ids.append(item_id) - repre_docs = legacy_io.find( - { - "type": "representation", - "_id": {"$in": repre_ids} - }, - {"parent": 1} + project_name = legacy_io.active_project() + repre_docs = get_representations( + project_name, representation_ids=repre_ids, fields=["parent"] ) version_ids = [] @@ -97,10 +101,9 @@ class SceneInventoryView(QtWidgets.QTreeView): if version_id not in version_ids: version_ids.append(version_id) - loaded_versions = legacy_io.find({ - "_id": {"$in": version_ids}, - "type": {"$in": ["version", "hero_version"]} - }) + loaded_versions = get_versions( + project_name, version_ids=version_ids, hero=True + ) loaded_hero_versions = [] versions_by_parent_id = collections.defaultdict(list) @@ -114,10 +117,9 @@ class SceneInventoryView(QtWidgets.QTreeView): if parent_id not in version_parents: version_parents.append(parent_id) - all_versions = legacy_io.find({ - "type": {"$in": ["hero_version", "version"]}, - "parent": {"$in": version_parents} - }) + all_versions = get_versions( + project_name, subset_ids=version_parents, hero=True + ) hero_versions = [] versions = [] for version in all_versions: @@ -150,12 +152,10 @@ class SceneInventoryView(QtWidgets.QTreeView): if item_id not in repre_ids: repre_ids.append(item_id) - repre_docs = legacy_io.find( - { - "type": "representation", - "_id": {"$in": repre_ids} - }, - {"parent": 1} + repre_docs = get_representations( + project_name, + representation_ids=repre_ids, + fields=["parent"] ) version_ids = [] @@ -165,13 +165,13 @@ class SceneInventoryView(QtWidgets.QTreeView): version_id_by_repre_id[repre_doc["_id"]] = version_id if version_id not in version_ids: version_ids.append(version_id) - hero_versions = legacy_io.find( - { - "_id": {"$in": version_ids}, - "type": "hero_version" - }, - {"version_id": 1} + + hero_versions = get_hero_versions( + project_name, + version_ids=version_ids, + fields=["version_id"] ) + version_ids = set() for hero_version in hero_versions: version_id = hero_version["version_id"] @@ -183,12 +183,10 @@ class SceneInventoryView(QtWidgets.QTreeView): if current_version_id == hero_version_id: version_id_by_repre_id[_repre_id] = version_id - version_docs = legacy_io.find( - { - "_id": {"$in": list(version_ids)}, - "type": "version" - }, - {"name": 1} + version_docs = get_versions( + project_name, + version_ids=version_ids, + fields=["name"] ) version_name_by_id = {} for version_doc in version_docs: @@ -370,10 +368,9 @@ class SceneInventoryView(QtWidgets.QTreeView): active_site = self.sync_server.get_active_site(project_name) remote_site = self.sync_server.get_remote_site(project_name) - repre_docs = legacy_io.find({ - "type": "representation", - "_id": {"$in": repre_ids} - }) + repre_docs = get_representations( + project_name, representation_ids=repre_ids + ) repre_docs_by_id = { repre_doc["_id"]: repre_doc for repre_doc in repre_docs @@ -658,25 +655,35 @@ class SceneInventoryView(QtWidgets.QTreeView): active = items[-1] + project_name = legacy_io.active_project() # Get available versions for active representation - representation_id = ObjectId(active["representation"]) - representation = legacy_io.find_one({"_id": representation_id}) - version = legacy_io.find_one({ - "_id": representation["parent"] - }) + repre_doc = get_representation_by_id( + project_name, + active["representation"], + fields=["parent"] + ) - versions = list(legacy_io.find( - { - "parent": version["parent"], - "type": "version" - }, - sort=[("name", 1)] + repre_version_doc = get_version_by_id( + project_name, + repre_doc["parent"], + fields=["parent"] + ) + + version_docs = list(get_versions( + project_name, + subset_ids=[repre_version_doc["parent"]], + hero=True + )) + hero_version = None + standard_versions = [] + for version_doc in version_docs: + if version_doc["type"] == "hero_version": + hero_version = version_doc + else: + standard_versions.append(version_doc) + versions = list(reversed( + sorted(standard_versions, key=lambda item: item["name"]) )) - - hero_version = legacy_io.find_one({ - "parent": version["parent"], - "type": "hero_version" - }) if hero_version: _version_id = hero_version["version_id"] for _version in versions: @@ -703,7 +710,7 @@ class SceneInventoryView(QtWidgets.QTreeView): all_versions = [] if hero_version: all_versions.append(hero_version) - all_versions.extend(reversed(versions)) + all_versions.extend(versions) if current_item: index = all_versions.index(current_item) diff --git a/openpype/tools/standalonepublish/app.py b/openpype/tools/standalonepublish/app.py index 1ad5cd119e..3ceeb3ad48 100644 --- a/openpype/tools/standalonepublish/app.py +++ b/openpype/tools/standalonepublish/app.py @@ -6,6 +6,8 @@ import signal from bson.objectid import ObjectId from Qt import QtWidgets, QtCore, QtGui +from openpype.client import get_asset_by_id + from .widgets import ( AssetWidget, FamilyWidget, ComponentsWidget, ShadowWidget ) @@ -126,17 +128,6 @@ class Window(QtWidgets.QDialog): if event: super().resizeEvent(event) - def get_avalon_parent(self, entity): - ''' Avalon DB entities helper - get all parents (exclude project). - ''' - parent_id = entity['data']['visualParent'] - parents = [] - if parent_id is not None: - parent = self.db.find_one({'_id': parent_id}) - parents.extend(self.get_avalon_parent(parent)) - parents.append(parent['name']) - return parents - def on_project_change(self, project_name): self.widget_family.refresh() @@ -152,7 +143,10 @@ class Window(QtWidgets.QDialog): ] if len(selected) == 1: self.valid_parent = True - asset = self.db.find_one({"_id": selected[0], "type": "asset"}) + project_name = self.db.active_project() + asset = get_asset_by_id( + project_name, selected[0], fields=["name"] + ) self.widget_family.change_asset(asset['name']) else: self.valid_parent = False diff --git a/openpype/tools/standalonepublish/widgets/model_asset.py b/openpype/tools/standalonepublish/widgets/model_asset.py index 02e9073555..abfc0a2145 100644 --- a/openpype/tools/standalonepublish/widgets/model_asset.py +++ b/openpype/tools/standalonepublish/widgets/model_asset.py @@ -4,6 +4,7 @@ import collections from Qt import QtCore, QtGui import qtawesome +from openpype.client import get_assets from openpype.style import ( get_default_entity_icon_color, get_deprecated_entity_font_color, @@ -104,17 +105,18 @@ class AssetModel(TreeModel): def refresh(self): """Refresh the data for the model.""" + project_name = self.dbcon.active_project() self.clear() - if ( - self.dbcon.active_project() is None or - self.dbcon.active_project() == '' - ): + if not project_name: return self.beginResetModel() # Get all assets in current project sorted by name - db_assets = self.dbcon.find({"type": "asset"}).sort("name", 1) + asset_docs = get_assets(project_name) + db_assets = list( + sorted(asset_docs, key=lambda item: item["name"]) + ) # Group the assets by their visual parent's id assets_by_parent = collections.defaultdict(list) diff --git a/openpype/tools/standalonepublish/widgets/widget_asset.py b/openpype/tools/standalonepublish/widgets/widget_asset.py index 8b43cd7cf8..73114f7960 100644 --- a/openpype/tools/standalonepublish/widgets/widget_asset.py +++ b/openpype/tools/standalonepublish/widgets/widget_asset.py @@ -2,6 +2,10 @@ import contextlib from Qt import QtWidgets, QtCore import qtawesome +from openpype.client import ( + get_project, + get_asset_by_id, +) from openpype.tools.utils import PlaceholderLineEdit from openpype.style import get_default_tools_icon_color @@ -218,7 +222,8 @@ class AssetWidget(QtWidgets.QWidget): self.view = view def collect_data(self): - project = self.dbcon.find_one({'type': 'project'}) + project_name = self.dbcon.active_project() + project = get_project(project_name, fields=["name"]) asset = self.get_active_asset() try: @@ -241,9 +246,16 @@ class AssetWidget(QtWidgets.QWidget): return ent_parents output = [] - if entity.get('data', {}).get('visualParent', None) is None: + parent_asset_id = entity.get('data', {}).get('visualParent', None) + if parent_asset_id is None: return output - parent = self.dbcon.find_one({'_id': entity['data']['visualParent']}) + + project_name = self.dbcon.active_project() + parent = get_asset_by_id( + project_name, + parent_asset_id, + fields=["name", "data.visualParent"] + ) output.append(parent['name']) output.extend(self.get_parents(parent)) return output @@ -349,9 +361,10 @@ class AssetWidget(QtWidgets.QWidget): tasks = [] selected = self.get_selected_assets() if len(selected) == 1: - asset = self.dbcon.find_one({ - "_id": selected[0], "type": "asset" - }) + project_name = self.dbcon.active_project() + asset = get_asset_by_id( + project_name, selected[0], fields=["data.tasks"] + ) if asset: tasks = asset.get('data', {}).get('tasks', []) self.task_model.set_tasks(tasks) @@ -400,7 +413,7 @@ class AssetWidget(QtWidgets.QWidget): # Select mode = selection_model.Select | selection_model.Rows - for index in lib.iter_model_rows( + for index in _iter_model_rows( self.proxy, column=0, include_root=False ): # stop iteration if there are no assets to process diff --git a/openpype/tools/standalonepublish/widgets/widget_family.py b/openpype/tools/standalonepublish/widgets/widget_family.py index 08cd45bbf2..1736be84ab 100644 --- a/openpype/tools/standalonepublish/widgets/widget_family.py +++ b/openpype/tools/standalonepublish/widgets/widget_family.py @@ -1,14 +1,21 @@ import re from Qt import QtWidgets, QtCore -from . import HelpRole, FamilyRole, ExistsRole, PluginRole, PluginKeyRole -from . import FamilyDescriptionWidget +from openpype.client import ( + get_asset_by_name, + get_subset_by_name, + get_subsets, + get_last_version_by_subset_id, +) from openpype.api import get_project_settings from openpype.pipeline import LegacyCreator from openpype.lib import TaskNotSetError from openpype.pipeline.create import SUBSET_NAME_ALLOWED_SYMBOLS +from . import HelpRole, FamilyRole, ExistsRole, PluginRole, PluginKeyRole +from . import FamilyDescriptionWidget + class FamilyWidget(QtWidgets.QWidget): @@ -180,12 +187,9 @@ class FamilyWidget(QtWidgets.QWidget): asset_doc = None if asset_name != self.NOT_SELECTED: # Get the assets from the database which match with the name - asset_doc = self.dbcon.find_one( - { - "type": "asset", - "name": asset_name - }, - {"_id": 1} + project_name = self.dbcon.active_project() + asset_doc = get_asset_by_name( + project_name, asset_name, fields=["_id"] ) # Get plugin and family @@ -200,14 +204,13 @@ class FamilyWidget(QtWidgets.QWidget): return # Get the asset from the database which match with the name - asset_doc = self.dbcon.find_one( - {"name": asset_name, "type": "asset"}, - projection={"_id": 1} + project_name = self.dbcon.active_project() + asset_doc = get_asset_by_name( + project_name, asset_name, fields=["_id"] ) # Get plugin plugin = item.data(PluginRole) if asset_doc and plugin: - project_name = self.dbcon.Session["AVALON_PROJECT"] asset_id = asset_doc["_id"] task_name = self.dbcon.Session["AVALON_TASK"] @@ -231,14 +234,14 @@ class FamilyWidget(QtWidgets.QWidget): self.input_result.setText("Select task please") # Get all subsets of the current asset - subset_docs = self.dbcon.find( - { - "type": "subset", - "parent": asset_id - }, - {"name": 1} + subset_docs = get_subsets( + project_name, asset_ids=[asset_id], fields=["name"] ) - existing_subset_names = set(subset_docs.distinct("name")) + + existing_subset_names = { + subset_doc["name"] + for subset_doc in subset_docs + } # Defaults to dropdown defaults = [] @@ -296,47 +299,37 @@ class FamilyWidget(QtWidgets.QWidget): if not auto_version: return + project_name = self.dbcon.active_project() asset_name = self.asset_name subset_name = self.input_result.text() version = 1 asset_doc = None subset_doc = None - versions = None if ( asset_name != self.NOT_SELECTED and subset_name.strip() != '' ): - asset_doc = self.dbcon.find_one( - { - 'type': 'asset', - 'name': asset_name - }, - {"_id": 1} + asset_doc = get_asset_by_name( + project_name, asset_name, fields=["_id"] ) if asset_doc: - subset_doc = self.dbcon.find_one( - { - 'type': 'subset', - 'parent': asset_doc['_id'], - 'name': subset_name - }, - {"_id": 1} + subset_doc = get_subset_by_name( + project_name, + subset_name, + asset_doc['_id'], + fields=["_id"] ) if subset_doc: - versions = self.dbcon.find( - { - 'type': 'version', - 'parent': subset_doc['_id'] - }, - {"name": 1} - ).distinct("name") - - if versions: - versions = sorted(versions) - version = int(versions[-1]) + 1 + last_version = get_last_version_by_subset_id( + project_name, + subset_doc["_id"], + fields=["name"] + ) + if last_version: + version = last_version["name"] + 1 self.version_spinbox.setValue(version) diff --git a/openpype/tools/texture_copy/app.py b/openpype/tools/texture_copy/app.py index fd8d6dc02e..746a72b3ec 100644 --- a/openpype/tools/texture_copy/app.py +++ b/openpype/tools/texture_copy/app.py @@ -4,6 +4,7 @@ import click import speedcopy +from openpype.client import get_project, get_asset_by_name from openpype.lib import Terminal from openpype.api import Anatomy from openpype.pipeline import legacy_io @@ -29,20 +30,6 @@ class TextureCopy: if os.path.splitext(x)[1].lower() in texture_extensions) return textures - def _get_project(self, project_name): - project = legacy_io.find_one({ - 'type': 'project', - 'name': project_name - }) - return project - - def _get_asset(self, asset_name): - asset = legacy_io.find_one({ - 'type': 'asset', - 'name': asset_name - }) - return asset - def _get_destination_path(self, asset, project): project_name = project["name"] hierarchy = "" @@ -88,11 +75,12 @@ class TextureCopy: t.echo("!!! {}".format(e)) exit(1) - def process(self, asset, project, path): + def process(self, asset_name, project_name, path): """ Process all textures found in path and copy them to asset under project. """ + t.echo(">>> Looking for textures ...") textures = self._get_textures(path) if len(textures) < 1: @@ -101,14 +89,14 @@ class TextureCopy: else: t.echo(">>> Found {} textures ...".format(len(textures))) - project = self._get_project(project) + project = get_project(project_name) if not project: - t.echo("!!! Project name [ {} ] not found.".format(project)) + t.echo("!!! Project name [ {} ] not found.".format(project_name)) exit(1) - asset = self._get_asset(asset) - if not project: - t.echo("!!! Asset [ {} ] not found in project".format(asset)) + asset = get_asset_by_name(project_name, asset_name) + if not asset: + t.echo("!!! Asset [ {} ] not found in project".format(asset_name)) exit(1) t.echo((">>> Project [ {} ] and " "asset [ {} ] seems to be OK ...").format(project['name'], diff --git a/openpype/tools/utils/assets_widget.py b/openpype/tools/utils/assets_widget.py index 82bdcd63a2..772946e9e1 100644 --- a/openpype/tools/utils/assets_widget.py +++ b/openpype/tools/utils/assets_widget.py @@ -5,6 +5,10 @@ import Qt from Qt import QtWidgets, QtCore, QtGui import qtawesome +from openpype.client import ( + get_project, + get_assets, +) from openpype.style import ( get_objected_colors, get_default_tools_icon_color, @@ -525,21 +529,18 @@ class AssetModel(QtGui.QStandardItemModel): self._doc_fetched.emit() def _fetch_asset_docs(self): - if not self.dbcon.Session.get("AVALON_PROJECT"): + project_name = self.dbcon.current_project() + if not project_name: return [] - project_doc = self.dbcon.find_one( - {"type": "project"}, - {"_id": True} - ) + project_doc = get_project(project_name, fields=["_id"]) if not project_doc: return [] # Get all assets sorted by name - return list(self.dbcon.find( - {"type": "asset"}, - self._asset_projection - )) + return list( + get_assets(project_name, fields=self._asset_projection.keys()) + ) def _stop_fetch_thread(self): self._refreshing = False diff --git a/openpype/tools/utils/delegates.py b/openpype/tools/utils/delegates.py index 71f817a1d7..d6c2d69e76 100644 --- a/openpype/tools/utils/delegates.py +++ b/openpype/tools/utils/delegates.py @@ -6,15 +6,19 @@ import numbers import Qt from Qt import QtWidgets, QtGui, QtCore -from openpype.pipeline import HeroVersionType -from .models import TreeModel -from . import lib - if Qt.__binding__ == "PySide": from PySide.QtGui import QStyleOptionViewItemV4 elif Qt.__binding__ == "PyQt4": from PyQt4.QtGui import QStyleOptionViewItemV4 +from openpype.client import ( + get_versions, + get_hero_versions, +) +from openpype.pipeline import HeroVersionType +from .models import TreeModel +from . import lib + log = logging.getLogger(__name__) @@ -114,26 +118,24 @@ class VersionDelegate(QtWidgets.QStyledItemDelegate): "Version is not integer" ) + project_name = self.dbcon.active_project() # Add all available versions to the editor parent_id = item["version_document"]["parent"] - version_docs = list(self.dbcon.find( - { - "type": "version", - "parent": parent_id - }, - sort=[("name", 1)] + version_docs = list(sorted( + get_versions(project_name, subset_ids=[parent_id]), + key=lambda item: item["name"] )) - hero_version_doc = self.dbcon.find_one( - { - "type": "hero_version", - "parent": parent_id - }, { - "name": 1, - "data.tags": 1, - "version_id": 1 - } + hero_versions = list( + get_hero_versions( + project_name, + subset_ids=[parent_id], + fields=["name", "data.tags", "version_id"] + ) ) + hero_version_doc = None + if hero_versions: + hero_version_doc = hero_versions[0] doc_for_hero_version = None diff --git a/openpype/tools/utils/lib.py b/openpype/tools/utils/lib.py index 20fea6600b..ea1362945f 100644 --- a/openpype/tools/utils/lib.py +++ b/openpype/tools/utils/lib.py @@ -6,6 +6,10 @@ import collections from Qt import QtWidgets, QtCore, QtGui import qtawesome +from openpype.client import ( + get_project, + get_asset_by_name, +) from openpype.style import ( get_default_entity_icon_color, get_objected_colors, @@ -430,9 +434,8 @@ class FamilyConfigCache: database = getattr(self.dbcon, "database", None) if database is None: database = self.dbcon._database - asset_doc = database[project_name].find_one( - {"type": "asset", "name": asset_name}, - {"data.tasks": True} + asset_doc = get_asset_by_name( + project_name, asset_name, fields=["data.tasks"] ) or {} tasks_info = asset_doc.get("data", {}).get("tasks") or {} task_type = tasks_info.get(task_name, {}).get("type") @@ -500,10 +503,7 @@ class GroupsConfig: project_name = self.dbcon.Session.get("AVALON_PROJECT") if project_name: # Get pre-defined group name and appearance from project config - project_doc = self.dbcon.find_one( - {"type": "project"}, - projection={"config.groups": True} - ) + project_doc = get_project(project_name, fields=["config.groups"]) if project_doc: group_configs = project_doc["config"].get("groups") or [] diff --git a/openpype/tools/utils/tasks_widget.py b/openpype/tools/utils/tasks_widget.py index eab183d5f3..0353f3dd2f 100644 --- a/openpype/tools/utils/tasks_widget.py +++ b/openpype/tools/utils/tasks_widget.py @@ -1,6 +1,10 @@ from Qt import QtWidgets, QtCore, QtGui import qtawesome +from openpype.client import ( + get_project, + get_asset_by_id, +) from openpype.style import get_disabled_entity_icon_color from openpype.tools.utils.lib import get_task_icon @@ -47,7 +51,8 @@ class TasksModel(QtGui.QStandardItemModel): # Get the project configured icons from database project_doc = {} if self._context_is_valid(): - project_doc = self.dbcon.find_one({"type": "project"}) + project_name = self.dbcon.active_project() + project_doc = get_project(project_name) self._loaded_project_name = self._get_current_project() self._project_doc = project_doc @@ -71,9 +76,9 @@ class TasksModel(QtGui.QStandardItemModel): def set_asset_id(self, asset_id): asset_doc = None if self._context_is_valid(): - asset_doc = self.dbcon.find_one( - {"_id": asset_id}, - {"data.tasks": True} + project_name = self._get_current_project() + asset_doc = get_asset_by_id( + project_name, asset_id, fields=["data.tasks"] ) self._set_asset(asset_doc) diff --git a/openpype/tools/workfiles/files_widget.py b/openpype/tools/workfiles/files_widget.py index 977111b71b..a7e54471dc 100644 --- a/openpype/tools/workfiles/files_widget.py +++ b/openpype/tools/workfiles/files_widget.py @@ -1,10 +1,12 @@ import os import logging import shutil +import copy import Qt from Qt import QtWidgets, QtCore +from openpype.client import get_asset_by_id from openpype.tools.utils import PlaceholderLineEdit from openpype.tools.utils.delegates import PrettyTimeDelegate from openpype.lib import ( @@ -89,7 +91,9 @@ class FilesWidget(QtWidgets.QWidget): self._task_type = None # Pype's anatomy object for current project - self.anatomy = Anatomy(legacy_io.Session["AVALON_PROJECT"]) + project_name = legacy_io.Session["AVALON_PROJECT"] + self.anatomy = Anatomy(project_name) + self.project_name = project_name # Template key used to get work template from anatomy templates self.template_key = "work" @@ -97,6 +101,7 @@ class FilesWidget(QtWidgets.QWidget): self._workfiles_root = None self._workdir_path = None self.host = registered_host() + self.host_name = os.environ["AVALON_APP"] # Whether to automatically select the latest modified # file on a refresh of the files model. @@ -384,7 +389,10 @@ class FilesWidget(QtWidgets.QWidget): return None if self._asset_doc is None: - self._asset_doc = legacy_io.find_one({"_id": self._asset_id}) + self._asset_doc = get_asset_by_id( + self.project_name, self._asset_id + ) + return self._asset_doc def _get_session(self): @@ -393,8 +401,8 @@ class FilesWidget(QtWidgets.QWidget): session = legacy_io.Session.copy() self.template_key = get_workfile_template_key( self._task_type, - session["AVALON_APP"], - project_name=session["AVALON_PROJECT"] + self.host_name, + project_name=self.project_name ) changes = compute_session_changes( session, @@ -427,6 +435,21 @@ class FilesWidget(QtWidgets.QWidget): template_key=self.template_key ) + def _get_event_context_data(self): + asset_id = None + asset_name = None + asset_doc = self._get_asset_doc() + if asset_doc: + asset_id = asset_doc["_id"] + asset_name = asset_doc["name"] + return { + "project_name": self.project_name, + "asset_id": asset_id, + "asset_name": asset_name, + "task_name": self._task_name, + "host_name": self.host_name + } + def open_file(self, filepath): host = self.host if host.has_unsaved_changes(): @@ -450,8 +473,21 @@ class FilesWidget(QtWidgets.QWidget): # Save current scene, continue to open file host.save_file(current_file) + event_data_before = self._get_event_context_data() + event_data_before["filepath"] = filepath + event_data_after = copy.deepcopy(event_data_before) + emit_event( + "workfile.open.before", + event_data_before, + source="workfiles.tool" + ) self._enter_session() host.open_file(filepath) + emit_event( + "workfile.open.after", + event_data_after, + source="workfiles.tool" + ) self.file_opened.emit() def save_changes_prompt(self): @@ -564,9 +600,14 @@ class FilesWidget(QtWidgets.QWidget): src_path = self._get_selected_filepath() # Trigger before save event + event_data_before = self._get_event_context_data() + event_data_before.update({ + "filename": work_filename, + "workdir_path": self._workdir_path + }) emit_event( "workfile.save.before", - {"filename": work_filename, "workdir_path": self._workdir_path}, + event_data_before, source="workfiles.tool" ) @@ -599,15 +640,20 @@ class FilesWidget(QtWidgets.QWidget): # Create extra folders create_workdir_extra_folders( self._workdir_path, - legacy_io.Session["AVALON_APP"], + self.host_name, self._task_type, self._task_name, - legacy_io.Session["AVALON_PROJECT"] + self.project_name ) + event_data_after = self._get_event_context_data() + event_data_after.update({ + "filename": work_filename, + "workdir_path": self._workdir_path + }) # Trigger after save events emit_event( "workfile.save.after", - {"filename": work_filename, "workdir_path": self._workdir_path}, + event_data_after, source="workfiles.tool" ) diff --git a/openpype/tools/workfiles/model.py b/openpype/tools/workfiles/model.py index 8f9dd8c6ba..d5b7cef339 100644 --- a/openpype/tools/workfiles/model.py +++ b/openpype/tools/workfiles/model.py @@ -4,6 +4,11 @@ import logging from Qt import QtCore, QtGui import qtawesome +from openpype.client import ( + get_subsets, + get_versions, + get_representations, +) from openpype.style import ( get_default_entity_icon_color, get_disabled_entity_icon_color, @@ -215,6 +220,7 @@ class PublishFilesModel(QtGui.QStandardItemModel): self._dbcon = dbcon self._anatomy = anatomy + self._file_extensions = extensions self._invalid_context_item = None @@ -234,6 +240,10 @@ class PublishFilesModel(QtGui.QStandardItemModel): self._asset_id = None self._task_name = None + @property + def project_name(self): + return self._dbcon.Session["AVALON_PROJECT"] + def _set_item_invalid(self, item): item.setFlags(QtCore.Qt.NoItemFlags) item.setData(self._invalid_icon, QtCore.Qt.DecorationRole) @@ -285,15 +295,11 @@ class PublishFilesModel(QtGui.QStandardItemModel): def _get_workfie_representations(self): output = [] # Get subset docs of asset - subset_docs = self._dbcon.find( - { - "type": "subset", - "parent": self._asset_id - }, - { - "_id": True, - "name": True - } + subset_docs = get_subsets( + self.project_name, + asset_ids=[self._asset_id], + fields=["_id", "name"] + ) subset_ids = [subset_doc["_id"] for subset_doc in subset_docs] @@ -301,17 +307,12 @@ class PublishFilesModel(QtGui.QStandardItemModel): return output # Get version docs of subsets with their families - version_docs = self._dbcon.find( - { - "type": "version", - "parent": {"$in": subset_ids} - }, - { - "_id": True, - "data.families": True, - "parent": True - } + version_docs = get_versions( + self.project_name, + subset_ids=subset_ids, + fields=["_id", "parent", "data.families"] ) + # Filter versions if they contain 'workfile' family filtered_versions = [] for version_doc in version_docs: @@ -327,13 +328,10 @@ class PublishFilesModel(QtGui.QStandardItemModel): # Query representations of filtered versions and add filter for # extension extensions = [ext.replace(".", "") for ext in self._file_extensions] - repre_docs = self._dbcon.find( - { - "type": "representation", - "parent": {"$in": version_ids}, - "context.ext": {"$in": extensions} - } + repre_docs = get_representations( + self.project_name, version_ids, extensions ) + # Filter queried representations by task name if task is set filtered_repre_docs = [] for repre_doc in repre_docs: diff --git a/openpype/tools/workfiles/save_as_dialog.py b/openpype/tools/workfiles/save_as_dialog.py index 3e97d6c938..b62fd2c889 100644 --- a/openpype/tools/workfiles/save_as_dialog.py +++ b/openpype/tools/workfiles/save_as_dialog.py @@ -5,6 +5,10 @@ import logging from Qt import QtWidgets, QtCore +from openpype.client import ( + get_project, + get_asset_by_name, +) from openpype.lib import ( get_last_workfile_with_version, get_workdir_data, @@ -22,29 +26,19 @@ def build_workfile_data(session): """Get the data required for workfile formatting from avalon `session`""" # Set work file data for template formatting + project_name = session["AVALON_PROJECT"] asset_name = session["AVALON_ASSET"] task_name = session["AVALON_TASK"] host_name = session["AVALON_APP"] - project_doc = legacy_io.find_one( - {"type": "project"}, - { - "name": True, - "data.code": True, - "config.tasks": True, - } + project_doc = get_project( + project_name, fields=["name", "data.code", "config.tasks"] + ) + asset_doc = get_asset_by_name( + project_name, + asset_name, + fields=["name", "data.tasks", "data.parents"] ) - asset_doc = legacy_io.find_one( - { - "type": "asset", - "name": asset_name - }, - { - "name": True, - "data.tasks": True, - "data.parents": True - } - ) data = get_workdir_data(project_doc, asset_doc, task_name, host_name) data.update({ "version": 1, diff --git a/openpype/tools/workfiles/window.py b/openpype/tools/workfiles/window.py index 02a22af26c..9f4cea2f8a 100644 --- a/openpype/tools/workfiles/window.py +++ b/openpype/tools/workfiles/window.py @@ -2,6 +2,7 @@ import os import datetime from Qt import QtCore, QtWidgets +from openpype.client import get_asset_by_id, get_asset_by_name from openpype import style from openpype.lib import ( get_workfile_doc, @@ -223,6 +224,10 @@ class Window(QtWidgets.QMainWindow): self._first_show = True self._context_to_set = None + @property + def project_name(self): + return legacy_io.Session["AVALON_PROJECT"] + def showEvent(self, event): super(Window, self).showEvent(event) if self._first_show: @@ -296,7 +301,8 @@ class Window(QtWidgets.QMainWindow): if not workfile_doc: workdir, filename = os.path.split(filepath) asset_id = self.assets_widget.get_selected_asset_id() - asset_doc = legacy_io.find_one({"_id": asset_id}) + project_name = legacy_io.active_project() + asset_doc = get_asset_by_id(project_name, asset_id) task_name = self.tasks_widget.get_selected_task_name() create_workfile_doc( asset_doc, task_name, filename, workdir, legacy_io @@ -322,14 +328,13 @@ class Window(QtWidgets.QMainWindow): self._context_to_set, context = None, self._context_to_set if "asset" in context: - asset_doc = legacy_io.find_one( - { - "name": context["asset"], - "type": "asset" - }, - {"_id": 1} - ) or {} - asset_id = asset_doc.get("_id") + asset_doc = get_asset_by_name( + self.project_name, context["asset"], fields=["_id"] + ) + + asset_id = None + if asset_doc: + asset_id = asset_doc["_id"] # Select the asset self.assets_widget.select_asset(asset_id) self.tasks_widget.set_asset_id(asset_id) diff --git a/openpype/version.py b/openpype/version.py index 4b0a688cbf..79e3b445f9 100644 --- a/openpype/version.py +++ b/openpype/version.py @@ -1,3 +1,3 @@ # -*- coding: utf-8 -*- """Package declaring Pype version.""" -__version__ = "3.11.0-nightly.2" +__version__ = "3.11.2-nightly.1" diff --git a/pyproject.toml b/pyproject.toml index 4289c74ebe..4b297fe042 100644 --- a/pyproject.toml +++ b/pyproject.toml @@ -1,6 +1,6 @@ [tool.poetry] name = "OpenPype" -version = "3.11.0-nightly.2" # OpenPype +version = "3.11.2-nightly.1" # OpenPype description = "Open VFX and Animation pipeline with support." authors = ["OpenPype Team "] license = "MIT License" diff --git a/website/docs/admin_settings_project_anatomy.md b/website/docs/admin_settings_project_anatomy.md index 6e0b49f152..106faeb806 100644 --- a/website/docs/admin_settings_project_anatomy.md +++ b/website/docs/admin_settings_project_anatomy.md @@ -68,6 +68,7 @@ We have a few required anatomy templates for OpenPype to work properly, however | `representation` | Representation name | | `frame` | Frame number for sequence files. | | `app` | Application Name | +| `user` | User's login name (can be overridden in local settings) | | `output` | | | `comment` | | diff --git a/website/docs/artist_hosts_maya_multiverse.md b/website/docs/artist_hosts_maya_multiverse.md index e6520bafa0..a173e79125 100644 --- a/website/docs/artist_hosts_maya_multiverse.md +++ b/website/docs/artist_hosts_maya_multiverse.md @@ -65,6 +65,25 @@ the one depicted here: ![Maya - Multiverse Setup](assets/maya-multiverse_setup.png) + +``` +{ + "MULTIVERSE_PATH": "/Path/to/Multiverse-{MULTIVERSE_VERSION}", + "MAYA_MODULE_PATH": "{MULTIVERSE}/Maya;{MAYA_MODULE_PATH}" +} + +{ + "MULTIVERSE_VERSION": "7.1.0-py27" +} + +``` + +The Multiverse Maya module file (.mod) pointed above contains all the necessary +environment variables to run Multiverse. + +The OpenPype settings will contain blocks to enable/disable the Multiverse +Creators and Loader, along with sensible studio setting. + For more information about setup of Multiverse please refer to the relative page on the [Multiverse official documentation](https://multi-verse.io/docs). @@ -94,7 +113,7 @@ You can choose the USD file format in the Creators' set nodes: - Assets: `.usd` (default) or `.usda` or `.usdz` - Compositions: `.usda` (default) or `.usd` - Overrides: `.usda` (default) or `.usd` -- Looks: `.ma` +- Looks: `.ma` (default) ![Maya - Multiverse Asset Creator](assets/maya-multiverse_openpype_asset_creator.png) diff --git a/website/docs/assets/maya-multiverse_setup.png b/website/docs/assets/maya-multiverse_setup.png index 8aa89ef7e5..72bdb0d379 100644 Binary files a/website/docs/assets/maya-multiverse_setup.png and b/website/docs/assets/maya-multiverse_setup.png differ diff --git a/website/docs/module_ftrack.md b/website/docs/module_ftrack.md index fd9687ed9d..667782754f 100644 --- a/website/docs/module_ftrack.md +++ b/website/docs/module_ftrack.md @@ -26,7 +26,7 @@ You can only use our Ftrack Actions and publish to Ftrack if each artist is logg ### Custom Attributes After successfully connecting OpenPype with you Ftrack, you can right click on any project in Ftrack and you should see a bunch of actions available. The most important one is called `OpenPype Admin` and contains multiple options inside. -To prepare Ftrack for working with OpenPype you'll need to run [OpenPype Admin - Create/Update Avalon Attributes](manager_ftrack_actions.md#create-update-avalon-attributes), which creates and sets the Custom Attributes necessary for OpenPype to function. +To prepare Ftrack for working with OpenPype you'll need to run [OpenPype Admin - Create/Update Custom Attributes](manager_ftrack_actions.md#create-update-avalon-attributes), which creates and sets the Custom Attributes necessary for OpenPype to function. diff --git a/website/src/pages/index.js b/website/src/pages/index.js index 115102ed04..0886706015 100644 --- a/website/src/pages/index.js +++ b/website/src/pages/index.js @@ -361,7 +361,7 @@ function Home() { - DaVinci Resolve (Beta) + Resolve (Beta) @@ -374,6 +374,16 @@ function Home() { Ftrack + + + Shotgrid (Beta) + + + + + Kitsu (Beta) + + Clockify @@ -384,12 +394,7 @@ function Home() { Deadline - - - Muster - - - + Royal Render @@ -399,30 +404,30 @@ function Home() { Slack - - -

In development by us or OpenPype community.

- - +

Planned or in development by us and OpenPype community.

+ + diff --git a/website/static/img/app_hibob.png b/website/static/img/app_hibob.png new file mode 100644 index 0000000000..91dd8d3f6b Binary files /dev/null and b/website/static/img/app_hibob.png differ