diff --git a/.github/ISSUE_TEMPLATE/bug_report.yml b/.github/ISSUE_TEMPLATE/bug_report.yml index d7e49de5cb..0762eb2f20 100644 --- a/.github/ISSUE_TEMPLATE/bug_report.yml +++ b/.github/ISSUE_TEMPLATE/bug_report.yml @@ -35,6 +35,16 @@ body: label: Version description: What version are you running? Look to OpenPype Tray options: + - 3.17.0 + - 3.16.7 + - 3.16.7-nightly.2 + - 3.16.7-nightly.1 + - 3.16.6 + - 3.16.6-nightly.1 + - 3.16.5 + - 3.16.5-nightly.5 + - 3.16.5-nightly.4 + - 3.16.5-nightly.3 - 3.16.5-nightly.2 - 3.16.5-nightly.1 - 3.16.4 @@ -125,16 +135,6 @@ body: - 3.14.10-nightly.3 - 3.14.10-nightly.2 - 3.14.10-nightly.1 - - 3.14.9 - - 3.14.9-nightly.5 - - 3.14.9-nightly.4 - - 3.14.9-nightly.3 - - 3.14.9-nightly.2 - - 3.14.9-nightly.1 - - 3.14.8 - - 3.14.8-nightly.4 - - 3.14.8-nightly.3 - - 3.14.8-nightly.2 validations: required: true - type: dropdown diff --git a/CHANGELOG.md b/CHANGELOG.md index f1948b1a3f..4bcf66a210 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -1,6 +1,1289 @@ # Changelog +## [3.17.0](https://github.com/ynput/OpenPype/tree/3.17.0) + + +[Full Changelog](https://github.com/ynput/OpenPype/compare/3.16.7...3.17.0) + +### **🚀 Enhancements** + + +
+Chore: Remove schema from OpenPype root #5355 + +Remove unused schema directory in root of repository which was moved inside openpype/pipeline/schema. + + +___ + +
+ + +
+Igniter: Allow custom Qt scale factor rounding policy #5554 + +Do not force `PassThrough` rounding policy if different policy is defined via env variable. + + +___ + +
+ +### **🐛 Bug fixes** + + +
+Chore: Lower urllib3 to support older OpenSSL #5538 + +Lowered `urllib3` to `1.26.16` to support older OpenSSL. + + +___ + +
+ + +
+Chore: Do not try to add schema to zip files #5557 + +Do not add `schema` folder to zip file. This fixes issue cause by https://github.com/ynput/OpenPype/pull/5355 . + + +___ + +
+ + +
+Chore: Lower click dependency version #5629 + +Lower click version to support older versions of python. + + +___ + +
+ +### **Merged pull requests** + + +
+Bump certifi from 2023.5.7 to 2023.7.22 #5351 + +Bumps [certifi](https://github.com/certifi/python-certifi) from 2023.5.7 to 2023.7.22. +
+Commits + +
+
+ + +[![Dependabot compatibility score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=certifi&package-manager=pip&previous-version=2023.5.7&new-version=2023.7.22)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores) + +You can trigger a rebase of this PR by commenting `@dependabot rebase`. + +[//]: # (dependabot-automerge-start) +[//]: # (dependabot-automerge-end) + +--- + +
+Dependabot commands and options +
+ +You can trigger Dependabot actions by commenting on this PR: +- `@dependabot rebase` will rebase this PR +- `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it +- `@dependabot merge` will merge this PR after your CI passes on it +- `@dependabot squash and merge` will squash and merge this PR after your CI passes on it +- `@dependabot cancel merge` will cancel a previously requested merge and block automerging +- `@dependabot reopen` will reopen this PR if it is closed +- `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually +- `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself) +- `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself) +- `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself) +You can disable automated security fix PRs for this repo from the [Security Alerts page](https://github.com/ynput/OpenPype/network/alerts). + +
+ +> **Note** +> Automatic rebases have been disabled on this pull request as it has been open for over 30 days. + +___ + +
+ + + + +## [3.16.7](https://github.com/ynput/OpenPype/tree/3.16.7) + + +[Full Changelog](https://github.com/ynput/OpenPype/compare/3.16.6...3.16.7) + +### **🆕 New features** + + +
+Maya: Extract active view as thumbnail when no thumbnail set #5426 + +This sets the Maya instance's thumbnail to the current active view if no thumbnail was set yet. + + +___ + +
+ + +
+Maya: Implement USD publish and load using native `mayaUsdPlugin` #5573 + +Implement Creator and Loaders for extraction and loading of USD files using Maya's own `mayaUsdPlugin`.Also adds support to load a `usd` file into an Arnold Standin (`aiStandin`) and assigning looks to it. + + +___ + +
+ + +
+AYON: Ignore separated modules #5619 + +Do not load already separated modules from default directory. + + +___ + +
+ +### **🚀 Enhancements** + + +
+Maya: Reduce amount of code for Collect Looks #5253 + +- Refactor `get_file_node_files` because popping from `paths` by index should have been done in reversed order anyway. It's now changed to not need popping at all. +- Removed unused `RENDERER_NODE_TYPES` and if-branch which collected `node_attrs` list which was unused + collected members which was also done outside of the if branch and thus generated no extra data. +- Collected all materials from look set attributes at once instead of multiple queries +- Collected all file nodes in history from a single query instead of per type +- Restructured assignment of `instance.data["resources"]` to be more readable +- Cached `PXR_NODES` only ones (Note: plugin load is checked on discovery of the collect look plugin) instead of querying plugin load and its nodes per file node per attribute +- Removed some debug logs or combined some messages + + +___ + +
+ + +
+AYON: Mark deprecated settings in Maya #5627 + +Added deprecated info to docstrings of maya colormanagement settings.Resolves: https://github.com/ynput/OpenPype/issues/5556 + + +___ + +
+ + +
+Max: switching versions of maxScene maintain parentage/links with the loaders #5424 + +When using scene inventory to manage or update the version of the loading objects, the linked modifiers or parentage of the objects would be kept.Meanwhile, loaded objects from all loaders no longer parented to the container with OP Data. + + +___ + +
+ + +
+3ds max: small tweaks to obj extractor and model publishing flow #5605 + +There migh be situation where OBJ Extractor passes without failure, but no obj file is produced. This is adding simple check directly into the extractor to catch it earlier then in the integration phase. Also switched `Validate USD Plugin` to optional, because it was always run no matter if the Extract USD was enabled or not, hindering testing (and publishing). + + +___ + +
+ + +
+TVPaint: Plugin can be reopened #5610 + +TVPaint plugin can be reopened. + + +___ + +
+ + +
+Maya: Remove context prompt #5632 + +More of a plea than a PR, but could we please remove the context prompt in Maya when switching tasks? + + +___ + +
+ + +
+General: Create a desktop icon is checked #5636 + +In OP Installer `Create a desktop icon` is checked by default. +___ + +
+ +### **🐛 Bug fixes** + + +
+Maya: Extract look is not AYON compatible - OP-5375 #5341 + +The textures that would use hardlinking are going through texture processors. Currently all texture processors are hardcoded to copy texture instead of respecting the settings of forcing to copy.The texture processors were last modified 4 months ago, so effectively all clients that are on any pipeline updated in the last 4 months wont be utilizing hardlinking at all, since the hardcoded texture processors will copy texture no matter the OS.This opts for completely disabling the hardlinking feature, while we figure out what to do about it. + + +___ + +
+ + +
+Maya: Multiverse USD Override inherit from correct new style creator #5566 + +Fix Creator for Multiverse USD Override by inheriting from correct new style creator class type + + +___ + +
+ + +
+Max: Bug Fix Alembic Loaders with Ornatrix #5434 + +Bugfix the alembic loader with both ornatrix alembic and max alembic supportsAdd the ornatrix alembic loaders for loading the alembic with Ornatrix-related modifiers. + + +___ + +
+ + +
+AYON: Avoid creation of duplicated links #5593 + +Handle cases when an existing link should be recreated and do not create the same link multitple times during single publishing. + + +___ + +
+ + +
+Extract Review: Multilayer specification for ffmpeg #5613 + +Extract review is specifying layer name when exr is multilayer. + + +___ + +
+ + +
+Fussion: added support for Fusion 17 #5614 + +Fusion 17 still uses Python 3.6 which causes issues with some our delivered libraries. Vendorized necessary set for Python 3.6 + + +___ + +
+ + +
+Publisher: Fix screenshot widget #5615 + +Use correct super method name.EDITED:Removed fade animation which is not triggered at some cases, e.g. in Nuke the animation does not start. I do expect that is caused by `exec_` on the dialog, which blocks event processing to the animation, even when I've added the window as parent it still didn't trigger registered callback.Modified how the "empty" space is not filled by using paths instead of clear mode on painter. Added render hints to add antialiasing. + + +___ + +
+ + +
+Photoshop: auto_images without alpha will not fail #5620 + +ExtractReview caused issue on `auto_image` instance without alpha channel, this fixes it. + + +___ + +
+ + +
+Fix - _id key used instead of id in get_last_version_by_subset_name #5626 + +Just 'id' is not returned because value in fields. Caused KeyError. + + +___ + +
+ + +
+Bugfix: create symlinks for ssl libs on Centos 7 #5633 + +Docker build was missing `libssl.1.1.so` and `libcrypto.1.1.so` symlinks needed by the executable itself, because Python is now explicitly built with OpenSSL 1.1.1 + + +___ + +
+ +### **📃 Documentation** + + +
+Documentation/local settings #5102 + +I completed the "Working with local settings" page. I updated the screenshot, wrote an explanation for each empty category, and if available, linked the more detailed pages already existing. I also added the "Environments" category. + + +___ + +
+ + + + +## [3.16.6](https://github.com/ynput/OpenPype/tree/3.16.6) + + +[Full Changelog](https://github.com/ynput/OpenPype/compare/3.16.5...3.16.6) + +### **🆕 New features** + + +
+Workfiles tool: Refactor workfiles tool (for AYON) #5550 + +Refactored workfiles tool to new tool. Separated backend and frontend logic. Refactored logic is AYON-centric and is used only in AYON mode, so it does not affect OpenPype. + + +___ + +
+ + +
+AfterEffects: added validator for missing files in FootageItems #5590 + +Published composition in AE could contain multiple FootageItems as a layers. If FootageItem contains imported file and it doesn't exist, render triggered by Publish process will silently fail and no output is generated. This could cause failure later in the process with unclear reason. (In `ExtractReview`).This PR adds validation to protect from this. + + +___ + +
+ +### **🚀 Enhancements** + + +
+Maya: Yeti Cache Include viewport preview settings from source #5561 + +When publishing and loading yeti caches persist the display output and preview colors + settings to ensure consistency in the view + + +___ + +
+ + +
+Houdini: validate colorspace in review rop #5322 + +Adding a validator that checks if 'OCIO Colorspace' parameter on review rop was set to a valid value.It is a step towards managing colorspace in review ropvalid values are the ones in the dropdown menuthis validator also provides some helper actions This PR is related to #4836 and #4833 + + +___ + +
+ + +
+Colorspace: adding abstraction of publishing related functions #5497 + +The functionality of Colorspace has been abstracted for greater usability. + + +___ + +
+ + +
+Nuke: removing redundant workfile colorspace attributes #5580 + +Nuke root workfile colorspace data type knobs are long time configured automatically via config roles or the default values are also working well. Therefore there is no need for pipeline managed knobs. + + +___ + +
+ + +
+Ftrack: Less verbose logs for Ftrack integration in artist facing logs #5596 + +- Reduce artist-facing logs for component integration for Ftrack +- Avoid "Comment is not set" log in artist facing report for Kitsu and Ftrack +- Remove info log about `ffprobe` inspecting a file (changed to debug log) +- interesting to see however that it ffprobes the same jpeg twice - but maybe once for thumbnail? + + +___ + +
+ +### **🐛 Bug fixes** + + +
+Maya: Fix rig validators for new out_SET and controls_SET names #5595 + +Fix usage of `out_SET` and `controls_SET` since #5310 because they can now be prefixed by the subset name. + + +___ + +
+ + +
+TrayPublisher: set default frame values to sequential data #5530 + +We are inheriting default frame handles and fps data either from project or setting them to 0. This is just for case a production will decide not to injest the sequential representations with asset based metadata. + + +___ + +
+ + +
+Publisher: Screenshot opacity value fix #5576 + +Fix opacity value. + + +___ + +
+ + +
+AfterEffects: fix imports of image sequences #5581 + +#4602 broke imports of image sequences. + + +___ + +
+ + +
+AYON: Fix representation context conversion #5591 + +Do not fix `"folder"` key in representation context until it is needed. + + +___ + +
+ + +
+ayon-nuke: default factory to lists #5594 + +Default factory were missing in settings schemas for complicated objects like lists and it was causing settings to be failing saving. + + +___ + +
+ + +
+Maya: Fix look assigner showing no asset if 'not found' representations are present #5597 + +Fix Maya Look assigner failing to show any content if it finds an invalid container for which it can't find the asset in the current project. (This can happen when e.g. loading something from a library project).There was logic already to avoid this but there was a bug where it used variable `_id` which did not exist and likely had to be `asset_id`.I've fixed that and improved the logged message a bit, e.g.: +``` +// Warning: openpype.hosts.maya.tools.mayalookassigner.commands : Id found on 22 nodes for which no asset is found database, skipping '641d78ec85c3c5b102e836b0' +``` +Example not found representation in Loader:The issue isn't necessarily related to NOT FOUND representations but in essence boils down to finding nodes with asset ids that do not exist in the current project which could very well just be local meshes in your scene.**Note:**I've excluded logging the nodes themselves because that tends to be a very long list of nodes. Only downside to removing that is that it's unclear which nodes are related to that `id`. If there are any ideas on how to still provide a concise informational message about that that'd be great so I could add it. Things I had considered: +- Report the containers, issue here is that it's about asset ids on nodes which don't HAVE to be in containers - it could be local geometry +- Report the namespaces, issue here is that it could be nodes without namespaces (plus potentially not about ALL nodes in a namespace) +- Report the short names of the nodes; it's shorter and readable but still likely a lot of nodes.@tokejepsen @LiborBatek any other ideas? + + +___ + +
+ + +
+Photoshop: fixed blank Flatten image #5600 + +Flatten image is simplified publishing approach where all visible layers are "flatten" and published together. This image could be used as a reference etc.This is implemented by auto creator which wasn't updated after first publish. This would result in missing newly created layers after `auto_image` instance was created. + + +___ + +
+ + +
+Blender: Remove Hardcoded Subset Name for Reviews #5603 + +Fixes hardcoded subset name for Reviews in Blender. + + +___ + +
+ + +
+TVPaint: Fix tool callbacks #5608 + +Do not wait for callback to finish. + + +___ + +
+ +### **🔀 Refactored code** + + +
+Chore: Remove unused variables and cleanup #5588 + +Removing some unused variables. In some cases the unused variables _seemed like they should've been used - maybe?_ so please **double check the code whether it doesn't hint to an already existing bug**.Also tweaked some other small bugs in code + tweaked logging levels. + + +___ + +
+ +### **Merged pull requests** + + +
+Chore: Loader log deprecation warning for 'fname' attribute #5587 + +Since https://github.com/ynput/OpenPype/pull/4602 the `fname` attribute on the `LoaderPlugin` should've been deprecated and set for removal over time. However, no deprecation warning was logged whatsoever and thus one usage appears to have sneaked in (fixed with this PR) and a new one tried to sneak in with a recent PR + + +___ + +
+ + + + +## [3.16.5](https://github.com/ynput/OpenPype/tree/3.16.5) + + +[Full Changelog](https://github.com/ynput/OpenPype/compare/3.16.4...3.16.5) + +### **🆕 New features** + + +
+Attribute Definitions: Multiselection enum def #5547 + +Added `multiselection` option to `EnumDef`. + + +___ + +
+ +### **🚀 Enhancements** + + +
+Farm: adding target collector #5494 + +Enhancing farm publishing workflow. + + +___ + +
+ + +
+Maya: Optimize validate plug-in path attributes #5522 + +- Optimize query (use `cmds.ls` once) +- Add Select Invalid action +- Improve validation report +- Avoid "Unknown object type" errors + + +___ + +
+ + +
+Maya: Remove Validate Instance Attributes plug-in #5525 + +Remove Validate Instance Attributes plug-in. + + +___ + +
+ + +
+Enhancement: Tweak logging for artist facing reports #5537 + +Tweak the logging of publishing for global, deadline, maya and a fusion plugin to have a cleaner artist-facing report. +- Fix context being reported correctly from CollectContext +- Fix ValidateMeshArnoldAttributes: fix when arnold is not loaded, fix applying settings, fix for when ai attributes do not exist + + +___ + +
+ + +
+AYON: Update settings #5544 + +Updated settings in AYON addons and conversion of AYON settings in OpenPype. + + +___ + +
+ + +
+Chore: Removed Ass export script #5560 + +Removed Arnold render script, which was obsolete and unused. + + +___ + +
+ + +
+Nuke: Allow for knob values to be validated against multiple values. #5042 + +Knob values can now be validated against multiple values, so you can allow write nodes to be `exr` and `png`, or `16-bit` and `32-bit`. + + +___ + +
+ + +
+Enhancement: Cosmetics for Higher version of publish already exists validation error #5190 + +Fix double spaces in message.Example output **after** the PR: + + +___ + +
+ + +
+Nuke: publish existing frames on farm #5409 + +This PR proposes adding a fourth option in Nuke render publish called "Use Existing Frames - Farm". This would be useful when the farm is busy or when the artist lacks enough farm licenses. Additionally, some artists prefer rendering on the farm but still want to check frames before publishing.By adding the "Use Existing Frames - Farm" option, artists will have more flexibility and control over their render publishing process. This enhancement will streamline the workflow and improve efficiency for Nuke users. + + +___ + +
+ + +
+Unreal: Create project in temp location and move to final when done #5476 + +Create Unreal project in local temporary folder and when done, move it to final destination. + + +___ + +
+ + +
+TrayPublisher: adding audio product type into default presets #5489 + +Adding Audio product type into default presets so anybody can publish audio to their shots. + + +___ + +
+ + +
+Global: avoiding cleanup of flagged representation #5502 + +Publishing folder can be flagged as persistent at representation level. + + +___ + +
+ + +
+General: missing tag could raise error #5511 + +- avoiding potential situation where missing Tag key could raise error + + +___ + +
+ + +
+Chore: Queued event system #5514 + +Implemented event system with more expected behavior of event system. If an event is triggered during other event callback, it is not processed immediately but waits until all callbacks of previous events are done. The event system also allows to not trigger events directly once `emit_event` is called which gives option to process events in custom loops. + + +___ + +
+ + +
+Publisher: Tweak log message to provide plugin name after "Plugin" #5521 + +Fix logged message for settings automatically applied to plugin attributes + + +___ + +
+ + +
+Houdini: Improve VDB Selection #5523 + +Improves VDB selection if selection is `SopNode`: return the selected sop nodeif selection is `ObjNode`: get the output node with the minimum 'outputidx' or the node with display flag + + +___ + +
+ + +
+Maya: Refactor/tweak Validate Instance In same Context plug-in #5526 + +- Chore/Refactor: Re-use existing select invalid and repair actions +- Enhancement: provide more elaborate PublishValidationError report +- Bugfix: fix "optional" support by using `OptionalPyblishPluginMixin` base class. + + +___ + +
+ + +
+Enhancement: Update houdini main menu #5527 + +This PR adds two updates: +- dynamic main menu +- dynamic asset name and task + + +___ + +
+ + +
+Houdini: Reset FPS when clicking Set Frame Range #5528 + +_Similar to Maya,_ Make `Set Frame Range` resets FPS, issue https://github.com/ynput/OpenPype/issues/5516 + + +___ + +
+ + +
+Enhancement: Deadline plugins optimize, cleanup and fix optional support for validate deadline pools #5531 + +- Fix optional support of validate deadline pools +- Query deadline webservice only once per URL for verification, and once for available deadline pools instead of for every instance +- Use `deadlineUrl` in `instance.data` when validating pools if it is set. +- Code cleanup: Re-use existing `requests_get` implementation + + +___ + +
+ + +
+Chore: PowerShell script for docker build #5535 + +Added PowerShell script to run docker build. + + +___ + +
+ + +
+AYON: Deadline expand userpaths in executables list #5540 + +Expande `~` paths in executables list. + + +___ + +
+ + +
+Chore: Use correct git url #5542 + +Fixed github url in README.md. + + +___ + +
+ + +
+Chore: Create plugin does not expect system settings #5553 + +System settings are not passed to initialization of create plugin initialization (and `apply_settings`). + + +___ + +
+ + +
+Chore: Allow custom Qt scale factor rounding policy #5555 + +Do not force `PassThrough` rounding policy if different policy is defined via env variable. + + +___ + +
+ + +
+Houdini: Fix outdated containers pop-up on opening last workfile on launch #5567 + +Fix Houdini not showing outdated containers pop-up on scene open when launching with last workfile argument + + +___ + +
+ + +
+Houdini: Improve errors e.g. raise PublishValidationError or cosmetics #5568 + +Improve errors e.g. raise PublishValidationError or cosmeticsThis also fixes the Increment Current File plug-in since due to an invalid import it was previously broken + + +___ + +
+ + +
+Fusion: Code updates #5569 + +Update fusion code which contains obsolete code. Removed `switch_ui.py` script from fusion with related script in scripts. + + +___ + +
+ +### **🐛 Bug fixes** + + +
+Maya: Validate Shape Zero fix repair action + provide informational artist-facing report #5524 + +Refactor to PublishValidationError to allow the RepairAction to work + provide informational report message + + +___ + +
+ + +
+Maya: Fix attribute definitions for `CreateYetiCache` #5574 + +Fix attribute definitions for `CreateYetiCache` + + +___ + +
+ + +
+Max: Optional Renderable Camera Validator for Render Instance #5286 + +Optional validation to check on renderable camera being set up correctly for deadline submission.If not being set up correctly, it wont pass the validation and user can perform repair actions. + + +___ + +
+ + +
+Max: Adding custom modifiers back to the loaded objects #5378 + +The custom parameters OpenpypeData doesn't show in the loaded container when it is being loaded through the loader. + + +___ + +
+ + +
+Houdini: Use default_variant to Houdini Node TAB Creator #5421 + +Use the default variant of the creator plugins on the interactive creator from the TAB node search instead of hard-coding it to `Main`. + + +___ + +
+ + +
+Nuke: adding inherited colorspace from instance #5454 + +Thumbnails are extracted with inherited colorspace collected from rendering write node. + + +___ + +
+ + +
+Add kitsu credentials to deadline publish job #5455 + +This PR hopefully fixes this issue #5440 + + +___ + +
+ + +
+AYON: Fill entities during editorial #5475 + +Fill entities and update template data on instances during extract AYON hierarchy. + + +___ + +
+ + +
+Ftrack: Fix version 0 when integrating to Ftrack - OP-6595 #5477 + +Fix publishing version 0 to Ftrack. + + +___ + +
+ + +
+OCIO: windows unc path support in Nuke and Hiero #5479 + +Hiero and Nuke is not supporting windows unc path formatting in OCIO environment variable. + + +___ + +
+ + +
+Deadline: Added super call to init #5480 + +DL 10.3 requires plugin inheriting from DeadlinePlugin to call super's **init** explicitly. + + +___ + +
+ + +
+Nuke: fixing thumbnail and monitor out root attributes #5483 + +Nuke Root Colorspace settings for Thumbnail and Monitor Out schema was gradually changed between version 12, 13, 14 and we needed to address those changes individually for particular version. + + +___ + +
+ + +
+Nuke: fixing missing `instance_id` error #5484 + +Workfiles with Instances created in old publisher workflow were rising error during converting method since they were missing `instance_id` key introduced in new publisher workflow. + + +___ + +
+ + +
+Nuke: existing frames validator is repairing render target #5486 + +Nuke is now correctly repairing render target after the existing frames validator finds missing frames and repair action is used. + + +___ + +
+ + +
+added UE to extract burnins families #5487 + +This PR fixes missing burnins in reviewables when rendering from UE. +___ + +
+ + +
+Harmony: refresh code for current Deadline #5493 + +- Added support in Deadline Plug-in for new versions of Harmony, in particular version 21 and 22. +- Remove review=False flag on render instance +- Add farm=True flag on render instance +- Fix is_in_tests function call in Harmony Deadline submission plugin +- Force HarmonyOpenPype.py Deadline Python plug-in to py3 +- Fix cosmetics/hound in HarmonyOpenPype.py Deadline Python plug-in + + +___ + +
+ + +
+Publisher: Fix multiselection value #5505 + +Selection of multiple instances in Publisher does not cause that all instances change all publish attributes to the same value. + + +___ + +
+ + +
+Publisher: Avoid warnings on thumbnails if source image also has alpha channel #5510 + +Avoids the following warning from `ExtractThumbnailFromSource`: +``` +// pyblish.ExtractThumbnailFromSource : oiiotool WARNING: -o : Can't save 4 channels to jpeg... saving only R,G,B +``` + + + +___ + +
+ + +
+Update ayon-python-api #5512 + +Update ayon python api and related callbacks. + + +___ + +
+ + +
+Max: Fixing the bug of falling back to use workfile for Arnold or any renderers except Redshift #5520 + +Fix the bug of falling back to use workfile for Arnold + + +___ + +
+ + +
+General: Fix Validate Publish Dir Validator #5534 + +Nonsensical "family" key was used instead of real value (as 'render' etc.) which would result in wrong translation of intermediate family names.Updated docstring. + + +___ + +
+ + +
+have the addons loading respect a custom AYON_ADDONS_DIR #5539 + +When using a custom AYON_ADDONS_DIR environment variable that variable is used in the launcher correctly and downloads and extracts addons to there, however when running Ayon does not respect this environment variable + + +___ + +
+ + +
+Deadline: files on representation cannot be single item list #5545 + +Further logic expects that single item files will be only 'string' not 'list' (eg. repre["files"] = "abc.exr" not repre["files"] = ["abc.exr"].This would cause an issue in ExtractReview later.This could happen if DL rendered single frame file with different frame value. + + +___ + +
+ + +
+Webpublisher: better encode list values for click #5546 + +Targets could be a list, original implementation pushed it as a separate items, it must be added as `--targets webpulish --targets filepublish`.`wepublish_routes` handles triggering from UI, changes in `publish_functions` handle triggering from cmd (for tests, api access). + + +___ + +
+ + +
+Houdini: Introduce imprint function for correct version in hda loader #5548 + +Resolve #5478 + + +___ + +
+ + +
+AYON: Fill entities during editorial (2) #5549 + +Fix changes made in https://github.com/ynput/OpenPype/pull/5475. + + +___ + +
+ + +
+Max: OP Data updates in Loaders #5563 + +Fix the bug on the loaders not being able to load the objects when iterating key and values with the dict.Max prefers list over the list in dict. + + +___ + +
+ + +
+Create Plugins: Better check of overriden '__init__' method #5571 + +Create plugins do not log warning messages about each create plugin because of wrong `__init__` method check. + + +___ + +
+ +### **Merged pull requests** + + +
+Tests: fix unit tests #5533 + +Fixed failing tests.Updated Unreal's validator to match removed general one which had a couple of issues fixed. + + +___ + +
+ + + + ## [3.16.4](https://github.com/ynput/OpenPype/tree/3.16.4) diff --git a/Dockerfile.centos7 b/Dockerfile.centos7 index 9217140f20..ab1d3f8253 100644 --- a/Dockerfile.centos7 +++ b/Dockerfile.centos7 @@ -109,6 +109,8 @@ RUN source $HOME/.bashrc \ RUN cp /usr/lib64/libffi* ./build/exe.linux-x86_64-3.9/lib \ && cp /usr/lib64/openssl11/libssl* ./build/exe.linux-x86_64-3.9/lib \ && cp /usr/lib64/openssl11/libcrypto* ./build/exe.linux-x86_64-3.9/lib \ + && ln -sr ./build/exe.linux-x86_64-3.9/lib/libssl.so ./build/exe.linux-x86_64-3.9/lib/libssl.1.1.so \ + && ln -sr ./build/exe.linux-x86_64-3.9/lib/libcrypto.so ./build/exe.linux-x86_64-3.9/lib/libcrypto.1.1.so \ && cp /root/.pyenv/versions/${OPENPYPE_PYTHON_VERSION}/lib/libpython* ./build/exe.linux-x86_64-3.9/lib \ && cp /usr/lib64/libxcb* ./build/exe.linux-x86_64-3.9/vendor/python/PySide2/Qt/lib diff --git a/README.md b/README.md index 6caed8061c..ce98f845e6 100644 --- a/README.md +++ b/README.md @@ -62,7 +62,7 @@ development tools like [CMake](https://cmake.org/) and [Visual Studio](https://v #### Clone repository: ```sh -git clone --recurse-submodules git@github.com:Pypeclub/OpenPype.git +git clone --recurse-submodules git@github.com:ynput/OpenPype.git ``` #### To build OpenPype: @@ -144,6 +144,10 @@ sudo ./tools/docker_build.sh centos7 If all is successful, you'll find built OpenPype in `./build/` folder. +Docker build can be also started from Windows machine, just use `./tools/docker_build.ps1` instead of shell script. + +This could be used even for building linux build (with argument `centos7` or `debian`) + #### Manual build You will need [Python >= 3.9](https://www.python.org/downloads/) and [git](https://git-scm.com/downloads). You'll also need [curl](https://curl.se) on systems that doesn't have one preinstalled. diff --git a/igniter/__init__.py b/igniter/__init__.py index 16ffb940f6..085a825860 100644 --- a/igniter/__init__.py +++ b/igniter/__init__.py @@ -34,7 +34,11 @@ def _get_qt_app(): if attr is not None: QtWidgets.QApplication.setAttribute(attr) - if hasattr(QtWidgets.QApplication, "setHighDpiScaleFactorRoundingPolicy"): + policy = os.getenv("QT_SCALE_FACTOR_ROUNDING_POLICY") + if ( + hasattr(QtWidgets.QApplication, "setHighDpiScaleFactorRoundingPolicy") + and not policy + ): QtWidgets.QApplication.setHighDpiScaleFactorRoundingPolicy( QtCore.Qt.HighDpiScaleFactorRoundingPolicy.PassThrough ) diff --git a/igniter/bootstrap_repos.py b/igniter/bootstrap_repos.py index 408764e1a8..e7b440f812 100644 --- a/igniter/bootstrap_repos.py +++ b/igniter/bootstrap_repos.py @@ -589,7 +589,7 @@ class BootstrapRepos: self.registry = OpenPypeSettingsRegistry() self.zip_filter = [".pyc", "__pycache__"] self.openpype_filter = [ - "openpype", "schema", "LICENSE" + "openpype", "LICENSE" ] # dummy progress reporter diff --git a/inno_setup.iss b/inno_setup.iss index 418bedbd4d..d9a41d22ee 100644 --- a/inno_setup.iss +++ b/inno_setup.iss @@ -36,7 +36,7 @@ WizardStyle=modern Name: "english"; MessagesFile: "compiler:Default.isl" [Tasks] -Name: "desktopicon"; Description: "{cm:CreateDesktopIcon}"; GroupDescription: "{cm:AdditionalIcons}"; Flags: unchecked +Name: "desktopicon"; Description: "{cm:CreateDesktopIcon}"; GroupDescription: "{cm:AdditionalIcons}" [InstallDelete] ; clean everything in previous installation folder @@ -53,4 +53,3 @@ Name: "{autodesktop}\{#MyAppName} {#AppVer}"; Filename: "{app}\openpype_gui.exe" [Run] Filename: "{app}\openpype_gui.exe"; Description: "{cm:LaunchProgram,OpenPype}"; Flags: nowait postinstall skipifsilent - diff --git a/openpype/client/server/conversion_utils.py b/openpype/client/server/conversion_utils.py index a6c190a0fc..f67a1ef9c4 100644 --- a/openpype/client/server/conversion_utils.py +++ b/openpype/client/server/conversion_utils.py @@ -663,10 +663,13 @@ def convert_v4_representation_to_v3(representation): if isinstance(context, six.string_types): context = json.loads(context) - if "folder" in context: - _c_folder = context.pop("folder") + if "asset" not in context and "folder" in context: + _c_folder = context["folder"] context["asset"] = _c_folder["name"] + elif "asset" in context and "folder" not in context: + context["folder"] = {"name": context["asset"]} + if "product" in context: _c_product = context.pop("product") context["family"] = _c_product["type"] @@ -959,9 +962,11 @@ def convert_create_representation_to_v4(representation, con): converted_representation["files"] = new_files context = representation["context"] - context["folder"] = { - "name": context.pop("asset", None) - } + if "folder" not in context: + context["folder"] = { + "name": context.get("asset") + } + context["product"] = { "type": context.pop("family", None), "name": context.pop("subset", None), @@ -1285,7 +1290,7 @@ def convert_update_representation_to_v4( if "context" in update_data: context = update_data["context"] - if "asset" in context: + if "folder" not in context and "asset" in context: context["folder"] = {"name": context.pop("asset")} if "family" in context or "subset" in context: diff --git a/openpype/client/server/entities.py b/openpype/client/server/entities.py index 39322627bb..3ee62a3172 100644 --- a/openpype/client/server/entities.py +++ b/openpype/client/server/entities.py @@ -422,7 +422,7 @@ def get_last_version_by_subset_name( if not subset: return None return get_last_version_by_subset_id( - project_name, subset["id"], fields=fields + project_name, subset["_id"], fields=fields ) diff --git a/openpype/hooks/pre_ocio_hook.py b/openpype/hooks/pre_ocio_hook.py index 1307ed9f76..add3a0adaf 100644 --- a/openpype/hooks/pre_ocio_hook.py +++ b/openpype/hooks/pre_ocio_hook.py @@ -45,6 +45,9 @@ class OCIOEnvHook(PreLaunchHook): if config_data: ocio_path = config_data["path"] + if self.host_name in ["nuke", "hiero"]: + ocio_path = ocio_path.replace("\\", "/") + self.log.info( f"Setting OCIO environment to config path: {ocio_path}") diff --git a/openpype/hosts/aftereffects/api/extension.zxp b/openpype/hosts/aftereffects/api/extension.zxp index 358e9740d3..933dc7dc6c 100644 Binary files a/openpype/hosts/aftereffects/api/extension.zxp and b/openpype/hosts/aftereffects/api/extension.zxp differ diff --git a/openpype/hosts/aftereffects/api/extension/CSXS/manifest.xml b/openpype/hosts/aftereffects/api/extension/CSXS/manifest.xml index 0057758320..7329a9e723 100644 --- a/openpype/hosts/aftereffects/api/extension/CSXS/manifest.xml +++ b/openpype/hosts/aftereffects/api/extension/CSXS/manifest.xml @@ -1,5 +1,5 @@ - @@ -10,22 +10,22 @@ - + - + - - + + - + - - + + - + @@ -63,7 +63,7 @@ 550 400 --> - + ./icons/iconNormal.png @@ -71,9 +71,9 @@ ./icons/iconDisabled.png ./icons/iconDarkNormal.png ./icons/iconDarkRollover.png - + - \ No newline at end of file + diff --git a/openpype/hosts/aftereffects/api/extension/jsx/hostscript.jsx b/openpype/hosts/aftereffects/api/extension/jsx/hostscript.jsx index bc443930df..c00844e637 100644 --- a/openpype/hosts/aftereffects/api/extension/jsx/hostscript.jsx +++ b/openpype/hosts/aftereffects/api/extension/jsx/hostscript.jsx @@ -215,6 +215,8 @@ function _getItem(item, comps, folders, footages){ * Refactor */ var item_type = ''; + var path = ''; + var containing_comps = []; if (item instanceof FolderItem){ item_type = 'folder'; if (!folders){ @@ -222,10 +224,18 @@ function _getItem(item, comps, folders, footages){ } } if (item instanceof FootageItem){ - item_type = 'footage'; if (!footages){ return "{}"; } + item_type = 'footage'; + if (item.file){ + path = item.file.fsName; + } + if (item.usedIn){ + for (j = 0; j < item.usedIn.length; ++j){ + containing_comps.push(item.usedIn[j].id); + } + } } if (item instanceof CompItem){ item_type = 'comp'; @@ -236,7 +246,9 @@ function _getItem(item, comps, folders, footages){ var item = {"name": item.name, "id": item.id, - "type": item_type}; + "type": item_type, + "path": path, + "containing_comps": containing_comps}; return JSON.stringify(item); } diff --git a/openpype/hosts/aftereffects/api/ws_stub.py b/openpype/hosts/aftereffects/api/ws_stub.py index f5b96fa63a..18f530e272 100644 --- a/openpype/hosts/aftereffects/api/ws_stub.py +++ b/openpype/hosts/aftereffects/api/ws_stub.py @@ -37,6 +37,9 @@ class AEItem(object): height = attr.ib(default=None) is_placeholder = attr.ib(default=False) uuid = attr.ib(default=False) + path = attr.ib(default=False) # path to FootageItem to validate + # list of composition Footage is in + containing_comps = attr.ib(factory=list) class AfterEffectsServerStub(): @@ -704,7 +707,10 @@ class AfterEffectsServerStub(): d.get("instance_id"), d.get("width"), d.get("height"), - d.get("is_placeholder")) + d.get("is_placeholder"), + d.get("uuid"), + d.get("path"), + d.get("containing_comps"),) ret.append(item) return ret diff --git a/openpype/hosts/aftereffects/plugins/create/create_render.py b/openpype/hosts/aftereffects/plugins/create/create_render.py index dcf424b44f..fbe600ae68 100644 --- a/openpype/hosts/aftereffects/plugins/create/create_render.py +++ b/openpype/hosts/aftereffects/plugins/create/create_render.py @@ -164,7 +164,7 @@ class RenderCreator(Creator): api.get_stub().rename_item(comp_id, new_comp_name) - def apply_settings(self, project_settings, system_settings): + def apply_settings(self, project_settings): plugin_settings = ( project_settings["aftereffects"]["create"]["RenderCreator"] ) diff --git a/openpype/hosts/aftereffects/plugins/load/load_file.py b/openpype/hosts/aftereffects/plugins/load/load_file.py index def7c927ab..8d52aac546 100644 --- a/openpype/hosts/aftereffects/plugins/load/load_file.py +++ b/openpype/hosts/aftereffects/plugins/load/load_file.py @@ -31,13 +31,8 @@ class FileLoader(api.AfterEffectsLoader): path = self.filepath_from_context(context) - repr_cont = context["representation"]["context"] - if "#" not in path: - frame = repr_cont.get("frame") - if frame: - padding = len(frame) - path = path.replace(frame, "#" * padding) - import_options['sequence'] = True + if len(context["representation"]["files"]) > 1: + import_options['sequence'] = True if not path: repr_id = context["representation"]["_id"] diff --git a/openpype/hosts/aftereffects/plugins/publish/collect_render.py b/openpype/hosts/aftereffects/plugins/publish/collect_render.py index aa46461915..49874d6cff 100644 --- a/openpype/hosts/aftereffects/plugins/publish/collect_render.py +++ b/openpype/hosts/aftereffects/plugins/publish/collect_render.py @@ -138,7 +138,6 @@ class CollectAERender(publish.AbstractCollectRender): fam = "render.farm" if fam not in instance.families: instance.families.append(fam) - instance.toBeRenderedOn = "deadline" instance.renderer = "aerender" instance.farm = True # to skip integrate if "review" in instance.families: diff --git a/openpype/hosts/aftereffects/plugins/publish/help/validate_footage_items.xml b/openpype/hosts/aftereffects/plugins/publish/help/validate_footage_items.xml new file mode 100644 index 0000000000..01c8966015 --- /dev/null +++ b/openpype/hosts/aftereffects/plugins/publish/help/validate_footage_items.xml @@ -0,0 +1,14 @@ + + + +Footage item missing + +## Footage item missing + + FootageItem `{name}` contains missing `{path}`. Render will not produce any frames and AE will stop react to any integration +### How to repair? + +Remove `{name}` or provide missing file. + + + diff --git a/openpype/hosts/aftereffects/plugins/publish/validate_footage_items.py b/openpype/hosts/aftereffects/plugins/publish/validate_footage_items.py new file mode 100644 index 0000000000..40a08a2c3f --- /dev/null +++ b/openpype/hosts/aftereffects/plugins/publish/validate_footage_items.py @@ -0,0 +1,49 @@ +# -*- coding: utf-8 -*- +"""Validate presence of footage items in composition +Requires: +""" +import os + +import pyblish.api + +from openpype.pipeline import ( + PublishXmlValidationError +) +from openpype.hosts.aftereffects.api import get_stub + + +class ValidateFootageItems(pyblish.api.InstancePlugin): + """ + Validates if FootageItems contained in composition exist. + + AE fails silently and doesn't render anything if footage item file is + missing. This will result in nonresponsiveness of AE UI as it expects + reaction from user, but it will not provide dialog. + This validator tries to check existence of the files. + It will not protect from missing frame in multiframes though + (as AE api doesn't provide this information and it cannot be told how many + frames should be there easily). Missing frame is replaced by placeholder. + """ + + order = pyblish.api.ValidatorOrder + label = "Validate Footage Items" + families = ["render.farm", "render.local", "render"] + hosts = ["aftereffects"] + optional = True + + def process(self, instance): + """Plugin entry point.""" + + comp_id = instance.data["comp_id"] + for footage_item in get_stub().get_items(comps=False, folders=False, + footages=True): + self.log.info(footage_item) + if comp_id not in footage_item.containing_comps: + continue + + path = footage_item.path + if path and not os.path.exists(path): + msg = f"File {path} not found." + formatting = {"name": footage_item.name, "path": path} + raise PublishXmlValidationError(self, msg, + formatting_data=formatting) diff --git a/openpype/hosts/blender/plugins/load/load_blend.py b/openpype/hosts/blender/plugins/load/load_blend.py index 99f291a5a7..fa41f4374b 100644 --- a/openpype/hosts/blender/plugins/load/load_blend.py +++ b/openpype/hosts/blender/plugins/load/load_blend.py @@ -119,7 +119,7 @@ class BlendLoader(plugin.AssetLoader): context: Full parenthood of representation to load options: Additional settings dictionary """ - libpath = self.fname + libpath = self.filepath_from_context(context) asset = context["asset"]["name"] subset = context["subset"]["name"] diff --git a/openpype/hosts/blender/plugins/load/load_camera_abc.py b/openpype/hosts/blender/plugins/load/load_camera_abc.py index e5afecff66..05d3fb764d 100644 --- a/openpype/hosts/blender/plugins/load/load_camera_abc.py +++ b/openpype/hosts/blender/plugins/load/load_camera_abc.py @@ -100,7 +100,7 @@ class AbcCameraLoader(plugin.AssetLoader): asset_group = bpy.data.objects.new(group_name, object_data=None) avalon_container.objects.link(asset_group) - objects = self._process(libpath, asset_group, group_name) + self._process(libpath, asset_group, group_name) objects = [] nodes = list(asset_group.children) diff --git a/openpype/hosts/blender/plugins/load/load_camera_fbx.py b/openpype/hosts/blender/plugins/load/load_camera_fbx.py index b9d05dda0a..3cca6e7fd3 100644 --- a/openpype/hosts/blender/plugins/load/load_camera_fbx.py +++ b/openpype/hosts/blender/plugins/load/load_camera_fbx.py @@ -103,7 +103,7 @@ class FbxCameraLoader(plugin.AssetLoader): asset_group = bpy.data.objects.new(group_name, object_data=None) avalon_container.objects.link(asset_group) - objects = self._process(libpath, asset_group, group_name) + self._process(libpath, asset_group, group_name) objects = [] nodes = list(asset_group.children) diff --git a/openpype/hosts/blender/plugins/publish/collect_review.py b/openpype/hosts/blender/plugins/publish/collect_review.py index 6459927015..3bf2e39e24 100644 --- a/openpype/hosts/blender/plugins/publish/collect_review.py +++ b/openpype/hosts/blender/plugins/publish/collect_review.py @@ -39,15 +39,11 @@ class CollectReview(pyblish.api.InstancePlugin): ] if not instance.data.get("remove"): - - task = instance.context.data["task"] - # Store focal length in `burninDataMembers` burninData = instance.data.setdefault("burninDataMembers", {}) burninData["focalLength"] = focal_length instance.data.update({ - "subset": f"{task}Review", "review_camera": camera, "frameStart": instance.context.data["frameStart"], "frameEnd": instance.context.data["frameEnd"], diff --git a/openpype/hosts/blender/plugins/publish/extract_abc.py b/openpype/hosts/blender/plugins/publish/extract_abc.py index f4babc94d3..87159e53f0 100644 --- a/openpype/hosts/blender/plugins/publish/extract_abc.py +++ b/openpype/hosts/blender/plugins/publish/extract_abc.py @@ -21,8 +21,6 @@ class ExtractABC(publish.Extractor): filename = f"{instance.name}.abc" filepath = os.path.join(stagingdir, filename) - context = bpy.context - # Perform extraction self.log.info("Performing extraction..") diff --git a/openpype/hosts/blender/plugins/publish/extract_abc_animation.py b/openpype/hosts/blender/plugins/publish/extract_abc_animation.py index e141ccaa44..44b2ba3761 100644 --- a/openpype/hosts/blender/plugins/publish/extract_abc_animation.py +++ b/openpype/hosts/blender/plugins/publish/extract_abc_animation.py @@ -20,8 +20,6 @@ class ExtractAnimationABC(publish.Extractor): filename = f"{instance.name}.abc" filepath = os.path.join(stagingdir, filename) - context = bpy.context - # Perform extraction self.log.info("Performing extraction..") diff --git a/openpype/hosts/blender/plugins/publish/extract_camera_abc.py b/openpype/hosts/blender/plugins/publish/extract_camera_abc.py index a21a59b151..036be7bf3c 100644 --- a/openpype/hosts/blender/plugins/publish/extract_camera_abc.py +++ b/openpype/hosts/blender/plugins/publish/extract_camera_abc.py @@ -21,16 +21,11 @@ class ExtractCameraABC(publish.Extractor): filename = f"{instance.name}.abc" filepath = os.path.join(stagingdir, filename) - context = bpy.context - # Perform extraction self.log.info("Performing extraction..") plugin.deselect_all() - selected = [] - active = None - asset_group = None for obj in instance: if obj.get(AVALON_PROPERTY): diff --git a/openpype/hosts/flame/plugins/load/load_clip.py b/openpype/hosts/flame/plugins/load/load_clip.py index 338833b449..ca4eab0f63 100644 --- a/openpype/hosts/flame/plugins/load/load_clip.py +++ b/openpype/hosts/flame/plugins/load/load_clip.py @@ -48,7 +48,6 @@ class LoadClip(opfapi.ClipLoader): self.fpd = fproject.current_workspace.desktop # load clip to timeline and get main variables - namespace = namespace version = context['version'] version_data = version.get("data", {}) version_name = version.get("name", None) diff --git a/openpype/hosts/flame/plugins/load/load_clip_batch.py b/openpype/hosts/flame/plugins/load/load_clip_batch.py index ca43b94ee9..1f3a017d72 100644 --- a/openpype/hosts/flame/plugins/load/load_clip_batch.py +++ b/openpype/hosts/flame/plugins/load/load_clip_batch.py @@ -45,7 +45,6 @@ class LoadClipBatch(opfapi.ClipLoader): self.batch = options.get("batch") or flame.batch # load clip to timeline and get main variables - namespace = namespace version = context['version'] version_data = version.get("data", {}) version_name = version.get("name", None) diff --git a/openpype/hosts/flame/plugins/publish/collect_timeline_instances.py b/openpype/hosts/flame/plugins/publish/collect_timeline_instances.py index 23fdf5e785..e14f960a2b 100644 --- a/openpype/hosts/flame/plugins/publish/collect_timeline_instances.py +++ b/openpype/hosts/flame/plugins/publish/collect_timeline_instances.py @@ -325,7 +325,6 @@ class CollectTimelineInstances(pyblish.api.ContextPlugin): def _create_shot_instance(self, context, clip_name, **data): master_layer = data.get("heroTrack") hierarchy_data = data.get("hierarchyData") - asset = data.get("asset") if not master_layer: return diff --git a/openpype/hosts/fusion/addon.py b/openpype/hosts/fusion/addon.py index 45683cfbde..8343f3c79d 100644 --- a/openpype/hosts/fusion/addon.py +++ b/openpype/hosts/fusion/addon.py @@ -60,8 +60,9 @@ class FusionAddon(OpenPypeModule, IHostAddon): return [] return [os.path.join(FUSION_HOST_DIR, "hooks")] - def add_implementation_envs(self, env, _app): + def add_implementation_envs(self, env, app): # Set default values if are not already set via settings + defaults = {"OPENPYPE_LOG_NO_COLORS": "Yes"} for key, value in defaults.items(): if not env.get(key): diff --git a/openpype/hosts/fusion/deploy/MenuScripts/openpype_menu.py b/openpype/hosts/fusion/deploy/MenuScripts/openpype_menu.py index 685e58d58f..1c58ee50e4 100644 --- a/openpype/hosts/fusion/deploy/MenuScripts/openpype_menu.py +++ b/openpype/hosts/fusion/deploy/MenuScripts/openpype_menu.py @@ -1,6 +1,19 @@ import os import sys +if sys.version_info < (3, 7): + # hack to handle discrepancy between distributed libraries and Python 3.6 + # mostly because wrong version of urllib3 + # TODO remove when not necessary + from openpype import PACKAGE_DIR + FUSION_HOST_DIR = os.path.join(PACKAGE_DIR, "hosts", "fusion") + + vendor_path = os.path.join(FUSION_HOST_DIR, "vendor") + if vendor_path not in sys.path: + sys.path.insert(0, vendor_path) + + print(f"Added vendorized libraries from {vendor_path}") + from openpype.lib import Logger from openpype.pipeline import ( install_host, diff --git a/openpype/hosts/fusion/deploy/Scripts/Comp/OpenPype/32bit/backgrounds_selected_to32bit.py b/openpype/hosts/fusion/deploy/Scripts/Comp/OpenPype/32bit/backgrounds_selected_to32bit.py deleted file mode 100644 index 1a0a9911ea..0000000000 --- a/openpype/hosts/fusion/deploy/Scripts/Comp/OpenPype/32bit/backgrounds_selected_to32bit.py +++ /dev/null @@ -1,16 +0,0 @@ -from openpype.hosts.fusion.api import ( - comp_lock_and_undo_chunk, - get_current_comp -) - - -def main(): - comp = get_current_comp() - """Set all selected backgrounds to 32 bit""" - with comp_lock_and_undo_chunk(comp, 'Selected Backgrounds to 32bit'): - tools = comp.GetToolList(True, "Background").values() - for tool in tools: - tool.Depth = 5 - - -main() diff --git a/openpype/hosts/fusion/deploy/Scripts/Comp/OpenPype/32bit/backgrounds_to32bit.py b/openpype/hosts/fusion/deploy/Scripts/Comp/OpenPype/32bit/backgrounds_to32bit.py deleted file mode 100644 index c2eea505e5..0000000000 --- a/openpype/hosts/fusion/deploy/Scripts/Comp/OpenPype/32bit/backgrounds_to32bit.py +++ /dev/null @@ -1,16 +0,0 @@ -from openpype.hosts.fusion.api import ( - comp_lock_and_undo_chunk, - get_current_comp -) - - -def main(): - comp = get_current_comp() - """Set all backgrounds to 32 bit""" - with comp_lock_and_undo_chunk(comp, 'Backgrounds to 32bit'): - tools = comp.GetToolList(False, "Background").values() - for tool in tools: - tool.Depth = 5 - - -main() diff --git a/openpype/hosts/fusion/deploy/Scripts/Comp/OpenPype/32bit/loaders_selected_to32bit.py b/openpype/hosts/fusion/deploy/Scripts/Comp/OpenPype/32bit/loaders_selected_to32bit.py deleted file mode 100644 index 2118767f4d..0000000000 --- a/openpype/hosts/fusion/deploy/Scripts/Comp/OpenPype/32bit/loaders_selected_to32bit.py +++ /dev/null @@ -1,16 +0,0 @@ -from openpype.hosts.fusion.api import ( - comp_lock_and_undo_chunk, - get_current_comp -) - - -def main(): - comp = get_current_comp() - """Set all selected loaders to 32 bit""" - with comp_lock_and_undo_chunk(comp, 'Selected Loaders to 32bit'): - tools = comp.GetToolList(True, "Loader").values() - for tool in tools: - tool.Depth = 5 - - -main() diff --git a/openpype/hosts/fusion/deploy/Scripts/Comp/OpenPype/32bit/loaders_to32bit.py b/openpype/hosts/fusion/deploy/Scripts/Comp/OpenPype/32bit/loaders_to32bit.py deleted file mode 100644 index 7dd1f66a5e..0000000000 --- a/openpype/hosts/fusion/deploy/Scripts/Comp/OpenPype/32bit/loaders_to32bit.py +++ /dev/null @@ -1,16 +0,0 @@ -from openpype.hosts.fusion.api import ( - comp_lock_and_undo_chunk, - get_current_comp -) - - -def main(): - comp = get_current_comp() - """Set all loaders to 32 bit""" - with comp_lock_and_undo_chunk(comp, 'Loaders to 32bit'): - tools = comp.GetToolList(False, "Loader").values() - for tool in tools: - tool.Depth = 5 - - -main() diff --git a/openpype/hosts/fusion/deploy/Scripts/Comp/OpenPype/switch_ui.py b/openpype/hosts/fusion/deploy/Scripts/Comp/OpenPype/switch_ui.py deleted file mode 100644 index 87322235f5..0000000000 --- a/openpype/hosts/fusion/deploy/Scripts/Comp/OpenPype/switch_ui.py +++ /dev/null @@ -1,200 +0,0 @@ -import os -import sys -import glob -import logging - -from qtpy import QtWidgets, QtCore - -import qtawesome as qta - -from openpype.client import get_assets -from openpype import style -from openpype.pipeline import ( - install_host, - get_current_project_name, -) -from openpype.hosts.fusion import api -from openpype.pipeline.context_tools import get_workdir_from_session - -log = logging.getLogger("Fusion Switch Shot") - - -class App(QtWidgets.QWidget): - - def __init__(self, parent=None): - - ################################################ - # |---------------------| |------------------| # - # |Comp | |Asset | # - # |[..][ v]| |[ v]| # - # |---------------------| |------------------| # - # | Update existing comp [ ] | # - # |------------------------------------------| # - # | Switch | # - # |------------------------------------------| # - ################################################ - - QtWidgets.QWidget.__init__(self, parent) - - layout = QtWidgets.QVBoxLayout() - - # Comp related input - comp_hlayout = QtWidgets.QHBoxLayout() - comp_label = QtWidgets.QLabel("Comp file") - comp_label.setFixedWidth(50) - comp_box = QtWidgets.QComboBox() - - button_icon = qta.icon("fa.folder", color="white") - open_from_dir = QtWidgets.QPushButton() - open_from_dir.setIcon(button_icon) - - comp_box.setFixedHeight(25) - open_from_dir.setFixedWidth(25) - open_from_dir.setFixedHeight(25) - - comp_hlayout.addWidget(comp_label) - comp_hlayout.addWidget(comp_box) - comp_hlayout.addWidget(open_from_dir) - - # Asset related input - asset_hlayout = QtWidgets.QHBoxLayout() - asset_label = QtWidgets.QLabel("Shot") - asset_label.setFixedWidth(50) - - asset_box = QtWidgets.QComboBox() - asset_box.setLineEdit(QtWidgets.QLineEdit()) - asset_box.setFixedHeight(25) - - refresh_icon = qta.icon("fa.refresh", color="white") - refresh_btn = QtWidgets.QPushButton() - refresh_btn.setIcon(refresh_icon) - - asset_box.setFixedHeight(25) - refresh_btn.setFixedWidth(25) - refresh_btn.setFixedHeight(25) - - asset_hlayout.addWidget(asset_label) - asset_hlayout.addWidget(asset_box) - asset_hlayout.addWidget(refresh_btn) - - # Options - options = QtWidgets.QHBoxLayout() - options.setAlignment(QtCore.Qt.AlignLeft) - - current_comp_check = QtWidgets.QCheckBox() - current_comp_check.setChecked(True) - current_comp_label = QtWidgets.QLabel("Use current comp") - - options.addWidget(current_comp_label) - options.addWidget(current_comp_check) - - accept_btn = QtWidgets.QPushButton("Switch") - - layout.addLayout(options) - layout.addLayout(comp_hlayout) - layout.addLayout(asset_hlayout) - layout.addWidget(accept_btn) - - self._open_from_dir = open_from_dir - self._comps = comp_box - self._assets = asset_box - self._use_current = current_comp_check - self._accept_btn = accept_btn - self._refresh_btn = refresh_btn - - self.setWindowTitle("Fusion Switch Shot") - self.setLayout(layout) - - self.resize(260, 140) - self.setMinimumWidth(260) - self.setFixedHeight(140) - - self.connections() - - # Update ui to correct state - self._on_use_current_comp() - self._refresh() - - def connections(self): - self._use_current.clicked.connect(self._on_use_current_comp) - self._open_from_dir.clicked.connect(self._on_open_from_dir) - self._refresh_btn.clicked.connect(self._refresh) - self._accept_btn.clicked.connect(self._on_switch) - - def _on_use_current_comp(self): - state = self._use_current.isChecked() - self._open_from_dir.setEnabled(not state) - self._comps.setEnabled(not state) - - def _on_open_from_dir(self): - - start_dir = get_workdir_from_session() - comp_file, _ = QtWidgets.QFileDialog.getOpenFileName( - self, "Choose comp", start_dir) - - if not comp_file: - return - - # Create completer - self.populate_comp_box([comp_file]) - self._refresh() - - def _refresh(self): - # Clear any existing items - self._assets.clear() - - asset_names = self.collect_asset_names() - completer = QtWidgets.QCompleter(asset_names) - - self._assets.setCompleter(completer) - self._assets.addItems(asset_names) - - def _on_switch(self): - - if not self._use_current.isChecked(): - file_name = self._comps.itemData(self._comps.currentIndex()) - else: - comp = api.get_current_comp() - file_name = comp.GetAttrs("COMPS_FileName") - - asset = self._assets.currentText() - - import colorbleed.scripts.fusion_switch_shot as switch_shot - switch_shot.switch(asset_name=asset, filepath=file_name, new=True) - - def collect_slap_comps(self, directory): - items = glob.glob("{}/*.comp".format(directory)) - return items - - def collect_asset_names(self): - project_name = get_current_project_name() - asset_docs = get_assets(project_name, fields=["name"]) - asset_names = { - asset_doc["name"] - for asset_doc in asset_docs - } - return list(asset_names) - - def populate_comp_box(self, files): - """Ensure we display the filename only but the path is stored as well - - Args: - files (list): list of full file path [path/to/item/item.ext,] - - Returns: - None - """ - - for f in files: - filename = os.path.basename(f) - self._comps.addItem(filename, userData=f) - - -if __name__ == '__main__': - install_host(api) - - app = QtWidgets.QApplication(sys.argv) - window = App() - window.setStyleSheet(style.load_stylesheet()) - window.show() - sys.exit(app.exec_()) diff --git a/openpype/hosts/fusion/deploy/Scripts/Comp/OpenPype/update_loader_ranges.py b/openpype/hosts/fusion/deploy/Scripts/Comp/OpenPype/update_loader_ranges.py deleted file mode 100644 index 3d2d1ecfa6..0000000000 --- a/openpype/hosts/fusion/deploy/Scripts/Comp/OpenPype/update_loader_ranges.py +++ /dev/null @@ -1,40 +0,0 @@ -"""Forces Fusion to 'retrigger' the Loader to update. - -Warning: - This might change settings like 'Reverse', 'Loop', trims and other - settings of the Loader. So use this at your own risk. - -""" -from openpype.hosts.fusion.api.pipeline import ( - get_current_comp, - comp_lock_and_undo_chunk -) - - -def update_loader_ranges(): - comp = get_current_comp() - with comp_lock_and_undo_chunk(comp, "Reload clip time ranges"): - tools = comp.GetToolList(True, "Loader").values() - for tool in tools: - - # Get tool attributes - tool_a = tool.GetAttrs() - clipTable = tool_a['TOOLST_Clip_Name'] - altclipTable = tool_a['TOOLST_AltClip_Name'] - startTime = tool_a['TOOLNT_Clip_Start'] - old_global_in = tool.GlobalIn[comp.CurrentTime] - - # Reapply - for index, _ in clipTable.items(): - time = startTime[index] - tool.Clip[time] = tool.Clip[time] - - for index, _ in altclipTable.items(): - time = startTime[index] - tool.ProxyFilename[time] = tool.ProxyFilename[time] - - tool.GlobalIn[comp.CurrentTime] = old_global_in - - -if __name__ == '__main__': - update_loader_ranges() diff --git a/openpype/hosts/fusion/deploy/fusion_shared.prefs b/openpype/hosts/fusion/deploy/fusion_shared.prefs index b379ea7c66..93b08aa886 100644 --- a/openpype/hosts/fusion/deploy/fusion_shared.prefs +++ b/openpype/hosts/fusion/deploy/fusion_shared.prefs @@ -5,7 +5,7 @@ Global = { Map = { ["OpenPype:"] = "$(OPENPYPE_FUSION)/deploy", ["Config:"] = "UserPaths:Config;OpenPype:Config", - ["Scripts:"] = "UserPaths:Scripts;Reactor:System/Scripts;OpenPype:Scripts", + ["Scripts:"] = "UserPaths:Scripts;Reactor:System/Scripts", }, }, Script = { diff --git a/openpype/hosts/fusion/plugins/create/create_saver.py b/openpype/hosts/fusion/plugins/create/create_saver.py index 04898d0a45..39edca4de3 100644 --- a/openpype/hosts/fusion/plugins/create/create_saver.py +++ b/openpype/hosts/fusion/plugins/create/create_saver.py @@ -30,10 +30,6 @@ class CreateSaver(NewCreator): instance_attributes = [ "reviewable" ] - default_variants = [ - "Main", - "Mask" - ] # TODO: This should be renamed together with Nuke so it is aligned temp_rendering_path_template = ( @@ -250,11 +246,7 @@ class CreateSaver(NewCreator): label="Review", ) - def apply_settings( - self, - project_settings, - system_settings - ): + def apply_settings(self, project_settings): """Method called on initialization of plugin to apply settings.""" # plugin settings diff --git a/openpype/hosts/fusion/plugins/publish/collect_instances.py b/openpype/hosts/fusion/plugins/publish/collect_instances.py index 6016baa2a9..4d6da79b77 100644 --- a/openpype/hosts/fusion/plugins/publish/collect_instances.py +++ b/openpype/hosts/fusion/plugins/publish/collect_instances.py @@ -85,5 +85,5 @@ class CollectInstanceData(pyblish.api.InstancePlugin): # Add review family if the instance is marked as 'review' # This could be done through a 'review' Creator attribute. if instance.data.get("review", False): - self.log.info("Adding review family..") + self.log.debug("Adding review family..") instance.data["families"].append("review") diff --git a/openpype/hosts/fusion/plugins/publish/collect_render.py b/openpype/hosts/fusion/plugins/publish/collect_render.py index a20a142701..341f3f191a 100644 --- a/openpype/hosts/fusion/plugins/publish/collect_render.py +++ b/openpype/hosts/fusion/plugins/publish/collect_render.py @@ -108,7 +108,6 @@ class CollectFusionRender( fam = "render.farm" if fam not in instance.families: instance.families.append(fam) - instance.toBeRenderedOn = "deadline" instance.farm = True # to skip integrate if "review" in instance.families: # to skip ExtractReview locally diff --git a/openpype/hosts/fusion/vendor/attr/__init__.py b/openpype/hosts/fusion/vendor/attr/__init__.py new file mode 100644 index 0000000000..b1ce7fe248 --- /dev/null +++ b/openpype/hosts/fusion/vendor/attr/__init__.py @@ -0,0 +1,78 @@ +from __future__ import absolute_import, division, print_function + +import sys + +from functools import partial + +from . import converters, exceptions, filters, setters, validators +from ._cmp import cmp_using +from ._config import get_run_validators, set_run_validators +from ._funcs import asdict, assoc, astuple, evolve, has, resolve_types +from ._make import ( + NOTHING, + Attribute, + Factory, + attrib, + attrs, + fields, + fields_dict, + make_class, + validate, +) +from ._version_info import VersionInfo + + +__version__ = "21.2.0" +__version_info__ = VersionInfo._from_version_string(__version__) + +__title__ = "attrs" +__description__ = "Classes Without Boilerplate" +__url__ = "https://www.attrs.org/" +__uri__ = __url__ +__doc__ = __description__ + " <" + __uri__ + ">" + +__author__ = "Hynek Schlawack" +__email__ = "hs@ox.cx" + +__license__ = "MIT" +__copyright__ = "Copyright (c) 2015 Hynek Schlawack" + + +s = attributes = attrs +ib = attr = attrib +dataclass = partial(attrs, auto_attribs=True) # happy Easter ;) + +__all__ = [ + "Attribute", + "Factory", + "NOTHING", + "asdict", + "assoc", + "astuple", + "attr", + "attrib", + "attributes", + "attrs", + "cmp_using", + "converters", + "evolve", + "exceptions", + "fields", + "fields_dict", + "filters", + "get_run_validators", + "has", + "ib", + "make_class", + "resolve_types", + "s", + "set_run_validators", + "setters", + "validate", + "validators", +] + +if sys.version_info[:2] >= (3, 6): + from ._next_gen import define, field, frozen, mutable + + __all__.extend((define, field, frozen, mutable)) diff --git a/openpype/hosts/fusion/vendor/attr/__init__.pyi b/openpype/hosts/fusion/vendor/attr/__init__.pyi new file mode 100644 index 0000000000..3503b073b4 --- /dev/null +++ b/openpype/hosts/fusion/vendor/attr/__init__.pyi @@ -0,0 +1,475 @@ +import sys + +from typing import ( + Any, + Callable, + Dict, + Generic, + List, + Mapping, + Optional, + Sequence, + Tuple, + Type, + TypeVar, + Union, + overload, +) + +# `import X as X` is required to make these public +from . import converters as converters +from . import exceptions as exceptions +from . import filters as filters +from . import setters as setters +from . import validators as validators +from ._version_info import VersionInfo + + +__version__: str +__version_info__: VersionInfo +__title__: str +__description__: str +__url__: str +__uri__: str +__author__: str +__email__: str +__license__: str +__copyright__: str + +_T = TypeVar("_T") +_C = TypeVar("_C", bound=type) + +_EqOrderType = Union[bool, Callable[[Any], Any]] +_ValidatorType = Callable[[Any, Attribute[_T], _T], Any] +_ConverterType = Callable[[Any], Any] +_FilterType = Callable[[Attribute[_T], _T], bool] +_ReprType = Callable[[Any], str] +_ReprArgType = Union[bool, _ReprType] +_OnSetAttrType = Callable[[Any, Attribute[Any], Any], Any] +_OnSetAttrArgType = Union[ + _OnSetAttrType, List[_OnSetAttrType], setters._NoOpType +] +_FieldTransformer = Callable[[type, List[Attribute[Any]]], List[Attribute[Any]]] +# FIXME: in reality, if multiple validators are passed they must be in a list +# or tuple, but those are invariant and so would prevent subtypes of +# _ValidatorType from working when passed in a list or tuple. +_ValidatorArgType = Union[_ValidatorType[_T], Sequence[_ValidatorType[_T]]] + +# _make -- + +NOTHING: object + +# NOTE: Factory lies about its return type to make this possible: +# `x: List[int] # = Factory(list)` +# Work around mypy issue #4554 in the common case by using an overload. +if sys.version_info >= (3, 8): + from typing import Literal + + @overload + def Factory(factory: Callable[[], _T]) -> _T: ... + @overload + def Factory( + factory: Callable[[Any], _T], + takes_self: Literal[True], + ) -> _T: ... + @overload + def Factory( + factory: Callable[[], _T], + takes_self: Literal[False], + ) -> _T: ... +else: + @overload + def Factory(factory: Callable[[], _T]) -> _T: ... + @overload + def Factory( + factory: Union[Callable[[Any], _T], Callable[[], _T]], + takes_self: bool = ..., + ) -> _T: ... + +# Static type inference support via __dataclass_transform__ implemented as per: +# https://github.com/microsoft/pyright/blob/1.1.135/specs/dataclass_transforms.md +# This annotation must be applied to all overloads of "define" and "attrs" +# +# NOTE: This is a typing construct and does not exist at runtime. Extensions +# wrapping attrs decorators should declare a separate __dataclass_transform__ +# signature in the extension module using the specification linked above to +# provide pyright support. +def __dataclass_transform__( + *, + eq_default: bool = True, + order_default: bool = False, + kw_only_default: bool = False, + field_descriptors: Tuple[Union[type, Callable[..., Any]], ...] = (()), +) -> Callable[[_T], _T]: ... + +class Attribute(Generic[_T]): + name: str + default: Optional[_T] + validator: Optional[_ValidatorType[_T]] + repr: _ReprArgType + cmp: _EqOrderType + eq: _EqOrderType + order: _EqOrderType + hash: Optional[bool] + init: bool + converter: Optional[_ConverterType] + metadata: Dict[Any, Any] + type: Optional[Type[_T]] + kw_only: bool + on_setattr: _OnSetAttrType + + def evolve(self, **changes: Any) -> "Attribute[Any]": ... + +# NOTE: We had several choices for the annotation to use for type arg: +# 1) Type[_T] +# - Pros: Handles simple cases correctly +# - Cons: Might produce less informative errors in the case of conflicting +# TypeVars e.g. `attr.ib(default='bad', type=int)` +# 2) Callable[..., _T] +# - Pros: Better error messages than #1 for conflicting TypeVars +# - Cons: Terrible error messages for validator checks. +# e.g. attr.ib(type=int, validator=validate_str) +# -> error: Cannot infer function type argument +# 3) type (and do all of the work in the mypy plugin) +# - Pros: Simple here, and we could customize the plugin with our own errors. +# - Cons: Would need to write mypy plugin code to handle all the cases. +# We chose option #1. + +# `attr` lies about its return type to make the following possible: +# attr() -> Any +# attr(8) -> int +# attr(validator=) -> Whatever the callable expects. +# This makes this type of assignments possible: +# x: int = attr(8) +# +# This form catches explicit None or no default but with no other arguments +# returns Any. +@overload +def attrib( + default: None = ..., + validator: None = ..., + repr: _ReprArgType = ..., + cmp: Optional[_EqOrderType] = ..., + hash: Optional[bool] = ..., + init: bool = ..., + metadata: Optional[Mapping[Any, Any]] = ..., + type: None = ..., + converter: None = ..., + factory: None = ..., + kw_only: bool = ..., + eq: Optional[_EqOrderType] = ..., + order: Optional[_EqOrderType] = ..., + on_setattr: Optional[_OnSetAttrArgType] = ..., +) -> Any: ... + +# This form catches an explicit None or no default and infers the type from the +# other arguments. +@overload +def attrib( + default: None = ..., + validator: Optional[_ValidatorArgType[_T]] = ..., + repr: _ReprArgType = ..., + cmp: Optional[_EqOrderType] = ..., + hash: Optional[bool] = ..., + init: bool = ..., + metadata: Optional[Mapping[Any, Any]] = ..., + type: Optional[Type[_T]] = ..., + converter: Optional[_ConverterType] = ..., + factory: Optional[Callable[[], _T]] = ..., + kw_only: bool = ..., + eq: Optional[_EqOrderType] = ..., + order: Optional[_EqOrderType] = ..., + on_setattr: Optional[_OnSetAttrArgType] = ..., +) -> _T: ... + +# This form catches an explicit default argument. +@overload +def attrib( + default: _T, + validator: Optional[_ValidatorArgType[_T]] = ..., + repr: _ReprArgType = ..., + cmp: Optional[_EqOrderType] = ..., + hash: Optional[bool] = ..., + init: bool = ..., + metadata: Optional[Mapping[Any, Any]] = ..., + type: Optional[Type[_T]] = ..., + converter: Optional[_ConverterType] = ..., + factory: Optional[Callable[[], _T]] = ..., + kw_only: bool = ..., + eq: Optional[_EqOrderType] = ..., + order: Optional[_EqOrderType] = ..., + on_setattr: Optional[_OnSetAttrArgType] = ..., +) -> _T: ... + +# This form covers type=non-Type: e.g. forward references (str), Any +@overload +def attrib( + default: Optional[_T] = ..., + validator: Optional[_ValidatorArgType[_T]] = ..., + repr: _ReprArgType = ..., + cmp: Optional[_EqOrderType] = ..., + hash: Optional[bool] = ..., + init: bool = ..., + metadata: Optional[Mapping[Any, Any]] = ..., + type: object = ..., + converter: Optional[_ConverterType] = ..., + factory: Optional[Callable[[], _T]] = ..., + kw_only: bool = ..., + eq: Optional[_EqOrderType] = ..., + order: Optional[_EqOrderType] = ..., + on_setattr: Optional[_OnSetAttrArgType] = ..., +) -> Any: ... +@overload +def field( + *, + default: None = ..., + validator: None = ..., + repr: _ReprArgType = ..., + hash: Optional[bool] = ..., + init: bool = ..., + metadata: Optional[Mapping[Any, Any]] = ..., + converter: None = ..., + factory: None = ..., + kw_only: bool = ..., + eq: Optional[bool] = ..., + order: Optional[bool] = ..., + on_setattr: Optional[_OnSetAttrArgType] = ..., +) -> Any: ... + +# This form catches an explicit None or no default and infers the type from the +# other arguments. +@overload +def field( + *, + default: None = ..., + validator: Optional[_ValidatorArgType[_T]] = ..., + repr: _ReprArgType = ..., + hash: Optional[bool] = ..., + init: bool = ..., + metadata: Optional[Mapping[Any, Any]] = ..., + converter: Optional[_ConverterType] = ..., + factory: Optional[Callable[[], _T]] = ..., + kw_only: bool = ..., + eq: Optional[_EqOrderType] = ..., + order: Optional[_EqOrderType] = ..., + on_setattr: Optional[_OnSetAttrArgType] = ..., +) -> _T: ... + +# This form catches an explicit default argument. +@overload +def field( + *, + default: _T, + validator: Optional[_ValidatorArgType[_T]] = ..., + repr: _ReprArgType = ..., + hash: Optional[bool] = ..., + init: bool = ..., + metadata: Optional[Mapping[Any, Any]] = ..., + converter: Optional[_ConverterType] = ..., + factory: Optional[Callable[[], _T]] = ..., + kw_only: bool = ..., + eq: Optional[_EqOrderType] = ..., + order: Optional[_EqOrderType] = ..., + on_setattr: Optional[_OnSetAttrArgType] = ..., +) -> _T: ... + +# This form covers type=non-Type: e.g. forward references (str), Any +@overload +def field( + *, + default: Optional[_T] = ..., + validator: Optional[_ValidatorArgType[_T]] = ..., + repr: _ReprArgType = ..., + hash: Optional[bool] = ..., + init: bool = ..., + metadata: Optional[Mapping[Any, Any]] = ..., + converter: Optional[_ConverterType] = ..., + factory: Optional[Callable[[], _T]] = ..., + kw_only: bool = ..., + eq: Optional[_EqOrderType] = ..., + order: Optional[_EqOrderType] = ..., + on_setattr: Optional[_OnSetAttrArgType] = ..., +) -> Any: ... +@overload +@__dataclass_transform__(order_default=True, field_descriptors=(attrib, field)) +def attrs( + maybe_cls: _C, + these: Optional[Dict[str, Any]] = ..., + repr_ns: Optional[str] = ..., + repr: bool = ..., + cmp: Optional[_EqOrderType] = ..., + hash: Optional[bool] = ..., + init: bool = ..., + slots: bool = ..., + frozen: bool = ..., + weakref_slot: bool = ..., + str: bool = ..., + auto_attribs: bool = ..., + kw_only: bool = ..., + cache_hash: bool = ..., + auto_exc: bool = ..., + eq: Optional[_EqOrderType] = ..., + order: Optional[_EqOrderType] = ..., + auto_detect: bool = ..., + collect_by_mro: bool = ..., + getstate_setstate: Optional[bool] = ..., + on_setattr: Optional[_OnSetAttrArgType] = ..., + field_transformer: Optional[_FieldTransformer] = ..., +) -> _C: ... +@overload +@__dataclass_transform__(order_default=True, field_descriptors=(attrib, field)) +def attrs( + maybe_cls: None = ..., + these: Optional[Dict[str, Any]] = ..., + repr_ns: Optional[str] = ..., + repr: bool = ..., + cmp: Optional[_EqOrderType] = ..., + hash: Optional[bool] = ..., + init: bool = ..., + slots: bool = ..., + frozen: bool = ..., + weakref_slot: bool = ..., + str: bool = ..., + auto_attribs: bool = ..., + kw_only: bool = ..., + cache_hash: bool = ..., + auto_exc: bool = ..., + eq: Optional[_EqOrderType] = ..., + order: Optional[_EqOrderType] = ..., + auto_detect: bool = ..., + collect_by_mro: bool = ..., + getstate_setstate: Optional[bool] = ..., + on_setattr: Optional[_OnSetAttrArgType] = ..., + field_transformer: Optional[_FieldTransformer] = ..., +) -> Callable[[_C], _C]: ... +@overload +@__dataclass_transform__(field_descriptors=(attrib, field)) +def define( + maybe_cls: _C, + *, + these: Optional[Dict[str, Any]] = ..., + repr: bool = ..., + hash: Optional[bool] = ..., + init: bool = ..., + slots: bool = ..., + frozen: bool = ..., + weakref_slot: bool = ..., + str: bool = ..., + auto_attribs: bool = ..., + kw_only: bool = ..., + cache_hash: bool = ..., + auto_exc: bool = ..., + eq: Optional[bool] = ..., + order: Optional[bool] = ..., + auto_detect: bool = ..., + getstate_setstate: Optional[bool] = ..., + on_setattr: Optional[_OnSetAttrArgType] = ..., + field_transformer: Optional[_FieldTransformer] = ..., +) -> _C: ... +@overload +@__dataclass_transform__(field_descriptors=(attrib, field)) +def define( + maybe_cls: None = ..., + *, + these: Optional[Dict[str, Any]] = ..., + repr: bool = ..., + hash: Optional[bool] = ..., + init: bool = ..., + slots: bool = ..., + frozen: bool = ..., + weakref_slot: bool = ..., + str: bool = ..., + auto_attribs: bool = ..., + kw_only: bool = ..., + cache_hash: bool = ..., + auto_exc: bool = ..., + eq: Optional[bool] = ..., + order: Optional[bool] = ..., + auto_detect: bool = ..., + getstate_setstate: Optional[bool] = ..., + on_setattr: Optional[_OnSetAttrArgType] = ..., + field_transformer: Optional[_FieldTransformer] = ..., +) -> Callable[[_C], _C]: ... + +mutable = define +frozen = define # they differ only in their defaults + +# TODO: add support for returning NamedTuple from the mypy plugin +class _Fields(Tuple[Attribute[Any], ...]): + def __getattr__(self, name: str) -> Attribute[Any]: ... + +def fields(cls: type) -> _Fields: ... +def fields_dict(cls: type) -> Dict[str, Attribute[Any]]: ... +def validate(inst: Any) -> None: ... +def resolve_types( + cls: _C, + globalns: Optional[Dict[str, Any]] = ..., + localns: Optional[Dict[str, Any]] = ..., + attribs: Optional[List[Attribute[Any]]] = ..., +) -> _C: ... + +# TODO: add support for returning a proper attrs class from the mypy plugin +# we use Any instead of _CountingAttr so that e.g. `make_class('Foo', +# [attr.ib()])` is valid +def make_class( + name: str, + attrs: Union[List[str], Tuple[str, ...], Dict[str, Any]], + bases: Tuple[type, ...] = ..., + repr_ns: Optional[str] = ..., + repr: bool = ..., + cmp: Optional[_EqOrderType] = ..., + hash: Optional[bool] = ..., + init: bool = ..., + slots: bool = ..., + frozen: bool = ..., + weakref_slot: bool = ..., + str: bool = ..., + auto_attribs: bool = ..., + kw_only: bool = ..., + cache_hash: bool = ..., + auto_exc: bool = ..., + eq: Optional[_EqOrderType] = ..., + order: Optional[_EqOrderType] = ..., + collect_by_mro: bool = ..., + on_setattr: Optional[_OnSetAttrArgType] = ..., + field_transformer: Optional[_FieldTransformer] = ..., +) -> type: ... + +# _funcs -- + +# TODO: add support for returning TypedDict from the mypy plugin +# FIXME: asdict/astuple do not honor their factory args. Waiting on one of +# these: +# https://github.com/python/mypy/issues/4236 +# https://github.com/python/typing/issues/253 +def asdict( + inst: Any, + recurse: bool = ..., + filter: Optional[_FilterType[Any]] = ..., + dict_factory: Type[Mapping[Any, Any]] = ..., + retain_collection_types: bool = ..., + value_serializer: Optional[Callable[[type, Attribute[Any], Any], Any]] = ..., +) -> Dict[str, Any]: ... + +# TODO: add support for returning NamedTuple from the mypy plugin +def astuple( + inst: Any, + recurse: bool = ..., + filter: Optional[_FilterType[Any]] = ..., + tuple_factory: Type[Sequence[Any]] = ..., + retain_collection_types: bool = ..., +) -> Tuple[Any, ...]: ... +def has(cls: type) -> bool: ... +def assoc(inst: _T, **changes: Any) -> _T: ... +def evolve(inst: _T, **changes: Any) -> _T: ... + +# _config -- + +def set_run_validators(run: bool) -> None: ... +def get_run_validators() -> bool: ... + +# aliases -- + +s = attributes = attrs +ib = attr = attrib +dataclass = attrs # Technically, partial(attrs, auto_attribs=True) ;) diff --git a/openpype/hosts/fusion/vendor/attr/_cmp.py b/openpype/hosts/fusion/vendor/attr/_cmp.py new file mode 100644 index 0000000000..b747b603f1 --- /dev/null +++ b/openpype/hosts/fusion/vendor/attr/_cmp.py @@ -0,0 +1,152 @@ +from __future__ import absolute_import, division, print_function + +import functools + +from ._compat import new_class +from ._make import _make_ne + + +_operation_names = {"eq": "==", "lt": "<", "le": "<=", "gt": ">", "ge": ">="} + + +def cmp_using( + eq=None, + lt=None, + le=None, + gt=None, + ge=None, + require_same_type=True, + class_name="Comparable", +): + """ + Create a class that can be passed into `attr.ib`'s ``eq``, ``order``, and + ``cmp`` arguments to customize field comparison. + + The resulting class will have a full set of ordering methods if + at least one of ``{lt, le, gt, ge}`` and ``eq`` are provided. + + :param Optional[callable] eq: `callable` used to evaluate equality + of two objects. + :param Optional[callable] lt: `callable` used to evaluate whether + one object is less than another object. + :param Optional[callable] le: `callable` used to evaluate whether + one object is less than or equal to another object. + :param Optional[callable] gt: `callable` used to evaluate whether + one object is greater than another object. + :param Optional[callable] ge: `callable` used to evaluate whether + one object is greater than or equal to another object. + + :param bool require_same_type: When `True`, equality and ordering methods + will return `NotImplemented` if objects are not of the same type. + + :param Optional[str] class_name: Name of class. Defaults to 'Comparable'. + + See `comparison` for more details. + + .. versionadded:: 21.1.0 + """ + + body = { + "__slots__": ["value"], + "__init__": _make_init(), + "_requirements": [], + "_is_comparable_to": _is_comparable_to, + } + + # Add operations. + num_order_functions = 0 + has_eq_function = False + + if eq is not None: + has_eq_function = True + body["__eq__"] = _make_operator("eq", eq) + body["__ne__"] = _make_ne() + + if lt is not None: + num_order_functions += 1 + body["__lt__"] = _make_operator("lt", lt) + + if le is not None: + num_order_functions += 1 + body["__le__"] = _make_operator("le", le) + + if gt is not None: + num_order_functions += 1 + body["__gt__"] = _make_operator("gt", gt) + + if ge is not None: + num_order_functions += 1 + body["__ge__"] = _make_operator("ge", ge) + + type_ = new_class(class_name, (object,), {}, lambda ns: ns.update(body)) + + # Add same type requirement. + if require_same_type: + type_._requirements.append(_check_same_type) + + # Add total ordering if at least one operation was defined. + if 0 < num_order_functions < 4: + if not has_eq_function: + # functools.total_ordering requires __eq__ to be defined, + # so raise early error here to keep a nice stack. + raise ValueError( + "eq must be define is order to complete ordering from " + "lt, le, gt, ge." + ) + type_ = functools.total_ordering(type_) + + return type_ + + +def _make_init(): + """ + Create __init__ method. + """ + + def __init__(self, value): + """ + Initialize object with *value*. + """ + self.value = value + + return __init__ + + +def _make_operator(name, func): + """ + Create operator method. + """ + + def method(self, other): + if not self._is_comparable_to(other): + return NotImplemented + + result = func(self.value, other.value) + if result is NotImplemented: + return NotImplemented + + return result + + method.__name__ = "__%s__" % (name,) + method.__doc__ = "Return a %s b. Computed by attrs." % ( + _operation_names[name], + ) + + return method + + +def _is_comparable_to(self, other): + """ + Check whether `other` is comparable to `self`. + """ + for func in self._requirements: + if not func(self, other): + return False + return True + + +def _check_same_type(self, other): + """ + Return True if *self* and *other* are of the same type, False otherwise. + """ + return other.value.__class__ is self.value.__class__ diff --git a/openpype/hosts/fusion/vendor/attr/_cmp.pyi b/openpype/hosts/fusion/vendor/attr/_cmp.pyi new file mode 100644 index 0000000000..7093550f0f --- /dev/null +++ b/openpype/hosts/fusion/vendor/attr/_cmp.pyi @@ -0,0 +1,14 @@ +from typing import Type + +from . import _CompareWithType + + +def cmp_using( + eq: Optional[_CompareWithType], + lt: Optional[_CompareWithType], + le: Optional[_CompareWithType], + gt: Optional[_CompareWithType], + ge: Optional[_CompareWithType], + require_same_type: bool, + class_name: str, +) -> Type: ... diff --git a/openpype/hosts/fusion/vendor/attr/_compat.py b/openpype/hosts/fusion/vendor/attr/_compat.py new file mode 100644 index 0000000000..6939f338da --- /dev/null +++ b/openpype/hosts/fusion/vendor/attr/_compat.py @@ -0,0 +1,242 @@ +from __future__ import absolute_import, division, print_function + +import platform +import sys +import types +import warnings + + +PY2 = sys.version_info[0] == 2 +PYPY = platform.python_implementation() == "PyPy" + + +if PYPY or sys.version_info[:2] >= (3, 6): + ordered_dict = dict +else: + from collections import OrderedDict + + ordered_dict = OrderedDict + + +if PY2: + from collections import Mapping, Sequence + + from UserDict import IterableUserDict + + # We 'bundle' isclass instead of using inspect as importing inspect is + # fairly expensive (order of 10-15 ms for a modern machine in 2016) + def isclass(klass): + return isinstance(klass, (type, types.ClassType)) + + def new_class(name, bases, kwds, exec_body): + """ + A minimal stub of types.new_class that we need for make_class. + """ + ns = {} + exec_body(ns) + + return type(name, bases, ns) + + # TYPE is used in exceptions, repr(int) is different on Python 2 and 3. + TYPE = "type" + + def iteritems(d): + return d.iteritems() + + # Python 2 is bereft of a read-only dict proxy, so we make one! + class ReadOnlyDict(IterableUserDict): + """ + Best-effort read-only dict wrapper. + """ + + def __setitem__(self, key, val): + # We gently pretend we're a Python 3 mappingproxy. + raise TypeError( + "'mappingproxy' object does not support item assignment" + ) + + def update(self, _): + # We gently pretend we're a Python 3 mappingproxy. + raise AttributeError( + "'mappingproxy' object has no attribute 'update'" + ) + + def __delitem__(self, _): + # We gently pretend we're a Python 3 mappingproxy. + raise TypeError( + "'mappingproxy' object does not support item deletion" + ) + + def clear(self): + # We gently pretend we're a Python 3 mappingproxy. + raise AttributeError( + "'mappingproxy' object has no attribute 'clear'" + ) + + def pop(self, key, default=None): + # We gently pretend we're a Python 3 mappingproxy. + raise AttributeError( + "'mappingproxy' object has no attribute 'pop'" + ) + + def popitem(self): + # We gently pretend we're a Python 3 mappingproxy. + raise AttributeError( + "'mappingproxy' object has no attribute 'popitem'" + ) + + def setdefault(self, key, default=None): + # We gently pretend we're a Python 3 mappingproxy. + raise AttributeError( + "'mappingproxy' object has no attribute 'setdefault'" + ) + + def __repr__(self): + # Override to be identical to the Python 3 version. + return "mappingproxy(" + repr(self.data) + ")" + + def metadata_proxy(d): + res = ReadOnlyDict() + res.data.update(d) # We blocked update, so we have to do it like this. + return res + + def just_warn(*args, **kw): # pragma: no cover + """ + We only warn on Python 3 because we are not aware of any concrete + consequences of not setting the cell on Python 2. + """ + + +else: # Python 3 and later. + from collections.abc import Mapping, Sequence # noqa + + def just_warn(*args, **kw): + """ + We only warn on Python 3 because we are not aware of any concrete + consequences of not setting the cell on Python 2. + """ + warnings.warn( + "Running interpreter doesn't sufficiently support code object " + "introspection. Some features like bare super() or accessing " + "__class__ will not work with slotted classes.", + RuntimeWarning, + stacklevel=2, + ) + + def isclass(klass): + return isinstance(klass, type) + + TYPE = "class" + + def iteritems(d): + return d.items() + + new_class = types.new_class + + def metadata_proxy(d): + return types.MappingProxyType(dict(d)) + + +def make_set_closure_cell(): + """Return a function of two arguments (cell, value) which sets + the value stored in the closure cell `cell` to `value`. + """ + # pypy makes this easy. (It also supports the logic below, but + # why not do the easy/fast thing?) + if PYPY: + + def set_closure_cell(cell, value): + cell.__setstate__((value,)) + + return set_closure_cell + + # Otherwise gotta do it the hard way. + + # Create a function that will set its first cellvar to `value`. + def set_first_cellvar_to(value): + x = value + return + + # This function will be eliminated as dead code, but + # not before its reference to `x` forces `x` to be + # represented as a closure cell rather than a local. + def force_x_to_be_a_cell(): # pragma: no cover + return x + + try: + # Extract the code object and make sure our assumptions about + # the closure behavior are correct. + if PY2: + co = set_first_cellvar_to.func_code + else: + co = set_first_cellvar_to.__code__ + if co.co_cellvars != ("x",) or co.co_freevars != (): + raise AssertionError # pragma: no cover + + # Convert this code object to a code object that sets the + # function's first _freevar_ (not cellvar) to the argument. + if sys.version_info >= (3, 8): + # CPython 3.8+ has an incompatible CodeType signature + # (added a posonlyargcount argument) but also added + # CodeType.replace() to do this without counting parameters. + set_first_freevar_code = co.replace( + co_cellvars=co.co_freevars, co_freevars=co.co_cellvars + ) + else: + args = [co.co_argcount] + if not PY2: + args.append(co.co_kwonlyargcount) + args.extend( + [ + co.co_nlocals, + co.co_stacksize, + co.co_flags, + co.co_code, + co.co_consts, + co.co_names, + co.co_varnames, + co.co_filename, + co.co_name, + co.co_firstlineno, + co.co_lnotab, + # These two arguments are reversed: + co.co_cellvars, + co.co_freevars, + ] + ) + set_first_freevar_code = types.CodeType(*args) + + def set_closure_cell(cell, value): + # Create a function using the set_first_freevar_code, + # whose first closure cell is `cell`. Calling it will + # change the value of that cell. + setter = types.FunctionType( + set_first_freevar_code, {}, "setter", (), (cell,) + ) + # And call it to set the cell. + setter(value) + + # Make sure it works on this interpreter: + def make_func_with_cell(): + x = None + + def func(): + return x # pragma: no cover + + return func + + if PY2: + cell = make_func_with_cell().func_closure[0] + else: + cell = make_func_with_cell().__closure__[0] + set_closure_cell(cell, 100) + if cell.cell_contents != 100: + raise AssertionError # pragma: no cover + + except Exception: + return just_warn + else: + return set_closure_cell + + +set_closure_cell = make_set_closure_cell() diff --git a/openpype/hosts/fusion/vendor/attr/_config.py b/openpype/hosts/fusion/vendor/attr/_config.py new file mode 100644 index 0000000000..8ec920962d --- /dev/null +++ b/openpype/hosts/fusion/vendor/attr/_config.py @@ -0,0 +1,23 @@ +from __future__ import absolute_import, division, print_function + + +__all__ = ["set_run_validators", "get_run_validators"] + +_run_validators = True + + +def set_run_validators(run): + """ + Set whether or not validators are run. By default, they are run. + """ + if not isinstance(run, bool): + raise TypeError("'run' must be bool.") + global _run_validators + _run_validators = run + + +def get_run_validators(): + """ + Return whether or not validators are run. + """ + return _run_validators diff --git a/openpype/hosts/fusion/vendor/attr/_funcs.py b/openpype/hosts/fusion/vendor/attr/_funcs.py new file mode 100644 index 0000000000..fda508c5c4 --- /dev/null +++ b/openpype/hosts/fusion/vendor/attr/_funcs.py @@ -0,0 +1,395 @@ +from __future__ import absolute_import, division, print_function + +import copy + +from ._compat import iteritems +from ._make import NOTHING, _obj_setattr, fields +from .exceptions import AttrsAttributeNotFoundError + + +def asdict( + inst, + recurse=True, + filter=None, + dict_factory=dict, + retain_collection_types=False, + value_serializer=None, +): + """ + Return the ``attrs`` attribute values of *inst* as a dict. + + Optionally recurse into other ``attrs``-decorated classes. + + :param inst: Instance of an ``attrs``-decorated class. + :param bool recurse: Recurse into classes that are also + ``attrs``-decorated. + :param callable filter: A callable whose return code determines whether an + attribute or element is included (``True``) or dropped (``False``). Is + called with the `attr.Attribute` as the first argument and the + value as the second argument. + :param callable dict_factory: A callable to produce dictionaries from. For + example, to produce ordered dictionaries instead of normal Python + dictionaries, pass in ``collections.OrderedDict``. + :param bool retain_collection_types: Do not convert to ``list`` when + encountering an attribute whose type is ``tuple`` or ``set``. Only + meaningful if ``recurse`` is ``True``. + :param Optional[callable] value_serializer: A hook that is called for every + attribute or dict key/value. It receives the current instance, field + and value and must return the (updated) value. The hook is run *after* + the optional *filter* has been applied. + + :rtype: return type of *dict_factory* + + :raise attr.exceptions.NotAnAttrsClassError: If *cls* is not an ``attrs`` + class. + + .. versionadded:: 16.0.0 *dict_factory* + .. versionadded:: 16.1.0 *retain_collection_types* + .. versionadded:: 20.3.0 *value_serializer* + """ + attrs = fields(inst.__class__) + rv = dict_factory() + for a in attrs: + v = getattr(inst, a.name) + if filter is not None and not filter(a, v): + continue + + if value_serializer is not None: + v = value_serializer(inst, a, v) + + if recurse is True: + if has(v.__class__): + rv[a.name] = asdict( + v, + True, + filter, + dict_factory, + retain_collection_types, + value_serializer, + ) + elif isinstance(v, (tuple, list, set, frozenset)): + cf = v.__class__ if retain_collection_types is True else list + rv[a.name] = cf( + [ + _asdict_anything( + i, + filter, + dict_factory, + retain_collection_types, + value_serializer, + ) + for i in v + ] + ) + elif isinstance(v, dict): + df = dict_factory + rv[a.name] = df( + ( + _asdict_anything( + kk, + filter, + df, + retain_collection_types, + value_serializer, + ), + _asdict_anything( + vv, + filter, + df, + retain_collection_types, + value_serializer, + ), + ) + for kk, vv in iteritems(v) + ) + else: + rv[a.name] = v + else: + rv[a.name] = v + return rv + + +def _asdict_anything( + val, + filter, + dict_factory, + retain_collection_types, + value_serializer, +): + """ + ``asdict`` only works on attrs instances, this works on anything. + """ + if getattr(val.__class__, "__attrs_attrs__", None) is not None: + # Attrs class. + rv = asdict( + val, + True, + filter, + dict_factory, + retain_collection_types, + value_serializer, + ) + elif isinstance(val, (tuple, list, set, frozenset)): + cf = val.__class__ if retain_collection_types is True else list + rv = cf( + [ + _asdict_anything( + i, + filter, + dict_factory, + retain_collection_types, + value_serializer, + ) + for i in val + ] + ) + elif isinstance(val, dict): + df = dict_factory + rv = df( + ( + _asdict_anything( + kk, filter, df, retain_collection_types, value_serializer + ), + _asdict_anything( + vv, filter, df, retain_collection_types, value_serializer + ), + ) + for kk, vv in iteritems(val) + ) + else: + rv = val + if value_serializer is not None: + rv = value_serializer(None, None, rv) + + return rv + + +def astuple( + inst, + recurse=True, + filter=None, + tuple_factory=tuple, + retain_collection_types=False, +): + """ + Return the ``attrs`` attribute values of *inst* as a tuple. + + Optionally recurse into other ``attrs``-decorated classes. + + :param inst: Instance of an ``attrs``-decorated class. + :param bool recurse: Recurse into classes that are also + ``attrs``-decorated. + :param callable filter: A callable whose return code determines whether an + attribute or element is included (``True``) or dropped (``False``). Is + called with the `attr.Attribute` as the first argument and the + value as the second argument. + :param callable tuple_factory: A callable to produce tuples from. For + example, to produce lists instead of tuples. + :param bool retain_collection_types: Do not convert to ``list`` + or ``dict`` when encountering an attribute which type is + ``tuple``, ``dict`` or ``set``. Only meaningful if ``recurse`` is + ``True``. + + :rtype: return type of *tuple_factory* + + :raise attr.exceptions.NotAnAttrsClassError: If *cls* is not an ``attrs`` + class. + + .. versionadded:: 16.2.0 + """ + attrs = fields(inst.__class__) + rv = [] + retain = retain_collection_types # Very long. :/ + for a in attrs: + v = getattr(inst, a.name) + if filter is not None and not filter(a, v): + continue + if recurse is True: + if has(v.__class__): + rv.append( + astuple( + v, + recurse=True, + filter=filter, + tuple_factory=tuple_factory, + retain_collection_types=retain, + ) + ) + elif isinstance(v, (tuple, list, set, frozenset)): + cf = v.__class__ if retain is True else list + rv.append( + cf( + [ + astuple( + j, + recurse=True, + filter=filter, + tuple_factory=tuple_factory, + retain_collection_types=retain, + ) + if has(j.__class__) + else j + for j in v + ] + ) + ) + elif isinstance(v, dict): + df = v.__class__ if retain is True else dict + rv.append( + df( + ( + astuple( + kk, + tuple_factory=tuple_factory, + retain_collection_types=retain, + ) + if has(kk.__class__) + else kk, + astuple( + vv, + tuple_factory=tuple_factory, + retain_collection_types=retain, + ) + if has(vv.__class__) + else vv, + ) + for kk, vv in iteritems(v) + ) + ) + else: + rv.append(v) + else: + rv.append(v) + + return rv if tuple_factory is list else tuple_factory(rv) + + +def has(cls): + """ + Check whether *cls* is a class with ``attrs`` attributes. + + :param type cls: Class to introspect. + :raise TypeError: If *cls* is not a class. + + :rtype: bool + """ + return getattr(cls, "__attrs_attrs__", None) is not None + + +def assoc(inst, **changes): + """ + Copy *inst* and apply *changes*. + + :param inst: Instance of a class with ``attrs`` attributes. + :param changes: Keyword changes in the new copy. + + :return: A copy of inst with *changes* incorporated. + + :raise attr.exceptions.AttrsAttributeNotFoundError: If *attr_name* couldn't + be found on *cls*. + :raise attr.exceptions.NotAnAttrsClassError: If *cls* is not an ``attrs`` + class. + + .. deprecated:: 17.1.0 + Use `evolve` instead. + """ + import warnings + + warnings.warn( + "assoc is deprecated and will be removed after 2018/01.", + DeprecationWarning, + stacklevel=2, + ) + new = copy.copy(inst) + attrs = fields(inst.__class__) + for k, v in iteritems(changes): + a = getattr(attrs, k, NOTHING) + if a is NOTHING: + raise AttrsAttributeNotFoundError( + "{k} is not an attrs attribute on {cl}.".format( + k=k, cl=new.__class__ + ) + ) + _obj_setattr(new, k, v) + return new + + +def evolve(inst, **changes): + """ + Create a new instance, based on *inst* with *changes* applied. + + :param inst: Instance of a class with ``attrs`` attributes. + :param changes: Keyword changes in the new copy. + + :return: A copy of inst with *changes* incorporated. + + :raise TypeError: If *attr_name* couldn't be found in the class + ``__init__``. + :raise attr.exceptions.NotAnAttrsClassError: If *cls* is not an ``attrs`` + class. + + .. versionadded:: 17.1.0 + """ + cls = inst.__class__ + attrs = fields(cls) + for a in attrs: + if not a.init: + continue + attr_name = a.name # To deal with private attributes. + init_name = attr_name if attr_name[0] != "_" else attr_name[1:] + if init_name not in changes: + changes[init_name] = getattr(inst, attr_name) + + return cls(**changes) + + +def resolve_types(cls, globalns=None, localns=None, attribs=None): + """ + Resolve any strings and forward annotations in type annotations. + + This is only required if you need concrete types in `Attribute`'s *type* + field. In other words, you don't need to resolve your types if you only + use them for static type checking. + + With no arguments, names will be looked up in the module in which the class + was created. If this is not what you want, e.g. if the name only exists + inside a method, you may pass *globalns* or *localns* to specify other + dictionaries in which to look up these names. See the docs of + `typing.get_type_hints` for more details. + + :param type cls: Class to resolve. + :param Optional[dict] globalns: Dictionary containing global variables. + :param Optional[dict] localns: Dictionary containing local variables. + :param Optional[list] attribs: List of attribs for the given class. + This is necessary when calling from inside a ``field_transformer`` + since *cls* is not an ``attrs`` class yet. + + :raise TypeError: If *cls* is not a class. + :raise attr.exceptions.NotAnAttrsClassError: If *cls* is not an ``attrs`` + class and you didn't pass any attribs. + :raise NameError: If types cannot be resolved because of missing variables. + + :returns: *cls* so you can use this function also as a class decorator. + Please note that you have to apply it **after** `attr.s`. That means + the decorator has to come in the line **before** `attr.s`. + + .. versionadded:: 20.1.0 + .. versionadded:: 21.1.0 *attribs* + + """ + try: + # Since calling get_type_hints is expensive we cache whether we've + # done it already. + cls.__attrs_types_resolved__ + except AttributeError: + import typing + + hints = typing.get_type_hints(cls, globalns=globalns, localns=localns) + for field in fields(cls) if attribs is None else attribs: + if field.name in hints: + # Since fields have been frozen we must work around it. + _obj_setattr(field, "type", hints[field.name]) + cls.__attrs_types_resolved__ = True + + # Return the class so you can use it as a decorator too. + return cls diff --git a/openpype/hosts/fusion/vendor/attr/_make.py b/openpype/hosts/fusion/vendor/attr/_make.py new file mode 100644 index 0000000000..a1912b1233 --- /dev/null +++ b/openpype/hosts/fusion/vendor/attr/_make.py @@ -0,0 +1,3052 @@ +from __future__ import absolute_import, division, print_function + +import copy +import inspect +import linecache +import sys +import threading +import uuid +import warnings + +from operator import itemgetter + +from . import _config, setters +from ._compat import ( + PY2, + PYPY, + isclass, + iteritems, + metadata_proxy, + new_class, + ordered_dict, + set_closure_cell, +) +from .exceptions import ( + DefaultAlreadySetError, + FrozenInstanceError, + NotAnAttrsClassError, + PythonTooOldError, + UnannotatedAttributeError, +) + + +if not PY2: + import typing + + +# This is used at least twice, so cache it here. +_obj_setattr = object.__setattr__ +_init_converter_pat = "__attr_converter_%s" +_init_factory_pat = "__attr_factory_{}" +_tuple_property_pat = ( + " {attr_name} = _attrs_property(_attrs_itemgetter({index}))" +) +_classvar_prefixes = ( + "typing.ClassVar", + "t.ClassVar", + "ClassVar", + "typing_extensions.ClassVar", +) +# we don't use a double-underscore prefix because that triggers +# name mangling when trying to create a slot for the field +# (when slots=True) +_hash_cache_field = "_attrs_cached_hash" + +_empty_metadata_singleton = metadata_proxy({}) + +# Unique object for unequivocal getattr() defaults. +_sentinel = object() + + +class _Nothing(object): + """ + Sentinel class to indicate the lack of a value when ``None`` is ambiguous. + + ``_Nothing`` is a singleton. There is only ever one of it. + + .. versionchanged:: 21.1.0 ``bool(NOTHING)`` is now False. + """ + + _singleton = None + + def __new__(cls): + if _Nothing._singleton is None: + _Nothing._singleton = super(_Nothing, cls).__new__(cls) + return _Nothing._singleton + + def __repr__(self): + return "NOTHING" + + def __bool__(self): + return False + + def __len__(self): + return 0 # __bool__ for Python 2 + + +NOTHING = _Nothing() +""" +Sentinel to indicate the lack of a value when ``None`` is ambiguous. +""" + + +class _CacheHashWrapper(int): + """ + An integer subclass that pickles / copies as None + + This is used for non-slots classes with ``cache_hash=True``, to avoid + serializing a potentially (even likely) invalid hash value. Since ``None`` + is the default value for uncalculated hashes, whenever this is copied, + the copy's value for the hash should automatically reset. + + See GH #613 for more details. + """ + + if PY2: + # For some reason `type(None)` isn't callable in Python 2, but we don't + # actually need a constructor for None objects, we just need any + # available function that returns None. + def __reduce__(self, _none_constructor=getattr, _args=(0, "", None)): + return _none_constructor, _args + + else: + + def __reduce__(self, _none_constructor=type(None), _args=()): + return _none_constructor, _args + + +def attrib( + default=NOTHING, + validator=None, + repr=True, + cmp=None, + hash=None, + init=True, + metadata=None, + type=None, + converter=None, + factory=None, + kw_only=False, + eq=None, + order=None, + on_setattr=None, +): + """ + Create a new attribute on a class. + + .. warning:: + + Does *not* do anything unless the class is also decorated with + `attr.s`! + + :param default: A value that is used if an ``attrs``-generated ``__init__`` + is used and no value is passed while instantiating or the attribute is + excluded using ``init=False``. + + If the value is an instance of `Factory`, its callable will be + used to construct a new value (useful for mutable data types like lists + or dicts). + + If a default is not set (or set manually to `attr.NOTHING`), a value + *must* be supplied when instantiating; otherwise a `TypeError` + will be raised. + + The default can also be set using decorator notation as shown below. + + :type default: Any value + + :param callable factory: Syntactic sugar for + ``default=attr.Factory(factory)``. + + :param validator: `callable` that is called by ``attrs``-generated + ``__init__`` methods after the instance has been initialized. They + receive the initialized instance, the `Attribute`, and the + passed value. + + The return value is *not* inspected so the validator has to throw an + exception itself. + + If a `list` is passed, its items are treated as validators and must + all pass. + + Validators can be globally disabled and re-enabled using + `get_run_validators`. + + The validator can also be set using decorator notation as shown below. + + :type validator: `callable` or a `list` of `callable`\\ s. + + :param repr: Include this attribute in the generated ``__repr__`` + method. If ``True``, include the attribute; if ``False``, omit it. By + default, the built-in ``repr()`` function is used. To override how the + attribute value is formatted, pass a ``callable`` that takes a single + value and returns a string. Note that the resulting string is used + as-is, i.e. it will be used directly *instead* of calling ``repr()`` + (the default). + :type repr: a `bool` or a `callable` to use a custom function. + + :param eq: If ``True`` (default), include this attribute in the + generated ``__eq__`` and ``__ne__`` methods that check two instances + for equality. To override how the attribute value is compared, + pass a ``callable`` that takes a single value and returns the value + to be compared. + :type eq: a `bool` or a `callable`. + + :param order: If ``True`` (default), include this attributes in the + generated ``__lt__``, ``__le__``, ``__gt__`` and ``__ge__`` methods. + To override how the attribute value is ordered, + pass a ``callable`` that takes a single value and returns the value + to be ordered. + :type order: a `bool` or a `callable`. + + :param cmp: Setting *cmp* is equivalent to setting *eq* and *order* to the + same value. Must not be mixed with *eq* or *order*. + :type cmp: a `bool` or a `callable`. + + :param Optional[bool] hash: Include this attribute in the generated + ``__hash__`` method. If ``None`` (default), mirror *eq*'s value. This + is the correct behavior according the Python spec. Setting this value + to anything else than ``None`` is *discouraged*. + :param bool init: Include this attribute in the generated ``__init__`` + method. It is possible to set this to ``False`` and set a default + value. In that case this attributed is unconditionally initialized + with the specified default value or factory. + :param callable converter: `callable` that is called by + ``attrs``-generated ``__init__`` methods to convert attribute's value + to the desired format. It is given the passed-in value, and the + returned value will be used as the new value of the attribute. The + value is converted before being passed to the validator, if any. + :param metadata: An arbitrary mapping, to be used by third-party + components. See `extending_metadata`. + :param type: The type of the attribute. In Python 3.6 or greater, the + preferred method to specify the type is using a variable annotation + (see `PEP 526 `_). + This argument is provided for backward compatibility. + Regardless of the approach used, the type will be stored on + ``Attribute.type``. + + Please note that ``attrs`` doesn't do anything with this metadata by + itself. You can use it as part of your own code or for + `static type checking `. + :param kw_only: Make this attribute keyword-only (Python 3+) + in the generated ``__init__`` (if ``init`` is ``False``, this + parameter is ignored). + :param on_setattr: Allows to overwrite the *on_setattr* setting from + `attr.s`. If left `None`, the *on_setattr* value from `attr.s` is used. + Set to `attr.setters.NO_OP` to run **no** `setattr` hooks for this + attribute -- regardless of the setting in `attr.s`. + :type on_setattr: `callable`, or a list of callables, or `None`, or + `attr.setters.NO_OP` + + .. versionadded:: 15.2.0 *convert* + .. versionadded:: 16.3.0 *metadata* + .. versionchanged:: 17.1.0 *validator* can be a ``list`` now. + .. versionchanged:: 17.1.0 + *hash* is ``None`` and therefore mirrors *eq* by default. + .. versionadded:: 17.3.0 *type* + .. deprecated:: 17.4.0 *convert* + .. versionadded:: 17.4.0 *converter* as a replacement for the deprecated + *convert* to achieve consistency with other noun-based arguments. + .. versionadded:: 18.1.0 + ``factory=f`` is syntactic sugar for ``default=attr.Factory(f)``. + .. versionadded:: 18.2.0 *kw_only* + .. versionchanged:: 19.2.0 *convert* keyword argument removed. + .. versionchanged:: 19.2.0 *repr* also accepts a custom callable. + .. deprecated:: 19.2.0 *cmp* Removal on or after 2021-06-01. + .. versionadded:: 19.2.0 *eq* and *order* + .. versionadded:: 20.1.0 *on_setattr* + .. versionchanged:: 20.3.0 *kw_only* backported to Python 2 + .. versionchanged:: 21.1.0 + *eq*, *order*, and *cmp* also accept a custom callable + .. versionchanged:: 21.1.0 *cmp* undeprecated + """ + eq, eq_key, order, order_key = _determine_attrib_eq_order( + cmp, eq, order, True + ) + + if hash is not None and hash is not True and hash is not False: + raise TypeError( + "Invalid value for hash. Must be True, False, or None." + ) + + if factory is not None: + if default is not NOTHING: + raise ValueError( + "The `default` and `factory` arguments are mutually " + "exclusive." + ) + if not callable(factory): + raise ValueError("The `factory` argument must be a callable.") + default = Factory(factory) + + if metadata is None: + metadata = {} + + # Apply syntactic sugar by auto-wrapping. + if isinstance(on_setattr, (list, tuple)): + on_setattr = setters.pipe(*on_setattr) + + if validator and isinstance(validator, (list, tuple)): + validator = and_(*validator) + + if converter and isinstance(converter, (list, tuple)): + converter = pipe(*converter) + + return _CountingAttr( + default=default, + validator=validator, + repr=repr, + cmp=None, + hash=hash, + init=init, + converter=converter, + metadata=metadata, + type=type, + kw_only=kw_only, + eq=eq, + eq_key=eq_key, + order=order, + order_key=order_key, + on_setattr=on_setattr, + ) + + +def _compile_and_eval(script, globs, locs=None, filename=""): + """ + "Exec" the script with the given global (globs) and local (locs) variables. + """ + bytecode = compile(script, filename, "exec") + eval(bytecode, globs, locs) + + +def _make_method(name, script, filename, globs=None): + """ + Create the method with the script given and return the method object. + """ + locs = {} + if globs is None: + globs = {} + + _compile_and_eval(script, globs, locs, filename) + + # In order of debuggers like PDB being able to step through the code, + # we add a fake linecache entry. + linecache.cache[filename] = ( + len(script), + None, + script.splitlines(True), + filename, + ) + + return locs[name] + + +def _make_attr_tuple_class(cls_name, attr_names): + """ + Create a tuple subclass to hold `Attribute`s for an `attrs` class. + + The subclass is a bare tuple with properties for names. + + class MyClassAttributes(tuple): + __slots__ = () + x = property(itemgetter(0)) + """ + attr_class_name = "{}Attributes".format(cls_name) + attr_class_template = [ + "class {}(tuple):".format(attr_class_name), + " __slots__ = ()", + ] + if attr_names: + for i, attr_name in enumerate(attr_names): + attr_class_template.append( + _tuple_property_pat.format(index=i, attr_name=attr_name) + ) + else: + attr_class_template.append(" pass") + globs = {"_attrs_itemgetter": itemgetter, "_attrs_property": property} + _compile_and_eval("\n".join(attr_class_template), globs) + return globs[attr_class_name] + + +# Tuple class for extracted attributes from a class definition. +# `base_attrs` is a subset of `attrs`. +_Attributes = _make_attr_tuple_class( + "_Attributes", + [ + # all attributes to build dunder methods for + "attrs", + # attributes that have been inherited + "base_attrs", + # map inherited attributes to their originating classes + "base_attrs_map", + ], +) + + +def _is_class_var(annot): + """ + Check whether *annot* is a typing.ClassVar. + + The string comparison hack is used to avoid evaluating all string + annotations which would put attrs-based classes at a performance + disadvantage compared to plain old classes. + """ + annot = str(annot) + + # Annotation can be quoted. + if annot.startswith(("'", '"')) and annot.endswith(("'", '"')): + annot = annot[1:-1] + + return annot.startswith(_classvar_prefixes) + + +def _has_own_attribute(cls, attrib_name): + """ + Check whether *cls* defines *attrib_name* (and doesn't just inherit it). + + Requires Python 3. + """ + attr = getattr(cls, attrib_name, _sentinel) + if attr is _sentinel: + return False + + for base_cls in cls.__mro__[1:]: + a = getattr(base_cls, attrib_name, None) + if attr is a: + return False + + return True + + +def _get_annotations(cls): + """ + Get annotations for *cls*. + """ + if _has_own_attribute(cls, "__annotations__"): + return cls.__annotations__ + + return {} + + +def _counter_getter(e): + """ + Key function for sorting to avoid re-creating a lambda for every class. + """ + return e[1].counter + + +def _collect_base_attrs(cls, taken_attr_names): + """ + Collect attr.ibs from base classes of *cls*, except *taken_attr_names*. + """ + base_attrs = [] + base_attr_map = {} # A dictionary of base attrs to their classes. + + # Traverse the MRO and collect attributes. + for base_cls in reversed(cls.__mro__[1:-1]): + for a in getattr(base_cls, "__attrs_attrs__", []): + if a.inherited or a.name in taken_attr_names: + continue + + a = a.evolve(inherited=True) + base_attrs.append(a) + base_attr_map[a.name] = base_cls + + # For each name, only keep the freshest definition i.e. the furthest at the + # back. base_attr_map is fine because it gets overwritten with every new + # instance. + filtered = [] + seen = set() + for a in reversed(base_attrs): + if a.name in seen: + continue + filtered.insert(0, a) + seen.add(a.name) + + return filtered, base_attr_map + + +def _collect_base_attrs_broken(cls, taken_attr_names): + """ + Collect attr.ibs from base classes of *cls*, except *taken_attr_names*. + + N.B. *taken_attr_names* will be mutated. + + Adhere to the old incorrect behavior. + + Notably it collects from the front and considers inherited attributes which + leads to the buggy behavior reported in #428. + """ + base_attrs = [] + base_attr_map = {} # A dictionary of base attrs to their classes. + + # Traverse the MRO and collect attributes. + for base_cls in cls.__mro__[1:-1]: + for a in getattr(base_cls, "__attrs_attrs__", []): + if a.name in taken_attr_names: + continue + + a = a.evolve(inherited=True) + taken_attr_names.add(a.name) + base_attrs.append(a) + base_attr_map[a.name] = base_cls + + return base_attrs, base_attr_map + + +def _transform_attrs( + cls, these, auto_attribs, kw_only, collect_by_mro, field_transformer +): + """ + Transform all `_CountingAttr`s on a class into `Attribute`s. + + If *these* is passed, use that and don't look for them on the class. + + *collect_by_mro* is True, collect them in the correct MRO order, otherwise + use the old -- incorrect -- order. See #428. + + Return an `_Attributes`. + """ + cd = cls.__dict__ + anns = _get_annotations(cls) + + if these is not None: + ca_list = [(name, ca) for name, ca in iteritems(these)] + + if not isinstance(these, ordered_dict): + ca_list.sort(key=_counter_getter) + elif auto_attribs is True: + ca_names = { + name + for name, attr in cd.items() + if isinstance(attr, _CountingAttr) + } + ca_list = [] + annot_names = set() + for attr_name, type in anns.items(): + if _is_class_var(type): + continue + annot_names.add(attr_name) + a = cd.get(attr_name, NOTHING) + + if not isinstance(a, _CountingAttr): + if a is NOTHING: + a = attrib() + else: + a = attrib(default=a) + ca_list.append((attr_name, a)) + + unannotated = ca_names - annot_names + if len(unannotated) > 0: + raise UnannotatedAttributeError( + "The following `attr.ib`s lack a type annotation: " + + ", ".join( + sorted(unannotated, key=lambda n: cd.get(n).counter) + ) + + "." + ) + else: + ca_list = sorted( + ( + (name, attr) + for name, attr in cd.items() + if isinstance(attr, _CountingAttr) + ), + key=lambda e: e[1].counter, + ) + + own_attrs = [ + Attribute.from_counting_attr( + name=attr_name, ca=ca, type=anns.get(attr_name) + ) + for attr_name, ca in ca_list + ] + + if collect_by_mro: + base_attrs, base_attr_map = _collect_base_attrs( + cls, {a.name for a in own_attrs} + ) + else: + base_attrs, base_attr_map = _collect_base_attrs_broken( + cls, {a.name for a in own_attrs} + ) + + attr_names = [a.name for a in base_attrs + own_attrs] + + AttrsClass = _make_attr_tuple_class(cls.__name__, attr_names) + + if kw_only: + own_attrs = [a.evolve(kw_only=True) for a in own_attrs] + base_attrs = [a.evolve(kw_only=True) for a in base_attrs] + + attrs = AttrsClass(base_attrs + own_attrs) + + # Mandatory vs non-mandatory attr order only matters when they are part of + # the __init__ signature and when they aren't kw_only (which are moved to + # the end and can be mandatory or non-mandatory in any order, as they will + # be specified as keyword args anyway). Check the order of those attrs: + had_default = False + for a in (a for a in attrs if a.init is not False and a.kw_only is False): + if had_default is True and a.default is NOTHING: + raise ValueError( + "No mandatory attributes allowed after an attribute with a " + "default value or factory. Attribute in question: %r" % (a,) + ) + + if had_default is False and a.default is not NOTHING: + had_default = True + + if field_transformer is not None: + attrs = field_transformer(cls, attrs) + return _Attributes((attrs, base_attrs, base_attr_map)) + + +if PYPY: + + def _frozen_setattrs(self, name, value): + """ + Attached to frozen classes as __setattr__. + """ + if isinstance(self, BaseException) and name in ( + "__cause__", + "__context__", + ): + BaseException.__setattr__(self, name, value) + return + + raise FrozenInstanceError() + + +else: + + def _frozen_setattrs(self, name, value): + """ + Attached to frozen classes as __setattr__. + """ + raise FrozenInstanceError() + + +def _frozen_delattrs(self, name): + """ + Attached to frozen classes as __delattr__. + """ + raise FrozenInstanceError() + + +class _ClassBuilder(object): + """ + Iteratively build *one* class. + """ + + __slots__ = ( + "_attr_names", + "_attrs", + "_base_attr_map", + "_base_names", + "_cache_hash", + "_cls", + "_cls_dict", + "_delete_attribs", + "_frozen", + "_has_pre_init", + "_has_post_init", + "_is_exc", + "_on_setattr", + "_slots", + "_weakref_slot", + "_has_own_setattr", + "_has_custom_setattr", + ) + + def __init__( + self, + cls, + these, + slots, + frozen, + weakref_slot, + getstate_setstate, + auto_attribs, + kw_only, + cache_hash, + is_exc, + collect_by_mro, + on_setattr, + has_custom_setattr, + field_transformer, + ): + attrs, base_attrs, base_map = _transform_attrs( + cls, + these, + auto_attribs, + kw_only, + collect_by_mro, + field_transformer, + ) + + self._cls = cls + self._cls_dict = dict(cls.__dict__) if slots else {} + self._attrs = attrs + self._base_names = set(a.name for a in base_attrs) + self._base_attr_map = base_map + self._attr_names = tuple(a.name for a in attrs) + self._slots = slots + self._frozen = frozen + self._weakref_slot = weakref_slot + self._cache_hash = cache_hash + self._has_pre_init = bool(getattr(cls, "__attrs_pre_init__", False)) + self._has_post_init = bool(getattr(cls, "__attrs_post_init__", False)) + self._delete_attribs = not bool(these) + self._is_exc = is_exc + self._on_setattr = on_setattr + + self._has_custom_setattr = has_custom_setattr + self._has_own_setattr = False + + self._cls_dict["__attrs_attrs__"] = self._attrs + + if frozen: + self._cls_dict["__setattr__"] = _frozen_setattrs + self._cls_dict["__delattr__"] = _frozen_delattrs + + self._has_own_setattr = True + + if getstate_setstate: + ( + self._cls_dict["__getstate__"], + self._cls_dict["__setstate__"], + ) = self._make_getstate_setstate() + + def __repr__(self): + return "<_ClassBuilder(cls={cls})>".format(cls=self._cls.__name__) + + def build_class(self): + """ + Finalize class based on the accumulated configuration. + + Builder cannot be used after calling this method. + """ + if self._slots is True: + return self._create_slots_class() + else: + return self._patch_original_class() + + def _patch_original_class(self): + """ + Apply accumulated methods and return the class. + """ + cls = self._cls + base_names = self._base_names + + # Clean class of attribute definitions (`attr.ib()`s). + if self._delete_attribs: + for name in self._attr_names: + if ( + name not in base_names + and getattr(cls, name, _sentinel) is not _sentinel + ): + try: + delattr(cls, name) + except AttributeError: + # This can happen if a base class defines a class + # variable and we want to set an attribute with the + # same name by using only a type annotation. + pass + + # Attach our dunder methods. + for name, value in self._cls_dict.items(): + setattr(cls, name, value) + + # If we've inherited an attrs __setattr__ and don't write our own, + # reset it to object's. + if not self._has_own_setattr and getattr( + cls, "__attrs_own_setattr__", False + ): + cls.__attrs_own_setattr__ = False + + if not self._has_custom_setattr: + cls.__setattr__ = object.__setattr__ + + return cls + + def _create_slots_class(self): + """ + Build and return a new class with a `__slots__` attribute. + """ + cd = { + k: v + for k, v in iteritems(self._cls_dict) + if k not in tuple(self._attr_names) + ("__dict__", "__weakref__") + } + + # If our class doesn't have its own implementation of __setattr__ + # (either from the user or by us), check the bases, if one of them has + # an attrs-made __setattr__, that needs to be reset. We don't walk the + # MRO because we only care about our immediate base classes. + # XXX: This can be confused by subclassing a slotted attrs class with + # XXX: a non-attrs class and subclass the resulting class with an attrs + # XXX: class. See `test_slotted_confused` for details. For now that's + # XXX: OK with us. + if not self._has_own_setattr: + cd["__attrs_own_setattr__"] = False + + if not self._has_custom_setattr: + for base_cls in self._cls.__bases__: + if base_cls.__dict__.get("__attrs_own_setattr__", False): + cd["__setattr__"] = object.__setattr__ + break + + # Traverse the MRO to collect existing slots + # and check for an existing __weakref__. + existing_slots = dict() + weakref_inherited = False + for base_cls in self._cls.__mro__[1:-1]: + if base_cls.__dict__.get("__weakref__", None) is not None: + weakref_inherited = True + existing_slots.update( + { + name: getattr(base_cls, name) + for name in getattr(base_cls, "__slots__", []) + } + ) + + base_names = set(self._base_names) + + names = self._attr_names + if ( + self._weakref_slot + and "__weakref__" not in getattr(self._cls, "__slots__", ()) + and "__weakref__" not in names + and not weakref_inherited + ): + names += ("__weakref__",) + + # We only add the names of attributes that aren't inherited. + # Setting __slots__ to inherited attributes wastes memory. + slot_names = [name for name in names if name not in base_names] + # There are slots for attributes from current class + # that are defined in parent classes. + # As their descriptors may be overriden by a child class, + # we collect them here and update the class dict + reused_slots = { + slot: slot_descriptor + for slot, slot_descriptor in iteritems(existing_slots) + if slot in slot_names + } + slot_names = [name for name in slot_names if name not in reused_slots] + cd.update(reused_slots) + if self._cache_hash: + slot_names.append(_hash_cache_field) + cd["__slots__"] = tuple(slot_names) + + qualname = getattr(self._cls, "__qualname__", None) + if qualname is not None: + cd["__qualname__"] = qualname + + # Create new class based on old class and our methods. + cls = type(self._cls)(self._cls.__name__, self._cls.__bases__, cd) + + # The following is a fix for + # https://github.com/python-attrs/attrs/issues/102. On Python 3, + # if a method mentions `__class__` or uses the no-arg super(), the + # compiler will bake a reference to the class in the method itself + # as `method.__closure__`. Since we replace the class with a + # clone, we rewrite these references so it keeps working. + for item in cls.__dict__.values(): + if isinstance(item, (classmethod, staticmethod)): + # Class- and staticmethods hide their functions inside. + # These might need to be rewritten as well. + closure_cells = getattr(item.__func__, "__closure__", None) + elif isinstance(item, property): + # Workaround for property `super()` shortcut (PY3-only). + # There is no universal way for other descriptors. + closure_cells = getattr(item.fget, "__closure__", None) + else: + closure_cells = getattr(item, "__closure__", None) + + if not closure_cells: # Catch None or the empty list. + continue + for cell in closure_cells: + try: + match = cell.cell_contents is self._cls + except ValueError: # ValueError: Cell is empty + pass + else: + if match: + set_closure_cell(cell, cls) + + return cls + + def add_repr(self, ns): + self._cls_dict["__repr__"] = self._add_method_dunders( + _make_repr(self._attrs, ns=ns) + ) + return self + + def add_str(self): + repr = self._cls_dict.get("__repr__") + if repr is None: + raise ValueError( + "__str__ can only be generated if a __repr__ exists." + ) + + def __str__(self): + return self.__repr__() + + self._cls_dict["__str__"] = self._add_method_dunders(__str__) + return self + + def _make_getstate_setstate(self): + """ + Create custom __setstate__ and __getstate__ methods. + """ + # __weakref__ is not writable. + state_attr_names = tuple( + an for an in self._attr_names if an != "__weakref__" + ) + + def slots_getstate(self): + """ + Automatically created by attrs. + """ + return tuple(getattr(self, name) for name in state_attr_names) + + hash_caching_enabled = self._cache_hash + + def slots_setstate(self, state): + """ + Automatically created by attrs. + """ + __bound_setattr = _obj_setattr.__get__(self, Attribute) + for name, value in zip(state_attr_names, state): + __bound_setattr(name, value) + + # The hash code cache is not included when the object is + # serialized, but it still needs to be initialized to None to + # indicate that the first call to __hash__ should be a cache + # miss. + if hash_caching_enabled: + __bound_setattr(_hash_cache_field, None) + + return slots_getstate, slots_setstate + + def make_unhashable(self): + self._cls_dict["__hash__"] = None + return self + + def add_hash(self): + self._cls_dict["__hash__"] = self._add_method_dunders( + _make_hash( + self._cls, + self._attrs, + frozen=self._frozen, + cache_hash=self._cache_hash, + ) + ) + + return self + + def add_init(self): + self._cls_dict["__init__"] = self._add_method_dunders( + _make_init( + self._cls, + self._attrs, + self._has_pre_init, + self._has_post_init, + self._frozen, + self._slots, + self._cache_hash, + self._base_attr_map, + self._is_exc, + self._on_setattr is not None + and self._on_setattr is not setters.NO_OP, + attrs_init=False, + ) + ) + + return self + + def add_attrs_init(self): + self._cls_dict["__attrs_init__"] = self._add_method_dunders( + _make_init( + self._cls, + self._attrs, + self._has_pre_init, + self._has_post_init, + self._frozen, + self._slots, + self._cache_hash, + self._base_attr_map, + self._is_exc, + self._on_setattr is not None + and self._on_setattr is not setters.NO_OP, + attrs_init=True, + ) + ) + + return self + + def add_eq(self): + cd = self._cls_dict + + cd["__eq__"] = self._add_method_dunders( + _make_eq(self._cls, self._attrs) + ) + cd["__ne__"] = self._add_method_dunders(_make_ne()) + + return self + + def add_order(self): + cd = self._cls_dict + + cd["__lt__"], cd["__le__"], cd["__gt__"], cd["__ge__"] = ( + self._add_method_dunders(meth) + for meth in _make_order(self._cls, self._attrs) + ) + + return self + + def add_setattr(self): + if self._frozen: + return self + + sa_attrs = {} + for a in self._attrs: + on_setattr = a.on_setattr or self._on_setattr + if on_setattr and on_setattr is not setters.NO_OP: + sa_attrs[a.name] = a, on_setattr + + if not sa_attrs: + return self + + if self._has_custom_setattr: + # We need to write a __setattr__ but there already is one! + raise ValueError( + "Can't combine custom __setattr__ with on_setattr hooks." + ) + + # docstring comes from _add_method_dunders + def __setattr__(self, name, val): + try: + a, hook = sa_attrs[name] + except KeyError: + nval = val + else: + nval = hook(self, a, val) + + _obj_setattr(self, name, nval) + + self._cls_dict["__attrs_own_setattr__"] = True + self._cls_dict["__setattr__"] = self._add_method_dunders(__setattr__) + self._has_own_setattr = True + + return self + + def _add_method_dunders(self, method): + """ + Add __module__ and __qualname__ to a *method* if possible. + """ + try: + method.__module__ = self._cls.__module__ + except AttributeError: + pass + + try: + method.__qualname__ = ".".join( + (self._cls.__qualname__, method.__name__) + ) + except AttributeError: + pass + + try: + method.__doc__ = "Method generated by attrs for class %s." % ( + self._cls.__qualname__, + ) + except AttributeError: + pass + + return method + + +_CMP_DEPRECATION = ( + "The usage of `cmp` is deprecated and will be removed on or after " + "2021-06-01. Please use `eq` and `order` instead." +) + + +def _determine_attrs_eq_order(cmp, eq, order, default_eq): + """ + Validate the combination of *cmp*, *eq*, and *order*. Derive the effective + values of eq and order. If *eq* is None, set it to *default_eq*. + """ + if cmp is not None and any((eq is not None, order is not None)): + raise ValueError("Don't mix `cmp` with `eq' and `order`.") + + # cmp takes precedence due to bw-compatibility. + if cmp is not None: + return cmp, cmp + + # If left None, equality is set to the specified default and ordering + # mirrors equality. + if eq is None: + eq = default_eq + + if order is None: + order = eq + + if eq is False and order is True: + raise ValueError("`order` can only be True if `eq` is True too.") + + return eq, order + + +def _determine_attrib_eq_order(cmp, eq, order, default_eq): + """ + Validate the combination of *cmp*, *eq*, and *order*. Derive the effective + values of eq and order. If *eq* is None, set it to *default_eq*. + """ + if cmp is not None and any((eq is not None, order is not None)): + raise ValueError("Don't mix `cmp` with `eq' and `order`.") + + def decide_callable_or_boolean(value): + """ + Decide whether a key function is used. + """ + if callable(value): + value, key = True, value + else: + key = None + return value, key + + # cmp takes precedence due to bw-compatibility. + if cmp is not None: + cmp, cmp_key = decide_callable_or_boolean(cmp) + return cmp, cmp_key, cmp, cmp_key + + # If left None, equality is set to the specified default and ordering + # mirrors equality. + if eq is None: + eq, eq_key = default_eq, None + else: + eq, eq_key = decide_callable_or_boolean(eq) + + if order is None: + order, order_key = eq, eq_key + else: + order, order_key = decide_callable_or_boolean(order) + + if eq is False and order is True: + raise ValueError("`order` can only be True if `eq` is True too.") + + return eq, eq_key, order, order_key + + +def _determine_whether_to_implement( + cls, flag, auto_detect, dunders, default=True +): + """ + Check whether we should implement a set of methods for *cls*. + + *flag* is the argument passed into @attr.s like 'init', *auto_detect* the + same as passed into @attr.s and *dunders* is a tuple of attribute names + whose presence signal that the user has implemented it themselves. + + Return *default* if no reason for either for or against is found. + + auto_detect must be False on Python 2. + """ + if flag is True or flag is False: + return flag + + if flag is None and auto_detect is False: + return default + + # Logically, flag is None and auto_detect is True here. + for dunder in dunders: + if _has_own_attribute(cls, dunder): + return False + + return default + + +def attrs( + maybe_cls=None, + these=None, + repr_ns=None, + repr=None, + cmp=None, + hash=None, + init=None, + slots=False, + frozen=False, + weakref_slot=True, + str=False, + auto_attribs=False, + kw_only=False, + cache_hash=False, + auto_exc=False, + eq=None, + order=None, + auto_detect=False, + collect_by_mro=False, + getstate_setstate=None, + on_setattr=None, + field_transformer=None, +): + r""" + A class decorator that adds `dunder + `_\ -methods according to the + specified attributes using `attr.ib` or the *these* argument. + + :param these: A dictionary of name to `attr.ib` mappings. This is + useful to avoid the definition of your attributes within the class body + because you can't (e.g. if you want to add ``__repr__`` methods to + Django models) or don't want to. + + If *these* is not ``None``, ``attrs`` will *not* search the class body + for attributes and will *not* remove any attributes from it. + + If *these* is an ordered dict (`dict` on Python 3.6+, + `collections.OrderedDict` otherwise), the order is deduced from + the order of the attributes inside *these*. Otherwise the order + of the definition of the attributes is used. + + :type these: `dict` of `str` to `attr.ib` + + :param str repr_ns: When using nested classes, there's no way in Python 2 + to automatically detect that. Therefore it's possible to set the + namespace explicitly for a more meaningful ``repr`` output. + :param bool auto_detect: Instead of setting the *init*, *repr*, *eq*, + *order*, and *hash* arguments explicitly, assume they are set to + ``True`` **unless any** of the involved methods for one of the + arguments is implemented in the *current* class (i.e. it is *not* + inherited from some base class). + + So for example by implementing ``__eq__`` on a class yourself, + ``attrs`` will deduce ``eq=False`` and will create *neither* + ``__eq__`` *nor* ``__ne__`` (but Python classes come with a sensible + ``__ne__`` by default, so it *should* be enough to only implement + ``__eq__`` in most cases). + + .. warning:: + + If you prevent ``attrs`` from creating the ordering methods for you + (``order=False``, e.g. by implementing ``__le__``), it becomes + *your* responsibility to make sure its ordering is sound. The best + way is to use the `functools.total_ordering` decorator. + + + Passing ``True`` or ``False`` to *init*, *repr*, *eq*, *order*, + *cmp*, or *hash* overrides whatever *auto_detect* would determine. + + *auto_detect* requires Python 3. Setting it ``True`` on Python 2 raises + a `PythonTooOldError`. + + :param bool repr: Create a ``__repr__`` method with a human readable + representation of ``attrs`` attributes.. + :param bool str: Create a ``__str__`` method that is identical to + ``__repr__``. This is usually not necessary except for + `Exception`\ s. + :param Optional[bool] eq: If ``True`` or ``None`` (default), add ``__eq__`` + and ``__ne__`` methods that check two instances for equality. + + They compare the instances as if they were tuples of their ``attrs`` + attributes if and only if the types of both classes are *identical*! + :param Optional[bool] order: If ``True``, add ``__lt__``, ``__le__``, + ``__gt__``, and ``__ge__`` methods that behave like *eq* above and + allow instances to be ordered. If ``None`` (default) mirror value of + *eq*. + :param Optional[bool] cmp: Setting *cmp* is equivalent to setting *eq* + and *order* to the same value. Must not be mixed with *eq* or *order*. + :param Optional[bool] hash: If ``None`` (default), the ``__hash__`` method + is generated according how *eq* and *frozen* are set. + + 1. If *both* are True, ``attrs`` will generate a ``__hash__`` for you. + 2. If *eq* is True and *frozen* is False, ``__hash__`` will be set to + None, marking it unhashable (which it is). + 3. If *eq* is False, ``__hash__`` will be left untouched meaning the + ``__hash__`` method of the base class will be used (if base class is + ``object``, this means it will fall back to id-based hashing.). + + Although not recommended, you can decide for yourself and force + ``attrs`` to create one (e.g. if the class is immutable even though you + didn't freeze it programmatically) by passing ``True`` or not. Both of + these cases are rather special and should be used carefully. + + See our documentation on `hashing`, Python's documentation on + `object.__hash__`, and the `GitHub issue that led to the default \ + behavior `_ for more + details. + :param bool init: Create a ``__init__`` method that initializes the + ``attrs`` attributes. Leading underscores are stripped for the argument + name. If a ``__attrs_pre_init__`` method exists on the class, it will + be called before the class is initialized. If a ``__attrs_post_init__`` + method exists on the class, it will be called after the class is fully + initialized. + + If ``init`` is ``False``, an ``__attrs_init__`` method will be + injected instead. This allows you to define a custom ``__init__`` + method that can do pre-init work such as ``super().__init__()``, + and then call ``__attrs_init__()`` and ``__attrs_post_init__()``. + :param bool slots: Create a `slotted class ` that's more + memory-efficient. Slotted classes are generally superior to the default + dict classes, but have some gotchas you should know about, so we + encourage you to read the `glossary entry `. + :param bool frozen: Make instances immutable after initialization. If + someone attempts to modify a frozen instance, + `attr.exceptions.FrozenInstanceError` is raised. + + .. note:: + + 1. This is achieved by installing a custom ``__setattr__`` method + on your class, so you can't implement your own. + + 2. True immutability is impossible in Python. + + 3. This *does* have a minor a runtime performance `impact + ` when initializing new instances. In other words: + ``__init__`` is slightly slower with ``frozen=True``. + + 4. If a class is frozen, you cannot modify ``self`` in + ``__attrs_post_init__`` or a self-written ``__init__``. You can + circumvent that limitation by using + ``object.__setattr__(self, "attribute_name", value)``. + + 5. Subclasses of a frozen class are frozen too. + + :param bool weakref_slot: Make instances weak-referenceable. This has no + effect unless ``slots`` is also enabled. + :param bool auto_attribs: If ``True``, collect `PEP 526`_-annotated + attributes (Python 3.6 and later only) from the class body. + + In this case, you **must** annotate every field. If ``attrs`` + encounters a field that is set to an `attr.ib` but lacks a type + annotation, an `attr.exceptions.UnannotatedAttributeError` is + raised. Use ``field_name: typing.Any = attr.ib(...)`` if you don't + want to set a type. + + If you assign a value to those attributes (e.g. ``x: int = 42``), that + value becomes the default value like if it were passed using + ``attr.ib(default=42)``. Passing an instance of `Factory` also + works as expected in most cases (see warning below). + + Attributes annotated as `typing.ClassVar`, and attributes that are + neither annotated nor set to an `attr.ib` are **ignored**. + + .. warning:: + For features that use the attribute name to create decorators (e.g. + `validators `), you still *must* assign `attr.ib` to + them. Otherwise Python will either not find the name or try to use + the default value to call e.g. ``validator`` on it. + + These errors can be quite confusing and probably the most common bug + report on our bug tracker. + + .. _`PEP 526`: https://www.python.org/dev/peps/pep-0526/ + :param bool kw_only: Make all attributes keyword-only (Python 3+) + in the generated ``__init__`` (if ``init`` is ``False``, this + parameter is ignored). + :param bool cache_hash: Ensure that the object's hash code is computed + only once and stored on the object. If this is set to ``True``, + hashing must be either explicitly or implicitly enabled for this + class. If the hash code is cached, avoid any reassignments of + fields involved in hash code computation or mutations of the objects + those fields point to after object creation. If such changes occur, + the behavior of the object's hash code is undefined. + :param bool auto_exc: If the class subclasses `BaseException` + (which implicitly includes any subclass of any exception), the + following happens to behave like a well-behaved Python exceptions + class: + + - the values for *eq*, *order*, and *hash* are ignored and the + instances compare and hash by the instance's ids (N.B. ``attrs`` will + *not* remove existing implementations of ``__hash__`` or the equality + methods. It just won't add own ones.), + - all attributes that are either passed into ``__init__`` or have a + default value are additionally available as a tuple in the ``args`` + attribute, + - the value of *str* is ignored leaving ``__str__`` to base classes. + :param bool collect_by_mro: Setting this to `True` fixes the way ``attrs`` + collects attributes from base classes. The default behavior is + incorrect in certain cases of multiple inheritance. It should be on by + default but is kept off for backward-compatability. + + See issue `#428 `_ for + more details. + + :param Optional[bool] getstate_setstate: + .. note:: + This is usually only interesting for slotted classes and you should + probably just set *auto_detect* to `True`. + + If `True`, ``__getstate__`` and + ``__setstate__`` are generated and attached to the class. This is + necessary for slotted classes to be pickleable. If left `None`, it's + `True` by default for slotted classes and ``False`` for dict classes. + + If *auto_detect* is `True`, and *getstate_setstate* is left `None`, + and **either** ``__getstate__`` or ``__setstate__`` is detected directly + on the class (i.e. not inherited), it is set to `False` (this is usually + what you want). + + :param on_setattr: A callable that is run whenever the user attempts to set + an attribute (either by assignment like ``i.x = 42`` or by using + `setattr` like ``setattr(i, "x", 42)``). It receives the same arguments + as validators: the instance, the attribute that is being modified, and + the new value. + + If no exception is raised, the attribute is set to the return value of + the callable. + + If a list of callables is passed, they're automatically wrapped in an + `attr.setters.pipe`. + + :param Optional[callable] field_transformer: + A function that is called with the original class object and all + fields right before ``attrs`` finalizes the class. You can use + this, e.g., to automatically add converters or validators to + fields based on their types. See `transform-fields` for more details. + + .. versionadded:: 16.0.0 *slots* + .. versionadded:: 16.1.0 *frozen* + .. versionadded:: 16.3.0 *str* + .. versionadded:: 16.3.0 Support for ``__attrs_post_init__``. + .. versionchanged:: 17.1.0 + *hash* supports ``None`` as value which is also the default now. + .. versionadded:: 17.3.0 *auto_attribs* + .. versionchanged:: 18.1.0 + If *these* is passed, no attributes are deleted from the class body. + .. versionchanged:: 18.1.0 If *these* is ordered, the order is retained. + .. versionadded:: 18.2.0 *weakref_slot* + .. deprecated:: 18.2.0 + ``__lt__``, ``__le__``, ``__gt__``, and ``__ge__`` now raise a + `DeprecationWarning` if the classes compared are subclasses of + each other. ``__eq`` and ``__ne__`` never tried to compared subclasses + to each other. + .. versionchanged:: 19.2.0 + ``__lt__``, ``__le__``, ``__gt__``, and ``__ge__`` now do not consider + subclasses comparable anymore. + .. versionadded:: 18.2.0 *kw_only* + .. versionadded:: 18.2.0 *cache_hash* + .. versionadded:: 19.1.0 *auto_exc* + .. deprecated:: 19.2.0 *cmp* Removal on or after 2021-06-01. + .. versionadded:: 19.2.0 *eq* and *order* + .. versionadded:: 20.1.0 *auto_detect* + .. versionadded:: 20.1.0 *collect_by_mro* + .. versionadded:: 20.1.0 *getstate_setstate* + .. versionadded:: 20.1.0 *on_setattr* + .. versionadded:: 20.3.0 *field_transformer* + .. versionchanged:: 21.1.0 + ``init=False`` injects ``__attrs_init__`` + .. versionchanged:: 21.1.0 Support for ``__attrs_pre_init__`` + .. versionchanged:: 21.1.0 *cmp* undeprecated + """ + if auto_detect and PY2: + raise PythonTooOldError( + "auto_detect only works on Python 3 and later." + ) + + eq_, order_ = _determine_attrs_eq_order(cmp, eq, order, None) + hash_ = hash # work around the lack of nonlocal + + if isinstance(on_setattr, (list, tuple)): + on_setattr = setters.pipe(*on_setattr) + + def wrap(cls): + + if getattr(cls, "__class__", None) is None: + raise TypeError("attrs only works with new-style classes.") + + is_frozen = frozen or _has_frozen_base_class(cls) + is_exc = auto_exc is True and issubclass(cls, BaseException) + has_own_setattr = auto_detect and _has_own_attribute( + cls, "__setattr__" + ) + + if has_own_setattr and is_frozen: + raise ValueError("Can't freeze a class with a custom __setattr__.") + + builder = _ClassBuilder( + cls, + these, + slots, + is_frozen, + weakref_slot, + _determine_whether_to_implement( + cls, + getstate_setstate, + auto_detect, + ("__getstate__", "__setstate__"), + default=slots, + ), + auto_attribs, + kw_only, + cache_hash, + is_exc, + collect_by_mro, + on_setattr, + has_own_setattr, + field_transformer, + ) + if _determine_whether_to_implement( + cls, repr, auto_detect, ("__repr__",) + ): + builder.add_repr(repr_ns) + if str is True: + builder.add_str() + + eq = _determine_whether_to_implement( + cls, eq_, auto_detect, ("__eq__", "__ne__") + ) + if not is_exc and eq is True: + builder.add_eq() + if not is_exc and _determine_whether_to_implement( + cls, order_, auto_detect, ("__lt__", "__le__", "__gt__", "__ge__") + ): + builder.add_order() + + builder.add_setattr() + + if ( + hash_ is None + and auto_detect is True + and _has_own_attribute(cls, "__hash__") + ): + hash = False + else: + hash = hash_ + if hash is not True and hash is not False and hash is not None: + # Can't use `hash in` because 1 == True for example. + raise TypeError( + "Invalid value for hash. Must be True, False, or None." + ) + elif hash is False or (hash is None and eq is False) or is_exc: + # Don't do anything. Should fall back to __object__'s __hash__ + # which is by id. + if cache_hash: + raise TypeError( + "Invalid value for cache_hash. To use hash caching," + " hashing must be either explicitly or implicitly " + "enabled." + ) + elif hash is True or ( + hash is None and eq is True and is_frozen is True + ): + # Build a __hash__ if told so, or if it's safe. + builder.add_hash() + else: + # Raise TypeError on attempts to hash. + if cache_hash: + raise TypeError( + "Invalid value for cache_hash. To use hash caching," + " hashing must be either explicitly or implicitly " + "enabled." + ) + builder.make_unhashable() + + if _determine_whether_to_implement( + cls, init, auto_detect, ("__init__",) + ): + builder.add_init() + else: + builder.add_attrs_init() + if cache_hash: + raise TypeError( + "Invalid value for cache_hash. To use hash caching," + " init must be True." + ) + + return builder.build_class() + + # maybe_cls's type depends on the usage of the decorator. It's a class + # if it's used as `@attrs` but ``None`` if used as `@attrs()`. + if maybe_cls is None: + return wrap + else: + return wrap(maybe_cls) + + +_attrs = attrs +""" +Internal alias so we can use it in functions that take an argument called +*attrs*. +""" + + +if PY2: + + def _has_frozen_base_class(cls): + """ + Check whether *cls* has a frozen ancestor by looking at its + __setattr__. + """ + return ( + getattr(cls.__setattr__, "__module__", None) + == _frozen_setattrs.__module__ + and cls.__setattr__.__name__ == _frozen_setattrs.__name__ + ) + + +else: + + def _has_frozen_base_class(cls): + """ + Check whether *cls* has a frozen ancestor by looking at its + __setattr__. + """ + return cls.__setattr__ == _frozen_setattrs + + +def _generate_unique_filename(cls, func_name): + """ + Create a "filename" suitable for a function being generated. + """ + unique_id = uuid.uuid4() + extra = "" + count = 1 + + while True: + unique_filename = "".format( + func_name, + cls.__module__, + getattr(cls, "__qualname__", cls.__name__), + extra, + ) + # To handle concurrency we essentially "reserve" our spot in + # the linecache with a dummy line. The caller can then + # set this value correctly. + cache_line = (1, None, (str(unique_id),), unique_filename) + if ( + linecache.cache.setdefault(unique_filename, cache_line) + == cache_line + ): + return unique_filename + + # Looks like this spot is taken. Try again. + count += 1 + extra = "-{0}".format(count) + + +def _make_hash(cls, attrs, frozen, cache_hash): + attrs = tuple( + a for a in attrs if a.hash is True or (a.hash is None and a.eq is True) + ) + + tab = " " + + unique_filename = _generate_unique_filename(cls, "hash") + type_hash = hash(unique_filename) + + hash_def = "def __hash__(self" + hash_func = "hash((" + closing_braces = "))" + if not cache_hash: + hash_def += "):" + else: + if not PY2: + hash_def += ", *" + + hash_def += ( + ", _cache_wrapper=" + + "__import__('attr._make')._make._CacheHashWrapper):" + ) + hash_func = "_cache_wrapper(" + hash_func + closing_braces += ")" + + method_lines = [hash_def] + + def append_hash_computation_lines(prefix, indent): + """ + Generate the code for actually computing the hash code. + Below this will either be returned directly or used to compute + a value which is then cached, depending on the value of cache_hash + """ + + method_lines.extend( + [ + indent + prefix + hash_func, + indent + " %d," % (type_hash,), + ] + ) + + for a in attrs: + method_lines.append(indent + " self.%s," % a.name) + + method_lines.append(indent + " " + closing_braces) + + if cache_hash: + method_lines.append(tab + "if self.%s is None:" % _hash_cache_field) + if frozen: + append_hash_computation_lines( + "object.__setattr__(self, '%s', " % _hash_cache_field, tab * 2 + ) + method_lines.append(tab * 2 + ")") # close __setattr__ + else: + append_hash_computation_lines( + "self.%s = " % _hash_cache_field, tab * 2 + ) + method_lines.append(tab + "return self.%s" % _hash_cache_field) + else: + append_hash_computation_lines("return ", tab) + + script = "\n".join(method_lines) + return _make_method("__hash__", script, unique_filename) + + +def _add_hash(cls, attrs): + """ + Add a hash method to *cls*. + """ + cls.__hash__ = _make_hash(cls, attrs, frozen=False, cache_hash=False) + return cls + + +def _make_ne(): + """ + Create __ne__ method. + """ + + def __ne__(self, other): + """ + Check equality and either forward a NotImplemented or + return the result negated. + """ + result = self.__eq__(other) + if result is NotImplemented: + return NotImplemented + + return not result + + return __ne__ + + +def _make_eq(cls, attrs): + """ + Create __eq__ method for *cls* with *attrs*. + """ + attrs = [a for a in attrs if a.eq] + + unique_filename = _generate_unique_filename(cls, "eq") + lines = [ + "def __eq__(self, other):", + " if other.__class__ is not self.__class__:", + " return NotImplemented", + ] + + # We can't just do a big self.x = other.x and... clause due to + # irregularities like nan == nan is false but (nan,) == (nan,) is true. + globs = {} + if attrs: + lines.append(" return (") + others = [" ) == ("] + for a in attrs: + if a.eq_key: + cmp_name = "_%s_key" % (a.name,) + # Add the key function to the global namespace + # of the evaluated function. + globs[cmp_name] = a.eq_key + lines.append( + " %s(self.%s)," + % ( + cmp_name, + a.name, + ) + ) + others.append( + " %s(other.%s)," + % ( + cmp_name, + a.name, + ) + ) + else: + lines.append(" self.%s," % (a.name,)) + others.append(" other.%s," % (a.name,)) + + lines += others + [" )"] + else: + lines.append(" return True") + + script = "\n".join(lines) + + return _make_method("__eq__", script, unique_filename, globs) + + +def _make_order(cls, attrs): + """ + Create ordering methods for *cls* with *attrs*. + """ + attrs = [a for a in attrs if a.order] + + def attrs_to_tuple(obj): + """ + Save us some typing. + """ + return tuple( + key(value) if key else value + for value, key in ( + (getattr(obj, a.name), a.order_key) for a in attrs + ) + ) + + def __lt__(self, other): + """ + Automatically created by attrs. + """ + if other.__class__ is self.__class__: + return attrs_to_tuple(self) < attrs_to_tuple(other) + + return NotImplemented + + def __le__(self, other): + """ + Automatically created by attrs. + """ + if other.__class__ is self.__class__: + return attrs_to_tuple(self) <= attrs_to_tuple(other) + + return NotImplemented + + def __gt__(self, other): + """ + Automatically created by attrs. + """ + if other.__class__ is self.__class__: + return attrs_to_tuple(self) > attrs_to_tuple(other) + + return NotImplemented + + def __ge__(self, other): + """ + Automatically created by attrs. + """ + if other.__class__ is self.__class__: + return attrs_to_tuple(self) >= attrs_to_tuple(other) + + return NotImplemented + + return __lt__, __le__, __gt__, __ge__ + + +def _add_eq(cls, attrs=None): + """ + Add equality methods to *cls* with *attrs*. + """ + if attrs is None: + attrs = cls.__attrs_attrs__ + + cls.__eq__ = _make_eq(cls, attrs) + cls.__ne__ = _make_ne() + + return cls + + +_already_repring = threading.local() + + +def _make_repr(attrs, ns): + """ + Make a repr method that includes relevant *attrs*, adding *ns* to the full + name. + """ + + # Figure out which attributes to include, and which function to use to + # format them. The a.repr value can be either bool or a custom callable. + attr_names_with_reprs = tuple( + (a.name, repr if a.repr is True else a.repr) + for a in attrs + if a.repr is not False + ) + + def __repr__(self): + """ + Automatically created by attrs. + """ + try: + working_set = _already_repring.working_set + except AttributeError: + working_set = set() + _already_repring.working_set = working_set + + if id(self) in working_set: + return "..." + real_cls = self.__class__ + if ns is None: + qualname = getattr(real_cls, "__qualname__", None) + if qualname is not None: + class_name = qualname.rsplit(">.", 1)[-1] + else: + class_name = real_cls.__name__ + else: + class_name = ns + "." + real_cls.__name__ + + # Since 'self' remains on the stack (i.e.: strongly referenced) for the + # duration of this call, it's safe to depend on id(...) stability, and + # not need to track the instance and therefore worry about properties + # like weakref- or hash-ability. + working_set.add(id(self)) + try: + result = [class_name, "("] + first = True + for name, attr_repr in attr_names_with_reprs: + if first: + first = False + else: + result.append(", ") + result.extend( + (name, "=", attr_repr(getattr(self, name, NOTHING))) + ) + return "".join(result) + ")" + finally: + working_set.remove(id(self)) + + return __repr__ + + +def _add_repr(cls, ns=None, attrs=None): + """ + Add a repr method to *cls*. + """ + if attrs is None: + attrs = cls.__attrs_attrs__ + + cls.__repr__ = _make_repr(attrs, ns) + return cls + + +def fields(cls): + """ + Return the tuple of ``attrs`` attributes for a class. + + The tuple also allows accessing the fields by their names (see below for + examples). + + :param type cls: Class to introspect. + + :raise TypeError: If *cls* is not a class. + :raise attr.exceptions.NotAnAttrsClassError: If *cls* is not an ``attrs`` + class. + + :rtype: tuple (with name accessors) of `attr.Attribute` + + .. versionchanged:: 16.2.0 Returned tuple allows accessing the fields + by name. + """ + if not isclass(cls): + raise TypeError("Passed object must be a class.") + attrs = getattr(cls, "__attrs_attrs__", None) + if attrs is None: + raise NotAnAttrsClassError( + "{cls!r} is not an attrs-decorated class.".format(cls=cls) + ) + return attrs + + +def fields_dict(cls): + """ + Return an ordered dictionary of ``attrs`` attributes for a class, whose + keys are the attribute names. + + :param type cls: Class to introspect. + + :raise TypeError: If *cls* is not a class. + :raise attr.exceptions.NotAnAttrsClassError: If *cls* is not an ``attrs`` + class. + + :rtype: an ordered dict where keys are attribute names and values are + `attr.Attribute`\\ s. This will be a `dict` if it's + naturally ordered like on Python 3.6+ or an + :class:`~collections.OrderedDict` otherwise. + + .. versionadded:: 18.1.0 + """ + if not isclass(cls): + raise TypeError("Passed object must be a class.") + attrs = getattr(cls, "__attrs_attrs__", None) + if attrs is None: + raise NotAnAttrsClassError( + "{cls!r} is not an attrs-decorated class.".format(cls=cls) + ) + return ordered_dict(((a.name, a) for a in attrs)) + + +def validate(inst): + """ + Validate all attributes on *inst* that have a validator. + + Leaves all exceptions through. + + :param inst: Instance of a class with ``attrs`` attributes. + """ + if _config._run_validators is False: + return + + for a in fields(inst.__class__): + v = a.validator + if v is not None: + v(inst, a, getattr(inst, a.name)) + + +def _is_slot_cls(cls): + return "__slots__" in cls.__dict__ + + +def _is_slot_attr(a_name, base_attr_map): + """ + Check if the attribute name comes from a slot class. + """ + return a_name in base_attr_map and _is_slot_cls(base_attr_map[a_name]) + + +def _make_init( + cls, + attrs, + pre_init, + post_init, + frozen, + slots, + cache_hash, + base_attr_map, + is_exc, + has_global_on_setattr, + attrs_init, +): + if frozen and has_global_on_setattr: + raise ValueError("Frozen classes can't use on_setattr.") + + needs_cached_setattr = cache_hash or frozen + filtered_attrs = [] + attr_dict = {} + for a in attrs: + if not a.init and a.default is NOTHING: + continue + + filtered_attrs.append(a) + attr_dict[a.name] = a + + if a.on_setattr is not None: + if frozen is True: + raise ValueError("Frozen classes can't use on_setattr.") + + needs_cached_setattr = True + elif ( + has_global_on_setattr and a.on_setattr is not setters.NO_OP + ) or _is_slot_attr(a.name, base_attr_map): + needs_cached_setattr = True + + unique_filename = _generate_unique_filename(cls, "init") + + script, globs, annotations = _attrs_to_init_script( + filtered_attrs, + frozen, + slots, + pre_init, + post_init, + cache_hash, + base_attr_map, + is_exc, + needs_cached_setattr, + has_global_on_setattr, + attrs_init, + ) + if cls.__module__ in sys.modules: + # This makes typing.get_type_hints(CLS.__init__) resolve string types. + globs.update(sys.modules[cls.__module__].__dict__) + + globs.update({"NOTHING": NOTHING, "attr_dict": attr_dict}) + + if needs_cached_setattr: + # Save the lookup overhead in __init__ if we need to circumvent + # setattr hooks. + globs["_cached_setattr"] = _obj_setattr + + init = _make_method( + "__attrs_init__" if attrs_init else "__init__", + script, + unique_filename, + globs, + ) + init.__annotations__ = annotations + + return init + + +def _setattr(attr_name, value_var, has_on_setattr): + """ + Use the cached object.setattr to set *attr_name* to *value_var*. + """ + return "_setattr('%s', %s)" % (attr_name, value_var) + + +def _setattr_with_converter(attr_name, value_var, has_on_setattr): + """ + Use the cached object.setattr to set *attr_name* to *value_var*, but run + its converter first. + """ + return "_setattr('%s', %s(%s))" % ( + attr_name, + _init_converter_pat % (attr_name,), + value_var, + ) + + +def _assign(attr_name, value, has_on_setattr): + """ + Unless *attr_name* has an on_setattr hook, use normal assignment. Otherwise + relegate to _setattr. + """ + if has_on_setattr: + return _setattr(attr_name, value, True) + + return "self.%s = %s" % (attr_name, value) + + +def _assign_with_converter(attr_name, value_var, has_on_setattr): + """ + Unless *attr_name* has an on_setattr hook, use normal assignment after + conversion. Otherwise relegate to _setattr_with_converter. + """ + if has_on_setattr: + return _setattr_with_converter(attr_name, value_var, True) + + return "self.%s = %s(%s)" % ( + attr_name, + _init_converter_pat % (attr_name,), + value_var, + ) + + +if PY2: + + def _unpack_kw_only_py2(attr_name, default=None): + """ + Unpack *attr_name* from _kw_only dict. + """ + if default is not None: + arg_default = ", %s" % default + else: + arg_default = "" + return "%s = _kw_only.pop('%s'%s)" % ( + attr_name, + attr_name, + arg_default, + ) + + def _unpack_kw_only_lines_py2(kw_only_args): + """ + Unpack all *kw_only_args* from _kw_only dict and handle errors. + + Given a list of strings "{attr_name}" and "{attr_name}={default}" + generates list of lines of code that pop attrs from _kw_only dict and + raise TypeError similar to builtin if required attr is missing or + extra key is passed. + + >>> print("\n".join(_unpack_kw_only_lines_py2(["a", "b=42"]))) + try: + a = _kw_only.pop('a') + b = _kw_only.pop('b', 42) + except KeyError as _key_error: + raise TypeError( + ... + if _kw_only: + raise TypeError( + ... + """ + lines = ["try:"] + lines.extend( + " " + _unpack_kw_only_py2(*arg.split("=")) + for arg in kw_only_args + ) + lines += """\ +except KeyError as _key_error: + raise TypeError( + '__init__() missing required keyword-only argument: %s' % _key_error + ) +if _kw_only: + raise TypeError( + '__init__() got an unexpected keyword argument %r' + % next(iter(_kw_only)) + ) +""".split( + "\n" + ) + return lines + + +def _attrs_to_init_script( + attrs, + frozen, + slots, + pre_init, + post_init, + cache_hash, + base_attr_map, + is_exc, + needs_cached_setattr, + has_global_on_setattr, + attrs_init, +): + """ + Return a script of an initializer for *attrs* and a dict of globals. + + The globals are expected by the generated script. + + If *frozen* is True, we cannot set the attributes directly so we use + a cached ``object.__setattr__``. + """ + lines = [] + if pre_init: + lines.append("self.__attrs_pre_init__()") + + if needs_cached_setattr: + lines.append( + # Circumvent the __setattr__ descriptor to save one lookup per + # assignment. + # Note _setattr will be used again below if cache_hash is True + "_setattr = _cached_setattr.__get__(self, self.__class__)" + ) + + if frozen is True: + if slots is True: + fmt_setter = _setattr + fmt_setter_with_converter = _setattr_with_converter + else: + # Dict frozen classes assign directly to __dict__. + # But only if the attribute doesn't come from an ancestor slot + # class. + # Note _inst_dict will be used again below if cache_hash is True + lines.append("_inst_dict = self.__dict__") + + def fmt_setter(attr_name, value_var, has_on_setattr): + if _is_slot_attr(attr_name, base_attr_map): + return _setattr(attr_name, value_var, has_on_setattr) + + return "_inst_dict['%s'] = %s" % (attr_name, value_var) + + def fmt_setter_with_converter( + attr_name, value_var, has_on_setattr + ): + if has_on_setattr or _is_slot_attr(attr_name, base_attr_map): + return _setattr_with_converter( + attr_name, value_var, has_on_setattr + ) + + return "_inst_dict['%s'] = %s(%s)" % ( + attr_name, + _init_converter_pat % (attr_name,), + value_var, + ) + + else: + # Not frozen. + fmt_setter = _assign + fmt_setter_with_converter = _assign_with_converter + + args = [] + kw_only_args = [] + attrs_to_validate = [] + + # This is a dictionary of names to validator and converter callables. + # Injecting this into __init__ globals lets us avoid lookups. + names_for_globals = {} + annotations = {"return": None} + + for a in attrs: + if a.validator: + attrs_to_validate.append(a) + + attr_name = a.name + has_on_setattr = a.on_setattr is not None or ( + a.on_setattr is not setters.NO_OP and has_global_on_setattr + ) + arg_name = a.name.lstrip("_") + + has_factory = isinstance(a.default, Factory) + if has_factory and a.default.takes_self: + maybe_self = "self" + else: + maybe_self = "" + + if a.init is False: + if has_factory: + init_factory_name = _init_factory_pat.format(a.name) + if a.converter is not None: + lines.append( + fmt_setter_with_converter( + attr_name, + init_factory_name + "(%s)" % (maybe_self,), + has_on_setattr, + ) + ) + conv_name = _init_converter_pat % (a.name,) + names_for_globals[conv_name] = a.converter + else: + lines.append( + fmt_setter( + attr_name, + init_factory_name + "(%s)" % (maybe_self,), + has_on_setattr, + ) + ) + names_for_globals[init_factory_name] = a.default.factory + else: + if a.converter is not None: + lines.append( + fmt_setter_with_converter( + attr_name, + "attr_dict['%s'].default" % (attr_name,), + has_on_setattr, + ) + ) + conv_name = _init_converter_pat % (a.name,) + names_for_globals[conv_name] = a.converter + else: + lines.append( + fmt_setter( + attr_name, + "attr_dict['%s'].default" % (attr_name,), + has_on_setattr, + ) + ) + elif a.default is not NOTHING and not has_factory: + arg = "%s=attr_dict['%s'].default" % (arg_name, attr_name) + if a.kw_only: + kw_only_args.append(arg) + else: + args.append(arg) + + if a.converter is not None: + lines.append( + fmt_setter_with_converter( + attr_name, arg_name, has_on_setattr + ) + ) + names_for_globals[ + _init_converter_pat % (a.name,) + ] = a.converter + else: + lines.append(fmt_setter(attr_name, arg_name, has_on_setattr)) + + elif has_factory: + arg = "%s=NOTHING" % (arg_name,) + if a.kw_only: + kw_only_args.append(arg) + else: + args.append(arg) + lines.append("if %s is not NOTHING:" % (arg_name,)) + + init_factory_name = _init_factory_pat.format(a.name) + if a.converter is not None: + lines.append( + " " + + fmt_setter_with_converter( + attr_name, arg_name, has_on_setattr + ) + ) + lines.append("else:") + lines.append( + " " + + fmt_setter_with_converter( + attr_name, + init_factory_name + "(" + maybe_self + ")", + has_on_setattr, + ) + ) + names_for_globals[ + _init_converter_pat % (a.name,) + ] = a.converter + else: + lines.append( + " " + fmt_setter(attr_name, arg_name, has_on_setattr) + ) + lines.append("else:") + lines.append( + " " + + fmt_setter( + attr_name, + init_factory_name + "(" + maybe_self + ")", + has_on_setattr, + ) + ) + names_for_globals[init_factory_name] = a.default.factory + else: + if a.kw_only: + kw_only_args.append(arg_name) + else: + args.append(arg_name) + + if a.converter is not None: + lines.append( + fmt_setter_with_converter( + attr_name, arg_name, has_on_setattr + ) + ) + names_for_globals[ + _init_converter_pat % (a.name,) + ] = a.converter + else: + lines.append(fmt_setter(attr_name, arg_name, has_on_setattr)) + + if a.init is True: + if a.type is not None and a.converter is None: + annotations[arg_name] = a.type + elif a.converter is not None and not PY2: + # Try to get the type from the converter. + sig = None + try: + sig = inspect.signature(a.converter) + except (ValueError, TypeError): # inspect failed + pass + if sig: + sig_params = list(sig.parameters.values()) + if ( + sig_params + and sig_params[0].annotation + is not inspect.Parameter.empty + ): + annotations[arg_name] = sig_params[0].annotation + + if attrs_to_validate: # we can skip this if there are no validators. + names_for_globals["_config"] = _config + lines.append("if _config._run_validators is True:") + for a in attrs_to_validate: + val_name = "__attr_validator_" + a.name + attr_name = "__attr_" + a.name + lines.append( + " %s(self, %s, self.%s)" % (val_name, attr_name, a.name) + ) + names_for_globals[val_name] = a.validator + names_for_globals[attr_name] = a + + if post_init: + lines.append("self.__attrs_post_init__()") + + # because this is set only after __attrs_post_init is called, a crash + # will result if post-init tries to access the hash code. This seemed + # preferable to setting this beforehand, in which case alteration to + # field values during post-init combined with post-init accessing the + # hash code would result in silent bugs. + if cache_hash: + if frozen: + if slots: + # if frozen and slots, then _setattr defined above + init_hash_cache = "_setattr('%s', %s)" + else: + # if frozen and not slots, then _inst_dict defined above + init_hash_cache = "_inst_dict['%s'] = %s" + else: + init_hash_cache = "self.%s = %s" + lines.append(init_hash_cache % (_hash_cache_field, "None")) + + # For exceptions we rely on BaseException.__init__ for proper + # initialization. + if is_exc: + vals = ",".join("self." + a.name for a in attrs if a.init) + + lines.append("BaseException.__init__(self, %s)" % (vals,)) + + args = ", ".join(args) + if kw_only_args: + if PY2: + lines = _unpack_kw_only_lines_py2(kw_only_args) + lines + + args += "%s**_kw_only" % (", " if args else "",) # leading comma + else: + args += "%s*, %s" % ( + ", " if args else "", # leading comma + ", ".join(kw_only_args), # kw_only args + ) + return ( + """\ +def {init_name}(self, {args}): + {lines} +""".format( + init_name=("__attrs_init__" if attrs_init else "__init__"), + args=args, + lines="\n ".join(lines) if lines else "pass", + ), + names_for_globals, + annotations, + ) + + +class Attribute(object): + """ + *Read-only* representation of an attribute. + + Instances of this class are frequently used for introspection purposes + like: + + - `fields` returns a tuple of them. + - Validators get them passed as the first argument. + - The *field transformer* hook receives a list of them. + + :attribute name: The name of the attribute. + :attribute inherited: Whether or not that attribute has been inherited from + a base class. + + Plus *all* arguments of `attr.ib` (except for ``factory`` + which is only syntactic sugar for ``default=Factory(...)``. + + .. versionadded:: 20.1.0 *inherited* + .. versionadded:: 20.1.0 *on_setattr* + .. versionchanged:: 20.2.0 *inherited* is not taken into account for + equality checks and hashing anymore. + .. versionadded:: 21.1.0 *eq_key* and *order_key* + + For the full version history of the fields, see `attr.ib`. + """ + + __slots__ = ( + "name", + "default", + "validator", + "repr", + "eq", + "eq_key", + "order", + "order_key", + "hash", + "init", + "metadata", + "type", + "converter", + "kw_only", + "inherited", + "on_setattr", + ) + + def __init__( + self, + name, + default, + validator, + repr, + cmp, # XXX: unused, remove along with other cmp code. + hash, + init, + inherited, + metadata=None, + type=None, + converter=None, + kw_only=False, + eq=None, + eq_key=None, + order=None, + order_key=None, + on_setattr=None, + ): + eq, eq_key, order, order_key = _determine_attrib_eq_order( + cmp, eq_key or eq, order_key or order, True + ) + + # Cache this descriptor here to speed things up later. + bound_setattr = _obj_setattr.__get__(self, Attribute) + + # Despite the big red warning, people *do* instantiate `Attribute` + # themselves. + bound_setattr("name", name) + bound_setattr("default", default) + bound_setattr("validator", validator) + bound_setattr("repr", repr) + bound_setattr("eq", eq) + bound_setattr("eq_key", eq_key) + bound_setattr("order", order) + bound_setattr("order_key", order_key) + bound_setattr("hash", hash) + bound_setattr("init", init) + bound_setattr("converter", converter) + bound_setattr( + "metadata", + ( + metadata_proxy(metadata) + if metadata + else _empty_metadata_singleton + ), + ) + bound_setattr("type", type) + bound_setattr("kw_only", kw_only) + bound_setattr("inherited", inherited) + bound_setattr("on_setattr", on_setattr) + + def __setattr__(self, name, value): + raise FrozenInstanceError() + + @classmethod + def from_counting_attr(cls, name, ca, type=None): + # type holds the annotated value. deal with conflicts: + if type is None: + type = ca.type + elif ca.type is not None: + raise ValueError( + "Type annotation and type argument cannot both be present" + ) + inst_dict = { + k: getattr(ca, k) + for k in Attribute.__slots__ + if k + not in ( + "name", + "validator", + "default", + "type", + "inherited", + ) # exclude methods and deprecated alias + } + return cls( + name=name, + validator=ca._validator, + default=ca._default, + type=type, + cmp=None, + inherited=False, + **inst_dict + ) + + @property + def cmp(self): + """ + Simulate the presence of a cmp attribute and warn. + """ + warnings.warn(_CMP_DEPRECATION, DeprecationWarning, stacklevel=2) + + return self.eq and self.order + + # Don't use attr.evolve since fields(Attribute) doesn't work + def evolve(self, **changes): + """ + Copy *self* and apply *changes*. + + This works similarly to `attr.evolve` but that function does not work + with ``Attribute``. + + It is mainly meant to be used for `transform-fields`. + + .. versionadded:: 20.3.0 + """ + new = copy.copy(self) + + new._setattrs(changes.items()) + + return new + + # Don't use _add_pickle since fields(Attribute) doesn't work + def __getstate__(self): + """ + Play nice with pickle. + """ + return tuple( + getattr(self, name) if name != "metadata" else dict(self.metadata) + for name in self.__slots__ + ) + + def __setstate__(self, state): + """ + Play nice with pickle. + """ + self._setattrs(zip(self.__slots__, state)) + + def _setattrs(self, name_values_pairs): + bound_setattr = _obj_setattr.__get__(self, Attribute) + for name, value in name_values_pairs: + if name != "metadata": + bound_setattr(name, value) + else: + bound_setattr( + name, + metadata_proxy(value) + if value + else _empty_metadata_singleton, + ) + + +_a = [ + Attribute( + name=name, + default=NOTHING, + validator=None, + repr=True, + cmp=None, + eq=True, + order=False, + hash=(name != "metadata"), + init=True, + inherited=False, + ) + for name in Attribute.__slots__ +] + +Attribute = _add_hash( + _add_eq( + _add_repr(Attribute, attrs=_a), + attrs=[a for a in _a if a.name != "inherited"], + ), + attrs=[a for a in _a if a.hash and a.name != "inherited"], +) + + +class _CountingAttr(object): + """ + Intermediate representation of attributes that uses a counter to preserve + the order in which the attributes have been defined. + + *Internal* data structure of the attrs library. Running into is most + likely the result of a bug like a forgotten `@attr.s` decorator. + """ + + __slots__ = ( + "counter", + "_default", + "repr", + "eq", + "eq_key", + "order", + "order_key", + "hash", + "init", + "metadata", + "_validator", + "converter", + "type", + "kw_only", + "on_setattr", + ) + __attrs_attrs__ = tuple( + Attribute( + name=name, + default=NOTHING, + validator=None, + repr=True, + cmp=None, + hash=True, + init=True, + kw_only=False, + eq=True, + eq_key=None, + order=False, + order_key=None, + inherited=False, + on_setattr=None, + ) + for name in ( + "counter", + "_default", + "repr", + "eq", + "order", + "hash", + "init", + "on_setattr", + ) + ) + ( + Attribute( + name="metadata", + default=None, + validator=None, + repr=True, + cmp=None, + hash=False, + init=True, + kw_only=False, + eq=True, + eq_key=None, + order=False, + order_key=None, + inherited=False, + on_setattr=None, + ), + ) + cls_counter = 0 + + def __init__( + self, + default, + validator, + repr, + cmp, + hash, + init, + converter, + metadata, + type, + kw_only, + eq, + eq_key, + order, + order_key, + on_setattr, + ): + _CountingAttr.cls_counter += 1 + self.counter = _CountingAttr.cls_counter + self._default = default + self._validator = validator + self.converter = converter + self.repr = repr + self.eq = eq + self.eq_key = eq_key + self.order = order + self.order_key = order_key + self.hash = hash + self.init = init + self.metadata = metadata + self.type = type + self.kw_only = kw_only + self.on_setattr = on_setattr + + def validator(self, meth): + """ + Decorator that adds *meth* to the list of validators. + + Returns *meth* unchanged. + + .. versionadded:: 17.1.0 + """ + if self._validator is None: + self._validator = meth + else: + self._validator = and_(self._validator, meth) + return meth + + def default(self, meth): + """ + Decorator that allows to set the default for an attribute. + + Returns *meth* unchanged. + + :raises DefaultAlreadySetError: If default has been set before. + + .. versionadded:: 17.1.0 + """ + if self._default is not NOTHING: + raise DefaultAlreadySetError() + + self._default = Factory(meth, takes_self=True) + + return meth + + +_CountingAttr = _add_eq(_add_repr(_CountingAttr)) + + +class Factory(object): + """ + Stores a factory callable. + + If passed as the default value to `attr.ib`, the factory is used to + generate a new value. + + :param callable factory: A callable that takes either none or exactly one + mandatory positional argument depending on *takes_self*. + :param bool takes_self: Pass the partially initialized instance that is + being initialized as a positional argument. + + .. versionadded:: 17.1.0 *takes_self* + """ + + __slots__ = ("factory", "takes_self") + + def __init__(self, factory, takes_self=False): + """ + `Factory` is part of the default machinery so if we want a default + value here, we have to implement it ourselves. + """ + self.factory = factory + self.takes_self = takes_self + + def __getstate__(self): + """ + Play nice with pickle. + """ + return tuple(getattr(self, name) for name in self.__slots__) + + def __setstate__(self, state): + """ + Play nice with pickle. + """ + for name, value in zip(self.__slots__, state): + setattr(self, name, value) + + +_f = [ + Attribute( + name=name, + default=NOTHING, + validator=None, + repr=True, + cmp=None, + eq=True, + order=False, + hash=True, + init=True, + inherited=False, + ) + for name in Factory.__slots__ +] + +Factory = _add_hash(_add_eq(_add_repr(Factory, attrs=_f), attrs=_f), attrs=_f) + + +def make_class(name, attrs, bases=(object,), **attributes_arguments): + """ + A quick way to create a new class called *name* with *attrs*. + + :param str name: The name for the new class. + + :param attrs: A list of names or a dictionary of mappings of names to + attributes. + + If *attrs* is a list or an ordered dict (`dict` on Python 3.6+, + `collections.OrderedDict` otherwise), the order is deduced from + the order of the names or attributes inside *attrs*. Otherwise the + order of the definition of the attributes is used. + :type attrs: `list` or `dict` + + :param tuple bases: Classes that the new class will subclass. + + :param attributes_arguments: Passed unmodified to `attr.s`. + + :return: A new class with *attrs*. + :rtype: type + + .. versionadded:: 17.1.0 *bases* + .. versionchanged:: 18.1.0 If *attrs* is ordered, the order is retained. + """ + if isinstance(attrs, dict): + cls_dict = attrs + elif isinstance(attrs, (list, tuple)): + cls_dict = dict((a, attrib()) for a in attrs) + else: + raise TypeError("attrs argument must be a dict or a list.") + + pre_init = cls_dict.pop("__attrs_pre_init__", None) + post_init = cls_dict.pop("__attrs_post_init__", None) + user_init = cls_dict.pop("__init__", None) + + body = {} + if pre_init is not None: + body["__attrs_pre_init__"] = pre_init + if post_init is not None: + body["__attrs_post_init__"] = post_init + if user_init is not None: + body["__init__"] = user_init + + type_ = new_class(name, bases, {}, lambda ns: ns.update(body)) + + # For pickling to work, the __module__ variable needs to be set to the + # frame where the class is created. Bypass this step in environments where + # sys._getframe is not defined (Jython for example) or sys._getframe is not + # defined for arguments greater than 0 (IronPython). + try: + type_.__module__ = sys._getframe(1).f_globals.get( + "__name__", "__main__" + ) + except (AttributeError, ValueError): + pass + + # We do it here for proper warnings with meaningful stacklevel. + cmp = attributes_arguments.pop("cmp", None) + ( + attributes_arguments["eq"], + attributes_arguments["order"], + ) = _determine_attrs_eq_order( + cmp, + attributes_arguments.get("eq"), + attributes_arguments.get("order"), + True, + ) + + return _attrs(these=cls_dict, **attributes_arguments)(type_) + + +# These are required by within this module so we define them here and merely +# import into .validators / .converters. + + +@attrs(slots=True, hash=True) +class _AndValidator(object): + """ + Compose many validators to a single one. + """ + + _validators = attrib() + + def __call__(self, inst, attr, value): + for v in self._validators: + v(inst, attr, value) + + +def and_(*validators): + """ + A validator that composes multiple validators into one. + + When called on a value, it runs all wrapped validators. + + :param callables validators: Arbitrary number of validators. + + .. versionadded:: 17.1.0 + """ + vals = [] + for validator in validators: + vals.extend( + validator._validators + if isinstance(validator, _AndValidator) + else [validator] + ) + + return _AndValidator(tuple(vals)) + + +def pipe(*converters): + """ + A converter that composes multiple converters into one. + + When called on a value, it runs all wrapped converters, returning the + *last* value. + + Type annotations will be inferred from the wrapped converters', if + they have any. + + :param callables converters: Arbitrary number of converters. + + .. versionadded:: 20.1.0 + """ + + def pipe_converter(val): + for converter in converters: + val = converter(val) + + return val + + if not PY2: + if not converters: + # If the converter list is empty, pipe_converter is the identity. + A = typing.TypeVar("A") + pipe_converter.__annotations__ = {"val": A, "return": A} + else: + # Get parameter type. + sig = None + try: + sig = inspect.signature(converters[0]) + except (ValueError, TypeError): # inspect failed + pass + if sig: + params = list(sig.parameters.values()) + if ( + params + and params[0].annotation is not inspect.Parameter.empty + ): + pipe_converter.__annotations__["val"] = params[ + 0 + ].annotation + # Get return type. + sig = None + try: + sig = inspect.signature(converters[-1]) + except (ValueError, TypeError): # inspect failed + pass + if sig and sig.return_annotation is not inspect.Signature().empty: + pipe_converter.__annotations__[ + "return" + ] = sig.return_annotation + + return pipe_converter diff --git a/openpype/hosts/fusion/vendor/attr/_next_gen.py b/openpype/hosts/fusion/vendor/attr/_next_gen.py new file mode 100644 index 0000000000..fab0af966a --- /dev/null +++ b/openpype/hosts/fusion/vendor/attr/_next_gen.py @@ -0,0 +1,158 @@ +""" +These are Python 3.6+-only and keyword-only APIs that call `attr.s` and +`attr.ib` with different default values. +""" + +from functools import partial + +from attr.exceptions import UnannotatedAttributeError + +from . import setters +from ._make import NOTHING, _frozen_setattrs, attrib, attrs + + +def define( + maybe_cls=None, + *, + these=None, + repr=None, + hash=None, + init=None, + slots=True, + frozen=False, + weakref_slot=True, + str=False, + auto_attribs=None, + kw_only=False, + cache_hash=False, + auto_exc=True, + eq=None, + order=False, + auto_detect=True, + getstate_setstate=None, + on_setattr=None, + field_transformer=None, +): + r""" + The only behavioral differences are the handling of the *auto_attribs* + option: + + :param Optional[bool] auto_attribs: If set to `True` or `False`, it behaves + exactly like `attr.s`. If left `None`, `attr.s` will try to guess: + + 1. If any attributes are annotated and no unannotated `attr.ib`\ s + are found, it assumes *auto_attribs=True*. + 2. Otherwise it assumes *auto_attribs=False* and tries to collect + `attr.ib`\ s. + + and that mutable classes (``frozen=False``) validate on ``__setattr__``. + + .. versionadded:: 20.1.0 + """ + + def do_it(cls, auto_attribs): + return attrs( + maybe_cls=cls, + these=these, + repr=repr, + hash=hash, + init=init, + slots=slots, + frozen=frozen, + weakref_slot=weakref_slot, + str=str, + auto_attribs=auto_attribs, + kw_only=kw_only, + cache_hash=cache_hash, + auto_exc=auto_exc, + eq=eq, + order=order, + auto_detect=auto_detect, + collect_by_mro=True, + getstate_setstate=getstate_setstate, + on_setattr=on_setattr, + field_transformer=field_transformer, + ) + + def wrap(cls): + """ + Making this a wrapper ensures this code runs during class creation. + + We also ensure that frozen-ness of classes is inherited. + """ + nonlocal frozen, on_setattr + + had_on_setattr = on_setattr not in (None, setters.NO_OP) + + # By default, mutable classes validate on setattr. + if frozen is False and on_setattr is None: + on_setattr = setters.validate + + # However, if we subclass a frozen class, we inherit the immutability + # and disable on_setattr. + for base_cls in cls.__bases__: + if base_cls.__setattr__ is _frozen_setattrs: + if had_on_setattr: + raise ValueError( + "Frozen classes can't use on_setattr " + "(frozen-ness was inherited)." + ) + + on_setattr = setters.NO_OP + break + + if auto_attribs is not None: + return do_it(cls, auto_attribs) + + try: + return do_it(cls, True) + except UnannotatedAttributeError: + return do_it(cls, False) + + # maybe_cls's type depends on the usage of the decorator. It's a class + # if it's used as `@attrs` but ``None`` if used as `@attrs()`. + if maybe_cls is None: + return wrap + else: + return wrap(maybe_cls) + + +mutable = define +frozen = partial(define, frozen=True, on_setattr=None) + + +def field( + *, + default=NOTHING, + validator=None, + repr=True, + hash=None, + init=True, + metadata=None, + converter=None, + factory=None, + kw_only=False, + eq=None, + order=None, + on_setattr=None, +): + """ + Identical to `attr.ib`, except keyword-only and with some arguments + removed. + + .. versionadded:: 20.1.0 + """ + return attrib( + default=default, + validator=validator, + repr=repr, + hash=hash, + init=init, + metadata=metadata, + converter=converter, + factory=factory, + kw_only=kw_only, + eq=eq, + order=order, + on_setattr=on_setattr, + ) diff --git a/openpype/hosts/fusion/vendor/attr/_version_info.py b/openpype/hosts/fusion/vendor/attr/_version_info.py new file mode 100644 index 0000000000..014e78a1b4 --- /dev/null +++ b/openpype/hosts/fusion/vendor/attr/_version_info.py @@ -0,0 +1,85 @@ +from __future__ import absolute_import, division, print_function + +from functools import total_ordering + +from ._funcs import astuple +from ._make import attrib, attrs + + +@total_ordering +@attrs(eq=False, order=False, slots=True, frozen=True) +class VersionInfo(object): + """ + A version object that can be compared to tuple of length 1--4: + + >>> attr.VersionInfo(19, 1, 0, "final") <= (19, 2) + True + >>> attr.VersionInfo(19, 1, 0, "final") < (19, 1, 1) + True + >>> vi = attr.VersionInfo(19, 2, 0, "final") + >>> vi < (19, 1, 1) + False + >>> vi < (19,) + False + >>> vi == (19, 2,) + True + >>> vi == (19, 2, 1) + False + + .. versionadded:: 19.2 + """ + + year = attrib(type=int) + minor = attrib(type=int) + micro = attrib(type=int) + releaselevel = attrib(type=str) + + @classmethod + def _from_version_string(cls, s): + """ + Parse *s* and return a _VersionInfo. + """ + v = s.split(".") + if len(v) == 3: + v.append("final") + + return cls( + year=int(v[0]), minor=int(v[1]), micro=int(v[2]), releaselevel=v[3] + ) + + def _ensure_tuple(self, other): + """ + Ensure *other* is a tuple of a valid length. + + Returns a possibly transformed *other* and ourselves as a tuple of + the same length as *other*. + """ + + if self.__class__ is other.__class__: + other = astuple(other) + + if not isinstance(other, tuple): + raise NotImplementedError + + if not (1 <= len(other) <= 4): + raise NotImplementedError + + return astuple(self)[: len(other)], other + + def __eq__(self, other): + try: + us, them = self._ensure_tuple(other) + except NotImplementedError: + return NotImplemented + + return us == them + + def __lt__(self, other): + try: + us, them = self._ensure_tuple(other) + except NotImplementedError: + return NotImplemented + + # Since alphabetically "dev0" < "final" < "post1" < "post2", we don't + # have to do anything special with releaselevel for now. + return us < them diff --git a/openpype/hosts/fusion/vendor/attr/_version_info.pyi b/openpype/hosts/fusion/vendor/attr/_version_info.pyi new file mode 100644 index 0000000000..45ced08633 --- /dev/null +++ b/openpype/hosts/fusion/vendor/attr/_version_info.pyi @@ -0,0 +1,9 @@ +class VersionInfo: + @property + def year(self) -> int: ... + @property + def minor(self) -> int: ... + @property + def micro(self) -> int: ... + @property + def releaselevel(self) -> str: ... diff --git a/openpype/hosts/fusion/vendor/attr/converters.py b/openpype/hosts/fusion/vendor/attr/converters.py new file mode 100644 index 0000000000..2777db6d0a --- /dev/null +++ b/openpype/hosts/fusion/vendor/attr/converters.py @@ -0,0 +1,111 @@ +""" +Commonly useful converters. +""" + +from __future__ import absolute_import, division, print_function + +from ._compat import PY2 +from ._make import NOTHING, Factory, pipe + + +if not PY2: + import inspect + import typing + + +__all__ = [ + "pipe", + "optional", + "default_if_none", +] + + +def optional(converter): + """ + A converter that allows an attribute to be optional. An optional attribute + is one which can be set to ``None``. + + Type annotations will be inferred from the wrapped converter's, if it + has any. + + :param callable converter: the converter that is used for non-``None`` + values. + + .. versionadded:: 17.1.0 + """ + + def optional_converter(val): + if val is None: + return None + return converter(val) + + if not PY2: + sig = None + try: + sig = inspect.signature(converter) + except (ValueError, TypeError): # inspect failed + pass + if sig: + params = list(sig.parameters.values()) + if params and params[0].annotation is not inspect.Parameter.empty: + optional_converter.__annotations__["val"] = typing.Optional[ + params[0].annotation + ] + if sig.return_annotation is not inspect.Signature.empty: + optional_converter.__annotations__["return"] = typing.Optional[ + sig.return_annotation + ] + + return optional_converter + + +def default_if_none(default=NOTHING, factory=None): + """ + A converter that allows to replace ``None`` values by *default* or the + result of *factory*. + + :param default: Value to be used if ``None`` is passed. Passing an instance + of `attr.Factory` is supported, however the ``takes_self`` option + is *not*. + :param callable factory: A callable that takes no parameters whose result + is used if ``None`` is passed. + + :raises TypeError: If **neither** *default* or *factory* is passed. + :raises TypeError: If **both** *default* and *factory* are passed. + :raises ValueError: If an instance of `attr.Factory` is passed with + ``takes_self=True``. + + .. versionadded:: 18.2.0 + """ + if default is NOTHING and factory is None: + raise TypeError("Must pass either `default` or `factory`.") + + if default is not NOTHING and factory is not None: + raise TypeError( + "Must pass either `default` or `factory` but not both." + ) + + if factory is not None: + default = Factory(factory) + + if isinstance(default, Factory): + if default.takes_self: + raise ValueError( + "`takes_self` is not supported by default_if_none." + ) + + def default_if_none_converter(val): + if val is not None: + return val + + return default.factory() + + else: + + def default_if_none_converter(val): + if val is not None: + return val + + return default + + return default_if_none_converter diff --git a/openpype/hosts/fusion/vendor/attr/converters.pyi b/openpype/hosts/fusion/vendor/attr/converters.pyi new file mode 100644 index 0000000000..84a57590b0 --- /dev/null +++ b/openpype/hosts/fusion/vendor/attr/converters.pyi @@ -0,0 +1,13 @@ +from typing import Callable, Optional, TypeVar, overload + +from . import _ConverterType + + +_T = TypeVar("_T") + +def pipe(*validators: _ConverterType) -> _ConverterType: ... +def optional(converter: _ConverterType) -> _ConverterType: ... +@overload +def default_if_none(default: _T) -> _ConverterType: ... +@overload +def default_if_none(*, factory: Callable[[], _T]) -> _ConverterType: ... diff --git a/openpype/hosts/fusion/vendor/attr/exceptions.py b/openpype/hosts/fusion/vendor/attr/exceptions.py new file mode 100644 index 0000000000..f6f9861bea --- /dev/null +++ b/openpype/hosts/fusion/vendor/attr/exceptions.py @@ -0,0 +1,92 @@ +from __future__ import absolute_import, division, print_function + + +class FrozenError(AttributeError): + """ + A frozen/immutable instance or attribute have been attempted to be + modified. + + It mirrors the behavior of ``namedtuples`` by using the same error message + and subclassing `AttributeError`. + + .. versionadded:: 20.1.0 + """ + + msg = "can't set attribute" + args = [msg] + + +class FrozenInstanceError(FrozenError): + """ + A frozen instance has been attempted to be modified. + + .. versionadded:: 16.1.0 + """ + + +class FrozenAttributeError(FrozenError): + """ + A frozen attribute has been attempted to be modified. + + .. versionadded:: 20.1.0 + """ + + +class AttrsAttributeNotFoundError(ValueError): + """ + An ``attrs`` function couldn't find an attribute that the user asked for. + + .. versionadded:: 16.2.0 + """ + + +class NotAnAttrsClassError(ValueError): + """ + A non-``attrs`` class has been passed into an ``attrs`` function. + + .. versionadded:: 16.2.0 + """ + + +class DefaultAlreadySetError(RuntimeError): + """ + A default has been set using ``attr.ib()`` and is attempted to be reset + using the decorator. + + .. versionadded:: 17.1.0 + """ + + +class UnannotatedAttributeError(RuntimeError): + """ + A class with ``auto_attribs=True`` has an ``attr.ib()`` without a type + annotation. + + .. versionadded:: 17.3.0 + """ + + +class PythonTooOldError(RuntimeError): + """ + It was attempted to use an ``attrs`` feature that requires a newer Python + version. + + .. versionadded:: 18.2.0 + """ + + +class NotCallableError(TypeError): + """ + A ``attr.ib()`` requiring a callable has been set with a value + that is not callable. + + .. versionadded:: 19.2.0 + """ + + def __init__(self, msg, value): + super(TypeError, self).__init__(msg, value) + self.msg = msg + self.value = value + + def __str__(self): + return str(self.msg) diff --git a/openpype/hosts/fusion/vendor/attr/exceptions.pyi b/openpype/hosts/fusion/vendor/attr/exceptions.pyi new file mode 100644 index 0000000000..a800fb26bb --- /dev/null +++ b/openpype/hosts/fusion/vendor/attr/exceptions.pyi @@ -0,0 +1,18 @@ +from typing import Any + + +class FrozenError(AttributeError): + msg: str = ... + +class FrozenInstanceError(FrozenError): ... +class FrozenAttributeError(FrozenError): ... +class AttrsAttributeNotFoundError(ValueError): ... +class NotAnAttrsClassError(ValueError): ... +class DefaultAlreadySetError(RuntimeError): ... +class UnannotatedAttributeError(RuntimeError): ... +class PythonTooOldError(RuntimeError): ... + +class NotCallableError(TypeError): + msg: str = ... + value: Any = ... + def __init__(self, msg: str, value: Any) -> None: ... diff --git a/openpype/hosts/fusion/vendor/attr/filters.py b/openpype/hosts/fusion/vendor/attr/filters.py new file mode 100644 index 0000000000..dc47e8fa38 --- /dev/null +++ b/openpype/hosts/fusion/vendor/attr/filters.py @@ -0,0 +1,52 @@ +""" +Commonly useful filters for `attr.asdict`. +""" + +from __future__ import absolute_import, division, print_function + +from ._compat import isclass +from ._make import Attribute + + +def _split_what(what): + """ + Returns a tuple of `frozenset`s of classes and attributes. + """ + return ( + frozenset(cls for cls in what if isclass(cls)), + frozenset(cls for cls in what if isinstance(cls, Attribute)), + ) + + +def include(*what): + """ + Whitelist *what*. + + :param what: What to whitelist. + :type what: `list` of `type` or `attr.Attribute`\\ s + + :rtype: `callable` + """ + cls, attrs = _split_what(what) + + def include_(attribute, value): + return value.__class__ in cls or attribute in attrs + + return include_ + + +def exclude(*what): + """ + Blacklist *what*. + + :param what: What to blacklist. + :type what: `list` of classes or `attr.Attribute`\\ s. + + :rtype: `callable` + """ + cls, attrs = _split_what(what) + + def exclude_(attribute, value): + return value.__class__ not in cls and attribute not in attrs + + return exclude_ diff --git a/openpype/hosts/fusion/vendor/attr/filters.pyi b/openpype/hosts/fusion/vendor/attr/filters.pyi new file mode 100644 index 0000000000..f7b63f1bb4 --- /dev/null +++ b/openpype/hosts/fusion/vendor/attr/filters.pyi @@ -0,0 +1,7 @@ +from typing import Any, Union + +from . import Attribute, _FilterType + + +def include(*what: Union[type, Attribute[Any]]) -> _FilterType[Any]: ... +def exclude(*what: Union[type, Attribute[Any]]) -> _FilterType[Any]: ... diff --git a/openpype/hosts/fusion/vendor/attr/py.typed b/openpype/hosts/fusion/vendor/attr/py.typed new file mode 100644 index 0000000000..e69de29bb2 diff --git a/openpype/hosts/fusion/vendor/attr/setters.py b/openpype/hosts/fusion/vendor/attr/setters.py new file mode 100644 index 0000000000..240014b3c1 --- /dev/null +++ b/openpype/hosts/fusion/vendor/attr/setters.py @@ -0,0 +1,77 @@ +""" +Commonly used hooks for on_setattr. +""" + +from __future__ import absolute_import, division, print_function + +from . import _config +from .exceptions import FrozenAttributeError + + +def pipe(*setters): + """ + Run all *setters* and return the return value of the last one. + + .. versionadded:: 20.1.0 + """ + + def wrapped_pipe(instance, attrib, new_value): + rv = new_value + + for setter in setters: + rv = setter(instance, attrib, rv) + + return rv + + return wrapped_pipe + + +def frozen(_, __, ___): + """ + Prevent an attribute to be modified. + + .. versionadded:: 20.1.0 + """ + raise FrozenAttributeError() + + +def validate(instance, attrib, new_value): + """ + Run *attrib*'s validator on *new_value* if it has one. + + .. versionadded:: 20.1.0 + """ + if _config._run_validators is False: + return new_value + + v = attrib.validator + if not v: + return new_value + + v(instance, attrib, new_value) + + return new_value + + +def convert(instance, attrib, new_value): + """ + Run *attrib*'s converter -- if it has one -- on *new_value* and return the + result. + + .. versionadded:: 20.1.0 + """ + c = attrib.converter + if c: + return c(new_value) + + return new_value + + +NO_OP = object() +""" +Sentinel for disabling class-wide *on_setattr* hooks for certain attributes. + +Does not work in `pipe` or within lists. + +.. versionadded:: 20.1.0 +""" diff --git a/openpype/hosts/fusion/vendor/attr/setters.pyi b/openpype/hosts/fusion/vendor/attr/setters.pyi new file mode 100644 index 0000000000..a921e07deb --- /dev/null +++ b/openpype/hosts/fusion/vendor/attr/setters.pyi @@ -0,0 +1,20 @@ +from typing import Any, NewType, NoReturn, TypeVar, cast + +from . import Attribute, _OnSetAttrType + + +_T = TypeVar("_T") + +def frozen( + instance: Any, attribute: Attribute[Any], new_value: Any +) -> NoReturn: ... +def pipe(*setters: _OnSetAttrType) -> _OnSetAttrType: ... +def validate(instance: Any, attribute: Attribute[_T], new_value: _T) -> _T: ... + +# convert is allowed to return Any, because they can be chained using pipe. +def convert( + instance: Any, attribute: Attribute[Any], new_value: Any +) -> Any: ... + +_NoOpType = NewType("_NoOpType", object) +NO_OP: _NoOpType diff --git a/openpype/hosts/fusion/vendor/attr/validators.py b/openpype/hosts/fusion/vendor/attr/validators.py new file mode 100644 index 0000000000..b9a73054e9 --- /dev/null +++ b/openpype/hosts/fusion/vendor/attr/validators.py @@ -0,0 +1,379 @@ +""" +Commonly useful validators. +""" + +from __future__ import absolute_import, division, print_function + +import re + +from ._make import _AndValidator, and_, attrib, attrs +from .exceptions import NotCallableError + + +__all__ = [ + "and_", + "deep_iterable", + "deep_mapping", + "in_", + "instance_of", + "is_callable", + "matches_re", + "optional", + "provides", +] + + +@attrs(repr=False, slots=True, hash=True) +class _InstanceOfValidator(object): + type = attrib() + + def __call__(self, inst, attr, value): + """ + We use a callable class to be able to change the ``__repr__``. + """ + if not isinstance(value, self.type): + raise TypeError( + "'{name}' must be {type!r} (got {value!r} that is a " + "{actual!r}).".format( + name=attr.name, + type=self.type, + actual=value.__class__, + value=value, + ), + attr, + self.type, + value, + ) + + def __repr__(self): + return "".format( + type=self.type + ) + + +def instance_of(type): + """ + A validator that raises a `TypeError` if the initializer is called + with a wrong type for this particular attribute (checks are performed using + `isinstance` therefore it's also valid to pass a tuple of types). + + :param type: The type to check for. + :type type: type or tuple of types + + :raises TypeError: With a human readable error message, the attribute + (of type `attr.Attribute`), the expected type, and the value it + got. + """ + return _InstanceOfValidator(type) + + +@attrs(repr=False, frozen=True, slots=True) +class _MatchesReValidator(object): + regex = attrib() + flags = attrib() + match_func = attrib() + + def __call__(self, inst, attr, value): + """ + We use a callable class to be able to change the ``__repr__``. + """ + if not self.match_func(value): + raise ValueError( + "'{name}' must match regex {regex!r}" + " ({value!r} doesn't)".format( + name=attr.name, regex=self.regex.pattern, value=value + ), + attr, + self.regex, + value, + ) + + def __repr__(self): + return "".format( + regex=self.regex + ) + + +def matches_re(regex, flags=0, func=None): + r""" + A validator that raises `ValueError` if the initializer is called + with a string that doesn't match *regex*. + + :param str regex: a regex string to match against + :param int flags: flags that will be passed to the underlying re function + (default 0) + :param callable func: which underlying `re` function to call (options + are `re.fullmatch`, `re.search`, `re.match`, default + is ``None`` which means either `re.fullmatch` or an emulation of + it on Python 2). For performance reasons, they won't be used directly + but on a pre-`re.compile`\ ed pattern. + + .. versionadded:: 19.2.0 + """ + fullmatch = getattr(re, "fullmatch", None) + valid_funcs = (fullmatch, None, re.search, re.match) + if func not in valid_funcs: + raise ValueError( + "'func' must be one of %s." + % ( + ", ".join( + sorted( + e and e.__name__ or "None" for e in set(valid_funcs) + ) + ), + ) + ) + + pattern = re.compile(regex, flags) + if func is re.match: + match_func = pattern.match + elif func is re.search: + match_func = pattern.search + else: + if fullmatch: + match_func = pattern.fullmatch + else: + pattern = re.compile(r"(?:{})\Z".format(regex), flags) + match_func = pattern.match + + return _MatchesReValidator(pattern, flags, match_func) + + +@attrs(repr=False, slots=True, hash=True) +class _ProvidesValidator(object): + interface = attrib() + + def __call__(self, inst, attr, value): + """ + We use a callable class to be able to change the ``__repr__``. + """ + if not self.interface.providedBy(value): + raise TypeError( + "'{name}' must provide {interface!r} which {value!r} " + "doesn't.".format( + name=attr.name, interface=self.interface, value=value + ), + attr, + self.interface, + value, + ) + + def __repr__(self): + return "".format( + interface=self.interface + ) + + +def provides(interface): + """ + A validator that raises a `TypeError` if the initializer is called + with an object that does not provide the requested *interface* (checks are + performed using ``interface.providedBy(value)`` (see `zope.interface + `_). + + :param interface: The interface to check for. + :type interface: ``zope.interface.Interface`` + + :raises TypeError: With a human readable error message, the attribute + (of type `attr.Attribute`), the expected interface, and the + value it got. + """ + return _ProvidesValidator(interface) + + +@attrs(repr=False, slots=True, hash=True) +class _OptionalValidator(object): + validator = attrib() + + def __call__(self, inst, attr, value): + if value is None: + return + + self.validator(inst, attr, value) + + def __repr__(self): + return "".format( + what=repr(self.validator) + ) + + +def optional(validator): + """ + A validator that makes an attribute optional. An optional attribute is one + which can be set to ``None`` in addition to satisfying the requirements of + the sub-validator. + + :param validator: A validator (or a list of validators) that is used for + non-``None`` values. + :type validator: callable or `list` of callables. + + .. versionadded:: 15.1.0 + .. versionchanged:: 17.1.0 *validator* can be a list of validators. + """ + if isinstance(validator, list): + return _OptionalValidator(_AndValidator(validator)) + return _OptionalValidator(validator) + + +@attrs(repr=False, slots=True, hash=True) +class _InValidator(object): + options = attrib() + + def __call__(self, inst, attr, value): + try: + in_options = value in self.options + except TypeError: # e.g. `1 in "abc"` + in_options = False + + if not in_options: + raise ValueError( + "'{name}' must be in {options!r} (got {value!r})".format( + name=attr.name, options=self.options, value=value + ) + ) + + def __repr__(self): + return "".format( + options=self.options + ) + + +def in_(options): + """ + A validator that raises a `ValueError` if the initializer is called + with a value that does not belong in the options provided. The check is + performed using ``value in options``. + + :param options: Allowed options. + :type options: list, tuple, `enum.Enum`, ... + + :raises ValueError: With a human readable error message, the attribute (of + type `attr.Attribute`), the expected options, and the value it + got. + + .. versionadded:: 17.1.0 + """ + return _InValidator(options) + + +@attrs(repr=False, slots=False, hash=True) +class _IsCallableValidator(object): + def __call__(self, inst, attr, value): + """ + We use a callable class to be able to change the ``__repr__``. + """ + if not callable(value): + message = ( + "'{name}' must be callable " + "(got {value!r} that is a {actual!r})." + ) + raise NotCallableError( + msg=message.format( + name=attr.name, value=value, actual=value.__class__ + ), + value=value, + ) + + def __repr__(self): + return "" + + +def is_callable(): + """ + A validator that raises a `attr.exceptions.NotCallableError` if the + initializer is called with a value for this particular attribute + that is not callable. + + .. versionadded:: 19.1.0 + + :raises `attr.exceptions.NotCallableError`: With a human readable error + message containing the attribute (`attr.Attribute`) name, + and the value it got. + """ + return _IsCallableValidator() + + +@attrs(repr=False, slots=True, hash=True) +class _DeepIterable(object): + member_validator = attrib(validator=is_callable()) + iterable_validator = attrib( + default=None, validator=optional(is_callable()) + ) + + def __call__(self, inst, attr, value): + """ + We use a callable class to be able to change the ``__repr__``. + """ + if self.iterable_validator is not None: + self.iterable_validator(inst, attr, value) + + for member in value: + self.member_validator(inst, attr, member) + + def __repr__(self): + iterable_identifier = ( + "" + if self.iterable_validator is None + else " {iterable!r}".format(iterable=self.iterable_validator) + ) + return ( + "" + ).format( + iterable_identifier=iterable_identifier, + member=self.member_validator, + ) + + +def deep_iterable(member_validator, iterable_validator=None): + """ + A validator that performs deep validation of an iterable. + + :param member_validator: Validator to apply to iterable members + :param iterable_validator: Validator to apply to iterable itself + (optional) + + .. versionadded:: 19.1.0 + + :raises TypeError: if any sub-validators fail + """ + return _DeepIterable(member_validator, iterable_validator) + + +@attrs(repr=False, slots=True, hash=True) +class _DeepMapping(object): + key_validator = attrib(validator=is_callable()) + value_validator = attrib(validator=is_callable()) + mapping_validator = attrib(default=None, validator=optional(is_callable())) + + def __call__(self, inst, attr, value): + """ + We use a callable class to be able to change the ``__repr__``. + """ + if self.mapping_validator is not None: + self.mapping_validator(inst, attr, value) + + for key in value: + self.key_validator(inst, attr, key) + self.value_validator(inst, attr, value[key]) + + def __repr__(self): + return ( + "" + ).format(key=self.key_validator, value=self.value_validator) + + +def deep_mapping(key_validator, value_validator, mapping_validator=None): + """ + A validator that performs deep validation of a dictionary. + + :param key_validator: Validator to apply to dictionary keys + :param value_validator: Validator to apply to dictionary values + :param mapping_validator: Validator to apply to top-level mapping + attribute (optional) + + .. versionadded:: 19.1.0 + + :raises TypeError: if any sub-validators fail + """ + return _DeepMapping(key_validator, value_validator, mapping_validator) diff --git a/openpype/hosts/fusion/vendor/attr/validators.pyi b/openpype/hosts/fusion/vendor/attr/validators.pyi new file mode 100644 index 0000000000..fe92aac421 --- /dev/null +++ b/openpype/hosts/fusion/vendor/attr/validators.pyi @@ -0,0 +1,68 @@ +from typing import ( + Any, + AnyStr, + Callable, + Container, + Iterable, + List, + Mapping, + Match, + Optional, + Tuple, + Type, + TypeVar, + Union, + overload, +) + +from . import _ValidatorType + + +_T = TypeVar("_T") +_T1 = TypeVar("_T1") +_T2 = TypeVar("_T2") +_T3 = TypeVar("_T3") +_I = TypeVar("_I", bound=Iterable) +_K = TypeVar("_K") +_V = TypeVar("_V") +_M = TypeVar("_M", bound=Mapping) + +# To be more precise on instance_of use some overloads. +# If there are more than 3 items in the tuple then we fall back to Any +@overload +def instance_of(type: Type[_T]) -> _ValidatorType[_T]: ... +@overload +def instance_of(type: Tuple[Type[_T]]) -> _ValidatorType[_T]: ... +@overload +def instance_of( + type: Tuple[Type[_T1], Type[_T2]] +) -> _ValidatorType[Union[_T1, _T2]]: ... +@overload +def instance_of( + type: Tuple[Type[_T1], Type[_T2], Type[_T3]] +) -> _ValidatorType[Union[_T1, _T2, _T3]]: ... +@overload +def instance_of(type: Tuple[type, ...]) -> _ValidatorType[Any]: ... +def provides(interface: Any) -> _ValidatorType[Any]: ... +def optional( + validator: Union[_ValidatorType[_T], List[_ValidatorType[_T]]] +) -> _ValidatorType[Optional[_T]]: ... +def in_(options: Container[_T]) -> _ValidatorType[_T]: ... +def and_(*validators: _ValidatorType[_T]) -> _ValidatorType[_T]: ... +def matches_re( + regex: AnyStr, + flags: int = ..., + func: Optional[ + Callable[[AnyStr, AnyStr, int], Optional[Match[AnyStr]]] + ] = ..., +) -> _ValidatorType[AnyStr]: ... +def deep_iterable( + member_validator: _ValidatorType[_T], + iterable_validator: Optional[_ValidatorType[_I]] = ..., +) -> _ValidatorType[_I]: ... +def deep_mapping( + key_validator: _ValidatorType[_K], + value_validator: _ValidatorType[_V], + mapping_validator: Optional[_ValidatorType[_M]] = ..., +) -> _ValidatorType[_M]: ... +def is_callable() -> _ValidatorType[_T]: ... diff --git a/openpype/hosts/fusion/vendor/urllib3/__init__.py b/openpype/hosts/fusion/vendor/urllib3/__init__.py new file mode 100644 index 0000000000..fe86b59d78 --- /dev/null +++ b/openpype/hosts/fusion/vendor/urllib3/__init__.py @@ -0,0 +1,85 @@ +""" +Python HTTP library with thread-safe connection pooling, file post support, user friendly, and more +""" +from __future__ import absolute_import + +# Set default logging handler to avoid "No handler found" warnings. +import logging +import warnings +from logging import NullHandler + +from . import exceptions +from ._version import __version__ +from .connectionpool import HTTPConnectionPool, HTTPSConnectionPool, connection_from_url +from .filepost import encode_multipart_formdata +from .poolmanager import PoolManager, ProxyManager, proxy_from_url +from .response import HTTPResponse +from .util.request import make_headers +from .util.retry import Retry +from .util.timeout import Timeout +from .util.url import get_host + +__author__ = "Andrey Petrov (andrey.petrov@shazow.net)" +__license__ = "MIT" +__version__ = __version__ + +__all__ = ( + "HTTPConnectionPool", + "HTTPSConnectionPool", + "PoolManager", + "ProxyManager", + "HTTPResponse", + "Retry", + "Timeout", + "add_stderr_logger", + "connection_from_url", + "disable_warnings", + "encode_multipart_formdata", + "get_host", + "make_headers", + "proxy_from_url", +) + +logging.getLogger(__name__).addHandler(NullHandler()) + + +def add_stderr_logger(level=logging.DEBUG): + """ + Helper for quickly adding a StreamHandler to the logger. Useful for + debugging. + + Returns the handler after adding it. + """ + # This method needs to be in this __init__.py to get the __name__ correct + # even if urllib3 is vendored within another package. + logger = logging.getLogger(__name__) + handler = logging.StreamHandler() + handler.setFormatter(logging.Formatter("%(asctime)s %(levelname)s %(message)s")) + logger.addHandler(handler) + logger.setLevel(level) + logger.debug("Added a stderr logging handler to logger: %s", __name__) + return handler + + +# ... Clean up. +del NullHandler + + +# All warning filters *must* be appended unless you're really certain that they +# shouldn't be: otherwise, it's very hard for users to use most Python +# mechanisms to silence them. +# SecurityWarning's always go off by default. +warnings.simplefilter("always", exceptions.SecurityWarning, append=True) +# SubjectAltNameWarning's should go off once per host +warnings.simplefilter("default", exceptions.SubjectAltNameWarning, append=True) +# InsecurePlatformWarning's don't vary between requests, so we keep it default. +warnings.simplefilter("default", exceptions.InsecurePlatformWarning, append=True) +# SNIMissingWarnings should go off only once. +warnings.simplefilter("default", exceptions.SNIMissingWarning, append=True) + + +def disable_warnings(category=exceptions.HTTPWarning): + """ + Helper for quickly disabling all urllib3 warnings. + """ + warnings.simplefilter("ignore", category) diff --git a/openpype/hosts/fusion/vendor/urllib3/_collections.py b/openpype/hosts/fusion/vendor/urllib3/_collections.py new file mode 100644 index 0000000000..da9857e986 --- /dev/null +++ b/openpype/hosts/fusion/vendor/urllib3/_collections.py @@ -0,0 +1,337 @@ +from __future__ import absolute_import + +try: + from collections.abc import Mapping, MutableMapping +except ImportError: + from collections import Mapping, MutableMapping +try: + from threading import RLock +except ImportError: # Platform-specific: No threads available + + class RLock: + def __enter__(self): + pass + + def __exit__(self, exc_type, exc_value, traceback): + pass + + +from collections import OrderedDict + +from .exceptions import InvalidHeader +from .packages import six +from .packages.six import iterkeys, itervalues + +__all__ = ["RecentlyUsedContainer", "HTTPHeaderDict"] + + +_Null = object() + + +class RecentlyUsedContainer(MutableMapping): + """ + Provides a thread-safe dict-like container which maintains up to + ``maxsize`` keys while throwing away the least-recently-used keys beyond + ``maxsize``. + + :param maxsize: + Maximum number of recent elements to retain. + + :param dispose_func: + Every time an item is evicted from the container, + ``dispose_func(value)`` is called. Callback which will get called + """ + + ContainerCls = OrderedDict + + def __init__(self, maxsize=10, dispose_func=None): + self._maxsize = maxsize + self.dispose_func = dispose_func + + self._container = self.ContainerCls() + self.lock = RLock() + + def __getitem__(self, key): + # Re-insert the item, moving it to the end of the eviction line. + with self.lock: + item = self._container.pop(key) + self._container[key] = item + return item + + def __setitem__(self, key, value): + evicted_value = _Null + with self.lock: + # Possibly evict the existing value of 'key' + evicted_value = self._container.get(key, _Null) + self._container[key] = value + + # If we didn't evict an existing value, we might have to evict the + # least recently used item from the beginning of the container. + if len(self._container) > self._maxsize: + _key, evicted_value = self._container.popitem(last=False) + + if self.dispose_func and evicted_value is not _Null: + self.dispose_func(evicted_value) + + def __delitem__(self, key): + with self.lock: + value = self._container.pop(key) + + if self.dispose_func: + self.dispose_func(value) + + def __len__(self): + with self.lock: + return len(self._container) + + def __iter__(self): + raise NotImplementedError( + "Iteration over this class is unlikely to be threadsafe." + ) + + def clear(self): + with self.lock: + # Copy pointers to all values, then wipe the mapping + values = list(itervalues(self._container)) + self._container.clear() + + if self.dispose_func: + for value in values: + self.dispose_func(value) + + def keys(self): + with self.lock: + return list(iterkeys(self._container)) + + +class HTTPHeaderDict(MutableMapping): + """ + :param headers: + An iterable of field-value pairs. Must not contain multiple field names + when compared case-insensitively. + + :param kwargs: + Additional field-value pairs to pass in to ``dict.update``. + + A ``dict`` like container for storing HTTP Headers. + + Field names are stored and compared case-insensitively in compliance with + RFC 7230. Iteration provides the first case-sensitive key seen for each + case-insensitive pair. + + Using ``__setitem__`` syntax overwrites fields that compare equal + case-insensitively in order to maintain ``dict``'s api. For fields that + compare equal, instead create a new ``HTTPHeaderDict`` and use ``.add`` + in a loop. + + If multiple fields that are equal case-insensitively are passed to the + constructor or ``.update``, the behavior is undefined and some will be + lost. + + >>> headers = HTTPHeaderDict() + >>> headers.add('Set-Cookie', 'foo=bar') + >>> headers.add('set-cookie', 'baz=quxx') + >>> headers['content-length'] = '7' + >>> headers['SET-cookie'] + 'foo=bar, baz=quxx' + >>> headers['Content-Length'] + '7' + """ + + def __init__(self, headers=None, **kwargs): + super(HTTPHeaderDict, self).__init__() + self._container = OrderedDict() + if headers is not None: + if isinstance(headers, HTTPHeaderDict): + self._copy_from(headers) + else: + self.extend(headers) + if kwargs: + self.extend(kwargs) + + def __setitem__(self, key, val): + self._container[key.lower()] = [key, val] + return self._container[key.lower()] + + def __getitem__(self, key): + val = self._container[key.lower()] + return ", ".join(val[1:]) + + def __delitem__(self, key): + del self._container[key.lower()] + + def __contains__(self, key): + return key.lower() in self._container + + def __eq__(self, other): + if not isinstance(other, Mapping) and not hasattr(other, "keys"): + return False + if not isinstance(other, type(self)): + other = type(self)(other) + return dict((k.lower(), v) for k, v in self.itermerged()) == dict( + (k.lower(), v) for k, v in other.itermerged() + ) + + def __ne__(self, other): + return not self.__eq__(other) + + if six.PY2: # Python 2 + iterkeys = MutableMapping.iterkeys + itervalues = MutableMapping.itervalues + + __marker = object() + + def __len__(self): + return len(self._container) + + def __iter__(self): + # Only provide the originally cased names + for vals in self._container.values(): + yield vals[0] + + def pop(self, key, default=__marker): + """D.pop(k[,d]) -> v, remove specified key and return the corresponding value. + If key is not found, d is returned if given, otherwise KeyError is raised. + """ + # Using the MutableMapping function directly fails due to the private marker. + # Using ordinary dict.pop would expose the internal structures. + # So let's reinvent the wheel. + try: + value = self[key] + except KeyError: + if default is self.__marker: + raise + return default + else: + del self[key] + return value + + def discard(self, key): + try: + del self[key] + except KeyError: + pass + + def add(self, key, val): + """Adds a (name, value) pair, doesn't overwrite the value if it already + exists. + + >>> headers = HTTPHeaderDict(foo='bar') + >>> headers.add('Foo', 'baz') + >>> headers['foo'] + 'bar, baz' + """ + key_lower = key.lower() + new_vals = [key, val] + # Keep the common case aka no item present as fast as possible + vals = self._container.setdefault(key_lower, new_vals) + if new_vals is not vals: + vals.append(val) + + def extend(self, *args, **kwargs): + """Generic import function for any type of header-like object. + Adapted version of MutableMapping.update in order to insert items + with self.add instead of self.__setitem__ + """ + if len(args) > 1: + raise TypeError( + "extend() takes at most 1 positional " + "arguments ({0} given)".format(len(args)) + ) + other = args[0] if len(args) >= 1 else () + + if isinstance(other, HTTPHeaderDict): + for key, val in other.iteritems(): + self.add(key, val) + elif isinstance(other, Mapping): + for key in other: + self.add(key, other[key]) + elif hasattr(other, "keys"): + for key in other.keys(): + self.add(key, other[key]) + else: + for key, value in other: + self.add(key, value) + + for key, value in kwargs.items(): + self.add(key, value) + + def getlist(self, key, default=__marker): + """Returns a list of all the values for the named field. Returns an + empty list if the key doesn't exist.""" + try: + vals = self._container[key.lower()] + except KeyError: + if default is self.__marker: + return [] + return default + else: + return vals[1:] + + # Backwards compatibility for httplib + getheaders = getlist + getallmatchingheaders = getlist + iget = getlist + + # Backwards compatibility for http.cookiejar + get_all = getlist + + def __repr__(self): + return "%s(%s)" % (type(self).__name__, dict(self.itermerged())) + + def _copy_from(self, other): + for key in other: + val = other.getlist(key) + if isinstance(val, list): + # Don't need to convert tuples + val = list(val) + self._container[key.lower()] = [key] + val + + def copy(self): + clone = type(self)() + clone._copy_from(self) + return clone + + def iteritems(self): + """Iterate over all header lines, including duplicate ones.""" + for key in self: + vals = self._container[key.lower()] + for val in vals[1:]: + yield vals[0], val + + def itermerged(self): + """Iterate over all headers, merging duplicate ones together.""" + for key in self: + val = self._container[key.lower()] + yield val[0], ", ".join(val[1:]) + + def items(self): + return list(self.iteritems()) + + @classmethod + def from_httplib(cls, message): # Python 2 + """Read headers from a Python 2 httplib message object.""" + # python2.7 does not expose a proper API for exporting multiheaders + # efficiently. This function re-reads raw lines from the message + # object and extracts the multiheaders properly. + obs_fold_continued_leaders = (" ", "\t") + headers = [] + + for line in message.headers: + if line.startswith(obs_fold_continued_leaders): + if not headers: + # We received a header line that starts with OWS as described + # in RFC-7230 S3.2.4. This indicates a multiline header, but + # there exists no previous header to which we can attach it. + raise InvalidHeader( + "Header continuation with no previous header: %s" % line + ) + else: + key, value = headers[-1] + headers[-1] = (key, value + " " + line.strip()) + continue + + key, value = line.split(":", 1) + headers.append((key, value.strip())) + + return cls(headers) diff --git a/openpype/hosts/fusion/vendor/urllib3/_version.py b/openpype/hosts/fusion/vendor/urllib3/_version.py new file mode 100644 index 0000000000..e8ebee957f --- /dev/null +++ b/openpype/hosts/fusion/vendor/urllib3/_version.py @@ -0,0 +1,2 @@ +# This file is protected via CODEOWNERS +__version__ = "1.26.6" diff --git a/openpype/hosts/fusion/vendor/urllib3/connection.py b/openpype/hosts/fusion/vendor/urllib3/connection.py new file mode 100644 index 0000000000..4c996659c8 --- /dev/null +++ b/openpype/hosts/fusion/vendor/urllib3/connection.py @@ -0,0 +1,539 @@ +from __future__ import absolute_import + +import datetime +import logging +import os +import re +import socket +import warnings +from socket import error as SocketError +from socket import timeout as SocketTimeout + +from .packages import six +from .packages.six.moves.http_client import HTTPConnection as _HTTPConnection +from .packages.six.moves.http_client import HTTPException # noqa: F401 +from .util.proxy import create_proxy_ssl_context + +try: # Compiled with SSL? + import ssl + + BaseSSLError = ssl.SSLError +except (ImportError, AttributeError): # Platform-specific: No SSL. + ssl = None + + class BaseSSLError(BaseException): + pass + + +try: + # Python 3: not a no-op, we're adding this to the namespace so it can be imported. + ConnectionError = ConnectionError +except NameError: + # Python 2 + class ConnectionError(Exception): + pass + + +try: # Python 3: + # Not a no-op, we're adding this to the namespace so it can be imported. + BrokenPipeError = BrokenPipeError +except NameError: # Python 2: + + class BrokenPipeError(Exception): + pass + + +from ._collections import HTTPHeaderDict # noqa (historical, removed in v2) +from ._version import __version__ +from .exceptions import ( + ConnectTimeoutError, + NewConnectionError, + SubjectAltNameWarning, + SystemTimeWarning, +) +from .packages.ssl_match_hostname import CertificateError, match_hostname +from .util import SKIP_HEADER, SKIPPABLE_HEADERS, connection +from .util.ssl_ import ( + assert_fingerprint, + create_urllib3_context, + resolve_cert_reqs, + resolve_ssl_version, + ssl_wrap_socket, +) + +log = logging.getLogger(__name__) + +port_by_scheme = {"http": 80, "https": 443} + +# When it comes time to update this value as a part of regular maintenance +# (ie test_recent_date is failing) update it to ~6 months before the current date. +RECENT_DATE = datetime.date(2020, 7, 1) + +_CONTAINS_CONTROL_CHAR_RE = re.compile(r"[^-!#$%&'*+.^_`|~0-9a-zA-Z]") + + +class HTTPConnection(_HTTPConnection, object): + """ + Based on :class:`http.client.HTTPConnection` but provides an extra constructor + backwards-compatibility layer between older and newer Pythons. + + Additional keyword parameters are used to configure attributes of the connection. + Accepted parameters include: + + - ``strict``: See the documentation on :class:`urllib3.connectionpool.HTTPConnectionPool` + - ``source_address``: Set the source address for the current connection. + - ``socket_options``: Set specific options on the underlying socket. If not specified, then + defaults are loaded from ``HTTPConnection.default_socket_options`` which includes disabling + Nagle's algorithm (sets TCP_NODELAY to 1) unless the connection is behind a proxy. + + For example, if you wish to enable TCP Keep Alive in addition to the defaults, + you might pass: + + .. code-block:: python + + HTTPConnection.default_socket_options + [ + (socket.SOL_SOCKET, socket.SO_KEEPALIVE, 1), + ] + + Or you may want to disable the defaults by passing an empty list (e.g., ``[]``). + """ + + default_port = port_by_scheme["http"] + + #: Disable Nagle's algorithm by default. + #: ``[(socket.IPPROTO_TCP, socket.TCP_NODELAY, 1)]`` + default_socket_options = [(socket.IPPROTO_TCP, socket.TCP_NODELAY, 1)] + + #: Whether this connection verifies the host's certificate. + is_verified = False + + def __init__(self, *args, **kw): + if not six.PY2: + kw.pop("strict", None) + + # Pre-set source_address. + self.source_address = kw.get("source_address") + + #: The socket options provided by the user. If no options are + #: provided, we use the default options. + self.socket_options = kw.pop("socket_options", self.default_socket_options) + + # Proxy options provided by the user. + self.proxy = kw.pop("proxy", None) + self.proxy_config = kw.pop("proxy_config", None) + + _HTTPConnection.__init__(self, *args, **kw) + + @property + def host(self): + """ + Getter method to remove any trailing dots that indicate the hostname is an FQDN. + + In general, SSL certificates don't include the trailing dot indicating a + fully-qualified domain name, and thus, they don't validate properly when + checked against a domain name that includes the dot. In addition, some + servers may not expect to receive the trailing dot when provided. + + However, the hostname with trailing dot is critical to DNS resolution; doing a + lookup with the trailing dot will properly only resolve the appropriate FQDN, + whereas a lookup without a trailing dot will search the system's search domain + list. Thus, it's important to keep the original host around for use only in + those cases where it's appropriate (i.e., when doing DNS lookup to establish the + actual TCP connection across which we're going to send HTTP requests). + """ + return self._dns_host.rstrip(".") + + @host.setter + def host(self, value): + """ + Setter for the `host` property. + + We assume that only urllib3 uses the _dns_host attribute; httplib itself + only uses `host`, and it seems reasonable that other libraries follow suit. + """ + self._dns_host = value + + def _new_conn(self): + """Establish a socket connection and set nodelay settings on it. + + :return: New socket connection. + """ + extra_kw = {} + if self.source_address: + extra_kw["source_address"] = self.source_address + + if self.socket_options: + extra_kw["socket_options"] = self.socket_options + + try: + conn = connection.create_connection( + (self._dns_host, self.port), self.timeout, **extra_kw + ) + + except SocketTimeout: + raise ConnectTimeoutError( + self, + "Connection to %s timed out. (connect timeout=%s)" + % (self.host, self.timeout), + ) + + except SocketError as e: + raise NewConnectionError( + self, "Failed to establish a new connection: %s" % e + ) + + return conn + + def _is_using_tunnel(self): + # Google App Engine's httplib does not define _tunnel_host + return getattr(self, "_tunnel_host", None) + + def _prepare_conn(self, conn): + self.sock = conn + if self._is_using_tunnel(): + # TODO: Fix tunnel so it doesn't depend on self.sock state. + self._tunnel() + # Mark this connection as not reusable + self.auto_open = 0 + + def connect(self): + conn = self._new_conn() + self._prepare_conn(conn) + + def putrequest(self, method, url, *args, **kwargs): + """ """ + # Empty docstring because the indentation of CPython's implementation + # is broken but we don't want this method in our documentation. + match = _CONTAINS_CONTROL_CHAR_RE.search(method) + if match: + raise ValueError( + "Method cannot contain non-token characters %r (found at least %r)" + % (method, match.group()) + ) + + return _HTTPConnection.putrequest(self, method, url, *args, **kwargs) + + def putheader(self, header, *values): + """ """ + if not any(isinstance(v, str) and v == SKIP_HEADER for v in values): + _HTTPConnection.putheader(self, header, *values) + elif six.ensure_str(header.lower()) not in SKIPPABLE_HEADERS: + raise ValueError( + "urllib3.util.SKIP_HEADER only supports '%s'" + % ("', '".join(map(str.title, sorted(SKIPPABLE_HEADERS))),) + ) + + def request(self, method, url, body=None, headers=None): + if headers is None: + headers = {} + else: + # Avoid modifying the headers passed into .request() + headers = headers.copy() + if "user-agent" not in (six.ensure_str(k.lower()) for k in headers): + headers["User-Agent"] = _get_default_user_agent() + super(HTTPConnection, self).request(method, url, body=body, headers=headers) + + def request_chunked(self, method, url, body=None, headers=None): + """ + Alternative to the common request method, which sends the + body with chunked encoding and not as one block + """ + headers = headers or {} + header_keys = set([six.ensure_str(k.lower()) for k in headers]) + skip_accept_encoding = "accept-encoding" in header_keys + skip_host = "host" in header_keys + self.putrequest( + method, url, skip_accept_encoding=skip_accept_encoding, skip_host=skip_host + ) + if "user-agent" not in header_keys: + self.putheader("User-Agent", _get_default_user_agent()) + for header, value in headers.items(): + self.putheader(header, value) + if "transfer-encoding" not in header_keys: + self.putheader("Transfer-Encoding", "chunked") + self.endheaders() + + if body is not None: + stringish_types = six.string_types + (bytes,) + if isinstance(body, stringish_types): + body = (body,) + for chunk in body: + if not chunk: + continue + if not isinstance(chunk, bytes): + chunk = chunk.encode("utf8") + len_str = hex(len(chunk))[2:] + to_send = bytearray(len_str.encode()) + to_send += b"\r\n" + to_send += chunk + to_send += b"\r\n" + self.send(to_send) + + # After the if clause, to always have a closed body + self.send(b"0\r\n\r\n") + + +class HTTPSConnection(HTTPConnection): + """ + Many of the parameters to this constructor are passed to the underlying SSL + socket by means of :py:func:`urllib3.util.ssl_wrap_socket`. + """ + + default_port = port_by_scheme["https"] + + cert_reqs = None + ca_certs = None + ca_cert_dir = None + ca_cert_data = None + ssl_version = None + assert_fingerprint = None + tls_in_tls_required = False + + def __init__( + self, + host, + port=None, + key_file=None, + cert_file=None, + key_password=None, + strict=None, + timeout=socket._GLOBAL_DEFAULT_TIMEOUT, + ssl_context=None, + server_hostname=None, + **kw + ): + + HTTPConnection.__init__(self, host, port, strict=strict, timeout=timeout, **kw) + + self.key_file = key_file + self.cert_file = cert_file + self.key_password = key_password + self.ssl_context = ssl_context + self.server_hostname = server_hostname + + # Required property for Google AppEngine 1.9.0 which otherwise causes + # HTTPS requests to go out as HTTP. (See Issue #356) + self._protocol = "https" + + def set_cert( + self, + key_file=None, + cert_file=None, + cert_reqs=None, + key_password=None, + ca_certs=None, + assert_hostname=None, + assert_fingerprint=None, + ca_cert_dir=None, + ca_cert_data=None, + ): + """ + This method should only be called once, before the connection is used. + """ + # If cert_reqs is not provided we'll assume CERT_REQUIRED unless we also + # have an SSLContext object in which case we'll use its verify_mode. + if cert_reqs is None: + if self.ssl_context is not None: + cert_reqs = self.ssl_context.verify_mode + else: + cert_reqs = resolve_cert_reqs(None) + + self.key_file = key_file + self.cert_file = cert_file + self.cert_reqs = cert_reqs + self.key_password = key_password + self.assert_hostname = assert_hostname + self.assert_fingerprint = assert_fingerprint + self.ca_certs = ca_certs and os.path.expanduser(ca_certs) + self.ca_cert_dir = ca_cert_dir and os.path.expanduser(ca_cert_dir) + self.ca_cert_data = ca_cert_data + + def connect(self): + # Add certificate verification + conn = self._new_conn() + hostname = self.host + tls_in_tls = False + + if self._is_using_tunnel(): + if self.tls_in_tls_required: + conn = self._connect_tls_proxy(hostname, conn) + tls_in_tls = True + + self.sock = conn + + # Calls self._set_hostport(), so self.host is + # self._tunnel_host below. + self._tunnel() + # Mark this connection as not reusable + self.auto_open = 0 + + # Override the host with the one we're requesting data from. + hostname = self._tunnel_host + + server_hostname = hostname + if self.server_hostname is not None: + server_hostname = self.server_hostname + + is_time_off = datetime.date.today() < RECENT_DATE + if is_time_off: + warnings.warn( + ( + "System time is way off (before {0}). This will probably " + "lead to SSL verification errors" + ).format(RECENT_DATE), + SystemTimeWarning, + ) + + # Wrap socket using verification with the root certs in + # trusted_root_certs + default_ssl_context = False + if self.ssl_context is None: + default_ssl_context = True + self.ssl_context = create_urllib3_context( + ssl_version=resolve_ssl_version(self.ssl_version), + cert_reqs=resolve_cert_reqs(self.cert_reqs), + ) + + context = self.ssl_context + context.verify_mode = resolve_cert_reqs(self.cert_reqs) + + # Try to load OS default certs if none are given. + # Works well on Windows (requires Python3.4+) + if ( + not self.ca_certs + and not self.ca_cert_dir + and not self.ca_cert_data + and default_ssl_context + and hasattr(context, "load_default_certs") + ): + context.load_default_certs() + + self.sock = ssl_wrap_socket( + sock=conn, + keyfile=self.key_file, + certfile=self.cert_file, + key_password=self.key_password, + ca_certs=self.ca_certs, + ca_cert_dir=self.ca_cert_dir, + ca_cert_data=self.ca_cert_data, + server_hostname=server_hostname, + ssl_context=context, + tls_in_tls=tls_in_tls, + ) + + # If we're using all defaults and the connection + # is TLSv1 or TLSv1.1 we throw a DeprecationWarning + # for the host. + if ( + default_ssl_context + and self.ssl_version is None + and hasattr(self.sock, "version") + and self.sock.version() in {"TLSv1", "TLSv1.1"} + ): + warnings.warn( + "Negotiating TLSv1/TLSv1.1 by default is deprecated " + "and will be disabled in urllib3 v2.0.0. Connecting to " + "'%s' with '%s' can be enabled by explicitly opting-in " + "with 'ssl_version'" % (self.host, self.sock.version()), + DeprecationWarning, + ) + + if self.assert_fingerprint: + assert_fingerprint( + self.sock.getpeercert(binary_form=True), self.assert_fingerprint + ) + elif ( + context.verify_mode != ssl.CERT_NONE + and not getattr(context, "check_hostname", False) + and self.assert_hostname is not False + ): + # While urllib3 attempts to always turn off hostname matching from + # the TLS library, this cannot always be done. So we check whether + # the TLS Library still thinks it's matching hostnames. + cert = self.sock.getpeercert() + if not cert.get("subjectAltName", ()): + warnings.warn( + ( + "Certificate for {0} has no `subjectAltName`, falling back to check for a " + "`commonName` for now. This feature is being removed by major browsers and " + "deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 " + "for details.)".format(hostname) + ), + SubjectAltNameWarning, + ) + _match_hostname(cert, self.assert_hostname or server_hostname) + + self.is_verified = ( + context.verify_mode == ssl.CERT_REQUIRED + or self.assert_fingerprint is not None + ) + + def _connect_tls_proxy(self, hostname, conn): + """ + Establish a TLS connection to the proxy using the provided SSL context. + """ + proxy_config = self.proxy_config + ssl_context = proxy_config.ssl_context + if ssl_context: + # If the user provided a proxy context, we assume CA and client + # certificates have already been set + return ssl_wrap_socket( + sock=conn, + server_hostname=hostname, + ssl_context=ssl_context, + ) + + ssl_context = create_proxy_ssl_context( + self.ssl_version, + self.cert_reqs, + self.ca_certs, + self.ca_cert_dir, + self.ca_cert_data, + ) + # By default urllib3's SSLContext disables `check_hostname` and uses + # a custom check. For proxies we're good with relying on the default + # verification. + ssl_context.check_hostname = True + + # If no cert was provided, use only the default options for server + # certificate validation + return ssl_wrap_socket( + sock=conn, + ca_certs=self.ca_certs, + ca_cert_dir=self.ca_cert_dir, + ca_cert_data=self.ca_cert_data, + server_hostname=hostname, + ssl_context=ssl_context, + ) + + +def _match_hostname(cert, asserted_hostname): + try: + match_hostname(cert, asserted_hostname) + except CertificateError as e: + log.warning( + "Certificate did not match expected hostname: %s. Certificate: %s", + asserted_hostname, + cert, + ) + # Add cert to exception and reraise so client code can inspect + # the cert when catching the exception, if they want to + e._peer_cert = cert + raise + + +def _get_default_user_agent(): + return "python-urllib3/%s" % __version__ + + +class DummyConnection(object): + """Used to detect a failed ConnectionCls import.""" + + pass + + +if not ssl: + HTTPSConnection = DummyConnection # noqa: F811 + + +VerifiedHTTPSConnection = HTTPSConnection diff --git a/openpype/hosts/fusion/vendor/urllib3/connectionpool.py b/openpype/hosts/fusion/vendor/urllib3/connectionpool.py new file mode 100644 index 0000000000..459bbe095b --- /dev/null +++ b/openpype/hosts/fusion/vendor/urllib3/connectionpool.py @@ -0,0 +1,1067 @@ +from __future__ import absolute_import + +import errno +import logging +import socket +import sys +import warnings +from socket import error as SocketError +from socket import timeout as SocketTimeout + +from .connection import ( + BaseSSLError, + BrokenPipeError, + DummyConnection, + HTTPConnection, + HTTPException, + HTTPSConnection, + VerifiedHTTPSConnection, + port_by_scheme, +) +from .exceptions import ( + ClosedPoolError, + EmptyPoolError, + HeaderParsingError, + HostChangedError, + InsecureRequestWarning, + LocationValueError, + MaxRetryError, + NewConnectionError, + ProtocolError, + ProxyError, + ReadTimeoutError, + SSLError, + TimeoutError, +) +from .packages import six +from .packages.six.moves import queue +from .packages.ssl_match_hostname import CertificateError +from .request import RequestMethods +from .response import HTTPResponse +from .util.connection import is_connection_dropped +from .util.proxy import connection_requires_http_tunnel +from .util.queue import LifoQueue +from .util.request import set_file_position +from .util.response import assert_header_parsing +from .util.retry import Retry +from .util.timeout import Timeout +from .util.url import Url, _encode_target +from .util.url import _normalize_host as normalize_host +from .util.url import get_host, parse_url + +xrange = six.moves.xrange + +log = logging.getLogger(__name__) + +_Default = object() + + +# Pool objects +class ConnectionPool(object): + """ + Base class for all connection pools, such as + :class:`.HTTPConnectionPool` and :class:`.HTTPSConnectionPool`. + + .. note:: + ConnectionPool.urlopen() does not normalize or percent-encode target URIs + which is useful if your target server doesn't support percent-encoded + target URIs. + """ + + scheme = None + QueueCls = LifoQueue + + def __init__(self, host, port=None): + if not host: + raise LocationValueError("No host specified.") + + self.host = _normalize_host(host, scheme=self.scheme) + self._proxy_host = host.lower() + self.port = port + + def __str__(self): + return "%s(host=%r, port=%r)" % (type(self).__name__, self.host, self.port) + + def __enter__(self): + return self + + def __exit__(self, exc_type, exc_val, exc_tb): + self.close() + # Return False to re-raise any potential exceptions + return False + + def close(self): + """ + Close all pooled connections and disable the pool. + """ + pass + + +# This is taken from http://hg.python.org/cpython/file/7aaba721ebc0/Lib/socket.py#l252 +_blocking_errnos = {errno.EAGAIN, errno.EWOULDBLOCK} + + +class HTTPConnectionPool(ConnectionPool, RequestMethods): + """ + Thread-safe connection pool for one host. + + :param host: + Host used for this HTTP Connection (e.g. "localhost"), passed into + :class:`http.client.HTTPConnection`. + + :param port: + Port used for this HTTP Connection (None is equivalent to 80), passed + into :class:`http.client.HTTPConnection`. + + :param strict: + Causes BadStatusLine to be raised if the status line can't be parsed + as a valid HTTP/1.0 or 1.1 status line, passed into + :class:`http.client.HTTPConnection`. + + .. note:: + Only works in Python 2. This parameter is ignored in Python 3. + + :param timeout: + Socket timeout in seconds for each individual connection. This can + be a float or integer, which sets the timeout for the HTTP request, + or an instance of :class:`urllib3.util.Timeout` which gives you more + fine-grained control over request timeouts. After the constructor has + been parsed, this is always a `urllib3.util.Timeout` object. + + :param maxsize: + Number of connections to save that can be reused. More than 1 is useful + in multithreaded situations. If ``block`` is set to False, more + connections will be created but they will not be saved once they've + been used. + + :param block: + If set to True, no more than ``maxsize`` connections will be used at + a time. When no free connections are available, the call will block + until a connection has been released. This is a useful side effect for + particular multithreaded situations where one does not want to use more + than maxsize connections per host to prevent flooding. + + :param headers: + Headers to include with all requests, unless other headers are given + explicitly. + + :param retries: + Retry configuration to use by default with requests in this pool. + + :param _proxy: + Parsed proxy URL, should not be used directly, instead, see + :class:`urllib3.ProxyManager` + + :param _proxy_headers: + A dictionary with proxy headers, should not be used directly, + instead, see :class:`urllib3.ProxyManager` + + :param \\**conn_kw: + Additional parameters are used to create fresh :class:`urllib3.connection.HTTPConnection`, + :class:`urllib3.connection.HTTPSConnection` instances. + """ + + scheme = "http" + ConnectionCls = HTTPConnection + ResponseCls = HTTPResponse + + def __init__( + self, + host, + port=None, + strict=False, + timeout=Timeout.DEFAULT_TIMEOUT, + maxsize=1, + block=False, + headers=None, + retries=None, + _proxy=None, + _proxy_headers=None, + _proxy_config=None, + **conn_kw + ): + ConnectionPool.__init__(self, host, port) + RequestMethods.__init__(self, headers) + + self.strict = strict + + if not isinstance(timeout, Timeout): + timeout = Timeout.from_float(timeout) + + if retries is None: + retries = Retry.DEFAULT + + self.timeout = timeout + self.retries = retries + + self.pool = self.QueueCls(maxsize) + self.block = block + + self.proxy = _proxy + self.proxy_headers = _proxy_headers or {} + self.proxy_config = _proxy_config + + # Fill the queue up so that doing get() on it will block properly + for _ in xrange(maxsize): + self.pool.put(None) + + # These are mostly for testing and debugging purposes. + self.num_connections = 0 + self.num_requests = 0 + self.conn_kw = conn_kw + + if self.proxy: + # Enable Nagle's algorithm for proxies, to avoid packet fragmentation. + # We cannot know if the user has added default socket options, so we cannot replace the + # list. + self.conn_kw.setdefault("socket_options", []) + + self.conn_kw["proxy"] = self.proxy + self.conn_kw["proxy_config"] = self.proxy_config + + def _new_conn(self): + """ + Return a fresh :class:`HTTPConnection`. + """ + self.num_connections += 1 + log.debug( + "Starting new HTTP connection (%d): %s:%s", + self.num_connections, + self.host, + self.port or "80", + ) + + conn = self.ConnectionCls( + host=self.host, + port=self.port, + timeout=self.timeout.connect_timeout, + strict=self.strict, + **self.conn_kw + ) + return conn + + def _get_conn(self, timeout=None): + """ + Get a connection. Will return a pooled connection if one is available. + + If no connections are available and :prop:`.block` is ``False``, then a + fresh connection is returned. + + :param timeout: + Seconds to wait before giving up and raising + :class:`urllib3.exceptions.EmptyPoolError` if the pool is empty and + :prop:`.block` is ``True``. + """ + conn = None + try: + conn = self.pool.get(block=self.block, timeout=timeout) + + except AttributeError: # self.pool is None + raise ClosedPoolError(self, "Pool is closed.") + + except queue.Empty: + if self.block: + raise EmptyPoolError( + self, + "Pool reached maximum size and no more connections are allowed.", + ) + pass # Oh well, we'll create a new connection then + + # If this is a persistent connection, check if it got disconnected + if conn and is_connection_dropped(conn): + log.debug("Resetting dropped connection: %s", self.host) + conn.close() + if getattr(conn, "auto_open", 1) == 0: + # This is a proxied connection that has been mutated by + # http.client._tunnel() and cannot be reused (since it would + # attempt to bypass the proxy) + conn = None + + return conn or self._new_conn() + + def _put_conn(self, conn): + """ + Put a connection back into the pool. + + :param conn: + Connection object for the current host and port as returned by + :meth:`._new_conn` or :meth:`._get_conn`. + + If the pool is already full, the connection is closed and discarded + because we exceeded maxsize. If connections are discarded frequently, + then maxsize should be increased. + + If the pool is closed, then the connection will be closed and discarded. + """ + try: + self.pool.put(conn, block=False) + return # Everything is dandy, done. + except AttributeError: + # self.pool is None. + pass + except queue.Full: + # This should never happen if self.block == True + log.warning("Connection pool is full, discarding connection: %s", self.host) + + # Connection never got put back into the pool, close it. + if conn: + conn.close() + + def _validate_conn(self, conn): + """ + Called right before a request is made, after the socket is created. + """ + pass + + def _prepare_proxy(self, conn): + # Nothing to do for HTTP connections. + pass + + def _get_timeout(self, timeout): + """Helper that always returns a :class:`urllib3.util.Timeout`""" + if timeout is _Default: + return self.timeout.clone() + + if isinstance(timeout, Timeout): + return timeout.clone() + else: + # User passed us an int/float. This is for backwards compatibility, + # can be removed later + return Timeout.from_float(timeout) + + def _raise_timeout(self, err, url, timeout_value): + """Is the error actually a timeout? Will raise a ReadTimeout or pass""" + + if isinstance(err, SocketTimeout): + raise ReadTimeoutError( + self, url, "Read timed out. (read timeout=%s)" % timeout_value + ) + + # See the above comment about EAGAIN in Python 3. In Python 2 we have + # to specifically catch it and throw the timeout error + if hasattr(err, "errno") and err.errno in _blocking_errnos: + raise ReadTimeoutError( + self, url, "Read timed out. (read timeout=%s)" % timeout_value + ) + + # Catch possible read timeouts thrown as SSL errors. If not the + # case, rethrow the original. We need to do this because of: + # http://bugs.python.org/issue10272 + if "timed out" in str(err) or "did not complete (read)" in str( + err + ): # Python < 2.7.4 + raise ReadTimeoutError( + self, url, "Read timed out. (read timeout=%s)" % timeout_value + ) + + def _make_request( + self, conn, method, url, timeout=_Default, chunked=False, **httplib_request_kw + ): + """ + Perform a request on a given urllib connection object taken from our + pool. + + :param conn: + a connection from one of our connection pools + + :param timeout: + Socket timeout in seconds for the request. This can be a + float or integer, which will set the same timeout value for + the socket connect and the socket read, or an instance of + :class:`urllib3.util.Timeout`, which gives you more fine-grained + control over your timeouts. + """ + self.num_requests += 1 + + timeout_obj = self._get_timeout(timeout) + timeout_obj.start_connect() + conn.timeout = timeout_obj.connect_timeout + + # Trigger any extra validation we need to do. + try: + self._validate_conn(conn) + except (SocketTimeout, BaseSSLError) as e: + # Py2 raises this as a BaseSSLError, Py3 raises it as socket timeout. + self._raise_timeout(err=e, url=url, timeout_value=conn.timeout) + raise + + # conn.request() calls http.client.*.request, not the method in + # urllib3.request. It also calls makefile (recv) on the socket. + try: + if chunked: + conn.request_chunked(method, url, **httplib_request_kw) + else: + conn.request(method, url, **httplib_request_kw) + + # We are swallowing BrokenPipeError (errno.EPIPE) since the server is + # legitimately able to close the connection after sending a valid response. + # With this behaviour, the received response is still readable. + except BrokenPipeError: + # Python 3 + pass + except IOError as e: + # Python 2 and macOS/Linux + # EPIPE and ESHUTDOWN are BrokenPipeError on Python 2, and EPROTOTYPE is needed on macOS + # https://erickt.github.io/blog/2014/11/19/adventures-in-debugging-a-potential-osx-kernel-bug/ + if e.errno not in { + errno.EPIPE, + errno.ESHUTDOWN, + errno.EPROTOTYPE, + }: + raise + + # Reset the timeout for the recv() on the socket + read_timeout = timeout_obj.read_timeout + + # App Engine doesn't have a sock attr + if getattr(conn, "sock", None): + # In Python 3 socket.py will catch EAGAIN and return None when you + # try and read into the file pointer created by http.client, which + # instead raises a BadStatusLine exception. Instead of catching + # the exception and assuming all BadStatusLine exceptions are read + # timeouts, check for a zero timeout before making the request. + if read_timeout == 0: + raise ReadTimeoutError( + self, url, "Read timed out. (read timeout=%s)" % read_timeout + ) + if read_timeout is Timeout.DEFAULT_TIMEOUT: + conn.sock.settimeout(socket.getdefaulttimeout()) + else: # None or a value + conn.sock.settimeout(read_timeout) + + # Receive the response from the server + try: + try: + # Python 2.7, use buffering of HTTP responses + httplib_response = conn.getresponse(buffering=True) + except TypeError: + # Python 3 + try: + httplib_response = conn.getresponse() + except BaseException as e: + # Remove the TypeError from the exception chain in + # Python 3 (including for exceptions like SystemExit). + # Otherwise it looks like a bug in the code. + six.raise_from(e, None) + except (SocketTimeout, BaseSSLError, SocketError) as e: + self._raise_timeout(err=e, url=url, timeout_value=read_timeout) + raise + + # AppEngine doesn't have a version attr. + http_version = getattr(conn, "_http_vsn_str", "HTTP/?") + log.debug( + '%s://%s:%s "%s %s %s" %s %s', + self.scheme, + self.host, + self.port, + method, + url, + http_version, + httplib_response.status, + httplib_response.length, + ) + + try: + assert_header_parsing(httplib_response.msg) + except (HeaderParsingError, TypeError) as hpe: # Platform-specific: Python 3 + log.warning( + "Failed to parse headers (url=%s): %s", + self._absolute_url(url), + hpe, + exc_info=True, + ) + + return httplib_response + + def _absolute_url(self, path): + return Url(scheme=self.scheme, host=self.host, port=self.port, path=path).url + + def close(self): + """ + Close all pooled connections and disable the pool. + """ + if self.pool is None: + return + # Disable access to the pool + old_pool, self.pool = self.pool, None + + try: + while True: + conn = old_pool.get(block=False) + if conn: + conn.close() + + except queue.Empty: + pass # Done. + + def is_same_host(self, url): + """ + Check if the given ``url`` is a member of the same host as this + connection pool. + """ + if url.startswith("/"): + return True + + # TODO: Add optional support for socket.gethostbyname checking. + scheme, host, port = get_host(url) + if host is not None: + host = _normalize_host(host, scheme=scheme) + + # Use explicit default port for comparison when none is given + if self.port and not port: + port = port_by_scheme.get(scheme) + elif not self.port and port == port_by_scheme.get(scheme): + port = None + + return (scheme, host, port) == (self.scheme, self.host, self.port) + + def urlopen( + self, + method, + url, + body=None, + headers=None, + retries=None, + redirect=True, + assert_same_host=True, + timeout=_Default, + pool_timeout=None, + release_conn=None, + chunked=False, + body_pos=None, + **response_kw + ): + """ + Get a connection from the pool and perform an HTTP request. This is the + lowest level call for making a request, so you'll need to specify all + the raw details. + + .. note:: + + More commonly, it's appropriate to use a convenience method provided + by :class:`.RequestMethods`, such as :meth:`request`. + + .. note:: + + `release_conn` will only behave as expected if + `preload_content=False` because we want to make + `preload_content=False` the default behaviour someday soon without + breaking backwards compatibility. + + :param method: + HTTP request method (such as GET, POST, PUT, etc.) + + :param url: + The URL to perform the request on. + + :param body: + Data to send in the request body, either :class:`str`, :class:`bytes`, + an iterable of :class:`str`/:class:`bytes`, or a file-like object. + + :param headers: + Dictionary of custom headers to send, such as User-Agent, + If-None-Match, etc. If None, pool headers are used. If provided, + these headers completely replace any pool-specific headers. + + :param retries: + Configure the number of retries to allow before raising a + :class:`~urllib3.exceptions.MaxRetryError` exception. + + Pass ``None`` to retry until you receive a response. Pass a + :class:`~urllib3.util.retry.Retry` object for fine-grained control + over different types of retries. + Pass an integer number to retry connection errors that many times, + but no other types of errors. Pass zero to never retry. + + If ``False``, then retries are disabled and any exception is raised + immediately. Also, instead of raising a MaxRetryError on redirects, + the redirect response will be returned. + + :type retries: :class:`~urllib3.util.retry.Retry`, False, or an int. + + :param redirect: + If True, automatically handle redirects (status codes 301, 302, + 303, 307, 308). Each redirect counts as a retry. Disabling retries + will disable redirect, too. + + :param assert_same_host: + If ``True``, will make sure that the host of the pool requests is + consistent else will raise HostChangedError. When ``False``, you can + use the pool on an HTTP proxy and request foreign hosts. + + :param timeout: + If specified, overrides the default timeout for this one + request. It may be a float (in seconds) or an instance of + :class:`urllib3.util.Timeout`. + + :param pool_timeout: + If set and the pool is set to block=True, then this method will + block for ``pool_timeout`` seconds and raise EmptyPoolError if no + connection is available within the time period. + + :param release_conn: + If False, then the urlopen call will not release the connection + back into the pool once a response is received (but will release if + you read the entire contents of the response such as when + `preload_content=True`). This is useful if you're not preloading + the response's content immediately. You will need to call + ``r.release_conn()`` on the response ``r`` to return the connection + back into the pool. If None, it takes the value of + ``response_kw.get('preload_content', True)``. + + :param chunked: + If True, urllib3 will send the body using chunked transfer + encoding. Otherwise, urllib3 will send the body using the standard + content-length form. Defaults to False. + + :param int body_pos: + Position to seek to in file-like body in the event of a retry or + redirect. Typically this won't need to be set because urllib3 will + auto-populate the value when needed. + + :param \\**response_kw: + Additional parameters are passed to + :meth:`urllib3.response.HTTPResponse.from_httplib` + """ + + parsed_url = parse_url(url) + destination_scheme = parsed_url.scheme + + if headers is None: + headers = self.headers + + if not isinstance(retries, Retry): + retries = Retry.from_int(retries, redirect=redirect, default=self.retries) + + if release_conn is None: + release_conn = response_kw.get("preload_content", True) + + # Check host + if assert_same_host and not self.is_same_host(url): + raise HostChangedError(self, url, retries) + + # Ensure that the URL we're connecting to is properly encoded + if url.startswith("/"): + url = six.ensure_str(_encode_target(url)) + else: + url = six.ensure_str(parsed_url.url) + + conn = None + + # Track whether `conn` needs to be released before + # returning/raising/recursing. Update this variable if necessary, and + # leave `release_conn` constant throughout the function. That way, if + # the function recurses, the original value of `release_conn` will be + # passed down into the recursive call, and its value will be respected. + # + # See issue #651 [1] for details. + # + # [1] + release_this_conn = release_conn + + http_tunnel_required = connection_requires_http_tunnel( + self.proxy, self.proxy_config, destination_scheme + ) + + # Merge the proxy headers. Only done when not using HTTP CONNECT. We + # have to copy the headers dict so we can safely change it without those + # changes being reflected in anyone else's copy. + if not http_tunnel_required: + headers = headers.copy() + headers.update(self.proxy_headers) + + # Must keep the exception bound to a separate variable or else Python 3 + # complains about UnboundLocalError. + err = None + + # Keep track of whether we cleanly exited the except block. This + # ensures we do proper cleanup in finally. + clean_exit = False + + # Rewind body position, if needed. Record current position + # for future rewinds in the event of a redirect/retry. + body_pos = set_file_position(body, body_pos) + + try: + # Request a connection from the queue. + timeout_obj = self._get_timeout(timeout) + conn = self._get_conn(timeout=pool_timeout) + + conn.timeout = timeout_obj.connect_timeout + + is_new_proxy_conn = self.proxy is not None and not getattr( + conn, "sock", None + ) + if is_new_proxy_conn and http_tunnel_required: + self._prepare_proxy(conn) + + # Make the request on the httplib connection object. + httplib_response = self._make_request( + conn, + method, + url, + timeout=timeout_obj, + body=body, + headers=headers, + chunked=chunked, + ) + + # If we're going to release the connection in ``finally:``, then + # the response doesn't need to know about the connection. Otherwise + # it will also try to release it and we'll have a double-release + # mess. + response_conn = conn if not release_conn else None + + # Pass method to Response for length checking + response_kw["request_method"] = method + + # Import httplib's response into our own wrapper object + response = self.ResponseCls.from_httplib( + httplib_response, + pool=self, + connection=response_conn, + retries=retries, + **response_kw + ) + + # Everything went great! + clean_exit = True + + except EmptyPoolError: + # Didn't get a connection from the pool, no need to clean up + clean_exit = True + release_this_conn = False + raise + + except ( + TimeoutError, + HTTPException, + SocketError, + ProtocolError, + BaseSSLError, + SSLError, + CertificateError, + ) as e: + # Discard the connection for these exceptions. It will be + # replaced during the next _get_conn() call. + clean_exit = False + if isinstance(e, (BaseSSLError, CertificateError)): + e = SSLError(e) + elif isinstance(e, (SocketError, NewConnectionError)) and self.proxy: + e = ProxyError("Cannot connect to proxy.", e) + elif isinstance(e, (SocketError, HTTPException)): + e = ProtocolError("Connection aborted.", e) + + retries = retries.increment( + method, url, error=e, _pool=self, _stacktrace=sys.exc_info()[2] + ) + retries.sleep() + + # Keep track of the error for the retry warning. + err = e + + finally: + if not clean_exit: + # We hit some kind of exception, handled or otherwise. We need + # to throw the connection away unless explicitly told not to. + # Close the connection, set the variable to None, and make sure + # we put the None back in the pool to avoid leaking it. + conn = conn and conn.close() + release_this_conn = True + + if release_this_conn: + # Put the connection back to be reused. If the connection is + # expired then it will be None, which will get replaced with a + # fresh connection during _get_conn. + self._put_conn(conn) + + if not conn: + # Try again + log.warning( + "Retrying (%r) after connection broken by '%r': %s", retries, err, url + ) + return self.urlopen( + method, + url, + body, + headers, + retries, + redirect, + assert_same_host, + timeout=timeout, + pool_timeout=pool_timeout, + release_conn=release_conn, + chunked=chunked, + body_pos=body_pos, + **response_kw + ) + + # Handle redirect? + redirect_location = redirect and response.get_redirect_location() + if redirect_location: + if response.status == 303: + method = "GET" + + try: + retries = retries.increment(method, url, response=response, _pool=self) + except MaxRetryError: + if retries.raise_on_redirect: + response.drain_conn() + raise + return response + + response.drain_conn() + retries.sleep_for_retry(response) + log.debug("Redirecting %s -> %s", url, redirect_location) + return self.urlopen( + method, + redirect_location, + body, + headers, + retries=retries, + redirect=redirect, + assert_same_host=assert_same_host, + timeout=timeout, + pool_timeout=pool_timeout, + release_conn=release_conn, + chunked=chunked, + body_pos=body_pos, + **response_kw + ) + + # Check if we should retry the HTTP response. + has_retry_after = bool(response.getheader("Retry-After")) + if retries.is_retry(method, response.status, has_retry_after): + try: + retries = retries.increment(method, url, response=response, _pool=self) + except MaxRetryError: + if retries.raise_on_status: + response.drain_conn() + raise + return response + + response.drain_conn() + retries.sleep(response) + log.debug("Retry: %s", url) + return self.urlopen( + method, + url, + body, + headers, + retries=retries, + redirect=redirect, + assert_same_host=assert_same_host, + timeout=timeout, + pool_timeout=pool_timeout, + release_conn=release_conn, + chunked=chunked, + body_pos=body_pos, + **response_kw + ) + + return response + + +class HTTPSConnectionPool(HTTPConnectionPool): + """ + Same as :class:`.HTTPConnectionPool`, but HTTPS. + + :class:`.HTTPSConnection` uses one of ``assert_fingerprint``, + ``assert_hostname`` and ``host`` in this order to verify connections. + If ``assert_hostname`` is False, no verification is done. + + The ``key_file``, ``cert_file``, ``cert_reqs``, ``ca_certs``, + ``ca_cert_dir``, ``ssl_version``, ``key_password`` are only used if :mod:`ssl` + is available and are fed into :meth:`urllib3.util.ssl_wrap_socket` to upgrade + the connection socket into an SSL socket. + """ + + scheme = "https" + ConnectionCls = HTTPSConnection + + def __init__( + self, + host, + port=None, + strict=False, + timeout=Timeout.DEFAULT_TIMEOUT, + maxsize=1, + block=False, + headers=None, + retries=None, + _proxy=None, + _proxy_headers=None, + key_file=None, + cert_file=None, + cert_reqs=None, + key_password=None, + ca_certs=None, + ssl_version=None, + assert_hostname=None, + assert_fingerprint=None, + ca_cert_dir=None, + **conn_kw + ): + + HTTPConnectionPool.__init__( + self, + host, + port, + strict, + timeout, + maxsize, + block, + headers, + retries, + _proxy, + _proxy_headers, + **conn_kw + ) + + self.key_file = key_file + self.cert_file = cert_file + self.cert_reqs = cert_reqs + self.key_password = key_password + self.ca_certs = ca_certs + self.ca_cert_dir = ca_cert_dir + self.ssl_version = ssl_version + self.assert_hostname = assert_hostname + self.assert_fingerprint = assert_fingerprint + + def _prepare_conn(self, conn): + """ + Prepare the ``connection`` for :meth:`urllib3.util.ssl_wrap_socket` + and establish the tunnel if proxy is used. + """ + + if isinstance(conn, VerifiedHTTPSConnection): + conn.set_cert( + key_file=self.key_file, + key_password=self.key_password, + cert_file=self.cert_file, + cert_reqs=self.cert_reqs, + ca_certs=self.ca_certs, + ca_cert_dir=self.ca_cert_dir, + assert_hostname=self.assert_hostname, + assert_fingerprint=self.assert_fingerprint, + ) + conn.ssl_version = self.ssl_version + return conn + + def _prepare_proxy(self, conn): + """ + Establishes a tunnel connection through HTTP CONNECT. + + Tunnel connection is established early because otherwise httplib would + improperly set Host: header to proxy's IP:port. + """ + + conn.set_tunnel(self._proxy_host, self.port, self.proxy_headers) + + if self.proxy.scheme == "https": + conn.tls_in_tls_required = True + + conn.connect() + + def _new_conn(self): + """ + Return a fresh :class:`http.client.HTTPSConnection`. + """ + self.num_connections += 1 + log.debug( + "Starting new HTTPS connection (%d): %s:%s", + self.num_connections, + self.host, + self.port or "443", + ) + + if not self.ConnectionCls or self.ConnectionCls is DummyConnection: + raise SSLError( + "Can't connect to HTTPS URL because the SSL module is not available." + ) + + actual_host = self.host + actual_port = self.port + if self.proxy is not None: + actual_host = self.proxy.host + actual_port = self.proxy.port + + conn = self.ConnectionCls( + host=actual_host, + port=actual_port, + timeout=self.timeout.connect_timeout, + strict=self.strict, + cert_file=self.cert_file, + key_file=self.key_file, + key_password=self.key_password, + **self.conn_kw + ) + + return self._prepare_conn(conn) + + def _validate_conn(self, conn): + """ + Called right before a request is made, after the socket is created. + """ + super(HTTPSConnectionPool, self)._validate_conn(conn) + + # Force connect early to allow us to validate the connection. + if not getattr(conn, "sock", None): # AppEngine might not have `.sock` + conn.connect() + + if not conn.is_verified: + warnings.warn( + ( + "Unverified HTTPS request is being made to host '%s'. " + "Adding certificate verification is strongly advised. See: " + "https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html" + "#ssl-warnings" % conn.host + ), + InsecureRequestWarning, + ) + + +def connection_from_url(url, **kw): + """ + Given a url, return an :class:`.ConnectionPool` instance of its host. + + This is a shortcut for not having to parse out the scheme, host, and port + of the url before creating an :class:`.ConnectionPool` instance. + + :param url: + Absolute URL string that must include the scheme. Port is optional. + + :param \\**kw: + Passes additional parameters to the constructor of the appropriate + :class:`.ConnectionPool`. Useful for specifying things like + timeout, maxsize, headers, etc. + + Example:: + + >>> conn = connection_from_url('http://google.com/') + >>> r = conn.request('GET', '/') + """ + scheme, host, port = get_host(url) + port = port or port_by_scheme.get(scheme, 80) + if scheme == "https": + return HTTPSConnectionPool(host, port=port, **kw) + else: + return HTTPConnectionPool(host, port=port, **kw) + + +def _normalize_host(host, scheme): + """ + Normalize hosts for comparisons and use with sockets. + """ + + host = normalize_host(host, scheme) + + # httplib doesn't like it when we include brackets in IPv6 addresses + # Specifically, if we include brackets but also pass the port then + # httplib crazily doubles up the square brackets on the Host header. + # Instead, we need to make sure we never pass ``None`` as the port. + # However, for backward compatibility reasons we can't actually + # *assert* that. See http://bugs.python.org/issue28539 + if host.startswith("[") and host.endswith("]"): + host = host[1:-1] + return host diff --git a/openpype/hosts/fusion/vendor/urllib3/contrib/__init__.py b/openpype/hosts/fusion/vendor/urllib3/contrib/__init__.py new file mode 100644 index 0000000000..e69de29bb2 diff --git a/openpype/hosts/fusion/vendor/urllib3/contrib/_appengine_environ.py b/openpype/hosts/fusion/vendor/urllib3/contrib/_appengine_environ.py new file mode 100644 index 0000000000..8765b907d7 --- /dev/null +++ b/openpype/hosts/fusion/vendor/urllib3/contrib/_appengine_environ.py @@ -0,0 +1,36 @@ +""" +This module provides means to detect the App Engine environment. +""" + +import os + + +def is_appengine(): + return is_local_appengine() or is_prod_appengine() + + +def is_appengine_sandbox(): + """Reports if the app is running in the first generation sandbox. + + The second generation runtimes are technically still in a sandbox, but it + is much less restrictive, so generally you shouldn't need to check for it. + see https://cloud.google.com/appengine/docs/standard/runtimes + """ + return is_appengine() and os.environ["APPENGINE_RUNTIME"] == "python27" + + +def is_local_appengine(): + return "APPENGINE_RUNTIME" in os.environ and os.environ.get( + "SERVER_SOFTWARE", "" + ).startswith("Development/") + + +def is_prod_appengine(): + return "APPENGINE_RUNTIME" in os.environ and os.environ.get( + "SERVER_SOFTWARE", "" + ).startswith("Google App Engine/") + + +def is_prod_appengine_mvms(): + """Deprecated.""" + return False diff --git a/openpype/hosts/fusion/vendor/urllib3/contrib/_securetransport/__init__.py b/openpype/hosts/fusion/vendor/urllib3/contrib/_securetransport/__init__.py new file mode 100644 index 0000000000..e69de29bb2 diff --git a/openpype/hosts/fusion/vendor/urllib3/contrib/_securetransport/bindings.py b/openpype/hosts/fusion/vendor/urllib3/contrib/_securetransport/bindings.py new file mode 100644 index 0000000000..11524d400b --- /dev/null +++ b/openpype/hosts/fusion/vendor/urllib3/contrib/_securetransport/bindings.py @@ -0,0 +1,519 @@ +""" +This module uses ctypes to bind a whole bunch of functions and constants from +SecureTransport. The goal here is to provide the low-level API to +SecureTransport. These are essentially the C-level functions and constants, and +they're pretty gross to work with. + +This code is a bastardised version of the code found in Will Bond's oscrypto +library. An enormous debt is owed to him for blazing this trail for us. For +that reason, this code should be considered to be covered both by urllib3's +license and by oscrypto's: + + Copyright (c) 2015-2016 Will Bond + + Permission is hereby granted, free of charge, to any person obtaining a + copy of this software and associated documentation files (the "Software"), + to deal in the Software without restriction, including without limitation + the rights to use, copy, modify, merge, publish, distribute, sublicense, + and/or sell copies of the Software, and to permit persons to whom the + Software is furnished to do so, subject to the following conditions: + + The above copyright notice and this permission notice shall be included in + all copies or substantial portions of the Software. + + THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR + IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, + FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE + AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER + LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING + FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER + DEALINGS IN THE SOFTWARE. +""" +from __future__ import absolute_import + +import platform +from ctypes import ( + CDLL, + CFUNCTYPE, + POINTER, + c_bool, + c_byte, + c_char_p, + c_int32, + c_long, + c_size_t, + c_uint32, + c_ulong, + c_void_p, +) +from ctypes.util import find_library + +from urllib3.packages.six import raise_from + +if platform.system() != "Darwin": + raise ImportError("Only macOS is supported") + +version = platform.mac_ver()[0] +version_info = tuple(map(int, version.split("."))) +if version_info < (10, 8): + raise OSError( + "Only OS X 10.8 and newer are supported, not %s.%s" + % (version_info[0], version_info[1]) + ) + + +def load_cdll(name, macos10_16_path): + """Loads a CDLL by name, falling back to known path on 10.16+""" + try: + # Big Sur is technically 11 but we use 10.16 due to the Big Sur + # beta being labeled as 10.16. + if version_info >= (10, 16): + path = macos10_16_path + else: + path = find_library(name) + if not path: + raise OSError # Caught and reraised as 'ImportError' + return CDLL(path, use_errno=True) + except OSError: + raise_from(ImportError("The library %s failed to load" % name), None) + + +Security = load_cdll( + "Security", "/System/Library/Frameworks/Security.framework/Security" +) +CoreFoundation = load_cdll( + "CoreFoundation", + "/System/Library/Frameworks/CoreFoundation.framework/CoreFoundation", +) + + +Boolean = c_bool +CFIndex = c_long +CFStringEncoding = c_uint32 +CFData = c_void_p +CFString = c_void_p +CFArray = c_void_p +CFMutableArray = c_void_p +CFDictionary = c_void_p +CFError = c_void_p +CFType = c_void_p +CFTypeID = c_ulong + +CFTypeRef = POINTER(CFType) +CFAllocatorRef = c_void_p + +OSStatus = c_int32 + +CFDataRef = POINTER(CFData) +CFStringRef = POINTER(CFString) +CFArrayRef = POINTER(CFArray) +CFMutableArrayRef = POINTER(CFMutableArray) +CFDictionaryRef = POINTER(CFDictionary) +CFArrayCallBacks = c_void_p +CFDictionaryKeyCallBacks = c_void_p +CFDictionaryValueCallBacks = c_void_p + +SecCertificateRef = POINTER(c_void_p) +SecExternalFormat = c_uint32 +SecExternalItemType = c_uint32 +SecIdentityRef = POINTER(c_void_p) +SecItemImportExportFlags = c_uint32 +SecItemImportExportKeyParameters = c_void_p +SecKeychainRef = POINTER(c_void_p) +SSLProtocol = c_uint32 +SSLCipherSuite = c_uint32 +SSLContextRef = POINTER(c_void_p) +SecTrustRef = POINTER(c_void_p) +SSLConnectionRef = c_uint32 +SecTrustResultType = c_uint32 +SecTrustOptionFlags = c_uint32 +SSLProtocolSide = c_uint32 +SSLConnectionType = c_uint32 +SSLSessionOption = c_uint32 + + +try: + Security.SecItemImport.argtypes = [ + CFDataRef, + CFStringRef, + POINTER(SecExternalFormat), + POINTER(SecExternalItemType), + SecItemImportExportFlags, + POINTER(SecItemImportExportKeyParameters), + SecKeychainRef, + POINTER(CFArrayRef), + ] + Security.SecItemImport.restype = OSStatus + + Security.SecCertificateGetTypeID.argtypes = [] + Security.SecCertificateGetTypeID.restype = CFTypeID + + Security.SecIdentityGetTypeID.argtypes = [] + Security.SecIdentityGetTypeID.restype = CFTypeID + + Security.SecKeyGetTypeID.argtypes = [] + Security.SecKeyGetTypeID.restype = CFTypeID + + Security.SecCertificateCreateWithData.argtypes = [CFAllocatorRef, CFDataRef] + Security.SecCertificateCreateWithData.restype = SecCertificateRef + + Security.SecCertificateCopyData.argtypes = [SecCertificateRef] + Security.SecCertificateCopyData.restype = CFDataRef + + Security.SecCopyErrorMessageString.argtypes = [OSStatus, c_void_p] + Security.SecCopyErrorMessageString.restype = CFStringRef + + Security.SecIdentityCreateWithCertificate.argtypes = [ + CFTypeRef, + SecCertificateRef, + POINTER(SecIdentityRef), + ] + Security.SecIdentityCreateWithCertificate.restype = OSStatus + + Security.SecKeychainCreate.argtypes = [ + c_char_p, + c_uint32, + c_void_p, + Boolean, + c_void_p, + POINTER(SecKeychainRef), + ] + Security.SecKeychainCreate.restype = OSStatus + + Security.SecKeychainDelete.argtypes = [SecKeychainRef] + Security.SecKeychainDelete.restype = OSStatus + + Security.SecPKCS12Import.argtypes = [ + CFDataRef, + CFDictionaryRef, + POINTER(CFArrayRef), + ] + Security.SecPKCS12Import.restype = OSStatus + + SSLReadFunc = CFUNCTYPE(OSStatus, SSLConnectionRef, c_void_p, POINTER(c_size_t)) + SSLWriteFunc = CFUNCTYPE( + OSStatus, SSLConnectionRef, POINTER(c_byte), POINTER(c_size_t) + ) + + Security.SSLSetIOFuncs.argtypes = [SSLContextRef, SSLReadFunc, SSLWriteFunc] + Security.SSLSetIOFuncs.restype = OSStatus + + Security.SSLSetPeerID.argtypes = [SSLContextRef, c_char_p, c_size_t] + Security.SSLSetPeerID.restype = OSStatus + + Security.SSLSetCertificate.argtypes = [SSLContextRef, CFArrayRef] + Security.SSLSetCertificate.restype = OSStatus + + Security.SSLSetCertificateAuthorities.argtypes = [SSLContextRef, CFTypeRef, Boolean] + Security.SSLSetCertificateAuthorities.restype = OSStatus + + Security.SSLSetConnection.argtypes = [SSLContextRef, SSLConnectionRef] + Security.SSLSetConnection.restype = OSStatus + + Security.SSLSetPeerDomainName.argtypes = [SSLContextRef, c_char_p, c_size_t] + Security.SSLSetPeerDomainName.restype = OSStatus + + Security.SSLHandshake.argtypes = [SSLContextRef] + Security.SSLHandshake.restype = OSStatus + + Security.SSLRead.argtypes = [SSLContextRef, c_char_p, c_size_t, POINTER(c_size_t)] + Security.SSLRead.restype = OSStatus + + Security.SSLWrite.argtypes = [SSLContextRef, c_char_p, c_size_t, POINTER(c_size_t)] + Security.SSLWrite.restype = OSStatus + + Security.SSLClose.argtypes = [SSLContextRef] + Security.SSLClose.restype = OSStatus + + Security.SSLGetNumberSupportedCiphers.argtypes = [SSLContextRef, POINTER(c_size_t)] + Security.SSLGetNumberSupportedCiphers.restype = OSStatus + + Security.SSLGetSupportedCiphers.argtypes = [ + SSLContextRef, + POINTER(SSLCipherSuite), + POINTER(c_size_t), + ] + Security.SSLGetSupportedCiphers.restype = OSStatus + + Security.SSLSetEnabledCiphers.argtypes = [ + SSLContextRef, + POINTER(SSLCipherSuite), + c_size_t, + ] + Security.SSLSetEnabledCiphers.restype = OSStatus + + Security.SSLGetNumberEnabledCiphers.argtype = [SSLContextRef, POINTER(c_size_t)] + Security.SSLGetNumberEnabledCiphers.restype = OSStatus + + Security.SSLGetEnabledCiphers.argtypes = [ + SSLContextRef, + POINTER(SSLCipherSuite), + POINTER(c_size_t), + ] + Security.SSLGetEnabledCiphers.restype = OSStatus + + Security.SSLGetNegotiatedCipher.argtypes = [SSLContextRef, POINTER(SSLCipherSuite)] + Security.SSLGetNegotiatedCipher.restype = OSStatus + + Security.SSLGetNegotiatedProtocolVersion.argtypes = [ + SSLContextRef, + POINTER(SSLProtocol), + ] + Security.SSLGetNegotiatedProtocolVersion.restype = OSStatus + + Security.SSLCopyPeerTrust.argtypes = [SSLContextRef, POINTER(SecTrustRef)] + Security.SSLCopyPeerTrust.restype = OSStatus + + Security.SecTrustSetAnchorCertificates.argtypes = [SecTrustRef, CFArrayRef] + Security.SecTrustSetAnchorCertificates.restype = OSStatus + + Security.SecTrustSetAnchorCertificatesOnly.argstypes = [SecTrustRef, Boolean] + Security.SecTrustSetAnchorCertificatesOnly.restype = OSStatus + + Security.SecTrustEvaluate.argtypes = [SecTrustRef, POINTER(SecTrustResultType)] + Security.SecTrustEvaluate.restype = OSStatus + + Security.SecTrustGetCertificateCount.argtypes = [SecTrustRef] + Security.SecTrustGetCertificateCount.restype = CFIndex + + Security.SecTrustGetCertificateAtIndex.argtypes = [SecTrustRef, CFIndex] + Security.SecTrustGetCertificateAtIndex.restype = SecCertificateRef + + Security.SSLCreateContext.argtypes = [ + CFAllocatorRef, + SSLProtocolSide, + SSLConnectionType, + ] + Security.SSLCreateContext.restype = SSLContextRef + + Security.SSLSetSessionOption.argtypes = [SSLContextRef, SSLSessionOption, Boolean] + Security.SSLSetSessionOption.restype = OSStatus + + Security.SSLSetProtocolVersionMin.argtypes = [SSLContextRef, SSLProtocol] + Security.SSLSetProtocolVersionMin.restype = OSStatus + + Security.SSLSetProtocolVersionMax.argtypes = [SSLContextRef, SSLProtocol] + Security.SSLSetProtocolVersionMax.restype = OSStatus + + try: + Security.SSLSetALPNProtocols.argtypes = [SSLContextRef, CFArrayRef] + Security.SSLSetALPNProtocols.restype = OSStatus + except AttributeError: + # Supported only in 10.12+ + pass + + Security.SecCopyErrorMessageString.argtypes = [OSStatus, c_void_p] + Security.SecCopyErrorMessageString.restype = CFStringRef + + Security.SSLReadFunc = SSLReadFunc + Security.SSLWriteFunc = SSLWriteFunc + Security.SSLContextRef = SSLContextRef + Security.SSLProtocol = SSLProtocol + Security.SSLCipherSuite = SSLCipherSuite + Security.SecIdentityRef = SecIdentityRef + Security.SecKeychainRef = SecKeychainRef + Security.SecTrustRef = SecTrustRef + Security.SecTrustResultType = SecTrustResultType + Security.SecExternalFormat = SecExternalFormat + Security.OSStatus = OSStatus + + Security.kSecImportExportPassphrase = CFStringRef.in_dll( + Security, "kSecImportExportPassphrase" + ) + Security.kSecImportItemIdentity = CFStringRef.in_dll( + Security, "kSecImportItemIdentity" + ) + + # CoreFoundation time! + CoreFoundation.CFRetain.argtypes = [CFTypeRef] + CoreFoundation.CFRetain.restype = CFTypeRef + + CoreFoundation.CFRelease.argtypes = [CFTypeRef] + CoreFoundation.CFRelease.restype = None + + CoreFoundation.CFGetTypeID.argtypes = [CFTypeRef] + CoreFoundation.CFGetTypeID.restype = CFTypeID + + CoreFoundation.CFStringCreateWithCString.argtypes = [ + CFAllocatorRef, + c_char_p, + CFStringEncoding, + ] + CoreFoundation.CFStringCreateWithCString.restype = CFStringRef + + CoreFoundation.CFStringGetCStringPtr.argtypes = [CFStringRef, CFStringEncoding] + CoreFoundation.CFStringGetCStringPtr.restype = c_char_p + + CoreFoundation.CFStringGetCString.argtypes = [ + CFStringRef, + c_char_p, + CFIndex, + CFStringEncoding, + ] + CoreFoundation.CFStringGetCString.restype = c_bool + + CoreFoundation.CFDataCreate.argtypes = [CFAllocatorRef, c_char_p, CFIndex] + CoreFoundation.CFDataCreate.restype = CFDataRef + + CoreFoundation.CFDataGetLength.argtypes = [CFDataRef] + CoreFoundation.CFDataGetLength.restype = CFIndex + + CoreFoundation.CFDataGetBytePtr.argtypes = [CFDataRef] + CoreFoundation.CFDataGetBytePtr.restype = c_void_p + + CoreFoundation.CFDictionaryCreate.argtypes = [ + CFAllocatorRef, + POINTER(CFTypeRef), + POINTER(CFTypeRef), + CFIndex, + CFDictionaryKeyCallBacks, + CFDictionaryValueCallBacks, + ] + CoreFoundation.CFDictionaryCreate.restype = CFDictionaryRef + + CoreFoundation.CFDictionaryGetValue.argtypes = [CFDictionaryRef, CFTypeRef] + CoreFoundation.CFDictionaryGetValue.restype = CFTypeRef + + CoreFoundation.CFArrayCreate.argtypes = [ + CFAllocatorRef, + POINTER(CFTypeRef), + CFIndex, + CFArrayCallBacks, + ] + CoreFoundation.CFArrayCreate.restype = CFArrayRef + + CoreFoundation.CFArrayCreateMutable.argtypes = [ + CFAllocatorRef, + CFIndex, + CFArrayCallBacks, + ] + CoreFoundation.CFArrayCreateMutable.restype = CFMutableArrayRef + + CoreFoundation.CFArrayAppendValue.argtypes = [CFMutableArrayRef, c_void_p] + CoreFoundation.CFArrayAppendValue.restype = None + + CoreFoundation.CFArrayGetCount.argtypes = [CFArrayRef] + CoreFoundation.CFArrayGetCount.restype = CFIndex + + CoreFoundation.CFArrayGetValueAtIndex.argtypes = [CFArrayRef, CFIndex] + CoreFoundation.CFArrayGetValueAtIndex.restype = c_void_p + + CoreFoundation.kCFAllocatorDefault = CFAllocatorRef.in_dll( + CoreFoundation, "kCFAllocatorDefault" + ) + CoreFoundation.kCFTypeArrayCallBacks = c_void_p.in_dll( + CoreFoundation, "kCFTypeArrayCallBacks" + ) + CoreFoundation.kCFTypeDictionaryKeyCallBacks = c_void_p.in_dll( + CoreFoundation, "kCFTypeDictionaryKeyCallBacks" + ) + CoreFoundation.kCFTypeDictionaryValueCallBacks = c_void_p.in_dll( + CoreFoundation, "kCFTypeDictionaryValueCallBacks" + ) + + CoreFoundation.CFTypeRef = CFTypeRef + CoreFoundation.CFArrayRef = CFArrayRef + CoreFoundation.CFStringRef = CFStringRef + CoreFoundation.CFDictionaryRef = CFDictionaryRef + +except (AttributeError): + raise ImportError("Error initializing ctypes") + + +class CFConst(object): + """ + A class object that acts as essentially a namespace for CoreFoundation + constants. + """ + + kCFStringEncodingUTF8 = CFStringEncoding(0x08000100) + + +class SecurityConst(object): + """ + A class object that acts as essentially a namespace for Security constants. + """ + + kSSLSessionOptionBreakOnServerAuth = 0 + + kSSLProtocol2 = 1 + kSSLProtocol3 = 2 + kTLSProtocol1 = 4 + kTLSProtocol11 = 7 + kTLSProtocol12 = 8 + # SecureTransport does not support TLS 1.3 even if there's a constant for it + kTLSProtocol13 = 10 + kTLSProtocolMaxSupported = 999 + + kSSLClientSide = 1 + kSSLStreamType = 0 + + kSecFormatPEMSequence = 10 + + kSecTrustResultInvalid = 0 + kSecTrustResultProceed = 1 + # This gap is present on purpose: this was kSecTrustResultConfirm, which + # is deprecated. + kSecTrustResultDeny = 3 + kSecTrustResultUnspecified = 4 + kSecTrustResultRecoverableTrustFailure = 5 + kSecTrustResultFatalTrustFailure = 6 + kSecTrustResultOtherError = 7 + + errSSLProtocol = -9800 + errSSLWouldBlock = -9803 + errSSLClosedGraceful = -9805 + errSSLClosedNoNotify = -9816 + errSSLClosedAbort = -9806 + + errSSLXCertChainInvalid = -9807 + errSSLCrypto = -9809 + errSSLInternal = -9810 + errSSLCertExpired = -9814 + errSSLCertNotYetValid = -9815 + errSSLUnknownRootCert = -9812 + errSSLNoRootCert = -9813 + errSSLHostNameMismatch = -9843 + errSSLPeerHandshakeFail = -9824 + errSSLPeerUserCancelled = -9839 + errSSLWeakPeerEphemeralDHKey = -9850 + errSSLServerAuthCompleted = -9841 + errSSLRecordOverflow = -9847 + + errSecVerifyFailed = -67808 + errSecNoTrustSettings = -25263 + errSecItemNotFound = -25300 + errSecInvalidTrustSettings = -25262 + + # Cipher suites. We only pick the ones our default cipher string allows. + # Source: https://developer.apple.com/documentation/security/1550981-ssl_cipher_suite_values + TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384 = 0xC02C + TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 = 0xC030 + TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256 = 0xC02B + TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256 = 0xC02F + TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305_SHA256 = 0xCCA9 + TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305_SHA256 = 0xCCA8 + TLS_DHE_RSA_WITH_AES_256_GCM_SHA384 = 0x009F + TLS_DHE_RSA_WITH_AES_128_GCM_SHA256 = 0x009E + TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA384 = 0xC024 + TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384 = 0xC028 + TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA = 0xC00A + TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA = 0xC014 + TLS_DHE_RSA_WITH_AES_256_CBC_SHA256 = 0x006B + TLS_DHE_RSA_WITH_AES_256_CBC_SHA = 0x0039 + TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA256 = 0xC023 + TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256 = 0xC027 + TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA = 0xC009 + TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA = 0xC013 + TLS_DHE_RSA_WITH_AES_128_CBC_SHA256 = 0x0067 + TLS_DHE_RSA_WITH_AES_128_CBC_SHA = 0x0033 + TLS_RSA_WITH_AES_256_GCM_SHA384 = 0x009D + TLS_RSA_WITH_AES_128_GCM_SHA256 = 0x009C + TLS_RSA_WITH_AES_256_CBC_SHA256 = 0x003D + TLS_RSA_WITH_AES_128_CBC_SHA256 = 0x003C + TLS_RSA_WITH_AES_256_CBC_SHA = 0x0035 + TLS_RSA_WITH_AES_128_CBC_SHA = 0x002F + TLS_AES_128_GCM_SHA256 = 0x1301 + TLS_AES_256_GCM_SHA384 = 0x1302 + TLS_AES_128_CCM_8_SHA256 = 0x1305 + TLS_AES_128_CCM_SHA256 = 0x1304 diff --git a/openpype/hosts/fusion/vendor/urllib3/contrib/_securetransport/low_level.py b/openpype/hosts/fusion/vendor/urllib3/contrib/_securetransport/low_level.py new file mode 100644 index 0000000000..ed8120190c --- /dev/null +++ b/openpype/hosts/fusion/vendor/urllib3/contrib/_securetransport/low_level.py @@ -0,0 +1,396 @@ +""" +Low-level helpers for the SecureTransport bindings. + +These are Python functions that are not directly related to the high-level APIs +but are necessary to get them to work. They include a whole bunch of low-level +CoreFoundation messing about and memory management. The concerns in this module +are almost entirely about trying to avoid memory leaks and providing +appropriate and useful assistance to the higher-level code. +""" +import base64 +import ctypes +import itertools +import os +import re +import ssl +import struct +import tempfile + +from .bindings import CFConst, CoreFoundation, Security + +# This regular expression is used to grab PEM data out of a PEM bundle. +_PEM_CERTS_RE = re.compile( + b"-----BEGIN CERTIFICATE-----\n(.*?)\n-----END CERTIFICATE-----", re.DOTALL +) + + +def _cf_data_from_bytes(bytestring): + """ + Given a bytestring, create a CFData object from it. This CFData object must + be CFReleased by the caller. + """ + return CoreFoundation.CFDataCreate( + CoreFoundation.kCFAllocatorDefault, bytestring, len(bytestring) + ) + + +def _cf_dictionary_from_tuples(tuples): + """ + Given a list of Python tuples, create an associated CFDictionary. + """ + dictionary_size = len(tuples) + + # We need to get the dictionary keys and values out in the same order. + keys = (t[0] for t in tuples) + values = (t[1] for t in tuples) + cf_keys = (CoreFoundation.CFTypeRef * dictionary_size)(*keys) + cf_values = (CoreFoundation.CFTypeRef * dictionary_size)(*values) + + return CoreFoundation.CFDictionaryCreate( + CoreFoundation.kCFAllocatorDefault, + cf_keys, + cf_values, + dictionary_size, + CoreFoundation.kCFTypeDictionaryKeyCallBacks, + CoreFoundation.kCFTypeDictionaryValueCallBacks, + ) + + +def _cfstr(py_bstr): + """ + Given a Python binary data, create a CFString. + The string must be CFReleased by the caller. + """ + c_str = ctypes.c_char_p(py_bstr) + cf_str = CoreFoundation.CFStringCreateWithCString( + CoreFoundation.kCFAllocatorDefault, + c_str, + CFConst.kCFStringEncodingUTF8, + ) + return cf_str + + +def _create_cfstring_array(lst): + """ + Given a list of Python binary data, create an associated CFMutableArray. + The array must be CFReleased by the caller. + + Raises an ssl.SSLError on failure. + """ + cf_arr = None + try: + cf_arr = CoreFoundation.CFArrayCreateMutable( + CoreFoundation.kCFAllocatorDefault, + 0, + ctypes.byref(CoreFoundation.kCFTypeArrayCallBacks), + ) + if not cf_arr: + raise MemoryError("Unable to allocate memory!") + for item in lst: + cf_str = _cfstr(item) + if not cf_str: + raise MemoryError("Unable to allocate memory!") + try: + CoreFoundation.CFArrayAppendValue(cf_arr, cf_str) + finally: + CoreFoundation.CFRelease(cf_str) + except BaseException as e: + if cf_arr: + CoreFoundation.CFRelease(cf_arr) + raise ssl.SSLError("Unable to allocate array: %s" % (e,)) + return cf_arr + + +def _cf_string_to_unicode(value): + """ + Creates a Unicode string from a CFString object. Used entirely for error + reporting. + + Yes, it annoys me quite a lot that this function is this complex. + """ + value_as_void_p = ctypes.cast(value, ctypes.POINTER(ctypes.c_void_p)) + + string = CoreFoundation.CFStringGetCStringPtr( + value_as_void_p, CFConst.kCFStringEncodingUTF8 + ) + if string is None: + buffer = ctypes.create_string_buffer(1024) + result = CoreFoundation.CFStringGetCString( + value_as_void_p, buffer, 1024, CFConst.kCFStringEncodingUTF8 + ) + if not result: + raise OSError("Error copying C string from CFStringRef") + string = buffer.value + if string is not None: + string = string.decode("utf-8") + return string + + +def _assert_no_error(error, exception_class=None): + """ + Checks the return code and throws an exception if there is an error to + report + """ + if error == 0: + return + + cf_error_string = Security.SecCopyErrorMessageString(error, None) + output = _cf_string_to_unicode(cf_error_string) + CoreFoundation.CFRelease(cf_error_string) + + if output is None or output == u"": + output = u"OSStatus %s" % error + + if exception_class is None: + exception_class = ssl.SSLError + + raise exception_class(output) + + +def _cert_array_from_pem(pem_bundle): + """ + Given a bundle of certs in PEM format, turns them into a CFArray of certs + that can be used to validate a cert chain. + """ + # Normalize the PEM bundle's line endings. + pem_bundle = pem_bundle.replace(b"\r\n", b"\n") + + der_certs = [ + base64.b64decode(match.group(1)) for match in _PEM_CERTS_RE.finditer(pem_bundle) + ] + if not der_certs: + raise ssl.SSLError("No root certificates specified") + + cert_array = CoreFoundation.CFArrayCreateMutable( + CoreFoundation.kCFAllocatorDefault, + 0, + ctypes.byref(CoreFoundation.kCFTypeArrayCallBacks), + ) + if not cert_array: + raise ssl.SSLError("Unable to allocate memory!") + + try: + for der_bytes in der_certs: + certdata = _cf_data_from_bytes(der_bytes) + if not certdata: + raise ssl.SSLError("Unable to allocate memory!") + cert = Security.SecCertificateCreateWithData( + CoreFoundation.kCFAllocatorDefault, certdata + ) + CoreFoundation.CFRelease(certdata) + if not cert: + raise ssl.SSLError("Unable to build cert object!") + + CoreFoundation.CFArrayAppendValue(cert_array, cert) + CoreFoundation.CFRelease(cert) + except Exception: + # We need to free the array before the exception bubbles further. + # We only want to do that if an error occurs: otherwise, the caller + # should free. + CoreFoundation.CFRelease(cert_array) + + return cert_array + + +def _is_cert(item): + """ + Returns True if a given CFTypeRef is a certificate. + """ + expected = Security.SecCertificateGetTypeID() + return CoreFoundation.CFGetTypeID(item) == expected + + +def _is_identity(item): + """ + Returns True if a given CFTypeRef is an identity. + """ + expected = Security.SecIdentityGetTypeID() + return CoreFoundation.CFGetTypeID(item) == expected + + +def _temporary_keychain(): + """ + This function creates a temporary Mac keychain that we can use to work with + credentials. This keychain uses a one-time password and a temporary file to + store the data. We expect to have one keychain per socket. The returned + SecKeychainRef must be freed by the caller, including calling + SecKeychainDelete. + + Returns a tuple of the SecKeychainRef and the path to the temporary + directory that contains it. + """ + # Unfortunately, SecKeychainCreate requires a path to a keychain. This + # means we cannot use mkstemp to use a generic temporary file. Instead, + # we're going to create a temporary directory and a filename to use there. + # This filename will be 8 random bytes expanded into base64. We also need + # some random bytes to password-protect the keychain we're creating, so we + # ask for 40 random bytes. + random_bytes = os.urandom(40) + filename = base64.b16encode(random_bytes[:8]).decode("utf-8") + password = base64.b16encode(random_bytes[8:]) # Must be valid UTF-8 + tempdirectory = tempfile.mkdtemp() + + keychain_path = os.path.join(tempdirectory, filename).encode("utf-8") + + # We now want to create the keychain itself. + keychain = Security.SecKeychainRef() + status = Security.SecKeychainCreate( + keychain_path, len(password), password, False, None, ctypes.byref(keychain) + ) + _assert_no_error(status) + + # Having created the keychain, we want to pass it off to the caller. + return keychain, tempdirectory + + +def _load_items_from_file(keychain, path): + """ + Given a single file, loads all the trust objects from it into arrays and + the keychain. + Returns a tuple of lists: the first list is a list of identities, the + second a list of certs. + """ + certificates = [] + identities = [] + result_array = None + + with open(path, "rb") as f: + raw_filedata = f.read() + + try: + filedata = CoreFoundation.CFDataCreate( + CoreFoundation.kCFAllocatorDefault, raw_filedata, len(raw_filedata) + ) + result_array = CoreFoundation.CFArrayRef() + result = Security.SecItemImport( + filedata, # cert data + None, # Filename, leaving it out for now + None, # What the type of the file is, we don't care + None, # what's in the file, we don't care + 0, # import flags + None, # key params, can include passphrase in the future + keychain, # The keychain to insert into + ctypes.byref(result_array), # Results + ) + _assert_no_error(result) + + # A CFArray is not very useful to us as an intermediary + # representation, so we are going to extract the objects we want + # and then free the array. We don't need to keep hold of keys: the + # keychain already has them! + result_count = CoreFoundation.CFArrayGetCount(result_array) + for index in range(result_count): + item = CoreFoundation.CFArrayGetValueAtIndex(result_array, index) + item = ctypes.cast(item, CoreFoundation.CFTypeRef) + + if _is_cert(item): + CoreFoundation.CFRetain(item) + certificates.append(item) + elif _is_identity(item): + CoreFoundation.CFRetain(item) + identities.append(item) + finally: + if result_array: + CoreFoundation.CFRelease(result_array) + + CoreFoundation.CFRelease(filedata) + + return (identities, certificates) + + +def _load_client_cert_chain(keychain, *paths): + """ + Load certificates and maybe keys from a number of files. Has the end goal + of returning a CFArray containing one SecIdentityRef, and then zero or more + SecCertificateRef objects, suitable for use as a client certificate trust + chain. + """ + # Ok, the strategy. + # + # This relies on knowing that macOS will not give you a SecIdentityRef + # unless you have imported a key into a keychain. This is a somewhat + # artificial limitation of macOS (for example, it doesn't necessarily + # affect iOS), but there is nothing inside Security.framework that lets you + # get a SecIdentityRef without having a key in a keychain. + # + # So the policy here is we take all the files and iterate them in order. + # Each one will use SecItemImport to have one or more objects loaded from + # it. We will also point at a keychain that macOS can use to work with the + # private key. + # + # Once we have all the objects, we'll check what we actually have. If we + # already have a SecIdentityRef in hand, fab: we'll use that. Otherwise, + # we'll take the first certificate (which we assume to be our leaf) and + # ask the keychain to give us a SecIdentityRef with that cert's associated + # key. + # + # We'll then return a CFArray containing the trust chain: one + # SecIdentityRef and then zero-or-more SecCertificateRef objects. The + # responsibility for freeing this CFArray will be with the caller. This + # CFArray must remain alive for the entire connection, so in practice it + # will be stored with a single SSLSocket, along with the reference to the + # keychain. + certificates = [] + identities = [] + + # Filter out bad paths. + paths = (path for path in paths if path) + + try: + for file_path in paths: + new_identities, new_certs = _load_items_from_file(keychain, file_path) + identities.extend(new_identities) + certificates.extend(new_certs) + + # Ok, we have everything. The question is: do we have an identity? If + # not, we want to grab one from the first cert we have. + if not identities: + new_identity = Security.SecIdentityRef() + status = Security.SecIdentityCreateWithCertificate( + keychain, certificates[0], ctypes.byref(new_identity) + ) + _assert_no_error(status) + identities.append(new_identity) + + # We now want to release the original certificate, as we no longer + # need it. + CoreFoundation.CFRelease(certificates.pop(0)) + + # We now need to build a new CFArray that holds the trust chain. + trust_chain = CoreFoundation.CFArrayCreateMutable( + CoreFoundation.kCFAllocatorDefault, + 0, + ctypes.byref(CoreFoundation.kCFTypeArrayCallBacks), + ) + for item in itertools.chain(identities, certificates): + # ArrayAppendValue does a CFRetain on the item. That's fine, + # because the finally block will release our other refs to them. + CoreFoundation.CFArrayAppendValue(trust_chain, item) + + return trust_chain + finally: + for obj in itertools.chain(identities, certificates): + CoreFoundation.CFRelease(obj) + + +TLS_PROTOCOL_VERSIONS = { + "SSLv2": (0, 2), + "SSLv3": (3, 0), + "TLSv1": (3, 1), + "TLSv1.1": (3, 2), + "TLSv1.2": (3, 3), +} + + +def _build_tls_unknown_ca_alert(version): + """ + Builds a TLS alert record for an unknown CA. + """ + ver_maj, ver_min = TLS_PROTOCOL_VERSIONS[version] + severity_fatal = 0x02 + description_unknown_ca = 0x30 + msg = struct.pack(">BB", severity_fatal, description_unknown_ca) + msg_len = len(msg) + record_type_alert = 0x15 + record = struct.pack(">BBBH", record_type_alert, ver_maj, ver_min, msg_len) + msg + return record diff --git a/openpype/hosts/fusion/vendor/urllib3/contrib/appengine.py b/openpype/hosts/fusion/vendor/urllib3/contrib/appengine.py new file mode 100644 index 0000000000..f91bdd6e77 --- /dev/null +++ b/openpype/hosts/fusion/vendor/urllib3/contrib/appengine.py @@ -0,0 +1,314 @@ +""" +This module provides a pool manager that uses Google App Engine's +`URLFetch Service `_. + +Example usage:: + + from urllib3 import PoolManager + from urllib3.contrib.appengine import AppEngineManager, is_appengine_sandbox + + if is_appengine_sandbox(): + # AppEngineManager uses AppEngine's URLFetch API behind the scenes + http = AppEngineManager() + else: + # PoolManager uses a socket-level API behind the scenes + http = PoolManager() + + r = http.request('GET', 'https://google.com/') + +There are `limitations `_ to the URLFetch service and it may not be +the best choice for your application. There are three options for using +urllib3 on Google App Engine: + +1. You can use :class:`AppEngineManager` with URLFetch. URLFetch is + cost-effective in many circumstances as long as your usage is within the + limitations. +2. You can use a normal :class:`~urllib3.PoolManager` by enabling sockets. + Sockets also have `limitations and restrictions + `_ and have a lower free quota than URLFetch. + To use sockets, be sure to specify the following in your ``app.yaml``:: + + env_variables: + GAE_USE_SOCKETS_HTTPLIB : 'true' + +3. If you are using `App Engine Flexible +`_, you can use the standard +:class:`PoolManager` without any configuration or special environment variables. +""" + +from __future__ import absolute_import + +import io +import logging +import warnings + +from ..exceptions import ( + HTTPError, + HTTPWarning, + MaxRetryError, + ProtocolError, + SSLError, + TimeoutError, +) +from ..packages.six.moves.urllib.parse import urljoin +from ..request import RequestMethods +from ..response import HTTPResponse +from ..util.retry import Retry +from ..util.timeout import Timeout +from . import _appengine_environ + +try: + from google.appengine.api import urlfetch +except ImportError: + urlfetch = None + + +log = logging.getLogger(__name__) + + +class AppEnginePlatformWarning(HTTPWarning): + pass + + +class AppEnginePlatformError(HTTPError): + pass + + +class AppEngineManager(RequestMethods): + """ + Connection manager for Google App Engine sandbox applications. + + This manager uses the URLFetch service directly instead of using the + emulated httplib, and is subject to URLFetch limitations as described in + the App Engine documentation `here + `_. + + Notably it will raise an :class:`AppEnginePlatformError` if: + * URLFetch is not available. + * If you attempt to use this on App Engine Flexible, as full socket + support is available. + * If a request size is more than 10 megabytes. + * If a response size is more than 32 megabytes. + * If you use an unsupported request method such as OPTIONS. + + Beyond those cases, it will raise normal urllib3 errors. + """ + + def __init__( + self, + headers=None, + retries=None, + validate_certificate=True, + urlfetch_retries=True, + ): + if not urlfetch: + raise AppEnginePlatformError( + "URLFetch is not available in this environment." + ) + + warnings.warn( + "urllib3 is using URLFetch on Google App Engine sandbox instead " + "of sockets. To use sockets directly instead of URLFetch see " + "https://urllib3.readthedocs.io/en/1.26.x/reference/urllib3.contrib.html.", + AppEnginePlatformWarning, + ) + + RequestMethods.__init__(self, headers) + self.validate_certificate = validate_certificate + self.urlfetch_retries = urlfetch_retries + + self.retries = retries or Retry.DEFAULT + + def __enter__(self): + return self + + def __exit__(self, exc_type, exc_val, exc_tb): + # Return False to re-raise any potential exceptions + return False + + def urlopen( + self, + method, + url, + body=None, + headers=None, + retries=None, + redirect=True, + timeout=Timeout.DEFAULT_TIMEOUT, + **response_kw + ): + + retries = self._get_retries(retries, redirect) + + try: + follow_redirects = redirect and retries.redirect != 0 and retries.total + response = urlfetch.fetch( + url, + payload=body, + method=method, + headers=headers or {}, + allow_truncated=False, + follow_redirects=self.urlfetch_retries and follow_redirects, + deadline=self._get_absolute_timeout(timeout), + validate_certificate=self.validate_certificate, + ) + except urlfetch.DeadlineExceededError as e: + raise TimeoutError(self, e) + + except urlfetch.InvalidURLError as e: + if "too large" in str(e): + raise AppEnginePlatformError( + "URLFetch request too large, URLFetch only " + "supports requests up to 10mb in size.", + e, + ) + raise ProtocolError(e) + + except urlfetch.DownloadError as e: + if "Too many redirects" in str(e): + raise MaxRetryError(self, url, reason=e) + raise ProtocolError(e) + + except urlfetch.ResponseTooLargeError as e: + raise AppEnginePlatformError( + "URLFetch response too large, URLFetch only supports" + "responses up to 32mb in size.", + e, + ) + + except urlfetch.SSLCertificateError as e: + raise SSLError(e) + + except urlfetch.InvalidMethodError as e: + raise AppEnginePlatformError( + "URLFetch does not support method: %s" % method, e + ) + + http_response = self._urlfetch_response_to_http_response( + response, retries=retries, **response_kw + ) + + # Handle redirect? + redirect_location = redirect and http_response.get_redirect_location() + if redirect_location: + # Check for redirect response + if self.urlfetch_retries and retries.raise_on_redirect: + raise MaxRetryError(self, url, "too many redirects") + else: + if http_response.status == 303: + method = "GET" + + try: + retries = retries.increment( + method, url, response=http_response, _pool=self + ) + except MaxRetryError: + if retries.raise_on_redirect: + raise MaxRetryError(self, url, "too many redirects") + return http_response + + retries.sleep_for_retry(http_response) + log.debug("Redirecting %s -> %s", url, redirect_location) + redirect_url = urljoin(url, redirect_location) + return self.urlopen( + method, + redirect_url, + body, + headers, + retries=retries, + redirect=redirect, + timeout=timeout, + **response_kw + ) + + # Check if we should retry the HTTP response. + has_retry_after = bool(http_response.getheader("Retry-After")) + if retries.is_retry(method, http_response.status, has_retry_after): + retries = retries.increment(method, url, response=http_response, _pool=self) + log.debug("Retry: %s", url) + retries.sleep(http_response) + return self.urlopen( + method, + url, + body=body, + headers=headers, + retries=retries, + redirect=redirect, + timeout=timeout, + **response_kw + ) + + return http_response + + def _urlfetch_response_to_http_response(self, urlfetch_resp, **response_kw): + + if is_prod_appengine(): + # Production GAE handles deflate encoding automatically, but does + # not remove the encoding header. + content_encoding = urlfetch_resp.headers.get("content-encoding") + + if content_encoding == "deflate": + del urlfetch_resp.headers["content-encoding"] + + transfer_encoding = urlfetch_resp.headers.get("transfer-encoding") + # We have a full response's content, + # so let's make sure we don't report ourselves as chunked data. + if transfer_encoding == "chunked": + encodings = transfer_encoding.split(",") + encodings.remove("chunked") + urlfetch_resp.headers["transfer-encoding"] = ",".join(encodings) + + original_response = HTTPResponse( + # In order for decoding to work, we must present the content as + # a file-like object. + body=io.BytesIO(urlfetch_resp.content), + msg=urlfetch_resp.header_msg, + headers=urlfetch_resp.headers, + status=urlfetch_resp.status_code, + **response_kw + ) + + return HTTPResponse( + body=io.BytesIO(urlfetch_resp.content), + headers=urlfetch_resp.headers, + status=urlfetch_resp.status_code, + original_response=original_response, + **response_kw + ) + + def _get_absolute_timeout(self, timeout): + if timeout is Timeout.DEFAULT_TIMEOUT: + return None # Defer to URLFetch's default. + if isinstance(timeout, Timeout): + if timeout._read is not None or timeout._connect is not None: + warnings.warn( + "URLFetch does not support granular timeout settings, " + "reverting to total or default URLFetch timeout.", + AppEnginePlatformWarning, + ) + return timeout.total + return timeout + + def _get_retries(self, retries, redirect): + if not isinstance(retries, Retry): + retries = Retry.from_int(retries, redirect=redirect, default=self.retries) + + if retries.connect or retries.read or retries.redirect: + warnings.warn( + "URLFetch only supports total retries and does not " + "recognize connect, read, or redirect retry parameters.", + AppEnginePlatformWarning, + ) + + return retries + + +# Alias methods from _appengine_environ to maintain public API interface. + +is_appengine = _appengine_environ.is_appengine +is_appengine_sandbox = _appengine_environ.is_appengine_sandbox +is_local_appengine = _appengine_environ.is_local_appengine +is_prod_appengine = _appengine_environ.is_prod_appengine +is_prod_appengine_mvms = _appengine_environ.is_prod_appengine_mvms diff --git a/openpype/hosts/fusion/vendor/urllib3/contrib/ntlmpool.py b/openpype/hosts/fusion/vendor/urllib3/contrib/ntlmpool.py new file mode 100644 index 0000000000..41a8fd174c --- /dev/null +++ b/openpype/hosts/fusion/vendor/urllib3/contrib/ntlmpool.py @@ -0,0 +1,130 @@ +""" +NTLM authenticating pool, contributed by erikcederstran + +Issue #10, see: http://code.google.com/p/urllib3/issues/detail?id=10 +""" +from __future__ import absolute_import + +import warnings +from logging import getLogger + +from ntlm import ntlm + +from .. import HTTPSConnectionPool +from ..packages.six.moves.http_client import HTTPSConnection + +warnings.warn( + "The 'urllib3.contrib.ntlmpool' module is deprecated and will be removed " + "in urllib3 v2.0 release, urllib3 is not able to support it properly due " + "to reasons listed in issue: https://github.com/urllib3/urllib3/issues/2282. " + "If you are a user of this module please comment in the mentioned issue.", + DeprecationWarning, +) + +log = getLogger(__name__) + + +class NTLMConnectionPool(HTTPSConnectionPool): + """ + Implements an NTLM authentication version of an urllib3 connection pool + """ + + scheme = "https" + + def __init__(self, user, pw, authurl, *args, **kwargs): + """ + authurl is a random URL on the server that is protected by NTLM. + user is the Windows user, probably in the DOMAIN\\username format. + pw is the password for the user. + """ + super(NTLMConnectionPool, self).__init__(*args, **kwargs) + self.authurl = authurl + self.rawuser = user + user_parts = user.split("\\", 1) + self.domain = user_parts[0].upper() + self.user = user_parts[1] + self.pw = pw + + def _new_conn(self): + # Performs the NTLM handshake that secures the connection. The socket + # must be kept open while requests are performed. + self.num_connections += 1 + log.debug( + "Starting NTLM HTTPS connection no. %d: https://%s%s", + self.num_connections, + self.host, + self.authurl, + ) + + headers = {"Connection": "Keep-Alive"} + req_header = "Authorization" + resp_header = "www-authenticate" + + conn = HTTPSConnection(host=self.host, port=self.port) + + # Send negotiation message + headers[req_header] = "NTLM %s" % ntlm.create_NTLM_NEGOTIATE_MESSAGE( + self.rawuser + ) + log.debug("Request headers: %s", headers) + conn.request("GET", self.authurl, None, headers) + res = conn.getresponse() + reshdr = dict(res.getheaders()) + log.debug("Response status: %s %s", res.status, res.reason) + log.debug("Response headers: %s", reshdr) + log.debug("Response data: %s [...]", res.read(100)) + + # Remove the reference to the socket, so that it can not be closed by + # the response object (we want to keep the socket open) + res.fp = None + + # Server should respond with a challenge message + auth_header_values = reshdr[resp_header].split(", ") + auth_header_value = None + for s in auth_header_values: + if s[:5] == "NTLM ": + auth_header_value = s[5:] + if auth_header_value is None: + raise Exception( + "Unexpected %s response header: %s" % (resp_header, reshdr[resp_header]) + ) + + # Send authentication message + ServerChallenge, NegotiateFlags = ntlm.parse_NTLM_CHALLENGE_MESSAGE( + auth_header_value + ) + auth_msg = ntlm.create_NTLM_AUTHENTICATE_MESSAGE( + ServerChallenge, self.user, self.domain, self.pw, NegotiateFlags + ) + headers[req_header] = "NTLM %s" % auth_msg + log.debug("Request headers: %s", headers) + conn.request("GET", self.authurl, None, headers) + res = conn.getresponse() + log.debug("Response status: %s %s", res.status, res.reason) + log.debug("Response headers: %s", dict(res.getheaders())) + log.debug("Response data: %s [...]", res.read()[:100]) + if res.status != 200: + if res.status == 401: + raise Exception("Server rejected request: wrong username or password") + raise Exception("Wrong server response: %s %s" % (res.status, res.reason)) + + res.fp = None + log.debug("Connection established") + return conn + + def urlopen( + self, + method, + url, + body=None, + headers=None, + retries=3, + redirect=True, + assert_same_host=True, + ): + if headers is None: + headers = {} + headers["Connection"] = "Keep-Alive" + return super(NTLMConnectionPool, self).urlopen( + method, url, body, headers, retries, redirect, assert_same_host + ) diff --git a/openpype/hosts/fusion/vendor/urllib3/contrib/pyopenssl.py b/openpype/hosts/fusion/vendor/urllib3/contrib/pyopenssl.py new file mode 100644 index 0000000000..def83afdb2 --- /dev/null +++ b/openpype/hosts/fusion/vendor/urllib3/contrib/pyopenssl.py @@ -0,0 +1,511 @@ +""" +TLS with SNI_-support for Python 2. Follow these instructions if you would +like to verify TLS certificates in Python 2. Note, the default libraries do +*not* do certificate checking; you need to do additional work to validate +certificates yourself. + +This needs the following packages installed: + +* `pyOpenSSL`_ (tested with 16.0.0) +* `cryptography`_ (minimum 1.3.4, from pyopenssl) +* `idna`_ (minimum 2.0, from cryptography) + +However, pyopenssl depends on cryptography, which depends on idna, so while we +use all three directly here we end up having relatively few packages required. + +You can install them with the following command: + +.. code-block:: bash + + $ python -m pip install pyopenssl cryptography idna + +To activate certificate checking, call +:func:`~urllib3.contrib.pyopenssl.inject_into_urllib3` from your Python code +before you begin making HTTP requests. This can be done in a ``sitecustomize`` +module, or at any other time before your application begins using ``urllib3``, +like this: + +.. code-block:: python + + try: + import urllib3.contrib.pyopenssl + urllib3.contrib.pyopenssl.inject_into_urllib3() + except ImportError: + pass + +Now you can use :mod:`urllib3` as you normally would, and it will support SNI +when the required modules are installed. + +Activating this module also has the positive side effect of disabling SSL/TLS +compression in Python 2 (see `CRIME attack`_). + +.. _sni: https://en.wikipedia.org/wiki/Server_Name_Indication +.. _crime attack: https://en.wikipedia.org/wiki/CRIME_(security_exploit) +.. _pyopenssl: https://www.pyopenssl.org +.. _cryptography: https://cryptography.io +.. _idna: https://github.com/kjd/idna +""" +from __future__ import absolute_import + +import OpenSSL.SSL +from cryptography import x509 +from cryptography.hazmat.backends.openssl import backend as openssl_backend +from cryptography.hazmat.backends.openssl.x509 import _Certificate + +try: + from cryptography.x509 import UnsupportedExtension +except ImportError: + # UnsupportedExtension is gone in cryptography >= 2.1.0 + class UnsupportedExtension(Exception): + pass + + +from io import BytesIO +from socket import error as SocketError +from socket import timeout + +try: # Platform-specific: Python 2 + from socket import _fileobject +except ImportError: # Platform-specific: Python 3 + _fileobject = None + from ..packages.backports.makefile import backport_makefile + +import logging +import ssl +import sys + +from .. import util +from ..packages import six +from ..util.ssl_ import PROTOCOL_TLS_CLIENT + +__all__ = ["inject_into_urllib3", "extract_from_urllib3"] + +# SNI always works. +HAS_SNI = True + +# Map from urllib3 to PyOpenSSL compatible parameter-values. +_openssl_versions = { + util.PROTOCOL_TLS: OpenSSL.SSL.SSLv23_METHOD, + PROTOCOL_TLS_CLIENT: OpenSSL.SSL.SSLv23_METHOD, + ssl.PROTOCOL_TLSv1: OpenSSL.SSL.TLSv1_METHOD, +} + +if hasattr(ssl, "PROTOCOL_SSLv3") and hasattr(OpenSSL.SSL, "SSLv3_METHOD"): + _openssl_versions[ssl.PROTOCOL_SSLv3] = OpenSSL.SSL.SSLv3_METHOD + +if hasattr(ssl, "PROTOCOL_TLSv1_1") and hasattr(OpenSSL.SSL, "TLSv1_1_METHOD"): + _openssl_versions[ssl.PROTOCOL_TLSv1_1] = OpenSSL.SSL.TLSv1_1_METHOD + +if hasattr(ssl, "PROTOCOL_TLSv1_2") and hasattr(OpenSSL.SSL, "TLSv1_2_METHOD"): + _openssl_versions[ssl.PROTOCOL_TLSv1_2] = OpenSSL.SSL.TLSv1_2_METHOD + + +_stdlib_to_openssl_verify = { + ssl.CERT_NONE: OpenSSL.SSL.VERIFY_NONE, + ssl.CERT_OPTIONAL: OpenSSL.SSL.VERIFY_PEER, + ssl.CERT_REQUIRED: OpenSSL.SSL.VERIFY_PEER + + OpenSSL.SSL.VERIFY_FAIL_IF_NO_PEER_CERT, +} +_openssl_to_stdlib_verify = dict((v, k) for k, v in _stdlib_to_openssl_verify.items()) + +# OpenSSL will only write 16K at a time +SSL_WRITE_BLOCKSIZE = 16384 + +orig_util_HAS_SNI = util.HAS_SNI +orig_util_SSLContext = util.ssl_.SSLContext + + +log = logging.getLogger(__name__) + + +def inject_into_urllib3(): + "Monkey-patch urllib3 with PyOpenSSL-backed SSL-support." + + _validate_dependencies_met() + + util.SSLContext = PyOpenSSLContext + util.ssl_.SSLContext = PyOpenSSLContext + util.HAS_SNI = HAS_SNI + util.ssl_.HAS_SNI = HAS_SNI + util.IS_PYOPENSSL = True + util.ssl_.IS_PYOPENSSL = True + + +def extract_from_urllib3(): + "Undo monkey-patching by :func:`inject_into_urllib3`." + + util.SSLContext = orig_util_SSLContext + util.ssl_.SSLContext = orig_util_SSLContext + util.HAS_SNI = orig_util_HAS_SNI + util.ssl_.HAS_SNI = orig_util_HAS_SNI + util.IS_PYOPENSSL = False + util.ssl_.IS_PYOPENSSL = False + + +def _validate_dependencies_met(): + """ + Verifies that PyOpenSSL's package-level dependencies have been met. + Throws `ImportError` if they are not met. + """ + # Method added in `cryptography==1.1`; not available in older versions + from cryptography.x509.extensions import Extensions + + if getattr(Extensions, "get_extension_for_class", None) is None: + raise ImportError( + "'cryptography' module missing required functionality. " + "Try upgrading to v1.3.4 or newer." + ) + + # pyOpenSSL 0.14 and above use cryptography for OpenSSL bindings. The _x509 + # attribute is only present on those versions. + from OpenSSL.crypto import X509 + + x509 = X509() + if getattr(x509, "_x509", None) is None: + raise ImportError( + "'pyOpenSSL' module missing required functionality. " + "Try upgrading to v0.14 or newer." + ) + + +def _dnsname_to_stdlib(name): + """ + Converts a dNSName SubjectAlternativeName field to the form used by the + standard library on the given Python version. + + Cryptography produces a dNSName as a unicode string that was idna-decoded + from ASCII bytes. We need to idna-encode that string to get it back, and + then on Python 3 we also need to convert to unicode via UTF-8 (the stdlib + uses PyUnicode_FromStringAndSize on it, which decodes via UTF-8). + + If the name cannot be idna-encoded then we return None signalling that + the name given should be skipped. + """ + + def idna_encode(name): + """ + Borrowed wholesale from the Python Cryptography Project. It turns out + that we can't just safely call `idna.encode`: it can explode for + wildcard names. This avoids that problem. + """ + import idna + + try: + for prefix in [u"*.", u"."]: + if name.startswith(prefix): + name = name[len(prefix) :] + return prefix.encode("ascii") + idna.encode(name) + return idna.encode(name) + except idna.core.IDNAError: + return None + + # Don't send IPv6 addresses through the IDNA encoder. + if ":" in name: + return name + + name = idna_encode(name) + if name is None: + return None + elif sys.version_info >= (3, 0): + name = name.decode("utf-8") + return name + + +def get_subj_alt_name(peer_cert): + """ + Given an PyOpenSSL certificate, provides all the subject alternative names. + """ + # Pass the cert to cryptography, which has much better APIs for this. + if hasattr(peer_cert, "to_cryptography"): + cert = peer_cert.to_cryptography() + else: + # This is technically using private APIs, but should work across all + # relevant versions before PyOpenSSL got a proper API for this. + cert = _Certificate(openssl_backend, peer_cert._x509) + + # We want to find the SAN extension. Ask Cryptography to locate it (it's + # faster than looping in Python) + try: + ext = cert.extensions.get_extension_for_class(x509.SubjectAlternativeName).value + except x509.ExtensionNotFound: + # No such extension, return the empty list. + return [] + except ( + x509.DuplicateExtension, + UnsupportedExtension, + x509.UnsupportedGeneralNameType, + UnicodeError, + ) as e: + # A problem has been found with the quality of the certificate. Assume + # no SAN field is present. + log.warning( + "A problem was encountered with the certificate that prevented " + "urllib3 from finding the SubjectAlternativeName field. This can " + "affect certificate validation. The error was %s", + e, + ) + return [] + + # We want to return dNSName and iPAddress fields. We need to cast the IPs + # back to strings because the match_hostname function wants them as + # strings. + # Sadly the DNS names need to be idna encoded and then, on Python 3, UTF-8 + # decoded. This is pretty frustrating, but that's what the standard library + # does with certificates, and so we need to attempt to do the same. + # We also want to skip over names which cannot be idna encoded. + names = [ + ("DNS", name) + for name in map(_dnsname_to_stdlib, ext.get_values_for_type(x509.DNSName)) + if name is not None + ] + names.extend( + ("IP Address", str(name)) for name in ext.get_values_for_type(x509.IPAddress) + ) + + return names + + +class WrappedSocket(object): + """API-compatibility wrapper for Python OpenSSL's Connection-class. + + Note: _makefile_refs, _drop() and _reuse() are needed for the garbage + collector of pypy. + """ + + def __init__(self, connection, socket, suppress_ragged_eofs=True): + self.connection = connection + self.socket = socket + self.suppress_ragged_eofs = suppress_ragged_eofs + self._makefile_refs = 0 + self._closed = False + + def fileno(self): + return self.socket.fileno() + + # Copy-pasted from Python 3.5 source code + def _decref_socketios(self): + if self._makefile_refs > 0: + self._makefile_refs -= 1 + if self._closed: + self.close() + + def recv(self, *args, **kwargs): + try: + data = self.connection.recv(*args, **kwargs) + except OpenSSL.SSL.SysCallError as e: + if self.suppress_ragged_eofs and e.args == (-1, "Unexpected EOF"): + return b"" + else: + raise SocketError(str(e)) + except OpenSSL.SSL.ZeroReturnError: + if self.connection.get_shutdown() == OpenSSL.SSL.RECEIVED_SHUTDOWN: + return b"" + else: + raise + except OpenSSL.SSL.WantReadError: + if not util.wait_for_read(self.socket, self.socket.gettimeout()): + raise timeout("The read operation timed out") + else: + return self.recv(*args, **kwargs) + + # TLS 1.3 post-handshake authentication + except OpenSSL.SSL.Error as e: + raise ssl.SSLError("read error: %r" % e) + else: + return data + + def recv_into(self, *args, **kwargs): + try: + return self.connection.recv_into(*args, **kwargs) + except OpenSSL.SSL.SysCallError as e: + if self.suppress_ragged_eofs and e.args == (-1, "Unexpected EOF"): + return 0 + else: + raise SocketError(str(e)) + except OpenSSL.SSL.ZeroReturnError: + if self.connection.get_shutdown() == OpenSSL.SSL.RECEIVED_SHUTDOWN: + return 0 + else: + raise + except OpenSSL.SSL.WantReadError: + if not util.wait_for_read(self.socket, self.socket.gettimeout()): + raise timeout("The read operation timed out") + else: + return self.recv_into(*args, **kwargs) + + # TLS 1.3 post-handshake authentication + except OpenSSL.SSL.Error as e: + raise ssl.SSLError("read error: %r" % e) + + def settimeout(self, timeout): + return self.socket.settimeout(timeout) + + def _send_until_done(self, data): + while True: + try: + return self.connection.send(data) + except OpenSSL.SSL.WantWriteError: + if not util.wait_for_write(self.socket, self.socket.gettimeout()): + raise timeout() + continue + except OpenSSL.SSL.SysCallError as e: + raise SocketError(str(e)) + + def sendall(self, data): + total_sent = 0 + while total_sent < len(data): + sent = self._send_until_done( + data[total_sent : total_sent + SSL_WRITE_BLOCKSIZE] + ) + total_sent += sent + + def shutdown(self): + # FIXME rethrow compatible exceptions should we ever use this + self.connection.shutdown() + + def close(self): + if self._makefile_refs < 1: + try: + self._closed = True + return self.connection.close() + except OpenSSL.SSL.Error: + return + else: + self._makefile_refs -= 1 + + def getpeercert(self, binary_form=False): + x509 = self.connection.get_peer_certificate() + + if not x509: + return x509 + + if binary_form: + return OpenSSL.crypto.dump_certificate(OpenSSL.crypto.FILETYPE_ASN1, x509) + + return { + "subject": ((("commonName", x509.get_subject().CN),),), + "subjectAltName": get_subj_alt_name(x509), + } + + def version(self): + return self.connection.get_protocol_version_name() + + def _reuse(self): + self._makefile_refs += 1 + + def _drop(self): + if self._makefile_refs < 1: + self.close() + else: + self._makefile_refs -= 1 + + +if _fileobject: # Platform-specific: Python 2 + + def makefile(self, mode, bufsize=-1): + self._makefile_refs += 1 + return _fileobject(self, mode, bufsize, close=True) + + +else: # Platform-specific: Python 3 + makefile = backport_makefile + +WrappedSocket.makefile = makefile + + +class PyOpenSSLContext(object): + """ + I am a wrapper class for the PyOpenSSL ``Context`` object. I am responsible + for translating the interface of the standard library ``SSLContext`` object + to calls into PyOpenSSL. + """ + + def __init__(self, protocol): + self.protocol = _openssl_versions[protocol] + self._ctx = OpenSSL.SSL.Context(self.protocol) + self._options = 0 + self.check_hostname = False + + @property + def options(self): + return self._options + + @options.setter + def options(self, value): + self._options = value + self._ctx.set_options(value) + + @property + def verify_mode(self): + return _openssl_to_stdlib_verify[self._ctx.get_verify_mode()] + + @verify_mode.setter + def verify_mode(self, value): + self._ctx.set_verify(_stdlib_to_openssl_verify[value], _verify_callback) + + def set_default_verify_paths(self): + self._ctx.set_default_verify_paths() + + def set_ciphers(self, ciphers): + if isinstance(ciphers, six.text_type): + ciphers = ciphers.encode("utf-8") + self._ctx.set_cipher_list(ciphers) + + def load_verify_locations(self, cafile=None, capath=None, cadata=None): + if cafile is not None: + cafile = cafile.encode("utf-8") + if capath is not None: + capath = capath.encode("utf-8") + try: + self._ctx.load_verify_locations(cafile, capath) + if cadata is not None: + self._ctx.load_verify_locations(BytesIO(cadata)) + except OpenSSL.SSL.Error as e: + raise ssl.SSLError("unable to load trusted certificates: %r" % e) + + def load_cert_chain(self, certfile, keyfile=None, password=None): + self._ctx.use_certificate_chain_file(certfile) + if password is not None: + if not isinstance(password, six.binary_type): + password = password.encode("utf-8") + self._ctx.set_passwd_cb(lambda *_: password) + self._ctx.use_privatekey_file(keyfile or certfile) + + def set_alpn_protocols(self, protocols): + protocols = [six.ensure_binary(p) for p in protocols] + return self._ctx.set_alpn_protos(protocols) + + def wrap_socket( + self, + sock, + server_side=False, + do_handshake_on_connect=True, + suppress_ragged_eofs=True, + server_hostname=None, + ): + cnx = OpenSSL.SSL.Connection(self._ctx, sock) + + if isinstance(server_hostname, six.text_type): # Platform-specific: Python 3 + server_hostname = server_hostname.encode("utf-8") + + if server_hostname is not None: + cnx.set_tlsext_host_name(server_hostname) + + cnx.set_connect_state() + + while True: + try: + cnx.do_handshake() + except OpenSSL.SSL.WantReadError: + if not util.wait_for_read(sock, sock.gettimeout()): + raise timeout("select timed out") + continue + except OpenSSL.SSL.Error as e: + raise ssl.SSLError("bad handshake: %r" % e) + break + + return WrappedSocket(cnx, sock) + + +def _verify_callback(cnx, x509, err_no, err_depth, return_code): + return err_no == 0 diff --git a/openpype/hosts/fusion/vendor/urllib3/contrib/securetransport.py b/openpype/hosts/fusion/vendor/urllib3/contrib/securetransport.py new file mode 100644 index 0000000000..554c015fed --- /dev/null +++ b/openpype/hosts/fusion/vendor/urllib3/contrib/securetransport.py @@ -0,0 +1,922 @@ +""" +SecureTranport support for urllib3 via ctypes. + +This makes platform-native TLS available to urllib3 users on macOS without the +use of a compiler. This is an important feature because the Python Package +Index is moving to become a TLSv1.2-or-higher server, and the default OpenSSL +that ships with macOS is not capable of doing TLSv1.2. The only way to resolve +this is to give macOS users an alternative solution to the problem, and that +solution is to use SecureTransport. + +We use ctypes here because this solution must not require a compiler. That's +because pip is not allowed to require a compiler either. + +This is not intended to be a seriously long-term solution to this problem. +The hope is that PEP 543 will eventually solve this issue for us, at which +point we can retire this contrib module. But in the short term, we need to +solve the impending tire fire that is Python on Mac without this kind of +contrib module. So...here we are. + +To use this module, simply import and inject it:: + + import urllib3.contrib.securetransport + urllib3.contrib.securetransport.inject_into_urllib3() + +Happy TLSing! + +This code is a bastardised version of the code found in Will Bond's oscrypto +library. An enormous debt is owed to him for blazing this trail for us. For +that reason, this code should be considered to be covered both by urllib3's +license and by oscrypto's: + +.. code-block:: + + Copyright (c) 2015-2016 Will Bond + + Permission is hereby granted, free of charge, to any person obtaining a + copy of this software and associated documentation files (the "Software"), + to deal in the Software without restriction, including without limitation + the rights to use, copy, modify, merge, publish, distribute, sublicense, + and/or sell copies of the Software, and to permit persons to whom the + Software is furnished to do so, subject to the following conditions: + + The above copyright notice and this permission notice shall be included in + all copies or substantial portions of the Software. + + THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR + IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, + FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE + AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER + LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING + FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER + DEALINGS IN THE SOFTWARE. +""" +from __future__ import absolute_import + +import contextlib +import ctypes +import errno +import os.path +import shutil +import socket +import ssl +import struct +import threading +import weakref + +import six + +from .. import util +from ..util.ssl_ import PROTOCOL_TLS_CLIENT +from ._securetransport.bindings import CoreFoundation, Security, SecurityConst +from ._securetransport.low_level import ( + _assert_no_error, + _build_tls_unknown_ca_alert, + _cert_array_from_pem, + _create_cfstring_array, + _load_client_cert_chain, + _temporary_keychain, +) + +try: # Platform-specific: Python 2 + from socket import _fileobject +except ImportError: # Platform-specific: Python 3 + _fileobject = None + from ..packages.backports.makefile import backport_makefile + +__all__ = ["inject_into_urllib3", "extract_from_urllib3"] + +# SNI always works +HAS_SNI = True + +orig_util_HAS_SNI = util.HAS_SNI +orig_util_SSLContext = util.ssl_.SSLContext + +# This dictionary is used by the read callback to obtain a handle to the +# calling wrapped socket. This is a pretty silly approach, but for now it'll +# do. I feel like I should be able to smuggle a handle to the wrapped socket +# directly in the SSLConnectionRef, but for now this approach will work I +# guess. +# +# We need to lock around this structure for inserts, but we don't do it for +# reads/writes in the callbacks. The reasoning here goes as follows: +# +# 1. It is not possible to call into the callbacks before the dictionary is +# populated, so once in the callback the id must be in the dictionary. +# 2. The callbacks don't mutate the dictionary, they only read from it, and +# so cannot conflict with any of the insertions. +# +# This is good: if we had to lock in the callbacks we'd drastically slow down +# the performance of this code. +_connection_refs = weakref.WeakValueDictionary() +_connection_ref_lock = threading.Lock() + +# Limit writes to 16kB. This is OpenSSL's limit, but we'll cargo-cult it over +# for no better reason than we need *a* limit, and this one is right there. +SSL_WRITE_BLOCKSIZE = 16384 + +# This is our equivalent of util.ssl_.DEFAULT_CIPHERS, but expanded out to +# individual cipher suites. We need to do this because this is how +# SecureTransport wants them. +CIPHER_SUITES = [ + SecurityConst.TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384, + SecurityConst.TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256, + SecurityConst.TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384, + SecurityConst.TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256, + SecurityConst.TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305_SHA256, + SecurityConst.TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305_SHA256, + SecurityConst.TLS_DHE_RSA_WITH_AES_256_GCM_SHA384, + SecurityConst.TLS_DHE_RSA_WITH_AES_128_GCM_SHA256, + SecurityConst.TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA384, + SecurityConst.TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA, + SecurityConst.TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA256, + SecurityConst.TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA, + SecurityConst.TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384, + SecurityConst.TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA, + SecurityConst.TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256, + SecurityConst.TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA, + SecurityConst.TLS_DHE_RSA_WITH_AES_256_CBC_SHA256, + SecurityConst.TLS_DHE_RSA_WITH_AES_256_CBC_SHA, + SecurityConst.TLS_DHE_RSA_WITH_AES_128_CBC_SHA256, + SecurityConst.TLS_DHE_RSA_WITH_AES_128_CBC_SHA, + SecurityConst.TLS_AES_256_GCM_SHA384, + SecurityConst.TLS_AES_128_GCM_SHA256, + SecurityConst.TLS_RSA_WITH_AES_256_GCM_SHA384, + SecurityConst.TLS_RSA_WITH_AES_128_GCM_SHA256, + SecurityConst.TLS_AES_128_CCM_8_SHA256, + SecurityConst.TLS_AES_128_CCM_SHA256, + SecurityConst.TLS_RSA_WITH_AES_256_CBC_SHA256, + SecurityConst.TLS_RSA_WITH_AES_128_CBC_SHA256, + SecurityConst.TLS_RSA_WITH_AES_256_CBC_SHA, + SecurityConst.TLS_RSA_WITH_AES_128_CBC_SHA, +] + +# Basically this is simple: for PROTOCOL_SSLv23 we turn it into a low of +# TLSv1 and a high of TLSv1.2. For everything else, we pin to that version. +# TLSv1 to 1.2 are supported on macOS 10.8+ +_protocol_to_min_max = { + util.PROTOCOL_TLS: (SecurityConst.kTLSProtocol1, SecurityConst.kTLSProtocol12), + PROTOCOL_TLS_CLIENT: (SecurityConst.kTLSProtocol1, SecurityConst.kTLSProtocol12), +} + +if hasattr(ssl, "PROTOCOL_SSLv2"): + _protocol_to_min_max[ssl.PROTOCOL_SSLv2] = ( + SecurityConst.kSSLProtocol2, + SecurityConst.kSSLProtocol2, + ) +if hasattr(ssl, "PROTOCOL_SSLv3"): + _protocol_to_min_max[ssl.PROTOCOL_SSLv3] = ( + SecurityConst.kSSLProtocol3, + SecurityConst.kSSLProtocol3, + ) +if hasattr(ssl, "PROTOCOL_TLSv1"): + _protocol_to_min_max[ssl.PROTOCOL_TLSv1] = ( + SecurityConst.kTLSProtocol1, + SecurityConst.kTLSProtocol1, + ) +if hasattr(ssl, "PROTOCOL_TLSv1_1"): + _protocol_to_min_max[ssl.PROTOCOL_TLSv1_1] = ( + SecurityConst.kTLSProtocol11, + SecurityConst.kTLSProtocol11, + ) +if hasattr(ssl, "PROTOCOL_TLSv1_2"): + _protocol_to_min_max[ssl.PROTOCOL_TLSv1_2] = ( + SecurityConst.kTLSProtocol12, + SecurityConst.kTLSProtocol12, + ) + + +def inject_into_urllib3(): + """ + Monkey-patch urllib3 with SecureTransport-backed SSL-support. + """ + util.SSLContext = SecureTransportContext + util.ssl_.SSLContext = SecureTransportContext + util.HAS_SNI = HAS_SNI + util.ssl_.HAS_SNI = HAS_SNI + util.IS_SECURETRANSPORT = True + util.ssl_.IS_SECURETRANSPORT = True + + +def extract_from_urllib3(): + """ + Undo monkey-patching by :func:`inject_into_urllib3`. + """ + util.SSLContext = orig_util_SSLContext + util.ssl_.SSLContext = orig_util_SSLContext + util.HAS_SNI = orig_util_HAS_SNI + util.ssl_.HAS_SNI = orig_util_HAS_SNI + util.IS_SECURETRANSPORT = False + util.ssl_.IS_SECURETRANSPORT = False + + +def _read_callback(connection_id, data_buffer, data_length_pointer): + """ + SecureTransport read callback. This is called by ST to request that data + be returned from the socket. + """ + wrapped_socket = None + try: + wrapped_socket = _connection_refs.get(connection_id) + if wrapped_socket is None: + return SecurityConst.errSSLInternal + base_socket = wrapped_socket.socket + + requested_length = data_length_pointer[0] + + timeout = wrapped_socket.gettimeout() + error = None + read_count = 0 + + try: + while read_count < requested_length: + if timeout is None or timeout >= 0: + if not util.wait_for_read(base_socket, timeout): + raise socket.error(errno.EAGAIN, "timed out") + + remaining = requested_length - read_count + buffer = (ctypes.c_char * remaining).from_address( + data_buffer + read_count + ) + chunk_size = base_socket.recv_into(buffer, remaining) + read_count += chunk_size + if not chunk_size: + if not read_count: + return SecurityConst.errSSLClosedGraceful + break + except (socket.error) as e: + error = e.errno + + if error is not None and error != errno.EAGAIN: + data_length_pointer[0] = read_count + if error == errno.ECONNRESET or error == errno.EPIPE: + return SecurityConst.errSSLClosedAbort + raise + + data_length_pointer[0] = read_count + + if read_count != requested_length: + return SecurityConst.errSSLWouldBlock + + return 0 + except Exception as e: + if wrapped_socket is not None: + wrapped_socket._exception = e + return SecurityConst.errSSLInternal + + +def _write_callback(connection_id, data_buffer, data_length_pointer): + """ + SecureTransport write callback. This is called by ST to request that data + actually be sent on the network. + """ + wrapped_socket = None + try: + wrapped_socket = _connection_refs.get(connection_id) + if wrapped_socket is None: + return SecurityConst.errSSLInternal + base_socket = wrapped_socket.socket + + bytes_to_write = data_length_pointer[0] + data = ctypes.string_at(data_buffer, bytes_to_write) + + timeout = wrapped_socket.gettimeout() + error = None + sent = 0 + + try: + while sent < bytes_to_write: + if timeout is None or timeout >= 0: + if not util.wait_for_write(base_socket, timeout): + raise socket.error(errno.EAGAIN, "timed out") + chunk_sent = base_socket.send(data) + sent += chunk_sent + + # This has some needless copying here, but I'm not sure there's + # much value in optimising this data path. + data = data[chunk_sent:] + except (socket.error) as e: + error = e.errno + + if error is not None and error != errno.EAGAIN: + data_length_pointer[0] = sent + if error == errno.ECONNRESET or error == errno.EPIPE: + return SecurityConst.errSSLClosedAbort + raise + + data_length_pointer[0] = sent + + if sent != bytes_to_write: + return SecurityConst.errSSLWouldBlock + + return 0 + except Exception as e: + if wrapped_socket is not None: + wrapped_socket._exception = e + return SecurityConst.errSSLInternal + + +# We need to keep these two objects references alive: if they get GC'd while +# in use then SecureTransport could attempt to call a function that is in freed +# memory. That would be...uh...bad. Yeah, that's the word. Bad. +_read_callback_pointer = Security.SSLReadFunc(_read_callback) +_write_callback_pointer = Security.SSLWriteFunc(_write_callback) + + +class WrappedSocket(object): + """ + API-compatibility wrapper for Python's OpenSSL wrapped socket object. + + Note: _makefile_refs, _drop(), and _reuse() are needed for the garbage + collector of PyPy. + """ + + def __init__(self, socket): + self.socket = socket + self.context = None + self._makefile_refs = 0 + self._closed = False + self._exception = None + self._keychain = None + self._keychain_dir = None + self._client_cert_chain = None + + # We save off the previously-configured timeout and then set it to + # zero. This is done because we use select and friends to handle the + # timeouts, but if we leave the timeout set on the lower socket then + # Python will "kindly" call select on that socket again for us. Avoid + # that by forcing the timeout to zero. + self._timeout = self.socket.gettimeout() + self.socket.settimeout(0) + + @contextlib.contextmanager + def _raise_on_error(self): + """ + A context manager that can be used to wrap calls that do I/O from + SecureTransport. If any of the I/O callbacks hit an exception, this + context manager will correctly propagate the exception after the fact. + This avoids silently swallowing those exceptions. + + It also correctly forces the socket closed. + """ + self._exception = None + + # We explicitly don't catch around this yield because in the unlikely + # event that an exception was hit in the block we don't want to swallow + # it. + yield + if self._exception is not None: + exception, self._exception = self._exception, None + self.close() + raise exception + + def _set_ciphers(self): + """ + Sets up the allowed ciphers. By default this matches the set in + util.ssl_.DEFAULT_CIPHERS, at least as supported by macOS. This is done + custom and doesn't allow changing at this time, mostly because parsing + OpenSSL cipher strings is going to be a freaking nightmare. + """ + ciphers = (Security.SSLCipherSuite * len(CIPHER_SUITES))(*CIPHER_SUITES) + result = Security.SSLSetEnabledCiphers( + self.context, ciphers, len(CIPHER_SUITES) + ) + _assert_no_error(result) + + def _set_alpn_protocols(self, protocols): + """ + Sets up the ALPN protocols on the context. + """ + if not protocols: + return + protocols_arr = _create_cfstring_array(protocols) + try: + result = Security.SSLSetALPNProtocols(self.context, protocols_arr) + _assert_no_error(result) + finally: + CoreFoundation.CFRelease(protocols_arr) + + def _custom_validate(self, verify, trust_bundle): + """ + Called when we have set custom validation. We do this in two cases: + first, when cert validation is entirely disabled; and second, when + using a custom trust DB. + Raises an SSLError if the connection is not trusted. + """ + # If we disabled cert validation, just say: cool. + if not verify: + return + + successes = ( + SecurityConst.kSecTrustResultUnspecified, + SecurityConst.kSecTrustResultProceed, + ) + try: + trust_result = self._evaluate_trust(trust_bundle) + if trust_result in successes: + return + reason = "error code: %d" % (trust_result,) + except Exception as e: + # Do not trust on error + reason = "exception: %r" % (e,) + + # SecureTransport does not send an alert nor shuts down the connection. + rec = _build_tls_unknown_ca_alert(self.version()) + self.socket.sendall(rec) + # close the connection immediately + # l_onoff = 1, activate linger + # l_linger = 0, linger for 0 seoncds + opts = struct.pack("ii", 1, 0) + self.socket.setsockopt(socket.SOL_SOCKET, socket.SO_LINGER, opts) + self.close() + raise ssl.SSLError("certificate verify failed, %s" % reason) + + def _evaluate_trust(self, trust_bundle): + # We want data in memory, so load it up. + if os.path.isfile(trust_bundle): + with open(trust_bundle, "rb") as f: + trust_bundle = f.read() + + cert_array = None + trust = Security.SecTrustRef() + + try: + # Get a CFArray that contains the certs we want. + cert_array = _cert_array_from_pem(trust_bundle) + + # Ok, now the hard part. We want to get the SecTrustRef that ST has + # created for this connection, shove our CAs into it, tell ST to + # ignore everything else it knows, and then ask if it can build a + # chain. This is a buuuunch of code. + result = Security.SSLCopyPeerTrust(self.context, ctypes.byref(trust)) + _assert_no_error(result) + if not trust: + raise ssl.SSLError("Failed to copy trust reference") + + result = Security.SecTrustSetAnchorCertificates(trust, cert_array) + _assert_no_error(result) + + result = Security.SecTrustSetAnchorCertificatesOnly(trust, True) + _assert_no_error(result) + + trust_result = Security.SecTrustResultType() + result = Security.SecTrustEvaluate(trust, ctypes.byref(trust_result)) + _assert_no_error(result) + finally: + if trust: + CoreFoundation.CFRelease(trust) + + if cert_array is not None: + CoreFoundation.CFRelease(cert_array) + + return trust_result.value + + def handshake( + self, + server_hostname, + verify, + trust_bundle, + min_version, + max_version, + client_cert, + client_key, + client_key_passphrase, + alpn_protocols, + ): + """ + Actually performs the TLS handshake. This is run automatically by + wrapped socket, and shouldn't be needed in user code. + """ + # First, we do the initial bits of connection setup. We need to create + # a context, set its I/O funcs, and set the connection reference. + self.context = Security.SSLCreateContext( + None, SecurityConst.kSSLClientSide, SecurityConst.kSSLStreamType + ) + result = Security.SSLSetIOFuncs( + self.context, _read_callback_pointer, _write_callback_pointer + ) + _assert_no_error(result) + + # Here we need to compute the handle to use. We do this by taking the + # id of self modulo 2**31 - 1. If this is already in the dictionary, we + # just keep incrementing by one until we find a free space. + with _connection_ref_lock: + handle = id(self) % 2147483647 + while handle in _connection_refs: + handle = (handle + 1) % 2147483647 + _connection_refs[handle] = self + + result = Security.SSLSetConnection(self.context, handle) + _assert_no_error(result) + + # If we have a server hostname, we should set that too. + if server_hostname: + if not isinstance(server_hostname, bytes): + server_hostname = server_hostname.encode("utf-8") + + result = Security.SSLSetPeerDomainName( + self.context, server_hostname, len(server_hostname) + ) + _assert_no_error(result) + + # Setup the ciphers. + self._set_ciphers() + + # Setup the ALPN protocols. + self._set_alpn_protocols(alpn_protocols) + + # Set the minimum and maximum TLS versions. + result = Security.SSLSetProtocolVersionMin(self.context, min_version) + _assert_no_error(result) + + result = Security.SSLSetProtocolVersionMax(self.context, max_version) + _assert_no_error(result) + + # If there's a trust DB, we need to use it. We do that by telling + # SecureTransport to break on server auth. We also do that if we don't + # want to validate the certs at all: we just won't actually do any + # authing in that case. + if not verify or trust_bundle is not None: + result = Security.SSLSetSessionOption( + self.context, SecurityConst.kSSLSessionOptionBreakOnServerAuth, True + ) + _assert_no_error(result) + + # If there's a client cert, we need to use it. + if client_cert: + self._keychain, self._keychain_dir = _temporary_keychain() + self._client_cert_chain = _load_client_cert_chain( + self._keychain, client_cert, client_key + ) + result = Security.SSLSetCertificate(self.context, self._client_cert_chain) + _assert_no_error(result) + + while True: + with self._raise_on_error(): + result = Security.SSLHandshake(self.context) + + if result == SecurityConst.errSSLWouldBlock: + raise socket.timeout("handshake timed out") + elif result == SecurityConst.errSSLServerAuthCompleted: + self._custom_validate(verify, trust_bundle) + continue + else: + _assert_no_error(result) + break + + def fileno(self): + return self.socket.fileno() + + # Copy-pasted from Python 3.5 source code + def _decref_socketios(self): + if self._makefile_refs > 0: + self._makefile_refs -= 1 + if self._closed: + self.close() + + def recv(self, bufsiz): + buffer = ctypes.create_string_buffer(bufsiz) + bytes_read = self.recv_into(buffer, bufsiz) + data = buffer[:bytes_read] + return data + + def recv_into(self, buffer, nbytes=None): + # Read short on EOF. + if self._closed: + return 0 + + if nbytes is None: + nbytes = len(buffer) + + buffer = (ctypes.c_char * nbytes).from_buffer(buffer) + processed_bytes = ctypes.c_size_t(0) + + with self._raise_on_error(): + result = Security.SSLRead( + self.context, buffer, nbytes, ctypes.byref(processed_bytes) + ) + + # There are some result codes that we want to treat as "not always + # errors". Specifically, those are errSSLWouldBlock, + # errSSLClosedGraceful, and errSSLClosedNoNotify. + if result == SecurityConst.errSSLWouldBlock: + # If we didn't process any bytes, then this was just a time out. + # However, we can get errSSLWouldBlock in situations when we *did* + # read some data, and in those cases we should just read "short" + # and return. + if processed_bytes.value == 0: + # Timed out, no data read. + raise socket.timeout("recv timed out") + elif result in ( + SecurityConst.errSSLClosedGraceful, + SecurityConst.errSSLClosedNoNotify, + ): + # The remote peer has closed this connection. We should do so as + # well. Note that we don't actually return here because in + # principle this could actually be fired along with return data. + # It's unlikely though. + self.close() + else: + _assert_no_error(result) + + # Ok, we read and probably succeeded. We should return whatever data + # was actually read. + return processed_bytes.value + + def settimeout(self, timeout): + self._timeout = timeout + + def gettimeout(self): + return self._timeout + + def send(self, data): + processed_bytes = ctypes.c_size_t(0) + + with self._raise_on_error(): + result = Security.SSLWrite( + self.context, data, len(data), ctypes.byref(processed_bytes) + ) + + if result == SecurityConst.errSSLWouldBlock and processed_bytes.value == 0: + # Timed out + raise socket.timeout("send timed out") + else: + _assert_no_error(result) + + # We sent, and probably succeeded. Tell them how much we sent. + return processed_bytes.value + + def sendall(self, data): + total_sent = 0 + while total_sent < len(data): + sent = self.send(data[total_sent : total_sent + SSL_WRITE_BLOCKSIZE]) + total_sent += sent + + def shutdown(self): + with self._raise_on_error(): + Security.SSLClose(self.context) + + def close(self): + # TODO: should I do clean shutdown here? Do I have to? + if self._makefile_refs < 1: + self._closed = True + if self.context: + CoreFoundation.CFRelease(self.context) + self.context = None + if self._client_cert_chain: + CoreFoundation.CFRelease(self._client_cert_chain) + self._client_cert_chain = None + if self._keychain: + Security.SecKeychainDelete(self._keychain) + CoreFoundation.CFRelease(self._keychain) + shutil.rmtree(self._keychain_dir) + self._keychain = self._keychain_dir = None + return self.socket.close() + else: + self._makefile_refs -= 1 + + def getpeercert(self, binary_form=False): + # Urgh, annoying. + # + # Here's how we do this: + # + # 1. Call SSLCopyPeerTrust to get hold of the trust object for this + # connection. + # 2. Call SecTrustGetCertificateAtIndex for index 0 to get the leaf. + # 3. To get the CN, call SecCertificateCopyCommonName and process that + # string so that it's of the appropriate type. + # 4. To get the SAN, we need to do something a bit more complex: + # a. Call SecCertificateCopyValues to get the data, requesting + # kSecOIDSubjectAltName. + # b. Mess about with this dictionary to try to get the SANs out. + # + # This is gross. Really gross. It's going to be a few hundred LoC extra + # just to repeat something that SecureTransport can *already do*. So my + # operating assumption at this time is that what we want to do is + # instead to just flag to urllib3 that it shouldn't do its own hostname + # validation when using SecureTransport. + if not binary_form: + raise ValueError("SecureTransport only supports dumping binary certs") + trust = Security.SecTrustRef() + certdata = None + der_bytes = None + + try: + # Grab the trust store. + result = Security.SSLCopyPeerTrust(self.context, ctypes.byref(trust)) + _assert_no_error(result) + if not trust: + # Probably we haven't done the handshake yet. No biggie. + return None + + cert_count = Security.SecTrustGetCertificateCount(trust) + if not cert_count: + # Also a case that might happen if we haven't handshaked. + # Handshook? Handshaken? + return None + + leaf = Security.SecTrustGetCertificateAtIndex(trust, 0) + assert leaf + + # Ok, now we want the DER bytes. + certdata = Security.SecCertificateCopyData(leaf) + assert certdata + + data_length = CoreFoundation.CFDataGetLength(certdata) + data_buffer = CoreFoundation.CFDataGetBytePtr(certdata) + der_bytes = ctypes.string_at(data_buffer, data_length) + finally: + if certdata: + CoreFoundation.CFRelease(certdata) + if trust: + CoreFoundation.CFRelease(trust) + + return der_bytes + + def version(self): + protocol = Security.SSLProtocol() + result = Security.SSLGetNegotiatedProtocolVersion( + self.context, ctypes.byref(protocol) + ) + _assert_no_error(result) + if protocol.value == SecurityConst.kTLSProtocol13: + raise ssl.SSLError("SecureTransport does not support TLS 1.3") + elif protocol.value == SecurityConst.kTLSProtocol12: + return "TLSv1.2" + elif protocol.value == SecurityConst.kTLSProtocol11: + return "TLSv1.1" + elif protocol.value == SecurityConst.kTLSProtocol1: + return "TLSv1" + elif protocol.value == SecurityConst.kSSLProtocol3: + return "SSLv3" + elif protocol.value == SecurityConst.kSSLProtocol2: + return "SSLv2" + else: + raise ssl.SSLError("Unknown TLS version: %r" % protocol) + + def _reuse(self): + self._makefile_refs += 1 + + def _drop(self): + if self._makefile_refs < 1: + self.close() + else: + self._makefile_refs -= 1 + + +if _fileobject: # Platform-specific: Python 2 + + def makefile(self, mode, bufsize=-1): + self._makefile_refs += 1 + return _fileobject(self, mode, bufsize, close=True) + + +else: # Platform-specific: Python 3 + + def makefile(self, mode="r", buffering=None, *args, **kwargs): + # We disable buffering with SecureTransport because it conflicts with + # the buffering that ST does internally (see issue #1153 for more). + buffering = 0 + return backport_makefile(self, mode, buffering, *args, **kwargs) + + +WrappedSocket.makefile = makefile + + +class SecureTransportContext(object): + """ + I am a wrapper class for the SecureTransport library, to translate the + interface of the standard library ``SSLContext`` object to calls into + SecureTransport. + """ + + def __init__(self, protocol): + self._min_version, self._max_version = _protocol_to_min_max[protocol] + self._options = 0 + self._verify = False + self._trust_bundle = None + self._client_cert = None + self._client_key = None + self._client_key_passphrase = None + self._alpn_protocols = None + + @property + def check_hostname(self): + """ + SecureTransport cannot have its hostname checking disabled. For more, + see the comment on getpeercert() in this file. + """ + return True + + @check_hostname.setter + def check_hostname(self, value): + """ + SecureTransport cannot have its hostname checking disabled. For more, + see the comment on getpeercert() in this file. + """ + pass + + @property + def options(self): + # TODO: Well, crap. + # + # So this is the bit of the code that is the most likely to cause us + # trouble. Essentially we need to enumerate all of the SSL options that + # users might want to use and try to see if we can sensibly translate + # them, or whether we should just ignore them. + return self._options + + @options.setter + def options(self, value): + # TODO: Update in line with above. + self._options = value + + @property + def verify_mode(self): + return ssl.CERT_REQUIRED if self._verify else ssl.CERT_NONE + + @verify_mode.setter + def verify_mode(self, value): + self._verify = True if value == ssl.CERT_REQUIRED else False + + def set_default_verify_paths(self): + # So, this has to do something a bit weird. Specifically, what it does + # is nothing. + # + # This means that, if we had previously had load_verify_locations + # called, this does not undo that. We need to do that because it turns + # out that the rest of the urllib3 code will attempt to load the + # default verify paths if it hasn't been told about any paths, even if + # the context itself was sometime earlier. We resolve that by just + # ignoring it. + pass + + def load_default_certs(self): + return self.set_default_verify_paths() + + def set_ciphers(self, ciphers): + # For now, we just require the default cipher string. + if ciphers != util.ssl_.DEFAULT_CIPHERS: + raise ValueError("SecureTransport doesn't support custom cipher strings") + + def load_verify_locations(self, cafile=None, capath=None, cadata=None): + # OK, we only really support cadata and cafile. + if capath is not None: + raise ValueError("SecureTransport does not support cert directories") + + # Raise if cafile does not exist. + if cafile is not None: + with open(cafile): + pass + + self._trust_bundle = cafile or cadata + + def load_cert_chain(self, certfile, keyfile=None, password=None): + self._client_cert = certfile + self._client_key = keyfile + self._client_cert_passphrase = password + + def set_alpn_protocols(self, protocols): + """ + Sets the ALPN protocols that will later be set on the context. + + Raises a NotImplementedError if ALPN is not supported. + """ + if not hasattr(Security, "SSLSetALPNProtocols"): + raise NotImplementedError( + "SecureTransport supports ALPN only in macOS 10.12+" + ) + self._alpn_protocols = [six.ensure_binary(p) for p in protocols] + + def wrap_socket( + self, + sock, + server_side=False, + do_handshake_on_connect=True, + suppress_ragged_eofs=True, + server_hostname=None, + ): + # So, what do we do here? Firstly, we assert some properties. This is a + # stripped down shim, so there is some functionality we don't support. + # See PEP 543 for the real deal. + assert not server_side + assert do_handshake_on_connect + assert suppress_ragged_eofs + + # Ok, we're good to go. Now we want to create the wrapped socket object + # and store it in the appropriate place. + wrapped_socket = WrappedSocket(sock) + + # Now we can handshake + wrapped_socket.handshake( + server_hostname, + self._verify, + self._trust_bundle, + self._min_version, + self._max_version, + self._client_cert, + self._client_key, + self._client_key_passphrase, + self._alpn_protocols, + ) + return wrapped_socket diff --git a/openpype/hosts/fusion/vendor/urllib3/contrib/socks.py b/openpype/hosts/fusion/vendor/urllib3/contrib/socks.py new file mode 100644 index 0000000000..c326e80dd1 --- /dev/null +++ b/openpype/hosts/fusion/vendor/urllib3/contrib/socks.py @@ -0,0 +1,216 @@ +# -*- coding: utf-8 -*- +""" +This module contains provisional support for SOCKS proxies from within +urllib3. This module supports SOCKS4, SOCKS4A (an extension of SOCKS4), and +SOCKS5. To enable its functionality, either install PySocks or install this +module with the ``socks`` extra. + +The SOCKS implementation supports the full range of urllib3 features. It also +supports the following SOCKS features: + +- SOCKS4A (``proxy_url='socks4a://...``) +- SOCKS4 (``proxy_url='socks4://...``) +- SOCKS5 with remote DNS (``proxy_url='socks5h://...``) +- SOCKS5 with local DNS (``proxy_url='socks5://...``) +- Usernames and passwords for the SOCKS proxy + +.. note:: + It is recommended to use ``socks5h://`` or ``socks4a://`` schemes in + your ``proxy_url`` to ensure that DNS resolution is done from the remote + server instead of client-side when connecting to a domain name. + +SOCKS4 supports IPv4 and domain names with the SOCKS4A extension. SOCKS5 +supports IPv4, IPv6, and domain names. + +When connecting to a SOCKS4 proxy the ``username`` portion of the ``proxy_url`` +will be sent as the ``userid`` section of the SOCKS request: + +.. code-block:: python + + proxy_url="socks4a://@proxy-host" + +When connecting to a SOCKS5 proxy the ``username`` and ``password`` portion +of the ``proxy_url`` will be sent as the username/password to authenticate +with the proxy: + +.. code-block:: python + + proxy_url="socks5h://:@proxy-host" + +""" +from __future__ import absolute_import + +try: + import socks +except ImportError: + import warnings + + from ..exceptions import DependencyWarning + + warnings.warn( + ( + "SOCKS support in urllib3 requires the installation of optional " + "dependencies: specifically, PySocks. For more information, see " + "https://urllib3.readthedocs.io/en/1.26.x/contrib.html#socks-proxies" + ), + DependencyWarning, + ) + raise + +from socket import error as SocketError +from socket import timeout as SocketTimeout + +from ..connection import HTTPConnection, HTTPSConnection +from ..connectionpool import HTTPConnectionPool, HTTPSConnectionPool +from ..exceptions import ConnectTimeoutError, NewConnectionError +from ..poolmanager import PoolManager +from ..util.url import parse_url + +try: + import ssl +except ImportError: + ssl = None + + +class SOCKSConnection(HTTPConnection): + """ + A plain-text HTTP connection that connects via a SOCKS proxy. + """ + + def __init__(self, *args, **kwargs): + self._socks_options = kwargs.pop("_socks_options") + super(SOCKSConnection, self).__init__(*args, **kwargs) + + def _new_conn(self): + """ + Establish a new connection via the SOCKS proxy. + """ + extra_kw = {} + if self.source_address: + extra_kw["source_address"] = self.source_address + + if self.socket_options: + extra_kw["socket_options"] = self.socket_options + + try: + conn = socks.create_connection( + (self.host, self.port), + proxy_type=self._socks_options["socks_version"], + proxy_addr=self._socks_options["proxy_host"], + proxy_port=self._socks_options["proxy_port"], + proxy_username=self._socks_options["username"], + proxy_password=self._socks_options["password"], + proxy_rdns=self._socks_options["rdns"], + timeout=self.timeout, + **extra_kw + ) + + except SocketTimeout: + raise ConnectTimeoutError( + self, + "Connection to %s timed out. (connect timeout=%s)" + % (self.host, self.timeout), + ) + + except socks.ProxyError as e: + # This is fragile as hell, but it seems to be the only way to raise + # useful errors here. + if e.socket_err: + error = e.socket_err + if isinstance(error, SocketTimeout): + raise ConnectTimeoutError( + self, + "Connection to %s timed out. (connect timeout=%s)" + % (self.host, self.timeout), + ) + else: + raise NewConnectionError( + self, "Failed to establish a new connection: %s" % error + ) + else: + raise NewConnectionError( + self, "Failed to establish a new connection: %s" % e + ) + + except SocketError as e: # Defensive: PySocks should catch all these. + raise NewConnectionError( + self, "Failed to establish a new connection: %s" % e + ) + + return conn + + +# We don't need to duplicate the Verified/Unverified distinction from +# urllib3/connection.py here because the HTTPSConnection will already have been +# correctly set to either the Verified or Unverified form by that module. This +# means the SOCKSHTTPSConnection will automatically be the correct type. +class SOCKSHTTPSConnection(SOCKSConnection, HTTPSConnection): + pass + + +class SOCKSHTTPConnectionPool(HTTPConnectionPool): + ConnectionCls = SOCKSConnection + + +class SOCKSHTTPSConnectionPool(HTTPSConnectionPool): + ConnectionCls = SOCKSHTTPSConnection + + +class SOCKSProxyManager(PoolManager): + """ + A version of the urllib3 ProxyManager that routes connections via the + defined SOCKS proxy. + """ + + pool_classes_by_scheme = { + "http": SOCKSHTTPConnectionPool, + "https": SOCKSHTTPSConnectionPool, + } + + def __init__( + self, + proxy_url, + username=None, + password=None, + num_pools=10, + headers=None, + **connection_pool_kw + ): + parsed = parse_url(proxy_url) + + if username is None and password is None and parsed.auth is not None: + split = parsed.auth.split(":") + if len(split) == 2: + username, password = split + if parsed.scheme == "socks5": + socks_version = socks.PROXY_TYPE_SOCKS5 + rdns = False + elif parsed.scheme == "socks5h": + socks_version = socks.PROXY_TYPE_SOCKS5 + rdns = True + elif parsed.scheme == "socks4": + socks_version = socks.PROXY_TYPE_SOCKS4 + rdns = False + elif parsed.scheme == "socks4a": + socks_version = socks.PROXY_TYPE_SOCKS4 + rdns = True + else: + raise ValueError("Unable to determine SOCKS version from %s" % proxy_url) + + self.proxy_url = proxy_url + + socks_options = { + "socks_version": socks_version, + "proxy_host": parsed.host, + "proxy_port": parsed.port, + "username": username, + "password": password, + "rdns": rdns, + } + connection_pool_kw["_socks_options"] = socks_options + + super(SOCKSProxyManager, self).__init__( + num_pools, headers, **connection_pool_kw + ) + + self.pool_classes_by_scheme = SOCKSProxyManager.pool_classes_by_scheme diff --git a/openpype/hosts/fusion/vendor/urllib3/exceptions.py b/openpype/hosts/fusion/vendor/urllib3/exceptions.py new file mode 100644 index 0000000000..cba6f3f560 --- /dev/null +++ b/openpype/hosts/fusion/vendor/urllib3/exceptions.py @@ -0,0 +1,323 @@ +from __future__ import absolute_import + +from .packages.six.moves.http_client import IncompleteRead as httplib_IncompleteRead + +# Base Exceptions + + +class HTTPError(Exception): + """Base exception used by this module.""" + + pass + + +class HTTPWarning(Warning): + """Base warning used by this module.""" + + pass + + +class PoolError(HTTPError): + """Base exception for errors caused within a pool.""" + + def __init__(self, pool, message): + self.pool = pool + HTTPError.__init__(self, "%s: %s" % (pool, message)) + + def __reduce__(self): + # For pickling purposes. + return self.__class__, (None, None) + + +class RequestError(PoolError): + """Base exception for PoolErrors that have associated URLs.""" + + def __init__(self, pool, url, message): + self.url = url + PoolError.__init__(self, pool, message) + + def __reduce__(self): + # For pickling purposes. + return self.__class__, (None, self.url, None) + + +class SSLError(HTTPError): + """Raised when SSL certificate fails in an HTTPS connection.""" + + pass + + +class ProxyError(HTTPError): + """Raised when the connection to a proxy fails.""" + + def __init__(self, message, error, *args): + super(ProxyError, self).__init__(message, error, *args) + self.original_error = error + + +class DecodeError(HTTPError): + """Raised when automatic decoding based on Content-Type fails.""" + + pass + + +class ProtocolError(HTTPError): + """Raised when something unexpected happens mid-request/response.""" + + pass + + +#: Renamed to ProtocolError but aliased for backwards compatibility. +ConnectionError = ProtocolError + + +# Leaf Exceptions + + +class MaxRetryError(RequestError): + """Raised when the maximum number of retries is exceeded. + + :param pool: The connection pool + :type pool: :class:`~urllib3.connectionpool.HTTPConnectionPool` + :param string url: The requested Url + :param exceptions.Exception reason: The underlying error + + """ + + def __init__(self, pool, url, reason=None): + self.reason = reason + + message = "Max retries exceeded with url: %s (Caused by %r)" % (url, reason) + + RequestError.__init__(self, pool, url, message) + + +class HostChangedError(RequestError): + """Raised when an existing pool gets a request for a foreign host.""" + + def __init__(self, pool, url, retries=3): + message = "Tried to open a foreign host with url: %s" % url + RequestError.__init__(self, pool, url, message) + self.retries = retries + + +class TimeoutStateError(HTTPError): + """Raised when passing an invalid state to a timeout""" + + pass + + +class TimeoutError(HTTPError): + """Raised when a socket timeout error occurs. + + Catching this error will catch both :exc:`ReadTimeoutErrors + ` and :exc:`ConnectTimeoutErrors `. + """ + + pass + + +class ReadTimeoutError(TimeoutError, RequestError): + """Raised when a socket timeout occurs while receiving data from a server""" + + pass + + +# This timeout error does not have a URL attached and needs to inherit from the +# base HTTPError +class ConnectTimeoutError(TimeoutError): + """Raised when a socket timeout occurs while connecting to a server""" + + pass + + +class NewConnectionError(ConnectTimeoutError, PoolError): + """Raised when we fail to establish a new connection. Usually ECONNREFUSED.""" + + pass + + +class EmptyPoolError(PoolError): + """Raised when a pool runs out of connections and no more are allowed.""" + + pass + + +class ClosedPoolError(PoolError): + """Raised when a request enters a pool after the pool has been closed.""" + + pass + + +class LocationValueError(ValueError, HTTPError): + """Raised when there is something wrong with a given URL input.""" + + pass + + +class LocationParseError(LocationValueError): + """Raised when get_host or similar fails to parse the URL input.""" + + def __init__(self, location): + message = "Failed to parse: %s" % location + HTTPError.__init__(self, message) + + self.location = location + + +class URLSchemeUnknown(LocationValueError): + """Raised when a URL input has an unsupported scheme.""" + + def __init__(self, scheme): + message = "Not supported URL scheme %s" % scheme + super(URLSchemeUnknown, self).__init__(message) + + self.scheme = scheme + + +class ResponseError(HTTPError): + """Used as a container for an error reason supplied in a MaxRetryError.""" + + GENERIC_ERROR = "too many error responses" + SPECIFIC_ERROR = "too many {status_code} error responses" + + +class SecurityWarning(HTTPWarning): + """Warned when performing security reducing actions""" + + pass + + +class SubjectAltNameWarning(SecurityWarning): + """Warned when connecting to a host with a certificate missing a SAN.""" + + pass + + +class InsecureRequestWarning(SecurityWarning): + """Warned when making an unverified HTTPS request.""" + + pass + + +class SystemTimeWarning(SecurityWarning): + """Warned when system time is suspected to be wrong""" + + pass + + +class InsecurePlatformWarning(SecurityWarning): + """Warned when certain TLS/SSL configuration is not available on a platform.""" + + pass + + +class SNIMissingWarning(HTTPWarning): + """Warned when making a HTTPS request without SNI available.""" + + pass + + +class DependencyWarning(HTTPWarning): + """ + Warned when an attempt is made to import a module with missing optional + dependencies. + """ + + pass + + +class ResponseNotChunked(ProtocolError, ValueError): + """Response needs to be chunked in order to read it as chunks.""" + + pass + + +class BodyNotHttplibCompatible(HTTPError): + """ + Body should be :class:`http.client.HTTPResponse` like + (have an fp attribute which returns raw chunks) for read_chunked(). + """ + + pass + + +class IncompleteRead(HTTPError, httplib_IncompleteRead): + """ + Response length doesn't match expected Content-Length + + Subclass of :class:`http.client.IncompleteRead` to allow int value + for ``partial`` to avoid creating large objects on streamed reads. + """ + + def __init__(self, partial, expected): + super(IncompleteRead, self).__init__(partial, expected) + + def __repr__(self): + return "IncompleteRead(%i bytes read, %i more expected)" % ( + self.partial, + self.expected, + ) + + +class InvalidChunkLength(HTTPError, httplib_IncompleteRead): + """Invalid chunk length in a chunked response.""" + + def __init__(self, response, length): + super(InvalidChunkLength, self).__init__( + response.tell(), response.length_remaining + ) + self.response = response + self.length = length + + def __repr__(self): + return "InvalidChunkLength(got length %r, %i bytes read)" % ( + self.length, + self.partial, + ) + + +class InvalidHeader(HTTPError): + """The header provided was somehow invalid.""" + + pass + + +class ProxySchemeUnknown(AssertionError, URLSchemeUnknown): + """ProxyManager does not support the supplied scheme""" + + # TODO(t-8ch): Stop inheriting from AssertionError in v2.0. + + def __init__(self, scheme): + # 'localhost' is here because our URL parser parses + # localhost:8080 -> scheme=localhost, remove if we fix this. + if scheme == "localhost": + scheme = None + if scheme is None: + message = "Proxy URL had no scheme, should start with http:// or https://" + else: + message = ( + "Proxy URL had unsupported scheme %s, should use http:// or https://" + % scheme + ) + super(ProxySchemeUnknown, self).__init__(message) + + +class ProxySchemeUnsupported(ValueError): + """Fetching HTTPS resources through HTTPS proxies is unsupported""" + + pass + + +class HeaderParsingError(HTTPError): + """Raised by assert_header_parsing, but we convert it to a log.warning statement.""" + + def __init__(self, defects, unparsed_data): + message = "%s, unparsed data: %r" % (defects or "Unknown", unparsed_data) + super(HeaderParsingError, self).__init__(message) + + +class UnrewindableBodyError(HTTPError): + """urllib3 encountered an error when trying to rewind a body""" + + pass diff --git a/openpype/hosts/fusion/vendor/urllib3/fields.py b/openpype/hosts/fusion/vendor/urllib3/fields.py new file mode 100644 index 0000000000..9d630f491d --- /dev/null +++ b/openpype/hosts/fusion/vendor/urllib3/fields.py @@ -0,0 +1,274 @@ +from __future__ import absolute_import + +import email.utils +import mimetypes +import re + +from .packages import six + + +def guess_content_type(filename, default="application/octet-stream"): + """ + Guess the "Content-Type" of a file. + + :param filename: + The filename to guess the "Content-Type" of using :mod:`mimetypes`. + :param default: + If no "Content-Type" can be guessed, default to `default`. + """ + if filename: + return mimetypes.guess_type(filename)[0] or default + return default + + +def format_header_param_rfc2231(name, value): + """ + Helper function to format and quote a single header parameter using the + strategy defined in RFC 2231. + + Particularly useful for header parameters which might contain + non-ASCII values, like file names. This follows + `RFC 2388 Section 4.4 `_. + + :param name: + The name of the parameter, a string expected to be ASCII only. + :param value: + The value of the parameter, provided as ``bytes`` or `str``. + :ret: + An RFC-2231-formatted unicode string. + """ + if isinstance(value, six.binary_type): + value = value.decode("utf-8") + + if not any(ch in value for ch in '"\\\r\n'): + result = u'%s="%s"' % (name, value) + try: + result.encode("ascii") + except (UnicodeEncodeError, UnicodeDecodeError): + pass + else: + return result + + if six.PY2: # Python 2: + value = value.encode("utf-8") + + # encode_rfc2231 accepts an encoded string and returns an ascii-encoded + # string in Python 2 but accepts and returns unicode strings in Python 3 + value = email.utils.encode_rfc2231(value, "utf-8") + value = "%s*=%s" % (name, value) + + if six.PY2: # Python 2: + value = value.decode("utf-8") + + return value + + +_HTML5_REPLACEMENTS = { + u"\u0022": u"%22", + # Replace "\" with "\\". + u"\u005C": u"\u005C\u005C", +} + +# All control characters from 0x00 to 0x1F *except* 0x1B. +_HTML5_REPLACEMENTS.update( + { + six.unichr(cc): u"%{:02X}".format(cc) + for cc in range(0x00, 0x1F + 1) + if cc not in (0x1B,) + } +) + + +def _replace_multiple(value, needles_and_replacements): + def replacer(match): + return needles_and_replacements[match.group(0)] + + pattern = re.compile( + r"|".join([re.escape(needle) for needle in needles_and_replacements.keys()]) + ) + + result = pattern.sub(replacer, value) + + return result + + +def format_header_param_html5(name, value): + """ + Helper function to format and quote a single header parameter using the + HTML5 strategy. + + Particularly useful for header parameters which might contain + non-ASCII values, like file names. This follows the `HTML5 Working Draft + Section 4.10.22.7`_ and matches the behavior of curl and modern browsers. + + .. _HTML5 Working Draft Section 4.10.22.7: + https://w3c.github.io/html/sec-forms.html#multipart-form-data + + :param name: + The name of the parameter, a string expected to be ASCII only. + :param value: + The value of the parameter, provided as ``bytes`` or `str``. + :ret: + A unicode string, stripped of troublesome characters. + """ + if isinstance(value, six.binary_type): + value = value.decode("utf-8") + + value = _replace_multiple(value, _HTML5_REPLACEMENTS) + + return u'%s="%s"' % (name, value) + + +# For backwards-compatibility. +format_header_param = format_header_param_html5 + + +class RequestField(object): + """ + A data container for request body parameters. + + :param name: + The name of this request field. Must be unicode. + :param data: + The data/value body. + :param filename: + An optional filename of the request field. Must be unicode. + :param headers: + An optional dict-like object of headers to initially use for the field. + :param header_formatter: + An optional callable that is used to encode and format the headers. By + default, this is :func:`format_header_param_html5`. + """ + + def __init__( + self, + name, + data, + filename=None, + headers=None, + header_formatter=format_header_param_html5, + ): + self._name = name + self._filename = filename + self.data = data + self.headers = {} + if headers: + self.headers = dict(headers) + self.header_formatter = header_formatter + + @classmethod + def from_tuples(cls, fieldname, value, header_formatter=format_header_param_html5): + """ + A :class:`~urllib3.fields.RequestField` factory from old-style tuple parameters. + + Supports constructing :class:`~urllib3.fields.RequestField` from + parameter of key/value strings AND key/filetuple. A filetuple is a + (filename, data, MIME type) tuple where the MIME type is optional. + For example:: + + 'foo': 'bar', + 'fakefile': ('foofile.txt', 'contents of foofile'), + 'realfile': ('barfile.txt', open('realfile').read()), + 'typedfile': ('bazfile.bin', open('bazfile').read(), 'image/jpeg'), + 'nonamefile': 'contents of nonamefile field', + + Field names and filenames must be unicode. + """ + if isinstance(value, tuple): + if len(value) == 3: + filename, data, content_type = value + else: + filename, data = value + content_type = guess_content_type(filename) + else: + filename = None + content_type = None + data = value + + request_param = cls( + fieldname, data, filename=filename, header_formatter=header_formatter + ) + request_param.make_multipart(content_type=content_type) + + return request_param + + def _render_part(self, name, value): + """ + Overridable helper function to format a single header parameter. By + default, this calls ``self.header_formatter``. + + :param name: + The name of the parameter, a string expected to be ASCII only. + :param value: + The value of the parameter, provided as a unicode string. + """ + + return self.header_formatter(name, value) + + def _render_parts(self, header_parts): + """ + Helper function to format and quote a single header. + + Useful for single headers that are composed of multiple items. E.g., + 'Content-Disposition' fields. + + :param header_parts: + A sequence of (k, v) tuples or a :class:`dict` of (k, v) to format + as `k1="v1"; k2="v2"; ...`. + """ + parts = [] + iterable = header_parts + if isinstance(header_parts, dict): + iterable = header_parts.items() + + for name, value in iterable: + if value is not None: + parts.append(self._render_part(name, value)) + + return u"; ".join(parts) + + def render_headers(self): + """ + Renders the headers for this request field. + """ + lines = [] + + sort_keys = ["Content-Disposition", "Content-Type", "Content-Location"] + for sort_key in sort_keys: + if self.headers.get(sort_key, False): + lines.append(u"%s: %s" % (sort_key, self.headers[sort_key])) + + for header_name, header_value in self.headers.items(): + if header_name not in sort_keys: + if header_value: + lines.append(u"%s: %s" % (header_name, header_value)) + + lines.append(u"\r\n") + return u"\r\n".join(lines) + + def make_multipart( + self, content_disposition=None, content_type=None, content_location=None + ): + """ + Makes this request field into a multipart request field. + + This method overrides "Content-Disposition", "Content-Type" and + "Content-Location" headers to the request parameter. + + :param content_type: + The 'Content-Type' of the request body. + :param content_location: + The 'Content-Location' of the request body. + + """ + self.headers["Content-Disposition"] = content_disposition or u"form-data" + self.headers["Content-Disposition"] += u"; ".join( + [ + u"", + self._render_parts( + ((u"name", self._name), (u"filename", self._filename)) + ), + ] + ) + self.headers["Content-Type"] = content_type + self.headers["Content-Location"] = content_location diff --git a/openpype/hosts/fusion/vendor/urllib3/filepost.py b/openpype/hosts/fusion/vendor/urllib3/filepost.py new file mode 100644 index 0000000000..36c9252c64 --- /dev/null +++ b/openpype/hosts/fusion/vendor/urllib3/filepost.py @@ -0,0 +1,98 @@ +from __future__ import absolute_import + +import binascii +import codecs +import os +from io import BytesIO + +from .fields import RequestField +from .packages import six +from .packages.six import b + +writer = codecs.lookup("utf-8")[3] + + +def choose_boundary(): + """ + Our embarrassingly-simple replacement for mimetools.choose_boundary. + """ + boundary = binascii.hexlify(os.urandom(16)) + if not six.PY2: + boundary = boundary.decode("ascii") + return boundary + + +def iter_field_objects(fields): + """ + Iterate over fields. + + Supports list of (k, v) tuples and dicts, and lists of + :class:`~urllib3.fields.RequestField`. + + """ + if isinstance(fields, dict): + i = six.iteritems(fields) + else: + i = iter(fields) + + for field in i: + if isinstance(field, RequestField): + yield field + else: + yield RequestField.from_tuples(*field) + + +def iter_fields(fields): + """ + .. deprecated:: 1.6 + + Iterate over fields. + + The addition of :class:`~urllib3.fields.RequestField` makes this function + obsolete. Instead, use :func:`iter_field_objects`, which returns + :class:`~urllib3.fields.RequestField` objects. + + Supports list of (k, v) tuples and dicts. + """ + if isinstance(fields, dict): + return ((k, v) for k, v in six.iteritems(fields)) + + return ((k, v) for k, v in fields) + + +def encode_multipart_formdata(fields, boundary=None): + """ + Encode a dictionary of ``fields`` using the multipart/form-data MIME format. + + :param fields: + Dictionary of fields or list of (key, :class:`~urllib3.fields.RequestField`). + + :param boundary: + If not specified, then a random boundary will be generated using + :func:`urllib3.filepost.choose_boundary`. + """ + body = BytesIO() + if boundary is None: + boundary = choose_boundary() + + for field in iter_field_objects(fields): + body.write(b("--%s\r\n" % (boundary))) + + writer(body).write(field.render_headers()) + data = field.data + + if isinstance(data, int): + data = str(data) # Backwards compatibility + + if isinstance(data, six.text_type): + writer(body).write(data) + else: + body.write(data) + + body.write(b"\r\n") + + body.write(b("--%s--\r\n" % (boundary))) + + content_type = str("multipart/form-data; boundary=%s" % boundary) + + return body.getvalue(), content_type diff --git a/openpype/hosts/fusion/vendor/urllib3/packages/__init__.py b/openpype/hosts/fusion/vendor/urllib3/packages/__init__.py new file mode 100644 index 0000000000..fce4caa65d --- /dev/null +++ b/openpype/hosts/fusion/vendor/urllib3/packages/__init__.py @@ -0,0 +1,5 @@ +from __future__ import absolute_import + +from . import ssl_match_hostname + +__all__ = ("ssl_match_hostname",) diff --git a/openpype/hosts/fusion/vendor/urllib3/packages/backports/__init__.py b/openpype/hosts/fusion/vendor/urllib3/packages/backports/__init__.py new file mode 100644 index 0000000000..e69de29bb2 diff --git a/openpype/hosts/fusion/vendor/urllib3/packages/backports/makefile.py b/openpype/hosts/fusion/vendor/urllib3/packages/backports/makefile.py new file mode 100644 index 0000000000..b8fb2154b6 --- /dev/null +++ b/openpype/hosts/fusion/vendor/urllib3/packages/backports/makefile.py @@ -0,0 +1,51 @@ +# -*- coding: utf-8 -*- +""" +backports.makefile +~~~~~~~~~~~~~~~~~~ + +Backports the Python 3 ``socket.makefile`` method for use with anything that +wants to create a "fake" socket object. +""" +import io +from socket import SocketIO + + +def backport_makefile( + self, mode="r", buffering=None, encoding=None, errors=None, newline=None +): + """ + Backport of ``socket.makefile`` from Python 3.5. + """ + if not set(mode) <= {"r", "w", "b"}: + raise ValueError("invalid mode %r (only r, w, b allowed)" % (mode,)) + writing = "w" in mode + reading = "r" in mode or not writing + assert reading or writing + binary = "b" in mode + rawmode = "" + if reading: + rawmode += "r" + if writing: + rawmode += "w" + raw = SocketIO(self, rawmode) + self._makefile_refs += 1 + if buffering is None: + buffering = -1 + if buffering < 0: + buffering = io.DEFAULT_BUFFER_SIZE + if buffering == 0: + if not binary: + raise ValueError("unbuffered streams must be binary") + return raw + if reading and writing: + buffer = io.BufferedRWPair(raw, raw, buffering) + elif reading: + buffer = io.BufferedReader(raw, buffering) + else: + assert writing + buffer = io.BufferedWriter(raw, buffering) + if binary: + return buffer + text = io.TextIOWrapper(buffer, encoding, errors, newline) + text.mode = mode + return text diff --git a/openpype/hosts/fusion/vendor/urllib3/packages/six.py b/openpype/hosts/fusion/vendor/urllib3/packages/six.py new file mode 100644 index 0000000000..ba50acb062 --- /dev/null +++ b/openpype/hosts/fusion/vendor/urllib3/packages/six.py @@ -0,0 +1,1077 @@ +# Copyright (c) 2010-2020 Benjamin Peterson +# +# Permission is hereby granted, free of charge, to any person obtaining a copy +# of this software and associated documentation files (the "Software"), to deal +# in the Software without restriction, including without limitation the rights +# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell +# copies of the Software, and to permit persons to whom the Software is +# furnished to do so, subject to the following conditions: +# +# The above copyright notice and this permission notice shall be included in all +# copies or substantial portions of the Software. +# +# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE +# SOFTWARE. + +"""Utilities for writing code that runs on Python 2 and 3""" + +from __future__ import absolute_import + +import functools +import itertools +import operator +import sys +import types + +__author__ = "Benjamin Peterson " +__version__ = "1.16.0" + + +# Useful for very coarse version differentiation. +PY2 = sys.version_info[0] == 2 +PY3 = sys.version_info[0] == 3 +PY34 = sys.version_info[0:2] >= (3, 4) + +if PY3: + string_types = (str,) + integer_types = (int,) + class_types = (type,) + text_type = str + binary_type = bytes + + MAXSIZE = sys.maxsize +else: + string_types = (basestring,) + integer_types = (int, long) + class_types = (type, types.ClassType) + text_type = unicode + binary_type = str + + if sys.platform.startswith("java"): + # Jython always uses 32 bits. + MAXSIZE = int((1 << 31) - 1) + else: + # It's possible to have sizeof(long) != sizeof(Py_ssize_t). + class X(object): + def __len__(self): + return 1 << 31 + + try: + len(X()) + except OverflowError: + # 32-bit + MAXSIZE = int((1 << 31) - 1) + else: + # 64-bit + MAXSIZE = int((1 << 63) - 1) + del X + +if PY34: + from importlib.util import spec_from_loader +else: + spec_from_loader = None + + +def _add_doc(func, doc): + """Add documentation to a function.""" + func.__doc__ = doc + + +def _import_module(name): + """Import module, returning the module after the last dot.""" + __import__(name) + return sys.modules[name] + + +class _LazyDescr(object): + def __init__(self, name): + self.name = name + + def __get__(self, obj, tp): + result = self._resolve() + setattr(obj, self.name, result) # Invokes __set__. + try: + # This is a bit ugly, but it avoids running this again by + # removing this descriptor. + delattr(obj.__class__, self.name) + except AttributeError: + pass + return result + + +class MovedModule(_LazyDescr): + def __init__(self, name, old, new=None): + super(MovedModule, self).__init__(name) + if PY3: + if new is None: + new = name + self.mod = new + else: + self.mod = old + + def _resolve(self): + return _import_module(self.mod) + + def __getattr__(self, attr): + _module = self._resolve() + value = getattr(_module, attr) + setattr(self, attr, value) + return value + + +class _LazyModule(types.ModuleType): + def __init__(self, name): + super(_LazyModule, self).__init__(name) + self.__doc__ = self.__class__.__doc__ + + def __dir__(self): + attrs = ["__doc__", "__name__"] + attrs += [attr.name for attr in self._moved_attributes] + return attrs + + # Subclasses should override this + _moved_attributes = [] + + +class MovedAttribute(_LazyDescr): + def __init__(self, name, old_mod, new_mod, old_attr=None, new_attr=None): + super(MovedAttribute, self).__init__(name) + if PY3: + if new_mod is None: + new_mod = name + self.mod = new_mod + if new_attr is None: + if old_attr is None: + new_attr = name + else: + new_attr = old_attr + self.attr = new_attr + else: + self.mod = old_mod + if old_attr is None: + old_attr = name + self.attr = old_attr + + def _resolve(self): + module = _import_module(self.mod) + return getattr(module, self.attr) + + +class _SixMetaPathImporter(object): + + """ + A meta path importer to import six.moves and its submodules. + + This class implements a PEP302 finder and loader. It should be compatible + with Python 2.5 and all existing versions of Python3 + """ + + def __init__(self, six_module_name): + self.name = six_module_name + self.known_modules = {} + + def _add_module(self, mod, *fullnames): + for fullname in fullnames: + self.known_modules[self.name + "." + fullname] = mod + + def _get_module(self, fullname): + return self.known_modules[self.name + "." + fullname] + + def find_module(self, fullname, path=None): + if fullname in self.known_modules: + return self + return None + + def find_spec(self, fullname, path, target=None): + if fullname in self.known_modules: + return spec_from_loader(fullname, self) + return None + + def __get_module(self, fullname): + try: + return self.known_modules[fullname] + except KeyError: + raise ImportError("This loader does not know module " + fullname) + + def load_module(self, fullname): + try: + # in case of a reload + return sys.modules[fullname] + except KeyError: + pass + mod = self.__get_module(fullname) + if isinstance(mod, MovedModule): + mod = mod._resolve() + else: + mod.__loader__ = self + sys.modules[fullname] = mod + return mod + + def is_package(self, fullname): + """ + Return true, if the named module is a package. + + We need this method to get correct spec objects with + Python 3.4 (see PEP451) + """ + return hasattr(self.__get_module(fullname), "__path__") + + def get_code(self, fullname): + """Return None + + Required, if is_package is implemented""" + self.__get_module(fullname) # eventually raises ImportError + return None + + get_source = get_code # same as get_code + + def create_module(self, spec): + return self.load_module(spec.name) + + def exec_module(self, module): + pass + + +_importer = _SixMetaPathImporter(__name__) + + +class _MovedItems(_LazyModule): + + """Lazy loading of moved objects""" + + __path__ = [] # mark as package + + +_moved_attributes = [ + MovedAttribute("cStringIO", "cStringIO", "io", "StringIO"), + MovedAttribute("filter", "itertools", "builtins", "ifilter", "filter"), + MovedAttribute( + "filterfalse", "itertools", "itertools", "ifilterfalse", "filterfalse" + ), + MovedAttribute("input", "__builtin__", "builtins", "raw_input", "input"), + MovedAttribute("intern", "__builtin__", "sys"), + MovedAttribute("map", "itertools", "builtins", "imap", "map"), + MovedAttribute("getcwd", "os", "os", "getcwdu", "getcwd"), + MovedAttribute("getcwdb", "os", "os", "getcwd", "getcwdb"), + MovedAttribute("getoutput", "commands", "subprocess"), + MovedAttribute("range", "__builtin__", "builtins", "xrange", "range"), + MovedAttribute( + "reload_module", "__builtin__", "importlib" if PY34 else "imp", "reload" + ), + MovedAttribute("reduce", "__builtin__", "functools"), + MovedAttribute("shlex_quote", "pipes", "shlex", "quote"), + MovedAttribute("StringIO", "StringIO", "io"), + MovedAttribute("UserDict", "UserDict", "collections"), + MovedAttribute("UserList", "UserList", "collections"), + MovedAttribute("UserString", "UserString", "collections"), + MovedAttribute("xrange", "__builtin__", "builtins", "xrange", "range"), + MovedAttribute("zip", "itertools", "builtins", "izip", "zip"), + MovedAttribute( + "zip_longest", "itertools", "itertools", "izip_longest", "zip_longest" + ), + MovedModule("builtins", "__builtin__"), + MovedModule("configparser", "ConfigParser"), + MovedModule( + "collections_abc", + "collections", + "collections.abc" if sys.version_info >= (3, 3) else "collections", + ), + MovedModule("copyreg", "copy_reg"), + MovedModule("dbm_gnu", "gdbm", "dbm.gnu"), + MovedModule("dbm_ndbm", "dbm", "dbm.ndbm"), + MovedModule( + "_dummy_thread", + "dummy_thread", + "_dummy_thread" if sys.version_info < (3, 9) else "_thread", + ), + MovedModule("http_cookiejar", "cookielib", "http.cookiejar"), + MovedModule("http_cookies", "Cookie", "http.cookies"), + MovedModule("html_entities", "htmlentitydefs", "html.entities"), + MovedModule("html_parser", "HTMLParser", "html.parser"), + MovedModule("http_client", "httplib", "http.client"), + MovedModule("email_mime_base", "email.MIMEBase", "email.mime.base"), + MovedModule("email_mime_image", "email.MIMEImage", "email.mime.image"), + MovedModule("email_mime_multipart", "email.MIMEMultipart", "email.mime.multipart"), + MovedModule( + "email_mime_nonmultipart", "email.MIMENonMultipart", "email.mime.nonmultipart" + ), + MovedModule("email_mime_text", "email.MIMEText", "email.mime.text"), + MovedModule("BaseHTTPServer", "BaseHTTPServer", "http.server"), + MovedModule("CGIHTTPServer", "CGIHTTPServer", "http.server"), + MovedModule("SimpleHTTPServer", "SimpleHTTPServer", "http.server"), + MovedModule("cPickle", "cPickle", "pickle"), + MovedModule("queue", "Queue"), + MovedModule("reprlib", "repr"), + MovedModule("socketserver", "SocketServer"), + MovedModule("_thread", "thread", "_thread"), + MovedModule("tkinter", "Tkinter"), + MovedModule("tkinter_dialog", "Dialog", "tkinter.dialog"), + MovedModule("tkinter_filedialog", "FileDialog", "tkinter.filedialog"), + MovedModule("tkinter_scrolledtext", "ScrolledText", "tkinter.scrolledtext"), + MovedModule("tkinter_simpledialog", "SimpleDialog", "tkinter.simpledialog"), + MovedModule("tkinter_tix", "Tix", "tkinter.tix"), + MovedModule("tkinter_ttk", "ttk", "tkinter.ttk"), + MovedModule("tkinter_constants", "Tkconstants", "tkinter.constants"), + MovedModule("tkinter_dnd", "Tkdnd", "tkinter.dnd"), + MovedModule("tkinter_colorchooser", "tkColorChooser", "tkinter.colorchooser"), + MovedModule("tkinter_commondialog", "tkCommonDialog", "tkinter.commondialog"), + MovedModule("tkinter_tkfiledialog", "tkFileDialog", "tkinter.filedialog"), + MovedModule("tkinter_font", "tkFont", "tkinter.font"), + MovedModule("tkinter_messagebox", "tkMessageBox", "tkinter.messagebox"), + MovedModule("tkinter_tksimpledialog", "tkSimpleDialog", "tkinter.simpledialog"), + MovedModule("urllib_parse", __name__ + ".moves.urllib_parse", "urllib.parse"), + MovedModule("urllib_error", __name__ + ".moves.urllib_error", "urllib.error"), + MovedModule("urllib", __name__ + ".moves.urllib", __name__ + ".moves.urllib"), + MovedModule("urllib_robotparser", "robotparser", "urllib.robotparser"), + MovedModule("xmlrpc_client", "xmlrpclib", "xmlrpc.client"), + MovedModule("xmlrpc_server", "SimpleXMLRPCServer", "xmlrpc.server"), +] +# Add windows specific modules. +if sys.platform == "win32": + _moved_attributes += [ + MovedModule("winreg", "_winreg"), + ] + +for attr in _moved_attributes: + setattr(_MovedItems, attr.name, attr) + if isinstance(attr, MovedModule): + _importer._add_module(attr, "moves." + attr.name) +del attr + +_MovedItems._moved_attributes = _moved_attributes + +moves = _MovedItems(__name__ + ".moves") +_importer._add_module(moves, "moves") + + +class Module_six_moves_urllib_parse(_LazyModule): + + """Lazy loading of moved objects in six.moves.urllib_parse""" + + +_urllib_parse_moved_attributes = [ + MovedAttribute("ParseResult", "urlparse", "urllib.parse"), + MovedAttribute("SplitResult", "urlparse", "urllib.parse"), + MovedAttribute("parse_qs", "urlparse", "urllib.parse"), + MovedAttribute("parse_qsl", "urlparse", "urllib.parse"), + MovedAttribute("urldefrag", "urlparse", "urllib.parse"), + MovedAttribute("urljoin", "urlparse", "urllib.parse"), + MovedAttribute("urlparse", "urlparse", "urllib.parse"), + MovedAttribute("urlsplit", "urlparse", "urllib.parse"), + MovedAttribute("urlunparse", "urlparse", "urllib.parse"), + MovedAttribute("urlunsplit", "urlparse", "urllib.parse"), + MovedAttribute("quote", "urllib", "urllib.parse"), + MovedAttribute("quote_plus", "urllib", "urllib.parse"), + MovedAttribute("unquote", "urllib", "urllib.parse"), + MovedAttribute("unquote_plus", "urllib", "urllib.parse"), + MovedAttribute( + "unquote_to_bytes", "urllib", "urllib.parse", "unquote", "unquote_to_bytes" + ), + MovedAttribute("urlencode", "urllib", "urllib.parse"), + MovedAttribute("splitquery", "urllib", "urllib.parse"), + MovedAttribute("splittag", "urllib", "urllib.parse"), + MovedAttribute("splituser", "urllib", "urllib.parse"), + MovedAttribute("splitvalue", "urllib", "urllib.parse"), + MovedAttribute("uses_fragment", "urlparse", "urllib.parse"), + MovedAttribute("uses_netloc", "urlparse", "urllib.parse"), + MovedAttribute("uses_params", "urlparse", "urllib.parse"), + MovedAttribute("uses_query", "urlparse", "urllib.parse"), + MovedAttribute("uses_relative", "urlparse", "urllib.parse"), +] +for attr in _urllib_parse_moved_attributes: + setattr(Module_six_moves_urllib_parse, attr.name, attr) +del attr + +Module_six_moves_urllib_parse._moved_attributes = _urllib_parse_moved_attributes + +_importer._add_module( + Module_six_moves_urllib_parse(__name__ + ".moves.urllib_parse"), + "moves.urllib_parse", + "moves.urllib.parse", +) + + +class Module_six_moves_urllib_error(_LazyModule): + + """Lazy loading of moved objects in six.moves.urllib_error""" + + +_urllib_error_moved_attributes = [ + MovedAttribute("URLError", "urllib2", "urllib.error"), + MovedAttribute("HTTPError", "urllib2", "urllib.error"), + MovedAttribute("ContentTooShortError", "urllib", "urllib.error"), +] +for attr in _urllib_error_moved_attributes: + setattr(Module_six_moves_urllib_error, attr.name, attr) +del attr + +Module_six_moves_urllib_error._moved_attributes = _urllib_error_moved_attributes + +_importer._add_module( + Module_six_moves_urllib_error(__name__ + ".moves.urllib.error"), + "moves.urllib_error", + "moves.urllib.error", +) + + +class Module_six_moves_urllib_request(_LazyModule): + + """Lazy loading of moved objects in six.moves.urllib_request""" + + +_urllib_request_moved_attributes = [ + MovedAttribute("urlopen", "urllib2", "urllib.request"), + MovedAttribute("install_opener", "urllib2", "urllib.request"), + MovedAttribute("build_opener", "urllib2", "urllib.request"), + MovedAttribute("pathname2url", "urllib", "urllib.request"), + MovedAttribute("url2pathname", "urllib", "urllib.request"), + MovedAttribute("getproxies", "urllib", "urllib.request"), + MovedAttribute("Request", "urllib2", "urllib.request"), + MovedAttribute("OpenerDirector", "urllib2", "urllib.request"), + MovedAttribute("HTTPDefaultErrorHandler", "urllib2", "urllib.request"), + MovedAttribute("HTTPRedirectHandler", "urllib2", "urllib.request"), + MovedAttribute("HTTPCookieProcessor", "urllib2", "urllib.request"), + MovedAttribute("ProxyHandler", "urllib2", "urllib.request"), + MovedAttribute("BaseHandler", "urllib2", "urllib.request"), + MovedAttribute("HTTPPasswordMgr", "urllib2", "urllib.request"), + MovedAttribute("HTTPPasswordMgrWithDefaultRealm", "urllib2", "urllib.request"), + MovedAttribute("AbstractBasicAuthHandler", "urllib2", "urllib.request"), + MovedAttribute("HTTPBasicAuthHandler", "urllib2", "urllib.request"), + MovedAttribute("ProxyBasicAuthHandler", "urllib2", "urllib.request"), + MovedAttribute("AbstractDigestAuthHandler", "urllib2", "urllib.request"), + MovedAttribute("HTTPDigestAuthHandler", "urllib2", "urllib.request"), + MovedAttribute("ProxyDigestAuthHandler", "urllib2", "urllib.request"), + MovedAttribute("HTTPHandler", "urllib2", "urllib.request"), + MovedAttribute("HTTPSHandler", "urllib2", "urllib.request"), + MovedAttribute("FileHandler", "urllib2", "urllib.request"), + MovedAttribute("FTPHandler", "urllib2", "urllib.request"), + MovedAttribute("CacheFTPHandler", "urllib2", "urllib.request"), + MovedAttribute("UnknownHandler", "urllib2", "urllib.request"), + MovedAttribute("HTTPErrorProcessor", "urllib2", "urllib.request"), + MovedAttribute("urlretrieve", "urllib", "urllib.request"), + MovedAttribute("urlcleanup", "urllib", "urllib.request"), + MovedAttribute("URLopener", "urllib", "urllib.request"), + MovedAttribute("FancyURLopener", "urllib", "urllib.request"), + MovedAttribute("proxy_bypass", "urllib", "urllib.request"), + MovedAttribute("parse_http_list", "urllib2", "urllib.request"), + MovedAttribute("parse_keqv_list", "urllib2", "urllib.request"), +] +for attr in _urllib_request_moved_attributes: + setattr(Module_six_moves_urllib_request, attr.name, attr) +del attr + +Module_six_moves_urllib_request._moved_attributes = _urllib_request_moved_attributes + +_importer._add_module( + Module_six_moves_urllib_request(__name__ + ".moves.urllib.request"), + "moves.urllib_request", + "moves.urllib.request", +) + + +class Module_six_moves_urllib_response(_LazyModule): + + """Lazy loading of moved objects in six.moves.urllib_response""" + + +_urllib_response_moved_attributes = [ + MovedAttribute("addbase", "urllib", "urllib.response"), + MovedAttribute("addclosehook", "urllib", "urllib.response"), + MovedAttribute("addinfo", "urllib", "urllib.response"), + MovedAttribute("addinfourl", "urllib", "urllib.response"), +] +for attr in _urllib_response_moved_attributes: + setattr(Module_six_moves_urllib_response, attr.name, attr) +del attr + +Module_six_moves_urllib_response._moved_attributes = _urllib_response_moved_attributes + +_importer._add_module( + Module_six_moves_urllib_response(__name__ + ".moves.urllib.response"), + "moves.urllib_response", + "moves.urllib.response", +) + + +class Module_six_moves_urllib_robotparser(_LazyModule): + + """Lazy loading of moved objects in six.moves.urllib_robotparser""" + + +_urllib_robotparser_moved_attributes = [ + MovedAttribute("RobotFileParser", "robotparser", "urllib.robotparser"), +] +for attr in _urllib_robotparser_moved_attributes: + setattr(Module_six_moves_urllib_robotparser, attr.name, attr) +del attr + +Module_six_moves_urllib_robotparser._moved_attributes = ( + _urllib_robotparser_moved_attributes +) + +_importer._add_module( + Module_six_moves_urllib_robotparser(__name__ + ".moves.urllib.robotparser"), + "moves.urllib_robotparser", + "moves.urllib.robotparser", +) + + +class Module_six_moves_urllib(types.ModuleType): + + """Create a six.moves.urllib namespace that resembles the Python 3 namespace""" + + __path__ = [] # mark as package + parse = _importer._get_module("moves.urllib_parse") + error = _importer._get_module("moves.urllib_error") + request = _importer._get_module("moves.urllib_request") + response = _importer._get_module("moves.urllib_response") + robotparser = _importer._get_module("moves.urllib_robotparser") + + def __dir__(self): + return ["parse", "error", "request", "response", "robotparser"] + + +_importer._add_module( + Module_six_moves_urllib(__name__ + ".moves.urllib"), "moves.urllib" +) + + +def add_move(move): + """Add an item to six.moves.""" + setattr(_MovedItems, move.name, move) + + +def remove_move(name): + """Remove item from six.moves.""" + try: + delattr(_MovedItems, name) + except AttributeError: + try: + del moves.__dict__[name] + except KeyError: + raise AttributeError("no such move, %r" % (name,)) + + +if PY3: + _meth_func = "__func__" + _meth_self = "__self__" + + _func_closure = "__closure__" + _func_code = "__code__" + _func_defaults = "__defaults__" + _func_globals = "__globals__" +else: + _meth_func = "im_func" + _meth_self = "im_self" + + _func_closure = "func_closure" + _func_code = "func_code" + _func_defaults = "func_defaults" + _func_globals = "func_globals" + + +try: + advance_iterator = next +except NameError: + + def advance_iterator(it): + return it.next() + + +next = advance_iterator + + +try: + callable = callable +except NameError: + + def callable(obj): + return any("__call__" in klass.__dict__ for klass in type(obj).__mro__) + + +if PY3: + + def get_unbound_function(unbound): + return unbound + + create_bound_method = types.MethodType + + def create_unbound_method(func, cls): + return func + + Iterator = object +else: + + def get_unbound_function(unbound): + return unbound.im_func + + def create_bound_method(func, obj): + return types.MethodType(func, obj, obj.__class__) + + def create_unbound_method(func, cls): + return types.MethodType(func, None, cls) + + class Iterator(object): + def next(self): + return type(self).__next__(self) + + callable = callable +_add_doc( + get_unbound_function, """Get the function out of a possibly unbound function""" +) + + +get_method_function = operator.attrgetter(_meth_func) +get_method_self = operator.attrgetter(_meth_self) +get_function_closure = operator.attrgetter(_func_closure) +get_function_code = operator.attrgetter(_func_code) +get_function_defaults = operator.attrgetter(_func_defaults) +get_function_globals = operator.attrgetter(_func_globals) + + +if PY3: + + def iterkeys(d, **kw): + return iter(d.keys(**kw)) + + def itervalues(d, **kw): + return iter(d.values(**kw)) + + def iteritems(d, **kw): + return iter(d.items(**kw)) + + def iterlists(d, **kw): + return iter(d.lists(**kw)) + + viewkeys = operator.methodcaller("keys") + + viewvalues = operator.methodcaller("values") + + viewitems = operator.methodcaller("items") +else: + + def iterkeys(d, **kw): + return d.iterkeys(**kw) + + def itervalues(d, **kw): + return d.itervalues(**kw) + + def iteritems(d, **kw): + return d.iteritems(**kw) + + def iterlists(d, **kw): + return d.iterlists(**kw) + + viewkeys = operator.methodcaller("viewkeys") + + viewvalues = operator.methodcaller("viewvalues") + + viewitems = operator.methodcaller("viewitems") + +_add_doc(iterkeys, "Return an iterator over the keys of a dictionary.") +_add_doc(itervalues, "Return an iterator over the values of a dictionary.") +_add_doc(iteritems, "Return an iterator over the (key, value) pairs of a dictionary.") +_add_doc( + iterlists, "Return an iterator over the (key, [values]) pairs of a dictionary." +) + + +if PY3: + + def b(s): + return s.encode("latin-1") + + def u(s): + return s + + unichr = chr + import struct + + int2byte = struct.Struct(">B").pack + del struct + byte2int = operator.itemgetter(0) + indexbytes = operator.getitem + iterbytes = iter + import io + + StringIO = io.StringIO + BytesIO = io.BytesIO + del io + _assertCountEqual = "assertCountEqual" + if sys.version_info[1] <= 1: + _assertRaisesRegex = "assertRaisesRegexp" + _assertRegex = "assertRegexpMatches" + _assertNotRegex = "assertNotRegexpMatches" + else: + _assertRaisesRegex = "assertRaisesRegex" + _assertRegex = "assertRegex" + _assertNotRegex = "assertNotRegex" +else: + + def b(s): + return s + + # Workaround for standalone backslash + + def u(s): + return unicode(s.replace(r"\\", r"\\\\"), "unicode_escape") + + unichr = unichr + int2byte = chr + + def byte2int(bs): + return ord(bs[0]) + + def indexbytes(buf, i): + return ord(buf[i]) + + iterbytes = functools.partial(itertools.imap, ord) + import StringIO + + StringIO = BytesIO = StringIO.StringIO + _assertCountEqual = "assertItemsEqual" + _assertRaisesRegex = "assertRaisesRegexp" + _assertRegex = "assertRegexpMatches" + _assertNotRegex = "assertNotRegexpMatches" +_add_doc(b, """Byte literal""") +_add_doc(u, """Text literal""") + + +def assertCountEqual(self, *args, **kwargs): + return getattr(self, _assertCountEqual)(*args, **kwargs) + + +def assertRaisesRegex(self, *args, **kwargs): + return getattr(self, _assertRaisesRegex)(*args, **kwargs) + + +def assertRegex(self, *args, **kwargs): + return getattr(self, _assertRegex)(*args, **kwargs) + + +def assertNotRegex(self, *args, **kwargs): + return getattr(self, _assertNotRegex)(*args, **kwargs) + + +if PY3: + exec_ = getattr(moves.builtins, "exec") + + def reraise(tp, value, tb=None): + try: + if value is None: + value = tp() + if value.__traceback__ is not tb: + raise value.with_traceback(tb) + raise value + finally: + value = None + tb = None + + +else: + + def exec_(_code_, _globs_=None, _locs_=None): + """Execute code in a namespace.""" + if _globs_ is None: + frame = sys._getframe(1) + _globs_ = frame.f_globals + if _locs_ is None: + _locs_ = frame.f_locals + del frame + elif _locs_ is None: + _locs_ = _globs_ + exec ("""exec _code_ in _globs_, _locs_""") + + exec_( + """def reraise(tp, value, tb=None): + try: + raise tp, value, tb + finally: + tb = None +""" + ) + + +if sys.version_info[:2] > (3,): + exec_( + """def raise_from(value, from_value): + try: + raise value from from_value + finally: + value = None +""" + ) +else: + + def raise_from(value, from_value): + raise value + + +print_ = getattr(moves.builtins, "print", None) +if print_ is None: + + def print_(*args, **kwargs): + """The new-style print function for Python 2.4 and 2.5.""" + fp = kwargs.pop("file", sys.stdout) + if fp is None: + return + + def write(data): + if not isinstance(data, basestring): + data = str(data) + # If the file has an encoding, encode unicode with it. + if ( + isinstance(fp, file) + and isinstance(data, unicode) + and fp.encoding is not None + ): + errors = getattr(fp, "errors", None) + if errors is None: + errors = "strict" + data = data.encode(fp.encoding, errors) + fp.write(data) + + want_unicode = False + sep = kwargs.pop("sep", None) + if sep is not None: + if isinstance(sep, unicode): + want_unicode = True + elif not isinstance(sep, str): + raise TypeError("sep must be None or a string") + end = kwargs.pop("end", None) + if end is not None: + if isinstance(end, unicode): + want_unicode = True + elif not isinstance(end, str): + raise TypeError("end must be None or a string") + if kwargs: + raise TypeError("invalid keyword arguments to print()") + if not want_unicode: + for arg in args: + if isinstance(arg, unicode): + want_unicode = True + break + if want_unicode: + newline = unicode("\n") + space = unicode(" ") + else: + newline = "\n" + space = " " + if sep is None: + sep = space + if end is None: + end = newline + for i, arg in enumerate(args): + if i: + write(sep) + write(arg) + write(end) + + +if sys.version_info[:2] < (3, 3): + _print = print_ + + def print_(*args, **kwargs): + fp = kwargs.get("file", sys.stdout) + flush = kwargs.pop("flush", False) + _print(*args, **kwargs) + if flush and fp is not None: + fp.flush() + + +_add_doc(reraise, """Reraise an exception.""") + +if sys.version_info[0:2] < (3, 4): + # This does exactly the same what the :func:`py3:functools.update_wrapper` + # function does on Python versions after 3.2. It sets the ``__wrapped__`` + # attribute on ``wrapper`` object and it doesn't raise an error if any of + # the attributes mentioned in ``assigned`` and ``updated`` are missing on + # ``wrapped`` object. + def _update_wrapper( + wrapper, + wrapped, + assigned=functools.WRAPPER_ASSIGNMENTS, + updated=functools.WRAPPER_UPDATES, + ): + for attr in assigned: + try: + value = getattr(wrapped, attr) + except AttributeError: + continue + else: + setattr(wrapper, attr, value) + for attr in updated: + getattr(wrapper, attr).update(getattr(wrapped, attr, {})) + wrapper.__wrapped__ = wrapped + return wrapper + + _update_wrapper.__doc__ = functools.update_wrapper.__doc__ + + def wraps( + wrapped, + assigned=functools.WRAPPER_ASSIGNMENTS, + updated=functools.WRAPPER_UPDATES, + ): + return functools.partial( + _update_wrapper, wrapped=wrapped, assigned=assigned, updated=updated + ) + + wraps.__doc__ = functools.wraps.__doc__ + +else: + wraps = functools.wraps + + +def with_metaclass(meta, *bases): + """Create a base class with a metaclass.""" + # This requires a bit of explanation: the basic idea is to make a dummy + # metaclass for one level of class instantiation that replaces itself with + # the actual metaclass. + class metaclass(type): + def __new__(cls, name, this_bases, d): + if sys.version_info[:2] >= (3, 7): + # This version introduced PEP 560 that requires a bit + # of extra care (we mimic what is done by __build_class__). + resolved_bases = types.resolve_bases(bases) + if resolved_bases is not bases: + d["__orig_bases__"] = bases + else: + resolved_bases = bases + return meta(name, resolved_bases, d) + + @classmethod + def __prepare__(cls, name, this_bases): + return meta.__prepare__(name, bases) + + return type.__new__(metaclass, "temporary_class", (), {}) + + +def add_metaclass(metaclass): + """Class decorator for creating a class with a metaclass.""" + + def wrapper(cls): + orig_vars = cls.__dict__.copy() + slots = orig_vars.get("__slots__") + if slots is not None: + if isinstance(slots, str): + slots = [slots] + for slots_var in slots: + orig_vars.pop(slots_var) + orig_vars.pop("__dict__", None) + orig_vars.pop("__weakref__", None) + if hasattr(cls, "__qualname__"): + orig_vars["__qualname__"] = cls.__qualname__ + return metaclass(cls.__name__, cls.__bases__, orig_vars) + + return wrapper + + +def ensure_binary(s, encoding="utf-8", errors="strict"): + """Coerce **s** to six.binary_type. + + For Python 2: + - `unicode` -> encoded to `str` + - `str` -> `str` + + For Python 3: + - `str` -> encoded to `bytes` + - `bytes` -> `bytes` + """ + if isinstance(s, binary_type): + return s + if isinstance(s, text_type): + return s.encode(encoding, errors) + raise TypeError("not expecting type '%s'" % type(s)) + + +def ensure_str(s, encoding="utf-8", errors="strict"): + """Coerce *s* to `str`. + + For Python 2: + - `unicode` -> encoded to `str` + - `str` -> `str` + + For Python 3: + - `str` -> `str` + - `bytes` -> decoded to `str` + """ + # Optimization: Fast return for the common case. + if type(s) is str: + return s + if PY2 and isinstance(s, text_type): + return s.encode(encoding, errors) + elif PY3 and isinstance(s, binary_type): + return s.decode(encoding, errors) + elif not isinstance(s, (text_type, binary_type)): + raise TypeError("not expecting type '%s'" % type(s)) + return s + + +def ensure_text(s, encoding="utf-8", errors="strict"): + """Coerce *s* to six.text_type. + + For Python 2: + - `unicode` -> `unicode` + - `str` -> `unicode` + + For Python 3: + - `str` -> `str` + - `bytes` -> decoded to `str` + """ + if isinstance(s, binary_type): + return s.decode(encoding, errors) + elif isinstance(s, text_type): + return s + else: + raise TypeError("not expecting type '%s'" % type(s)) + + +def python_2_unicode_compatible(klass): + """ + A class decorator that defines __unicode__ and __str__ methods under Python 2. + Under Python 3 it does nothing. + + To support Python 2 and 3 with a single code base, define a __str__ method + returning text and apply this decorator to the class. + """ + if PY2: + if "__str__" not in klass.__dict__: + raise ValueError( + "@python_2_unicode_compatible cannot be applied " + "to %s because it doesn't define __str__()." % klass.__name__ + ) + klass.__unicode__ = klass.__str__ + klass.__str__ = lambda self: self.__unicode__().encode("utf-8") + return klass + + +# Complete the moves implementation. +# This code is at the end of this module to speed up module loading. +# Turn this module into a package. +__path__ = [] # required for PEP 302 and PEP 451 +__package__ = __name__ # see PEP 366 @ReservedAssignment +if globals().get("__spec__") is not None: + __spec__.submodule_search_locations = [] # PEP 451 @UndefinedVariable +# Remove other six meta path importers, since they cause problems. This can +# happen if six is removed from sys.modules and then reloaded. (Setuptools does +# this for some reason.) +if sys.meta_path: + for i, importer in enumerate(sys.meta_path): + # Here's some real nastiness: Another "instance" of the six module might + # be floating around. Therefore, we can't use isinstance() to check for + # the six meta path importer, since the other six instance will have + # inserted an importer with different class. + if ( + type(importer).__name__ == "_SixMetaPathImporter" + and importer.name == __name__ + ): + del sys.meta_path[i] + break + del i, importer +# Finally, add the importer to the meta path import hook. +sys.meta_path.append(_importer) diff --git a/openpype/hosts/fusion/vendor/urllib3/packages/ssl_match_hostname/__init__.py b/openpype/hosts/fusion/vendor/urllib3/packages/ssl_match_hostname/__init__.py new file mode 100644 index 0000000000..ef3fde5206 --- /dev/null +++ b/openpype/hosts/fusion/vendor/urllib3/packages/ssl_match_hostname/__init__.py @@ -0,0 +1,24 @@ +import sys + +try: + # Our match_hostname function is the same as 3.10's, so we only want to + # import the match_hostname function if it's at least that good. + # We also fallback on Python 3.10+ because our code doesn't emit + # deprecation warnings and is the same as Python 3.10 otherwise. + if sys.version_info < (3, 5) or sys.version_info >= (3, 10): + raise ImportError("Fallback to vendored code") + + from ssl import CertificateError, match_hostname +except ImportError: + try: + # Backport of the function from a pypi module + from backports.ssl_match_hostname import ( # type: ignore + CertificateError, + match_hostname, + ) + except ImportError: + # Our vendored copy + from ._implementation import CertificateError, match_hostname # type: ignore + +# Not needed, but documenting what we provide. +__all__ = ("CertificateError", "match_hostname") diff --git a/openpype/hosts/fusion/vendor/urllib3/packages/ssl_match_hostname/_implementation.py b/openpype/hosts/fusion/vendor/urllib3/packages/ssl_match_hostname/_implementation.py new file mode 100644 index 0000000000..689208d3c6 --- /dev/null +++ b/openpype/hosts/fusion/vendor/urllib3/packages/ssl_match_hostname/_implementation.py @@ -0,0 +1,160 @@ +"""The match_hostname() function from Python 3.3.3, essential when using SSL.""" + +# Note: This file is under the PSF license as the code comes from the python +# stdlib. http://docs.python.org/3/license.html + +import re +import sys + +# ipaddress has been backported to 2.6+ in pypi. If it is installed on the +# system, use it to handle IPAddress ServerAltnames (this was added in +# python-3.5) otherwise only do DNS matching. This allows +# backports.ssl_match_hostname to continue to be used in Python 2.7. +try: + import ipaddress +except ImportError: + ipaddress = None + +__version__ = "3.5.0.1" + + +class CertificateError(ValueError): + pass + + +def _dnsname_match(dn, hostname, max_wildcards=1): + """Matching according to RFC 6125, section 6.4.3 + + http://tools.ietf.org/html/rfc6125#section-6.4.3 + """ + pats = [] + if not dn: + return False + + # Ported from python3-syntax: + # leftmost, *remainder = dn.split(r'.') + parts = dn.split(r".") + leftmost = parts[0] + remainder = parts[1:] + + wildcards = leftmost.count("*") + if wildcards > max_wildcards: + # Issue #17980: avoid denials of service by refusing more + # than one wildcard per fragment. A survey of established + # policy among SSL implementations showed it to be a + # reasonable choice. + raise CertificateError( + "too many wildcards in certificate DNS name: " + repr(dn) + ) + + # speed up common case w/o wildcards + if not wildcards: + return dn.lower() == hostname.lower() + + # RFC 6125, section 6.4.3, subitem 1. + # The client SHOULD NOT attempt to match a presented identifier in which + # the wildcard character comprises a label other than the left-most label. + if leftmost == "*": + # When '*' is a fragment by itself, it matches a non-empty dotless + # fragment. + pats.append("[^.]+") + elif leftmost.startswith("xn--") or hostname.startswith("xn--"): + # RFC 6125, section 6.4.3, subitem 3. + # The client SHOULD NOT attempt to match a presented identifier + # where the wildcard character is embedded within an A-label or + # U-label of an internationalized domain name. + pats.append(re.escape(leftmost)) + else: + # Otherwise, '*' matches any dotless string, e.g. www* + pats.append(re.escape(leftmost).replace(r"\*", "[^.]*")) + + # add the remaining fragments, ignore any wildcards + for frag in remainder: + pats.append(re.escape(frag)) + + pat = re.compile(r"\A" + r"\.".join(pats) + r"\Z", re.IGNORECASE) + return pat.match(hostname) + + +def _to_unicode(obj): + if isinstance(obj, str) and sys.version_info < (3,): + obj = unicode(obj, encoding="ascii", errors="strict") + return obj + + +def _ipaddress_match(ipname, host_ip): + """Exact matching of IP addresses. + + RFC 6125 explicitly doesn't define an algorithm for this + (section 1.7.2 - "Out of Scope"). + """ + # OpenSSL may add a trailing newline to a subjectAltName's IP address + # Divergence from upstream: ipaddress can't handle byte str + ip = ipaddress.ip_address(_to_unicode(ipname).rstrip()) + return ip == host_ip + + +def match_hostname(cert, hostname): + """Verify that *cert* (in decoded format as returned by + SSLSocket.getpeercert()) matches the *hostname*. RFC 2818 and RFC 6125 + rules are followed, but IP addresses are not accepted for *hostname*. + + CertificateError is raised on failure. On success, the function + returns nothing. + """ + if not cert: + raise ValueError( + "empty or no certificate, match_hostname needs a " + "SSL socket or SSL context with either " + "CERT_OPTIONAL or CERT_REQUIRED" + ) + try: + # Divergence from upstream: ipaddress can't handle byte str + host_ip = ipaddress.ip_address(_to_unicode(hostname)) + except ValueError: + # Not an IP address (common case) + host_ip = None + except UnicodeError: + # Divergence from upstream: Have to deal with ipaddress not taking + # byte strings. addresses should be all ascii, so we consider it not + # an ipaddress in this case + host_ip = None + except AttributeError: + # Divergence from upstream: Make ipaddress library optional + if ipaddress is None: + host_ip = None + else: + raise + dnsnames = [] + san = cert.get("subjectAltName", ()) + for key, value in san: + if key == "DNS": + if host_ip is None and _dnsname_match(value, hostname): + return + dnsnames.append(value) + elif key == "IP Address": + if host_ip is not None and _ipaddress_match(value, host_ip): + return + dnsnames.append(value) + if not dnsnames: + # The subject is only checked when there is no dNSName entry + # in subjectAltName + for sub in cert.get("subject", ()): + for key, value in sub: + # XXX according to RFC 2818, the most specific Common Name + # must be used. + if key == "commonName": + if _dnsname_match(value, hostname): + return + dnsnames.append(value) + if len(dnsnames) > 1: + raise CertificateError( + "hostname %r " + "doesn't match either of %s" % (hostname, ", ".join(map(repr, dnsnames))) + ) + elif len(dnsnames) == 1: + raise CertificateError("hostname %r doesn't match %r" % (hostname, dnsnames[0])) + else: + raise CertificateError( + "no appropriate commonName or subjectAltName fields were found" + ) diff --git a/openpype/hosts/fusion/vendor/urllib3/poolmanager.py b/openpype/hosts/fusion/vendor/urllib3/poolmanager.py new file mode 100644 index 0000000000..3a31a285bf --- /dev/null +++ b/openpype/hosts/fusion/vendor/urllib3/poolmanager.py @@ -0,0 +1,536 @@ +from __future__ import absolute_import + +import collections +import functools +import logging + +from ._collections import RecentlyUsedContainer +from .connectionpool import HTTPConnectionPool, HTTPSConnectionPool, port_by_scheme +from .exceptions import ( + LocationValueError, + MaxRetryError, + ProxySchemeUnknown, + ProxySchemeUnsupported, + URLSchemeUnknown, +) +from .packages import six +from .packages.six.moves.urllib.parse import urljoin +from .request import RequestMethods +from .util.proxy import connection_requires_http_tunnel +from .util.retry import Retry +from .util.url import parse_url + +__all__ = ["PoolManager", "ProxyManager", "proxy_from_url"] + + +log = logging.getLogger(__name__) + +SSL_KEYWORDS = ( + "key_file", + "cert_file", + "cert_reqs", + "ca_certs", + "ssl_version", + "ca_cert_dir", + "ssl_context", + "key_password", +) + +# All known keyword arguments that could be provided to the pool manager, its +# pools, or the underlying connections. This is used to construct a pool key. +_key_fields = ( + "key_scheme", # str + "key_host", # str + "key_port", # int + "key_timeout", # int or float or Timeout + "key_retries", # int or Retry + "key_strict", # bool + "key_block", # bool + "key_source_address", # str + "key_key_file", # str + "key_key_password", # str + "key_cert_file", # str + "key_cert_reqs", # str + "key_ca_certs", # str + "key_ssl_version", # str + "key_ca_cert_dir", # str + "key_ssl_context", # instance of ssl.SSLContext or urllib3.util.ssl_.SSLContext + "key_maxsize", # int + "key_headers", # dict + "key__proxy", # parsed proxy url + "key__proxy_headers", # dict + "key__proxy_config", # class + "key_socket_options", # list of (level (int), optname (int), value (int or str)) tuples + "key__socks_options", # dict + "key_assert_hostname", # bool or string + "key_assert_fingerprint", # str + "key_server_hostname", # str +) + +#: The namedtuple class used to construct keys for the connection pool. +#: All custom key schemes should include the fields in this key at a minimum. +PoolKey = collections.namedtuple("PoolKey", _key_fields) + +_proxy_config_fields = ("ssl_context", "use_forwarding_for_https") +ProxyConfig = collections.namedtuple("ProxyConfig", _proxy_config_fields) + + +def _default_key_normalizer(key_class, request_context): + """ + Create a pool key out of a request context dictionary. + + According to RFC 3986, both the scheme and host are case-insensitive. + Therefore, this function normalizes both before constructing the pool + key for an HTTPS request. If you wish to change this behaviour, provide + alternate callables to ``key_fn_by_scheme``. + + :param key_class: + The class to use when constructing the key. This should be a namedtuple + with the ``scheme`` and ``host`` keys at a minimum. + :type key_class: namedtuple + :param request_context: + A dictionary-like object that contain the context for a request. + :type request_context: dict + + :return: A namedtuple that can be used as a connection pool key. + :rtype: PoolKey + """ + # Since we mutate the dictionary, make a copy first + context = request_context.copy() + context["scheme"] = context["scheme"].lower() + context["host"] = context["host"].lower() + + # These are both dictionaries and need to be transformed into frozensets + for key in ("headers", "_proxy_headers", "_socks_options"): + if key in context and context[key] is not None: + context[key] = frozenset(context[key].items()) + + # The socket_options key may be a list and needs to be transformed into a + # tuple. + socket_opts = context.get("socket_options") + if socket_opts is not None: + context["socket_options"] = tuple(socket_opts) + + # Map the kwargs to the names in the namedtuple - this is necessary since + # namedtuples can't have fields starting with '_'. + for key in list(context.keys()): + context["key_" + key] = context.pop(key) + + # Default to ``None`` for keys missing from the context + for field in key_class._fields: + if field not in context: + context[field] = None + + return key_class(**context) + + +#: A dictionary that maps a scheme to a callable that creates a pool key. +#: This can be used to alter the way pool keys are constructed, if desired. +#: Each PoolManager makes a copy of this dictionary so they can be configured +#: globally here, or individually on the instance. +key_fn_by_scheme = { + "http": functools.partial(_default_key_normalizer, PoolKey), + "https": functools.partial(_default_key_normalizer, PoolKey), +} + +pool_classes_by_scheme = {"http": HTTPConnectionPool, "https": HTTPSConnectionPool} + + +class PoolManager(RequestMethods): + """ + Allows for arbitrary requests while transparently keeping track of + necessary connection pools for you. + + :param num_pools: + Number of connection pools to cache before discarding the least + recently used pool. + + :param headers: + Headers to include with all requests, unless other headers are given + explicitly. + + :param \\**connection_pool_kw: + Additional parameters are used to create fresh + :class:`urllib3.connectionpool.ConnectionPool` instances. + + Example:: + + >>> manager = PoolManager(num_pools=2) + >>> r = manager.request('GET', 'http://google.com/') + >>> r = manager.request('GET', 'http://google.com/mail') + >>> r = manager.request('GET', 'http://yahoo.com/') + >>> len(manager.pools) + 2 + + """ + + proxy = None + proxy_config = None + + def __init__(self, num_pools=10, headers=None, **connection_pool_kw): + RequestMethods.__init__(self, headers) + self.connection_pool_kw = connection_pool_kw + self.pools = RecentlyUsedContainer(num_pools, dispose_func=lambda p: p.close()) + + # Locally set the pool classes and keys so other PoolManagers can + # override them. + self.pool_classes_by_scheme = pool_classes_by_scheme + self.key_fn_by_scheme = key_fn_by_scheme.copy() + + def __enter__(self): + return self + + def __exit__(self, exc_type, exc_val, exc_tb): + self.clear() + # Return False to re-raise any potential exceptions + return False + + def _new_pool(self, scheme, host, port, request_context=None): + """ + Create a new :class:`urllib3.connectionpool.ConnectionPool` based on host, port, scheme, and + any additional pool keyword arguments. + + If ``request_context`` is provided, it is provided as keyword arguments + to the pool class used. This method is used to actually create the + connection pools handed out by :meth:`connection_from_url` and + companion methods. It is intended to be overridden for customization. + """ + pool_cls = self.pool_classes_by_scheme[scheme] + if request_context is None: + request_context = self.connection_pool_kw.copy() + + # Although the context has everything necessary to create the pool, + # this function has historically only used the scheme, host, and port + # in the positional args. When an API change is acceptable these can + # be removed. + for key in ("scheme", "host", "port"): + request_context.pop(key, None) + + if scheme == "http": + for kw in SSL_KEYWORDS: + request_context.pop(kw, None) + + return pool_cls(host, port, **request_context) + + def clear(self): + """ + Empty our store of pools and direct them all to close. + + This will not affect in-flight connections, but they will not be + re-used after completion. + """ + self.pools.clear() + + def connection_from_host(self, host, port=None, scheme="http", pool_kwargs=None): + """ + Get a :class:`urllib3.connectionpool.ConnectionPool` based on the host, port, and scheme. + + If ``port`` isn't given, it will be derived from the ``scheme`` using + ``urllib3.connectionpool.port_by_scheme``. If ``pool_kwargs`` is + provided, it is merged with the instance's ``connection_pool_kw`` + variable and used to create the new connection pool, if one is + needed. + """ + + if not host: + raise LocationValueError("No host specified.") + + request_context = self._merge_pool_kwargs(pool_kwargs) + request_context["scheme"] = scheme or "http" + if not port: + port = port_by_scheme.get(request_context["scheme"].lower(), 80) + request_context["port"] = port + request_context["host"] = host + + return self.connection_from_context(request_context) + + def connection_from_context(self, request_context): + """ + Get a :class:`urllib3.connectionpool.ConnectionPool` based on the request context. + + ``request_context`` must at least contain the ``scheme`` key and its + value must be a key in ``key_fn_by_scheme`` instance variable. + """ + scheme = request_context["scheme"].lower() + pool_key_constructor = self.key_fn_by_scheme.get(scheme) + if not pool_key_constructor: + raise URLSchemeUnknown(scheme) + pool_key = pool_key_constructor(request_context) + + return self.connection_from_pool_key(pool_key, request_context=request_context) + + def connection_from_pool_key(self, pool_key, request_context=None): + """ + Get a :class:`urllib3.connectionpool.ConnectionPool` based on the provided pool key. + + ``pool_key`` should be a namedtuple that only contains immutable + objects. At a minimum it must have the ``scheme``, ``host``, and + ``port`` fields. + """ + with self.pools.lock: + # If the scheme, host, or port doesn't match existing open + # connections, open a new ConnectionPool. + pool = self.pools.get(pool_key) + if pool: + return pool + + # Make a fresh ConnectionPool of the desired type + scheme = request_context["scheme"] + host = request_context["host"] + port = request_context["port"] + pool = self._new_pool(scheme, host, port, request_context=request_context) + self.pools[pool_key] = pool + + return pool + + def connection_from_url(self, url, pool_kwargs=None): + """ + Similar to :func:`urllib3.connectionpool.connection_from_url`. + + If ``pool_kwargs`` is not provided and a new pool needs to be + constructed, ``self.connection_pool_kw`` is used to initialize + the :class:`urllib3.connectionpool.ConnectionPool`. If ``pool_kwargs`` + is provided, it is used instead. Note that if a new pool does not + need to be created for the request, the provided ``pool_kwargs`` are + not used. + """ + u = parse_url(url) + return self.connection_from_host( + u.host, port=u.port, scheme=u.scheme, pool_kwargs=pool_kwargs + ) + + def _merge_pool_kwargs(self, override): + """ + Merge a dictionary of override values for self.connection_pool_kw. + + This does not modify self.connection_pool_kw and returns a new dict. + Any keys in the override dictionary with a value of ``None`` are + removed from the merged dictionary. + """ + base_pool_kwargs = self.connection_pool_kw.copy() + if override: + for key, value in override.items(): + if value is None: + try: + del base_pool_kwargs[key] + except KeyError: + pass + else: + base_pool_kwargs[key] = value + return base_pool_kwargs + + def _proxy_requires_url_absolute_form(self, parsed_url): + """ + Indicates if the proxy requires the complete destination URL in the + request. Normally this is only needed when not using an HTTP CONNECT + tunnel. + """ + if self.proxy is None: + return False + + return not connection_requires_http_tunnel( + self.proxy, self.proxy_config, parsed_url.scheme + ) + + def _validate_proxy_scheme_url_selection(self, url_scheme): + """ + Validates that were not attempting to do TLS in TLS connections on + Python2 or with unsupported SSL implementations. + """ + if self.proxy is None or url_scheme != "https": + return + + if self.proxy.scheme != "https": + return + + if six.PY2 and not self.proxy_config.use_forwarding_for_https: + raise ProxySchemeUnsupported( + "Contacting HTTPS destinations through HTTPS proxies " + "'via CONNECT tunnels' is not supported in Python 2" + ) + + def urlopen(self, method, url, redirect=True, **kw): + """ + Same as :meth:`urllib3.HTTPConnectionPool.urlopen` + with custom cross-host redirect logic and only sends the request-uri + portion of the ``url``. + + The given ``url`` parameter must be absolute, such that an appropriate + :class:`urllib3.connectionpool.ConnectionPool` can be chosen for it. + """ + u = parse_url(url) + self._validate_proxy_scheme_url_selection(u.scheme) + + conn = self.connection_from_host(u.host, port=u.port, scheme=u.scheme) + + kw["assert_same_host"] = False + kw["redirect"] = False + + if "headers" not in kw: + kw["headers"] = self.headers.copy() + + if self._proxy_requires_url_absolute_form(u): + response = conn.urlopen(method, url, **kw) + else: + response = conn.urlopen(method, u.request_uri, **kw) + + redirect_location = redirect and response.get_redirect_location() + if not redirect_location: + return response + + # Support relative URLs for redirecting. + redirect_location = urljoin(url, redirect_location) + + # RFC 7231, Section 6.4.4 + if response.status == 303: + method = "GET" + + retries = kw.get("retries") + if not isinstance(retries, Retry): + retries = Retry.from_int(retries, redirect=redirect) + + # Strip headers marked as unsafe to forward to the redirected location. + # Check remove_headers_on_redirect to avoid a potential network call within + # conn.is_same_host() which may use socket.gethostbyname() in the future. + if retries.remove_headers_on_redirect and not conn.is_same_host( + redirect_location + ): + headers = list(six.iterkeys(kw["headers"])) + for header in headers: + if header.lower() in retries.remove_headers_on_redirect: + kw["headers"].pop(header, None) + + try: + retries = retries.increment(method, url, response=response, _pool=conn) + except MaxRetryError: + if retries.raise_on_redirect: + response.drain_conn() + raise + return response + + kw["retries"] = retries + kw["redirect"] = redirect + + log.info("Redirecting %s -> %s", url, redirect_location) + + response.drain_conn() + return self.urlopen(method, redirect_location, **kw) + + +class ProxyManager(PoolManager): + """ + Behaves just like :class:`PoolManager`, but sends all requests through + the defined proxy, using the CONNECT method for HTTPS URLs. + + :param proxy_url: + The URL of the proxy to be used. + + :param proxy_headers: + A dictionary containing headers that will be sent to the proxy. In case + of HTTP they are being sent with each request, while in the + HTTPS/CONNECT case they are sent only once. Could be used for proxy + authentication. + + :param proxy_ssl_context: + The proxy SSL context is used to establish the TLS connection to the + proxy when using HTTPS proxies. + + :param use_forwarding_for_https: + (Defaults to False) If set to True will forward requests to the HTTPS + proxy to be made on behalf of the client instead of creating a TLS + tunnel via the CONNECT method. **Enabling this flag means that request + and response headers and content will be visible from the HTTPS proxy** + whereas tunneling keeps request and response headers and content + private. IP address, target hostname, SNI, and port are always visible + to an HTTPS proxy even when this flag is disabled. + + Example: + >>> proxy = urllib3.ProxyManager('http://localhost:3128/') + >>> r1 = proxy.request('GET', 'http://google.com/') + >>> r2 = proxy.request('GET', 'http://httpbin.org/') + >>> len(proxy.pools) + 1 + >>> r3 = proxy.request('GET', 'https://httpbin.org/') + >>> r4 = proxy.request('GET', 'https://twitter.com/') + >>> len(proxy.pools) + 3 + + """ + + def __init__( + self, + proxy_url, + num_pools=10, + headers=None, + proxy_headers=None, + proxy_ssl_context=None, + use_forwarding_for_https=False, + **connection_pool_kw + ): + + if isinstance(proxy_url, HTTPConnectionPool): + proxy_url = "%s://%s:%i" % ( + proxy_url.scheme, + proxy_url.host, + proxy_url.port, + ) + proxy = parse_url(proxy_url) + + if proxy.scheme not in ("http", "https"): + raise ProxySchemeUnknown(proxy.scheme) + + if not proxy.port: + port = port_by_scheme.get(proxy.scheme, 80) + proxy = proxy._replace(port=port) + + self.proxy = proxy + self.proxy_headers = proxy_headers or {} + self.proxy_ssl_context = proxy_ssl_context + self.proxy_config = ProxyConfig(proxy_ssl_context, use_forwarding_for_https) + + connection_pool_kw["_proxy"] = self.proxy + connection_pool_kw["_proxy_headers"] = self.proxy_headers + connection_pool_kw["_proxy_config"] = self.proxy_config + + super(ProxyManager, self).__init__(num_pools, headers, **connection_pool_kw) + + def connection_from_host(self, host, port=None, scheme="http", pool_kwargs=None): + if scheme == "https": + return super(ProxyManager, self).connection_from_host( + host, port, scheme, pool_kwargs=pool_kwargs + ) + + return super(ProxyManager, self).connection_from_host( + self.proxy.host, self.proxy.port, self.proxy.scheme, pool_kwargs=pool_kwargs + ) + + def _set_proxy_headers(self, url, headers=None): + """ + Sets headers needed by proxies: specifically, the Accept and Host + headers. Only sets headers not provided by the user. + """ + headers_ = {"Accept": "*/*"} + + netloc = parse_url(url).netloc + if netloc: + headers_["Host"] = netloc + + if headers: + headers_.update(headers) + return headers_ + + def urlopen(self, method, url, redirect=True, **kw): + "Same as HTTP(S)ConnectionPool.urlopen, ``url`` must be absolute." + u = parse_url(url) + if not connection_requires_http_tunnel(self.proxy, self.proxy_config, u.scheme): + # For connections using HTTP CONNECT, httplib sets the necessary + # headers on the CONNECT to the proxy. If we're not using CONNECT, + # we'll definitely need to set 'Host' at the very least. + headers = kw.get("headers", self.headers) + kw["headers"] = self._set_proxy_headers(url, headers) + + return super(ProxyManager, self).urlopen(method, url, redirect=redirect, **kw) + + +def proxy_from_url(url, **kw): + return ProxyManager(proxy_url=url, **kw) diff --git a/openpype/hosts/fusion/vendor/urllib3/request.py b/openpype/hosts/fusion/vendor/urllib3/request.py new file mode 100644 index 0000000000..398386a5b9 --- /dev/null +++ b/openpype/hosts/fusion/vendor/urllib3/request.py @@ -0,0 +1,170 @@ +from __future__ import absolute_import + +from .filepost import encode_multipart_formdata +from .packages.six.moves.urllib.parse import urlencode + +__all__ = ["RequestMethods"] + + +class RequestMethods(object): + """ + Convenience mixin for classes who implement a :meth:`urlopen` method, such + as :class:`urllib3.HTTPConnectionPool` and + :class:`urllib3.PoolManager`. + + Provides behavior for making common types of HTTP request methods and + decides which type of request field encoding to use. + + Specifically, + + :meth:`.request_encode_url` is for sending requests whose fields are + encoded in the URL (such as GET, HEAD, DELETE). + + :meth:`.request_encode_body` is for sending requests whose fields are + encoded in the *body* of the request using multipart or www-form-urlencoded + (such as for POST, PUT, PATCH). + + :meth:`.request` is for making any kind of request, it will look up the + appropriate encoding format and use one of the above two methods to make + the request. + + Initializer parameters: + + :param headers: + Headers to include with all requests, unless other headers are given + explicitly. + """ + + _encode_url_methods = {"DELETE", "GET", "HEAD", "OPTIONS"} + + def __init__(self, headers=None): + self.headers = headers or {} + + def urlopen( + self, + method, + url, + body=None, + headers=None, + encode_multipart=True, + multipart_boundary=None, + **kw + ): # Abstract + raise NotImplementedError( + "Classes extending RequestMethods must implement " + "their own ``urlopen`` method." + ) + + def request(self, method, url, fields=None, headers=None, **urlopen_kw): + """ + Make a request using :meth:`urlopen` with the appropriate encoding of + ``fields`` based on the ``method`` used. + + This is a convenience method that requires the least amount of manual + effort. It can be used in most situations, while still having the + option to drop down to more specific methods when necessary, such as + :meth:`request_encode_url`, :meth:`request_encode_body`, + or even the lowest level :meth:`urlopen`. + """ + method = method.upper() + + urlopen_kw["request_url"] = url + + if method in self._encode_url_methods: + return self.request_encode_url( + method, url, fields=fields, headers=headers, **urlopen_kw + ) + else: + return self.request_encode_body( + method, url, fields=fields, headers=headers, **urlopen_kw + ) + + def request_encode_url(self, method, url, fields=None, headers=None, **urlopen_kw): + """ + Make a request using :meth:`urlopen` with the ``fields`` encoded in + the url. This is useful for request methods like GET, HEAD, DELETE, etc. + """ + if headers is None: + headers = self.headers + + extra_kw = {"headers": headers} + extra_kw.update(urlopen_kw) + + if fields: + url += "?" + urlencode(fields) + + return self.urlopen(method, url, **extra_kw) + + def request_encode_body( + self, + method, + url, + fields=None, + headers=None, + encode_multipart=True, + multipart_boundary=None, + **urlopen_kw + ): + """ + Make a request using :meth:`urlopen` with the ``fields`` encoded in + the body. This is useful for request methods like POST, PUT, PATCH, etc. + + When ``encode_multipart=True`` (default), then + :func:`urllib3.encode_multipart_formdata` is used to encode + the payload with the appropriate content type. Otherwise + :func:`urllib.parse.urlencode` is used with the + 'application/x-www-form-urlencoded' content type. + + Multipart encoding must be used when posting files, and it's reasonably + safe to use it in other times too. However, it may break request + signing, such as with OAuth. + + Supports an optional ``fields`` parameter of key/value strings AND + key/filetuple. A filetuple is a (filename, data, MIME type) tuple where + the MIME type is optional. For example:: + + fields = { + 'foo': 'bar', + 'fakefile': ('foofile.txt', 'contents of foofile'), + 'realfile': ('barfile.txt', open('realfile').read()), + 'typedfile': ('bazfile.bin', open('bazfile').read(), + 'image/jpeg'), + 'nonamefile': 'contents of nonamefile field', + } + + When uploading a file, providing a filename (the first parameter of the + tuple) is optional but recommended to best mimic behavior of browsers. + + Note that if ``headers`` are supplied, the 'Content-Type' header will + be overwritten because it depends on the dynamic random boundary string + which is used to compose the body of the request. The random boundary + string can be explicitly set with the ``multipart_boundary`` parameter. + """ + if headers is None: + headers = self.headers + + extra_kw = {"headers": {}} + + if fields: + if "body" in urlopen_kw: + raise TypeError( + "request got values for both 'fields' and 'body', can only specify one." + ) + + if encode_multipart: + body, content_type = encode_multipart_formdata( + fields, boundary=multipart_boundary + ) + else: + body, content_type = ( + urlencode(fields), + "application/x-www-form-urlencoded", + ) + + extra_kw["body"] = body + extra_kw["headers"] = {"Content-Type": content_type} + + extra_kw["headers"].update(headers) + extra_kw.update(urlopen_kw) + + return self.urlopen(method, url, **extra_kw) diff --git a/openpype/hosts/fusion/vendor/urllib3/response.py b/openpype/hosts/fusion/vendor/urllib3/response.py new file mode 100644 index 0000000000..38693f4fc6 --- /dev/null +++ b/openpype/hosts/fusion/vendor/urllib3/response.py @@ -0,0 +1,821 @@ +from __future__ import absolute_import + +import io +import logging +import zlib +from contextlib import contextmanager +from socket import error as SocketError +from socket import timeout as SocketTimeout + +try: + import brotli +except ImportError: + brotli = None + +from ._collections import HTTPHeaderDict +from .connection import BaseSSLError, HTTPException +from .exceptions import ( + BodyNotHttplibCompatible, + DecodeError, + HTTPError, + IncompleteRead, + InvalidChunkLength, + InvalidHeader, + ProtocolError, + ReadTimeoutError, + ResponseNotChunked, + SSLError, +) +from .packages import six +from .util.response import is_fp_closed, is_response_to_head + +log = logging.getLogger(__name__) + + +class DeflateDecoder(object): + def __init__(self): + self._first_try = True + self._data = b"" + self._obj = zlib.decompressobj() + + def __getattr__(self, name): + return getattr(self._obj, name) + + def decompress(self, data): + if not data: + return data + + if not self._first_try: + return self._obj.decompress(data) + + self._data += data + try: + decompressed = self._obj.decompress(data) + if decompressed: + self._first_try = False + self._data = None + return decompressed + except zlib.error: + self._first_try = False + self._obj = zlib.decompressobj(-zlib.MAX_WBITS) + try: + return self.decompress(self._data) + finally: + self._data = None + + +class GzipDecoderState(object): + + FIRST_MEMBER = 0 + OTHER_MEMBERS = 1 + SWALLOW_DATA = 2 + + +class GzipDecoder(object): + def __init__(self): + self._obj = zlib.decompressobj(16 + zlib.MAX_WBITS) + self._state = GzipDecoderState.FIRST_MEMBER + + def __getattr__(self, name): + return getattr(self._obj, name) + + def decompress(self, data): + ret = bytearray() + if self._state == GzipDecoderState.SWALLOW_DATA or not data: + return bytes(ret) + while True: + try: + ret += self._obj.decompress(data) + except zlib.error: + previous_state = self._state + # Ignore data after the first error + self._state = GzipDecoderState.SWALLOW_DATA + if previous_state == GzipDecoderState.OTHER_MEMBERS: + # Allow trailing garbage acceptable in other gzip clients + return bytes(ret) + raise + data = self._obj.unused_data + if not data: + return bytes(ret) + self._state = GzipDecoderState.OTHER_MEMBERS + self._obj = zlib.decompressobj(16 + zlib.MAX_WBITS) + + +if brotli is not None: + + class BrotliDecoder(object): + # Supports both 'brotlipy' and 'Brotli' packages + # since they share an import name. The top branches + # are for 'brotlipy' and bottom branches for 'Brotli' + def __init__(self): + self._obj = brotli.Decompressor() + if hasattr(self._obj, "decompress"): + self.decompress = self._obj.decompress + else: + self.decompress = self._obj.process + + def flush(self): + if hasattr(self._obj, "flush"): + return self._obj.flush() + return b"" + + +class MultiDecoder(object): + """ + From RFC7231: + If one or more encodings have been applied to a representation, the + sender that applied the encodings MUST generate a Content-Encoding + header field that lists the content codings in the order in which + they were applied. + """ + + def __init__(self, modes): + self._decoders = [_get_decoder(m.strip()) for m in modes.split(",")] + + def flush(self): + return self._decoders[0].flush() + + def decompress(self, data): + for d in reversed(self._decoders): + data = d.decompress(data) + return data + + +def _get_decoder(mode): + if "," in mode: + return MultiDecoder(mode) + + if mode == "gzip": + return GzipDecoder() + + if brotli is not None and mode == "br": + return BrotliDecoder() + + return DeflateDecoder() + + +class HTTPResponse(io.IOBase): + """ + HTTP Response container. + + Backwards-compatible with :class:`http.client.HTTPResponse` but the response ``body`` is + loaded and decoded on-demand when the ``data`` property is accessed. This + class is also compatible with the Python standard library's :mod:`io` + module, and can hence be treated as a readable object in the context of that + framework. + + Extra parameters for behaviour not present in :class:`http.client.HTTPResponse`: + + :param preload_content: + If True, the response's body will be preloaded during construction. + + :param decode_content: + If True, will attempt to decode the body based on the + 'content-encoding' header. + + :param original_response: + When this HTTPResponse wrapper is generated from an :class:`http.client.HTTPResponse` + object, it's convenient to include the original for debug purposes. It's + otherwise unused. + + :param retries: + The retries contains the last :class:`~urllib3.util.retry.Retry` that + was used during the request. + + :param enforce_content_length: + Enforce content length checking. Body returned by server must match + value of Content-Length header, if present. Otherwise, raise error. + """ + + CONTENT_DECODERS = ["gzip", "deflate"] + if brotli is not None: + CONTENT_DECODERS += ["br"] + REDIRECT_STATUSES = [301, 302, 303, 307, 308] + + def __init__( + self, + body="", + headers=None, + status=0, + version=0, + reason=None, + strict=0, + preload_content=True, + decode_content=True, + original_response=None, + pool=None, + connection=None, + msg=None, + retries=None, + enforce_content_length=False, + request_method=None, + request_url=None, + auto_close=True, + ): + + if isinstance(headers, HTTPHeaderDict): + self.headers = headers + else: + self.headers = HTTPHeaderDict(headers) + self.status = status + self.version = version + self.reason = reason + self.strict = strict + self.decode_content = decode_content + self.retries = retries + self.enforce_content_length = enforce_content_length + self.auto_close = auto_close + + self._decoder = None + self._body = None + self._fp = None + self._original_response = original_response + self._fp_bytes_read = 0 + self.msg = msg + self._request_url = request_url + + if body and isinstance(body, (six.string_types, bytes)): + self._body = body + + self._pool = pool + self._connection = connection + + if hasattr(body, "read"): + self._fp = body + + # Are we using the chunked-style of transfer encoding? + self.chunked = False + self.chunk_left = None + tr_enc = self.headers.get("transfer-encoding", "").lower() + # Don't incur the penalty of creating a list and then discarding it + encodings = (enc.strip() for enc in tr_enc.split(",")) + if "chunked" in encodings: + self.chunked = True + + # Determine length of response + self.length_remaining = self._init_length(request_method) + + # If requested, preload the body. + if preload_content and not self._body: + self._body = self.read(decode_content=decode_content) + + def get_redirect_location(self): + """ + Should we redirect and where to? + + :returns: Truthy redirect location string if we got a redirect status + code and valid location. ``None`` if redirect status and no + location. ``False`` if not a redirect status code. + """ + if self.status in self.REDIRECT_STATUSES: + return self.headers.get("location") + + return False + + def release_conn(self): + if not self._pool or not self._connection: + return + + self._pool._put_conn(self._connection) + self._connection = None + + def drain_conn(self): + """ + Read and discard any remaining HTTP response data in the response connection. + + Unread data in the HTTPResponse connection blocks the connection from being released back to the pool. + """ + try: + self.read() + except (HTTPError, SocketError, BaseSSLError, HTTPException): + pass + + @property + def data(self): + # For backwards-compat with earlier urllib3 0.4 and earlier. + if self._body: + return self._body + + if self._fp: + return self.read(cache_content=True) + + @property + def connection(self): + return self._connection + + def isclosed(self): + return is_fp_closed(self._fp) + + def tell(self): + """ + Obtain the number of bytes pulled over the wire so far. May differ from + the amount of content returned by :meth:``urllib3.response.HTTPResponse.read`` + if bytes are encoded on the wire (e.g, compressed). + """ + return self._fp_bytes_read + + def _init_length(self, request_method): + """ + Set initial length value for Response content if available. + """ + length = self.headers.get("content-length") + + if length is not None: + if self.chunked: + # This Response will fail with an IncompleteRead if it can't be + # received as chunked. This method falls back to attempt reading + # the response before raising an exception. + log.warning( + "Received response with both Content-Length and " + "Transfer-Encoding set. This is expressly forbidden " + "by RFC 7230 sec 3.3.2. Ignoring Content-Length and " + "attempting to process response as Transfer-Encoding: " + "chunked." + ) + return None + + try: + # RFC 7230 section 3.3.2 specifies multiple content lengths can + # be sent in a single Content-Length header + # (e.g. Content-Length: 42, 42). This line ensures the values + # are all valid ints and that as long as the `set` length is 1, + # all values are the same. Otherwise, the header is invalid. + lengths = set([int(val) for val in length.split(",")]) + if len(lengths) > 1: + raise InvalidHeader( + "Content-Length contained multiple " + "unmatching values (%s)" % length + ) + length = lengths.pop() + except ValueError: + length = None + else: + if length < 0: + length = None + + # Convert status to int for comparison + # In some cases, httplib returns a status of "_UNKNOWN" + try: + status = int(self.status) + except ValueError: + status = 0 + + # Check for responses that shouldn't include a body + if status in (204, 304) or 100 <= status < 200 or request_method == "HEAD": + length = 0 + + return length + + def _init_decoder(self): + """ + Set-up the _decoder attribute if necessary. + """ + # Note: content-encoding value should be case-insensitive, per RFC 7230 + # Section 3.2 + content_encoding = self.headers.get("content-encoding", "").lower() + if self._decoder is None: + if content_encoding in self.CONTENT_DECODERS: + self._decoder = _get_decoder(content_encoding) + elif "," in content_encoding: + encodings = [ + e.strip() + for e in content_encoding.split(",") + if e.strip() in self.CONTENT_DECODERS + ] + if len(encodings): + self._decoder = _get_decoder(content_encoding) + + DECODER_ERROR_CLASSES = (IOError, zlib.error) + if brotli is not None: + DECODER_ERROR_CLASSES += (brotli.error,) + + def _decode(self, data, decode_content, flush_decoder): + """ + Decode the data passed in and potentially flush the decoder. + """ + if not decode_content: + return data + + try: + if self._decoder: + data = self._decoder.decompress(data) + except self.DECODER_ERROR_CLASSES as e: + content_encoding = self.headers.get("content-encoding", "").lower() + raise DecodeError( + "Received response with content-encoding: %s, but " + "failed to decode it." % content_encoding, + e, + ) + if flush_decoder: + data += self._flush_decoder() + + return data + + def _flush_decoder(self): + """ + Flushes the decoder. Should only be called if the decoder is actually + being used. + """ + if self._decoder: + buf = self._decoder.decompress(b"") + return buf + self._decoder.flush() + + return b"" + + @contextmanager + def _error_catcher(self): + """ + Catch low-level python exceptions, instead re-raising urllib3 + variants, so that low-level exceptions are not leaked in the + high-level api. + + On exit, release the connection back to the pool. + """ + clean_exit = False + + try: + try: + yield + + except SocketTimeout: + # FIXME: Ideally we'd like to include the url in the ReadTimeoutError but + # there is yet no clean way to get at it from this context. + raise ReadTimeoutError(self._pool, None, "Read timed out.") + + except BaseSSLError as e: + # FIXME: Is there a better way to differentiate between SSLErrors? + if "read operation timed out" not in str(e): + # SSL errors related to framing/MAC get wrapped and reraised here + raise SSLError(e) + + raise ReadTimeoutError(self._pool, None, "Read timed out.") + + except (HTTPException, SocketError) as e: + # This includes IncompleteRead. + raise ProtocolError("Connection broken: %r" % e, e) + + # If no exception is thrown, we should avoid cleaning up + # unnecessarily. + clean_exit = True + finally: + # If we didn't terminate cleanly, we need to throw away our + # connection. + if not clean_exit: + # The response may not be closed but we're not going to use it + # anymore so close it now to ensure that the connection is + # released back to the pool. + if self._original_response: + self._original_response.close() + + # Closing the response may not actually be sufficient to close + # everything, so if we have a hold of the connection close that + # too. + if self._connection: + self._connection.close() + + # If we hold the original response but it's closed now, we should + # return the connection back to the pool. + if self._original_response and self._original_response.isclosed(): + self.release_conn() + + def read(self, amt=None, decode_content=None, cache_content=False): + """ + Similar to :meth:`http.client.HTTPResponse.read`, but with two additional + parameters: ``decode_content`` and ``cache_content``. + + :param amt: + How much of the content to read. If specified, caching is skipped + because it doesn't make sense to cache partial content as the full + response. + + :param decode_content: + If True, will attempt to decode the body based on the + 'content-encoding' header. + + :param cache_content: + If True, will save the returned data such that the same result is + returned despite of the state of the underlying file object. This + is useful if you want the ``.data`` property to continue working + after having ``.read()`` the file object. (Overridden if ``amt`` is + set.) + """ + self._init_decoder() + if decode_content is None: + decode_content = self.decode_content + + if self._fp is None: + return + + flush_decoder = False + fp_closed = getattr(self._fp, "closed", False) + + with self._error_catcher(): + if amt is None: + # cStringIO doesn't like amt=None + data = self._fp.read() if not fp_closed else b"" + flush_decoder = True + else: + cache_content = False + data = self._fp.read(amt) if not fp_closed else b"" + if ( + amt != 0 and not data + ): # Platform-specific: Buggy versions of Python. + # Close the connection when no data is returned + # + # This is redundant to what httplib/http.client _should_ + # already do. However, versions of python released before + # December 15, 2012 (http://bugs.python.org/issue16298) do + # not properly close the connection in all cases. There is + # no harm in redundantly calling close. + self._fp.close() + flush_decoder = True + if self.enforce_content_length and self.length_remaining not in ( + 0, + None, + ): + # This is an edge case that httplib failed to cover due + # to concerns of backward compatibility. We're + # addressing it here to make sure IncompleteRead is + # raised during streaming, so all calls with incorrect + # Content-Length are caught. + raise IncompleteRead(self._fp_bytes_read, self.length_remaining) + + if data: + self._fp_bytes_read += len(data) + if self.length_remaining is not None: + self.length_remaining -= len(data) + + data = self._decode(data, decode_content, flush_decoder) + + if cache_content: + self._body = data + + return data + + def stream(self, amt=2 ** 16, decode_content=None): + """ + A generator wrapper for the read() method. A call will block until + ``amt`` bytes have been read from the connection or until the + connection is closed. + + :param amt: + How much of the content to read. The generator will return up to + much data per iteration, but may return less. This is particularly + likely when using compressed data. However, the empty string will + never be returned. + + :param decode_content: + If True, will attempt to decode the body based on the + 'content-encoding' header. + """ + if self.chunked and self.supports_chunked_reads(): + for line in self.read_chunked(amt, decode_content=decode_content): + yield line + else: + while not is_fp_closed(self._fp): + data = self.read(amt=amt, decode_content=decode_content) + + if data: + yield data + + @classmethod + def from_httplib(ResponseCls, r, **response_kw): + """ + Given an :class:`http.client.HTTPResponse` instance ``r``, return a + corresponding :class:`urllib3.response.HTTPResponse` object. + + Remaining parameters are passed to the HTTPResponse constructor, along + with ``original_response=r``. + """ + headers = r.msg + + if not isinstance(headers, HTTPHeaderDict): + if six.PY2: + # Python 2.7 + headers = HTTPHeaderDict.from_httplib(headers) + else: + headers = HTTPHeaderDict(headers.items()) + + # HTTPResponse objects in Python 3 don't have a .strict attribute + strict = getattr(r, "strict", 0) + resp = ResponseCls( + body=r, + headers=headers, + status=r.status, + version=r.version, + reason=r.reason, + strict=strict, + original_response=r, + **response_kw + ) + return resp + + # Backwards-compatibility methods for http.client.HTTPResponse + def getheaders(self): + return self.headers + + def getheader(self, name, default=None): + return self.headers.get(name, default) + + # Backwards compatibility for http.cookiejar + def info(self): + return self.headers + + # Overrides from io.IOBase + def close(self): + if not self.closed: + self._fp.close() + + if self._connection: + self._connection.close() + + if not self.auto_close: + io.IOBase.close(self) + + @property + def closed(self): + if not self.auto_close: + return io.IOBase.closed.__get__(self) + elif self._fp is None: + return True + elif hasattr(self._fp, "isclosed"): + return self._fp.isclosed() + elif hasattr(self._fp, "closed"): + return self._fp.closed + else: + return True + + def fileno(self): + if self._fp is None: + raise IOError("HTTPResponse has no file to get a fileno from") + elif hasattr(self._fp, "fileno"): + return self._fp.fileno() + else: + raise IOError( + "The file-like object this HTTPResponse is wrapped " + "around has no file descriptor" + ) + + def flush(self): + if ( + self._fp is not None + and hasattr(self._fp, "flush") + and not getattr(self._fp, "closed", False) + ): + return self._fp.flush() + + def readable(self): + # This method is required for `io` module compatibility. + return True + + def readinto(self, b): + # This method is required for `io` module compatibility. + temp = self.read(len(b)) + if len(temp) == 0: + return 0 + else: + b[: len(temp)] = temp + return len(temp) + + def supports_chunked_reads(self): + """ + Checks if the underlying file-like object looks like a + :class:`http.client.HTTPResponse` object. We do this by testing for + the fp attribute. If it is present we assume it returns raw chunks as + processed by read_chunked(). + """ + return hasattr(self._fp, "fp") + + def _update_chunk_length(self): + # First, we'll figure out length of a chunk and then + # we'll try to read it from socket. + if self.chunk_left is not None: + return + line = self._fp.fp.readline() + line = line.split(b";", 1)[0] + try: + self.chunk_left = int(line, 16) + except ValueError: + # Invalid chunked protocol response, abort. + self.close() + raise InvalidChunkLength(self, line) + + def _handle_chunk(self, amt): + returned_chunk = None + if amt is None: + chunk = self._fp._safe_read(self.chunk_left) + returned_chunk = chunk + self._fp._safe_read(2) # Toss the CRLF at the end of the chunk. + self.chunk_left = None + elif amt < self.chunk_left: + value = self._fp._safe_read(amt) + self.chunk_left = self.chunk_left - amt + returned_chunk = value + elif amt == self.chunk_left: + value = self._fp._safe_read(amt) + self._fp._safe_read(2) # Toss the CRLF at the end of the chunk. + self.chunk_left = None + returned_chunk = value + else: # amt > self.chunk_left + returned_chunk = self._fp._safe_read(self.chunk_left) + self._fp._safe_read(2) # Toss the CRLF at the end of the chunk. + self.chunk_left = None + return returned_chunk + + def read_chunked(self, amt=None, decode_content=None): + """ + Similar to :meth:`HTTPResponse.read`, but with an additional + parameter: ``decode_content``. + + :param amt: + How much of the content to read. If specified, caching is skipped + because it doesn't make sense to cache partial content as the full + response. + + :param decode_content: + If True, will attempt to decode the body based on the + 'content-encoding' header. + """ + self._init_decoder() + # FIXME: Rewrite this method and make it a class with a better structured logic. + if not self.chunked: + raise ResponseNotChunked( + "Response is not chunked. " + "Header 'transfer-encoding: chunked' is missing." + ) + if not self.supports_chunked_reads(): + raise BodyNotHttplibCompatible( + "Body should be http.client.HTTPResponse like. " + "It should have have an fp attribute which returns raw chunks." + ) + + with self._error_catcher(): + # Don't bother reading the body of a HEAD request. + if self._original_response and is_response_to_head(self._original_response): + self._original_response.close() + return + + # If a response is already read and closed + # then return immediately. + if self._fp.fp is None: + return + + while True: + self._update_chunk_length() + if self.chunk_left == 0: + break + chunk = self._handle_chunk(amt) + decoded = self._decode( + chunk, decode_content=decode_content, flush_decoder=False + ) + if decoded: + yield decoded + + if decode_content: + # On CPython and PyPy, we should never need to flush the + # decoder. However, on Jython we *might* need to, so + # lets defensively do it anyway. + decoded = self._flush_decoder() + if decoded: # Platform-specific: Jython. + yield decoded + + # Chunk content ends with \r\n: discard it. + while True: + line = self._fp.fp.readline() + if not line: + # Some sites may not end with '\r\n'. + break + if line == b"\r\n": + break + + # We read everything; close the "file". + if self._original_response: + self._original_response.close() + + def geturl(self): + """ + Returns the URL that was the source of this response. + If the request that generated this response redirected, this method + will return the final redirect location. + """ + if self.retries is not None and len(self.retries.history): + return self.retries.history[-1].redirect_location + else: + return self._request_url + + def __iter__(self): + buffer = [] + for chunk in self.stream(decode_content=True): + if b"\n" in chunk: + chunk = chunk.split(b"\n") + yield b"".join(buffer) + chunk[0] + b"\n" + for x in chunk[1:-1]: + yield x + b"\n" + if chunk[-1]: + buffer = [chunk[-1]] + else: + buffer = [] + else: + buffer.append(chunk) + if buffer: + yield b"".join(buffer) diff --git a/openpype/hosts/fusion/vendor/urllib3/util/__init__.py b/openpype/hosts/fusion/vendor/urllib3/util/__init__.py new file mode 100644 index 0000000000..4547fc522b --- /dev/null +++ b/openpype/hosts/fusion/vendor/urllib3/util/__init__.py @@ -0,0 +1,49 @@ +from __future__ import absolute_import + +# For backwards compatibility, provide imports that used to be here. +from .connection import is_connection_dropped +from .request import SKIP_HEADER, SKIPPABLE_HEADERS, make_headers +from .response import is_fp_closed +from .retry import Retry +from .ssl_ import ( + ALPN_PROTOCOLS, + HAS_SNI, + IS_PYOPENSSL, + IS_SECURETRANSPORT, + PROTOCOL_TLS, + SSLContext, + assert_fingerprint, + resolve_cert_reqs, + resolve_ssl_version, + ssl_wrap_socket, +) +from .timeout import Timeout, current_time +from .url import Url, get_host, parse_url, split_first +from .wait import wait_for_read, wait_for_write + +__all__ = ( + "HAS_SNI", + "IS_PYOPENSSL", + "IS_SECURETRANSPORT", + "SSLContext", + "PROTOCOL_TLS", + "ALPN_PROTOCOLS", + "Retry", + "Timeout", + "Url", + "assert_fingerprint", + "current_time", + "is_connection_dropped", + "is_fp_closed", + "get_host", + "parse_url", + "make_headers", + "resolve_cert_reqs", + "resolve_ssl_version", + "split_first", + "ssl_wrap_socket", + "wait_for_read", + "wait_for_write", + "SKIP_HEADER", + "SKIPPABLE_HEADERS", +) diff --git a/openpype/hosts/fusion/vendor/urllib3/util/connection.py b/openpype/hosts/fusion/vendor/urllib3/util/connection.py new file mode 100644 index 0000000000..bdc240c50c --- /dev/null +++ b/openpype/hosts/fusion/vendor/urllib3/util/connection.py @@ -0,0 +1,150 @@ +from __future__ import absolute_import + +import socket + +from urllib3.exceptions import LocationParseError + +from ..contrib import _appengine_environ +from ..packages import six +from .wait import NoWayToWaitForSocketError, wait_for_read + + +def is_connection_dropped(conn): # Platform-specific + """ + Returns True if the connection is dropped and should be closed. + + :param conn: + :class:`http.client.HTTPConnection` object. + + Note: For platforms like AppEngine, this will always return ``False`` to + let the platform handle connection recycling transparently for us. + """ + sock = getattr(conn, "sock", False) + if sock is False: # Platform-specific: AppEngine + return False + if sock is None: # Connection already closed (such as by httplib). + return True + try: + # Returns True if readable, which here means it's been dropped + return wait_for_read(sock, timeout=0.0) + except NoWayToWaitForSocketError: # Platform-specific: AppEngine + return False + + +# This function is copied from socket.py in the Python 2.7 standard +# library test suite. Added to its signature is only `socket_options`. +# One additional modification is that we avoid binding to IPv6 servers +# discovered in DNS if the system doesn't have IPv6 functionality. +def create_connection( + address, + timeout=socket._GLOBAL_DEFAULT_TIMEOUT, + source_address=None, + socket_options=None, +): + """Connect to *address* and return the socket object. + + Convenience function. Connect to *address* (a 2-tuple ``(host, + port)``) and return the socket object. Passing the optional + *timeout* parameter will set the timeout on the socket instance + before attempting to connect. If no *timeout* is supplied, the + global default timeout setting returned by :func:`socket.getdefaulttimeout` + is used. If *source_address* is set it must be a tuple of (host, port) + for the socket to bind as a source address before making the connection. + An host of '' or port 0 tells the OS to use the default. + """ + + host, port = address + if host.startswith("["): + host = host.strip("[]") + err = None + + # Using the value from allowed_gai_family() in the context of getaddrinfo lets + # us select whether to work with IPv4 DNS records, IPv6 records, or both. + # The original create_connection function always returns all records. + family = allowed_gai_family() + + try: + host.encode("idna") + except UnicodeError: + return six.raise_from( + LocationParseError(u"'%s', label empty or too long" % host), None + ) + + for res in socket.getaddrinfo(host, port, family, socket.SOCK_STREAM): + af, socktype, proto, canonname, sa = res + sock = None + try: + sock = socket.socket(af, socktype, proto) + + # If provided, set socket level options before connecting. + _set_socket_options(sock, socket_options) + + if timeout is not socket._GLOBAL_DEFAULT_TIMEOUT: + sock.settimeout(timeout) + if source_address: + sock.bind(source_address) + sock.connect(sa) + return sock + + except socket.error as e: + err = e + if sock is not None: + sock.close() + sock = None + + if err is not None: + raise err + + raise socket.error("getaddrinfo returns an empty list") + + +def _set_socket_options(sock, options): + if options is None: + return + + for opt in options: + sock.setsockopt(*opt) + + +def allowed_gai_family(): + """This function is designed to work in the context of + getaddrinfo, where family=socket.AF_UNSPEC is the default and + will perform a DNS search for both IPv6 and IPv4 records.""" + + family = socket.AF_INET + if HAS_IPV6: + family = socket.AF_UNSPEC + return family + + +def _has_ipv6(host): + """Returns True if the system can bind an IPv6 address.""" + sock = None + has_ipv6 = False + + # App Engine doesn't support IPV6 sockets and actually has a quota on the + # number of sockets that can be used, so just early out here instead of + # creating a socket needlessly. + # See https://github.com/urllib3/urllib3/issues/1446 + if _appengine_environ.is_appengine_sandbox(): + return False + + if socket.has_ipv6: + # has_ipv6 returns true if cPython was compiled with IPv6 support. + # It does not tell us if the system has IPv6 support enabled. To + # determine that we must bind to an IPv6 address. + # https://github.com/urllib3/urllib3/pull/611 + # https://bugs.python.org/issue658327 + try: + sock = socket.socket(socket.AF_INET6) + sock.bind((host, 0)) + has_ipv6 = True + except Exception: + pass + + if sock: + sock.close() + return has_ipv6 + + +HAS_IPV6 = _has_ipv6("::1") diff --git a/openpype/hosts/fusion/vendor/urllib3/util/proxy.py b/openpype/hosts/fusion/vendor/urllib3/util/proxy.py new file mode 100644 index 0000000000..34f884d5b3 --- /dev/null +++ b/openpype/hosts/fusion/vendor/urllib3/util/proxy.py @@ -0,0 +1,56 @@ +from .ssl_ import create_urllib3_context, resolve_cert_reqs, resolve_ssl_version + + +def connection_requires_http_tunnel( + proxy_url=None, proxy_config=None, destination_scheme=None +): + """ + Returns True if the connection requires an HTTP CONNECT through the proxy. + + :param URL proxy_url: + URL of the proxy. + :param ProxyConfig proxy_config: + Proxy configuration from poolmanager.py + :param str destination_scheme: + The scheme of the destination. (i.e https, http, etc) + """ + # If we're not using a proxy, no way to use a tunnel. + if proxy_url is None: + return False + + # HTTP destinations never require tunneling, we always forward. + if destination_scheme == "http": + return False + + # Support for forwarding with HTTPS proxies and HTTPS destinations. + if ( + proxy_url.scheme == "https" + and proxy_config + and proxy_config.use_forwarding_for_https + ): + return False + + # Otherwise always use a tunnel. + return True + + +def create_proxy_ssl_context( + ssl_version, cert_reqs, ca_certs=None, ca_cert_dir=None, ca_cert_data=None +): + """ + Generates a default proxy ssl context if one hasn't been provided by the + user. + """ + ssl_context = create_urllib3_context( + ssl_version=resolve_ssl_version(ssl_version), + cert_reqs=resolve_cert_reqs(cert_reqs), + ) + if ( + not ca_certs + and not ca_cert_dir + and not ca_cert_data + and hasattr(ssl_context, "load_default_certs") + ): + ssl_context.load_default_certs() + + return ssl_context diff --git a/openpype/hosts/fusion/vendor/urllib3/util/queue.py b/openpype/hosts/fusion/vendor/urllib3/util/queue.py new file mode 100644 index 0000000000..41784104ee --- /dev/null +++ b/openpype/hosts/fusion/vendor/urllib3/util/queue.py @@ -0,0 +1,22 @@ +import collections + +from ..packages import six +from ..packages.six.moves import queue + +if six.PY2: + # Queue is imported for side effects on MS Windows. See issue #229. + import Queue as _unused_module_Queue # noqa: F401 + + +class LifoQueue(queue.Queue): + def _init(self, _): + self.queue = collections.deque() + + def _qsize(self, len=len): + return len(self.queue) + + def _put(self, item): + self.queue.append(item) + + def _get(self): + return self.queue.pop() diff --git a/openpype/hosts/fusion/vendor/urllib3/util/request.py b/openpype/hosts/fusion/vendor/urllib3/util/request.py new file mode 100644 index 0000000000..25103383ec --- /dev/null +++ b/openpype/hosts/fusion/vendor/urllib3/util/request.py @@ -0,0 +1,143 @@ +from __future__ import absolute_import + +from base64 import b64encode + +from ..exceptions import UnrewindableBodyError +from ..packages.six import b, integer_types + +# Pass as a value within ``headers`` to skip +# emitting some HTTP headers that are added automatically. +# The only headers that are supported are ``Accept-Encoding``, +# ``Host``, and ``User-Agent``. +SKIP_HEADER = "@@@SKIP_HEADER@@@" +SKIPPABLE_HEADERS = frozenset(["accept-encoding", "host", "user-agent"]) + +ACCEPT_ENCODING = "gzip,deflate" +try: + import brotli as _unused_module_brotli # noqa: F401 +except ImportError: + pass +else: + ACCEPT_ENCODING += ",br" + +_FAILEDTELL = object() + + +def make_headers( + keep_alive=None, + accept_encoding=None, + user_agent=None, + basic_auth=None, + proxy_basic_auth=None, + disable_cache=None, +): + """ + Shortcuts for generating request headers. + + :param keep_alive: + If ``True``, adds 'connection: keep-alive' header. + + :param accept_encoding: + Can be a boolean, list, or string. + ``True`` translates to 'gzip,deflate'. + List will get joined by comma. + String will be used as provided. + + :param user_agent: + String representing the user-agent you want, such as + "python-urllib3/0.6" + + :param basic_auth: + Colon-separated username:password string for 'authorization: basic ...' + auth header. + + :param proxy_basic_auth: + Colon-separated username:password string for 'proxy-authorization: basic ...' + auth header. + + :param disable_cache: + If ``True``, adds 'cache-control: no-cache' header. + + Example:: + + >>> make_headers(keep_alive=True, user_agent="Batman/1.0") + {'connection': 'keep-alive', 'user-agent': 'Batman/1.0'} + >>> make_headers(accept_encoding=True) + {'accept-encoding': 'gzip,deflate'} + """ + headers = {} + if accept_encoding: + if isinstance(accept_encoding, str): + pass + elif isinstance(accept_encoding, list): + accept_encoding = ",".join(accept_encoding) + else: + accept_encoding = ACCEPT_ENCODING + headers["accept-encoding"] = accept_encoding + + if user_agent: + headers["user-agent"] = user_agent + + if keep_alive: + headers["connection"] = "keep-alive" + + if basic_auth: + headers["authorization"] = "Basic " + b64encode(b(basic_auth)).decode("utf-8") + + if proxy_basic_auth: + headers["proxy-authorization"] = "Basic " + b64encode( + b(proxy_basic_auth) + ).decode("utf-8") + + if disable_cache: + headers["cache-control"] = "no-cache" + + return headers + + +def set_file_position(body, pos): + """ + If a position is provided, move file to that point. + Otherwise, we'll attempt to record a position for future use. + """ + if pos is not None: + rewind_body(body, pos) + elif getattr(body, "tell", None) is not None: + try: + pos = body.tell() + except (IOError, OSError): + # This differentiates from None, allowing us to catch + # a failed `tell()` later when trying to rewind the body. + pos = _FAILEDTELL + + return pos + + +def rewind_body(body, body_pos): + """ + Attempt to rewind body to a certain position. + Primarily used for request redirects and retries. + + :param body: + File-like object that supports seek. + + :param int pos: + Position to seek to in file. + """ + body_seek = getattr(body, "seek", None) + if body_seek is not None and isinstance(body_pos, integer_types): + try: + body_seek(body_pos) + except (IOError, OSError): + raise UnrewindableBodyError( + "An error occurred when rewinding request body for redirect/retry." + ) + elif body_pos is _FAILEDTELL: + raise UnrewindableBodyError( + "Unable to record file position for rewinding " + "request body during a redirect/retry." + ) + else: + raise ValueError( + "body_pos must be of type integer, instead it was %s." % type(body_pos) + ) diff --git a/openpype/hosts/fusion/vendor/urllib3/util/response.py b/openpype/hosts/fusion/vendor/urllib3/util/response.py new file mode 100644 index 0000000000..5ea609cced --- /dev/null +++ b/openpype/hosts/fusion/vendor/urllib3/util/response.py @@ -0,0 +1,107 @@ +from __future__ import absolute_import + +from email.errors import MultipartInvariantViolationDefect, StartBoundaryNotFoundDefect + +from ..exceptions import HeaderParsingError +from ..packages.six.moves import http_client as httplib + + +def is_fp_closed(obj): + """ + Checks whether a given file-like object is closed. + + :param obj: + The file-like object to check. + """ + + try: + # Check `isclosed()` first, in case Python3 doesn't set `closed`. + # GH Issue #928 + return obj.isclosed() + except AttributeError: + pass + + try: + # Check via the official file-like-object way. + return obj.closed + except AttributeError: + pass + + try: + # Check if the object is a container for another file-like object that + # gets released on exhaustion (e.g. HTTPResponse). + return obj.fp is None + except AttributeError: + pass + + raise ValueError("Unable to determine whether fp is closed.") + + +def assert_header_parsing(headers): + """ + Asserts whether all headers have been successfully parsed. + Extracts encountered errors from the result of parsing headers. + + Only works on Python 3. + + :param http.client.HTTPMessage headers: Headers to verify. + + :raises urllib3.exceptions.HeaderParsingError: + If parsing errors are found. + """ + + # This will fail silently if we pass in the wrong kind of parameter. + # To make debugging easier add an explicit check. + if not isinstance(headers, httplib.HTTPMessage): + raise TypeError("expected httplib.Message, got {0}.".format(type(headers))) + + defects = getattr(headers, "defects", None) + get_payload = getattr(headers, "get_payload", None) + + unparsed_data = None + if get_payload: + # get_payload is actually email.message.Message.get_payload; + # we're only interested in the result if it's not a multipart message + if not headers.is_multipart(): + payload = get_payload() + + if isinstance(payload, (bytes, str)): + unparsed_data = payload + if defects: + # httplib is assuming a response body is available + # when parsing headers even when httplib only sends + # header data to parse_headers() This results in + # defects on multipart responses in particular. + # See: https://github.com/urllib3/urllib3/issues/800 + + # So we ignore the following defects: + # - StartBoundaryNotFoundDefect: + # The claimed start boundary was never found. + # - MultipartInvariantViolationDefect: + # A message claimed to be a multipart but no subparts were found. + defects = [ + defect + for defect in defects + if not isinstance( + defect, (StartBoundaryNotFoundDefect, MultipartInvariantViolationDefect) + ) + ] + + if defects or unparsed_data: + raise HeaderParsingError(defects=defects, unparsed_data=unparsed_data) + + +def is_response_to_head(response): + """ + Checks whether the request of a response has been a HEAD-request. + Handles the quirks of AppEngine. + + :param http.client.HTTPResponse response: + Response to check if the originating request + used 'HEAD' as a method. + """ + # FIXME: Can we do this somehow without accessing private httplib _method? + method = response._method + if isinstance(method, int): # Platform-specific: Appengine + return method == 3 + return method.upper() == "HEAD" diff --git a/openpype/hosts/fusion/vendor/urllib3/util/retry.py b/openpype/hosts/fusion/vendor/urllib3/util/retry.py new file mode 100644 index 0000000000..c7dc42f1d6 --- /dev/null +++ b/openpype/hosts/fusion/vendor/urllib3/util/retry.py @@ -0,0 +1,602 @@ +from __future__ import absolute_import + +import email +import logging +import re +import time +import warnings +from collections import namedtuple +from itertools import takewhile + +from ..exceptions import ( + ConnectTimeoutError, + InvalidHeader, + MaxRetryError, + ProtocolError, + ProxyError, + ReadTimeoutError, + ResponseError, +) +from ..packages import six + +log = logging.getLogger(__name__) + + +# Data structure for representing the metadata of requests that result in a retry. +RequestHistory = namedtuple( + "RequestHistory", ["method", "url", "error", "status", "redirect_location"] +) + + +# TODO: In v2 we can remove this sentinel and metaclass with deprecated options. +_Default = object() + + +class _RetryMeta(type): + @property + def DEFAULT_METHOD_WHITELIST(cls): + warnings.warn( + "Using 'Retry.DEFAULT_METHOD_WHITELIST' is deprecated and " + "will be removed in v2.0. Use 'Retry.DEFAULT_ALLOWED_METHODS' instead", + DeprecationWarning, + ) + return cls.DEFAULT_ALLOWED_METHODS + + @DEFAULT_METHOD_WHITELIST.setter + def DEFAULT_METHOD_WHITELIST(cls, value): + warnings.warn( + "Using 'Retry.DEFAULT_METHOD_WHITELIST' is deprecated and " + "will be removed in v2.0. Use 'Retry.DEFAULT_ALLOWED_METHODS' instead", + DeprecationWarning, + ) + cls.DEFAULT_ALLOWED_METHODS = value + + @property + def DEFAULT_REDIRECT_HEADERS_BLACKLIST(cls): + warnings.warn( + "Using 'Retry.DEFAULT_REDIRECT_HEADERS_BLACKLIST' is deprecated and " + "will be removed in v2.0. Use 'Retry.DEFAULT_REMOVE_HEADERS_ON_REDIRECT' instead", + DeprecationWarning, + ) + return cls.DEFAULT_REMOVE_HEADERS_ON_REDIRECT + + @DEFAULT_REDIRECT_HEADERS_BLACKLIST.setter + def DEFAULT_REDIRECT_HEADERS_BLACKLIST(cls, value): + warnings.warn( + "Using 'Retry.DEFAULT_REDIRECT_HEADERS_BLACKLIST' is deprecated and " + "will be removed in v2.0. Use 'Retry.DEFAULT_REMOVE_HEADERS_ON_REDIRECT' instead", + DeprecationWarning, + ) + cls.DEFAULT_REMOVE_HEADERS_ON_REDIRECT = value + + +@six.add_metaclass(_RetryMeta) +class Retry(object): + """Retry configuration. + + Each retry attempt will create a new Retry object with updated values, so + they can be safely reused. + + Retries can be defined as a default for a pool:: + + retries = Retry(connect=5, read=2, redirect=5) + http = PoolManager(retries=retries) + response = http.request('GET', 'http://example.com/') + + Or per-request (which overrides the default for the pool):: + + response = http.request('GET', 'http://example.com/', retries=Retry(10)) + + Retries can be disabled by passing ``False``:: + + response = http.request('GET', 'http://example.com/', retries=False) + + Errors will be wrapped in :class:`~urllib3.exceptions.MaxRetryError` unless + retries are disabled, in which case the causing exception will be raised. + + :param int total: + Total number of retries to allow. Takes precedence over other counts. + + Set to ``None`` to remove this constraint and fall back on other + counts. + + Set to ``0`` to fail on the first retry. + + Set to ``False`` to disable and imply ``raise_on_redirect=False``. + + :param int connect: + How many connection-related errors to retry on. + + These are errors raised before the request is sent to the remote server, + which we assume has not triggered the server to process the request. + + Set to ``0`` to fail on the first retry of this type. + + :param int read: + How many times to retry on read errors. + + These errors are raised after the request was sent to the server, so the + request may have side-effects. + + Set to ``0`` to fail on the first retry of this type. + + :param int redirect: + How many redirects to perform. Limit this to avoid infinite redirect + loops. + + A redirect is a HTTP response with a status code 301, 302, 303, 307 or + 308. + + Set to ``0`` to fail on the first retry of this type. + + Set to ``False`` to disable and imply ``raise_on_redirect=False``. + + :param int status: + How many times to retry on bad status codes. + + These are retries made on responses, where status code matches + ``status_forcelist``. + + Set to ``0`` to fail on the first retry of this type. + + :param int other: + How many times to retry on other errors. + + Other errors are errors that are not connect, read, redirect or status errors. + These errors might be raised after the request was sent to the server, so the + request might have side-effects. + + Set to ``0`` to fail on the first retry of this type. + + If ``total`` is not set, it's a good idea to set this to 0 to account + for unexpected edge cases and avoid infinite retry loops. + + :param iterable allowed_methods: + Set of uppercased HTTP method verbs that we should retry on. + + By default, we only retry on methods which are considered to be + idempotent (multiple requests with the same parameters end with the + same state). See :attr:`Retry.DEFAULT_ALLOWED_METHODS`. + + Set to a ``False`` value to retry on any verb. + + .. warning:: + + Previously this parameter was named ``method_whitelist``, that + usage is deprecated in v1.26.0 and will be removed in v2.0. + + :param iterable status_forcelist: + A set of integer HTTP status codes that we should force a retry on. + A retry is initiated if the request method is in ``allowed_methods`` + and the response status code is in ``status_forcelist``. + + By default, this is disabled with ``None``. + + :param float backoff_factor: + A backoff factor to apply between attempts after the second try + (most errors are resolved immediately by a second try without a + delay). urllib3 will sleep for:: + + {backoff factor} * (2 ** ({number of total retries} - 1)) + + seconds. If the backoff_factor is 0.1, then :func:`.sleep` will sleep + for [0.0s, 0.2s, 0.4s, ...] between retries. It will never be longer + than :attr:`Retry.BACKOFF_MAX`. + + By default, backoff is disabled (set to 0). + + :param bool raise_on_redirect: Whether, if the number of redirects is + exhausted, to raise a MaxRetryError, or to return a response with a + response code in the 3xx range. + + :param bool raise_on_status: Similar meaning to ``raise_on_redirect``: + whether we should raise an exception, or return a response, + if status falls in ``status_forcelist`` range and retries have + been exhausted. + + :param tuple history: The history of the request encountered during + each call to :meth:`~Retry.increment`. The list is in the order + the requests occurred. Each list item is of class :class:`RequestHistory`. + + :param bool respect_retry_after_header: + Whether to respect Retry-After header on status codes defined as + :attr:`Retry.RETRY_AFTER_STATUS_CODES` or not. + + :param iterable remove_headers_on_redirect: + Sequence of headers to remove from the request when a response + indicating a redirect is returned before firing off the redirected + request. + """ + + #: Default methods to be used for ``allowed_methods`` + DEFAULT_ALLOWED_METHODS = frozenset( + ["HEAD", "GET", "PUT", "DELETE", "OPTIONS", "TRACE"] + ) + + #: Default status codes to be used for ``status_forcelist`` + RETRY_AFTER_STATUS_CODES = frozenset([413, 429, 503]) + + #: Default headers to be used for ``remove_headers_on_redirect`` + DEFAULT_REMOVE_HEADERS_ON_REDIRECT = frozenset(["Authorization"]) + + #: Maximum backoff time. + BACKOFF_MAX = 120 + + def __init__( + self, + total=10, + connect=None, + read=None, + redirect=None, + status=None, + other=None, + allowed_methods=_Default, + status_forcelist=None, + backoff_factor=0, + raise_on_redirect=True, + raise_on_status=True, + history=None, + respect_retry_after_header=True, + remove_headers_on_redirect=_Default, + # TODO: Deprecated, remove in v2.0 + method_whitelist=_Default, + ): + + if method_whitelist is not _Default: + if allowed_methods is not _Default: + raise ValueError( + "Using both 'allowed_methods' and " + "'method_whitelist' together is not allowed. " + "Instead only use 'allowed_methods'" + ) + warnings.warn( + "Using 'method_whitelist' with Retry is deprecated and " + "will be removed in v2.0. Use 'allowed_methods' instead", + DeprecationWarning, + stacklevel=2, + ) + allowed_methods = method_whitelist + if allowed_methods is _Default: + allowed_methods = self.DEFAULT_ALLOWED_METHODS + if remove_headers_on_redirect is _Default: + remove_headers_on_redirect = self.DEFAULT_REMOVE_HEADERS_ON_REDIRECT + + self.total = total + self.connect = connect + self.read = read + self.status = status + self.other = other + + if redirect is False or total is False: + redirect = 0 + raise_on_redirect = False + + self.redirect = redirect + self.status_forcelist = status_forcelist or set() + self.allowed_methods = allowed_methods + self.backoff_factor = backoff_factor + self.raise_on_redirect = raise_on_redirect + self.raise_on_status = raise_on_status + self.history = history or tuple() + self.respect_retry_after_header = respect_retry_after_header + self.remove_headers_on_redirect = frozenset( + [h.lower() for h in remove_headers_on_redirect] + ) + + def new(self, **kw): + params = dict( + total=self.total, + connect=self.connect, + read=self.read, + redirect=self.redirect, + status=self.status, + other=self.other, + status_forcelist=self.status_forcelist, + backoff_factor=self.backoff_factor, + raise_on_redirect=self.raise_on_redirect, + raise_on_status=self.raise_on_status, + history=self.history, + remove_headers_on_redirect=self.remove_headers_on_redirect, + respect_retry_after_header=self.respect_retry_after_header, + ) + + # TODO: If already given in **kw we use what's given to us + # If not given we need to figure out what to pass. We decide + # based on whether our class has the 'method_whitelist' property + # and if so we pass the deprecated 'method_whitelist' otherwise + # we use 'allowed_methods'. Remove in v2.0 + if "method_whitelist" not in kw and "allowed_methods" not in kw: + if "method_whitelist" in self.__dict__: + warnings.warn( + "Using 'method_whitelist' with Retry is deprecated and " + "will be removed in v2.0. Use 'allowed_methods' instead", + DeprecationWarning, + ) + params["method_whitelist"] = self.allowed_methods + else: + params["allowed_methods"] = self.allowed_methods + + params.update(kw) + return type(self)(**params) + + @classmethod + def from_int(cls, retries, redirect=True, default=None): + """Backwards-compatibility for the old retries format.""" + if retries is None: + retries = default if default is not None else cls.DEFAULT + + if isinstance(retries, Retry): + return retries + + redirect = bool(redirect) and None + new_retries = cls(retries, redirect=redirect) + log.debug("Converted retries value: %r -> %r", retries, new_retries) + return new_retries + + def get_backoff_time(self): + """Formula for computing the current backoff + + :rtype: float + """ + # We want to consider only the last consecutive errors sequence (Ignore redirects). + consecutive_errors_len = len( + list( + takewhile(lambda x: x.redirect_location is None, reversed(self.history)) + ) + ) + if consecutive_errors_len <= 1: + return 0 + + backoff_value = self.backoff_factor * (2 ** (consecutive_errors_len - 1)) + return min(self.BACKOFF_MAX, backoff_value) + + def parse_retry_after(self, retry_after): + # Whitespace: https://tools.ietf.org/html/rfc7230#section-3.2.4 + if re.match(r"^\s*[0-9]+\s*$", retry_after): + seconds = int(retry_after) + else: + retry_date_tuple = email.utils.parsedate_tz(retry_after) + if retry_date_tuple is None: + raise InvalidHeader("Invalid Retry-After header: %s" % retry_after) + if retry_date_tuple[9] is None: # Python 2 + # Assume UTC if no timezone was specified + # On Python2.7, parsedate_tz returns None for a timezone offset + # instead of 0 if no timezone is given, where mktime_tz treats + # a None timezone offset as local time. + retry_date_tuple = retry_date_tuple[:9] + (0,) + retry_date_tuple[10:] + + retry_date = email.utils.mktime_tz(retry_date_tuple) + seconds = retry_date - time.time() + + if seconds < 0: + seconds = 0 + + return seconds + + def get_retry_after(self, response): + """Get the value of Retry-After in seconds.""" + + retry_after = response.getheader("Retry-After") + + if retry_after is None: + return None + + return self.parse_retry_after(retry_after) + + def sleep_for_retry(self, response=None): + retry_after = self.get_retry_after(response) + if retry_after: + time.sleep(retry_after) + return True + + return False + + def _sleep_backoff(self): + backoff = self.get_backoff_time() + if backoff <= 0: + return + time.sleep(backoff) + + def sleep(self, response=None): + """Sleep between retry attempts. + + This method will respect a server's ``Retry-After`` response header + and sleep the duration of the time requested. If that is not present, it + will use an exponential backoff. By default, the backoff factor is 0 and + this method will return immediately. + """ + + if self.respect_retry_after_header and response: + slept = self.sleep_for_retry(response) + if slept: + return + + self._sleep_backoff() + + def _is_connection_error(self, err): + """Errors when we're fairly sure that the server did not receive the + request, so it should be safe to retry. + """ + if isinstance(err, ProxyError): + err = err.original_error + return isinstance(err, ConnectTimeoutError) + + def _is_read_error(self, err): + """Errors that occur after the request has been started, so we should + assume that the server began processing it. + """ + return isinstance(err, (ReadTimeoutError, ProtocolError)) + + def _is_method_retryable(self, method): + """Checks if a given HTTP method should be retried upon, depending if + it is included in the allowed_methods + """ + # TODO: For now favor if the Retry implementation sets its own method_whitelist + # property outside of our constructor to avoid breaking custom implementations. + if "method_whitelist" in self.__dict__: + warnings.warn( + "Using 'method_whitelist' with Retry is deprecated and " + "will be removed in v2.0. Use 'allowed_methods' instead", + DeprecationWarning, + ) + allowed_methods = self.method_whitelist + else: + allowed_methods = self.allowed_methods + + if allowed_methods and method.upper() not in allowed_methods: + return False + return True + + def is_retry(self, method, status_code, has_retry_after=False): + """Is this method/status code retryable? (Based on allowlists and control + variables such as the number of total retries to allow, whether to + respect the Retry-After header, whether this header is present, and + whether the returned status code is on the list of status codes to + be retried upon on the presence of the aforementioned header) + """ + if not self._is_method_retryable(method): + return False + + if self.status_forcelist and status_code in self.status_forcelist: + return True + + return ( + self.total + and self.respect_retry_after_header + and has_retry_after + and (status_code in self.RETRY_AFTER_STATUS_CODES) + ) + + def is_exhausted(self): + """Are we out of retries?""" + retry_counts = ( + self.total, + self.connect, + self.read, + self.redirect, + self.status, + self.other, + ) + retry_counts = list(filter(None, retry_counts)) + if not retry_counts: + return False + + return min(retry_counts) < 0 + + def increment( + self, + method=None, + url=None, + response=None, + error=None, + _pool=None, + _stacktrace=None, + ): + """Return a new Retry object with incremented retry counters. + + :param response: A response object, or None, if the server did not + return a response. + :type response: :class:`~urllib3.response.HTTPResponse` + :param Exception error: An error encountered during the request, or + None if the response was received successfully. + + :return: A new ``Retry`` object. + """ + if self.total is False and error: + # Disabled, indicate to re-raise the error. + raise six.reraise(type(error), error, _stacktrace) + + total = self.total + if total is not None: + total -= 1 + + connect = self.connect + read = self.read + redirect = self.redirect + status_count = self.status + other = self.other + cause = "unknown" + status = None + redirect_location = None + + if error and self._is_connection_error(error): + # Connect retry? + if connect is False: + raise six.reraise(type(error), error, _stacktrace) + elif connect is not None: + connect -= 1 + + elif error and self._is_read_error(error): + # Read retry? + if read is False or not self._is_method_retryable(method): + raise six.reraise(type(error), error, _stacktrace) + elif read is not None: + read -= 1 + + elif error: + # Other retry? + if other is not None: + other -= 1 + + elif response and response.get_redirect_location(): + # Redirect retry? + if redirect is not None: + redirect -= 1 + cause = "too many redirects" + redirect_location = response.get_redirect_location() + status = response.status + + else: + # Incrementing because of a server error like a 500 in + # status_forcelist and the given method is in the allowed_methods + cause = ResponseError.GENERIC_ERROR + if response and response.status: + if status_count is not None: + status_count -= 1 + cause = ResponseError.SPECIFIC_ERROR.format(status_code=response.status) + status = response.status + + history = self.history + ( + RequestHistory(method, url, error, status, redirect_location), + ) + + new_retry = self.new( + total=total, + connect=connect, + read=read, + redirect=redirect, + status=status_count, + other=other, + history=history, + ) + + if new_retry.is_exhausted(): + raise MaxRetryError(_pool, url, error or ResponseError(cause)) + + log.debug("Incremented Retry for (url='%s'): %r", url, new_retry) + + return new_retry + + def __repr__(self): + return ( + "{cls.__name__}(total={self.total}, connect={self.connect}, " + "read={self.read}, redirect={self.redirect}, status={self.status})" + ).format(cls=type(self), self=self) + + def __getattr__(self, item): + if item == "method_whitelist": + # TODO: Remove this deprecated alias in v2.0 + warnings.warn( + "Using 'method_whitelist' with Retry is deprecated and " + "will be removed in v2.0. Use 'allowed_methods' instead", + DeprecationWarning, + ) + return self.allowed_methods + try: + return getattr(super(Retry, self), item) + except AttributeError: + return getattr(Retry, item) + + +# For backwards compatibility (equivalent to pre-v1.9): +Retry.DEFAULT = Retry(3) diff --git a/openpype/hosts/fusion/vendor/urllib3/util/ssl_.py b/openpype/hosts/fusion/vendor/urllib3/util/ssl_.py new file mode 100644 index 0000000000..8f867812a5 --- /dev/null +++ b/openpype/hosts/fusion/vendor/urllib3/util/ssl_.py @@ -0,0 +1,495 @@ +from __future__ import absolute_import + +import hmac +import os +import sys +import warnings +from binascii import hexlify, unhexlify +from hashlib import md5, sha1, sha256 + +from ..exceptions import ( + InsecurePlatformWarning, + ProxySchemeUnsupported, + SNIMissingWarning, + SSLError, +) +from ..packages import six +from .url import BRACELESS_IPV6_ADDRZ_RE, IPV4_RE + +SSLContext = None +SSLTransport = None +HAS_SNI = False +IS_PYOPENSSL = False +IS_SECURETRANSPORT = False +ALPN_PROTOCOLS = ["http/1.1"] + +# Maps the length of a digest to a possible hash function producing this digest +HASHFUNC_MAP = {32: md5, 40: sha1, 64: sha256} + + +def _const_compare_digest_backport(a, b): + """ + Compare two digests of equal length in constant time. + + The digests must be of type str/bytes. + Returns True if the digests match, and False otherwise. + """ + result = abs(len(a) - len(b)) + for left, right in zip(bytearray(a), bytearray(b)): + result |= left ^ right + return result == 0 + + +_const_compare_digest = getattr(hmac, "compare_digest", _const_compare_digest_backport) + +try: # Test for SSL features + import ssl + from ssl import CERT_REQUIRED, wrap_socket +except ImportError: + pass + +try: + from ssl import HAS_SNI # Has SNI? +except ImportError: + pass + +try: + from .ssltransport import SSLTransport +except ImportError: + pass + + +try: # Platform-specific: Python 3.6 + from ssl import PROTOCOL_TLS + + PROTOCOL_SSLv23 = PROTOCOL_TLS +except ImportError: + try: + from ssl import PROTOCOL_SSLv23 as PROTOCOL_TLS + + PROTOCOL_SSLv23 = PROTOCOL_TLS + except ImportError: + PROTOCOL_SSLv23 = PROTOCOL_TLS = 2 + +try: + from ssl import PROTOCOL_TLS_CLIENT +except ImportError: + PROTOCOL_TLS_CLIENT = PROTOCOL_TLS + + +try: + from ssl import OP_NO_COMPRESSION, OP_NO_SSLv2, OP_NO_SSLv3 +except ImportError: + OP_NO_SSLv2, OP_NO_SSLv3 = 0x1000000, 0x2000000 + OP_NO_COMPRESSION = 0x20000 + + +try: # OP_NO_TICKET was added in Python 3.6 + from ssl import OP_NO_TICKET +except ImportError: + OP_NO_TICKET = 0x4000 + + +# A secure default. +# Sources for more information on TLS ciphers: +# +# - https://wiki.mozilla.org/Security/Server_Side_TLS +# - https://www.ssllabs.com/projects/best-practices/index.html +# - https://hynek.me/articles/hardening-your-web-servers-ssl-ciphers/ +# +# The general intent is: +# - prefer cipher suites that offer perfect forward secrecy (DHE/ECDHE), +# - prefer ECDHE over DHE for better performance, +# - prefer any AES-GCM and ChaCha20 over any AES-CBC for better performance and +# security, +# - prefer AES-GCM over ChaCha20 because hardware-accelerated AES is common, +# - disable NULL authentication, MD5 MACs, DSS, and other +# insecure ciphers for security reasons. +# - NOTE: TLS 1.3 cipher suites are managed through a different interface +# not exposed by CPython (yet!) and are enabled by default if they're available. +DEFAULT_CIPHERS = ":".join( + [ + "ECDHE+AESGCM", + "ECDHE+CHACHA20", + "DHE+AESGCM", + "DHE+CHACHA20", + "ECDH+AESGCM", + "DH+AESGCM", + "ECDH+AES", + "DH+AES", + "RSA+AESGCM", + "RSA+AES", + "!aNULL", + "!eNULL", + "!MD5", + "!DSS", + ] +) + +try: + from ssl import SSLContext # Modern SSL? +except ImportError: + + class SSLContext(object): # Platform-specific: Python 2 + def __init__(self, protocol_version): + self.protocol = protocol_version + # Use default values from a real SSLContext + self.check_hostname = False + self.verify_mode = ssl.CERT_NONE + self.ca_certs = None + self.options = 0 + self.certfile = None + self.keyfile = None + self.ciphers = None + + def load_cert_chain(self, certfile, keyfile): + self.certfile = certfile + self.keyfile = keyfile + + def load_verify_locations(self, cafile=None, capath=None, cadata=None): + self.ca_certs = cafile + + if capath is not None: + raise SSLError("CA directories not supported in older Pythons") + + if cadata is not None: + raise SSLError("CA data not supported in older Pythons") + + def set_ciphers(self, cipher_suite): + self.ciphers = cipher_suite + + def wrap_socket(self, socket, server_hostname=None, server_side=False): + warnings.warn( + "A true SSLContext object is not available. This prevents " + "urllib3 from configuring SSL appropriately and may cause " + "certain SSL connections to fail. You can upgrade to a newer " + "version of Python to solve this. For more information, see " + "https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html" + "#ssl-warnings", + InsecurePlatformWarning, + ) + kwargs = { + "keyfile": self.keyfile, + "certfile": self.certfile, + "ca_certs": self.ca_certs, + "cert_reqs": self.verify_mode, + "ssl_version": self.protocol, + "server_side": server_side, + } + return wrap_socket(socket, ciphers=self.ciphers, **kwargs) + + +def assert_fingerprint(cert, fingerprint): + """ + Checks if given fingerprint matches the supplied certificate. + + :param cert: + Certificate as bytes object. + :param fingerprint: + Fingerprint as string of hexdigits, can be interspersed by colons. + """ + + fingerprint = fingerprint.replace(":", "").lower() + digest_length = len(fingerprint) + hashfunc = HASHFUNC_MAP.get(digest_length) + if not hashfunc: + raise SSLError("Fingerprint of invalid length: {0}".format(fingerprint)) + + # We need encode() here for py32; works on py2 and p33. + fingerprint_bytes = unhexlify(fingerprint.encode()) + + cert_digest = hashfunc(cert).digest() + + if not _const_compare_digest(cert_digest, fingerprint_bytes): + raise SSLError( + 'Fingerprints did not match. Expected "{0}", got "{1}".'.format( + fingerprint, hexlify(cert_digest) + ) + ) + + +def resolve_cert_reqs(candidate): + """ + Resolves the argument to a numeric constant, which can be passed to + the wrap_socket function/method from the ssl module. + Defaults to :data:`ssl.CERT_REQUIRED`. + If given a string it is assumed to be the name of the constant in the + :mod:`ssl` module or its abbreviation. + (So you can specify `REQUIRED` instead of `CERT_REQUIRED`. + If it's neither `None` nor a string we assume it is already the numeric + constant which can directly be passed to wrap_socket. + """ + if candidate is None: + return CERT_REQUIRED + + if isinstance(candidate, str): + res = getattr(ssl, candidate, None) + if res is None: + res = getattr(ssl, "CERT_" + candidate) + return res + + return candidate + + +def resolve_ssl_version(candidate): + """ + like resolve_cert_reqs + """ + if candidate is None: + return PROTOCOL_TLS + + if isinstance(candidate, str): + res = getattr(ssl, candidate, None) + if res is None: + res = getattr(ssl, "PROTOCOL_" + candidate) + return res + + return candidate + + +def create_urllib3_context( + ssl_version=None, cert_reqs=None, options=None, ciphers=None +): + """All arguments have the same meaning as ``ssl_wrap_socket``. + + By default, this function does a lot of the same work that + ``ssl.create_default_context`` does on Python 3.4+. It: + + - Disables SSLv2, SSLv3, and compression + - Sets a restricted set of server ciphers + + If you wish to enable SSLv3, you can do:: + + from urllib3.util import ssl_ + context = ssl_.create_urllib3_context() + context.options &= ~ssl_.OP_NO_SSLv3 + + You can do the same to enable compression (substituting ``COMPRESSION`` + for ``SSLv3`` in the last line above). + + :param ssl_version: + The desired protocol version to use. This will default to + PROTOCOL_SSLv23 which will negotiate the highest protocol that both + the server and your installation of OpenSSL support. + :param cert_reqs: + Whether to require the certificate verification. This defaults to + ``ssl.CERT_REQUIRED``. + :param options: + Specific OpenSSL options. These default to ``ssl.OP_NO_SSLv2``, + ``ssl.OP_NO_SSLv3``, ``ssl.OP_NO_COMPRESSION``, and ``ssl.OP_NO_TICKET``. + :param ciphers: + Which cipher suites to allow the server to select. + :returns: + Constructed SSLContext object with specified options + :rtype: SSLContext + """ + # PROTOCOL_TLS is deprecated in Python 3.10 + if not ssl_version or ssl_version == PROTOCOL_TLS: + ssl_version = PROTOCOL_TLS_CLIENT + + context = SSLContext(ssl_version) + + context.set_ciphers(ciphers or DEFAULT_CIPHERS) + + # Setting the default here, as we may have no ssl module on import + cert_reqs = ssl.CERT_REQUIRED if cert_reqs is None else cert_reqs + + if options is None: + options = 0 + # SSLv2 is easily broken and is considered harmful and dangerous + options |= OP_NO_SSLv2 + # SSLv3 has several problems and is now dangerous + options |= OP_NO_SSLv3 + # Disable compression to prevent CRIME attacks for OpenSSL 1.0+ + # (issue #309) + options |= OP_NO_COMPRESSION + # TLSv1.2 only. Unless set explicitly, do not request tickets. + # This may save some bandwidth on wire, and although the ticket is encrypted, + # there is a risk associated with it being on wire, + # if the server is not rotating its ticketing keys properly. + options |= OP_NO_TICKET + + context.options |= options + + # Enable post-handshake authentication for TLS 1.3, see GH #1634. PHA is + # necessary for conditional client cert authentication with TLS 1.3. + # The attribute is None for OpenSSL <= 1.1.0 or does not exist in older + # versions of Python. We only enable on Python 3.7.4+ or if certificate + # verification is enabled to work around Python issue #37428 + # See: https://bugs.python.org/issue37428 + if (cert_reqs == ssl.CERT_REQUIRED or sys.version_info >= (3, 7, 4)) and getattr( + context, "post_handshake_auth", None + ) is not None: + context.post_handshake_auth = True + + def disable_check_hostname(): + if ( + getattr(context, "check_hostname", None) is not None + ): # Platform-specific: Python 3.2 + # We do our own verification, including fingerprints and alternative + # hostnames. So disable it here + context.check_hostname = False + + # The order of the below lines setting verify_mode and check_hostname + # matter due to safe-guards SSLContext has to prevent an SSLContext with + # check_hostname=True, verify_mode=NONE/OPTIONAL. This is made even more + # complex because we don't know whether PROTOCOL_TLS_CLIENT will be used + # or not so we don't know the initial state of the freshly created SSLContext. + if cert_reqs == ssl.CERT_REQUIRED: + context.verify_mode = cert_reqs + disable_check_hostname() + else: + disable_check_hostname() + context.verify_mode = cert_reqs + + # Enable logging of TLS session keys via defacto standard environment variable + # 'SSLKEYLOGFILE', if the feature is available (Python 3.8+). Skip empty values. + if hasattr(context, "keylog_filename"): + sslkeylogfile = os.environ.get("SSLKEYLOGFILE") + if sslkeylogfile: + context.keylog_filename = sslkeylogfile + + return context + + +def ssl_wrap_socket( + sock, + keyfile=None, + certfile=None, + cert_reqs=None, + ca_certs=None, + server_hostname=None, + ssl_version=None, + ciphers=None, + ssl_context=None, + ca_cert_dir=None, + key_password=None, + ca_cert_data=None, + tls_in_tls=False, +): + """ + All arguments except for server_hostname, ssl_context, and ca_cert_dir have + the same meaning as they do when using :func:`ssl.wrap_socket`. + + :param server_hostname: + When SNI is supported, the expected hostname of the certificate + :param ssl_context: + A pre-made :class:`SSLContext` object. If none is provided, one will + be created using :func:`create_urllib3_context`. + :param ciphers: + A string of ciphers we wish the client to support. + :param ca_cert_dir: + A directory containing CA certificates in multiple separate files, as + supported by OpenSSL's -CApath flag or the capath argument to + SSLContext.load_verify_locations(). + :param key_password: + Optional password if the keyfile is encrypted. + :param ca_cert_data: + Optional string containing CA certificates in PEM format suitable for + passing as the cadata parameter to SSLContext.load_verify_locations() + :param tls_in_tls: + Use SSLTransport to wrap the existing socket. + """ + context = ssl_context + if context is None: + # Note: This branch of code and all the variables in it are no longer + # used by urllib3 itself. We should consider deprecating and removing + # this code. + context = create_urllib3_context(ssl_version, cert_reqs, ciphers=ciphers) + + if ca_certs or ca_cert_dir or ca_cert_data: + try: + context.load_verify_locations(ca_certs, ca_cert_dir, ca_cert_data) + except (IOError, OSError) as e: + raise SSLError(e) + + elif ssl_context is None and hasattr(context, "load_default_certs"): + # try to load OS default certs; works well on Windows (require Python3.4+) + context.load_default_certs() + + # Attempt to detect if we get the goofy behavior of the + # keyfile being encrypted and OpenSSL asking for the + # passphrase via the terminal and instead error out. + if keyfile and key_password is None and _is_key_file_encrypted(keyfile): + raise SSLError("Client private key is encrypted, password is required") + + if certfile: + if key_password is None: + context.load_cert_chain(certfile, keyfile) + else: + context.load_cert_chain(certfile, keyfile, key_password) + + try: + if hasattr(context, "set_alpn_protocols"): + context.set_alpn_protocols(ALPN_PROTOCOLS) + except NotImplementedError: # Defensive: in CI, we always have set_alpn_protocols + pass + + # If we detect server_hostname is an IP address then the SNI + # extension should not be used according to RFC3546 Section 3.1 + use_sni_hostname = server_hostname and not is_ipaddress(server_hostname) + # SecureTransport uses server_hostname in certificate verification. + send_sni = (use_sni_hostname and HAS_SNI) or ( + IS_SECURETRANSPORT and server_hostname + ) + # Do not warn the user if server_hostname is an invalid SNI hostname. + if not HAS_SNI and use_sni_hostname: + warnings.warn( + "An HTTPS request has been made, but the SNI (Server Name " + "Indication) extension to TLS is not available on this platform. " + "This may cause the server to present an incorrect TLS " + "certificate, which can cause validation failures. You can upgrade to " + "a newer version of Python to solve this. For more information, see " + "https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html" + "#ssl-warnings", + SNIMissingWarning, + ) + + if send_sni: + ssl_sock = _ssl_wrap_socket_impl( + sock, context, tls_in_tls, server_hostname=server_hostname + ) + else: + ssl_sock = _ssl_wrap_socket_impl(sock, context, tls_in_tls) + return ssl_sock + + +def is_ipaddress(hostname): + """Detects whether the hostname given is an IPv4 or IPv6 address. + Also detects IPv6 addresses with Zone IDs. + + :param str hostname: Hostname to examine. + :return: True if the hostname is an IP address, False otherwise. + """ + if not six.PY2 and isinstance(hostname, bytes): + # IDN A-label bytes are ASCII compatible. + hostname = hostname.decode("ascii") + return bool(IPV4_RE.match(hostname) or BRACELESS_IPV6_ADDRZ_RE.match(hostname)) + + +def _is_key_file_encrypted(key_file): + """Detects if a key file is encrypted or not.""" + with open(key_file, "r") as f: + for line in f: + # Look for Proc-Type: 4,ENCRYPTED + if "ENCRYPTED" in line: + return True + + return False + + +def _ssl_wrap_socket_impl(sock, ssl_context, tls_in_tls, server_hostname=None): + if tls_in_tls: + if not SSLTransport: + # Import error, ssl is not available. + raise ProxySchemeUnsupported( + "TLS in TLS requires support for the 'ssl' module" + ) + + SSLTransport._validate_ssl_context_for_tls_in_tls(ssl_context) + return SSLTransport(sock, ssl_context, server_hostname) + + if server_hostname: + return ssl_context.wrap_socket(sock, server_hostname=server_hostname) + else: + return ssl_context.wrap_socket(sock) diff --git a/openpype/hosts/fusion/vendor/urllib3/util/ssltransport.py b/openpype/hosts/fusion/vendor/urllib3/util/ssltransport.py new file mode 100644 index 0000000000..c2186bced9 --- /dev/null +++ b/openpype/hosts/fusion/vendor/urllib3/util/ssltransport.py @@ -0,0 +1,221 @@ +import io +import socket +import ssl + +from urllib3.exceptions import ProxySchemeUnsupported +from urllib3.packages import six + +SSL_BLOCKSIZE = 16384 + + +class SSLTransport: + """ + The SSLTransport wraps an existing socket and establishes an SSL connection. + + Contrary to Python's implementation of SSLSocket, it allows you to chain + multiple TLS connections together. It's particularly useful if you need to + implement TLS within TLS. + + The class supports most of the socket API operations. + """ + + @staticmethod + def _validate_ssl_context_for_tls_in_tls(ssl_context): + """ + Raises a ProxySchemeUnsupported if the provided ssl_context can't be used + for TLS in TLS. + + The only requirement is that the ssl_context provides the 'wrap_bio' + methods. + """ + + if not hasattr(ssl_context, "wrap_bio"): + if six.PY2: + raise ProxySchemeUnsupported( + "TLS in TLS requires SSLContext.wrap_bio() which isn't " + "supported on Python 2" + ) + else: + raise ProxySchemeUnsupported( + "TLS in TLS requires SSLContext.wrap_bio() which isn't " + "available on non-native SSLContext" + ) + + def __init__( + self, socket, ssl_context, server_hostname=None, suppress_ragged_eofs=True + ): + """ + Create an SSLTransport around socket using the provided ssl_context. + """ + self.incoming = ssl.MemoryBIO() + self.outgoing = ssl.MemoryBIO() + + self.suppress_ragged_eofs = suppress_ragged_eofs + self.socket = socket + + self.sslobj = ssl_context.wrap_bio( + self.incoming, self.outgoing, server_hostname=server_hostname + ) + + # Perform initial handshake. + self._ssl_io_loop(self.sslobj.do_handshake) + + def __enter__(self): + return self + + def __exit__(self, *_): + self.close() + + def fileno(self): + return self.socket.fileno() + + def read(self, len=1024, buffer=None): + return self._wrap_ssl_read(len, buffer) + + def recv(self, len=1024, flags=0): + if flags != 0: + raise ValueError("non-zero flags not allowed in calls to recv") + return self._wrap_ssl_read(len) + + def recv_into(self, buffer, nbytes=None, flags=0): + if flags != 0: + raise ValueError("non-zero flags not allowed in calls to recv_into") + if buffer and (nbytes is None): + nbytes = len(buffer) + elif nbytes is None: + nbytes = 1024 + return self.read(nbytes, buffer) + + def sendall(self, data, flags=0): + if flags != 0: + raise ValueError("non-zero flags not allowed in calls to sendall") + count = 0 + with memoryview(data) as view, view.cast("B") as byte_view: + amount = len(byte_view) + while count < amount: + v = self.send(byte_view[count:]) + count += v + + def send(self, data, flags=0): + if flags != 0: + raise ValueError("non-zero flags not allowed in calls to send") + response = self._ssl_io_loop(self.sslobj.write, data) + return response + + def makefile( + self, mode="r", buffering=None, encoding=None, errors=None, newline=None + ): + """ + Python's httpclient uses makefile and buffered io when reading HTTP + messages and we need to support it. + + This is unfortunately a copy and paste of socket.py makefile with small + changes to point to the socket directly. + """ + if not set(mode) <= {"r", "w", "b"}: + raise ValueError("invalid mode %r (only r, w, b allowed)" % (mode,)) + + writing = "w" in mode + reading = "r" in mode or not writing + assert reading or writing + binary = "b" in mode + rawmode = "" + if reading: + rawmode += "r" + if writing: + rawmode += "w" + raw = socket.SocketIO(self, rawmode) + self.socket._io_refs += 1 + if buffering is None: + buffering = -1 + if buffering < 0: + buffering = io.DEFAULT_BUFFER_SIZE + if buffering == 0: + if not binary: + raise ValueError("unbuffered streams must be binary") + return raw + if reading and writing: + buffer = io.BufferedRWPair(raw, raw, buffering) + elif reading: + buffer = io.BufferedReader(raw, buffering) + else: + assert writing + buffer = io.BufferedWriter(raw, buffering) + if binary: + return buffer + text = io.TextIOWrapper(buffer, encoding, errors, newline) + text.mode = mode + return text + + def unwrap(self): + self._ssl_io_loop(self.sslobj.unwrap) + + def close(self): + self.socket.close() + + def getpeercert(self, binary_form=False): + return self.sslobj.getpeercert(binary_form) + + def version(self): + return self.sslobj.version() + + def cipher(self): + return self.sslobj.cipher() + + def selected_alpn_protocol(self): + return self.sslobj.selected_alpn_protocol() + + def selected_npn_protocol(self): + return self.sslobj.selected_npn_protocol() + + def shared_ciphers(self): + return self.sslobj.shared_ciphers() + + def compression(self): + return self.sslobj.compression() + + def settimeout(self, value): + self.socket.settimeout(value) + + def gettimeout(self): + return self.socket.gettimeout() + + def _decref_socketios(self): + self.socket._decref_socketios() + + def _wrap_ssl_read(self, len, buffer=None): + try: + return self._ssl_io_loop(self.sslobj.read, len, buffer) + except ssl.SSLError as e: + if e.errno == ssl.SSL_ERROR_EOF and self.suppress_ragged_eofs: + return 0 # eof, return 0. + else: + raise + + def _ssl_io_loop(self, func, *args): + """Performs an I/O loop between incoming/outgoing and the socket.""" + should_loop = True + ret = None + + while should_loop: + errno = None + try: + ret = func(*args) + except ssl.SSLError as e: + if e.errno not in (ssl.SSL_ERROR_WANT_READ, ssl.SSL_ERROR_WANT_WRITE): + # WANT_READ, and WANT_WRITE are expected, others are not. + raise e + errno = e.errno + + buf = self.outgoing.read() + self.socket.sendall(buf) + + if errno is None: + should_loop = False + elif errno == ssl.SSL_ERROR_WANT_READ: + buf = self.socket.recv(SSL_BLOCKSIZE) + if buf: + self.incoming.write(buf) + else: + self.incoming.write_eof() + return ret diff --git a/openpype/hosts/fusion/vendor/urllib3/util/timeout.py b/openpype/hosts/fusion/vendor/urllib3/util/timeout.py new file mode 100644 index 0000000000..ff69593b05 --- /dev/null +++ b/openpype/hosts/fusion/vendor/urllib3/util/timeout.py @@ -0,0 +1,268 @@ +from __future__ import absolute_import + +import time + +# The default socket timeout, used by httplib to indicate that no timeout was +# specified by the user +from socket import _GLOBAL_DEFAULT_TIMEOUT + +from ..exceptions import TimeoutStateError + +# A sentinel value to indicate that no timeout was specified by the user in +# urllib3 +_Default = object() + + +# Use time.monotonic if available. +current_time = getattr(time, "monotonic", time.time) + + +class Timeout(object): + """Timeout configuration. + + Timeouts can be defined as a default for a pool: + + .. code-block:: python + + timeout = Timeout(connect=2.0, read=7.0) + http = PoolManager(timeout=timeout) + response = http.request('GET', 'http://example.com/') + + Or per-request (which overrides the default for the pool): + + .. code-block:: python + + response = http.request('GET', 'http://example.com/', timeout=Timeout(10)) + + Timeouts can be disabled by setting all the parameters to ``None``: + + .. code-block:: python + + no_timeout = Timeout(connect=None, read=None) + response = http.request('GET', 'http://example.com/, timeout=no_timeout) + + + :param total: + This combines the connect and read timeouts into one; the read timeout + will be set to the time leftover from the connect attempt. In the + event that both a connect timeout and a total are specified, or a read + timeout and a total are specified, the shorter timeout will be applied. + + Defaults to None. + + :type total: int, float, or None + + :param connect: + The maximum amount of time (in seconds) to wait for a connection + attempt to a server to succeed. Omitting the parameter will default the + connect timeout to the system default, probably `the global default + timeout in socket.py + `_. + None will set an infinite timeout for connection attempts. + + :type connect: int, float, or None + + :param read: + The maximum amount of time (in seconds) to wait between consecutive + read operations for a response from the server. Omitting the parameter + will default the read timeout to the system default, probably `the + global default timeout in socket.py + `_. + None will set an infinite timeout. + + :type read: int, float, or None + + .. note:: + + Many factors can affect the total amount of time for urllib3 to return + an HTTP response. + + For example, Python's DNS resolver does not obey the timeout specified + on the socket. Other factors that can affect total request time include + high CPU load, high swap, the program running at a low priority level, + or other behaviors. + + In addition, the read and total timeouts only measure the time between + read operations on the socket connecting the client and the server, + not the total amount of time for the request to return a complete + response. For most requests, the timeout is raised because the server + has not sent the first byte in the specified time. This is not always + the case; if a server streams one byte every fifteen seconds, a timeout + of 20 seconds will not trigger, even though the request will take + several minutes to complete. + + If your goal is to cut off any request after a set amount of wall clock + time, consider having a second "watcher" thread to cut off a slow + request. + """ + + #: A sentinel object representing the default timeout value + DEFAULT_TIMEOUT = _GLOBAL_DEFAULT_TIMEOUT + + def __init__(self, total=None, connect=_Default, read=_Default): + self._connect = self._validate_timeout(connect, "connect") + self._read = self._validate_timeout(read, "read") + self.total = self._validate_timeout(total, "total") + self._start_connect = None + + def __repr__(self): + return "%s(connect=%r, read=%r, total=%r)" % ( + type(self).__name__, + self._connect, + self._read, + self.total, + ) + + # __str__ provided for backwards compatibility + __str__ = __repr__ + + @classmethod + def _validate_timeout(cls, value, name): + """Check that a timeout attribute is valid. + + :param value: The timeout value to validate + :param name: The name of the timeout attribute to validate. This is + used to specify in error messages. + :return: The validated and casted version of the given value. + :raises ValueError: If it is a numeric value less than or equal to + zero, or the type is not an integer, float, or None. + """ + if value is _Default: + return cls.DEFAULT_TIMEOUT + + if value is None or value is cls.DEFAULT_TIMEOUT: + return value + + if isinstance(value, bool): + raise ValueError( + "Timeout cannot be a boolean value. It must " + "be an int, float or None." + ) + try: + float(value) + except (TypeError, ValueError): + raise ValueError( + "Timeout value %s was %s, but it must be an " + "int, float or None." % (name, value) + ) + + try: + if value <= 0: + raise ValueError( + "Attempted to set %s timeout to %s, but the " + "timeout cannot be set to a value less " + "than or equal to 0." % (name, value) + ) + except TypeError: + # Python 3 + raise ValueError( + "Timeout value %s was %s, but it must be an " + "int, float or None." % (name, value) + ) + + return value + + @classmethod + def from_float(cls, timeout): + """Create a new Timeout from a legacy timeout value. + + The timeout value used by httplib.py sets the same timeout on the + connect(), and recv() socket requests. This creates a :class:`Timeout` + object that sets the individual timeouts to the ``timeout`` value + passed to this function. + + :param timeout: The legacy timeout value. + :type timeout: integer, float, sentinel default object, or None + :return: Timeout object + :rtype: :class:`Timeout` + """ + return Timeout(read=timeout, connect=timeout) + + def clone(self): + """Create a copy of the timeout object + + Timeout properties are stored per-pool but each request needs a fresh + Timeout object to ensure each one has its own start/stop configured. + + :return: a copy of the timeout object + :rtype: :class:`Timeout` + """ + # We can't use copy.deepcopy because that will also create a new object + # for _GLOBAL_DEFAULT_TIMEOUT, which socket.py uses as a sentinel to + # detect the user default. + return Timeout(connect=self._connect, read=self._read, total=self.total) + + def start_connect(self): + """Start the timeout clock, used during a connect() attempt + + :raises urllib3.exceptions.TimeoutStateError: if you attempt + to start a timer that has been started already. + """ + if self._start_connect is not None: + raise TimeoutStateError("Timeout timer has already been started.") + self._start_connect = current_time() + return self._start_connect + + def get_connect_duration(self): + """Gets the time elapsed since the call to :meth:`start_connect`. + + :return: Elapsed time in seconds. + :rtype: float + :raises urllib3.exceptions.TimeoutStateError: if you attempt + to get duration for a timer that hasn't been started. + """ + if self._start_connect is None: + raise TimeoutStateError( + "Can't get connect duration for timer that has not started." + ) + return current_time() - self._start_connect + + @property + def connect_timeout(self): + """Get the value to use when setting a connection timeout. + + This will be a positive float or integer, the value None + (never timeout), or the default system timeout. + + :return: Connect timeout. + :rtype: int, float, :attr:`Timeout.DEFAULT_TIMEOUT` or None + """ + if self.total is None: + return self._connect + + if self._connect is None or self._connect is self.DEFAULT_TIMEOUT: + return self.total + + return min(self._connect, self.total) + + @property + def read_timeout(self): + """Get the value for the read timeout. + + This assumes some time has elapsed in the connection timeout and + computes the read timeout appropriately. + + If self.total is set, the read timeout is dependent on the amount of + time taken by the connect timeout. If the connection time has not been + established, a :exc:`~urllib3.exceptions.TimeoutStateError` will be + raised. + + :return: Value to use for the read timeout. + :rtype: int, float, :attr:`Timeout.DEFAULT_TIMEOUT` or None + :raises urllib3.exceptions.TimeoutStateError: If :meth:`start_connect` + has not yet been called on this object. + """ + if ( + self.total is not None + and self.total is not self.DEFAULT_TIMEOUT + and self._read is not None + and self._read is not self.DEFAULT_TIMEOUT + ): + # In case the connect timeout has not yet been established. + if self._start_connect is None: + return self._read + return max(0, min(self.total - self.get_connect_duration(), self._read)) + elif self.total is not None and self.total is not self.DEFAULT_TIMEOUT: + return max(0, self.total - self.get_connect_duration()) + else: + return self._read diff --git a/openpype/hosts/fusion/vendor/urllib3/util/url.py b/openpype/hosts/fusion/vendor/urllib3/util/url.py new file mode 100644 index 0000000000..81a03da9e3 --- /dev/null +++ b/openpype/hosts/fusion/vendor/urllib3/util/url.py @@ -0,0 +1,432 @@ +from __future__ import absolute_import + +import re +from collections import namedtuple + +from ..exceptions import LocationParseError +from ..packages import six + +url_attrs = ["scheme", "auth", "host", "port", "path", "query", "fragment"] + +# We only want to normalize urls with an HTTP(S) scheme. +# urllib3 infers URLs without a scheme (None) to be http. +NORMALIZABLE_SCHEMES = ("http", "https", None) + +# Almost all of these patterns were derived from the +# 'rfc3986' module: https://github.com/python-hyper/rfc3986 +PERCENT_RE = re.compile(r"%[a-fA-F0-9]{2}") +SCHEME_RE = re.compile(r"^(?:[a-zA-Z][a-zA-Z0-9+-]*:|/)") +URI_RE = re.compile( + r"^(?:([a-zA-Z][a-zA-Z0-9+.-]*):)?" + r"(?://([^\\/?#]*))?" + r"([^?#]*)" + r"(?:\?([^#]*))?" + r"(?:#(.*))?$", + re.UNICODE | re.DOTALL, +) + +IPV4_PAT = r"(?:[0-9]{1,3}\.){3}[0-9]{1,3}" +HEX_PAT = "[0-9A-Fa-f]{1,4}" +LS32_PAT = "(?:{hex}:{hex}|{ipv4})".format(hex=HEX_PAT, ipv4=IPV4_PAT) +_subs = {"hex": HEX_PAT, "ls32": LS32_PAT} +_variations = [ + # 6( h16 ":" ) ls32 + "(?:%(hex)s:){6}%(ls32)s", + # "::" 5( h16 ":" ) ls32 + "::(?:%(hex)s:){5}%(ls32)s", + # [ h16 ] "::" 4( h16 ":" ) ls32 + "(?:%(hex)s)?::(?:%(hex)s:){4}%(ls32)s", + # [ *1( h16 ":" ) h16 ] "::" 3( h16 ":" ) ls32 + "(?:(?:%(hex)s:)?%(hex)s)?::(?:%(hex)s:){3}%(ls32)s", + # [ *2( h16 ":" ) h16 ] "::" 2( h16 ":" ) ls32 + "(?:(?:%(hex)s:){0,2}%(hex)s)?::(?:%(hex)s:){2}%(ls32)s", + # [ *3( h16 ":" ) h16 ] "::" h16 ":" ls32 + "(?:(?:%(hex)s:){0,3}%(hex)s)?::%(hex)s:%(ls32)s", + # [ *4( h16 ":" ) h16 ] "::" ls32 + "(?:(?:%(hex)s:){0,4}%(hex)s)?::%(ls32)s", + # [ *5( h16 ":" ) h16 ] "::" h16 + "(?:(?:%(hex)s:){0,5}%(hex)s)?::%(hex)s", + # [ *6( h16 ":" ) h16 ] "::" + "(?:(?:%(hex)s:){0,6}%(hex)s)?::", +] + +UNRESERVED_PAT = r"ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789._!\-~" +IPV6_PAT = "(?:" + "|".join([x % _subs for x in _variations]) + ")" +ZONE_ID_PAT = "(?:%25|%)(?:[" + UNRESERVED_PAT + "]|%[a-fA-F0-9]{2})+" +IPV6_ADDRZ_PAT = r"\[" + IPV6_PAT + r"(?:" + ZONE_ID_PAT + r")?\]" +REG_NAME_PAT = r"(?:[^\[\]%:/?#]|%[a-fA-F0-9]{2})*" +TARGET_RE = re.compile(r"^(/[^?#]*)(?:\?([^#]*))?(?:#.*)?$") + +IPV4_RE = re.compile("^" + IPV4_PAT + "$") +IPV6_RE = re.compile("^" + IPV6_PAT + "$") +IPV6_ADDRZ_RE = re.compile("^" + IPV6_ADDRZ_PAT + "$") +BRACELESS_IPV6_ADDRZ_RE = re.compile("^" + IPV6_ADDRZ_PAT[2:-2] + "$") +ZONE_ID_RE = re.compile("(" + ZONE_ID_PAT + r")\]$") + +_HOST_PORT_PAT = ("^(%s|%s|%s)(?::([0-9]{0,5}))?$") % ( + REG_NAME_PAT, + IPV4_PAT, + IPV6_ADDRZ_PAT, +) +_HOST_PORT_RE = re.compile(_HOST_PORT_PAT, re.UNICODE | re.DOTALL) + +UNRESERVED_CHARS = set( + "ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789._-~" +) +SUB_DELIM_CHARS = set("!$&'()*+,;=") +USERINFO_CHARS = UNRESERVED_CHARS | SUB_DELIM_CHARS | {":"} +PATH_CHARS = USERINFO_CHARS | {"@", "/"} +QUERY_CHARS = FRAGMENT_CHARS = PATH_CHARS | {"?"} + + +class Url(namedtuple("Url", url_attrs)): + """ + Data structure for representing an HTTP URL. Used as a return value for + :func:`parse_url`. Both the scheme and host are normalized as they are + both case-insensitive according to RFC 3986. + """ + + __slots__ = () + + def __new__( + cls, + scheme=None, + auth=None, + host=None, + port=None, + path=None, + query=None, + fragment=None, + ): + if path and not path.startswith("/"): + path = "/" + path + if scheme is not None: + scheme = scheme.lower() + return super(Url, cls).__new__( + cls, scheme, auth, host, port, path, query, fragment + ) + + @property + def hostname(self): + """For backwards-compatibility with urlparse. We're nice like that.""" + return self.host + + @property + def request_uri(self): + """Absolute path including the query string.""" + uri = self.path or "/" + + if self.query is not None: + uri += "?" + self.query + + return uri + + @property + def netloc(self): + """Network location including host and port""" + if self.port: + return "%s:%d" % (self.host, self.port) + return self.host + + @property + def url(self): + """ + Convert self into a url + + This function should more or less round-trip with :func:`.parse_url`. The + returned url may not be exactly the same as the url inputted to + :func:`.parse_url`, but it should be equivalent by the RFC (e.g., urls + with a blank port will have : removed). + + Example: :: + + >>> U = parse_url('http://google.com/mail/') + >>> U.url + 'http://google.com/mail/' + >>> Url('http', 'username:password', 'host.com', 80, + ... '/path', 'query', 'fragment').url + 'http://username:password@host.com:80/path?query#fragment' + """ + scheme, auth, host, port, path, query, fragment = self + url = u"" + + # We use "is not None" we want things to happen with empty strings (or 0 port) + if scheme is not None: + url += scheme + u"://" + if auth is not None: + url += auth + u"@" + if host is not None: + url += host + if port is not None: + url += u":" + str(port) + if path is not None: + url += path + if query is not None: + url += u"?" + query + if fragment is not None: + url += u"#" + fragment + + return url + + def __str__(self): + return self.url + + +def split_first(s, delims): + """ + .. deprecated:: 1.25 + + Given a string and an iterable of delimiters, split on the first found + delimiter. Return two split parts and the matched delimiter. + + If not found, then the first part is the full input string. + + Example:: + + >>> split_first('foo/bar?baz', '?/=') + ('foo', 'bar?baz', '/') + >>> split_first('foo/bar?baz', '123') + ('foo/bar?baz', '', None) + + Scales linearly with number of delims. Not ideal for large number of delims. + """ + min_idx = None + min_delim = None + for d in delims: + idx = s.find(d) + if idx < 0: + continue + + if min_idx is None or idx < min_idx: + min_idx = idx + min_delim = d + + if min_idx is None or min_idx < 0: + return s, "", None + + return s[:min_idx], s[min_idx + 1 :], min_delim + + +def _encode_invalid_chars(component, allowed_chars, encoding="utf-8"): + """Percent-encodes a URI component without reapplying + onto an already percent-encoded component. + """ + if component is None: + return component + + component = six.ensure_text(component) + + # Normalize existing percent-encoded bytes. + # Try to see if the component we're encoding is already percent-encoded + # so we can skip all '%' characters but still encode all others. + component, percent_encodings = PERCENT_RE.subn( + lambda match: match.group(0).upper(), component + ) + + uri_bytes = component.encode("utf-8", "surrogatepass") + is_percent_encoded = percent_encodings == uri_bytes.count(b"%") + encoded_component = bytearray() + + for i in range(0, len(uri_bytes)): + # Will return a single character bytestring on both Python 2 & 3 + byte = uri_bytes[i : i + 1] + byte_ord = ord(byte) + if (is_percent_encoded and byte == b"%") or ( + byte_ord < 128 and byte.decode() in allowed_chars + ): + encoded_component += byte + continue + encoded_component.extend(b"%" + (hex(byte_ord)[2:].encode().zfill(2).upper())) + + return encoded_component.decode(encoding) + + +def _remove_path_dot_segments(path): + # See http://tools.ietf.org/html/rfc3986#section-5.2.4 for pseudo-code + segments = path.split("/") # Turn the path into a list of segments + output = [] # Initialize the variable to use to store output + + for segment in segments: + # '.' is the current directory, so ignore it, it is superfluous + if segment == ".": + continue + # Anything other than '..', should be appended to the output + elif segment != "..": + output.append(segment) + # In this case segment == '..', if we can, we should pop the last + # element + elif output: + output.pop() + + # If the path starts with '/' and the output is empty or the first string + # is non-empty + if path.startswith("/") and (not output or output[0]): + output.insert(0, "") + + # If the path starts with '/.' or '/..' ensure we add one more empty + # string to add a trailing '/' + if path.endswith(("/.", "/..")): + output.append("") + + return "/".join(output) + + +def _normalize_host(host, scheme): + if host: + if isinstance(host, six.binary_type): + host = six.ensure_str(host) + + if scheme in NORMALIZABLE_SCHEMES: + is_ipv6 = IPV6_ADDRZ_RE.match(host) + if is_ipv6: + match = ZONE_ID_RE.search(host) + if match: + start, end = match.span(1) + zone_id = host[start:end] + + if zone_id.startswith("%25") and zone_id != "%25": + zone_id = zone_id[3:] + else: + zone_id = zone_id[1:] + zone_id = "%" + _encode_invalid_chars(zone_id, UNRESERVED_CHARS) + return host[:start].lower() + zone_id + host[end:] + else: + return host.lower() + elif not IPV4_RE.match(host): + return six.ensure_str( + b".".join([_idna_encode(label) for label in host.split(".")]) + ) + return host + + +def _idna_encode(name): + if name and any([ord(x) > 128 for x in name]): + try: + import idna + except ImportError: + six.raise_from( + LocationParseError("Unable to parse URL without the 'idna' module"), + None, + ) + try: + return idna.encode(name.lower(), strict=True, std3_rules=True) + except idna.IDNAError: + six.raise_from( + LocationParseError(u"Name '%s' is not a valid IDNA label" % name), None + ) + return name.lower().encode("ascii") + + +def _encode_target(target): + """Percent-encodes a request target so that there are no invalid characters""" + path, query = TARGET_RE.match(target).groups() + target = _encode_invalid_chars(path, PATH_CHARS) + query = _encode_invalid_chars(query, QUERY_CHARS) + if query is not None: + target += "?" + query + return target + + +def parse_url(url): + """ + Given a url, return a parsed :class:`.Url` namedtuple. Best-effort is + performed to parse incomplete urls. Fields not provided will be None. + This parser is RFC 3986 compliant. + + The parser logic and helper functions are based heavily on + work done in the ``rfc3986`` module. + + :param str url: URL to parse into a :class:`.Url` namedtuple. + + Partly backwards-compatible with :mod:`urlparse`. + + Example:: + + >>> parse_url('http://google.com/mail/') + Url(scheme='http', host='google.com', port=None, path='/mail/', ...) + >>> parse_url('google.com:80') + Url(scheme=None, host='google.com', port=80, path=None, ...) + >>> parse_url('/foo?bar') + Url(scheme=None, host=None, port=None, path='/foo', query='bar', ...) + """ + if not url: + # Empty + return Url() + + source_url = url + if not SCHEME_RE.search(url): + url = "//" + url + + try: + scheme, authority, path, query, fragment = URI_RE.match(url).groups() + normalize_uri = scheme is None or scheme.lower() in NORMALIZABLE_SCHEMES + + if scheme: + scheme = scheme.lower() + + if authority: + auth, _, host_port = authority.rpartition("@") + auth = auth or None + host, port = _HOST_PORT_RE.match(host_port).groups() + if auth and normalize_uri: + auth = _encode_invalid_chars(auth, USERINFO_CHARS) + if port == "": + port = None + else: + auth, host, port = None, None, None + + if port is not None: + port = int(port) + if not (0 <= port <= 65535): + raise LocationParseError(url) + + host = _normalize_host(host, scheme) + + if normalize_uri and path: + path = _remove_path_dot_segments(path) + path = _encode_invalid_chars(path, PATH_CHARS) + if normalize_uri and query: + query = _encode_invalid_chars(query, QUERY_CHARS) + if normalize_uri and fragment: + fragment = _encode_invalid_chars(fragment, FRAGMENT_CHARS) + + except (ValueError, AttributeError): + return six.raise_from(LocationParseError(source_url), None) + + # For the sake of backwards compatibility we put empty + # string values for path if there are any defined values + # beyond the path in the URL. + # TODO: Remove this when we break backwards compatibility. + if not path: + if query is not None or fragment is not None: + path = "" + else: + path = None + + # Ensure that each part of the URL is a `str` for + # backwards compatibility. + if isinstance(url, six.text_type): + ensure_func = six.ensure_text + else: + ensure_func = six.ensure_str + + def ensure_type(x): + return x if x is None else ensure_func(x) + + return Url( + scheme=ensure_type(scheme), + auth=ensure_type(auth), + host=ensure_type(host), + port=port, + path=ensure_type(path), + query=ensure_type(query), + fragment=ensure_type(fragment), + ) + + +def get_host(url): + """ + Deprecated. Use :func:`parse_url` instead. + """ + p = parse_url(url) + return p.scheme or "http", p.hostname, p.port diff --git a/openpype/hosts/fusion/vendor/urllib3/util/wait.py b/openpype/hosts/fusion/vendor/urllib3/util/wait.py new file mode 100644 index 0000000000..c280646c7b --- /dev/null +++ b/openpype/hosts/fusion/vendor/urllib3/util/wait.py @@ -0,0 +1,153 @@ +import errno +import select +import sys +from functools import partial + +try: + from time import monotonic +except ImportError: + from time import time as monotonic + +__all__ = ["NoWayToWaitForSocketError", "wait_for_read", "wait_for_write"] + + +class NoWayToWaitForSocketError(Exception): + pass + + +# How should we wait on sockets? +# +# There are two types of APIs you can use for waiting on sockets: the fancy +# modern stateful APIs like epoll/kqueue, and the older stateless APIs like +# select/poll. The stateful APIs are more efficient when you have a lots of +# sockets to keep track of, because you can set them up once and then use them +# lots of times. But we only ever want to wait on a single socket at a time +# and don't want to keep track of state, so the stateless APIs are actually +# more efficient. So we want to use select() or poll(). +# +# Now, how do we choose between select() and poll()? On traditional Unixes, +# select() has a strange calling convention that makes it slow, or fail +# altogether, for high-numbered file descriptors. The point of poll() is to fix +# that, so on Unixes, we prefer poll(). +# +# On Windows, there is no poll() (or at least Python doesn't provide a wrapper +# for it), but that's OK, because on Windows, select() doesn't have this +# strange calling convention; plain select() works fine. +# +# So: on Windows we use select(), and everywhere else we use poll(). We also +# fall back to select() in case poll() is somehow broken or missing. + +if sys.version_info >= (3, 5): + # Modern Python, that retries syscalls by default + def _retry_on_intr(fn, timeout): + return fn(timeout) + + +else: + # Old and broken Pythons. + def _retry_on_intr(fn, timeout): + if timeout is None: + deadline = float("inf") + else: + deadline = monotonic() + timeout + + while True: + try: + return fn(timeout) + # OSError for 3 <= pyver < 3.5, select.error for pyver <= 2.7 + except (OSError, select.error) as e: + # 'e.args[0]' incantation works for both OSError and select.error + if e.args[0] != errno.EINTR: + raise + else: + timeout = deadline - monotonic() + if timeout < 0: + timeout = 0 + if timeout == float("inf"): + timeout = None + continue + + +def select_wait_for_socket(sock, read=False, write=False, timeout=None): + if not read and not write: + raise RuntimeError("must specify at least one of read=True, write=True") + rcheck = [] + wcheck = [] + if read: + rcheck.append(sock) + if write: + wcheck.append(sock) + # When doing a non-blocking connect, most systems signal success by + # marking the socket writable. Windows, though, signals success by marked + # it as "exceptional". We paper over the difference by checking the write + # sockets for both conditions. (The stdlib selectors module does the same + # thing.) + fn = partial(select.select, rcheck, wcheck, wcheck) + rready, wready, xready = _retry_on_intr(fn, timeout) + return bool(rready or wready or xready) + + +def poll_wait_for_socket(sock, read=False, write=False, timeout=None): + if not read and not write: + raise RuntimeError("must specify at least one of read=True, write=True") + mask = 0 + if read: + mask |= select.POLLIN + if write: + mask |= select.POLLOUT + poll_obj = select.poll() + poll_obj.register(sock, mask) + + # For some reason, poll() takes timeout in milliseconds + def do_poll(t): + if t is not None: + t *= 1000 + return poll_obj.poll(t) + + return bool(_retry_on_intr(do_poll, timeout)) + + +def null_wait_for_socket(*args, **kwargs): + raise NoWayToWaitForSocketError("no select-equivalent available") + + +def _have_working_poll(): + # Apparently some systems have a select.poll that fails as soon as you try + # to use it, either due to strange configuration or broken monkeypatching + # from libraries like eventlet/greenlet. + try: + poll_obj = select.poll() + _retry_on_intr(poll_obj.poll, 0) + except (AttributeError, OSError): + return False + else: + return True + + +def wait_for_socket(*args, **kwargs): + # We delay choosing which implementation to use until the first time we're + # called. We could do it at import time, but then we might make the wrong + # decision if someone goes wild with monkeypatching select.poll after + # we're imported. + global wait_for_socket + if _have_working_poll(): + wait_for_socket = poll_wait_for_socket + elif hasattr(select, "select"): + wait_for_socket = select_wait_for_socket + else: # Platform-specific: Appengine. + wait_for_socket = null_wait_for_socket + return wait_for_socket(*args, **kwargs) + + +def wait_for_read(sock, timeout=None): + """Waits for reading to be available on a given socket. + Returns True if the socket is readable, or False if the timeout expired. + """ + return wait_for_socket(sock, read=True, timeout=timeout) + + +def wait_for_write(sock, timeout=None): + """Waits for writing to be available on a given socket. + Returns True if the socket is readable, or False if the timeout expired. + """ + return wait_for_socket(sock, write=True, timeout=timeout) diff --git a/openpype/hosts/harmony/plugins/load/load_template.py b/openpype/hosts/harmony/plugins/load/load_template.py index f3c69a9104..a78a1bf1ec 100644 --- a/openpype/hosts/harmony/plugins/load/load_template.py +++ b/openpype/hosts/harmony/plugins/load/load_template.py @@ -82,7 +82,6 @@ class TemplateLoader(load.LoaderPlugin): node = harmony.find_node_by_name(node_name, "GROUP") self_name = self.__class__.__name__ - update_and_replace = False if is_representation_from_latest(representation): self._set_green(node) else: diff --git a/openpype/hosts/harmony/plugins/publish/collect_farm_render.py b/openpype/hosts/harmony/plugins/publish/collect_farm_render.py index 5e9b9094a7..af825c052a 100644 --- a/openpype/hosts/harmony/plugins/publish/collect_farm_render.py +++ b/openpype/hosts/harmony/plugins/publish/collect_farm_render.py @@ -147,13 +147,13 @@ class CollectFarmRender(publish.AbstractCollectRender): attachTo=False, setMembers=[node], publish=info[4], - review=False, renderer=None, priority=50, name=node.split("/")[1], family="render.farm", families=["render.farm"], + farm=True, resolutionWidth=context.data["resolutionWidth"], resolutionHeight=context.data["resolutionHeight"], @@ -174,7 +174,6 @@ class CollectFarmRender(publish.AbstractCollectRender): outputFormat=info[1], outputStartFrame=info[3], leadingZeros=info[2], - toBeRenderedOn='deadline', ignoreFrameHandleCheck=True ) diff --git a/openpype/hosts/hiero/api/plugin.py b/openpype/hosts/hiero/api/plugin.py index 65a4009756..52f96261b2 100644 --- a/openpype/hosts/hiero/api/plugin.py +++ b/openpype/hosts/hiero/api/plugin.py @@ -317,20 +317,6 @@ class Spacer(QtWidgets.QWidget): self.setLayout(layout) -def get_reference_node_parents(ref): - """Return all parent reference nodes of reference node - - Args: - ref (str): reference node. - - Returns: - list: The upstream parent reference nodes. - - """ - parents = [] - return parents - - class SequenceLoader(LoaderPlugin): """A basic SequenceLoader for Resolve diff --git a/openpype/hosts/hiero/plugins/publish/collect_clip_effects.py b/openpype/hosts/hiero/plugins/publish/collect_clip_effects.py index d455ad4a4e..fcb1ab27a0 100644 --- a/openpype/hosts/hiero/plugins/publish/collect_clip_effects.py +++ b/openpype/hosts/hiero/plugins/publish/collect_clip_effects.py @@ -43,7 +43,6 @@ class CollectClipEffects(pyblish.api.InstancePlugin): if review and review_track_index == _track_index: continue for sitem in sub_track_items: - effect = None # make sure this subtrack item is relative of track item if ((track_item not in sitem.linkedItems()) and (len(sitem.linkedItems()) > 0)): @@ -53,7 +52,6 @@ class CollectClipEffects(pyblish.api.InstancePlugin): continue effect = self.add_effect(_track_index, sitem) - if effect: effects.update(effect) diff --git a/openpype/hosts/houdini/api/colorspace.py b/openpype/hosts/houdini/api/colorspace.py index 7047644225..cc40b9df1c 100644 --- a/openpype/hosts/houdini/api/colorspace.py +++ b/openpype/hosts/houdini/api/colorspace.py @@ -1,7 +1,7 @@ import attr import hou from openpype.hosts.houdini.api.lib import get_color_management_preferences - +from openpype.pipeline.colorspace import get_display_view_colorspace_name @attr.s class LayerMetadata(object): @@ -54,3 +54,16 @@ class ARenderProduct(object): ) ] return colorspace_data + + +def get_default_display_view_colorspace(): + """Returns the colorspace attribute of the default (display, view) pair. + + It's used for 'ociocolorspace' parm in OpenGL Node.""" + + prefs = get_color_management_preferences() + return get_display_view_colorspace_name( + config_path=prefs["config"], + display=prefs["display"], + view=prefs["view"] + ) diff --git a/openpype/hosts/houdini/api/creator_node_shelves.py b/openpype/hosts/houdini/api/creator_node_shelves.py index 7c6122cffe..1f9fef7417 100644 --- a/openpype/hosts/houdini/api/creator_node_shelves.py +++ b/openpype/hosts/houdini/api/creator_node_shelves.py @@ -57,28 +57,31 @@ def create_interactive(creator_identifier, **kwargs): list: The created instances. """ - - # TODO Use Qt instead - result, variant = hou.ui.readInput('Define variant name', - buttons=("Ok", "Cancel"), - initial_contents='Main', - title="Define variant", - help="Set the variant for the " - "publish instance", - close_choice=1) - if result == 1: - # User interrupted - return - variant = variant.strip() - if not variant: - raise RuntimeError("Empty variant value entered.") - host = registered_host() context = CreateContext(host) creator = context.manual_creators.get(creator_identifier) if not creator: - raise RuntimeError("Invalid creator identifier: " - "{}".format(creator_identifier)) + raise RuntimeError("Invalid creator identifier: {}".format( + creator_identifier) + ) + + # TODO Use Qt instead + result, variant = hou.ui.readInput( + "Define variant name", + buttons=("Ok", "Cancel"), + initial_contents=creator.get_default_variant(), + title="Define variant", + help="Set the variant for the publish instance", + close_choice=1 + ) + + if result == 1: + # User interrupted + return + + variant = variant.strip() + if not variant: + raise RuntimeError("Empty variant value entered.") # TODO: Once more elaborate unique create behavior should exist per Creator # instead of per network editor area then we should move this from here diff --git a/openpype/hosts/houdini/api/lib.py b/openpype/hosts/houdini/api/lib.py index b03f8c8fc1..a3f691e1fc 100644 --- a/openpype/hosts/houdini/api/lib.py +++ b/openpype/hosts/houdini/api/lib.py @@ -22,9 +22,12 @@ log = logging.getLogger(__name__) JSON_PREFIX = "JSON:::" -def get_asset_fps(): +def get_asset_fps(asset_doc=None): """Return current asset fps.""" - return get_current_project_asset()["data"].get("fps") + + if asset_doc is None: + asset_doc = get_current_project_asset(fields=["data.fps"]) + return asset_doc["data"]["fps"] def set_id(node, unique_id, overwrite=False): @@ -472,14 +475,19 @@ def maintained_selection(): def reset_framerange(): - """Set frame range to current asset""" + """Set frame range and FPS to current asset""" + # Get asset data project_name = get_current_project_name() asset_name = get_current_asset_name() # Get the asset ID from the database for the asset of current context asset_doc = get_asset_by_name(project_name, asset_name) asset_data = asset_doc["data"] + # Get FPS + fps = get_asset_fps(asset_doc) + + # Get Start and End Frames frame_start = asset_data.get("frameStart") frame_end = asset_data.get("frameEnd") @@ -493,6 +501,9 @@ def reset_framerange(): frame_start -= int(handle_start) frame_end += int(handle_end) + # Set frame range and FPS + print("Setting scene FPS to {}".format(int(fps))) + set_scene_fps(fps) hou.playbar.setFrameRange(frame_start, frame_end) hou.playbar.setPlaybackRange(frame_start, frame_end) hou.setFrame(frame_start) @@ -638,3 +649,101 @@ def get_color_management_preferences(): "display": hou.Color.ocio_defaultDisplay(), "view": hou.Color.ocio_defaultView() } + + +def get_obj_node_output(obj_node): + """Find output node. + + If the node has any output node return the + output node with the minimum `outputidx`. + When no output is present return the node + with the display flag set. If no output node is + detected then None is returned. + + Arguments: + node (hou.Node): The node to retrieve a single + the output node for. + + Returns: + Optional[hou.Node]: The child output node. + + """ + + outputs = obj_node.subnetOutputs() + if not outputs: + return + + elif len(outputs) == 1: + return outputs[0] + + else: + return min(outputs, + key=lambda node: node.evalParm('outputidx')) + + +def get_output_children(output_node, include_sops=True): + """Recursively return a list of all output nodes + contained in this node including this node. + + It works in a similar manner to output_node.allNodes(). + """ + out_list = [output_node] + + if output_node.childTypeCategory() == hou.objNodeTypeCategory(): + for child in output_node.children(): + out_list += get_output_children(child, include_sops=include_sops) + + elif include_sops and \ + output_node.childTypeCategory() == hou.sopNodeTypeCategory(): + out = get_obj_node_output(output_node) + if out: + out_list += [out] + + return out_list + + +def get_resolution_from_doc(doc): + """Get resolution from the given asset document. """ + + if not doc or "data" not in doc: + print("Entered document is not valid. \"{}\"".format(str(doc))) + return None + + resolution_width = doc["data"].get("resolutionWidth") + resolution_height = doc["data"].get("resolutionHeight") + + # Make sure both width and height are set + if resolution_width is None or resolution_height is None: + print("No resolution information found for \"{}\"".format(doc["name"])) + return None + + return int(resolution_width), int(resolution_height) + + +def set_camera_resolution(camera, asset_doc=None): + """Apply resolution to camera from asset document of the publish""" + + if not asset_doc: + asset_doc = get_current_project_asset() + + resolution = get_resolution_from_doc(asset_doc) + + if resolution: + print("Setting camera resolution: {} -> {}x{}".format( + camera.name(), resolution[0], resolution[1] + )) + camera.parm("resx").set(resolution[0]) + camera.parm("resy").set(resolution[1]) + + +def get_camera_from_container(container): + """Get camera from container node. """ + + cameras = container.recursiveGlob( + "*", + filter=hou.nodeTypeFilter.ObjCamera, + include_subnets=False + ) + + assert len(cameras) == 1, "Camera instance must have only one camera" + return cameras[0] diff --git a/openpype/hosts/houdini/api/pipeline.py b/openpype/hosts/houdini/api/pipeline.py index 8a26bbb504..6aa65deb89 100644 --- a/openpype/hosts/houdini/api/pipeline.py +++ b/openpype/hosts/houdini/api/pipeline.py @@ -14,6 +14,7 @@ import pyblish.api from openpype.pipeline import ( register_creator_plugin_path, register_loader_plugin_path, + register_inventory_action_path, AVALON_CONTAINER_ID, ) from openpype.pipeline.load import any_outdated_containers @@ -25,7 +26,6 @@ from openpype.lib import ( emit_event, ) -from .lib import get_asset_fps log = logging.getLogger("openpype.hosts.houdini") @@ -56,6 +56,7 @@ class HoudiniHost(HostBase, IWorkfileHost, ILoadHost, IPublishHost): pyblish.api.register_plugin_path(PUBLISH_PATH) register_loader_plugin_path(LOAD_PATH) register_creator_plugin_path(CREATE_PATH) + register_inventory_action_path(INVENTORY_PATH) log.info("Installing callbacks ... ") # register_event_callback("init", on_init) @@ -304,6 +305,28 @@ def on_save(): lib.set_id(node, new_id, overwrite=False) +def _show_outdated_content_popup(): + # Get main window + parent = lib.get_main_window() + if parent is None: + log.info("Skipping outdated content pop-up " + "because Houdini window can't be found.") + else: + from openpype.widgets import popup + + # Show outdated pop-up + def _on_show_inventory(): + from openpype.tools.utils import host_tools + host_tools.show_scene_inventory(parent=parent) + + dialog = popup.Popup(parent=parent) + dialog.setWindowTitle("Houdini scene has outdated content") + dialog.setMessage("There are outdated containers in " + "your Houdini scene.") + dialog.on_clicked.connect(_on_show_inventory) + dialog.show() + + def on_open(): if not hou.isUIAvailable(): @@ -317,28 +340,18 @@ def on_open(): lib.validate_fps() if any_outdated_containers(): - from openpype.widgets import popup - - log.warning("Scene has outdated content.") - - # Get main window parent = lib.get_main_window() if parent is None: - log.info("Skipping outdated content pop-up " - "because Houdini window can't be found.") + # When opening Houdini with last workfile on launch the UI hasn't + # initialized yet completely when the `on_open` callback triggers. + # We defer the dialog popup to wait for the UI to become available. + # We assume it will open because `hou.isUIAvailable()` returns True + import hdefereval + hdefereval.executeDeferred(_show_outdated_content_popup) else: + _show_outdated_content_popup() - # Show outdated pop-up - def _on_show_inventory(): - from openpype.tools.utils import host_tools - host_tools.show_scene_inventory(parent=parent) - - dialog = popup.Popup(parent=parent) - dialog.setWindowTitle("Houdini scene has outdated content") - dialog.setMessage("There are outdated containers in " - "your Houdini scene.") - dialog.on_clicked.connect(_on_show_inventory) - dialog.show() + log.warning("Scene has outdated content.") def on_new(): @@ -385,11 +398,6 @@ def _set_context_settings(): None """ - # Set new scene fps - fps = get_asset_fps() - print("Setting scene FPS to %i" % fps) - lib.set_scene_fps(fps) - lib.reset_framerange() diff --git a/openpype/hosts/houdini/api/plugin.py b/openpype/hosts/houdini/api/plugin.py index 70c837205e..730a627dc3 100644 --- a/openpype/hosts/houdini/api/plugin.py +++ b/openpype/hosts/houdini/api/plugin.py @@ -296,7 +296,7 @@ class HoudiniCreator(NewCreator, HoudiniCreatorBase): """ return [hou.ropNodeTypeCategory()] - def apply_settings(self, project_settings, system_settings): + def apply_settings(self, project_settings): """Method called on initialization of plugin to apply settings.""" settings_name = self.settings_name diff --git a/openpype/hosts/houdini/plugins/create/create_review.py b/openpype/hosts/houdini/plugins/create/create_review.py index ab06b30c35..60c34a358b 100644 --- a/openpype/hosts/houdini/plugins/create/create_review.py +++ b/openpype/hosts/houdini/plugins/create/create_review.py @@ -3,6 +3,9 @@ from openpype.hosts.houdini.api import plugin from openpype.lib import EnumDef, BoolDef, NumberDef +import os +import hou + class CreateReview(plugin.HoudiniCreator): """Review with OpenGL ROP""" @@ -13,7 +16,6 @@ class CreateReview(plugin.HoudiniCreator): icon = "video-camera" def create(self, subset_name, instance_data, pre_create_data): - import hou instance_data.pop("active", None) instance_data.update({"node_type": "opengl"}) @@ -82,6 +84,11 @@ class CreateReview(plugin.HoudiniCreator): instance_node.setParms(parms) + # Set OCIO Colorspace to the default output colorspace + # if there's OCIO + if os.getenv("OCIO"): + self.set_colorcorrect_to_default_view_space(instance_node) + to_lock = ["id", "family"] self.lock_parameters(instance_node, to_lock) @@ -123,3 +130,23 @@ class CreateReview(plugin.HoudiniCreator): minimum=0.0001, decimals=3) ] + + def set_colorcorrect_to_default_view_space(self, + instance_node): + """Set ociocolorspace to the default output space.""" + from openpype.hosts.houdini.api.colorspace import get_default_display_view_colorspace # noqa + + # set Color Correction parameter to OpenColorIO + instance_node.setParms({"colorcorrect": 2}) + + # Get default view space for ociocolorspace parm. + default_view_space = get_default_display_view_colorspace() + instance_node.setParms( + {"ociocolorspace": default_view_space} + ) + + self.log.debug( + "'OCIO Colorspace' parm on '{}' has been set to " + "the default view color space '{}'" + .format(instance_node, default_view_space) + ) diff --git a/openpype/hosts/houdini/plugins/create/create_staticmesh.py b/openpype/hosts/houdini/plugins/create/create_staticmesh.py new file mode 100644 index 0000000000..ea0b36f03f --- /dev/null +++ b/openpype/hosts/houdini/plugins/create/create_staticmesh.py @@ -0,0 +1,143 @@ +# -*- coding: utf-8 -*- +"""Creator for Unreal Static Meshes.""" +from openpype.hosts.houdini.api import plugin +from openpype.lib import BoolDef, EnumDef + +import hou + + +class CreateStaticMesh(plugin.HoudiniCreator): + """Static Meshes as FBX. """ + + identifier = "io.openpype.creators.houdini.staticmesh.fbx" + label = "Static Mesh (FBX)" + family = "staticMesh" + icon = "fa5s.cubes" + + default_variants = ["Main"] + + def create(self, subset_name, instance_data, pre_create_data): + + instance_data.update({"node_type": "filmboxfbx"}) + + instance = super(CreateStaticMesh, self).create( + subset_name, + instance_data, + pre_create_data) + + # get the created rop node + instance_node = hou.node(instance.get("instance_node")) + + # prepare parms + output_path = hou.text.expandString( + "$HIP/pyblish/{}.fbx".format(subset_name) + ) + + parms = { + "startnode": self.get_selection(), + "sopoutput": output_path, + # vertex cache format + "vcformat": pre_create_data.get("vcformat"), + "convertunits": pre_create_data.get("convertunits"), + # set render range to use frame range start-end frame + "trange": 1, + "createsubnetroot": pre_create_data.get("createsubnetroot") + } + + # set parms + instance_node.setParms(parms) + + # Lock any parameters in this list + to_lock = ["family", "id"] + self.lock_parameters(instance_node, to_lock) + + def get_network_categories(self): + return [ + hou.ropNodeTypeCategory(), + hou.sopNodeTypeCategory() + ] + + def get_pre_create_attr_defs(self): + """Add settings for users. """ + + attrs = super(CreateStaticMesh, self).get_pre_create_attr_defs() + createsubnetroot = BoolDef("createsubnetroot", + tooltip="Create an extra root for the " + "Export node when it's a " + "subnetwork. This causes the " + "exporting subnetwork node to be " + "represented in the FBX file.", + default=False, + label="Create Root for Subnet") + vcformat = EnumDef("vcformat", + items={ + 0: "Maya Compatible (MC)", + 1: "3DS MAX Compatible (PC2)" + }, + default=0, + label="Vertex Cache Format") + convert_units = BoolDef("convertunits", + tooltip="When on, the FBX is converted" + "from the current Houdini " + "system units to the native " + "FBX unit of centimeters.", + default=False, + label="Convert Units") + + return attrs + [createsubnetroot, vcformat, convert_units] + + def get_dynamic_data( + self, variant, task_name, asset_doc, project_name, host_name, instance + ): + """ + The default subset name templates for Unreal include {asset} and thus + we should pass that along as dynamic data. + """ + dynamic_data = super(CreateStaticMesh, self).get_dynamic_data( + variant, task_name, asset_doc, project_name, host_name, instance + ) + dynamic_data["asset"] = asset_doc["name"] + return dynamic_data + + def get_selection(self): + """Selection Logic. + + how self.selected_nodes should be processed to get + the desirable node from selection. + + Returns: + str : node path + """ + + selection = "" + + if self.selected_nodes: + selected_node = self.selected_nodes[0] + + # Accept sop level nodes (e.g. /obj/geo1/box1) + if isinstance(selected_node, hou.SopNode): + selection = selected_node.path() + self.log.debug( + "Valid SopNode selection, 'Export' in filmboxfbx" + " will be set to '%s'.", selected_node + ) + + # Accept object level nodes (e.g. /obj/geo1) + elif isinstance(selected_node, hou.ObjNode): + selection = selected_node.path() + self.log.debug( + "Valid ObjNode selection, 'Export' in filmboxfbx " + "will be set to the child path '%s'.", selection + ) + + else: + self.log.debug( + "Selection isn't valid. 'Export' in " + "filmboxfbx will be empty." + ) + else: + self.log.debug( + "No Selection. 'Export' in filmboxfbx will be empty." + ) + + return selection diff --git a/openpype/hosts/houdini/plugins/create/create_vbd_cache.py b/openpype/hosts/houdini/plugins/create/create_vbd_cache.py index c015cebd49..9c96e48e3a 100644 --- a/openpype/hosts/houdini/plugins/create/create_vbd_cache.py +++ b/openpype/hosts/houdini/plugins/create/create_vbd_cache.py @@ -33,7 +33,7 @@ class CreateVDBCache(plugin.HoudiniCreator): } if self.selected_nodes: - parms["soppath"] = self.selected_nodes[0].path() + parms["soppath"] = self.get_sop_node_path(self.selected_nodes[0]) instance_node.setParms(parms) @@ -42,3 +42,63 @@ class CreateVDBCache(plugin.HoudiniCreator): hou.ropNodeTypeCategory(), hou.sopNodeTypeCategory() ] + + def get_sop_node_path(self, selected_node): + """Get Sop Path of the selected node. + + Although Houdini allows ObjNode path on `sop_path` for the + the ROP node, we prefer it set to the SopNode path explicitly. + """ + + # Allow sop level paths (e.g. /obj/geo1/box1) + if isinstance(selected_node, hou.SopNode): + self.log.debug( + "Valid SopNode selection, 'SOP Path' in ROP will" + " be set to '%s'.", selected_node.path() + ) + return selected_node.path() + + # Allow object level paths to Geometry nodes (e.g. /obj/geo1) + # but do not allow other object level nodes types like cameras, etc. + elif isinstance(selected_node, hou.ObjNode) and \ + selected_node.type().name() == "geo": + + # Try to find output node. + sop_node = self.get_obj_output(selected_node) + if sop_node: + self.log.debug( + "Valid ObjNode selection, 'SOP Path' in ROP will " + "be set to the child path '%s'.", sop_node.path() + ) + return sop_node.path() + + self.log.debug( + "Selection isn't valid. 'SOP Path' in ROP will be empty." + ) + return "" + + def get_obj_output(self, obj_node): + """Try to find output node. + + If any output nodes are present, return the output node with + the minimum 'outputidx' + If no output nodes are present, return the node with display flag + If no nodes are present at all, return None + """ + + outputs = obj_node.subnetOutputs() + + # if obj_node is empty + if not outputs: + return + + # if obj_node has one output child whether its + # sop output node or a node with the render flag + elif len(outputs) == 1: + return outputs[0] + + # if there are more than one, then it has multiple output nodes + # return the one with the minimum 'outputidx' + else: + return min(outputs, + key=lambda node: node.evalParm('outputidx')) diff --git a/openpype/hosts/houdini/plugins/inventory/set_camera_resolution.py b/openpype/hosts/houdini/plugins/inventory/set_camera_resolution.py new file mode 100644 index 0000000000..18ececb019 --- /dev/null +++ b/openpype/hosts/houdini/plugins/inventory/set_camera_resolution.py @@ -0,0 +1,26 @@ +from openpype.pipeline import InventoryAction +from openpype.hosts.houdini.api.lib import ( + get_camera_from_container, + set_camera_resolution +) +from openpype.pipeline.context_tools import get_current_project_asset + + +class SetCameraResolution(InventoryAction): + + label = "Set Camera Resolution" + icon = "desktop" + color = "orange" + + @staticmethod + def is_compatible(container): + return ( + container.get("loader") == "CameraLoader" + ) + + def process(self, containers): + asset_doc = get_current_project_asset() + for container in containers: + node = container["node"] + camera = get_camera_from_container(node) + set_camera_resolution(camera, asset_doc) diff --git a/openpype/hosts/houdini/plugins/load/load_bgeo.py b/openpype/hosts/houdini/plugins/load/load_bgeo.py index 22680178c0..489bf944ed 100644 --- a/openpype/hosts/houdini/plugins/load/load_bgeo.py +++ b/openpype/hosts/houdini/plugins/load/load_bgeo.py @@ -34,7 +34,6 @@ class BgeoLoader(load.LoaderPlugin): # Create a new geo node container = obj.createNode("geo", node_name=node_name) - is_sequence = bool(context["representation"]["context"].get("frame")) # Remove the file node, it only loads static meshes # Houdini 17 has removed the file node from the geo node diff --git a/openpype/hosts/houdini/plugins/load/load_camera.py b/openpype/hosts/houdini/plugins/load/load_camera.py index 7b4a04809e..e16146a267 100644 --- a/openpype/hosts/houdini/plugins/load/load_camera.py +++ b/openpype/hosts/houdini/plugins/load/load_camera.py @@ -4,6 +4,13 @@ from openpype.pipeline import ( ) from openpype.hosts.houdini.api import pipeline +from openpype.hosts.houdini.api.lib import ( + set_camera_resolution, + get_camera_from_container +) + +import hou + ARCHIVE_EXPRESSION = ('__import__("_alembic_hom_extensions")' '.alembicGetCameraDict') @@ -25,7 +32,15 @@ def transfer_non_default_values(src, dest, ignore=None): channel expression and ignore certain Parm types. """ - import hou + + ignore_types = { + hou.parmTemplateType.Toggle, + hou.parmTemplateType.Menu, + hou.parmTemplateType.Button, + hou.parmTemplateType.FolderSet, + hou.parmTemplateType.Separator, + hou.parmTemplateType.Label, + } src.updateParmStates() @@ -62,14 +77,6 @@ def transfer_non_default_values(src, dest, ignore=None): continue # Ignore folders, separators, etc. - ignore_types = { - hou.parmTemplateType.Toggle, - hou.parmTemplateType.Menu, - hou.parmTemplateType.Button, - hou.parmTemplateType.FolderSet, - hou.parmTemplateType.Separator, - hou.parmTemplateType.Label, - } if parm.parmTemplate().type() in ignore_types: continue @@ -90,13 +97,8 @@ class CameraLoader(load.LoaderPlugin): def load(self, context, name=None, namespace=None, data=None): - import os - import hou - # Format file name, Houdini only wants forward slashes - file_path = self.filepath_from_context(context) - file_path = os.path.normpath(file_path) - file_path = file_path.replace("\\", "/") + file_path = self.filepath_from_context(context).replace("\\", "/") # Get the root node obj = hou.node("/obj") @@ -106,19 +108,21 @@ class CameraLoader(load.LoaderPlugin): node_name = "{}_{}".format(namespace, name) if namespace else name # Create a archive node - container = self.create_and_connect(obj, "alembicarchive", node_name) + node = self.create_and_connect(obj, "alembicarchive", node_name) # TODO: add FPS of project / asset - container.setParms({"fileName": file_path, - "channelRef": True}) + node.setParms({"fileName": file_path, "channelRef": True}) # Apply some magic - container.parm("buildHierarchy").pressButton() - container.moveToGoodPosition() + node.parm("buildHierarchy").pressButton() + node.moveToGoodPosition() # Create an alembic xform node - nodes = [container] + nodes = [node] + camera = get_camera_from_container(node) + self._match_maya_render_mask(camera) + set_camera_resolution(camera, asset_doc=context["asset"]) self[:] = nodes return pipeline.containerise(node_name, @@ -143,14 +147,14 @@ class CameraLoader(load.LoaderPlugin): # Store the cam temporarily next to the Alembic Archive # so that we can preserve parm values the user set on it # after build hierarchy was triggered. - old_camera = self._get_camera(node) + old_camera = get_camera_from_container(node) temp_camera = old_camera.copyTo(node.parent()) # Rebuild node.parm("buildHierarchy").pressButton() # Apply values to the new camera - new_camera = self._get_camera(node) + new_camera = get_camera_from_container(node) transfer_non_default_values(temp_camera, new_camera, # The hidden uniform scale attribute @@ -158,6 +162,9 @@ class CameraLoader(load.LoaderPlugin): # "icon_scale" just skip that completely ignore={"scale"}) + self._match_maya_render_mask(new_camera) + set_camera_resolution(new_camera) + temp_camera.destroy() def remove(self, container): @@ -165,15 +172,6 @@ class CameraLoader(load.LoaderPlugin): node = container["node"] node.destroy() - def _get_camera(self, node): - import hou - cameras = node.recursiveGlob("*", - filter=hou.nodeTypeFilter.ObjCamera, - include_subnets=False) - - assert len(cameras) == 1, "Camera instance must have only one camera" - return cameras[0] - def create_and_connect(self, node, node_type, name=None): """Create a node within a node which and connect it to the input @@ -194,5 +192,20 @@ class CameraLoader(load.LoaderPlugin): new_node.moveToGoodPosition() return new_node - def switch(self, container, representation): - self.update(container, representation) + def _match_maya_render_mask(self, camera): + """Workaround to match Maya render mask in Houdini""" + + # print("Setting match maya render mask ") + parm = camera.parm("aperture") + expression = parm.expression() + expression = expression.replace("return ", "aperture = ") + expression += """ +# Match maya render mask (logic from Houdini's own FBX importer) +node = hou.pwd() +resx = node.evalParm('resx') +resy = node.evalParm('resy') +aspect = node.evalParm('aspect') +aperture *= min(1, (resx / resy * aspect) / 1.5) +return aperture +""" + parm.setExpression(expression, language=hou.exprLanguage.Python) diff --git a/openpype/hosts/houdini/plugins/load/load_fbx.py b/openpype/hosts/houdini/plugins/load/load_fbx.py new file mode 100644 index 0000000000..cac22d62d4 --- /dev/null +++ b/openpype/hosts/houdini/plugins/load/load_fbx.py @@ -0,0 +1,139 @@ +# -*- coding: utf-8 -*- +"""Fbx Loader for houdini. """ +from openpype.pipeline import ( + load, + get_representation_path, +) +from openpype.hosts.houdini.api import pipeline + + +class FbxLoader(load.LoaderPlugin): + """Load fbx files. """ + + label = "Load FBX" + icon = "code-fork" + color = "orange" + + order = -10 + + families = ["staticMesh", "fbx"] + representations = ["fbx"] + + def load(self, context, name=None, namespace=None, data=None): + + # get file path from context + file_path = self.filepath_from_context(context) + file_path = file_path.replace("\\", "/") + + # get necessary data + namespace, node_name = self.get_node_name(context, name, namespace) + + # create load tree + nodes = self.create_load_node_tree(file_path, node_name, name) + + self[:] = nodes + + # Call containerise function which does some automations for you + # like moving created nodes to the AVALON_CONTAINERS subnetwork + containerised_nodes = pipeline.containerise( + node_name, + namespace, + nodes, + context, + self.__class__.__name__, + suffix="", + ) + + return containerised_nodes + + def update(self, container, representation): + + node = container["node"] + try: + file_node = next( + n for n in node.children() if n.type().name() == "file" + ) + except StopIteration: + self.log.error("Could not find node of type `file`") + return + + # Update the file path from representation + file_path = get_representation_path(representation) + file_path = file_path.replace("\\", "/") + + file_node.setParms({"file": file_path}) + + # Update attribute + node.setParms({"representation": str(representation["_id"])}) + + def remove(self, container): + + node = container["node"] + node.destroy() + + def switch(self, container, representation): + self.update(container, representation) + + def get_node_name(self, context, name=None, namespace=None): + """Define node name.""" + + if not namespace: + namespace = context["asset"]["name"] + + if namespace: + node_name = "{}_{}".format(namespace, name) + else: + node_name = name + + return namespace, node_name + + def create_load_node_tree(self, file_path, node_name, subset_name): + """Create Load network. + + you can start building your tree at any obj level. + it'll be much easier to build it in the root obj level. + + Afterwards, your tree will be automatically moved to + '/obj/AVALON_CONTAINERS' subnetwork. + """ + import hou + + # Get the root obj level + obj = hou.node("/obj") + + # Create a new obj geo node + parent_node = obj.createNode("geo", node_name=node_name) + + # In older houdini, + # when reating a new obj geo node, a default file node will be + # automatically created. + # so, we will delete it if exists. + file_node = parent_node.node("file1") + if file_node: + file_node.destroy() + + # Create a new file node + file_node = parent_node.createNode("file", node_name=node_name) + file_node.setParms({"file": file_path}) + + # Create attribute delete + attribdelete_name = "attribdelete_{}".format(subset_name) + attribdelete = parent_node.createNode("attribdelete", + node_name=attribdelete_name) + attribdelete.setParms({"ptdel": "fbx_*"}) + attribdelete.setInput(0, file_node) + + # Create a Null node + null_name = "OUT_{}".format(subset_name) + null = parent_node.createNode("null", node_name=null_name) + null.setInput(0, attribdelete) + + # Ensure display flag is on the file_node input node and not on the OUT + # node to optimize "debug" displaying in the viewport. + file_node.setDisplayFlag(True) + + # Set new position for children nodes + parent_node.layoutChildren() + + # Return all the nodes + return [parent_node, file_node, attribdelete, null] diff --git a/openpype/hosts/houdini/plugins/load/load_hda.py b/openpype/hosts/houdini/plugins/load/load_hda.py index 57edc341a3..9630716253 100644 --- a/openpype/hosts/houdini/plugins/load/load_hda.py +++ b/openpype/hosts/houdini/plugins/load/load_hda.py @@ -59,6 +59,9 @@ class HdaLoader(load.LoaderPlugin): def_paths = [d.libraryFilePath() for d in defs] new = def_paths.index(file_path) defs[new].setIsPreferred(True) + hda_node.setParms({ + "representation": str(representation["_id"]) + }) def remove(self, container): node = container["node"] diff --git a/openpype/hosts/houdini/plugins/publish/collect_output_node.py b/openpype/hosts/houdini/plugins/publish/collect_output_node.py index 601ed17b39..bca3d9fdc1 100644 --- a/openpype/hosts/houdini/plugins/publish/collect_output_node.py +++ b/openpype/hosts/houdini/plugins/publish/collect_output_node.py @@ -1,5 +1,7 @@ import pyblish.api +from openpype.pipeline.publish import KnownPublishError + class CollectOutputSOPPath(pyblish.api.InstancePlugin): """Collect the out node's SOP/COP Path value.""" @@ -12,7 +14,8 @@ class CollectOutputSOPPath(pyblish.api.InstancePlugin): "imagesequence", "usd", "usdrender", - "redshiftproxy" + "redshiftproxy", + "staticMesh" ] hosts = ["houdini"] @@ -57,9 +60,13 @@ class CollectOutputSOPPath(pyblish.api.InstancePlugin): elif node_type == "Redshift_Proxy_Output": out_node = node.parm("RS_archive_sopPath").evalAsNode() + + elif node_type == "filmboxfbx": + out_node = node.parm("startnode").evalAsNode() + else: - raise ValueError( - "ROP node type '%s' is" " not supported." % node_type + raise KnownPublishError( + "ROP node type '{}' is not supported.".format(node_type) ) if not out_node: diff --git a/openpype/hosts/houdini/plugins/publish/collect_staticmesh_type.py b/openpype/hosts/houdini/plugins/publish/collect_staticmesh_type.py new file mode 100644 index 0000000000..db9efec7a1 --- /dev/null +++ b/openpype/hosts/houdini/plugins/publish/collect_staticmesh_type.py @@ -0,0 +1,20 @@ +# -*- coding: utf-8 -*- +"""Collector for staticMesh types. """ + +import pyblish.api + + +class CollectStaticMeshType(pyblish.api.InstancePlugin): + """Collect data type for fbx instance.""" + + hosts = ["houdini"] + families = ["staticMesh"] + label = "Collect type of staticMesh" + + order = pyblish.api.CollectorOrder + + def process(self, instance): + + if instance.data["creator_identifier"] == "io.openpype.creators.houdini.staticmesh.fbx": # noqa: E501 + # Marking this instance as FBX triggers the FBX extractor. + instance.data["families"] += ["fbx"] diff --git a/openpype/hosts/houdini/plugins/publish/collect_vray_rop.py b/openpype/hosts/houdini/plugins/publish/collect_vray_rop.py index d4fe37f993..277f922ba4 100644 --- a/openpype/hosts/houdini/plugins/publish/collect_vray_rop.py +++ b/openpype/hosts/houdini/plugins/publish/collect_vray_rop.py @@ -80,14 +80,9 @@ class CollectVrayROPRenderProducts(pyblish.api.InstancePlugin): def get_beauty_render_product(self, prefix, suffix=""): """Return the beauty output filename if render element enabled """ + # Remove aov suffix from the product: `prefix.aov_suffix` -> `prefix` aov_parm = ".{}".format(suffix) - beauty_product = None - if aov_parm in prefix: - beauty_product = prefix.replace(aov_parm, "") - else: - beauty_product = prefix - - return beauty_product + return prefix.replace(aov_parm, "") def get_render_element_name(self, node, prefix, suffix=""): """Return the output filename using the AOV prefix and suffix diff --git a/openpype/hosts/houdini/plugins/publish/extract_fbx.py b/openpype/hosts/houdini/plugins/publish/extract_fbx.py new file mode 100644 index 0000000000..7993b3352f --- /dev/null +++ b/openpype/hosts/houdini/plugins/publish/extract_fbx.py @@ -0,0 +1,53 @@ +# -*- coding: utf-8 -*- +"""Fbx Extractor for houdini. """ + +import os +import pyblish.api +from openpype.pipeline import publish +from openpype.hosts.houdini.api.lib import render_rop + +import hou + + +class ExtractFBX(publish.Extractor): + + label = "Extract FBX" + families = ["fbx"] + hosts = ["houdini"] + + order = pyblish.api.ExtractorOrder + 0.1 + + def process(self, instance): + + # get rop node + ropnode = hou.node(instance.data.get("instance_node")) + output_file = ropnode.evalParm("sopoutput") + + # get staging_dir and file_name + staging_dir = os.path.normpath(os.path.dirname(output_file)) + file_name = os.path.basename(output_file) + + # render rop + self.log.debug("Writing FBX '%s' to '%s'", file_name, staging_dir) + render_rop(ropnode) + + # prepare representation + representation = { + "name": "fbx", + "ext": "fbx", + "files": file_name, + "stagingDir": staging_dir + } + + # A single frame may also be rendered without start/end frame. + if "frameStart" in instance.data and "frameEnd" in instance.data: + representation["frameStart"] = instance.data["frameStart"] + representation["frameEnd"] = instance.data["frameEnd"] + + # set value type for 'representations' key to list + if "representations" not in instance.data: + instance.data["representations"] = [] + + # update instance data + instance.data["stagingDir"] = staging_dir + instance.data["representations"].append(representation) diff --git a/openpype/hosts/houdini/plugins/publish/increment_current_file.py b/openpype/hosts/houdini/plugins/publish/increment_current_file.py index 2493b28bc1..3569de7693 100644 --- a/openpype/hosts/houdini/plugins/publish/increment_current_file.py +++ b/openpype/hosts/houdini/plugins/publish/increment_current_file.py @@ -2,7 +2,7 @@ import pyblish.api from openpype.lib import version_up from openpype.pipeline import registered_host -from openpype.action import get_errored_plugins_from_data +from openpype.pipeline.publish import get_errored_plugins_from_context from openpype.hosts.houdini.api import HoudiniHost from openpype.pipeline.publish import KnownPublishError @@ -27,7 +27,7 @@ class IncrementCurrentFile(pyblish.api.ContextPlugin): def process(self, context): - errored_plugins = get_errored_plugins_from_data(context) + errored_plugins = get_errored_plugins_from_context(context) if any( plugin.__name__ == "HoudiniSubmitPublishDeadline" for plugin in errored_plugins @@ -40,9 +40,10 @@ class IncrementCurrentFile(pyblish.api.ContextPlugin): # Filename must not have changed since collecting host = registered_host() # type: HoudiniHost current_file = host.current_file() - assert ( - context.data["currentFile"] == current_file - ), "Collected filename mismatches from current scene name." + if context.data["currentFile"] != current_file: + raise KnownPublishError( + "Collected filename mismatches from current scene name." + ) new_filepath = version_up(current_file) host.save_workfile(new_filepath) diff --git a/openpype/hosts/houdini/plugins/publish/validate_animation_settings.py b/openpype/hosts/houdini/plugins/publish/validate_animation_settings.py index 4878738ed3..79387fbef5 100644 --- a/openpype/hosts/houdini/plugins/publish/validate_animation_settings.py +++ b/openpype/hosts/houdini/plugins/publish/validate_animation_settings.py @@ -1,5 +1,6 @@ import pyblish.api +from openpype.pipeline.publish import PublishValidationError from openpype.hosts.houdini.api import lib import hou @@ -30,7 +31,7 @@ class ValidateAnimationSettings(pyblish.api.InstancePlugin): invalid = self.get_invalid(instance) if invalid: - raise RuntimeError( + raise PublishValidationError( "Output settings do no match for '%s'" % instance ) diff --git a/openpype/hosts/houdini/plugins/publish/validate_fbx_output_node.py b/openpype/hosts/houdini/plugins/publish/validate_fbx_output_node.py new file mode 100644 index 0000000000..894dad7d72 --- /dev/null +++ b/openpype/hosts/houdini/plugins/publish/validate_fbx_output_node.py @@ -0,0 +1,140 @@ +# -*- coding: utf-8 -*- +import pyblish.api +from openpype.pipeline import PublishValidationError +from openpype.hosts.houdini.api.action import ( + SelectInvalidAction, + SelectROPAction, +) +from openpype.hosts.houdini.api.lib import get_obj_node_output +import hou + + +class ValidateFBXOutputNode(pyblish.api.InstancePlugin): + """Validate the instance Output Node. + + This will ensure: + - The Output Node Path is set. + - The Output Node Path refers to an existing object. + - The Output Node is a Sop or Obj node. + - The Output Node has geometry data. + - The Output Node doesn't include invalid primitive types. + """ + + order = pyblish.api.ValidatorOrder + families = ["fbx"] + hosts = ["houdini"] + label = "Validate FBX Output Node" + actions = [SelectROPAction, SelectInvalidAction] + + def process(self, instance): + + invalid = self.get_invalid(instance) + if invalid: + nodes = [n.path() for n in invalid] + raise PublishValidationError( + "See log for details. " + "Invalid nodes: {0}".format(nodes), + title="Invalid output node(s)" + ) + + @classmethod + def get_invalid(cls, instance): + output_node = instance.data.get("output_node") + + # Check if The Output Node Path is set and + # refers to an existing object. + if output_node is None: + rop_node = hou.node(instance.data["instance_node"]) + cls.log.error( + "Output node in '%s' does not exist. " + "Ensure a valid output path is set.", rop_node.path() + ) + + return [rop_node] + + # Check if the Output Node is a Sop or an Obj node + # also, list all sop output nodes inside as well as + # invalid empty nodes. + all_out_sops = [] + invalid = [] + + # if output_node is an ObjSubnet or an ObjNetwork + if output_node.childTypeCategory() == hou.objNodeTypeCategory(): + for node in output_node.allSubChildren(): + if node.type().name() == "geo": + out = get_obj_node_output(node) + if out: + all_out_sops.append(out) + else: + invalid.append(node) # empty_objs + cls.log.error( + "Geo Obj Node '%s' is empty!", + node.path() + ) + if not all_out_sops: + invalid.append(output_node) # empty_objs + cls.log.error( + "Output Node '%s' is empty!", + node.path() + ) + + # elif output_node is an ObjNode + elif output_node.type().name() == "geo": + out = get_obj_node_output(output_node) + if out: + all_out_sops.append(out) + else: + invalid.append(node) # empty_objs + cls.log.error( + "Output Node '%s' is empty!", + node.path() + ) + + # elif output_node is a SopNode + elif output_node.type().category().name() == "Sop": + all_out_sops.append(output_node) + + # Then it's a wrong node type + else: + cls.log.error( + "Output node %s is not a SOP or OBJ Geo or OBJ SubNet node. " + "Instead found category type: %s %s", + output_node.path(), output_node.type().category().name(), + output_node.type().name() + ) + return [output_node] + + # Check if all output sop nodes have geometry + # and don't contain invalid prims + invalid_prim_types = ["VDB", "Volume"] + for sop_node in all_out_sops: + # Empty Geometry test + if not hasattr(sop_node, "geometry"): + invalid.append(sop_node) # empty_geometry + cls.log.error( + "Sop node '%s' doesn't include any prims.", + sop_node.path() + ) + continue + + frame = instance.data.get("frameStart", 0) + geo = sop_node.geometryAtFrame(frame) + if len(geo.iterPrims()) == 0: + invalid.append(sop_node) # empty_geometry + cls.log.error( + "Sop node '%s' doesn't include any prims.", + sop_node.path() + ) + continue + + # Invalid Prims test + for prim_type in invalid_prim_types: + if geo.countPrimType(prim_type) > 0: + invalid.append(sop_node) # invalid_prims + cls.log.error( + "Sop node '%s' includes invalid prims of type '%s'.", + sop_node.path(), prim_type + ) + + if invalid: + return invalid diff --git a/openpype/hosts/houdini/plugins/publish/validate_mesh_is_static.py b/openpype/hosts/houdini/plugins/publish/validate_mesh_is_static.py new file mode 100644 index 0000000000..b499682e0b --- /dev/null +++ b/openpype/hosts/houdini/plugins/publish/validate_mesh_is_static.py @@ -0,0 +1,59 @@ +# -*- coding: utf-8 -*- +"""Validator for correct naming of Static Meshes.""" +import pyblish.api +from openpype.pipeline import ( + PublishValidationError, + OptionalPyblishPluginMixin +) +from openpype.pipeline.publish import ValidateContentsOrder + +from openpype.hosts.houdini.api.action import SelectInvalidAction +from openpype.hosts.houdini.api.lib import get_output_children + + +class ValidateMeshIsStatic(pyblish.api.InstancePlugin, + OptionalPyblishPluginMixin): + """Validate mesh is static. + + It checks if output node is time dependent. + """ + + families = ["staticMesh"] + hosts = ["houdini"] + label = "Validate Mesh is Static" + order = ValidateContentsOrder + 0.1 + actions = [SelectInvalidAction] + + def process(self, instance): + + invalid = self.get_invalid(instance) + if invalid: + nodes = [n.path() for n in invalid] + raise PublishValidationError( + "See log for details. " + "Invalid nodes: {0}".format(nodes) + ) + + @classmethod + def get_invalid(cls, instance): + + invalid = [] + + output_node = instance.data.get("output_node") + if output_node is None: + cls.log.debug( + "No Output Node, skipping check.." + ) + return + + all_outputs = get_output_children(output_node) + + for output in all_outputs: + if output.isTimeDependent(): + invalid.append(output) + cls.log.error( + "Output node '%s' is time dependent.", + output.path() + ) + + return invalid diff --git a/openpype/hosts/houdini/plugins/publish/validate_remote_publish.py b/openpype/hosts/houdini/plugins/publish/validate_remote_publish.py index 4e8e5fc0e8..4f71d79382 100644 --- a/openpype/hosts/houdini/plugins/publish/validate_remote_publish.py +++ b/openpype/hosts/houdini/plugins/publish/validate_remote_publish.py @@ -36,11 +36,11 @@ class ValidateRemotePublishOutNode(pyblish.api.ContextPlugin): if node.parm("shellexec").eval(): self.raise_error("Must not execute in shell") if node.parm("prerender").eval() != cmd: - self.raise_error(("REMOTE_PUBLISH node does not have " - "correct prerender script.")) + self.raise_error("REMOTE_PUBLISH node does not have " + "correct prerender script.") if node.parm("lprerender").eval() != "python": - self.raise_error(("REMOTE_PUBLISH node prerender script " - "type not set to 'python'")) + self.raise_error("REMOTE_PUBLISH node prerender script " + "type not set to 'python'") @classmethod def repair(cls, context): @@ -48,5 +48,4 @@ class ValidateRemotePublishOutNode(pyblish.api.ContextPlugin): lib.create_remote_publish_node(force=True) def raise_error(self, message): - self.log.error(message) - raise PublishValidationError(message, title=self.label) + raise PublishValidationError(message) diff --git a/openpype/hosts/houdini/plugins/publish/validate_review_colorspace.py b/openpype/hosts/houdini/plugins/publish/validate_review_colorspace.py new file mode 100644 index 0000000000..03ecd1b052 --- /dev/null +++ b/openpype/hosts/houdini/plugins/publish/validate_review_colorspace.py @@ -0,0 +1,90 @@ +# -*- coding: utf-8 -*- +import pyblish.api +from openpype.pipeline import ( + PublishValidationError, + OptionalPyblishPluginMixin +) +from openpype.pipeline.publish import RepairAction +from openpype.hosts.houdini.api.action import SelectROPAction + +import os +import hou + + +class SetDefaultViewSpaceAction(RepairAction): + label = "Set default view colorspace" + icon = "mdi.monitor" + + +class ValidateReviewColorspace(pyblish.api.InstancePlugin, + OptionalPyblishPluginMixin): + """Validate Review Colorspace parameters. + + It checks if 'OCIO Colorspace' parameter was set to valid value. + """ + + order = pyblish.api.ValidatorOrder + 0.1 + families = ["review"] + hosts = ["houdini"] + label = "Validate Review Colorspace" + actions = [SetDefaultViewSpaceAction, SelectROPAction] + + optional = True + + def process(self, instance): + + if not self.is_active(instance.data): + return + + if os.getenv("OCIO") is None: + self.log.debug( + "Using Houdini's Default Color Management, " + " skipping check.." + ) + return + + rop_node = hou.node(instance.data["instance_node"]) + if rop_node.evalParm("colorcorrect") != 2: + # any colorspace settings other than default requires + # 'Color Correct' parm to be set to 'OpenColorIO' + raise PublishValidationError( + "'Color Correction' parm on '{}' ROP must be set to" + " 'OpenColorIO'".format(rop_node.path()) + ) + + if rop_node.evalParm("ociocolorspace") not in \ + hou.Color.ocio_spaces(): + + raise PublishValidationError( + "Invalid value: Colorspace name doesn't exist.\n" + "Check 'OCIO Colorspace' parameter on '{}' ROP" + .format(rop_node.path()) + ) + + @classmethod + def repair(cls, instance): + """Set Default View Space Action. + + It is a helper action more than a repair action, + used to set colorspace on opengl node to the default view. + """ + from openpype.hosts.houdini.api.colorspace import get_default_display_view_colorspace # noqa + + rop_node = hou.node(instance.data["instance_node"]) + + if rop_node.evalParm("colorcorrect") != 2: + rop_node.setParms({"colorcorrect": 2}) + cls.log.debug( + "'Color Correction' parm on '{}' has been set to" + " 'OpenColorIO'".format(rop_node.path()) + ) + + # Get default view colorspace name + default_view_space = get_default_display_view_colorspace() + + rop_node.setParms({"ociocolorspace": default_view_space}) + cls.log.info( + "'OCIO Colorspace' parm on '{}' has been set to " + "the default view color space '{}'" + .format(rop_node, default_view_space) + ) diff --git a/openpype/hosts/houdini/plugins/publish/validate_sop_output_node.py b/openpype/hosts/houdini/plugins/publish/validate_sop_output_node.py index d9dee38680..9590e37d26 100644 --- a/openpype/hosts/houdini/plugins/publish/validate_sop_output_node.py +++ b/openpype/hosts/houdini/plugins/publish/validate_sop_output_node.py @@ -24,7 +24,7 @@ class ValidateSopOutputNode(pyblish.api.InstancePlugin): order = pyblish.api.ValidatorOrder families = ["pointcache", "vdbcache"] hosts = ["houdini"] - label = "Validate Output Node" + label = "Validate Output Node (SOP)" actions = [SelectROPAction, SelectInvalidAction] def process(self, instance): diff --git a/openpype/hosts/houdini/plugins/publish/validate_subset_name.py b/openpype/hosts/houdini/plugins/publish/validate_subset_name.py new file mode 100644 index 0000000000..bb3648f361 --- /dev/null +++ b/openpype/hosts/houdini/plugins/publish/validate_subset_name.py @@ -0,0 +1,93 @@ +# -*- coding: utf-8 -*- +"""Validator for correct naming of Static Meshes.""" +import pyblish.api +from openpype.pipeline import ( + PublishValidationError, + OptionalPyblishPluginMixin +) +from openpype.pipeline.publish import ( + ValidateContentsOrder, + RepairAction, +) +from openpype.hosts.houdini.api.action import SelectInvalidAction +from openpype.pipeline.create import get_subset_name + +import hou + + +class FixSubsetNameAction(RepairAction): + label = "Fix Subset Name" + + +class ValidateSubsetName(pyblish.api.InstancePlugin, + OptionalPyblishPluginMixin): + """Validate Subset name. + + """ + + families = ["staticMesh"] + hosts = ["houdini"] + label = "Validate Subset Name" + order = ValidateContentsOrder + 0.1 + actions = [FixSubsetNameAction, SelectInvalidAction] + + optional = True + + def process(self, instance): + + if not self.is_active(instance.data): + return + + invalid = self.get_invalid(instance) + if invalid: + nodes = [n.path() for n in invalid] + raise PublishValidationError( + "See log for details. " + "Invalid nodes: {0}".format(nodes) + ) + + @classmethod + def get_invalid(cls, instance): + + invalid = [] + + rop_node = hou.node(instance.data["instance_node"]) + + # Check subset name + subset_name = get_subset_name( + family=instance.data["family"], + variant=instance.data["variant"], + task_name=instance.data["task"], + asset_doc=instance.data["assetEntity"], + dynamic_data={"asset": instance.data["asset"]} + ) + + if instance.data.get("subset") != subset_name: + invalid.append(rop_node) + cls.log.error( + "Invalid subset name on rop node '%s' should be '%s'.", + rop_node.path(), subset_name + ) + + return invalid + + @classmethod + def repair(cls, instance): + rop_node = hou.node(instance.data["instance_node"]) + + # Check subset name + subset_name = get_subset_name( + family=instance.data["family"], + variant=instance.data["variant"], + task_name=instance.data["task"], + asset_doc=instance.data["assetEntity"], + dynamic_data={"asset": instance.data["asset"]} + ) + + instance.data["subset"] = subset_name + rop_node.parm("subset").set(subset_name) + + cls.log.debug( + "Subset name on rop node '%s' has been set to '%s'.", + rop_node.path(), subset_name + ) diff --git a/openpype/hosts/houdini/plugins/publish/validate_unreal_staticmesh_naming.py b/openpype/hosts/houdini/plugins/publish/validate_unreal_staticmesh_naming.py new file mode 100644 index 0000000000..ae3c7e5602 --- /dev/null +++ b/openpype/hosts/houdini/plugins/publish/validate_unreal_staticmesh_naming.py @@ -0,0 +1,97 @@ +# -*- coding: utf-8 -*- +"""Validator for correct naming of Static Meshes.""" +import pyblish.api +from openpype.pipeline import ( + PublishValidationError, + OptionalPyblishPluginMixin +) +from openpype.pipeline.publish import ValidateContentsOrder + +from openpype.hosts.houdini.api.action import SelectInvalidAction +from openpype.hosts.houdini.api.lib import get_output_children + +import hou + + +class ValidateUnrealStaticMeshName(pyblish.api.InstancePlugin, + OptionalPyblishPluginMixin): + """Validate name of Unreal Static Mesh. + + This validator checks if output node name has a collision prefix: + - UBX + - UCP + - USP + - UCX + + This validator also checks if subset name is correct + - {static mesh prefix}_{Asset-Name}{Variant}. + + """ + + families = ["staticMesh"] + hosts = ["houdini"] + label = "Unreal Static Mesh Name (FBX)" + order = ValidateContentsOrder + 0.1 + actions = [SelectInvalidAction] + + optional = True + collision_prefixes = [] + static_mesh_prefix = "" + + @classmethod + def apply_settings(cls, project_settings, system_settings): + + settings = ( + project_settings["houdini"]["create"]["CreateStaticMesh"] + ) + cls.collision_prefixes = settings["collision_prefixes"] + cls.static_mesh_prefix = settings["static_mesh_prefix"] + + def process(self, instance): + + if not self.is_active(instance.data): + return + + invalid = self.get_invalid(instance) + if invalid: + nodes = [n.path() for n in invalid] + raise PublishValidationError( + "See log for details. " + "Invalid nodes: {0}".format(nodes) + ) + + @classmethod + def get_invalid(cls, instance): + + invalid = [] + + rop_node = hou.node(instance.data["instance_node"]) + output_node = instance.data.get("output_node") + if output_node is None: + cls.log.debug( + "No Output Node, skipping check.." + ) + return + + if rop_node.evalParm("buildfrompath"): + # This validator doesn't support naming check if + # building hierarchy from path' is used + cls.log.info( + "Using 'Build Hierarchy from Path Attribute', skipping check.." + ) + return + + # Check nodes names + all_outputs = get_output_children(output_node, include_sops=False) + for output in all_outputs: + for prefix in cls.collision_prefixes: + if output.name().startswith(prefix): + invalid.append(output) + cls.log.error( + "Invalid node name: Node '%s' " + "includes a collision prefix '%s'", + output.path(), prefix + ) + break + + return invalid diff --git a/openpype/hosts/houdini/plugins/publish/validate_usd_render_product_names.py b/openpype/hosts/houdini/plugins/publish/validate_usd_render_product_names.py index 02c44ab94e..1daa96f2b9 100644 --- a/openpype/hosts/houdini/plugins/publish/validate_usd_render_product_names.py +++ b/openpype/hosts/houdini/plugins/publish/validate_usd_render_product_names.py @@ -24,7 +24,7 @@ class ValidateUSDRenderProductNames(pyblish.api.InstancePlugin): if not os.path.isabs(filepath): invalid.append( - "Output file path is not " "absolute path: %s" % filepath + "Output file path is not absolute path: %s" % filepath ) if invalid: diff --git a/openpype/hosts/houdini/startup/MainMenuCommon.xml b/openpype/hosts/houdini/startup/MainMenuCommon.xml index 47a4653d5d..5818a117eb 100644 --- a/openpype/hosts/houdini/startup/MainMenuCommon.xml +++ b/openpype/hosts/houdini/startup/MainMenuCommon.xml @@ -2,7 +2,19 @@ - + + + + + + diff --git a/openpype/hosts/max/api/lib.py b/openpype/hosts/max/api/lib.py index ccd4cd67e1..4a150067e1 100644 --- a/openpype/hosts/max/api/lib.py +++ b/openpype/hosts/max/api/lib.py @@ -6,7 +6,7 @@ from typing import Any, Dict, Union import six from openpype.pipeline.context_tools import ( - get_current_project, get_current_project_asset,) + get_current_project, get_current_project_asset) from pymxs import runtime as rt JSON_PREFIX = "JSON::" @@ -312,3 +312,114 @@ def set_timeline(frameStart, frameEnd): """ rt.animationRange = rt.interval(frameStart, frameEnd) return rt.animationRange + + +def unique_namespace(namespace, format="%02d", + prefix="", suffix="", con_suffix="CON"): + """Return unique namespace + + Arguments: + namespace (str): Name of namespace to consider + format (str, optional): Formatting of the given iteration number + suffix (str, optional): Only consider namespaces with this suffix. + con_suffix: max only, for finding the name of the master container + + >>> unique_namespace("bar") + # bar01 + >>> unique_namespace(":hello") + # :hello01 + >>> unique_namespace("bar:", suffix="_NS") + # bar01_NS: + + """ + + def current_namespace(): + current = namespace + # When inside a namespace Max adds no trailing : + if not current.endswith(":"): + current += ":" + return current + + # Always check against the absolute namespace root + # There's no clash with :x if we're defining namespace :a:x + ROOT = ":" if namespace.startswith(":") else current_namespace() + + # Strip trailing `:` tokens since we might want to add a suffix + start = ":" if namespace.startswith(":") else "" + end = ":" if namespace.endswith(":") else "" + namespace = namespace.strip(":") + if ":" in namespace: + # Split off any nesting that we don't uniqify anyway. + parents, namespace = namespace.rsplit(":", 1) + start += parents + ":" + ROOT += start + + iteration = 1 + increment_version = True + while increment_version: + nr_namespace = namespace + format % iteration + unique = prefix + nr_namespace + suffix + container_name = f"{unique}:{namespace}{con_suffix}" + if not rt.getNodeByName(container_name): + name_space = start + unique + end + increment_version = False + return name_space + else: + increment_version = True + iteration += 1 + + +def get_namespace(container_name): + """Get the namespace and name of the sub-container + + Args: + container_name (str): the name of master container + + Raises: + RuntimeError: when there is no master container found + + Returns: + namespace (str): namespace of the sub-container + name (str): name of the sub-container + """ + node = rt.getNodeByName(container_name) + if not node: + raise RuntimeError("Master Container Not Found..") + name = rt.getUserProp(node, "name") + namespace = rt.getUserProp(node, "namespace") + return namespace, name + + +def object_transform_set(container_children): + """A function which allows to store the transform of + previous loaded object(s) + Args: + container_children(list): A list of nodes + + Returns: + transform_set (dict): A dict with all transform data of + the previous loaded object(s) + """ + transform_set = {} + for node in container_children: + name = f"{node.name}.transform" + transform_set[name] = node.pos + name = f"{node.name}.scale" + transform_set[name] = node.scale + return transform_set + + +def get_plugins() -> list: + """Get all loaded plugins in 3dsMax + + Returns: + plugin_info_list: a list of loaded plugins + """ + manager = rt.PluginManager + count = manager.pluginDllCount + plugin_info_list = [] + for p in range(1, count + 1): + plugin_info = manager.pluginDllName(p) + plugin_info_list.append(plugin_info) + + return plugin_info_list diff --git a/openpype/hosts/max/api/lib_rendersettings.py b/openpype/hosts/max/api/lib_rendersettings.py index 1b62edabee..26e176aa8d 100644 --- a/openpype/hosts/max/api/lib_rendersettings.py +++ b/openpype/hosts/max/api/lib_rendersettings.py @@ -37,13 +37,10 @@ class RenderSettings(object): def set_render_camera(self, selection): for sel in selection: # to avoid Attribute Error from pymxs wrapper - found = False if rt.classOf(sel) in rt.Camera.classes: - found = True rt.viewport.setCamera(sel) - break - if not found: - raise RuntimeError("Camera not found") + return + raise RuntimeError("Active Camera not found") def render_output(self, container): folder = rt.maxFilePath @@ -113,7 +110,8 @@ class RenderSettings(object): # for setting up renderable camera arv = rt.MAXToAOps.ArnoldRenderView() render_camera = rt.viewport.GetCamera() - arv.setOption("Camera", str(render_camera)) + if render_camera: + arv.setOption("Camera", str(render_camera)) # TODO: add AOVs and extension img_fmt = self._project_settings["max"]["RenderSettings"]["image_format"] # noqa diff --git a/openpype/hosts/max/api/pipeline.py b/openpype/hosts/max/api/pipeline.py index 03b85a4066..bd680a3d84 100644 --- a/openpype/hosts/max/api/pipeline.py +++ b/openpype/hosts/max/api/pipeline.py @@ -15,6 +15,7 @@ from openpype.pipeline import ( ) from openpype.hosts.max.api.menu import OpenPypeMenu from openpype.hosts.max.api import lib +from openpype.hosts.max.api.plugin import MS_CUSTOM_ATTRIB from openpype.hosts.max import MAX_HOST_DIR from pymxs import runtime as rt # noqa @@ -152,21 +153,87 @@ def ls() -> list: yield lib.read(container) -def containerise(name: str, nodes: list, context, loader=None, suffix="_CON"): +def containerise(name: str, nodes: list, context, + namespace=None, loader=None, suffix="_CON"): data = { "schema": "openpype:container-2.0", "id": AVALON_CONTAINER_ID, "name": name, - "namespace": "", + "namespace": namespace or "", "loader": loader, "representation": context["representation"]["_id"], } - - container_name = f"{name}{suffix}" + container_name = f"{namespace}:{name}{suffix}" container = rt.container(name=container_name) - for node in nodes: - node.Parent = container - + import_custom_attribute_data(container, nodes) if not lib.imprint(container_name, data): print(f"imprinting of {container_name} failed.") return container + + +def load_custom_attribute_data(): + """Re-loading the Openpype/AYON custom parameter built by the creator + + Returns: + attribute: re-loading the custom OP attributes set in Maxscript + """ + return rt.Execute(MS_CUSTOM_ATTRIB) + + +def import_custom_attribute_data(container: str, selections: list): + """Importing the Openpype/AYON custom parameter built by the creator + + Args: + container (str): target container which adds custom attributes + selections (list): nodes to be added into + group in custom attributes + """ + attrs = load_custom_attribute_data() + modifier = rt.EmptyModifier() + rt.addModifier(container, modifier) + container.modifiers[0].name = "OP Data" + rt.custAttributes.add(container.modifiers[0], attrs) + node_list = [] + sel_list = [] + for i in selections: + node_ref = rt.NodeTransformMonitor(node=i) + node_list.append(node_ref) + sel_list.append(str(i)) + + # Setting the property + rt.setProperty( + container.modifiers[0].openPypeData, + "all_handles", node_list) + rt.setProperty( + container.modifiers[0].openPypeData, + "sel_list", sel_list) + + +def update_custom_attribute_data(container: str, selections: list): + """Updating the Openpype/AYON custom parameter built by the creator + + Args: + container (str): target container which adds custom attributes + selections (list): nodes to be added into + group in custom attributes + """ + if container.modifiers[0].name == "OP Data": + rt.deleteModifier(container, container.modifiers[0]) + import_custom_attribute_data(container, selections) + + +def get_previous_loaded_object(container: str): + """Get previous loaded_object through the OP data + + Args: + container (str): the container which stores the OP data + + Returns: + node_list(list): list of nodes which are previously loaded + """ + node_list = [] + sel_list = rt.getProperty(container.modifiers[0].openPypeData, "sel_list") + for obj in rt.Objects: + if str(obj) in sel_list: + node_list.append(obj) + return node_list diff --git a/openpype/hosts/max/plugins/create/create_render.py b/openpype/hosts/max/plugins/create/create_render.py index 235046684e..9cc3c8da8a 100644 --- a/openpype/hosts/max/plugins/create/create_render.py +++ b/openpype/hosts/max/plugins/create/create_render.py @@ -14,7 +14,6 @@ class CreateRender(plugin.MaxCreator): def create(self, subset_name, instance_data, pre_create_data): from pymxs import runtime as rt - sel_obj = list(rt.selection) file = rt.maxFileName filename, _ = os.path.splitext(file) instance_data["AssetName"] = filename diff --git a/openpype/hosts/max/plugins/load/load_camera_fbx.py b/openpype/hosts/max/plugins/load/load_camera_fbx.py index 62284b23d9..ce1427a980 100644 --- a/openpype/hosts/max/plugins/load/load_camera_fbx.py +++ b/openpype/hosts/max/plugins/load/load_camera_fbx.py @@ -1,7 +1,16 @@ import os from openpype.hosts.max.api import lib, maintained_selection -from openpype.hosts.max.api.pipeline import containerise +from openpype.hosts.max.api.lib import ( + unique_namespace, + get_namespace, + object_transform_set +) +from openpype.hosts.max.api.pipeline import ( + containerise, + get_previous_loaded_object, + update_custom_attribute_data +) from openpype.pipeline import get_representation_path, load @@ -16,51 +25,66 @@ class FbxLoader(load.LoaderPlugin): def load(self, context, name=None, namespace=None, data=None): from pymxs import runtime as rt - filepath = self.filepath_from_context(context) filepath = os.path.normpath(filepath) rt.FBXImporterSetParam("Animation", True) rt.FBXImporterSetParam("Camera", True) rt.FBXImporterSetParam("AxisConversionMethod", True) + rt.FBXImporterSetParam("Mode", rt.Name("create")) rt.FBXImporterSetParam("Preserveinstances", True) rt.ImportFile( filepath, rt.name("noPrompt"), using=rt.FBXIMP) - container = rt.GetNodeByName(f"{name}") - if not container: - container = rt.Container() - container.name = f"{name}" + namespace = unique_namespace( + name + "_", + suffix="_", + ) + selections = rt.GetCurrentSelection() - for selection in rt.GetCurrentSelection(): - selection.Parent = container + for selection in selections: + selection.name = f"{namespace}:{selection.name}" return containerise( - name, [container], context, loader=self.__class__.__name__) + name, selections, context, + namespace, loader=self.__class__.__name__) def update(self, container, representation): from pymxs import runtime as rt path = get_representation_path(representation) - node = rt.GetNodeByName(container["instance_node"]) - rt.Select(node.Children) - fbx_reimport_cmd = ( - f""" + node_name = container["instance_node"] + node = rt.getNodeByName(node_name) + namespace, _ = get_namespace(node_name) -FBXImporterSetParam "Animation" true -FBXImporterSetParam "Cameras" true -FBXImporterSetParam "AxisConversionMethod" true -FbxExporterSetParam "UpAxis" "Y" -FbxExporterSetParam "Preserveinstances" true + node_list = get_previous_loaded_object(node) + rt.Select(node_list) + prev_fbx_objects = rt.GetCurrentSelection() + transform_data = object_transform_set(prev_fbx_objects) + for prev_fbx_obj in prev_fbx_objects: + if rt.isValidNode(prev_fbx_obj): + rt.Delete(prev_fbx_obj) -importFile @"{path}" #noPrompt using:FBXIMP - """) - rt.Execute(fbx_reimport_cmd) - - with maintained_selection(): - rt.Select(node) + rt.FBXImporterSetParam("Animation", True) + rt.FBXImporterSetParam("Camera", True) + rt.FBXImporterSetParam("Mode", rt.Name("merge")) + rt.FBXImporterSetParam("AxisConversionMethod", True) + rt.FBXImporterSetParam("Preserveinstances", True) + rt.ImportFile( + path, rt.name("noPrompt"), using=rt.FBXIMP) + current_fbx_objects = rt.GetCurrentSelection() + fbx_objects = [] + for fbx_object in current_fbx_objects: + fbx_object.name = f"{namespace}:{fbx_object.name}" + fbx_objects.append(fbx_object) + fbx_transform = f"{fbx_object.name}.transform" + if fbx_transform in transform_data.keys(): + fbx_object.pos = transform_data[fbx_transform] or 0 + fbx_object.scale = transform_data[ + f"{fbx_object.name}.scale"] or 0 + update_custom_attribute_data(node, fbx_objects) lib.imprint(container["instance_node"], { "representation": str(representation["_id"]) }) diff --git a/openpype/hosts/max/plugins/load/load_max_scene.py b/openpype/hosts/max/plugins/load/load_max_scene.py index 76cd3bf367..0b5f0a2858 100644 --- a/openpype/hosts/max/plugins/load/load_max_scene.py +++ b/openpype/hosts/max/plugins/load/load_max_scene.py @@ -1,7 +1,15 @@ import os from openpype.hosts.max.api import lib -from openpype.hosts.max.api.pipeline import containerise +from openpype.hosts.max.api.lib import ( + unique_namespace, + get_namespace, + object_transform_set +) +from openpype.hosts.max.api.pipeline import ( + containerise, get_previous_loaded_object, + update_custom_attribute_data +) from openpype.pipeline import get_representation_path, load @@ -19,36 +27,63 @@ class MaxSceneLoader(load.LoaderPlugin): def load(self, context, name=None, namespace=None, data=None): from pymxs import runtime as rt - path = self.filepath_from_context(context) path = os.path.normpath(path) # import the max scene by using "merge file" path = path.replace('\\', '/') - rt.MergeMaxFile(path) + rt.MergeMaxFile(path, quiet=True, includeFullGroup=True) max_objects = rt.getLastMergedNodes() - max_container = rt.Container(name=f"{name}") - for max_object in max_objects: - max_object.Parent = max_container - + max_object_names = [obj.name for obj in max_objects] + # implement the OP/AYON custom attributes before load + max_container = [] + namespace = unique_namespace( + name + "_", + suffix="_", + ) + for max_obj, obj_name in zip(max_objects, max_object_names): + max_obj.name = f"{namespace}:{obj_name}" + max_container.append(rt.getNodeByName(max_obj.name)) return containerise( - name, [max_container], context, loader=self.__class__.__name__) + name, max_container, context, + namespace, loader=self.__class__.__name__) def update(self, container, representation): from pymxs import runtime as rt path = get_representation_path(representation) node_name = container["instance_node"] + node = rt.getNodeByName(node_name) + namespace, _ = get_namespace(node_name) + # delete the old container with attribute + # delete old duplicate + # use the modifier OP data to delete the data + node_list = get_previous_loaded_object(node) + rt.select(node_list) + prev_max_objects = rt.GetCurrentSelection() + transform_data = object_transform_set(prev_max_objects) - rt.MergeMaxFile(path, - rt.Name("noRedraw"), - rt.Name("deleteOldDups"), - rt.Name("useSceneMtlDups")) + for prev_max_obj in prev_max_objects: + if rt.isValidNode(prev_max_obj): # noqa + rt.Delete(prev_max_obj) + rt.MergeMaxFile(path, quiet=True) - max_objects = rt.getLastMergedNodes() - container_node = rt.GetNodeByName(node_name) - for max_object in max_objects: - max_object.Parent = container_node + current_max_objects = rt.getLastMergedNodes() + current_max_object_names = [obj.name for obj + in current_max_objects] + + max_objects = [] + for max_obj, obj_name in zip(current_max_objects, + current_max_object_names): + max_obj.name = f"{namespace}:{obj_name}" + max_objects.append(max_obj) + max_transform = f"{max_obj.name}.transform" + if max_transform in transform_data.keys(): + max_obj.pos = transform_data[max_transform] or 0 + max_obj.scale = transform_data[ + f"{max_obj.name}.scale"] or 0 + + update_custom_attribute_data(node, max_objects) lib.imprint(container["instance_node"], { "representation": str(representation["_id"]) }) diff --git a/openpype/hosts/max/plugins/load/load_model.py b/openpype/hosts/max/plugins/load/load_model.py index cff82a593c..c41608c860 100644 --- a/openpype/hosts/max/plugins/load/load_model.py +++ b/openpype/hosts/max/plugins/load/load_model.py @@ -1,15 +1,20 @@ import os from openpype.pipeline import load, get_representation_path -from openpype.hosts.max.api.pipeline import containerise +from openpype.hosts.max.api.pipeline import ( + containerise, + get_previous_loaded_object +) from openpype.hosts.max.api import lib -from openpype.hosts.max.api.lib import maintained_selection +from openpype.hosts.max.api.lib import ( + maintained_selection, unique_namespace +) class ModelAbcLoader(load.LoaderPlugin): """Loading model with the Alembic loader.""" families = ["model"] - label = "Load Model(Alembic)" + label = "Load Model with Alembic" representations = ["abc"] order = -10 icon = "code-fork" @@ -30,7 +35,7 @@ class ModelAbcLoader(load.LoaderPlugin): rt.AlembicImport.CustomAttributes = True rt.AlembicImport.UVs = True rt.AlembicImport.VertexColors = True - rt.importFile(file_path, rt.name("noPrompt")) + rt.importFile(file_path, rt.name("noPrompt"), using=rt.AlembicImport) abc_after = { c @@ -46,8 +51,22 @@ class ModelAbcLoader(load.LoaderPlugin): abc_container = abc_containers.pop() + namespace = unique_namespace( + name + "_", + suffix="_", + ) + abc_objects = [] + for abc_object in abc_container.Children: + abc_object.name = f"{namespace}:{abc_object.name}" + abc_objects.append(abc_object) + # rename the abc container with namespace + abc_container_name = f"{namespace}:{name}" + abc_container.name = abc_container_name + abc_objects.append(abc_container) + return containerise( - name, [abc_container], context, loader=self.__class__.__name__ + name, abc_objects, context, + namespace, loader=self.__class__.__name__ ) def update(self, container, representation): @@ -55,22 +74,19 @@ class ModelAbcLoader(load.LoaderPlugin): path = get_representation_path(representation) node = rt.GetNodeByName(container["instance_node"]) - rt.Select(node.Children) - - for alembic in rt.Selection: - abc = rt.GetNodeByName(alembic.name) - rt.Select(abc.Children) - for abc_con in rt.Selection: - container = rt.GetNodeByName(abc_con.name) - container.source = path - rt.Select(container.Children) - for abc_obj in rt.Selection: - alembic_obj = rt.GetNodeByName(abc_obj.name) - alembic_obj.source = path - + node_list = [n for n in get_previous_loaded_object(node) + if rt.ClassOf(n) == rt.AlembicContainer] with maintained_selection(): - rt.Select(node) + rt.Select(node_list) + for alembic in rt.Selection: + abc = rt.GetNodeByName(alembic.name) + rt.Select(abc.Children) + for abc_con in abc.Children: + abc_con.source = path + rt.Select(abc_con.Children) + for abc_obj in abc_con.Children: + abc_obj.source = path lib.imprint( container["instance_node"], {"representation": str(representation["_id"])}, diff --git a/openpype/hosts/max/plugins/load/load_model_fbx.py b/openpype/hosts/max/plugins/load/load_model_fbx.py index 12f526ab95..71fc382eed 100644 --- a/openpype/hosts/max/plugins/load/load_model_fbx.py +++ b/openpype/hosts/max/plugins/load/load_model_fbx.py @@ -1,7 +1,15 @@ import os from openpype.pipeline import load, get_representation_path -from openpype.hosts.max.api.pipeline import containerise +from openpype.hosts.max.api.pipeline import ( + containerise, get_previous_loaded_object, + update_custom_attribute_data +) from openpype.hosts.max.api import lib +from openpype.hosts.max.api.lib import ( + unique_namespace, + get_namespace, + object_transform_set +) from openpype.hosts.max.api.lib import maintained_selection @@ -16,45 +24,68 @@ class FbxModelLoader(load.LoaderPlugin): def load(self, context, name=None, namespace=None, data=None): from pymxs import runtime as rt - - filepath = os.path.normpath(self.filepath_from_context(context)) + filepath = self.filepath_from_context(context) + filepath = os.path.normpath(filepath) rt.FBXImporterSetParam("Animation", False) rt.FBXImporterSetParam("Cameras", False) + rt.FBXImporterSetParam("Mode", rt.Name("create")) rt.FBXImporterSetParam("Preserveinstances", True) - rt.importFile(filepath, rt.name("noPrompt"), using=rt.FBXIMP) + rt.importFile( + filepath, rt.name("noPrompt"), using=rt.FBXIMP) - container = rt.GetNodeByName(name) - if not container: - container = rt.Container() - container.name = name + namespace = unique_namespace( + name + "_", + suffix="_", + ) + selections = rt.GetCurrentSelection() - for selection in rt.GetCurrentSelection(): - selection.Parent = container + for selection in selections: + selection.name = f"{namespace}:{selection.name}" return containerise( - name, [container], context, loader=self.__class__.__name__ - ) + name, selections, context, + namespace, loader=self.__class__.__name__) def update(self, container, representation): from pymxs import runtime as rt + path = get_representation_path(representation) - node = rt.getNodeByName(container["instance_node"]) - rt.select(node.Children) + node_name = container["instance_node"] + node = rt.getNodeByName(node_name) + if not node: + rt.Container(name=node_name) + namespace, _ = get_namespace(node_name) + + node_list = get_previous_loaded_object(node) + rt.Select(node_list) + prev_fbx_objects = rt.GetCurrentSelection() + transform_data = object_transform_set(prev_fbx_objects) + for prev_fbx_obj in prev_fbx_objects: + if rt.isValidNode(prev_fbx_obj): + rt.Delete(prev_fbx_obj) rt.FBXImporterSetParam("Animation", False) rt.FBXImporterSetParam("Cameras", False) - rt.FBXImporterSetParam("AxisConversionMethod", True) - rt.FBXImporterSetParam("UpAxis", "Y") + rt.FBXImporterSetParam("Mode", rt.Name("create")) rt.FBXImporterSetParam("Preserveinstances", True) rt.importFile(path, rt.name("noPrompt"), using=rt.FBXIMP) + current_fbx_objects = rt.GetCurrentSelection() + fbx_objects = [] + for fbx_object in current_fbx_objects: + fbx_object.name = f"{namespace}:{fbx_object.name}" + fbx_objects.append(fbx_object) + fbx_transform = f"{fbx_object.name}.transform" + if fbx_transform in transform_data.keys(): + fbx_object.pos = transform_data[fbx_transform] or 0 + fbx_object.scale = transform_data[ + f"{fbx_object.name}.scale"] or 0 with maintained_selection(): rt.Select(node) - - lib.imprint( - container["instance_node"], - {"representation": str(representation["_id"])}, - ) + update_custom_attribute_data(node, fbx_objects) + lib.imprint(container["instance_node"], { + "representation": str(representation["_id"]) + }) def switch(self, container, representation): self.update(container, representation) diff --git a/openpype/hosts/max/plugins/load/load_model_obj.py b/openpype/hosts/max/plugins/load/load_model_obj.py index 18a19414fa..aedb288a2d 100644 --- a/openpype/hosts/max/plugins/load/load_model_obj.py +++ b/openpype/hosts/max/plugins/load/load_model_obj.py @@ -1,8 +1,18 @@ import os from openpype.hosts.max.api import lib +from openpype.hosts.max.api.lib import ( + unique_namespace, + get_namespace, + maintained_selection, + object_transform_set +) from openpype.hosts.max.api.lib import maintained_selection -from openpype.hosts.max.api.pipeline import containerise +from openpype.hosts.max.api.pipeline import ( + containerise, + get_previous_loaded_object, + update_custom_attribute_data +) from openpype.pipeline import get_representation_path, load @@ -22,36 +32,46 @@ class ObjLoader(load.LoaderPlugin): self.log.debug("Executing command to import..") rt.Execute(f'importFile @"{filepath}" #noPrompt using:ObjImp') + + namespace = unique_namespace( + name + "_", + suffix="_", + ) # create "missing" container for obj import - container = rt.Container() - container.name = name - + selections = rt.GetCurrentSelection() # get current selection - for selection in rt.GetCurrentSelection(): - selection.Parent = container - - asset = rt.GetNodeByName(name) - + for selection in selections: + selection.name = f"{namespace}:{selection.name}" return containerise( - name, [asset], context, loader=self.__class__.__name__) + name, selections, context, + namespace, loader=self.__class__.__name__) def update(self, container, representation): from pymxs import runtime as rt path = get_representation_path(representation) node_name = container["instance_node"] - node = rt.GetNodeByName(node_name) - - instance_name, _ = node_name.split("_") - container = rt.GetNodeByName(instance_name) - for child in container.Children: - rt.Delete(child) + node = rt.getNodeByName(node_name) + namespace, _ = get_namespace(node_name) + node_list = get_previous_loaded_object(node) + rt.Select(node_list) + previous_objects = rt.GetCurrentSelection() + transform_data = object_transform_set(previous_objects) + for prev_obj in previous_objects: + if rt.isValidNode(prev_obj): + rt.Delete(prev_obj) rt.Execute(f'importFile @"{path}" #noPrompt using:ObjImp') # get current selection - for selection in rt.GetCurrentSelection(): - selection.Parent = container - + selections = rt.GetCurrentSelection() + for selection in selections: + selection.name = f"{namespace}:{selection.name}" + selection_transform = f"{selection.name}.transform" + if selection_transform in transform_data.keys(): + selection.pos = transform_data[selection_transform] or 0 + selection.scale = transform_data[ + f"{selection.name}.scale"] or 0 + update_custom_attribute_data(node, selections) with maintained_selection(): rt.Select(node) diff --git a/openpype/hosts/max/plugins/load/load_model_usd.py b/openpype/hosts/max/plugins/load/load_model_usd.py index 48b50b9b18..bce4bd4a9a 100644 --- a/openpype/hosts/max/plugins/load/load_model_usd.py +++ b/openpype/hosts/max/plugins/load/load_model_usd.py @@ -1,8 +1,20 @@ import os +from pymxs import runtime as rt +from openpype.pipeline.load import LoadError from openpype.hosts.max.api import lib +from openpype.hosts.max.api.lib import ( + unique_namespace, + get_namespace, + object_transform_set, + get_plugins +) from openpype.hosts.max.api.lib import maintained_selection -from openpype.hosts.max.api.pipeline import containerise +from openpype.hosts.max.api.pipeline import ( + containerise, + get_previous_loaded_object, + update_custom_attribute_data +) from openpype.pipeline import get_representation_path, load @@ -17,36 +29,54 @@ class ModelUSDLoader(load.LoaderPlugin): color = "orange" def load(self, context, name=None, namespace=None, data=None): - from pymxs import runtime as rt - # asset_filepath + plugin_info = get_plugins() + if "usdimport.dli" not in plugin_info: + raise LoadError("No USDImporter loaded/installed in Max..") filepath = os.path.normpath(self.filepath_from_context(context)) import_options = rt.USDImporter.CreateOptions() base_filename = os.path.basename(filepath) - filename, ext = os.path.splitext(base_filename) + _, ext = os.path.splitext(base_filename) log_filepath = filepath.replace(ext, "txt") rt.LogPath = log_filepath rt.LogLevel = rt.Name("info") rt.USDImporter.importFile(filepath, importOptions=import_options) - + namespace = unique_namespace( + name + "_", + suffix="_", + ) asset = rt.GetNodeByName(name) + usd_objects = [] + + for usd_asset in asset.Children: + usd_asset.name = f"{namespace}:{usd_asset.name}" + usd_objects.append(usd_asset) + + asset_name = f"{namespace}:{name}" + asset.name = asset_name + # need to get the correct container after renamed + asset = rt.GetNodeByName(asset_name) + usd_objects.append(asset) return containerise( - name, [asset], context, loader=self.__class__.__name__) + name, usd_objects, context, + namespace, loader=self.__class__.__name__) def update(self, container, representation): - from pymxs import runtime as rt - path = get_representation_path(representation) node_name = container["instance_node"] node = rt.GetNodeByName(node_name) - for n in node.Children: - for r in n.Children: - rt.Delete(r) + namespace, name = get_namespace(node_name) + node_list = get_previous_loaded_object(node) + rt.Select(node_list) + prev_objects = [sel for sel in rt.GetCurrentSelection() + if sel != rt.Container + and sel.name != node_name] + transform_data = object_transform_set(prev_objects) + for n in prev_objects: rt.Delete(n) - instance_name, _ = node_name.split("_") import_options = rt.USDImporter.CreateOptions() base_filename = os.path.basename(path) @@ -55,12 +85,23 @@ class ModelUSDLoader(load.LoaderPlugin): rt.LogPath = log_filepath rt.LogLevel = rt.Name("info") - rt.USDImporter.importFile(path, - importOptions=import_options) + rt.USDImporter.importFile( + path, importOptions=import_options) - asset = rt.GetNodeByName(instance_name) - asset.Parent = node + asset = rt.GetNodeByName(name) + usd_objects = [] + for children in asset.Children: + children.name = f"{namespace}:{children.name}" + usd_objects.append(children) + children_transform = f"{children.name}.transform" + if children_transform in transform_data.keys(): + children.pos = transform_data[children_transform] or 0 + children.scale = transform_data[ + f"{children.name}.scale"] or 0 + asset.name = f"{namespace}:{asset.name}" + usd_objects.append(asset) + update_custom_attribute_data(node, usd_objects) with maintained_selection(): rt.Select(node) @@ -72,7 +113,5 @@ class ModelUSDLoader(load.LoaderPlugin): self.update(container, representation) def remove(self, container): - from pymxs import runtime as rt - node = rt.GetNodeByName(container["instance_node"]) rt.Delete(node) diff --git a/openpype/hosts/max/plugins/load/load_pointcache.py b/openpype/hosts/max/plugins/load/load_pointcache.py index 290503e053..3c2dfe8c25 100644 --- a/openpype/hosts/max/plugins/load/load_pointcache.py +++ b/openpype/hosts/max/plugins/load/load_pointcache.py @@ -7,7 +7,11 @@ Because of limited api, alembics can be only loaded, but not easily updated. import os from openpype.pipeline import load, get_representation_path from openpype.hosts.max.api import lib, maintained_selection -from openpype.hosts.max.api.pipeline import containerise +from openpype.hosts.max.api.lib import unique_namespace +from openpype.hosts.max.api.pipeline import ( + containerise, + get_previous_loaded_object +) class AbcLoader(load.LoaderPlugin): @@ -33,7 +37,7 @@ class AbcLoader(load.LoaderPlugin): } rt.AlembicImport.ImportToRoot = False - rt.importFile(file_path, rt.name("noPrompt")) + rt.importFile(file_path, rt.name("noPrompt"), using=rt.AlembicImport) abc_after = { c @@ -48,13 +52,27 @@ class AbcLoader(load.LoaderPlugin): self.log.error("Something failed when loading.") abc_container = abc_containers.pop() - - for abc in rt.GetCurrentSelection(): + selections = rt.GetCurrentSelection() + for abc in selections: for cam_shape in abc.Children: - cam_shape.playbackType = 2 + cam_shape.playbackType = 0 + + namespace = unique_namespace( + name + "_", + suffix="_", + ) + abc_objects = [] + for abc_object in abc_container.Children: + abc_object.name = f"{namespace}:{abc_object.name}" + abc_objects.append(abc_object) + # rename the abc container with namespace + abc_container_name = f"{namespace}:{name}" + abc_container.name = abc_container_name + abc_objects.append(abc_container) return containerise( - name, [abc_container], context, loader=self.__class__.__name__ + name, abc_objects, context, + namespace, loader=self.__class__.__name__ ) def update(self, container, representation): @@ -62,29 +80,23 @@ class AbcLoader(load.LoaderPlugin): path = get_representation_path(representation) node = rt.GetNodeByName(container["instance_node"]) - - alembic_objects = self.get_container_children(node, "AlembicObject") - for alembic_object in alembic_objects: - alembic_object.source = path - - lib.imprint( - container["instance_node"], - {"representation": str(representation["_id"])}, - ) - + abc_container = [n for n in get_previous_loaded_object(node) + if rt.ClassOf(n) == rt.AlembicContainer] with maintained_selection(): - rt.Select(node.Children) + rt.Select(abc_container) for alembic in rt.Selection: abc = rt.GetNodeByName(alembic.name) rt.Select(abc.Children) - for abc_con in rt.Selection: - container = rt.GetNodeByName(abc_con.name) - container.source = path - rt.Select(container.Children) - for abc_obj in rt.Selection: - alembic_obj = rt.GetNodeByName(abc_obj.name) - alembic_obj.source = path + for abc_con in abc.Children: + abc_con.source = path + rt.Select(abc_con.Children) + for abc_obj in abc_con.Children: + abc_obj.source = path + lib.imprint( + container["instance_node"], + {"representation": str(representation["_id"])}, + ) def switch(self, container, representation): self.update(container, representation) diff --git a/openpype/hosts/max/plugins/load/load_pointcache_ornatrix.py b/openpype/hosts/max/plugins/load/load_pointcache_ornatrix.py new file mode 100644 index 0000000000..96060a6a6f --- /dev/null +++ b/openpype/hosts/max/plugins/load/load_pointcache_ornatrix.py @@ -0,0 +1,108 @@ +import os +from openpype.pipeline import load, get_representation_path +from openpype.pipeline.load import LoadError +from openpype.hosts.max.api.pipeline import ( + containerise, + get_previous_loaded_object, + update_custom_attribute_data +) + +from openpype.hosts.max.api.lib import ( + unique_namespace, + get_namespace, + object_transform_set, + get_plugins +) +from openpype.hosts.max.api import lib +from pymxs import runtime as rt + + +class OxAbcLoader(load.LoaderPlugin): + """Ornatrix Alembic loader.""" + + families = ["camera", "animation", "pointcache"] + label = "Load Alembic with Ornatrix" + representations = ["abc"] + order = -10 + icon = "code-fork" + color = "orange" + postfix = "param" + + def load(self, context, name=None, namespace=None, data=None): + plugin_list = get_plugins() + if "ephere.plugins.autodesk.max.ornatrix.dlo" not in plugin_list: + raise LoadError("Ornatrix plugin not " + "found/installed in Max yet..") + + file_path = os.path.normpath(self.filepath_from_context(context)) + rt.AlembicImport.ImportToRoot = True + rt.AlembicImport.CustomAttributes = True + rt.importFile( + file_path, rt.name("noPrompt"), + using=rt.Ornatrix_Alembic_Importer) + + scene_object = [] + for obj in rt.rootNode.Children: + obj_type = rt.ClassOf(obj) + if str(obj_type).startswith("Ox_"): + scene_object.append(obj) + + namespace = unique_namespace( + name + "_", + suffix="_", + ) + abc_container = [] + for abc in scene_object: + abc.name = f"{namespace}:{abc.name}" + abc_container.append(abc) + + return containerise( + name, abc_container, context, + namespace, loader=self.__class__.__name__ + ) + + def update(self, container, representation): + path = get_representation_path(representation) + node_name = container["instance_node"] + namespace, name = get_namespace(node_name) + node = rt.getNodeByName(node_name) + node_list = get_previous_loaded_object(node) + rt.Select(node_list) + selections = rt.getCurrentSelection() + transform_data = object_transform_set(selections) + for prev_obj in selections: + if rt.isValidNode(prev_obj): + rt.Delete(prev_obj) + + rt.AlembicImport.ImportToRoot = False + rt.AlembicImport.CustomAttributes = True + rt.importFile( + path, rt.name("noPrompt"), + using=rt.Ornatrix_Alembic_Importer) + + scene_object = [] + for obj in rt.rootNode.Children: + obj_type = rt.ClassOf(obj) + if str(obj_type).startswith("Ox_"): + scene_object.append(obj) + ox_abc_objects = [] + for abc in scene_object: + abc.Parent = container + abc.name = f"{namespace}:{abc.name}" + ox_abc_objects.append(abc) + ox_transform = f"{abc.name}.transform" + if ox_transform in transform_data.keys(): + abc.pos = transform_data[ox_transform] or 0 + abc.scale = transform_data[f"{abc.name}.scale"] or 0 + update_custom_attribute_data(node, ox_abc_objects) + lib.imprint( + container["instance_node"], + {"representation": str(representation["_id"])}, + ) + + def switch(self, container, representation): + self.update(container, representation) + + def remove(self, container): + node = rt.GetNodeByName(container["instance_node"]) + rt.Delete(node) diff --git a/openpype/hosts/max/plugins/load/load_pointcloud.py b/openpype/hosts/max/plugins/load/load_pointcloud.py index 2a1175167a..e0317a2e22 100644 --- a/openpype/hosts/max/plugins/load/load_pointcloud.py +++ b/openpype/hosts/max/plugins/load/load_pointcloud.py @@ -1,7 +1,15 @@ import os from openpype.hosts.max.api import lib, maintained_selection -from openpype.hosts.max.api.pipeline import containerise +from openpype.hosts.max.api.lib import ( + unique_namespace, + +) +from openpype.hosts.max.api.pipeline import ( + containerise, + get_previous_loaded_object, + update_custom_attribute_data +) from openpype.pipeline import get_representation_path, load @@ -13,19 +21,24 @@ class PointCloudLoader(load.LoaderPlugin): order = -8 icon = "code-fork" color = "green" + postfix = "param" def load(self, context, name=None, namespace=None, data=None): """load point cloud by tyCache""" from pymxs import runtime as rt - filepath = os.path.normpath(self.filepath_from_context(context)) obj = rt.tyCache() obj.filename = filepath - prt_container = rt.GetNodeByName(obj.name) + namespace = unique_namespace( + name + "_", + suffix="_", + ) + obj.name = f"{namespace}:{obj.name}" return containerise( - name, [prt_container], context, loader=self.__class__.__name__) + name, [obj], context, + namespace, loader=self.__class__.__name__) def update(self, container, representation): """update the container""" @@ -33,15 +46,16 @@ class PointCloudLoader(load.LoaderPlugin): path = get_representation_path(representation) node = rt.GetNodeByName(container["instance_node"]) + node_list = get_previous_loaded_object(node) + update_custom_attribute_data( + node, node_list) with maintained_selection(): - rt.Select(node.Children) + rt.Select(node_list) for prt in rt.Selection: - prt_object = rt.GetNodeByName(prt.name) - prt_object.filename = path - - lib.imprint(container["instance_node"], { - "representation": str(representation["_id"]) - }) + prt.filename = path + lib.imprint(container["instance_node"], { + "representation": str(representation["_id"]) + }) def switch(self, container, representation): self.update(container, representation) diff --git a/openpype/hosts/max/plugins/load/load_redshift_proxy.py b/openpype/hosts/max/plugins/load/load_redshift_proxy.py index 31692f6367..daf6d3e169 100644 --- a/openpype/hosts/max/plugins/load/load_redshift_proxy.py +++ b/openpype/hosts/max/plugins/load/load_redshift_proxy.py @@ -5,8 +5,17 @@ from openpype.pipeline import ( load, get_representation_path ) -from openpype.hosts.max.api.pipeline import containerise +from openpype.pipeline.load import LoadError +from openpype.hosts.max.api.pipeline import ( + containerise, + update_custom_attribute_data, + get_previous_loaded_object +) from openpype.hosts.max.api import lib +from openpype.hosts.max.api.lib import ( + unique_namespace, + get_plugins +) class RedshiftProxyLoader(load.LoaderPlugin): @@ -21,7 +30,9 @@ class RedshiftProxyLoader(load.LoaderPlugin): def load(self, context, name=None, namespace=None, data=None): from pymxs import runtime as rt - + plugin_info = get_plugins() + if "redshift4max.dlr" not in plugin_info: + raise LoadError("Redshift not loaded/installed in Max..") filepath = self.filepath_from_context(context) rs_proxy = rt.RedshiftProxy() rs_proxy.file = filepath @@ -30,24 +41,27 @@ class RedshiftProxyLoader(load.LoaderPlugin): if collections: rs_proxy.is_sequence = True - container = rt.container() - container.name = name - rs_proxy.Parent = container - - asset = rt.getNodeByName(name) + namespace = unique_namespace( + name + "_", + suffix="_", + ) + rs_proxy.name = f"{namespace}:{rs_proxy.name}" return containerise( - name, [asset], context, loader=self.__class__.__name__) + name, [rs_proxy], context, + namespace, loader=self.__class__.__name__) def update(self, container, representation): from pymxs import runtime as rt path = get_representation_path(representation) node = rt.getNodeByName(container["instance_node"]) - for children in node.Children: - children_node = rt.getNodeByName(children.name) - for proxy in children_node.Children: - proxy.file = path + node_list = get_previous_loaded_object(node) + rt.Select(node_list) + update_custom_attribute_data( + node, rt.Selection) + for proxy in rt.Selection: + proxy.file = path lib.imprint(container["instance_node"], { "representation": str(representation["_id"]) diff --git a/openpype/hosts/max/plugins/publish/collect_render.py b/openpype/hosts/max/plugins/publish/collect_render.py index db5c84fad9..2dfa1520a9 100644 --- a/openpype/hosts/max/plugins/publish/collect_render.py +++ b/openpype/hosts/max/plugins/publish/collect_render.py @@ -30,10 +30,12 @@ class CollectRender(pyblish.api.InstancePlugin): asset = get_current_asset_name() files_by_aov = RenderProducts().get_beauty(instance.name) - folder = folder.replace("\\", "/") aovs = RenderProducts().get_aovs(instance.name) files_by_aov.update(aovs) + camera = rt.viewport.GetCamera() + instance.data["cameras"] = [camera.name] if camera else None # noqa + if "expectedFiles" not in instance.data: instance.data["expectedFiles"] = list() instance.data["files"] = list() diff --git a/openpype/hosts/max/plugins/publish/extract_camera_abc.py b/openpype/hosts/max/plugins/publish/extract_camera_abc.py index b42732e70d..b1918c53e0 100644 --- a/openpype/hosts/max/plugins/publish/extract_camera_abc.py +++ b/openpype/hosts/max/plugins/publish/extract_camera_abc.py @@ -22,8 +22,6 @@ class ExtractCameraAlembic(publish.Extractor, OptionalPyblishPluginMixin): start = float(instance.data.get("frameStartHandle", 1)) end = float(instance.data.get("frameEndHandle", 1)) - container = instance.data["instance_node"] - self.log.info("Extracting Camera ...") stagingdir = self.staging_dir(instance) diff --git a/openpype/hosts/max/plugins/publish/extract_camera_fbx.py b/openpype/hosts/max/plugins/publish/extract_camera_fbx.py index 06ac3da093..537c88eb4d 100644 --- a/openpype/hosts/max/plugins/publish/extract_camera_fbx.py +++ b/openpype/hosts/max/plugins/publish/extract_camera_fbx.py @@ -19,9 +19,8 @@ class ExtractCameraFbx(publish.Extractor, OptionalPyblishPluginMixin): def process(self, instance): if not self.is_active(instance.data): return - container = instance.data["instance_node"] - self.log.info("Extracting Camera ...") + self.log.debug("Extracting Camera ...") stagingdir = self.staging_dir(instance) filename = "{name}.fbx".format(**instance.data) diff --git a/openpype/hosts/max/plugins/publish/extract_max_scene_raw.py b/openpype/hosts/max/plugins/publish/extract_max_scene_raw.py index de5db9ab56..a7a889c587 100644 --- a/openpype/hosts/max/plugins/publish/extract_max_scene_raw.py +++ b/openpype/hosts/max/plugins/publish/extract_max_scene_raw.py @@ -18,10 +18,9 @@ class ExtractMaxSceneRaw(publish.Extractor, OptionalPyblishPluginMixin): def process(self, instance): if not self.is_active(instance.data): return - container = instance.data["instance_node"] # publish the raw scene for camera - self.log.info("Extracting Raw Max Scene ...") + self.log.debug("Extracting Raw Max Scene ...") stagingdir = self.staging_dir(instance) filename = "{name}.max".format(**instance.data) diff --git a/openpype/hosts/max/plugins/publish/extract_model.py b/openpype/hosts/max/plugins/publish/extract_model.py index c7ecf7efc9..38f4848c5e 100644 --- a/openpype/hosts/max/plugins/publish/extract_model.py +++ b/openpype/hosts/max/plugins/publish/extract_model.py @@ -20,9 +20,7 @@ class ExtractModel(publish.Extractor, OptionalPyblishPluginMixin): if not self.is_active(instance.data): return - container = instance.data["instance_node"] - - self.log.info("Extracting Geometry ...") + self.log.debug("Extracting Geometry ...") stagingdir = self.staging_dir(instance) filename = "{name}.abc".format(**instance.data) diff --git a/openpype/hosts/max/plugins/publish/extract_model_fbx.py b/openpype/hosts/max/plugins/publish/extract_model_fbx.py index 56c2cadd94..fd48ed5007 100644 --- a/openpype/hosts/max/plugins/publish/extract_model_fbx.py +++ b/openpype/hosts/max/plugins/publish/extract_model_fbx.py @@ -20,10 +20,7 @@ class ExtractModelFbx(publish.Extractor, OptionalPyblishPluginMixin): if not self.is_active(instance.data): return - container = instance.data["instance_node"] - - - self.log.info("Extracting Geometry ...") + self.log.debug("Extracting Geometry ...") stagingdir = self.staging_dir(instance) filename = "{name}.fbx".format(**instance.data) diff --git a/openpype/hosts/max/plugins/publish/extract_model_obj.py b/openpype/hosts/max/plugins/publish/extract_model_obj.py index 4fde65cf22..a5d9ad6597 100644 --- a/openpype/hosts/max/plugins/publish/extract_model_obj.py +++ b/openpype/hosts/max/plugins/publish/extract_model_obj.py @@ -3,6 +3,7 @@ import pyblish.api from openpype.pipeline import publish, OptionalPyblishPluginMixin from pymxs import runtime as rt from openpype.hosts.max.api import maintained_selection +from openpype.pipeline.publish import KnownPublishError class ExtractModelObj(publish.Extractor, OptionalPyblishPluginMixin): @@ -20,15 +21,14 @@ class ExtractModelObj(publish.Extractor, OptionalPyblishPluginMixin): if not self.is_active(instance.data): return - container = instance.data["instance_node"] - - self.log.info("Extracting Geometry ...") + self.log.debug("Extracting Geometry ...") stagingdir = self.staging_dir(instance) filename = "{name}.obj".format(**instance.data) filepath = os.path.join(stagingdir, filename) self.log.info("Writing OBJ '%s' to '%s'" % (filepath, stagingdir)) + self.log.info("Performing Extraction ...") with maintained_selection(): # select and export node_list = instance.data["members"] @@ -40,7 +40,10 @@ class ExtractModelObj(publish.Extractor, OptionalPyblishPluginMixin): using=rt.ObjExp, ) - self.log.info("Performing Extraction ...") + if not os.path.exists(filepath): + raise KnownPublishError( + "File {} wasn't produced by 3ds max, please check the logs.") + if "representations" not in instance.data: instance.data["representations"] = [] diff --git a/openpype/hosts/max/plugins/publish/extract_pointcache.py b/openpype/hosts/max/plugins/publish/extract_pointcache.py index 5a99a8b845..c3de623bc0 100644 --- a/openpype/hosts/max/plugins/publish/extract_pointcache.py +++ b/openpype/hosts/max/plugins/publish/extract_pointcache.py @@ -54,8 +54,6 @@ class ExtractAlembic(publish.Extractor): start = float(instance.data.get("frameStartHandle", 1)) end = float(instance.data.get("frameEndHandle", 1)) - container = instance.data["instance_node"] - self.log.debug("Extracting pointcache ...") parent_dir = self.staging_dir(instance) diff --git a/openpype/hosts/max/plugins/publish/extract_redshift_proxy.py b/openpype/hosts/max/plugins/publish/extract_redshift_proxy.py index ab569ecbcb..f67ed30c6b 100644 --- a/openpype/hosts/max/plugins/publish/extract_redshift_proxy.py +++ b/openpype/hosts/max/plugins/publish/extract_redshift_proxy.py @@ -16,11 +16,10 @@ class ExtractRedshiftProxy(publish.Extractor): families = ["redshiftproxy"] def process(self, instance): - container = instance.data["instance_node"] start = int(instance.context.data.get("frameStart")) end = int(instance.context.data.get("frameEnd")) - self.log.info("Extracting Redshift Proxy...") + self.log.debug("Extracting Redshift Proxy...") stagingdir = self.staging_dir(instance) rs_filename = "{name}.rs".format(**instance.data) rs_filepath = os.path.join(stagingdir, rs_filename) diff --git a/openpype/hosts/max/plugins/publish/validate_no_max_content.py b/openpype/hosts/max/plugins/publish/validate_no_max_content.py index c6a27dace3..73e12e75c9 100644 --- a/openpype/hosts/max/plugins/publish/validate_no_max_content.py +++ b/openpype/hosts/max/plugins/publish/validate_no_max_content.py @@ -13,7 +13,6 @@ class ValidateMaxContents(pyblish.api.InstancePlugin): order = pyblish.api.ValidatorOrder families = ["camera", "maxScene", - "maxrender", "review"] hosts = ["max"] label = "Max Scene Contents" diff --git a/openpype/hosts/max/plugins/publish/validate_renderable_camera.py b/openpype/hosts/max/plugins/publish/validate_renderable_camera.py new file mode 100644 index 0000000000..61321661b5 --- /dev/null +++ b/openpype/hosts/max/plugins/publish/validate_renderable_camera.py @@ -0,0 +1,46 @@ +# -*- coding: utf-8 -*- +import pyblish.api +from openpype.pipeline import ( + PublishValidationError, + OptionalPyblishPluginMixin) +from openpype.pipeline.publish import RepairAction +from openpype.hosts.max.api.lib import get_current_renderer + +from pymxs import runtime as rt + + +class ValidateRenderableCamera(pyblish.api.InstancePlugin, + OptionalPyblishPluginMixin): + """Validates Renderable Camera + + Check if the renderable camera used for rendering + """ + + order = pyblish.api.ValidatorOrder + families = ["maxrender"] + hosts = ["max"] + label = "Renderable Camera" + optional = True + actions = [RepairAction] + + def process(self, instance): + if not self.is_active(instance.data): + return + if not instance.data["cameras"]: + raise PublishValidationError( + "No renderable Camera found in scene." + ) + + @classmethod + def repair(cls, instance): + + rt.viewport.setType(rt.Name("view_camera")) + camera = rt.viewport.GetCamera() + cls.log.info(f"Camera {camera} set as renderable camera") + renderer_class = get_current_renderer() + renderer = str(renderer_class).split(":")[0] + if renderer == "Arnold": + arv = rt.MAXToAOps.ArnoldRenderView() + arv.setOption("Camera", str(camera)) + arv.close() + instance.data["cameras"] = [camera.name] diff --git a/openpype/hosts/max/plugins/publish/validate_resolution_setting.py b/openpype/hosts/max/plugins/publish/validate_resolution_setting.py index 5fcb843b20..5ac41b10a0 100644 --- a/openpype/hosts/max/plugins/publish/validate_resolution_setting.py +++ b/openpype/hosts/max/plugins/publish/validate_resolution_setting.py @@ -6,11 +6,6 @@ from openpype.pipeline import ( from pymxs import runtime as rt from openpype.hosts.max.api.lib import reset_scene_resolution -from openpype.pipeline.context_tools import ( - get_current_project_asset, - get_current_project -) - class ValidateResolutionSetting(pyblish.api.InstancePlugin, OptionalPyblishPluginMixin): @@ -43,22 +38,16 @@ class ValidateResolutionSetting(pyblish.api.InstancePlugin, "on asset or shot.") def get_db_resolution(self, instance): - data = ["data.resolutionWidth", "data.resolutionHeight"] - project_resolution = get_current_project(fields=data) - project_resolution_data = project_resolution["data"] - asset_resolution = get_current_project_asset(fields=data) - asset_resolution_data = asset_resolution["data"] - # Set project resolution - project_width = int( - project_resolution_data.get("resolutionWidth", 1920)) - project_height = int( - project_resolution_data.get("resolutionHeight", 1080)) - width = int( - asset_resolution_data.get("resolutionWidth", project_width)) - height = int( - asset_resolution_data.get("resolutionHeight", project_height)) + asset_doc = instance.data["assetEntity"] + project_doc = instance.context.data["projectEntity"] + for data in [asset_doc["data"], project_doc["data"]]: + if "resolutionWidth" in data and "resolutionHeight" in data: + width = data["resolutionWidth"] + height = data["resolutionHeight"] + return int(width), int(height) - return width, height + # Defaults if not found in asset document or project document + return 1920, 1080 @classmethod def repair(cls, instance): diff --git a/openpype/hosts/max/plugins/publish/validate_usd_plugin.py b/openpype/hosts/max/plugins/publish/validate_usd_plugin.py index 9957e62736..36c4291925 100644 --- a/openpype/hosts/max/plugins/publish/validate_usd_plugin.py +++ b/openpype/hosts/max/plugins/publish/validate_usd_plugin.py @@ -1,9 +1,13 @@ # -*- coding: utf-8 -*- """Validator for USD plugin.""" -from openpype.pipeline import PublishValidationError from pyblish.api import InstancePlugin, ValidatorOrder from pymxs import runtime as rt +from openpype.pipeline import ( + OptionalPyblishPluginMixin, + PublishValidationError +) + def get_plugins() -> list: """Get plugin list from 3ds max.""" @@ -17,17 +21,25 @@ def get_plugins() -> list: return plugin_info_list -class ValidateUSDPlugin(InstancePlugin): +class ValidateUSDPlugin(OptionalPyblishPluginMixin, + InstancePlugin): """Validates if USD plugin is installed or loaded in 3ds max.""" order = ValidatorOrder - 0.01 families = ["model"] hosts = ["max"] - label = "USD Plugin" + label = "Validate USD Plugin loaded" + optional = True def process(self, instance): """Plugin entry point.""" + for sc in ValidateUSDPlugin.__subclasses__(): + self.log.info(sc) + + if not self.is_active(instance.data): + return + plugin_info = get_plugins() usd_import = "usdimport.dli" if usd_import not in plugin_info: diff --git a/openpype/hosts/maya/api/lib_rendersettings.py b/openpype/hosts/maya/api/lib_rendersettings.py index f54633c04d..42cf29d0a7 100644 --- a/openpype/hosts/maya/api/lib_rendersettings.py +++ b/openpype/hosts/maya/api/lib_rendersettings.py @@ -177,12 +177,7 @@ class RenderSettings(object): # list all the aovs all_rs_aovs = cmds.ls(type='RedshiftAOV') for rs_aov in redshift_aovs: - rs_layername = rs_aov - if " " in rs_aov: - rs_renderlayer = rs_aov.replace(" ", "") - rs_layername = "rsAov_{}".format(rs_renderlayer) - else: - rs_layername = "rsAov_{}".format(rs_aov) + rs_layername = "rsAov_{}".format(rs_aov.replace(" ", "")) if rs_layername in all_rs_aovs: continue cmds.rsCreateAov(type=rs_aov) @@ -317,7 +312,7 @@ class RenderSettings(object): separators = [cmds.menuItem(i, query=True, label=True) for i in items] # noqa: E501 try: sep_idx = separators.index(aov_separator) - except ValueError as e: + except ValueError: six.reraise( CreatorError, CreatorError( diff --git a/openpype/hosts/maya/api/pipeline.py b/openpype/hosts/maya/api/pipeline.py index 60495ac652..3647ec0b6b 100644 --- a/openpype/hosts/maya/api/pipeline.py +++ b/openpype/hosts/maya/api/pipeline.py @@ -659,17 +659,6 @@ def on_task_changed(): lib.set_context_settings() lib.update_content_on_context_change() - msg = " project: {}\n asset: {}\n task:{}".format( - get_current_project_name(), - get_current_asset_name(), - get_current_task_name() - ) - - lib.show_message( - "Context was changed", - ("Context was changed to:\n{}".format(msg)), - ) - def before_workfile_open(): if handle_workfile_locks(): diff --git a/openpype/hosts/maya/api/plugin.py b/openpype/hosts/maya/api/plugin.py index 00d6602ef9..79fcf9bc8b 100644 --- a/openpype/hosts/maya/api/plugin.py +++ b/openpype/hosts/maya/api/plugin.py @@ -129,18 +129,49 @@ class MayaCreatorBase(object): shared_data["maya_cached_legacy_subsets"] = cache_legacy return shared_data + def get_publish_families(self): + """Return families for the instances of this creator. + + Allow a Creator to define multiple families so that a creator can + e.g. specify `usd` and `usdMaya` and another USD creator can also + specify `usd` but apply different extractors like `usdMultiverse`. + + There is no need to override this method if you only have the + primary family defined by the `family` property as that will always + be set. + + Returns: + list: families for instances of this creator + + """ + return [] + def imprint_instance_node(self, node, data): # We never store the instance_node as value on the node since # it's the node name itself data.pop("instance_node", None) + # Don't store `families` since it's up to the creator itself + # to define the initial publish families - not a stored attribute of + # `families` + data.pop("families", None) + # We store creator attributes at the root level and assume they # will not clash in names with `subset`, `task`, etc. and other # default names. This is just so these attributes in many cases # are still editable in the maya UI by artists. - # pop to move to end of dict to sort attributes last on the node + # note: pop to move to end of dict to sort attributes last on the node creator_attributes = data.pop("creator_attributes", {}) + + # We only flatten value types which `imprint` function supports + json_creator_attributes = {} + for key, value in dict(creator_attributes).items(): + if isinstance(value, (list, tuple, dict)): + creator_attributes.pop(key) + json_creator_attributes[key] = value + + # Flatten remaining creator attributes to the node itself data.update(creator_attributes) # We know the "publish_attributes" will be complex data of @@ -150,6 +181,10 @@ class MayaCreatorBase(object): data.pop("publish_attributes", {}) ) + # Persist the non-flattened creator attributes (special value types, + # like multiselection EnumDef) + data["creator_attributes"] = json.dumps(json_creator_attributes) + # Since we flattened the data structure for creator attributes we want # to correctly detect which flattened attributes should end back in the # creator attributes when reading the data from the node, so we store @@ -170,15 +205,22 @@ class MayaCreatorBase(object): # being read as 'data' node_data.pop("cbId", None) + # Make sure we convert any creator attributes from the json string + creator_attributes = node_data.get("creator_attributes") + if creator_attributes: + node_data["creator_attributes"] = json.loads(creator_attributes) + else: + node_data["creator_attributes"] = {} + # Move the relevant attributes into "creator_attributes" that # we flattened originally - node_data["creator_attributes"] = {} creator_attribute_keys = node_data.pop("__creator_attributes_keys", "").split(",") for key in creator_attribute_keys: if key in node_data: node_data["creator_attributes"][key] = node_data.pop(key) + # Make sure we convert any publish attributes from the json string publish_attributes = node_data.get("publish_attributes") if publish_attributes: node_data["publish_attributes"] = json.loads(publish_attributes) @@ -186,6 +228,11 @@ class MayaCreatorBase(object): # Explicitly re-parse the node name node_data["instance_node"] = node + # If the creator plug-in specifies + families = self.get_publish_families() + if families: + node_data["families"] = families + return node_data def _default_collect_instances(self): @@ -230,6 +277,14 @@ class MayaCreator(NewCreator, MayaCreatorBase): if pre_create_data.get("use_selection"): members = cmds.ls(selection=True) + # Allow a Creator to define multiple families + publish_families = self.get_publish_families() + if publish_families: + families = instance_data.setdefault("families", []) + for family in self.get_publish_families(): + if family not in families: + families.append(family) + with lib.undo_chunk(): instance_node = cmds.sets(members, name=subset_name) instance_data["instance_node"] = instance_node @@ -260,7 +315,7 @@ class MayaCreator(NewCreator, MayaCreatorBase): default=True) ] - def apply_settings(self, project_settings, system_settings): + def apply_settings(self, project_settings): """Method called on initialization of plugin to apply settings.""" settings_name = self.settings_name @@ -683,7 +738,6 @@ class ReferenceLoader(Loader): loaded_containers.append(container) self._organize_containers(nodes, container) c += 1 - namespace = None return loaded_containers diff --git a/openpype/hosts/maya/plugins/create/create_animation.py b/openpype/hosts/maya/plugins/create/create_animation.py index 214ac18aef..115c73c0d3 100644 --- a/openpype/hosts/maya/plugins/create/create_animation.py +++ b/openpype/hosts/maya/plugins/create/create_animation.py @@ -81,10 +81,8 @@ class CreateAnimation(plugin.MayaHiddenCreator): return defs - def apply_settings(self, project_settings, system_settings): - super(CreateAnimation, self).apply_settings( - project_settings, system_settings - ) + def apply_settings(self, project_settings): + super(CreateAnimation, self).apply_settings(project_settings) # Hardcoding creator to be enabled due to existing settings would # disable the creator causing the creator plugin to not be # discoverable. diff --git a/openpype/hosts/maya/plugins/create/create_look.py b/openpype/hosts/maya/plugins/create/create_look.py index 385ae81e01..11a69151fd 100644 --- a/openpype/hosts/maya/plugins/create/create_look.py +++ b/openpype/hosts/maya/plugins/create/create_look.py @@ -37,13 +37,7 @@ class CreateLook(plugin.MayaCreator): label="Convert textures to .rstex", tooltip="Whether to generate Redshift .rstex files for " "your textures", - default=self.rs_tex), - BoolDef("forceCopy", - label="Force Copy", - tooltip="Enable users to force a copy instead of hardlink." - "\nNote: On Windows copy is always forced due to " - "bugs in windows' implementation of hardlinks.", - default=False) + default=self.rs_tex) ] def get_pre_create_attr_defs(self): diff --git a/openpype/hosts/maya/plugins/create/create_maya_usd.py b/openpype/hosts/maya/plugins/create/create_maya_usd.py new file mode 100644 index 0000000000..cc9a14bd3a --- /dev/null +++ b/openpype/hosts/maya/plugins/create/create_maya_usd.py @@ -0,0 +1,102 @@ +from openpype.hosts.maya.api import plugin, lib +from openpype.lib import ( + BoolDef, + EnumDef, + TextDef +) + +from maya import cmds + + +class CreateMayaUsd(plugin.MayaCreator): + """Create Maya USD Export""" + + identifier = "io.openpype.creators.maya.mayausd" + label = "Maya USD" + family = "usd" + icon = "cubes" + description = "Create Maya USD Export" + + cache = {} + + def get_publish_families(self): + return ["usd", "mayaUsd"] + + def get_instance_attr_defs(self): + + if "jobContextItems" not in self.cache: + # Query once instead of per instance + job_context_items = {} + try: + cmds.loadPlugin("mayaUsdPlugin", quiet=True) + job_context_items = { + cmds.mayaUSDListJobContexts(jobContext=name): name + for name in cmds.mayaUSDListJobContexts(export=True) or [] + } + except RuntimeError: + # Likely `mayaUsdPlugin` plug-in not available + self.log.warning("Unable to retrieve available job " + "contexts for `mayaUsdPlugin` exports") + + if not job_context_items: + # enumdef multiselection may not be empty + job_context_items = [""] + + self.cache["jobContextItems"] = job_context_items + + defs = lib.collect_animation_defs() + defs.extend([ + EnumDef("defaultUSDFormat", + label="File format", + items={ + "usdc": "Binary", + "usda": "ASCII" + }, + default="usdc"), + BoolDef("stripNamespaces", + label="Strip Namespaces", + tooltip=( + "Remove namespaces during export. By default, " + "namespaces are exported to the USD file in the " + "following format: nameSpaceExample_pPlatonic1" + ), + default=True), + BoolDef("mergeTransformAndShape", + label="Merge Transform and Shape", + tooltip=( + "Combine Maya transform and shape into a single USD" + "prim that has transform and geometry, for all" + " \"geometric primitives\" (gprims).\n" + "This results in smaller and faster scenes. Gprims " + "will be \"unpacked\" back into transform and shape " + "nodes when imported into Maya from USD." + ), + default=True), + BoolDef("includeUserDefinedAttributes", + label="Include User Defined Attributes", + tooltip=( + "Whether to include all custom maya attributes found " + "on nodes as metadata (userProperties) in USD." + ), + default=False), + TextDef("attr", + label="Custom Attributes", + default="", + placeholder="attr1, attr2"), + TextDef("attrPrefix", + label="Custom Attributes Prefix", + default="", + placeholder="prefix1, prefix2"), + EnumDef("jobContext", + label="Job Context", + items=self.cache["jobContextItems"], + tooltip=( + "Specifies an additional export context to handle.\n" + "These usually contain extra schemas, primitives,\n" + "and materials that are to be exported for a " + "specific\ntask, a target renderer for example." + ), + multiselection=True), + ]) + + return defs diff --git a/openpype/hosts/maya/plugins/create/create_multiverse_usd.py b/openpype/hosts/maya/plugins/create/create_multiverse_usd.py index 0b0ad3bccb..2963d4d5b6 100644 --- a/openpype/hosts/maya/plugins/create/create_multiverse_usd.py +++ b/openpype/hosts/maya/plugins/create/create_multiverse_usd.py @@ -14,6 +14,10 @@ class CreateMultiverseUsd(plugin.MayaCreator): label = "Multiverse USD Asset" family = "usd" icon = "cubes" + description = "Create Multiverse USD Asset" + + def get_publish_families(self): + return ["usd", "mvUsd"] def get_instance_attr_defs(self): diff --git a/openpype/hosts/maya/plugins/create/create_multiverse_usd_over.py b/openpype/hosts/maya/plugins/create/create_multiverse_usd_over.py index e1534dd68c..166dbf6515 100644 --- a/openpype/hosts/maya/plugins/create/create_multiverse_usd_over.py +++ b/openpype/hosts/maya/plugins/create/create_multiverse_usd_over.py @@ -6,7 +6,7 @@ from openpype.lib import ( ) -class CreateMultiverseUsdOver(plugin.Creator): +class CreateMultiverseUsdOver(plugin.MayaCreator): """Create Multiverse USD Override""" identifier = "io.openpype.creators.maya.mvusdoverride" diff --git a/openpype/hosts/maya/plugins/create/create_render.py b/openpype/hosts/maya/plugins/create/create_render.py index cc5c1eb205..6266689af4 100644 --- a/openpype/hosts/maya/plugins/create/create_render.py +++ b/openpype/hosts/maya/plugins/create/create_render.py @@ -34,7 +34,7 @@ class CreateRenderlayer(plugin.RenderlayerCreator): render_settings = {} @classmethod - def apply_settings(cls, project_settings, system_settings): + def apply_settings(cls, project_settings): cls.render_settings = project_settings["maya"]["RenderSettings"] def create(self, subset_name, instance_data, pre_create_data): diff --git a/openpype/hosts/maya/plugins/create/create_unreal_skeletalmesh.py b/openpype/hosts/maya/plugins/create/create_unreal_skeletalmesh.py index 4e2a99eced..3c9a79156a 100644 --- a/openpype/hosts/maya/plugins/create/create_unreal_skeletalmesh.py +++ b/openpype/hosts/maya/plugins/create/create_unreal_skeletalmesh.py @@ -21,7 +21,7 @@ class CreateUnrealSkeletalMesh(plugin.MayaCreator): # Defined in settings joint_hints = set() - def apply_settings(self, project_settings, system_settings): + def apply_settings(self, project_settings): """Apply project settings to creator""" settings = ( project_settings["maya"]["create"]["CreateUnrealSkeletalMesh"] diff --git a/openpype/hosts/maya/plugins/create/create_unreal_staticmesh.py b/openpype/hosts/maya/plugins/create/create_unreal_staticmesh.py index 3f96d91a54..025b39fa55 100644 --- a/openpype/hosts/maya/plugins/create/create_unreal_staticmesh.py +++ b/openpype/hosts/maya/plugins/create/create_unreal_staticmesh.py @@ -16,7 +16,7 @@ class CreateUnrealStaticMesh(plugin.MayaCreator): # Defined in settings collision_prefixes = [] - def apply_settings(self, project_settings, system_settings): + def apply_settings(self, project_settings): """Apply project settings to creator""" settings = project_settings["maya"]["create"]["CreateUnrealStaticMesh"] self.collision_prefixes = settings["collision_prefixes"] diff --git a/openpype/hosts/maya/plugins/create/create_vrayscene.py b/openpype/hosts/maya/plugins/create/create_vrayscene.py index d601dceb54..2726979d30 100644 --- a/openpype/hosts/maya/plugins/create/create_vrayscene.py +++ b/openpype/hosts/maya/plugins/create/create_vrayscene.py @@ -22,7 +22,7 @@ class CreateVRayScene(plugin.RenderlayerCreator): singleton_node_name = "vraysceneMain" @classmethod - def apply_settings(cls, project_settings, system_settings): + def apply_settings(cls, project_settings): cls.render_settings = project_settings["maya"]["RenderSettings"] def create(self, subset_name, instance_data, pre_create_data): diff --git a/openpype/hosts/maya/plugins/create/create_yeti_cache.py b/openpype/hosts/maya/plugins/create/create_yeti_cache.py index 395aa62325..ca002392d4 100644 --- a/openpype/hosts/maya/plugins/create/create_yeti_cache.py +++ b/openpype/hosts/maya/plugins/create/create_yeti_cache.py @@ -13,8 +13,7 @@ class CreateYetiCache(plugin.MayaCreator): family = "yeticache" icon = "pagelines" - def __init__(self, *args, **kwargs): - super(CreateYetiCache, self).__init__(*args, **kwargs) + def get_instance_attr_defs(self): defs = [ NumberDef("preroll", @@ -36,3 +35,5 @@ class CreateYetiCache(plugin.MayaCreator): default=3, decimals=0) ) + + return defs diff --git a/openpype/hosts/maya/plugins/inventory/import_reference.py b/openpype/hosts/maya/plugins/inventory/import_reference.py index ecc424209d..3f3b85ba6c 100644 --- a/openpype/hosts/maya/plugins/inventory/import_reference.py +++ b/openpype/hosts/maya/plugins/inventory/import_reference.py @@ -12,7 +12,6 @@ class ImportReference(InventoryAction): color = "#d8d8d8" def process(self, containers): - references = cmds.ls(type="reference") for container in containers: if container["loader"] != "ReferenceLoader": print("Not a reference, skipping") diff --git a/openpype/hosts/maya/plugins/load/load_arnold_standin.py b/openpype/hosts/maya/plugins/load/load_arnold_standin.py index b5cc4d629b..2e1329f201 100644 --- a/openpype/hosts/maya/plugins/load/load_arnold_standin.py +++ b/openpype/hosts/maya/plugins/load/load_arnold_standin.py @@ -17,6 +17,7 @@ from openpype.hosts.maya.api.lib import ( ) from openpype.hosts.maya.api.pipeline import containerise + def is_sequence(files): sequence = False collections, remainder = clique.assemble(files, minimum_items=1) @@ -29,11 +30,12 @@ def get_current_session_fps(): session_fps = float(legacy_io.Session.get('AVALON_FPS', 25)) return convert_to_maya_fps(session_fps) + class ArnoldStandinLoader(load.LoaderPlugin): """Load as Arnold standin""" - families = ["ass", "animation", "model", "proxyAbc", "pointcache"] - representations = ["ass", "abc"] + families = ["ass", "animation", "model", "proxyAbc", "pointcache", "usd"] + representations = ["ass", "abc", "usda", "usdc", "usd"] label = "Load as Arnold standin" order = -5 diff --git a/openpype/hosts/maya/plugins/load/load_maya_usd.py b/openpype/hosts/maya/plugins/load/load_maya_usd.py new file mode 100644 index 0000000000..2fb1a625a5 --- /dev/null +++ b/openpype/hosts/maya/plugins/load/load_maya_usd.py @@ -0,0 +1,108 @@ +# -*- coding: utf-8 -*- +import maya.cmds as cmds + +from openpype.pipeline import ( + load, + get_representation_path, +) +from openpype.pipeline.load import get_representation_path_from_context +from openpype.hosts.maya.api.lib import ( + namespaced, + unique_namespace +) +from openpype.hosts.maya.api.pipeline import containerise + + +class MayaUsdLoader(load.LoaderPlugin): + """Read USD data in a Maya USD Proxy""" + + families = ["model", "usd", "pointcache", "animation"] + representations = ["usd", "usda", "usdc", "usdz", "abc"] + + label = "Load USD to Maya Proxy" + order = -1 + icon = "code-fork" + color = "orange" + + def load(self, context, name=None, namespace=None, options=None): + asset = context['asset']['name'] + namespace = namespace or unique_namespace( + asset + "_", + prefix="_" if asset[0].isdigit() else "", + suffix="_", + ) + + # Make sure we can load the plugin + cmds.loadPlugin("mayaUsdPlugin", quiet=True) + + path = get_representation_path_from_context(context) + + # Create the shape + cmds.namespace(addNamespace=namespace) + with namespaced(namespace, new=False): + transform = cmds.createNode("transform", + name=name, + skipSelect=True) + proxy = cmds.createNode('mayaUsdProxyShape', + name="{}Shape".format(name), + parent=transform, + skipSelect=True) + + cmds.connectAttr("time1.outTime", "{}.time".format(proxy)) + cmds.setAttr("{}.filePath".format(proxy), path, type="string") + + # By default, we force the proxy to not use a shared stage because + # when doing so Maya will quite easily allow to save into the + # loaded usd file. Since we are loading published files we want to + # avoid altering them. Unshared stages also save their edits into + # the workfile as an artist might expect it to do. + cmds.setAttr("{}.shareStage".format(proxy), False) + # cmds.setAttr("{}.shareStage".format(proxy), lock=True) + + nodes = [transform, proxy] + self[:] = nodes + + return containerise( + name=name, + namespace=namespace, + nodes=nodes, + context=context, + loader=self.__class__.__name__) + + def update(self, container, representation): + # type: (dict, dict) -> None + """Update container with specified representation.""" + node = container['objectName'] + assert cmds.objExists(node), "Missing container" + + members = cmds.sets(node, query=True) or [] + shapes = cmds.ls(members, type="mayaUsdProxyShape") + + path = get_representation_path(representation) + for shape in shapes: + cmds.setAttr("{}.filePath".format(shape), path, type="string") + + cmds.setAttr("{}.representation".format(node), + str(representation["_id"]), + type="string") + + def switch(self, container, representation): + self.update(container, representation) + + def remove(self, container): + # type: (dict) -> None + """Remove loaded container.""" + # Delete container and its contents + if cmds.objExists(container['objectName']): + members = cmds.sets(container['objectName'], query=True) or [] + cmds.delete([container['objectName']] + members) + + # Remove the namespace, if empty + namespace = container['namespace'] + if cmds.namespace(exists=namespace): + members = cmds.namespaceInfo(namespace, listNamespace=True) + if not members: + cmds.namespace(removeNamespace=namespace) + else: + self.log.warning("Namespace not deleted because it " + "still has members: %s", namespace) diff --git a/openpype/hosts/maya/plugins/load/load_multiverse_usd.py b/openpype/hosts/maya/plugins/load/load_multiverse_usd.py index d08fcd904e..cad42b55f9 100644 --- a/openpype/hosts/maya/plugins/load/load_multiverse_usd.py +++ b/openpype/hosts/maya/plugins/load/load_multiverse_usd.py @@ -43,8 +43,6 @@ class MultiverseUsdLoader(load.LoaderPlugin): import multiverse # Create the shape - shape = None - transform = None with maintained_selection(): cmds.namespace(addNamespace=namespace) with namespaced(namespace, new=False): diff --git a/openpype/hosts/maya/plugins/load/load_reference.py b/openpype/hosts/maya/plugins/load/load_reference.py index 91767249e0..61f337f501 100644 --- a/openpype/hosts/maya/plugins/load/load_reference.py +++ b/openpype/hosts/maya/plugins/load/load_reference.py @@ -205,7 +205,7 @@ class ReferenceLoader(openpype.hosts.maya.api.plugin.ReferenceLoader): cmds.setAttr("{}.selectHandleZ".format(group_name), cz) if family == "rig": - self._post_process_rig(name, namespace, context, options) + self._post_process_rig(namespace, context, options) else: if "translate" in options: if not attach_to_root and new_nodes: @@ -229,7 +229,7 @@ class ReferenceLoader(openpype.hosts.maya.api.plugin.ReferenceLoader): members = get_container_members(container) self._lock_camera_transforms(members) - def _post_process_rig(self, name, namespace, context, options): + def _post_process_rig(self, namespace, context, options): nodes = self[:] create_rig_animation_instance( diff --git a/openpype/hosts/maya/plugins/load/load_xgen.py b/openpype/hosts/maya/plugins/load/load_xgen.py index 323f8d7eda..2ad6ad55bc 100644 --- a/openpype/hosts/maya/plugins/load/load_xgen.py +++ b/openpype/hosts/maya/plugins/load/load_xgen.py @@ -53,8 +53,6 @@ class XgenLoader(openpype.hosts.maya.api.plugin.ReferenceLoader): ) # Reference xgen. Xgen does not like being referenced in under a group. - new_nodes = [] - with maintained_selection(): nodes = cmds.file( maya_filepath, diff --git a/openpype/hosts/maya/plugins/load/load_yeti_cache.py b/openpype/hosts/maya/plugins/load/load_yeti_cache.py index 5cded13d4e..4a11ea9a2c 100644 --- a/openpype/hosts/maya/plugins/load/load_yeti_cache.py +++ b/openpype/hosts/maya/plugins/load/load_yeti_cache.py @@ -15,6 +15,16 @@ from openpype.hosts.maya.api import lib from openpype.hosts.maya.api.pipeline import containerise +# Do not reset these values on update but only apply on first load +# to preserve any potential local overrides +SKIP_UPDATE_ATTRS = { + "displayOutput", + "viewportDensity", + "viewportWidth", + "viewportLength", +} + + def set_attribute(node, attr, value): """Wrapper of set attribute which ignores None values""" if value is None: @@ -205,6 +215,8 @@ class YetiCacheLoader(load.LoaderPlugin): yeti_node = yeti_nodes[0] for attr, value in node_settings["attrs"].items(): + if attr in SKIP_UPDATE_ATTRS: + continue set_attribute(attr, value, yeti_node) cmds.setAttr("{}.representation".format(container_node), @@ -311,7 +323,6 @@ class YetiCacheLoader(load.LoaderPlugin): # Update attributes with defaults attributes = node_settings["attrs"] attributes.update({ - "viewportDensity": 0.1, "verbosity": 2, "fileMode": 1, @@ -321,6 +332,9 @@ class YetiCacheLoader(load.LoaderPlugin): "visibleInRefractions": True }) + if "viewportDensity" not in attributes: + attributes["viewportDensity"] = 0.1 + # Apply attributes to pgYetiMaya node for attr, value in attributes.items(): set_attribute(attr, value, yeti_node) diff --git a/openpype/hosts/maya/plugins/publish/collect_animation.py b/openpype/hosts/maya/plugins/publish/collect_animation.py index 8f523f770b..26a0a01c8b 100644 --- a/openpype/hosts/maya/plugins/publish/collect_animation.py +++ b/openpype/hosts/maya/plugins/publish/collect_animation.py @@ -58,17 +58,3 @@ class CollectAnimationOutputGeometry(pyblish.api.InstancePlugin): if instance.data.get("farm"): instance.data["families"].append("publish.farm") - # Collect user defined attributes. - if not instance.data.get("includeUserDefinedAttributes", False): - return - - user_defined_attributes = set() - for node in hierarchy: - attrs = cmds.listAttr(node, userDefined=True) or list() - shapes = cmds.listRelatives(node, shapes=True) or list() - for shape in shapes: - attrs.extend(cmds.listAttr(shape, userDefined=True) or list()) - - user_defined_attributes.update(attrs) - - instance.data["userDefinedAttributes"] = list(user_defined_attributes) diff --git a/openpype/hosts/maya/plugins/publish/collect_assembly.py b/openpype/hosts/maya/plugins/publish/collect_assembly.py index 2aef9ab908..f64d6bee44 100644 --- a/openpype/hosts/maya/plugins/publish/collect_assembly.py +++ b/openpype/hosts/maya/plugins/publish/collect_assembly.py @@ -35,14 +35,11 @@ class CollectAssembly(pyblish.api.InstancePlugin): # Get all content from the instance instance_lookup = set(cmds.ls(instance, type="transform", long=True)) data = defaultdict(list) - self.log.info(instance_lookup) hierarchy_nodes = [] for container in containers: - self.log.info(container) root = lib.get_container_transforms(container, root=True) - self.log.info(root) if not root or root not in instance_lookup: continue diff --git a/openpype/hosts/maya/plugins/publish/collect_history.py b/openpype/hosts/maya/plugins/publish/collect_history.py index 71f0169971..d4e8c6298b 100644 --- a/openpype/hosts/maya/plugins/publish/collect_history.py +++ b/openpype/hosts/maya/plugins/publish/collect_history.py @@ -18,7 +18,6 @@ class CollectMayaHistory(pyblish.api.InstancePlugin): hosts = ["maya"] label = "Maya History" families = ["rig"] - verbose = False def process(self, instance): diff --git a/openpype/hosts/maya/plugins/publish/collect_instances.py b/openpype/hosts/maya/plugins/publish/collect_instances.py index 5f914b40d7..5058da3d01 100644 --- a/openpype/hosts/maya/plugins/publish/collect_instances.py +++ b/openpype/hosts/maya/plugins/publish/collect_instances.py @@ -28,6 +28,8 @@ class CollectNewInstances(pyblish.api.InstancePlugin): order = pyblish.api.CollectorOrder hosts = ["maya"] + valid_empty_families = {"workfile", "renderlayer"} + def process(self, instance): objset = instance.data.get("instance_node") @@ -58,7 +60,7 @@ class CollectNewInstances(pyblish.api.InstancePlugin): instance[:] = members_hierarchy - elif instance.data["family"] != "workfile": + elif instance.data["family"] not in self.valid_empty_families: self.log.warning("Empty instance: \"%s\" " % objset) # Store the exact members of the object set instance.data["setMembers"] = members diff --git a/openpype/hosts/maya/plugins/publish/collect_look.py b/openpype/hosts/maya/plugins/publish/collect_look.py index b3da920566..db042963c6 100644 --- a/openpype/hosts/maya/plugins/publish/collect_look.py +++ b/openpype/hosts/maya/plugins/publish/collect_look.py @@ -17,11 +17,6 @@ SHAPE_ATTRS = ["castsShadows", "visibleInRefractions", "doubleSided", "opposite"] - -RENDERER_NODE_TYPES = [ - # redshift - "RedshiftMeshParameters" -] SHAPE_ATTRS = set(SHAPE_ATTRS) @@ -36,12 +31,13 @@ def get_pxr_multitexture_file_attrs(node): FILE_NODES = { + # maya "file": "fileTextureName", - + # arnold (mtoa) "aiImage": "filename", - + # redshift "RedshiftNormalMap": "tex0", - + # renderman "PxrBump": "filename", "PxrNormalMap": "filename", "PxrMultiTexture": get_pxr_multitexture_file_attrs, @@ -49,6 +45,22 @@ FILE_NODES = { "PxrTexture": "filename" } +# Keep only node types that actually exist +all_node_types = set(cmds.allNodeTypes()) +for node_type in list(FILE_NODES.keys()): + if node_type not in all_node_types: + FILE_NODES.pop(node_type) +del all_node_types + +# Cache pixar dependency node types so we can perform a type lookup against it +PXR_NODES = set() +if cmds.pluginInfo("RenderMan_for_Maya", query=True, loaded=True): + PXR_NODES = set( + cmds.pluginInfo("RenderMan_for_Maya", + query=True, + dependNode=True) + ) + def get_attributes(dictionary, attr, node=None): # type: (dict, str, str) -> list @@ -232,20 +244,17 @@ def get_file_node_files(node): """ paths = get_file_node_paths(node) - sequences = [] - replaces = [] + + # For sequences get all files and filter to only existing files + result = [] for index, path in enumerate(paths): if node_uses_image_sequence(node, path): glob_pattern = seq_to_glob(path) - sequences.extend(glob.glob(glob_pattern)) - replaces.append(index) + result.extend(glob.glob(glob_pattern)) + elif os.path.exists(path): + result.append(path) - for index in replaces: - paths.pop(index) - - paths.extend(sequences) - - return [p for p in paths if os.path.exists(p)] + return result class CollectLook(pyblish.api.InstancePlugin): @@ -260,7 +269,7 @@ class CollectLook(pyblish.api.InstancePlugin): membership relations. Collects: - lookAttribtutes (list): Nodes in instance with their altered attributes + lookAttributes (list): Nodes in instance with their altered attributes lookSetRelations (list): Sets and their memberships lookSets (list): List of set names included in the look @@ -286,7 +295,10 @@ class CollectLook(pyblish.api.InstancePlugin): """ self.log.debug("Looking for look associations " - "for %s" % instance.data['name']) + "for %s" % instance.data['name']) + + # Lookup set (optimization) + instance_lookup = set(cmds.ls(instance, long=True)) # Discover related object sets self.log.debug("Gathering sets ...") @@ -296,68 +308,24 @@ class CollectLook(pyblish.api.InstancePlugin): instance_lookup = set(cmds.ls(instance, long=True)) self.log.debug("Gathering set relations ...") - # Ensure iteration happen in a list so we can remove keys from the + # Ensure iteration happen in a list to allow removing keys from the # dict within the loop - - # skipped types of attribute on render specific nodes - disabled_types = ["message", "TdataCompound"] - for obj_set in list(sets): self.log.debug("From {}".format(obj_set)) - - # if node is specified as renderer node type, it will be - # serialized with its attributes. - if cmds.nodeType(obj_set) in RENDERER_NODE_TYPES: - self.log.debug("- {} is {}".format( - obj_set, cmds.nodeType(obj_set))) - - node_attrs = [] - - # serialize its attributes so they can be recreated on look - # load. - for attr in cmds.listAttr(obj_set): - # skip publishedNodeInfo attributes as they break - # getAttr() and we don't need them anyway - if attr.startswith("publishedNodeInfo"): - continue - - # skip attributes types defined in 'disabled_type' list - if cmds.getAttr("{}.{}".format(obj_set, attr), type=True) in disabled_types: # noqa - continue - - node_attrs.append(( - attr, - cmds.getAttr("{}.{}".format(obj_set, attr)), - cmds.getAttr( - "{}.{}".format(obj_set, attr), type=True) - )) - - for member in cmds.ls( - cmds.sets(obj_set, query=True), long=True): - member_data = self.collect_member_data(member, - instance_lookup) - if not member_data: - continue - - # Add information of the node to the members list - sets[obj_set]["members"].append(member_data) - # Get all nodes of the current objectSet (shadingEngine) for member in cmds.ls(cmds.sets(obj_set, query=True), long=True): member_data = self.collect_member_data(member, instance_lookup) - if not member_data: - continue - - # Add information of the node to the members list - sets[obj_set]["members"].append(member_data) + if member_data: + # Add information of the node to the members list + sets[obj_set]["members"].append(member_data) # Remove sets that didn't have any members assigned in the end # Thus the data will be limited to only what we need. - self.log.debug("obj_set {}".format(sets[obj_set])) if not sets[obj_set]["members"]: - self.log.info( - "Removing redundant set information: {}".format(obj_set)) + self.log.debug( + "Removing redundant set information: {}".format(obj_set) + ) sets.pop(obj_set, None) self.log.debug("Gathering attribute changes to instance members..") @@ -382,35 +350,28 @@ class CollectLook(pyblish.api.InstancePlugin): "rman__displacement" ] if look_sets: - materials = [] + self.log.debug("Found look sets: {}".format(look_sets)) + # Get all material attrs for all look sets to retrieve their inputs + existing_attrs = [] for look in look_sets: - for at in shader_attrs: - try: - con = cmds.listConnections("{}.{}".format(look, at)) - except ValueError: - # skip attributes that are invalid in current - # context. For example in the case where - # Arnold is not enabled. - continue - if con: - materials.extend(con) + for attr in shader_attrs: + if cmds.attributeQuery(attr, node=look, exists=True): + existing_attrs.append("{}.{}".format(look, attr)) + materials = cmds.listConnections(existing_attrs, + source=True, + destination=False) or [] - self.log.info("Found materials:\n{}".format(materials)) + self.log.debug("Found materials:\n{}".format(materials)) - self.log.info("Found the following sets:\n{}".format(look_sets)) + self.log.debug("Found the following sets:\n{}".format(look_sets)) # Get the entire node chain of the look sets - # history = cmds.listHistory(look_sets) - history = [] - for material in materials: - history.extend(cmds.listHistory(material, ac=True)) - - # handle VrayPluginNodeMtl node - see #1397 - vray_plugin_nodes = cmds.ls( - history, type="VRayPluginNodeMtl", long=True) - for vray_node in vray_plugin_nodes: - history.extend(cmds.listHistory(vray_node, ac=True)) + # history = cmds.listHistory(look_sets, allConnections=True) + history = cmds.listHistory(materials, allConnections=True) + # Since we retrieved history only of the connected materials + # connected to the look sets above we now add direct history + # for some of the look sets directly # handling render attribute sets render_set_types = [ "VRayDisplacement", @@ -428,20 +389,26 @@ class CollectLook(pyblish.api.InstancePlugin): or [] ) - all_supported_nodes = FILE_NODES.keys() - files = [] - for node_type in all_supported_nodes: - files.extend(cmds.ls(history, type=node_type, long=True)) + # Ensure unique entries only + history = list(set(history)) + + files = cmds.ls(history, + # It's important only node types are passed that + # exist (e.g. for loaded plugins) because otherwise + # the result will turn back empty + type=list(FILE_NODES.keys()), + long=True) + + # Sort for log readability + files.sort() self.log.debug("Collected file nodes:\n{}".format(files)) # Collect textures if any file nodes are found - instance.data["resources"] = [] - for n in files: - for res in self.collect_resources(n): - instance.data["resources"].append(res) - - self.log.debug("Collected resources: {}".format( - instance.data["resources"])) + resources = [] + for node in files: # sort for log readability + resources.extend(self.collect_resources(node)) + instance.data["resources"] = resources + self.log.debug("Collected resources: {}".format(resources)) # Log warning when no relevant sets were retrieved for the look. if ( @@ -456,7 +423,7 @@ class CollectLook(pyblish.api.InstancePlugin): instance.extend(shader for shader in look_sets if shader not in instance_lookup) - self.log.info("Collected look for %s" % instance) + self.log.debug("Collected look for %s" % instance) def collect_sets(self, instance): """Collect all objectSets which are of importance for publishing @@ -536,14 +503,14 @@ class CollectLook(pyblish.api.InstancePlugin): # Collect changes to "custom" attributes node_attrs = get_look_attrs(node) - self.log.debug( - "Node \"{0}\" attributes: {1}".format(node, node_attrs) - ) - # Only include if there are any properties we care about if not node_attrs: continue + self.log.debug( + "Node \"{0}\" attributes: {1}".format(node, node_attrs) + ) + node_attributes = {} for attr in node_attrs: if not cmds.attributeQuery(attr, node=node, exists=True): @@ -574,14 +541,14 @@ class CollectLook(pyblish.api.InstancePlugin): Returns: dict """ - self.log.debug("processing: {}".format(node)) - all_supported_nodes = FILE_NODES.keys() - if cmds.nodeType(node) not in all_supported_nodes: + if cmds.nodeType(node) not in FILE_NODES: self.log.error( "Unsupported file node: {}".format(cmds.nodeType(node))) raise AssertionError("Unsupported file node") - self.log.debug(" - got {}".format(cmds.nodeType(node))) + self.log.debug( + "Collecting resource: {} ({})".format(node, cmds.nodeType(node)) + ) attributes = get_attributes(FILE_NODES, cmds.nodeType(node), node) for attribute in attributes: @@ -589,39 +556,34 @@ class CollectLook(pyblish.api.InstancePlugin): node, attribute )) - computed_attribute = "{}.{}".format(node, attribute) - if attribute == "fileTextureName": - computed_attribute = node + ".computedFileTextureNamePattern" - self.log.info(" - file source: {}".format(source)) + self.log.debug(" - file source: {}".format(source)) color_space_attr = "{}.colorSpace".format(node) try: color_space = cmds.getAttr(color_space_attr) except ValueError: # node doesn't have colorspace attribute color_space = "Raw" + # Compare with the computed file path, e.g. the one with # the pattern in it, to generate some logging information # about this difference - computed_source = cmds.getAttr(computed_attribute) - if source != computed_source: - self.log.debug("Detected computed file pattern difference " - "from original pattern: {0} " - "({1} -> {2})".format(node, - source, - computed_source)) + # Only for file nodes with `fileTextureName` attribute + if attribute == "fileTextureName": + computed_source = cmds.getAttr( + "{}.computedFileTextureNamePattern".format(node) + ) + if source != computed_source: + self.log.debug("Detected computed file pattern difference " + "from original pattern: {0} " + "({1} -> {2})".format(node, + source, + computed_source)) # renderman allows nodes to have filename attribute empty while # you can have another incoming connection from different node. - pxr_nodes = set() - if cmds.pluginInfo("RenderMan_for_Maya", query=True, loaded=True): - pxr_nodes = set( - cmds.pluginInfo("RenderMan_for_Maya", - query=True, - dependNode=True) - ) - if not source and cmds.nodeType(node) in pxr_nodes: - self.log.info("Renderman: source is empty, skipping...") + if not source and cmds.nodeType(node) in PXR_NODES: + self.log.debug("Renderman: source is empty, skipping...") continue # We replace backslashes with forward slashes because V-Ray # can't handle the UDIM files with the backslashes in the @@ -630,14 +592,14 @@ class CollectLook(pyblish.api.InstancePlugin): files = get_file_node_files(node) if len(files) == 0: - self.log.error("No valid files found from node `%s`" % node) + self.log.debug("No valid files found from node `%s`" % node) - self.log.info("collection of resource done:") - self.log.info(" - node: {}".format(node)) - self.log.info(" - attribute: {}".format(attribute)) - self.log.info(" - source: {}".format(source)) - self.log.info(" - file: {}".format(files)) - self.log.info(" - color space: {}".format(color_space)) + self.log.debug("collection of resource done:") + self.log.debug(" - node: {}".format(node)) + self.log.debug(" - attribute: {}".format(attribute)) + self.log.debug(" - source: {}".format(source)) + self.log.debug(" - file: {}".format(files)) + self.log.debug(" - color space: {}".format(color_space)) # Define the resource yield { diff --git a/openpype/hosts/maya/plugins/publish/collect_multiverse_look.py b/openpype/hosts/maya/plugins/publish/collect_multiverse_look.py index 33fc7a025f..bcb979edfc 100644 --- a/openpype/hosts/maya/plugins/publish/collect_multiverse_look.py +++ b/openpype/hosts/maya/plugins/publish/collect_multiverse_look.py @@ -268,7 +268,7 @@ class CollectMultiverseLookData(pyblish.api.InstancePlugin): cmds.loadPlugin("MultiverseForMaya", quiet=True) import multiverse - self.log.info("Processing mvLook for '{}'".format(instance)) + self.log.debug("Processing mvLook for '{}'".format(instance)) nodes = set() for node in instance: @@ -281,13 +281,12 @@ class CollectMultiverseLookData(pyblish.api.InstancePlugin): long=True) nodes.update(nodes_of_interest) - files = [] sets = {} instance.data["resources"] = [] publishMipMap = instance.data["publishMipMap"] for node in nodes: - self.log.info("Getting resources for '{}'".format(node)) + self.log.debug("Getting resources for '{}'".format(node)) # We know what nodes need to be collected, now we need to # extract the materials overrides. @@ -380,12 +379,12 @@ class CollectMultiverseLookData(pyblish.api.InstancePlugin): if len(files) == 0: self.log.error("No valid files found from node `%s`" % node) - self.log.info("collection of resource done:") - self.log.info(" - node: {}".format(node)) - self.log.info(" - attribute: {}".format(fname_attrib)) - self.log.info(" - source: {}".format(source)) - self.log.info(" - file: {}".format(files)) - self.log.info(" - color space: {}".format(color_space)) + self.log.debug("collection of resource done:") + self.log.debug(" - node: {}".format(node)) + self.log.debug(" - attribute: {}".format(fname_attrib)) + self.log.debug(" - source: {}".format(source)) + self.log.debug(" - file: {}".format(files)) + self.log.debug(" - color space: {}".format(color_space)) # Define the resource resource = {"node": node, @@ -406,14 +405,14 @@ class CollectMultiverseLookData(pyblish.api.InstancePlugin): extra_files = [] self.log.debug("Expecting MipMaps, going to look for them.") for fname in files: - self.log.info("Checking '{}' for mipmaps".format(fname)) + self.log.debug("Checking '{}' for mipmaps".format(fname)) if is_mipmap(fname): self.log.debug(" - file is already MipMap, skipping.") continue mipmap = get_mipmap(fname) if mipmap: - self.log.info(" mipmap found for '{}'".format(fname)) + self.log.debug(" mipmap found for '{}'".format(fname)) extra_files.append(mipmap) else: self.log.warning(" no mipmap found for '{}'".format(fname)) diff --git a/openpype/hosts/maya/plugins/publish/collect_pointcache.py b/openpype/hosts/maya/plugins/publish/collect_pointcache.py index bb9065792f..5578a57f31 100644 --- a/openpype/hosts/maya/plugins/publish/collect_pointcache.py +++ b/openpype/hosts/maya/plugins/publish/collect_pointcache.py @@ -45,18 +45,3 @@ class CollectPointcache(pyblish.api.InstancePlugin): if proxy_set: instance.remove(proxy_set) instance.data["setMembers"].remove(proxy_set) - - # Collect user defined attributes. - if not instance.data.get("includeUserDefinedAttributes", False): - return - - user_defined_attributes = set() - for node in instance: - attrs = cmds.listAttr(node, userDefined=True) or list() - shapes = cmds.listRelatives(node, shapes=True) or list() - for shape in shapes: - attrs.extend(cmds.listAttr(shape, userDefined=True) or list()) - - user_defined_attributes.update(attrs) - - instance.data["userDefinedAttributes"] = list(user_defined_attributes) diff --git a/openpype/hosts/maya/plugins/publish/collect_render.py b/openpype/hosts/maya/plugins/publish/collect_render.py index c17a8789e4..886c2b4caa 100644 --- a/openpype/hosts/maya/plugins/publish/collect_render.py +++ b/openpype/hosts/maya/plugins/publish/collect_render.py @@ -105,7 +105,7 @@ class CollectMayaRender(pyblish.api.InstancePlugin): "family": cmds.getAttr("{}.family".format(s)), } ) - self.log.info(" -> attach render to: {}".format(s)) + self.log.debug(" -> attach render to: {}".format(s)) layer_name = layer.name() @@ -137,10 +137,10 @@ class CollectMayaRender(pyblish.api.InstancePlugin): has_cameras = any(product.camera for product in render_products) assert has_cameras, "No render cameras found." - self.log.info("multipart: {}".format( + self.log.debug("multipart: {}".format( multipart)) assert expected_files, "no file names were generated, this is a bug" - self.log.info( + self.log.debug( "expected files: {}".format( json.dumps(expected_files, indent=4, sort_keys=True) ) @@ -157,10 +157,10 @@ class CollectMayaRender(pyblish.api.InstancePlugin): # append full path aov_dict = {} - default_render_folder = context.data.get("project_settings")\ - .get("maya")\ - .get("RenderSettings")\ - .get("default_render_image_folder") or "" + image_directory = os.path.join( + cmds.workspace(query=True, rootDirectory=True), + cmds.workspace(fileRuleEntry="images") + ) # replace relative paths with absolute. Render products are # returned as list of dictionaries. publish_meta_path = None @@ -168,14 +168,13 @@ class CollectMayaRender(pyblish.api.InstancePlugin): full_paths = [] aov_first_key = list(aov.keys())[0] for file in aov[aov_first_key]: - full_path = os.path.join(workspace, default_render_folder, - file) + full_path = os.path.join(image_directory, file) full_path = full_path.replace("\\", "/") full_paths.append(full_path) publish_meta_path = os.path.dirname(full_path) aov_dict[aov_first_key] = full_paths full_exp_files = [aov_dict] - self.log.info(full_exp_files) + self.log.debug(full_exp_files) if publish_meta_path is None: raise KnownPublishError("Unable to detect any expected output " @@ -227,7 +226,7 @@ class CollectMayaRender(pyblish.api.InstancePlugin): if platform.system().lower() in ["linux", "darwin"]: common_publish_meta_path = "/" + common_publish_meta_path - self.log.info( + self.log.debug( "Publish meta path: {}".format(common_publish_meta_path)) # Get layer specific settings, might be overrides @@ -300,7 +299,7 @@ class CollectMayaRender(pyblish.api.InstancePlugin): ) if rr_settings["enabled"]: data["rrPathName"] = instance.data.get("rrPathName") - self.log.info(data["rrPathName"]) + self.log.debug(data["rrPathName"]) if self.sync_workfile_version: data["version"] = context.data["version"] diff --git a/openpype/hosts/maya/plugins/publish/collect_render_layer_aovs.py b/openpype/hosts/maya/plugins/publish/collect_render_layer_aovs.py index c3dc31ead9..035c531a9b 100644 --- a/openpype/hosts/maya/plugins/publish/collect_render_layer_aovs.py +++ b/openpype/hosts/maya/plugins/publish/collect_render_layer_aovs.py @@ -37,7 +37,7 @@ class CollectRenderLayerAOVS(pyblish.api.InstancePlugin): # Get renderer renderer = instance.data["renderer"] - self.log.info("Renderer found: {}".format(renderer)) + self.log.debug("Renderer found: {}".format(renderer)) rp_node_types = {"vray": ["VRayRenderElement", "VRayRenderElementSet"], "arnold": ["aiAOV"], @@ -66,8 +66,8 @@ class CollectRenderLayerAOVS(pyblish.api.InstancePlugin): result.append(render_pass) - self.log.info("Found {} render elements / AOVs for " - "'{}'".format(len(result), instance.data["subset"])) + self.log.debug("Found {} render elements / AOVs for " + "'{}'".format(len(result), instance.data["subset"])) instance.data["renderPasses"] = result diff --git a/openpype/hosts/maya/plugins/publish/collect_renderable_camera.py b/openpype/hosts/maya/plugins/publish/collect_renderable_camera.py index d1c3cf3b2c..4443e2e0db 100644 --- a/openpype/hosts/maya/plugins/publish/collect_renderable_camera.py +++ b/openpype/hosts/maya/plugins/publish/collect_renderable_camera.py @@ -21,11 +21,12 @@ class CollectRenderableCamera(pyblish.api.InstancePlugin): else: layer = instance.data["renderlayer"] - self.log.info("layer: {}".format(layer)) cameras = cmds.ls(type="camera", long=True) - renderable = [c for c in cameras if - get_attr_in_layer("%s.renderable" % c, layer)] + renderable = [cam for cam in cameras if + get_attr_in_layer("{}.renderable".format(cam), layer)] - self.log.info("Found cameras %s: %s" % (len(renderable), renderable)) + self.log.debug( + "Found renderable cameras %s: %s", len(renderable), renderable + ) instance.data["cameras"] = renderable diff --git a/openpype/hosts/maya/plugins/publish/collect_rig_sets.py b/openpype/hosts/maya/plugins/publish/collect_rig_sets.py new file mode 100644 index 0000000000..36a4211af1 --- /dev/null +++ b/openpype/hosts/maya/plugins/publish/collect_rig_sets.py @@ -0,0 +1,39 @@ +import pyblish.api +from maya import cmds + + +class CollectRigSets(pyblish.api.InstancePlugin): + """Ensure rig contains pipeline-critical content + + Every rig must contain at least two object sets: + "controls_SET" - Set of all animatable controls + "out_SET" - Set of all cacheable meshes + + """ + + order = pyblish.api.CollectorOrder + 0.05 + label = "Collect Rig Sets" + hosts = ["maya"] + families = ["rig"] + + accepted_output = ["mesh", "transform"] + accepted_controllers = ["transform"] + + def process(self, instance): + + # Find required sets by suffix + searching = {"controls_SET", "out_SET"} + found = {} + for node in cmds.ls(instance, exactType="objectSet"): + for suffix in searching: + if node.endswith(suffix): + found[suffix] = node + searching.remove(suffix) + break + if not searching: + break + + self.log.debug("Found sets: {}".format(found)) + rig_sets = instance.data.setdefault("rig_sets", {}) + for name, objset in found.items(): + rig_sets[name] = objset diff --git a/openpype/hosts/maya/plugins/publish/collect_unreal_staticmesh.py b/openpype/hosts/maya/plugins/publish/collect_unreal_staticmesh.py index 79d0856fa0..03b6c4a188 100644 --- a/openpype/hosts/maya/plugins/publish/collect_unreal_staticmesh.py +++ b/openpype/hosts/maya/plugins/publish/collect_unreal_staticmesh.py @@ -19,7 +19,7 @@ class CollectUnrealStaticMesh(pyblish.api.InstancePlugin): instance.data["geometryMembers"] = cmds.sets( geometry_set, query=True) - self.log.info("geometry: {}".format( + self.log.debug("geometry: {}".format( pformat(instance.data.get("geometryMembers")))) collision_set = [ @@ -29,7 +29,7 @@ class CollectUnrealStaticMesh(pyblish.api.InstancePlugin): instance.data["collisionMembers"] = cmds.sets( collision_set, query=True) - self.log.info("collisions: {}".format( + self.log.debug("collisions: {}".format( pformat(instance.data.get("collisionMembers")))) frame = cmds.currentTime(query=True) diff --git a/openpype/hosts/maya/plugins/publish/collect_user_defined_attributes.py b/openpype/hosts/maya/plugins/publish/collect_user_defined_attributes.py new file mode 100644 index 0000000000..16fef2e168 --- /dev/null +++ b/openpype/hosts/maya/plugins/publish/collect_user_defined_attributes.py @@ -0,0 +1,39 @@ +from maya import cmds + +import pyblish.api + + +class CollectUserDefinedAttributes(pyblish.api.InstancePlugin): + """Collect user defined attributes for nodes in instance.""" + + order = pyblish.api.CollectorOrder + 0.45 + families = ["pointcache", "animation", "usd"] + label = "Collect User Defined Attributes" + hosts = ["maya"] + + def process(self, instance): + + # Collect user defined attributes. + if not instance.data.get("includeUserDefinedAttributes", False): + return + + if "out_hierarchy" in instance.data: + # animation family + nodes = instance.data["out_hierarchy"] + else: + nodes = instance[:] + if not nodes: + return + + shapes = cmds.listRelatives(nodes, shapes=True, fullPath=True) or [] + nodes = set(nodes).union(shapes) + + attrs = cmds.listAttr(list(nodes), userDefined=True) or [] + user_defined_attributes = list(sorted(set(attrs))) + instance.data["userDefinedAttributes"] = user_defined_attributes + + self.log.debug( + "Collected user defined attributes: {}".format( + ", ".join(user_defined_attributes) + ) + ) diff --git a/openpype/hosts/maya/plugins/publish/collect_xgen.py b/openpype/hosts/maya/plugins/publish/collect_xgen.py index 46968f7d1a..45648e1776 100644 --- a/openpype/hosts/maya/plugins/publish/collect_xgen.py +++ b/openpype/hosts/maya/plugins/publish/collect_xgen.py @@ -67,5 +67,5 @@ class CollectXgen(pyblish.api.InstancePlugin): data["transfers"] = transfers - self.log.info(data) + self.log.debug(data) instance.data.update(data) diff --git a/openpype/hosts/maya/plugins/publish/collect_yeti_cache.py b/openpype/hosts/maya/plugins/publish/collect_yeti_cache.py index e6b5ca4260..4dcda29050 100644 --- a/openpype/hosts/maya/plugins/publish/collect_yeti_cache.py +++ b/openpype/hosts/maya/plugins/publish/collect_yeti_cache.py @@ -4,12 +4,23 @@ import pyblish.api from openpype.hosts.maya.api import lib -SETTINGS = {"renderDensity", - "renderWidth", - "renderLength", - "increaseRenderBounds", - "imageSearchPath", - "cbId"} + +SETTINGS = { + # Preview + "displayOutput", + "colorR", "colorG", "colorB", + "viewportDensity", + "viewportWidth", + "viewportLength", + # Render attributes + "renderDensity", + "renderWidth", + "renderLength", + "increaseRenderBounds", + "imageSearchPath", + # Pipeline specific + "cbId" +} class CollectYetiCache(pyblish.api.InstancePlugin): @@ -39,10 +50,6 @@ class CollectYetiCache(pyblish.api.InstancePlugin): # Get yeti nodes and their transforms yeti_shapes = cmds.ls(instance, type="pgYetiMaya") for shape in yeti_shapes: - shape_data = {"transform": None, - "name": shape, - "cbId": lib.get_id(shape), - "attrs": None} # Get specific node attributes attr_data = {} @@ -58,9 +65,12 @@ class CollectYetiCache(pyblish.api.InstancePlugin): parent = cmds.listRelatives(shape, parent=True)[0] transform_data = {"name": parent, "cbId": lib.get_id(parent)} - # Store collected data - shape_data["attrs"] = attr_data - shape_data["transform"] = transform_data + shape_data = { + "transform": transform_data, + "name": shape, + "cbId": lib.get_id(shape), + "attrs": attr_data, + } settings["nodes"].append(shape_data) diff --git a/openpype/hosts/maya/plugins/publish/collect_yeti_rig.py b/openpype/hosts/maya/plugins/publish/collect_yeti_rig.py index bc15edd9e0..df761cde13 100644 --- a/openpype/hosts/maya/plugins/publish/collect_yeti_rig.py +++ b/openpype/hosts/maya/plugins/publish/collect_yeti_rig.py @@ -119,7 +119,6 @@ class CollectYetiRig(pyblish.api.InstancePlugin): texture_filenames = [] if image_search_paths: - # TODO: Somehow this uses OS environment path separator, `:` vs `;` # Later on check whether this is pipeline OS cross-compatible. image_search_paths = [p for p in @@ -130,13 +129,13 @@ class CollectYetiRig(pyblish.api.InstancePlugin): # List all related textures texture_filenames = cmds.pgYetiCommand(node, listTextures=True) - self.log.info("Found %i texture(s)" % len(texture_filenames)) + self.log.debug("Found %i texture(s)" % len(texture_filenames)) # Get all reference nodes reference_nodes = cmds.pgYetiGraph(node, listNodes=True, type="reference") - self.log.info("Found %i reference node(s)" % len(reference_nodes)) + self.log.debug("Found %i reference node(s)" % len(reference_nodes)) if texture_filenames and not image_search_paths: raise ValueError("pgYetiMaya node '%s' is missing the path to the " diff --git a/openpype/hosts/maya/plugins/publish/extract_active_view_thumbnail.py b/openpype/hosts/maya/plugins/publish/extract_active_view_thumbnail.py new file mode 100644 index 0000000000..483ae6d9d3 --- /dev/null +++ b/openpype/hosts/maya/plugins/publish/extract_active_view_thumbnail.py @@ -0,0 +1,60 @@ +import maya.api.OpenMaya as om +import maya.api.OpenMayaUI as omui + +import pyblish.api +import tempfile + +from openpype.hosts.maya.api.lib import IS_HEADLESS + + +class ExtractActiveViewThumbnail(pyblish.api.InstancePlugin): + """Set instance thumbnail to a screengrab of current active viewport. + + This makes it so that if an instance does not have a thumbnail set yet that + it will get a thumbnail of the currently active view at the time of + publishing as a fallback. + + """ + order = pyblish.api.ExtractorOrder + 0.49 + label = "Active View Thumbnail" + families = ["workfile"] + hosts = ["maya"] + + def process(self, instance): + if IS_HEADLESS: + self.log.debug( + "Skip extraction of active view thumbnail, due to being in" + "headless mode." + ) + return + + thumbnail = instance.data.get("thumbnailPath") + if not thumbnail: + view_thumbnail = self.get_view_thumbnail(instance) + if not view_thumbnail: + return + + self.log.debug("Setting instance thumbnail path to: {}".format( + view_thumbnail + )) + instance.data["thumbnailPath"] = view_thumbnail + + def get_view_thumbnail(self, instance): + cache_key = "__maya_view_thumbnail" + context = instance.context + + if cache_key not in context.data: + # Generate only a single thumbnail, even for multiple instances + with tempfile.NamedTemporaryFile(suffix="_thumbnail.jpg", + delete=False) as f: + path = f.name + + view = omui.M3dView.active3dView() + image = om.MImage() + view.readColorBuffer(image, True) + image.writeToFile(path, "jpg") + self.log.debug("Generated thumbnail: {}".format(path)) + + context.data["cleanupFullPaths"].append(path) + context.data[cache_key] = path + return context.data[cache_key] diff --git a/openpype/hosts/maya/plugins/publish/extract_arnold_scene_source.py b/openpype/hosts/maya/plugins/publish/extract_arnold_scene_source.py index 102f0e46a2..46cc9090bb 100644 --- a/openpype/hosts/maya/plugins/publish/extract_arnold_scene_source.py +++ b/openpype/hosts/maya/plugins/publish/extract_arnold_scene_source.py @@ -100,7 +100,7 @@ class ExtractArnoldSceneSource(publish.Extractor): instance.data["representations"].append(representation) - self.log.info( + self.log.debug( "Extracted instance {} to: {}".format(instance.name, staging_dir) ) @@ -126,7 +126,7 @@ class ExtractArnoldSceneSource(publish.Extractor): instance.data["representations"].append(representation) def _extract(self, nodes, attribute_data, kwargs): - self.log.info( + self.log.debug( "Writing {} with:\n{}".format(kwargs["filename"], kwargs) ) filenames = [] @@ -180,12 +180,12 @@ class ExtractArnoldSceneSource(publish.Extractor): with lib.attribute_values(attribute_data): with lib.maintained_selection(): - self.log.info( + self.log.debug( "Writing: {}".format(duplicate_nodes) ) cmds.select(duplicate_nodes, noExpand=True) - self.log.info( + self.log.debug( "Extracting ass sequence with: {}".format(kwargs) ) @@ -194,6 +194,6 @@ class ExtractArnoldSceneSource(publish.Extractor): for file in exported_files: filenames.append(os.path.split(file)[1]) - self.log.info("Exported: {}".format(filenames)) + self.log.debug("Exported: {}".format(filenames)) return filenames, nodes_by_id diff --git a/openpype/hosts/maya/plugins/publish/extract_assembly.py b/openpype/hosts/maya/plugins/publish/extract_assembly.py index 9b2978d192..86ffdcef24 100644 --- a/openpype/hosts/maya/plugins/publish/extract_assembly.py +++ b/openpype/hosts/maya/plugins/publish/extract_assembly.py @@ -27,7 +27,7 @@ class ExtractAssembly(publish.Extractor): json_filename = "{}.json".format(instance.name) json_path = os.path.join(staging_dir, json_filename) - self.log.info("Dumping scene data for debugging ..") + self.log.debug("Dumping scene data for debugging ..") with open(json_path, "w") as filepath: json.dump(instance.data["scenedata"], filepath, ensure_ascii=False) diff --git a/openpype/hosts/maya/plugins/publish/extract_camera_alembic.py b/openpype/hosts/maya/plugins/publish/extract_camera_alembic.py index aa445a0387..4ec1399df4 100644 --- a/openpype/hosts/maya/plugins/publish/extract_camera_alembic.py +++ b/openpype/hosts/maya/plugins/publish/extract_camera_alembic.py @@ -94,7 +94,7 @@ class ExtractCameraAlembic(publish.Extractor): "Attributes to bake must be specified as a list" ) for attr in self.bake_attributes: - self.log.info("Adding {} attribute".format(attr)) + self.log.debug("Adding {} attribute".format(attr)) job_str += " -attr {0}".format(attr) with lib.evaluation("off"): @@ -112,5 +112,5 @@ class ExtractCameraAlembic(publish.Extractor): } instance.data["representations"].append(representation) - self.log.info("Extracted instance '{0}' to: {1}".format( + self.log.debug("Extracted instance '{0}' to: {1}".format( instance.name, path)) diff --git a/openpype/hosts/maya/plugins/publish/extract_camera_mayaScene.py b/openpype/hosts/maya/plugins/publish/extract_camera_mayaScene.py index 30e6b89f2f..a50a8f0dfa 100644 --- a/openpype/hosts/maya/plugins/publish/extract_camera_mayaScene.py +++ b/openpype/hosts/maya/plugins/publish/extract_camera_mayaScene.py @@ -111,7 +111,7 @@ class ExtractCameraMayaScene(publish.Extractor): for family in self.families: try: self.scene_type = ext_mapping[family] - self.log.info( + self.log.debug( "Using {} as scene type".format(self.scene_type)) break except KeyError: @@ -151,7 +151,7 @@ class ExtractCameraMayaScene(publish.Extractor): with lib.evaluation("off"): with lib.suspended_refresh(): if bake_to_worldspace: - self.log.info( + self.log.debug( "Performing camera bakes: {}".format(transform)) baked = lib.bake_to_world_space( transform, @@ -186,7 +186,7 @@ class ExtractCameraMayaScene(publish.Extractor): unlock(plug) cmds.setAttr(plug, value) - self.log.info("Performing extraction..") + self.log.debug("Performing extraction..") cmds.select(cmds.ls(members, dag=True, shapes=True, long=True), noExpand=True) cmds.file(path, @@ -217,5 +217,5 @@ class ExtractCameraMayaScene(publish.Extractor): } instance.data["representations"].append(representation) - self.log.info("Extracted instance '{0}' to: {1}".format( + self.log.debug("Extracted instance '{0}' to: {1}".format( instance.name, path)) diff --git a/openpype/hosts/maya/plugins/publish/extract_fbx.py b/openpype/hosts/maya/plugins/publish/extract_fbx.py index 9af3acef65..4f7eaf57bf 100644 --- a/openpype/hosts/maya/plugins/publish/extract_fbx.py +++ b/openpype/hosts/maya/plugins/publish/extract_fbx.py @@ -33,11 +33,11 @@ class ExtractFBX(publish.Extractor): # to format it into a string in a mel expression path = path.replace('\\', '/') - self.log.info("Extracting FBX to: {0}".format(path)) + self.log.debug("Extracting FBX to: {0}".format(path)) members = instance.data["setMembers"] - self.log.info("Members: {0}".format(members)) - self.log.info("Instance: {0}".format(instance[:])) + self.log.debug("Members: {0}".format(members)) + self.log.debug("Instance: {0}".format(instance[:])) fbx_exporter.set_options_from_instance(instance) @@ -58,4 +58,4 @@ class ExtractFBX(publish.Extractor): } instance.data["representations"].append(representation) - self.log.info("Extract FBX successful to: {0}".format(path)) + self.log.debug("Extract FBX successful to: {0}".format(path)) diff --git a/openpype/hosts/maya/plugins/publish/extract_gltf.py b/openpype/hosts/maya/plugins/publish/extract_gltf.py index ac258ffb3d..6d72d28525 100644 --- a/openpype/hosts/maya/plugins/publish/extract_gltf.py +++ b/openpype/hosts/maya/plugins/publish/extract_gltf.py @@ -20,14 +20,10 @@ class ExtractGLB(publish.Extractor): filename = "{0}.glb".format(instance.name) path = os.path.join(staging_dir, filename) - self.log.info("Extracting GLB to: {}".format(path)) - cmds.loadPlugin("maya2glTF", quiet=True) nodes = instance[:] - self.log.info("Instance: {0}".format(nodes)) - start_frame = instance.data('frameStart') or \ int(cmds.playbackOptions(query=True, animationStartTime=True))# noqa @@ -48,6 +44,7 @@ class ExtractGLB(publish.Extractor): "vno": True # visibleNodeOnly } + self.log.debug("Extracting GLB to: {}".format(path)) with lib.maintained_selection(): cmds.select(nodes, hi=True, noExpand=True) extract_gltf(staging_dir, @@ -65,4 +62,4 @@ class ExtractGLB(publish.Extractor): } instance.data["representations"].append(representation) - self.log.info("Extract GLB successful to: {0}".format(path)) + self.log.debug("Extract GLB successful to: {0}".format(path)) diff --git a/openpype/hosts/maya/plugins/publish/extract_gpu_cache.py b/openpype/hosts/maya/plugins/publish/extract_gpu_cache.py index 422f5ad019..16436c6fe4 100644 --- a/openpype/hosts/maya/plugins/publish/extract_gpu_cache.py +++ b/openpype/hosts/maya/plugins/publish/extract_gpu_cache.py @@ -60,6 +60,6 @@ class ExtractGPUCache(publish.Extractor): instance.data["representations"].append(representation) - self.log.info( + self.log.debug( "Extracted instance {} to: {}".format(instance.name, staging_dir) ) diff --git a/openpype/hosts/maya/plugins/publish/extract_import_reference.py b/openpype/hosts/maya/plugins/publish/extract_import_reference.py index 8bb82be9b6..1fdee28d0c 100644 --- a/openpype/hosts/maya/plugins/publish/extract_import_reference.py +++ b/openpype/hosts/maya/plugins/publish/extract_import_reference.py @@ -30,8 +30,8 @@ class ExtractImportReference(publish.Extractor, tmp_format = "_tmp" @classmethod - def apply_settings(cls, project_setting, system_settings): - cls.active = project_setting["deadline"]["publish"]["MayaSubmitDeadline"]["import_reference"] # noqa + def apply_settings(cls, project_settings): + cls.active = project_settings["deadline"]["publish"]["MayaSubmitDeadline"]["import_reference"] # noqa def process(self, instance): if not self.is_active(instance.data): @@ -46,7 +46,7 @@ class ExtractImportReference(publish.Extractor, for family in self.families: try: self.scene_type = ext_mapping[family] - self.log.info( + self.log.debug( "Using {} as scene type".format(self.scene_type)) break @@ -69,7 +69,7 @@ class ExtractImportReference(publish.Extractor, reference_path = os.path.join(dir_path, ref_scene_name) tmp_path = os.path.dirname(current_name) + "/" + ref_scene_name - self.log.info("Performing extraction..") + self.log.debug("Performing extraction..") # This generates script for mayapy to take care of reference # importing outside current session. It is passing current scene @@ -111,7 +111,7 @@ print("*** Done") # process until handles are closed by context manager. with tempfile.TemporaryDirectory() as tmp_dir_name: tmp_script_path = os.path.join(tmp_dir_name, "import_ref.py") - self.log.info("Using script file: {}".format(tmp_script_path)) + self.log.debug("Using script file: {}".format(tmp_script_path)) with open(tmp_script_path, "wt") as tmp: tmp.write(script) @@ -149,9 +149,9 @@ print("*** Done") "stagingDir": os.path.dirname(current_name), "outputName": "imported" } - self.log.info("%s" % ref_representation) + self.log.debug(ref_representation) instance.data["representations"].append(ref_representation) - self.log.info("Extracted instance '%s' to : '%s'" % (ref_scene_name, - reference_path)) + self.log.debug("Extracted instance '%s' to : '%s'" % (ref_scene_name, + reference_path)) diff --git a/openpype/hosts/maya/plugins/publish/extract_layout.py b/openpype/hosts/maya/plugins/publish/extract_layout.py index bf5b4fc0e7..75920b44a2 100644 --- a/openpype/hosts/maya/plugins/publish/extract_layout.py +++ b/openpype/hosts/maya/plugins/publish/extract_layout.py @@ -23,7 +23,7 @@ class ExtractLayout(publish.Extractor): stagingdir = self.staging_dir(instance) # Perform extraction - self.log.info("Performing extraction..") + self.log.debug("Performing extraction..") if "representations" not in instance.data: instance.data["representations"] = [] @@ -64,7 +64,7 @@ class ExtractLayout(publish.Extractor): fields=["parent", "context.family"] ) - self.log.info(representation) + self.log.debug(representation) version_id = representation.get("parent") family = representation.get("context").get("family") @@ -159,5 +159,5 @@ class ExtractLayout(publish.Extractor): } instance.data["representations"].append(json_representation) - self.log.info("Extracted instance '%s' to: %s", - instance.name, json_representation) + self.log.debug("Extracted instance '%s' to: %s", + instance.name, json_representation) diff --git a/openpype/hosts/maya/plugins/publish/extract_look.py b/openpype/hosts/maya/plugins/publish/extract_look.py index b13568c781..74fcb58d29 100644 --- a/openpype/hosts/maya/plugins/publish/extract_look.py +++ b/openpype/hosts/maya/plugins/publish/extract_look.py @@ -6,7 +6,6 @@ import contextlib import json import logging import os -import platform import tempfile import six import attr @@ -25,6 +24,7 @@ from openpype.lib import ( from openpype.pipeline import legacy_io, publish, KnownPublishError from openpype.hosts.maya.api import lib +from openpype import AYON_SERVER_ENABLED # Modes for transfer COPY = 1 @@ -57,6 +57,12 @@ def find_paths_by_hash(texture_hash): str: path to texture if found. """ + if AYON_SERVER_ENABLED: + raise KnownPublishError( + "This is a bug. \"find_paths_by_hash\" is not compatible with " + "AYON." + ) + key = "data.sourceHashes.{0}".format(texture_hash) return legacy_io.distinct(key, {"type": "version"}) @@ -307,7 +313,7 @@ class MakeTX(TextureProcessor): render_colorspace = color_management["rendering_space"] - self.log.info("tx: converting colorspace {0} " + self.log.debug("tx: converting colorspace {0} " "-> {1}".format(colorspace, render_colorspace)) args.extend(["--colorconvert", colorspace, render_colorspace]) @@ -331,7 +337,7 @@ class MakeTX(TextureProcessor): if not os.path.exists(resources_dir): os.makedirs(resources_dir) - self.log.info("Generating .tx file for %s .." % source) + self.log.debug("Generating .tx file for %s .." % source) subprocess_args = maketx_args + [ "-v", # verbose @@ -421,7 +427,7 @@ class ExtractLook(publish.Extractor): for family in self.families: try: self.scene_type = ext_mapping[family] - self.log.info( + self.log.debug( "Using {} as scene type".format(self.scene_type)) break except KeyError: @@ -453,7 +459,7 @@ class ExtractLook(publish.Extractor): relationships = lookdata["relationships"] sets = list(relationships.keys()) if not sets: - self.log.info("No sets found for the look") + self.log.debug("No sets found for the look") return # Specify texture processing executables to activate @@ -485,7 +491,7 @@ class ExtractLook(publish.Extractor): remap = results["attrRemap"] # Extract in correct render layer - self.log.info("Extracting look maya scene file: {}".format(maya_path)) + self.log.debug("Extracting look maya scene file: {}".format(maya_path)) layer = instance.data.get("renderlayer", "defaultRenderLayer") with lib.renderlayer(layer): # TODO: Ensure membership edits don't become renderlayer overrides @@ -511,12 +517,12 @@ class ExtractLook(publish.Extractor): ) # Write the JSON data - self.log.info("Extract json..") data = { "attributes": lookdata["attributes"], "relationships": relationships } + self.log.debug("Extracting json file: {}".format(json_path)) with open(json_path, "w") as f: json.dump(data, f) @@ -557,8 +563,8 @@ class ExtractLook(publish.Extractor): # Source hash for the textures instance.data["sourceHashes"] = hashes - self.log.info("Extracted instance '%s' to: %s" % (instance.name, - maya_path)) + self.log.debug("Extracted instance '%s' to: %s" % (instance.name, + maya_path)) def _set_resource_result_colorspace(self, resource, colorspace): """Update resource resulting colorspace after texture processing""" @@ -589,14 +595,19 @@ class ExtractLook(publish.Extractor): resources = instance.data["resources"] color_management = lib.get_color_management_preferences() - # Temporary fix to NOT create hardlinks on windows machines - if platform.system().lower() == "windows": - self.log.info( + # TODO: Temporary disable all hardlinking, due to the feature not being + # used or properly working. + self.log.info( + "Forcing copy instead of hardlink." + ) + force_copy = True + + if not force_copy and platform.system().lower() == "windows": + # Temporary fix to NOT create hardlinks on windows machines + self.log.warning( "Forcing copy instead of hardlink due to issues on Windows..." ) force_copy = True - else: - force_copy = instance.data.get("forceCopy", False) destinations_cache = {} @@ -671,11 +682,11 @@ class ExtractLook(publish.Extractor): destination = get_resource_destination_cached(source) if force_copy or texture_result.transfer_mode == COPY: transfers.append((source, destination)) - self.log.info('file will be copied {} -> {}'.format( + self.log.debug('file will be copied {} -> {}'.format( source, destination)) elif texture_result.transfer_mode == HARDLINK: hardlinks.append((source, destination)) - self.log.info('file will be hardlinked {} -> {}'.format( + self.log.debug('file will be hardlinked {} -> {}'.format( source, destination)) # Store the hashes from hash to destination to include in the @@ -707,7 +718,7 @@ class ExtractLook(publish.Extractor): color_space_attr = "{}.colorSpace".format(node) remap[color_space_attr] = resource["result_color_space"] - self.log.info("Finished remapping destinations ...") + self.log.debug("Finished remapping destinations ...") return { "fileTransfers": transfers, @@ -815,8 +826,8 @@ class ExtractLook(publish.Extractor): if not processed_result: raise RuntimeError("Texture Processor {} returned " "no result.".format(processor)) - self.log.info("Generated processed " - "texture: {}".format(processed_result.path)) + self.log.debug("Generated processed " + "texture: {}".format(processed_result.path)) # TODO: Currently all processors force copy instead of allowing # hardlinks using source hashes. This should be refactored @@ -827,7 +838,7 @@ class ExtractLook(publish.Extractor): if not force_copy: existing = self._get_existing_hashed_texture(filepath) if existing: - self.log.info("Found hash in database, preparing hardlink..") + self.log.debug("Found hash in database, preparing hardlink..") return TextureResult( path=filepath, file_hash=texture_hash, diff --git a/openpype/hosts/maya/plugins/publish/extract_maya_scene_raw.py b/openpype/hosts/maya/plugins/publish/extract_maya_scene_raw.py index d87d6c208a..ab170fe48c 100644 --- a/openpype/hosts/maya/plugins/publish/extract_maya_scene_raw.py +++ b/openpype/hosts/maya/plugins/publish/extract_maya_scene_raw.py @@ -34,7 +34,7 @@ class ExtractMayaSceneRaw(publish.Extractor): for family in self.families: try: self.scene_type = ext_mapping[family] - self.log.info( + self.log.debug( "Using {} as scene type".format(self.scene_type)) break except KeyError: @@ -63,7 +63,7 @@ class ExtractMayaSceneRaw(publish.Extractor): selection += self._get_loaded_containers(members) # Perform extraction - self.log.info("Performing extraction ...") + self.log.debug("Performing extraction ...") with maintained_selection(): cmds.select(selection, noExpand=True) cmds.file(path, @@ -87,7 +87,8 @@ class ExtractMayaSceneRaw(publish.Extractor): } instance.data["representations"].append(representation) - self.log.info("Extracted instance '%s' to: %s" % (instance.name, path)) + self.log.debug("Extracted instance '%s' to: %s" % (instance.name, + path)) @staticmethod def _get_loaded_containers(members): diff --git a/openpype/hosts/maya/plugins/publish/extract_maya_usd.py b/openpype/hosts/maya/plugins/publish/extract_maya_usd.py new file mode 100644 index 0000000000..8c32ac1e39 --- /dev/null +++ b/openpype/hosts/maya/plugins/publish/extract_maya_usd.py @@ -0,0 +1,293 @@ +import os +import six +import json +import contextlib + +from maya import cmds + +import pyblish.api +from openpype.pipeline import publish +from openpype.hosts.maya.api.lib import maintained_selection + + +@contextlib.contextmanager +def usd_export_attributes(nodes, attrs=None, attr_prefixes=None, mapping=None): + """Define attributes for the given nodes that should be exported. + + MayaUSDExport will export custom attributes if the Maya node has a + string attribute `USD_UserExportedAttributesJson` that provides an + export mapping for the maya attributes. This context manager will try + to autogenerate such an attribute during the export to include attributes + for the export. + + Arguments: + nodes (List[str]): Nodes to process. + attrs (Optional[List[str]]): Full name of attributes to include. + attr_prefixes (Optional[List[str]]): Prefixes of attributes to include. + mapping (Optional[Dict[Dict]]): A mapping per attribute name for the + conversion to a USD attribute, including renaming, defining type, + converting attribute precision, etc. This match the usual + `USD_UserExportedAttributesJson` json mapping of `mayaUSDExport`. + When no mapping provided for an attribute it will use `{}` as + value. + + Examples: + >>> with usd_export_attributes( + >>> ["pCube1"], attrs="myDoubleAttributeAsFloat", mapping={ + >>> "myDoubleAttributeAsFloat": { + >>> "usdAttrName": "my:namespace:attrib", + >>> "translateMayaDoubleToUsdSinglePrecision": True, + >>> } + >>> }) + + """ + # todo: this might be better done with a custom export chaser + # see `chaser` argument for `mayaUSDExport` + + import maya.api.OpenMaya as om + + if not attrs and not attr_prefixes: + # context manager does nothing + yield + return + + if attrs is None: + attrs = [] + if attr_prefixes is None: + attr_prefixes = [] + if mapping is None: + mapping = {} + + usd_json_attr = "USD_UserExportedAttributesJson" + strings = attrs + ["{}*".format(prefix) for prefix in attr_prefixes] + context_state = {} + for node in set(nodes): + node_attrs = cmds.listAttr(node, st=strings) + if not node_attrs: + # Nothing to do for this node + continue + + node_attr_data = {} + for node_attr in set(node_attrs): + node_attr_data[node_attr] = mapping.get(node_attr, {}) + + if cmds.attributeQuery(usd_json_attr, node=node, exists=True): + existing_node_attr_value = cmds.getAttr( + "{}.{}".format(node, usd_json_attr) + ) + if existing_node_attr_value and existing_node_attr_value != "{}": + # Any existing attribute mappings in an existing + # `USD_UserExportedAttributesJson` attribute always take + # precedence over what this function tries to imprint + existing_node_attr_data = json.loads(existing_node_attr_value) + node_attr_data.update(existing_node_attr_data) + + context_state[node] = json.dumps(node_attr_data) + + sel = om.MSelectionList() + dg_mod = om.MDGModifier() + fn_string = om.MFnStringData() + fn_typed = om.MFnTypedAttribute() + try: + for node, value in context_state.items(): + data = fn_string.create(value) + sel.clear() + if cmds.attributeQuery(usd_json_attr, node=node, exists=True): + # Set the attribute value + sel.add("{}.{}".format(node, usd_json_attr)) + plug = sel.getPlug(0) + dg_mod.newPlugValue(plug, data) + else: + # Create attribute with the value as default value + sel.add(node) + node_obj = sel.getDependNode(0) + attr_obj = fn_typed.create(usd_json_attr, + usd_json_attr, + om.MFnData.kString, + data) + dg_mod.addAttribute(node_obj, attr_obj) + dg_mod.doIt() + yield + finally: + dg_mod.undoIt() + + +class ExtractMayaUsd(publish.Extractor): + """Extractor for Maya USD Asset data. + + Upon publish a .usd (or .usdz) asset file will typically be written. + """ + + label = "Extract Maya USD Asset" + hosts = ["maya"] + families = ["mayaUsd"] + + @property + def options(self): + """Overridable options for Maya USD Export + + Given in the following format + - {NAME: EXPECTED TYPE} + + If the overridden option's type does not match, + the option is not included and a warning is logged. + + """ + + # TODO: Support more `mayaUSDExport` parameters + return { + "defaultUSDFormat": str, + "stripNamespaces": bool, + "mergeTransformAndShape": bool, + "exportDisplayColor": bool, + "exportColorSets": bool, + "exportInstances": bool, + "exportUVs": bool, + "exportVisibility": bool, + "exportComponentTags": bool, + "exportRefsAsInstanceable": bool, + "eulerFilter": bool, + "renderableOnly": bool, + "jobContext": (list, None) # optional list + # "worldspace": bool, + } + + @property + def default_options(self): + """The default options for Maya USD Export.""" + + # TODO: Support more `mayaUSDExport` parameters + return { + "defaultUSDFormat": "usdc", + "stripNamespaces": False, + "mergeTransformAndShape": False, + "exportDisplayColor": False, + "exportColorSets": True, + "exportInstances": True, + "exportUVs": True, + "exportVisibility": True, + "exportComponentTags": True, + "exportRefsAsInstanceable": False, + "eulerFilter": True, + "renderableOnly": False, + "jobContext": None + # "worldspace": False + } + + def parse_overrides(self, instance, options): + """Inspect data of instance to determine overridden options""" + + for key in instance.data: + if key not in self.options: + continue + + # Ensure the data is of correct type + value = instance.data[key] + if isinstance(value, six.text_type): + value = str(value) + if not isinstance(value, self.options[key]): + self.log.warning( + "Overridden attribute {key} was of " + "the wrong type: {invalid_type} " + "- should have been {valid_type}".format( + key=key, + invalid_type=type(value).__name__, + valid_type=self.options[key].__name__)) + continue + + options[key] = value + + return options + + def filter_members(self, members): + # Can be overridden by inherited classes + return members + + def process(self, instance): + + # Load plugin first + cmds.loadPlugin("mayaUsdPlugin", quiet=True) + + # Define output file path + staging_dir = self.staging_dir(instance) + file_name = "{0}.usd".format(instance.name) + file_path = os.path.join(staging_dir, file_name) + file_path = file_path.replace('\\', '/') + + # Parse export options + options = self.default_options + options = self.parse_overrides(instance, options) + self.log.debug("Export options: {0}".format(options)) + + # Perform extraction + self.log.debug("Performing extraction ...") + + members = instance.data("setMembers") + self.log.debug('Collected objects: {}'.format(members)) + members = self.filter_members(members) + if not members: + self.log.error('No members!') + return + + start = instance.data["frameStartHandle"] + end = instance.data["frameEndHandle"] + + def parse_attr_str(attr_str): + result = list() + for attr in attr_str.split(","): + attr = attr.strip() + if not attr: + continue + result.append(attr) + return result + + attrs = parse_attr_str(instance.data.get("attr", "")) + attrs += instance.data.get("userDefinedAttributes", []) + attrs += ["cbId"] + attr_prefixes = parse_attr_str(instance.data.get("attrPrefix", "")) + + self.log.debug('Exporting USD: {} / {}'.format(file_path, members)) + with maintained_selection(): + with usd_export_attributes(instance[:], + attrs=attrs, + attr_prefixes=attr_prefixes): + cmds.mayaUSDExport(file=file_path, + frameRange=(start, end), + frameStride=instance.data.get("step", 1.0), + exportRoots=members, + **options) + + representation = { + 'name': "usd", + 'ext': "usd", + 'files': file_name, + 'stagingDir': staging_dir + } + instance.data.setdefault("representations", []).append(representation) + + self.log.debug( + "Extracted instance {} to {}".format(instance.name, file_path) + ) + + +class ExtractMayaUsdAnim(ExtractMayaUsd): + """Extractor for Maya USD Animation Sparse Cache data. + + This will extract the sparse cache data from the scene and generate a + USD file with all the animation data. + + Upon publish a .usd sparse cache will be written. + """ + label = "Extract Maya USD Animation Sparse Cache" + families = ["animation", "mayaUsd"] + match = pyblish.api.Subset + + def filter_members(self, members): + out_set = next((i for i in members if i.endswith("out_SET")), None) + + if out_set is None: + self.log.warning("Expecting out_SET") + return None + + members = cmds.ls(cmds.sets(out_set, query=True), long=True) + return members diff --git a/openpype/hosts/maya/plugins/publish/extract_model.py b/openpype/hosts/maya/plugins/publish/extract_model.py index 5137dffd94..29c952ebbc 100644 --- a/openpype/hosts/maya/plugins/publish/extract_model.py +++ b/openpype/hosts/maya/plugins/publish/extract_model.py @@ -44,7 +44,7 @@ class ExtractModel(publish.Extractor, for family in self.families: try: self.scene_type = ext_mapping[family] - self.log.info( + self.log.debug( "Using {} as scene type".format(self.scene_type)) break except KeyError: @@ -56,7 +56,7 @@ class ExtractModel(publish.Extractor, path = os.path.join(stagingdir, filename) # Perform extraction - self.log.info("Performing extraction ...") + self.log.debug("Performing extraction ...") # Get only the shape contents we need in such a way that we avoid # taking along intermediateObjects @@ -102,4 +102,5 @@ class ExtractModel(publish.Extractor, } instance.data["representations"].append(representation) - self.log.info("Extracted instance '%s' to: %s" % (instance.name, path)) + self.log.debug("Extracted instance '%s' to: %s" % (instance.name, + path)) diff --git a/openpype/hosts/maya/plugins/publish/extract_multiverse_look.py b/openpype/hosts/maya/plugins/publish/extract_multiverse_look.py index 6fe7cf0d55..c2bebeaee6 100644 --- a/openpype/hosts/maya/plugins/publish/extract_multiverse_look.py +++ b/openpype/hosts/maya/plugins/publish/extract_multiverse_look.py @@ -101,10 +101,10 @@ class ExtractMultiverseLook(publish.Extractor): # Parse export options options = self.default_options - self.log.info("Export options: {0}".format(options)) + self.log.debug("Export options: {0}".format(options)) # Perform extraction - self.log.info("Performing extraction ...") + self.log.debug("Performing extraction ...") with maintained_selection(): members = instance.data("setMembers") @@ -114,7 +114,7 @@ class ExtractMultiverseLook(publish.Extractor): type="mvUsdCompoundShape", noIntermediate=True, long=True) - self.log.info('Collected object {}'.format(members)) + self.log.debug('Collected object {}'.format(members)) if len(members) > 1: self.log.error('More than one member: {}'.format(members)) @@ -153,5 +153,5 @@ class ExtractMultiverseLook(publish.Extractor): } instance.data["representations"].append(representation) - self.log.info("Extracted instance {} to {}".format( + self.log.debug("Extracted instance {} to {}".format( instance.name, file_path)) diff --git a/openpype/hosts/maya/plugins/publish/extract_multiverse_usd.py b/openpype/hosts/maya/plugins/publish/extract_multiverse_usd.py index 4399eacda1..60185bb152 100644 --- a/openpype/hosts/maya/plugins/publish/extract_multiverse_usd.py +++ b/openpype/hosts/maya/plugins/publish/extract_multiverse_usd.py @@ -28,7 +28,7 @@ class ExtractMultiverseUsd(publish.Extractor): label = "Extract Multiverse USD Asset" hosts = ["maya"] - families = ["usd"] + families = ["mvUsd"] scene_type = "usd" file_formats = ["usd", "usda", "usdz"] @@ -150,7 +150,6 @@ class ExtractMultiverseUsd(publish.Extractor): return options def get_default_options(self): - self.log.info("ExtractMultiverseUsd get_default_options") return self.default_options def filter_members(self, members): @@ -173,19 +172,19 @@ class ExtractMultiverseUsd(publish.Extractor): # Parse export options options = self.get_default_options() options = self.parse_overrides(instance, options) - self.log.info("Export options: {0}".format(options)) + self.log.debug("Export options: {0}".format(options)) # Perform extraction - self.log.info("Performing extraction ...") + self.log.debug("Performing extraction ...") with maintained_selection(): members = instance.data("setMembers") - self.log.info('Collected objects: {}'.format(members)) + self.log.debug('Collected objects: {}'.format(members)) members = self.filter_members(members) if not members: self.log.error('No members!') return - self.log.info(' - filtered: {}'.format(members)) + self.log.debug(' - filtered: {}'.format(members)) import multiverse @@ -229,7 +228,7 @@ class ExtractMultiverseUsd(publish.Extractor): self.log.debug(" - {}={}".format(key, value)) setattr(asset_write_opts, key, value) - self.log.info('WriteAsset: {} / {}'.format(file_path, members)) + self.log.debug('WriteAsset: {} / {}'.format(file_path, members)) multiverse.WriteAsset(file_path, members, asset_write_opts) if "representations" not in instance.data: @@ -243,7 +242,7 @@ class ExtractMultiverseUsd(publish.Extractor): } instance.data["representations"].append(representation) - self.log.info("Extracted instance {} to {}".format( + self.log.debug("Extracted instance {} to {}".format( instance.name, file_path)) diff --git a/openpype/hosts/maya/plugins/publish/extract_multiverse_usd_comp.py b/openpype/hosts/maya/plugins/publish/extract_multiverse_usd_comp.py index a62729c198..7966c4fa93 100644 --- a/openpype/hosts/maya/plugins/publish/extract_multiverse_usd_comp.py +++ b/openpype/hosts/maya/plugins/publish/extract_multiverse_usd_comp.py @@ -105,14 +105,14 @@ class ExtractMultiverseUsdComposition(publish.Extractor): # Parse export options options = self.default_options options = self.parse_overrides(instance, options) - self.log.info("Export options: {0}".format(options)) + self.log.debug("Export options: {0}".format(options)) # Perform extraction - self.log.info("Performing extraction ...") + self.log.debug("Performing extraction ...") with maintained_selection(): members = instance.data("setMembers") - self.log.info('Collected object {}'.format(members)) + self.log.debug('Collected object {}'.format(members)) import multiverse @@ -175,5 +175,5 @@ class ExtractMultiverseUsdComposition(publish.Extractor): } instance.data["representations"].append(representation) - self.log.info("Extracted instance {} to {}".format( - instance.name, file_path)) + self.log.debug("Extracted instance {} to {}".format(instance.name, + file_path)) diff --git a/openpype/hosts/maya/plugins/publish/extract_multiverse_usd_over.py b/openpype/hosts/maya/plugins/publish/extract_multiverse_usd_over.py index cf610ac6b4..e4a97db6e4 100644 --- a/openpype/hosts/maya/plugins/publish/extract_multiverse_usd_over.py +++ b/openpype/hosts/maya/plugins/publish/extract_multiverse_usd_over.py @@ -87,10 +87,10 @@ class ExtractMultiverseUsdOverride(publish.Extractor): # Parse export options options = self.default_options - self.log.info("Export options: {0}".format(options)) + self.log.debug("Export options: {0}".format(options)) # Perform extraction - self.log.info("Performing extraction ...") + self.log.debug("Performing extraction ...") with maintained_selection(): members = instance.data("setMembers") @@ -100,7 +100,7 @@ class ExtractMultiverseUsdOverride(publish.Extractor): type="mvUsdCompoundShape", noIntermediate=True, long=True) - self.log.info("Collected object {}".format(members)) + self.log.debug("Collected object {}".format(members)) # TODO: Deal with asset, composition, override with options. import multiverse @@ -153,5 +153,5 @@ class ExtractMultiverseUsdOverride(publish.Extractor): } instance.data["representations"].append(representation) - self.log.info("Extracted instance {} to {}".format( + self.log.debug("Extracted instance {} to {}".format( instance.name, file_path)) diff --git a/openpype/hosts/maya/plugins/publish/extract_obj.py b/openpype/hosts/maya/plugins/publish/extract_obj.py index 518b0f0ff8..ca94130d09 100644 --- a/openpype/hosts/maya/plugins/publish/extract_obj.py +++ b/openpype/hosts/maya/plugins/publish/extract_obj.py @@ -30,7 +30,7 @@ class ExtractObj(publish.Extractor): # The export requires forward slashes because we need to # format it into a string in a mel expression - self.log.info("Extracting OBJ to: {0}".format(path)) + self.log.debug("Extracting OBJ to: {0}".format(path)) members = instance.data("setMembers") members = cmds.ls(members, @@ -39,8 +39,8 @@ class ExtractObj(publish.Extractor): type=("mesh", "nurbsCurve"), noIntermediate=True, long=True) - self.log.info("Members: {0}".format(members)) - self.log.info("Instance: {0}".format(instance[:])) + self.log.debug("Members: {0}".format(members)) + self.log.debug("Instance: {0}".format(instance[:])) if not cmds.pluginInfo('objExport', query=True, loaded=True): cmds.loadPlugin('objExport') @@ -74,4 +74,4 @@ class ExtractObj(publish.Extractor): } instance.data["representations"].append(representation) - self.log.info("Extract OBJ successful to: {0}".format(path)) + self.log.debug("Extract OBJ successful to: {0}".format(path)) diff --git a/openpype/hosts/maya/plugins/publish/extract_playblast.py b/openpype/hosts/maya/plugins/publish/extract_playblast.py index 9580c13841..cfab239da3 100644 --- a/openpype/hosts/maya/plugins/publish/extract_playblast.py +++ b/openpype/hosts/maya/plugins/publish/extract_playblast.py @@ -48,7 +48,7 @@ class ExtractPlayblast(publish.Extractor): self.log.debug("playblast path {}".format(path)) def process(self, instance): - self.log.info("Extracting capture..") + self.log.debug("Extracting capture..") # get scene fps fps = instance.data.get("fps") or instance.context.data.get("fps") @@ -62,7 +62,7 @@ class ExtractPlayblast(publish.Extractor): if end is None: end = cmds.playbackOptions(query=True, animationEndTime=True) - self.log.info("start: {}, end: {}".format(start, end)) + self.log.debug("start: {}, end: {}".format(start, end)) # get cameras camera = instance.data["review_camera"] @@ -119,7 +119,7 @@ class ExtractPlayblast(publish.Extractor): filename = "{0}".format(instance.name) path = os.path.join(stagingdir, filename) - self.log.info("Outputting images to %s" % path) + self.log.debug("Outputting images to %s" % path) preset["filename"] = path preset["overwrite"] = True @@ -237,7 +237,7 @@ class ExtractPlayblast(publish.Extractor): self.log.debug("collection head {}".format(filebase)) if filebase in filename: frame_collection = collection - self.log.info( + self.log.debug( "we found collection of interest {}".format( str(frame_collection))) diff --git a/openpype/hosts/maya/plugins/publish/extract_pointcache.py b/openpype/hosts/maya/plugins/publish/extract_pointcache.py index 9537a11ee4..0cc802fa7a 100644 --- a/openpype/hosts/maya/plugins/publish/extract_pointcache.py +++ b/openpype/hosts/maya/plugins/publish/extract_pointcache.py @@ -107,13 +107,14 @@ class ExtractAlembic(publish.Extractor): } instance.data["representations"].append(representation) - instance.context.data["cleanupFullPaths"].append(path) + if not instance.data.get("stagingDir_persistent", False): + instance.context.data["cleanupFullPaths"].append(path) - self.log.info("Extracted {} to {}".format(instance, dirname)) + self.log.debug("Extracted {} to {}".format(instance, dirname)) # Extract proxy. if not instance.data.get("proxy"): - self.log.info("No proxy nodes found. Skipping proxy extraction.") + self.log.debug("No proxy nodes found. Skipping proxy extraction.") return path = path.replace(".abc", "_proxy.abc") diff --git a/openpype/hosts/maya/plugins/publish/extract_proxy_abc.py b/openpype/hosts/maya/plugins/publish/extract_proxy_abc.py index cf6351fdca..d9bec87cfd 100644 --- a/openpype/hosts/maya/plugins/publish/extract_proxy_abc.py +++ b/openpype/hosts/maya/plugins/publish/extract_proxy_abc.py @@ -32,7 +32,7 @@ class ExtractProxyAlembic(publish.Extractor): attr_prefixes = instance.data.get("attrPrefix", "").split(";") attr_prefixes = [value for value in attr_prefixes if value.strip()] - self.log.info("Extracting Proxy Alembic..") + self.log.debug("Extracting Proxy Alembic..") dirname = self.staging_dir(instance) filename = "{name}.abc".format(**instance.data) @@ -80,9 +80,10 @@ class ExtractProxyAlembic(publish.Extractor): } instance.data["representations"].append(representation) - instance.context.data["cleanupFullPaths"].append(path) + if not instance.data.get("stagingDir_persistent", False): + instance.context.data["cleanupFullPaths"].append(path) - self.log.info("Extracted {} to {}".format(instance, dirname)) + self.log.debug("Extracted {} to {}".format(instance, dirname)) # remove the bounding box bbox_master = cmds.ls("bbox_grp") cmds.delete(bbox_master) diff --git a/openpype/hosts/maya/plugins/publish/extract_redshift_proxy.py b/openpype/hosts/maya/plugins/publish/extract_redshift_proxy.py index 834b335fc5..3868270b79 100644 --- a/openpype/hosts/maya/plugins/publish/extract_redshift_proxy.py +++ b/openpype/hosts/maya/plugins/publish/extract_redshift_proxy.py @@ -59,7 +59,7 @@ class ExtractRedshiftProxy(publish.Extractor): # vertex_colors = instance.data.get("vertexColors", False) # Write out rs file - self.log.info("Writing: '%s'" % file_path) + self.log.debug("Writing: '%s'" % file_path) with maintained_selection(): cmds.select(instance.data["setMembers"], noExpand=True) cmds.file(file_path, @@ -82,5 +82,5 @@ class ExtractRedshiftProxy(publish.Extractor): } instance.data["representations"].append(representation) - self.log.info("Extracted instance '%s' to: %s" - % (instance.name, staging_dir)) + self.log.debug("Extracted instance '%s' to: %s" + % (instance.name, staging_dir)) diff --git a/openpype/hosts/maya/plugins/publish/extract_rendersetup.py b/openpype/hosts/maya/plugins/publish/extract_rendersetup.py index 5970c038a4..7e21f5282e 100644 --- a/openpype/hosts/maya/plugins/publish/extract_rendersetup.py +++ b/openpype/hosts/maya/plugins/publish/extract_rendersetup.py @@ -37,5 +37,5 @@ class ExtractRenderSetup(publish.Extractor): } instance.data["representations"].append(representation) - self.log.info( + self.log.debug( "Extracted instance '%s' to: %s" % (instance.name, json_path)) diff --git a/openpype/hosts/maya/plugins/publish/extract_rig.py b/openpype/hosts/maya/plugins/publish/extract_rig.py index be57b9de07..1ffc9a7dae 100644 --- a/openpype/hosts/maya/plugins/publish/extract_rig.py +++ b/openpype/hosts/maya/plugins/publish/extract_rig.py @@ -27,7 +27,7 @@ class ExtractRig(publish.Extractor): for family in self.families: try: self.scene_type = ext_mapping[family] - self.log.info( + self.log.debug( "Using '.{}' as scene type".format(self.scene_type)) break except AttributeError: @@ -39,7 +39,7 @@ class ExtractRig(publish.Extractor): path = os.path.join(dir_path, filename) # Perform extraction - self.log.info("Performing extraction ...") + self.log.debug("Performing extraction ...") with maintained_selection(): cmds.select(instance, noExpand=True) cmds.file(path, @@ -63,4 +63,4 @@ class ExtractRig(publish.Extractor): } instance.data["representations"].append(representation) - self.log.info("Extracted instance '%s' to: %s" % (instance.name, path)) + self.log.debug("Extracted instance '%s' to: %s", instance.name, path) diff --git a/openpype/hosts/maya/plugins/publish/extract_thumbnail.py b/openpype/hosts/maya/plugins/publish/extract_thumbnail.py index 4160ac4cb2..c0be3d77db 100644 --- a/openpype/hosts/maya/plugins/publish/extract_thumbnail.py +++ b/openpype/hosts/maya/plugins/publish/extract_thumbnail.py @@ -24,7 +24,7 @@ class ExtractThumbnail(publish.Extractor): families = ["review"] def process(self, instance): - self.log.info("Extracting capture..") + self.log.debug("Extracting capture..") camera = instance.data["review_camera"] @@ -92,11 +92,10 @@ class ExtractThumbnail(publish.Extractor): "Create temp directory {} for thumbnail".format(dst_staging) ) # Store new staging to cleanup paths - instance.context.data["cleanupFullPaths"].append(dst_staging) filename = "{0}".format(instance.name) path = os.path.join(dst_staging, filename) - self.log.info("Outputting images to %s" % path) + self.log.debug("Outputting images to %s" % path) preset["filename"] = path preset["overwrite"] = True @@ -159,7 +158,7 @@ class ExtractThumbnail(publish.Extractor): _, thumbnail = os.path.split(playblast) - self.log.info("file list {}".format(thumbnail)) + self.log.debug("file list {}".format(thumbnail)) if "representations" not in instance.data: instance.data["representations"] = [] diff --git a/openpype/hosts/maya/plugins/publish/extract_unreal_skeletalmesh_abc.py b/openpype/hosts/maya/plugins/publish/extract_unreal_skeletalmesh_abc.py index 4a797eb462..9c2f55a1ef 100644 --- a/openpype/hosts/maya/plugins/publish/extract_unreal_skeletalmesh_abc.py +++ b/openpype/hosts/maya/plugins/publish/extract_unreal_skeletalmesh_abc.py @@ -57,9 +57,9 @@ class ExtractUnrealSkeletalMeshAbc(publish.Extractor): # to format it into a string in a mel expression path = path.replace('\\', '/') - self.log.info("Extracting ABC to: {0}".format(path)) - self.log.info("Members: {0}".format(nodes)) - self.log.info("Instance: {0}".format(instance[:])) + self.log.debug("Extracting ABC to: {0}".format(path)) + self.log.debug("Members: {0}".format(nodes)) + self.log.debug("Instance: {0}".format(instance[:])) options = { "step": instance.data.get("step", 1.0), @@ -74,7 +74,7 @@ class ExtractUnrealSkeletalMeshAbc(publish.Extractor): "worldSpace": instance.data.get("worldSpace", True) } - self.log.info("Options: {}".format(options)) + self.log.debug("Options: {}".format(options)) if int(cmds.about(version=True)) >= 2017: # Since Maya 2017 alembic supports multiple uv sets - write them. @@ -105,4 +105,4 @@ class ExtractUnrealSkeletalMeshAbc(publish.Extractor): } instance.data["representations"].append(representation) - self.log.info("Extract ABC successful to: {0}".format(path)) + self.log.debug("Extract ABC successful to: {0}".format(path)) diff --git a/openpype/hosts/maya/plugins/publish/extract_unreal_skeletalmesh_fbx.py b/openpype/hosts/maya/plugins/publish/extract_unreal_skeletalmesh_fbx.py index b162ce47f7..96175a07d7 100644 --- a/openpype/hosts/maya/plugins/publish/extract_unreal_skeletalmesh_fbx.py +++ b/openpype/hosts/maya/plugins/publish/extract_unreal_skeletalmesh_fbx.py @@ -46,9 +46,9 @@ class ExtractUnrealSkeletalMeshFbx(publish.Extractor): # to format it into a string in a mel expression path = path.replace('\\', '/') - self.log.info("Extracting FBX to: {0}".format(path)) - self.log.info("Members: {0}".format(to_extract)) - self.log.info("Instance: {0}".format(instance[:])) + self.log.debug("Extracting FBX to: {0}".format(path)) + self.log.debug("Members: {0}".format(to_extract)) + self.log.debug("Instance: {0}".format(instance[:])) fbx_exporter.set_options_from_instance(instance) @@ -70,7 +70,7 @@ class ExtractUnrealSkeletalMeshFbx(publish.Extractor): renamed_to_extract.append("|".join(node_path)) with renamed(original_parent, parent_node): - self.log.info("Extracting: {}".format(renamed_to_extract, path)) + self.log.debug("Extracting: {}".format(renamed_to_extract, path)) fbx_exporter.export(renamed_to_extract, path) if "representations" not in instance.data: @@ -84,4 +84,4 @@ class ExtractUnrealSkeletalMeshFbx(publish.Extractor): } instance.data["representations"].append(representation) - self.log.info("Extract FBX successful to: {0}".format(path)) + self.log.debug("Extract FBX successful to: {0}".format(path)) diff --git a/openpype/hosts/maya/plugins/publish/extract_unreal_staticmesh.py b/openpype/hosts/maya/plugins/publish/extract_unreal_staticmesh.py index 44f0615a27..26ab0827e4 100644 --- a/openpype/hosts/maya/plugins/publish/extract_unreal_staticmesh.py +++ b/openpype/hosts/maya/plugins/publish/extract_unreal_staticmesh.py @@ -37,15 +37,15 @@ class ExtractUnrealStaticMesh(publish.Extractor): # to format it into a string in a mel expression path = path.replace('\\', '/') - self.log.info("Extracting FBX to: {0}".format(path)) - self.log.info("Members: {0}".format(members)) - self.log.info("Instance: {0}".format(instance[:])) + self.log.debug("Extracting FBX to: {0}".format(path)) + self.log.debug("Members: {0}".format(members)) + self.log.debug("Instance: {0}".format(instance[:])) fbx_exporter.set_options_from_instance(instance) with maintained_selection(): with parent_nodes(members): - self.log.info("Un-parenting: {}".format(members)) + self.log.debug("Un-parenting: {}".format(members)) fbx_exporter.export(members, path) if "representations" not in instance.data: @@ -59,4 +59,4 @@ class ExtractUnrealStaticMesh(publish.Extractor): } instance.data["representations"].append(representation) - self.log.info("Extract FBX successful to: {0}".format(path)) + self.log.debug("Extract FBX successful to: {0}".format(path)) diff --git a/openpype/hosts/maya/plugins/publish/extract_vrayproxy.py b/openpype/hosts/maya/plugins/publish/extract_vrayproxy.py index df16c6c357..21dfcfffc5 100644 --- a/openpype/hosts/maya/plugins/publish/extract_vrayproxy.py +++ b/openpype/hosts/maya/plugins/publish/extract_vrayproxy.py @@ -43,7 +43,7 @@ class ExtractVRayProxy(publish.Extractor): vertex_colors = instance.data.get("vertexColors", False) # Write out vrmesh file - self.log.info("Writing: '%s'" % file_path) + self.log.debug("Writing: '%s'" % file_path) with maintained_selection(): cmds.select(instance.data["setMembers"], noExpand=True) cmds.vrayCreateProxy(exportType=1, @@ -68,5 +68,5 @@ class ExtractVRayProxy(publish.Extractor): } instance.data["representations"].append(representation) - self.log.info("Extracted instance '%s' to: %s" - % (instance.name, staging_dir)) + self.log.debug("Extracted instance '%s' to: %s" + % (instance.name, staging_dir)) diff --git a/openpype/hosts/maya/plugins/publish/extract_vrayscene.py b/openpype/hosts/maya/plugins/publish/extract_vrayscene.py index 8442df1611..b0615149a9 100644 --- a/openpype/hosts/maya/plugins/publish/extract_vrayscene.py +++ b/openpype/hosts/maya/plugins/publish/extract_vrayscene.py @@ -20,13 +20,13 @@ class ExtractVrayscene(publish.Extractor): def process(self, instance): """Plugin entry point.""" if instance.data.get("exportOnFarm"): - self.log.info("vrayscenes will be exported on farm.") + self.log.debug("vrayscenes will be exported on farm.") raise NotImplementedError( "exporting vrayscenes is not implemented") # handle sequence if instance.data.get("vraySceneMultipleFiles"): - self.log.info("vrayscenes will be exported on farm.") + self.log.debug("vrayscenes will be exported on farm.") raise NotImplementedError( "exporting vrayscene sequences not implemented yet") @@ -40,7 +40,6 @@ class ExtractVrayscene(publish.Extractor): layer_name = instance.data.get("layer") staging_dir = self.staging_dir(instance) - self.log.info("staging: {}".format(staging_dir)) template = cmds.getAttr("{}.vrscene_filename".format(node)) start_frame = instance.data.get( "frameStartHandle") if instance.data.get( @@ -56,21 +55,21 @@ class ExtractVrayscene(publish.Extractor): staging_dir, "vrayscene", *formatted_name.split("/")) # Write out vrscene file - self.log.info("Writing: '%s'" % file_path) + self.log.debug("Writing: '%s'" % file_path) with maintained_selection(): if "*" not in instance.data["setMembers"]: - self.log.info( + self.log.debug( "Exporting: {}".format(instance.data["setMembers"])) set_members = instance.data["setMembers"] cmds.select(set_members, noExpand=True) else: - self.log.info("Exporting all ...") + self.log.debug("Exporting all ...") set_members = cmds.ls( long=True, objectsOnly=True, geometry=True, lights=True, cameras=True) cmds.select(set_members, noExpand=True) - self.log.info("Appending layer name {}".format(layer_name)) + self.log.debug("Appending layer name {}".format(layer_name)) set_members.append(layer_name) export_in_rs_layer( @@ -93,8 +92,8 @@ class ExtractVrayscene(publish.Extractor): } instance.data["representations"].append(representation) - self.log.info("Extracted instance '%s' to: %s" - % (instance.name, staging_dir)) + self.log.debug("Extracted instance '%s' to: %s" + % (instance.name, staging_dir)) @staticmethod def format_vray_output_filename( diff --git a/openpype/hosts/maya/plugins/publish/extract_workfile_xgen.py b/openpype/hosts/maya/plugins/publish/extract_workfile_xgen.py index 0d2a97bc4b..4bd01c2df2 100644 --- a/openpype/hosts/maya/plugins/publish/extract_workfile_xgen.py +++ b/openpype/hosts/maya/plugins/publish/extract_workfile_xgen.py @@ -241,7 +241,7 @@ class ExtractWorkfileXgen(publish.Extractor): data[palette] = {attr: old_value} cmds.setAttr(node_attr, value, type="string") - self.log.info( + self.log.debug( "Setting \"{}\" on \"{}\"".format(value, node_attr) ) diff --git a/openpype/hosts/maya/plugins/publish/extract_xgen.py b/openpype/hosts/maya/plugins/publish/extract_xgen.py index 3c9d0bd344..8409330e49 100644 --- a/openpype/hosts/maya/plugins/publish/extract_xgen.py +++ b/openpype/hosts/maya/plugins/publish/extract_xgen.py @@ -77,7 +77,7 @@ class ExtractXgen(publish.Extractor): xgenm.exportPalette( instance.data["xgmPalette"].replace("|", ""), temp_xgen_path ) - self.log.info("Extracted to {}".format(temp_xgen_path)) + self.log.debug("Extracted to {}".format(temp_xgen_path)) # Import xgen onto the duplicate. with maintained_selection(): @@ -118,7 +118,7 @@ class ExtractXgen(publish.Extractor): expressions=True ) - self.log.info("Extracted to {}".format(maya_filepath)) + self.log.debug("Extracted to {}".format(maya_filepath)) if os.path.exists(temp_xgen_path): os.remove(temp_xgen_path) diff --git a/openpype/hosts/maya/plugins/publish/extract_yeti_cache.py b/openpype/hosts/maya/plugins/publish/extract_yeti_cache.py index b61f599cab..b113e02219 100644 --- a/openpype/hosts/maya/plugins/publish/extract_yeti_cache.py +++ b/openpype/hosts/maya/plugins/publish/extract_yeti_cache.py @@ -39,7 +39,7 @@ class ExtractYetiCache(publish.Extractor): else: kwargs.update({"samples": samples}) - self.log.info( + self.log.debug( "Writing out cache {} - {}".format(start_frame, end_frame)) # Start writing the files for snap shot # will be replace by the Yeti node name @@ -53,7 +53,7 @@ class ExtractYetiCache(publish.Extractor): cache_files = [x for x in os.listdir(dirname) if x.endswith(".fur")] - self.log.info("Writing metadata file") + self.log.debug("Writing metadata file") settings = instance.data["fursettings"] fursettings_path = os.path.join(dirname, "yeti.fursettings") with open(fursettings_path, "w") as fp: @@ -63,7 +63,7 @@ class ExtractYetiCache(publish.Extractor): if "representations" not in instance.data: instance.data["representations"] = [] - self.log.info("cache files: {}".format(cache_files[0])) + self.log.debug("cache files: {}".format(cache_files[0])) # Workaround: We do not explicitly register these files with the # representation solely so that we can write multiple sequences @@ -87,4 +87,4 @@ class ExtractYetiCache(publish.Extractor): } ) - self.log.info("Extracted {} to {}".format(instance, dirname)) + self.log.debug("Extracted {} to {}".format(instance, dirname)) diff --git a/openpype/hosts/maya/plugins/publish/extract_yeti_rig.py b/openpype/hosts/maya/plugins/publish/extract_yeti_rig.py index 9a46c31177..da67cb911f 100644 --- a/openpype/hosts/maya/plugins/publish/extract_yeti_rig.py +++ b/openpype/hosts/maya/plugins/publish/extract_yeti_rig.py @@ -109,7 +109,7 @@ class ExtractYetiRig(publish.Extractor): for family in self.families: try: self.scene_type = ext_mapping[family] - self.log.info( + self.log.debug( "Using {} as scene type".format(self.scene_type)) break except KeyError: @@ -127,7 +127,7 @@ class ExtractYetiRig(publish.Extractor): maya_path = os.path.join(dirname, "yeti_rig.{}".format(self.scene_type)) - self.log.info("Writing metadata file") + self.log.debug("Writing metadata file: {}".format(settings_path)) image_search_path = resources_dir = instance.data["resourcesDir"] @@ -147,7 +147,7 @@ class ExtractYetiRig(publish.Extractor): dst = os.path.join(image_search_path, os.path.basename(file)) instance.data['transfers'].append([src, dst]) - self.log.info("adding transfer {} -> {}". format(src, dst)) + self.log.debug("adding transfer {} -> {}". format(src, dst)) # Ensure the imageSearchPath is being remapped to the publish folder attr_value = {"%s.imageSearchPath" % n: str(image_search_path) for @@ -182,7 +182,7 @@ class ExtractYetiRig(publish.Extractor): if "representations" not in instance.data: instance.data["representations"] = [] - self.log.info("rig file: {}".format(maya_path)) + self.log.debug("rig file: {}".format(maya_path)) instance.data["representations"].append( { 'name': self.scene_type, @@ -191,7 +191,7 @@ class ExtractYetiRig(publish.Extractor): 'stagingDir': dirname } ) - self.log.info("settings file: {}".format(settings_path)) + self.log.debug("settings file: {}".format(settings_path)) instance.data["representations"].append( { 'name': 'rigsettings', @@ -201,6 +201,6 @@ class ExtractYetiRig(publish.Extractor): } ) - self.log.info("Extracted {} to {}".format(instance, dirname)) + self.log.debug("Extracted {} to {}".format(instance, dirname)) cmds.select(clear=True) diff --git a/openpype/hosts/maya/plugins/publish/reset_xgen_attributes.py b/openpype/hosts/maya/plugins/publish/reset_xgen_attributes.py index d8e8554b68..759aa23258 100644 --- a/openpype/hosts/maya/plugins/publish/reset_xgen_attributes.py +++ b/openpype/hosts/maya/plugins/publish/reset_xgen_attributes.py @@ -23,7 +23,7 @@ class ResetXgenAttributes(pyblish.api.InstancePlugin): for palette, data in xgen_attributes.items(): for attr, value in data.items(): node_attr = "{}.{}".format(palette, attr) - self.log.info( + self.log.debug( "Setting \"{}\" on \"{}\"".format(value, node_attr) ) cmds.setAttr(node_attr, value, type="string") @@ -32,5 +32,5 @@ class ResetXgenAttributes(pyblish.api.InstancePlugin): # Need to save the scene, cause the attribute changes above does not # mark the scene as modified so user can exit without committing the # changes. - self.log.info("Saving changes.") + self.log.debug("Saving changes.") cmds.file(save=True) diff --git a/openpype/hosts/maya/plugins/publish/submit_maya_muster.py b/openpype/hosts/maya/plugins/publish/submit_maya_muster.py index 8e219eae85..c174fa7a33 100644 --- a/openpype/hosts/maya/plugins/publish/submit_maya_muster.py +++ b/openpype/hosts/maya/plugins/publish/submit_maya_muster.py @@ -215,9 +215,9 @@ class MayaSubmitMuster(pyblish.api.InstancePlugin): :rtype: int :raises: Exception if template ID isn't found """ - self.log.info("Trying to find template for [{}]".format(renderer)) + self.log.debug("Trying to find template for [{}]".format(renderer)) mapped = _get_template_id(renderer) - self.log.info("got id [{}]".format(mapped)) + self.log.debug("got id [{}]".format(mapped)) return self._templates.get(mapped) def _submit(self, payload): @@ -249,7 +249,6 @@ class MayaSubmitMuster(pyblish.api.InstancePlugin): Authenticate with Muster, collect all data, prepare path for post render publish job and submit job to farm. """ - instance.data["toBeRenderedOn"] = "muster" # setup muster environment self.MUSTER_REST_URL = os.environ.get("MUSTER_REST_URL") @@ -454,8 +453,8 @@ class MayaSubmitMuster(pyblish.api.InstancePlugin): self.preflight_check(instance) - self.log.info("Submitting ...") - self.log.info(json.dumps(payload, indent=4, sort_keys=True)) + self.log.debug("Submitting ...") + self.log.debug(json.dumps(payload, indent=4, sort_keys=True)) response = self._submit(payload) # response = requests.post(url, json=payload) diff --git a/openpype/hosts/maya/plugins/publish/validate_assembly_name.py b/openpype/hosts/maya/plugins/publish/validate_assembly_name.py index bcc40760e0..00588cd300 100644 --- a/openpype/hosts/maya/plugins/publish/validate_assembly_name.py +++ b/openpype/hosts/maya/plugins/publish/validate_assembly_name.py @@ -20,7 +20,7 @@ class ValidateAssemblyName(pyblish.api.InstancePlugin): @classmethod def get_invalid(cls, instance): - cls.log.info("Checking name of {}".format(instance.name)) + cls.log.debug("Checking name of {}".format(instance.name)) content_instance = instance.data.get("setMembers", None) if not content_instance: diff --git a/openpype/hosts/maya/plugins/publish/validate_assembly_namespaces.py b/openpype/hosts/maya/plugins/publish/validate_assembly_namespaces.py index 41ef78aab4..06577f38f7 100644 --- a/openpype/hosts/maya/plugins/publish/validate_assembly_namespaces.py +++ b/openpype/hosts/maya/plugins/publish/validate_assembly_namespaces.py @@ -23,7 +23,7 @@ class ValidateAssemblyNamespaces(pyblish.api.InstancePlugin): def process(self, instance): - self.log.info("Checking namespace for %s" % instance.name) + self.log.debug("Checking namespace for %s" % instance.name) if self.get_invalid(instance): raise PublishValidationError("Nested namespaces found") diff --git a/openpype/hosts/maya/plugins/publish/validate_frame_range.py b/openpype/hosts/maya/plugins/publish/validate_frame_range.py index c6184ed348..a7043b8407 100644 --- a/openpype/hosts/maya/plugins/publish/validate_frame_range.py +++ b/openpype/hosts/maya/plugins/publish/validate_frame_range.py @@ -47,10 +47,10 @@ class ValidateFrameRange(pyblish.api.InstancePlugin, context = instance.context if instance.data.get("tileRendering"): - self.log.info(( + self.log.debug( "Skipping frame range validation because " "tile rendering is enabled." - )) + ) return frame_start_handle = int(context.data.get("frameStartHandle")) diff --git a/openpype/hosts/maya/plugins/publish/validate_glsl_material.py b/openpype/hosts/maya/plugins/publish/validate_glsl_material.py index 10c48da404..3b386c3def 100644 --- a/openpype/hosts/maya/plugins/publish/validate_glsl_material.py +++ b/openpype/hosts/maya/plugins/publish/validate_glsl_material.py @@ -75,7 +75,7 @@ class ValidateGLSLMaterial(pyblish.api.InstancePlugin): """ meshes = cmds.ls(instance, type="mesh", long=True) - cls.log.info("meshes: {}".format(meshes)) + cls.log.debug("meshes: {}".format(meshes)) # load the glsl shader plugin cmds.loadPlugin("glslShader", quiet=True) @@ -96,8 +96,8 @@ class ValidateGLSLMaterial(pyblish.api.InstancePlugin): cls.log.warning("ogsfx shader file " "not found in {}".format(ogsfx_path)) - cls.log.info("Find the ogsfx shader file in " - "default maya directory...") + cls.log.debug("Searching the ogsfx shader file in " + "default maya directory...") # re-direct to search the ogsfx path in maya_dir ogsfx_path = os.getenv("MAYA_APP_DIR") + ogsfx_path if not os.path.exists(ogsfx_path): @@ -130,8 +130,8 @@ class ValidateGLSLMaterial(pyblish.api.InstancePlugin): @classmethod def pbs_shader_conversion(cls, main_shader, glsl): - cls.log.info("StringrayPBS detected " - "-> Can do texture conversion") + cls.log.debug("StringrayPBS detected " + "-> Can do texture conversion") for shader in main_shader: # get the file textures related to the PBS Shader @@ -168,8 +168,8 @@ class ValidateGLSLMaterial(pyblish.api.InstancePlugin): @classmethod def arnold_shader_conversion(cls, main_shader, glsl): - cls.log.info("aiStandardSurface detected " - "-> Can do texture conversion") + cls.log.debug("aiStandardSurface detected " + "-> Can do texture conversion") for shader in main_shader: # get the file textures related to the PBS Shader diff --git a/openpype/hosts/maya/plugins/publish/validate_instance_attributes.py b/openpype/hosts/maya/plugins/publish/validate_instance_attributes.py deleted file mode 100644 index f870c9f8c4..0000000000 --- a/openpype/hosts/maya/plugins/publish/validate_instance_attributes.py +++ /dev/null @@ -1,60 +0,0 @@ -from maya import cmds - -import pyblish.api -from openpype.pipeline.publish import ( - ValidateContentsOrder, PublishValidationError, RepairAction -) -from openpype.pipeline import discover_legacy_creator_plugins -from openpype.hosts.maya.api.lib import imprint - - -class ValidateInstanceAttributes(pyblish.api.InstancePlugin): - """Validate Instance Attributes. - - New attributes can be introduced as new features come in. Old instances - will need to be updated with these attributes for the documentation to make - sense, and users do not have to recreate the instances. - """ - - order = ValidateContentsOrder - hosts = ["maya"] - families = ["*"] - label = "Instance Attributes" - plugins_by_family = { - p.family: p for p in discover_legacy_creator_plugins() - } - actions = [RepairAction] - - @classmethod - def get_missing_attributes(self, instance): - plugin = self.plugins_by_family[instance.data["family"]] - subset = instance.data["subset"] - asset = instance.data["asset"] - objset = instance.data["objset"] - - missing_attributes = {} - for key, value in plugin(subset, asset).data.items(): - if not cmds.objExists("{}.{}".format(objset, key)): - missing_attributes[key] = value - - return missing_attributes - - def process(self, instance): - objset = instance.data.get("objset") - if objset is None: - self.log.debug( - "Skipping {} because no objectset found.".format(instance) - ) - return - - missing_attributes = self.get_missing_attributes(instance) - if missing_attributes: - raise PublishValidationError( - "Missing attributes on {}:\n{}".format( - objset, missing_attributes - ) - ) - - @classmethod - def repair(cls, instance): - imprint(instance.data["objset"], cls.get_missing_attributes(instance)) diff --git a/openpype/hosts/maya/plugins/publish/validate_instance_in_context.py b/openpype/hosts/maya/plugins/publish/validate_instance_in_context.py index b257add7e8..4ded57137c 100644 --- a/openpype/hosts/maya/plugins/publish/validate_instance_in_context.py +++ b/openpype/hosts/maya/plugins/publish/validate_instance_in_context.py @@ -3,94 +3,19 @@ from __future__ import absolute_import import pyblish.api +import openpype.hosts.maya.api.action from openpype.pipeline.publish import ( - ValidateContentsOrder, PublishValidationError + RepairAction, + ValidateContentsOrder, + PublishValidationError, + OptionalPyblishPluginMixin ) from maya import cmds -class SelectInvalidInstances(pyblish.api.Action): - """Select invalid instances in Outliner.""" - - label = "Select Instances" - icon = "briefcase" - on = "failed" - - def process(self, context, plugin): - """Process invalid validators and select invalid instances.""" - # Get the errored instances - failed = [] - for result in context.data["results"]: - if ( - result["error"] is None - or result["instance"] is None - or result["instance"] in failed - or result["plugin"] != plugin - ): - continue - - failed.append(result["instance"]) - - # Apply pyblish.logic to get the instances for the plug-in - instances = pyblish.api.instances_by_plugin(failed, plugin) - - if instances: - self.log.info( - "Selecting invalid nodes: %s" % ", ".join( - [str(x) for x in instances] - ) - ) - self.select(instances) - else: - self.log.info("No invalid nodes found.") - self.deselect() - - def select(self, instances): - cmds.select(instances, replace=True, noExpand=True) - - def deselect(self): - cmds.select(deselect=True) - - -class RepairSelectInvalidInstances(pyblish.api.Action): - """Repair the instance asset.""" - - label = "Repair" - icon = "wrench" - on = "failed" - - def process(self, context, plugin): - # Get the errored instances - failed = [] - for result in context.data["results"]: - if result["error"] is None: - continue - if result["instance"] is None: - continue - if result["instance"] in failed: - continue - if result["plugin"] != plugin: - continue - - failed.append(result["instance"]) - - # Apply pyblish.logic to get the instances for the plug-in - instances = pyblish.api.instances_by_plugin(failed, plugin) - - context_asset = context.data["assetEntity"]["name"] - for instance in instances: - self.set_attribute(instance, context_asset) - - def set_attribute(self, instance, context_asset): - cmds.setAttr( - instance.data.get("name") + ".asset", - context_asset, - type="string" - ) - - -class ValidateInstanceInContext(pyblish.api.InstancePlugin): +class ValidateInstanceInContext(pyblish.api.InstancePlugin, + OptionalPyblishPluginMixin): """Validator to check if instance asset match context asset. When working in per-shot style you always publish data in context of @@ -104,11 +29,49 @@ class ValidateInstanceInContext(pyblish.api.InstancePlugin): label = "Instance in same Context" optional = True hosts = ["maya"] - actions = [SelectInvalidInstances, RepairSelectInvalidInstances] + actions = [ + openpype.hosts.maya.api.action.SelectInvalidAction, RepairAction + ] def process(self, instance): + if not self.is_active(instance.data): + return + asset = instance.data.get("asset") - context_asset = instance.context.data["assetEntity"]["name"] - msg = "{} has asset {}".format(instance.name, asset) + context_asset = self.get_context_asset(instance) if asset != context_asset: - raise PublishValidationError(msg) + raise PublishValidationError( + message=( + "Instance '{}' publishes to different asset than current " + "context: {}. Current context: {}".format( + instance.name, asset, context_asset + ) + ), + description=( + "## Publishing to a different asset\n" + "There are publish instances present which are publishing " + "into a different asset than your current context.\n\n" + "Usually this is not what you want but there can be cases " + "where you might want to publish into another asset or " + "shot. If that's the case you can disable the validation " + "on the instance to ignore it." + ) + ) + + @classmethod + def get_invalid(cls, instance): + return [instance.data["instance_node"]] + + @classmethod + def repair(cls, instance): + context_asset = cls.get_context_asset(instance) + instance_node = instance.data["instance_node"] + cmds.setAttr( + "{}.asset".format(instance_node), + context_asset, + type="string" + ) + + @staticmethod + def get_context_asset(instance): + return instance.context.data["assetEntity"]["name"] diff --git a/openpype/hosts/maya/plugins/publish/validate_instancer_content.py b/openpype/hosts/maya/plugins/publish/validate_instancer_content.py index 2f14693ef2..236adfb03d 100644 --- a/openpype/hosts/maya/plugins/publish/validate_instancer_content.py +++ b/openpype/hosts/maya/plugins/publish/validate_instancer_content.py @@ -21,7 +21,7 @@ class ValidateInstancerContent(pyblish.api.InstancePlugin): members = instance.data['setMembers'] export_members = instance.data['exactExportMembers'] - self.log.info("Contents {0}".format(members)) + self.log.debug("Contents {0}".format(members)) if not len(members) == len(cmds.ls(members, type="instancer")): self.log.error("Instancer can only contain instancers") diff --git a/openpype/hosts/maya/plugins/publish/validate_instancer_frame_ranges.py b/openpype/hosts/maya/plugins/publish/validate_instancer_frame_ranges.py index fcfcdce8b6..714c6229d6 100644 --- a/openpype/hosts/maya/plugins/publish/validate_instancer_frame_ranges.py +++ b/openpype/hosts/maya/plugins/publish/validate_instancer_frame_ranges.py @@ -5,8 +5,6 @@ import pyblish.api from openpype.pipeline.publish import PublishValidationError -VERBOSE = False - def is_cache_resource(resource): """Return whether resource is a cacheFile resource""" @@ -73,9 +71,6 @@ class ValidateInstancerFrameRanges(pyblish.api.InstancePlugin): xml = all_files.pop(0) assert xml.endswith(".xml") - if VERBOSE: - cls.log.info("Checking: {0}".format(all_files)) - # Ensure all files exist (including ticks) # The remainder file paths should be the .mcx or .mcc files valdidate_files(all_files) @@ -129,8 +124,8 @@ class ValidateInstancerFrameRanges(pyblish.api.InstancePlugin): # for the frames required by the time range. if ticks: ticks = list(sorted(ticks)) - cls.log.info("Found ticks: {0} " - "(substeps: {1})".format(ticks, len(ticks))) + cls.log.debug("Found ticks: {0} " + "(substeps: {1})".format(ticks, len(ticks))) # Check all frames except the last since we don't # require subframes after our time range. diff --git a/openpype/hosts/maya/plugins/publish/validate_maya_units.py b/openpype/hosts/maya/plugins/publish/validate_maya_units.py index 1d5619795f..ae6dc093a9 100644 --- a/openpype/hosts/maya/plugins/publish/validate_maya_units.py +++ b/openpype/hosts/maya/plugins/publish/validate_maya_units.py @@ -37,7 +37,7 @@ class ValidateMayaUnits(pyblish.api.ContextPlugin): ) @classmethod - def apply_settings(cls, project_settings, system_settings): + def apply_settings(cls, project_settings): """Apply project settings to creator""" settings = ( project_settings["maya"]["publish"]["ValidateMayaUnits"] diff --git a/openpype/hosts/maya/plugins/publish/validate_mesh_arnold_attributes.py b/openpype/hosts/maya/plugins/publish/validate_mesh_arnold_attributes.py index 55624726ea..bde78a98b8 100644 --- a/openpype/hosts/maya/plugins/publish/validate_mesh_arnold_attributes.py +++ b/openpype/hosts/maya/plugins/publish/validate_mesh_arnold_attributes.py @@ -36,28 +36,34 @@ class ValidateMeshArnoldAttributes(pyblish.api.InstancePlugin, optional = True - @classmethod - def apply_settings(cls, project_settings, system_settings): - # todo: this should not be done this way - attr = "defaultRenderGlobals.currentRenderer" - cls.active = cmds.getAttr(attr).lower() == "arnold" + # cache (will be `dict` when cached) + arnold_mesh_defaults = None @classmethod def get_default_attributes(cls): + + if cls.arnold_mesh_defaults is not None: + # Use from cache + return cls.arnold_mesh_defaults + # Get default arnold attribute values for mesh type. defaults = {} with delete_after() as tmp: - transform = cmds.createNode("transform") + transform = cmds.createNode("transform", skipSelect=True) tmp.append(transform) - mesh = cmds.createNode("mesh", parent=transform) - for attr in cmds.listAttr(mesh, string="ai*"): + mesh = cmds.createNode("mesh", parent=transform, skipSelect=True) + arnold_attributes = cmds.listAttr(mesh, + string="ai*", + fromPlugin=True) or [] + for attr in arnold_attributes: plug = "{}.{}".format(mesh, attr) try: defaults[attr] = get_attribute(plug) except PublishValidationError: cls.log.debug("Ignoring arnold attribute: {}".format(attr)) + cls.arnold_mesh_defaults = defaults # assign cache return defaults @classmethod @@ -109,6 +115,10 @@ class ValidateMeshArnoldAttributes(pyblish.api.InstancePlugin, if not self.is_active(instance.data): return + if not cmds.pluginInfo("mtoa", query=True, loaded=True): + # Arnold attributes only exist if plug-in is loaded + return + invalid = self.get_invalid_attributes(instance, compute=True) if invalid: raise PublishValidationError( diff --git a/openpype/hosts/maya/plugins/publish/validate_model_name.py b/openpype/hosts/maya/plugins/publish/validate_model_name.py index 6948dcf724..f4c1aa39c7 100644 --- a/openpype/hosts/maya/plugins/publish/validate_model_name.py +++ b/openpype/hosts/maya/plugins/publish/validate_model_name.py @@ -125,7 +125,7 @@ class ValidateModelName(pyblish.api.InstancePlugin, r = re.compile(regex) for obj in filtered: - cls.log.info("testing: {}".format(obj)) + cls.log.debug("testing: {}".format(obj)) m = r.match(obj) if m is None: cls.log.error("invalid name on: {}".format(obj)) diff --git a/openpype/hosts/maya/plugins/publish/validate_mvlook_contents.py b/openpype/hosts/maya/plugins/publish/validate_mvlook_contents.py index 68784a165d..ad0fcafc56 100644 --- a/openpype/hosts/maya/plugins/publish/validate_mvlook_contents.py +++ b/openpype/hosts/maya/plugins/publish/validate_mvlook_contents.py @@ -35,12 +35,12 @@ class ValidateMvLookContents(pyblish.api.InstancePlugin, publishMipMap = instance.data["publishMipMap"] enforced = True if intent in self.enforced_intents: - self.log.info("This validation will be enforced: '{}'" - .format(intent)) + self.log.debug("This validation will be enforced: '{}'" + .format(intent)) else: enforced = False - self.log.info("This validation will NOT be enforced: '{}'" - .format(intent)) + self.log.debug("This validation will NOT be enforced: '{}'" + .format(intent)) if not instance[:]: raise PublishValidationError("Instance is empty") @@ -75,8 +75,9 @@ class ValidateMvLookContents(pyblish.api.InstancePlugin, self.log.warning(msg) if invalid: - raise PublishValidationError("'{}' has invalid look " - "content".format(instance.name)) + raise PublishValidationError( + "'{}' has invalid look content".format(instance.name) + ) def valid_file(self, fname): self.log.debug("Checking validity of '{}'".format(fname)) diff --git a/openpype/hosts/maya/plugins/publish/validate_plugin_path_attributes.py b/openpype/hosts/maya/plugins/publish/validate_plugin_path_attributes.py index 78334cd01f..9f47bf7a3d 100644 --- a/openpype/hosts/maya/plugins/publish/validate_plugin_path_attributes.py +++ b/openpype/hosts/maya/plugins/publish/validate_plugin_path_attributes.py @@ -4,6 +4,8 @@ from maya import cmds import pyblish.api +from openpype.hosts.maya.api.lib import pairwise +from openpype.hosts.maya.api.action import SelectInvalidAction from openpype.pipeline.publish import ( ValidateContentsOrder, PublishValidationError @@ -19,31 +21,33 @@ class ValidatePluginPathAttributes(pyblish.api.InstancePlugin): hosts = ['maya'] families = ["workfile"] label = "Plug-in Path Attributes" + actions = [SelectInvalidAction] - def get_invalid(self, instance): + # Attributes are defined in project settings + attribute = [] + + @classmethod + def get_invalid(cls, instance): invalid = list() - # get the project setting - validate_path = ( - instance.context.data["project_settings"]["maya"]["publish"] - ) - file_attr = validate_path["ValidatePluginPathAttributes"]["attribute"] + file_attr = cls.attribute if not file_attr: return invalid - # get the nodes and file attributes - for node, attr in file_attr.items(): - # check the related nodes - targets = cmds.ls(type=node) + # Consider only valid node types to avoid "Unknown object type" warning + all_node_types = set(cmds.allNodeTypes()) + node_types = [key for key in file_attr.keys() if key in all_node_types] - for target in targets: - # get the filepath - file_attr = "{}.{}".format(target, attr) - filepath = cmds.getAttr(file_attr) + for node, node_type in pairwise(cmds.ls(type=node_types, + showType=True)): + # get the filepath + file_attr = "{}.{}".format(node, file_attr[node_type]) + filepath = cmds.getAttr(file_attr) - if filepath and not os.path.exists(filepath): - self.log.error("File {0} not exists".format(filepath)) # noqa - invalid.append(target) + if filepath and not os.path.exists(filepath): + cls.log.error("{} '{}' uses non-existing filepath: {}" + .format(node_type, node, filepath)) + invalid.append(node) return invalid @@ -51,5 +55,16 @@ class ValidatePluginPathAttributes(pyblish.api.InstancePlugin): """Process all directories Set as Filenames in Non-Maya Nodes""" invalid = self.get_invalid(instance) if invalid: - raise PublishValidationError("Non-existent Path " - "found: {0}".format(invalid)) + raise PublishValidationError( + title="Plug-in Path Attributes", + message="Non-existent filepath found on nodes: {}".format( + ", ".join(invalid) + ), + description=( + "## Plug-in nodes use invalid filepaths\n" + "The workfile contains nodes from plug-ins that use " + "filepaths which do not exist.\n\n" + "Please make sure their filepaths are correct and the " + "files exist on disk." + ) + ) diff --git a/openpype/hosts/maya/plugins/publish/validate_render_image_rule.py b/openpype/hosts/maya/plugins/publish/validate_render_image_rule.py index f9aa7f82d0..030e41ca1f 100644 --- a/openpype/hosts/maya/plugins/publish/validate_render_image_rule.py +++ b/openpype/hosts/maya/plugins/publish/validate_render_image_rule.py @@ -1,4 +1,7 @@ +import os + import pyblish.api + from maya import cmds from openpype.pipeline.publish import ( @@ -22,8 +25,12 @@ class ValidateRenderImageRule(pyblish.api.InstancePlugin): def process(self, instance): - required_images_rule = self.get_default_render_image_folder(instance) - current_images_rule = cmds.workspace(fileRuleEntry="images") + required_images_rule = os.path.normpath( + self.get_default_render_image_folder(instance) + ) + current_images_rule = os.path.normpath( + cmds.workspace(fileRuleEntry="images") + ) if current_images_rule != required_images_rule: raise PublishValidationError( @@ -42,8 +49,17 @@ class ValidateRenderImageRule(pyblish.api.InstancePlugin): cmds.workspace(fileRule=("images", required_images_rule)) cmds.workspace(saveWorkspace=True) - @staticmethod - def get_default_render_image_folder(instance): + @classmethod + def get_default_render_image_folder(cls, instance): + staging_dir = instance.data.get("stagingDir") + if staging_dir: + cls.log.debug( + "Staging dir found: \"{}\". Ignoring setting from " + "`project_settings/maya/RenderSettings/" + "default_render_image_folder`.".format(staging_dir) + ) + return staging_dir + return instance.context.data.get('project_settings')\ .get('maya') \ .get('RenderSettings') \ diff --git a/openpype/hosts/maya/plugins/publish/validate_rig_contents.py b/openpype/hosts/maya/plugins/publish/validate_rig_contents.py index 7b5392f8f9..23f031a5db 100644 --- a/openpype/hosts/maya/plugins/publish/validate_rig_contents.py +++ b/openpype/hosts/maya/plugins/publish/validate_rig_contents.py @@ -2,7 +2,9 @@ import pyblish.api from maya import cmds from openpype.pipeline.publish import ( - PublishValidationError, ValidateContentsOrder) + PublishValidationError, + ValidateContentsOrder +) class ValidateRigContents(pyblish.api.InstancePlugin): @@ -24,31 +26,45 @@ class ValidateRigContents(pyblish.api.InstancePlugin): def process(self, instance): - objectsets = ("controls_SET", "out_SET") - missing = [obj for obj in objectsets if obj not in instance] - assert not missing, ("%s is missing %s" % (instance, missing)) + # Find required sets by suffix + required = ["controls_SET", "out_SET"] + missing = [ + key for key in required if key not in instance.data["rig_sets"] + ] + if missing: + raise PublishValidationError( + "%s is missing sets: %s" % (instance, ", ".join(missing)) + ) + + controls_set = instance.data["rig_sets"]["controls_SET"] + out_set = instance.data["rig_sets"]["out_SET"] # Ensure there are at least some transforms or dag nodes # in the rig instance set_members = instance.data['setMembers'] if not cmds.ls(set_members, type="dagNode", long=True): raise PublishValidationError( - ("No dag nodes in the pointcache instance. " - "(Empty instance?)")) + "No dag nodes in the pointcache instance. " + "(Empty instance?)" + ) # Ensure contents in sets and retrieve long path for all objects - output_content = cmds.sets("out_SET", query=True) or [] - assert output_content, "Must have members in rig out_SET" + output_content = cmds.sets(out_set, query=True) or [] + if not output_content: + raise PublishValidationError("Must have members in rig out_SET") output_content = cmds.ls(output_content, long=True) - controls_content = cmds.sets("controls_SET", query=True) or [] - assert controls_content, "Must have members in rig controls_SET" + controls_content = cmds.sets(controls_set, query=True) or [] + if not controls_content: + raise PublishValidationError( + "Must have members in rig controls_SET" + ) controls_content = cmds.ls(controls_content, long=True) # Validate members are inside the hierarchy from root node - root_node = cmds.ls(set_members, assemblies=True) - hierarchy = cmds.listRelatives(root_node, allDescendents=True, - fullPath=True) + root_nodes = cmds.ls(set_members, assemblies=True, long=True) + hierarchy = cmds.listRelatives(root_nodes, allDescendents=True, + fullPath=True) + root_nodes hierarchy = set(hierarchy) invalid_hierarchy = [] diff --git a/openpype/hosts/maya/plugins/publish/validate_rig_controllers.py b/openpype/hosts/maya/plugins/publish/validate_rig_controllers.py index 7bbf4257ab..a3828f871b 100644 --- a/openpype/hosts/maya/plugins/publish/validate_rig_controllers.py +++ b/openpype/hosts/maya/plugins/publish/validate_rig_controllers.py @@ -52,22 +52,30 @@ class ValidateRigControllers(pyblish.api.InstancePlugin): def process(self, instance): invalid = self.get_invalid(instance) if invalid: - raise PublishValidationError('{} failed, see log ' - 'information'.format(self.label)) + raise PublishValidationError( + '{} failed, see log information'.format(self.label) + ) @classmethod def get_invalid(cls, instance): - controllers_sets = [i for i in instance if i == "controls_SET"] - controls = cmds.sets(controllers_sets, query=True) - assert controls, "Must have 'controls_SET' in rig instance" + controls_set = instance.data["rig_sets"].get("controls_SET") + if not controls_set: + cls.log.error( + "Must have 'controls_SET' in rig instance" + ) + return [instance.data["instance_node"]] + + controls = cmds.sets(controls_set, query=True) # Ensure all controls are within the top group lookup = set(instance[:]) - assert all(control in lookup for control in cmds.ls(controls, - long=True)), ( - "All controls must be inside the rig's group." - ) + if not all(control in lookup for control in cmds.ls(controls, + long=True)): + cls.log.error( + "All controls must be inside the rig's group." + ) + return [controls_set] # Validate all controls has_connections = list() @@ -181,9 +189,17 @@ class ValidateRigControllers(pyblish.api.InstancePlugin): @classmethod def repair(cls, instance): + controls_set = instance.data["rig_sets"].get("controls_SET") + if not controls_set: + cls.log.error( + "Unable to repair because no 'controls_SET' found in rig " + "instance: {}".format(instance) + ) + return + # Use a single undo chunk with undo_chunk(): - controls = cmds.sets("controls_SET", query=True) + controls = cmds.sets(controls_set, query=True) for control in controls: # Lock visibility diff --git a/openpype/hosts/maya/plugins/publish/validate_rig_controllers_arnold_attributes.py b/openpype/hosts/maya/plugins/publish/validate_rig_controllers_arnold_attributes.py index 842c1de01b..03f6a5f1ab 100644 --- a/openpype/hosts/maya/plugins/publish/validate_rig_controllers_arnold_attributes.py +++ b/openpype/hosts/maya/plugins/publish/validate_rig_controllers_arnold_attributes.py @@ -56,11 +56,11 @@ class ValidateRigControllersArnoldAttributes(pyblish.api.InstancePlugin): @classmethod def get_invalid(cls, instance): - controllers_sets = [i for i in instance if i == "controls_SET"] - if not controllers_sets: + controls_set = instance.data["rig_sets"].get("controls_SET") + if not controls_set: return [] - controls = cmds.sets(controllers_sets, query=True) or [] + controls = cmds.sets(controls_set, query=True) or [] if not controls: return [] diff --git a/openpype/hosts/maya/plugins/publish/validate_rig_out_set_node_ids.py b/openpype/hosts/maya/plugins/publish/validate_rig_out_set_node_ids.py index 39f0941faa..fbd510c683 100644 --- a/openpype/hosts/maya/plugins/publish/validate_rig_out_set_node_ids.py +++ b/openpype/hosts/maya/plugins/publish/validate_rig_out_set_node_ids.py @@ -38,16 +38,19 @@ class ValidateRigOutSetNodeIds(pyblish.api.InstancePlugin): # if a deformer has been created on the shape invalid = self.get_invalid(instance) if invalid: - raise PublishValidationError("Nodes found with mismatching " - "IDs: {0}".format(invalid)) + raise PublishValidationError( + "Nodes found with mismatching IDs: {0}".format(invalid) + ) @classmethod def get_invalid(cls, instance): """Get all nodes which do not match the criteria""" - invalid = [] + out_set = instance.data["rig_sets"].get("out_SET") + if not out_set: + return [] - out_set = next(x for x in instance if x.endswith("out_SET")) + invalid = [] members = cmds.sets(out_set, query=True) shapes = cmds.ls(members, dag=True, diff --git a/openpype/hosts/maya/plugins/publish/validate_rig_output_ids.py b/openpype/hosts/maya/plugins/publish/validate_rig_output_ids.py index cbc750bace..24fb36eb8b 100644 --- a/openpype/hosts/maya/plugins/publish/validate_rig_output_ids.py +++ b/openpype/hosts/maya/plugins/publish/validate_rig_output_ids.py @@ -47,7 +47,10 @@ class ValidateRigOutputIds(pyblish.api.InstancePlugin): invalid = {} if compute: - out_set = next(x for x in instance if "out_SET" in x) + out_set = instance.data["rig_sets"].get("out_SET") + if not out_set: + instance.data["mismatched_output_ids"] = invalid + return invalid instance_nodes = cmds.sets(out_set, query=True, nodesOnly=True) instance_nodes = cmds.ls(instance_nodes, long=True) diff --git a/openpype/hosts/maya/plugins/publish/validate_shape_zero.py b/openpype/hosts/maya/plugins/publish/validate_shape_zero.py index 7a7e9a0aee..c7af6a60db 100644 --- a/openpype/hosts/maya/plugins/publish/validate_shape_zero.py +++ b/openpype/hosts/maya/plugins/publish/validate_shape_zero.py @@ -7,6 +7,7 @@ from openpype.hosts.maya.api import lib from openpype.pipeline.publish import ( ValidateContentsOrder, RepairAction, + PublishValidationError ) @@ -67,5 +68,30 @@ class ValidateShapeZero(pyblish.api.Validator): invalid = self.get_invalid(instance) if invalid: - raise ValueError("Shapes found with non-zero component tweaks: " - "{0}".format(invalid)) + raise PublishValidationError( + title="Shape Component Tweaks", + message="Shapes found with non-zero component tweaks: '{}'" + "".format(", ".join(invalid)), + description=( + "## Shapes found with component tweaks\n" + "Shapes were detected that have component tweaks on their " + "components. Please remove the component tweaks to " + "continue.\n\n" + "### Repair\n" + "The repair action will try to *freeze* the component " + "tweaks into the shapes, which is usually the correct fix " + "if the mesh has no construction history (= has its " + "history deleted)."), + detail=( + "Maya allows to store component tweaks within shape nodes " + "which are applied between its `inMesh` and `outMesh` " + "connections resulting in the output of a shape node " + "differing from the input. We usually want to avoid this " + "for published meshes (in particular for Maya scenes) as " + "it can have unintended results when using these meshes " + "as intermediate meshes since it applies positional " + "differences without being visible edits in the node " + "graph.\n\n" + "These tweaks are traditionally stored in the `.pnts` " + "attribute of shapes.") + ) diff --git a/openpype/hosts/maya/plugins/publish/validate_skeletalmesh_hierarchy.py b/openpype/hosts/maya/plugins/publish/validate_skeletalmesh_hierarchy.py index 398b6fb7bf..9084374c76 100644 --- a/openpype/hosts/maya/plugins/publish/validate_skeletalmesh_hierarchy.py +++ b/openpype/hosts/maya/plugins/publish/validate_skeletalmesh_hierarchy.py @@ -28,7 +28,7 @@ class ValidateSkeletalMeshHierarchy(pyblish.api.InstancePlugin): parent.split("|")[1] for parent in (joints_parents + geo_parents) } - self.log.info(parents_set) + self.log.debug(parents_set) if len(set(parents_set)) > 2: raise PublishXmlValidationError( diff --git a/openpype/hosts/maya/plugins/publish/validate_unreal_staticmesh_naming.py b/openpype/hosts/maya/plugins/publish/validate_unreal_staticmesh_naming.py index b2cb2ebda2..5ba256f9f5 100644 --- a/openpype/hosts/maya/plugins/publish/validate_unreal_staticmesh_naming.py +++ b/openpype/hosts/maya/plugins/publish/validate_unreal_staticmesh_naming.py @@ -140,12 +140,12 @@ class ValidateUnrealStaticMeshName(pyblish.api.InstancePlugin, return if not self.validate_mesh and not self.validate_collision: - self.log.info("Validation of both mesh and collision names" - "is disabled.") + self.log.debug("Validation of both mesh and collision names" + "is disabled.") return if not instance.data.get("collisionMembers", None): - self.log.info("There are no collision objects to validate") + self.log.debug("There are no collision objects to validate") return invalid = self.get_invalid(instance) diff --git a/openpype/hosts/maya/plugins/publish/validate_vray_distributed_rendering.py b/openpype/hosts/maya/plugins/publish/validate_vray_distributed_rendering.py index a71849da00..14571203ea 100644 --- a/openpype/hosts/maya/plugins/publish/validate_vray_distributed_rendering.py +++ b/openpype/hosts/maya/plugins/publish/validate_vray_distributed_rendering.py @@ -52,6 +52,6 @@ class ValidateVRayDistributedRendering(pyblish.api.InstancePlugin): renderlayer = instance.data.get("renderlayer") with lib.renderlayer(renderlayer): - cls.log.info("Enabling Distributed Rendering " - "ignore in batch mode..") + cls.log.debug("Enabling Distributed Rendering " + "ignore in batch mode..") cmds.setAttr(cls.ignored_attr, True) diff --git a/openpype/hosts/maya/plugins/publish/validate_yeti_renderscript_callbacks.py b/openpype/hosts/maya/plugins/publish/validate_yeti_renderscript_callbacks.py index 06250f5779..a8085418e7 100644 --- a/openpype/hosts/maya/plugins/publish/validate_yeti_renderscript_callbacks.py +++ b/openpype/hosts/maya/plugins/publish/validate_yeti_renderscript_callbacks.py @@ -54,7 +54,7 @@ class ValidateYetiRenderScriptCallbacks(pyblish.api.InstancePlugin): # has any yeti callback set or not since if the callback # is there it wouldn't error and if it weren't then # nothing happens because there are no yeti nodes. - cls.log.info( + cls.log.debug( "Yeti is loaded but no yeti nodes were found. " "Callback validation skipped.." ) @@ -62,7 +62,7 @@ class ValidateYetiRenderScriptCallbacks(pyblish.api.InstancePlugin): renderer = instance.data["renderer"] if renderer == "redshift": - cls.log.info("Redshift ignores any pre and post render callbacks") + cls.log.debug("Redshift ignores any pre and post render callbacks") return False callback_lookup = cls.callbacks.get(renderer, {}) diff --git a/openpype/hosts/maya/plugins/publish/validate_yeti_rig_input_in_instance.py b/openpype/hosts/maya/plugins/publish/validate_yeti_rig_input_in_instance.py index 96fb475a0a..50a27589ad 100644 --- a/openpype/hosts/maya/plugins/publish/validate_yeti_rig_input_in_instance.py +++ b/openpype/hosts/maya/plugins/publish/validate_yeti_rig_input_in_instance.py @@ -37,8 +37,8 @@ class ValidateYetiRigInputShapesInInstance(pyblish.api.Validator): # Allow publish without input meshes. if not shapes: - cls.log.info("Found no input meshes for %s, skipping ..." - % instance) + cls.log.debug("Found no input meshes for %s, skipping ..." + % instance) return [] # check if input node is part of groomRig instance diff --git a/openpype/hosts/maya/tools/mayalookassigner/arnold_standin.py b/openpype/hosts/maya/tools/mayalookassigner/arnold_standin.py index 0ce2b21dcd..076b0047bb 100644 --- a/openpype/hosts/maya/tools/mayalookassigner/arnold_standin.py +++ b/openpype/hosts/maya/tools/mayalookassigner/arnold_standin.py @@ -10,6 +10,7 @@ from openpype.client import get_last_version_by_subset_name from openpype.hosts.maya import api from . import lib from .alembic import get_alembic_ids_cache +from .usd import is_usd_lib_supported, get_usd_ids_cache log = logging.getLogger(__name__) @@ -74,6 +75,13 @@ def get_nodes_by_id(standin): # Support alembic files directly return get_alembic_ids_cache(path) + elif ( + is_usd_lib_supported and + any(path.endswith(ext) for ext in [".usd", ".usda", ".usdc"]) + ): + # Support usd files directly + return get_usd_ids_cache(path) + json_path = None for f in os.listdir(os.path.dirname(path)): if f.endswith(".json"): diff --git a/openpype/hosts/maya/tools/mayalookassigner/commands.py b/openpype/hosts/maya/tools/mayalookassigner/commands.py index a1290aa68d..5cc4f84931 100644 --- a/openpype/hosts/maya/tools/mayalookassigner/commands.py +++ b/openpype/hosts/maya/tools/mayalookassigner/commands.py @@ -138,8 +138,13 @@ def create_items_from_nodes(nodes): asset_doc = asset_docs_by_id.get(asset_id) # Skip if asset id is not found if not asset_doc: - log.warning("Id not found in the database, skipping '%s'." % _id) - log.warning("Nodes: %s" % id_nodes) + log.warning( + "Id found on {num} nodes for which no asset is found database," + " skipping '{asset_id}'".format( + num=len(nodes), + asset_id=asset_id + ) + ) continue # Collect available look subsets for this asset diff --git a/openpype/hosts/maya/tools/mayalookassigner/usd.py b/openpype/hosts/maya/tools/mayalookassigner/usd.py new file mode 100644 index 0000000000..6b5cb2f0f5 --- /dev/null +++ b/openpype/hosts/maya/tools/mayalookassigner/usd.py @@ -0,0 +1,38 @@ +from collections import defaultdict + +try: + from pxr import Usd + is_usd_lib_supported = True +except ImportError: + is_usd_lib_supported = False + + +def get_usd_ids_cache(path): + # type: (str) -> dict + """Build a id to node mapping in a USD file. + + Nodes without IDs are ignored. + + Returns: + dict: Mapping of id to nodes in the USD file. + + """ + if not is_usd_lib_supported: + raise RuntimeError("No pxr.Usd python library available.") + + stage = Usd.Stage.Open(path) + ids = {} + for prim in stage.Traverse(): + attr = prim.GetAttribute("userProperties:cbId") + if not attr.IsValid(): + continue + value = attr.Get() + if not value: + continue + path = str(prim.GetPath()) + ids[path] = value + + cache = defaultdict(list) + for path, value in ids.items(): + cache[value].append(path) + return dict(cache) diff --git a/openpype/hosts/maya/tools/mayalookassigner/widgets.py b/openpype/hosts/maya/tools/mayalookassigner/widgets.py index f2df17e68c..82c37e2104 100644 --- a/openpype/hosts/maya/tools/mayalookassigner/widgets.py +++ b/openpype/hosts/maya/tools/mayalookassigner/widgets.py @@ -90,15 +90,13 @@ class AssetOutliner(QtWidgets.QWidget): def get_all_assets(self): """Add all items from the current scene""" - items = [] with preserve_expanded_rows(self.view): with preserve_selection(self.view): self.clear() nodes = commands.get_all_asset_nodes() items = commands.create_items_from_nodes(nodes) self.add_items(items) - - return len(items) > 0 + return len(items) > 0 def get_selected_assets(self): """Add all selected items from the current scene""" diff --git a/openpype/hosts/nuke/api/lib.py b/openpype/hosts/nuke/api/lib.py index 4a1e109b17..41e6a27cef 100644 --- a/openpype/hosts/nuke/api/lib.py +++ b/openpype/hosts/nuke/api/lib.py @@ -2041,6 +2041,7 @@ class WorkfileSettings(object): ) workfile_settings = imageio_host["workfile"] + viewer_process_settings = imageio_host["viewer"]["viewerProcess"] if not config_data: # TODO: backward compatibility for old projects - remove later @@ -2091,6 +2092,15 @@ class WorkfileSettings(object): workfile_settings.pop("colorManagement", None) workfile_settings.pop("OCIO_config", None) + # get monitor lut from settings respecting Nuke version differences + monitor_lut = workfile_settings.pop("monitorLut", None) + monitor_lut_data = self._get_monitor_settings( + viewer_process_settings, monitor_lut) + + # set monitor related knobs luts (MonitorOut, Thumbnails) + for knob, value_ in monitor_lut_data.items(): + workfile_settings[knob] = value_ + # then set the rest for knob, value_ in workfile_settings.items(): # skip unfilled ocio config path @@ -2107,8 +2117,9 @@ class WorkfileSettings(object): # set ocio config path if config_data: + config_path = config_data["path"].replace("\\", "/") log.info("OCIO config path found: `{}`".format( - config_data["path"])) + config_path)) # check if there's a mismatch between environment and settings correct_settings = self._is_settings_matching_environment( @@ -2118,6 +2129,40 @@ class WorkfileSettings(object): if correct_settings: self._set_ocio_config_path_to_workfile(config_data) + def _get_monitor_settings(self, viewer_lut, monitor_lut): + """ Get monitor settings from viewer and monitor lut + + Args: + viewer_lut (str): viewer lut string + monitor_lut (str): monitor lut string + + Returns: + dict: monitor settings + """ + output_data = {} + m_display, m_viewer = get_viewer_config_from_string(monitor_lut) + v_display, v_viewer = get_viewer_config_from_string(viewer_lut) + + # set monitor lut differently for nuke version 14 + if nuke.NUKE_VERSION_MAJOR >= 14: + output_data["monitorOutLUT"] = create_viewer_profile_string( + m_viewer, m_display, path_like=False) + # monitorLut=thumbnails - viewerProcess makes more sense + output_data["monitorLut"] = create_viewer_profile_string( + v_viewer, v_display, path_like=False) + + if nuke.NUKE_VERSION_MAJOR == 13: + output_data["monitorOutLUT"] = create_viewer_profile_string( + m_viewer, m_display, path_like=False) + # monitorLut=thumbnails - viewerProcess makes more sense + output_data["monitorLut"] = create_viewer_profile_string( + v_viewer, v_display, path_like=True) + if nuke.NUKE_VERSION_MAJOR <= 12: + output_data["monitorLut"] = create_viewer_profile_string( + m_viewer, m_display, path_like=True) + + return output_data + def _is_settings_matching_environment(self, config_data): """ Check if OCIO config path is different from environment @@ -2177,6 +2222,7 @@ Reopening Nuke should synchronize these paths and resolve any discrepancies. """ # replace path with env var if possible ocio_path = self._replace_ocio_path_with_env_var(config_data) + ocio_path = ocio_path.replace("\\", "/") log.info("Setting OCIO config path to: `{}`".format( ocio_path)) @@ -2232,7 +2278,7 @@ Reopening Nuke should synchronize these paths and resolve any discrepancies. Returns: str: OCIO config path with environment variable TCL expression """ - config_path = config_data["path"] + config_path = config_data["path"].replace("\\", "/") config_template = config_data["template"] included_vars = self._get_included_vars(config_template) @@ -3320,11 +3366,11 @@ def get_viewer_config_from_string(input_string): display = split[0] elif "(" in viewer: pattern = r"([\w\d\s\.\-]+).*[(](.*)[)]" - result = re.findall(pattern, viewer) + result_ = re.findall(pattern, viewer) try: - result = result.pop() - display = str(result[1]).rstrip() - viewer = str(result[0]).rstrip() + result_ = result_.pop() + display = str(result_[1]).rstrip() + viewer = str(result_[0]).rstrip() except IndexError: raise IndexError(( "Viewer Input string is not correct. " @@ -3332,3 +3378,22 @@ def get_viewer_config_from_string(input_string): ).format(input_string)) return (display, viewer) + + +def create_viewer_profile_string(viewer, display=None, path_like=False): + """Convert viewer and display to string + + Args: + viewer (str): viewer name + display (Optional[str]): display name + path_like (Optional[bool]): if True, return path like string + + Returns: + str: viewer config string + """ + if not display: + return viewer + + if path_like: + return "{}/{}".format(display, viewer) + return "{} ({})".format(viewer, display) diff --git a/openpype/hosts/nuke/api/plugin.py b/openpype/hosts/nuke/api/plugin.py index 6d48c09d60..a0e1525cd0 100644 --- a/openpype/hosts/nuke/api/plugin.py +++ b/openpype/hosts/nuke/api/plugin.py @@ -379,11 +379,7 @@ class NukeWriteCreator(NukeCreator): sys.exc_info()[2] ) - def apply_settings( - self, - project_settings, - system_settings - ): + def apply_settings(self, project_settings): """Method called on initialization of plugin to apply settings.""" # plugin settings diff --git a/openpype/hosts/nuke/plugins/load/load_camera_abc.py b/openpype/hosts/nuke/plugins/load/load_camera_abc.py index fec4ee556e..2939ceebae 100644 --- a/openpype/hosts/nuke/plugins/load/load_camera_abc.py +++ b/openpype/hosts/nuke/plugins/load/load_camera_abc.py @@ -112,8 +112,6 @@ class AlembicCameraLoader(load.LoaderPlugin): version_doc = get_version_by_id(project_name, representation["parent"]) object_name = container['objectName'] - # get corresponding node - camera_node = nuke.toNode(object_name) # get main variables version_data = version_doc.get("data", {}) diff --git a/openpype/hosts/nuke/plugins/publish/extract_review_data_lut.py b/openpype/hosts/nuke/plugins/publish/extract_review_data_lut.py index e4b7b155cd..2a26ed82fb 100644 --- a/openpype/hosts/nuke/plugins/publish/extract_review_data_lut.py +++ b/openpype/hosts/nuke/plugins/publish/extract_review_data_lut.py @@ -20,7 +20,6 @@ class ExtractReviewDataLut(publish.Extractor): hosts = ["nuke"] def process(self, instance): - families = instance.data["families"] self.log.info("Creating staging dir...") if "representations" in instance.data: staging_dir = instance.data[ diff --git a/openpype/hosts/nuke/plugins/publish/extract_thumbnail.py b/openpype/hosts/nuke/plugins/publish/extract_thumbnail.py index d57d55f85d..b20df4ffe2 100644 --- a/openpype/hosts/nuke/plugins/publish/extract_thumbnail.py +++ b/openpype/hosts/nuke/plugins/publish/extract_thumbnail.py @@ -91,8 +91,6 @@ class ExtractThumbnail(publish.Extractor): if collection: # get path - fname = os.path.basename(collection.format( - "{head}{padding}{tail}")) fhead = collection.format("{head}") thumb_fname = list(collection)[mid_frame] diff --git a/openpype/hosts/nuke/plugins/publish/validate_rendered_frames.py b/openpype/hosts/nuke/plugins/publish/validate_rendered_frames.py index 45c20412c8..9a35b61a0e 100644 --- a/openpype/hosts/nuke/plugins/publish/validate_rendered_frames.py +++ b/openpype/hosts/nuke/plugins/publish/validate_rendered_frames.py @@ -14,27 +14,26 @@ class RepairActionBase(pyblish.api.Action): # Get the errored instances return get_errored_instances_from_context(context, plugin=plugin) - def repair_knob(self, instances, state): + def repair_knob(self, context, instances, state): + create_context = context.data["create_context"] for instance in instances: - node = instance.data["transientData"]["node"] - files_remove = [os.path.join(instance.data["outputDir"], f) - for r in instance.data.get("representations", []) - for f in r.get("files", []) - ] - self.log.info("Files to be removed: {}".format(files_remove)) - for f in files_remove: - os.remove(f) - self.log.debug("removing file: {}".format(f)) - node["render"].setValue(state) + # Reset the render knob + instance_id = instance.data.get("instance_id") + created_instance = create_context.get_instance_by_id( + instance_id + ) + created_instance.creator_attributes["render_target"] = state self.log.info("Rendering toggled to `{}`".format(state)) + create_context.save_changes() + class RepairCollectionActionToLocal(RepairActionBase): label = "Repair - rerender with \"Local\"" def process(self, context, plugin): instances = self.get_instance(context, plugin) - self.repair_knob(instances, "Local") + self.repair_knob(context, instances, "local") class RepairCollectionActionToFarm(RepairActionBase): @@ -42,7 +41,7 @@ class RepairCollectionActionToFarm(RepairActionBase): def process(self, context, plugin): instances = self.get_instance(context, plugin) - self.repair_knob(instances, "On farm") + self.repair_knob(context, instances, "farm") class ValidateRenderedFrames(pyblish.api.InstancePlugin): diff --git a/openpype/hosts/nuke/plugins/publish/validate_write_nodes.py b/openpype/hosts/nuke/plugins/publish/validate_write_nodes.py index aeecea655f..2a925fbeff 100644 --- a/openpype/hosts/nuke/plugins/publish/validate_write_nodes.py +++ b/openpype/hosts/nuke/plugins/publish/validate_write_nodes.py @@ -1,3 +1,5 @@ +from collections import defaultdict + import pyblish.api from openpype.pipeline.publish import get_errored_instances_from_context from openpype.hosts.nuke.api.lib import ( @@ -87,6 +89,11 @@ class ValidateNukeWriteNode( correct_data )) + # Collect key values of same type in a list. + values_by_name = defaultdict(list) + for knob_data in correct_data["knobs"]: + values_by_name[knob_data["name"]].append(knob_data["value"]) + for knob_data in correct_data["knobs"]: knob_type = knob_data["type"] self.log.debug("__ knob_type: {}".format( @@ -105,28 +112,33 @@ class ValidateNukeWriteNode( ) key = knob_data["name"] - value = knob_data["value"] + values = values_by_name[key] node_value = write_node[key].value() # fix type differences - if type(node_value) in (int, float): - try: - if isinstance(value, list): - value = color_gui_to_int(value) - else: - value = float(value) - node_value = float(node_value) - except ValueError: - value = str(value) - else: - value = str(value) - node_value = str(node_value) + fixed_values = [] + for value in values: + if type(node_value) in (int, float): + try: - self.log.debug("__ key: {} | value: {}".format( - key, value + if isinstance(value, list): + value = color_gui_to_int(value) + else: + value = float(value) + node_value = float(node_value) + except ValueError: + value = str(value) + else: + value = str(value) + node_value = str(node_value) + + fixed_values.append(value) + + self.log.debug("__ key: {} | values: {}".format( + key, fixed_values )) if ( - node_value != value + node_value not in fixed_values and key != "file" and key != "tile_color" ): diff --git a/openpype/hosts/photoshop/plugins/create/create_flatten_image.py b/openpype/hosts/photoshop/plugins/create/create_flatten_image.py index 3bc61c8184..e4229788bd 100644 --- a/openpype/hosts/photoshop/plugins/create/create_flatten_image.py +++ b/openpype/hosts/photoshop/plugins/create/create_flatten_image.py @@ -4,6 +4,7 @@ from openpype.lib import BoolDef import openpype.hosts.photoshop.api as api from openpype.hosts.photoshop.lib import PSAutoCreator from openpype.pipeline.create import get_subset_name +from openpype.lib import prepare_template_data from openpype.client import get_asset_by_name @@ -37,19 +38,14 @@ class AutoImageCreator(PSAutoCreator): asset_doc = get_asset_by_name(project_name, asset_name) if existing_instance is None: - subset_name = get_subset_name( - self.family, self.default_variant, task_name, asset_doc, + subset_name = self.get_subset_name( + self.default_variant, task_name, asset_doc, project_name, host_name ) - publishable_ids = [layer.id for layer in api.stub().get_layers() - if layer.visible] data = { "asset": asset_name, "task": task_name, - # ids are "virtual" layers, won't get grouped as 'members' do - # same difference in color coded layers in WP - "ids": publishable_ids } if not self.active_on_create: @@ -69,8 +65,8 @@ class AutoImageCreator(PSAutoCreator): existing_instance["asset"] != asset_name or existing_instance["task"] != task_name ): - subset_name = get_subset_name( - self.family, self.default_variant, task_name, asset_doc, + subset_name = self.get_subset_name( + self.default_variant, task_name, asset_doc, project_name, host_name ) @@ -98,7 +94,7 @@ class AutoImageCreator(PSAutoCreator): ) ] - def apply_settings(self, project_settings, system_settings): + def apply_settings(self, project_settings): plugin_settings = ( project_settings["photoshop"]["create"]["AutoImageCreator"] ) @@ -118,3 +114,29 @@ class AutoImageCreator(PSAutoCreator): Artist might disable this instance from publishing or from creating review for it though. """ + + def get_subset_name( + self, + variant, + task_name, + asset_doc, + project_name, + host_name=None, + instance=None + ): + dynamic_data = prepare_template_data({"layer": "{layer}"}) + subset_name = get_subset_name( + self.family, variant, task_name, asset_doc, + project_name, host_name, dynamic_data=dynamic_data + ) + return self._clean_subset_name(subset_name) + + def _clean_subset_name(self, subset_name): + """Clean all variants leftover {layer} from subset name.""" + dynamic_data = prepare_template_data({"layer": "{layer}"}) + for value in dynamic_data.values(): + if value in subset_name: + return (subset_name.replace(value, "") + .replace("__", "_") + .replace("..", ".")) + return subset_name diff --git a/openpype/hosts/photoshop/plugins/create/create_image.py b/openpype/hosts/photoshop/plugins/create/create_image.py index f3165fca57..af20d456e0 100644 --- a/openpype/hosts/photoshop/plugins/create/create_image.py +++ b/openpype/hosts/photoshop/plugins/create/create_image.py @@ -94,12 +94,17 @@ class ImageCreator(Creator): name = self._clean_highlights(stub, directory) layer_names_in_hierarchy.append(name) - data.update({"subset": subset_name}) - data.update({"members": [str(group.id)]}) - data.update({"layer_name": layer_name}) - data.update({"long_name": "_".join(layer_names_in_hierarchy)}) + data_update = { + "subset": subset_name, + "members": [str(group.id)], + "layer_name": layer_name, + "long_name": "_".join(layer_names_in_hierarchy) + } + data.update(data_update) - creator_attributes = {"mark_for_review": self.mark_for_review} + mark_for_review = (pre_create_data.get("mark_for_review") or + self.mark_for_review) + creator_attributes = {"mark_for_review": mark_for_review} data.update({"creator_attributes": creator_attributes}) if not self.active_on_create: @@ -124,8 +129,6 @@ class ImageCreator(Creator): if creator_id == self.identifier: instance_data = self._handle_legacy(instance_data) - layer = api.stub().get_layer(instance_data["members"][0]) - instance_data["layer"] = layer instance = CreatedInstance.from_existing( instance_data, self ) @@ -171,7 +174,7 @@ class ImageCreator(Creator): ) ] - def apply_settings(self, project_settings, system_settings): + def apply_settings(self, project_settings): plugin_settings = ( project_settings["photoshop"]["create"]["ImageCreator"] ) diff --git a/openpype/hosts/photoshop/plugins/create/create_review.py b/openpype/hosts/photoshop/plugins/create/create_review.py index 064485d465..63751d94e4 100644 --- a/openpype/hosts/photoshop/plugins/create/create_review.py +++ b/openpype/hosts/photoshop/plugins/create/create_review.py @@ -18,7 +18,7 @@ class ReviewCreator(PSAutoCreator): it will get recreated in next publish either way). """ - def apply_settings(self, project_settings, system_settings): + def apply_settings(self, project_settings): plugin_settings = ( project_settings["photoshop"]["create"]["ReviewCreator"] ) diff --git a/openpype/hosts/photoshop/plugins/create/create_workfile.py b/openpype/hosts/photoshop/plugins/create/create_workfile.py index d498f0549c..1b255de3a3 100644 --- a/openpype/hosts/photoshop/plugins/create/create_workfile.py +++ b/openpype/hosts/photoshop/plugins/create/create_workfile.py @@ -19,7 +19,7 @@ class WorkfileCreator(PSAutoCreator): in next publish automatically). """ - def apply_settings(self, project_settings, system_settings): + def apply_settings(self, project_settings): plugin_settings = ( project_settings["photoshop"]["create"]["WorkfileCreator"] ) diff --git a/openpype/hosts/photoshop/plugins/publish/collect_auto_image.py b/openpype/hosts/photoshop/plugins/publish/collect_auto_image.py index f1d8419608..77f1a3e91f 100644 --- a/openpype/hosts/photoshop/plugins/publish/collect_auto_image.py +++ b/openpype/hosts/photoshop/plugins/publish/collect_auto_image.py @@ -16,7 +16,6 @@ class CollectAutoImage(pyblish.api.ContextPlugin): targets = ["automated"] def process(self, context): - family = "image" for instance in context: creator_identifier = instance.data.get("creator_identifier") if creator_identifier and creator_identifier == "auto_image": diff --git a/openpype/hosts/photoshop/plugins/publish/collect_auto_image_refresh.py b/openpype/hosts/photoshop/plugins/publish/collect_auto_image_refresh.py new file mode 100644 index 0000000000..741fb0e9cd --- /dev/null +++ b/openpype/hosts/photoshop/plugins/publish/collect_auto_image_refresh.py @@ -0,0 +1,24 @@ +import pyblish.api + +from openpype.hosts.photoshop import api as photoshop + + +class CollectAutoImageRefresh(pyblish.api.ContextPlugin): + """Refreshes auto_image instance with currently visible layers.. + """ + + label = "Collect Auto Image Refresh" + order = pyblish.api.CollectorOrder + hosts = ["photoshop"] + order = pyblish.api.CollectorOrder + 0.2 + + def process(self, context): + for instance in context: + creator_identifier = instance.data.get("creator_identifier") + if creator_identifier and creator_identifier == "auto_image": + self.log.debug("Auto image instance found, won't create new") + # refresh existing auto image instance with current visible + publishable_ids = [layer.id for layer in photoshop.stub().get_layers() # noqa + if layer.visible] + instance.data["ids"] = publishable_ids + return diff --git a/openpype/hosts/photoshop/plugins/publish/collect_image.py b/openpype/hosts/photoshop/plugins/publish/collect_image.py new file mode 100644 index 0000000000..64727cef33 --- /dev/null +++ b/openpype/hosts/photoshop/plugins/publish/collect_image.py @@ -0,0 +1,20 @@ +import pyblish.api + +from openpype.hosts.photoshop import api + + +class CollectImage(pyblish.api.InstancePlugin): + """Collect layer metadata into a instance. + + Used later in validation + """ + order = pyblish.api.CollectorOrder + 0.200 + label = 'Collect Image' + + hosts = ["photoshop"] + families = ["image"] + + def process(self, instance): + if instance.data.get("members"): + layer = api.stub().get_layer(instance.data["members"][0]) + instance.data["layer"] = layer diff --git a/openpype/hosts/photoshop/plugins/publish/extract_image.py b/openpype/hosts/photoshop/plugins/publish/extract_image.py index cdb28c742d..680f580cc0 100644 --- a/openpype/hosts/photoshop/plugins/publish/extract_image.py +++ b/openpype/hosts/photoshop/plugins/publish/extract_image.py @@ -45,9 +45,11 @@ class ExtractImage(pyblish.api.ContextPlugin): # Perform extraction files = {} ids = set() - layer = instance.data.get("layer") - if layer: - ids.add(layer.id) + # real layers and groups + members = instance.data("members") + if members: + ids.update(set([int(member) for member in members])) + # virtual groups collected by color coding or auto_image add_ids = instance.data.pop("ids", None) if add_ids: ids.update(set(add_ids)) diff --git a/openpype/hosts/photoshop/plugins/publish/extract_review.py b/openpype/hosts/photoshop/plugins/publish/extract_review.py index 4aa7a05bd1..d5dac417d7 100644 --- a/openpype/hosts/photoshop/plugins/publish/extract_review.py +++ b/openpype/hosts/photoshop/plugins/publish/extract_review.py @@ -1,4 +1,5 @@ import os +import shutil from PIL import Image from openpype.lib import ( @@ -55,6 +56,7 @@ class ExtractReview(publish.Extractor): } if instance.data["family"] != "review": + self.log.debug("Existing extracted file from image family used.") # enable creation of review, without this jpg review would clash # with jpg of the image family output_name = repre_name @@ -62,8 +64,15 @@ class ExtractReview(publish.Extractor): repre_skeleton.update({"name": repre_name, "outputName": output_name}) - if self.make_image_sequence and len(layers) > 1: - self.log.info("Extract layers to image sequence.") + img_file = self.output_seq_filename % 0 + self._prepare_file_for_image_family(img_file, instance, + staging_dir) + repre_skeleton.update({ + "files": img_file, + }) + processed_img_names = [img_file] + elif self.make_image_sequence and len(layers) > 1: + self.log.debug("Extract layers to image sequence.") img_list = self._save_sequence_images(staging_dir, layers) repre_skeleton.update({ @@ -72,17 +81,17 @@ class ExtractReview(publish.Extractor): "fps": fps, "files": img_list, }) - instance.data["representations"].append(repre_skeleton) processed_img_names = img_list else: - self.log.info("Extract layers to flatten image.") - img_list = self._save_flatten_image(staging_dir, layers) + self.log.debug("Extract layers to flatten image.") + img_file = self._save_flatten_image(staging_dir, layers) repre_skeleton.update({ - "files": img_list, + "files": img_file, }) - instance.data["representations"].append(repre_skeleton) - processed_img_names = [img_list] + processed_img_names = [img_file] + + instance.data["representations"].append(repre_skeleton) ffmpeg_args = get_ffmpeg_tool_args("ffmpeg") @@ -111,6 +120,39 @@ class ExtractReview(publish.Extractor): self.log.info(f"Extracted {instance} to {staging_dir}") + def _prepare_file_for_image_family(self, img_file, instance, staging_dir): + """Converts existing file for image family to .jpg + + Image instance could have its own separate review (instance per layer + for example). This uses extracted file instead of extracting again. + Args: + img_file (str): name of output file (with 0000 value for ffmpeg + later) + instance: + staging_dir (str): temporary folder where extracted file is located + """ + repre_file = instance.data["representations"][0] + source_file_path = os.path.join(repre_file["stagingDir"], + repre_file["files"]) + if not os.path.exists(source_file_path): + raise RuntimeError(f"{source_file_path} doesn't exist for " + "review to create from") + _, ext = os.path.splitext(repre_file["files"]) + if ext != ".jpg": + im = Image.open(source_file_path) + if (im.mode in ('RGBA', 'LA') or ( + im.mode == 'P' and 'transparency' in im.info)): + # without this it produces messy low quality jpg + rgb_im = Image.new("RGBA", (im.width, im.height), "#ffffff") + rgb_im.alpha_composite(im) + rgb_im.convert("RGB").save(os.path.join(staging_dir, img_file)) + else: + im.save(os.path.join(staging_dir, img_file)) + else: + # handles already .jpg + shutil.copy(source_file_path, + os.path.join(staging_dir, img_file)) + def _generate_mov(self, ffmpeg_path, instance, fps, no_of_frames, source_files_pattern, staging_dir): """Generates .mov to upload to Ftrack. @@ -218,6 +260,11 @@ class ExtractReview(publish.Extractor): (list) of PSItem """ layers = [] + # creating review for existing 'image' instance + if instance.data["family"] == "image" and instance.data.get("layer"): + layers.append(instance.data["layer"]) + return layers + for image_instance in instance.context: if image_instance.data["family"] != "image": continue diff --git a/openpype/hosts/resolve/api/plugin.py b/openpype/hosts/resolve/api/plugin.py index 59c27f29da..e2bd76ffa2 100644 --- a/openpype/hosts/resolve/api/plugin.py +++ b/openpype/hosts/resolve/api/plugin.py @@ -413,8 +413,6 @@ class ClipLoader: if self.with_handles: source_in -= handle_start source_out += handle_end - handle_start = 0 - handle_end = 0 # make track item from source in bin as item timeline_item = lib.create_timeline_item( @@ -433,14 +431,6 @@ class ClipLoader: self.data["path"], self.active_bin) _clip_property = media_pool_item.GetClipProperty - # get handles - handle_start = self.data["versionData"].get("handleStart") - handle_end = self.data["versionData"].get("handleEnd") - if handle_start is None: - handle_start = int(self.data["assetData"]["handleStart"]) - if handle_end is None: - handle_end = int(self.data["assetData"]["handleEnd"]) - source_in = int(_clip_property("Start")) source_out = int(_clip_property("End")) diff --git a/openpype/hosts/standalonepublisher/plugins/publish/extract_thumbnail.py b/openpype/hosts/standalonepublisher/plugins/publish/extract_thumbnail.py index b99503b3c8..a2afd160fa 100644 --- a/openpype/hosts/standalonepublisher/plugins/publish/extract_thumbnail.py +++ b/openpype/hosts/standalonepublisher/plugins/publish/extract_thumbnail.py @@ -49,8 +49,6 @@ class ExtractThumbnailSP(pyblish.api.InstancePlugin): else: first_filename = files - staging_dir = None - # Convert to jpeg if not yet full_input_path = os.path.join( thumbnail_repre["stagingDir"], first_filename diff --git a/openpype/hosts/traypublisher/plugins/create/create_movie_batch.py b/openpype/hosts/traypublisher/plugins/create/create_movie_batch.py index 1bed07f785..3454b6e135 100644 --- a/openpype/hosts/traypublisher/plugins/create/create_movie_batch.py +++ b/openpype/hosts/traypublisher/plugins/create/create_movie_batch.py @@ -36,7 +36,7 @@ class BatchMovieCreator(TrayPublishCreator): # Position batch creator after simple creators order = 110 - def apply_settings(self, project_settings, system_settings): + def apply_settings(self, project_settings): creator_settings = ( project_settings["traypublisher"]["create"]["BatchMovieCreator"] ) diff --git a/openpype/hosts/traypublisher/plugins/publish/collect_frame_range_asset_entity.py b/openpype/hosts/traypublisher/plugins/publish/collect_missing_frame_range_asset_entity.py similarity index 83% rename from openpype/hosts/traypublisher/plugins/publish/collect_frame_range_asset_entity.py rename to openpype/hosts/traypublisher/plugins/publish/collect_missing_frame_range_asset_entity.py index c18e10e438..72379ea4e1 100644 --- a/openpype/hosts/traypublisher/plugins/publish/collect_frame_range_asset_entity.py +++ b/openpype/hosts/traypublisher/plugins/publish/collect_missing_frame_range_asset_entity.py @@ -2,16 +2,18 @@ import pyblish.api from openpype.pipeline import OptionalPyblishPluginMixin -class CollectFrameDataFromAssetEntity(pyblish.api.InstancePlugin, - OptionalPyblishPluginMixin): - """Collect Frame Range data From Asset Entity +class CollectMissingFrameDataFromAssetEntity( + pyblish.api.InstancePlugin, + OptionalPyblishPluginMixin +): + """Collect Missing Frame Range data From Asset Entity Frame range data will only be collected if the keys are not yet collected for the instance. """ order = pyblish.api.CollectorOrder + 0.491 - label = "Collect Frame Data From Asset Entity" + label = "Collect Missing Frame Data From Asset Entity" families = ["plate", "pointcache", "vdbcache", "online", "render"] diff --git a/openpype/hosts/traypublisher/plugins/publish/validate_frame_ranges.py b/openpype/hosts/traypublisher/plugins/publish/validate_frame_ranges.py index b962ea464a..09de2d8db2 100644 --- a/openpype/hosts/traypublisher/plugins/publish/validate_frame_ranges.py +++ b/openpype/hosts/traypublisher/plugins/publish/validate_frame_ranges.py @@ -15,7 +15,7 @@ class ValidateFrameRange(OptionalPyblishPluginMixin, label = "Validate Frame Range" hosts = ["traypublisher"] - families = ["render"] + families = ["render", "plate"] order = ValidateContentsOrder optional = True diff --git a/openpype/hosts/tvpaint/api/communication_server.py b/openpype/hosts/tvpaint/api/communication_server.py index 6f76c25e0c..d67ef8f798 100644 --- a/openpype/hosts/tvpaint/api/communication_server.py +++ b/openpype/hosts/tvpaint/api/communication_server.py @@ -11,7 +11,7 @@ import filecmp import tempfile import threading import shutil -from queue import Queue + from contextlib import closing from aiohttp import web @@ -319,19 +319,19 @@ class QtTVPaintRpc(BaseTVPaintRpc): async def workfiles_tool(self): log.info("Triggering Workfile tool") item = MainThreadItem(self.tools_helper.show_workfiles) - self._execute_in_main_thread(item) + self._execute_in_main_thread(item, wait=False) return async def loader_tool(self): log.info("Triggering Loader tool") item = MainThreadItem(self.tools_helper.show_loader) - self._execute_in_main_thread(item) + self._execute_in_main_thread(item, wait=False) return async def publish_tool(self): log.info("Triggering Publish tool") item = MainThreadItem(self.tools_helper.show_publisher_tool) - self._execute_in_main_thread(item) + self._execute_in_main_thread(item, wait=False) return async def scene_inventory_tool(self): @@ -350,13 +350,13 @@ class QtTVPaintRpc(BaseTVPaintRpc): async def library_loader_tool(self): log.info("Triggering Library loader tool") item = MainThreadItem(self.tools_helper.show_library_loader) - self._execute_in_main_thread(item) + self._execute_in_main_thread(item, wait=False) return async def experimental_tools(self): log.info("Triggering Library loader tool") item = MainThreadItem(self.tools_helper.show_experimental_tools_dialog) - self._execute_in_main_thread(item) + self._execute_in_main_thread(item, wait=False) return async def _async_execute_in_main_thread(self, item, **kwargs): @@ -867,7 +867,7 @@ class QtCommunicator(BaseCommunicator): def __init__(self, qt_app): super().__init__() - self.callback_queue = Queue() + self.callback_queue = collections.deque() self.qt_app = qt_app def _create_routes(self): @@ -880,14 +880,14 @@ class QtCommunicator(BaseCommunicator): def execute_in_main_thread(self, main_thread_item, wait=True): """Add `MainThreadItem` to callback queue and wait for result.""" - self.callback_queue.put(main_thread_item) + self.callback_queue.append(main_thread_item) if wait: return main_thread_item.wait() return async def async_execute_in_main_thread(self, main_thread_item, wait=True): """Add `MainThreadItem` to callback queue and wait for result.""" - self.callback_queue.put(main_thread_item) + self.callback_queue.append(main_thread_item) if wait: return await main_thread_item.async_wait() @@ -904,9 +904,9 @@ class QtCommunicator(BaseCommunicator): self._exit() return None - if self.callback_queue.empty(): - return None - return self.callback_queue.get() + if self.callback_queue: + return self.callback_queue.popleft() + return None def _on_client_connect(self): super()._on_client_connect() diff --git a/openpype/hosts/tvpaint/plugins/create/create_render.py b/openpype/hosts/tvpaint/plugins/create/create_render.py index 2369c7329f..b7a7c208d9 100644 --- a/openpype/hosts/tvpaint/plugins/create/create_render.py +++ b/openpype/hosts/tvpaint/plugins/create/create_render.py @@ -139,7 +139,7 @@ class CreateRenderlayer(TVPaintCreator): # - Mark by default instance for review mark_for_review = True - def apply_settings(self, project_settings, system_settings): + def apply_settings(self, project_settings): plugin_settings = ( project_settings["tvpaint"]["create"]["create_render_layer"] ) @@ -387,7 +387,7 @@ class CreateRenderPass(TVPaintCreator): # Settings mark_for_review = True - def apply_settings(self, project_settings, system_settings): + def apply_settings(self, project_settings): plugin_settings = ( project_settings["tvpaint"]["create"]["create_render_pass"] ) @@ -690,7 +690,7 @@ class TVPaintAutoDetectRenderCreator(TVPaintCreator): group_idx_offset = 10 group_idx_padding = 3 - def apply_settings(self, project_settings, system_settings): + def apply_settings(self, project_settings): plugin_settings = ( project_settings ["tvpaint"] @@ -1029,7 +1029,7 @@ class TVPaintSceneRenderCreator(TVPaintAutoCreator): mark_for_review = True active_on_create = False - def apply_settings(self, project_settings, system_settings): + def apply_settings(self, project_settings): plugin_settings = ( project_settings["tvpaint"]["create"]["create_render_scene"] ) diff --git a/openpype/hosts/tvpaint/plugins/create/create_review.py b/openpype/hosts/tvpaint/plugins/create/create_review.py index 886dae7c39..7bb7510a8e 100644 --- a/openpype/hosts/tvpaint/plugins/create/create_review.py +++ b/openpype/hosts/tvpaint/plugins/create/create_review.py @@ -12,7 +12,7 @@ class TVPaintReviewCreator(TVPaintAutoCreator): # Settings active_on_create = True - def apply_settings(self, project_settings, system_settings): + def apply_settings(self, project_settings): plugin_settings = ( project_settings["tvpaint"]["create"]["create_review"] ) diff --git a/openpype/hosts/tvpaint/plugins/create/create_workfile.py b/openpype/hosts/tvpaint/plugins/create/create_workfile.py index 41347576d5..c3982c0eca 100644 --- a/openpype/hosts/tvpaint/plugins/create/create_workfile.py +++ b/openpype/hosts/tvpaint/plugins/create/create_workfile.py @@ -9,7 +9,7 @@ class TVPaintWorkfileCreator(TVPaintAutoCreator): label = "Workfile" icon = "fa.file-o" - def apply_settings(self, project_settings, system_settings): + def apply_settings(self, project_settings): plugin_settings = ( project_settings["tvpaint"]["create"]["create_workfile"] ) diff --git a/openpype/hosts/tvpaint/plugins/load/load_reference_image.py b/openpype/hosts/tvpaint/plugins/load/load_reference_image.py index edc116a8e4..3707ef97aa 100644 --- a/openpype/hosts/tvpaint/plugins/load/load_reference_image.py +++ b/openpype/hosts/tvpaint/plugins/load/load_reference_image.py @@ -171,7 +171,7 @@ class LoadImage(plugin.Loader): george_script = "\n".join(george_script_lines) execute_george_through_file(george_script) - def _remove_container(self, container, members=None): + def _remove_container(self, container): if not container: return representation = container["representation"] diff --git a/openpype/hosts/tvpaint/plugins/publish/extract_sequence.py b/openpype/hosts/tvpaint/plugins/publish/extract_sequence.py index 8a610cf388..fd568b2826 100644 --- a/openpype/hosts/tvpaint/plugins/publish/extract_sequence.py +++ b/openpype/hosts/tvpaint/plugins/publish/extract_sequence.py @@ -6,7 +6,10 @@ from PIL import Image import pyblish.api -from openpype.pipeline.publish import KnownPublishError +from openpype.pipeline.publish import ( + KnownPublishError, + get_publish_instance_families, +) from openpype.hosts.tvpaint.api.lib import ( execute_george, execute_george_through_file, @@ -63,7 +66,6 @@ class ExtractSequence(pyblish.api.Extractor): "ignoreLayersTransparency", False ) - family_lowered = instance.data["family"].lower() mark_in = instance.context.data["sceneMarkIn"] mark_out = instance.context.data["sceneMarkOut"] @@ -76,11 +78,9 @@ class ExtractSequence(pyblish.api.Extractor): # Frame start/end may be stored as float frame_start = int(instance.data["frameStart"]) - frame_end = int(instance.data["frameEnd"]) # Handles are not stored per instance but on Context handle_start = instance.context.data["handleStart"] - handle_end = instance.context.data["handleEnd"] scene_bg_color = instance.context.data["sceneBgColor"] @@ -143,8 +143,9 @@ class ExtractSequence(pyblish.api.Extractor): ) # Fill tags and new families from project settings + instance_families = get_publish_instance_families(instance) tags = [] - if "review" in instance.data["families"]: + if "review" in instance_families: tags.append("review") # Sequence of one frame diff --git a/openpype/hosts/tvpaint/tvpaint_plugin/plugin_code/library.cpp b/openpype/hosts/tvpaint/tvpaint_plugin/plugin_code/library.cpp index 88106bc770..ec45a45123 100644 --- a/openpype/hosts/tvpaint/tvpaint_plugin/plugin_code/library.cpp +++ b/openpype/hosts/tvpaint/tvpaint_plugin/plugin_code/library.cpp @@ -573,56 +573,6 @@ void FAR PASCAL PI_Close( PIFilter* iFilter ) } -/**************************************************************************************/ -// we have something to do ! - -int FAR PASCAL PI_Parameters( PIFilter* iFilter, char* iArg ) -{ - if( !iArg ) - { - - // If the requester is not open, we open it. - if( Data.mReq == 0) - { - // Create empty requester because menu items are defined with - // `define_menu` callback - DWORD req = TVOpenFilterReqEx( - iFilter, - 185, - 20, - NULL, - NULL, - PIRF_STANDARD_REQ | PIRF_COLLAPSABLE_REQ, - FILTERREQ_NO_TBAR - ); - if( req == 0 ) - { - TVWarning( iFilter, TXT_REQUESTER_ERROR ); - return 0; - } - - - Data.mReq = req; - // This is a very simple requester, so we create it's content right here instead - // of waiting for the PICBREQ_OPEN message... - // Not recommended for more complex requesters. (see the other examples) - - // Sets the title of the requester. - TVSetReqTitle( iFilter, Data.mReq, TXT_REQUESTER ); - // Request to listen to ticks - TVGrabTicks(iFilter, req, PITICKS_FLAG_ON); - } - else - { - // If it is already open, we just put it on front of all other requesters. - TVReqToFront( iFilter, Data.mReq ); - } - } - - return 1; -} - - int newMenuItemsProcess(PIFilter* iFilter) { // Menu items defined with `define_menu` should be propagated. @@ -702,6 +652,62 @@ int newMenuItemsProcess(PIFilter* iFilter) { return 1; } + +/**************************************************************************************/ +// we have something to do ! + +int FAR PASCAL PI_Parameters( PIFilter* iFilter, char* iArg ) +{ + if( !iArg ) + { + + // If the requester is not open, we open it. + if( Data.mReq == 0) + { + // Create empty requester because menu items are defined with + // `define_menu` callback + DWORD req = TVOpenFilterReqEx( + iFilter, + 185, + 20, + NULL, + NULL, + PIRF_STANDARD_REQ | PIRF_COLLAPSABLE_REQ, + FILTERREQ_NO_TBAR + ); + if( req == 0 ) + { + TVWarning( iFilter, TXT_REQUESTER_ERROR ); + return 0; + } + + Data.mReq = req; + + // This is a very simple requester, so we create it's content right here instead + // of waiting for the PICBREQ_OPEN message... + // Not recommended for more complex requesters. (see the other examples) + + // Sets the title of the requester. + TVSetReqTitle( iFilter, Data.mReq, TXT_REQUESTER ); + // Request to listen to ticks + TVGrabTicks(iFilter, req, PITICKS_FLAG_ON); + + if ( Data.firstParams == true ) { + Data.firstParams = false; + } else { + newMenuItemsProcess(iFilter); + } + } + else + { + // If it is already open, we just put it on front of all other requesters. + TVReqToFront( iFilter, Data.mReq ); + } + } + + return 1; +} + /**************************************************************************************/ // something happened that needs our attention. // Global variable where current button up data are stored diff --git a/openpype/hosts/tvpaint/tvpaint_plugin/plugin_files/windows_x64/plugin/OpenPypePlugin.dll b/openpype/hosts/tvpaint/tvpaint_plugin/plugin_files/windows_x64/plugin/OpenPypePlugin.dll index 7081778bee..9c6e969e24 100644 Binary files a/openpype/hosts/tvpaint/tvpaint_plugin/plugin_files/windows_x64/plugin/OpenPypePlugin.dll and b/openpype/hosts/tvpaint/tvpaint_plugin/plugin_files/windows_x64/plugin/OpenPypePlugin.dll differ diff --git a/openpype/hosts/tvpaint/tvpaint_plugin/plugin_files/windows_x86/plugin/OpenPypePlugin.dll b/openpype/hosts/tvpaint/tvpaint_plugin/plugin_files/windows_x86/plugin/OpenPypePlugin.dll index 0f2afec245..b573476a21 100644 Binary files a/openpype/hosts/tvpaint/tvpaint_plugin/plugin_files/windows_x86/plugin/OpenPypePlugin.dll and b/openpype/hosts/tvpaint/tvpaint_plugin/plugin_files/windows_x86/plugin/OpenPypePlugin.dll differ diff --git a/openpype/hosts/unreal/hooks/pre_workfile_preparation.py b/openpype/hosts/unreal/hooks/pre_workfile_preparation.py index 202d7854f6..a635bd4cab 100644 --- a/openpype/hosts/unreal/hooks/pre_workfile_preparation.py +++ b/openpype/hosts/unreal/hooks/pre_workfile_preparation.py @@ -2,6 +2,8 @@ """Hook to launch Unreal and prepare projects.""" import os import copy +import shutil +import tempfile from pathlib import Path from qtpy import QtCore @@ -224,10 +226,24 @@ class UnrealPrelaunchHook(PreLaunchHook): project_file = project_path / unreal_project_filename if not project_file.is_file(): - self.exec_ue_project_gen(engine_version, - unreal_project_name, - engine_path, - project_path) + with tempfile.TemporaryDirectory() as temp_dir: + self.exec_ue_project_gen(engine_version, + unreal_project_name, + engine_path, + Path(temp_dir)) + try: + self.log.info(( + f"Moving from {temp_dir} to " + f"{project_path.as_posix()}" + )) + shutil.copytree( + temp_dir, project_path, dirs_exist_ok=True) + + except shutil.Error as e: + raise ApplicationLaunchFailed(( + f"{self.signature} Cannot copy directory {temp_dir} " + f"to {project_path.as_posix()} - {e}" + )) from e self.launch_context.env["AYON_UNREAL_VERSION"] = engine_version # Append project file to launch arguments diff --git a/openpype/hosts/unreal/plugins/publish/extract_uasset.py b/openpype/hosts/unreal/plugins/publish/extract_uasset.py index 48b62faa97..0dd7ff4a0d 100644 --- a/openpype/hosts/unreal/plugins/publish/extract_uasset.py +++ b/openpype/hosts/unreal/plugins/publish/extract_uasset.py @@ -19,9 +19,8 @@ class ExtractUAsset(publish.Extractor): "umap" if "umap" in instance.data.get("families") else "uasset") ar = unreal.AssetRegistryHelpers.get_asset_registry() - self.log.info("Performing extraction..") + self.log.debug("Performing extraction..") staging_dir = self.staging_dir(instance) - filename = f"{instance.name}.{extension}" members = instance.data.get("members", []) diff --git a/openpype/hosts/unreal/plugins/publish/validate_sequence_frames.py b/openpype/hosts/unreal/plugins/publish/validate_sequence_frames.py index 76bb25fac3..96485d5a2d 100644 --- a/openpype/hosts/unreal/plugins/publish/validate_sequence_frames.py +++ b/openpype/hosts/unreal/plugins/publish/validate_sequence_frames.py @@ -1,4 +1,6 @@ import clique +import os +import re import pyblish.api @@ -21,7 +23,19 @@ class ValidateSequenceFrames(pyblish.api.InstancePlugin): representations = instance.data.get("representations") for repr in representations: data = instance.data.get("assetEntity", {}).get("data", {}) - patterns = [clique.PATTERNS["frames"]] + repr_files = repr["files"] + if isinstance(repr_files, str): + continue + + ext = repr.get("ext") + if not ext: + _, ext = os.path.splitext(repr_files[0]) + elif not ext.startswith("."): + ext = ".{}".format(ext) + pattern = r"\D?(?P(?P0*)\d+){}$".format( + re.escape(ext)) + patterns = [pattern] + collections, remainder = clique.assemble( repr["files"], minimum_items=1, patterns=patterns) @@ -30,6 +44,10 @@ class ValidateSequenceFrames(pyblish.api.InstancePlugin): collection = collections[0] frames = list(collection.indexes) + if instance.data.get("slate"): + # Slate is not part of the frame range + frames = frames[1:] + current_range = (frames[0], frames[-1]) required_range = (data["clipIn"], data["clipOut"]) diff --git a/openpype/hosts/webpublisher/webserver_service/webpublish_routes.py b/openpype/hosts/webpublisher/webserver_service/webpublish_routes.py index e56f245d27..20d585e906 100644 --- a/openpype/hosts/webpublisher/webserver_service/webpublish_routes.py +++ b/openpype/hosts/webpublisher/webserver_service/webpublish_routes.py @@ -280,13 +280,14 @@ class BatchPublishEndpoint(WebpublishApiEndpoint): for key, value in add_args.items(): # Skip key values where value is None - if value is not None: - args.append("--{}".format(key)) - # Extend list into arguments (targets can be a list) - if isinstance(value, (tuple, list)): - args.extend(value) - else: - args.append(value) + if value is None: + continue + arg_key = "--{}".format(key) + if not isinstance(value, (tuple, list)): + value = [value] + + for item in value: + args += [arg_key, item] log.info("args:: {}".format(args)) if add_to_queue: diff --git a/openpype/lib/attribute_definitions.py b/openpype/lib/attribute_definitions.py index 6054d2a92a..a71d6cc72a 100644 --- a/openpype/lib/attribute_definitions.py +++ b/openpype/lib/attribute_definitions.py @@ -424,17 +424,25 @@ class TextDef(AbstractAttrDef): class EnumDef(AbstractAttrDef): - """Enumeration of single item from items. + """Enumeration of items. + + Enumeration of single item from items. Or list of items if multiselection + is enabled. Args: - items: Items definition that can be converted using - 'prepare_enum_items'. - default: Default value. Must be one key(value) from passed items. + items (Union[list[str], list[dict[str, Any]]): Items definition that + can be converted using 'prepare_enum_items'. + default (Optional[Any]): Default value. Must be one key(value) from + passed items or list of values for multiselection. + multiselection (Optional[bool]): If True, multiselection is allowed. + Output is list of selected items. """ type = "enum" - def __init__(self, key, items, default=None, **kwargs): + def __init__( + self, key, items, default=None, multiselection=False, **kwargs + ): if not items: raise ValueError(( "Empty 'items' value. {} must have" @@ -443,30 +451,44 @@ class EnumDef(AbstractAttrDef): items = self.prepare_enum_items(items) item_values = [item["value"] for item in items] - if default not in item_values: - for value in item_values: - default = value - break + item_values_set = set(item_values) + if multiselection: + if default is None: + default = [] + default = list(item_values_set.intersection(default)) + + elif default not in item_values: + default = next(iter(item_values), None) super(EnumDef, self).__init__(key, default=default, **kwargs) self.items = items - self._item_values = set(item_values) + self._item_values = item_values_set + self.multiselection = multiselection def __eq__(self, other): if not super(EnumDef, self).__eq__(other): return False - return self.items == other.items + return ( + self.items == other.items + and self.multiselection == other.multiselection + ) def convert_value(self, value): - if value in self._item_values: - return value - return self.default + if not self.multiselection: + if value in self._item_values: + return value + return self.default + + if value is None: + return copy.deepcopy(self.default) + return list(self._item_values.intersection(value)) def serialize(self): data = super(EnumDef, self).serialize() data["items"] = copy.deepcopy(self.items) + data["multiselection"] = self.multiselection return data @staticmethod diff --git a/openpype/lib/events.py b/openpype/lib/events.py index dca58fcf93..496b765a05 100644 --- a/openpype/lib/events.py +++ b/openpype/lib/events.py @@ -3,6 +3,7 @@ import os import re import copy import inspect +import collections import logging import weakref from uuid import uuid4 @@ -340,8 +341,8 @@ class EventSystem(object): event.emit() return event - def emit_event(self, event): - """Emit event object. + def _process_event(self, event): + """Process event topic and trigger callbacks. Args: event (Event): Prepared event with topic and data. @@ -356,6 +357,91 @@ class EventSystem(object): for callback in invalid_callbacks: self._registered_callbacks.remove(callback) + def emit_event(self, event): + """Emit event object. + + Args: + event (Event): Prepared event with topic and data. + """ + + self._process_event(event) + + +class QueuedEventSystem(EventSystem): + """Events are automatically processed in queue. + + If callback triggers another event, the event is not processed until + all callbacks of previous event are processed. + + Allows to implement custom event process loop by changing 'auto_execute'. + + Note: + This probably should be default behavior of 'EventSystem'. Changing it + now could cause problems in existing code. + + Args: + auto_execute (Optional[bool]): If 'True', events are processed + automatically. Custom loop calling 'process_next_event' + must be implemented when set to 'False'. + """ + + def __init__(self, auto_execute=True): + super(QueuedEventSystem, self).__init__() + self._event_queue = collections.deque() + self._current_event = None + self._auto_execute = auto_execute + + def __len__(self): + return self.count() + + def count(self): + """Get number of events in queue. + + Returns: + int: Number of events in queue. + """ + + return len(self._event_queue) + + def process_next_event(self): + """Process next event in queue. + + Should be used only if 'auto_execute' is set to 'False'. Only single + event is processed. + + Returns: + Union[Event, None]: Processed event. + """ + + if self._current_event is not None: + raise ValueError("An event is already in progress.") + + if not self._event_queue: + return None + event = self._event_queue.popleft() + self._current_event = event + self._process_event(event) + self._current_event = None + return event + + def emit_event(self, event): + """Emit event object. + + Args: + event (Event): Prepared event with topic and data. + """ + + if not self._auto_execute or self._current_event is not None: + self._event_queue.append(event) + return + + self._event_queue.append(event) + while self._event_queue: + event = self._event_queue.popleft() + self._current_event = event + self._process_event(event) + self._current_event = None + class GlobalEventSystem: """Event system living in global scope of process. diff --git a/openpype/lib/python_module_tools.py b/openpype/lib/python_module_tools.py index a10263f991..bedf19562d 100644 --- a/openpype/lib/python_module_tools.py +++ b/openpype/lib/python_module_tools.py @@ -270,8 +270,8 @@ def is_func_signature_supported(func, *args, **kwargs): Args: func (function): A function where the signature should be tested. - *args (tuple[Any]): Positional arguments for function signature. - **kwargs (dict[str, Any]): Keyword arguments for function signature. + *args (Any): Positional arguments for function signature. + **kwargs (Any): Keyword arguments for function signature. Returns: bool: Function can pass in arguments. diff --git a/openpype/lib/transcoding.py b/openpype/lib/transcoding.py index 2bae28786e..97c8dd41ab 100644 --- a/openpype/lib/transcoding.py +++ b/openpype/lib/transcoding.py @@ -315,6 +315,92 @@ def parse_oiio_xml_output(xml_string, logger=None): return output +def get_review_info_by_layer_name(channel_names): + """Get channels info grouped by layer name. + + Finds all layers in channel names and returns list of dictionaries with + information about channels in layer. + Example output (not real world example): + [ + { + "name": "Main", + "review_channels": { + "R": "Main.red", + "G": "Main.green", + "B": "Main.blue", + "A": None, + } + }, + { + "name": "Composed", + "review_channels": { + "R": "Composed.R", + "G": "Composed.G", + "B": "Composed.B", + "A": "Composed.A", + } + }, + ... + ] + + Args: + channel_names (list[str]): List of channel names. + + Returns: + list[dict]: List of channels information. + """ + + layer_names_order = [] + rgba_by_layer_name = collections.defaultdict(dict) + channels_by_layer_name = collections.defaultdict(dict) + + for channel_name in channel_names: + layer_name = "" + last_part = channel_name + if "." in channel_name: + layer_name, last_part = channel_name.rsplit(".", 1) + + channels_by_layer_name[layer_name][channel_name] = last_part + if last_part.lower() not in { + "r", "red", + "g", "green", + "b", "blue", + "a", "alpha" + }: + continue + + if layer_name not in layer_names_order: + layer_names_order.append(layer_name) + # R, G, B or A + channel = last_part[0].upper() + rgba_by_layer_name[layer_name][channel] = channel_name + + # Put empty layer to the beginning of the list + # - if input has R, G, B, A channels they should be used for review + if "" in layer_names_order: + layer_names_order.remove("") + layer_names_order.insert(0, "") + + output = [] + for layer_name in layer_names_order: + rgba_layer_info = rgba_by_layer_name[layer_name] + red = rgba_layer_info.get("R") + green = rgba_layer_info.get("G") + blue = rgba_layer_info.get("B") + if not red or not green or not blue: + continue + output.append({ + "name": layer_name, + "review_channels": { + "R": red, + "G": green, + "B": blue, + "A": rgba_layer_info.get("A"), + } + }) + return output + + def get_convert_rgb_channels(channel_names): """Get first available RGB(A) group from channels info. @@ -323,7 +409,7 @@ def get_convert_rgb_channels(channel_names): # Ideal situation channels_info: [ "R", "G", "B", "A" - } + ] ``` Result will be `("R", "G", "B", "A")` @@ -331,50 +417,60 @@ def get_convert_rgb_channels(channel_names): # Not ideal situation channels_info: [ "beauty.red", - "beuaty.green", + "beauty.green", "beauty.blue", "depth.Z" ] ``` Result will be `("beauty.red", "beauty.green", "beauty.blue", None)` + Args: + channel_names (list[str]): List of channel names. + Returns: - NoneType: There is not channel combination that matches RGB - combination. - tuple: Tuple of 4 channel names defying channel names for R, G, B, A - where A can be None. + Union[NoneType, tuple[str, str, str, Union[str, None]]]: Tuple of + 4 channel names defying channel names for R, G, B, A or None + if there is not any layer with RGB combination. """ - rgb_by_main_name = collections.defaultdict(dict) - main_name_order = [""] - for channel_name in channel_names: - name_parts = channel_name.split(".") - rgb_part = name_parts.pop(-1).lower() - main_name = ".".join(name_parts) - if rgb_part in ("r", "red"): - rgb_by_main_name[main_name]["R"] = channel_name - elif rgb_part in ("g", "green"): - rgb_by_main_name[main_name]["G"] = channel_name - elif rgb_part in ("b", "blue"): - rgb_by_main_name[main_name]["B"] = channel_name - elif rgb_part in ("a", "alpha"): - rgb_by_main_name[main_name]["A"] = channel_name - else: - continue - if main_name not in main_name_order: - main_name_order.append(main_name) - output = None - for main_name in main_name_order: - colors = rgb_by_main_name.get(main_name) or {} - red = colors.get("R") - green = colors.get("G") - blue = colors.get("B") - alpha = colors.get("A") - if red is not None and green is not None and blue is not None: - output = (red, green, blue, alpha) - break + channels_info = get_review_info_by_layer_name(channel_names) + for item in channels_info: + review_channels = item["review_channels"] + return ( + review_channels["R"], + review_channels["G"], + review_channels["B"], + review_channels["A"] + ) + return None - return output + +def get_review_layer_name(src_filepath): + """Find layer name that could be used for review. + + Args: + src_filepath (str): Path to input file. + + Returns: + Union[str, None]: Layer name of None. + """ + + ext = os.path.splitext(src_filepath)[-1].lower() + if ext != ".exr": + return None + + # Load info about file from oiio tool + input_info = get_oiio_info_for_input(src_filepath) + if not input_info: + return None + + channel_names = input_info["channelnames"] + channels_info = get_review_info_by_layer_name(channel_names) + for item in channels_info: + # Layer name can be '', when review channels are 'R', 'G', 'B' + # without layer + return item["name"] or None + return None def should_convert_for_ffmpeg(src_filepath): @@ -395,7 +491,7 @@ def should_convert_for_ffmpeg(src_filepath): if not is_oiio_supported(): return None - # Load info about info from oiio tool + # Load info about file from oiio tool input_info = get_oiio_info_for_input(src_filepath) if not input_info: return None @@ -724,7 +820,7 @@ def get_ffprobe_data(path_to_file, logger=None): """ if not logger: logger = logging.getLogger(__name__) - logger.info( + logger.debug( "Getting information about input \"{}\".".format(path_to_file) ) ffprobe_args = get_ffmpeg_tool_args("ffprobe") diff --git a/openpype/modules/base.py b/openpype/modules/base.py index 9b3637c48a..a3c21718b9 100644 --- a/openpype/modules/base.py +++ b/openpype/modules/base.py @@ -59,6 +59,14 @@ IGNORED_DEFAULT_FILENAMES = ( "example_addons", "default_modules", ) +# Modules that won't be loaded in AYON mode from "./openpype/modules" +# - the same modules are ignored in "./server_addon/create_ayon_addons.py" +IGNORED_FILENAMES_IN_AYON = { + "ftrack", + "shotgrid", + "sync_server", + "slack", +} # Inherit from `object` for Python 2 hosts @@ -373,10 +381,12 @@ def _load_ayon_addons(openpype_modules, modules_key, log): addons_info = _get_ayon_addons_information() if not addons_info: return v3_addons_to_skip - addons_dir = os.path.join( - appdirs.user_data_dir("AYON", "Ynput"), - "addons" - ) + addons_dir = os.environ.get("AYON_ADDONS_DIR") + if not addons_dir: + addons_dir = os.path.join( + appdirs.user_data_dir("AYON", "Ynput"), + "addons" + ) if not os.path.exists(addons_dir): log.warning("Addons directory does not exists. Path \"{}\"".format( addons_dir @@ -390,9 +400,9 @@ def _load_ayon_addons(openpype_modules, modules_key, log): folder_name = "{}_{}".format(addon_name, addon_version) addon_dir = os.path.join(addons_dir, folder_name) if not os.path.exists(addon_dir): - log.warning(( - "Directory for addon {} {} does not exists. Path \"{}\"" - ).format(addon_name, addon_version, addon_dir)) + log.debug(( + "No localized client code found for addon {} {}." + ).format(addon_name, addon_version)) continue sys.path.insert(0, addon_dir) @@ -481,6 +491,10 @@ def _load_modules(): is_in_current_dir = dirpath == current_dir is_in_host_dir = dirpath == hosts_dir + ignored_current_dir_filenames = set(IGNORED_DEFAULT_FILENAMES) + if AYON_SERVER_ENABLED: + ignored_current_dir_filenames |= IGNORED_FILENAMES_IN_AYON + for filename in os.listdir(dirpath): # Ignore filenames if filename in IGNORED_FILENAMES: @@ -488,7 +502,7 @@ def _load_modules(): if ( is_in_current_dir - and filename in IGNORED_DEFAULT_FILENAMES + and filename in ignored_current_dir_filenames ): continue diff --git a/openpype/modules/deadline/plugins/publish/collect_deadline_server_from_instance.py b/openpype/modules/deadline/plugins/publish/collect_deadline_server_from_instance.py index eadfc3c83e..9b4f89c129 100644 --- a/openpype/modules/deadline/plugins/publish/collect_deadline_server_from_instance.py +++ b/openpype/modules/deadline/plugins/publish/collect_deadline_server_from_instance.py @@ -8,6 +8,7 @@ attribute or using default server if that attribute doesn't exists. from maya import cmds import pyblish.api +from openpype.pipeline.publish import KnownPublishError class CollectDeadlineServerFromInstance(pyblish.api.InstancePlugin): @@ -23,7 +24,7 @@ class CollectDeadlineServerFromInstance(pyblish.api.InstancePlugin): instance.data["deadlineUrl"] = self._collect_deadline_url(instance) instance.data["deadlineUrl"] = \ instance.data["deadlineUrl"].strip().rstrip("/") - self.log.info( + self.log.debug( "Using {} for submission.".format(instance.data["deadlineUrl"])) def _collect_deadline_url(self, render_instance): @@ -81,13 +82,14 @@ class CollectDeadlineServerFromInstance(pyblish.api.InstancePlugin): if k in default_servers } - msg = ( - "\"{}\" server on instance is not enabled in project settings." - " Enabled project servers:\n{}".format( - instance_server, project_enabled_servers + if instance_server not in project_enabled_servers: + msg = ( + "\"{}\" server on instance is not enabled in project settings." + " Enabled project servers:\n{}".format( + instance_server, project_enabled_servers + ) ) - ) - assert instance_server in project_enabled_servers, msg + raise KnownPublishError(msg) self.log.debug("Using project approved server.") return project_enabled_servers[instance_server] diff --git a/openpype/modules/deadline/plugins/publish/help/validate_deadline_pools.xml b/openpype/modules/deadline/plugins/publish/help/validate_deadline_pools.xml index 0e7d72910e..aa21df3734 100644 --- a/openpype/modules/deadline/plugins/publish/help/validate_deadline_pools.xml +++ b/openpype/modules/deadline/plugins/publish/help/validate_deadline_pools.xml @@ -1,31 +1,31 @@ - Scene setting + Deadline Pools - ## Invalid Deadline pools found +## Invalid Deadline pools found - Configured pools don't match what is set in Deadline. +Configured pools don't match available pools in Deadline. - {invalid_value_str} +### How to repair? - ### How to repair? +If your instance had deadline pools set on creation, remove or +change them. - If your instance had deadline pools set on creation, remove or - change them. +In other cases inform admin to change them in Settings. - In other cases inform admin to change them in Settings. +Available deadline pools: + +{pools_str} - Available deadline pools {pools_str}. - ### __Detailed Info__ +### __Detailed Info__ - This error is shown when deadline pool is not on Deadline anymore. It - could happen in case of republish old workfile which was created with - previous deadline pools, - or someone changed pools on Deadline side, but didn't modify Openpype - Settings. +This error is shown when a configured pool is not available on Deadline. It +can happen when publishing old workfiles which were created with previous +deadline pools, or someone changed the available pools in Deadline, +but didn't modify Openpype Settings to match the changes. \ No newline at end of file diff --git a/openpype/modules/deadline/plugins/publish/submit_celaction_deadline.py b/openpype/modules/deadline/plugins/publish/submit_celaction_deadline.py index ee28612b44..47a0a25755 100644 --- a/openpype/modules/deadline/plugins/publish/submit_celaction_deadline.py +++ b/openpype/modules/deadline/plugins/publish/submit_celaction_deadline.py @@ -27,7 +27,7 @@ class CelactionSubmitDeadline(pyblish.api.InstancePlugin): deadline_job_delay = "00:00:08:00" def process(self, instance): - instance.data["toBeRenderedOn"] = "deadline" + context = instance.context # get default deadline webservice url from deadline module @@ -183,10 +183,10 @@ class CelactionSubmitDeadline(pyblish.api.InstancePlugin): } plugin = payload["JobInfo"]["Plugin"] - self.log.info("using render plugin : {}".format(plugin)) + self.log.debug("using render plugin : {}".format(plugin)) - self.log.info("Submitting..") - self.log.info(json.dumps(payload, indent=4, sort_keys=True)) + self.log.debug("Submitting..") + self.log.debug(json.dumps(payload, indent=4, sort_keys=True)) # adding expectied files to instance.data self.expected_files(instance, render_path) diff --git a/openpype/modules/deadline/plugins/publish/submit_fusion_deadline.py b/openpype/modules/deadline/plugins/publish/submit_fusion_deadline.py index a48596c6bf..70aa12956d 100644 --- a/openpype/modules/deadline/plugins/publish/submit_fusion_deadline.py +++ b/openpype/modules/deadline/plugins/publish/submit_fusion_deadline.py @@ -233,8 +233,8 @@ class FusionSubmitDeadline( ) for index, key in enumerate(environment) }) - self.log.info("Submitting..") - self.log.info(json.dumps(payload, indent=4, sort_keys=True)) + self.log.debug("Submitting..") + self.log.debug(json.dumps(payload, indent=4, sort_keys=True)) # E.g. http://192.168.0.1:8082/api/jobs url = "{}/api/jobs".format(deadline_url) diff --git a/openpype/modules/deadline/plugins/publish/submit_harmony_deadline.py b/openpype/modules/deadline/plugins/publish/submit_harmony_deadline.py index 2c37268f04..17e672334c 100644 --- a/openpype/modules/deadline/plugins/publish/submit_harmony_deadline.py +++ b/openpype/modules/deadline/plugins/publish/submit_harmony_deadline.py @@ -265,7 +265,7 @@ class HarmonySubmitDeadline( job_info.SecondaryPool = self._instance.data.get("secondaryPool") job_info.ChunkSize = self.chunk_size batch_name = os.path.basename(self._instance.data["source"]) - if is_in_tests: + if is_in_tests(): batch_name += datetime.now().strftime("%d%m%Y%H%M%S") job_info.BatchName = batch_name job_info.Department = self.department @@ -369,7 +369,7 @@ class HarmonySubmitDeadline( # rendering, we need to unzip it. published_scene = Path( self.from_published_scene(False)) - self.log.info(f"Processing {published_scene.as_posix()}") + self.log.debug(f"Processing {published_scene.as_posix()}") xstage_path = self._unzip_scene_file(published_scene) render_path = xstage_path.parent / "renders" diff --git a/openpype/modules/deadline/plugins/publish/submit_houdini_remote_publish.py b/openpype/modules/deadline/plugins/publish/submit_houdini_remote_publish.py index 68aa653804..39c0c3afe4 100644 --- a/openpype/modules/deadline/plugins/publish/submit_houdini_remote_publish.py +++ b/openpype/modules/deadline/plugins/publish/submit_houdini_remote_publish.py @@ -162,7 +162,7 @@ class HoudiniSubmitPublishDeadline(pyblish.api.ContextPlugin): ) # Submit - self.log.info("Submitting..") + self.log.debug("Submitting..") self.log.debug(json.dumps(payload, indent=4, sort_keys=True)) # E.g. http://192.168.0.1:8082/api/jobs diff --git a/openpype/modules/deadline/plugins/publish/submit_houdini_render_deadline.py b/openpype/modules/deadline/plugins/publish/submit_houdini_render_deadline.py index 108c377078..8f21a920be 100644 --- a/openpype/modules/deadline/plugins/publish/submit_houdini_render_deadline.py +++ b/openpype/modules/deadline/plugins/publish/submit_houdini_render_deadline.py @@ -141,4 +141,3 @@ class HoudiniSubmitDeadline(abstract_submit_deadline.AbstractSubmitDeadline): # Store output dir for unified publisher (filesequence) output_dir = os.path.dirname(instance.data["files"][0]) instance.data["outputDir"] = output_dir - instance.data["toBeRenderedOn"] = "deadline" diff --git a/openpype/modules/deadline/plugins/publish/submit_max_deadline.py b/openpype/modules/deadline/plugins/publish/submit_max_deadline.py index d8725e853c..63c6e4a0c7 100644 --- a/openpype/modules/deadline/plugins/publish/submit_max_deadline.py +++ b/openpype/modules/deadline/plugins/publish/submit_max_deadline.py @@ -176,7 +176,6 @@ class MaxSubmitDeadline(abstract_submit_deadline.AbstractSubmitDeadline, first_file = next(self._iter_expected_files(files)) output_dir = os.path.dirname(first_file) instance.data["outputDir"] = output_dir - instance.data["toBeRenderedOn"] = "deadline" filename = os.path.basename(filepath) @@ -238,7 +237,10 @@ class MaxSubmitDeadline(abstract_submit_deadline.AbstractSubmitDeadline, if renderer == "Redshift_Renderer": plugin_data["redshift_SeparateAovFiles"] = instance.data.get( "separateAovFiles") - + if instance.data["cameras"]: + plugin_info["Camera0"] = None + plugin_info["Camera"] = instance.data["cameras"][0] + plugin_info["Camera1"] = instance.data["cameras"][0] self.log.debug("plugin data:{}".format(plugin_data)) plugin_info.update(plugin_data) diff --git a/openpype/modules/deadline/plugins/publish/submit_maya_deadline.py b/openpype/modules/deadline/plugins/publish/submit_maya_deadline.py index 75d24b28f0..7775191b12 100644 --- a/openpype/modules/deadline/plugins/publish/submit_maya_deadline.py +++ b/openpype/modules/deadline/plugins/publish/submit_maya_deadline.py @@ -290,7 +290,6 @@ class MayaSubmitDeadline(abstract_submit_deadline.AbstractSubmitDeadline, def process_submission(self): instance = self._instance - context = instance.context filepath = self.scene_path # publish if `use_publish` else workfile @@ -300,20 +299,17 @@ class MayaSubmitDeadline(abstract_submit_deadline.AbstractSubmitDeadline, first_file = next(iter_expected_files(expected_files)) output_dir = os.path.dirname(first_file) instance.data["outputDir"] = output_dir - instance.data["toBeRenderedOn"] = "deadline" # Patch workfile (only when use_published is enabled) if self.use_published: self._patch_workfile() # Gather needed data ------------------------------------------------ - workspace = context.data["workspaceDir"] - default_render_file = instance.context.data.get('project_settings')\ - .get('maya')\ - .get('RenderSettings')\ - .get('default_render_image_folder') filename = os.path.basename(filepath) - dirname = os.path.join(workspace, default_render_file) + dirname = os.path.join( + cmds.workspace(query=True, rootDirectory=True), + cmds.workspace(fileRuleEntry="images") + ) # Fill in common data to payload ------------------------------------ # TODO: Replace these with collected data from CollectRender @@ -335,12 +331,6 @@ class MayaSubmitDeadline(abstract_submit_deadline.AbstractSubmitDeadline, payload = self._get_vray_render_payload(payload_data) - elif "assscene" in instance.data["families"]: - self.log.debug("Submitting Arnold .ass standalone render..") - ass_export_payload = self._get_arnold_export_payload(payload_data) - export_job = self.submit(ass_export_payload) - - payload = self._get_arnold_render_payload(payload_data) else: self.log.debug("Submitting MayaBatch render..") payload = self._get_maya_payload(payload_data) @@ -434,7 +424,7 @@ class MayaSubmitDeadline(abstract_submit_deadline.AbstractSubmitDeadline, new_job_info.update(tiles_data["JobInfo"]) new_plugin_info.update(tiles_data["PluginInfo"]) - self.log.info("hashing {} - {}".format(file_index, file)) + self.log.debug("hashing {} - {}".format(file_index, file)) job_hash = hashlib.sha256( ("{}_{}".format(file_index, file)).encode("utf-8")) @@ -450,7 +440,7 @@ class MayaSubmitDeadline(abstract_submit_deadline.AbstractSubmitDeadline, ) file_index += 1 - self.log.info( + self.log.debug( "Submitting tile job(s) [{}] ...".format(len(frame_payloads))) # Submit frame tile jobs @@ -560,7 +550,7 @@ class MayaSubmitDeadline(abstract_submit_deadline.AbstractSubmitDeadline, assembly_job_ids = [] num_assemblies = len(assembly_payloads) for i, payload in enumerate(assembly_payloads): - self.log.info( + self.log.debug( "submitting assembly job {} of {}".format(i + 1, num_assemblies) ) @@ -636,53 +626,6 @@ class MayaSubmitDeadline(abstract_submit_deadline.AbstractSubmitDeadline, return job_info, attr.asdict(plugin_info) - def _get_arnold_export_payload(self, data): - - try: - from openpype.scripts import export_maya_ass_job - except Exception: - raise AssertionError( - "Expected module 'export_maya_ass_job' to be available") - - module_path = export_maya_ass_job.__file__ - if module_path.endswith(".pyc"): - module_path = module_path[: -len(".pyc")] + ".py" - - script = os.path.normpath(module_path) - - job_info = copy.deepcopy(self.job_info) - job_info.Name = self._job_info_label("Export") - - # Force a single frame Python job - job_info.Plugin = "Python" - job_info.Frames = 1 - - renderlayer = self._instance.data["setMembers"] - - # add required env vars for the export script - envs = { - "AVALON_APP_NAME": os.environ.get("AVALON_APP_NAME"), - "OPENPYPE_ASS_EXPORT_RENDER_LAYER": renderlayer, - "OPENPYPE_ASS_EXPORT_SCENE_FILE": self.scene_path, - "OPENPYPE_ASS_EXPORT_OUTPUT": job_info.OutputFilename[0], - "OPENPYPE_ASS_EXPORT_START": int(self._instance.data["frameStartHandle"]), # noqa - "OPENPYPE_ASS_EXPORT_END": int(self._instance.data["frameEndHandle"]), # noqa - "OPENPYPE_ASS_EXPORT_STEP": 1 - } - for key, value in envs.items(): - if not value: - continue - job_info.EnvironmentKeyValue[key] = value - - plugin_info = PythonPluginInfo( - ScriptFile=script, - Version="3.6", - Arguments="", - SingleFrameOnly="True" - ) - - return job_info, attr.asdict(plugin_info) - def _get_vray_render_payload(self, data): # Job Info diff --git a/openpype/modules/deadline/plugins/publish/submit_nuke_deadline.py b/openpype/modules/deadline/plugins/publish/submit_nuke_deadline.py index cfdeb4968b..0295c2b760 100644 --- a/openpype/modules/deadline/plugins/publish/submit_nuke_deadline.py +++ b/openpype/modules/deadline/plugins/publish/submit_nuke_deadline.py @@ -97,7 +97,6 @@ class NukeSubmitDeadline(pyblish.api.InstancePlugin, instance.data["suspend_publish"] = instance.data["attributeValues"][ "suspend_publish"] - instance.data["toBeRenderedOn"] = "deadline" families = instance.data["families"] node = instance.data["transientData"]["node"] @@ -244,7 +243,7 @@ class NukeSubmitDeadline(pyblish.api.InstancePlugin, # resolve any limit groups limit_groups = self.get_limit_groups() - self.log.info("Limit groups: `{}`".format(limit_groups)) + self.log.debug("Limit groups: `{}`".format(limit_groups)) payload = { "JobInfo": { @@ -387,10 +386,10 @@ class NukeSubmitDeadline(pyblish.api.InstancePlugin, }) plugin = payload["JobInfo"]["Plugin"] - self.log.info("using render plugin : {}".format(plugin)) + self.log.debug("using render plugin : {}".format(plugin)) - self.log.info("Submitting..") - self.log.info(json.dumps(payload, indent=4, sort_keys=True)) + self.log.debug("Submitting..") + self.log.debug(json.dumps(payload, indent=4, sort_keys=True)) # adding expectied files to instance.data self.expected_files( diff --git a/openpype/modules/deadline/plugins/publish/submit_publish_job.py b/openpype/modules/deadline/plugins/publish/submit_publish_job.py index bf4411ef43..909975f7ab 100644 --- a/openpype/modules/deadline/plugins/publish/submit_publish_job.py +++ b/openpype/modules/deadline/plugins/publish/submit_publish_job.py @@ -317,7 +317,7 @@ class ProcessSubmittedJobOnFarm(pyblish.api.InstancePlugin, # remove secondary pool payload["JobInfo"].pop("SecondaryPool", None) - self.log.info("Submitting Deadline job ...") + self.log.debug("Submitting Deadline publish job ...") url = "{}/api/jobs".format(self.deadline_url) response = requests.post(url, json=payload, timeout=10) @@ -345,6 +345,151 @@ class ProcessSubmittedJobOnFarm(pyblish.api.InstancePlugin, self.log.debug("Skipping local instance.") return + data = instance.data.copy() + context = instance.context + self.context = context + self.anatomy = instance.context.data["anatomy"] + + asset = data.get("asset") or context.data["asset"] + subset = data.get("subset") + + start = instance.data.get("frameStart") + if start is None: + start = context.data["frameStart"] + + end = instance.data.get("frameEnd") + if end is None: + end = context.data["frameEnd"] + + handle_start = instance.data.get("handleStart") + if handle_start is None: + handle_start = context.data["handleStart"] + + handle_end = instance.data.get("handleEnd") + if handle_end is None: + handle_end = context.data["handleEnd"] + + fps = instance.data.get("fps") + if fps is None: + fps = context.data["fps"] + + if data.get("extendFrames", False): + start, end = self._extend_frames( + asset, + subset, + start, + end, + data["overrideExistingFrame"]) + + try: + source = data["source"] + except KeyError: + source = context.data["currentFile"] + + success, rootless_path = ( + self.anatomy.find_root_template_from_path(source) + ) + if success: + source = rootless_path + + else: + # `rootless_path` is not set to `source` if none of roots match + self.log.warning(( + "Could not find root path for remapping \"{}\"." + " This may cause issues." + ).format(source)) + + family = "render" + if ("prerender" in instance.data["families"] or + "prerender.farm" in instance.data["families"]): + family = "prerender" + families = [family] + + # pass review to families if marked as review + do_not_add_review = False + if data.get("review"): + families.append("review") + elif data.get("review") is False: + self.log.debug("Instance has review explicitly disabled.") + do_not_add_review = True + + instance_skeleton_data = { + "family": family, + "subset": subset, + "families": families, + "asset": asset, + "frameStart": start, + "frameEnd": end, + "handleStart": handle_start, + "handleEnd": handle_end, + "frameStartHandle": start - handle_start, + "frameEndHandle": end + handle_end, + "comment": instance.data["comment"], + "fps": fps, + "source": source, + "extendFrames": data.get("extendFrames"), + "overrideExistingFrame": data.get("overrideExistingFrame"), + "pixelAspect": data.get("pixelAspect", 1), + "resolutionWidth": data.get("resolutionWidth", 1920), + "resolutionHeight": data.get("resolutionHeight", 1080), + "multipartExr": data.get("multipartExr", False), + "jobBatchName": data.get("jobBatchName", ""), + "useSequenceForReview": data.get("useSequenceForReview", True), + # map inputVersions `ObjectId` -> `str` so json supports it + "inputVersions": list(map(str, data.get("inputVersions", []))), + "colorspace": instance.data.get("colorspace"), + "stagingDir_persistent": instance.data.get( + "stagingDir_persistent", False + ) + } + + # skip locking version if we are creating v01 + instance_version = instance.data.get("version") # take this if exists + if instance_version != 1: + instance_skeleton_data["version"] = instance_version + + # transfer specific families from original instance to new render + for item in self.families_transfer: + if item in instance.data.get("families", []): + instance_skeleton_data["families"] += [item] + + # transfer specific properties from original instance based on + # mapping dictionary `instance_transfer` + for key, values in self.instance_transfer.items(): + if key in instance.data.get("families", []): + for v in values: + instance_skeleton_data[v] = instance.data.get(v) + + # look into instance data if representations are not having any + # which are having tag `publish_on_farm` and include them + for repre in instance.data.get("representations", []): + staging_dir = repre.get("stagingDir") + if staging_dir: + success, rootless_staging_dir = ( + self.anatomy.find_root_template_from_path( + staging_dir + ) + ) + if success: + repre["stagingDir"] = rootless_staging_dir + else: + self.log.warning(( + "Could not find root path for remapping \"{}\"." + " This may cause issues on farm." + ).format(staging_dir)) + repre["stagingDir"] = staging_dir + + if "publish_on_farm" in repre.get("tags"): + # create representations attribute of not there + if "representations" not in instance_skeleton_data.keys(): + instance_skeleton_data["representations"] = [] + + instance_skeleton_data["representations"].append(repre) + + instances = None + assert data.get("expectedFiles"), ("Submission from old Pype version" + " - missing expectedFiles") + anatomy = instance.context.data["anatomy"] instance_skeleton_data = create_skeleton_instance( @@ -454,7 +599,7 @@ class ProcessSubmittedJobOnFarm(pyblish.api.InstancePlugin, import getpass render_job = {} - self.log.info("Faking job data ...") + self.log.debug("Faking job data ...") render_job["Props"] = {} # Render job doesn't exist because we do not have prior submission. # We still use data from it so lets fake it. diff --git a/openpype/modules/deadline/plugins/publish/validate_deadline_connection.py b/openpype/modules/deadline/plugins/publish/validate_deadline_connection.py index a30401e7dc..a7b300beff 100644 --- a/openpype/modules/deadline/plugins/publish/validate_deadline_connection.py +++ b/openpype/modules/deadline/plugins/publish/validate_deadline_connection.py @@ -1,8 +1,7 @@ -import os -import requests - import pyblish.api +from openpype_modules.deadline.abstract_submit_deadline import requests_get + class ValidateDeadlineConnection(pyblish.api.InstancePlugin): """Validate Deadline Web Service is running""" @@ -12,34 +11,25 @@ class ValidateDeadlineConnection(pyblish.api.InstancePlugin): hosts = ["maya", "nuke"] families = ["renderlayer", "render"] + # cache + responses = {} + def process(self, instance): # get default deadline webservice url from deadline module deadline_url = instance.context.data["defaultDeadline"] # if custom one is set in instance, use that if instance.data.get("deadlineUrl"): deadline_url = instance.data.get("deadlineUrl") - self.log.info( - "We have deadline URL on instance {}".format( - deadline_url)) + self.log.debug( + "We have deadline URL on instance {}".format(deadline_url) + ) assert deadline_url, "Requires Deadline Webservice URL" - # Check response - response = self._requests_get(deadline_url) + if deadline_url not in self.responses: + self.responses[deadline_url] = requests_get(deadline_url) + + response = self.responses[deadline_url] assert response.ok, "Response must be ok" assert response.text.startswith("Deadline Web Service "), ( "Web service did not respond with 'Deadline Web Service'" ) - - def _requests_get(self, *args, **kwargs): - """ Wrapper for requests, disabling SSL certificate validation if - DONT_VERIFY_SSL environment variable is found. This is useful when - Deadline or Muster server are running with self-signed certificates - and their certificate is not added to trusted certificates on - client machines. - - WARNING: disabling SSL certificate validation is defeating one line - of defense SSL is providing and it is not recommended. - """ - if 'verify' not in kwargs: - kwargs['verify'] = False if os.getenv("OPENPYPE_DONT_VERIFY_SSL", True) else True # noqa - return requests.get(*args, **kwargs) diff --git a/openpype/modules/deadline/plugins/publish/validate_deadline_pools.py b/openpype/modules/deadline/plugins/publish/validate_deadline_pools.py index 594f0ef866..949caff7d8 100644 --- a/openpype/modules/deadline/plugins/publish/validate_deadline_pools.py +++ b/openpype/modules/deadline/plugins/publish/validate_deadline_pools.py @@ -25,33 +25,58 @@ class ValidateDeadlinePools(OptionalPyblishPluginMixin, "maxrender"] optional = True + # cache + pools_per_url = {} + def process(self, instance): + if not self.is_active(instance.data): + return + if not instance.data.get("farm"): self.log.debug("Skipping local instance.") return - # get default deadline webservice url from deadline module - deadline_url = instance.context.data["defaultDeadline"] - self.log.info("deadline_url::{}".format(deadline_url)) - pools = DeadlineModule.get_deadline_pools(deadline_url, log=self.log) - self.log.info("pools::{}".format(pools)) - - formatting_data = { - "pools_str": ",".join(pools) - } + deadline_url = self.get_deadline_url(instance) + pools = self.get_pools(deadline_url) + invalid_pools = {} primary_pool = instance.data.get("primaryPool") if primary_pool and primary_pool not in pools: - msg = "Configured primary '{}' not present on Deadline".format( - instance.data["primaryPool"]) - formatting_data["invalid_value_str"] = msg - raise PublishXmlValidationError(self, msg, - formatting_data=formatting_data) + invalid_pools["primary"] = primary_pool secondary_pool = instance.data.get("secondaryPool") if secondary_pool and secondary_pool not in pools: - msg = "Configured secondary '{}' not present on Deadline".format( - instance.data["secondaryPool"]) - formatting_data["invalid_value_str"] = msg - raise PublishXmlValidationError(self, msg, - formatting_data=formatting_data) + invalid_pools["secondary"] = secondary_pool + + if invalid_pools: + message = "\n".join( + "{} pool '{}' not available on Deadline".format(key.title(), + pool) + for key, pool in invalid_pools.items() + ) + raise PublishXmlValidationError( + plugin=self, + message=message, + formatting_data={"pools_str": ", ".join(pools)} + ) + + def get_deadline_url(self, instance): + # get default deadline webservice url from deadline module + deadline_url = instance.context.data["defaultDeadline"] + if instance.data.get("deadlineUrl"): + # if custom one is set in instance, use that + deadline_url = instance.data.get("deadlineUrl") + return deadline_url + + def get_pools(self, deadline_url): + if deadline_url not in self.pools_per_url: + self.log.debug( + "Querying available pools for Deadline url: {}".format( + deadline_url) + ) + pools = DeadlineModule.get_deadline_pools(deadline_url, + log=self.log) + self.log.info("Available pools: {}".format(pools)) + self.pools_per_url[deadline_url] = pools + + return self.pools_per_url[deadline_url] diff --git a/openpype/modules/deadline/plugins/publish/validate_expected_and_rendered_files.py b/openpype/modules/deadline/plugins/publish/validate_expected_and_rendered_files.py index 9f1f7bc518..5d37e7357e 100644 --- a/openpype/modules/deadline/plugins/publish/validate_expected_and_rendered_files.py +++ b/openpype/modules/deadline/plugins/publish/validate_expected_and_rendered_files.py @@ -70,7 +70,10 @@ class ValidateExpectedFiles(pyblish.api.InstancePlugin): # Update the representation expected files self.log.info("Update range from actual job range " "to frame list: {}".format(frame_list)) - repre["files"] = sorted(job_expected_files) + # single item files must be string not list + repre["files"] = (sorted(job_expected_files) + if len(job_expected_files) > 1 else + list(job_expected_files)[0]) # Update the expected files expected_files = job_expected_files diff --git a/openpype/modules/deadline/repository/custom/plugins/Ayon/Ayon.py b/openpype/modules/deadline/repository/custom/plugins/Ayon/Ayon.py index 1544acc2a4..a29acf9823 100644 --- a/openpype/modules/deadline/repository/custom/plugins/Ayon/Ayon.py +++ b/openpype/modules/deadline/repository/custom/plugins/Ayon/Ayon.py @@ -91,7 +91,13 @@ class AyonDeadlinePlugin(DeadlinePlugin): # clean '\ ' for MacOS pasting if platform.system().lower() == "darwin": exe_list = exe_list.replace("\\ ", " ") - exe = FileUtils.SearchFileList(exe_list) + + expanded_paths = [] + for path in exe_list.split(";"): + if path.startswith("~"): + path = os.path.expanduser(path) + expanded_paths.append(path) + exe = FileUtils.SearchFileList(";".join(expanded_paths)) if exe == "": self.FailRender( diff --git a/openpype/modules/deadline/repository/custom/plugins/GlobalJobPreLoad.py b/openpype/modules/deadline/repository/custom/plugins/GlobalJobPreLoad.py index 5f7e1f1032..e9b81369ca 100644 --- a/openpype/modules/deadline/repository/custom/plugins/GlobalJobPreLoad.py +++ b/openpype/modules/deadline/repository/custom/plugins/GlobalJobPreLoad.py @@ -385,6 +385,12 @@ def inject_openpype_environment(deadlinePlugin): for key, value in contents.items(): deadlinePlugin.SetProcessEnvironmentVariable(key, value) + if "PATH" in contents: + # Set os.environ[PATH] so studio settings' path entries + # can be used to define search path for executables. + print(f">>> Setting 'PATH' Environment to: {contents['PATH']}") + os.environ["PATH"] = contents["PATH"] + script_url = job.GetJobPluginInfoKeyValue("ScriptFilename") if script_url: script_url = script_url.format(**contents).replace("\\", "/") @@ -509,6 +515,12 @@ def inject_ayon_environment(deadlinePlugin): for key, value in contents.items(): deadlinePlugin.SetProcessEnvironmentVariable(key, value) + if "PATH" in contents: + # Set os.environ[PATH] so studio settings' path entries + # can be used to define search path for executables. + print(f">>> Setting 'PATH' Environment to: {contents['PATH']}") + os.environ["PATH"] = contents["PATH"] + script_url = job.GetJobPluginInfoKeyValue("ScriptFilename") if script_url: script_url = script_url.format(**contents).replace("\\", "/") @@ -547,7 +559,14 @@ def get_ayon_executable(): # clean '\ ' for MacOS pasting if platform.system().lower() == "darwin": exe_list = exe_list.replace("\\ ", " ") - return exe_list + + # Expand user paths + expanded_paths = [] + for path in exe_list.split(";"): + if path.startswith("~"): + path = os.path.expanduser(path) + expanded_paths.append(path) + return ";".join(expanded_paths) def inject_render_job_id(deadlinePlugin): diff --git a/openpype/modules/deadline/repository/custom/plugins/HarmonyOpenPype/HarmonyOpenPype.param b/openpype/modules/deadline/repository/custom/plugins/HarmonyOpenPype/HarmonyOpenPype.param index ff2949766c..43a54a464e 100644 --- a/openpype/modules/deadline/repository/custom/plugins/HarmonyOpenPype/HarmonyOpenPype.param +++ b/openpype/modules/deadline/repository/custom/plugins/HarmonyOpenPype/HarmonyOpenPype.param @@ -77,4 +77,22 @@ CategoryOrder=0 Index=4 Label=Harmony 20 Render Executable Description=The path to the Harmony Render executable file used for rendering. Enter alternative paths on separate lines. -Default=c:\Program Files (x86)\Toon Boom Animation\Toon Boom Harmony 20 Premium\win64\bin\HarmonyPremium.exe;/Applications/Toon Boom Harmony 20 Premium/Harmony Premium.app/Contents/MacOS/Harmony Premium;/usr/local/ToonBoomAnimation/harmonyPremium_20/lnx86_64/bin/HarmonyPremium \ No newline at end of file +Default=c:\Program Files (x86)\Toon Boom Animation\Toon Boom Harmony 20 Premium\win64\bin\HarmonyPremium.exe;/Applications/Toon Boom Harmony 20 Premium/Harmony Premium.app/Contents/MacOS/Harmony Premium;/usr/local/ToonBoomAnimation/harmonyPremium_20/lnx86_64/bin/HarmonyPremium + +[Harmony_RenderExecutable_21] +Type=multilinemultifilename +Category=Render Executables +CategoryOrder=0 +Index=4 +Label=Harmony 21 Render Executable +Description=The path to the Harmony Render executable file used for rendering. Enter alternative paths on separate lines. +Default=c:\Program Files (x86)\Toon Boom Animation\Toon Boom Harmony 21 Premium\win64\bin\HarmonyPremium.exe;/Applications/Toon Boom Harmony 21 Premium/Harmony Premium.app/Contents/MacOS/Harmony Premium;/usr/local/ToonBoomAnimation/harmonyPremium_21/lnx86_64/bin/HarmonyPremium + +[Harmony_RenderExecutable_22] +Type=multilinemultifilename +Category=Render Executables +CategoryOrder=0 +Index=4 +Label=Harmony 22 Render Executable +Description=The path to the Harmony Render executable file used for rendering. Enter alternative paths on separate lines. +Default=c:\Program Files (x86)\Toon Boom Animation\Toon Boom Harmony 22 Premium\win64\bin\HarmonyPremium.exe;/Applications/Toon Boom Harmony 22 Premium/Harmony Premium.app/Contents/MacOS/Harmony Premium;/usr/local/ToonBoomAnimation/harmonyPremium_22/lnx86_64/bin/HarmonyPremium diff --git a/openpype/modules/deadline/repository/custom/plugins/HarmonyOpenPype/HarmonyOpenPype.py b/openpype/modules/deadline/repository/custom/plugins/HarmonyOpenPype/HarmonyOpenPype.py index 2f6e9cf379..32ed76b58d 100644 --- a/openpype/modules/deadline/repository/custom/plugins/HarmonyOpenPype/HarmonyOpenPype.py +++ b/openpype/modules/deadline/repository/custom/plugins/HarmonyOpenPype/HarmonyOpenPype.py @@ -1,3 +1,4 @@ +#!/usr/bin/env python3 from System import * from System.Diagnostics import * from System.IO import * diff --git a/openpype/modules/ftrack/plugins/publish/integrate_ftrack_api.py b/openpype/modules/ftrack/plugins/publish/integrate_ftrack_api.py index 4d474fab10..858c0bb2d6 100644 --- a/openpype/modules/ftrack/plugins/publish/integrate_ftrack_api.py +++ b/openpype/modules/ftrack/plugins/publish/integrate_ftrack_api.py @@ -27,8 +27,8 @@ class IntegrateFtrackApi(pyblish.api.InstancePlugin): def process(self, instance): component_list = instance.data.get("ftrackComponentsList") if not component_list: - self.log.info( - "Instance don't have components to integrate to Ftrack." + self.log.debug( + "Instance doesn't have components to integrate to Ftrack." " Skipping." ) return @@ -37,7 +37,7 @@ class IntegrateFtrackApi(pyblish.api.InstancePlugin): task_entity, parent_entity = self.get_instance_entities( instance, context) if parent_entity is None: - self.log.info(( + self.log.debug(( "Skipping ftrack integration. Instance \"{}\" does not" " have specified ftrack entities." ).format(str(instance))) @@ -323,7 +323,7 @@ class IntegrateFtrackApi(pyblish.api.InstancePlugin): "type_id": asset_type_id, "context_id": parent_id } - self.log.info("Created new Asset with data: {}.".format(asset_data)) + self.log.debug("Created new Asset with data: {}.".format(asset_data)) session.create("Asset", asset_data) session.commit() return self._query_asset(session, asset_name, asset_type_id, parent_id) @@ -384,7 +384,7 @@ class IntegrateFtrackApi(pyblish.api.InstancePlugin): if comment: new_asset_version_data["comment"] = comment - self.log.info("Created new AssetVersion with data {}".format( + self.log.debug("Created new AssetVersion with data {}".format( new_asset_version_data )) session.create("AssetVersion", new_asset_version_data) @@ -555,7 +555,7 @@ class IntegrateFtrackApi(pyblish.api.InstancePlugin): location=location ) data["component"] = component_entity - self.log.info( + self.log.debug( ( "Created new Component with path: {0}, data: {1}," " metadata: {2}, location: {3}" diff --git a/openpype/modules/ftrack/plugins/publish/integrate_ftrack_description.py b/openpype/modules/ftrack/plugins/publish/integrate_ftrack_description.py index 6ed02bc8b6..ceaff8ff54 100644 --- a/openpype/modules/ftrack/plugins/publish/integrate_ftrack_description.py +++ b/openpype/modules/ftrack/plugins/publish/integrate_ftrack_description.py @@ -40,7 +40,7 @@ class IntegrateFtrackDescription(pyblish.api.InstancePlugin): comment = instance.data["comment"] if not comment: - self.log.info("Comment is not set.") + self.log.debug("Comment is not set.") else: self.log.debug("Comment is set to `{}`".format(comment)) diff --git a/openpype/modules/ftrack/plugins/publish/integrate_ftrack_note.py b/openpype/modules/ftrack/plugins/publish/integrate_ftrack_note.py index 6e82897d89..10b7932cdf 100644 --- a/openpype/modules/ftrack/plugins/publish/integrate_ftrack_note.py +++ b/openpype/modules/ftrack/plugins/publish/integrate_ftrack_note.py @@ -47,7 +47,7 @@ class IntegrateFtrackNote(pyblish.api.InstancePlugin): app_label = context.data["appLabel"] comment = instance.data["comment"] if not comment: - self.log.info("Comment is not set.") + self.log.debug("Comment is not set.") else: self.log.debug("Comment is set to `{}`".format(comment)) @@ -127,14 +127,14 @@ class IntegrateFtrackNote(pyblish.api.InstancePlugin): note_text = StringTemplate.format_template(template, format_data) if not note_text.solved: - self.log.warning(( + self.log.debug(( "Note template require more keys then can be provided." "\nTemplate: {}\nMissing values for keys:{}\nData: {}" ).format(template, note_text.missing_keys, format_data)) continue if not note_text: - self.log.info(( + self.log.debug(( "Note for AssetVersion {} would be empty. Skipping." "\nTemplate: {}\nData: {}" ).format(asset_version["id"], template, format_data)) diff --git a/openpype/modules/kitsu/plugins/publish/integrate_kitsu_note.py b/openpype/modules/kitsu/plugins/publish/integrate_kitsu_note.py index 6e5dd056f3..b66e1f01e0 100644 --- a/openpype/modules/kitsu/plugins/publish/integrate_kitsu_note.py +++ b/openpype/modules/kitsu/plugins/publish/integrate_kitsu_note.py @@ -121,7 +121,7 @@ class IntegrateKitsuNote(pyblish.api.ContextPlugin): publish_comment = self.format_publish_comment(instance) if not publish_comment: - self.log.info("Comment is not set.") + self.log.debug("Comment is not set.") else: self.log.debug("Comment is `{}`".format(publish_comment)) diff --git a/openpype/pipeline/colorspace.py b/openpype/pipeline/colorspace.py index a4901b7dfd..ae16c13635 100644 --- a/openpype/pipeline/colorspace.py +++ b/openpype/pipeline/colorspace.py @@ -16,6 +16,8 @@ from openpype.lib import ( Logger ) from openpype.pipeline import Anatomy +from openpype.lib.transcoding import VIDEO_EXTENSIONS, IMAGE_EXTENSIONS + log = Logger.get_logger(__name__) @@ -868,3 +870,173 @@ def _get_imageio_settings(project_settings, host_name): imageio_host = project_settings.get(host_name, {}).get("imageio", {}) return imageio_global, imageio_host + + +def get_colorspace_settings_from_publish_context(context_data): + """Returns solved settings for the host context. + + Args: + context_data (publish.Context.data): publishing context data + + Returns: + tuple | bool: config, file rules or None + """ + if "imageioSettings" in context_data and context_data["imageioSettings"]: + return context_data["imageioSettings"] + + project_name = context_data["projectName"] + host_name = context_data["hostName"] + anatomy_data = context_data["anatomyData"] + project_settings_ = context_data["project_settings"] + + config_data = get_imageio_config( + project_name, host_name, + project_settings=project_settings_, + anatomy_data=anatomy_data + ) + + # caching invalid state, so it's not recalculated all the time + file_rules = None + if config_data: + file_rules = get_imageio_file_rules( + project_name, host_name, + project_settings=project_settings_ + ) + + # caching settings for future instance processing + context_data["imageioSettings"] = (config_data, file_rules) + + return config_data, file_rules + + +def set_colorspace_data_to_representation( + representation, context_data, + colorspace=None, + log=None +): + """Sets colorspace data to representation. + + Args: + representation (dict): publishing representation + context_data (publish.Context.data): publishing context data + colorspace (str, optional): colorspace name. Defaults to None. + log (logging.Logger, optional): logger instance. Defaults to None. + + Example: + ``` + { + # for other publish plugins and loaders + "colorspace": "linear", + "config": { + # for future references in case need + "path": "/abs/path/to/config.ocio", + # for other plugins within remote publish cases + "template": "{project[root]}/path/to/config.ocio" + } + } + ``` + + """ + log = log or Logger.get_logger(__name__) + + file_ext = representation["ext"] + + # check if `file_ext` in lower case is in CachedData.allowed_exts + if file_ext.lstrip(".").lower() not in CachedData.allowed_exts: + log.debug( + "Extension '{}' is not in allowed extensions.".format(file_ext) + ) + return + + # get colorspace settings + config_data, file_rules = get_colorspace_settings_from_publish_context( + context_data) + + # in case host color management is not enabled + if not config_data: + log.warning("Host's colorspace management is disabled.") + return + + log.debug("Config data is: `{}`".format(config_data)) + + project_name = context_data["projectName"] + host_name = context_data["hostName"] + project_settings = context_data["project_settings"] + + # get one filename + filename = representation["files"] + if isinstance(filename, list): + filename = filename[0] + + # get matching colorspace from rules + colorspace = colorspace or get_imageio_colorspace_from_filepath( + filename, host_name, project_name, + config_data=config_data, + file_rules=file_rules, + project_settings=project_settings + ) + + # infuse data to representation + if colorspace: + colorspace_data = { + "colorspace": colorspace, + "config": config_data + } + + # update data key + representation["colorspaceData"] = colorspace_data + + +def get_display_view_colorspace_name(config_path, display, view): + """Returns the colorspace attribute of the (display, view) pair. + + Args: + config_path (str): path string leading to config.ocio + display (str): display name e.g. "ACES" + view (str): view name e.g. "sRGB" + + Returns: + view color space name (str) e.g. "Output - sRGB" + """ + + if not compatibility_check(): + # python environment is not compatible with PyOpenColorIO + # needs to be run in subprocess + return get_display_view_colorspace_subprocess(config_path, + display, view) + + from openpype.scripts.ocio_wrapper import _get_display_view_colorspace_name # noqa + + return _get_display_view_colorspace_name(config_path, display, view) + + +def get_display_view_colorspace_subprocess(config_path, display, view): + """Returns the colorspace attribute of the (display, view) pair + via subprocess. + + Args: + config_path (str): path string leading to config.ocio + display (str): display name e.g. "ACES" + view (str): view name e.g. "sRGB" + + Returns: + view color space name (str) e.g. "Output - sRGB" + """ + + with _make_temp_json_file() as tmp_json_path: + # Prepare subprocess arguments + args = [ + "run", get_ocio_config_script_path(), + "config", "get_display_view_colorspace_name", + "--in_path", config_path, + "--out_path", tmp_json_path, + "--display", display, + "--view", view + ] + log.debug("Executing: {}".format(" ".join(args))) + + run_openpype_process(*args, logger=log) + + # return default view colorspace name + with open(tmp_json_path, "r") as f: + return json.load(f) diff --git a/openpype/pipeline/create/context.py b/openpype/pipeline/create/context.py index 3076efcde7..f9e3f86652 100644 --- a/openpype/pipeline/create/context.py +++ b/openpype/pipeline/create/context.py @@ -1774,7 +1774,7 @@ class CreateContext: self.creator_discover_result = report for creator_class in report.plugins: if inspect.isabstract(creator_class): - self.log.info( + self.log.debug( "Skipping abstract Creator {}".format(str(creator_class)) ) continue @@ -1804,6 +1804,7 @@ class CreateContext: self, self.headless ) + if not creator.enabled: disabled_creators[creator_identifier] = creator continue diff --git a/openpype/pipeline/create/creator_plugins.py b/openpype/pipeline/create/creator_plugins.py index 38d6b6f465..6aa08cae70 100644 --- a/openpype/pipeline/create/creator_plugins.py +++ b/openpype/pipeline/create/creator_plugins.py @@ -1,16 +1,12 @@ import copy import collections -from abc import ( - ABCMeta, - abstractmethod, - abstractproperty -) +from abc import ABCMeta, abstractmethod import six from openpype.settings import get_system_settings, get_project_settings -from openpype.lib import Logger +from openpype.lib import Logger, is_func_signature_supported from openpype.pipeline.plugin_discover import ( discover, register_plugin, @@ -84,7 +80,8 @@ class SubsetConvertorPlugin(object): def host(self): return self._create_context.host - @abstractproperty + @property + @abstractmethod def identifier(self): """Converted identifier. @@ -161,7 +158,6 @@ class BaseCreator: Args: project_settings (Dict[str, Any]): Project settings. - system_settings (Dict[str, Any]): System settings. create_context (CreateContext): Context which initialized creator. headless (bool): Running in headless mode. """ @@ -208,10 +204,41 @@ class BaseCreator: # - we may use UI inside processing this attribute should be checked self.headless = headless - self.apply_settings(project_settings, system_settings) + expect_system_settings = False + if is_func_signature_supported( + self.apply_settings, project_settings + ): + self.apply_settings(project_settings) + else: + expect_system_settings = True + # Backwards compatibility for system settings + self.apply_settings(project_settings, system_settings) - def apply_settings(self, project_settings, system_settings): - """Method called on initialization of plugin to apply settings.""" + init_use_base = any( + self.__class__.__init__ is cls.__init__ + for cls in { + BaseCreator, + Creator, + HiddenCreator, + AutoCreator, + } + ) + if not init_use_base or expect_system_settings: + self.log.warning(( + "WARNING: Source - Create plugin {}." + " System settings argument will not be passed to" + " '__init__' and 'apply_settings' methods in future versions" + " of OpenPype. Planned version to drop the support" + " is 3.16.6 or 3.17.0. Please contact Ynput core team if you" + " need to keep system settings." + ).format(self.__class__.__name__)) + + def apply_settings(self, project_settings): + """Method called on initialization of plugin to apply settings. + + Args: + project_settings (dict[str, Any]): Project settings. + """ pass @@ -224,7 +251,8 @@ class BaseCreator: return self.family - @abstractproperty + @property + @abstractmethod def family(self): """Family that plugin represents.""" diff --git a/openpype/pipeline/load/plugins.py b/openpype/pipeline/load/plugins.py index f87fb3312d..8acfcfdb6c 100644 --- a/openpype/pipeline/load/plugins.py +++ b/openpype/pipeline/load/plugins.py @@ -234,6 +234,19 @@ class LoaderPlugin(list): """ return cls.options or [] + @property + def fname(self): + """Backwards compatibility with deprecation warning""" + + self.log.warning(( + "DEPRECATION WARNING: Source - Loader plugin {}." + " The 'fname' property on the Loader plugin will be removed in" + " future versions of OpenPype. Planned version to drop the support" + " is 3.16.6 or 3.17.0." + ).format(self.__class__.__name__)) + if hasattr(self, "_fname"): + return self._fname + class SubsetLoaderPlugin(LoaderPlugin): """Load subset into host application diff --git a/openpype/pipeline/load/utils.py b/openpype/pipeline/load/utils.py index 42418be40e..b10d6032b3 100644 --- a/openpype/pipeline/load/utils.py +++ b/openpype/pipeline/load/utils.py @@ -318,7 +318,8 @@ def load_with_repre_context( # Backwards compatibility: Originally the loader's __init__ required the # representation context to set `fname` attribute to the filename to load - loader.fname = get_representation_path_from_context(repre_context) + # Deprecated - to be removed in OpenPype 3.16.6 or 3.17.0. + loader._fname = get_representation_path_from_context(repre_context) return loader.load(repre_context, name, namespace, options) diff --git a/openpype/pipeline/publish/__init__.py b/openpype/pipeline/publish/__init__.py index 0c57915c05..3a82d6f565 100644 --- a/openpype/pipeline/publish/__init__.py +++ b/openpype/pipeline/publish/__init__.py @@ -40,6 +40,7 @@ from .lib import ( apply_plugin_settings_automatically, get_plugin_settings, get_publish_instance_label, + get_publish_instance_families, ) from .abstract_expected_files import ExpectedFiles @@ -87,6 +88,7 @@ __all__ = ( "apply_plugin_settings_automatically", "get_plugin_settings", "get_publish_instance_label", + "get_publish_instance_families", "ExpectedFiles", diff --git a/openpype/pipeline/publish/abstract_collect_render.py b/openpype/pipeline/publish/abstract_collect_render.py index 6877d556c3..8a26402bd8 100644 --- a/openpype/pipeline/publish/abstract_collect_render.py +++ b/openpype/pipeline/publish/abstract_collect_render.py @@ -75,7 +75,6 @@ class RenderInstance(object): tilesY = attr.ib(default=0) # number of tiles in Y # submit_publish_job - toBeRenderedOn = attr.ib(default=None) deadlineSubmissionJob = attr.ib(default=None) anatomyData = attr.ib(default=None) outputDir = attr.ib(default=None) diff --git a/openpype/pipeline/publish/lib.py b/openpype/pipeline/publish/lib.py index 815761cd0f..4d9443f635 100644 --- a/openpype/pipeline/publish/lib.py +++ b/openpype/pipeline/publish/lib.py @@ -952,6 +952,7 @@ def replace_with_published_scene_path(instance, replace_in_path=True): return file_path + def add_repre_files_for_cleanup(instance, repre): """ Explicitly mark repre files to be deleted. @@ -960,7 +961,16 @@ def add_repre_files_for_cleanup(instance, repre): """ files = repre["files"] staging_dir = repre.get("stagingDir") - if not staging_dir or instance.data.get("stagingDir_persistent"): + + # first make sure representation level is not persistent + if ( + not staging_dir + or repre.get("stagingDir_persistent") + ): + return + + # then look into instance level if it's not persistent + if instance.data.get("stagingDir_persistent"): return if isinstance(files, str): @@ -992,3 +1002,27 @@ def get_publish_instance_label(instance): or instance.data.get("name") or str(instance) ) + + +def get_publish_instance_families(instance): + """Get all families of the instance. + + Look for families under 'family' and 'families' keys in instance data. + Value of 'family' is used as first family and then all other families + in random order. + + Args: + pyblish.api.Instance: Instance to get families from. + + Returns: + list[str]: List of families. + """ + + family = instance.data.get("family") + families = set(instance.data.get("families") or []) + output = [] + if family: + output.append(family) + families.discard(family) + output.extend(families) + return output diff --git a/openpype/pipeline/publish/publish_plugins.py b/openpype/pipeline/publish/publish_plugins.py index ba3be6397e..ae6cbc42d1 100644 --- a/openpype/pipeline/publish/publish_plugins.py +++ b/openpype/pipeline/publish/publish_plugins.py @@ -1,6 +1,5 @@ import inspect from abc import ABCMeta -from pprint import pformat import pyblish.api from pyblish.plugin import MetaPlugin, ExplicitMetaPlugin from openpype.lib.transcoding import VIDEO_EXTENSIONS, IMAGE_EXTENSIONS @@ -14,9 +13,8 @@ from .lib import ( ) from openpype.pipeline.colorspace import ( - get_imageio_colorspace_from_filepath, - get_imageio_config, - get_imageio_file_rules + get_colorspace_settings_from_publish_context, + set_colorspace_data_to_representation ) @@ -306,12 +304,8 @@ class ColormanagedPyblishPluginMixin(object): matching colorspace from rules. Finally, it infuses this data into the representation. """ - allowed_ext = set( - ext.lstrip(".") for ext in IMAGE_EXTENSIONS.union(VIDEO_EXTENSIONS) - ) - @staticmethod - def get_colorspace_settings(context): + def get_colorspace_settings(self, context): """Returns solved settings for the host context. Args: @@ -320,50 +314,18 @@ class ColormanagedPyblishPluginMixin(object): Returns: tuple | bool: config, file rules or None """ - if "imageioSettings" in context.data: - return context.data["imageioSettings"] - - project_name = context.data["projectName"] - host_name = context.data["hostName"] - anatomy_data = context.data["anatomyData"] - project_settings_ = context.data["project_settings"] - - config_data = get_imageio_config( - project_name, host_name, - project_settings=project_settings_, - anatomy_data=anatomy_data - ) - - # in case host color management is not enabled - if not config_data: - return None - - file_rules = get_imageio_file_rules( - project_name, host_name, - project_settings=project_settings_ - ) - - # caching settings for future instance processing - context.data["imageioSettings"] = (config_data, file_rules) - - return config_data, file_rules + return get_colorspace_settings_from_publish_context(context.data) def set_representation_colorspace( self, representation, context, colorspace=None, - colorspace_settings=None ): """Sets colorspace data to representation. Args: representation (dict): publishing representation context (publish.Context): publishing context - config_data (dict): host resolved config data - file_rules (dict): host resolved file rules data colorspace (str, optional): colorspace name. Defaults to None. - colorspace_settings (tuple[dict, dict], optional): - Settings for config_data and file_rules. - Defaults to None. Example: ``` @@ -380,64 +342,10 @@ class ColormanagedPyblishPluginMixin(object): ``` """ - ext = representation["ext"] - # check extension - self.log.debug("__ ext: `{}`".format(ext)) - # check if ext in lower case is in self.allowed_ext - if ext.lstrip(".").lower() not in self.allowed_ext: - self.log.debug( - "Extension '{}' is not in allowed extensions.".format(ext) - ) - return - - if colorspace_settings is None: - colorspace_settings = self.get_colorspace_settings(context) - - # in case host color management is not enabled - if not colorspace_settings: - self.log.warning("Host's colorspace management is disabled.") - return - - # unpack colorspace settings - config_data, file_rules = colorspace_settings - - if not config_data: - # warn in case no colorspace path was defined - self.log.warning("No colorspace management was defined") - return - - self.log.debug("Config data is: `{}`".format(config_data)) - - project_name = context.data["projectName"] - host_name = context.data["hostName"] - project_settings = context.data["project_settings"] - - # get one filename - filename = representation["files"] - if isinstance(filename, list): - filename = filename[0] - - self.log.debug("__ filename: `{}`".format(filename)) - - # get matching colorspace from rules - colorspace = colorspace or get_imageio_colorspace_from_filepath( - filename, host_name, project_name, - config_data=config_data, - file_rules=file_rules, - project_settings=project_settings + # using cached settings if available + set_colorspace_data_to_representation( + representation, context.data, + colorspace, + log=self.log ) - self.log.debug("__ colorspace: `{}`".format(colorspace)) - - # infuse data to representation - if colorspace: - colorspace_data = { - "colorspace": colorspace, - "config": config_data - } - - # update data key - representation["colorspaceData"] = colorspace_data - - self.log.debug("__ colorspace_data: `{}`".format( - pformat(colorspace_data))) diff --git a/openpype/plugins/publish/cleanup.py b/openpype/plugins/publish/cleanup.py index 573cd829e4..6c122ddf09 100644 --- a/openpype/plugins/publish/cleanup.py +++ b/openpype/plugins/publish/cleanup.py @@ -69,7 +69,7 @@ class CleanUp(pyblish.api.InstancePlugin): skip_cleanup_filepaths.add(os.path.normpath(path)) if self.remove_temp_renders: - self.log.info("Cleaning renders new...") + self.log.debug("Cleaning renders new...") self.clean_renders(instance, skip_cleanup_filepaths) if [ef for ef in self.exclude_families @@ -95,10 +95,12 @@ class CleanUp(pyblish.api.InstancePlugin): return if instance.data.get("stagingDir_persistent"): - self.log.info("Staging dir: %s should be persistent" % staging_dir) + self.log.debug( + "Staging dir {} should be persistent".format(staging_dir) + ) return - self.log.info("Removing staging directory {}".format(staging_dir)) + self.log.debug("Removing staging directory {}".format(staging_dir)) shutil.rmtree(staging_dir) def clean_renders(self, instance, skip_cleanup_filepaths): diff --git a/openpype/plugins/publish/cleanup_farm.py b/openpype/plugins/publish/cleanup_farm.py index 8052f13734..e655437ced 100644 --- a/openpype/plugins/publish/cleanup_farm.py +++ b/openpype/plugins/publish/cleanup_farm.py @@ -26,10 +26,10 @@ class CleanUpFarm(pyblish.api.ContextPlugin): # Skip process if is not in list of source hosts in which this # plugin should run if src_host_name not in self.allowed_hosts: - self.log.info(( + self.log.debug( "Source host \"{}\" is not in list of enabled hosts {}." - " Skipping" - ).format(str(src_host_name), str(self.allowed_hosts))) + " Skipping".format(src_host_name, self.allowed_hosts) + ) return self.log.debug("Preparing filepaths to remove") @@ -47,7 +47,7 @@ class CleanUpFarm(pyblish.api.ContextPlugin): dirpaths_to_remove.add(os.path.normpath(staging_dir)) if not dirpaths_to_remove: - self.log.info("Nothing to remove. Skipping") + self.log.debug("Nothing to remove. Skipping") return self.log.debug("Filepaths to remove are:\n{}".format( diff --git a/openpype/plugins/publish/collect_audio.py b/openpype/plugins/publish/collect_audio.py index 3a0ddb3281..6aaadfc568 100644 --- a/openpype/plugins/publish/collect_audio.py +++ b/openpype/plugins/publish/collect_audio.py @@ -53,8 +53,8 @@ class CollectAudio(pyblish.api.ContextPlugin): ): # Skip instances that already have audio filled if instance.data.get("audio"): - self.log.info( - "Skipping Audio collecion. It is already collected" + self.log.debug( + "Skipping Audio collection. It is already collected" ) continue filtered_instances.append(instance) @@ -70,7 +70,7 @@ class CollectAudio(pyblish.api.ContextPlugin): instances_by_asset_name[asset_name].append(instance) asset_names = set(instances_by_asset_name.keys()) - self.log.info(( + self.log.debug(( "Searching for audio subset '{subset}' in assets {assets}" ).format( subset=self.audio_subset_name, @@ -100,7 +100,7 @@ class CollectAudio(pyblish.api.ContextPlugin): "offset": 0, "filename": repre_path }] - self.log.info("Audio Data added to instance ...") + self.log.debug("Audio Data added to instance ...") def query_representations(self, project_name, asset_names): """Query representations related to audio subsets for passed assets. diff --git a/openpype/plugins/publish/collect_current_context.py b/openpype/plugins/publish/collect_current_context.py index 166d75e5de..8b12a3f77f 100644 --- a/openpype/plugins/publish/collect_current_context.py +++ b/openpype/plugins/publish/collect_current_context.py @@ -39,5 +39,12 @@ class CollectCurrentContext(pyblish.api.ContextPlugin): # - 'task' -> 'taskName' self.log.info(( - "Collected project context\nProject: {}\nAsset: {}\nTask: {}" - ).format(project_name, asset_name, task_name)) + "Collected project context\n" + "Project: {project_name}\n" + "Asset: {asset_name}\n" + "Task: {task_name}" + ).format( + project_name=context.data["projectName"], + asset_name=context.data["asset"], + task_name=context.data["task"] + )) diff --git a/openpype/plugins/publish/collect_farm_target.py b/openpype/plugins/publish/collect_farm_target.py new file mode 100644 index 0000000000..adcd842b48 --- /dev/null +++ b/openpype/plugins/publish/collect_farm_target.py @@ -0,0 +1,35 @@ +# -*- coding: utf-8 -*- +import pyblish.api + + +class CollectFarmTarget(pyblish.api.InstancePlugin): + """Collects the render target for the instance + """ + + order = pyblish.api.CollectorOrder + 0.499 + label = "Collect Farm Target" + targets = ["local"] + + def process(self, instance): + if not instance.data.get("farm"): + return + + context = instance.context + + farm_name = "" + op_modules = context.data.get("openPypeModules") + + for farm_renderer in ["deadline", "royalrender", "muster"]: + op_module = op_modules.get(farm_renderer, False) + + if op_module and op_module.enabled: + farm_name = farm_renderer + elif not op_module: + self.log.error("Cannot get OpenPype {0} module.".format( + farm_renderer)) + + if farm_name: + self.log.debug("Collected render target: {0}".format(farm_name)) + instance.data["toBeRenderedOn"] = farm_name + else: + AssertionError("No OpenPype renderer module found") diff --git a/openpype/plugins/publish/collect_hierarchy.py b/openpype/plugins/publish/collect_hierarchy.py index 687397be8a..b5fd1e4bb9 100644 --- a/openpype/plugins/publish/collect_hierarchy.py +++ b/openpype/plugins/publish/collect_hierarchy.py @@ -24,7 +24,7 @@ class CollectHierarchy(pyblish.api.ContextPlugin): final_context[project_name]['entity_type'] = 'Project' for instance in context: - self.log.info("Processing instance: `{}` ...".format(instance)) + self.log.debug("Processing instance: `{}` ...".format(instance)) # shot data dict shot_data = {} diff --git a/openpype/plugins/publish/collect_input_representations_to_versions.py b/openpype/plugins/publish/collect_input_representations_to_versions.py index 54a3214647..2b8c745d3d 100644 --- a/openpype/plugins/publish/collect_input_representations_to_versions.py +++ b/openpype/plugins/publish/collect_input_representations_to_versions.py @@ -46,3 +46,10 @@ class CollectInputRepresentationsToVersions(pyblish.api.ContextPlugin): version_id = representation_id_to_version_id.get(repre_id) if version_id: input_versions.append(version_id) + else: + self.log.debug( + "Representation id {} skipped because its version is " + "not found in current project. Likely it is loaded " + "from a library project or uses a deleted " + "representation or version.".format(repre_id) + ) diff --git a/openpype/plugins/publish/collect_rendered_files.py b/openpype/plugins/publish/collect_rendered_files.py index 6c8d1e9ca5..8a5a5a83f1 100644 --- a/openpype/plugins/publish/collect_rendered_files.py +++ b/openpype/plugins/publish/collect_rendered_files.py @@ -91,25 +91,28 @@ class CollectRenderedFiles(pyblish.api.ContextPlugin): # now we can just add instances from json file and we are done for instance_data in data.get("instances"): - self.log.info(" - processing instance for {}".format( + self.log.debug(" - processing instance for {}".format( instance_data.get("subset"))) instance = self._context.create_instance( instance_data.get("subset") ) - self.log.info("Filling stagingDir...") + self.log.debug("Filling stagingDir...") self._fill_staging_dir(instance_data, anatomy) instance.data.update(instance_data) # stash render job id for later validation instance.data["render_job_id"] = data.get("job").get("_id") - + staging_dir_persistent = instance.data.get( + "stagingDir_persistent", False + ) representations = [] for repre_data in instance_data.get("representations") or []: self._fill_staging_dir(repre_data, anatomy) representations.append(repre_data) - add_repre_files_for_cleanup(instance, repre_data) + if not staging_dir_persistent: + add_repre_files_for_cleanup(instance, repre_data) instance.data["representations"] = representations @@ -121,9 +124,11 @@ class CollectRenderedFiles(pyblish.api.ContextPlugin): "offset": 0 }] }) - self.log.info( + self.log.debug( f"Adding audio to instance: {instance.data['audio']}") + return staging_dir_persistent + def process(self, context): self._context = context @@ -137,11 +142,11 @@ class CollectRenderedFiles(pyblish.api.ContextPlugin): # Using already collected Anatomy anatomy = context.data["anatomy"] - self.log.info("Getting root setting for project \"{}\"".format( + self.log.debug("Getting root setting for project \"{}\"".format( anatomy.project_name )) - self.log.info("anatomy: {}".format(anatomy.roots)) + self.log.debug("anatomy: {}".format(anatomy.roots)) try: session_is_set = False for path in paths: @@ -156,13 +161,16 @@ class CollectRenderedFiles(pyblish.api.ContextPlugin): if remapped: session_data["AVALON_WORKDIR"] = remapped - self.log.info("Setting session using data from file") + self.log.debug("Setting session using data from file") legacy_io.Session.update(session_data) os.environ.update(session_data) session_is_set = True - self._process_path(data, anatomy) - context.data["cleanupFullPaths"].append(path) - context.data["cleanupEmptyDirs"].append(os.path.dirname(path)) + staging_dir_persistent = self._process_path(data, anatomy) + if not staging_dir_persistent: + context.data["cleanupFullPaths"].append(path) + context.data["cleanupEmptyDirs"].append( + os.path.dirname(path) + ) except Exception as e: self.log.error(e, exc_info=True) raise Exception("Error") from e diff --git a/openpype/plugins/publish/collect_scene_version.py b/openpype/plugins/publish/collect_scene_version.py index 70a0aca296..7920c1e82b 100644 --- a/openpype/plugins/publish/collect_scene_version.py +++ b/openpype/plugins/publish/collect_scene_version.py @@ -63,4 +63,6 @@ class CollectSceneVersion(pyblish.api.ContextPlugin): "filename: {}".format(filename)) context.data['version'] = int(version) - self.log.info('Scene Version: %s' % context.data.get('version')) + self.log.debug( + "Collected scene version: {}".format(context.data.get('version')) + ) diff --git a/openpype/plugins/publish/collect_sequence_frame_data.py b/openpype/plugins/publish/collect_sequence_frame_data.py index c200b245e9..6c2bfbf358 100644 --- a/openpype/plugins/publish/collect_sequence_frame_data.py +++ b/openpype/plugins/publish/collect_sequence_frame_data.py @@ -50,4 +50,7 @@ class CollectSequenceFrameData(pyblish.api.InstancePlugin): return { "frameStart": repres_frames[0], "frameEnd": repres_frames[-1], + "handleStart": 0, + "handleEnd": 0, + "fps": instance.context.data["assetEntity"]["data"]["fps"] } diff --git a/openpype/plugins/publish/extract_burnin.py b/openpype/plugins/publish/extract_burnin.py index e5b37ee3b4..dc8aab6ce4 100644 --- a/openpype/plugins/publish/extract_burnin.py +++ b/openpype/plugins/publish/extract_burnin.py @@ -83,7 +83,7 @@ class ExtractBurnin(publish.Extractor): return if not instance.data.get("representations"): - self.log.info( + self.log.debug( "Instance does not have filled representations. Skipping") return @@ -135,11 +135,11 @@ class ExtractBurnin(publish.Extractor): burnin_defs, repre["tags"] ) if not repre_burnin_defs: - self.log.info(( + self.log.debug( "Skipped representation. All burnin definitions from" - " selected profile does not match to representation's" - " tags. \"{}\"" - ).format(str(repre["tags"]))) + " selected profile do not match to representation's" + " tags. \"{}\"".format(repre["tags"]) + ) continue filtered_repres.append((repre, repre_burnin_defs)) @@ -164,7 +164,7 @@ class ExtractBurnin(publish.Extractor): logger=self.log) if not profile: - self.log.info(( + self.log.debug(( "Skipped instance. None of profiles in presets are for" " Host: \"{}\" | Families: \"{}\" | Task \"{}\"" " | Task type \"{}\" | Subset \"{}\" " @@ -176,7 +176,7 @@ class ExtractBurnin(publish.Extractor): # Pre-filter burnin definitions by instance families burnin_defs = self.filter_burnins_defs(profile, instance) if not burnin_defs: - self.log.info(( + self.log.debug(( "Skipped instance. Burnin definitions are not set for profile" " Host: \"{}\" | Families: \"{}\" | Task \"{}\"" " | Profile \"{}\"" @@ -223,10 +223,10 @@ class ExtractBurnin(publish.Extractor): # If result is None the requirement of conversion can't be # determined if do_convert is None: - self.log.info(( + self.log.debug( "Can't determine if representation requires conversion." " Skipped." - )) + ) continue # Do conversion if needed diff --git a/openpype/plugins/publish/extract_color_transcode.py b/openpype/plugins/publish/extract_color_transcode.py index f7c8af9318..dbf1b6c8a6 100644 --- a/openpype/plugins/publish/extract_color_transcode.py +++ b/openpype/plugins/publish/extract_color_transcode.py @@ -320,7 +320,7 @@ class ExtractOIIOTranscode(publish.Extractor): logger=self.log) if not profile: - self.log.info(( + self.log.debug(( "Skipped instance. None of profiles in presets are for" " Host: \"{}\" | Families: \"{}\" | Task \"{}\"" " | Task type \"{}\" | Subset \"{}\" " diff --git a/openpype/plugins/publish/extract_colorspace_data.py b/openpype/plugins/publish/extract_colorspace_data.py index 363df28fb5..8873dcd637 100644 --- a/openpype/plugins/publish/extract_colorspace_data.py +++ b/openpype/plugins/publish/extract_colorspace_data.py @@ -30,7 +30,7 @@ class ExtractColorspaceData(publish.Extractor, def process(self, instance): representations = instance.data.get("representations") if not representations: - self.log.info("No representations at instance : `{}`".format( + self.log.debug("No representations at instance : `{}`".format( instance)) return diff --git a/openpype/plugins/publish/extract_hierarchy_avalon.py b/openpype/plugins/publish/extract_hierarchy_avalon.py index 1d57545bc0..d70f0cbdd7 100644 --- a/openpype/plugins/publish/extract_hierarchy_avalon.py +++ b/openpype/plugins/publish/extract_hierarchy_avalon.py @@ -21,7 +21,7 @@ class ExtractHierarchyToAvalon(pyblish.api.ContextPlugin): return if "hierarchyContext" not in context.data: - self.log.info("skipping IntegrateHierarchyToAvalon") + self.log.debug("skipping ExtractHierarchyToAvalon") return if not legacy_io.Session: diff --git a/openpype/plugins/publish/extract_hierarchy_to_ayon.py b/openpype/plugins/publish/extract_hierarchy_to_ayon.py index 915650ae41..0d9131718b 100644 --- a/openpype/plugins/publish/extract_hierarchy_to_ayon.py +++ b/openpype/plugins/publish/extract_hierarchy_to_ayon.py @@ -8,6 +8,11 @@ from ayon_api import slugify_string from ayon_api.entity_hub import EntityHub from openpype import AYON_SERVER_ENABLED +from openpype.client import get_assets +from openpype.pipeline.template_data import ( + get_asset_template_data, + get_task_template_data, +) def _default_json_parse(value): @@ -27,13 +32,51 @@ class ExtractHierarchyToAYON(pyblish.api.ContextPlugin): hierarchy_context = context.data.get("hierarchyContext") if not hierarchy_context: - self.log.info("Skipping") + self.log.debug("Skipping ExtractHierarchyToAYON") return project_name = context.data["projectName"] + self._create_hierarchy(context, project_name) + self._fill_instance_entities(context, project_name) + + def _fill_instance_entities(self, context, project_name): + instances_by_asset_name = collections.defaultdict(list) + for instance in context: + if instance.data.get("publish") is False: + continue + + instance_entity = instance.data.get("assetEntity") + if instance_entity: + continue + + # Skip if instance asset does not match + instance_asset_name = instance.data.get("asset") + instances_by_asset_name[instance_asset_name].append(instance) + + project_doc = context.data["projectEntity"] + asset_docs = get_assets( + project_name, asset_names=instances_by_asset_name.keys() + ) + asset_docs_by_name = { + asset_doc["name"]: asset_doc + for asset_doc in asset_docs + } + for asset_name, instances in instances_by_asset_name.items(): + asset_doc = asset_docs_by_name[asset_name] + asset_data = get_asset_template_data(asset_doc, project_name) + for instance in instances: + task_name = instance.data.get("task") + template_data = get_task_template_data( + project_doc, asset_doc, task_name) + template_data.update(copy.deepcopy(asset_data)) + + instance.data["anatomyData"].update(template_data) + instance.data["assetEntity"] = asset_doc + + def _create_hierarchy(self, context, project_name): hierarchy_context = self._filter_hierarchy(context) if not hierarchy_context: - self.log.info("All folders were filtered out") + self.log.debug("All folders were filtered out") return self.log.debug("Hierarchy_context: {}".format( diff --git a/openpype/plugins/publish/extract_review.py b/openpype/plugins/publish/extract_review.py index 9cc456872e..0ae941511c 100644 --- a/openpype/plugins/publish/extract_review.py +++ b/openpype/plugins/publish/extract_review.py @@ -21,6 +21,7 @@ from openpype.lib.transcoding import ( IMAGE_EXTENSIONS, get_ffprobe_streams, should_convert_for_ffmpeg, + get_review_layer_name, convert_input_paths_for_ffmpeg, get_transcode_temp_directory, ) @@ -266,6 +267,8 @@ class ExtractReview(pyblish.api.InstancePlugin): )) continue + layer_name = get_review_layer_name(first_input_path) + # Do conversion if needed # - change staging dir of source representation # - must be set back after output definitions processing @@ -284,7 +287,8 @@ class ExtractReview(pyblish.api.InstancePlugin): instance, repre, src_repre_staging_dir, - filtered_output_defs + filtered_output_defs, + layer_name ) finally: @@ -298,7 +302,12 @@ class ExtractReview(pyblish.api.InstancePlugin): shutil.rmtree(new_staging_dir) def _render_output_definitions( - self, instance, repre, src_repre_staging_dir, output_definitions + self, + instance, + repre, + src_repre_staging_dir, + output_definitions, + layer_name ): fill_data = copy.deepcopy(instance.data["anatomyData"]) for _output_def in output_definitions: @@ -370,7 +379,12 @@ class ExtractReview(pyblish.api.InstancePlugin): try: # temporary until oiiotool is supported cross platform ffmpeg_args = self._ffmpeg_arguments( - output_def, instance, new_repre, temp_data, fill_data + output_def, + instance, + new_repre, + temp_data, + fill_data, + layer_name, ) except ZeroDivisionError: # TODO recalculate width and height using OIIO before @@ -531,7 +545,13 @@ class ExtractReview(pyblish.api.InstancePlugin): } def _ffmpeg_arguments( - self, output_def, instance, new_repre, temp_data, fill_data + self, + output_def, + instance, + new_repre, + temp_data, + fill_data, + layer_name ): """Prepares ffmpeg arguments for expected extraction. @@ -599,6 +619,10 @@ class ExtractReview(pyblish.api.InstancePlugin): duration_seconds = float(output_frames_len / temp_data["fps"]) + # Define which layer should be used + if layer_name: + ffmpeg_input_args.extend(["-layer", layer_name]) + if temp_data["input_is_sequence"]: # Set start frame of input sequence (just frame in filename) # - definition of input filepath diff --git a/openpype/plugins/publish/extract_review_slate.py b/openpype/plugins/publish/extract_review_slate.py index 886384fee6..d89fbb90c4 100644 --- a/openpype/plugins/publish/extract_review_slate.py +++ b/openpype/plugins/publish/extract_review_slate.py @@ -15,6 +15,7 @@ from openpype.lib import ( get_ffmpeg_format_args, ) from openpype.pipeline import publish +from openpype.pipeline.publish import KnownPublishError class ExtractReviewSlate(publish.Extractor): @@ -46,7 +47,7 @@ class ExtractReviewSlate(publish.Extractor): "*": inst_data["slateFrame"] } - self.log.info("_ slates_data: {}".format(pformat(slates_data))) + self.log.debug("_ slates_data: {}".format(pformat(slates_data))) if "reviewToWidth" in inst_data: use_legacy_code = True @@ -76,7 +77,7 @@ class ExtractReviewSlate(publish.Extractor): ) # get slate data slate_path = self._get_slate_path(input_file, slates_data) - self.log.info("_ slate_path: {}".format(slate_path)) + self.log.debug("_ slate_path: {}".format(slate_path)) slate_width, slate_height = self._get_slates_resolution(slate_path) @@ -93,9 +94,10 @@ class ExtractReviewSlate(publish.Extractor): # Raise exception of any stream didn't define input resolution if input_width is None: - raise AssertionError(( + raise KnownPublishError( "FFprobe couldn't read resolution from input file: \"{}\"" - ).format(input_path)) + .format(input_path) + ) ( audio_codec, diff --git a/openpype/plugins/publish/extract_scanline_exr.py b/openpype/plugins/publish/extract_scanline_exr.py index 9f22794a79..747155689b 100644 --- a/openpype/plugins/publish/extract_scanline_exr.py +++ b/openpype/plugins/publish/extract_scanline_exr.py @@ -29,24 +29,24 @@ class ExtractScanlineExr(pyblish.api.InstancePlugin): representations_new = [] for repre in representations: - self.log.info( + self.log.debug( "Processing representation {}".format(repre.get("name"))) tags = repre.get("tags", []) if "toScanline" not in tags: - self.log.info(" - missing toScanline tag") + self.log.debug(" - missing toScanline tag") continue # run only on exrs if repre.get("ext") != "exr": - self.log.info("- not EXR files") + self.log.debug("- not EXR files") continue if not isinstance(repre['files'], (list, tuple)): input_files = [repre['files']] - self.log.info("We have a single frame") + self.log.debug("We have a single frame") else: input_files = repre['files'] - self.log.info("We have a sequence") + self.log.debug("We have a sequence") stagingdir = os.path.normpath(repre.get("stagingDir")) @@ -68,7 +68,7 @@ class ExtractScanlineExr(pyblish.api.InstancePlugin): ] subprocess_exr = " ".join(oiio_cmd) - self.log.info(f"running: {subprocess_exr}") + self.log.debug(f"running: {subprocess_exr}") run_subprocess(subprocess_exr, logger=self.log) # raise error if there is no ouptput diff --git a/openpype/plugins/publish/extract_thumbnail.py b/openpype/plugins/publish/extract_thumbnail.py index b72a6d02ad..de101ac7ac 100644 --- a/openpype/plugins/publish/extract_thumbnail.py +++ b/openpype/plugins/publish/extract_thumbnail.py @@ -43,12 +43,12 @@ class ExtractThumbnail(pyblish.api.InstancePlugin): # Skip if instance have 'review' key in data set to 'False' if not self._is_review_instance(instance): - self.log.info("Skipping - no review set on instance.") + self.log.debug("Skipping - no review set on instance.") return # Check if already has thumbnail created if self._already_has_thumbnail(instance_repres): - self.log.info("Thumbnail representation already present.") + self.log.debug("Thumbnail representation already present.") return # skip crypto passes. @@ -58,15 +58,15 @@ class ExtractThumbnail(pyblish.api.InstancePlugin): # representation that can be determined much earlier and # with better precision. if "crypto" in subset_name.lower(): - self.log.info("Skipping crypto passes.") + self.log.debug("Skipping crypto passes.") return filtered_repres = self._get_filtered_repres(instance) if not filtered_repres: - self.log.info(( - "Instance don't have representations" - " that can be used as source for thumbnail. Skipping" - )) + self.log.info( + "Instance doesn't have representations that can be used " + "as source for thumbnail. Skipping thumbnail extraction." + ) return # Create temp directory for thumbnail @@ -107,10 +107,10 @@ class ExtractThumbnail(pyblish.api.InstancePlugin): # oiiotool isn't available if not thumbnail_created: if oiio_supported: - self.log.info(( + self.log.debug( "Converting with FFMPEG because input" " can't be read by OIIO." - )) + ) thumbnail_created = self.create_thumbnail_ffmpeg( full_input_path, full_output_path @@ -165,8 +165,8 @@ class ExtractThumbnail(pyblish.api.InstancePlugin): continue if not repre.get("files"): - self.log.info(( - "Representation \"{}\" don't have files. Skipping" + self.log.debug(( + "Representation \"{}\" doesn't have files. Skipping" ).format(repre["name"])) continue @@ -174,7 +174,7 @@ class ExtractThumbnail(pyblish.api.InstancePlugin): return filtered_repres def create_thumbnail_oiio(self, src_path, dst_path): - self.log.info("Extracting thumbnail {}".format(dst_path)) + self.log.debug("Extracting thumbnail with OIIO: {}".format(dst_path)) oiio_cmd = get_oiio_tool_args( "oiiotool", "-a", src_path, @@ -192,7 +192,7 @@ class ExtractThumbnail(pyblish.api.InstancePlugin): return False def create_thumbnail_ffmpeg(self, src_path, dst_path): - self.log.info("outputting {}".format(dst_path)) + self.log.debug("Extracting thumbnail with FFMPEG: {}".format(dst_path)) ffmpeg_path_args = get_ffmpeg_tool_args("ffmpeg") ffmpeg_args = self.ffmpeg_args or {} @@ -225,7 +225,7 @@ class ExtractThumbnail(pyblish.api.InstancePlugin): return True except Exception: self.log.warning( - "Failed to create thubmnail using ffmpeg", + "Failed to create thumbnail using ffmpeg", exc_info=True ) return False diff --git a/openpype/plugins/publish/extract_thumbnail_from_source.py b/openpype/plugins/publish/extract_thumbnail_from_source.py index 1b9f0a8bae..401a5d615d 100644 --- a/openpype/plugins/publish/extract_thumbnail_from_source.py +++ b/openpype/plugins/publish/extract_thumbnail_from_source.py @@ -49,7 +49,7 @@ class ExtractThumbnailFromSource(pyblish.api.InstancePlugin): # Check if already has thumbnail created if self._instance_has_thumbnail(instance): - self.log.info("Thumbnail representation already present.") + self.log.debug("Thumbnail representation already present.") return dst_filepath = self._create_thumbnail( @@ -98,7 +98,7 @@ class ExtractThumbnailFromSource(pyblish.api.InstancePlugin): thumbnail_created = False oiio_supported = is_oiio_supported() - self.log.info("Thumbnail source: {}".format(thumbnail_source)) + self.log.debug("Thumbnail source: {}".format(thumbnail_source)) src_basename = os.path.basename(thumbnail_source) dst_filename = os.path.splitext(src_basename)[0] + "_thumb.jpg" full_output_path = os.path.join(dst_staging, dst_filename) @@ -115,10 +115,10 @@ class ExtractThumbnailFromSource(pyblish.api.InstancePlugin): # oiiotool isn't available if not thumbnail_created: if oiio_supported: - self.log.info(( + self.log.info( "Converting with FFMPEG because input" " can't be read by OIIO." - )) + ) thumbnail_created = self.create_thumbnail_ffmpeg( thumbnail_source, full_output_path @@ -143,20 +143,20 @@ class ExtractThumbnailFromSource(pyblish.api.InstancePlugin): return False def create_thumbnail_oiio(self, src_path, dst_path): - self.log.info("outputting {}".format(dst_path)) + self.log.debug("Outputting thumbnail with OIIO: {}".format(dst_path)) oiio_cmd = get_oiio_tool_args( "oiiotool", "-a", src_path, "--ch", "R,G,B", "-o", dst_path ) - self.log.info("Running: {}".format(" ".join(oiio_cmd))) + self.log.debug("Running: {}".format(" ".join(oiio_cmd))) try: run_subprocess(oiio_cmd, logger=self.log) return True except Exception: self.log.warning( - "Failed to create thubmnail using oiiotool", + "Failed to create thumbnail using oiiotool", exc_info=True ) return False @@ -173,13 +173,13 @@ class ExtractThumbnailFromSource(pyblish.api.InstancePlugin): dst_path ) - self.log.info("Running: {}".format(" ".join(ffmpeg_cmd))) + self.log.debug("Running: {}".format(" ".join(ffmpeg_cmd))) try: run_subprocess(ffmpeg_cmd, logger=self.log) return True except Exception: self.log.warning( - "Failed to create thubmnail using ffmpeg", + "Failed to create thumbnail using ffmpeg", exc_info=True ) return False diff --git a/openpype/plugins/publish/extract_trim_video_audio.py b/openpype/plugins/publish/extract_trim_video_audio.py index 2907ae1839..5e00cfc96f 100644 --- a/openpype/plugins/publish/extract_trim_video_audio.py +++ b/openpype/plugins/publish/extract_trim_video_audio.py @@ -36,7 +36,7 @@ class ExtractTrimVideoAudio(publish.Extractor): # get staging dir staging_dir = self.staging_dir(instance) - self.log.info("Staging dir set to: `{}`".format(staging_dir)) + self.log.debug("Staging dir set to: `{}`".format(staging_dir)) # Generate mov file. fps = instance.data["fps"] @@ -59,7 +59,7 @@ class ExtractTrimVideoAudio(publish.Extractor): extensions = [output_file_type] for ext in extensions: - self.log.info("Processing ext: `{}`".format(ext)) + self.log.debug("Processing ext: `{}`".format(ext)) if not ext.startswith("."): ext = "." + ext @@ -98,7 +98,7 @@ class ExtractTrimVideoAudio(publish.Extractor): ffmpeg_args.append(clip_trimed_path) joined_args = " ".join(ffmpeg_args) - self.log.info(f"Processing: {joined_args}") + self.log.debug(f"Processing: {joined_args}") run_subprocess( ffmpeg_args, logger=self.log ) diff --git a/openpype/plugins/publish/integrate.py b/openpype/plugins/publish/integrate.py index be07cffe72..7e48155b9e 100644 --- a/openpype/plugins/publish/integrate.py +++ b/openpype/plugins/publish/integrate.py @@ -155,13 +155,13 @@ class IntegrateAsset(pyblish.api.InstancePlugin): # Instance should be integrated on a farm if instance.data.get("farm"): - self.log.info( + self.log.debug( "Instance is marked to be processed on farm. Skipping") return # Instance is marked to not get integrated if not instance.data.get("integrate", True): - self.log.info("Instance is marked to skip integrating. Skipping") + self.log.debug("Instance is marked to skip integrating. Skipping") return filtered_repres = self.filter_representations(instance) @@ -306,7 +306,7 @@ class IntegrateAsset(pyblish.api.InstancePlugin): # increase if the file transaction takes a long time. op_session.commit() - self.log.info("Subset {subset[name]} and Version {version[name]} " + self.log.info("Subset '{subset[name]}' version {version[name]} " "written to database..".format(subset=subset, version=version)) @@ -392,8 +392,13 @@ class IntegrateAsset(pyblish.api.InstancePlugin): p["representation"]["_id"]: p for p in prepared_representations } - self.log.info("Registered {} representations" - "".format(len(prepared_representations))) + self.log.info( + "Registered {} representations: {}".format( + len(prepared_representations), + ", ".join(p["representation"]["name"] + for p in prepared_representations) + ) + ) def prepare_subset(self, instance, op_session, project_name): asset_doc = instance.data["assetEntity"] diff --git a/openpype/plugins/publish/integrate_hero_version.py b/openpype/plugins/publish/integrate_hero_version.py index 6c21664b78..9f0f7fe7f3 100644 --- a/openpype/plugins/publish/integrate_hero_version.py +++ b/openpype/plugins/publish/integrate_hero_version.py @@ -275,10 +275,10 @@ class IntegrateHeroVersion(pyblish.api.InstancePlugin): backup_hero_publish_dir = _backup_hero_publish_dir break except Exception: - self.log.info(( + self.log.info( "Could not remove previous backup folder." - " Trying to add index to folder name" - )) + " Trying to add index to folder name." + ) _backup_hero_publish_dir = ( backup_hero_publish_dir + str(idx) diff --git a/openpype/plugins/publish/integrate_inputlinks_ayon.py b/openpype/plugins/publish/integrate_inputlinks_ayon.py index 180524cd08..28684aa889 100644 --- a/openpype/plugins/publish/integrate_inputlinks_ayon.py +++ b/openpype/plugins/publish/integrate_inputlinks_ayon.py @@ -1,7 +1,11 @@ import collections import pyblish.api -from ayon_api import create_link, make_sure_link_type_exists +from ayon_api import ( + create_link, + make_sure_link_type_exists, + get_versions_links, +) from openpype import AYON_SERVER_ENABLED @@ -124,6 +128,33 @@ class IntegrateInputLinksAYON(pyblish.api.ContextPlugin): version_entity["_id"], ) + def _get_existing_links(self, project_name, link_type, entity_ids): + """Find all existing links for given version ids. + + Args: + project_name (str): Name of project. + link_type (str): Type of link. + entity_ids (set[str]): Set of version ids. + + Returns: + dict[str, set[str]]: Existing links by version id. + """ + + output = collections.defaultdict(set) + if not entity_ids: + return output + + existing_in_links = get_versions_links( + project_name, entity_ids, [link_type], "output" + ) + + for entity_id, links in existing_in_links.items(): + if not links: + continue + for link in links: + output[entity_id].add(link["entityId"]) + return output + def create_links_on_server(self, context, new_links): """Create new links on server. @@ -144,16 +175,32 @@ class IntegrateInputLinksAYON(pyblish.api.ContextPlugin): # Create link themselves for link_type, items in new_links.items(): + mapping = collections.defaultdict(set) + # Make sure there are no duplicates of src > dst ids for item in items: - input_id, output_id = item - create_link( - project_name, - link_type, - input_id, - "version", - output_id, - "version" - ) + _input_id, _output_id = item + mapping[_input_id].add(_output_id) + + existing_links_by_in_id = self._get_existing_links( + project_name, link_type, set(mapping.keys()) + ) + + for input_id, output_ids in mapping.items(): + existing_links = existing_links_by_in_id[input_id] + for output_id in output_ids: + # Skip creation of link if already exists + # NOTE: AYON server does not support + # to have same links + if output_id in existing_links: + continue + create_link( + project_name, + link_type, + input_id, + "version", + output_id, + "version" + ) if not AYON_SERVER_ENABLED: diff --git a/openpype/plugins/publish/integrate_thumbnail.py b/openpype/plugins/publish/integrate_thumbnail.py index 9929d8f754..0c12255d38 100644 --- a/openpype/plugins/publish/integrate_thumbnail.py +++ b/openpype/plugins/publish/integrate_thumbnail.py @@ -41,7 +41,9 @@ class IntegrateThumbnails(pyblish.api.ContextPlugin): def process(self, context): if AYON_SERVER_ENABLED: - self.log.info("AYON is enabled. Skipping v3 thumbnail integration") + self.log.debug( + "AYON is enabled. Skipping v3 thumbnail integration" + ) return # Filter instances which can be used for integration @@ -74,14 +76,14 @@ class IntegrateThumbnails(pyblish.api.ContextPlugin): thumbnail_template = anatomy.templates["publish"]["thumbnail"] if not thumbnail_template: - self.log.info("Thumbnail template is not filled. Skipping.") + self.log.debug("Thumbnail template is not filled. Skipping.") return if ( not thumbnail_root and thumbnail_root_format_key in thumbnail_template ): - self.log.warning(("{} is not set. Skipping.").format(env_key)) + self.log.warning("{} is not set. Skipping.".format(env_key)) return # Collect verion ids from all filtered instance diff --git a/openpype/plugins/publish/integrate_thumbnail_ayon.py b/openpype/plugins/publish/integrate_thumbnail_ayon.py index ba5664c69f..cf05327ce8 100644 --- a/openpype/plugins/publish/integrate_thumbnail_ayon.py +++ b/openpype/plugins/publish/integrate_thumbnail_ayon.py @@ -35,13 +35,13 @@ class IntegrateThumbnailsAYON(pyblish.api.ContextPlugin): def process(self, context): if not AYON_SERVER_ENABLED: - self.log.info("AYON is not enabled. Skipping") + self.log.debug("AYON is not enabled. Skipping") return # Filter instances which can be used for integration filtered_instance_items = self._prepare_instances(context) if not filtered_instance_items: - self.log.info( + self.log.debug( "All instances were filtered. Thumbnail integration skipped." ) return @@ -110,7 +110,7 @@ class IntegrateThumbnailsAYON(pyblish.api.ContextPlugin): # Skip instance if thumbnail path is not available for it if not thumbnail_path: - self.log.info(( + self.log.debug(( "Skipping thumbnail integration for instance \"{}\"." " Instance and context" " thumbnail paths are not available." diff --git a/openpype/plugins/publish/validate_asset_docs.py b/openpype/plugins/publish/validate_asset_docs.py index 9a1ca5b8de..8dfd783c39 100644 --- a/openpype/plugins/publish/validate_asset_docs.py +++ b/openpype/plugins/publish/validate_asset_docs.py @@ -22,11 +22,11 @@ class ValidateAssetDocs(pyblish.api.InstancePlugin): return if instance.data.get("assetEntity"): - self.log.info("Instance has set asset document in its data.") + self.log.debug("Instance has set asset document in its data.") elif instance.data.get("newAssetPublishing"): # skip if it is editorial - self.log.info("Editorial instance is no need to check...") + self.log.debug("Editorial instance has no need to check...") else: raise PublishValidationError(( diff --git a/openpype/plugins/publish/validate_editorial_asset_name.py b/openpype/plugins/publish/validate_editorial_asset_name.py index 4f8a1abf2e..fca0d8e7f5 100644 --- a/openpype/plugins/publish/validate_editorial_asset_name.py +++ b/openpype/plugins/publish/validate_editorial_asset_name.py @@ -56,7 +56,7 @@ class ValidateEditorialAssetName(pyblish.api.ContextPlugin): } continue - self.log.info("correct asset: {}".format(asset)) + self.log.debug("correct asset: {}".format(asset)) if assets_missing_name: wrong_names = {} diff --git a/openpype/plugins/publish/validate_file_saved.py b/openpype/plugins/publish/validate_file_saved.py index 448eaccf57..94aadc9358 100644 --- a/openpype/plugins/publish/validate_file_saved.py +++ b/openpype/plugins/publish/validate_file_saved.py @@ -1,5 +1,7 @@ import pyblish.api +from openpype.pipeline.publish import PublishValidationError + class ValidateCurrentSaveFile(pyblish.api.ContextPlugin): """File must be saved before publishing""" @@ -12,4 +14,4 @@ class ValidateCurrentSaveFile(pyblish.api.ContextPlugin): current_file = context.data["currentFile"] if not current_file: - raise RuntimeError("File not saved") + raise PublishValidationError("File not saved") diff --git a/openpype/plugins/publish/validate_filesequences.py b/openpype/plugins/publish/validate_filesequences.py index 8a877d79bb..0ac281022d 100644 --- a/openpype/plugins/publish/validate_filesequences.py +++ b/openpype/plugins/publish/validate_filesequences.py @@ -1,5 +1,7 @@ import pyblish.api +from openpype.pipeline.publish import PublishValidationError + class ValidateFileSequences(pyblish.api.ContextPlugin): """Validates whether any file sequences were collected.""" @@ -10,4 +12,5 @@ class ValidateFileSequences(pyblish.api.ContextPlugin): label = "Validate File Sequences" def process(self, context): - assert context, "Nothing collected." + if not context: + raise PublishValidationError("Nothing collected.") diff --git a/openpype/plugins/publish/validate_intent.py b/openpype/plugins/publish/validate_intent.py index 23d57bb2b7..832c7cc0a1 100644 --- a/openpype/plugins/publish/validate_intent.py +++ b/openpype/plugins/publish/validate_intent.py @@ -1,7 +1,7 @@ -import os import pyblish.api from openpype.lib import filter_profiles +from openpype.pipeline.publish import PublishValidationError class ValidateIntent(pyblish.api.ContextPlugin): @@ -51,12 +51,10 @@ class ValidateIntent(pyblish.api.ContextPlugin): )) return - msg = ( - "Please make sure that you select the intent of this publish." - ) - intent = context.data.get("intent") or {} self.log.debug(str(intent)) intent_value = intent.get("value") if not intent_value: - raise AssertionError(msg) + raise PublishValidationError( + "Please make sure that you select the intent of this publish." + ) diff --git a/openpype/plugins/publish/validate_publish_dir.py b/openpype/plugins/publish/validate_publish_dir.py index 2f41127548..0eb93da583 100644 --- a/openpype/plugins/publish/validate_publish_dir.py +++ b/openpype/plugins/publish/validate_publish_dir.py @@ -7,12 +7,12 @@ from openpype.pipeline.publish import ( class ValidatePublishDir(pyblish.api.InstancePlugin): - """Validates if 'publishDir' is a project directory + """Validates if files are being published into a project directory - 'publishDir' is collected based on publish templates. In specific cases - ('source' template) source folder of items is used as a 'publishDir', this - validates if it is inside any project dir for the project. - (eg. files are not published from local folder, unaccessible for studio' + In specific cases ('source' template - in place publishing) source folder + of published items is used as a regular `publish` dir. + This validates if it is inside any project dir for the project. + (eg. files are not published from local folder, inaccessible for studio') """ @@ -44,23 +44,27 @@ class ValidatePublishDir(pyblish.api.InstancePlugin): anatomy = instance.context.data["anatomy"] + # original_dirname must be convertable to rootless path + # in other case it is path inside of root folder for the project success, _ = anatomy.find_root_template_from_path(original_dirname) - - formatting_data = { - "original_dirname": original_dirname, - } - msg = "Path '{}' not in project folder.".format(original_dirname) + \ - " Please publish from inside of project folder." if not success: - raise PublishXmlValidationError(self, msg, key="not_in_dir", - formatting_data=formatting_data) + raise PublishXmlValidationError( + plugin=self, + message=( + "Path '{}' not in project folder. Please publish from " + "inside of project folder.".format(original_dirname) + ), + key="not_in_dir", + formatting_data={"original_dirname": original_dirname} + ) def _get_template_name_from_instance(self, instance): + """Find template which will be used during integration.""" project_name = instance.context.data["projectName"] host_name = instance.context.data["hostName"] anatomy_data = instance.data["anatomyData"] family = anatomy_data["family"] - family = self.family_mapping.get("family") or family + family = self.family_mapping.get(family) or family task_info = anatomy_data.get("task") or {} return get_publish_template_name( diff --git a/openpype/plugins/publish/validate_version.py b/openpype/plugins/publish/validate_version.py index 2b919a3119..84d52fab73 100644 --- a/openpype/plugins/publish/validate_version.py +++ b/openpype/plugins/publish/validate_version.py @@ -25,16 +25,16 @@ class ValidateVersion(pyblish.api.InstancePlugin): # TODO: Remove full non-html version upon drop of old publisher msg = ( "Version '{0}' from instance '{1}' that you are " - " trying to publish is lower or equal to an existing version " - " in the database. Version in database: '{2}'." + "trying to publish is lower or equal to an existing version " + "in the database. Version in database: '{2}'." "Please version up your workfile to a higher version number " "than: '{2}'." ).format(version, instance.data["name"], latest_version) msg_html = ( "Version {0} from instance {1} that you are " - " trying to publish is lower or equal to an existing version " - " in the database. Version in database: {2}.

" + "trying to publish is lower or equal to an existing version " + "in the database. Version in database: {2}.

" "Please version up your workfile to a higher version number " "than: {2}." ).format(version, instance.data["name"], latest_version) diff --git a/openpype/scripts/export_maya_ass_job.py b/openpype/scripts/export_maya_ass_job.py deleted file mode 100644 index 16e841ce96..0000000000 --- a/openpype/scripts/export_maya_ass_job.py +++ /dev/null @@ -1,105 +0,0 @@ -"""This module is used for command line exporting of ASS files. - -WARNING: -This need to be rewriten to be able use it in Pype 3! -""" - -import os -import argparse -import logging -import subprocess -import platform - -try: - from shutil import which -except ImportError: - # we are in python < 3.3 - def which(command): - path = os.getenv('PATH') - for p in path.split(os.path.pathsep): - p = os.path.join(p, command) - if os.path.exists(p) and os.access(p, os.X_OK): - return p - -handler = logging.basicConfig() -log = logging.getLogger("Publish Image Sequences") -log.setLevel(logging.DEBUG) - -error_format = "Failed {plugin.__name__}: {error} -- {error.traceback}" - - -def __main__(): - parser = argparse.ArgumentParser() - parser.add_argument("--paths", - nargs="*", - default=[], - help="The filepaths to publish. This can be a " - "directory or a path to a .json publish " - "configuration.") - parser.add_argument("--gui", - default=False, - action="store_true", - help="Whether to run Pyblish in GUI mode.") - - parser.add_argument("--pype", help="Pype root") - - kwargs, args = parser.parse_known_args() - - print("Running pype ...") - auto_pype_root = os.path.dirname(os.path.abspath(__file__)) - auto_pype_root = os.path.abspath(auto_pype_root + "../../../../..") - - auto_pype_root = os.environ.get('OPENPYPE_SETUP_PATH') or auto_pype_root - if os.environ.get('OPENPYPE_SETUP_PATH'): - print("Got Pype location from environment: {}".format( - os.environ.get('OPENPYPE_SETUP_PATH'))) - - pype_command = "openpype.ps1" - if platform.system().lower() == "linux": - pype_command = "pype" - elif platform.system().lower() == "windows": - pype_command = "openpype.bat" - - if kwargs.pype: - pype_root = kwargs.pype - else: - # test if pype.bat / pype is in the PATH - # if it is, which() will return its path and we use that. - # if not, we use auto_pype_root path. Caveat of that one is - # that it can be UNC path and that will not work on windows. - - pype_path = which(pype_command) - - if pype_path: - pype_root = os.path.dirname(pype_path) - else: - pype_root = auto_pype_root - - print("Set pype root to: {}".format(pype_root)) - print("Paths: {}".format(kwargs.paths or [os.getcwd()])) - - # paths = kwargs.paths or [os.environ.get("OPENPYPE_METADATA_FILE")] or [os.getcwd()] # noqa - - mayabatch = os.environ.get("AVALON_APP_NAME").replace("maya", "mayabatch") - args = [ - os.path.join(pype_root, pype_command), - "launch", - "--app", - mayabatch, - "-script", - os.path.join(pype_root, "repos", "pype", - "pype", "scripts", "export_maya_ass_sequence.mel") - ] - - print("Pype command: {}".format(" ".join(args))) - # Forcing forwaring the environment because environment inheritance does - # not always work. - # Cast all values in environment to str to be safe - env = {k: str(v) for k, v in os.environ.items()} - exit_code = subprocess.call(args, env=env) - if exit_code != 0: - raise RuntimeError("Publishing failed.") - - -if __name__ == '__main__': - __main__() diff --git a/openpype/scripts/export_maya_ass_sequence.mel b/openpype/scripts/export_maya_ass_sequence.mel deleted file mode 100644 index b3b9a8543e..0000000000 --- a/openpype/scripts/export_maya_ass_sequence.mel +++ /dev/null @@ -1,67 +0,0 @@ -/* - Script to export specified layer as ass files. - -Attributes: - - scene_file (str): Name of the scene to load. - start (int): Start frame. - end (int): End frame. - step (int): Step size. - output_path (str): File output path. - render_layer (str): Name of render layer. - -*/ - -$scene_file=`getenv "OPENPYPE_ASS_EXPORT_SCENE_FILE"`; -$step=`getenv "OPENPYPE_ASS_EXPORT_STEP"`; -$start=`getenv "OPENPYPE_ASS_EXPORT_START"`; -$end=`getenv "OPENPYPE_ASS_EXPORT_END"`; -$file_path=`getenv "OPENPYPE_ASS_EXPORT_OUTPUT"`; -$render_layer = `getenv "OPENPYPE_ASS_EXPORT_RENDER_LAYER"`; - -print("*** ASS Export Plugin\n"); - -if ($scene_file == "") { - print("!!! cannot determine scene file\n"); - quit -a -ex -1; -} - -if ($step == "") { - print("!!! cannot determine step size\n"); - quit -a -ex -1; -} - -if ($start == "") { - print("!!! cannot determine start frame\n"); - quit -a -ex -1; -} - -if ($end == "") { - print("!!! cannot determine end frame\n"); - quit -a -ex -1; -} - -if ($file_path == "") { - print("!!! cannot determine output file\n"); - quit -a -ex -1; -} - -if ($render_layer == "") { - print("!!! cannot determine render layer\n"); - quit -a -ex -1; -} - - -print(">>> Opening Scene [ " + $scene_file + " ]\n"); - -// open scene -file -o -f $scene_file; - -// switch to render layer -print(">>> Switching layer [ "+ $render_layer + " ]\n"); -editRenderLayerGlobals -currentRenderLayer $render_layer; - -// export -print(">>> Exporting to [ " + $file_path + " ]\n"); -arnoldExportAss -mask 255 -sl 1 -ll 1 -bb 1 -sf $start -se $end -b -fs $step; -print("--- Done\n"); diff --git a/openpype/scripts/fusion_switch_shot.py b/openpype/scripts/fusion_switch_shot.py deleted file mode 100644 index 1cc728226f..0000000000 --- a/openpype/scripts/fusion_switch_shot.py +++ /dev/null @@ -1,241 +0,0 @@ -import os -import re -import sys -import logging - -from openpype.client import get_asset_by_name, get_versions - -# Pipeline imports -from openpype.hosts.fusion import api -import openpype.hosts.fusion.api.lib as fusion_lib - -# Config imports -from openpype.lib import version_up -from openpype.pipeline import ( - install_host, - registered_host, - legacy_io, - get_current_project_name, -) - -from openpype.pipeline.context_tools import get_workdir_from_session -from openpype.pipeline.version_start import get_versioning_start - -log = logging.getLogger("Update Slap Comp") - - -def _format_version_folder(folder): - """Format a version folder based on the filepath - - Args: - folder: file path to a folder - - Returns: - str: new version folder name - """ - - new_version = get_versioning_start( - get_current_project_name(), - "fusion", - family="workfile" - ) - if os.path.isdir(folder): - re_version = re.compile(r"v\d+$") - versions = [i for i in os.listdir(folder) if os.path.isdir(i) - and re_version.match(i)] - if versions: - # ensure the "v" is not included - new_version = int(max(versions)[1:]) + 1 - - version_folder = "v{:03d}".format(new_version) - - return version_folder - - -def _get_fusion_instance(): - fusion = getattr(sys.modules["__main__"], "fusion", None) - if fusion is None: - try: - # Support for FuScript.exe, BlackmagicFusion module for py2 only - import BlackmagicFusion as bmf - fusion = bmf.scriptapp("Fusion") - except ImportError: - raise RuntimeError("Could not find a Fusion instance") - return fusion - - -def _format_filepath(session): - - project = session["AVALON_PROJECT"] - asset = session["AVALON_ASSET"] - - # Save updated slap comp - work_path = get_workdir_from_session(session) - walk_to_dir = os.path.join(work_path, "scenes", "slapcomp") - slapcomp_dir = os.path.abspath(walk_to_dir) - - # Ensure destination exists - if not os.path.isdir(slapcomp_dir): - log.warning("Folder did not exist, creating folder structure") - os.makedirs(slapcomp_dir) - - # Compute output path - new_filename = "{}_{}_slapcomp_v001.comp".format(project, asset) - new_filepath = os.path.join(slapcomp_dir, new_filename) - - # Create new unqiue filepath - if os.path.exists(new_filepath): - new_filepath = version_up(new_filepath) - - return new_filepath - - -def _update_savers(comp, session): - """Update all savers of the current comp to ensure the output is correct - - Args: - comp (object): current comp instance - session (dict): the current Avalon session - - Returns: - None - """ - - new_work = get_workdir_from_session(session) - renders = os.path.join(new_work, "renders") - version_folder = _format_version_folder(renders) - renders_version = os.path.join(renders, version_folder) - - comp.Print("New renders to: %s\n" % renders) - - with api.comp_lock_and_undo_chunk(comp): - savers = comp.GetToolList(False, "Saver").values() - for saver in savers: - filepath = saver.GetAttrs("TOOLST_Clip_Name")[1.0] - filename = os.path.basename(filepath) - new_path = os.path.join(renders_version, filename) - saver["Clip"] = new_path - - -def update_frame_range(comp, representations): - """Update the frame range of the comp and render length - - The start and end frame are based on the lowest start frame and the highest - end frame - - Args: - comp (object): current focused comp - representations (list) collection of dicts - - Returns: - None - - """ - - version_ids = [r["parent"] for r in representations] - project_name = get_current_project_name() - versions = list(get_versions(project_name, version_ids=version_ids)) - - start = min(v["data"]["frameStart"] for v in versions) - end = max(v["data"]["frameEnd"] for v in versions) - - fusion_lib.update_frame_range(start, end, comp=comp) - - -def switch(asset_name, filepath=None, new=True): - """Switch the current containers of the file to the other asset (shot) - - Args: - filepath (str): file path of the comp file - asset_name (str): name of the asset (shot) - new (bool): Save updated comp under a different name - - Returns: - comp path (str): new filepath of the updated comp - - """ - - # If filepath provided, ensure it is valid absolute path - if filepath is not None: - if not os.path.isabs(filepath): - filepath = os.path.abspath(filepath) - - assert os.path.exists(filepath), "%s must exist " % filepath - - # Assert asset name exists - # It is better to do this here then to wait till switch_shot does it - project_name = get_current_project_name() - asset = get_asset_by_name(project_name, asset_name) - assert asset, "Could not find '%s' in the database" % asset_name - - # Go to comp - if not filepath: - current_comp = api.get_current_comp() - assert current_comp is not None, "Could not find current comp" - else: - fusion = _get_fusion_instance() - current_comp = fusion.LoadComp(filepath, quiet=True) - assert current_comp is not None, "Fusion could not load '%s'" % filepath - - host = registered_host() - containers = list(host.ls()) - assert containers, "Nothing to update" - - representations = [] - for container in containers: - try: - representation = fusion_lib.switch_item(container, - asset_name=asset_name) - representations.append(representation) - except Exception as e: - current_comp.Print("Error in switching! %s\n" % e.message) - - message = "Switched %i Loaders of the %i\n" % (len(representations), - len(containers)) - current_comp.Print(message) - - # Build the session to switch to - switch_to_session = legacy_io.Session.copy() - switch_to_session["AVALON_ASSET"] = asset['name'] - - if new: - comp_path = _format_filepath(switch_to_session) - - # Update savers output based on new session - _update_savers(current_comp, switch_to_session) - else: - comp_path = version_up(filepath) - - current_comp.Print(comp_path) - - current_comp.Print("\nUpdating frame range") - update_frame_range(current_comp, representations) - - current_comp.Save(comp_path) - - return comp_path - - -if __name__ == '__main__': - - import argparse - - parser = argparse.ArgumentParser(description="Switch to a shot within an" - "existing comp file") - - parser.add_argument("--file_path", - type=str, - default=True, - help="File path of the comp to use") - - parser.add_argument("--asset_name", - type=str, - default=True, - help="Name of the asset (shot) to switch") - - args, unknown = parser.parse_args() - - install_host(api) - switch(args.asset_name, args.file_path) - - sys.exit(0) diff --git a/openpype/scripts/ocio_wrapper.py b/openpype/scripts/ocio_wrapper.py index 56399f10a2..c362670126 100644 --- a/openpype/scripts/ocio_wrapper.py +++ b/openpype/scripts/ocio_wrapper.py @@ -319,5 +319,79 @@ def _get_config_file_rules_colorspace_from_filepath(config_path, filepath): return colorspace +def _get_display_view_colorspace_name(config_path, display, view): + """Returns the colorspace attribute of the (display, view) pair. + + Args: + config_path (str): path string leading to config.ocio + display (str): display name e.g. "ACES" + view (str): view name e.g. "sRGB" + + + Raises: + IOError: Input config does not exist. + + Returns: + view color space name (str) e.g. "Output - sRGB" + """ + + config_path = Path(config_path) + + if not config_path.is_file(): + raise IOError("Input path should be `config.ocio` file") + + config = ocio.Config.CreateFromFile(str(config_path)) + colorspace = config.getDisplayViewColorSpaceName(display, view) + + return colorspace + + +@config.command( + name="get_display_view_colorspace_name", + help=( + "return default view colorspace name " + "for the given display and view " + "--path input arg is required" + ) +) +@click.option("--in_path", required=True, + help="path where to read ocio config file", + type=click.Path(exists=True)) +@click.option("--out_path", required=True, + help="path where to write output json file", + type=click.Path()) +@click.option("--display", required=True, + help="display name", + type=click.STRING) +@click.option("--view", required=True, + help="view name", + type=click.STRING) +def get_display_view_colorspace_name(in_path, out_path, + display, view): + """Aggregate view colorspace name to file. + + Wrapper command for processes without access to OpenColorIO + + Args: + in_path (str): config file path string + out_path (str): temp json file path string + display (str): display name e.g. "ACES" + view (str): view name e.g. "sRGB" + + Example of use: + > pyton.exe ./ocio_wrapper.py config \ + get_display_view_colorspace_name --in_path= \ + --out_path= --display= --view= + """ + + out_data = _get_display_view_colorspace_name(in_path, + display, + view) + + with open(out_path, "w") as f: + json.dump(out_data, f) + + print(f"Display view colorspace saved to '{out_path}'") + if __name__ == '__main__': main() diff --git a/openpype/settings/ayon_settings.py b/openpype/settings/ayon_settings.py index 50abfe4839..9a4f0607e0 100644 --- a/openpype/settings/ayon_settings.py +++ b/openpype/settings/ayon_settings.py @@ -616,6 +616,23 @@ def _convert_maya_project_settings(ayon_settings, output): output["maya"] = ayon_maya +def _convert_3dsmax_project_settings(ayon_settings, output): + if "max" not in ayon_settings: + return + + ayon_max = ayon_settings["max"] + _convert_host_imageio(ayon_max) + if "PointCloud" in ayon_max: + point_cloud_attribute = ayon_max["PointCloud"]["attribute"] + new_point_cloud_attribute = { + item["name"]: item["value"] + for item in point_cloud_attribute + } + ayon_max["PointCloud"]["attribute"] = new_point_cloud_attribute + + output["max"] = ayon_max + + def _convert_nuke_knobs(knobs): new_knobs = [] for knob in knobs: @@ -737,6 +754,17 @@ def _convert_nuke_project_settings(ayon_settings, output): item_filter["subsets"] = item_filter.pop("product_names") item_filter["families"] = item_filter.pop("product_types") + reformat_nodes_config = item.get("reformat_nodes_config") or {} + reposition_nodes = reformat_nodes_config.get( + "reposition_nodes") or [] + + for reposition_node in reposition_nodes: + if "knobs" not in reposition_node: + continue + reposition_node["knobs"] = _convert_nuke_knobs( + reposition_node["knobs"] + ) + name = item.pop("name") new_review_data_outputs[name] = item ayon_publish["ExtractReviewDataMov"]["outputs"] = new_review_data_outputs @@ -1261,6 +1289,7 @@ def convert_project_settings(ayon_settings, default_settings): _convert_flame_project_settings(ayon_settings, output) _convert_fusion_project_settings(ayon_settings, output) _convert_maya_project_settings(ayon_settings, output) + _convert_3dsmax_project_settings(ayon_settings, output) _convert_nuke_project_settings(ayon_settings, output) _convert_hiero_project_settings(ayon_settings, output) _convert_photoshop_project_settings(ayon_settings, output) diff --git a/openpype/settings/defaults/project_settings/houdini.json b/openpype/settings/defaults/project_settings/houdini.json index 9d047c28bd..5392fc34dd 100644 --- a/openpype/settings/defaults/project_settings/houdini.json +++ b/openpype/settings/defaults/project_settings/houdini.json @@ -19,6 +19,19 @@ ], "ext": ".ass" }, + "CreateStaticMesh": { + "enabled": true, + "default_variants": [ + "Main" + ], + "static_mesh_prefix": "S", + "collision_prefixes": [ + "UBX", + "UCP", + "USP", + "UCX" + ] + }, "CreateAlembicCamera": { "enabled": true, "default_variants": [ @@ -93,10 +106,30 @@ "$JOB" ] }, + "ValidateReviewColorspace": { + "enabled": true, + "optional": true, + "active": true + }, "ValidateContainers": { "enabled": true, "optional": true, "active": true + }, + "ValidateSubsetName": { + "enabled": true, + "optional": true, + "active": true + }, + "ValidateMeshIsStatic": { + "enabled": true, + "optional": true, + "active": true + }, + "ValidateUnrealStaticMeshName": { + "enabled": false, + "optional": true, + "active": true } } } diff --git a/openpype/settings/defaults/project_settings/nuke.json b/openpype/settings/defaults/project_settings/nuke.json index b736c462ff..7961e77113 100644 --- a/openpype/settings/defaults/project_settings/nuke.json +++ b/openpype/settings/defaults/project_settings/nuke.json @@ -28,11 +28,7 @@ "colorManagement": "Nuke", "OCIO_config": "nuke-default", "workingSpaceLUT": "linear", - "monitorLut": "sRGB", - "int8Lut": "sRGB", - "int16Lut": "sRGB", - "logLut": "Cineon", - "floatLut": "linear" + "monitorLut": "sRGB" }, "nodes": { "requiredNodes": [ diff --git a/openpype/settings/entities/schemas/projects_schema/schemas/schema_houdini_create.json b/openpype/settings/entities/schemas/projects_schema/schemas/schema_houdini_create.json index 799bc0e81a..cd8c260124 100644 --- a/openpype/settings/entities/schemas/projects_schema/schemas/schema_houdini_create.json +++ b/openpype/settings/entities/schemas/projects_schema/schemas/schema_houdini_create.json @@ -39,6 +39,37 @@ ] }, + { + "type": "dict", + "collapsible": true, + "key": "CreateStaticMesh", + "label": "Create Static Mesh", + "checkbox_key": "enabled", + "children": [ + { + "type": "boolean", + "key": "enabled", + "label": "Enabled" + }, + { + "type": "list", + "key": "default_variants", + "label": "Default Variants", + "object_type": "text" + }, + { + "type": "text", + "key": "static_mesh_prefix", + "label": "Static Mesh Prefix" + }, + { + "type": "list", + "key": "collision_prefixes", + "label": "Collision Mesh Prefixes", + "object_type": "text" + } + ] + }, { "type": "schema_template", "name": "template_create_plugin", diff --git a/openpype/settings/entities/schemas/projects_schema/schemas/schema_houdini_publish.json b/openpype/settings/entities/schemas/projects_schema/schemas/schema_houdini_publish.json index aa6eaf5164..d5f70b0312 100644 --- a/openpype/settings/entities/schemas/projects_schema/schemas/schema_houdini_publish.json +++ b/openpype/settings/entities/schemas/projects_schema/schemas/schema_houdini_publish.json @@ -40,11 +40,27 @@ "type": "schema_template", "name": "template_publish_plugin", "template_data": [ + { + "key": "ValidateReviewColorspace", + "label": "Validate Review Colorspace" + }, { "key": "ValidateContainers", "label": "ValidateContainers" + }, + { + "key": "ValidateSubsetName", + "label": "Validate Subset Name" + }, + { + "key": "ValidateMeshIsStatic", + "label": "Validate Mesh is Static" + }, + { + "key": "ValidateUnrealStaticMeshName", + "label": "Validate Unreal Static Mesh Name" } ] } ] -} \ No newline at end of file +} diff --git a/openpype/settings/entities/schemas/projects_schema/schemas/schema_maya_render_settings.json b/openpype/settings/entities/schemas/projects_schema/schemas/schema_maya_render_settings.json index 636dfa114c..fc4e750e3b 100644 --- a/openpype/settings/entities/schemas/projects_schema/schemas/schema_maya_render_settings.json +++ b/openpype/settings/entities/schemas/projects_schema/schemas/schema_maya_render_settings.json @@ -12,7 +12,7 @@ { "type": "text", "key": "default_render_image_folder", - "label": "Default render image folder" + "label": "Default render image folder. This setting can be\noverwritten by custom staging directory profile;\n\"project_settings/global/tools/publish\n/custom_staging_dir_profiles\"." }, { "type": "boolean", diff --git a/openpype/settings/entities/schemas/projects_schema/schemas/schema_nuke_imageio.json b/openpype/settings/entities/schemas/projects_schema/schemas/schema_nuke_imageio.json index d4cd332ef8..af826fcf46 100644 --- a/openpype/settings/entities/schemas/projects_schema/schemas/schema_nuke_imageio.json +++ b/openpype/settings/entities/schemas/projects_schema/schemas/schema_nuke_imageio.json @@ -106,26 +106,6 @@ "type": "text", "key": "monitorLut", "label": "monitor" - }, - { - "type": "text", - "key": "int8Lut", - "label": "8-bit files" - }, - { - "type": "text", - "key": "int16Lut", - "label": "16-bit files" - }, - { - "type": "text", - "key": "logLut", - "label": "log files" - }, - { - "type": "text", - "key": "floatLut", - "label": "float files" } ] } diff --git a/openpype/style/style.css b/openpype/style/style.css index 5ce55aa658..ca368f84f8 100644 --- a/openpype/style/style.css +++ b/openpype/style/style.css @@ -1427,6 +1427,10 @@ CreateNextPageOverlay { background: rgba(0, 0, 0, 127); } +#OverlayFrameLabel { + font-size: 15pt; +} + #BreadcrumbsPathInput { padding: 2px; font-size: 9pt; diff --git a/openpype/tools/attribute_defs/widgets.py b/openpype/tools/attribute_defs/widgets.py index 7967416e9f..d9c55f4a64 100644 --- a/openpype/tools/attribute_defs/widgets.py +++ b/openpype/tools/attribute_defs/widgets.py @@ -19,6 +19,7 @@ from openpype.tools.utils import ( CustomTextComboBox, FocusSpinBox, FocusDoubleSpinBox, + MultiSelectionComboBox, ) from openpype.widgets.nice_checkbox import NiceCheckbox @@ -412,10 +413,19 @@ class EnumAttrWidget(_BaseAttrDefWidget): self._multivalue = False super(EnumAttrWidget, self).__init__(*args, **kwargs) + @property + def multiselection(self): + return self.attr_def.multiselection + def _ui_init(self): - input_widget = CustomTextComboBox(self) - combo_delegate = QtWidgets.QStyledItemDelegate(input_widget) - input_widget.setItemDelegate(combo_delegate) + if self.multiselection: + input_widget = MultiSelectionComboBox(self) + + else: + input_widget = CustomTextComboBox(self) + combo_delegate = QtWidgets.QStyledItemDelegate(input_widget) + input_widget.setItemDelegate(combo_delegate) + self._combo_delegate = combo_delegate if self.attr_def.tooltip: input_widget.setToolTip(self.attr_def.tooltip) @@ -427,9 +437,11 @@ class EnumAttrWidget(_BaseAttrDefWidget): if idx >= 0: input_widget.setCurrentIndex(idx) - input_widget.currentIndexChanged.connect(self._on_value_change) + if self.multiselection: + input_widget.value_changed.connect(self._on_value_change) + else: + input_widget.currentIndexChanged.connect(self._on_value_change) - self._combo_delegate = combo_delegate self._input_widget = input_widget self.main_layout.addWidget(input_widget, 0) @@ -442,17 +454,40 @@ class EnumAttrWidget(_BaseAttrDefWidget): self.value_changed.emit(new_value, self.attr_def.id) def current_value(self): + if self.multiselection: + return self._input_widget.value() idx = self._input_widget.currentIndex() return self._input_widget.itemData(idx) + def _multiselection_multivalue_prep(self, values): + final = None + multivalue = False + for value in values: + value = set(value) + if final is None: + final = value + elif multivalue or final != value: + final |= value + multivalue = True + return list(final), multivalue + def set_value(self, value, multivalue=False): if multivalue: - set_value = set(value) - if len(set_value) == 1: - multivalue = False - value = tuple(set_value)[0] + if self.multiselection: + value, multivalue = self._multiselection_multivalue_prep( + value) + else: + set_value = set(value) + if len(set_value) == 1: + multivalue = False + value = tuple(set_value)[0] - if not multivalue: + if self.multiselection: + self._input_widget.blockSignals(True) + self._input_widget.set_value(value) + self._input_widget.blockSignals(False) + + elif not multivalue: idx = self._input_widget.findData(value) cur_idx = self._input_widget.currentIndex() if idx != cur_idx and idx >= 0: diff --git a/openpype/tools/ayon_workfiles/__init__.py b/openpype/tools/ayon_workfiles/__init__.py new file mode 100644 index 0000000000..e69de29bb2 diff --git a/openpype/tools/ayon_workfiles/abstract.py b/openpype/tools/ayon_workfiles/abstract.py new file mode 100644 index 0000000000..e30a2c2499 --- /dev/null +++ b/openpype/tools/ayon_workfiles/abstract.py @@ -0,0 +1,984 @@ +import os +from abc import ABCMeta, abstractmethod + +import six +from openpype.style import get_default_entity_icon_color + + +class WorkfileInfo: + """Information about workarea file with possible additional from database. + + Args: + folder_id (str): Folder id. + task_id (str): Task id. + filepath (str): Filepath. + filesize (int): File size. + creation_time (int): Creation time (timestamp). + modification_time (int): Modification time (timestamp). + note (str): Note. + """ + + def __init__( + self, + folder_id, + task_id, + filepath, + filesize, + creation_time, + modification_time, + note, + ): + self.folder_id = folder_id + self.task_id = task_id + self.filepath = filepath + self.filesize = filesize + self.creation_time = creation_time + self.modification_time = modification_time + self.note = note + + def to_data(self): + """Converts WorkfileInfo item to data. + + Returns: + dict[str, Any]: Folder item data. + """ + + return { + "folder_id": self.folder_id, + "task_id": self.task_id, + "filepath": self.filepath, + "filesize": self.filesize, + "creation_time": self.creation_time, + "modification_time": self.modification_time, + "note": self.note, + } + + @classmethod + def from_data(cls, data): + """Re-creates WorkfileInfo item from data. + + Args: + data (dict[str, Any]): Workfile info item data. + + Returns: + WorkfileInfo: Workfile info item. + """ + + return cls(**data) + + +class FolderItem: + """Item representing folder entity on a server. + + Folder can be a child of another folder or a project. + + Args: + entity_id (str): Folder id. + parent_id (Union[str, None]): Parent folder id. If 'None' then project + is parent. + name (str): Name of folder. + label (str): Folder label. + icon_name (str): Name of icon from font awesome. + icon_color (str): Hex color string that will be used for icon. + """ + + def __init__( + self, entity_id, parent_id, name, label, icon_name, icon_color + ): + self.entity_id = entity_id + self.parent_id = parent_id + self.name = name + self.icon_name = icon_name or "fa.folder" + self.icon_color = icon_color or get_default_entity_icon_color() + self.label = label or name + + def to_data(self): + """Converts folder item to data. + + Returns: + dict[str, Any]: Folder item data. + """ + + return { + "entity_id": self.entity_id, + "parent_id": self.parent_id, + "name": self.name, + "label": self.label, + "icon_name": self.icon_name, + "icon_color": self.icon_color, + } + + @classmethod + def from_data(cls, data): + """Re-creates folder item from data. + + Args: + data (dict[str, Any]): Folder item data. + + Returns: + FolderItem: Folder item. + """ + + return cls(**data) + + +class TaskItem: + """Task item representing task entity on a server. + + Task is child of a folder. + + Task item has label that is used for display in UI. The label is by + default using task name and type. + + Args: + task_id (str): Task id. + name (str): Name of task. + task_type (str): Type of task. + parent_id (str): Parent folder id. + icon_name (str): Name of icon from font awesome. + icon_color (str): Hex color string that will be used for icon. + """ + + def __init__( + self, task_id, name, task_type, parent_id, icon_name, icon_color + ): + self.task_id = task_id + self.name = name + self.task_type = task_type + self.parent_id = parent_id + self.icon_name = icon_name or "fa.male" + self.icon_color = icon_color or get_default_entity_icon_color() + self._label = None + + @property + def id(self): + """Alias for task_id. + + Returns: + str: Task id. + """ + + return self.task_id + + @property + def label(self): + """Label of task item for UI. + + Returns: + str: Label of task item. + """ + + if self._label is None: + self._label = "{} ({})".format(self.name, self.task_type) + return self._label + + def to_data(self): + """Converts task item to data. + + Returns: + dict[str, Any]: Task item data. + """ + + return { + "task_id": self.task_id, + "name": self.name, + "parent_id": self.parent_id, + "task_type": self.task_type, + "icon_name": self.icon_name, + "icon_color": self.icon_color, + } + + @classmethod + def from_data(cls, data): + """Re-create task item from data. + + Args: + data (dict[str, Any]): Task item data. + + Returns: + TaskItem: Task item. + """ + + return cls(**data) + + +class FileItem: + """File item that represents a file. + + Can be used for both Workarea and Published workfile. Workarea file + will always exist on disk which is not the case for Published workfile. + + Args: + dirpath (str): Directory path of file. + filename (str): Filename. + modified (float): Modified timestamp. + representation_id (Optional[str]): Representation id of published + workfile. + filepath (Optional[str]): Prepared filepath. + exists (Optional[bool]): If file exists on disk. + """ + + def __init__( + self, + dirpath, + filename, + modified, + representation_id=None, + filepath=None, + exists=None + ): + self.filename = filename + self.dirpath = dirpath + self.modified = modified + self.representation_id = representation_id + self._filepath = filepath + self._exists = exists + + @property + def filepath(self): + """Filepath of file. + + Returns: + str: Full path to a file. + """ + + if self._filepath is None: + self._filepath = os.path.join(self.dirpath, self.filename) + return self._filepath + + @property + def exists(self): + """File is available. + + Returns: + bool: If file exists on disk. + """ + + if self._exists is None: + self._exists = os.path.exists(self.filepath) + return self._exists + + def to_data(self): + """Converts file item to data. + + Returns: + dict[str, Any]: File item data. + """ + + return { + "filename": self.filename, + "dirpath": self.dirpath, + "modified": self.modified, + "representation_id": self.representation_id, + "filepath": self.filepath, + "exists": self.exists, + } + + @classmethod + def from_data(cls, data): + """Re-creates file item from data. + + Args: + data (dict[str, Any]): File item data. + + Returns: + FileItem: File item. + """ + + required_keys = { + "filename", + "dirpath", + "modified", + "representation_id" + } + missing_keys = required_keys - set(data.keys()) + if missing_keys: + raise KeyError("Missing keys: {}".format(missing_keys)) + + return cls(**{ + key: data[key] + for key in required_keys + }) + + +class WorkareaFilepathResult: + """Result of workarea file formatting. + + Args: + root (str): Root path of workarea. + filename (str): Filename. + exists (bool): True if file exists. + filepath (str): Filepath. If not provided it will be constructed + from root and filename. + """ + + def __init__(self, root, filename, exists, filepath=None): + if not filepath and root and filename: + filepath = os.path.join(root, filename) + self.root = root + self.filename = filename + self.exists = exists + self.filepath = filepath + + +@six.add_metaclass(ABCMeta) +class AbstractWorkfilesCommon(object): + @abstractmethod + def is_host_valid(self): + """Host is valid for workfiles tool work. + + Returns: + bool: True if host is valid. + """ + + pass + + @abstractmethod + def get_workfile_extensions(self): + """Get possible workfile extensions. + + Defined by host implementation. + + Returns: + Iterable[str]: List of extensions. + """ + + pass + + @abstractmethod + def is_save_enabled(self): + """Is workfile save enabled. + + Returns: + bool: True if save is enabled. + """ + + pass + + @abstractmethod + def set_save_enabled(self, enabled): + """Enable or disabled workfile save. + + Args: + enabled (bool): Enable save workfile when True. + """ + + pass + + +class AbstractWorkfilesBackend(AbstractWorkfilesCommon): + # Current context + @abstractmethod + def get_host_name(self): + """Name of host. + + Returns: + str: Name of host. + """ + pass + + @abstractmethod + def get_current_project_name(self): + """Project name from current context of host. + + Returns: + str: Name of project. + """ + + pass + + @abstractmethod + def get_current_folder_id(self): + """Folder id from current context of host. + + Returns: + Union[str, None]: Folder id or None if host does not have + any context. + """ + + pass + + @abstractmethod + def get_current_task_name(self): + """Task name from current context of host. + + Returns: + Union[str, None]: Task name or None if host does not have + any context. + """ + + pass + + @abstractmethod + def get_current_workfile(self): + """Current workfile from current context of host. + + Returns: + Union[str, None]: Path to workfile or None if host does + not have opened specific file. + """ + + pass + + @property + @abstractmethod + def project_anatomy(self): + """Project anatomy for current project. + + Returns: + Anatomy: Project anatomy. + """ + + pass + + @property + @abstractmethod + def project_settings(self): + """Project settings for current project. + + Returns: + dict[str, Any]: Project settings. + """ + + pass + + @abstractmethod + def get_folder_entity(self, folder_id): + """Get folder entity by id. + + Args: + folder_id (str): Folder id. + + Returns: + dict[str, Any]: Folder entity data. + """ + + pass + + @abstractmethod + def get_task_entity(self, task_id): + """Get task entity by id. + + Args: + task_id (str): Task id. + + Returns: + dict[str, Any]: Task entity data. + """ + + pass + + def emit_event(self, topic, data=None, source=None): + """Emit event. + + Args: + topic (str): Event topic used for callbacks filtering. + data (Optional[dict[str, Any]]): Event data. + source (Optional[str]): Event source. + """ + + pass + + +class AbstractWorkfilesFrontend(AbstractWorkfilesCommon): + """UI controller abstraction that is used for workfiles tool frontend. + + Abstraction to provide data for UI and to handle UI events. + + Provide access to abstract backend data, like folders and tasks. Cares + about handling of selection, keep information about current UI selection + and have ability to tell what selection should UI show. + + Selection is separated into 2 parts, first is what UI elements tell + about selection, and second is what UI should show as selected. + """ + + @abstractmethod + def register_event_callback(self, topic, callback): + """Register event callback. + + Listen for events with given topic. + + Args: + topic (str): Name of topic. + callback (Callable): Callback that will be called when event + is triggered. + """ + + pass + + # Host information + @abstractmethod + def get_workfile_extensions(self): + """Each host can define extensions that can be used for workfile. + + Returns: + List[str]: File extensions that can be used as workfile for + current host. + """ + + pass + + # Selection information + @abstractmethod + def get_selected_folder_id(self): + """Currently selected folder id. + + Returns: + Union[str, None]: Folder id or None if no folder is selected. + """ + + pass + + @abstractmethod + def set_selected_folder(self, folder_id): + """Change selected folder. + + This deselects currently selected task. + + Args: + folder_id (Union[str, None]): Folder id or None if no folder + is selected. + """ + + pass + + @abstractmethod + def get_selected_task_id(self): + """Currently selected task id. + + Returns: + Union[str, None]: Task id or None if no folder is selected. + """ + + pass + + @abstractmethod + def get_selected_task_name(self): + """Currently selected task name. + + Returns: + Union[str, None]: Task name or None if no folder is selected. + """ + + pass + + @abstractmethod + def set_selected_task(self, folder_id, task_id, task_name): + """Change selected task. + + Args: + folder_id (Union[str, None]): Folder id or None if no folder + is selected. + task_id (Union[str, None]): Task id or None if no task + is selected. + task_name (Union[str, None]): Task name or None if no task + is selected. + """ + + pass + + @abstractmethod + def get_selected_workfile_path(self): + """Currently selected workarea workile. + + Returns: + Union[str, None]: Selected workfile path. + """ + + pass + + @abstractmethod + def set_selected_workfile_path(self, path): + """Change selected workfile path. + + Args: + path (Union[str, None]): Selected workfile path. + """ + + pass + + @abstractmethod + def get_selected_representation_id(self): + """Currently selected workfile representation id. + + Returns: + Union[str, None]: Representation id or None if no representation + is selected. + """ + + pass + + @abstractmethod + def set_selected_representation_id(self, representation_id): + """Change selected representation. + + Args: + representation_id (Union[str, None]): Selected workfile + representation id. + """ + + pass + + def get_selected_context(self): + """Obtain selected context. + + Returns: + dict[str, Union[str, None]]: Selected context. + """ + + return { + "folder_id": self.get_selected_folder_id(), + "task_id": self.get_selected_task_id(), + "task_name": self.get_selected_task_name(), + "workfile_path": self.get_selected_workfile_path(), + "representation_id": self.get_selected_representation_id(), + } + + # Expected selection + # - expected selection is used to restore selection after refresh + # or when current context should be used + @abstractmethod + def set_expected_selection( + self, + folder_id, + task_name, + workfile_name=None, + representation_id=None + ): + """Define what should be selected in UI. + + Expected selection provide a way to define/change selection of + sequential UI elements. For example, if folder and task should be + selected a task element should wait until folder element has selected + folder. + + Triggers 'expected_selection.changed' event. + + Args: + folder_id (str): Folder id. + task_name (str): Task name. + workfile_name (Optional[str]): Workfile name. Used for workarea + files UI element. + representation_id (Optional[str]): Representation id. Used for + published filed UI element. + """ + + pass + + @abstractmethod + def get_expected_selection_data(self): + """Data of expected selection. + + TODOs: + Return defined object instead of dict. + + Returns: + dict[str, Any]: Expected selection data. + """ + + pass + + @abstractmethod + def expected_folder_selected(self, folder_id): + """Expected folder was selected in UI. + + Args: + folder_id (str): Folder id which was selected. + """ + + pass + + @abstractmethod + def expected_task_selected(self, folder_id, task_name): + """Expected task was selected in UI. + + Args: + folder_id (str): Folder id under which task is. + task_name (str): Task name which was selected. + """ + + pass + + @abstractmethod + def expected_representation_selected(self, representation_id): + """Expected representation was selected in UI. + + Args: + representation_id (str): Representation id which was selected. + """ + + pass + + @abstractmethod + def expected_workfile_selected(self, workfile_path): + """Expected workfile was selected in UI. + + Args: + workfile_path (str): Workfile path which was selected. + """ + + pass + + @abstractmethod + def go_to_current_context(self): + """Set expected selection to current context.""" + + pass + + # Model functions + @abstractmethod + def get_folder_items(self, sender): + """Folder items to visualize project hierarchy. + + This function may trigger events 'folders.refresh.started' and + 'folders.refresh.finished' which will contain 'sender' value in data. + That may help to avoid re-refresh of folder items in UI elements. + + Args: + sender (str): Who requested folder items. + + Returns: + list[FolderItem]: Minimum possible information needed + for visualisation of folder hierarchy. + """ + + pass + + @abstractmethod + def get_task_items(self, folder_id, sender): + """Task items. + + This function may trigger events 'tasks.refresh.started' and + 'tasks.refresh.finished' which will contain 'sender' value in data. + That may help to avoid re-refresh of task items in UI elements. + + Args: + folder_id (str): Folder ID for which are tasks requested. + sender (str): Who requested folder items. + + Returns: + list[TaskItem]: Minimum possible information needed + for visualisation of tasks. + """ + + pass + + @abstractmethod + def has_unsaved_changes(self): + """Has host unsaved change in currently running session. + + Returns: + bool: Has unsaved changes. + """ + + pass + + @abstractmethod + def get_workarea_dir_by_context(self, folder_id, task_id): + """Get workarea directory by context. + + Args: + folder_id (str): Folder id. + task_id (str): Task id. + + Returns: + str: Workarea directory. + """ + + pass + + @abstractmethod + def get_workarea_file_items(self, folder_id, task_id): + """Get workarea file items. + + Args: + folder_id (str): Folder id. + task_id (str): Task id. + + Returns: + list[FileItem]: List of workarea file items. + """ + + pass + + @abstractmethod + def get_workarea_save_as_data(self, folder_id, task_id): + """Prepare data for Save As operation. + + Todos: + Return defined object instead of dict. + + Args: + folder_id (str): Folder id. + task_id (str): Task id. + + Returns: + dict[str, Any]: Data for Save As operation. + """ + + pass + + @abstractmethod + def fill_workarea_filepath( + self, + folder_id, + task_id, + extension, + use_last_version, + version, + comment, + ): + """Calculate workfile path for passed context. + + Args: + folder_id (str): Folder id. + task_id (str): Task id. + extension (str): File extension. + use_last_version (bool): Use last version. + version (int): Version used if 'use_last_version' if 'False'. + comment (str): User's comment (subversion). + + Returns: + WorkareaFilepathResult: Result of the operation. + """ + + pass + + @abstractmethod + def get_published_file_items(self, folder_id, task_id): + """Get published file items. + + Args: + folder_id (str): Folder id. + task_id (Union[str, None]): Task id. + + Returns: + list[FileItem]: List of published file items. + """ + + pass + + @abstractmethod + def get_workfile_info(self, folder_id, task_id, filepath): + """Workfile info from database. + + Args: + folder_id (str): Folder id. + task_id (str): Task id. + filepath (str): Workfile path. + + Returns: + Union[WorkfileInfo, None]: Workfile info or None if was passed + invalid context. + """ + + pass + + @abstractmethod + def save_workfile_info(self, folder_id, task_id, filepath, note): + """Save workfile info to database. + + At this moment the only information which can be saved about + workfile is 'note'. + + Args: + folder_id (str): Folder id. + task_id (str): Task id. + filepath (str): Workfile path. + note (str): Note. + """ + + pass + + # General commands + @abstractmethod + def refresh(self): + """Refresh everything, models, ui etc. + + Triggers 'controller.refresh.started' event at the beginning and + 'controller.refresh.finished' at the end. + """ + + pass + + # Controller actions + @abstractmethod + def open_workfile(self, filepath): + """Open a workfile. + + Args: + filepath (str): Workfile path. + """ + + pass + + @abstractmethod + def save_current_workfile(self): + """Save state of current workfile.""" + + pass + + @abstractmethod + def save_as_workfile( + self, + folder_id, + task_id, + workdir, + filename, + template_key, + ): + """Save current state of workfile to workarea. + + Args: + folder_id (str): Folder id. + task_id (str): Task id. + workdir (str): Workarea directory. + filename (str): Workarea filename. + template_key (str): Template key used to get the workdir + and filename. + """ + + pass + + @abstractmethod + def copy_workfile_representation( + self, + representation_id, + representation_filepath, + folder_id, + task_id, + workdir, + filename, + template_key, + ): + """Action to copy published workfile representation to workarea. + + Triggers 'copy_representation.started' event on start and + 'copy_representation.finished' event with '{"failed": bool}'. + + Args: + representation_id (str): Representation id. + representation_filepath (str): Path to representation file. + folder_id (str): Folder id. + task_id (str): Task id. + workdir (str): Workarea directory. + filename (str): Workarea filename. + template_key (str): Template key. + """ + + pass + + @abstractmethod + def duplicate_workfile(self, src_filepath, workdir, filename): + """Duplicate workfile. + + Workfiles is not opened when done. + + Args: + src_filepath (str): Source workfile path. + workdir (str): Destination workdir. + filename (str): Destination filename. + """ + + pass diff --git a/openpype/tools/ayon_workfiles/control.py b/openpype/tools/ayon_workfiles/control.py new file mode 100644 index 0000000000..fc8819bff3 --- /dev/null +++ b/openpype/tools/ayon_workfiles/control.py @@ -0,0 +1,642 @@ +import os +import shutil + +import ayon_api + +from openpype.client import get_asset_by_id +from openpype.host import IWorkfileHost +from openpype.lib import Logger, emit_event +from openpype.lib.events import QueuedEventSystem +from openpype.settings import get_project_settings +from openpype.pipeline import Anatomy, registered_host +from openpype.pipeline.context_tools import ( + change_current_context, + get_current_host_name, + get_global_context, +) +from openpype.pipeline.workfile import create_workdir_extra_folders + +from .abstract import ( + AbstractWorkfilesFrontend, + AbstractWorkfilesBackend, +) +from .models import SelectionModel, EntitiesModel, WorkfilesModel + + +class ExpectedSelection: + def __init__(self): + self._folder_id = None + self._task_name = None + self._workfile_name = None + self._representation_id = None + self._folder_selected = True + self._task_selected = True + self._workfile_name_selected = True + self._representation_id_selected = True + + def set_expected_selection( + self, + folder_id, + task_name, + workfile_name=None, + representation_id=None + ): + self._folder_id = folder_id + self._task_name = task_name + self._workfile_name = workfile_name + self._representation_id = representation_id + self._folder_selected = False + self._task_selected = False + self._workfile_name_selected = workfile_name is None + self._representation_id_selected = representation_id is None + + def get_expected_selection_data(self): + return { + "folder_id": self._folder_id, + "task_name": self._task_name, + "workfile_name": self._workfile_name, + "representation_id": self._representation_id, + "folder_selected": self._folder_selected, + "task_selected": self._task_selected, + "workfile_name_selected": self._workfile_name_selected, + "representation_id_selected": self._representation_id_selected, + } + + def is_expected_folder_selected(self, folder_id): + return folder_id == self._folder_id and self._folder_selected + + def is_expected_task_selected(self, folder_id, task_name): + if not self.is_expected_folder_selected(folder_id): + return False + return task_name == self._task_name and self._task_selected + + def expected_folder_selected(self, folder_id): + if folder_id != self._folder_id: + return False + self._folder_selected = True + return True + + def expected_task_selected(self, folder_id, task_name): + if not self.is_expected_folder_selected(folder_id): + return False + + if task_name != self._task_name: + return False + + self._task_selected = True + return True + + def expected_workfile_selected(self, folder_id, task_name, workfile_name): + if not self.is_expected_task_selected(folder_id, task_name): + return False + + if workfile_name != self._workfile_name: + return False + self._workfile_name_selected = True + return True + + def expected_representation_selected( + self, folder_id, task_name, representation_id + ): + if not self.is_expected_task_selected(folder_id, task_name): + return False + if representation_id != self._representation_id: + return False + self._representation_id_selected = True + return True + + +class BaseWorkfileController( + AbstractWorkfilesFrontend, AbstractWorkfilesBackend +): + def __init__(self, host=None): + if host is None: + host = registered_host() + + host_is_valid = False + if host is not None: + missing_methods = ( + IWorkfileHost.get_missing_workfile_methods(host) + ) + host_is_valid = len(missing_methods) == 0 + + self._host = host + self._host_is_valid = host_is_valid + + self._project_anatomy = None + self._project_settings = None + self._event_system = None + self._log = None + + self._current_project_name = None + self._current_folder_name = None + self._current_folder_id = None + self._current_task_name = None + self._save_is_enabled = True + + # Expected selected folder and task + self._expected_selection = self._create_expected_selection_obj() + + self._selection_model = self._create_selection_model() + self._entities_model = self._create_entities_model() + self._workfiles_model = self._create_workfiles_model() + + @property + def log(self): + if self._log is None: + self._log = Logger.get_logger("WorkfilesUI") + return self._log + + def is_host_valid(self): + return self._host_is_valid + + def _create_expected_selection_obj(self): + return ExpectedSelection() + + def _create_selection_model(self): + return SelectionModel(self) + + def _create_entities_model(self): + return EntitiesModel(self) + + def _create_workfiles_model(self): + return WorkfilesModel(self) + + @property + def event_system(self): + """Inner event system for workfiles tool controller. + + Is used for communication with UI. Event system is created on demand. + + Returns: + QueuedEventSystem: Event system which can trigger callbacks + for topics. + """ + + if self._event_system is None: + self._event_system = QueuedEventSystem() + return self._event_system + + # ---------------------------------------------------- + # Implementation of methods required for backend logic + # ---------------------------------------------------- + @property + def project_settings(self): + if self._project_settings is None: + self._project_settings = get_project_settings( + self.get_current_project_name()) + return self._project_settings + + @property + def project_anatomy(self): + if self._project_anatomy is None: + self._project_anatomy = Anatomy(self.get_current_project_name()) + return self._project_anatomy + + def get_folder_entity(self, folder_id): + return self._entities_model.get_folder_entity(folder_id) + + def get_task_entity(self, task_id): + return self._entities_model.get_task_entity(task_id) + + # --------------------------------- + # Implementation of abstract methods + # --------------------------------- + def emit_event(self, topic, data=None, source=None): + """Use implemented event system to trigger event.""" + + if data is None: + data = {} + self.event_system.emit(topic, data, source) + + def register_event_callback(self, topic, callback): + self.event_system.add_callback(topic, callback) + + def is_save_enabled(self): + """Is workfile save enabled. + + Returns: + bool: True if save is enabled. + """ + + return self._save_is_enabled + + def set_save_enabled(self, enabled): + """Enable or disabled workfile save. + + Args: + enabled (bool): Enable save workfile when True. + """ + + if self._save_is_enabled == enabled: + return + + self._save_is_enabled = enabled + self._emit_event( + "workfile_save_enable.changed", + {"enabled": enabled} + ) + + # Host information + def get_workfile_extensions(self): + host = self._host + if isinstance(host, IWorkfileHost): + return host.get_workfile_extensions() + return host.file_extensions() + + def has_unsaved_changes(self): + host = self._host + if isinstance(host, IWorkfileHost): + return host.workfile_has_unsaved_changes() + return host.has_unsaved_changes() + + # Current context + def get_host_name(self): + host = self._host + if isinstance(host, IWorkfileHost): + return host.name + return get_current_host_name() + + def _get_host_current_context(self): + if hasattr(self._host, "get_current_context"): + return self._host.get_current_context() + return get_global_context() + + def get_current_project_name(self): + return self._current_project_name + + def get_current_folder_id(self): + return self._current_folder_id + + def get_current_task_name(self): + return self._current_task_name + + def get_current_workfile(self): + host = self._host + if isinstance(host, IWorkfileHost): + return host.get_current_workfile() + return host.current_file() + + # Selection information + def get_selected_folder_id(self): + return self._selection_model.get_selected_folder_id() + + def set_selected_folder(self, folder_id): + self._selection_model.set_selected_folder(folder_id) + + def get_selected_task_id(self): + return self._selection_model.get_selected_task_id() + + def get_selected_task_name(self): + return self._selection_model.get_selected_task_name() + + def set_selected_task(self, folder_id, task_id, task_name): + return self._selection_model.set_selected_task( + folder_id, task_id, task_name) + + def get_selected_workfile_path(self): + return self._selection_model.get_selected_workfile_path() + + def set_selected_workfile_path(self, path): + self._selection_model.set_selected_workfile_path(path) + + def get_selected_representation_id(self): + return self._selection_model.get_selected_representation_id() + + def set_selected_representation_id(self, representation_id): + self._selection_model.set_selected_representation_id( + representation_id) + + def set_expected_selection( + self, + folder_id, + task_name, + workfile_name=None, + representation_id=None + ): + self._expected_selection.set_expected_selection( + folder_id, task_name, workfile_name, representation_id + ) + self._trigger_expected_selection_changed() + + def expected_folder_selected(self, folder_id): + if self._expected_selection.expected_folder_selected(folder_id): + self._trigger_expected_selection_changed() + + def expected_task_selected(self, folder_id, task_name): + if self._expected_selection.expected_task_selected( + folder_id, task_name + ): + self._trigger_expected_selection_changed() + + def expected_workfile_selected(self, folder_id, task_name, workfile_name): + if self._expected_selection.expected_workfile_selected( + folder_id, task_name, workfile_name + ): + self._trigger_expected_selection_changed() + + def expected_representation_selected( + self, folder_id, task_name, representation_id + ): + if self._expected_selection.expected_representation_selected( + folder_id, task_name, representation_id + ): + self._trigger_expected_selection_changed() + + def get_expected_selection_data(self): + return self._expected_selection.get_expected_selection_data() + + def go_to_current_context(self): + self.set_expected_selection( + self._current_folder_id, self._current_task_name + ) + + # Model functions + def get_folder_items(self, sender): + return self._entities_model.get_folder_items(sender) + + def get_task_items(self, folder_id, sender): + return self._entities_model.get_tasks_items(folder_id, sender) + + def get_workarea_dir_by_context(self, folder_id, task_id): + return self._workfiles_model.get_workarea_dir_by_context( + folder_id, task_id) + + def get_workarea_file_items(self, folder_id, task_id): + return self._workfiles_model.get_workarea_file_items( + folder_id, task_id) + + def get_workarea_save_as_data(self, folder_id, task_id): + return self._workfiles_model.get_workarea_save_as_data( + folder_id, task_id) + + def fill_workarea_filepath( + self, + folder_id, + task_id, + extension, + use_last_version, + version, + comment, + ): + return self._workfiles_model.fill_workarea_filepath( + folder_id, + task_id, + extension, + use_last_version, + version, + comment, + ) + + def get_published_file_items(self, folder_id, task_id): + task_name = None + if task_id: + task = self.get_task_entity(task_id) + task_name = task.get("name") + + return self._workfiles_model.get_published_file_items( + folder_id, task_name) + + def get_workfile_info(self, folder_id, task_id, filepath): + return self._workfiles_model.get_workfile_info( + folder_id, task_id, filepath + ) + + def save_workfile_info(self, folder_id, task_id, filepath, note): + self._workfiles_model.save_workfile_info( + folder_id, task_id, filepath, note + ) + + def refresh(self): + if not self._host_is_valid: + self._emit_event("controller.refresh.started") + self._emit_event("controller.refresh.finished") + return + expected_folder_id = self.get_selected_folder_id() + expected_task_name = self.get_selected_task_name() + + self._emit_event("controller.refresh.started") + + context = self._get_host_current_context() + + project_name = context["project_name"] + folder_name = context["asset_name"] + task_name = context["task_name"] + folder_id = None + if folder_name: + folder = ayon_api.get_folder_by_name(project_name, folder_name) + if folder: + folder_id = folder["id"] + + self._project_settings = None + self._project_anatomy = None + + self._current_project_name = project_name + self._current_folder_name = folder_name + self._current_folder_id = folder_id + self._current_task_name = task_name + + if not expected_folder_id: + expected_folder_id = folder_id + expected_task_name = task_name + + self._expected_selection.set_expected_selection( + expected_folder_id, expected_task_name + ) + + self._entities_model.refresh() + + self._emit_event("controller.refresh.finished") + + # Controller actions + def open_workfile(self, filepath): + self._emit_event("open_workfile.started") + + failed = False + try: + self._host_open_workfile(filepath) + + except Exception: + failed = True + self.log.warning("Open of workfile failed", exc_info=True) + + self._emit_event( + "open_workfile.finished", + {"failed": failed}, + ) + + def save_current_workfile(self): + current_file = self.get_current_workfile() + self._host_save_workfile(current_file) + + def save_as_workfile( + self, + folder_id, + task_id, + workdir, + filename, + template_key, + ): + self._emit_event("save_as.started") + + failed = False + try: + self._save_as_workfile( + folder_id, + task_id, + workdir, + filename, + template_key, + ) + except Exception: + failed = True + self.log.warning("Save as failed", exc_info=True) + + self._emit_event( + "save_as.finished", + {"failed": failed}, + ) + + def copy_workfile_representation( + self, + representation_id, + representation_filepath, + folder_id, + task_id, + workdir, + filename, + template_key, + ): + self._emit_event("copy_representation.started") + + failed = False + try: + self._save_as_workfile( + folder_id, + task_id, + workdir, + filename, + template_key, + ) + except Exception: + failed = True + self.log.warning( + "Copy of workfile representation failed", exc_info=True + ) + + self._emit_event( + "copy_representation.finished", + {"failed": failed}, + ) + + def duplicate_workfile(self, src_filepath, workdir, filename): + self._emit_event("workfile_duplicate.started") + + failed = False + try: + dst_filepath = os.path.join(workdir, filename) + shutil.copy(src_filepath, dst_filepath) + except Exception: + failed = True + self.log.warning("Duplication of workfile failed", exc_info=True) + + self._emit_event( + "workfile_duplicate.finished", + {"failed": failed}, + ) + + # Helper host methods that resolve 'IWorkfileHost' interface + def _host_open_workfile(self, filepath): + host = self._host + if isinstance(host, IWorkfileHost): + host.open_workfile(filepath) + else: + host.open_file(filepath) + + def _host_save_workfile(self, filepath): + host = self._host + if isinstance(host, IWorkfileHost): + host.save_workfile(filepath) + else: + host.save_file(filepath) + + def _emit_event(self, topic, data=None): + self.emit_event(topic, data, "controller") + + # Expected selection + # - expected selection is used to restore selection after refresh + # or when current context should be used + def _trigger_expected_selection_changed(self): + self._emit_event( + "expected_selection_changed", + self._expected_selection.get_expected_selection_data(), + ) + + def _save_as_workfile( + self, + folder_id, + task_id, + workdir, + filename, + template_key, + src_filepath=None, + ): + # Trigger before save event + project_name = self.get_current_project_name() + folder = self.get_folder_entity(folder_id) + task = self.get_task_entity(task_id) + task_name = task["name"] + + # QUESTION should the data be different for 'before' and 'after'? + # NOTE keys should be OpenPype compatible + event_data = { + "project_name": project_name, + "folder_id": folder_id, + "asset_id": folder_id, + "asset_name": folder["name"], + "task_id": task_id, + "task_name": task_name, + "host_name": self.get_host_name(), + "filename": filename, + "workdir_path": workdir, + } + emit_event("workfile.save.before", event_data, source="workfiles.tool") + + # Create workfiles root folder + if not os.path.exists(workdir): + self.log.debug("Initializing work directory: %s", workdir) + os.makedirs(workdir) + + # Change context + if ( + folder_id != self.get_current_folder_id() + or task_name != self.get_current_task_name() + ): + # Use OpenPype asset-like object + asset_doc = get_asset_by_id(project_name, folder["id"]) + change_current_context( + asset_doc, + task["name"], + template_key=template_key + ) + + # Save workfile + dst_filepath = os.path.join(workdir, filename) + if src_filepath: + shutil.copyfile(src_filepath, dst_filepath) + self._host_open_workfile(dst_filepath) + else: + self._host_save_workfile(dst_filepath) + + # Create extra folders + create_workdir_extra_folders( + workdir, + self.get_host_name(), + task["taskType"], + task_name, + project_name + ) + + # Trigger after save events + emit_event("workfile.save.after", event_data, source="workfiles.tool") + self.refresh() diff --git a/openpype/tools/ayon_workfiles/models/__init__.py b/openpype/tools/ayon_workfiles/models/__init__.py new file mode 100644 index 0000000000..d906b9e7bd --- /dev/null +++ b/openpype/tools/ayon_workfiles/models/__init__.py @@ -0,0 +1,10 @@ +from .hierarchy import EntitiesModel +from .selection import SelectionModel +from .workfiles import WorkfilesModel + + +__all__ = ( + "SelectionModel", + "EntitiesModel", + "WorkfilesModel", +) diff --git a/openpype/tools/ayon_workfiles/models/hierarchy.py b/openpype/tools/ayon_workfiles/models/hierarchy.py new file mode 100644 index 0000000000..948c0b8a17 --- /dev/null +++ b/openpype/tools/ayon_workfiles/models/hierarchy.py @@ -0,0 +1,225 @@ +"""Hierarchy model that handles folders and tasks. + +The model can be extracted for common usage. In that case it will be required +to add more handling of project name changes. +""" + +import time +import collections +import contextlib + +import ayon_api + +from openpype.tools.ayon_workfiles.abstract import ( + FolderItem, + TaskItem, +) + + +def _get_task_items_from_tasks(tasks): + """ + + Returns: + TaskItem: Task item. + """ + + output = [] + for task in tasks: + folder_id = task["folderId"] + output.append(TaskItem( + task["id"], + task["name"], + task["type"], + folder_id, + None, + None + )) + return output + + +def _get_folder_item_from_hierarchy_item(item): + return FolderItem( + item["id"], + item["parentId"], + item["name"], + item["label"], + None, + None, + ) + + +class CacheItem: + def __init__(self, lifetime=120): + self._lifetime = lifetime + self._last_update = None + self._data = None + + @property + def is_valid(self): + if self._last_update is None: + return False + + return (time.time() - self._last_update) < self._lifetime + + def set_invalid(self, data=None): + self._last_update = None + self._data = data + + def get_data(self): + return self._data + + def update_data(self, data): + self._data = data + self._last_update = time.time() + + +class EntitiesModel(object): + event_source = "entities.model" + + def __init__(self, controller): + folders_cache = CacheItem() + folders_cache.set_invalid({}) + self._folders_cache = folders_cache + self._tasks_cache = {} + + self._folders_by_id = {} + self._tasks_by_id = {} + + self._folders_refreshing = False + self._tasks_refreshing = set() + self._controller = controller + + def reset(self): + self._folders_cache.set_invalid({}) + self._tasks_cache = {} + + self._folders_by_id = {} + self._tasks_by_id = {} + + def refresh(self): + self._refresh_folders_cache() + + def get_folder_items(self, sender): + if not self._folders_cache.is_valid: + self._refresh_folders_cache(sender) + return self._folders_cache.get_data() + + def get_tasks_items(self, folder_id, sender): + if not folder_id: + return [] + + task_cache = self._tasks_cache.get(folder_id) + if task_cache is None or not task_cache.is_valid: + self._refresh_tasks_cache(folder_id, sender) + task_cache = self._tasks_cache.get(folder_id) + return task_cache.get_data() + + def get_folder_entity(self, folder_id): + if folder_id not in self._folders_by_id: + entity = None + if folder_id: + project_name = self._controller.get_current_project_name() + entity = ayon_api.get_folder_by_id(project_name, folder_id) + self._folders_by_id[folder_id] = entity + return self._folders_by_id[folder_id] + + def get_task_entity(self, task_id): + if task_id not in self._tasks_by_id: + entity = None + if task_id: + project_name = self._controller.get_current_project_name() + entity = ayon_api.get_task_by_id(project_name, task_id) + self._tasks_by_id[task_id] = entity + return self._tasks_by_id[task_id] + + @contextlib.contextmanager + def _folder_refresh_event_manager(self, project_name, sender): + self._folders_refreshing = True + self._controller.emit_event( + "folders.refresh.started", + {"project_name": project_name, "sender": sender}, + self.event_source + ) + try: + yield + + finally: + self._controller.emit_event( + "folders.refresh.finished", + {"project_name": project_name, "sender": sender}, + self.event_source + ) + self._folders_refreshing = False + + @contextlib.contextmanager + def _task_refresh_event_manager( + self, project_name, folder_id, sender + ): + self._tasks_refreshing.add(folder_id) + self._controller.emit_event( + "tasks.refresh.started", + { + "project_name": project_name, + "folder_id": folder_id, + "sender": sender, + }, + self.event_source + ) + try: + yield + + finally: + self._controller.emit_event( + "tasks.refresh.finished", + { + "project_name": project_name, + "folder_id": folder_id, + "sender": sender, + }, + self.event_source + ) + self._tasks_refreshing.discard(folder_id) + + def _refresh_folders_cache(self, sender=None): + if self._folders_refreshing: + return + project_name = self._controller.get_current_project_name() + with self._folder_refresh_event_manager(project_name, sender): + folder_items = self._query_folders(project_name) + self._folders_cache.update_data(folder_items) + + def _query_folders(self, project_name): + hierarchy = ayon_api.get_folders_hierarchy(project_name) + + folder_items = {} + hierachy_queue = collections.deque(hierarchy["hierarchy"]) + while hierachy_queue: + item = hierachy_queue.popleft() + folder_item = _get_folder_item_from_hierarchy_item(item) + folder_items[folder_item.entity_id] = folder_item + hierachy_queue.extend(item["children"] or []) + return folder_items + + def _refresh_tasks_cache(self, folder_id, sender=None): + if folder_id in self._tasks_refreshing: + return + + project_name = self._controller.get_current_project_name() + with self._task_refresh_event_manager( + project_name, folder_id, sender + ): + cache_item = self._tasks_cache.get(folder_id) + if cache_item is None: + cache_item = CacheItem() + self._tasks_cache[folder_id] = cache_item + + task_items = self._query_tasks(project_name, folder_id) + cache_item.update_data(task_items) + + def _query_tasks(self, project_name, folder_id): + tasks = list(ayon_api.get_tasks( + project_name, + folder_ids=[folder_id], + fields={"id", "name", "label", "folderId", "type"} + )) + return _get_task_items_from_tasks(tasks) diff --git a/openpype/tools/ayon_workfiles/models/selection.py b/openpype/tools/ayon_workfiles/models/selection.py new file mode 100644 index 0000000000..ad034794d8 --- /dev/null +++ b/openpype/tools/ayon_workfiles/models/selection.py @@ -0,0 +1,91 @@ +class SelectionModel(object): + """Model handling selection changes. + + Triggering events: + - "selection.folder.changed" + - "selection.task.changed" + - "workarea.selection.changed" + - "selection.representation.changed" + """ + + event_source = "selection.model" + + def __init__(self, controller): + self._controller = controller + + self._folder_id = None + self._task_name = None + self._task_id = None + self._workfile_path = None + self._representation_id = None + + def get_selected_folder_id(self): + return self._folder_id + + def set_selected_folder(self, folder_id): + if folder_id == self._folder_id: + return + + self._folder_id = folder_id + self._controller.emit_event( + "selection.folder.changed", + {"folder_id": folder_id}, + self.event_source + ) + + def get_selected_task_name(self): + return self._task_name + + def get_selected_task_id(self): + return self._task_id + + def set_selected_task(self, folder_id, task_id, task_name): + if folder_id != self._folder_id: + self.set_selected_folder(folder_id) + + if task_id == self._task_id: + return + + self._task_name = task_name + self._task_id = task_id + self._controller.emit_event( + "selection.task.changed", + { + "folder_id": folder_id, + "task_name": task_name, + "task_id": task_id + }, + self.event_source + ) + + def get_selected_workfile_path(self): + return self._workfile_path + + def set_selected_workfile_path(self, path): + if path == self._workfile_path: + return + + self._workfile_path = path + self._controller.emit_event( + "workarea.selection.changed", + { + "path": path, + "folder_id": self._folder_id, + "task_name": self._task_name, + "task_id": self._task_id, + }, + self.event_source + ) + + def get_selected_representation_id(self): + return self._representation_id + + def set_selected_representation_id(self, representation_id): + if representation_id == self._representation_id: + return + self._representation_id = representation_id + self._controller.emit_event( + "selection.representation.changed", + {"representation_id": representation_id}, + self.event_source + ) diff --git a/openpype/tools/ayon_workfiles/models/workfiles.py b/openpype/tools/ayon_workfiles/models/workfiles.py new file mode 100644 index 0000000000..eb82f62de3 --- /dev/null +++ b/openpype/tools/ayon_workfiles/models/workfiles.py @@ -0,0 +1,711 @@ +import os +import re +import copy + +import arrow +import ayon_api +from ayon_api.operations import OperationsSession + +from openpype.client import get_project +from openpype.client.operations import ( + prepare_workfile_info_update_data, +) +from openpype.pipeline.template_data import ( + get_template_data, +) +from openpype.pipeline.workfile import ( + get_workdir_with_workdir_data, + get_workfile_template_key, + get_last_workfile_with_version, +) +from openpype.pipeline.version_start import get_versioning_start +from openpype.tools.ayon_workfiles.abstract import ( + WorkareaFilepathResult, + FileItem, + WorkfileInfo, +) + + +def get_folder_template_data(folder): + if not folder: + return {} + parts = folder["path"].split("/") + parts.pop(-1) + hierarchy = "/".join(parts) + return { + "asset": folder["name"], + "folder": { + "name": folder["name"], + "type": folder["folderType"], + "path": folder["path"], + }, + "hierarchy": hierarchy, + } + + +def get_task_template_data(task): + if not task: + return {} + return { + "task": { + "name": task["name"], + "type": task["taskType"] + } + } + + +class CommentMatcher(object): + """Use anatomy and work file data to parse comments from filenames""" + def __init__(self, extensions, file_template, data): + self.fname_regex = None + + if "{comment}" not in file_template: + # Don't look for comment if template doesn't allow it + return + + # Create a regex group for extensions + any_extension = "(?:{})".format( + "|".join(re.escape(ext.lstrip(".")) for ext in extensions) + ) + + # Use placeholders that will never be in the filename + temp_data = copy.deepcopy(data) + temp_data["comment"] = "<>" + temp_data["version"] = "<>" + temp_data["ext"] = "<>" + + fname_pattern = file_template.format_strict(temp_data) + fname_pattern = re.escape(fname_pattern) + + # Replace comment and version with something we can match with regex + replacements = { + "<>": "(.+)", + "<>": "[0-9]+", + "<>": any_extension, + } + for src, dest in replacements.items(): + fname_pattern = fname_pattern.replace(re.escape(src), dest) + + # Match from beginning to end of string to be safe + fname_pattern = "^{}$".format(fname_pattern) + + self.fname_regex = re.compile(fname_pattern) + + def parse_comment(self, filepath): + """Parse the {comment} part from a filename""" + if not self.fname_regex: + return + + fname = os.path.basename(filepath) + match = self.fname_regex.match(fname) + if match: + return match.group(1) + + +class WorkareaModel: + """Workfiles model looking for workfiles in workare folder. + + Workarea folder is usually task and host specific, defined by + anatomy templates. Is looking for files with extensions defined + by host integration. + """ + + def __init__(self, controller): + self._controller = controller + extensions = None + if controller.is_host_valid(): + extensions = controller.get_workfile_extensions() + self._extensions = extensions + self._base_data = None + self._fill_data_by_folder_id = {} + self._task_data_by_folder_id = {} + self._workdir_by_context = {} + + @property + def project_name(self): + return self._controller.get_current_project_name() + + def reset(self): + self._base_data = None + self._fill_data_by_folder_id = {} + self._task_data_by_folder_id = {} + + def _get_base_data(self): + if self._base_data is None: + base_data = get_template_data(get_project(self.project_name)) + base_data["app"] = self._controller.get_host_name() + self._base_data = base_data + return copy.deepcopy(self._base_data) + + def _get_folder_data(self, folder_id): + fill_data = self._fill_data_by_folder_id.get(folder_id) + if fill_data is None: + folder = self._controller.get_folder_entity(folder_id) + fill_data = get_folder_template_data(folder) + self._fill_data_by_folder_id[folder_id] = fill_data + return copy.deepcopy(fill_data) + + def _get_task_data(self, folder_id, task_id): + task_data = self._task_data_by_folder_id.setdefault(folder_id, {}) + if task_id not in task_data: + task = self._controller.get_task_entity(task_id) + if task: + task_data[task_id] = get_task_template_data(task) + return copy.deepcopy(task_data[task_id]) + + def _prepare_fill_data(self, folder_id, task_id): + if not folder_id or not task_id: + return {} + + base_data = self._get_base_data() + folder_data = self._get_folder_data(folder_id) + task_data = self._get_task_data(folder_id, task_id) + + base_data.update(folder_data) + base_data.update(task_data) + + return base_data + + def get_workarea_dir_by_context(self, folder_id, task_id): + if not folder_id or not task_id: + return None + folder_mapping = self._workdir_by_context.setdefault(folder_id, {}) + workdir = folder_mapping.get(task_id) + if workdir is not None: + return workdir + + workdir_data = self._prepare_fill_data(folder_id, task_id) + + workdir = get_workdir_with_workdir_data( + workdir_data, + self.project_name, + anatomy=self._controller.project_anatomy, + ) + folder_mapping[task_id] = workdir + return workdir + + def get_file_items(self, folder_id, task_id): + items = [] + if not folder_id or not task_id: + return items + + workdir = self.get_workarea_dir_by_context(folder_id, task_id) + if not os.path.exists(workdir): + return items + + for filename in os.listdir(workdir): + filepath = os.path.join(workdir, filename) + if not os.path.isfile(filepath): + continue + + ext = os.path.splitext(filename)[1].lower() + if ext not in self._extensions: + continue + + modified = os.path.getmtime(filepath) + items.append( + FileItem(workdir, filename, modified) + ) + return items + + def _get_template_key(self, fill_data): + task_type = fill_data.get("task", {}).get("type") + # TODO cache + return get_workfile_template_key( + task_type, + self._controller.get_host_name(), + project_name=self.project_name + ) + + def _get_last_workfile_version( + self, workdir, file_template, fill_data, extensions + ): + version = get_last_workfile_with_version( + workdir, str(file_template), fill_data, extensions + )[1] + + if version is None: + task_info = fill_data.get("task", {}) + version = get_versioning_start( + self.project_name, + self._controller.get_host_name(), + task_name=task_info.get("name"), + task_type=task_info.get("type"), + family="workfile", + project_settings=self._controller.project_settings, + ) + else: + version += 1 + return version + + def _get_comments_from_root( + self, + file_template, + extensions, + fill_data, + root, + current_filename, + ): + current_comment = None + comment_hints = set() + filenames = [] + if root and os.path.exists(root): + for filename in os.listdir(root): + path = os.path.join(root, filename) + if not os.path.isfile(path): + continue + + ext = os.path.splitext(filename)[-1].lower() + if ext in extensions: + filenames.append(filename) + + if not filenames: + return comment_hints, current_comment + + matcher = CommentMatcher(extensions, file_template, fill_data) + + for filename in filenames: + comment = matcher.parse_comment(filename) + if comment: + comment_hints.add(comment) + if filename == current_filename: + current_comment = comment + + return list(comment_hints), current_comment + + def _get_workdir(self, anatomy, template_key, fill_data): + template_info = anatomy.templates_obj[template_key] + directory_template = template_info["folder"] + return directory_template.format_strict(fill_data).normalized() + + def get_workarea_save_as_data(self, folder_id, task_id): + folder = None + task = None + if folder_id: + folder = self._controller.get_folder_entity(folder_id) + if task_id: + task = self._controller.get_task_entity(task_id) + + if not folder or not task: + return { + "template_key": None, + "template_has_version": None, + "template_has_comment": None, + "ext": None, + "workdir": None, + "comment": None, + "comment_hints": None, + "last_version": None, + "extensions": None, + } + + anatomy = self._controller.project_anatomy + fill_data = self._prepare_fill_data(folder_id, task_id) + template_key = self._get_template_key(fill_data) + + current_workfile = self._controller.get_current_workfile() + current_filename = None + current_ext = None + if current_workfile: + current_filename = os.path.basename(current_workfile) + current_ext = os.path.splitext(current_filename)[1].lower() + + extensions = self._extensions + if not current_ext and extensions: + current_ext = tuple(extensions)[0] + + workdir = self._get_workdir(anatomy, template_key, fill_data) + + template_info = anatomy.templates_obj[template_key] + file_template = template_info["file"] + + comment_hints, comment = self._get_comments_from_root( + file_template, + extensions, + fill_data, + workdir, + current_filename, + ) + last_version = self._get_last_workfile_version( + workdir, file_template, fill_data, extensions) + str_file_template = str(file_template) + template_has_version = "{version" in str_file_template + template_has_comment = "{comment" in str_file_template + + return { + "template_key": template_key, + "template_has_version": template_has_version, + "template_has_comment": template_has_comment, + "ext": current_ext, + "workdir": workdir, + "comment": comment, + "comment_hints": comment_hints, + "last_version": last_version, + "extensions": extensions, + } + + def fill_workarea_filepath( + self, + folder_id, + task_id, + extension, + use_last_version, + version, + comment, + ): + anatomy = self._controller.project_anatomy + fill_data = self._prepare_fill_data(folder_id, task_id) + template_key = self._get_template_key(fill_data) + + workdir = self._get_workdir(anatomy, template_key, fill_data) + + template_info = anatomy.templates_obj[template_key] + file_template = template_info["file"] + + if use_last_version: + version = self._get_last_workfile_version( + workdir, file_template, fill_data, self._extensions + ) + fill_data["version"] = version + fill_data["ext"] = extension.lstrip(".") + + if comment: + fill_data["comment"] = comment + + filename = file_template.format(fill_data) + if not filename.solved: + filename = None + + exists = False + if filename: + filepath = os.path.join(workdir, filename) + exists = os.path.exists(filepath) + + return WorkareaFilepathResult( + workdir, + filename, + exists + ) + + +class WorkfileEntitiesModel: + """Workfile entities model. + + Args: + control (AbstractWorkfileController): Controller object. + """ + + def __init__(self, controller): + self._controller = controller + self._cache = {} + self._items = {} + + def _get_workfile_info_identifier( + self, folder_id, task_id, rootless_path + ): + return "_".join([folder_id, task_id, rootless_path]) + + def _get_rootless_path(self, filepath): + anatomy = self._controller.project_anatomy + + workdir, filename = os.path.split(filepath) + success, rootless_dir = anatomy.find_root_template_from_path(workdir) + return "/".join([ + os.path.normpath(rootless_dir).replace("\\", "/"), + filename + ]) + + def _prepare_workfile_info_item( + self, folder_id, task_id, workfile_info, filepath + ): + note = "" + if workfile_info: + note = workfile_info["attrib"].get("description") or "" + + filestat = os.stat(filepath) + return WorkfileInfo( + folder_id, + task_id, + filepath, + filesize=filestat.st_size, + creation_time=filestat.st_ctime, + modification_time=filestat.st_mtime, + note=note + ) + + def _get_workfile_info(self, folder_id, task_id, identifier): + workfile_info = self._cache.get(identifier) + if workfile_info is not None: + return workfile_info + + for workfile_info in ayon_api.get_workfiles_info( + self._controller.get_current_project_name(), + task_ids=[task_id], + fields=["id", "path", "attrib"], + ): + workfile_identifier = self._get_workfile_info_identifier( + folder_id, task_id, workfile_info["path"] + ) + self._cache[workfile_identifier] = workfile_info + return self._cache.get(identifier) + + def get_workfile_info( + self, folder_id, task_id, filepath, rootless_path=None + ): + if not folder_id or not task_id or not filepath: + return None + + if rootless_path is None: + rootless_path = self._get_rootless_path(filepath) + + identifier = self._get_workfile_info_identifier( + folder_id, task_id, rootless_path) + item = self._items.get(identifier) + if item is None: + workfile_info = self._get_workfile_info( + folder_id, task_id, identifier + ) + item = self._prepare_workfile_info_item( + folder_id, task_id, workfile_info, filepath + ) + self._items[identifier] = item + return item + + def save_workfile_info(self, folder_id, task_id, filepath, note): + rootless_path = self._get_rootless_path(filepath) + identifier = self._get_workfile_info_identifier( + folder_id, task_id, rootless_path + ) + workfile_info = self._get_workfile_info( + folder_id, task_id, identifier + ) + if not workfile_info: + self._cache[identifier] = self._create_workfile_info_entity( + task_id, rootless_path, note) + self._items.pop(identifier, None) + return + + new_workfile_info = copy.deepcopy(workfile_info) + attrib = new_workfile_info.setdefault("attrib", {}) + attrib["description"] = note + update_data = prepare_workfile_info_update_data( + workfile_info, new_workfile_info + ) + self._cache[identifier] = new_workfile_info + self._items.pop(identifier, None) + if not update_data: + return + + project_name = self._controller.get_current_project_name() + + session = OperationsSession() + session.update_entity( + project_name, "workfile", workfile_info["id"], update_data + ) + session.commit() + + def _create_workfile_info_entity(self, task_id, rootless_path, note): + extension = os.path.splitext(rootless_path)[1] + + project_name = self._controller.get_current_project_name() + + workfile_info = { + "path": rootless_path, + "taskId": task_id, + "attrib": { + "extension": extension, + "description": note + } + } + + session = OperationsSession() + session.create_entity(project_name, "workfile", workfile_info) + session.commit() + return workfile_info + + +class PublishWorkfilesModel: + """Model for handling of published workfiles. + + Todos: + Cache workfiles products and representations for some time. + Note Representations won't change. Only what can change are + versions. + """ + + def __init__(self, controller): + self._controller = controller + self._cached_extensions = None + self._cached_repre_extensions = None + + @property + def _extensions(self): + if self._cached_extensions is None: + exts = self._controller.get_workfile_extensions() or [] + self._cached_extensions = exts + return self._cached_extensions + + @property + def _repre_extensions(self): + if self._cached_repre_extensions is None: + self._cached_repre_extensions = { + ext.lstrip(".") for ext in self._extensions + } + return self._cached_repre_extensions + + def _file_item_from_representation( + self, repre_entity, project_anatomy, task_name=None + ): + if task_name is not None: + task_info = repre_entity["context"].get("task") + if not task_info or task_info["name"] != task_name: + return None + + # Filter by extension + extensions = self._repre_extensions + workfile_path = None + for repre_file in repre_entity["files"]: + ext = ( + os.path.splitext(repre_file["name"])[1] + .lower() + .lstrip(".") + ) + if ext in extensions: + workfile_path = repre_file["path"] + break + + if not workfile_path: + return None + + try: + workfile_path = workfile_path.format( + root=project_anatomy.roots) + except Exception as exc: + print("Failed to format workfile path: {}".format(exc)) + + dirpath, filename = os.path.split(workfile_path) + created_at = arrow.get(repre_entity["createdAt"]) + return FileItem( + dirpath, + filename, + created_at.float_timestamp, + repre_entity["id"] + ) + + def get_file_items(self, folder_id, task_name): + # TODO refactor to use less server API calls + project_name = self._controller.get_current_project_name() + # Get subset docs of asset + product_entities = ayon_api.get_products( + project_name, + folder_ids=[folder_id], + product_types=["workfile"], + fields=["id", "name"] + ) + + output = [] + product_ids = {product["id"] for product in product_entities} + if not product_ids: + return output + + # Get version docs of subsets with their families + version_entities = ayon_api.get_versions( + project_name, + product_ids=product_ids, + fields=["id", "productId"] + ) + version_ids = {version["id"] for version in version_entities} + if not version_ids: + return output + + # Query representations of filtered versions and add filter for + # extension + repre_entities = ayon_api.get_representations( + project_name, + version_ids=version_ids + ) + project_anatomy = self._controller.project_anatomy + + # Filter queried representations by task name if task is set + file_items = [] + for repre_entity in repre_entities: + file_item = self._file_item_from_representation( + repre_entity, project_anatomy, task_name + ) + if file_item is not None: + file_items.append(file_item) + + return file_items + + +class WorkfilesModel: + """Workfiles model.""" + + def __init__(self, controller): + self._controller = controller + + self._entities_model = WorkfileEntitiesModel(controller) + self._workarea_model = WorkareaModel(controller) + self._published_model = PublishWorkfilesModel(controller) + + def get_workfile_info(self, folder_id, task_id, filepath): + return self._entities_model.get_workfile_info( + folder_id, task_id, filepath + ) + + def save_workfile_info(self, folder_id, task_id, filepath, note): + self._entities_model.save_workfile_info( + folder_id, task_id, filepath, note + ) + + def get_workarea_dir_by_context(self, folder_id, task_id): + """Workarea dir for passed context. + + The directory path is based on project anatomy templates. + + Args: + folder_id (str): Folder id. + task_id (str): Task id. + + Returns: + Union[str, None]: Workarea dir path or None for invalid context. + """ + + return self._workarea_model.get_workarea_dir_by_context( + folder_id, task_id) + + def get_workarea_file_items(self, folder_id, task_id): + """Workfile items for passed context from workarea. + + Args: + folder_id (Union[str, None]): Folder id. + task_id (Union[str, None]): Task id. + + Returns: + list[FileItem]: List of file items matching workarea of passed + context. + """ + + return self._workarea_model.get_file_items(folder_id, task_id) + + def get_workarea_save_as_data(self, folder_id, task_id): + return self._workarea_model.get_workarea_save_as_data( + folder_id, task_id) + + def fill_workarea_filepath(self, *args, **kwargs): + return self._workarea_model.fill_workarea_filepath( + *args, **kwargs + ) + + def get_published_file_items(self, folder_id, task_name): + """Published workfiles for passed context. + + Args: + folder_id (str): Folder id. + task_name (str): Task name. + + Returns: + list[FileItem]: List of files for published workfiles. + """ + + return self._published_model.get_file_items(folder_id, task_name) diff --git a/openpype/tools/ayon_workfiles/widgets/__init__.py b/openpype/tools/ayon_workfiles/widgets/__init__.py new file mode 100644 index 0000000000..f0c5ba1c40 --- /dev/null +++ b/openpype/tools/ayon_workfiles/widgets/__init__.py @@ -0,0 +1,6 @@ +from .window import WorkfilesToolWindow + + +__all__ = ( + "WorkfilesToolWindow", +) diff --git a/openpype/tools/ayon_workfiles/widgets/constants.py b/openpype/tools/ayon_workfiles/widgets/constants.py new file mode 100644 index 0000000000..fc74fd9bc4 --- /dev/null +++ b/openpype/tools/ayon_workfiles/widgets/constants.py @@ -0,0 +1,7 @@ +from qtpy import QtCore + + +ITEM_ID_ROLE = QtCore.Qt.UserRole + 1 +PARENT_ID_ROLE = QtCore.Qt.UserRole + 2 +ITEM_NAME_ROLE = QtCore.Qt.UserRole + 3 +TASK_TYPE_ROLE = QtCore.Qt.UserRole + 4 diff --git a/openpype/tools/ayon_workfiles/widgets/files_widget.py b/openpype/tools/ayon_workfiles/widgets/files_widget.py new file mode 100644 index 0000000000..fbf4dbc593 --- /dev/null +++ b/openpype/tools/ayon_workfiles/widgets/files_widget.py @@ -0,0 +1,398 @@ +import os + +import qtpy +from qtpy import QtWidgets, QtCore + +from .save_as_dialog import SaveAsDialog +from .files_widget_workarea import WorkAreaFilesWidget +from .files_widget_published import PublishedFilesWidget + + +class FilesWidget(QtWidgets.QWidget): + """A widget displaying files that allows to save and open files. + + Args: + controller (AbstractWorkfilesFrontend): The control object. + parent (QtWidgets.QWidget): The parent widget. + """ + + def __init__(self, controller, parent): + super(FilesWidget, self).__init__(parent) + + files_widget = QtWidgets.QStackedWidget(self) + workarea_widget = WorkAreaFilesWidget(controller, files_widget) + published_widget = PublishedFilesWidget(controller, files_widget) + files_widget.addWidget(workarea_widget) + files_widget.addWidget(published_widget) + + btns_widget = QtWidgets.QWidget(self) + + workarea_btns_widget = QtWidgets.QWidget(btns_widget) + workarea_btn_open = QtWidgets.QPushButton( + "Open", workarea_btns_widget) + workarea_btn_browse = QtWidgets.QPushButton( + "Browse", workarea_btns_widget) + workarea_btn_save = QtWidgets.QPushButton( + "Save As", workarea_btns_widget) + + workarea_btns_layout = QtWidgets.QHBoxLayout(workarea_btns_widget) + workarea_btns_layout.setContentsMargins(0, 0, 0, 0) + workarea_btns_layout.addWidget(workarea_btn_open, 1) + workarea_btns_layout.addWidget(workarea_btn_browse, 1) + workarea_btns_layout.addWidget(workarea_btn_save, 1) + + published_btns_widget = QtWidgets.QWidget(btns_widget) + published_btn_copy_n_open = QtWidgets.QPushButton( + "Copy && Open", published_btns_widget + ) + published_btn_change_context = QtWidgets.QPushButton( + "Choose different context", published_btns_widget + ) + published_btn_cancel = QtWidgets.QPushButton( + "Cancel", published_btns_widget + ) + + published_btns_layout = QtWidgets.QHBoxLayout(published_btns_widget) + published_btns_layout.setContentsMargins(0, 0, 0, 0) + published_btns_layout.addWidget(published_btn_copy_n_open, 1) + published_btns_layout.addWidget(published_btn_change_context, 1) + published_btns_layout.addWidget(published_btn_cancel, 1) + + btns_layout = QtWidgets.QVBoxLayout(btns_widget) + btns_layout.setContentsMargins(0, 0, 0, 0) + btns_layout.addWidget(workarea_btns_widget, 1) + btns_layout.addWidget(published_btns_widget, 1) + + main_layout = QtWidgets.QVBoxLayout(self) + main_layout.setContentsMargins(0, 0, 0, 0) + main_layout.addWidget(files_widget, 1) + main_layout.addWidget(btns_widget, 0) + + controller.register_event_callback( + "workarea.selection.changed", + self._on_workarea_path_changed + ) + controller.register_event_callback( + "selection.representation.changed", + self._on_published_repre_changed + ) + controller.register_event_callback( + "selection.task.changed", + self._on_task_changed + ) + controller.register_event_callback( + "copy_representation.finished", + self._on_copy_representation_finished, + ) + controller.register_event_callback( + "workfile_save_enable.changed", + self._on_workfile_save_enabled_change, + ) + + workarea_widget.open_current_requested.connect( + self._on_current_open_requests) + workarea_widget.duplicate_requested.connect( + self._on_duplicate_request) + workarea_btn_open.clicked.connect(self._on_workarea_open_clicked) + workarea_btn_browse.clicked.connect(self._on_workarea_browse_clicked) + workarea_btn_save.clicked.connect(self._on_workarea_save_clicked) + + published_widget.save_as_requested.connect(self._on_save_as_request) + published_btn_copy_n_open.clicked.connect( + self._on_published_save_clicked) + published_btn_change_context.clicked.connect( + self._on_published_change_context_clicked) + published_btn_cancel.clicked.connect( + self._on_published_cancel_clicked) + + self._selected_folder_id = None + self._selected_tak_name = None + + self._pre_select_folder_id = None + self._pre_select_task_name = None + + self._select_context_mode = False + self._valid_selected_context = False + self._valid_representation_id = False + self._tmp_text_filter = None + self._is_save_enabled = True + + self._controller = controller + self._files_widget = files_widget + self._workarea_widget = workarea_widget + self._published_widget = published_widget + self._workarea_btns_widget = workarea_btns_widget + self._published_btns_widget = published_btns_widget + + self._workarea_btn_open = workarea_btn_open + self._workarea_btn_browse = workarea_btn_browse + self._workarea_btn_save = workarea_btn_save + + self._published_widget = published_widget + self._published_btn_copy_n_open = published_btn_copy_n_open + self._published_btn_change_context = published_btn_change_context + self._published_btn_cancel = published_btn_cancel + + # Initial setup + workarea_btn_open.setEnabled(False) + published_btn_copy_n_open.setEnabled(False) + published_btn_change_context.setEnabled(False) + published_btn_cancel.setVisible(False) + + def set_published_mode(self, published_mode): + # Make sure context selection is disabled + self._set_select_contex_mode(False) + # Change current widget + self._files_widget.setCurrentWidget(( + self._published_widget + if published_mode + else self._workarea_widget + )) + # Pass the mode to the widgets, so they can start/stop handle events + self._workarea_widget.set_published_mode(published_mode) + self._published_widget.set_published_mode(published_mode) + + # Change available buttons + self._workarea_btns_widget.setVisible(not published_mode) + self._published_btns_widget.setVisible(published_mode) + + def set_text_filter(self, text_filter): + if self._select_context_mode: + self._tmp_text_filter = text_filter + return + self._workarea_widget.set_text_filter(text_filter) + self._published_widget.set_text_filter(text_filter) + + def _exec_save_as_dialog(self): + """Show SaveAs dialog using currently selected context. + + Returns: + Union[dict[str, Any], None]: Result of the dialog. + """ + + dialog = SaveAsDialog(self._controller, self) + dialog.update_context() + dialog.exec_() + return dialog.get_result() + + # ------------------------------------------------------------- + # Workarea workfiles + # ------------------------------------------------------------- + def _open_workfile(self, filepath): + if self._controller.has_unsaved_changes(): + result = self._save_changes_prompt() + if result is None: + return + + if result: + self._controller.save_current_workfile() + self._controller.open_workfile(filepath) + + def _on_workarea_open_clicked(self): + path = self._workarea_widget.get_selected_path() + if path: + self._open_workfile(path) + + def _on_current_open_requests(self): + self._on_workarea_open_clicked() + + def _on_duplicate_request(self): + filepath = self._workarea_widget.get_selected_path() + if filepath is None: + return + + result = self._exec_save_as_dialog() + if result is None: + return + self._controller.duplicate_workfile( + filepath, + result["workdir"], + result["filename"] + ) + + def _on_workarea_browse_clicked(self): + extnsions = self._controller.get_workfile_extensions() + ext_filter = "Work File (*{0})".format( + " *".join(extnsions) + ) + dir_key = "directory" + if qtpy.API in ("pyside", "pyside2", "pyside6"): + dir_key = "dir" + + selected_context = self._controller.get_selected_context() + workfile_root = self._controller.get_workarea_dir_by_context( + selected_context["folder_id"], selected_context["task_id"] + ) + # Find existing directory of workfile root + # - Qt will use 'cwd' instead, if path does not exist, which may lead + # to igniter directory + while workfile_root: + if os.path.exists(workfile_root): + break + workfile_root = os.path.dirname(workfile_root) + + kwargs = { + "caption": "Work Files", + "filter": ext_filter, + dir_key: workfile_root + } + + filepath = QtWidgets.QFileDialog.getOpenFileName(**kwargs)[0] + if filepath: + self._open_workfile(filepath) + + def _on_workarea_save_clicked(self): + result = self._exec_save_as_dialog() + if result is None: + return + self._controller.save_as_workfile( + result["folder_id"], + result["task_id"], + result["workdir"], + result["filename"], + result["template_key"], + ) + + def _on_workarea_path_changed(self, event): + valid_path = event["path"] is not None + self._workarea_btn_open.setEnabled(valid_path) + + # ------------------------------------------------------------- + # Published workfiles + # ------------------------------------------------------------- + def _update_published_btns_state(self): + enabled = ( + self._valid_representation_id + and self._valid_selected_context + and self._is_save_enabled + ) + self._published_btn_copy_n_open.setEnabled(enabled) + self._published_btn_change_context.setEnabled(enabled) + + def _update_workarea_btns_state(self): + enabled = self._is_save_enabled + self._workarea_btn_save.setEnabled(enabled) + + def _on_published_repre_changed(self, event): + self._valid_representation_id = event["representation_id"] is not None + self._update_published_btns_state() + + def _on_task_changed(self, event): + self._selected_folder_id = event["folder_id"] + self._selected_tak_name = event["task_name"] + self._valid_selected_context = ( + self._selected_folder_id is not None + and self._selected_tak_name is not None + ) + self._update_published_btns_state() + + def _on_published_save_clicked(self): + result = self._exec_save_as_dialog() + if result is None: + return + + repre_info = self._published_widget.get_selected_repre_info() + self._controller.copy_workfile_representation( + repre_info["representation_id"], + repre_info["filepath"], + result["folder_id"], + result["task_id"], + result["workdir"], + result["filename"], + result["template_key"], + ) + + def _on_save_as_request(self): + self._on_published_save_clicked() + + def _set_select_contex_mode(self, enabled): + if self._select_context_mode is enabled: + return + + if enabled: + self._pre_select_folder_id = self._selected_folder_id + self._pre_select_task_name = self._selected_tak_name + else: + self._pre_select_folder_id = None + self._pre_select_task_name = None + self._select_context_mode = enabled + self._published_btn_cancel.setVisible(enabled) + self._published_btn_change_context.setVisible(not enabled) + self._published_widget.set_select_context_mode(enabled) + + if not enabled and self._tmp_text_filter is not None: + self.set_text_filter(self._tmp_text_filter) + self._tmp_text_filter = None + + def _on_published_change_context_clicked(self): + self._set_select_contex_mode(True) + + def _should_set_pre_select_context(self): + if self._pre_select_folder_id is None: + return False + if self._pre_select_folder_id != self._selected_folder_id: + return True + if self._pre_select_task_name is None: + return False + return self._pre_select_task_name != self._selected_tak_name + + def _on_published_cancel_clicked(self): + folder_id = self._pre_select_folder_id + task_name = self._pre_select_task_name + representation_id = self._published_widget.get_selected_repre_id() + should_change_selection = self._should_set_pre_select_context() + self._set_select_contex_mode(False) + if should_change_selection: + self._controller.set_expected_selection( + folder_id, task_name, representation_id=representation_id + ) + + def _on_copy_representation_finished(self, event): + """Callback for when copy representation is finished. + + Make sure that select context mode is disabled when representation + copy is finished. + + Args: + event (Event): Event object. + """ + + if not event["failed"]: + self._set_select_contex_mode(False) + + def _on_workfile_save_enabled_change(self, event): + enabled = event["enabled"] + self._is_save_enabled = enabled + self._update_published_btns_state() + self._update_workarea_btns_state() + + def _save_changes_prompt(self): + """Ask user if wants to save changes to current file. + + Returns: + Union[bool, None]: True if user wants to save changes, False if + user does not want to save changes, None if user cancels + operation. + """ + messagebox = QtWidgets.QMessageBox(parent=self) + messagebox.setWindowFlags( + messagebox.windowFlags() | QtCore.Qt.FramelessWindowHint + ) + messagebox.setIcon(QtWidgets.QMessageBox.Warning) + messagebox.setWindowTitle("Unsaved Changes!") + messagebox.setText( + "There are unsaved changes to the current file." + "\nDo you want to save the changes?" + ) + messagebox.setStandardButtons( + QtWidgets.QMessageBox.Yes + | QtWidgets.QMessageBox.No + | QtWidgets.QMessageBox.Cancel + ) + + result = messagebox.exec_() + if result == QtWidgets.QMessageBox.Yes: + return True + if result == QtWidgets.QMessageBox.No: + return False + return None diff --git a/openpype/tools/ayon_workfiles/widgets/files_widget_published.py b/openpype/tools/ayon_workfiles/widgets/files_widget_published.py new file mode 100644 index 0000000000..bc59447777 --- /dev/null +++ b/openpype/tools/ayon_workfiles/widgets/files_widget_published.py @@ -0,0 +1,378 @@ +import qtawesome +from qtpy import QtWidgets, QtCore, QtGui + +from openpype.style import ( + get_default_entity_icon_color, + get_disabled_entity_icon_color, +) +from openpype.tools.utils.delegates import PrettyTimeDelegate + +from .utils import TreeView, BaseOverlayFrame + + +REPRE_ID_ROLE = QtCore.Qt.UserRole + 1 +FILEPATH_ROLE = QtCore.Qt.UserRole + 2 +DATE_MODIFIED_ROLE = QtCore.Qt.UserRole + 3 + + +class PublishedFilesModel(QtGui.QStandardItemModel): + """A model for displaying files. + + Args: + controller (AbstractWorkfilesFrontend): The control object. + """ + + def __init__(self, controller): + super(PublishedFilesModel, self).__init__() + + self.setColumnCount(2) + + self.setHeaderData(0, QtCore.Qt.Horizontal, "Name") + self.setHeaderData(1, QtCore.Qt.Horizontal, "Date Modified") + + controller.register_event_callback( + "selection.task.changed", + self._on_task_changed + ) + controller.register_event_callback( + "selection.folder.changed", + self._on_folder_changed + ) + + self._file_icon = qtawesome.icon( + "fa.file-o", + color=get_default_entity_icon_color() + ) + self._controller = controller + self._items_by_id = {} + self._missing_context_item = None + self._missing_context_used = False + self._empty_root_item = None + self._empty_item_used = False + + self._published_mode = False + self._context_select_mode = False + + self._last_folder_id = None + self._last_task_id = None + + self._add_empty_item() + + def _clear_items(self): + self._remove_missing_context_item() + self._remove_empty_item() + if self._items_by_id: + root = self.invisibleRootItem() + root.removeRows(0, root.rowCount()) + self._items_by_id = {} + + def set_published_mode(self, published_mode): + if self._published_mode == published_mode: + return + self._published_mode = published_mode + if published_mode: + self._fill_items() + elif self._context_select_mode: + self.set_select_context_mode(False) + + def set_select_context_mode(self, select_mode): + if self._context_select_mode is select_mode: + return + self._context_select_mode = select_mode + if not select_mode and self._published_mode: + self._fill_items() + + def get_index_by_representation_id(self, representation_id): + item = self._items_by_id.get(representation_id) + if item is None: + return QtCore.QModelIndex() + return self.indexFromItem(item) + + def _get_missing_context_item(self): + if self._missing_context_item is None: + message = "Select folder" + item = QtGui.QStandardItem(message) + icon = qtawesome.icon( + "fa.times", + color=get_disabled_entity_icon_color() + ) + item.setData(icon, QtCore.Qt.DecorationRole) + item.setFlags(QtCore.Qt.NoItemFlags) + item.setColumnCount(self.columnCount()) + self._missing_context_item = item + return self._missing_context_item + + def _add_missing_context_item(self): + if self._missing_context_used: + return + self._clear_items() + root_item = self.invisibleRootItem() + root_item.appendRow(self._get_missing_context_item()) + self._missing_context_used = True + + def _remove_missing_context_item(self): + if not self._missing_context_used: + return + root_item = self.invisibleRootItem() + root_item.takeRow(self._missing_context_item.row()) + self._missing_context_used = False + + def _get_empty_root_item(self): + if self._empty_root_item is None: + message = "Didn't find any published workfiles." + item = QtGui.QStandardItem(message) + icon = qtawesome.icon( + "fa.times", + color=get_disabled_entity_icon_color() + ) + item.setData(icon, QtCore.Qt.DecorationRole) + item.setFlags(QtCore.Qt.NoItemFlags) + item.setColumnCount(self.columnCount()) + self._empty_root_item = item + return self._empty_root_item + + def _add_empty_item(self): + if self._empty_item_used: + return + self._clear_items() + root_item = self.invisibleRootItem() + root_item.appendRow(self._get_empty_root_item()) + self._empty_item_used = True + + def _remove_empty_item(self): + if not self._empty_item_used: + return + root_item = self.invisibleRootItem() + root_item.takeRow(self._empty_root_item.row()) + self._empty_item_used = False + + def _on_folder_changed(self, event): + self._last_folder_id = event["folder_id"] + self._last_task_id = None + if self._context_select_mode: + return + + if self._published_mode: + self._fill_items() + + def _on_task_changed(self, event): + self._last_folder_id = event["folder_id"] + self._last_task_id = event["task_id"] + if self._context_select_mode: + return + + if self._published_mode: + self._fill_items() + + def _fill_items(self): + folder_id = self._last_folder_id + task_id = self._last_task_id + if not folder_id: + self._add_missing_context_item() + return + + file_items = self._controller.get_published_file_items( + folder_id, task_id + ) + root_item = self.invisibleRootItem() + if not file_items: + self._add_empty_item() + return + self._remove_empty_item() + self._remove_missing_context_item() + + items_to_remove = set(self._items_by_id.keys()) + new_items = [] + for file_item in file_items: + repre_id = file_item.representation_id + if repre_id in self._items_by_id: + items_to_remove.discard(repre_id) + item = self._items_by_id[repre_id] + else: + item = QtGui.QStandardItem() + new_items.append(item) + item.setColumnCount(self.columnCount()) + item.setData(self._file_icon, QtCore.Qt.DecorationRole) + item.setData(file_item.filename, QtCore.Qt.DisplayRole) + item.setData(repre_id, REPRE_ID_ROLE) + + if file_item.exists: + flags = QtCore.Qt.ItemIsEnabled | QtCore.Qt.ItemIsSelectable + else: + flags = QtCore.Qt.NoItemFlags + + item.setFlags(flags) + item.setData(file_item.filepath, FILEPATH_ROLE) + item.setData(file_item.modified, DATE_MODIFIED_ROLE) + + self._items_by_id[repre_id] = item + + if new_items: + root_item.appendRows(new_items) + + for repre_id in items_to_remove: + item = self._items_by_id.pop(repre_id) + root_item.removeRow(item.row()) + + if root_item.rowCount() == 0: + self._add_empty_item() + + def flags(self, index): + # Use flags of first column for all columns + if index.column() != 0: + index = self.index(index.row(), 0, index.parent()) + return super(PublishedFilesModel, self).flags(index) + + def data(self, index, role=None): + if role is None: + role = QtCore.Qt.DisplayRole + + # Handle roles for first column + if index.column() == 1: + if role == QtCore.Qt.DecorationRole: + return None + + if role in (QtCore.Qt.DisplayRole, QtCore.Qt.EditRole): + role = DATE_MODIFIED_ROLE + index = self.index(index.row(), 0, index.parent()) + + return super(PublishedFilesModel, self).data(index, role) + + +class SelectContextOverlay(BaseOverlayFrame): + """Overlay for files view when user should select context. + + Todos: + The look of this overlay should be improved, it is "not nice" now. + """ + + def __init__(self, parent): + super(SelectContextOverlay, self).__init__(parent) + + label_widget = QtWidgets.QLabel( + "Please choose context on the left
<", + self + ) + label_widget.setAlignment(QtCore.Qt.AlignCenter) + label_widget.setObjectName("OverlayFrameLabel") + + layout = QtWidgets.QHBoxLayout(self) + layout.addWidget(label_widget, 1, QtCore.Qt.AlignCenter) + + label_widget.setAttribute(QtCore.Qt.WA_TranslucentBackground) + + +class PublishedFilesWidget(QtWidgets.QWidget): + """Published workfiles widget. + + Args: + controller (AbstractWorkfilesFrontend): The control object. + parent (QtWidgets.QWidget): The parent widget. + """ + + selection_changed = QtCore.Signal() + save_as_requested = QtCore.Signal() + + def __init__(self, controller, parent): + super(PublishedFilesWidget, self).__init__(parent) + + view = TreeView(self) + view.setSortingEnabled(True) + view.setContextMenuPolicy(QtCore.Qt.CustomContextMenu) + # Smaller indentation + view.setIndentation(0) + + model = PublishedFilesModel(controller) + proxy_model = QtCore.QSortFilterProxyModel() + proxy_model.setSourceModel(model) + proxy_model.setSortCaseSensitivity(QtCore.Qt.CaseInsensitive) + proxy_model.setDynamicSortFilter(True) + + view.setModel(proxy_model) + + time_delegate = PrettyTimeDelegate() + view.setItemDelegateForColumn(1, time_delegate) + + # Default to a wider first filename column it is what we mostly care + # about and the date modified is relatively small anyway. + view.setColumnWidth(0, 330) + + select_overlay = SelectContextOverlay(view) + select_overlay.setVisible(False) + + main_layout = QtWidgets.QVBoxLayout(self) + main_layout.setContentsMargins(0, 0, 0, 0) + main_layout.addWidget(view, 1) + + selection_model = view.selectionModel() + selection_model.selectionChanged.connect(self._on_selection_change) + view.double_clicked_left.connect(self._on_left_double_click) + + controller.register_event_callback( + "expected_selection_changed", + self._on_expected_selection_change + ) + + self._view = view + self._select_overlay = select_overlay + self._model = model + self._proxy_model = proxy_model + self._time_delegate = time_delegate + self._controller = controller + + def set_published_mode(self, published_mode): + self._model.set_published_mode(published_mode) + + def set_select_context_mode(self, select_mode): + self._model.set_select_context_mode(select_mode) + self._select_overlay.setVisible(select_mode) + + def set_text_filter(self, text_filter): + self._proxy_model.setFilterFixedString(text_filter) + + def get_selected_repre_info(self): + selection_model = self._view.selectionModel() + representation_id = None + filepath = None + for index in selection_model.selectedIndexes(): + representation_id = index.data(REPRE_ID_ROLE) + filepath = index.data(FILEPATH_ROLE) + + return { + "representation_id": representation_id, + "filepath": filepath, + } + + def get_selected_repre_id(self): + return self.get_selected_repre_info()["representation_id"] + + def _on_selection_change(self): + repre_id = self.get_selected_repre_id() + self._controller.set_selected_representation_id(repre_id) + + def _on_left_double_click(self): + self.save_as_requested.emit() + + def _on_expected_selection_change(self, event): + if ( + event["representation_id_selected"] + or not event["folder_selected"] + or (event["task_name"] and not event["task_selected"]) + ): + return + + representation_id = event["representation_id"] + selected_repre_id = self.get_selected_repre_id() + if ( + representation_id is not None + and representation_id != selected_repre_id + ): + index = self._model.get_index_by_representation_id( + representation_id) + if index.isValid(): + proxy_index = self._proxy_model.mapFromSource(index) + self._view.setCurrentIndex(proxy_index) + + self._controller.expected_representation_selected( + event["folder_id"], event["task_name"], representation_id + ) diff --git a/openpype/tools/ayon_workfiles/widgets/files_widget_workarea.py b/openpype/tools/ayon_workfiles/widgets/files_widget_workarea.py new file mode 100644 index 0000000000..e8ccd094d1 --- /dev/null +++ b/openpype/tools/ayon_workfiles/widgets/files_widget_workarea.py @@ -0,0 +1,380 @@ +import qtawesome +from qtpy import QtWidgets, QtCore, QtGui + +from openpype.style import ( + get_default_entity_icon_color, + get_disabled_entity_icon_color, +) +from openpype.tools.utils.delegates import PrettyTimeDelegate + +from .utils import TreeView + +FILENAME_ROLE = QtCore.Qt.UserRole + 1 +FILEPATH_ROLE = QtCore.Qt.UserRole + 2 +DATE_MODIFIED_ROLE = QtCore.Qt.UserRole + 3 + + +class WorkAreaFilesModel(QtGui.QStandardItemModel): + """A model for workare workfiles. + + Args: + controller (AbstractWorkfilesFrontend): The control object. + """ + + def __init__(self, controller): + super(WorkAreaFilesModel, self).__init__() + + self.setColumnCount(2) + + self.setHeaderData(0, QtCore.Qt.Horizontal, "Name") + self.setHeaderData(1, QtCore.Qt.Horizontal, "Date Modified") + + controller.register_event_callback( + "selection.task.changed", + self._on_task_changed + ) + controller.register_event_callback( + "workfile_duplicate.finished", + self._on_duplicate_finished + ) + controller.register_event_callback( + "save_as.finished", + self._on_save_as_finished + ) + + self._file_icon = qtawesome.icon( + "fa.file-o", + color=get_default_entity_icon_color() + ) + self._controller = controller + self._items_by_filename = {} + self._missing_context_item = None + self._missing_context_used = False + self._empty_root_item = None + self._empty_item_used = False + self._published_mode = False + self._selected_folder_id = None + self._selected_task_id = None + + self._add_missing_context_item() + + def get_index_by_filename(self, filename): + item = self._items_by_filename.get(filename) + if item is None: + return QtCore.QModelIndex() + return self.indexFromItem(item) + + def _get_missing_context_item(self): + if self._missing_context_item is None: + message = "Select folder and task" + item = QtGui.QStandardItem(message) + icon = qtawesome.icon( + "fa.times", + color=get_disabled_entity_icon_color() + ) + item.setData(icon, QtCore.Qt.DecorationRole) + item.setFlags(QtCore.Qt.NoItemFlags) + item.setColumnCount(self.columnCount()) + self._missing_context_item = item + return self._missing_context_item + + def _clear_items(self): + self._remove_missing_context_item() + self._remove_empty_item() + if self._items_by_filename: + root = self.invisibleRootItem() + root.removeRows(0, root.rowCount()) + self._items_by_filename = {} + + def _add_missing_context_item(self): + if self._missing_context_used: + return + self._clear_items() + root_item = self.invisibleRootItem() + root_item.appendRow(self._get_missing_context_item()) + self._missing_context_used = True + + def _remove_missing_context_item(self): + if not self._missing_context_used: + return + root_item = self.invisibleRootItem() + root_item.takeRow(self._missing_context_item.row()) + self._missing_context_used = False + + def _get_empty_root_item(self): + if self._empty_root_item is None: + message = "Work Area is empty.." + item = QtGui.QStandardItem(message) + icon = qtawesome.icon( + "fa.exclamation-circle", + color=get_disabled_entity_icon_color() + ) + item.setData(icon, QtCore.Qt.DecorationRole) + item.setFlags(QtCore.Qt.NoItemFlags) + item.setColumnCount(self.columnCount()) + self._empty_root_item = item + return self._empty_root_item + + def _add_empty_item(self): + if self._empty_item_used: + return + self._clear_items() + root_item = self.invisibleRootItem() + root_item.appendRow(self._get_empty_root_item()) + self._empty_item_used = True + + def _remove_empty_item(self): + if not self._empty_item_used: + return + root_item = self.invisibleRootItem() + root_item.takeRow(self._empty_root_item.row()) + self._empty_item_used = False + + def _on_task_changed(self, event): + self._selected_folder_id = event["folder_id"] + self._selected_task_id = event["task_id"] + if not self._published_mode: + self._fill_items() + + def _on_duplicate_finished(self, event): + if event["failed"]: + return + + if not self._published_mode: + self._fill_items() + + def _on_save_as_finished(self, event): + if event["failed"]: + return + + if not self._published_mode: + self._fill_items() + + def _fill_items(self): + folder_id = self._selected_folder_id + task_id = self._selected_task_id + if not folder_id or not task_id: + self._add_missing_context_item() + return + + file_items = self._controller.get_workarea_file_items( + folder_id, task_id + ) + root_item = self.invisibleRootItem() + if not file_items: + self._add_empty_item() + return + self._remove_empty_item() + self._remove_missing_context_item() + + items_to_remove = set(self._items_by_filename.keys()) + new_items = [] + for file_item in file_items: + filename = file_item.filename + if filename in self._items_by_filename: + items_to_remove.discard(filename) + item = self._items_by_filename[filename] + else: + item = QtGui.QStandardItem() + new_items.append(item) + item.setColumnCount(self.columnCount()) + item.setFlags( + QtCore.Qt.ItemIsEnabled | QtCore.Qt.ItemIsSelectable + ) + item.setData(self._file_icon, QtCore.Qt.DecorationRole) + item.setData(file_item.filename, QtCore.Qt.DisplayRole) + item.setData(file_item.filename, FILENAME_ROLE) + + item.setData(file_item.filepath, FILEPATH_ROLE) + item.setData(file_item.modified, DATE_MODIFIED_ROLE) + + self._items_by_filename[file_item.filename] = item + + if new_items: + root_item.appendRows(new_items) + + for filename in items_to_remove: + item = self._items_by_filename.pop(filename) + root_item.removeRow(item.row()) + + if root_item.rowCount() == 0: + self._add_empty_item() + + def flags(self, index): + # Use flags of first column for all columns + if index.column() != 0: + index = self.index(index.row(), 0, index.parent()) + return super(WorkAreaFilesModel, self).flags(index) + + def data(self, index, role=None): + if role is None: + role = QtCore.Qt.DisplayRole + + # Handle roles for first column + if index.column() == 1: + if role == QtCore.Qt.DecorationRole: + return None + + if role in (QtCore.Qt.DisplayRole, QtCore.Qt.EditRole): + role = DATE_MODIFIED_ROLE + index = self.index(index.row(), 0, index.parent()) + + return super(WorkAreaFilesModel, self).data(index, role) + + def set_published_mode(self, published_mode): + if self._published_mode == published_mode: + return + self._published_mode = published_mode + if not published_mode: + self._fill_items() + + +class WorkAreaFilesWidget(QtWidgets.QWidget): + """Workarea files widget. + + Args: + controller (AbstractWorkfilesFrontend): The control object. + parent (QtWidgets.QWidget): The parent widget. + """ + + selection_changed = QtCore.Signal() + open_current_requested = QtCore.Signal() + duplicate_requested = QtCore.Signal() + + def __init__(self, controller, parent): + super(WorkAreaFilesWidget, self).__init__(parent) + + view = TreeView(self) + view.setSortingEnabled(True) + view.setContextMenuPolicy(QtCore.Qt.CustomContextMenu) + # Smaller indentation + view.setIndentation(0) + + model = WorkAreaFilesModel(controller) + proxy_model = QtCore.QSortFilterProxyModel() + proxy_model.setSourceModel(model) + proxy_model.setSortCaseSensitivity(QtCore.Qt.CaseInsensitive) + proxy_model.setDynamicSortFilter(True) + + view.setModel(proxy_model) + + time_delegate = PrettyTimeDelegate() + view.setItemDelegateForColumn(1, time_delegate) + + # Default to a wider first filename column it is what we mostly care + # about and the date modified is relatively small anyway. + view.setColumnWidth(0, 330) + + main_layout = QtWidgets.QVBoxLayout(self) + main_layout.setContentsMargins(0, 0, 0, 0) + main_layout.addWidget(view, 1) + + selection_model = view.selectionModel() + selection_model.selectionChanged.connect(self._on_selection_change) + view.double_clicked_left.connect(self._on_left_double_click) + view.customContextMenuRequested.connect(self._on_context_menu) + + controller.register_event_callback( + "expected_selection_changed", + self._on_expected_selection_change + ) + + self._view = view + self._model = model + self._proxy_model = proxy_model + self._time_delegate = time_delegate + self._controller = controller + + self._published_mode = False + + def set_published_mode(self, published_mode): + """Set the published mode. + + Widget should ignore most of events when in published mode is enabled. + + Args: + published_mode (bool): The published mode. + """ + + self._model.set_published_mode(published_mode) + self._published_mode = published_mode + + def set_text_filter(self, text_filter): + """Set the text filter. + + Args: + text_filter (str): The text filter. + """ + + self._proxy_model.setFilterFixedString(text_filter) + + def _get_selected_info(self): + selection_model = self._view.selectionModel() + filepath = None + filename = None + for index in selection_model.selectedIndexes(): + filepath = index.data(FILEPATH_ROLE) + filename = index.data(FILENAME_ROLE) + return { + "filepath": filepath, + "filename": filename, + } + + def get_selected_path(self): + """Selected filepath. + + Returns: + Union[str, None]: The selected filepath or None if nothing is + selected. + """ + return self._get_selected_info()["filepath"] + + def _on_selection_change(self): + filepath = self.get_selected_path() + self._controller.set_selected_workfile_path(filepath) + + def _on_left_double_click(self): + self.open_current_requested.emit() + + def _on_context_menu(self, point): + index = self._view.indexAt(point) + if not index.isValid(): + return + + if not index.flags() & QtCore.Qt.ItemIsEnabled: + return + + menu = QtWidgets.QMenu(self) + + # Duplicate + action = QtWidgets.QAction("Duplicate", menu) + tip = "Duplicate selected file." + action.setToolTip(tip) + action.setStatusTip(tip) + action.triggered.connect(self._on_duplicate_pressed) + menu.addAction(action) + + # Show the context action menu + global_point = self._view.mapToGlobal(point) + _ = menu.exec_(global_point) + + def _on_duplicate_pressed(self): + self.duplicate_requested.emit() + + def _on_expected_selection_change(self, event): + if event["workfile_name_selected"]: + return + + workfile_name = event["workfile_name"] + if ( + workfile_name is not None + and workfile_name != self._get_selected_info()["filename"] + ): + index = self._model.get_index_by_filename(workfile_name) + if index.isValid(): + proxy_index = self._proxy_model.mapFromSource(index) + self._view.setCurrentIndex(proxy_index) + + self._controller.expected_workfile_selected( + event["folder_id"], event["task_name"], workfile_name + ) diff --git a/openpype/tools/ayon_workfiles/widgets/folders_widget.py b/openpype/tools/ayon_workfiles/widgets/folders_widget.py new file mode 100644 index 0000000000..b35845f4b6 --- /dev/null +++ b/openpype/tools/ayon_workfiles/widgets/folders_widget.py @@ -0,0 +1,324 @@ +import uuid +import collections + +import qtawesome +from qtpy import QtWidgets, QtGui, QtCore + +from openpype.tools.utils import ( + RecursiveSortFilterProxyModel, + DeselectableTreeView, +) + +from .constants import ITEM_ID_ROLE, ITEM_NAME_ROLE + +SENDER_NAME = "qt_folders_model" + + +class FoldersRefreshThread(QtCore.QThread): + """Thread for refreshing folders. + + Call controller to get folders and emit signal when finished. + + Args: + controller (AbstractWorkfilesFrontend): The control object. + """ + + refresh_finished = QtCore.Signal(str) + + def __init__(self, controller): + super(FoldersRefreshThread, self).__init__() + self._id = uuid.uuid4().hex + self._controller = controller + self._result = None + + @property + def id(self): + """Thread id. + + Returns: + str: Unique id of the thread. + """ + + return self._id + + def run(self): + self._result = self._controller.get_folder_items(SENDER_NAME) + self.refresh_finished.emit(self.id) + + def get_result(self): + return self._result + + +class FoldersModel(QtGui.QStandardItemModel): + """Folders model which cares about refresh of folders. + + Args: + controller (AbstractWorkfilesFrontend): The control object. + """ + + refreshed = QtCore.Signal() + + def __init__(self, controller): + super(FoldersModel, self).__init__() + + self._controller = controller + self._items_by_id = {} + self._parent_id_by_id = {} + + self._refresh_threads = {} + self._current_refresh_thread = None + + self._has_content = False + self._is_refreshing = False + + @property + def is_refreshing(self): + """Model is refreshing. + + Returns: + bool: True if model is refreshing. + """ + return self._is_refreshing + + @property + def has_content(self): + """Has at least one folder. + + Returns: + bool: True if model has at least one folder. + """ + + return self._has_content + + def clear(self): + self._items_by_id = {} + self._parent_id_by_id = {} + self._has_content = False + super(FoldersModel, self).clear() + + def get_index_by_id(self, item_id): + """Get index by folder id. + + Returns: + QtCore.QModelIndex: Index of the folder. Can be invalid if folder + is not available. + """ + item = self._items_by_id.get(item_id) + if item is None: + return QtCore.QModelIndex() + return self.indexFromItem(item) + + def refresh(self): + """Refresh folders items. + + Refresh start thread because it can cause that controller can + start query from database if folders are not cached. + """ + + self._is_refreshing = True + + thread = FoldersRefreshThread(self._controller) + self._current_refresh_thread = thread.id + self._refresh_threads[thread.id] = thread + thread.refresh_finished.connect(self._on_refresh_thread) + thread.start() + + def _on_refresh_thread(self, thread_id): + """Callback when refresh thread is finished. + + Technically can be running multiple refresh threads at the same time, + to avoid using values from wrong thread, we check if thread id is + current refresh thread id. + + Folders are stored by id. + + Args: + thread_id (str): Thread id. + """ + + thread = self._refresh_threads.pop(thread_id) + if thread_id != self._current_refresh_thread: + return + + folder_items_by_id = thread.get_result() + if not folder_items_by_id: + if folder_items_by_id is not None: + self.clear() + self._is_refreshing = False + return + + self._has_content = True + + folder_ids = set(folder_items_by_id) + ids_to_remove = set(self._items_by_id) - folder_ids + + folder_items_by_parent = collections.defaultdict(list) + for folder_item in folder_items_by_id.values(): + folder_items_by_parent[folder_item.parent_id].append(folder_item) + + hierarchy_queue = collections.deque() + hierarchy_queue.append(None) + + while hierarchy_queue: + parent_id = hierarchy_queue.popleft() + folder_items = folder_items_by_parent[parent_id] + if parent_id is None: + parent_item = self.invisibleRootItem() + else: + parent_item = self._items_by_id[parent_id] + + new_items = [] + for folder_item in folder_items: + item_id = folder_item.entity_id + item = self._items_by_id.get(item_id) + if item is None: + is_new = True + item = QtGui.QStandardItem() + item.setEditable(False) + else: + is_new = self._parent_id_by_id[item_id] != parent_id + + icon = qtawesome.icon( + folder_item.icon_name, + color=folder_item.icon_color, + ) + item.setData(item_id, ITEM_ID_ROLE) + item.setData(folder_item.name, ITEM_NAME_ROLE) + item.setData(folder_item.label, QtCore.Qt.DisplayRole) + item.setData(icon, QtCore.Qt.DecorationRole) + if is_new: + new_items.append(item) + self._items_by_id[item_id] = item + self._parent_id_by_id[item_id] = parent_id + + hierarchy_queue.append(item_id) + + if new_items: + parent_item.appendRows(new_items) + + for item_id in ids_to_remove: + item = self._items_by_id[item_id] + parent_id = self._parent_id_by_id[item_id] + if parent_id is None: + parent_item = self.invisibleRootItem() + else: + parent_item = self._items_by_id[parent_id] + parent_item.takeChild(item.row()) + + for item_id in ids_to_remove: + self._items_by_id.pop(item_id) + self._parent_id_by_id.pop(item_id) + + self._is_refreshing = False + self.refreshed.emit() + + +class FoldersWidget(QtWidgets.QWidget): + """Folders widget. + + Widget that handles folders view, model and selection. + + Args: + controller (AbstractWorkfilesFrontend): The control object. + parent (QtWidgets.QWidget): The parent widget. + """ + + def __init__(self, controller, parent): + super(FoldersWidget, self).__init__(parent) + + folders_view = DeselectableTreeView(self) + folders_view.setHeaderHidden(True) + + folders_model = FoldersModel(controller) + folders_proxy_model = RecursiveSortFilterProxyModel() + folders_proxy_model.setSourceModel(folders_model) + + folders_view.setModel(folders_proxy_model) + + main_layout = QtWidgets.QHBoxLayout(self) + main_layout.setContentsMargins(0, 0, 0, 0) + main_layout.addWidget(folders_view, 1) + + controller.register_event_callback( + "folders.refresh.finished", + self._on_folders_refresh_finished + ) + controller.register_event_callback( + "controller.refresh.finished", + self._on_controller_refresh + ) + controller.register_event_callback( + "expected_selection_changed", + self._on_expected_selection_change + ) + + selection_model = folders_view.selectionModel() + selection_model.selectionChanged.connect(self._on_selection_change) + + folders_model.refreshed.connect(self._on_model_refresh) + + self._controller = controller + self._folders_view = folders_view + self._folders_model = folders_model + self._folders_proxy_model = folders_proxy_model + + self._expected_selection = None + + def set_name_filer(self, name): + self._folders_proxy_model.setFilterFixedString(name) + + def _clear(self): + self._folders_model.clear() + + def _on_folders_refresh_finished(self, event): + if event["sender"] != SENDER_NAME: + self._folders_model.refresh() + + def _on_controller_refresh(self): + self._update_expected_selection() + + def _update_expected_selection(self, expected_data=None): + if expected_data is None: + expected_data = self._controller.get_expected_selection_data() + + # We're done + if expected_data["folder_selected"]: + return + + folder_id = expected_data["folder_id"] + self._expected_selection = folder_id + if not self._folders_model.is_refreshing: + self._set_expected_selection() + + def _set_expected_selection(self): + folder_id = self._expected_selection + self._expected_selection = None + if ( + folder_id is not None + and folder_id != self._get_selected_item_id() + ): + index = self._folders_model.get_index_by_id(folder_id) + if index.isValid(): + proxy_index = self._folders_proxy_model.mapFromSource(index) + self._folders_view.setCurrentIndex(proxy_index) + self._controller.expected_folder_selected(folder_id) + + def _on_model_refresh(self): + if self._expected_selection: + self._set_expected_selection() + self._folders_proxy_model.sort(0) + + def _on_expected_selection_change(self, event): + self._update_expected_selection(event.data) + + def _get_selected_item_id(self): + selection_model = self._folders_view.selectionModel() + for index in selection_model.selectedIndexes(): + item_id = index.data(ITEM_ID_ROLE) + if item_id is not None: + return item_id + return None + + def _on_selection_change(self): + item_id = self._get_selected_item_id() + self._controller.set_selected_folder(item_id) diff --git a/openpype/tools/ayon_workfiles/widgets/save_as_dialog.py b/openpype/tools/ayon_workfiles/widgets/save_as_dialog.py new file mode 100644 index 0000000000..cdce73f030 --- /dev/null +++ b/openpype/tools/ayon_workfiles/widgets/save_as_dialog.py @@ -0,0 +1,351 @@ +from qtpy import QtWidgets, QtCore + +from openpype.tools.utils import PlaceholderLineEdit + + +class SubversionLineEdit(QtWidgets.QWidget): + """QLineEdit with QPushButton for drop down selection of list of strings""" + + text_changed = QtCore.Signal(str) + + def __init__(self, *args, **kwargs): + super(SubversionLineEdit, self).__init__(*args, **kwargs) + + input_field = PlaceholderLineEdit(self) + menu_btn = QtWidgets.QPushButton(self) + menu_btn.setFixedWidth(18) + + menu = QtWidgets.QMenu(self) + menu_btn.setMenu(menu) + + layout = QtWidgets.QHBoxLayout(self) + layout.setContentsMargins(0, 0, 0, 0) + layout.setSpacing(3) + + layout.addWidget(input_field, 1) + layout.addWidget(menu_btn, 0) + + input_field.textChanged.connect(self.text_changed) + + self.setFocusProxy(input_field) + + self._input_field = input_field + self._menu_btn = menu_btn + self._menu = menu + + def set_placeholder(self, placeholder): + self._input_field.setPlaceholderText(placeholder) + + def set_text(self, text): + self._input_field.setText(text) + + def set_values(self, values): + self._update(values) + + def _on_button_clicked(self): + self._menu.exec_() + + def _on_action_clicked(self, action): + self._input_field.setText(action.text()) + + def _update(self, values): + """Create optional predefined subset names + + Args: + default_names(list): all predefined names + + Returns: + None + """ + + menu = self._menu + button = self._menu_btn + + state = any(values) + button.setEnabled(state) + if state is False: + return + + # Include an empty string + values = [""] + sorted(values) + + # Get and destroy the action group + group = button.findChild(QtWidgets.QActionGroup) + if group: + group.deleteLater() + + # Build new action group + group = QtWidgets.QActionGroup(button) + for name in values: + action = group.addAction(name) + menu.addAction(action) + + group.triggered.connect(self._on_action_clicked) + + +class SaveAsDialog(QtWidgets.QDialog): + """Save as dialog to define a unique filename inside workdir. + + The filename is calculated in controller where UI sends values from + dialog inputs. + + Args: + controller (AbstractWorkfilesFrontend): The control object. + """ + + def __init__(self, controller, parent): + super(SaveAsDialog, self).__init__(parent=parent) + self.setWindowFlags(self.windowFlags() | QtCore.Qt.FramelessWindowHint) + + self._controller = controller + + self._folder_id = None + self._task_id = None + self._last_version = None + self._template_key = None + self._comment_value = None + self._version_value = None + self._ext_value = None + self._filename = None + self._workdir = None + + self._result = None + + # Btns widget + btns_widget = QtWidgets.QWidget(self) + + btn_ok = QtWidgets.QPushButton("Ok", btns_widget) + btn_cancel = QtWidgets.QPushButton("Cancel", btns_widget) + + btns_layout = QtWidgets.QHBoxLayout(btns_widget) + btns_layout.addWidget(btn_ok) + btns_layout.addWidget(btn_cancel) + + # Inputs widget + inputs_widget = QtWidgets.QWidget(self) + + # Version widget + version_widget = QtWidgets.QWidget(inputs_widget) + + # Version number input + version_input = QtWidgets.QSpinBox(version_widget) + version_input.setMinimum(1) + version_input.setMaximum(9999) + + # Last version checkbox + last_version_check = QtWidgets.QCheckBox( + "Next Available Version", version_widget + ) + last_version_check.setChecked(True) + + version_layout = QtWidgets.QHBoxLayout(version_widget) + version_layout.setContentsMargins(0, 0, 0, 0) + version_layout.addWidget(version_input) + version_layout.addWidget(last_version_check) + + # Preview widget + preview_widget = QtWidgets.QLabel("Preview filename", inputs_widget) + preview_widget.setWordWrap(True) + + # Subversion input + subversion_input = SubversionLineEdit(inputs_widget) + subversion_input.set_placeholder("Will be part of filename.") + + # Extensions combobox + extension_combobox = QtWidgets.QComboBox(inputs_widget) + # Add styled delegate to use stylesheets + extension_delegate = QtWidgets.QStyledItemDelegate() + extension_combobox.setItemDelegate(extension_delegate) + + version_label = QtWidgets.QLabel("Version:", inputs_widget) + subversion_label = QtWidgets.QLabel("Subversion:", inputs_widget) + extension_label = QtWidgets.QLabel("Extension:", inputs_widget) + preview_label = QtWidgets.QLabel("Preview:", inputs_widget) + + # Build inputs + inputs_layout = QtWidgets.QGridLayout(inputs_widget) + inputs_layout.addWidget(version_label, 0, 0) + inputs_layout.addWidget(version_widget, 0, 1) + inputs_layout.addWidget(subversion_label, 1, 0) + inputs_layout.addWidget(subversion_input, 1, 1) + inputs_layout.addWidget(extension_label, 2, 0) + inputs_layout.addWidget(extension_combobox, 2, 1) + inputs_layout.addWidget(preview_label, 3, 0) + inputs_layout.addWidget(preview_widget, 3, 1) + + # Build layout + main_layout = QtWidgets.QVBoxLayout(self) + main_layout.addWidget(inputs_widget) + main_layout.addWidget(btns_widget) + + # Signal callback registration + version_input.valueChanged.connect(self._on_version_spinbox_change) + last_version_check.stateChanged.connect( + self._on_version_checkbox_change + ) + + subversion_input.text_changed.connect(self._on_comment_change) + extension_combobox.currentIndexChanged.connect( + self._on_extension_change) + + btn_ok.pressed.connect(self._on_ok_pressed) + btn_cancel.pressed.connect(self._on_cancel_pressed) + + # Store objects + self._inputs_layout = inputs_layout + + self._btn_ok = btn_ok + self._btn_cancel = btn_cancel + + self._version_widget = version_widget + + self._version_input = version_input + self._last_version_check = last_version_check + + self._extension_delegate = extension_delegate + self._extension_combobox = extension_combobox + self._subversion_input = subversion_input + self._preview_widget = preview_widget + + self._version_label = version_label + self._subversion_label = subversion_label + self._extension_label = extension_label + self._preview_label = preview_label + + # Post init setup + + # Allow "Enter" key to accept the save. + btn_ok.setDefault(True) + + # Disable version input if last version is checked + version_input.setEnabled(not last_version_check.isChecked()) + + # Force default focus to comment, some hosts didn't automatically + # apply focus to this line edit (e.g. Houdini) + subversion_input.setFocus() + + def get_result(self): + return self._result + + def update_context(self): + # Add version only if template contains version key + # - since the version can be padded with "{version:0>4}" we only search + # for "{version". + selected_context = self._controller.get_selected_context() + folder_id = selected_context["folder_id"] + task_id = selected_context["task_id"] + data = self._controller.get_workarea_save_as_data(folder_id, task_id) + last_version = data["last_version"] + comment = data["comment"] + comment_hints = data["comment_hints"] + + template_has_version = data["template_has_version"] + template_has_comment = data["template_has_comment"] + + self._folder_id = folder_id + self._task_id = task_id + self._workdir = data["workdir"] + self._comment_value = data["comment"] + self._ext_value = data["ext"] + self._template_key = data["template_key"] + self._last_version = data["last_version"] + + self._extension_combobox.clear() + self._extension_combobox.addItems(data["extensions"]) + + self._version_input.setValue(last_version) + + vw_idx = self._inputs_layout.indexOf(self._version_widget) + self._version_label.setVisible(template_has_version) + self._version_widget.setVisible(template_has_version) + if template_has_version: + if vw_idx == -1: + self._inputs_layout.addWidget(self._version_label, 0, 0) + self._inputs_layout.addWidget(self._version_widget, 0, 1) + elif vw_idx != -1: + self._inputs_layout.takeAt(vw_idx) + self._inputs_layout.takeAt( + self._inputs_layout.indexOf(self._version_label) + ) + + cw_idx = self._inputs_layout.indexOf(self._subversion_input) + self._subversion_label.setVisible(template_has_comment) + self._subversion_input.setVisible(template_has_comment) + if template_has_comment: + if cw_idx == -1: + self._inputs_layout.addWidget(self._subversion_label, 1, 0) + self._inputs_layout.addWidget(self._subversion_input, 1, 1) + elif cw_idx != -1: + self._inputs_layout.takeAt(cw_idx) + self._inputs_layout.takeAt( + self._inputs_layout.indexOf(self._subversion_label) + ) + + if template_has_comment: + self._subversion_input.set_text(comment or "") + self._subversion_input.set_values(comment_hints) + self._update_filename() + + def _on_version_spinbox_change(self, value): + if value == self._version_value: + return + self._version_value = value + if not self._last_version_check.isChecked(): + self._update_filename() + + def _on_version_checkbox_change(self): + use_last_version = self._last_version_check.isChecked() + self._version_input.setEnabled(not use_last_version) + if use_last_version: + self._version_input.blockSignals(True) + self._version_input.setValue(self._last_version) + self._version_input.blockSignals(False) + self._update_filename() + + def _on_comment_change(self, text): + if self._comment_value == text: + return + self._comment_value = text + self._update_filename() + + def _on_extension_change(self): + ext = self._extension_combobox.currentText() + if ext == self._ext_value: + return + self._ext_value = ext + self._update_filename() + + def _on_ok_pressed(self): + self._result = { + "filename": self._filename, + "workdir": self._workdir, + "folder_id": self._folder_id, + "task_id": self._task_id, + "template_key": self._template_key, + } + self.close() + + def _on_cancel_pressed(self): + self.close() + + def _update_filename(self): + result = self._controller.fill_workarea_filepath( + self._folder_id, + self._task_id, + self._ext_value, + self._last_version_check.isChecked(), + self._version_value, + self._comment_value, + ) + self._filename = result.filename + self._btn_ok.setEnabled(not result.exists) + + if result.exists: + self._preview_widget.setText(( + "Cannot create \"{}\" because file exists!" + "" + ).format(result.filename)) + else: + self._preview_widget.setText( + "{}".format(result.filename) + ) diff --git a/openpype/tools/ayon_workfiles/widgets/side_panel.py b/openpype/tools/ayon_workfiles/widgets/side_panel.py new file mode 100644 index 0000000000..7f06576a00 --- /dev/null +++ b/openpype/tools/ayon_workfiles/widgets/side_panel.py @@ -0,0 +1,163 @@ +import datetime + +from qtpy import QtWidgets, QtCore + + +def file_size_to_string(file_size): + size = 0 + size_ending_mapping = { + "KB": 1024 ** 1, + "MB": 1024 ** 2, + "GB": 1024 ** 3 + } + ending = "B" + for _ending, _size in size_ending_mapping.items(): + if file_size < _size: + break + size = file_size / _size + ending = _ending + return "{:.2f} {}".format(size, ending) + + +class SidePanelWidget(QtWidgets.QWidget): + """Details about selected workfile. + + Todos: + At this moment only shows created and modified date of file + or its size. + + Args: + controller (AbstractWorkfilesFrontend): The control object. + parent (QtWidgets.QWidget): The parent widget. + """ + + published_workfile_message = ( + "INFO: Opened published workfiles will be stored in" + " temp directory on your machine. Current temp size: {}." + ) + + def __init__(self, controller, parent): + super(SidePanelWidget, self).__init__(parent) + + details_label = QtWidgets.QLabel("Details", self) + details_input = QtWidgets.QPlainTextEdit(self) + details_input.setReadOnly(True) + + artist_note_widget = QtWidgets.QWidget(self) + note_label = QtWidgets.QLabel("Artist note", artist_note_widget) + note_input = QtWidgets.QPlainTextEdit(artist_note_widget) + btn_note_save = QtWidgets.QPushButton("Save note", artist_note_widget) + + artist_note_layout = QtWidgets.QVBoxLayout(artist_note_widget) + artist_note_layout.setContentsMargins(0, 0, 0, 0) + artist_note_layout.addWidget(note_label, 0) + artist_note_layout.addWidget(note_input, 1) + artist_note_layout.addWidget( + btn_note_save, 0, alignment=QtCore.Qt.AlignRight + ) + + main_layout = QtWidgets.QVBoxLayout(self) + main_layout.setContentsMargins(0, 0, 0, 0) + main_layout.addWidget(details_label, 0) + main_layout.addWidget(details_input, 1) + main_layout.addWidget(artist_note_widget, 1) + + note_input.textChanged.connect(self._on_note_change) + btn_note_save.clicked.connect(self._on_save_click) + + controller.register_event_callback( + "workarea.selection.changed", self._on_selection_change + ) + + self._details_input = details_input + self._artist_note_widget = artist_note_widget + self._note_input = note_input + self._btn_note_save = btn_note_save + + self._folder_id = None + self._task_id = None + self._filepath = None + self._orig_note = "" + self._controller = controller + + self._set_context(None, None, None) + + def set_published_mode(self, published_mode): + """Change published mode. + + Args: + published_mode (bool): Published mode enabled. + """ + + self._artist_note_widget.setVisible(not published_mode) + + def _on_selection_change(self, event): + folder_id = event["folder_id"] + task_id = event["task_id"] + filepath = event["path"] + + self._set_context(folder_id, task_id, filepath) + + def _on_note_change(self): + text = self._note_input.toPlainText() + self._btn_note_save.setEnabled(self._orig_note != text) + + def _on_save_click(self): + note = self._note_input.toPlainText() + self._controller.save_workfile_info( + self._folder_id, + self._task_id, + self._filepath, + note + ) + self._orig_note = note + self._btn_note_save.setEnabled(False) + + def _set_context(self, folder_id, task_id, filepath): + workfile_info = None + # Check if folder, task and file are selected + if bool(folder_id) and bool(task_id) and bool(filepath): + workfile_info = self._controller.get_workfile_info( + folder_id, task_id, filepath + ) + enabled = workfile_info is not None + + self._details_input.setEnabled(enabled) + self._note_input.setEnabled(enabled) + self._btn_note_save.setEnabled(enabled) + + self._folder_id = folder_id + self._task_id = task_id + self._filepath = filepath + + # Disable inputs and remove texts if any required arguments are + # missing + if not enabled: + self._orig_note = "" + self._details_input.setPlainText("") + self._note_input.setPlainText("") + return + + note = workfile_info.note + size_value = file_size_to_string(workfile_info.filesize) + + # Append html string + datetime_format = "%b %d %Y %H:%M:%S" + creation_time = datetime.datetime.fromtimestamp( + workfile_info.creation_time) + modification_time = datetime.datetime.fromtimestamp( + workfile_info.modification_time) + lines = ( + "Size:", + size_value, + "Created:", + creation_time.strftime(datetime_format), + "Modified:", + modification_time.strftime(datetime_format) + ) + self._orig_note = note + self._note_input.setPlainText(note) + + # Set as empty string + self._details_input.setPlainText("") + self._details_input.appendHtml("
".join(lines)) diff --git a/openpype/tools/ayon_workfiles/widgets/tasks_widget.py b/openpype/tools/ayon_workfiles/widgets/tasks_widget.py new file mode 100644 index 0000000000..04f5b286b1 --- /dev/null +++ b/openpype/tools/ayon_workfiles/widgets/tasks_widget.py @@ -0,0 +1,420 @@ +import uuid +import qtawesome +from qtpy import QtWidgets, QtGui, QtCore + +from openpype.style import get_disabled_entity_icon_color +from openpype.tools.utils import DeselectableTreeView + +from .constants import ( + ITEM_NAME_ROLE, + ITEM_ID_ROLE, + PARENT_ID_ROLE, +) + +SENDER_NAME = "qt_tasks_model" + + +class RefreshThread(QtCore.QThread): + """Thread for refreshing tasks. + + Call controller to get tasks and emit signal when finished. + + Args: + controller (AbstractWorkfilesFrontend): The control object. + folder_id (str): Folder id. + """ + + refresh_finished = QtCore.Signal(str) + + def __init__(self, controller, folder_id): + super(RefreshThread, self).__init__() + self._id = uuid.uuid4().hex + self._controller = controller + self._folder_id = folder_id + self._result = None + + @property + def id(self): + return self._id + + def run(self): + self._result = self._controller.get_task_items( + self._folder_id, SENDER_NAME) + self.refresh_finished.emit(self.id) + + def get_result(self): + return self._result + + +class TasksModel(QtGui.QStandardItemModel): + """Tasks model which cares about refresh of tasks by folder id. + + Args: + controller (AbstractWorkfilesFrontend): The control object. + """ + + refreshed = QtCore.Signal() + + def __init__(self, controller): + super(TasksModel, self).__init__() + + self._controller = controller + + self._items_by_name = {} + self._has_content = False + self._is_refreshing = False + + self._invalid_selection_item_used = False + self._invalid_selection_item = None + self._empty_tasks_item_used = False + self._empty_tasks_item = None + + self._last_folder_id = None + + self._refresh_threads = {} + self._current_refresh_thread = None + + # Initial state + self._add_invalid_selection_item() + + def clear(self): + self._items_by_name = {} + self._has_content = False + self._remove_invalid_items() + super(TasksModel, self).clear() + + def refresh(self, folder_id): + """Refresh tasks for folder. + + Args: + folder_id (Union[str, None]): Folder id. + """ + + self._refresh(folder_id) + + def get_index_by_name(self, task_name): + """Find item by name and return its index. + + Returns: + QtCore.QModelIndex: Index of item. Is invalid if task is not + found by name. + """ + + item = self._items_by_name.get(task_name) + if item is None: + return QtCore.QModelIndex() + return self.indexFromItem(item) + + def get_last_folder_id(self): + """Get last refreshed folder id. + + Returns: + Union[str, None]: Folder id. + """ + + return self._last_folder_id + + def _get_invalid_selection_item(self): + if self._invalid_selection_item is None: + item = QtGui.QStandardItem("Select a folder") + item.setFlags(QtCore.Qt.NoItemFlags) + icon = qtawesome.icon( + "fa.times", + color=get_disabled_entity_icon_color() + ) + item.setData(icon, QtCore.Qt.DecorationRole) + self._invalid_selection_item = item + return self._invalid_selection_item + + def _get_empty_task_item(self): + if self._empty_tasks_item is None: + item = QtGui.QStandardItem("No task") + icon = qtawesome.icon( + "fa.exclamation-circle", + color=get_disabled_entity_icon_color() + ) + item.setData(icon, QtCore.Qt.DecorationRole) + item.setFlags(QtCore.Qt.NoItemFlags) + self._empty_tasks_item = item + return self._empty_tasks_item + + def _add_invalid_item(self, item): + self.clear() + root_item = self.invisibleRootItem() + root_item.appendRow(item) + + def _remove_invalid_item(self, item): + root_item = self.invisibleRootItem() + root_item.takeRow(item.row()) + + def _remove_invalid_items(self): + self._remove_invalid_selection_item() + self._remove_empty_task_item() + + def _add_invalid_selection_item(self): + if not self._invalid_selection_item_used: + self._add_invalid_item(self._get_invalid_selection_item()) + self._invalid_selection_item_used = True + + def _remove_invalid_selection_item(self): + if self._invalid_selection_item: + self._remove_invalid_item(self._get_invalid_selection_item()) + self._invalid_selection_item_used = False + + def _add_empty_task_item(self): + if not self._empty_tasks_item_used: + self._add_invalid_item(self._get_empty_task_item()) + self._empty_tasks_item_used = True + + def _remove_empty_task_item(self): + if self._empty_tasks_item_used: + self._remove_invalid_item(self._get_empty_task_item()) + self._empty_tasks_item_used = False + + def _refresh(self, folder_id): + self._is_refreshing = True + self._last_folder_id = folder_id + if not folder_id: + self._add_invalid_selection_item() + self._current_refresh_thread = None + self._is_refreshing = False + self.refreshed.emit() + return + + thread = RefreshThread(self._controller, folder_id) + self._current_refresh_thread = thread.id + self._refresh_threads[thread.id] = thread + thread.refresh_finished.connect(self._on_refresh_thread) + thread.start() + + def _on_refresh_thread(self, thread_id): + """Callback when refresh thread is finished. + + Technically can be running multiple refresh threads at the same time, + to avoid using values from wrong thread, we check if thread id is + current refresh thread id. + + Tasks are stored by name, so if a folder has same task name as + previously selected folder it keeps the selection. + + Args: + thread_id (str): Thread id. + """ + + thread = self._refresh_threads.pop(thread_id) + if thread_id != self._current_refresh_thread: + return + + task_items = thread.get_result() + # Task items are refreshed + if task_items is None: + return + + # No tasks are available on folder + if not task_items: + self._add_empty_task_item() + return + self._remove_invalid_items() + + new_items = [] + new_names = set() + for task_item in task_items: + name = task_item.name + new_names.add(name) + item = self._items_by_name.get(name) + if item is None: + item = QtGui.QStandardItem() + item.setEditable(False) + new_items.append(item) + self._items_by_name[name] = item + + # TODO cache locally + icon = qtawesome.icon( + task_item.icon_name, + color=task_item.icon_color, + ) + item.setData(task_item.label, QtCore.Qt.DisplayRole) + item.setData(name, ITEM_NAME_ROLE) + item.setData(task_item.id, ITEM_ID_ROLE) + item.setData(task_item.parent_id, PARENT_ID_ROLE) + item.setData(icon, QtCore.Qt.DecorationRole) + + root_item = self.invisibleRootItem() + + for name in set(self._items_by_name) - new_names: + item = self._items_by_name.pop(name) + root_item.removeRow(item.row()) + + if new_items: + root_item.appendRows(new_items) + + self._has_content = root_item.rowCount() > 0 + self._is_refreshing = False + self.refreshed.emit() + + @property + def is_refreshing(self): + """Model is refreshing. + + Returns: + bool: Model is refreshing + """ + + return self._is_refreshing + + @property + def has_content(self): + """Model has content. + + Returns: + bools: Have at least one task. + """ + + return self._has_content + + def headerData(self, section, orientation, role): + # Show nice labels in the header + if ( + role == QtCore.Qt.DisplayRole + and orientation == QtCore.Qt.Horizontal + ): + if section == 0: + return "Tasks" + + return super(TasksModel, self).headerData( + section, orientation, role + ) + + +class TasksWidget(QtWidgets.QWidget): + """Tasks widget. + + Widget that handles tasks view, model and selection. + + Args: + controller (AbstractWorkfilesFrontend): Workfiles controller. + """ + + def __init__(self, controller, parent): + super(TasksWidget, self).__init__(parent) + + tasks_view = DeselectableTreeView(self) + tasks_view.setIndentation(0) + + tasks_model = TasksModel(controller) + tasks_proxy_model = QtCore.QSortFilterProxyModel() + tasks_proxy_model.setSourceModel(tasks_model) + + tasks_view.setModel(tasks_proxy_model) + + main_layout = QtWidgets.QHBoxLayout(self) + main_layout.setContentsMargins(0, 0, 0, 0) + main_layout.addWidget(tasks_view, 1) + + controller.register_event_callback( + "tasks.refresh.finished", + self._on_tasks_refresh_finished + ) + controller.register_event_callback( + "selection.folder.changed", + self._folder_selection_changed + ) + controller.register_event_callback( + "expected_selection_changed", + self._on_expected_selection_change + ) + + selection_model = tasks_view.selectionModel() + selection_model.selectionChanged.connect(self._on_selection_change) + + tasks_model.refreshed.connect(self._on_tasks_model_refresh) + + self._controller = controller + self._tasks_view = tasks_view + self._tasks_model = tasks_model + self._tasks_proxy_model = tasks_proxy_model + + self._selected_folder_id = None + + self._expected_selection_data = None + + def _clear(self): + self._tasks_model.clear() + + def _on_tasks_refresh_finished(self, event): + """Tasks were refreshed in controller. + + Ignore if refresh was triggered by tasks model, or refreshed folder is + not the same as currently selected folder. + + Args: + event (Event): Event object. + """ + + # Refresh only if current folder id is the same + if ( + event["sender"] == SENDER_NAME + or event["folder_id"] != self._selected_folder_id + ): + return + self._tasks_model.refresh(self._selected_folder_id) + + def _folder_selection_changed(self, event): + self._selected_folder_id = event["folder_id"] + self._tasks_model.refresh(self._selected_folder_id) + + def _on_tasks_model_refresh(self): + if not self._set_expected_selection(): + self._on_selection_change() + self._tasks_proxy_model.sort(0) + + def _set_expected_selection(self): + if self._expected_selection_data is None: + return False + folder_id = self._expected_selection_data["folder_id"] + task_name = self._expected_selection_data["task_name"] + self._expected_selection_data = None + model_folder_id = self._tasks_model.get_last_folder_id() + if folder_id != model_folder_id: + return False + if task_name is not None: + index = self._tasks_model.get_index_by_name(task_name) + if index.isValid(): + proxy_index = self._tasks_proxy_model.mapFromSource(index) + self._tasks_view.setCurrentIndex(proxy_index) + self._controller.expected_task_selected(folder_id, task_name) + return True + + def _on_expected_selection_change(self, event): + if event["task_selected"] or not event["folder_selected"]: + return + + model_folder_id = self._tasks_model.get_last_folder_id() + folder_id = event["folder_id"] + self._expected_selection_data = { + "task_name": event["task_name"], + "folder_id": folder_id, + } + + if folder_id != model_folder_id or self._tasks_model.is_refreshing: + return + self._set_expected_selection() + + def _get_selected_item_ids(self): + selection_model = self._tasks_view.selectionModel() + for index in selection_model.selectedIndexes(): + task_id = index.data(ITEM_ID_ROLE) + task_name = index.data(ITEM_NAME_ROLE) + parent_id = index.data(PARENT_ID_ROLE) + if task_name is not None: + return parent_id, task_id, task_name + return self._selected_folder_id, None, None + + def _on_selection_change(self): + # Don't trigger task change during refresh + # - a task was deselected if that happens + # - can cause crash triggered during tasks refreshing + if self._tasks_model.is_refreshing: + return + parent_id, task_id, task_name = self._get_selected_item_ids() + self._controller.set_selected_task(parent_id, task_id, task_name) diff --git a/openpype/tools/ayon_workfiles/widgets/utils.py b/openpype/tools/ayon_workfiles/widgets/utils.py new file mode 100644 index 0000000000..6a61239f8d --- /dev/null +++ b/openpype/tools/ayon_workfiles/widgets/utils.py @@ -0,0 +1,94 @@ +from qtpy import QtWidgets, QtCore +from openpype.tools.flickcharm import FlickCharm + + +class TreeView(QtWidgets.QTreeView): + """Ultimate TreeView with flick charm and double click signals. + + Tree view have deselectable mode, which allows to deselect items by + clicking on item area without any items. + + Todos: + Add to tools utils. + """ + + double_clicked_left = QtCore.Signal() + double_clicked_right = QtCore.Signal() + + def __init__(self, *args, **kwargs): + super(TreeView, self).__init__(*args, **kwargs) + self._deselectable = False + + self._flick_charm_activated = False + self._flick_charm = FlickCharm(parent=self) + self._before_flick_scroll_mode = None + + def is_deselectable(self): + return self._deselectable + + def set_deselectable(self, deselectable): + self._deselectable = deselectable + + deselectable = property(is_deselectable, set_deselectable) + + def mousePressEvent(self, event): + if self._deselectable: + index = self.indexAt(event.pos()) + if not index.isValid(): + # clear the selection + self.clearSelection() + # clear the current index + self.setCurrentIndex(QtCore.QModelIndex()) + super(TreeView, self).mousePressEvent(event) + + def mouseDoubleClickEvent(self, event): + if event.button() == QtCore.Qt.LeftButton: + self.double_clicked_left.emit() + + elif event.button() == QtCore.Qt.RightButton: + self.double_clicked_right.emit() + + return super(TreeView, self).mouseDoubleClickEvent(event) + + def activate_flick_charm(self): + if self._flick_charm_activated: + return + self._flick_charm_activated = True + self._before_flick_scroll_mode = self.verticalScrollMode() + self._flick_charm.activateOn(self) + self.setVerticalScrollMode(QtWidgets.QAbstractItemView.ScrollPerPixel) + + def deactivate_flick_charm(self): + if not self._flick_charm_activated: + return + self._flick_charm_activated = False + self._flick_charm.deactivateFrom(self) + if self._before_flick_scroll_mode is not None: + self.setVerticalScrollMode(self._before_flick_scroll_mode) + + +class BaseOverlayFrame(QtWidgets.QFrame): + """Base frame for overlay widgets. + + Has implemented automated resize and event filtering. + """ + + def __init__(self, parent): + super(BaseOverlayFrame, self).__init__(parent) + self.setObjectName("OverlayFrame") + + self._parent = parent + + def setVisible(self, visible): + super(BaseOverlayFrame, self).setVisible(visible) + if visible: + self._parent.installEventFilter(self) + self.resize(self._parent.size()) + else: + self._parent.removeEventFilter(self) + + def eventFilter(self, obj, event): + if event.type() == QtCore.QEvent.Resize: + self.resize(obj.size()) + + return super(BaseOverlayFrame, self).eventFilter(obj, event) diff --git a/openpype/tools/ayon_workfiles/widgets/window.py b/openpype/tools/ayon_workfiles/widgets/window.py new file mode 100644 index 0000000000..ef352c8b18 --- /dev/null +++ b/openpype/tools/ayon_workfiles/widgets/window.py @@ -0,0 +1,400 @@ +from qtpy import QtCore, QtWidgets, QtGui + +from openpype import style, resources +from openpype.tools.utils import ( + PlaceholderLineEdit, + MessageOverlayObject, +) +from openpype.tools.utils.lib import get_qta_icon_by_name_and_color + +from openpype.tools.ayon_workfiles.control import BaseWorkfileController + +from .side_panel import SidePanelWidget +from .folders_widget import FoldersWidget +from .tasks_widget import TasksWidget +from .files_widget import FilesWidget +from .utils import BaseOverlayFrame + + +# TODO move to utils +# from openpype.tools.utils.lib import ( +# get_refresh_icon, get_go_to_current_icon) +def get_refresh_icon(): + return get_qta_icon_by_name_and_color( + "fa.refresh", style.get_default_tools_icon_color() + ) + + +def get_go_to_current_icon(): + return get_qta_icon_by_name_and_color( + "fa.arrow-down", style.get_default_tools_icon_color() + ) + + +class InvalidHostOverlay(BaseOverlayFrame): + def __init__(self, parent): + super(InvalidHostOverlay, self).__init__(parent) + + label_widget = QtWidgets.QLabel( + ( + "Workfiles tool is not supported in this host/DCCs." + "

This may be caused by a bug." + " Please contact your TD for more information." + ), + self + ) + label_widget.setAlignment(QtCore.Qt.AlignCenter) + label_widget.setObjectName("OverlayFrameLabel") + + layout = QtWidgets.QVBoxLayout(self) + layout.addStretch(2) + layout.addWidget(label_widget, 0, QtCore.Qt.AlignCenter) + layout.addStretch(3) + + label_widget.setAttribute(QtCore.Qt.WA_TranslucentBackground) + + +class WorkfilesToolWindow(QtWidgets.QWidget): + """WorkFiles Window. + + Main windows of workfiles tool. + + Args: + controller (AbstractWorkfilesFrontend): Frontend controller. + parent (Optional[QtWidgets.QWidget]): Parent widget. + """ + + title = "Work Files" + + def __init__(self, controller=None, parent=None): + super(WorkfilesToolWindow, self).__init__(parent=parent) + + if controller is None: + controller = BaseWorkfileController() + + self.setWindowTitle(self.title) + icon = QtGui.QIcon(resources.get_openpype_icon_filepath()) + self.setWindowIcon(icon) + flags = self.windowFlags() | QtCore.Qt.Window + self.setWindowFlags(flags) + + self._default_window_flags = flags + + self._folder_widget = None + self._folder_filter_input = None + + self._files_widget = None + + self._first_show = True + self._controller_refreshed = False + self._context_to_set = None + # Host validation should happen only once + self._host_is_valid = None + + self._controller = controller + + # Create pages widget and set it as central widget + pages_widget = QtWidgets.QStackedWidget(self) + + home_page_widget = QtWidgets.QWidget(pages_widget) + home_body_widget = QtWidgets.QWidget(home_page_widget) + + col_1_widget = self._create_col_1_widget(controller, parent) + tasks_widget = TasksWidget(controller, home_body_widget) + col_3_widget = self._create_col_3_widget(controller, home_body_widget) + side_panel = SidePanelWidget(controller, home_body_widget) + + pages_widget.addWidget(home_page_widget) + + # Build home + home_page_layout = QtWidgets.QVBoxLayout(home_page_widget) + home_page_layout.addWidget(home_body_widget) + + # Build home - body + body_layout = QtWidgets.QVBoxLayout(home_body_widget) + split_widget = QtWidgets.QSplitter(home_body_widget) + split_widget.addWidget(col_1_widget) + split_widget.addWidget(tasks_widget) + split_widget.addWidget(col_3_widget) + split_widget.addWidget(side_panel) + split_widget.setSizes([255, 160, 455, 175]) + + body_layout.addWidget(split_widget) + + main_layout = QtWidgets.QHBoxLayout(self) + main_layout.addWidget(pages_widget, 1) + + overlay_messages_widget = MessageOverlayObject(self) + overlay_invalid_host = InvalidHostOverlay(self) + overlay_invalid_host.setVisible(False) + + first_show_timer = QtCore.QTimer() + first_show_timer.setSingleShot(True) + first_show_timer.setInterval(50) + + first_show_timer.timeout.connect(self._on_first_show) + + controller.register_event_callback( + "save_as.finished", + self._on_save_as_finished, + ) + controller.register_event_callback( + "copy_representation.finished", + self._on_copy_representation_finished, + ) + controller.register_event_callback( + "workfile_duplicate.finished", + self._on_duplicate_finished + ) + controller.register_event_callback( + "open_workfile.finished", + self._on_open_finished + ) + controller.register_event_callback( + "controller.refresh.started", + self._on_controller_refresh_started, + ) + controller.register_event_callback( + "controller.refresh.finished", + self._on_controller_refresh_finished, + ) + + self._overlay_messages_widget = overlay_messages_widget + self._overlay_invalid_host = overlay_invalid_host + self._home_page_widget = home_page_widget + self._pages_widget = pages_widget + self._home_body_widget = home_body_widget + self._split_widget = split_widget + + self._tasks_widget = tasks_widget + self._side_panel = side_panel + + self._first_show_timer = first_show_timer + + self._post_init() + + def _post_init(self): + self._on_published_checkbox_changed() + + # Force focus on the open button by default, required for Houdini. + self._files_widget.setFocus() + + self.resize(1200, 600) + + def _create_col_1_widget(self, controller, parent): + col_widget = QtWidgets.QWidget(parent) + header_widget = QtWidgets.QWidget(col_widget) + + folder_filter_input = PlaceholderLineEdit(header_widget) + folder_filter_input.setPlaceholderText("Filter folders..") + + go_to_current_btn = QtWidgets.QPushButton(header_widget) + go_to_current_btn.setIcon(get_go_to_current_icon()) + go_to_current_btn_sp = go_to_current_btn.sizePolicy() + go_to_current_btn_sp.setVerticalPolicy(QtWidgets.QSizePolicy.Minimum) + go_to_current_btn.setSizePolicy(go_to_current_btn_sp) + + refresh_btn = QtWidgets.QPushButton(header_widget) + refresh_btn.setIcon(get_refresh_icon()) + refresh_btn_sp = refresh_btn.sizePolicy() + refresh_btn_sp.setVerticalPolicy(QtWidgets.QSizePolicy.Minimum) + refresh_btn.setSizePolicy(refresh_btn_sp) + + folder_widget = FoldersWidget(controller, col_widget) + + header_layout = QtWidgets.QHBoxLayout(header_widget) + header_layout.setContentsMargins(0, 0, 0, 0) + header_layout.addWidget(folder_filter_input, 1) + header_layout.addWidget(go_to_current_btn, 0) + header_layout.addWidget(refresh_btn, 0) + + col_layout = QtWidgets.QVBoxLayout(col_widget) + col_layout.setContentsMargins(0, 0, 0, 0) + col_layout.addWidget(header_widget, 0) + col_layout.addWidget(folder_widget, 1) + + folder_filter_input.textChanged.connect(self._on_folder_filter_change) + go_to_current_btn.clicked.connect(self._on_go_to_current_clicked) + refresh_btn.clicked.connect(self._on_refresh_clicked) + + self._folder_filter_input = folder_filter_input + self._folder_widget = folder_widget + + return col_widget + + def _create_col_3_widget(self, controller, parent): + col_widget = QtWidgets.QWidget(parent) + + header_widget = QtWidgets.QWidget(col_widget) + + files_filter_input = PlaceholderLineEdit(header_widget) + files_filter_input.setPlaceholderText("Filter files..") + + published_checkbox = QtWidgets.QCheckBox("Published", header_widget) + published_checkbox.setToolTip("Show published workfiles") + + header_layout = QtWidgets.QHBoxLayout(header_widget) + header_layout.setContentsMargins(0, 0, 0, 0) + header_layout.addWidget(files_filter_input, 1) + header_layout.addWidget(published_checkbox, 0) + + files_widget = FilesWidget(controller, col_widget) + + col_layout = QtWidgets.QVBoxLayout(col_widget) + col_layout.setContentsMargins(0, 0, 0, 0) + col_layout.addWidget(header_widget, 0) + col_layout.addWidget(files_widget, 1) + + files_filter_input.textChanged.connect( + self._on_file_text_filter_change) + published_checkbox.stateChanged.connect( + self._on_published_checkbox_changed + ) + + self._files_filter_input = files_filter_input + self._published_checkbox = published_checkbox + + self._files_widget = files_widget + + return col_widget + + def set_window_on_top(self, on_top): + """Set window on top of other windows. + + Args: + on_top (bool): Show on top of other windows. + """ + + flags = self._default_window_flags + if on_top: + flags |= QtCore.Qt.WindowStaysOnTopHint + if self.windowFlags() != flags: + self.setWindowFlags(flags) + + def ensure_visible(self, use_context=True, save=True, on_top=False): + """Ensure the window is visible. + + This method expects arguments for compatibility with previous variant + of Workfiles tool. + + Args: + use_context (Optional[bool]): DEPRECATED: This argument is + ignored. + save (Optional[bool]): Allow to save workfiles. + on_top (Optional[bool]): Show on top of other windows. + """ + + save = True if save is None else save + on_top = False if on_top is None else on_top + + is_visible = self.isVisible() + self._controller.set_save_enabled(save) + self.set_window_on_top(on_top) + + self.show() + self.raise_() + self.activateWindow() + if is_visible: + self.refresh() + + def refresh(self): + """Trigger refresh of workfiles tool controller.""" + + self._controller.refresh() + + def showEvent(self, event): + super(WorkfilesToolWindow, self).showEvent(event) + if self._first_show: + self._first_show = False + self._first_show_timer.start() + self.setStyleSheet(style.load_stylesheet()) + + def keyPressEvent(self, event): + """Custom keyPressEvent. + + Override keyPressEvent to do nothing so that Maya's panels won't + take focus when pressing "SHIFT" whilst mouse is over viewport or + outliner. This way users don't accidentally perform Maya commands + whilst trying to name an instance. + """ + + pass + + def _on_first_show(self): + if not self._controller_refreshed: + self.refresh() + + def _on_file_text_filter_change(self, text): + self._files_widget.set_text_filter(text) + + def _on_published_checkbox_changed(self): + """Publish mode changed. + + Tell children widgets about it so they can handle the mode. + """ + + published_mode = self._published_checkbox.isChecked() + self._files_widget.set_published_mode(published_mode) + self._side_panel.set_published_mode(published_mode) + + def _on_folder_filter_change(self, text): + self._folder_widget.set_name_filer(text) + + def _on_go_to_current_clicked(self): + self._controller.go_to_current_context() + + def _on_refresh_clicked(self): + self.refresh() + + def _on_controller_refresh_started(self): + self._controller_refreshed = True + + def _on_controller_refresh_finished(self): + if self._host_is_valid is None: + self._host_is_valid = self._controller.is_host_valid() + self._overlay_invalid_host.setVisible(not self._host_is_valid) + + if not self._host_is_valid: + return + + def _on_save_as_finished(self, event): + if event["failed"]: + self._overlay_messages_widget.add_message( + "Failed to save workfile", + "error", + ) + else: + self._overlay_messages_widget.add_message( + "Workfile saved" + ) + + def _on_copy_representation_finished(self, event): + if event["failed"]: + self._overlay_messages_widget.add_message( + "Failed to copy published workfile", + "error", + ) + else: + self._overlay_messages_widget.add_message( + "Publish workfile saved" + ) + + def _on_duplicate_finished(self, event): + if event["failed"]: + self._overlay_messages_widget.add_message( + "Failed to duplicate workfile", + "error", + ) + else: + self._overlay_messages_widget.add_message( + "Workfile duplicated" + ) + + def _on_open_finished(self, event): + if event["failed"]: + self._overlay_messages_widget.add_message( + "Failed to open workfile", + "error", + ) + else: + self.close() diff --git a/openpype/tools/publisher/widgets/screenshot_widget.py b/openpype/tools/publisher/widgets/screenshot_widget.py index 4ccf920571..3504b419b4 100644 --- a/openpype/tools/publisher/widgets/screenshot_widget.py +++ b/openpype/tools/publisher/widgets/screenshot_widget.py @@ -26,15 +26,6 @@ class ScreenMarquee(QtWidgets.QDialog): self.setCursor(QtCore.Qt.CrossCursor) self.setMouseTracking(True) - fade_anim = QtCore.QVariantAnimation() - fade_anim.setStartValue(0) - fade_anim.setEndValue(50) - fade_anim.setDuration(200) - fade_anim.setEasingCurve(QtCore.QEasingCurve.OutCubic) - fade_anim.start(QtCore.QAbstractAnimation.DeleteWhenStopped) - - fade_anim.valueChanged.connect(self._on_fade_anim) - app = QtWidgets.QApplication.instance() if hasattr(app, "screenAdded"): app.screenAdded.connect(self._on_screen_added) @@ -46,12 +37,10 @@ class ScreenMarquee(QtWidgets.QDialog): for screen in QtWidgets.QApplication.screens(): screen.geometryChanged.connect(self._fit_screen_geometry) - self._opacity = fade_anim.currentValue() + self._opacity = 50 self._click_pos = None self._capture_rect = None - self._fade_anim = fade_anim - def get_captured_pixmap(self): if self._capture_rect is None: return QtGui.QPixmap() @@ -68,28 +57,33 @@ class ScreenMarquee(QtWidgets.QDialog): click_pos = self.mapFromGlobal(self._click_pos) painter = QtGui.QPainter(self) + painter.setRenderHints( + QtGui.QPainter.Antialiasing + | QtGui.QPainter.SmoothPixmapTransform + ) # Draw background. Aside from aesthetics, this makes the full # tool region accept mouse events. painter.setBrush(QtGui.QColor(0, 0, 0, self._opacity)) painter.setPen(QtCore.Qt.NoPen) - painter.drawRect(event.rect()) + rect = event.rect() + fill_path = QtGui.QPainterPath() + fill_path.addRect(rect) # Clear the capture area if click_pos is not None: + sub_path = QtGui.QPainterPath() capture_rect = QtCore.QRect(click_pos, mouse_pos) - painter.setCompositionMode( - QtGui.QPainter.CompositionMode_Clear) - painter.drawRect(capture_rect) - painter.setCompositionMode( - QtGui.QPainter.CompositionMode_SourceOver) + sub_path.addRect(capture_rect) + fill_path = fill_path.subtracted(sub_path) - pen_color = QtGui.QColor(255, 255, 255, 64) + painter.drawPath(fill_path) + + pen_color = QtGui.QColor(255, 255, 255, self._opacity) pen = QtGui.QPen(pen_color, 1, QtCore.Qt.DotLine) painter.setPen(pen) # Draw cropping markers at click position - rect = event.rect() if click_pos is not None: painter.drawLine( rect.left(), click_pos.y(), @@ -109,6 +103,7 @@ class ScreenMarquee(QtWidgets.QDialog): mouse_pos.x(), rect.top(), mouse_pos.x(), rect.bottom() ) + painter.end() def mousePressEvent(self, event): """Mouse click event""" @@ -139,13 +134,13 @@ class ScreenMarquee(QtWidgets.QDialog): if event.key() == QtCore.Qt.Key_Escape: self._click_pos = None self._capture_rect = None + event.accept() self.close() return - return super(ScreenMarquee, self).mousePressEvent(event) + return super(ScreenMarquee, self).keyPressEvent(event) def showEvent(self, event): self._fit_screen_geometry() - self._fade_anim.start() def _fit_screen_geometry(self): # Compute the union of all screen geometries, and resize to fit. @@ -154,12 +149,6 @@ class ScreenMarquee(QtWidgets.QDialog): workspace_rect = workspace_rect.united(screen.geometry()) self.setGeometry(workspace_rect) - def _on_fade_anim(self): - """Animation callback for opacity.""" - - self._opacity = self._fade_anim.currentValue() - self.repaint() - def _on_screen_added(self): for screen in QtGui.QGuiApplication.screens(): screen.geometryChanged.connect(self._fit_screen_geometry) diff --git a/openpype/tools/settings/settings/item_widgets.py b/openpype/tools/settings/settings/item_widgets.py index 117eca7d6b..2fd13cbbd8 100644 --- a/openpype/tools/settings/settings/item_widgets.py +++ b/openpype/tools/settings/settings/item_widgets.py @@ -4,6 +4,7 @@ from qtpy import QtWidgets, QtCore, QtGui from openpype.widgets.sliders import NiceSlider from openpype.tools.settings import CHILD_OFFSET +from openpype.tools.utils import MultiSelectionComboBox from openpype.settings.entities.exceptions import BaseInvalidValue from .widgets import ( @@ -15,7 +16,6 @@ from .widgets import ( SettingsNiceCheckbox, SettingsLineEdit ) -from .multiselection_combobox import MultiSelectionComboBox from .wrapper_widgets import ( WrapperWidget, CollapsibleWrapper, diff --git a/openpype/tools/utils/__init__.py b/openpype/tools/utils/__init__.py index f35bfaee70..d343353112 100644 --- a/openpype/tools/utils/__init__.py +++ b/openpype/tools/utils/__init__.py @@ -38,6 +38,7 @@ from .models import ( from .overlay_messages import ( MessageOverlayObject, ) +from .multiselection_combobox import MultiSelectionComboBox __all__ = ( @@ -78,4 +79,6 @@ __all__ = ( "RecursiveSortFilterProxyModel", "MessageOverlayObject", + + "MultiSelectionComboBox", ) diff --git a/openpype/tools/utils/host_tools.py b/openpype/tools/utils/host_tools.py index bc4b7867c2..2ebc973a47 100644 --- a/openpype/tools/utils/host_tools.py +++ b/openpype/tools/utils/host_tools.py @@ -6,6 +6,8 @@ use singleton approach with global functions (using helper anyway). import os import pyblish.api + +from openpype import AYON_SERVER_ENABLED from openpype.host import IWorkfileHost, ILoadHost from openpype.lib import Logger from openpype.pipeline import ( @@ -46,17 +48,29 @@ class HostToolsHelper: self._log = Logger.get_logger(self.__class__.__name__) return self._log + def _init_ayon_workfiles_tool(self, parent): + from openpype.tools.ayon_workfiles.widgets import WorkfilesToolWindow + + workfiles_window = WorkfilesToolWindow(parent=parent) + self._workfiles_tool = workfiles_window + + def _init_openpype_workfiles_tool(self, parent): + from openpype.tools.workfiles.app import Window + + # Host validation + host = registered_host() + IWorkfileHost.validate_workfile_methods(host) + + workfiles_window = Window(parent=parent) + self._workfiles_tool = workfiles_window + def get_workfiles_tool(self, parent): """Create, cache and return workfiles tool window.""" if self._workfiles_tool is None: - from openpype.tools.workfiles.app import Window - - # Host validation - host = registered_host() - IWorkfileHost.validate_workfile_methods(host) - - workfiles_window = Window(parent=parent) - self._workfiles_tool = workfiles_window + if AYON_SERVER_ENABLED: + self._init_ayon_workfiles_tool(parent) + else: + self._init_openpype_workfiles_tool(parent) return self._workfiles_tool diff --git a/openpype/tools/utils/lib.py b/openpype/tools/utils/lib.py index 2df46c1eae..723e71e7aa 100644 --- a/openpype/tools/utils/lib.py +++ b/openpype/tools/utils/lib.py @@ -170,8 +170,12 @@ def get_openpype_qt_app(): if attr is not None: QtWidgets.QApplication.setAttribute(attr) - if hasattr( - QtWidgets.QApplication, "setHighDpiScaleFactorRoundingPolicy" + policy = os.getenv("QT_SCALE_FACTOR_ROUNDING_POLICY") + if ( + hasattr( + QtWidgets.QApplication, "setHighDpiScaleFactorRoundingPolicy" + ) + and not policy ): QtWidgets.QApplication.setHighDpiScaleFactorRoundingPolicy( QtCore.Qt.HighDpiScaleFactorRoundingPolicy.PassThrough diff --git a/openpype/tools/settings/settings/multiselection_combobox.py b/openpype/tools/utils/multiselection_combobox.py similarity index 84% rename from openpype/tools/settings/settings/multiselection_combobox.py rename to openpype/tools/utils/multiselection_combobox.py index d64fc83745..34361fca17 100644 --- a/openpype/tools/settings/settings/multiselection_combobox.py +++ b/openpype/tools/utils/multiselection_combobox.py @@ -1,9 +1,10 @@ from qtpy import QtCore, QtGui, QtWidgets -from openpype.tools.utils.lib import ( + +from .lib import ( checkstate_int_to_enum, checkstate_enum_to_int, ) -from openpype.tools.utils.constants import ( +from .constants import ( CHECKED_INT, UNCHECKED_INT, ITEM_IS_USER_TRISTATE, @@ -60,12 +61,25 @@ class MultiSelectionComboBox(QtWidgets.QComboBox): self._block_mouse_release_timer = QtCore.QTimer(self, singleShot=True) self._initial_mouse_pos = None self._separator = separator - self.placeholder_text = placeholder - self.delegate = ComboItemDelegate(self) - self.setItemDelegate(self.delegate) + self._placeholder_text = placeholder + delegate = ComboItemDelegate(self) + self.setItemDelegate(delegate) - self.lines = {} - self.item_height = None + self._lines = {} + self._item_height = None + self._custom_text = None + self._delegate = delegate + + def get_placeholder_text(self): + return self._placeholder_text + + def set_placeholder_text(self, text): + self._placeholder_text = text + self._update_size_hint() + + def set_custom_text(self, text): + self._custom_text = text + self._update_size_hint() def focusInEvent(self, event): self.focused_in.emit() @@ -158,7 +172,7 @@ class MultiSelectionComboBox(QtWidgets.QComboBox): if new_state is not None: model.setData(current_index, new_state, QtCore.Qt.CheckStateRole) self.view().update(current_index) - self.update_size_hint() + self._update_size_hint() self.value_changed.emit() return True @@ -182,25 +196,33 @@ class MultiSelectionComboBox(QtWidgets.QComboBox): self.initStyleOption(option) painter.drawComplexControl(QtWidgets.QStyle.CC_ComboBox, option) - # draw the icon and text items = self.checked_items_text() - if not items: - option.currentText = self.placeholder_text + # draw the icon and text + draw_text = True + combotext = None + if self._custom_text is not None: + combotext = self._custom_text + elif not items: + combotext = self._placeholder_text + else: + draw_text = False + if draw_text: + option.currentText = combotext option.palette.setCurrentColorGroup(QtGui.QPalette.Disabled) painter.drawControl(QtWidgets.QStyle.CE_ComboBoxLabel, option) return font_metricts = self.fontMetrics() - if self.item_height is None: + if self._item_height is None: self.updateGeometry() self.update() return - for line, items in self.lines.items(): + for line, items in self._lines.items(): top_y = ( option.rect.top() - + (line * self.item_height) + + (line * self._item_height) + self.top_bottom_margins ) left_x = option.rect.left() + self.left_offset @@ -210,7 +232,7 @@ class MultiSelectionComboBox(QtWidgets.QComboBox): label_rect.moveTop(top_y) label_rect.moveLeft(left_x) - label_rect.setHeight(self.item_height) + label_rect.setHeight(self._item_height) label_rect.setWidth( label_rect.width() + self.left_right_padding ) @@ -239,14 +261,18 @@ class MultiSelectionComboBox(QtWidgets.QComboBox): def resizeEvent(self, *args, **kwargs): super(MultiSelectionComboBox, self).resizeEvent(*args, **kwargs) - self.update_size_hint() + self._update_size_hint() - def update_size_hint(self): - self.lines = {} + def _update_size_hint(self): + if self._custom_text is not None: + self.update() + return + self._lines = {} items = self.checked_items_text() if not items: self.update() + self.repaint() return option = QtWidgets.QStyleOptionComboBox() @@ -259,7 +285,7 @@ class MultiSelectionComboBox(QtWidgets.QComboBox): total_width = option.rect.width() - btn_rect.width() line = 0 - self.lines = {line: []} + self._lines = {line: []} font_metricts = self.fontMetrics() default_left_x = 0 + self.left_offset @@ -270,18 +296,18 @@ class MultiSelectionComboBox(QtWidgets.QComboBox): right_x = left_x + width if right_x > total_width: left_x = int(default_left_x) - if self.lines.get(line): + if self._lines.get(line): line += 1 - self.lines[line] = [item] + self._lines[line] = [item] left_x += width else: - self.lines[line] = [item] + self._lines[line] = [item] line += 1 else: - if line in self.lines: - self.lines[line].append(item) + if line in self._lines: + self._lines[line].append(item) else: - self.lines[line] = [item] + self._lines[line] = [item] left_x = left_x + width + self.item_spacing self.update() @@ -289,18 +315,20 @@ class MultiSelectionComboBox(QtWidgets.QComboBox): def sizeHint(self): value = super(MultiSelectionComboBox, self).sizeHint() - lines = len(self.lines) - if lines == 0: - lines = 1 + lines = 1 + if self._custom_text is None: + lines = len(self._lines) + if lines == 0: + lines = 1 - if self.item_height is None: - self.item_height = ( + if self._item_height is None: + self._item_height = ( self.fontMetrics().height() + (2 * self.top_bottom_padding) + (2 * self.top_bottom_margins) ) value.setHeight( - (lines * self.item_height) + (lines * self._item_height) + (2 * self.top_bottom_margins) ) return value @@ -316,7 +344,7 @@ class MultiSelectionComboBox(QtWidgets.QComboBox): else: check_state = UNCHECKED_INT self.setItemData(idx, check_state, QtCore.Qt.CheckStateRole) - self.update_size_hint() + self._update_size_hint() def value(self): items = list() diff --git a/openpype/version.py b/openpype/version.py index 6d89e1eeae..483b70436a 100644 --- a/openpype/version.py +++ b/openpype/version.py @@ -1,3 +1,3 @@ # -*- coding: utf-8 -*- """Package declaring Pype version.""" -__version__ = "3.16.5-nightly.2" +__version__ = "3.17.1-nightly.1" diff --git a/poetry.lock b/poetry.lock index 5621d39988..d074a0c3d9 100644 --- a/poetry.lock +++ b/poetry.lock @@ -1,10 +1,9 @@ -# This file is automatically @generated by Poetry and should not be changed by hand. +# This file is automatically @generated by Poetry 1.5.1 and should not be changed by hand. [[package]] name = "acre" version = "1.0.0" description = "Lightweight cross-platform environment management Python package that makes it trivial to launch applications in their own configurable working environment." -category = "main" optional = false python-versions = ">=2.7" files = [] @@ -20,7 +19,6 @@ resolved_reference = "126f7a188cfe36718f707f42ebbc597e86aa86c3" name = "aiohttp" version = "3.8.4" description = "Async http client/server framework (asyncio)" -category = "main" optional = false python-versions = ">=3.6" files = [ @@ -129,7 +127,6 @@ speedups = ["Brotli", "aiodns", "cchardet"] name = "aiohttp-json-rpc" version = "0.13.3" description = "Implementation JSON-RPC 2.0 server and client using aiohttp on top of websockets transport" -category = "main" optional = false python-versions = ">=3.5" files = [ @@ -144,7 +141,6 @@ aiohttp = ">=3,<4" name = "aiohttp-middlewares" version = "2.2.0" description = "Collection of useful middlewares for aiohttp applications." -category = "main" optional = false python-versions = ">=3.7,<4.0" files = [ @@ -161,7 +157,6 @@ yarl = ">=1.5.1,<2.0.0" name = "aiosignal" version = "1.3.1" description = "aiosignal: a list of registered asynchronous callbacks" -category = "main" optional = false python-versions = ">=3.7" files = [ @@ -176,7 +171,6 @@ frozenlist = ">=1.1.0" name = "alabaster" version = "0.7.13" description = "A configurable sidebar-enabled Sphinx theme" -category = "dev" optional = false python-versions = ">=3.6" files = [ @@ -188,7 +182,6 @@ files = [ name = "ansicon" version = "1.89.0" description = "Python wrapper for loading Jason Hood's ANSICON" -category = "main" optional = false python-versions = "*" files = [ @@ -200,7 +193,6 @@ files = [ name = "appdirs" version = "1.4.4" description = "A small Python module for determining appropriate platform-specific dirs, e.g. a \"user data dir\"." -category = "main" optional = false python-versions = ">=2.7, !=3.0.*, !=3.1.*, !=3.2.*, !=3.3.*, !=3.4.*" files = [] @@ -216,7 +208,6 @@ resolved_reference = "8734277956c1df3b85385e6b308e954910533884" name = "arrow" version = "0.17.0" description = "Better dates & times for Python" -category = "main" optional = false python-versions = ">=2.7, !=3.0.*, !=3.1.*, !=3.2.*, !=3.3.*, !=3.4.*" files = [ @@ -231,7 +222,6 @@ python-dateutil = ">=2.7.0" name = "astroid" version = "2.15.5" description = "An abstract syntax tree for Python with inference support." -category = "dev" optional = false python-versions = ">=3.7.2" files = [ @@ -248,7 +238,6 @@ wrapt = {version = ">=1.11,<2", markers = "python_version < \"3.11\""} name = "async-timeout" version = "4.0.2" description = "Timeout context manager for asyncio programs" -category = "main" optional = false python-versions = ">=3.6" files = [ @@ -260,7 +249,6 @@ files = [ name = "atomicwrites" version = "1.4.1" description = "Atomic file writes." -category = "dev" optional = false python-versions = ">=2.7, !=3.0.*, !=3.1.*, !=3.2.*, !=3.3.*" files = [ @@ -271,7 +259,6 @@ files = [ name = "attrs" version = "23.1.0" description = "Classes Without Boilerplate" -category = "main" optional = false python-versions = ">=3.7" files = [ @@ -290,7 +277,6 @@ tests-no-zope = ["cloudpickle", "hypothesis", "mypy (>=1.1.1)", "pympler", "pyte name = "autopep8" version = "2.0.2" description = "A tool that automatically formats Python code to conform to the PEP 8 style guide" -category = "dev" optional = false python-versions = ">=3.6" files = [ @@ -306,7 +292,6 @@ tomli = {version = "*", markers = "python_version < \"3.11\""} name = "babel" version = "2.12.1" description = "Internationalization utilities" -category = "dev" optional = false python-versions = ">=3.7" files = [ @@ -318,7 +303,6 @@ files = [ name = "bcrypt" version = "4.0.1" description = "Modern password hashing for your software and your servers" -category = "main" optional = false python-versions = ">=3.6" files = [ @@ -353,7 +337,6 @@ typecheck = ["mypy"] name = "bidict" version = "0.22.1" description = "The bidirectional mapping library for Python." -category = "main" optional = false python-versions = ">=3.7" files = [ @@ -370,7 +353,6 @@ test = ["hypothesis", "pytest", "pytest-benchmark[histogram]", "pytest-cov", "py name = "blessed" version = "1.20.0" description = "Easy, practical library for making terminal apps, by providing an elegant, well-documented interface to Colors, Keyboard input, and screen Positioning capabilities." -category = "main" optional = false python-versions = ">=2.7" files = [ @@ -387,7 +369,6 @@ wcwidth = ">=0.1.4" name = "cachetools" version = "5.3.1" description = "Extensible memoizing collections and decorators" -category = "main" optional = false python-versions = ">=3.7" files = [ @@ -397,21 +378,19 @@ files = [ [[package]] name = "certifi" -version = "2023.5.7" +version = "2023.7.22" description = "Python package for providing Mozilla's CA Bundle." -category = "main" optional = false python-versions = ">=3.6" files = [ - {file = "certifi-2023.5.7-py3-none-any.whl", hash = "sha256:c6c2e98f5c7869efca1f8916fed228dd91539f9f1b444c314c06eef02980c716"}, - {file = "certifi-2023.5.7.tar.gz", hash = "sha256:0f0d56dc5a6ad56fd4ba36484d6cc34451e1c6548c61daad8c320169f91eddc7"}, + {file = "certifi-2023.7.22-py3-none-any.whl", hash = "sha256:92d6037539857d8206b8f6ae472e8b77db8058fec5937a1ef3f54304089edbb9"}, + {file = "certifi-2023.7.22.tar.gz", hash = "sha256:539cc1d13202e33ca466e88b2807e29f4c13049d6d87031a3c110744495cb082"}, ] [[package]] name = "cffi" version = "1.15.1" description = "Foreign Function Interface for Python calling C code." -category = "main" optional = false python-versions = "*" files = [ @@ -488,7 +467,6 @@ pycparser = "*" name = "cfgv" version = "3.3.1" description = "Validate configuration and produce human readable error messages." -category = "dev" optional = false python-versions = ">=3.6.1" files = [ @@ -500,7 +478,6 @@ files = [ name = "charset-normalizer" version = "3.1.0" description = "The Real First Universal Charset Detector. Open, modern and actively maintained alternative to Chardet." -category = "main" optional = false python-versions = ">=3.7.0" files = [ @@ -583,24 +560,19 @@ files = [ [[package]] name = "click" -version = "8.1.3" +version = "7.1.2" description = "Composable command line interface toolkit" -category = "main" optional = false -python-versions = ">=3.7" +python-versions = ">=2.7, !=3.0.*, !=3.1.*, !=3.2.*, !=3.3.*, !=3.4.*" files = [ - {file = "click-8.1.3-py3-none-any.whl", hash = "sha256:bb4d8133cb15a609f44e8213d9b391b0809795062913b383c62be0ee95b1db48"}, - {file = "click-8.1.3.tar.gz", hash = "sha256:7682dc8afb30297001674575ea00d1814d808d6a36af415a82bd481d37ba7b8e"}, + {file = "click-7.1.2-py2.py3-none-any.whl", hash = "sha256:dacca89f4bfadd5de3d7489b7c8a566eee0d3676333fbb50030263894c38c0dc"}, + {file = "click-7.1.2.tar.gz", hash = "sha256:d2b5255c7c6349bc1bd1e59e08cd12acbbd63ce649f2588755783aa94dfb6b1a"}, ] -[package.dependencies] -colorama = {version = "*", markers = "platform_system == \"Windows\""} - [[package]] name = "clique" version = "1.6.1" description = "Manage collections with common numerical component" -category = "main" optional = false python-versions = ">=2.7, <4.0" files = [ @@ -617,7 +589,6 @@ test = ["pytest (>=2.3.5,<5)", "pytest-cov (>=2,<3)", "pytest-runner (>=2.7,<3)" name = "colorama" version = "0.4.6" description = "Cross-platform colored terminal text." -category = "main" optional = false python-versions = "!=3.0.*,!=3.1.*,!=3.2.*,!=3.3.*,!=3.4.*,!=3.5.*,!=3.6.*,>=2.7" files = [ @@ -629,7 +600,6 @@ files = [ name = "commonmark" version = "0.9.1" description = "Python parser for the CommonMark Markdown spec" -category = "dev" optional = false python-versions = "*" files = [ @@ -644,7 +614,6 @@ test = ["flake8 (==3.7.8)", "hypothesis (==3.55.3)"] name = "coolname" version = "2.2.0" description = "Random name and slug generator" -category = "main" optional = false python-versions = "*" files = [ @@ -656,7 +625,6 @@ files = [ name = "coverage" version = "7.2.7" description = "Code coverage measurement for Python" -category = "dev" optional = false python-versions = ">=3.7" files = [ @@ -732,7 +700,6 @@ toml = ["tomli"] name = "cryptography" version = "39.0.0" description = "cryptography is a package which provides cryptographic recipes and primitives to Python developers." -category = "main" optional = false python-versions = ">=3.6" files = [ @@ -776,7 +743,6 @@ test = ["hypothesis (>=1.11.4,!=3.79.2)", "iso8601", "pretend", "pytest (>=6.2.0 name = "cx-freeze" version = "6.12.0" description = "Create standalone executables from Python scripts" -category = "dev" optional = false python-versions = ">=3.6" files = [ @@ -834,7 +800,6 @@ test = ["nose (==1.3.7)", "pygments (>=2.11.2)", "pytest (>=7.0.1)", "pytest-cov name = "cx-logging" version = "3.1.0" description = "Python and C interfaces for logging" -category = "dev" optional = false python-versions = ">=3.7" files = [ @@ -865,7 +830,6 @@ files = [ name = "dill" version = "0.3.6" description = "serialize all of python" -category = "dev" optional = false python-versions = ">=3.7" files = [ @@ -880,7 +844,6 @@ graph = ["objgraph (>=1.7.2)"] name = "distlib" version = "0.3.6" description = "Distribution utilities" -category = "dev" optional = false python-versions = "*" files = [ @@ -892,7 +855,6 @@ files = [ name = "dnspython" version = "2.3.0" description = "DNS toolkit" -category = "main" optional = false python-versions = ">=3.7,<4.0" files = [ @@ -913,7 +875,6 @@ wmi = ["wmi (>=1.5.1,<2.0.0)"] name = "docutils" version = "0.19" description = "Docutils -- Python Documentation Utilities" -category = "dev" optional = false python-versions = ">=3.7" files = [ @@ -925,7 +886,6 @@ files = [ name = "dropbox" version = "11.36.0" description = "Official Dropbox API Client" -category = "main" optional = false python-versions = "*" files = [ @@ -943,7 +903,6 @@ stone = ">=2" name = "enlighten" version = "1.11.2" description = "Enlighten Progress Bar" -category = "main" optional = false python-versions = "*" files = [ @@ -959,7 +918,6 @@ prefixed = ">=0.3.2" name = "evdev" version = "1.6.1" description = "Bindings to the Linux input handling subsystem" -category = "main" optional = false python-versions = "*" files = [ @@ -970,7 +928,6 @@ files = [ name = "filelock" version = "3.12.0" description = "A platform independent file lock." -category = "dev" optional = false python-versions = ">=3.7" files = [ @@ -986,7 +943,6 @@ testing = ["covdefaults (>=2.3)", "coverage (>=7.2.3)", "diff-cover (>=7.5)", "p name = "flake8" version = "6.0.0" description = "the modular source code checker: pep8 pyflakes and co" -category = "dev" optional = false python-versions = ">=3.8.1" files = [ @@ -1003,7 +959,6 @@ pyflakes = ">=3.0.0,<3.1.0" name = "frozenlist" version = "1.3.3" description = "A list-like structure which implements collections.abc.MutableSequence" -category = "main" optional = false python-versions = ">=3.7" files = [ @@ -1087,7 +1042,6 @@ files = [ name = "ftrack-python-api" version = "2.5.0" description = "Python API for ftrack." -category = "main" optional = false python-versions = ">=2.7.9, !=3.0.*, !=3.1.*, !=3.2.*, !=3.3.*, !=3.4.*, !=3.5.*" files = [ @@ -1110,7 +1064,6 @@ websocket-client = ">=0.40.0,<1" name = "future" version = "0.18.3" description = "Clean single-source support for Python 3 and 2" -category = "main" optional = false python-versions = ">=2.6, !=3.0.*, !=3.1.*, !=3.2.*" files = [ @@ -1121,7 +1074,6 @@ files = [ name = "gazu" version = "0.9.3" description = "Gazu is a client for Zou, the API to store the data of your CG production." -category = "main" optional = false python-versions = ">= 2.7, != 3.0.*, != 3.1.*, != 3.2.*, != 3.3.*, != 3.4.*, != 3.5.*, != 3.6.1, != 3.6.2" files = [ @@ -1141,7 +1093,6 @@ test = ["pytest", "pytest-cov", "requests-mock"] name = "gitdb" version = "4.0.10" description = "Git Object Database" -category = "dev" optional = false python-versions = ">=3.7" files = [ @@ -1156,7 +1107,6 @@ smmap = ">=3.0.1,<6" name = "gitpython" version = "3.1.31" description = "GitPython is a Python library used to interact with Git repositories" -category = "dev" optional = false python-versions = ">=3.7" files = [ @@ -1171,7 +1121,6 @@ gitdb = ">=4.0.1,<5" name = "google-api-core" version = "2.11.0" description = "Google API client core library" -category = "main" optional = false python-versions = ">=3.7" files = [ @@ -1194,7 +1143,6 @@ grpcio-gcp = ["grpcio-gcp (>=0.2.2,<1.0dev)"] name = "google-api-python-client" version = "1.12.11" description = "Google API Client Library for Python" -category = "main" optional = false python-versions = ">=2.7,!=3.0.*,!=3.1.*,!=3.2.*,!=3.3.*" files = [ @@ -1214,7 +1162,6 @@ uritemplate = ">=3.0.0,<4dev" name = "google-auth" version = "2.17.3" description = "Google Authentication Library" -category = "main" optional = false python-versions = ">=2.7,!=3.0.*,!=3.1.*,!=3.2.*,!=3.3.*,!=3.4.*,!=3.5.*" files = [ @@ -1239,7 +1186,6 @@ requests = ["requests (>=2.20.0,<3.0.0dev)"] name = "google-auth-httplib2" version = "0.1.0" description = "Google Authentication Library: httplib2 transport" -category = "main" optional = false python-versions = "*" files = [ @@ -1256,7 +1202,6 @@ six = "*" name = "googleapis-common-protos" version = "1.59.0" description = "Common protobufs used in Google APIs" -category = "main" optional = false python-versions = ">=3.7" files = [ @@ -1274,7 +1219,6 @@ grpc = ["grpcio (>=1.44.0,<2.0.0dev)"] name = "httplib2" version = "0.22.0" description = "A comprehensive HTTP client library." -category = "main" optional = false python-versions = ">=2.7, !=3.0.*, !=3.1.*, !=3.2.*, !=3.3.*" files = [ @@ -1289,7 +1233,6 @@ pyparsing = {version = ">=2.4.2,<3.0.0 || >3.0.0,<3.0.1 || >3.0.1,<3.0.2 || >3.0 name = "identify" version = "2.5.24" description = "File identification library for Python" -category = "dev" optional = false python-versions = ">=3.7" files = [ @@ -1304,7 +1247,6 @@ license = ["ukkonen"] name = "idna" version = "3.4" description = "Internationalized Domain Names in Applications (IDNA)" -category = "main" optional = false python-versions = ">=3.5" files = [ @@ -1316,7 +1258,6 @@ files = [ name = "imagesize" version = "1.4.1" description = "Getting image size from png/jpeg/jpeg2000/gif file" -category = "dev" optional = false python-versions = ">=2.7, !=3.0.*, !=3.1.*, !=3.2.*, !=3.3.*" files = [ @@ -1328,7 +1269,6 @@ files = [ name = "importlib-metadata" version = "6.6.0" description = "Read metadata from Python packages" -category = "main" optional = false python-versions = ">=3.7" files = [ @@ -1348,7 +1288,6 @@ testing = ["flake8 (<5)", "flufl.flake8", "importlib-resources (>=1.3)", "packag name = "iniconfig" version = "2.0.0" description = "brain-dead simple config-ini parsing" -category = "dev" optional = false python-versions = ">=3.7" files = [ @@ -1360,7 +1299,6 @@ files = [ name = "isort" version = "5.12.0" description = "A Python utility / library to sort Python imports." -category = "dev" optional = false python-versions = ">=3.8.0" files = [ @@ -1378,7 +1316,6 @@ requirements-deprecated-finder = ["pip-api", "pipreqs"] name = "jedi" version = "0.13.3" description = "An autocompletion tool for Python that can be used for text editors." -category = "dev" optional = false python-versions = ">=2.7, !=3.0.*, !=3.1.*, !=3.2.*, !=3.3.*" files = [ @@ -1396,7 +1333,6 @@ testing = ["colorama", "docopt", "pytest (>=3.1.0)"] name = "jeepney" version = "0.8.0" description = "Low-level, pure Python DBus protocol wrapper." -category = "main" optional = false python-versions = ">=3.7" files = [ @@ -1412,7 +1348,6 @@ trio = ["async_generator", "trio"] name = "jinja2" version = "3.1.2" description = "A very fast and expressive template engine." -category = "dev" optional = false python-versions = ">=3.7" files = [ @@ -1430,7 +1365,6 @@ i18n = ["Babel (>=2.7)"] name = "jinxed" version = "1.2.0" description = "Jinxed Terminal Library" -category = "main" optional = false python-versions = "*" files = [ @@ -1445,7 +1379,6 @@ ansicon = {version = "*", markers = "platform_system == \"Windows\""} name = "jsonschema" version = "2.6.0" description = "An implementation of JSON Schema validation for Python" -category = "main" optional = false python-versions = "*" files = [ @@ -1460,7 +1393,6 @@ format = ["rfc3987", "strict-rfc3339", "webcolors"] name = "keyring" version = "22.4.0" description = "Store and access your passwords safely." -category = "main" optional = false python-versions = ">=3.6" files = [ @@ -1482,7 +1414,6 @@ testing = ["pytest (>=4.6)", "pytest-black (>=0.3.7)", "pytest-checkdocs (>=1.2. name = "lazy-object-proxy" version = "1.9.0" description = "A fast and thorough lazy object proxy." -category = "dev" optional = false python-versions = ">=3.7" files = [ @@ -1528,7 +1459,6 @@ files = [ name = "lief" version = "0.13.1" description = "Library to instrument executable formats" -category = "dev" optional = false python-versions = ">=3.8" files = [ @@ -1561,7 +1491,6 @@ files = [ name = "linkify-it-py" version = "2.0.2" description = "Links recognition library with FULL unicode support." -category = "dev" optional = false python-versions = ">=3.7" files = [ @@ -1582,7 +1511,6 @@ test = ["coverage", "pytest", "pytest-cov"] name = "log4mongo" version = "1.7.0" description = "mongo database handler for python logging" -category = "main" optional = false python-versions = "*" files = [ @@ -1596,7 +1524,6 @@ pymongo = "*" name = "m2r2" version = "0.3.3.post2" description = "Markdown and reStructuredText in a single file." -category = "dev" optional = false python-versions = ">=3.7" files = [ @@ -1612,7 +1539,6 @@ mistune = "0.8.4" name = "markdown-it-py" version = "2.2.0" description = "Python port of markdown-it. Markdown parsing, done right!" -category = "dev" optional = false python-versions = ">=3.7" files = [ @@ -1637,7 +1563,6 @@ testing = ["coverage", "pytest", "pytest-cov", "pytest-regressions"] name = "markupsafe" version = "2.0.1" description = "Safely add untrusted strings to HTML/XML markup." -category = "dev" optional = false python-versions = ">=3.6" files = [ @@ -1716,7 +1641,6 @@ files = [ name = "mccabe" version = "0.7.0" description = "McCabe checker, plugin for flake8" -category = "dev" optional = false python-versions = ">=3.6" files = [ @@ -1728,7 +1652,6 @@ files = [ name = "mdit-py-plugins" version = "0.3.5" description = "Collection of plugins for markdown-it-py" -category = "dev" optional = false python-versions = ">=3.7" files = [ @@ -1748,7 +1671,6 @@ testing = ["coverage", "pytest", "pytest-cov", "pytest-regressions"] name = "mdurl" version = "0.1.2" description = "Markdown URL utilities" -category = "dev" optional = false python-versions = ">=3.7" files = [ @@ -1760,7 +1682,6 @@ files = [ name = "mistune" version = "0.8.4" description = "The fastest markdown parser in pure Python" -category = "dev" optional = false python-versions = "*" files = [ @@ -1772,7 +1693,6 @@ files = [ name = "multidict" version = "6.0.4" description = "multidict implementation" -category = "main" optional = false python-versions = ">=3.7" files = [ @@ -1856,7 +1776,6 @@ files = [ name = "myst-parser" version = "0.18.1" description = "An extended commonmark compliant parser, with bridges to docutils & sphinx." -category = "dev" optional = false python-versions = ">=3.7" files = [ @@ -1883,7 +1802,6 @@ testing = ["beautifulsoup4", "coverage[toml]", "pytest (>=6,<7)", "pytest-cov", name = "nodeenv" version = "1.8.0" description = "Node.js virtual environment builder" -category = "dev" optional = false python-versions = ">=2.7,!=3.0.*,!=3.1.*,!=3.2.*,!=3.3.*,!=3.4.*,!=3.5.*,!=3.6.*" files = [ @@ -1898,7 +1816,6 @@ setuptools = "*" name = "packaging" version = "23.1" description = "Core utilities for Python packages" -category = "main" optional = false python-versions = ">=3.7" files = [ @@ -1910,7 +1827,6 @@ files = [ name = "paramiko" version = "3.2.0" description = "SSH2 protocol library" -category = "main" optional = false python-versions = ">=3.6" files = [ @@ -1932,7 +1848,6 @@ invoke = ["invoke (>=2.0)"] name = "parso" version = "0.8.3" description = "A Python Parser" -category = "dev" optional = false python-versions = ">=3.6" files = [ @@ -1948,7 +1863,6 @@ testing = ["docopt", "pytest (<6.0.0)"] name = "patchelf" version = "0.17.2.1" description = "A small utility to modify the dynamic linker and RPATH of ELF executables." -category = "dev" optional = false python-versions = "*" files = [ @@ -1968,7 +1882,6 @@ test = ["importlib-metadata", "pytest"] name = "pathlib2" version = "2.3.7.post1" description = "Object-oriented filesystem paths" -category = "main" optional = false python-versions = "*" files = [ @@ -1983,7 +1896,6 @@ six = "*" name = "pillow" version = "9.5.0" description = "Python Imaging Library (Fork)" -category = "main" optional = false python-versions = ">=3.7" files = [ @@ -2063,7 +1975,6 @@ tests = ["check-manifest", "coverage", "defusedxml", "markdown2", "olefile", "pa name = "platformdirs" version = "3.5.1" description = "A small Python package for determining appropriate platform-specific dirs, e.g. a \"user data dir\"." -category = "dev" optional = false python-versions = ">=3.7" files = [ @@ -2079,7 +1990,6 @@ test = ["appdirs (==1.4.4)", "covdefaults (>=2.3)", "pytest (>=7.3.1)", "pytest- name = "pluggy" version = "1.0.0" description = "plugin and hook calling mechanisms for python" -category = "dev" optional = false python-versions = ">=3.6" files = [ @@ -2095,7 +2005,6 @@ testing = ["pytest", "pytest-benchmark"] name = "ply" version = "3.11" description = "Python Lex & Yacc" -category = "main" optional = false python-versions = "*" files = [ @@ -2107,7 +2016,6 @@ files = [ name = "pockets" version = "0.9.1" description = "A collection of helpful Python tools!" -category = "dev" optional = false python-versions = "*" files = [ @@ -2122,7 +2030,6 @@ six = ">=1.5.2" name = "pre-commit" version = "3.3.2" description = "A framework for managing and maintaining multi-language pre-commit hooks." -category = "dev" optional = false python-versions = ">=3.8" files = [ @@ -2141,7 +2048,6 @@ virtualenv = ">=20.10.0" name = "prefixed" version = "0.7.0" description = "Prefixed alternative numeric library" -category = "main" optional = false python-versions = "*" files = [ @@ -2153,7 +2059,6 @@ files = [ name = "protobuf" version = "4.23.2" description = "" -category = "main" optional = false python-versions = ">=3.7" files = [ @@ -2176,7 +2081,6 @@ files = [ name = "py" version = "1.11.0" description = "library with cross-python path, ini-parsing, io, code, log facilities" -category = "dev" optional = false python-versions = ">=2.7, !=3.0.*, !=3.1.*, !=3.2.*, !=3.3.*, !=3.4.*" files = [ @@ -2188,7 +2092,6 @@ files = [ name = "pyasn1" version = "0.5.0" description = "Pure-Python implementation of ASN.1 types and DER/BER/CER codecs (X.208)" -category = "main" optional = false python-versions = "!=3.0.*,!=3.1.*,!=3.2.*,!=3.3.*,!=3.4.*,!=3.5.*,>=2.7" files = [ @@ -2200,7 +2103,6 @@ files = [ name = "pyasn1-modules" version = "0.3.0" description = "A collection of ASN.1-based protocols modules" -category = "main" optional = false python-versions = "!=3.0.*,!=3.1.*,!=3.2.*,!=3.3.*,!=3.4.*,!=3.5.*,>=2.7" files = [ @@ -2215,7 +2117,6 @@ pyasn1 = ">=0.4.6,<0.6.0" name = "pyblish-base" version = "1.8.11" description = "Plug-in driven automation framework for content" -category = "main" optional = false python-versions = "*" files = [ @@ -2227,7 +2128,6 @@ files = [ name = "pycodestyle" version = "2.10.0" description = "Python style guide checker" -category = "dev" optional = false python-versions = ">=3.6" files = [ @@ -2239,7 +2139,6 @@ files = [ name = "pycparser" version = "2.21" description = "C parser in Python" -category = "main" optional = false python-versions = ">=2.7, !=3.0.*, !=3.1.*, !=3.2.*, !=3.3.*" files = [ @@ -2251,7 +2150,6 @@ files = [ name = "pydocstyle" version = "6.3.0" description = "Python docstring style checker" -category = "dev" optional = false python-versions = ">=3.6" files = [ @@ -2269,7 +2167,6 @@ toml = ["tomli (>=1.2.3)"] name = "pyflakes" version = "3.0.1" description = "passive checker of Python programs" -category = "dev" optional = false python-versions = ">=3.6" files = [ @@ -2281,7 +2178,6 @@ files = [ name = "pygments" version = "2.15.1" description = "Pygments is a syntax highlighting package written in Python." -category = "dev" optional = false python-versions = ">=3.7" files = [ @@ -2296,7 +2192,6 @@ plugins = ["importlib-metadata"] name = "pylint" version = "2.17.4" description = "python code static checker" -category = "dev" optional = false python-versions = ">=3.7.2" files = [ @@ -2323,7 +2218,6 @@ testutils = ["gitpython (>3)"] name = "pymongo" version = "3.13.0" description = "Python driver for MongoDB " -category = "main" optional = false python-versions = "*" files = [ @@ -2452,7 +2346,6 @@ zstd = ["zstandard"] name = "pynacl" version = "1.5.0" description = "Python binding to the Networking and Cryptography (NaCl) library" -category = "main" optional = false python-versions = ">=3.6" files = [ @@ -2479,7 +2372,6 @@ tests = ["hypothesis (>=3.27.0)", "pytest (>=3.2.1,!=3.3.0)"] name = "pynput" version = "1.7.6" description = "Monitor and control user input devices" -category = "main" optional = false python-versions = "*" files = [ @@ -2498,7 +2390,6 @@ six = "*" name = "pyobjc-core" version = "9.1.1" description = "Python<->ObjC Interoperability Module" -category = "main" optional = false python-versions = ">=3.7" files = [ @@ -2515,7 +2406,6 @@ files = [ name = "pyobjc-framework-applicationservices" version = "9.1.1" description = "Wrappers for the framework ApplicationServices on macOS" -category = "main" optional = false python-versions = ">=3.7" files = [ @@ -2537,7 +2427,6 @@ pyobjc-framework-Quartz = ">=9.1.1" name = "pyobjc-framework-cocoa" version = "9.1.1" description = "Wrappers for the Cocoa frameworks on macOS" -category = "main" optional = false python-versions = ">=3.7" files = [ @@ -2557,7 +2446,6 @@ pyobjc-core = ">=9.1.1" name = "pyobjc-framework-quartz" version = "9.1.1" description = "Wrappers for the Quartz frameworks on macOS" -category = "main" optional = false python-versions = ">=3.7" files = [ @@ -2578,7 +2466,6 @@ pyobjc-framework-Cocoa = ">=9.1.1" name = "pyparsing" version = "2.4.7" description = "Python parsing module" -category = "main" optional = false python-versions = ">=2.6, !=3.0.*, !=3.1.*, !=3.2.*" files = [ @@ -2590,7 +2477,6 @@ files = [ name = "pysftp" version = "0.2.9" description = "A friendly face on SFTP" -category = "main" optional = false python-versions = "*" files = [ @@ -2604,7 +2490,6 @@ paramiko = ">=1.17" name = "pytest" version = "6.2.5" description = "pytest: simple powerful testing with Python" -category = "dev" optional = false python-versions = ">=3.6" files = [ @@ -2629,7 +2514,6 @@ testing = ["argcomplete", "hypothesis (>=3.56)", "mock", "nose", "requests", "xm name = "pytest-cov" version = "4.1.0" description = "Pytest plugin for measuring coverage." -category = "dev" optional = false python-versions = ">=3.7" files = [ @@ -2648,7 +2532,6 @@ testing = ["fields", "hunter", "process-tests", "pytest-xdist", "six", "virtuale name = "pytest-print" version = "0.3.1" description = "pytest-print adds the printer fixture you can use to print messages to the user (directly to the pytest runner, not stdout)" -category = "dev" optional = false python-versions = ">=3.7" files = [ @@ -2666,7 +2549,6 @@ test = ["coverage (>=5)"] name = "python-dateutil" version = "2.8.2" description = "Extensions to the standard Python datetime module" -category = "main" optional = false python-versions = "!=3.0.*,!=3.1.*,!=3.2.*,>=2.7" files = [ @@ -2681,7 +2563,6 @@ six = ">=1.5" name = "python-engineio" version = "4.4.1" description = "Engine.IO server and client for Python" -category = "main" optional = false python-versions = ">=3.6" files = [ @@ -2697,7 +2578,6 @@ client = ["requests (>=2.21.0)", "websocket-client (>=0.54.0)"] name = "python-socketio" version = "5.8.0" description = "Socket.IO server and client for Python" -category = "main" optional = false python-versions = ">=3.6" files = [ @@ -2719,7 +2599,6 @@ client = ["requests (>=2.21.0)", "websocket-client (>=0.54.0)"] name = "python-xlib" version = "0.33" description = "Python X Library" -category = "main" optional = false python-versions = "*" files = [ @@ -2734,7 +2613,6 @@ six = ">=1.10.0" name = "python3-xlib" version = "0.15" description = "Python3 X Library" -category = "main" optional = false python-versions = "*" files = [ @@ -2745,7 +2623,6 @@ files = [ name = "pywin32" version = "301" description = "Python for Window Extensions" -category = "main" optional = false python-versions = "*" files = [ @@ -2765,7 +2642,6 @@ files = [ name = "pywin32-ctypes" version = "0.2.0" description = "" -category = "main" optional = false python-versions = "*" files = [ @@ -2777,7 +2653,6 @@ files = [ name = "pyyaml" version = "6.0" description = "YAML parser and emitter for Python" -category = "dev" optional = false python-versions = ">=3.6" files = [ @@ -2827,7 +2702,6 @@ files = [ name = "qt-py" version = "1.3.8" description = "Python 2 & 3 compatibility wrapper around all Qt bindings - PySide, PySide2, PyQt4 and PyQt5." -category = "main" optional = false python-versions = "*" files = [ @@ -2842,7 +2716,6 @@ types-PySide2 = "*" name = "qtawesome" version = "0.7.3" description = "FontAwesome icons in PyQt and PySide applications" -category = "main" optional = false python-versions = "*" files = [ @@ -2858,7 +2731,6 @@ six = "*" name = "qtpy" version = "2.3.1" description = "Provides an abstraction layer on top of the various Qt bindings (PyQt5/6 and PySide2/6)." -category = "main" optional = false python-versions = ">=3.7" files = [ @@ -2876,7 +2748,6 @@ test = ["pytest (>=6,!=7.0.0,!=7.0.1)", "pytest-cov (>=3.0.0)", "pytest-qt"] name = "recommonmark" version = "0.7.1" description = "A docutils-compatibility bridge to CommonMark, enabling you to write CommonMark inside of Docutils & Sphinx projects." -category = "dev" optional = false python-versions = "*" files = [ @@ -2893,7 +2764,6 @@ sphinx = ">=1.3.1" name = "requests" version = "2.31.0" description = "Python HTTP for Humans." -category = "main" optional = false python-versions = ">=3.7" files = [ @@ -2915,7 +2785,6 @@ use-chardet-on-py3 = ["chardet (>=3.0.2,<6)"] name = "revitron-sphinx-theme" version = "0.7.2" description = "Revitron theme for Sphinx" -category = "dev" optional = false python-versions = "*" files = [] @@ -2937,7 +2806,6 @@ resolved_reference = "c0779c66365d9d258d93575ebaff7db9d3aee282" name = "rsa" version = "4.9" description = "Pure-Python RSA implementation" -category = "main" optional = false python-versions = ">=3.6,<4" files = [ @@ -2952,7 +2820,6 @@ pyasn1 = ">=0.1.3" name = "secretstorage" version = "3.3.3" description = "Python bindings to FreeDesktop.org Secret Service API" -category = "main" optional = false python-versions = ">=3.6" files = [ @@ -2968,7 +2835,6 @@ jeepney = ">=0.6" name = "semver" version = "2.13.0" description = "Python helper for Semantic Versioning (http://semver.org/)" -category = "main" optional = false python-versions = ">=2.7, !=3.0.*, !=3.1.*, !=3.2.*, !=3.3.*" files = [ @@ -2980,7 +2846,6 @@ files = [ name = "setuptools" version = "65.7.0" description = "Easily download, build, install, upgrade, and uninstall Python packages" -category = "dev" optional = false python-versions = ">=3.7" files = [ @@ -2997,7 +2862,6 @@ testing-integration = ["build[virtualenv]", "filelock (>=3.4.0)", "jaraco.envs ( name = "shotgun-api3" version = "3.3.3" description = "Shotgun Python API" -category = "main" optional = false python-versions = "*" files = [] @@ -3013,7 +2877,6 @@ resolved_reference = "b9f066c0edbea6e0733242e18f32f75489064840" name = "six" version = "1.16.0" description = "Python 2 and 3 compatibility utilities" -category = "main" optional = false python-versions = ">=2.7, !=3.0.*, !=3.1.*, !=3.2.*" files = [ @@ -3025,7 +2888,6 @@ files = [ name = "slack-sdk" version = "3.21.3" description = "The Slack API Platform SDK for Python" -category = "main" optional = false python-versions = ">=3.6.0" files = [ @@ -3041,7 +2903,6 @@ testing = ["Flask (>=1,<2)", "Flask-Sockets (>=0.2,<1)", "Jinja2 (==3.0.3)", "We name = "smmap" version = "5.0.0" description = "A pure Python implementation of a sliding window memory map manager" -category = "dev" optional = false python-versions = ">=3.6" files = [ @@ -3053,7 +2914,6 @@ files = [ name = "snowballstemmer" version = "2.2.0" description = "This package provides 29 stemmers for 28 languages generated from Snowball algorithms." -category = "dev" optional = false python-versions = "*" files = [ @@ -3065,7 +2925,6 @@ files = [ name = "speedcopy" version = "2.1.4" description = "Replacement or alternative for python copyfile() utilizing server side copy on network shares for faster copying." -category = "main" optional = false python-versions = "*" files = [ @@ -3077,7 +2936,6 @@ files = [ name = "sphinx" version = "5.3.0" description = "Python documentation generator" -category = "dev" optional = false python-versions = ">=3.6" files = [ @@ -3113,7 +2971,6 @@ test = ["cython", "html5lib", "pytest (>=4.6)", "typed_ast"] name = "sphinx-autoapi" version = "2.1.0" description = "Sphinx API documentation generator" -category = "dev" optional = false python-versions = ">=3.7" files = [ @@ -3137,7 +2994,6 @@ go = ["sphinxcontrib-golangdomain"] name = "sphinxcontrib-applehelp" version = "1.0.4" description = "sphinxcontrib-applehelp is a Sphinx extension which outputs Apple help books" -category = "dev" optional = false python-versions = ">=3.8" files = [ @@ -3153,7 +3009,6 @@ test = ["pytest"] name = "sphinxcontrib-devhelp" version = "1.0.2" description = "sphinxcontrib-devhelp is a sphinx extension which outputs Devhelp document." -category = "dev" optional = false python-versions = ">=3.5" files = [ @@ -3169,7 +3024,6 @@ test = ["pytest"] name = "sphinxcontrib-htmlhelp" version = "2.0.1" description = "sphinxcontrib-htmlhelp is a sphinx extension which renders HTML help files" -category = "dev" optional = false python-versions = ">=3.8" files = [ @@ -3185,7 +3039,6 @@ test = ["html5lib", "pytest"] name = "sphinxcontrib-jsmath" version = "1.0.1" description = "A sphinx extension which renders display math in HTML via JavaScript" -category = "dev" optional = false python-versions = ">=3.5" files = [ @@ -3200,7 +3053,6 @@ test = ["flake8", "mypy", "pytest"] name = "sphinxcontrib-napoleon" version = "0.7" description = "Sphinx \"napoleon\" extension." -category = "dev" optional = false python-versions = "*" files = [ @@ -3216,7 +3068,6 @@ six = ">=1.5.2" name = "sphinxcontrib-qthelp" version = "1.0.3" description = "sphinxcontrib-qthelp is a sphinx extension which outputs QtHelp document." -category = "dev" optional = false python-versions = ">=3.5" files = [ @@ -3232,7 +3083,6 @@ test = ["pytest"] name = "sphinxcontrib-serializinghtml" version = "1.1.5" description = "sphinxcontrib-serializinghtml is a sphinx extension which outputs \"serialized\" HTML files (json and pickle)." -category = "dev" optional = false python-versions = ">=3.5" files = [ @@ -3248,7 +3098,6 @@ test = ["pytest"] name = "stone" version = "3.3.1" description = "Stone is an interface description language (IDL) for APIs." -category = "main" optional = false python-versions = "*" files = [ @@ -3265,7 +3114,6 @@ six = ">=1.12.0" name = "termcolor" version = "1.1.0" description = "ANSII Color formatting for output in terminal." -category = "main" optional = false python-versions = "*" files = [ @@ -3276,7 +3124,6 @@ files = [ name = "toml" version = "0.10.2" description = "Python Library for Tom's Obvious, Minimal Language" -category = "dev" optional = false python-versions = ">=2.6, !=3.0.*, !=3.1.*, !=3.2.*" files = [ @@ -3288,7 +3135,6 @@ files = [ name = "tomli" version = "2.0.1" description = "A lil' TOML parser" -category = "dev" optional = false python-versions = ">=3.7" files = [ @@ -3300,7 +3146,6 @@ files = [ name = "tomlkit" version = "0.11.8" description = "Style preserving TOML library" -category = "dev" optional = false python-versions = ">=3.7" files = [ @@ -3312,7 +3157,6 @@ files = [ name = "types-pyside2" version = "5.15.2.1.5" description = "The most accurate stubs for PySide2" -category = "main" optional = false python-versions = "*" files = [ @@ -3323,7 +3167,6 @@ files = [ name = "typing-extensions" version = "4.6.2" description = "Backported and Experimental Type Hints for Python 3.7+" -category = "dev" optional = false python-versions = ">=3.7" files = [ @@ -3335,7 +3178,6 @@ files = [ name = "uc-micro-py" version = "1.0.2" description = "Micro subset of unicode data files for linkify-it-py projects." -category = "dev" optional = false python-versions = ">=3.7" files = [ @@ -3350,7 +3192,6 @@ test = ["coverage", "pytest", "pytest-cov"] name = "unidecode" version = "1.2.0" description = "ASCII transliterations of Unicode text" -category = "main" optional = false python-versions = ">=2.7, !=3.0.*, !=3.1.*, !=3.2.*, !=3.3.*" files = [ @@ -3362,7 +3203,6 @@ files = [ name = "uritemplate" version = "3.0.1" description = "URI templates" -category = "main" optional = false python-versions = ">=2.7, !=3.0.*, !=3.1.*, !=3.2.*, !=3.3.*" files = [ @@ -3372,27 +3212,24 @@ files = [ [[package]] name = "urllib3" -version = "2.0.2" +version = "1.26.16" description = "HTTP library with thread-safe connection pooling, file post, and more." -category = "main" optional = false -python-versions = ">=3.7" +python-versions = ">=2.7, !=3.0.*, !=3.1.*, !=3.2.*, !=3.3.*, !=3.4.*, !=3.5.*" files = [ - {file = "urllib3-2.0.2-py3-none-any.whl", hash = "sha256:d055c2f9d38dc53c808f6fdc8eab7360b6fdbbde02340ed25cfbcd817c62469e"}, - {file = "urllib3-2.0.2.tar.gz", hash = "sha256:61717a1095d7e155cdb737ac7bb2f4324a858a1e2e6466f6d03ff630ca68d3cc"}, + {file = "urllib3-1.26.16-py2.py3-none-any.whl", hash = "sha256:8d36afa7616d8ab714608411b4a3b13e58f463aee519024578e062e141dce20f"}, + {file = "urllib3-1.26.16.tar.gz", hash = "sha256:8f135f6502756bde6b2a9b28989df5fbe87c9970cecaa69041edcce7f0589b14"}, ] [package.extras] -brotli = ["brotli (>=1.0.9)", "brotlicffi (>=0.8.0)"] -secure = ["certifi", "cryptography (>=1.9)", "idna (>=2.0.0)", "pyopenssl (>=17.1.0)", "urllib3-secure-extra"] -socks = ["pysocks (>=1.5.6,!=1.5.7,<2.0)"] -zstd = ["zstandard (>=0.18.0)"] +brotli = ["brotli (>=1.0.9)", "brotlicffi (>=0.8.0)", "brotlipy (>=0.6.0)"] +secure = ["certifi", "cryptography (>=1.3.4)", "idna (>=2.0.0)", "ipaddress", "pyOpenSSL (>=0.14)", "urllib3-secure-extra"] +socks = ["PySocks (>=1.5.6,!=1.5.7,<2.0)"] [[package]] name = "virtualenv" version = "20.23.0" description = "Virtual Python Environment builder" -category = "dev" optional = false python-versions = ">=3.7" files = [ @@ -3413,7 +3250,6 @@ test = ["covdefaults (>=2.3)", "coverage (>=7.2.3)", "coverage-enable-subprocess name = "wcwidth" version = "0.2.6" description = "Measures the displayed width of unicode strings in a terminal" -category = "main" optional = false python-versions = "*" files = [ @@ -3425,7 +3261,6 @@ files = [ name = "websocket-client" version = "0.59.0" description = "WebSocket client for Python with low level API options" -category = "main" optional = false python-versions = ">=2.6, !=3.0.*, !=3.1.*, !=3.2.*, !=3.3.*" files = [ @@ -3440,7 +3275,6 @@ six = "*" name = "wheel" version = "0.40.0" description = "A built-package format for Python" -category = "dev" optional = false python-versions = ">=3.7" files = [ @@ -3455,7 +3289,6 @@ test = ["pytest (>=6.0.0)"] name = "wrapt" version = "1.15.0" description = "Module for decorators, wrappers and monkey patching." -category = "dev" optional = false python-versions = "!=3.0.*,!=3.1.*,!=3.2.*,!=3.3.*,!=3.4.*,>=2.7" files = [ @@ -3540,7 +3373,6 @@ files = [ name = "wsrpc-aiohttp" version = "3.2.0" description = "WSRPC is the RPC over WebSocket for aiohttp" -category = "main" optional = false python-versions = ">3.5.*, <4" files = [ @@ -3561,7 +3393,6 @@ ujson = ["ujson"] name = "yarl" version = "1.9.2" description = "Yet another URL library" -category = "main" optional = false python-versions = ">=3.7" files = [ @@ -3649,7 +3480,6 @@ multidict = ">=4.0" name = "zipp" version = "3.15.0" description = "Backport of pathlib-compatible object wrapper for zip files" -category = "main" optional = false python-versions = ">=3.7" files = [ @@ -3666,5 +3496,5 @@ docs = [] [metadata] lock-version = "2.0" -python-versions = ">=3.9.1,<3.10" -content-hash = "d2b8da22dcd11e0b03f19b9b79e51f205156c5ce75e41cc0225392e9afd8803b" +python-versions = "~3.9" +content-hash = "bc3e256094db6e33894840bb6a5adda4473d3736b852433ad8d5bd478c7e0c1c" diff --git a/pyproject.toml b/pyproject.toml index a07c547123..d0b1ecf589 100644 --- a/pyproject.toml +++ b/pyproject.toml @@ -1,6 +1,6 @@ [tool.poetry] name = "OpenPype" -version = "3.16.4" # OpenPype +version = "3.17.0" # OpenPype description = "Open VFX and Animation pipeline with support." authors = ["OpenPype Team "] license = "MIT License" @@ -36,7 +36,7 @@ appdirs = { git = "https://github.com/ActiveState/appdirs.git", branch = "master blessed = "^1.17" # openpype terminal formatting coolname = "*" clique = "1.6.*" -Click = "^8" +Click = "7.1.2" dnspython = "^2.1.0" ftrack-python-api = "^2.3.3" arrow = "^0.17" @@ -56,6 +56,7 @@ QtPy = "^2.3.0" qtawesome = "0.7.3" speedcopy = "^2.1" six = "^1.15" +urllib3 = "1.26.16" semver = "^2.13.0" # for version resolution wsrpc_aiohttp = "^3.1.1" # websocket server pywin32 = { version = "301", markers = "sys_platform == 'win32'" } diff --git a/schema/application-1.0.json b/schema/application-1.0.json deleted file mode 100644 index 953abee569..0000000000 --- a/schema/application-1.0.json +++ /dev/null @@ -1,68 +0,0 @@ -{ - "$schema": "http://json-schema.org/draft-04/schema#", - - "title": "openpype:application-1.0", - "description": "An application definition.", - - "type": "object", - - "additionalProperties": true, - - "required": [ - "schema", - "label", - "application_dir", - "executable" - ], - - "properties": { - "schema": { - "description": "Schema identifier for payload", - "type": "string" - }, - "label": { - "description": "Nice name of application.", - "type": "string" - }, - "application_dir": { - "description": "Name of directory used for application resources.", - "type": "string" - }, - "executable": { - "description": "Name of callable executable, this is called to launch the application", - "type": "string" - }, - "description": { - "description": "Description of application.", - "type": "string" - }, - "environment": { - "description": "Key/value pairs for environment variables related to this application. Supports lists for paths, such as PYTHONPATH.", - "type": "object", - "items": { - "oneOf": [ - {"type": "string"}, - {"type": "array", "items": {"type": "string"}} - ] - } - }, - "default_dirs": { - "type": "array", - "items": { - "type": "string" - } - }, - "copy": { - "type": "object", - "patternProperties": { - "^.*$": { - "anyOf": [ - {"type": "string"}, - {"type": "null"} - ] - } - }, - "additionalProperties": false - } - } -} diff --git a/schema/asset-1.0.json b/schema/asset-1.0.json deleted file mode 100644 index ab104c002a..0000000000 --- a/schema/asset-1.0.json +++ /dev/null @@ -1,35 +0,0 @@ -{ - "$schema": "http://json-schema.org/draft-04/schema#", - - "title": "openpype:asset-1.0", - "description": "A unit of data", - - "type": "object", - - "additionalProperties": true, - - "required": [ - "schema", - "name", - "subsets" - ], - - "properties": { - "schema": { - "description": "Schema identifier for payload", - "type": "string" - }, - "name": { - "description": "Name of directory", - "type": "string" - }, - "subsets": { - "type": "array", - "items": { - "$ref": "subset.json" - } - } - }, - - "definitions": {} -} diff --git a/schema/asset-2.0.json b/schema/asset-2.0.json deleted file mode 100644 index b894d79792..0000000000 --- a/schema/asset-2.0.json +++ /dev/null @@ -1,55 +0,0 @@ -{ - "$schema": "http://json-schema.org/draft-04/schema#", - - "title": "openpype:asset-2.0", - "description": "A unit of data", - - "type": "object", - - "additionalProperties": true, - - "required": [ - "schema", - "type", - "name", - "silo", - "data" - ], - - "properties": { - "schema": { - "description": "Schema identifier for payload", - "type": "string", - "enum": ["openpype:asset-2.0"], - "example": "openpype:asset-2.0" - }, - "type": { - "description": "The type of document", - "type": "string", - "enum": ["asset"], - "example": "asset" - }, - "parent": { - "description": "Unique identifier to parent document", - "example": "592c33475f8c1b064c4d1696" - }, - "name": { - "description": "Name of asset", - "type": "string", - "pattern": "^[a-zA-Z0-9_.]*$", - "example": "Bruce" - }, - "silo": { - "description": "Group or container of asset", - "type": "string", - "example": "assets" - }, - "data": { - "description": "Document metadata", - "type": "object", - "example": {"key": "value"} - } - }, - - "definitions": {} -} diff --git a/schema/asset-3.0.json b/schema/asset-3.0.json deleted file mode 100644 index 948704d2a1..0000000000 --- a/schema/asset-3.0.json +++ /dev/null @@ -1,55 +0,0 @@ -{ - "$schema": "http://json-schema.org/draft-04/schema#", - - "title": "openpype:asset-3.0", - "description": "A unit of data", - - "type": "object", - - "additionalProperties": true, - - "required": [ - "schema", - "type", - "name", - "data" - ], - - "properties": { - "schema": { - "description": "Schema identifier for payload", - "type": "string", - "enum": ["openpype:asset-3.0"], - "example": "openpype:asset-3.0" - }, - "type": { - "description": "The type of document", - "type": "string", - "enum": ["asset"], - "example": "asset" - }, - "parent": { - "description": "Unique identifier to parent document", - "example": "592c33475f8c1b064c4d1696" - }, - "name": { - "description": "Name of asset", - "type": "string", - "pattern": "^[a-zA-Z0-9_.]*$", - "example": "Bruce" - }, - "silo": { - "description": "Group or container of asset", - "type": "string", - "pattern": "^[a-zA-Z0-9_.]*$", - "example": "assets" - }, - "data": { - "description": "Document metadata", - "type": "object", - "example": {"key": "value"} - } - }, - - "definitions": {} -} diff --git a/schema/config-1.0.json b/schema/config-1.0.json deleted file mode 100644 index 49398a57cd..0000000000 --- a/schema/config-1.0.json +++ /dev/null @@ -1,85 +0,0 @@ -{ - "$schema": "http://json-schema.org/draft-04/schema#", - - "title": "openpype:config-1.0", - "description": "A project configuration.", - - "type": "object", - - "additionalProperties": false, - "required": [ - "tasks", - "apps" - ], - - "properties": { - "schema": { - "description": "Schema identifier for payload", - "type": "string" - }, - "template": { - "type": "object", - "additionalProperties": false, - "patternProperties": { - "^.*$": { - "type": "string" - } - } - }, - "tasks": { - "type": "array", - "items": { - "type": "object", - "properties": { - "name": {"type": "string"}, - "icon": {"type": "string"}, - "group": {"type": "string"}, - "label": {"type": "string"} - }, - "required": ["name"] - } - }, - "apps": { - "type": "array", - "items": { - "type": "object", - "properties": { - "name": {"type": "string"}, - "icon": {"type": "string"}, - "group": {"type": "string"}, - "label": {"type": "string"} - }, - "required": ["name"] - } - }, - "families": { - "type": "array", - "items": { - "type": "object", - "properties": { - "name": {"type": "string"}, - "icon": {"type": "string"}, - "label": {"type": "string"}, - "hideFilter": {"type": "boolean"} - }, - "required": ["name"] - } - }, - "groups": { - "type": "array", - "items": { - "type": "object", - "properties": { - "name": {"type": "string"}, - "icon": {"type": "string"}, - "color": {"type": "string"}, - "order": {"type": ["integer", "number"]} - }, - "required": ["name"] - } - }, - "copy": { - "type": "object" - } - } -} diff --git a/schema/config-1.1.json b/schema/config-1.1.json deleted file mode 100644 index 6e15514aaf..0000000000 --- a/schema/config-1.1.json +++ /dev/null @@ -1,87 +0,0 @@ -{ - "$schema": "http://json-schema.org/draft-04/schema#", - - "title": "openpype:config-1.1", - "description": "A project configuration.", - - "type": "object", - - "additionalProperties": false, - "required": [ - "tasks", - "apps" - ], - - "properties": { - "schema": { - "description": "Schema identifier for payload", - "type": "string" - }, - "template": { - "type": "object", - "additionalProperties": false, - "patternProperties": { - "^.*$": { - "type": "string" - } - } - }, - "tasks": { - "type": "object", - "items": { - "type": "object", - "properties": { - "name": {"type": "string"}, - "icon": {"type": "string"}, - "group": {"type": "string"}, - "label": {"type": "string"} - }, - "required": [ - "short_name" - ] - } - }, - "apps": { - "type": "array", - "items": { - "type": "object", - "properties": { - "name": {"type": "string"}, - "icon": {"type": "string"}, - "group": {"type": "string"}, - "label": {"type": "string"} - }, - "required": ["name"] - } - }, - "families": { - "type": "array", - "items": { - "type": "object", - "properties": { - "name": {"type": "string"}, - "icon": {"type": "string"}, - "label": {"type": "string"}, - "hideFilter": {"type": "boolean"} - }, - "required": ["name"] - } - }, - "groups": { - "type": "array", - "items": { - "type": "object", - "properties": { - "name": {"type": "string"}, - "icon": {"type": "string"}, - "color": {"type": "string"}, - "order": {"type": ["integer", "number"]} - }, - "required": ["name"] - } - }, - "copy": { - "type": "object" - } - } -} diff --git a/schema/config-2.0.json b/schema/config-2.0.json deleted file mode 100644 index 54b226711a..0000000000 --- a/schema/config-2.0.json +++ /dev/null @@ -1,87 +0,0 @@ -{ - "$schema": "http://json-schema.org/draft-04/schema#", - - "title": "openpype:config-2.0", - "description": "A project configuration.", - - "type": "object", - - "additionalProperties": false, - "required": [ - "tasks", - "apps" - ], - - "properties": { - "schema": { - "description": "Schema identifier for payload", - "type": "string" - }, - "templates": { - "type": "object" - }, - "roots": { - "type": "object" - }, - "imageio": { - "type": "object" - }, - "tasks": { - "type": "object", - "items": { - "type": "object", - "properties": { - "name": {"type": "string"}, - "icon": {"type": "string"}, - "group": {"type": "string"}, - "label": {"type": "string"} - }, - "required": [ - "short_name" - ] - } - }, - "apps": { - "type": "array", - "items": { - "type": "object", - "properties": { - "name": {"type": "string"}, - "icon": {"type": "string"}, - "group": {"type": "string"}, - "label": {"type": "string"} - }, - "required": ["name"] - } - }, - "families": { - "type": "array", - "items": { - "type": "object", - "properties": { - "name": {"type": "string"}, - "icon": {"type": "string"}, - "label": {"type": "string"}, - "hideFilter": {"type": "boolean"} - }, - "required": ["name"] - } - }, - "groups": { - "type": "array", - "items": { - "type": "object", - "properties": { - "name": {"type": "string"}, - "icon": {"type": "string"}, - "color": {"type": "string"}, - "order": {"type": ["integer", "number"]} - }, - "required": ["name"] - } - }, - "copy": { - "type": "object" - } - } -} diff --git a/schema/container-1.0.json b/schema/container-1.0.json deleted file mode 100644 index 012e8499e6..0000000000 --- a/schema/container-1.0.json +++ /dev/null @@ -1,100 +0,0 @@ -{ - "$schema": "http://json-schema.org/draft-04/schema#", - - "title": "openpype:container-1.0", - "description": "A loaded asset", - - "type": "object", - - "additionalProperties": true, - - "required": [ - "id", - "objectName", - "name", - "author", - "loader", - "families", - "time", - "subset", - "asset", - "representation", - "version", - "silo", - "path", - "source" - ], - "properties": { - "id": { - "description": "Identifier for finding object in host", - "type": "string", - "enum": ["pyblish.mindbender.container"], - "example": "pyblish.mindbender.container" - }, - "objectName": { - "description": "Name of internal object, such as the objectSet in Maya.", - "type": "string", - "example": "Bruce_:rigDefault_CON" - }, - "name": { - "description": "Full name of application object", - "type": "string", - "example": "modelDefault" - }, - "author": { - "description": "Name of the author of the published version", - "type": "string", - "example": "Marcus Ottosson" - }, - "loader": { - "description": "Name of loader plug-in used to produce this container", - "type": "string", - "example": "ModelLoader" - }, - "families": { - "description": "Families associated with the this subset", - "type": "string", - "example": "mindbender.model" - }, - "time": { - "description": "File-system safe, formatted time", - "type": "string", - "example": "20170329T131545Z" - }, - "subset": { - "description": "Name of source subset", - "type": "string", - "example": "modelDefault" - }, - "asset": { - "description": "Name of source asset", - "type": "string" , - "example": "Bruce" - }, - "representation": { - "description": "Name of source representation", - "type": "string" , - "example": ".ma" - }, - "version": { - "description": "Version number", - "type": "number", - "example": 12 - }, - "silo": { - "description": "Silo of parent asset", - "type": "string", - "example": "assets" - }, - "path": { - "description": "Absolute path on disk", - "type": "string", - "example": "{root}/assets/Bruce/publish/rigDefault/v002" - }, - "source": { - "description": "Absolute path to file from which this version was published", - "type": "string", - "example": "{root}/assets/Bruce/work/rigging/maya/scenes/rig_v001.ma" - } - } -} diff --git a/schema/container-2.0.json b/schema/container-2.0.json deleted file mode 100644 index 1673ee5d1d..0000000000 --- a/schema/container-2.0.json +++ /dev/null @@ -1,59 +0,0 @@ -{ - "$schema": "http://json-schema.org/draft-04/schema#", - - "title": "openpype:container-2.0", - "description": "A loaded asset", - - "type": "object", - - "additionalProperties": true, - - "required": [ - "schema", - "id", - "objectName", - "name", - "namespace", - "loader", - "representation" - ], - "properties": { - "schema": { - "description": "Schema identifier for payload", - "type": "string", - "enum": ["openpype:container-2.0"], - "example": "openpype:container-2.0" - }, - "id": { - "description": "Identifier for finding object in host", - "type": "string", - "enum": ["pyblish.avalon.container"], - "example": "pyblish.avalon.container" - }, - "objectName": { - "description": "Name of internal object, such as the objectSet in Maya.", - "type": "string", - "example": "Bruce_:rigDefault_CON" - }, - "loader": { - "description": "Name of loader plug-in used to produce this container", - "type": "string", - "example": "ModelLoader" - }, - "name": { - "description": "Internal object name of container in application", - "type": "string", - "example": "modelDefault_01" - }, - "namespace": { - "description": "Internal namespace of container in application", - "type": "string", - "example": "Bruce_" - }, - "representation": { - "description": "Unique id of representation in database", - "type": "string", - "example": "59523f355f8c1b5f6c5e8348" - } - } -} diff --git a/schema/hero_version-1.0.json b/schema/hero_version-1.0.json deleted file mode 100644 index b720dc2887..0000000000 --- a/schema/hero_version-1.0.json +++ /dev/null @@ -1,44 +0,0 @@ -{ - "$schema": "http://json-schema.org/draft-04/schema#", - - "title": "openpype:hero_version-1.0", - "description": "Hero version of asset", - - "type": "object", - - "additionalProperties": true, - - "required": [ - "version_id", - "schema", - "type", - "parent" - ], - - "properties": { - "_id": { - "description": "Document's id (database will create it's if not entered)", - "example": "ObjectId(592c33475f8c1b064c4d1696)" - }, - "version_id": { - "description": "The version ID from which it was created", - "example": "ObjectId(592c33475f8c1b064c4d1695)" - }, - "schema": { - "description": "The schema associated with this document", - "type": "string", - "enum": ["openpype:hero_version-1.0"], - "example": "openpype:hero_version-1.0" - }, - "type": { - "description": "The type of document", - "type": "string", - "enum": ["hero_version"], - "example": "hero_version" - }, - "parent": { - "description": "Unique identifier to parent document", - "example": "ObjectId(592c33475f8c1b064c4d1697)" - } - } -} diff --git a/schema/inventory-1.0.json b/schema/inventory-1.0.json deleted file mode 100644 index 2fe78794ab..0000000000 --- a/schema/inventory-1.0.json +++ /dev/null @@ -1,10 +0,0 @@ -{ - "$schema": "http://json-schema.org/draft-04/schema#", - - "title": "openpype:config-1.0", - "description": "A project configuration.", - - "type": "object", - - "additionalProperties": true -} diff --git a/schema/inventory-1.1.json b/schema/inventory-1.1.json deleted file mode 100644 index b61a76b32a..0000000000 --- a/schema/inventory-1.1.json +++ /dev/null @@ -1,10 +0,0 @@ -{ - "$schema": "http://json-schema.org/draft-04/schema#", - - "title": "openpype:config-1.1", - "description": "A project configuration.", - - "type": "object", - - "additionalProperties": true -} diff --git a/schema/project-2.0.json b/schema/project-2.0.json deleted file mode 100644 index 0ed5a55599..0000000000 --- a/schema/project-2.0.json +++ /dev/null @@ -1,86 +0,0 @@ -{ - "$schema": "http://json-schema.org/draft-04/schema#", - - "title": "openpype:project-2.0", - "description": "A unit of data", - - "type": "object", - - "additionalProperties": true, - - "required": [ - "schema", - "type", - "name", - "data", - "config" - ], - - "properties": { - "schema": { - "description": "Schema identifier for payload", - "type": "string", - "enum": ["openpype:project-2.0"], - "example": "openpype:project-2.0" - }, - "type": { - "description": "The type of document", - "type": "string", - "enum": ["project"], - "example": "project" - }, - "parent": { - "description": "Unique identifier to parent document", - "example": "592c33475f8c1b064c4d1696" - }, - "name": { - "description": "Name of directory", - "type": "string", - "pattern": "^[a-zA-Z0-9_.]*$", - "example": "hulk" - }, - "data": { - "description": "Document metadata", - "type": "object", - "example": { - "fps": 24, - "width": 1920, - "height": 1080 - } - }, - "config": { - "type": "object", - "description": "Document metadata", - "example": { - "schema": "openpype:config-1.0", - "apps": [ - { - "name": "maya2016", - "label": "Autodesk Maya 2016" - }, - { - "name": "nuke10", - "label": "The Foundry Nuke 10.0" - } - ], - "tasks": [ - {"name": "model"}, - {"name": "render"}, - {"name": "animate"}, - {"name": "rig"}, - {"name": "lookdev"}, - {"name": "layout"} - ], - "template": { - "work": - "{root}/{project}/{silo}/{asset}/work/{task}/{app}", - "publish": - "{root}/{project}/{silo}/{asset}/publish/{subset}/v{version:0>3}/{subset}.{representation}" - } - }, - "$ref": "config-1.0.json" - } - }, - - "definitions": {} -} diff --git a/schema/project-2.1.json b/schema/project-2.1.json deleted file mode 100644 index 9413c9f691..0000000000 --- a/schema/project-2.1.json +++ /dev/null @@ -1,86 +0,0 @@ -{ - "$schema": "http://json-schema.org/draft-04/schema#", - - "title": "openpype:project-2.1", - "description": "A unit of data", - - "type": "object", - - "additionalProperties": true, - - "required": [ - "schema", - "type", - "name", - "data", - "config" - ], - - "properties": { - "schema": { - "description": "Schema identifier for payload", - "type": "string", - "enum": ["openpype:project-2.1"], - "example": "openpype:project-2.1" - }, - "type": { - "description": "The type of document", - "type": "string", - "enum": ["project"], - "example": "project" - }, - "parent": { - "description": "Unique identifier to parent document", - "example": "592c33475f8c1b064c4d1696" - }, - "name": { - "description": "Name of directory", - "type": "string", - "pattern": "^[a-zA-Z0-9_.]*$", - "example": "hulk" - }, - "data": { - "description": "Document metadata", - "type": "object", - "example": { - "fps": 24, - "width": 1920, - "height": 1080 - } - }, - "config": { - "type": "object", - "description": "Document metadata", - "example": { - "schema": "openpype:config-1.1", - "apps": [ - { - "name": "maya2016", - "label": "Autodesk Maya 2016" - }, - { - "name": "nuke10", - "label": "The Foundry Nuke 10.0" - } - ], - "tasks": { - "Model": {"short_name": "mdl"}, - "Render": {"short_name": "rnd"}, - "Animate": {"short_name": "anim"}, - "Rig": {"short_name": "rig"}, - "Lookdev": {"short_name": "look"}, - "Layout": {"short_name": "lay"} - }, - "template": { - "work": - "{root}/{project}/{silo}/{asset}/work/{task}/{app}", - "publish": - "{root}/{project}/{silo}/{asset}/publish/{subset}/v{version:0>3}/{subset}.{representation}" - } - }, - "$ref": "config-1.1.json" - } - }, - - "definitions": {} -} diff --git a/schema/project-3.0.json b/schema/project-3.0.json deleted file mode 100644 index be23e10c93..0000000000 --- a/schema/project-3.0.json +++ /dev/null @@ -1,59 +0,0 @@ -{ - "$schema": "http://json-schema.org/draft-04/schema#", - - "title": "openpype:project-3.0", - "description": "A unit of data", - - "type": "object", - - "additionalProperties": true, - - "required": [ - "schema", - "type", - "name", - "data", - "config" - ], - - "properties": { - "schema": { - "description": "Schema identifier for payload", - "type": "string", - "enum": ["openpype:project-3.0"], - "example": "openpype:project-3.0" - }, - "type": { - "description": "The type of document", - "type": "string", - "enum": ["project"], - "example": "project" - }, - "parent": { - "description": "Unique identifier to parent document", - "example": "592c33475f8c1b064c4d1696" - }, - "name": { - "description": "Name of directory", - "type": "string", - "pattern": "^[a-zA-Z0-9_.]*$", - "example": "hulk" - }, - "data": { - "description": "Document metadata", - "type": "object", - "example": { - "fps": 24, - "width": 1920, - "height": 1080 - } - }, - "config": { - "type": "object", - "description": "Document metadata", - "$ref": "config-2.0.json" - } - }, - - "definitions": {} -} diff --git a/schema/representation-1.0.json b/schema/representation-1.0.json deleted file mode 100644 index 347c585f52..0000000000 --- a/schema/representation-1.0.json +++ /dev/null @@ -1,28 +0,0 @@ -{ - "$schema": "http://json-schema.org/draft-04/schema#", - - "title": "openpype:representation-1.0", - "description": "The inverse of an instance", - - "type": "object", - - "additionalProperties": true, - - "required": [ - "schema", - "format", - "path" - ], - - "properties": { - "schema": {"type": "string"}, - "format": { - "description": "File extension, including '.'", - "type": "string" - }, - "path": { - "description": "Unformatted path to version.", - "type": "string" - } - } -} diff --git a/schema/representation-2.0.json b/schema/representation-2.0.json deleted file mode 100644 index f47c16a10a..0000000000 --- a/schema/representation-2.0.json +++ /dev/null @@ -1,78 +0,0 @@ -{ - "$schema": "http://json-schema.org/draft-04/schema#", - - "title": "openpype:representation-2.0", - "description": "The inverse of an instance", - - "type": "object", - - "additionalProperties": true, - - "required": [ - "schema", - "type", - "parent", - "name", - "data" - ], - - "properties": { - "schema": { - "description": "Schema identifier for payload", - "type": "string", - "enum": ["openpype:representation-2.0"], - "example": "openpype:representation-2.0" - }, - "type": { - "description": "The type of document", - "type": "string", - "enum": ["representation"], - "example": "representation" - }, - "parent": { - "description": "Unique identifier to parent document", - "example": "592c33475f8c1b064c4d1696" - }, - "name": { - "description": "Name of representation", - "type": "string", - "pattern": "^[a-zA-Z0-9_.]*$", - "example": "abc" - }, - "data": { - "description": "Document metadata", - "type": "object", - "example": { - "label": "Alembic" - } - }, - "dependencies": { - "description": "Other representation that this representation depends on", - "type": "array", - "items": {"type": "string"}, - "example": [ - "592d547a5f8c1b388093c145" - ] - }, - "context": { - "description": "Summary of the context to which this representation belong.", - "type": "object", - "properties": { - "project": {"type": "object"}, - "asset": {"type": "string"}, - "silo": {"type": ["string", "null"]}, - "subset": {"type": "string"}, - "version": {"type": "number"}, - "representation": {"type": "string"} - }, - "example": { - "project": "hulk", - "asset": "Bruce", - "silo": "assets", - "subset": "rigDefault", - "version": 12, - "representation": "ma" - } - } - } -} diff --git a/schema/session-1.0.json b/schema/session-1.0.json deleted file mode 100644 index 5ced0a6f08..0000000000 --- a/schema/session-1.0.json +++ /dev/null @@ -1,143 +0,0 @@ -{ - "$schema": "http://json-schema.org/draft-04/schema#", - - "title": "openpype:session-1.0", - "description": "The Avalon environment", - - "type": "object", - - "additionalProperties": true, - - "required": [ - "AVALON_PROJECTS", - "AVALON_PROJECT", - "AVALON_ASSET", - "AVALON_SILO", - "AVALON_CONFIG" - ], - - "properties": { - "AVALON_PROJECTS": { - "description": "Absolute path to root of project directories", - "type": "string", - "example": "/nas/projects" - }, - "AVALON_PROJECT": { - "description": "Name of project", - "type": "string", - "pattern": "^\\w*$", - "example": "Hulk" - }, - "AVALON_ASSET": { - "description": "Name of asset", - "type": "string", - "pattern": "^\\w*$", - "example": "Bruce" - }, - "AVALON_SILO": { - "description": "Name of asset group or container", - "type": "string", - "pattern": "^\\w*$", - "example": "assets" - }, - "AVALON_TASK": { - "description": "Name of task", - "type": "string", - "pattern": "^\\w*$", - "example": "modeling" - }, - "AVALON_CONFIG": { - "description": "Name of Avalon configuration", - "type": "string", - "pattern": "^\\w*$", - "example": "polly" - }, - "AVALON_APP": { - "description": "Name of application", - "type": "string", - "pattern": "^\\w*$", - "example": "maya2016" - }, - "AVALON_MONGO": { - "description": "Address to the asset database", - "type": "string", - "pattern": "^mongodb://[\\w/@:.]*$", - "example": "mongodb://localhost:27017", - "default": "mongodb://localhost:27017" - }, - "AVALON_DB": { - "description": "Name of database", - "type": "string", - "pattern": "^\\w*$", - "example": "avalon", - "default": "avalon" - }, - "AVALON_LABEL": { - "description": "Nice name of Avalon, used in e.g. graphical user interfaces", - "type": "string", - "example": "Mindbender", - "default": "Avalon" - }, - "AVALON_SENTRY": { - "description": "Address to Sentry", - "type": "string", - "pattern": "^http[\\w/@:.]*$", - "example": "https://5b872b280de742919b115bdc8da076a5:8d278266fe764361b8fa6024af004a9c@logs.mindbender.com/2", - "default": null - }, - "AVALON_DEADLINE": { - "description": "Address to Deadline", - "type": "string", - "pattern": "^http[\\w/@:.]*$", - "example": "http://192.168.99.101", - "default": null - }, - "AVALON_TIMEOUT": { - "description": "Wherever there is a need for a timeout, this is the default value.", - "type": "string", - "pattern": "^[0-9]*$", - "default": "1000", - "example": "1000" - }, - "AVALON_UPLOAD": { - "description": "Boolean of whether to upload published material to central asset repository", - "type": "string", - "default": null, - "example": "True" - }, - "AVALON_USERNAME": { - "description": "Generic username", - "type": "string", - "pattern": "^\\w*$", - "default": "avalon", - "example": "myself" - }, - "AVALON_PASSWORD": { - "description": "Generic password", - "type": "string", - "pattern": "^\\w*$", - "default": "secret", - "example": "abc123" - }, - "AVALON_INSTANCE_ID": { - "description": "Unique identifier for instances in a working file", - "type": "string", - "pattern": "^[\\w.]*$", - "default": "avalon.instance", - "example": "avalon.instance" - }, - "AVALON_CONTAINER_ID": { - "description": "Unique identifier for a loaded representation in a working file", - "type": "string", - "pattern": "^[\\w.]*$", - "default": "avalon.container", - "example": "avalon.container" - }, - "AVALON_DEBUG": { - "description": "Enable debugging mode. Some applications may use this for e.g. extended verbosity or mock plug-ins.", - "type": "string", - "default": null, - "example": "True" - } - } -} diff --git a/schema/session-2.0.json b/schema/session-2.0.json deleted file mode 100644 index 0a4d51beb2..0000000000 --- a/schema/session-2.0.json +++ /dev/null @@ -1,134 +0,0 @@ -{ - "$schema": "http://json-schema.org/draft-04/schema#", - - "title": "openpype:session-2.0", - "description": "The Avalon environment", - - "type": "object", - - "additionalProperties": true, - - "required": [ - "AVALON_PROJECT", - "AVALON_ASSET", - "AVALON_CONFIG" - ], - - "properties": { - "AVALON_PROJECTS": { - "description": "Absolute path to root of project directories", - "type": "string", - "example": "/nas/projects" - }, - "AVALON_PROJECT": { - "description": "Name of project", - "type": "string", - "pattern": "^\\w*$", - "example": "Hulk" - }, - "AVALON_ASSET": { - "description": "Name of asset", - "type": "string", - "pattern": "^\\w*$", - "example": "Bruce" - }, - "AVALON_SILO": { - "description": "Name of asset group or container", - "type": "string", - "pattern": "^\\w*$", - "example": "assets" - }, - "AVALON_TASK": { - "description": "Name of task", - "type": "string", - "pattern": "^\\w*$", - "example": "modeling" - }, - "AVALON_CONFIG": { - "description": "Name of Avalon configuration", - "type": "string", - "pattern": "^\\w*$", - "example": "polly" - }, - "AVALON_APP": { - "description": "Name of application", - "type": "string", - "pattern": "^\\w*$", - "example": "maya2016" - }, - "AVALON_DB": { - "description": "Name of database", - "type": "string", - "pattern": "^\\w*$", - "example": "avalon", - "default": "avalon" - }, - "AVALON_LABEL": { - "description": "Nice name of Avalon, used in e.g. graphical user interfaces", - "type": "string", - "example": "Mindbender", - "default": "Avalon" - }, - "AVALON_SENTRY": { - "description": "Address to Sentry", - "type": "string", - "pattern": "^http[\\w/@:.]*$", - "example": "https://5b872b280de742919b115bdc8da076a5:8d278266fe764361b8fa6024af004a9c@logs.mindbender.com/2", - "default": null - }, - "AVALON_DEADLINE": { - "description": "Address to Deadline", - "type": "string", - "pattern": "^http[\\w/@:.]*$", - "example": "http://192.168.99.101", - "default": null - }, - "AVALON_TIMEOUT": { - "description": "Wherever there is a need for a timeout, this is the default value.", - "type": "string", - "pattern": "^[0-9]*$", - "default": "1000", - "example": "1000" - }, - "AVALON_UPLOAD": { - "description": "Boolean of whether to upload published material to central asset repository", - "type": "string", - "default": null, - "example": "True" - }, - "AVALON_USERNAME": { - "description": "Generic username", - "type": "string", - "pattern": "^\\w*$", - "default": "avalon", - "example": "myself" - }, - "AVALON_PASSWORD": { - "description": "Generic password", - "type": "string", - "pattern": "^\\w*$", - "default": "secret", - "example": "abc123" - }, - "AVALON_INSTANCE_ID": { - "description": "Unique identifier for instances in a working file", - "type": "string", - "pattern": "^[\\w.]*$", - "default": "avalon.instance", - "example": "avalon.instance" - }, - "AVALON_CONTAINER_ID": { - "description": "Unique identifier for a loaded representation in a working file", - "type": "string", - "pattern": "^[\\w.]*$", - "default": "avalon.container", - "example": "avalon.container" - }, - "AVALON_DEBUG": { - "description": "Enable debugging mode. Some applications may use this for e.g. extended verbosity or mock plug-ins.", - "type": "string", - "default": null, - "example": "True" - } - } -} diff --git a/schema/session-3.0.json b/schema/session-3.0.json deleted file mode 100644 index 9f785939e4..0000000000 --- a/schema/session-3.0.json +++ /dev/null @@ -1,81 +0,0 @@ -{ - "$schema": "http://json-schema.org/draft-04/schema#", - - "title": "openpype:session-3.0", - "description": "The Avalon environment", - - "type": "object", - - "additionalProperties": true, - - "required": [ - "AVALON_PROJECT", - "AVALON_ASSET" - ], - - "properties": { - "AVALON_PROJECTS": { - "description": "Absolute path to root of project directories", - "type": "string", - "example": "/nas/projects" - }, - "AVALON_PROJECT": { - "description": "Name of project", - "type": "string", - "pattern": "^\\w*$", - "example": "Hulk" - }, - "AVALON_ASSET": { - "description": "Name of asset", - "type": "string", - "pattern": "^\\w*$", - "example": "Bruce" - }, - "AVALON_TASK": { - "description": "Name of task", - "type": "string", - "pattern": "^\\w*$", - "example": "modeling" - }, - "AVALON_APP": { - "description": "Name of host", - "type": "string", - "pattern": "^\\w*$", - "example": "maya2016" - }, - "AVALON_DB": { - "description": "Name of database", - "type": "string", - "pattern": "^\\w*$", - "example": "avalon", - "default": "avalon" - }, - "AVALON_LABEL": { - "description": "Nice name of Avalon, used in e.g. graphical user interfaces", - "type": "string", - "example": "Mindbender", - "default": "Avalon" - }, - "AVALON_TIMEOUT": { - "description": "Wherever there is a need for a timeout, this is the default value.", - "type": "string", - "pattern": "^[0-9]*$", - "default": "1000", - "example": "1000" - }, - "AVALON_INSTANCE_ID": { - "description": "Unique identifier for instances in a working file", - "type": "string", - "pattern": "^[\\w.]*$", - "default": "avalon.instance", - "example": "avalon.instance" - }, - "AVALON_CONTAINER_ID": { - "description": "Unique identifier for a loaded representation in a working file", - "type": "string", - "pattern": "^[\\w.]*$", - "default": "avalon.container", - "example": "avalon.container" - } - } -} diff --git a/schema/shaders-1.0.json b/schema/shaders-1.0.json deleted file mode 100644 index 7102ba1861..0000000000 --- a/schema/shaders-1.0.json +++ /dev/null @@ -1,32 +0,0 @@ -{ - "$schema": "http://json-schema.org/draft-04/schema#", - - "title": "openpype:shaders-1.0", - "description": "Relationships between shaders and Avalon IDs", - - "type": "object", - - "additionalProperties": true, - - "required": [ - "schema", - "shader" - ], - - "properties": { - "schema": { - "description": "Schema identifier for payload", - "type": "string" - }, - "shader": { - "description": "Name of directory", - "type": "array", - "items": { - "type": "str", - "description": "Avalon ID and optional face indexes, e.g. 'f9520572-ac1d-11e6-b39e-3085a99791c9.f[5002:5185]'" - } - } - }, - - "definitions": {} -} diff --git a/schema/subset-1.0.json b/schema/subset-1.0.json deleted file mode 100644 index a299a6d341..0000000000 --- a/schema/subset-1.0.json +++ /dev/null @@ -1,35 +0,0 @@ -{ - "$schema": "http://json-schema.org/draft-04/schema#", - - "title": "openpype:subset-1.0", - "description": "A container of instances", - - "type": "object", - - "additionalProperties": true, - - "required": [ - "schema", - "name", - "versions" - ], - - "properties": { - "schema": { - "description": "Schema identifier for payload", - "type": "string" - }, - "name": { - "description": "Name of directory", - "type": "string" - }, - "versions": { - "type": "array", - "items": { - "$ref": "version.json" - } - } - }, - - "definitions": {} -} diff --git a/schema/subset-2.0.json b/schema/subset-2.0.json deleted file mode 100644 index db256ec7fb..0000000000 --- a/schema/subset-2.0.json +++ /dev/null @@ -1,51 +0,0 @@ -{ - "$schema": "http://json-schema.org/draft-04/schema#", - - "title": "openpype:subset-2.0", - "description": "A container of instances", - - "type": "object", - - "additionalProperties": true, - - "required": [ - "schema", - "type", - "parent", - "name", - "data" - ], - - "properties": { - "schema": { - "description": "The schema associated with this document", - "type": "string", - "enum": ["openpype:subset-2.0"], - "example": "openpype:subset-2.0" - }, - "type": { - "description": "The type of document", - "type": "string", - "enum": ["subset"], - "example": "subset" - }, - "parent": { - "description": "Unique identifier to parent document", - "example": "592c33475f8c1b064c4d1696" - }, - "name": { - "description": "Name of directory", - "type": "string", - "pattern": "^[a-zA-Z0-9_.]*$", - "example": "shot01" - }, - "data": { - "type": "object", - "description": "Document metadata", - "example": { - "frameStart": 1000, - "frameEnd": 1201 - } - } - } -} diff --git a/schema/subset-3.0.json b/schema/subset-3.0.json deleted file mode 100644 index 1a0db53c04..0000000000 --- a/schema/subset-3.0.json +++ /dev/null @@ -1,62 +0,0 @@ -{ - "$schema": "http://json-schema.org/draft-04/schema#", - - "title": "openpype:subset-3.0", - "description": "A container of instances", - - "type": "object", - - "additionalProperties": true, - - "required": [ - "schema", - "type", - "parent", - "name", - "data" - ], - - "properties": { - "schema": { - "description": "The schema associated with this document", - "type": "string", - "enum": ["openpype:subset-3.0"], - "example": "openpype:subset-3.0" - }, - "type": { - "description": "The type of document", - "type": "string", - "enum": ["subset"], - "example": "subset" - }, - "parent": { - "description": "Unique identifier to parent document", - "example": "592c33475f8c1b064c4d1696" - }, - "name": { - "description": "Name of directory", - "type": "string", - "pattern": "^[a-zA-Z0-9_.]*$", - "example": "shot01" - }, - "data": { - "description": "Document metadata", - "type": "object", - "required": ["families"], - "properties": { - "families": { - "type": "array", - "items": {"type": "string"}, - "description": "One or more families associated with this subset" - } - }, - "example": { - "families" : [ - "avalon.camera" - ], - "frameStart": 1000, - "frameEnd": 1201 - } - } - } -} diff --git a/schema/thumbnail-1.0.json b/schema/thumbnail-1.0.json deleted file mode 100644 index 5bdf78a4b1..0000000000 --- a/schema/thumbnail-1.0.json +++ /dev/null @@ -1,42 +0,0 @@ -{ - "$schema": "http://json-schema.org/draft-04/schema#", - - "title": "openpype:thumbnail-1.0", - "description": "Entity with thumbnail data", - - "type": "object", - - "additionalProperties": true, - - "required": [ - "schema", - "type", - "data" - ], - - "properties": { - "schema": { - "description": "The schema associated with this document", - "type": "string", - "enum": ["openpype:thumbnail-1.0"], - "example": "openpype:thumbnail-1.0" - }, - "type": { - "description": "The type of document", - "type": "string", - "enum": ["thumbnail"], - "example": "thumbnail" - }, - "data": { - "description": "Thumbnail data", - "type": "object", - "example": { - "binary_data": "Binary({byte data of image})", - "template": "{thumbnail_root}/{project[name]}/{_id}{ext}}", - "template_data": { - "ext": ".jpg" - } - } - } - } -} diff --git a/schema/version-1.0.json b/schema/version-1.0.json deleted file mode 100644 index daa1997721..0000000000 --- a/schema/version-1.0.json +++ /dev/null @@ -1,50 +0,0 @@ -{ - "$schema": "http://json-schema.org/draft-04/schema#", - - "title": "openpype:version-1.0", - "description": "An individual version", - - "type": "object", - - "additionalProperties": true, - - "required": [ - "schema", - "version", - "path", - "time", - "author", - "source", - "representations" - ], - - "properties": { - "schema": {"type": "string"}, - "representations": { - "type": "array", - "items": { - "$ref": "representation.json" - } - }, - "time": { - "description": "ISO formatted, file-system compatible time", - "type": "string" - }, - "author": { - "description": "User logged on to the machine at time of publish", - "type": "string" - }, - "version": { - "description": "Number of this version", - "type": "number" - }, - "path": { - "description": "Unformatted path, e.g. '{root}/assets/Bruce/publish/lookdevDefault/v001", - "type": "string" - }, - "source": { - "description": "Original file from which this version was made.", - "type": "string" - } - } -} diff --git a/schema/version-2.0.json b/schema/version-2.0.json deleted file mode 100644 index 099e9be70a..0000000000 --- a/schema/version-2.0.json +++ /dev/null @@ -1,92 +0,0 @@ -{ - "$schema": "http://json-schema.org/draft-04/schema#", - - "title": "openpype:version-2.0", - "description": "An individual version", - - "type": "object", - - "additionalProperties": true, - - "required": [ - "schema", - "type", - "parent", - "name", - "data" - ], - - "properties": { - "schema": { - "description": "The schema associated with this document", - "type": "string", - "enum": ["openpype:version-2.0"], - "example": "openpype:version-2.0" - }, - "type": { - "description": "The type of document", - "type": "string", - "enum": ["version"], - "example": "version" - }, - "parent": { - "description": "Unique identifier to parent document", - "example": "592c33475f8c1b064c4d1696" - }, - "name": { - "description": "Number of version", - "type": "number", - "example": 12 - }, - "locations": { - "description": "Where on the planet this version can be found.", - "type": "array", - "items": {"type": "string"}, - "example": ["data.avalon.com"] - }, - "data": { - "description": "Document metadata", - "type": "object", - "required": ["families", "author", "source", "time"], - "properties": { - "time": { - "description": "ISO formatted, file-system compatible time", - "type": "string" - }, - "timeFormat": { - "description": "ISO format of time", - "type": "string" - }, - "author": { - "description": "User logged on to the machine at time of publish", - "type": "string" - }, - "version": { - "description": "Number of this version", - "type": "number" - }, - "path": { - "description": "Unformatted path, e.g. '{root}/assets/Bruce/publish/lookdevDefault/v001", - "type": "string" - }, - "source": { - "description": "Original file from which this version was made.", - "type": "string" - }, - "families": { - "type": "array", - "items": {"type": "string"}, - "description": "One or more families associated with this version" - } - }, - "example": { - "source" : "{root}/f02_prod/assets/BubbleWitch/work/modeling/marcus/maya/scenes/model_v001.ma", - "author" : "marcus", - "families" : [ - "avalon.model" - ], - "time" : "20170510T090203Z" - } - } - } -} diff --git a/schema/version-3.0.json b/schema/version-3.0.json deleted file mode 100644 index 3e07fc4499..0000000000 --- a/schema/version-3.0.json +++ /dev/null @@ -1,84 +0,0 @@ -{ - "$schema": "http://json-schema.org/draft-04/schema#", - - "title": "openpype:version-3.0", - "description": "An individual version", - - "type": "object", - - "additionalProperties": true, - - "required": [ - "schema", - "type", - "parent", - "name", - "data" - ], - - "properties": { - "schema": { - "description": "The schema associated with this document", - "type": "string", - "enum": ["openpype:version-3.0"], - "example": "openpype:version-3.0" - }, - "type": { - "description": "The type of document", - "type": "string", - "enum": ["version"], - "example": "version" - }, - "parent": { - "description": "Unique identifier to parent document", - "example": "592c33475f8c1b064c4d1696" - }, - "name": { - "description": "Number of version", - "type": "number", - "example": 12 - }, - "locations": { - "description": "Where on the planet this version can be found.", - "type": "array", - "items": {"type": "string"}, - "example": ["data.avalon.com"] - }, - "data": { - "description": "Document metadata", - "type": "object", - "required": ["author", "source", "time"], - "properties": { - "time": { - "description": "ISO formatted, file-system compatible time", - "type": "string" - }, - "timeFormat": { - "description": "ISO format of time", - "type": "string" - }, - "author": { - "description": "User logged on to the machine at time of publish", - "type": "string" - }, - "version": { - "description": "Number of this version", - "type": "number" - }, - "path": { - "description": "Unformatted path, e.g. '{root}/assets/Bruce/publish/lookdevDefault/v001", - "type": "string" - }, - "source": { - "description": "Original file from which this version was made.", - "type": "string" - } - }, - "example": { - "source" : "{root}/f02_prod/assets/BubbleWitch/work/modeling/marcus/maya/scenes/model_v001.ma", - "author" : "marcus", - "time" : "20170510T090203Z" - } - } - } -} diff --git a/schema/workfile-1.0.json b/schema/workfile-1.0.json deleted file mode 100644 index 5f9600ef20..0000000000 --- a/schema/workfile-1.0.json +++ /dev/null @@ -1,52 +0,0 @@ -{ - "$schema": "http://json-schema.org/draft-04/schema#", - - "title": "openpype:workfile-1.0", - "description": "Workfile additional information.", - - "type": "object", - - "additionalProperties": true, - - "required": [ - "schema", - "type", - "filename", - "task_name", - "parent" - ], - - "properties": { - "schema": { - "description": "Schema identifier for payload", - "type": "string", - "enum": ["openpype:workfile-1.0"], - "example": "openpype:workfile-1.0" - }, - "type": { - "description": "The type of document", - "type": "string", - "enum": ["workfile"], - "example": "workfile" - }, - "parent": { - "description": "Unique identifier to parent document", - "example": "592c33475f8c1b064c4d1696" - }, - "filename": { - "description": "Workfile's filename", - "type": "string", - "example": "kuba_each_case_Alpaca_01_animation_v001.ma" - }, - "task_name": { - "description": "Task name", - "type": "string", - "example": "animation" - }, - "data": { - "description": "Document metadata", - "type": "object", - "example": {"key": "value"} - } - } -} diff --git a/server_addon/aftereffects/server/settings/creator_plugins.py b/server_addon/aftereffects/server/settings/creator_plugins.py index ee52fadd40..9cb03b0b26 100644 --- a/server_addon/aftereffects/server/settings/creator_plugins.py +++ b/server_addon/aftereffects/server/settings/creator_plugins.py @@ -5,7 +5,7 @@ from ayon_server.settings import BaseSettingsModel class CreateRenderPlugin(BaseSettingsModel): mark_for_review: bool = Field(True, title="Review") - defaults: list[str] = Field( + default_variants: list[str] = Field( default_factory=list, title="Default Variants" ) diff --git a/server_addon/aftereffects/server/settings/main.py b/server_addon/aftereffects/server/settings/main.py index 04d2e51cc9..4edc46d259 100644 --- a/server_addon/aftereffects/server/settings/main.py +++ b/server_addon/aftereffects/server/settings/main.py @@ -40,7 +40,7 @@ DEFAULT_AFTEREFFECTS_SETTING = { "create": { "RenderCreator": { "mark_for_review": True, - "defaults": [ + "default_variants": [ "Main" ] } diff --git a/server_addon/aftereffects/server/version.py b/server_addon/aftereffects/server/version.py index a242f0e757..df0c92f1e2 100644 --- a/server_addon/aftereffects/server/version.py +++ b/server_addon/aftereffects/server/version.py @@ -1,3 +1,3 @@ # -*- coding: utf-8 -*- """Package declaring addon version.""" -__version__ = "0.1.1" +__version__ = "0.1.2" diff --git a/server_addon/core/server/settings/main.py b/server_addon/core/server/settings/main.py index d19d732e71..ca8f7e63ed 100644 --- a/server_addon/core/server/settings/main.py +++ b/server_addon/core/server/settings/main.py @@ -4,6 +4,7 @@ from ayon_server.settings import ( BaseSettingsModel, MultiplatformPathListModel, ensure_unique_names, + task_types_enum, ) from ayon_server.exceptions import BadRequestException @@ -38,13 +39,52 @@ class CoreImageIOConfigModel(BaseSettingsModel): class CoreImageIOBaseModel(BaseSettingsModel): activate_global_color_management: bool = Field( False, - title="Override global OCIO config" + title="Enable Color Management" ) ocio_config: CoreImageIOConfigModel = Field( - default_factory=CoreImageIOConfigModel, title="OCIO config" + default_factory=CoreImageIOConfigModel, + title="OCIO config" ) file_rules: CoreImageIOFileRulesModel = Field( - default_factory=CoreImageIOFileRulesModel, title="File Rules" + default_factory=CoreImageIOFileRulesModel, + title="File Rules" + ) + + +class VersionStartCategoryProfileModel(BaseSettingsModel): + _layout = "expanded" + host_names: list[str] = Field( + default_factory=list, + title="Host names" + ) + task_types: list[str] = Field( + default_factory=list, + title="Task types", + enum_resolver=task_types_enum + ) + task_names: list[str] = Field( + default_factory=list, + title="Task names" + ) + product_types: list[str] = Field( + default_factory=list, + title="Product types" + ) + product_names: list[str] = Field( + default_factory=list, + title="Product names" + ) + version_start: int = Field( + 1, + title="Version Start", + ge=0 + ) + + +class VersionStartCategoryModel(BaseSettingsModel): + profiles: list[VersionStartCategoryProfileModel] = Field( + default_factory=list, + title="Profiles" ) @@ -61,6 +101,10 @@ class CoreSettings(BaseSettingsModel): default_factory=GlobalToolsModel, title="Tools" ) + version_start_category: VersionStartCategoryModel = Field( + default_factory=VersionStartCategoryModel, + title="Version start" + ) imageio: CoreImageIOBaseModel = Field( default_factory=CoreImageIOBaseModel, title="Color Management (ImageIO)" @@ -131,6 +175,9 @@ DEFAULT_VALUES = { "studio_code": "", "environments": "{}", "tools": DEFAULT_TOOLS_VALUES, + "version_start_category": { + "profiles": [] + }, "publish": DEFAULT_PUBLISH_VALUES, "project_folder_structure": json.dumps({ "__project_root__": { diff --git a/server_addon/core/server/version.py b/server_addon/core/server/version.py index 485f44ac21..b3f4756216 100644 --- a/server_addon/core/server/version.py +++ b/server_addon/core/server/version.py @@ -1 +1 @@ -__version__ = "0.1.1" +__version__ = "0.1.2" diff --git a/server_addon/houdini/server/settings/publish_plugins.py b/server_addon/houdini/server/settings/publish_plugins.py index 7d35d7e634..58240b0205 100644 --- a/server_addon/houdini/server/settings/publish_plugins.py +++ b/server_addon/houdini/server/settings/publish_plugins.py @@ -21,10 +21,28 @@ class CreateArnoldAssModel(BaseSettingsModel): ext: str = Field(Title="Extension") +class CreateStaticMeshModel(BaseSettingsModel): + enabled: bool = Field(title="Enabled") + default_variants: list[str] = Field( + default_factory=list, + title="Default Products" + ) + static_mesh_prefixes: str = Field("S", title="Static Mesh Prefix") + collision_prefixes: list[str] = Field( + default_factory=list, + title="Collision Prefixes" + ) + + class CreatePluginsModel(BaseSettingsModel): CreateArnoldAss: CreateArnoldAssModel = Field( default_factory=CreateArnoldAssModel, title="Create Alembic Camera") + # "-" is not compatible in the new model + CreateStaticMesh: CreateStaticMeshModel = Field( + default_factory=CreateStaticMeshModel, + title="Create Static Mesh" + ) CreateAlembicCamera: CreatorModel = Field( default_factory=CreatorModel, title="Create Alembic Camera") @@ -63,6 +81,19 @@ DEFAULT_HOUDINI_CREATE_SETTINGS = { "default_variants": ["Main"], "ext": ".ass" }, + "CreateStaticMesh": { + "enabled": True, + "default_variants": [ + "Main" + ], + "static_mesh_prefix": "S", + "collision_prefixes": [ + "UBX", + "UCP", + "USP", + "UCX" + ] + }, "CreateAlembicCamera": { "enabled": True, "default_variants": ["Main"] @@ -120,7 +151,7 @@ class ValidateWorkfilePathsModel(BaseSettingsModel): ) -class ValidateContainersModel(BaseSettingsModel): +class BasicValidateModel(BaseSettingsModel): enabled: bool = Field(title="Enabled") optional: bool = Field(title="Optional") active: bool = Field(title="Active") @@ -130,9 +161,21 @@ class PublishPluginsModel(BaseSettingsModel): ValidateWorkfilePaths: ValidateWorkfilePathsModel = Field( default_factory=ValidateWorkfilePathsModel, title="Validate workfile paths settings.") - ValidateContainers: ValidateContainersModel = Field( - default_factory=ValidateContainersModel, + ValidateReviewColorspace: BasicValidateModel = Field( + default_factory=BasicValidateModel, + title="Validate Review Colorspace.") + ValidateContainers: BasicValidateModel = Field( + default_factory=BasicValidateModel, title="Validate Latest Containers.") + ValidateSubsetName: BasicValidateModel = Field( + default_factory=BasicValidateModel, + title="Validate Subset Name.") + ValidateMeshIsStatic: BasicValidateModel = Field( + default_factory=BasicValidateModel, + title="Validate Mesh is Static.") + ValidateUnrealStaticMeshName: BasicValidateModel = Field( + default_factory=BasicValidateModel, + title="Validate Unreal Static Mesh Name.") DEFAULT_HOUDINI_PUBLISH_SETTINGS = { @@ -148,9 +191,29 @@ DEFAULT_HOUDINI_PUBLISH_SETTINGS = { "$JOB" ] }, + "ValidateReviewColorspace": { + "enabled": True, + "optional": True, + "active": True + }, "ValidateContainers": { "enabled": True, "optional": True, "active": True + }, + "ValidateSubsetName": { + "enabled": True, + "optional": True, + "active": True + }, + "ValidateMeshIsStatic": { + "enabled": True, + "optional": True, + "active": True + }, + "ValidateUnrealStaticMeshName": { + "enabled": False, + "optional": True, + "active": True } } diff --git a/server_addon/houdini/server/version.py b/server_addon/houdini/server/version.py index 485f44ac21..ae7362549b 100644 --- a/server_addon/houdini/server/version.py +++ b/server_addon/houdini/server/version.py @@ -1 +1 @@ -__version__ = "0.1.1" +__version__ = "0.1.3" diff --git a/server_addon/max/server/settings/render_settings.py b/server_addon/max/server/settings/render_settings.py index 6c236d9f12..c00cb5e436 100644 --- a/server_addon/max/server/settings/render_settings.py +++ b/server_addon/max/server/settings/render_settings.py @@ -44,6 +44,6 @@ class RenderSettingsModel(BaseSettingsModel): DEFAULT_RENDER_SETTINGS = { "default_render_image_folder": "renders/3dsmax", "aov_separator": "underscore", - "image_format": "png", + "image_format": "exr", "multipass": True } diff --git a/server_addon/maya/server/settings/creators.py b/server_addon/maya/server/settings/creators.py index 9b97b92e59..11e2b8a36c 100644 --- a/server_addon/maya/server/settings/creators.py +++ b/server_addon/maya/server/settings/creators.py @@ -252,7 +252,9 @@ DEFAULT_CREATORS_SETTINGS = { }, "CreateUnrealSkeletalMesh": { "enabled": True, - "default_variants": [], + "default_variants": [ + "Main", + ], "joint_hints": "jnt_org" }, "CreateMultiverseLook": { diff --git a/server_addon/maya/server/settings/imageio.py b/server_addon/maya/server/settings/imageio.py index 7512bfe253..946a14c866 100644 --- a/server_addon/maya/server/settings/imageio.py +++ b/server_addon/maya/server/settings/imageio.py @@ -39,8 +39,10 @@ class ImageIOFileRulesModel(BaseSettingsModel): class ColorManagementPreferenceV2Model(BaseSettingsModel): - """Color Management Preference v2 (Maya 2022+).""" - _layout = "expanded" + """Color Management Preference v2 (Maya 2022+). + + Please migrate all to 'imageio/workfile' and enable it. + """ enabled: bool = Field(True, title="Use Color Management Preference v2") @@ -51,7 +53,6 @@ class ColorManagementPreferenceV2Model(BaseSettingsModel): class ColorManagementPreferenceModel(BaseSettingsModel): """Color Management Preference (legacy).""" - _layout = "expanded" renderSpace: str = Field(title="Rendering Space") viewTransform: str = Field(title="Viewer Transform ") @@ -89,11 +90,11 @@ class ImageIOSettings(BaseSettingsModel): # Deprecated colorManagementPreference_v2: ColorManagementPreferenceV2Model = Field( default_factory=ColorManagementPreferenceV2Model, - title="Color Management Preference v2 (Maya 2022+)" + title="DEPRECATED: Color Management Preference v2 (Maya 2022+)" ) colorManagementPreference: ColorManagementPreferenceModel = Field( default_factory=ColorManagementPreferenceModel, - title="Color Management Preference (legacy)" + title="DEPRECATED: Color Management Preference (legacy)" ) diff --git a/server_addon/nuke/server/settings/common.py b/server_addon/nuke/server/settings/common.py index 700f01f3dc..2bc3c9be81 100644 --- a/server_addon/nuke/server/settings/common.py +++ b/server_addon/nuke/server/settings/common.py @@ -89,12 +89,6 @@ knob_types_enum = [ class KnobModel(BaseSettingsModel): - """# TODO: new data structure - - v3 was having type, name, value but - ayon is not able to make it the same. Current model is - defining `type` as `text` and instead of `value` the key is `text`. - So if `type` is `boolean` then key is `boolean` (value). - """ _layout = "expanded" type: str = Field( diff --git a/server_addon/nuke/server/settings/create_plugins.py b/server_addon/nuke/server/settings/create_plugins.py index 0bbae4ee77..80aec51ae0 100644 --- a/server_addon/nuke/server/settings/create_plugins.py +++ b/server_addon/nuke/server/settings/create_plugins.py @@ -16,13 +16,10 @@ def instance_attributes_enum(): class PrenodeModel(BaseSettingsModel): - # TODO: missing in host api - # - good for `dependency` name: str = Field( title="Node name" ) - # TODO: `nodeclass` should be renamed to `nuke_node_class` nodeclass: str = Field( "", title="Node class" @@ -32,11 +29,8 @@ class PrenodeModel(BaseSettingsModel): title="Incoming dependency" ) - """# TODO: Changes in host api: - - Need complete rework of knob types in nuke integration. - - We could not support v3 style of settings. - """ knobs: list[KnobModel] = Field( + default_factory=list, title="Knobs", ) @@ -61,11 +55,8 @@ class CreateWriteRenderModel(BaseSettingsModel): title="Instance attributes" ) - """# TODO: Changes in host api: - - prenodes key was originally dict and now is list - (we could not support v3 style of settings) - """ prenodes: list[PrenodeModel] = Field( + default_factory=list, title="Preceding nodes", ) @@ -90,11 +81,8 @@ class CreateWritePrerenderModel(BaseSettingsModel): title="Instance attributes" ) - """# TODO: Changes in host api: - - prenodes key was originally dict and now is list - (we could not support v3 style of settings) - """ prenodes: list[PrenodeModel] = Field( + default_factory=list, title="Preceding nodes", ) @@ -119,11 +107,8 @@ class CreateWriteImageModel(BaseSettingsModel): title="Instance attributes" ) - """# TODO: Changes in host api: - - prenodes key was originally dict and now is list - (we could not support v3 style of settings) - """ prenodes: list[PrenodeModel] = Field( + default_factory=list, title="Preceding nodes", ) diff --git a/server_addon/nuke/server/settings/dirmap.py b/server_addon/nuke/server/settings/dirmap.py index 2da6d7bf60..7e3c443957 100644 --- a/server_addon/nuke/server/settings/dirmap.py +++ b/server_addon/nuke/server/settings/dirmap.py @@ -25,19 +25,6 @@ class DirmapSettings(BaseSettingsModel): ) -"""# TODO: -nuke is having originally implemented -following data inputs: - -"nuke-dirmap": { - "enabled": false, - "paths": { - "source-path": [], - "destination-path": [] - } -} -""" - DEFAULT_DIRMAP_SETTINGS = { "enabled": False, "paths": { diff --git a/server_addon/nuke/server/settings/imageio.py b/server_addon/nuke/server/settings/imageio.py index b43017ef8b..811b12104b 100644 --- a/server_addon/nuke/server/settings/imageio.py +++ b/server_addon/nuke/server/settings/imageio.py @@ -9,22 +9,17 @@ from .common import KnobModel class NodesModel(BaseSettingsModel): - """# TODO: This needs to be somehow labeled in settings panel - or at least it could show gist of configuration - """ _layout = "expanded" plugins: list[str] = Field( + default_factory=list, title="Used in plugins" ) - # TODO: rename `nukeNodeClass` to `nuke_node_class` nukeNodeClass: str = Field( title="Nuke Node Class", ) - """ # TODO: Need complete rework of knob types - in nuke integration. We could not support v3 style of settings. - """ knobs: list[KnobModel] = Field( + default_factory=list, title="Knobs", ) @@ -66,22 +61,6 @@ def ocio_configs_switcher_enum(): class WorkfileColorspaceSettings(BaseSettingsModel): """Nuke workfile colorspace preset. """ - """# TODO: enhance settings with host api: - we need to add mapping to resolve properly keys. - Nuke is excpecting camel case key names, - but for better code consistency we need to - be using snake_case: - - color_management = colorManagement - ocio_config = OCIO_config - working_space_name = workingSpaceLUT - monitor_name = monitorLut - monitor_out_name = monitorOutLut - int_8_name = int8Lut - int_16_name = int16Lut - log_name = logLut - float_name = floatLut - """ colorManagement: Literal["Nuke", "OCIO"] = Field( title="Color Management" @@ -100,18 +79,6 @@ class WorkfileColorspaceSettings(BaseSettingsModel): monitorLut: str = Field( title="Monitor" ) - int8Lut: str = Field( - title="8-bit files" - ) - int16Lut: str = Field( - title="16-bit files" - ) - logLut: str = Field( - title="Log files" - ) - floatLut: str = Field( - title="Float files" - ) class ReadColorspaceRulesItems(BaseSettingsModel): @@ -170,7 +137,7 @@ class ImageIOSettings(BaseSettingsModel): _isGroup: bool = True """# TODO: enhance settings with host api: - to restruture settings for simplification. + to restructure settings for simplification. now: nuke/imageio/viewer/viewerProcess future: nuke/imageio/viewer @@ -193,7 +160,7 @@ class ImageIOSettings(BaseSettingsModel): ) """# TODO: enhance settings with host api: - to restruture settings for simplification. + to restructure settings for simplification. now: nuke/imageio/baking/viewerProcess future: nuke/imageio/baking @@ -215,9 +182,9 @@ class ImageIOSettings(BaseSettingsModel): title="Nodes" ) """# TODO: enhance settings with host api: - - old settings are using `regexInputs` key but we + - [ ] old settings are using `regexInputs` key but we need to rename to `regex_inputs` - - no need for `inputs` middle part. It can stay + - [ ] no need for `inputs` middle part. It can stay directly on `regex_inputs` """ regexInputs: RegexInputsModel = Field( @@ -238,10 +205,6 @@ DEFAULT_IMAGEIO_SETTINGS = { "OCIO_config": "nuke-default", "workingSpaceLUT": "linear", "monitorLut": "sRGB", - "int8Lut": "sRGB", - "int16Lut": "sRGB", - "logLut": "Cineon", - "floatLut": "linear" }, "nodes": { "requiredNodes": [ diff --git a/server_addon/nuke/server/settings/loader_plugins.py b/server_addon/nuke/server/settings/loader_plugins.py index 6db381bffb..51e2c2149b 100644 --- a/server_addon/nuke/server/settings/loader_plugins.py +++ b/server_addon/nuke/server/settings/loader_plugins.py @@ -6,10 +6,6 @@ class LoadImageModel(BaseSettingsModel): enabled: bool = Field( title="Enabled" ) - """# TODO: v3 api used `_representation` - New api is hiding it so it had to be renamed - to `representations_include` - """ representations_include: list[str] = Field( default_factory=list, title="Include representations" @@ -33,10 +29,6 @@ class LoadClipModel(BaseSettingsModel): enabled: bool = Field( title="Enabled" ) - """# TODO: v3 api used `_representation` - New api is hiding it so it had to be renamed - to `representations_include` - """ representations_include: list[str] = Field( default_factory=list, title="Include representations" diff --git a/server_addon/nuke/server/settings/main.py b/server_addon/nuke/server/settings/main.py index 4687d48ac9..cdaaa3a9e2 100644 --- a/server_addon/nuke/server/settings/main.py +++ b/server_addon/nuke/server/settings/main.py @@ -59,9 +59,7 @@ class NukeSettings(BaseSettingsModel): default_factory=ImageIOSettings, title="Color Management (imageio)", ) - """# TODO: fix host api: - - rename `nuke-dirmap` to `dirmap` was inevitable - """ + dirmap: DirmapSettings = Field( default_factory=DirmapSettings, title="Nuke Directory Mapping", diff --git a/server_addon/nuke/server/settings/publish_plugins.py b/server_addon/nuke/server/settings/publish_plugins.py index 7e898f8c9a..c78685534f 100644 --- a/server_addon/nuke/server/settings/publish_plugins.py +++ b/server_addon/nuke/server/settings/publish_plugins.py @@ -28,11 +28,9 @@ def nuke_product_types_enum(): class NodeModel(BaseSettingsModel): - # TODO: missing in host api name: str = Field( title="Node name" ) - # TODO: `nodeclass` rename to `nuke_node_class` nodeclass: str = Field( "", title="Node class" @@ -41,11 +39,8 @@ class NodeModel(BaseSettingsModel): "", title="Incoming dependency" ) - """# TODO: Changes in host api: - - Need complete rework of knob types in nuke integration. - - We could not support v3 style of settings. - """ knobs: list[KnobModel] = Field( + default_factory=list, title="Knobs", ) @@ -99,12 +94,9 @@ class ExtractThumbnailModel(BaseSettingsModel): use_rendered: bool = Field(title="Use rendered images") bake_viewer_process: bool = Field(title="Bake view process") bake_viewer_input_process: bool = Field(title="Bake viewer input process") - """# TODO: needs to rewrite from v3 to ayon - - `nodes` in v3 was dict but now `prenodes` is list of dict - - also later `nodes` should be `prenodes` - """ nodes: list[NodeModel] = Field( + default_factory=list, title="Nodes (deprecated)" ) reposition_nodes: list[ThumbnailRepositionNodeModel] = Field( @@ -177,6 +169,7 @@ class ExtractReviewDataMovModel(BaseSettingsModel): enabled: bool = Field(title="Enabled") viewer_lut_raw: bool = Field(title="Viewer lut raw") outputs: list[BakingStreamModel] = Field( + default_factory=list, title="Baking streams" ) @@ -213,12 +206,6 @@ class ExctractSlateFrameParamModel(BaseSettingsModel): class ExtractSlateFrameModel(BaseSettingsModel): viewer_lut_raw: bool = Field(title="Viewer lut raw") - """# TODO: v3 api different model: - - not possible to replicate v3 model: - {"name": [bool, str]} - - not it is: - {"name": {"enabled": bool, "template": str}} - """ key_value_mapping: ExctractSlateFrameParamModel = Field( title="Key value mapping", default_factory=ExctractSlateFrameParamModel @@ -287,7 +274,6 @@ class PublishPuginsModel(BaseSettingsModel): title="Extract Slate Frame", default_factory=ExtractSlateFrameModel ) - # TODO: plugin should be renamed - `workfile` not `script` IncrementScriptVersion: IncrementScriptVersionModel = Field( title="Increment Workfile Version", default_factory=IncrementScriptVersionModel, diff --git a/server_addon/nuke/server/settings/scriptsmenu.py b/server_addon/nuke/server/settings/scriptsmenu.py index 9d1c32ebac..0b2d660da5 100644 --- a/server_addon/nuke/server/settings/scriptsmenu.py +++ b/server_addon/nuke/server/settings/scriptsmenu.py @@ -17,7 +17,6 @@ class ScriptsmenuSettings(BaseSettingsModel): """Nuke script menu project settings.""" _isGroup = True - # TODO: in api rename key `name` to `menu_name` name: str = Field(title="Menu Name") definition: list[ScriptsmenuSubmodel] = Field( default_factory=list, diff --git a/server_addon/nuke/server/settings/templated_workfile_build.py b/server_addon/nuke/server/settings/templated_workfile_build.py index e0245c8d06..0899be841e 100644 --- a/server_addon/nuke/server/settings/templated_workfile_build.py +++ b/server_addon/nuke/server/settings/templated_workfile_build.py @@ -28,6 +28,7 @@ class TemplatedWorkfileProfileModel(BaseSettingsModel): class TemplatedWorkfileBuildModel(BaseSettingsModel): + """Settings for templated workfile builder.""" profiles: list[TemplatedWorkfileProfileModel] = Field( default_factory=list ) diff --git a/server_addon/nuke/server/settings/workfile_builder.py b/server_addon/nuke/server/settings/workfile_builder.py index ee67c7c16a..3ae3b08788 100644 --- a/server_addon/nuke/server/settings/workfile_builder.py +++ b/server_addon/nuke/server/settings/workfile_builder.py @@ -48,20 +48,32 @@ class BuilderProfileModel(BaseSettingsModel): title="Task names" ) current_context: list[BuilderProfileItemModel] = Field( - title="Current context") + default_factory=list, + title="Current context" + ) linked_assets: list[BuilderProfileItemModel] = Field( - title="Linked assets/shots") + default_factory=list, + title="Linked assets/shots" + ) class WorkfileBuilderModel(BaseSettingsModel): + """[deprecated] use Template Workfile Build Settings instead. + """ create_first_version: bool = Field( title="Create first workfile") custom_templates: list[CustomTemplateModel] = Field( - title="Custom templates") + default_factory=list, + title="Custom templates" + ) builder_on_start: bool = Field( - title="Run Builder at first workfile") + default=False, + title="Run Builder at first workfile" + ) profiles: list[BuilderProfileModel] = Field( - title="Builder profiles") + default_factory=list, + title="Builder profiles" + ) DEFAULT_WORKFILE_BUILDER_SETTINGS = { diff --git a/server_addon/traypublisher/server/settings/simple_creators.py b/server_addon/traypublisher/server/settings/simple_creators.py index 94d6602738..8335b9d34e 100644 --- a/server_addon/traypublisher/server/settings/simple_creators.py +++ b/server_addon/traypublisher/server/settings/simple_creators.py @@ -288,5 +288,22 @@ DEFAULT_SIMPLE_CREATORS = [ "allow_multiple_items": True, "allow_version_control": False, "extensions": [] + }, + { + "product_type": "audio", + "identifier": "", + "label": "Audio ", + "icon": "fa5s.file-audio", + "default_variants": [ + "Main" + ], + "description": "Audio product", + "detailed_description": "Audio files for review or final delivery", + "allow_sequences": False, + "allow_multiple_items": False, + "allow_version_control": False, + "extensions": [ + ".wav" + ] } ] diff --git a/setup.cfg b/setup.cfg index 10cca3eb3f..ead9b25164 100644 --- a/setup.cfg +++ b/setup.cfg @@ -28,4 +28,11 @@ omit = /tests directory = ./coverage [tool:pytest] -norecursedirs = repos/* openpype/modules/ftrack/* \ No newline at end of file +norecursedirs = openpype/modules/ftrack/* + +[isort] +line_length = 79 +multi_line_output = 3 +include_trailing_comma = True +force_grid_wrap = 0 +combine_as_imports = True diff --git a/setup.py b/setup.py index 4b6f286730..6179de1d34 100644 --- a/setup.py +++ b/setup.py @@ -124,7 +124,6 @@ bin_includes = [ include_files = [ "igniter", "openpype", - "schema", "LICENSE", "README.md" ] diff --git a/tests/unit/openpype/hosts/unreal/plugins/publish/test_validate_sequence_frames.py b/tests/unit/openpype/hosts/unreal/plugins/publish/test_validate_sequence_frames.py index 17e47c9f64..f472b8052a 100644 --- a/tests/unit/openpype/hosts/unreal/plugins/publish/test_validate_sequence_frames.py +++ b/tests/unit/openpype/hosts/unreal/plugins/publish/test_validate_sequence_frames.py @@ -19,7 +19,7 @@ import logging from pyblish.api import Instance as PyblishInstance from tests.lib.testing_classes import BaseTest -from openpype.plugins.publish.validate_sequence_frames import ( +from openpype.hosts.unreal.plugins.publish.validate_sequence_frames import ( ValidateSequenceFrames ) @@ -38,7 +38,13 @@ class TestValidateSequenceFrames(BaseTest): data = { "frameStart": 1001, "frameEnd": 1002, - "representations": [] + "representations": [], + "assetEntity": { + "data": { + "clipIn": 1001, + "clipOut": 1002, + } + } } yield Instance @@ -58,6 +64,7 @@ class TestValidateSequenceFrames(BaseTest): ] instance.data["representations"] = representations instance.data["frameEnd"] = 1001 + instance.data["assetEntity"]["data"]["clipOut"] = 1001 plugin.process(instance) @@ -84,49 +91,11 @@ class TestValidateSequenceFrames(BaseTest): plugin.process(instance) - @pytest.mark.parametrize("files", - [["Main_beauty.1001.v001.exr", - "Main_beauty.1002.v001.exr"]]) - def test_validate_sequence_frames_wrong_name(self, instance, - plugin, files): - # tests for names with number inside, caused clique failure before - representations = [ - { - "ext": "exr", - "files": files, - } - ] - instance.data["representations"] = representations - - with pytest.raises(AssertionError) as excinfo: - plugin.process(instance) - assert ("Must detect single collection" in - str(excinfo.value)) - - @pytest.mark.parametrize("files", - [["Main_beauty.v001.1001.ass.gz", - "Main_beauty.v001.1002.ass.gz"]]) - def test_validate_sequence_frames_possible_wrong_name( - self, instance, plugin, files): - # currently pattern fails on extensions with dots - representations = [ - { - "files": files, - } - ] - instance.data["representations"] = representations - - with pytest.raises(AssertionError) as excinfo: - plugin.process(instance) - assert ("Must not have remainder" in - str(excinfo.value)) - @pytest.mark.parametrize("files", [["Main_beauty.v001.1001.ass.gz", "Main_beauty.v001.1002.ass.gz"]]) def test_validate_sequence_frames__correct_ext( self, instance, plugin, files): - # currently pattern fails on extensions with dots representations = [ { "ext": "ass.gz", @@ -147,6 +116,7 @@ class TestValidateSequenceFrames(BaseTest): ] instance.data["representations"] = representations instance.data["frameEnd"] = 1003 + instance.data["assetEntity"]["data"]["clipOut"] = 1003 plugin.process(instance) @@ -160,6 +130,7 @@ class TestValidateSequenceFrames(BaseTest): ] instance.data["representations"] = representations instance.data["frameEnd"] = 1003 + instance.data["assetEntity"]["data"]["clipOut"] = 1003 with pytest.raises(ValueError) as excinfo: plugin.process(instance) @@ -175,6 +146,7 @@ class TestValidateSequenceFrames(BaseTest): ] instance.data["representations"] = representations instance.data["frameEnd"] = 1003 + instance.data["assetEntity"]["data"]["clipOut"] = 1003 with pytest.raises(AssertionError) as excinfo: plugin.process(instance) @@ -195,6 +167,7 @@ class TestValidateSequenceFrames(BaseTest): instance.data["slate"] = True instance.data["representations"] = representations instance.data["frameEnd"] = 1003 + instance.data["assetEntity"]["data"]["clipOut"] = 1003 plugin.process(instance) diff --git a/tests/unit/openpype/lib/test_delivery.py b/tests/unit/openpype/lib/test_delivery.py index 04a71655e3..f1e435f3f8 100644 --- a/tests/unit/openpype/lib/test_delivery.py +++ b/tests/unit/openpype/lib/test_delivery.py @@ -1,6 +1,6 @@ # -*- coding: utf-8 -*- """Test suite for delivery functions.""" -from openpype.lib.delivery import collect_frames +from openpype.lib import collect_frames def test_collect_frames_multi_sequence(): @@ -153,4 +153,3 @@ def test_collect_frames_single_file(): print(ret) assert ret == expected, "Not matching" - diff --git a/tests/unit/openpype/lib/test_event_system.py b/tests/unit/openpype/lib/test_event_system.py new file mode 100644 index 0000000000..aa3f929065 --- /dev/null +++ b/tests/unit/openpype/lib/test_event_system.py @@ -0,0 +1,83 @@ +from openpype.lib.events import EventSystem, QueuedEventSystem + + +def test_default_event_system(): + output = [] + expected_output = [3, 2, 1] + event_system = EventSystem() + + def callback_1(): + event_system.emit("topic.2", {}, None) + output.append(1) + + def callback_2(): + event_system.emit("topic.3", {}, None) + output.append(2) + + def callback_3(): + output.append(3) + + event_system.add_callback("topic.1", callback_1) + event_system.add_callback("topic.2", callback_2) + event_system.add_callback("topic.3", callback_3) + + event_system.emit("topic.1", {}, None) + + assert output == expected_output, ( + "Callbacks were not called in correct order") + + +def test_base_event_system_queue(): + output = [] + expected_output = [1, 2, 3] + event_system = QueuedEventSystem() + + def callback_1(): + event_system.emit("topic.2", {}, None) + output.append(1) + + def callback_2(): + event_system.emit("topic.3", {}, None) + output.append(2) + + def callback_3(): + output.append(3) + + event_system.add_callback("topic.1", callback_1) + event_system.add_callback("topic.2", callback_2) + event_system.add_callback("topic.3", callback_3) + + event_system.emit("topic.1", {}, None) + + assert output == expected_output, ( + "Callbacks were not called in correct order") + + +def test_manual_event_system_queue(): + output = [] + expected_output = [1, 2, 3] + event_system = QueuedEventSystem(auto_execute=False) + + def callback_1(): + event_system.emit("topic.2", {}, None) + output.append(1) + + def callback_2(): + event_system.emit("topic.3", {}, None) + output.append(2) + + def callback_3(): + output.append(3) + + event_system.add_callback("topic.1", callback_1) + event_system.add_callback("topic.2", callback_2) + event_system.add_callback("topic.3", callback_3) + + event_system.emit("topic.1", {}, None) + + while True: + if event_system.process_next_event() is None: + break + + assert output == expected_output, ( + "Callbacks were not called in correct order") diff --git a/tests/unit/openpype/modules/sync_server/test_site_operations.py b/tests/unit/openpype/modules/sync_server/test_site_operations.py index 6a861100a4..c4a83e33a6 100644 --- a/tests/unit/openpype/modules/sync_server/test_site_operations.py +++ b/tests/unit/openpype/modules/sync_server/test_site_operations.py @@ -12,16 +12,19 @@ removes temporary databases (?) """ import pytest +from bson.objectid import ObjectId from tests.lib.testing_classes import ModuleUnitTest -from bson.objectid import ObjectId + +from openpype.modules.sync_server.utils import SiteAlreadyPresentError + class TestSiteOperation(ModuleUnitTest): REPRESENTATION_ID = "60e578d0c987036c6a7b741d" - TEST_FILES = [("1eCwPljuJeOI8A3aisfOIBKKjcmIycTEt", + TEST_FILES = [("1FHE70Hi7y05LLT_1O3Y6jGxwZGXKV9zX", "test_site_operations.zip", '')] @pytest.fixture(scope="module") @@ -71,7 +74,7 @@ class TestSiteOperation(ModuleUnitTest): @pytest.mark.usefixtures("setup_sync_server_module") def test_add_site_again(self, dbcon, setup_sync_server_module): """Depends on test_add_site, must throw exception.""" - with pytest.raises(ValueError): + with pytest.raises(SiteAlreadyPresentError): setup_sync_server_module.add_site(self.TEST_PROJECT_NAME, self.REPRESENTATION_ID, site_name='test_site') diff --git a/tools/docker_build.ps1 b/tools/docker_build.ps1 new file mode 100644 index 0000000000..392165288c --- /dev/null +++ b/tools/docker_build.ps1 @@ -0,0 +1,98 @@ +$current_dir = Get-Location +$script_dir = Split-Path -Path $MyInvocation.MyCommand.Definition -Parent +$repo_root = (Get-Item $script_dir).parent.FullName + +$env:PSModulePath = $env:PSModulePath + ";$($repo_root)\tools\modules\powershell" + +function Exit-WithCode($exitcode) { + # Only exit this host process if it's a child of another PowerShell parent process... + $parentPID = (Get-CimInstance -ClassName Win32_Process -Filter "ProcessId=$PID" | Select-Object -Property ParentProcessId).ParentProcessId + $parentProcName = (Get-CimInstance -ClassName Win32_Process -Filter "ProcessId=$parentPID" | Select-Object -Property Name).Name + if ('powershell.exe' -eq $parentProcName) { $host.SetShouldExit($exitcode) } + + exit $exitcode +} + +function Restore-Cwd() { + $tmp_current_dir = Get-Location + if ("$tmp_current_dir" -ne "$current_dir") { + Write-Color -Text ">>> ", "Restoring current directory" -Color Green, Gray + Set-Location -Path $current_dir + } +} + +function Get-Container { + if (-not (Test-Path -PathType Leaf -Path "$($repo_root)\build\docker-image.id")) { + Write-Color -Text "!!! ", "Docker command failed, cannot find image id." -Color Red, Yellow + Restore-Cwd + Exit-WithCode 1 + } + $id = Get-Content "$($repo_root)\build\docker-image.id" + Write-Color -Text ">>> ", "Creating container from image id ", "[", $id, "]" -Color Green, Gray, White, Cyan, White + $cid = docker create $id bash + if ($LASTEXITCODE -ne 0) { + Write-Color -Text "!!! ", "Cannot create container." -Color Red, Yellow + Restore-Cwd + Exit-WithCode 1 + } + return $cid +} + +function Change-Cwd() { + Set-Location -Path $repo_root +} + +function New-DockerBuild { + $version_file = Get-Content -Path "$($repo_root)\openpype\version.py" + $result = [regex]::Matches($version_file, '__version__ = "(?\d+\.\d+.\d+.*)"') + $openpype_version = $result[0].Groups['version'].Value + $startTime = [int][double]::Parse((Get-Date -UFormat %s)) + Write-Color -Text ">>> ", "Building OpenPype using Docker ..." -Color Green, Gray, White + $variant = $args[0] + if ($variant.Length -eq 0) { + $dockerfile = "$($repo_root)\Dockerfile" + } else { + $dockerfile = "$( $repo_root )\Dockerfile.$variant" + } + if (-not (Test-Path -PathType Leaf -Path $dockerfile)) { + Write-Color -Text "!!! ", "Dockerfile for specifed platform ", "[", $variant, "]", "doesn't exist." -Color Red, Yellow, Cyan, White, Cyan, Yellow + Restore-Cwd + Exit-WithCode 1 + } + Write-Color -Text ">>> ", "Using Dockerfile for ", "[ ", $variant, " ]" -Color Green, Gray, White, Cyan, White + + $build_dir = "$($repo_root)\build" + if (-not(Test-Path $build_dir)) { + New-Item -ItemType Directory -Path $build_dir + } + Write-Color -Text "--- ", "Cleaning build directory ..." -Color Yellow, Gray + try { + Remove-Item -Recurse -Force "$($build_dir)\*" + } catch { + Write-Color -Text "!!! ", "Cannot clean build directory, possibly because process is using it." -Color Red, Gray + Write-Color -Text $_.Exception.Message -Color Red + Exit-WithCode 1 + } + + Write-Color -Text ">>> ", "Running Docker build ..." -Color Green, Gray, White + docker build --pull --iidfile $repo_root/build/docker-image.id --build-arg BUILD_DATE=$(Get-Date -UFormat %Y-%m-%dT%H:%M:%SZ) --build-arg VERSION=$openpype_version -t pypeclub/openpype:$openpype_version -f $dockerfile . + if ($LASTEXITCODE -ne 0) { + Write-Color -Text "!!! ", "Docker command failed.", $LASTEXITCODE -Color Red, Yellow, Red + Restore-Cwd + Exit-WithCode 1 + } + Write-Color -Text ">>> ", "Copying build from container ..." -Color Green, Gray, White + $cid = Get-Container + + docker cp "$($cid):/opt/openpype/build/exe.linux-x86_64-3.9" "$($repo_root)/build" + docker cp "$($cid):/opt/openpype/build/build.log" "$($repo_root)/build" + + $endTime = [int][double]::Parse((Get-Date -UFormat %s)) + try { + New-BurntToastNotification -AppLogo "$openpype_root/openpype/resources/icons/openpype_icon.png" -Text "OpenPype build complete!", "All done in $( $endTime - $startTime ) secs. You will find OpenPype and build log in build directory." + } catch {} + Write-Color -Text "*** ", "All done in ", $($endTime - $startTime), " secs. You will find OpenPype and build log in ", "'.\build'", " directory." -Color Green, Gray, White, Gray, White, Gray +} + +Change-Cwd +New-DockerBuild $ARGS diff --git a/website/docs/admin_hosts_aftereffects.md b/website/docs/admin_hosts_aftereffects.md index 974428fe06..72fdb32faf 100644 --- a/website/docs/admin_hosts_aftereffects.md +++ b/website/docs/admin_hosts_aftereffects.md @@ -18,6 +18,10 @@ Location: Settings > Project > AfterEffects ## Publish plugins +### Collect Review + +Enable/disable creation of auto instance of review. + ### Validate Scene Settings #### Skip Resolution Check for Tasks @@ -28,6 +32,10 @@ Set regex pattern(s) to look for in a Task name to skip resolution check against Set regex pattern(s) to look for in a Task name to skip `frameStart`, `frameEnd` check against values from DB. +### ValidateContainers + +By default this validator will look loaded items with lower version than latest. This validator is context wide so it must be disabled in Context button. + ### AfterEffects Submit to Deadline * `Use Published scene` - Set to True (green) when Deadline should take published scene as a source instead of uploaded local one. diff --git a/website/docs/admin_hosts_photoshop.md b/website/docs/admin_hosts_photoshop.md index de684f01d2..d79789760e 100644 --- a/website/docs/admin_hosts_photoshop.md +++ b/website/docs/admin_hosts_photoshop.md @@ -33,7 +33,6 @@ Provides list of [variants](artist_concepts.md#variant) that will be shown to an Provides simplified publishing process. It will create single `image` instance for artist automatically. This instance will produce flatten image from all visible layers in a workfile. -- Subset template for flatten image - provide template for subset name for this instance (example `imageBeauty`) - Review - should be separate review created for this instance ### Create Review @@ -111,11 +110,11 @@ Set Byte limit for review file. Applicable if gigantic `image` instances are pro #### Extract jpg Options -Handles tags for produced `.jpg` representation. `Create review` and `Add review to Ftrack` are defaults. +Handles tags for produced `.jpg` representation. `Create review` and `Add review to Ftrack` are defaults. #### Extract mov Options -Handles tags for produced `.mov` representation. `Create review` and `Add review to Ftrack` are defaults. +Handles tags for produced `.mov` representation. `Create review` and `Add review to Ftrack` are defaults. ### Workfile Builder @@ -124,4 +123,4 @@ Allows to open prepared workfile for an artist when no workfile exists. Useful t Could be configured per `Task type`, eg. `composition` task type could use different `.psd` template file than `art` task. Workfile template must be accessible for all artists. -(Currently not handled by [SiteSync](module_site_sync.md)) \ No newline at end of file +(Currently not handled by [SiteSync](module_site_sync.md)) diff --git a/website/docs/admin_settings_local.md b/website/docs/admin_settings_local.md index b254beb53b..8935b29fb5 100644 --- a/website/docs/admin_settings_local.md +++ b/website/docs/admin_settings_local.md @@ -16,13 +16,19 @@ OpenPype stores some of it's settings and configuration in local file system. Th ## Categories ### OpenPype Mongo URL +The **Mongo URL** is the database URL given by your Studio. More details [here](artist_getting_started.md#mongodb). ### General +**OpenPype Username** : enter your username (if not provided, it uses computer session username by default). This username is used to sign your actions on **OpenPype**, for example the "author" on a publish. +**Admin permissions** : When enabled you do not need to enter a password (if defined in Studio Settings) to access to the **Admin** section. ### Experimental tools +Future version of existing tools or new ones. +### Environments +Local replacement of the environment data of each software and additional internal data necessary to be loaded correctly. ### Applications +Local override of software executable paths for each version. More details [here](admin_settings_system.md#applications). ### Project Settings - - +The **Project Settings** allows to determine the root folder. More details [here](module_site_sync.md#local-settings). diff --git a/website/docs/assets/admin_hosts_photoshop_settings.png b/website/docs/assets/admin_hosts_photoshop_settings.png index aaa6ecbed7..9478fbedf7 100644 Binary files a/website/docs/assets/admin_hosts_photoshop_settings.png and b/website/docs/assets/admin_hosts_photoshop_settings.png differ diff --git a/website/docs/assets/settings/settings_local.png b/website/docs/assets/settings/settings_local.png index d2cf1c920d..725c332747 100644 Binary files a/website/docs/assets/settings/settings_local.png and b/website/docs/assets/settings/settings_local.png differ diff --git a/website/docs/project_settings/settings_project_global.md b/website/docs/project_settings/settings_project_global.md index 5ddf247d98..e0481a8717 100644 --- a/website/docs/project_settings/settings_project_global.md +++ b/website/docs/project_settings/settings_project_global.md @@ -192,7 +192,7 @@ A profile may generate multiple outputs from a single input. Each output must de - Nuke extractor settings path: `project_settings/nuke/publish/ExtractReviewDataMov/outputs/baking/add_custom_tags` - Filtering by input length. Input may be video, sequence or single image. It is possible that `.mp4` should be created only when input is video or sequence and to create review `.png` when input is single frame. In some cases the output should be created even if it's single frame or multi frame input. - + ### Extract Burnin Plugin is responsible for adding burnins into review representations. @@ -226,13 +226,13 @@ A burnin profile may set multiple burnin outputs from one input. The burnin's na | **Bottom Centered** | Bottom center content. | str | "{username}" | | **Bottom Right** | Bottom right corner content. | str | "{frame_start}-{current_frame}-{frame_end}" | -Each burnin profile can be configured with additional family filtering and can -add additional tags to the burnin representation, these can be configured under +Each burnin profile can be configured with additional family filtering and can +add additional tags to the burnin representation, these can be configured under the profile's **Additional filtering** section. :::note Filename suffix -The filename suffix is appended to filename of the source representation. For -example, if the source representation has suffix **"h264"** and the burnin +The filename suffix is appended to filename of the source representation. For +example, if the source representation has suffix **"h264"** and the burnin suffix is **"client"** then the final suffix is **"h264_client"**. ::: @@ -343,6 +343,10 @@ One of the key advantages of this feature is that it allows users to choose the In some cases, these DCCs (Nuke, Houdini, Maya) automatically add a rendering path during the creation stage, which is then used in publishing. Creators and extractors of such DCCs need to use these profiles to fill paths in DCC's nodes to use this functionality. +:::note +Maya's setting `project_settings/maya/RenderSettings/default_render_image_folder` is be overwritten by the custom staging dir. +::: + The custom staging folder uses a path template configured in `project_anatomy/templates/others` with `transient` being a default example path that could be used. The template requires a 'folder' key for it to be usable as custom staging folder. ##### Known issues