diff --git a/CHANGELOG.md b/CHANGELOG.md index 0b69a8e2d5..5e76d7b76a 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -1,28 +1,36 @@ # Changelog -## [3.2.0-nightly.5](https://github.com/pypeclub/OpenPype/tree/HEAD) +## [3.2.0-nightly.7](https://github.com/pypeclub/OpenPype/tree/HEAD) [Full Changelog](https://github.com/pypeclub/OpenPype/compare/2.18.4...HEAD) **🚀 Enhancements** -- Settings UI copy/paste [\#1769](https://github.com/pypeclub/OpenPype/pull/1769) +- Nuke: ftrack family plugin settings preset [\#1805](https://github.com/pypeclub/OpenPype/pull/1805) +- Standalone publisher last project [\#1799](https://github.com/pypeclub/OpenPype/pull/1799) +- Ftrack Multiple notes as server action [\#1795](https://github.com/pypeclub/OpenPype/pull/1795) +- Settings conditional dict [\#1777](https://github.com/pypeclub/OpenPype/pull/1777) +- Settings application use python 2 only where needed [\#1776](https://github.com/pypeclub/OpenPype/pull/1776) - Workfile tool widths [\#1766](https://github.com/pypeclub/OpenPype/pull/1766) - Push hierarchical attributes care about task parent changes [\#1763](https://github.com/pypeclub/OpenPype/pull/1763) - Application executables with environment variables [\#1757](https://github.com/pypeclub/OpenPype/pull/1757) +- Deadline: Nuke submission additional attributes [\#1756](https://github.com/pypeclub/OpenPype/pull/1756) +- Settings schema without prefill [\#1753](https://github.com/pypeclub/OpenPype/pull/1753) - Settings Hosts enum [\#1739](https://github.com/pypeclub/OpenPype/pull/1739) - Validate containers settings [\#1736](https://github.com/pypeclub/OpenPype/pull/1736) - PS - added loader from sequence [\#1726](https://github.com/pypeclub/OpenPype/pull/1726) -- Autoupdate launcher [\#1725](https://github.com/pypeclub/OpenPype/pull/1725) -- Subset template and TVPaint subset template docs [\#1717](https://github.com/pypeclub/OpenPype/pull/1717) - Toggle Ftrack upload in StandalonePublisher [\#1708](https://github.com/pypeclub/OpenPype/pull/1708) -- Overscan color extract review [\#1701](https://github.com/pypeclub/OpenPype/pull/1701) -- Nuke: Prerender Frame Range by default [\#1699](https://github.com/pypeclub/OpenPype/pull/1699) -- Smoother edges of color triangle [\#1695](https://github.com/pypeclub/OpenPype/pull/1695) **🐛 Bug fixes** +- nuke: fixing wrong name of family folder when `used existing frames` [\#1803](https://github.com/pypeclub/OpenPype/pull/1803) +- Collect ftrack family bugs [\#1801](https://github.com/pypeclub/OpenPype/pull/1801) +- Invitee email can be None which break the Ftrack commit. [\#1788](https://github.com/pypeclub/OpenPype/pull/1788) +- Fix: staging and `--use-version` option [\#1786](https://github.com/pypeclub/OpenPype/pull/1786) +- Otio unrelated error on import [\#1782](https://github.com/pypeclub/OpenPype/pull/1782) - FFprobe streams order [\#1775](https://github.com/pypeclub/OpenPype/pull/1775) +- Fix - single file files are str only, cast it to list to count properly [\#1772](https://github.com/pypeclub/OpenPype/pull/1772) +- Environments in app executable for MacOS [\#1768](https://github.com/pypeclub/OpenPype/pull/1768) - Project specific environments [\#1767](https://github.com/pypeclub/OpenPype/pull/1767) - Settings UI with refresh button [\#1764](https://github.com/pypeclub/OpenPype/pull/1764) - Standalone publisher thumbnail extractor fix [\#1761](https://github.com/pypeclub/OpenPype/pull/1761) @@ -31,22 +39,18 @@ - hiero: precollect instances failing when audio selected [\#1743](https://github.com/pypeclub/OpenPype/pull/1743) - Hiero: creator instance error [\#1742](https://github.com/pypeclub/OpenPype/pull/1742) - Nuke: fixing render creator for no selection format failing [\#1741](https://github.com/pypeclub/OpenPype/pull/1741) +- StandalonePublisher: failing collector for editorial [\#1738](https://github.com/pypeclub/OpenPype/pull/1738) - Local settings UI crash on missing defaults [\#1737](https://github.com/pypeclub/OpenPype/pull/1737) - TVPaint white background on thumbnail [\#1735](https://github.com/pypeclub/OpenPype/pull/1735) -- Ftrack missing custom attribute message [\#1734](https://github.com/pypeclub/OpenPype/pull/1734) -- Launcher project changes [\#1733](https://github.com/pypeclub/OpenPype/pull/1733) -- Ftrack sync status [\#1732](https://github.com/pypeclub/OpenPype/pull/1732) -- TVPaint use layer name for default variant [\#1724](https://github.com/pypeclub/OpenPype/pull/1724) -- Default subset template for TVPaint review and workfile families [\#1716](https://github.com/pypeclub/OpenPype/pull/1716) -- Maya: Extract review hotfix [\#1714](https://github.com/pypeclub/OpenPype/pull/1714) -- Settings: Imageio improving granularity [\#1711](https://github.com/pypeclub/OpenPype/pull/1711) - Application without executables [\#1679](https://github.com/pypeclub/OpenPype/pull/1679) +- Unreal: launching on Linux [\#1672](https://github.com/pypeclub/OpenPype/pull/1672) **Merged pull requests:** +- Build: don't add Poetry to `PATH` [\#1808](https://github.com/pypeclub/OpenPype/pull/1808) - Bump prismjs from 1.23.0 to 1.24.0 in /website [\#1773](https://github.com/pypeclub/OpenPype/pull/1773) +- Bc/fix/docs [\#1771](https://github.com/pypeclub/OpenPype/pull/1771) - TVPaint ftrack family [\#1755](https://github.com/pypeclub/OpenPype/pull/1755) -- global: removing obsolete ftrack validator plugin [\#1710](https://github.com/pypeclub/OpenPype/pull/1710) ## [2.18.4](https://github.com/pypeclub/OpenPype/tree/2.18.4) (2021-06-24) @@ -65,18 +69,17 @@ - Tools names forwards compatibility [\#1727](https://github.com/pypeclub/OpenPype/pull/1727) +**Merged pull requests:** + +- global: removing obsolete ftrack validator plugin [\#1710](https://github.com/pypeclub/OpenPype/pull/1710) + ## [2.18.2](https://github.com/pypeclub/OpenPype/tree/2.18.2) (2021-06-16) [Full Changelog](https://github.com/pypeclub/OpenPype/compare/3.1.0...2.18.2) -**🚀 Enhancements** - -- StandalonePublisher: adding exception for adding `delete` tag to repre [\#1650](https://github.com/pypeclub/OpenPype/pull/1650) - **🐛 Bug fixes** - Maya: Extract review hotfix - 2.x backport [\#1713](https://github.com/pypeclub/OpenPype/pull/1713) -- StandalonePublisher: instance data attribute `keepSequence` [\#1668](https://github.com/pypeclub/OpenPype/pull/1668) **Merged pull requests:** @@ -94,14 +97,6 @@ - Sort applications and tools alphabetically in Settings UI [\#1689](https://github.com/pypeclub/OpenPype/pull/1689) - \#683 - Validate Frame Range in Standalone Publisher [\#1683](https://github.com/pypeclub/OpenPype/pull/1683) - Hiero: old container versions identify with red color [\#1682](https://github.com/pypeclub/OpenPype/pull/1682) -- Project Manger: Default name column width [\#1669](https://github.com/pypeclub/OpenPype/pull/1669) -- Remove outline in stylesheet [\#1667](https://github.com/pypeclub/OpenPype/pull/1667) -- TVPaint: Creator take layer name as default value for subset variant [\#1663](https://github.com/pypeclub/OpenPype/pull/1663) -- TVPaint custom subset template [\#1662](https://github.com/pypeclub/OpenPype/pull/1662) -- Editorial: conform assets validator [\#1659](https://github.com/pypeclub/OpenPype/pull/1659) -- Feature Slack integration [\#1657](https://github.com/pypeclub/OpenPype/pull/1657) -- Nuke - Publish simplification [\#1653](https://github.com/pypeclub/OpenPype/pull/1653) -- \#1333 - added tooltip hints to Pyblish buttons [\#1649](https://github.com/pypeclub/OpenPype/pull/1649) **🐛 Bug fixes** @@ -111,12 +106,6 @@ - Hiero: published whole edit mov [\#1687](https://github.com/pypeclub/OpenPype/pull/1687) - Ftrack subprocess handle of stdout/stderr [\#1675](https://github.com/pypeclub/OpenPype/pull/1675) - Settings list race condifiton and mutable dict list conversion [\#1671](https://github.com/pypeclub/OpenPype/pull/1671) -- Mac launch arguments fix [\#1660](https://github.com/pypeclub/OpenPype/pull/1660) -- Fix missing dbm python module [\#1652](https://github.com/pypeclub/OpenPype/pull/1652) -- Transparent branches in view on Mac [\#1648](https://github.com/pypeclub/OpenPype/pull/1648) -- Add asset on task item [\#1646](https://github.com/pypeclub/OpenPype/pull/1646) -- Project manager save and queue [\#1645](https://github.com/pypeclub/OpenPype/pull/1645) -- New project anatomy values [\#1644](https://github.com/pypeclub/OpenPype/pull/1644) **Merged pull requests:** diff --git a/igniter/bootstrap_repos.py b/igniter/bootstrap_repos.py index 6eaea27116..8c081b8614 100644 --- a/igniter/bootstrap_repos.py +++ b/igniter/bootstrap_repos.py @@ -657,7 +657,7 @@ class BootstrapRepos: ] # remove duplicates - openpype_versions = list(set(openpype_versions)) + openpype_versions = sorted(list(set(openpype_versions))) return openpype_versions diff --git a/openpype/cli.py b/openpype/cli.py index 48951c7287..ec5b04c468 100644 --- a/openpype/cli.py +++ b/openpype/cli.py @@ -15,6 +15,9 @@ from .pype_commands import PypeCommands expose_value=False, help="use specified version") @click.option("--use-staging", is_flag=True, expose_value=False, help="use staging variants") +@click.option("--list-versions", is_flag=True, expose_value=False, + help=("list all detected versions. Use With `--use-staging " + "to list staging versions.")) def main(ctx): """Pype is main command serving as entry point to pipeline system. diff --git a/openpype/hosts/houdini/plugins/publish/collect_instances.py b/openpype/hosts/houdini/plugins/publish/collect_instances.py index 413553c864..2e294face2 100644 --- a/openpype/hosts/houdini/plugins/publish/collect_instances.py +++ b/openpype/hosts/houdini/plugins/publish/collect_instances.py @@ -56,7 +56,7 @@ class CollectInstances(pyblish.api.ContextPlugin): # Create nice name if the instance has a frame range. label = data.get("name", node.name()) if "frameStart" in data and "frameEnd" in data: - frames = "[{startFrame} - {endFrame}]".format(**data) + frames = "[{frameStart} - {frameEnd}]".format(**data) label = "{} {}".format(label, frames) instance = context.create_instance(label) diff --git a/openpype/hosts/nuke/plugins/publish/precollect_instances.py b/openpype/hosts/nuke/plugins/publish/precollect_instances.py index 00d96c6cd1..c2c25d0627 100644 --- a/openpype/hosts/nuke/plugins/publish/precollect_instances.py +++ b/openpype/hosts/nuke/plugins/publish/precollect_instances.py @@ -70,8 +70,9 @@ class PreCollectNukeInstances(pyblish.api.ContextPlugin): review = False if "review" in node.knobs(): review = node["review"].value() + + if review: families.append("review") - families.append("ftrack") # Add all nodes in group instances. if node.Class() == "Group": @@ -81,18 +82,18 @@ class PreCollectNukeInstances(pyblish.api.ContextPlugin): if target == "Use existing frames": # Local rendering self.log.info("flagged for no render") - families.append(family) + families.append(families_ak.lower()) elif target == "Local": # Local rendering self.log.info("flagged for local render") families.append("{}.local".format(family)) + family = families_ak.lower() elif target == "On farm": # Farm rendering self.log.info("flagged for farm render") instance.data["transfer"] = False families.append("{}.farm".format(family)) - - family = families_ak.lower() + family = families_ak.lower() node.begin() for i in nuke.allNodes(): diff --git a/openpype/hosts/standalonepublisher/plugins/publish/collect_instances.py b/openpype/hosts/standalonepublisher/plugins/publish/collect_editorial_instances.py similarity index 96% rename from openpype/hosts/standalonepublisher/plugins/publish/collect_instances.py rename to openpype/hosts/standalonepublisher/plugins/publish/collect_editorial_instances.py index d753a3d9bb..3474cbcdde 100644 --- a/openpype/hosts/standalonepublisher/plugins/publish/collect_instances.py +++ b/openpype/hosts/standalonepublisher/plugins/publish/collect_editorial_instances.py @@ -8,7 +8,7 @@ class CollectInstances(pyblish.api.InstancePlugin): """Collect instances from editorial's OTIO sequence""" order = pyblish.api.CollectorOrder + 0.01 - label = "Collect Instances" + label = "Collect Editorial Instances" hosts = ["standalonepublisher"] families = ["editorial"] @@ -55,7 +55,7 @@ class CollectInstances(pyblish.api.InstancePlugin): fps = plib.get_asset()["data"]["fps"] tracks = timeline.each_child( - descended_from_type=otio.schema.track.Track + descended_from_type=otio.schema.Track ) # get data from avalon @@ -84,6 +84,9 @@ class CollectInstances(pyblish.api.InstancePlugin): if clip.name is None: continue + if isinstance(clip, otio.schema.Gap): + continue + # skip all generators like black ampty if isinstance( clip.media_reference, @@ -92,7 +95,7 @@ class CollectInstances(pyblish.api.InstancePlugin): # Transitions are ignored, because Clips have the full frame # range. - if isinstance(clip, otio.schema.transition.Transition): + if isinstance(clip, otio.schema.Transition): continue # basic unique asset name diff --git a/openpype/hosts/standalonepublisher/plugins/publish/collect_instance_resources.py b/openpype/hosts/standalonepublisher/plugins/publish/collect_editorial_resources.py similarity index 99% rename from openpype/hosts/standalonepublisher/plugins/publish/collect_instance_resources.py rename to openpype/hosts/standalonepublisher/plugins/publish/collect_editorial_resources.py index 565d066fd8..e262009637 100644 --- a/openpype/hosts/standalonepublisher/plugins/publish/collect_instance_resources.py +++ b/openpype/hosts/standalonepublisher/plugins/publish/collect_editorial_resources.py @@ -11,7 +11,7 @@ class CollectInstanceResources(pyblish.api.InstancePlugin): # must be after `CollectInstances` order = pyblish.api.CollectorOrder + 0.011 - label = "Collect Instance Resources" + label = "Collect Editorial Resources" hosts = ["standalonepublisher"] families = ["clip"] diff --git a/openpype/hosts/unreal/README.md b/openpype/hosts/unreal/README.md new file mode 100644 index 0000000000..0a69b9e0cf --- /dev/null +++ b/openpype/hosts/unreal/README.md @@ -0,0 +1,9 @@ +## Unreal Integration + +Supported Unreal Engine version is 4.26+ (mainly because of major Python changes done there). + +### Project naming +Unreal doesn't support project names starting with non-alphabetic character. So names like `123_myProject` are +invalid. If OpenPype detects such name it automatically prepends letter **P** to make it valid name, so `123_myProject` +will become `P123_myProject`. There is also soft-limit on project name length to be shorter than 20 characters. +Longer names will issue warning in Unreal Editor that there might be possible side effects. \ No newline at end of file diff --git a/openpype/hosts/unreal/api/lib.py b/openpype/hosts/unreal/api/lib.py index 7e706a2789..7e34c3ff15 100644 --- a/openpype/hosts/unreal/api/lib.py +++ b/openpype/hosts/unreal/api/lib.py @@ -1,38 +1,51 @@ +# -*- coding: utf-8 -*- +"""Unreal launching and project tools.""" import sys import os import platform import json from distutils import dir_util import subprocess +import re +from pathlib import Path +from collections import OrderedDict from openpype.api import get_project_settings -def get_engine_versions(): - """ +def get_engine_versions(env=None): + """Detect Unreal Engine versions. + This will try to detect location and versions of installed Unreal Engine. Location can be overridden by `UNREAL_ENGINE_LOCATION` environment variable. - Returns: + Args: + env (dict, optional): Environment to use. - dict: dictionary with version as a key and dir as value. + Returns: + OrderedDict: dictionary with version as a key and dir as value. + so the highest version is first. Example: - - >>> get_engine_version() + >>> get_engine_versions() { "4.23": "C:/Epic Games/UE_4.23", "4.24": "C:/Epic Games/UE_4.24" } - """ - try: - engine_locations = {} - root, dirs, files = next(os.walk(os.environ["UNREAL_ENGINE_LOCATION"])) - for dir in dirs: - if dir.startswith("UE_"): - ver = dir.split("_")[1] - engine_locations[ver] = os.path.join(root, dir) + """ + env = env or os.environ + engine_locations = {} + try: + root, dirs, _ = next(os.walk(env["UNREAL_ENGINE_LOCATION"])) + + for directory in dirs: + if directory.startswith("UE"): + try: + ver = re.split(r"[-_]", directory)[1] + except IndexError: + continue + engine_locations[ver] = os.path.join(root, directory) except KeyError: # environment variable not set pass @@ -40,32 +53,52 @@ def get_engine_versions(): # specified directory doesn't exists pass - # if we've got something, terminate autodetection process + # if we've got something, terminate auto-detection process if engine_locations: - return engine_locations + return OrderedDict(sorted(engine_locations.items())) # else kick in platform specific detection if platform.system().lower() == "windows": - return _win_get_engine_versions() - elif platform.system().lower() == "linux": + return OrderedDict(sorted(_win_get_engine_versions().items())) + if platform.system().lower() == "linux": # on linux, there is no installation and getting Unreal Engine involves # git clone. So we'll probably depend on `UNREAL_ENGINE_LOCATION`. pass - elif platform.system().lower() == "darwin": - return _darwin_get_engine_version() + if platform.system().lower() == "darwin": + return OrderedDict(sorted(_darwin_get_engine_version().items())) - return {} + return OrderedDict() + + +def get_editor_executable_path(engine_path: Path) -> Path: + """Get UE4 Editor executable path.""" + ue4_path = engine_path / "Engine/Binaries" + if platform.system().lower() == "windows": + ue4_path /= "Win64/UE4Editor.exe" + + elif platform.system().lower() == "linux": + ue4_path /= "Linux/UE4Editor" + + elif platform.system().lower() == "darwin": + ue4_path /= "Mac/UE4Editor" + + return ue4_path def _win_get_engine_versions(): - """ + """Get Unreal Engine versions on Windows. + If engines are installed via Epic Games Launcher then there is: `%PROGRAMDATA%/Epic/UnrealEngineLauncher/LauncherInstalled.dat` This file is JSON file listing installed stuff, Unreal engines are marked with `"AppName" = "UE_X.XX"`` like `UE_4.24` + + Returns: + dict: version as a key and path as a value. + """ install_json_path = os.path.join( - os.environ.get("PROGRAMDATA"), + os.getenv("PROGRAMDATA"), "Epic", "UnrealEngineLauncher", "LauncherInstalled.dat", @@ -75,11 +108,19 @@ def _win_get_engine_versions(): def _darwin_get_engine_version() -> dict: - """ + """Get Unreal Engine versions on MacOS. + It works the same as on Windows, just JSON file location is different. + + Returns: + dict: version as a key and path as a value. + + See Aslo: + :func:`_win_get_engine_versions`. + """ install_json_path = os.path.join( - os.environ.get("HOME"), + os.getenv("HOME"), "Library", "Application Support", "Epic", @@ -91,25 +132,26 @@ def _darwin_get_engine_version() -> dict: def _parse_launcher_locations(install_json_path: str) -> dict: - """ - This will parse locations from json file. + """This will parse locations from json file. + + Args: + install_json_path (str): Path to `LauncherInstalled.dat`. + + Returns: + dict: with unreal engine versions as keys and + paths to those engine installations as value. - :param install_json_path: path to `LauncherInstalled.dat` - :type install_json_path: str - :returns: returns dict with unreal engine versions as keys and - paths to those engine installations as value. - :rtype: dict """ engine_locations = {} if os.path.isfile(install_json_path): with open(install_json_path, "r") as ilf: try: install_data = json.load(ilf) - except json.JSONDecodeError: + except json.JSONDecodeError as e: raise Exception( "Invalid `LauncherInstalled.dat file. `" "Cannot determine Unreal Engine location." - ) + ) from e for installation in install_data.get("InstallationList", []): if installation.get("AppName").startswith("UE_"): @@ -121,55 +163,91 @@ def _parse_launcher_locations(install_json_path: str) -> dict: def create_unreal_project(project_name: str, ue_version: str, - pr_dir: str, - engine_path: str, - dev_mode: bool = False) -> None: - """ - This will create `.uproject` file at specified location. As there is no - way I know to create project via command line, this is easiest option. - Unreal project file is basically JSON file. If we find + pr_dir: Path, + engine_path: Path, + dev_mode: bool = False, + env: dict = None) -> None: + """This will create `.uproject` file at specified location. + + As there is no way I know to create project via command line, this is + easiest option. Unreal project file is basically JSON file. If we find `AVALON_UNREAL_PLUGIN` environment variable we assume this is location of Avalon Integration Plugin and we copy its content to project folder and enable this plugin. - :param project_name: project name - :type project_name: str - :param ue_version: unreal engine version (like 4.23) - :type ue_version: str - :param pr_dir: path to directory where project will be created - :type pr_dir: str - :param engine_path: Path to Unreal Engine installation - :type engine_path: str - :param dev_mode: Flag to trigger C++ style Unreal project needing - Visual Studio and other tools to compile plugins from - sources. This will trigger automatically if `Binaries` - directory is not found in plugin folders as this indicates - this is only source distribution of the plugin. Dev mode - is also set by preset file `unreal/project_setup.json` in - **OPENPYPE_CONFIG**. - :type dev_mode: bool - :returns: None - """ - preset = get_project_settings(project_name)["unreal"]["project_setup"] + Args: + project_name (str): Name of the project. + ue_version (str): Unreal engine version (like 4.23). + pr_dir (Path): Path to directory where project will be created. + engine_path (Path): Path to Unreal Engine installation. + dev_mode (bool, optional): Flag to trigger C++ style Unreal project + needing Visual Studio and other tools to compile plugins from + sources. This will trigger automatically if `Binaries` + directory is not found in plugin folders as this indicates + this is only source distribution of the plugin. Dev mode + is also set by preset file `unreal/project_setup.json` in + **OPENPYPE_CONFIG**. + env (dict, optional): Environment to use. If not set, `os.environ`. - if os.path.isdir(os.environ.get("AVALON_UNREAL_PLUGIN", "")): + Throws: + NotImplementedError: For unsupported platforms. + + Returns: + None + + """ + env = env or os.environ + preset = get_project_settings(project_name)["unreal"]["project_setup"] + ue_id = ".".join(ue_version.split(".")[:2]) + # get unreal engine identifier + # ------------------------------------------------------------------------- + # FIXME (antirotor): As of 4.26 this is problem with UE4 built from + # sources. In that case Engine ID is calculated per machine/user and not + # from Engine files as this code then reads. This then prevents UE4 + # to directly open project as it will complain about project being + # created in different UE4 version. When user convert such project + # to his UE4 version, Engine ID is replaced in uproject file. If some + # other user tries to open it, it will present him with similar error. + ue4_modules = Path() + if platform.system().lower() == "windows": + ue4_modules = Path(os.path.join(engine_path, "Engine", "Binaries", + "Win64", "UE4Editor.modules")) + + if platform.system().lower() == "linux": + ue4_modules = Path(os.path.join(engine_path, "Engine", "Binaries", + "Linux", "UE4Editor.modules")) + + if platform.system().lower() == "darwin": + ue4_modules = Path(os.path.join(engine_path, "Engine", "Binaries", + "Mac", "UE4Editor.modules")) + + if ue4_modules.exists(): + print("--- Loading Engine ID from modules file ...") + with open(ue4_modules, "r") as mp: + loaded_modules = json.load(mp) + + if loaded_modules.get("BuildId"): + ue_id = "{" + loaded_modules.get("BuildId") + "}" + + plugins_path = None + if os.path.isdir(env.get("AVALON_UNREAL_PLUGIN", "")): # copy plugin to correct path under project - plugins_path = os.path.join(pr_dir, "Plugins") - avalon_plugin_path = os.path.join(plugins_path, "Avalon") - if not os.path.isdir(avalon_plugin_path): - os.makedirs(avalon_plugin_path, exist_ok=True) + plugins_path = pr_dir / "Plugins" + avalon_plugin_path = plugins_path / "Avalon" + if not avalon_plugin_path.is_dir(): + avalon_plugin_path.mkdir(parents=True, exist_ok=True) dir_util._path_created = {} dir_util.copy_tree(os.environ.get("AVALON_UNREAL_PLUGIN"), - avalon_plugin_path) + avalon_plugin_path.as_posix()) - if (not os.path.isdir(os.path.join(avalon_plugin_path, "Binaries")) - or not os.path.join(avalon_plugin_path, "Intermediate")): + if not (avalon_plugin_path / "Binaries").is_dir() \ + or not (avalon_plugin_path / "Intermediate").is_dir(): dev_mode = True # data for project file data = { "FileVersion": 3, - "EngineAssociation": ue_version, + "EngineAssociation": ue_id, "Category": "", "Description": "", "Plugins": [ @@ -179,35 +257,6 @@ def create_unreal_project(project_name: str, ] } - if preset["install_unreal_python_engine"]: - # If `OPENPYPE_UNREAL_ENGINE_PYTHON_PLUGIN` is set, copy it from there - # to support offline installation. - # Otherwise clone UnrealEnginePython to Plugins directory - # https://github.com/20tab/UnrealEnginePython.git - uep_path = os.path.join(plugins_path, "UnrealEnginePython") - if os.environ.get("OPENPYPE_UNREAL_ENGINE_PYTHON_PLUGIN"): - - os.makedirs(uep_path, exist_ok=True) - dir_util._path_created = {} - dir_util.copy_tree( - os.environ.get("OPENPYPE_UNREAL_ENGINE_PYTHON_PLUGIN"), - uep_path) - else: - # WARNING: this will trigger dev_mode, because we need to compile - # this plugin. - dev_mode = True - import git - git.Repo.clone_from( - "https://github.com/20tab/UnrealEnginePython.git", - uep_path) - - data["Plugins"].append( - {"Name": "UnrealEnginePython", "Enabled": True}) - - if (not os.path.isdir(os.path.join(uep_path, "Binaries")) - or not os.path.join(uep_path, "Intermediate")): - dev_mode = True - if dev_mode or preset["dev_mode"]: # this will add project module and necessary source file to make it # C++ project and to (hopefully) make Unreal Editor to compile all @@ -220,51 +269,39 @@ def create_unreal_project(project_name: str, "AdditionalDependencies": ["Engine"], }] - if preset["install_unreal_python_engine"]: - # now we need to fix python path in: - # `UnrealEnginePython.Build.cs` - # to point to our python - with open(os.path.join( - uep_path, "Source", - "UnrealEnginePython", - "UnrealEnginePython.Build.cs"), mode="r") as f: - build_file = f.read() - - fix = build_file.replace( - 'private string pythonHome = "";', - 'private string pythonHome = "{}";'.format( - sys.base_prefix.replace("\\", "/"))) - - with open(os.path.join( - uep_path, "Source", - "UnrealEnginePython", - "UnrealEnginePython.Build.cs"), mode="w") as f: - f.write(fix) - # write project file - project_file = os.path.join(pr_dir, "{}.uproject".format(project_name)) + project_file = pr_dir / f"{project_name}.uproject" with open(project_file, mode="w") as pf: json.dump(data, pf, indent=4) - # UE < 4.26 have Python2 by default, so we need PySide - # but we will not need it in 4.26 and up - if int(ue_version.split(".")[1]) < 26: - # ensure we have PySide installed in engine - # TODO: make it work for other platforms 🍎 🐧 - if platform.system().lower() == "windows": - python_path = os.path.join(engine_path, "Engine", "Binaries", - "ThirdParty", "Python", "Win64", - "python.exe") + # ensure we have PySide2 installed in engine + python_path = None + if platform.system().lower() == "windows": + python_path = engine_path / ("Engine/Binaries/ThirdParty/" + "Python3/Win64/pythonw.exe") - subprocess.run([python_path, "-m", - "pip", "install", "pyside"]) + if platform.system().lower() == "linux": + python_path = engine_path / ("Engine/Binaries/ThirdParty/" + "Python3/Linux/bin/python3") + + if platform.system().lower() == "darwin": + python_path = engine_path / ("Engine/Binaries/ThirdParty/" + "Python3/Mac/bin/python3") + + if not python_path: + raise NotImplementedError("Unsupported platform") + if not python_path.exists(): + raise RuntimeError(f"Unreal Python not found at {python_path}") + subprocess.run( + [python_path.as_posix(), "-m", "pip", "install", "pyside2"]) if dev_mode or preset["dev_mode"]: _prepare_cpp_project(project_file, engine_path) -def _prepare_cpp_project(project_file: str, engine_path: str) -> None: - """ +def _prepare_cpp_project(project_file: Path, engine_path: Path) -> None: + """Prepare CPP Unreal Project. + This function will add source files needed for project to be rebuild along with the avalon integration plugin. @@ -273,19 +310,19 @@ def _prepare_cpp_project(project_file: str, engine_path: str) -> None: by some generator. This needs more research as manually writing those files is rather hackish. :skull_and_crossbones: - :param project_file: path to .uproject file - :type project_file: str - :param engine_path: path to unreal engine associated with project - :type engine_path: str + + Args: + project_file (str): Path to .uproject file. + engine_path (str): Path to unreal engine associated with project. + """ + project_name = project_file.stem + project_dir = project_file.parent + targets_dir = project_dir / "Source" + sources_dir = targets_dir / project_name - project_name = os.path.splitext(os.path.basename(project_file))[0] - project_dir = os.path.dirname(project_file) - targets_dir = os.path.join(project_dir, "Source") - sources_dir = os.path.join(targets_dir, project_name) - - os.makedirs(sources_dir, exist_ok=True) - os.makedirs(os.path.join(project_dir, "Content"), exist_ok=True) + sources_dir.mkdir(parents=True, exist_ok=True) + (project_dir / "Content").mkdir(parents=True, exist_ok=True) module_target = ''' using UnrealBuildTool; @@ -360,59 +397,59 @@ class {1}_API A{0}GameModeBase : public AGameModeBase }}; '''.format(project_name, project_name.upper()) - with open(os.path.join( - targets_dir, f"{project_name}.Target.cs"), mode="w") as f: + with open(targets_dir / f"{project_name}.Target.cs", mode="w") as f: f.write(module_target) - with open(os.path.join( - targets_dir, f"{project_name}Editor.Target.cs"), mode="w") as f: + with open(targets_dir / f"{project_name}Editor.Target.cs", mode="w") as f: f.write(editor_module_target) - with open(os.path.join( - sources_dir, f"{project_name}.Build.cs"), mode="w") as f: + with open(sources_dir / f"{project_name}.Build.cs", mode="w") as f: f.write(module_build) - with open(os.path.join( - sources_dir, f"{project_name}.cpp"), mode="w") as f: + with open(sources_dir / f"{project_name}.cpp", mode="w") as f: f.write(module_cpp) - with open(os.path.join( - sources_dir, f"{project_name}.h"), mode="w") as f: + with open(sources_dir / f"{project_name}.h", mode="w") as f: f.write(module_header) - with open(os.path.join( - sources_dir, f"{project_name}GameModeBase.cpp"), mode="w") as f: + with open(sources_dir / f"{project_name}GameModeBase.cpp", mode="w") as f: f.write(game_mode_cpp) - with open(os.path.join( - sources_dir, f"{project_name}GameModeBase.h"), mode="w") as f: + with open(sources_dir / f"{project_name}GameModeBase.h", mode="w") as f: f.write(game_mode_h) + u_build_tool = Path( + engine_path / "Engine/Binaries/DotNET/UnrealBuildTool.exe") + u_header_tool = None + + arch = "Win64" if platform.system().lower() == "windows": - u_build_tool = (f"{engine_path}/Engine/Binaries/DotNET/" - "UnrealBuildTool.exe") - u_header_tool = (f"{engine_path}/Engine/Binaries/Win64/" - f"UnrealHeaderTool.exe") + arch = "Win64" + u_header_tool = Path( + engine_path / "Engine/Binaries/Win64/UnrealHeaderTool.exe") elif platform.system().lower() == "linux": - # WARNING: there is no UnrealBuildTool on linux? - u_build_tool = "" - u_header_tool = "" + arch = "Linux" + u_header_tool = Path( + engine_path / "Engine/Binaries/Linux/UnrealHeaderTool") elif platform.system().lower() == "darwin": - # WARNING: there is no UnrealBuildTool on Mac? - u_build_tool = "" - u_header_tool = "" + # we need to test this out + arch = "Mac" + u_header_tool = Path( + engine_path / "Engine/Binaries/Mac/UnrealHeaderTool") - u_build_tool = u_build_tool.replace("\\", "/") - u_header_tool = u_header_tool.replace("\\", "/") + if not u_header_tool: + raise NotImplementedError("Unsupported platform") - command1 = [u_build_tool, "-projectfiles", f"-project={project_file}", - "-progress"] + command1 = [u_build_tool.as_posix(), "-projectfiles", + f"-project={project_file}", "-progress"] subprocess.run(command1) - command2 = [u_build_tool, f"-ModuleWithSuffix={project_name},3555" - "Win64", "Development", "-TargetType=Editor" - f'-Project="{project_file}"', f'"{project_file}"' + command2 = [u_build_tool.as_posix(), + f"-ModuleWithSuffix={project_name},3555", arch, + "Development", "-TargetType=Editor", + f'-Project={project_file}', + f'{project_file}', "-IgnoreJunk"] subprocess.run(command2) diff --git a/openpype/hosts/unreal/hooks/pre_workfile_preparation.py b/openpype/hosts/unreal/hooks/pre_workfile_preparation.py index f084cccfc3..01b8b6bc05 100644 --- a/openpype/hosts/unreal/hooks/pre_workfile_preparation.py +++ b/openpype/hosts/unreal/hooks/pre_workfile_preparation.py @@ -1,31 +1,49 @@ +# -*- coding: utf-8 -*- +"""Hook to launch Unreal and prepare projects.""" import os +from pathlib import Path from openpype.lib import ( PreLaunchHook, - ApplicationLaunchFailed + ApplicationLaunchFailed, + ApplicationNotFound ) from openpype.hosts.unreal.api import lib as unreal_lib class UnrealPrelaunchHook(PreLaunchHook): - """ + """Hook to handle launching Unreal. + This hook will check if current workfile path has Unreal project inside. IF not, it initialize it and finally it pass path to the project by environment variable to Unreal launcher shell script. - """ + """ def __init__(self, *args, **kwargs): super().__init__(*args, **kwargs) self.signature = "( {} )".format(self.__class__.__name__) def execute(self): + """Hook entry method.""" asset_name = self.data["asset_name"] task_name = self.data["task_name"] workdir = self.launch_context.env["AVALON_WORKDIR"] engine_version = self.app_name.split("/")[-1].replace("-", ".") unreal_project_name = f"{asset_name}_{task_name}" + try: + if int(engine_version.split(".")[0]) < 4 and \ + int(engine_version.split(".")[1]) < 26: + raise ApplicationLaunchFailed(( + f"{self.signature} Old unsupported version of UE4 " + f"detected - {engine_version}")) + except ValueError: + # there can be string in minor version and in that case + # int cast is failing. This probably happens only with + # early access versions and is of no concert for this check + # so lets keep it quite. + ... # Unreal is sensitive about project names longer then 20 chars if len(unreal_project_name) > 20: @@ -45,19 +63,21 @@ class UnrealPrelaunchHook(PreLaunchHook): )) unreal_project_name = f"P{unreal_project_name}" - project_path = os.path.join(workdir, unreal_project_name) + project_path = Path(os.path.join(workdir, unreal_project_name)) self.log.info(( f"{self.signature} requested UE4 version: " f"[ {engine_version} ]" )) - detected = unreal_lib.get_engine_versions() + detected = unreal_lib.get_engine_versions(self.launch_context.env) detected_str = ', '.join(detected.keys()) or 'none' self.log.info(( f"{self.signature} detected UE4 versions: " f"[ {detected_str} ]" )) + if not detected: + raise ApplicationNotFound("No Unreal Engines are found.") engine_version = ".".join(engine_version.split(".")[:2]) if engine_version not in detected.keys(): @@ -66,13 +86,14 @@ class UnrealPrelaunchHook(PreLaunchHook): f"detected [ {engine_version} ]" )) - os.makedirs(project_path, exist_ok=True) + ue4_path = unreal_lib.get_editor_executable_path( + Path(detected[engine_version])) - project_file = os.path.join( - project_path, - f"{unreal_project_name}.uproject" - ) - if not os.path.isfile(project_file): + self.launch_context.launch_args.append(ue4_path.as_posix()) + project_path.mkdir(parents=True, exist_ok=True) + + project_file = project_path / f"{unreal_project_name}.uproject" + if not project_file.is_file(): engine_path = detected[engine_version] self.log.info(( f"{self.signature} creating unreal " @@ -88,8 +109,9 @@ class UnrealPrelaunchHook(PreLaunchHook): unreal_project_name, engine_version, project_path, - engine_path=engine_path + engine_path=Path(engine_path) ) # Append project file to launch arguments - self.launch_context.launch_args.append(f"\"{project_file}\"") + self.launch_context.launch_args.append( + f"\"{project_file.as_posix()}\"") diff --git a/openpype/lib/applications.py b/openpype/lib/applications.py index 1eac7ea776..fb86d06150 100644 --- a/openpype/lib/applications.py +++ b/openpype/lib/applications.py @@ -449,6 +449,12 @@ class ApplicationExecutable: """Representation of executable loaded from settings.""" def __init__(self, executable): + # Try to format executable with environments + try: + executable = executable.format(**os.environ) + except Exception: + pass + # On MacOS check if exists path to executable when ends with `.app` # - it is common that path will lead to "/Applications/Blender" but # real path is "/Applications/Blender.app" @@ -460,12 +466,6 @@ class ApplicationExecutable: if os.path.exists(_executable): executable = _executable - # Try to format executable with environments - try: - executable = executable.format(**os.environ) - except Exception: - pass - self.executable_path = executable def __str__(self): diff --git a/openpype/modules/deadline/plugins/publish/submit_maya_deadline.py b/openpype/modules/deadline/plugins/publish/submit_maya_deadline.py index a5841f406c..a652da7786 100644 --- a/openpype/modules/deadline/plugins/publish/submit_maya_deadline.py +++ b/openpype/modules/deadline/plugins/publish/submit_maya_deadline.py @@ -271,6 +271,22 @@ class MayaSubmitDeadline(pyblish.api.InstancePlugin): ["DEADLINE_REST_URL"] ) + self._job_info = ( + context.data["project_settings"].get( + "deadline", {}).get( + "publish", {}).get( + "MayaSubmitDeadline", {}).get( + "jobInfo", {}) + ) + + self._plugin_info = ( + context.data["project_settings"].get( + "deadline", {}).get( + "publish", {}).get( + "MayaSubmitDeadline", {}).get( + "pluginInfo", {}) + ) + assert self._deadline_url, "Requires DEADLINE_REST_URL" context = instance.context @@ -407,7 +423,7 @@ class MayaSubmitDeadline(pyblish.api.InstancePlugin): self.payload_skeleton["JobInfo"]["Priority"] = \ self._instance.data.get("priority", 50) - if self.group != "none": + if self.group != "none" and self.group: self.payload_skeleton["JobInfo"]["Group"] = self.group if self.limit_groups: @@ -536,6 +552,10 @@ class MayaSubmitDeadline(pyblish.api.InstancePlugin): self.preflight_check(instance) + # add jobInfo and pluginInfo variables from Settings + payload["JobInfo"].update(self._job_info) + payload["PluginInfo"].update(self._plugin_info) + # Prepare tiles data ------------------------------------------------ if instance.data.get("tileRendering"): # if we have sequence of files, we need to create tile job for diff --git a/openpype/modules/ftrack/event_handlers_server/action_clone_review_session.py b/openpype/modules/ftrack/event_handlers_server/action_clone_review_session.py index d29316c795..59c8bffb75 100644 --- a/openpype/modules/ftrack/event_handlers_server/action_clone_review_session.py +++ b/openpype/modules/ftrack/event_handlers_server/action_clone_review_session.py @@ -16,11 +16,13 @@ def clone_review_session(session, entity): # Add all invitees. for invitee in entity["review_session_invitees"]: + # Make sure email is not None but string + email = invitee["email"] or "" session.create( "ReviewSessionInvitee", { "name": invitee["name"], - "email": invitee["email"], + "email": email, "review_session": review_session } ) diff --git a/openpype/modules/ftrack/event_handlers_server/action_multiple_notes.py b/openpype/modules/ftrack/event_handlers_server/action_multiple_notes.py new file mode 100644 index 0000000000..9ad7b1a969 --- /dev/null +++ b/openpype/modules/ftrack/event_handlers_server/action_multiple_notes.py @@ -0,0 +1,167 @@ +from openpype.modules.ftrack.lib import ServerAction + + +class MultipleNotesServer(ServerAction): + """Action adds same note for muliple AssetVersions. + + Note is added to selection of AssetVersions. Note is created with user + who triggered the action. It is possible to define note category of note. + """ + + identifier = "multiple.notes.server" + label = "Multiple Notes (Server)" + description = "Add same note to multiple Asset Versions" + + _none_category = "__NONE__" + + def discover(self, session, entities, event): + """Show action only on AssetVersions.""" + if not entities: + return False + + for entity in entities: + if entity.entity_type.lower() != "assetversion": + return False + return True + + def interface(self, session, entities, event): + event_source = event["source"] + user_info = event_source.get("user") or {} + user_id = user_info.get("id") + if not user_id: + return None + + values = event["data"].get("values") + if values: + return None + + note_label = { + "type": "label", + "value": "# Enter note: #" + } + + note_value = { + "name": "note", + "type": "textarea" + } + + category_label = { + "type": "label", + "value": "## Category: ##" + } + + category_data = [] + category_data.append({ + "label": "- None -", + "value": self._none_category + }) + all_categories = session.query( + "select id, name from NoteCategory" + ).all() + for cat in all_categories: + category_data.append({ + "label": cat["name"], + "value": cat["id"] + }) + category_value = { + "type": "enumerator", + "name": "category", + "data": category_data, + "value": self._none_category + } + + splitter = { + "type": "label", + "value": "---" + } + + return [ + note_label, + note_value, + splitter, + category_label, + category_value + ] + + def launch(self, session, entities, event): + if "values" not in event["data"]: + return None + + values = event["data"]["values"] + if len(values) <= 0 or "note" not in values: + return False + + # Get Note text + note_value = values["note"] + if note_value.lower().strip() == "": + return { + "success": True, + "message": "Note was not entered. Skipping" + } + + # Get User + event_source = event["source"] + user_info = event_source.get("user") or {} + user_id = user_info.get("id") + user = None + if user_id: + user = session.query( + 'User where id is "{}"'.format(user_id) + ).first() + + if not user: + return { + "success": False, + "message": "Couldn't get user information." + } + + # Logging message preparation + # - username + username = user.get("username") or "N/A" + + # - AssetVersion ids + asset_version_ids_str = ",".join([entity["id"] for entity in entities]) + + # Base note data + note_data = { + "content": note_value, + "author": user + } + + # Get category + category_id = values["category"] + if category_id == self._none_category: + category_id = None + + category_name = None + if category_id is not None: + category = session.query( + "select id, name from NoteCategory where id is \"{}\"".format( + category_id + ) + ).first() + if category: + note_data["category"] = category + category_name = category["name"] + + category_msg = "" + if category_name: + category_msg = " with category: \"{}\"".format(category_name) + + self.log.warning(( + "Creating note{} as User \"{}\" on " + "AssetVersions: {} with value \"{}\"" + ).format(category_msg, username, asset_version_ids_str, note_value)) + + # Create notes for entities + for entity in entities: + new_note = session.create("Note", note_data) + entity["notes"].append(new_note) + session.commit() + return True + + +def register(session): + '''Register plugin. Called when used as an plugin.''' + + MultipleNotesServer(session).register() diff --git a/openpype/modules/ftrack/plugins/publish/collect_ftrack_family.py b/openpype/modules/ftrack/plugins/publish/collect_ftrack_family.py index b505a429b5..8464a43ef7 100644 --- a/openpype/modules/ftrack/plugins/publish/collect_ftrack_family.py +++ b/openpype/modules/ftrack/plugins/publish/collect_ftrack_family.py @@ -51,7 +51,7 @@ class CollectFtrackFamily(pyblish.api.InstancePlugin): families = instance.data.get("families") add_ftrack_family = profile["add_ftrack_family"] - additional_filters = profile.get("additional_filters") + additional_filters = profile.get("advanced_filtering") if additional_filters: add_ftrack_family = self._get_add_ftrack_f_from_addit_filters( additional_filters, diff --git a/openpype/settings/defaults/project_settings/deadline.json b/openpype/settings/defaults/project_settings/deadline.json index 5861015f2c..2dba20d63c 100644 --- a/openpype/settings/defaults/project_settings/deadline.json +++ b/openpype/settings/defaults/project_settings/deadline.json @@ -3,9 +3,13 @@ "ValidateExpectedFiles": { "enabled": true, "active": true, - "families": ["render"], - "targets": ["deadline"], - "allow_user_override": true + "allow_user_override": true, + "families": [ + "render" + ], + "targets": [ + "deadline" + ] }, "MayaSubmitDeadline": { "enabled": true, @@ -15,7 +19,9 @@ "use_published": true, "asset_dependencies": true, "group": "none", - "limit": [] + "limit": [], + "jobInfo": {}, + "pluginInfo": {} }, "NukeSubmitDeadline": { "enabled": true, diff --git a/openpype/settings/defaults/project_settings/ftrack.json b/openpype/settings/defaults/project_settings/ftrack.json index 88f4e1e2e7..7cf5568662 100644 --- a/openpype/settings/defaults/project_settings/ftrack.json +++ b/openpype/settings/defaults/project_settings/ftrack.json @@ -229,7 +229,6 @@ "standalonepublisher" ], "families": [ - "review", "plate" ], "tasks": [], @@ -279,6 +278,25 @@ "tasks": [], "add_ftrack_family": true, "advanced_filtering": [] + }, + { + "hosts": [ + "nuke" + ], + "families": [ + "write", + "render" + ], + "tasks": [], + "add_ftrack_family": false, + "advanced_filtering": [ + { + "families": [ + "review" + ], + "add_ftrack_family": true + } + ] } ] }, diff --git a/openpype/settings/defaults/system_settings/applications.json b/openpype/settings/defaults/system_settings/applications.json index 224f9dc318..842c294599 100644 --- a/openpype/settings/defaults/system_settings/applications.json +++ b/openpype/settings/defaults/system_settings/applications.json @@ -1084,7 +1084,7 @@ "unreal": { "enabled": true, "label": "Unreal Editor", - "icon": "{}/app_icons/ue4.png'", + "icon": "{}/app_icons/ue4.png", "host_name": "unreal", "environment": {}, "variants": { diff --git a/openpype/settings/entities/__init__.py b/openpype/settings/entities/__init__.py index 94eb819f2b..c0eef15e69 100644 --- a/openpype/settings/entities/__init__.py +++ b/openpype/settings/entities/__init__.py @@ -111,6 +111,7 @@ from .enum_entity import ( from .list_entity import ListEntity from .dict_immutable_keys_entity import DictImmutableKeysEntity from .dict_mutable_keys_entity import DictMutableKeysEntity +from .dict_conditional import DictConditionalEntity from .anatomy_entities import AnatomyEntity @@ -166,5 +167,7 @@ __all__ = ( "DictMutableKeysEntity", + "DictConditionalEntity", + "AnatomyEntity" ) diff --git a/openpype/settings/entities/base_entity.py b/openpype/settings/entities/base_entity.py index 82705d1406..b4ebe885f5 100644 --- a/openpype/settings/entities/base_entity.py +++ b/openpype/settings/entities/base_entity.py @@ -136,6 +136,7 @@ class BaseItemEntity(BaseEntity): # Override state defines which values are used, saved and how. # TODO convert to private attribute self._override_state = OverrideState.NOT_DEFINED + self._ignore_missing_defaults = None # These attributes may change values during existence of an object # Default value, studio override values and project override values @@ -285,7 +286,7 @@ class BaseItemEntity(BaseEntity): pass @abstractmethod - def set_override_state(self, state): + def set_override_state(self, state, ignore_missing_defaults): """Set override state and trigger it on children. Method discard all changes in hierarchy and use values, metadata @@ -295,8 +296,15 @@ class BaseItemEntity(BaseEntity): Should start on root entity and when triggered then must be called on all entities in hierarchy. + Argument `ignore_missing_defaults` should be used when entity has + children that are not saved or used all the time but override statu + must be changed and children must have any default value. + Args: state (OverrideState): State to which should be data changed. + ignore_missing_defaults (bool): Ignore missing default values. + Entity won't raise `DefaultsNotDefined` and + `StudioDefaultsNotDefined`. """ pass diff --git a/openpype/settings/entities/dict_conditional.py b/openpype/settings/entities/dict_conditional.py new file mode 100644 index 0000000000..96065b670e --- /dev/null +++ b/openpype/settings/entities/dict_conditional.py @@ -0,0 +1,707 @@ +import copy + +from .lib import ( + OverrideState, + NOT_SET +) +from openpype.settings.constants import ( + METADATA_KEYS, + M_OVERRIDEN_KEY, + KEY_REGEX +) +from . import ( + BaseItemEntity, + ItemEntity, + GUIEntity +) +from .exceptions import ( + SchemaDuplicatedKeys, + EntitySchemaError, + InvalidKeySymbols +) + + +class DictConditionalEntity(ItemEntity): + """Entity represents dictionay with only one persistent key definition. + + The persistent key is enumerator which define rest of children under + dictionary. There is not possibility of shared children. + + Entity's keys can't be removed or added. But they may change based on + the persistent key. If you're change value manually (key by key) make sure + you'll change value of the persistent key as first. It is recommended to + use `set` method which handle this for you. + + It is possible to use entity similar way as `dict` object. Returned values + are not real settings values but entities representing the value. + """ + schema_types = ["dict-conditional"] + _default_label_wrap = { + "use_label_wrap": False, + "collapsible": False, + "collapsed": True + } + + def __getitem__(self, key): + """Return entity inder key.""" + if key == self.enum_key: + return self.enum_entity + return self.non_gui_children[self.current_enum][key] + + def __setitem__(self, key, value): + """Set value of item under key.""" + if key == self.enum_key: + child_obj = self.enum_entity + else: + child_obj = self.non_gui_children[self.current_enum][key] + child_obj.set(value) + + def __iter__(self): + """Iter through keys.""" + for key in self.keys(): + yield key + + def __contains__(self, key): + """Check if key is available.""" + if key == self.enum_key: + return True + return key in self.non_gui_children[self.current_enum] + + def get(self, key, default=None): + """Safe entity getter by key.""" + if key == self.enum_key: + return self.enum_entity + return self.non_gui_children[self.current_enum].get(key, default) + + def keys(self): + """Entity's keys.""" + keys = list(self.non_gui_children[self.current_enum].keys()) + keys.insert(0, [self.enum_key]) + return keys + + def values(self): + """Children entities.""" + values = [ + self.enum_entity + ] + for child_entiy in self.non_gui_children[self.current_enum].values(): + values.append(child_entiy) + return values + + def items(self): + """Children entities paired with their key (key, value).""" + items = [ + (self.enum_key, self.enum_entity) + ] + for key, value in self.non_gui_children[self.current_enum].items(): + items.append((key, value)) + return items + + def set(self, value): + """Set value.""" + new_value = self.convert_to_valid_type(value) + # First change value of enum key if available + if self.enum_key in new_value: + self.enum_entity.set(new_value.pop(self.enum_key)) + + for _key, _value in new_value.items(): + self.non_gui_children[self.current_enum][_key].set(_value) + + def _item_initalization(self): + self._default_metadata = NOT_SET + self._studio_override_metadata = NOT_SET + self._project_override_metadata = NOT_SET + + self._ignore_child_changes = False + + # `current_metadata` are still when schema is loaded + # - only metadata stored with dict item are gorup overrides in + # M_OVERRIDEN_KEY + self._current_metadata = {} + self._metadata_are_modified = False + + # Entity must be group or in group + if ( + self.group_item is None + and not self.is_dynamic_item + and not self.is_in_dynamic_item + ): + self.is_group = True + + # Children are stored by key as keys are immutable and are defined by + # schema + self.valid_value_types = (dict, ) + self.children = {} + self.non_gui_children = {} + self.gui_layout = {} + + if self.is_dynamic_item: + self.require_key = False + + self.enum_key = self.schema_data.get("enum_key") + self.enum_label = self.schema_data.get("enum_label") + self.enum_children = self.schema_data.get("enum_children") + + self.enum_entity = None + + self.highlight_content = self.schema_data.get( + "highlight_content", False + ) + self.show_borders = self.schema_data.get("show_borders", True) + + self._add_children() + + @property + def current_enum(self): + """Current value of enum entity. + + This value define what children are used. + """ + if self.enum_entity is None: + return None + return self.enum_entity.value + + def schema_validations(self): + """Validation of schema data.""" + # Enum key must be defined + if self.enum_key is None: + raise EntitySchemaError(self, "Key 'enum_key' is not set.") + + # Validate type of enum children + if not isinstance(self.enum_children, list): + raise EntitySchemaError( + self, "Key 'enum_children' must be a list. Got: {}".format( + str(type(self.enum_children)) + ) + ) + + # Without defined enum children entity has nothing to do + if not self.enum_children: + raise EntitySchemaError(self, ( + "Key 'enum_children' have empty value. Entity can't work" + " without children definitions." + )) + + children_def_keys = [] + for children_def in self.enum_children: + if not isinstance(children_def, dict): + raise EntitySchemaError(( + "Children definition under key 'enum_children' must" + " be a dictionary." + )) + + if "key" not in children_def: + raise EntitySchemaError(( + "Children definition under key 'enum_children' miss" + " 'key' definition." + )) + # We don't validate regex of these keys because they will be stored + # as value at the end. + key = children_def["key"] + if key in children_def_keys: + # TODO this hould probably be different exception? + raise SchemaDuplicatedKeys(self, key) + children_def_keys.append(key) + + # Validate key duplications per each enum item + for children in self.children.values(): + children_keys = set() + children_keys.add(self.enum_key) + for child_entity in children: + if not isinstance(child_entity, BaseItemEntity): + continue + elif child_entity.key not in children_keys: + children_keys.add(child_entity.key) + else: + raise SchemaDuplicatedKeys(self, child_entity.key) + + # Enum key must match key regex + if not KEY_REGEX.match(self.enum_key): + raise InvalidKeySymbols(self.path, self.enum_key) + + # Validate all remaining keys with key regex + for children_by_key in self.non_gui_children.values(): + for key in children_by_key.keys(): + if not KEY_REGEX.match(key): + raise InvalidKeySymbols(self.path, key) + + super(DictConditionalEntity, self).schema_validations() + # Trigger schema validation on children entities + for children in self.children.values(): + for child_obj in children: + child_obj.schema_validations() + + def on_change(self): + """Update metadata on change and pass change to parent.""" + self._update_current_metadata() + + for callback in self.on_change_callbacks: + callback() + self.parent.on_child_change(self) + + def on_child_change(self, child_obj): + """Trigger on change callback if child changes are not ignored.""" + if self._ignore_child_changes: + return + + if ( + child_obj is self.enum_entity + or child_obj in self.children[self.current_enum] + ): + self.on_change() + + def _add_children(self): + """Add children from schema data and repare enum items. + + Each enum item must have defined it's children. None are shared across + all enum items. + + Nice to have: Have ability to have shared keys across all enum items. + + All children are stored by their enum item. + """ + # Skip if are not defined + # - schema validations should raise and exception + if not self.enum_children or not self.enum_key: + return + + valid_enum_items = [] + for item in self.enum_children: + if isinstance(item, dict) and "key" in item: + valid_enum_items.append(item) + + enum_items = [] + for item in valid_enum_items: + item_key = item["key"] + item_label = item.get("label") or item_key + enum_items.append({item_key: item_label}) + + if not enum_items: + return + + # Create Enum child first + enum_key = self.enum_key or "invalid" + enum_schema = { + "type": "enum", + "multiselection": False, + "enum_items": enum_items, + "key": enum_key, + "label": self.enum_label or enum_key + } + + enum_entity = self.create_schema_object(enum_schema, self) + self.enum_entity = enum_entity + + # Create children per each enum item + for item in valid_enum_items: + item_key = item["key"] + # Make sure all keys have set value in these variables + # - key 'children' is optional + self.non_gui_children[item_key] = {} + self.children[item_key] = [] + self.gui_layout[item_key] = [] + + children = item.get("children") or [] + for children_schema in children: + child_obj = self.create_schema_object(children_schema, self) + self.children[item_key].append(child_obj) + self.gui_layout[item_key].append(child_obj) + if isinstance(child_obj, GUIEntity): + continue + + self.non_gui_children[item_key][child_obj.key] = child_obj + + def get_child_path(self, child_obj): + """Get hierarchical path of child entity. + + Child must be entity's direct children. This must be possible to get + for any children even if not from current enum value. + """ + if child_obj is self.enum_entity: + return "/".join([self.path, self.enum_key]) + + result_key = None + for children in self.non_gui_children.values(): + for key, _child_obj in children.items(): + if _child_obj is child_obj: + result_key = key + break + + if result_key is None: + raise ValueError("Didn't found child {}".format(child_obj)) + + return "/".join([self.path, result_key]) + + def _update_current_metadata(self): + current_metadata = {} + for key, child_obj in self.non_gui_children[self.current_enum].items(): + if self._override_state is OverrideState.DEFAULTS: + break + + if not child_obj.is_group: + continue + + if ( + self._override_state is OverrideState.STUDIO + and not child_obj.has_studio_override + ): + continue + + if ( + self._override_state is OverrideState.PROJECT + and not child_obj.has_project_override + ): + continue + + if M_OVERRIDEN_KEY not in current_metadata: + current_metadata[M_OVERRIDEN_KEY] = [] + current_metadata[M_OVERRIDEN_KEY].append(key) + + # Define if current metadata are avaialble for current override state + metadata = NOT_SET + if self._override_state is OverrideState.STUDIO: + metadata = self._studio_override_metadata + + elif self._override_state is OverrideState.PROJECT: + metadata = self._project_override_metadata + + if metadata is NOT_SET: + metadata = {} + + self._metadata_are_modified = current_metadata != metadata + self._current_metadata = current_metadata + + def set_override_state(self, state, ignore_missing_defaults): + # Trigger override state change of root if is not same + if self.root_item.override_state is not state: + self.root_item.set_override_state(state) + return + + # Change has/had override states + self._override_state = state + self._ignore_missing_defaults = ignore_missing_defaults + + # Set override state on enum entity first + self.enum_entity.set_override_state(state, ignore_missing_defaults) + + # Set override state on other enum children + # - these must not raise exception about missing defaults + for children_by_key in self.non_gui_children.values(): + for child_obj in children_by_key.values(): + child_obj.set_override_state(state, True) + + self._update_current_metadata() + + @property + def value(self): + output = { + self.enum_key: self.enum_entity.value + } + for key, child_obj in self.non_gui_children[self.current_enum].items(): + output[key] = child_obj.value + return output + + @property + def has_unsaved_changes(self): + if self._metadata_are_modified: + return True + + return self._child_has_unsaved_changes + + @property + def _child_has_unsaved_changes(self): + if self.enum_entity.has_unsaved_changes: + return True + + for child_obj in self.non_gui_children[self.current_enum].values(): + if child_obj.has_unsaved_changes: + return True + return False + + @property + def has_studio_override(self): + return self._child_has_studio_override + + @property + def _child_has_studio_override(self): + if self._override_state >= OverrideState.STUDIO: + if self.enum_entity.has_studio_override: + return True + + for child_obj in self.non_gui_children[self.current_enum].values(): + if child_obj.has_studio_override: + return True + return False + + @property + def has_project_override(self): + return self._child_has_project_override + + @property + def _child_has_project_override(self): + if self._override_state >= OverrideState.PROJECT: + if self.enum_entity.has_project_override: + return True + + for child_obj in self.non_gui_children[self.current_enum].values(): + if child_obj.has_project_override: + return True + return False + + def settings_value(self): + if self._override_state is OverrideState.NOT_DEFINED: + return NOT_SET + + if self._override_state is OverrideState.DEFAULTS: + children_items = [ + (self.enum_key, self.enum_entity) + ] + for item in self.non_gui_children[self.current_enum].items(): + children_items.append(item) + + output = {} + for key, child_obj in children_items: + child_value = child_obj.settings_value() + if not child_obj.is_file and not child_obj.file_item: + for _key, _value in child_value.items(): + new_key = "/".join([key, _key]) + output[new_key] = _value + else: + output[key] = child_value + return output + + if self.is_group: + if self._override_state is OverrideState.STUDIO: + if not self.has_studio_override: + return NOT_SET + elif self._override_state is OverrideState.PROJECT: + if not self.has_project_override: + return NOT_SET + + output = {} + children_items = [ + (self.enum_key, self.enum_entity) + ] + for item in self.non_gui_children[self.current_enum].items(): + children_items.append(item) + + for key, child_obj in children_items: + value = child_obj.settings_value() + if value is not NOT_SET: + output[key] = value + + if not output: + return NOT_SET + + output.update(self._current_metadata) + return output + + def _prepare_value(self, value): + if value is NOT_SET or self.enum_key not in value: + return NOT_SET, NOT_SET + + enum_value = value.get(self.enum_key) + if enum_value not in self.non_gui_children: + return NOT_SET, NOT_SET + + # Create copy of value before poping values + value = copy.deepcopy(value) + metadata = {} + for key in METADATA_KEYS: + if key in value: + metadata[key] = value.pop(key) + + enum_value = value.get(self.enum_key) + + old_metadata = metadata.get(M_OVERRIDEN_KEY) + if old_metadata: + old_metadata_set = set(old_metadata) + new_metadata = [] + non_gui_children = self.non_gui_children[enum_value] + for key in non_gui_children.keys(): + if key in old_metadata: + new_metadata.append(key) + old_metadata_set.remove(key) + + for key in old_metadata_set: + new_metadata.append(key) + metadata[M_OVERRIDEN_KEY] = new_metadata + + return value, metadata + + def update_default_value(self, value): + """Update default values. + + Not an api method, should be called by parent. + """ + value = self._check_update_value(value, "default") + self.has_default_value = value is not NOT_SET + # TODO add value validation + value, metadata = self._prepare_value(value) + self._default_metadata = metadata + + if value is NOT_SET: + self.enum_entity.update_default_value(value) + for children_by_key in self.non_gui_children.values(): + for child_obj in children_by_key.values(): + child_obj.update_default_value(value) + return + + value_keys = set(value.keys()) + enum_value = value[self.enum_key] + expected_keys = set(self.non_gui_children[enum_value].keys()) + expected_keys.add(self.enum_key) + unknown_keys = value_keys - expected_keys + if unknown_keys: + self.log.warning( + "{} Unknown keys in default values: {}".format( + self.path, + ", ".join("\"{}\"".format(key) for key in unknown_keys) + ) + ) + + self.enum_entity.update_default_value(enum_value) + for children_by_key in self.non_gui_children.values(): + for key, child_obj in children_by_key.items(): + child_value = value.get(key, NOT_SET) + child_obj.update_default_value(child_value) + + def update_studio_value(self, value): + """Update studio override values. + + Not an api method, should be called by parent. + """ + value = self._check_update_value(value, "studio override") + value, metadata = self._prepare_value(value) + self._studio_override_metadata = metadata + self.had_studio_override = metadata is not NOT_SET + + if value is NOT_SET: + self.enum_entity.update_studio_value(value) + for children_by_key in self.non_gui_children.values(): + for child_obj in children_by_key.values(): + child_obj.update_studio_value(value) + return + + value_keys = set(value.keys()) + enum_value = value[self.enum_key] + expected_keys = set(self.non_gui_children[enum_value]) + expected_keys.add(self.enum_key) + unknown_keys = value_keys - expected_keys + if unknown_keys: + self.log.warning( + "{} Unknown keys in studio overrides: {}".format( + self.path, + ", ".join("\"{}\"".format(key) for key in unknown_keys) + ) + ) + + self.enum_entity.update_studio_value(enum_value) + for children_by_key in self.non_gui_children.values(): + for key, child_obj in children_by_key.items(): + child_value = value.get(key, NOT_SET) + child_obj.update_studio_value(child_value) + + def update_project_value(self, value): + """Update project override values. + + Not an api method, should be called by parent. + """ + value = self._check_update_value(value, "project override") + value, metadata = self._prepare_value(value) + self._project_override_metadata = metadata + self.had_project_override = metadata is not NOT_SET + + if value is NOT_SET: + self.enum_entity.update_project_value(value) + for children_by_key in self.non_gui_children.values(): + for child_obj in children_by_key.values(): + child_obj.update_project_value(value) + return + + value_keys = set(value.keys()) + enum_value = value[self.enum_key] + expected_keys = set(self.non_gui_children[enum_value]) + expected_keys.add(self.enum_key) + unknown_keys = value_keys - expected_keys + if unknown_keys: + self.log.warning( + "{} Unknown keys in project overrides: {}".format( + self.path, + ", ".join("\"{}\"".format(key) for key in unknown_keys) + ) + ) + + self.enum_entity.update_project_value(enum_value) + for children_by_key in self.non_gui_children.values(): + for key, child_obj in children_by_key.items(): + child_value = value.get(key, NOT_SET) + child_obj.update_project_value(child_value) + + def _discard_changes(self, on_change_trigger): + self._ignore_child_changes = True + + self.enum_entity.discard_changes(on_change_trigger) + for children_by_key in self.non_gui_children.values(): + for child_obj in children_by_key.values(): + child_obj.discard_changes(on_change_trigger) + + self._ignore_child_changes = False + + def _add_to_studio_default(self, on_change_trigger): + self._ignore_child_changes = True + + self.enum_entity.add_to_studio_default(on_change_trigger) + for children_by_key in self.non_gui_children.values(): + for child_obj in children_by_key.values(): + child_obj.add_to_studio_default(on_change_trigger) + + self._ignore_child_changes = False + + self._update_current_metadata() + + self.parent.on_child_change(self) + + def _remove_from_studio_default(self, on_change_trigger): + self._ignore_child_changes = True + + self.enum_entity.remove_from_studio_default(on_change_trigger) + for children_by_key in self.non_gui_children.values(): + for child_obj in children_by_key.values(): + child_obj.remove_from_studio_default(on_change_trigger) + + self._ignore_child_changes = False + + def _add_to_project_override(self, on_change_trigger): + self._ignore_child_changes = True + + self.enum_entity.add_to_project_override(on_change_trigger) + for children_by_key in self.non_gui_children.values(): + for child_obj in children_by_key.values(): + child_obj.add_to_project_override(on_change_trigger) + + self._ignore_child_changes = False + + self._update_current_metadata() + + self.parent.on_child_change(self) + + def _remove_from_project_override(self, on_change_trigger): + if self._override_state is not OverrideState.PROJECT: + return + + self._ignore_child_changes = True + + self.enum_entity.remove_from_project_override(on_change_trigger) + for children_by_key in self.non_gui_children.values(): + for child_obj in children_by_key.values(): + child_obj.remove_from_project_override(on_change_trigger) + + self._ignore_child_changes = False + + def reset_callbacks(self): + """Reset registered callbacks on entity and children.""" + super(DictConditionalEntity, self).reset_callbacks() + for children in self.children.values(): + for child_entity in children: + child_entity.reset_callbacks() diff --git a/openpype/settings/entities/dict_immutable_keys_entity.py b/openpype/settings/entities/dict_immutable_keys_entity.py index c965dc3b5a..bde5304787 100644 --- a/openpype/settings/entities/dict_immutable_keys_entity.py +++ b/openpype/settings/entities/dict_immutable_keys_entity.py @@ -258,7 +258,7 @@ class DictImmutableKeysEntity(ItemEntity): self._metadata_are_modified = current_metadata != metadata self._current_metadata = current_metadata - def set_override_state(self, state): + def set_override_state(self, state, ignore_missing_defaults): # Trigger override state change of root if is not same if self.root_item.override_state is not state: self.root_item.set_override_state(state) @@ -266,9 +266,10 @@ class DictImmutableKeysEntity(ItemEntity): # Change has/had override states self._override_state = state + self._ignore_missing_defaults = ignore_missing_defaults for child_obj in self.non_gui_children.values(): - child_obj.set_override_state(state) + child_obj.set_override_state(state, ignore_missing_defaults) self._update_current_metadata() diff --git a/openpype/settings/entities/dict_mutable_keys_entity.py b/openpype/settings/entities/dict_mutable_keys_entity.py index 3c2645e3e5..c3df935269 100644 --- a/openpype/settings/entities/dict_mutable_keys_entity.py +++ b/openpype/settings/entities/dict_mutable_keys_entity.py @@ -154,7 +154,9 @@ class DictMutableKeysEntity(EndpointEntity): def add_key(self, key): new_child = self._add_key(key) - new_child.set_override_state(self._override_state) + new_child.set_override_state( + self._override_state, self._ignore_missing_defaults + ) self.on_change() return new_child @@ -320,7 +322,7 @@ class DictMutableKeysEntity(EndpointEntity): def _metadata_for_current_state(self): return self._get_metadata_for_state(self._override_state) - def set_override_state(self, state): + def set_override_state(self, state, ignore_missing_defaults): # Trigger override state change of root if is not same if self.root_item.override_state is not state: self.root_item.set_override_state(state) @@ -328,14 +330,22 @@ class DictMutableKeysEntity(EndpointEntity): # TODO change metadata self._override_state = state + self._ignore_missing_defaults = ignore_missing_defaults + # Ignore if is dynamic item and use default in that case if not self.is_dynamic_item and not self.is_in_dynamic_item: if state > OverrideState.DEFAULTS: - if not self.has_default_value: + if ( + not self.has_default_value + and not ignore_missing_defaults + ): raise DefaultsNotDefined(self) elif state > OverrideState.STUDIO: - if not self.had_studio_override: + if ( + not self.had_studio_override + and not ignore_missing_defaults + ): raise StudioDefaultsNotDefined(self) if state is OverrideState.STUDIO: @@ -426,7 +436,7 @@ class DictMutableKeysEntity(EndpointEntity): if label: children_label_by_id[child_entity.id] = label - child_entity.set_override_state(state) + child_entity.set_override_state(state, ignore_missing_defaults) self.children_label_by_id = children_label_by_id @@ -610,7 +620,9 @@ class DictMutableKeysEntity(EndpointEntity): if not self._can_discard_changes: return - self.set_override_state(self._override_state) + self.set_override_state( + self._override_state, self._ignore_missing_defaults + ) on_change_trigger.append(self.on_change) def _add_to_studio_default(self, _on_change_trigger): @@ -645,7 +657,9 @@ class DictMutableKeysEntity(EndpointEntity): if label: children_label_by_id[child_entity.id] = label - child_entity.set_override_state(self._override_state) + child_entity.set_override_state( + self._override_state, self._ignore_missing_defaults + ) self.children_label_by_id = children_label_by_id @@ -694,7 +708,9 @@ class DictMutableKeysEntity(EndpointEntity): if label: children_label_by_id[child_entity.id] = label - child_entity.set_override_state(self._override_state) + child_entity.set_override_state( + self._override_state, self._ignore_missing_defaults + ) self.children_label_by_id = children_label_by_id diff --git a/openpype/settings/entities/input_entities.py b/openpype/settings/entities/input_entities.py index 295333eb60..2abb7a2253 100644 --- a/openpype/settings/entities/input_entities.py +++ b/openpype/settings/entities/input_entities.py @@ -217,21 +217,28 @@ class InputEntity(EndpointEntity): return True return False - def set_override_state(self, state): + def set_override_state(self, state, ignore_missing_defaults): # Trigger override state change of root if is not same if self.root_item.override_state is not state: self.root_item.set_override_state(state) return self._override_state = state + self._ignore_missing_defaults = ignore_missing_defaults # Ignore if is dynamic item and use default in that case if not self.is_dynamic_item and not self.is_in_dynamic_item: if state > OverrideState.DEFAULTS: - if not self.has_default_value: + if ( + not self.has_default_value + and not ignore_missing_defaults + ): raise DefaultsNotDefined(self) elif state > OverrideState.STUDIO: - if not self.had_studio_override: + if ( + not self.had_studio_override + and not ignore_missing_defaults + ): raise StudioDefaultsNotDefined(self) if state is OverrideState.STUDIO: diff --git a/openpype/settings/entities/item_entities.py b/openpype/settings/entities/item_entities.py index 48336080b6..7e84f8c801 100644 --- a/openpype/settings/entities/item_entities.py +++ b/openpype/settings/entities/item_entities.py @@ -150,14 +150,15 @@ class PathEntity(ItemEntity): def value(self): return self.child_obj.value - def set_override_state(self, state): + def set_override_state(self, state, ignore_missing_defaults): # Trigger override state change of root if is not same if self.root_item.override_state is not state: self.root_item.set_override_state(state) return self._override_state = state - self.child_obj.set_override_state(state) + self._ignore_missing_defaults = ignore_missing_defaults + self.child_obj.set_override_state(state, ignore_missing_defaults) def update_default_value(self, value): self.child_obj.update_default_value(value) @@ -344,25 +345,32 @@ class ListStrictEntity(ItemEntity): return True return False - def set_override_state(self, state): + def set_override_state(self, state, ignore_missing_defaults): # Trigger override state change of root if is not same if self.root_item.override_state is not state: self.root_item.set_override_state(state) return self._override_state = state + self._ignore_missing_defaults = ignore_missing_defaults # Ignore if is dynamic item and use default in that case if not self.is_dynamic_item and not self.is_in_dynamic_item: if state > OverrideState.DEFAULTS: - if not self.has_default_value: + if ( + not self.has_default_value + and not ignore_missing_defaults + ): raise DefaultsNotDefined(self) elif state > OverrideState.STUDIO: - if not self.had_studio_override: + if ( + not self.had_studio_override + and not ignore_missing_defaults + ): raise StudioDefaultsNotDefined(self) for child_entity in self.children: - child_entity.set_override_state(state) + child_entity.set_override_state(state, ignore_missing_defaults) self.initial_value = self.settings_value() diff --git a/openpype/settings/entities/lib.py b/openpype/settings/entities/lib.py index 42a08232b9..e58281644a 100644 --- a/openpype/settings/entities/lib.py +++ b/openpype/settings/entities/lib.py @@ -147,7 +147,7 @@ class SchemasHub: crashed_item = self._crashed_on_load[schema_name] raise KeyError( "Unable to parse schema file \"{}\". {}".format( - crashed_item["filpath"], crashed_item["message"] + crashed_item["filepath"], crashed_item["message"] ) ) @@ -176,8 +176,8 @@ class SchemasHub: elif template_name in self._crashed_on_load: crashed_item = self._crashed_on_load[template_name] raise KeyError( - "Unable to parse templace file \"{}\". {}".format( - crashed_item["filpath"], crashed_item["message"] + "Unable to parse template file \"{}\". {}".format( + crashed_item["filepath"], crashed_item["message"] ) ) @@ -345,7 +345,7 @@ class SchemasHub: " One of them crashed on load \"{}\" {}" ).format( filename, - crashed_item["filpath"], + crashed_item["filepath"], crashed_item["message"] )) diff --git a/openpype/settings/entities/list_entity.py b/openpype/settings/entities/list_entity.py index 4b3f7a2659..64bbad28a7 100644 --- a/openpype/settings/entities/list_entity.py +++ b/openpype/settings/entities/list_entity.py @@ -102,7 +102,9 @@ class ListEntity(EndpointEntity): def add_new_item(self, idx=None, trigger_change=True): child_obj = self._add_new_item(idx) - child_obj.set_override_state(self._override_state) + child_obj.set_override_state( + self._override_state, self._ignore_missing_defaults + ) if trigger_change: self.on_child_change(child_obj) @@ -205,13 +207,14 @@ class ListEntity(EndpointEntity): self._has_project_override = True self.on_change() - def set_override_state(self, state): + def set_override_state(self, state, ignore_missing_defaults): # Trigger override state change of root if is not same if self.root_item.override_state is not state: self.root_item.set_override_state(state) return self._override_state = state + self._ignore_missing_defaults = ignore_missing_defaults while self.children: self.children.pop(0) @@ -219,11 +222,17 @@ class ListEntity(EndpointEntity): # Ignore if is dynamic item and use default in that case if not self.is_dynamic_item and not self.is_in_dynamic_item: if state > OverrideState.DEFAULTS: - if not self.has_default_value: + if ( + not self.has_default_value + and not ignore_missing_defaults + ): raise DefaultsNotDefined(self) elif state > OverrideState.STUDIO: - if not self.had_studio_override: + if ( + not self.had_studio_override + and not ignore_missing_defaults + ): raise StudioDefaultsNotDefined(self) value = NOT_SET @@ -257,7 +266,9 @@ class ListEntity(EndpointEntity): child_obj.update_studio_value(item) for child_obj in self.children: - child_obj.set_override_state(self._override_state) + child_obj.set_override_state( + self._override_state, ignore_missing_defaults + ) self.initial_value = self.settings_value() @@ -395,7 +406,9 @@ class ListEntity(EndpointEntity): if self.had_studio_override: child_obj.update_studio_value(item) - child_obj.set_override_state(self._override_state) + child_obj.set_override_state( + self._override_state, self._ignore_missing_defaults + ) if self._override_state >= OverrideState.PROJECT: self._has_project_override = self.had_project_override @@ -427,7 +440,9 @@ class ListEntity(EndpointEntity): for item in value: child_obj = self._add_new_item() child_obj.update_default_value(item) - child_obj.set_override_state(self._override_state) + child_obj.set_override_state( + self._override_state, self._ignore_missing_defaults + ) self._ignore_child_changes = False @@ -460,7 +475,10 @@ class ListEntity(EndpointEntity): child_obj.update_default_value(item) if self._has_studio_override: child_obj.update_studio_value(item) - child_obj.set_override_state(self._override_state) + child_obj.set_override_state( + self._override_state, + self._ignore_missing_defaults + ) self._ignore_child_changes = False diff --git a/openpype/settings/entities/root_entities.py b/openpype/settings/entities/root_entities.py index 5397bf21a1..00677480e8 100644 --- a/openpype/settings/entities/root_entities.py +++ b/openpype/settings/entities/root_entities.py @@ -217,7 +217,7 @@ class RootEntity(BaseItemEntity): schema_data, *args, **kwargs ) - def set_override_state(self, state): + def set_override_state(self, state, ignore_missing_defaults=None): """Set override state and trigger it on children. Method will discard all changes in hierarchy and use values, metadata @@ -226,9 +226,12 @@ class RootEntity(BaseItemEntity): Args: state (OverrideState): State to which should be data changed. """ + if not ignore_missing_defaults: + ignore_missing_defaults = False + self._override_state = state for child_obj in self.non_gui_children.values(): - child_obj.set_override_state(state) + child_obj.set_override_state(state, ignore_missing_defaults) def on_change(self): """Trigger callbacks on change.""" diff --git a/openpype/settings/entities/schemas/README.md b/openpype/settings/entities/schemas/README.md index bbd53fa46b..3c360b892f 100644 --- a/openpype/settings/entities/schemas/README.md +++ b/openpype/settings/entities/schemas/README.md @@ -181,6 +181,103 @@ } ``` +## dict-conditional +- is similar to `dict` but has only one child entity that will be always available +- the one entity is enumerator of possible values and based on value of the entity are defined and used other children entities +- each value of enumerator have defined children that will be used + - there is no way how to have shared entities across multiple enum items +- value from enumerator is also stored next to other values + - to define the key under which will be enum value stored use `enum_key` + - `enum_key` must match key regex and any enum item can't have children with same key + - `enum_label` is label of the entity for UI purposes +- enum items are define with `enum_children` + - it's a list where each item represents enum item + - all items in `enum_children` must have at least `key` key which represents value stored under `enum_key` + - items can define `label` for UI purposes + - most important part is that item can define `children` key where are definitions of it's children (`children` value works the same way as in `dict`) +- entity must have defined `"label"` if is not used as widget +- is set as group if any parent is not group +- if `"label"` is entetered there which will be shown in GUI + - item with label can be collapsible + - that can be set with key `"collapsible"` as `True`/`False` (Default: `True`) + - with key `"collapsed"` as `True`/`False` can be set that is collapsed when GUI is opened (Default: `False`) + - it is possible to add darker background with `"highlight_content"` (Default: `False`) + - darker background has limits of maximum applies after 3-4 nested highlighted items there is not difference in the color + - output is dictionary `{the "key": children values}` +``` +# Example +{ + "type": "dict-conditional", + "key": "my_key", + "label": "My Key", + "enum_key": "type", + "enum_label": "label", + "enum_children": [ + # Each item must be a dictionary with 'key' + { + "key": "action", + "label": "Action", + "children": [ + { + "type": "text", + "key": "key", + "label": "Key" + }, + { + "type": "text", + "key": "label", + "label": "Label" + }, + { + "type": "text", + "key": "command", + "label": "Comand" + } + ] + }, + { + "key": "menu", + "label": "Menu", + "children": [ + { + "key": "children", + "label": "Children", + "type": "list", + "object_type": "text" + } + ] + }, + { + # Separator does not have children as "separator" value is enough + "key": "separator", + "label": "Separator" + } + ] +} +``` + +How output of the schema could look like on save: +``` +{ + "type": "separator" +} + +{ + "type": "action", + "key": "action_1", + "label": "Action 1", + "command": "run command -arg" +} + +{ + "type": "menu", + "children": [ + "child_1", + "child_2" + ] +} +``` + ## Inputs for setting any kind of value (`Pure` inputs) - all these input must have defined `"key"` under which will be stored and `"label"` which will be shown next to input - unless they are used in different types of inputs (later) "as widgets" in that case `"key"` and `"label"` are not required as there is not place where to set them diff --git a/openpype/settings/entities/schemas/projects_schema/schema_project_deadline.json b/openpype/settings/entities/schemas/projects_schema/schema_project_deadline.json index 3281c9ce4d..27eeaef559 100644 --- a/openpype/settings/entities/schemas/projects_schema/schema_project_deadline.json +++ b/openpype/settings/entities/schemas/projects_schema/schema_project_deadline.json @@ -108,6 +108,16 @@ "key": "limit", "label": "Limit Groups", "object_type": "text" + }, + { + "type": "raw-json", + "key": "jobInfo", + "label": "Additional JobInfo data" + }, + { + "type": "raw-json", + "key": "pluginInfo", + "label": "Additional PluginInfo data" } ] }, diff --git a/openpype/settings/entities/schemas/projects_schema/schema_project_unreal.json b/openpype/settings/entities/schemas/projects_schema/schema_project_unreal.json index b6e94d9d03..4e197e9fc8 100644 --- a/openpype/settings/entities/schemas/projects_schema/schema_project_unreal.json +++ b/openpype/settings/entities/schemas/projects_schema/schema_project_unreal.json @@ -15,11 +15,6 @@ "type": "boolean", "key": "dev_mode", "label": "Dev mode" - }, - { - "type": "boolean", - "key": "install_unreal_python_engine", - "label": "Install unreal python engine" } ] } diff --git a/openpype/settings/entities/schemas/system_schema/example_schema.json b/openpype/settings/entities/schemas/system_schema/example_schema.json index a4ed56df32..c3287d7452 100644 --- a/openpype/settings/entities/schemas/system_schema/example_schema.json +++ b/openpype/settings/entities/schemas/system_schema/example_schema.json @@ -9,6 +9,54 @@ "label": "Color input", "type": "color" }, + { + "type": "dict-conditional", + "use_label_wrap": true, + "collapsible": true, + "key": "menu_items", + "label": "Menu items", + "enum_key": "type", + "enum_label": "Type", + "enum_children": [ + { + "key": "action", + "label": "Action", + "children": [ + { + "type": "text", + "key": "key", + "label": "Key" + }, + { + "type": "text", + "key": "label", + "label": "Label" + }, + { + "type": "text", + "key": "command", + "label": "Comand" + } + ] + }, + { + "key": "menu", + "label": "Menu", + "children": [ + { + "key": "children", + "label": "Children", + "type": "list", + "object_type": "text" + } + ] + }, + { + "key": "separator", + "label": "Separator" + } + ] + }, { "type": "dict", "key": "schema_template_exaples", diff --git a/openpype/tools/settings/settings/categories.py b/openpype/tools/settings/settings/categories.py index 392c749211..0aacd590db 100644 --- a/openpype/tools/settings/settings/categories.py +++ b/openpype/tools/settings/settings/categories.py @@ -11,6 +11,7 @@ from openpype.settings.entities import ( GUIEntity, DictImmutableKeysEntity, DictMutableKeysEntity, + DictConditionalEntity, ListEntity, PathEntity, ListStrictEntity, @@ -35,6 +36,7 @@ from .base import GUIWidget from .list_item_widget import ListWidget from .list_strict_widget import ListStrictWidget from .dict_mutable_widget import DictMutableKeysWidget +from .dict_conditional import DictConditionalWidget from .item_widgets import ( BoolWidget, DictImmutableKeysWidget, @@ -100,6 +102,9 @@ class SettingsCategoryWidget(QtWidgets.QWidget): if isinstance(entity, GUIEntity): return GUIWidget(*args) + elif isinstance(entity, DictConditionalEntity): + return DictConditionalWidget(*args) + elif isinstance(entity, DictImmutableKeysEntity): return DictImmutableKeysWidget(*args) diff --git a/openpype/tools/settings/settings/dict_conditional.py b/openpype/tools/settings/settings/dict_conditional.py new file mode 100644 index 0000000000..da2f53497e --- /dev/null +++ b/openpype/tools/settings/settings/dict_conditional.py @@ -0,0 +1,304 @@ +from Qt import QtWidgets + +from .widgets import ( + ExpandingWidget, + GridLabelWidget +) +from .wrapper_widgets import ( + WrapperWidget, + CollapsibleWrapper, + FormWrapper +) +from .base import BaseWidget +from openpype.tools.settings import CHILD_OFFSET + + +class DictConditionalWidget(BaseWidget): + def create_ui(self): + self.input_fields = [] + + self._content_by_enum_value = {} + self._last_enum_value = None + + self.label_widget = None + self.body_widget = None + self.content_widget = None + self.content_layout = None + + label = None + if self.entity.is_dynamic_item: + self._ui_as_dynamic_item() + + elif self.entity.use_label_wrap: + self._ui_label_wrap() + + else: + self._ui_item_base() + label = self.entity.label + + self._parent_widget_by_entity_id = {} + self._enum_key_by_wrapper_id = {} + self._added_wrapper_ids = set() + + self.content_layout.setColumnStretch(0, 0) + self.content_layout.setColumnStretch(1, 1) + + # Add enum entity to layout mapping + enum_entity = self.entity.enum_entity + self._parent_widget_by_entity_id[enum_entity.id] = self.content_widget + + # Add rest of entities to wrapper mappings + for enum_key, children in self.entity.gui_layout.items(): + parent_widget_by_entity_id = {} + + content_widget = QtWidgets.QWidget(self.content_widget) + content_layout = QtWidgets.QGridLayout(content_widget) + content_layout.setColumnStretch(0, 0) + content_layout.setColumnStretch(1, 1) + content_layout.setContentsMargins(0, 0, 0, 0) + content_layout.setSpacing(5) + + self._content_by_enum_value[enum_key] = { + "widget": content_widget, + "layout": content_layout + } + + self._prepare_entity_layouts( + children, + content_widget, + parent_widget_by_entity_id + ) + for item_id in parent_widget_by_entity_id.keys(): + self._enum_key_by_wrapper_id[item_id] = enum_key + self._parent_widget_by_entity_id.update(parent_widget_by_entity_id) + + enum_input_field = self.create_ui_for_entity( + self.category_widget, self.entity.enum_entity, self + ) + self.enum_input_field = enum_input_field + self.input_fields.append(enum_input_field) + + for item_key, children in self.entity.children.items(): + content_widget = self._content_by_enum_value[item_key]["widget"] + row = self.content_layout.rowCount() + self.content_layout.addWidget(content_widget, row, 0, 1, 2) + + for child_obj in children: + self.input_fields.append( + self.create_ui_for_entity( + self.category_widget, child_obj, self + ) + ) + + if self.entity.use_label_wrap and self.content_layout.count() == 0: + self.body_widget.hide_toolbox(True) + + self.entity_widget.add_widget_to_layout(self, label) + + def _prepare_entity_layouts( + self, gui_layout, widget, parent_widget_by_entity_id + ): + for child in gui_layout: + if not isinstance(child, dict): + parent_widget_by_entity_id[child.id] = widget + continue + + if child["type"] == "collapsible-wrap": + wrapper = CollapsibleWrapper(child, widget) + + elif child["type"] == "form": + wrapper = FormWrapper(child, widget) + + else: + raise KeyError( + "Unknown Wrapper type \"{}\"".format(child["type"]) + ) + + parent_widget_by_entity_id[wrapper.id] = widget + + self._prepare_entity_layouts( + child["children"], wrapper, parent_widget_by_entity_id + ) + + def _ui_item_base(self): + self.setObjectName("DictInvisible") + + self.content_widget = self + self.content_layout = QtWidgets.QGridLayout(self) + self.content_layout.setContentsMargins(0, 0, 0, 0) + self.content_layout.setSpacing(5) + + def _ui_as_dynamic_item(self): + content_widget = QtWidgets.QWidget(self) + content_widget.setObjectName("DictAsWidgetBody") + + show_borders = str(int(self.entity.show_borders)) + content_widget.setProperty("show_borders", show_borders) + + label_widget = QtWidgets.QLabel(self.entity.label) + + content_layout = QtWidgets.QGridLayout(content_widget) + content_layout.setContentsMargins(5, 5, 5, 5) + + main_layout = QtWidgets.QHBoxLayout(self) + main_layout.setContentsMargins(0, 0, 0, 0) + main_layout.setSpacing(5) + main_layout.addWidget(content_widget) + + self.label_widget = label_widget + self.content_widget = content_widget + self.content_layout = content_layout + + def _ui_label_wrap(self): + content_widget = QtWidgets.QWidget(self) + content_widget.setObjectName("ContentWidget") + + if self.entity.highlight_content: + content_state = "hightlighted" + bottom_margin = 5 + else: + content_state = "" + bottom_margin = 0 + content_widget.setProperty("content_state", content_state) + content_layout_margins = (CHILD_OFFSET, 5, 0, bottom_margin) + + body_widget = ExpandingWidget(self.entity.label, self) + label_widget = body_widget.label_widget + body_widget.set_content_widget(content_widget) + + content_layout = QtWidgets.QGridLayout(content_widget) + content_layout.setContentsMargins(*content_layout_margins) + + main_layout = QtWidgets.QHBoxLayout(self) + main_layout.setContentsMargins(0, 0, 0, 0) + main_layout.setSpacing(0) + main_layout.addWidget(body_widget) + + self.label_widget = label_widget + self.body_widget = body_widget + self.content_widget = content_widget + self.content_layout = content_layout + + if self.entity.collapsible: + if not self.entity.collapsed: + body_widget.toggle_content() + else: + body_widget.hide_toolbox(hide_content=False) + + def add_widget_to_layout(self, widget, label=None): + if not widget.entity: + map_id = widget.id + else: + map_id = widget.entity.id + + content_widget = self.content_widget + content_layout = self.content_layout + if map_id != self.entity.enum_entity.id: + enum_value = self._enum_key_by_wrapper_id[map_id] + content_widget = self._content_by_enum_value[enum_value]["widget"] + content_layout = self._content_by_enum_value[enum_value]["layout"] + + wrapper = self._parent_widget_by_entity_id[map_id] + if wrapper is not content_widget: + wrapper.add_widget_to_layout(widget, label) + if wrapper.id not in self._added_wrapper_ids: + self.add_widget_to_layout(wrapper) + self._added_wrapper_ids.add(wrapper.id) + return + + row = content_layout.rowCount() + if not label or isinstance(widget, WrapperWidget): + content_layout.addWidget(widget, row, 0, 1, 2) + else: + label_widget = GridLabelWidget(label, widget) + label_widget.input_field = widget + widget.label_widget = label_widget + content_layout.addWidget(label_widget, row, 0, 1, 1) + content_layout.addWidget(widget, row, 1, 1, 1) + + def set_entity_value(self): + for input_field in self.input_fields: + input_field.set_entity_value() + + self._on_entity_change() + + def hierarchical_style_update(self): + self.update_style() + for input_field in self.input_fields: + input_field.hierarchical_style_update() + + def update_style(self): + if not self.body_widget and not self.label_widget: + return + + if self.entity.group_item: + group_item = self.entity.group_item + has_unsaved_changes = group_item.has_unsaved_changes + has_project_override = group_item.has_project_override + has_studio_override = group_item.has_studio_override + else: + has_unsaved_changes = self.entity.has_unsaved_changes + has_project_override = self.entity.has_project_override + has_studio_override = self.entity.has_studio_override + + style_state = self.get_style_state( + self.is_invalid, + has_unsaved_changes, + has_project_override, + has_studio_override + ) + if self._style_state == style_state: + return + + self._style_state = style_state + + if self.body_widget: + if style_state: + child_style_state = "child-{}".format(style_state) + else: + child_style_state = "" + + self.body_widget.side_line_widget.setProperty( + "state", child_style_state + ) + self.body_widget.side_line_widget.style().polish( + self.body_widget.side_line_widget + ) + + # There is nothing to care if there is no label + if not self.label_widget: + return + + # Don't change label if is not group or under group item + if not self.entity.is_group and not self.entity.group_item: + return + + self.label_widget.setProperty("state", style_state) + self.label_widget.style().polish(self.label_widget) + + def _on_entity_change(self): + enum_value = self.enum_input_field.entity.value + if enum_value == self._last_enum_value: + return + + self._last_enum_value = enum_value + for item_key, content in self._content_by_enum_value.items(): + widget = content["widget"] + widget.setVisible(item_key == enum_value) + + @property + def is_invalid(self): + return self._is_invalid or self._child_invalid + + @property + def _child_invalid(self): + for input_field in self.input_fields: + if input_field.is_invalid: + return True + return False + + def get_invalid(self): + invalid = [] + for input_field in self.input_fields: + invalid.extend(input_field.get_invalid()) + return invalid diff --git a/openpype/tools/settings/settings/item_widgets.py b/openpype/tools/settings/settings/item_widgets.py index b23372e9ac..82afbb0a13 100644 --- a/openpype/tools/settings/settings/item_widgets.py +++ b/openpype/tools/settings/settings/item_widgets.py @@ -145,7 +145,7 @@ class DictImmutableKeysWidget(BaseWidget): self.content_widget = content_widget self.content_layout = content_layout - if len(self.input_fields) == 1 and self.checkbox_widget: + if len(self.input_fields) == 1 and self.checkbox_child: body_widget.hide_toolbox(hide_content=True) elif self.entity.collapsible: diff --git a/openpype/tools/standalonepublish/app.py b/openpype/tools/standalonepublish/app.py index 169abe530a..81a53c52b8 100644 --- a/openpype/tools/standalonepublish/app.py +++ b/openpype/tools/standalonepublish/app.py @@ -34,6 +34,13 @@ class Window(QtWidgets.QDialog): self._db = AvalonMongoDB() self._db.install() + try: + settings = QtCore.QSettings("pypeclub", "StandalonePublisher") + except Exception: + settings = None + + self._settings = settings + self.pyblish_paths = pyblish_paths self.setWindowTitle("Standalone Publish") @@ -44,7 +51,7 @@ class Window(QtWidgets.QDialog): self.valid_parent = False # assets widget - widget_assets = AssetWidget(dbcon=self._db, parent=self) + widget_assets = AssetWidget(self._db, settings, self) # family widget widget_family = FamilyWidget(dbcon=self._db, parent=self) diff --git a/openpype/tools/standalonepublish/widgets/widget_asset.py b/openpype/tools/standalonepublish/widgets/widget_asset.py index 4680e88344..c39d71b055 100644 --- a/openpype/tools/standalonepublish/widgets/widget_asset.py +++ b/openpype/tools/standalonepublish/widgets/widget_asset.py @@ -127,11 +127,12 @@ class AssetWidget(QtWidgets.QWidget): current_changed = QtCore.Signal() # on view current index change task_changed = QtCore.Signal() - def __init__(self, dbcon, parent=None): + def __init__(self, dbcon, settings, parent=None): super(AssetWidget, self).__init__(parent=parent) self.setContentsMargins(0, 0, 0, 0) self.dbcon = dbcon + self._settings = settings layout = QtWidgets.QVBoxLayout() layout.setContentsMargins(0, 0, 0, 0) @@ -139,6 +140,10 @@ class AssetWidget(QtWidgets.QWidget): # Project self.combo_projects = QtWidgets.QComboBox() + # Change delegate so stylysheets are applied + project_delegate = QtWidgets.QStyledItemDelegate(self.combo_projects) + self.combo_projects.setItemDelegate(project_delegate) + self._set_projects() self.combo_projects.currentTextChanged.connect(self.on_project_change) # Tree View @@ -198,6 +203,7 @@ class AssetWidget(QtWidgets.QWidget): self.selection_changed.connect(self._refresh_tasks) + self.project_delegate = project_delegate self.task_view = task_view self.task_model = task_model self.refreshButton = refresh @@ -237,15 +243,59 @@ class AssetWidget(QtWidgets.QWidget): output.extend(self.get_parents(parent)) return output + def _get_last_projects(self): + if not self._settings: + return [] + + project_names = [] + for project_name in self._settings.value("projects", "").split("|"): + if project_name: + project_names.append(project_name) + return project_names + + def _add_last_project(self, project_name): + if not self._settings: + return + + last_projects = [] + for _project_name in self._settings.value("projects", "").split("|"): + if _project_name: + last_projects.append(_project_name) + + if project_name in last_projects: + last_projects.remove(project_name) + + last_projects.insert(0, project_name) + while len(last_projects) > 5: + last_projects.pop(-1) + + self._settings.setValue("projects", "|".join(last_projects)) + def _set_projects(self): - projects = list() + project_names = list() for project in self.dbcon.projects(): - projects.append(project['name']) + project_name = project.get("name") + if project_name: + project_names.append(project_name) self.combo_projects.clear() - if len(projects) > 0: - self.combo_projects.addItems(projects) - self.dbcon.Session["AVALON_PROJECT"] = projects[0] + + if not project_names: + return + + sorted_project_names = list(sorted(project_names)) + self.combo_projects.addItems(list(sorted(sorted_project_names))) + + last_project = sorted_project_names[0] + for project_name in self._get_last_projects(): + if project_name in sorted_project_names: + last_project = project_name + break + + index = sorted_project_names.index(last_project) + self.combo_projects.setCurrentIndex(index) + + self.dbcon.Session["AVALON_PROJECT"] = last_project def on_project_change(self): projects = list() @@ -254,6 +304,7 @@ class AssetWidget(QtWidgets.QWidget): project_name = self.combo_projects.currentText() if project_name in projects: self.dbcon.Session["AVALON_PROJECT"] = project_name + self._add_last_project(project_name) self.project_changed.emit(project_name) diff --git a/openpype/version.py b/openpype/version.py index 0371d5f4e3..dabeacc084 100644 --- a/openpype/version.py +++ b/openpype/version.py @@ -1,3 +1,3 @@ # -*- coding: utf-8 -*- """Package declaring Pype version.""" -__version__ = "3.2.0-nightly.5" +__version__ = "3.2.0-nightly.7" diff --git a/start.py b/start.py index 8e7c195e95..1b5c25ae3a 100644 --- a/start.py +++ b/start.py @@ -135,18 +135,36 @@ if sys.__stdout__: def _print(message: str): if message.startswith("!!! "): print("{}{}".format(term.orangered2("!!! "), message[4:])) + return if message.startswith(">>> "): print("{}{}".format(term.aquamarine3(">>> "), message[4:])) + return if message.startswith("--- "): print("{}{}".format(term.darkolivegreen3("--- "), message[4:])) - if message.startswith(" "): - print("{}{}".format(term.darkseagreen3(" "), message[4:])) + return if message.startswith("*** "): print("{}{}".format(term.gold("*** "), message[4:])) + return if message.startswith(" - "): print("{}{}".format(term.wheat(" - "), message[4:])) + return if message.startswith(" . "): print("{}{}".format(term.tan(" . "), message[4:])) + return + if message.startswith(" - "): + print("{}{}".format(term.seagreen3(" - "), message[7:])) + return + if message.startswith(" ! "): + print("{}{}".format(term.goldenrod(" ! "), message[7:])) + return + if message.startswith(" * "): + print("{}{}".format(term.aquamarine1(" * "), message[7:])) + return + if message.startswith(" "): + print("{}{}".format(term.darkseagreen3(" "), message[4:])) + return + + print(message) else: def _print(message: str): print(message) @@ -175,6 +193,17 @@ silent_commands = ["run", "igniter", "standalonepublisher", "extractenvironments"] +def list_versions(openpype_versions: list, local_version=None) -> None: + """Print list of detected versions.""" + _print(" - Detected versions:") + for v in sorted(openpype_versions): + _print(f" - {v}: {v.path}") + if not openpype_versions: + _print(" ! none in repository detected") + if local_version: + _print(f" * local version {local_version}") + + def set_openpype_global_environments() -> None: """Set global OpenPype's environments.""" import acre @@ -303,22 +332,37 @@ def _process_arguments() -> tuple: # check for `--use-version=3.0.0` argument and `--use-staging` use_version = None use_staging = False + print_versions = False for arg in sys.argv: if arg == "--use-version": _print("!!! Please use option --use-version like:") _print(" --use-version=3.0.0") sys.exit(1) - m = re.search( - r"--use-version=(?P\d+\.\d+\.\d+(?:\S*)?)", arg) - if m and m.group('version'): - use_version = m.group('version') - sys.argv.remove(arg) - break + if arg.startswith("--use-version="): + m = re.search( + r"--use-version=(?P\d+\.\d+\.\d+(?:\S*)?)", arg) + if m and m.group('version'): + use_version = m.group('version') + _print(">>> Requested version [ {} ]".format(use_version)) + sys.argv.remove(arg) + if "+staging" in use_version: + use_staging = True + break + else: + _print("!!! Requested version isn't in correct format.") + _print((" Use --list-versions to find out" + " proper version string.")) + sys.exit(1) + if "--use-staging" in sys.argv: use_staging = True sys.argv.remove("--use-staging") + if "--list-versions" in sys.argv: + print_versions = True + sys.argv.remove("--list-versions") + # handle igniter # this is helper to run igniter before anything else if "igniter" in sys.argv: @@ -334,7 +378,7 @@ def _process_arguments() -> tuple: sys.argv.pop(idx) sys.argv.insert(idx, "tray") - return use_version, use_staging + return use_version, use_staging, print_versions def _determine_mongodb() -> str: @@ -487,7 +531,7 @@ def _find_frozen_openpype(use_version: str = None, openpype_version = openpype_versions[-1] except IndexError: _print(("!!! Something is wrong and we didn't " - "found it again.")) + "found it again.")) sys.exit(1) elif return_code != 2: _print(f" . finished ({return_code})") @@ -500,7 +544,7 @@ def _find_frozen_openpype(use_version: str = None, _print("*** Still no luck finding OpenPype.") _print(("*** We'll try to use the one coming " "with OpenPype installation.")) - version_path = _bootstrap_from_code(use_version) + version_path = _bootstrap_from_code(use_version, use_staging) openpype_version = OpenPypeVersion( version=BootstrapRepos.get_version(version_path), path=version_path) @@ -519,13 +563,8 @@ def _find_frozen_openpype(use_version: str = None, if found: openpype_version = sorted(found)[-1] if not openpype_version: - _print(f"!!! requested version {use_version} was not found.") - if openpype_versions: - _print(" - found: ") - for v in sorted(openpype_versions): - _print(f" - {v}: {v.path}") - - _print(f" - local version {local_version}") + _print(f"!!! Requested version {use_version} was not found.") + list_versions(openpype_versions, local_version) sys.exit(1) # test if latest detected is installed (in user data dir) @@ -560,7 +599,7 @@ def _find_frozen_openpype(use_version: str = None, return openpype_version.path -def _bootstrap_from_code(use_version): +def _bootstrap_from_code(use_version, use_staging): """Bootstrap live code (or the one coming with frozen OpenPype. Args: @@ -575,15 +614,33 @@ def _bootstrap_from_code(use_version): _openpype_root = OPENPYPE_ROOT if getattr(sys, 'frozen', False): local_version = bootstrap.get_version(Path(_openpype_root)) - _print(f" - running version: {local_version}") + switch_str = f" - will switch to {use_version}" if use_version else "" + _print(f" - booting version: {local_version}{switch_str}") assert local_version else: # get current version of OpenPype local_version = bootstrap.get_local_live_version() - if use_version and use_version != local_version: - version_to_use = None - openpype_versions = bootstrap.find_openpype(include_zips=True) + version_to_use = None + openpype_versions = bootstrap.find_openpype( + include_zips=True, staging=use_staging) + if use_staging and not use_version: + try: + version_to_use = openpype_versions[-1] + except IndexError: + _print("!!! No staging versions are found.") + list_versions(openpype_versions, local_version) + sys.exit(1) + if version_to_use.path.is_file(): + version_to_use.path = bootstrap.extract_openpype( + version_to_use) + bootstrap.add_paths_from_directory(version_to_use.path) + os.environ["OPENPYPE_VERSION"] = str(version_to_use) + version_path = version_to_use.path + os.environ["OPENPYPE_REPOS_ROOT"] = (version_path / "openpype").as_posix() # noqa: E501 + _openpype_root = version_to_use.path.as_posix() + + elif use_version and use_version != local_version: v: OpenPypeVersion found = [v for v in openpype_versions if str(v) == use_version] if found: @@ -600,13 +657,8 @@ def _bootstrap_from_code(use_version): os.environ["OPENPYPE_REPOS_ROOT"] = (version_path / "openpype").as_posix() # noqa: E501 _openpype_root = version_to_use.path.as_posix() else: - _print(f"!!! requested version {use_version} was not found.") - if openpype_versions: - _print(" - found: ") - for v in sorted(openpype_versions): - _print(f" - {v}: {v.path}") - - _print(f" - local version {local_version}") + _print(f"!!! Requested version {use_version} was not found.") + list_versions(openpype_versions, local_version) sys.exit(1) else: os.environ["OPENPYPE_VERSION"] = local_version @@ -675,11 +727,16 @@ def boot(): # Process arguments # ------------------------------------------------------------------------ - use_version, use_staging = _process_arguments() + use_version, use_staging, print_versions = _process_arguments() if os.getenv("OPENPYPE_VERSION"): - use_staging = "staging" in os.getenv("OPENPYPE_VERSION") - use_version = os.getenv("OPENPYPE_VERSION") + if use_version: + _print(("*** environment variable OPENPYPE_VERSION" + "is overridden by command line argument.")) + else: + _print(">>> version set by environment variable") + use_staging = "staging" in os.getenv("OPENPYPE_VERSION") + use_version = os.getenv("OPENPYPE_VERSION") # ------------------------------------------------------------------------ # Determine mongodb connection @@ -704,6 +761,24 @@ def boot(): if not os.getenv("OPENPYPE_PATH") and openpype_path: os.environ["OPENPYPE_PATH"] = openpype_path + if print_versions: + if not use_staging: + _print("--- This will list only non-staging versions detected.") + _print(" To see staging versions, use --use-staging argument.") + else: + _print("--- This will list only staging versions detected.") + _print(" To see other version, omit --use-staging argument.") + _openpype_root = OPENPYPE_ROOT + openpype_versions = bootstrap.find_openpype(include_zips=True, + staging=use_staging) + if getattr(sys, 'frozen', False): + local_version = bootstrap.get_version(Path(_openpype_root)) + else: + local_version = bootstrap.get_local_live_version() + + list_versions(openpype_versions, local_version) + sys.exit(1) + # ------------------------------------------------------------------------ # Find OpenPype versions # ------------------------------------------------------------------------ @@ -718,7 +793,7 @@ def boot(): _print(f"!!! {e}") sys.exit(1) else: - version_path = _bootstrap_from_code(use_version) + version_path = _bootstrap_from_code(use_version, use_staging) # set this to point either to `python` from venv in case of live code # or to `openpype` or `openpype_console` in case of frozen code @@ -754,7 +829,7 @@ def boot(): from openpype.version import __version__ assert version_path, "Version path not defined." - info = get_info() + info = get_info(use_staging) info.insert(0, f">>> Using OpenPype from [ {version_path} ]") t_width = 20 @@ -781,7 +856,7 @@ def boot(): sys.exit(1) -def get_info() -> list: +def get_info(use_staging=None) -> list: """Print additional information to console.""" from openpype.lib.mongo import get_default_components from openpype.lib.log import PypeLogger @@ -789,7 +864,7 @@ def get_info() -> list: components = get_default_components() inf = [] - if not getattr(sys, 'frozen', False): + if use_staging: inf.append(("OpenPype variant", "staging")) else: inf.append(("OpenPype variant", "production")) diff --git a/tests/unit/igniter/test_bootstrap_repos.py b/tests/unit/igniter/test_bootstrap_repos.py index 743131acfa..740a71a5ce 100644 --- a/tests/unit/igniter/test_bootstrap_repos.py +++ b/tests/unit/igniter/test_bootstrap_repos.py @@ -330,8 +330,8 @@ def test_find_openpype(fix_bootstrap, tmp_path_factory, monkeypatch, printer): assert result[-1].path == expected_path, ("not a latest version of " "OpenPype 3") + printer("testing finding OpenPype in OPENPYPE_PATH ...") monkeypatch.setenv("OPENPYPE_PATH", e_path.as_posix()) - result = fix_bootstrap.find_openpype(include_zips=True) # we should have results as file were created assert result is not None, "no OpenPype version found" @@ -349,6 +349,8 @@ def test_find_openpype(fix_bootstrap, tmp_path_factory, monkeypatch, printer): monkeypatch.delenv("OPENPYPE_PATH", raising=False) + printer("testing finding OpenPype in user data dir ...") + # mock appdirs user_data_dir def mock_user_data_dir(*args, **kwargs): """Mock local app data dir.""" @@ -373,18 +375,7 @@ def test_find_openpype(fix_bootstrap, tmp_path_factory, monkeypatch, printer): assert result[-1].path == expected_path, ("not a latest version of " "OpenPype 2") - result = fix_bootstrap.find_openpype(e_path, include_zips=True) - assert result is not None, "no OpenPype version found" - expected_path = Path( - e_path / "{}{}{}".format( - test_versions_1[5].prefix, - test_versions_1[5].version, - test_versions_1[5].suffix - ) - ) - assert result[-1].path == expected_path, ("not a latest version of " - "OpenPype 1") - + printer("testing finding OpenPype zip/dir precedence ...") result = fix_bootstrap.find_openpype(dir_path, include_zips=True) assert result is not None, "no OpenPype versions found" expected_path = Path( diff --git a/tools/build.ps1 b/tools/build.ps1 index c8c2f392ad..cc4253fe24 100644 --- a/tools/build.ps1 +++ b/tools/build.ps1 @@ -83,7 +83,8 @@ function Show-PSWarning() { function Install-Poetry() { Write-Host ">>> " -NoNewline -ForegroundColor Green Write-Host "Installing Poetry ... " - (Invoke-WebRequest -Uri https://raw.githubusercontent.com/python-poetry/poetry/master/get-poetry.py -UseBasicParsing).Content | python - + $env:POETRY_HOME="$openpype_root\.poetry" + (Invoke-WebRequest -Uri https://raw.githubusercontent.com/python-poetry/poetry/master/install-poetry.py -UseBasicParsing).Content | python - } $art = @" @@ -115,11 +116,9 @@ $openpype_root = (Get-Item $script_dir).parent.FullName $env:_INSIDE_OPENPYPE_TOOL = "1" -# make sure Poetry is in PATH if (-not (Test-Path 'env:POETRY_HOME')) { $env:POETRY_HOME = "$openpype_root\.poetry" } -$env:PATH = "$($env:PATH);$($env:POETRY_HOME)\bin" Set-Location -Path $openpype_root @@ -164,7 +163,7 @@ Write-Host " ]" -ForegroundColor white Write-Host ">>> " -NoNewline -ForegroundColor Green Write-Host "Reading Poetry ... " -NoNewline -if (-not (Test-Path -PathType Container -Path "$openpype_root\.poetry\bin")) { +if (-not (Test-Path -PathType Container -Path "$($env:POETRY_HOME)\bin")) { Write-Host "NOT FOUND" -ForegroundColor Yellow Write-Host "*** " -NoNewline -ForegroundColor Yellow Write-Host "We need to install Poetry create virtual env first ..." @@ -184,10 +183,10 @@ Write-Host ">>> " -NoNewline -ForegroundColor green Write-Host "Building OpenPype ..." $startTime = [int][double]::Parse((Get-Date -UFormat %s)) -$out = & poetry run python setup.py build 2>&1 +$out = & "$($env:POETRY_HOME)\bin\poetry" run python setup.py build 2>&1 +Set-Content -Path "$($openpype_root)\build\build.log" -Value $out if ($LASTEXITCODE -ne 0) { - Set-Content -Path "$($openpype_root)\build\build.log" -Value $out Write-Host "!!! " -NoNewLine -ForegroundColor Red Write-Host "Build failed. Check the log: " -NoNewline Write-Host ".\build\build.log" -ForegroundColor Yellow @@ -195,7 +194,7 @@ if ($LASTEXITCODE -ne 0) } Set-Content -Path "$($openpype_root)\build\build.log" -Value $out -& poetry run python "$($openpype_root)\tools\build_dependencies.py" +& "$($env:POETRY_HOME)\bin\poetry" run python "$($openpype_root)\tools\build_dependencies.py" Write-Host ">>> " -NoNewline -ForegroundColor green Write-Host "restoring current directory" diff --git a/tools/build.sh b/tools/build.sh index aa8f0121ea..c44e7157af 100755 --- a/tools/build.sh +++ b/tools/build.sh @@ -140,21 +140,6 @@ realpath () { echo $(cd $(dirname "$1") || return; pwd)/$(basename "$1") } -############################################################################## -# Install Poetry when needed -# Globals: -# PATH -# Arguments: -# None -# Returns: -# None -############################################################################### -install_poetry () { - echo -e "${BIGreen}>>>${RST} Installing Poetry ..." - command -v curl >/dev/null 2>&1 || { echo -e "${BIRed}!!!${RST}${BIYellow} Missing ${RST}${BIBlue}curl${BIYellow} command.${RST}"; return 1; } - curl -sSL https://raw.githubusercontent.com/python-poetry/poetry/master/get-poetry.py | python - -} - # Main main () { echo -e "${BGreen}" @@ -171,11 +156,9 @@ main () { _inside_openpype_tool="1" - # make sure Poetry is in PATH if [[ -z $POETRY_HOME ]]; then export POETRY_HOME="$openpype_root/.poetry" fi - export PATH="$POETRY_HOME/bin:$PATH" echo -e "${BIYellow}---${RST} Cleaning build directory ..." rm -rf "$openpype_root/build" && mkdir "$openpype_root/build" > /dev/null @@ -201,11 +184,11 @@ if [ "$disable_submodule_update" == 1 ]; then fi echo -e "${BIGreen}>>>${RST} Building ..." if [[ "$OSTYPE" == "linux-gnu"* ]]; then - poetry run python "$openpype_root/setup.py" build > "$openpype_root/build/build.log" || { echo -e "${BIRed}!!!${RST} Build failed, see the build log."; return; } + "$POETRY_HOME/bin/poetry" run python "$openpype_root/setup.py" build > "$openpype_root/build/build.log" || { echo -e "${BIRed}!!!${RST} Build failed, see the build log."; return; } elif [[ "$OSTYPE" == "darwin"* ]]; then - poetry run python "$openpype_root/setup.py" bdist_mac > "$openpype_root/build/build.log" || { echo -e "${BIRed}!!!${RST} Build failed, see the build log."; return; } + "$POETRY_HOME/bin/poetry" run python "$openpype_root/setup.py" bdist_mac > "$openpype_root/build/build.log" || { echo -e "${BIRed}!!!${RST} Build failed, see the build log."; return; } fi - poetry run python "$openpype_root/tools/build_dependencies.py" + "$POETRY_HOME/bin/poetry" run python "$openpype_root/tools/build_dependencies.py" if [[ "$OSTYPE" == "darwin"* ]]; then # fix code signing issue diff --git a/tools/build_win_installer.ps1 b/tools/build_win_installer.ps1 index 05ec0f9823..a0832e0135 100644 --- a/tools/build_win_installer.ps1 +++ b/tools/build_win_installer.ps1 @@ -64,14 +64,6 @@ function Show-PSWarning() { } } -function Install-Poetry() { - Write-Host ">>> " -NoNewline -ForegroundColor Green - Write-Host "Installing Poetry ... " - (Invoke-WebRequest -Uri https://raw.githubusercontent.com/python-poetry/poetry/master/get-poetry.py -UseBasicParsing).Content | python - - # add it to PATH - $env:PATH = "$($env:PATH);$($env:USERPROFILE)\.poetry\bin" -} - $art = @" . . .. . .. diff --git a/tools/create_env.ps1 b/tools/create_env.ps1 index 94a91ce48f..6c8124ccb2 100644 --- a/tools/create_env.ps1 +++ b/tools/create_env.ps1 @@ -49,9 +49,7 @@ function Install-Poetry() { Write-Host ">>> " -NoNewline -ForegroundColor Green Write-Host "Installing Poetry ... " $env:POETRY_HOME="$openpype_root\.poetry" - (Invoke-WebRequest -Uri https://raw.githubusercontent.com/python-poetry/poetry/master/get-poetry.py -UseBasicParsing).Content | python - - # add it to PATH - $env:PATH = "$($env:PATH);$openpype_root\.poetry\bin" + (Invoke-WebRequest -Uri https://raw.githubusercontent.com/python-poetry/poetry/master/install-poetry.py -UseBasicParsing).Content | python - } @@ -94,11 +92,10 @@ $current_dir = Get-Location $script_dir = Split-Path -Path $MyInvocation.MyCommand.Definition -Parent $openpype_root = (Get-Item $script_dir).parent.FullName -# make sure Poetry is in PATH if (-not (Test-Path 'env:POETRY_HOME')) { $env:POETRY_HOME = "$openpype_root\.poetry" } -$env:PATH = "$($env:PATH);$($env:POETRY_HOME)\bin" + Set-Location -Path $openpype_root @@ -145,7 +142,7 @@ Test-Python Write-Host ">>> " -NoNewline -ForegroundColor Green Write-Host "Reading Poetry ... " -NoNewline -if (-not (Test-Path -PathType Container -Path "$openpype_root\.poetry\bin")) { +if (-not (Test-Path -PathType Container -Path "$($env:POETRY_HOME)\bin")) { Write-Host "NOT FOUND" -ForegroundColor Yellow Install-Poetry Write-Host "INSTALLED" -ForegroundColor Cyan @@ -160,7 +157,7 @@ if (-not (Test-Path -PathType Leaf -Path "$($openpype_root)\poetry.lock")) { Write-Host ">>> " -NoNewline -ForegroundColor green Write-Host "Installing virtual environment from lock." } -& poetry install --no-root $poetry_verbosity +& "$env:POETRY_HOME\bin\poetry" install --no-root $poetry_verbosity --ansi if ($LASTEXITCODE -ne 0) { Write-Host "!!! " -ForegroundColor yellow -NoNewline Write-Host "Poetry command failed." diff --git a/tools/create_env.sh b/tools/create_env.sh index 226a26e199..cc9eddc317 100755 --- a/tools/create_env.sh +++ b/tools/create_env.sh @@ -109,8 +109,7 @@ install_poetry () { echo -e "${BIGreen}>>>${RST} Installing Poetry ..." export POETRY_HOME="$openpype_root/.poetry" command -v curl >/dev/null 2>&1 || { echo -e "${BIRed}!!!${RST}${BIYellow} Missing ${RST}${BIBlue}curl${BIYellow} command.${RST}"; return 1; } - curl -sSL https://raw.githubusercontent.com/python-poetry/poetry/master/get-poetry.py | python - - export PATH="$PATH:$POETRY_HOME/bin" + curl -sSL https://raw.githubusercontent.com/python-poetry/poetry/master/install-poetry.py | python - } ############################################################################## @@ -154,11 +153,10 @@ main () { # Directories openpype_root=$(realpath $(dirname $(dirname "${BASH_SOURCE[0]}"))) - # make sure Poetry is in PATH + if [[ -z $POETRY_HOME ]]; then export POETRY_HOME="$openpype_root/.poetry" fi - export PATH="$POETRY_HOME/bin:$PATH" pushd "$openpype_root" > /dev/null || return > /dev/null @@ -177,7 +175,7 @@ main () { echo -e "${BIGreen}>>>${RST} Installing dependencies ..." fi - poetry install --no-root $poetry_verbosity || { echo -e "${BIRed}!!!${RST} Poetry environment installation failed"; return; } + "$POETRY_HOME/bin/poetry" install --no-root $poetry_verbosity || { echo -e "${BIRed}!!!${RST} Poetry environment installation failed"; return; } echo -e "${BIGreen}>>>${RST} Cleaning cache files ..." clean_pyc @@ -186,10 +184,10 @@ main () { # cx_freeze will crash on missing __pychache__ on these but # reinstalling them solves the problem. echo -e "${BIGreen}>>>${RST} Fixing pycache bug ..." - poetry run python -m pip install --force-reinstall pip - poetry run pip install --force-reinstall setuptools - poetry run pip install --force-reinstall wheel - poetry run python -m pip install --force-reinstall pip + "$POETRY_HOME/bin/poetry" run python -m pip install --force-reinstall pip + "$POETRY_HOME/bin/poetry" run pip install --force-reinstall setuptools + "$POETRY_HOME/bin/poetry" run pip install --force-reinstall wheel + "$POETRY_HOME/bin/poetry" run python -m pip install --force-reinstall pip } main -3 diff --git a/tools/create_zip.ps1 b/tools/create_zip.ps1 index 1a7520eb11..c27857b480 100644 --- a/tools/create_zip.ps1 +++ b/tools/create_zip.ps1 @@ -45,11 +45,9 @@ $openpype_root = (Get-Item $script_dir).parent.FullName $env:_INSIDE_OPENPYPE_TOOL = "1" -# make sure Poetry is in PATH if (-not (Test-Path 'env:POETRY_HOME')) { $env:POETRY_HOME = "$openpype_root\.poetry" } -$env:PATH = "$($env:PATH);$($env:POETRY_HOME)\bin" Set-Location -Path $openpype_root @@ -87,7 +85,7 @@ if (-not $openpype_version) { Write-Host ">>> " -NoNewline -ForegroundColor Green Write-Host "Reading Poetry ... " -NoNewline -if (-not (Test-Path -PathType Container -Path "$openpype_root\.poetry\bin")) { +if (-not (Test-Path -PathType Container -Path "$($env:POETRY_HOME)\bin")) { Write-Host "NOT FOUND" -ForegroundColor Yellow Write-Host "*** " -NoNewline -ForegroundColor Yellow Write-Host "We need to install Poetry create virtual env first ..." @@ -107,5 +105,5 @@ Write-Host ">>> " -NoNewline -ForegroundColor green Write-Host "Generating zip from current sources ..." $env:PYTHONPATH="$($openpype_root);$($env:PYTHONPATH)" $env:OPENPYPE_ROOT="$($openpype_root)" -& poetry run python "$($openpype_root)\tools\create_zip.py" $ARGS +& "$($env:POETRY_HOME)\bin\poetry" run python "$($openpype_root)\tools\create_zip.py" $ARGS Set-Location -Path $current_dir diff --git a/tools/create_zip.sh b/tools/create_zip.sh index ec0276b040..85ee18a839 100755 --- a/tools/create_zip.sh +++ b/tools/create_zip.sh @@ -114,11 +114,9 @@ main () { _inside_openpype_tool="1" - # make sure Poetry is in PATH if [[ -z $POETRY_HOME ]]; then export POETRY_HOME="$openpype_root/.poetry" fi - export PATH="$POETRY_HOME/bin:$PATH" pushd "$openpype_root" > /dev/null || return > /dev/null @@ -134,7 +132,7 @@ main () { echo -e "${BIGreen}>>>${RST} Generating zip from current sources ..." PYTHONPATH="$openpype_root:$PYTHONPATH" OPENPYPE_ROOT="$openpype_root" - poetry run python3 "$openpype_root/tools/create_zip.py" "$@" + "$POETRY_HOME/bin/poetry" run python3 "$openpype_root/tools/create_zip.py" "$@" } main "$@" diff --git a/tools/fetch_thirdparty_libs.ps1 b/tools/fetch_thirdparty_libs.ps1 index 23f0b50c7a..16f7b70e7a 100644 --- a/tools/fetch_thirdparty_libs.ps1 +++ b/tools/fetch_thirdparty_libs.ps1 @@ -17,18 +17,15 @@ $openpype_root = (Get-Item $script_dir).parent.FullName $env:_INSIDE_OPENPYPE_TOOL = "1" -# make sure Poetry is in PATH if (-not (Test-Path 'env:POETRY_HOME')) { $env:POETRY_HOME = "$openpype_root\.poetry" } -$env:PATH = "$($env:PATH);$($env:POETRY_HOME)\bin" Set-Location -Path $openpype_root - Write-Host ">>> " -NoNewline -ForegroundColor Green Write-Host "Reading Poetry ... " -NoNewline -if (-not (Test-Path -PathType Container -Path "$openpype_root\.poetry\bin")) { +if (-not (Test-Path -PathType Container -Path "$($env:POETRY_HOME)\bin")) { Write-Host "NOT FOUND" -ForegroundColor Yellow Write-Host "*** " -NoNewline -ForegroundColor Yellow Write-Host "We need to install Poetry create virtual env first ..." @@ -37,5 +34,5 @@ if (-not (Test-Path -PathType Container -Path "$openpype_root\.poetry\bin")) { Write-Host "OK" -ForegroundColor Green } -& poetry run python "$($openpype_root)\tools\fetch_thirdparty_libs.py" +& "$($env:POETRY_HOME)\bin\poetry" run python "$($openpype_root)\tools\fetch_thirdparty_libs.py" Set-Location -Path $current_dir diff --git a/tools/fetch_thirdparty_libs.sh b/tools/fetch_thirdparty_libs.sh index 31f109ba68..93d0674965 100755 --- a/tools/fetch_thirdparty_libs.sh +++ b/tools/fetch_thirdparty_libs.sh @@ -1,8 +1,5 @@ #!/usr/bin/env bash -# Run Pype Tray - - art () { cat <<-EOF @@ -82,11 +79,9 @@ main () { _inside_openpype_tool="1" - # make sure Poetry is in PATH if [[ -z $POETRY_HOME ]]; then export POETRY_HOME="$openpype_root/.poetry" fi - export PATH="$POETRY_HOME/bin:$PATH" echo -e "${BIGreen}>>>${RST} Reading Poetry ... \c" if [ -f "$POETRY_HOME/bin/poetry" ]; then @@ -100,7 +95,7 @@ main () { pushd "$openpype_root" > /dev/null || return > /dev/null echo -e "${BIGreen}>>>${RST} Running Pype tool ..." - poetry run python "$openpype_root/tools/fetch_thirdparty_libs.py" + "$POETRY_HOME/bin/poetry" run python "$openpype_root/tools/fetch_thirdparty_libs.py" } main \ No newline at end of file diff --git a/tools/make_docs.ps1 b/tools/make_docs.ps1 index 2f9350eff0..45a11171ae 100644 --- a/tools/make_docs.ps1 +++ b/tools/make_docs.ps1 @@ -19,11 +19,9 @@ $openpype_root = (Get-Item $script_dir).parent.FullName $env:_INSIDE_OPENPYPE_TOOL = "1" -# make sure Poetry is in PATH if (-not (Test-Path 'env:POETRY_HOME')) { $env:POETRY_HOME = "$openpype_root\.poetry" } -$env:PATH = "$($env:PATH);$($env:POETRY_HOME)\bin" Set-Location -Path $openpype_root @@ -50,7 +48,7 @@ Write-Host $art -ForegroundColor DarkGreen Write-Host ">>> " -NoNewline -ForegroundColor Green Write-Host "Reading Poetry ... " -NoNewline -if (-not (Test-Path -PathType Container -Path "$openpype_root\.poetry\bin")) { +if (-not (Test-Path -PathType Container -Path "$($env:POETRY_HOME)\bin")) { Write-Host "NOT FOUND" -ForegroundColor Yellow Write-Host "*** " -NoNewline -ForegroundColor Yellow Write-Host "We need to install Poetry create virtual env first ..." @@ -63,10 +61,10 @@ Write-Host "This will not overwrite existing source rst files, only scan and add Set-Location -Path $openpype_root Write-Host ">>> " -NoNewline -ForegroundColor green Write-Host "Running apidoc ..." -& poetry run sphinx-apidoc -M -e -d 10 --ext-intersphinx --ext-todo --ext-coverage --ext-viewcode -o "$($openpype_root)\docs\source" igniter -& poetry run sphinx-apidoc.exe -M -e -d 10 --ext-intersphinx --ext-todo --ext-coverage --ext-viewcode -o "$($openpype_root)\docs\source" openpype vendor, openpype\vendor +& "$env:POETRY_HOME\bin\poetry" run sphinx-apidoc -M -e -d 10 --ext-intersphinx --ext-todo --ext-coverage --ext-viewcode -o "$($openpype_root)\docs\source" igniter +& "$env:POETRY_HOME\bin\poetry" run sphinx-apidoc.exe -M -e -d 10 --ext-intersphinx --ext-todo --ext-coverage --ext-viewcode -o "$($openpype_root)\docs\source" openpype vendor, openpype\vendor Write-Host ">>> " -NoNewline -ForegroundColor green Write-Host "Building html ..." -& poetry run python "$($openpype_root)\setup.py" build_sphinx +& "$env:POETRY_HOME\bin\poetry" run python "$($openpype_root)\setup.py" build_sphinx Set-Location -Path $current_dir diff --git a/tools/make_docs.sh b/tools/make_docs.sh index 9dfab26a38..52ee57dcf0 100755 --- a/tools/make_docs.sh +++ b/tools/make_docs.sh @@ -83,11 +83,9 @@ main () { _inside_openpype_tool="1" - # make sure Poetry is in PATH if [[ -z $POETRY_HOME ]]; then export POETRY_HOME="$openpype_root/.poetry" fi - export PATH="$POETRY_HOME/bin:$PATH" echo -e "${BIGreen}>>>${RST} Reading Poetry ... \c" if [ -f "$POETRY_HOME/bin/poetry" ]; then @@ -101,11 +99,11 @@ main () { pushd "$openpype_root" > /dev/null || return > /dev/null echo -e "${BIGreen}>>>${RST} Running apidoc ..." - poetry run sphinx-apidoc -M -e -d 10 --ext-intersphinx --ext-todo --ext-coverage --ext-viewcode -o "$openpype_root/docs/source" igniter - poetry run sphinx-apidoc -M -e -d 10 --ext-intersphinx --ext-todo --ext-coverage --ext-viewcode -o "$openpype_root/docs/source" openpype vendor, openpype\vendor + "$POETRY_HOME/bin/poetry" run sphinx-apidoc -M -e -d 10 --ext-intersphinx --ext-todo --ext-coverage --ext-viewcode -o "$openpype_root/docs/source" igniter + "$POETRY_HOME/bin/poetry" run sphinx-apidoc -M -e -d 10 --ext-intersphinx --ext-todo --ext-coverage --ext-viewcode -o "$openpype_root/docs/source" openpype vendor, openpype\vendor echo -e "${BIGreen}>>>${RST} Building html ..." - poetry run python3 "$openpype_root/setup.py" build_sphinx + "$POETRY_HOME/bin/poetry" run python3 "$openpype_root/setup.py" build_sphinx } main diff --git a/tools/run_project_manager.ps1 b/tools/run_project_manager.ps1 index 9886a80316..a9cfbb1e7b 100644 --- a/tools/run_project_manager.ps1 +++ b/tools/run_project_manager.ps1 @@ -47,7 +47,7 @@ Set-Location -Path $openpype_root Write-Host ">>> " -NoNewline -ForegroundColor Green Write-Host "Reading Poetry ... " -NoNewline -if (-not (Test-Path -PathType Container -Path "$openpype_root\.poetry\bin")) { +if (-not (Test-Path -PathType Container -Path "$($env:POETRY_HOME)\bin")) { Write-Host "NOT FOUND" -ForegroundColor Yellow Write-Host "*** " -NoNewline -ForegroundColor Yellow Write-Host "We need to install Poetry create virtual env first ..." @@ -56,5 +56,5 @@ if (-not (Test-Path -PathType Container -Path "$openpype_root\.poetry\bin")) { Write-Host "OK" -ForegroundColor Green } -& poetry run python "$($openpype_root)\start.py" projectmanager +& "$env:POETRY_HOME\bin\poetry" run python "$($openpype_root)\start.py" projectmanager Set-Location -Path $current_dir diff --git a/tools/run_projectmanager.sh b/tools/run_projectmanager.sh index 312f321d67..b5c858c34a 100755 --- a/tools/run_projectmanager.sh +++ b/tools/run_projectmanager.sh @@ -79,11 +79,9 @@ main () { _inside_openpype_tool="1" - # make sure Poetry is in PATH if [[ -z $POETRY_HOME ]]; then export POETRY_HOME="$openpype_root/.poetry" fi - export PATH="$POETRY_HOME/bin:$PATH" pushd "$openpype_root" > /dev/null || return > /dev/null @@ -97,7 +95,7 @@ main () { fi echo -e "${BIGreen}>>>${RST} Generating zip from current sources ..." - poetry run python "$openpype_root/start.py" projectmanager + "$POETRY_HOME/bin/poetry" run python "$openpype_root/start.py" projectmanager } main diff --git a/tools/run_settings.ps1 b/tools/run_settings.ps1 index 7477e546b3..1c0aa6e8f3 100644 --- a/tools/run_settings.ps1 +++ b/tools/run_settings.ps1 @@ -27,7 +27,7 @@ Set-Location -Path $openpype_root Write-Host ">>> " -NoNewline -ForegroundColor Green Write-Host "Reading Poetry ... " -NoNewline -if (-not (Test-Path -PathType Container -Path "$openpype_root\.poetry\bin")) { +if (-not (Test-Path -PathType Container -Path "$($env:POETRY_HOME)\bin")) { Write-Host "NOT FOUND" -ForegroundColor Yellow Write-Host "*** " -NoNewline -ForegroundColor Yellow Write-Host "We need to install Poetry create virtual env first ..." @@ -36,5 +36,5 @@ if (-not (Test-Path -PathType Container -Path "$openpype_root\.poetry\bin")) { Write-Host "OK" -ForegroundColor Green } -& poetry run python "$($openpype_root)\start.py" settings --dev +& "$env:POETRY_HOME\bin\poetry" run python "$($openpype_root)\start.py" settings --dev Set-Location -Path $current_dir \ No newline at end of file diff --git a/tools/run_settings.sh b/tools/run_settings.sh index 0287043bb6..5a465dce2c 100755 --- a/tools/run_settings.sh +++ b/tools/run_settings.sh @@ -79,11 +79,9 @@ main () { _inside_openpype_tool="1" - # make sure Poetry is in PATH if [[ -z $POETRY_HOME ]]; then export POETRY_HOME="$openpype_root/.poetry" fi - export PATH="$POETRY_HOME/bin:$PATH" pushd "$openpype_root" > /dev/null || return > /dev/null @@ -97,7 +95,7 @@ main () { fi echo -e "${BIGreen}>>>${RST} Generating zip from current sources ..." - poetry run python3 "$openpype_root/start.py" settings --dev + "$POETRY_HOME/bin/poetry" run python3 "$openpype_root/start.py" settings --dev } main diff --git a/tools/run_tests.ps1 b/tools/run_tests.ps1 index a6882e2a09..e631cb72df 100644 --- a/tools/run_tests.ps1 +++ b/tools/run_tests.ps1 @@ -59,11 +59,9 @@ $openpype_root = (Get-Item $script_dir).parent.FullName $env:_INSIDE_OPENPYPE_TOOL = "1" -# make sure Poetry is in PATH if (-not (Test-Path 'env:POETRY_HOME')) { $env:POETRY_HOME = "$openpype_root\.poetry" } -$env:PATH = "$($env:PATH);$($env:POETRY_HOME)\bin" Set-Location -Path $openpype_root @@ -83,7 +81,7 @@ Write-Host " ] ..." -ForegroundColor white Write-Host ">>> " -NoNewline -ForegroundColor Green Write-Host "Reading Poetry ... " -NoNewline -if (-not (Test-Path -PathType Container -Path "$openpype_root\.poetry\bin")) { +if (-not (Test-Path -PathType Container -Path "$($env:POETRY_HOME)\bin")) { Write-Host "NOT FOUND" -ForegroundColor Yellow Write-Host "*** " -NoNewline -ForegroundColor Yellow Write-Host "We need to install Poetry create virtual env first ..." @@ -102,7 +100,7 @@ Write-Host ">>> " -NoNewline -ForegroundColor green Write-Host "Testing OpenPype ..." $original_pythonpath = $env:PYTHONPATH $env:PYTHONPATH="$($openpype_root);$($env:PYTHONPATH)" -& poetry run pytest -x --capture=sys --print -W ignore::DeprecationWarning "$($openpype_root)/tests" +& "$env:POETRY_HOME\bin\poetry" run pytest -x --capture=sys --print -W ignore::DeprecationWarning "$($openpype_root)/tests" $env:PYTHONPATH = $original_pythonpath Write-Host ">>> " -NoNewline -ForegroundColor green diff --git a/tools/run_tests.sh b/tools/run_tests.sh index 90977edc83..8f8f82fd9c 100755 --- a/tools/run_tests.sh +++ b/tools/run_tests.sh @@ -98,11 +98,9 @@ main () { _inside_openpype_tool="1" - # make sure Poetry is in PATH if [[ -z $POETRY_HOME ]]; then export POETRY_HOME="$openpype_root/.poetry" fi - export PATH="$POETRY_HOME/bin:$PATH" echo -e "${BIGreen}>>>${RST} Reading Poetry ... \c" if [ -f "$POETRY_HOME/bin/poetry" ]; then @@ -118,7 +116,7 @@ main () { echo -e "${BIGreen}>>>${RST} Testing OpenPype ..." original_pythonpath=$PYTHONPATH export PYTHONPATH="$openpype_root:$PYTHONPATH" - poetry run pytest -x --capture=sys --print -W ignore::DeprecationWarning "$openpype_root/tests" + "$POETRY_HOME/bin/poetry" run pytest -x --capture=sys --print -W ignore::DeprecationWarning "$openpype_root/tests" PYTHONPATH=$original_pythonpath } diff --git a/tools/run_tray.ps1 b/tools/run_tray.ps1 index 533a791836..872c1524a6 100644 --- a/tools/run_tray.ps1 +++ b/tools/run_tray.ps1 @@ -26,7 +26,7 @@ Set-Location -Path $openpype_root Write-Host ">>> " -NoNewline -ForegroundColor Green Write-Host "Reading Poetry ... " -NoNewline -if (-not (Test-Path -PathType Container -Path "$openpype_root\.poetry\bin")) { +if (-not (Test-Path -PathType Container -Path "$($env:POETRY_HOME)\bin")) { Write-Host "NOT FOUND" -ForegroundColor Yellow Write-Host "*** " -NoNewline -ForegroundColor Yellow Write-Host "We need to install Poetry create virtual env first ..." @@ -35,5 +35,5 @@ if (-not (Test-Path -PathType Container -Path "$openpype_root\.poetry\bin")) { Write-Host "OK" -ForegroundColor Green } -& poetry run python "$($openpype_root)\start.py" tray --debug +& "$($env:POETRY_HOME)\bin\poetry" run python "$($openpype_root)\start.py" tray --debug Set-Location -Path $current_dir \ No newline at end of file diff --git a/tools/run_tray.sh b/tools/run_tray.sh index 339ff6f918..2eb9886063 100755 --- a/tools/run_tray.sh +++ b/tools/run_tray.sh @@ -56,11 +56,9 @@ main () { _inside_openpype_tool="1" - # make sure Poetry is in PATH if [[ -z $POETRY_HOME ]]; then export POETRY_HOME="$openpype_root/.poetry" fi - export PATH="$POETRY_HOME/bin:$PATH" echo -e "${BIGreen}>>>${RST} Reading Poetry ... \c" if [ -f "$POETRY_HOME/bin/poetry" ]; then @@ -74,7 +72,7 @@ main () { pushd "$openpype_root" > /dev/null || return > /dev/null echo -e "${BIGreen}>>>${RST} Running OpenPype Tray with debug option ..." - poetry run python3 "$openpype_root/start.py" tray --debug + "$POETRY_HOME/bin/poetry" run python3 "$openpype_root/start.py" tray --debug } main \ No newline at end of file diff --git a/website/docs/admin_openpype_commands.md b/website/docs/admin_openpype_commands.md index 324e0e8481..1a91e2e7fe 100644 --- a/website/docs/admin_openpype_commands.md +++ b/website/docs/admin_openpype_commands.md @@ -21,6 +21,8 @@ openpype_console --use-version=3.0.0-foo+bar `--use-staging` - to use staging versions of OpenPype. +`--list-versions [--use-staging]` - to list available versions. + For more information [see here](admin_use#run-openpype). ## Commands diff --git a/website/docs/admin_use.md b/website/docs/admin_use.md index 4a2b56e6f4..4ad08a0174 100644 --- a/website/docs/admin_use.md +++ b/website/docs/admin_use.md @@ -46,6 +46,16 @@ openpype_console --use-version=3.0.1 `--use-staging` - to specify you prefer staging version. In that case it will be used (if found) instead of production one. +:::tip List available versions +To list all available versions, use: + +```shell +openpype_console --list-versions +``` + +You can add `--use-staging` to list staging versions. +::: + ### Details When you run OpenPype from executable, few check are made: diff --git a/website/docs/dev_build.md b/website/docs/dev_build.md index 2c4bd1e9af..b3e0c24fc2 100644 --- a/website/docs/dev_build.md +++ b/website/docs/dev_build.md @@ -137,12 +137,12 @@ $ pyenv install -v 3.7.10 $ cd /path/to/pype-3 # set local python version -$ pyenv local 3.7.9 +$ pyenv local 3.7.10 ``` :::note Install build requirements for **Ubuntu** ```shell -sudo apt-get update; sudo apt-get install --no-install-recommends make build-essential libssl-dev zlib1g-dev libbz2-dev libreadline-dev libsqlite3-dev wget curl llvm libncurses5-dev xz-utils tk-dev libxml2-dev libxmlsec1-dev libffi-dev liblzma-dev git +sudo apt-get update; sudo apt-get install --no-install-recommends make build-essential libssl-dev zlib1g-dev libbz2-dev libreadline-dev libsqlite3-dev wget curl llvm libncurses5-dev xz-utils tk-dev libxml2-dev libxmlsec1-dev libffi-dev liblzma-dev git patchelf ``` In case you run in error about `xcb` when running Pype, diff --git a/website/docs/module_ftrack.md b/website/docs/module_ftrack.md index c3b9fd6bc2..005270b3b9 100644 --- a/website/docs/module_ftrack.md +++ b/website/docs/module_ftrack.md @@ -34,7 +34,7 @@ To prepare Ftrack for working with OpenPype you'll need to run [OpenPype Admin - Ftrack Event Server is the key to automation of many tasks like _status change_, _thumbnail update_, _automatic synchronization to Avalon database_ and many more. Event server should run at all times to perform the required processing as it is not possible to catch some of them retrospectively with enough certainty. ### Running event server -There are specific launch arguments for event server. With `openpype eventserver` you can launch event server but without prior preparation it will terminate immediately. The reason is that event server requires 3 pieces of information: _Ftrack server url_, _paths to events_ and _credentials (Username and API key)_. Ftrack server URL and Event path are set from OpenPype's environments by default, but the credentials must be done separatelly for security reasons. +There are specific launch arguments for event server. With `openpype_console eventserver` you can launch event server but without prior preparation it will terminate immediately. The reason is that event server requires 3 pieces of information: _Ftrack server url_, _paths to events_ and _credentials (Username and API key)_. Ftrack server URL and Event path are set from OpenPype's environments by default, but the credentials must be done separatelly for security reasons. @@ -56,7 +56,7 @@ There are specific launch arguments for event server. With `openpype eventserver - `--ftrack-url "https://yourdomain.ftrackapp.com/"` : Ftrack server URL _(it is not needed to enter if you have set `FTRACK_SERVER` in OpenPype' environments)_ - `--ftrack-events-path "//Paths/To/Events/"` : Paths to events folder. May contain multiple paths separated by `;`. _(it is not needed to enter if you have set `FTRACK_EVENTS_PATH` in OpenPype' environments)_ -So if you want to use OpenPype's environments then you can launch event server for first time with these arguments `openpype.exe eventserver --ftrack-user "my.username" --ftrack-api-key "00000aaa-11bb-22cc-33dd-444444eeeee" --store-credentials`. Since that time, if everything was entered correctly, you can launch event server with `openpype.exe eventserver`. +So if you want to use OpenPype's environments then you can launch event server for first time with these arguments `openpype_console.exe eventserver --ftrack-user "my.username" --ftrack-api-key "00000aaa-11bb-22cc-33dd-444444eeeee" --store-credentials`. Since that time, if everything was entered correctly, you can launch event server with `openpype_console.exe eventserver`. @@ -100,14 +100,17 @@ Event server should **not** run more than once! It may cause major issues. - create file: - `sudo vi /opt/OpenPype/run_event_server.sh` + `sudo vi /opt/openpype/run_event_server.sh` - add content to the file: ```sh -#!\usr\bin\env +#!/usr/bin/env export OPENPYPE_DEBUG=3 pushd /mnt/pipeline/prod/openpype-setup -. openpype eventserver --ftrack-user --ftrack-api-key +. openpype_console eventserver --ftrack-user --ftrack-api-key ``` +- change file permission: + `sudo chmod 0755 /opt/openpype/run_event_server.sh` + - create service file: `sudo vi /etc/systemd/system/openpype-ftrack-event-server.service` - add content to the service file @@ -145,7 +148,7 @@ WantedBy=multi-user.target @echo off set OPENPYPE_DEBUG=3 pushd \\path\to\file\ -call openpype.bat eventserver --ftrack-user --ftrack-api-key +openpype_console.exe eventserver --ftrack-user --ftrack-api-key ``` - download and install `nssm.cc` - create Windows service according to nssm.cc manual