Frame13/ayon-core
1
0
Fork 1
mirror of https://github.com/ynput/ayon-core.git synced 2025-12-25 05:14:40 +01:00
Code Issues Projects Releases Packages Wiki Activity Actions

Merge branch 'develop' into feature/OP-1933_nuke-toggle-baking-colorspace

Browse source
This commit is contained in:
Jakub Jezek 2021-11-12 11:48:51 +01:00
commit d337d3f54c
No known key found for this signature in database
GPG key ID: D8548FBF690B100A
33 changed files with 2626 additions and 2087 deletions
Download patch file Download diff file Expand all files Collapse all files

28
CHANGELOG.md
View file

@ -1,6 +1,6 @@
# Changelog
## [3.6.0-nightly.4](https://github.com/pypeclub/OpenPype/tree/HEAD)
## [3.6.0-nightly.5](https://github.com/pypeclub/OpenPype/tree/HEAD)
[Full Changelog](https://github.com/pypeclub/OpenPype/compare/3.5.0...HEAD)
@ -9,10 +9,17 @@
- Maya : Colorspace configuration [\#2170](https://github.com/pypeclub/OpenPype/pull/2170)
- Blender: Added support for audio [\#2168](https://github.com/pypeclub/OpenPype/pull/2168)
- Flame: a host basic integration [\#2165](https://github.com/pypeclub/OpenPype/pull/2165)
- Houdini: simple HDA workflow [\#2072](https://github.com/pypeclub/OpenPype/pull/2072)
**🚀 Enhancements**
- Add both side availability on Site Sync sites to Loader [\#2220](https://github.com/pypeclub/OpenPype/pull/2220)
- Tools: Center loader and library loader on show [\#2219](https://github.com/pypeclub/OpenPype/pull/2219)
- Maya : Validate shape zero [\#2212](https://github.com/pypeclub/OpenPype/pull/2212)
- Maya : validate unique names [\#2211](https://github.com/pypeclub/OpenPype/pull/2211)
- Tools: OpenPype stylesheet in workfiles tool [\#2208](https://github.com/pypeclub/OpenPype/pull/2208)
- Ftrack: Replace Queue with deque in event handlers logic [\#2204](https://github.com/pypeclub/OpenPype/pull/2204)
- Tools: New select context dialog [\#2200](https://github.com/pypeclub/OpenPype/pull/2200)
- Maya : Validate mesh ngons [\#2199](https://github.com/pypeclub/OpenPype/pull/2199)
- Delivery: Check 'frame' key in template for sequence delivery [\#2196](https://github.com/pypeclub/OpenPype/pull/2196)
- Usage of tools code [\#2185](https://github.com/pypeclub/OpenPype/pull/2185)
@ -21,11 +28,11 @@
- Tools: Experimental tools [\#2167](https://github.com/pypeclub/OpenPype/pull/2167)
- Loader: Refactor and use OpenPype stylesheets [\#2166](https://github.com/pypeclub/OpenPype/pull/2166)
- Add loader for linked smart objects in photoshop [\#2149](https://github.com/pypeclub/OpenPype/pull/2149)
- Burnins: DNxHD profiles handling [\#2142](https://github.com/pypeclub/OpenPype/pull/2142)
- Tools: Single access point for host tools [\#2139](https://github.com/pypeclub/OpenPype/pull/2139)
**🐛 Bug fixes**
- Maya : multiple subsets review broken [\#2210](https://github.com/pypeclub/OpenPype/pull/2210)
- Fix - different command used for Linux and Mac OS [\#2207](https://github.com/pypeclub/OpenPype/pull/2207)
- Tools: Workfiles tool don't use avalon widgets [\#2205](https://github.com/pypeclub/OpenPype/pull/2205)
- Ftrack: Fill missing ftrack id on mongo project [\#2203](https://github.com/pypeclub/OpenPype/pull/2203)
- Project Manager: Fix copying of tasks [\#2191](https://github.com/pypeclub/OpenPype/pull/2191)
@ -34,10 +41,7 @@
- MacOS: Launching of applications may cause Permissions error [\#2175](https://github.com/pypeclub/OpenPype/pull/2175)
- Maya: Aspect ratio [\#2174](https://github.com/pypeclub/OpenPype/pull/2174)
- Blender: Fix 'Deselect All' with object not in 'Object Mode' [\#2163](https://github.com/pypeclub/OpenPype/pull/2163)
- Tools: Stylesheets are applied after tool show [\#2161](https://github.com/pypeclub/OpenPype/pull/2161)
- Maya: Collect render - fix UNC path support 🐛 [\#2158](https://github.com/pypeclub/OpenPype/pull/2158)
- Maya: Fix hotbox broken by scriptsmenu [\#2151](https://github.com/pypeclub/OpenPype/pull/2151)
- Ftrack: Ignore save warnings exception in Prepare project action [\#2150](https://github.com/pypeclub/OpenPype/pull/2150)
- Added validator for source files for Standalone Publisher [\#2138](https://github.com/pypeclub/OpenPype/pull/2138)
**Merged pull requests:**
@ -45,7 +49,6 @@
- Settings: Site sync project settings improvement [\#2193](https://github.com/pypeclub/OpenPype/pull/2193)
- Add validate active site button to sync queue on a project [\#2176](https://github.com/pypeclub/OpenPype/pull/2176)
- Bump pillow from 8.2.0 to 8.3.2 [\#2162](https://github.com/pypeclub/OpenPype/pull/2162)
- Bump axios from 0.21.1 to 0.21.4 in /website [\#2059](https://github.com/pypeclub/OpenPype/pull/2059)
## [3.5.0](https://github.com/pypeclub/OpenPype/tree/3.5.0) (2021-10-17)
@ -62,8 +65,6 @@
- PYPE-1218 - changed namespace to contain subset name in Maya [\#2114](https://github.com/pypeclub/OpenPype/pull/2114)
- Added running configurable disk mapping command before start of OP [\#2091](https://github.com/pypeclub/OpenPype/pull/2091)
- SFTP provider [\#2073](https://github.com/pypeclub/OpenPype/pull/2073)
- Houdini: simple HDA workflow [\#2072](https://github.com/pypeclub/OpenPype/pull/2072)
- Maya: Validate setdress top group [\#2068](https://github.com/pypeclub/OpenPype/pull/2068)
**🚀 Enhancements**
@ -79,8 +80,6 @@
- Global: add global validators to settings [\#2078](https://github.com/pypeclub/OpenPype/pull/2078)
- Use CRF for burnin when available [\#2070](https://github.com/pypeclub/OpenPype/pull/2070)
- Project manager: Filter first item after selection of project [\#2069](https://github.com/pypeclub/OpenPype/pull/2069)
- Nuke: Adding `still` image family workflow [\#2064](https://github.com/pypeclub/OpenPype/pull/2064)
- Maya: validate authorized loaded plugins [\#2062](https://github.com/pypeclub/OpenPype/pull/2062)
**🐛 Bug fixes**
@ -102,8 +101,6 @@
- Deadline: Collect deadline server does not check existence of deadline key [\#2082](https://github.com/pypeclub/OpenPype/pull/2082)
- Blender: fixed Curves with modifiers in Rigs [\#2081](https://github.com/pypeclub/OpenPype/pull/2081)
- Nuke UI scaling [\#2077](https://github.com/pypeclub/OpenPype/pull/2077)
- Maya: Fix multi-camera renders [\#2065](https://github.com/pypeclub/OpenPype/pull/2065)
- Fix Sync Queue when project disabled [\#2063](https://github.com/pypeclub/OpenPype/pull/2063)
**Merged pull requests:**
@ -113,11 +110,6 @@
[Full Changelog](https://github.com/pypeclub/OpenPype/compare/CI/3.4.1-nightly.1...3.4.1)
**🐛 Bug fixes**
- Timers manger: Typo fix [\#2058](https://github.com/pypeclub/OpenPype/pull/2058)
- Hiero: Editorial fixes [\#2057](https://github.com/pypeclub/OpenPype/pull/2057)
## [3.4.0](https://github.com/pypeclub/OpenPype/tree/3.4.0) (2021-09-17)
[Full Changelog](https://github.com/pypeclub/OpenPype/compare/CI/3.4.0-nightly.6...3.4.0)

13
openpype/cli.py
View file

@ -166,7 +166,7 @@ def publish(debug, paths, targets):
@click.option("-p", "--project", help="Project")
@click.option("-t", "--targets", help="Targets", default=None,
multiple=True)
def remotepublishfromapp(debug, project, path, host, targets=None, user=None):
def remotepublishfromapp(debug, project, path, host, user=None, targets=None):
"""Start CLI publishing.
Publish collects json from paths provided as an argument.
@ -174,18 +174,19 @@ def remotepublishfromapp(debug, project, path, host, targets=None, user=None):
"""
if debug:
os.environ['OPENPYPE_DEBUG'] = '3'
PypeCommands.remotepublishfromapp(project, path, host, user,
targets=targets)
PypeCommands.remotepublishfromapp(
project, path, host, user, targets=targets
)
@main.command()
@click.argument("path")
@click.option("-d", "--debug", is_flag=True, help="Print debug messages")
@click.option("-h", "--host", help="Host")
@click.option("-u", "--user", help="User email address")
@click.option("-p", "--project", help="Project")
@click.option("-t", "--targets", help="Targets", default=None,
multiple=True)
def remotepublish(debug, project, path, host, targets=None, user=None):
def remotepublish(debug, project, path, user=None, targets=None):
"""Start CLI publishing.
Publish collects json from paths provided as an argument.
@ -193,7 +194,7 @@ def remotepublish(debug, project, path, host, targets=None, user=None):
"""
if debug:
os.environ['OPENPYPE_DEBUG'] = '3'
PypeCommands.remotepublish(project, path, host, user, targets=targets)
PypeCommands.remotepublish(project, path, user, targets=targets)
@main.command()

117
openpype/hosts/maya/api/__init__.py
View file

@ -13,6 +13,7 @@ from pyblish import api as pyblish
from openpype.lib import any_outdated
import openpype.hosts.maya
from openpype.hosts.maya.lib import copy_workspace_mel
from openpype.lib.path_tools import HostDirmap
from . import menu, lib
log = logging.getLogger("openpype.hosts.maya")
@ -30,7 +31,8 @@ def install():
project_settings = get_project_settings(os.getenv("AVALON_PROJECT"))
# process path mapping
process_dirmap(project_settings)
dirmap_processor = MayaDirmap("maya", project_settings)
dirmap_processor.process_dirmap()
pyblish.register_plugin_path(PUBLISH_PATH)
avalon.register_plugin_path(avalon.Loader, LOAD_PATH)
@ -60,110 +62,6 @@ def install():
avalon.data["familiesStateToggled"] = ["imagesequence"]
def process_dirmap(project_settings):
# type: (dict) -> None
"""Go through all paths in Settings and set them using `dirmap`.
If artists has Site Sync enabled, take dirmap mapping directly from
Local Settings when artist is syncing workfile locally.
Args:
project_settings (dict): Settings for current project.
"""
local_mapping = _get_local_sync_dirmap(project_settings)
if not project_settings["maya"].get("maya-dirmap") and not local_mapping:
return
mapping = local_mapping or \
project_settings["maya"]["maya-dirmap"]["paths"] \
or {}
mapping_enabled = project_settings["maya"]["maya-dirmap"]["enabled"] \
or bool(local_mapping)
if not mapping or not mapping_enabled:
return
if mapping.get("source-path") and mapping_enabled is True:
log.info("Processing directory mapping ...")
cmds.dirmap(en=True)
for k, sp in enumerate(mapping["source-path"]):
try:
print("{} -> {}".format(sp, mapping["destination-path"][k]))
cmds.dirmap(m=(sp, mapping["destination-path"][k]))
cmds.dirmap(m=(mapping["destination-path"][k], sp))
except IndexError:
# missing corresponding destination path
log.error(("invalid dirmap mapping, missing corresponding"
" destination directory."))
break
except RuntimeError:
log.error("invalid path {} -> {}, mapping not registered".format(
sp, mapping["destination-path"][k]
))
continue
def _get_local_sync_dirmap(project_settings):
"""
Returns dirmap if synch to local project is enabled.
Only valid mapping is from roots of remote site to local site set in
Local Settings.
Args:
project_settings (dict)
Returns:
dict : { "source-path": [XXX], "destination-path": [YYYY]}
"""
import json
mapping = {}
if not project_settings["global"]["sync_server"]["enabled"]:
log.debug("Site Sync not enabled")
return mapping
from openpype.settings.lib import get_site_local_overrides
from openpype.modules import ModulesManager
manager = ModulesManager()
sync_module = manager.modules_by_name["sync_server"]
project_name = os.getenv("AVALON_PROJECT")
sync_settings = sync_module.get_sync_project_setting(
os.getenv("AVALON_PROJECT"), exclude_locals=False, cached=False)
log.debug(json.dumps(sync_settings, indent=4))
active_site = sync_module.get_local_normalized_site(
sync_module.get_active_site(project_name))
remote_site = sync_module.get_local_normalized_site(
sync_module.get_remote_site(project_name))
log.debug("active {} - remote {}".format(active_site, remote_site))
if active_site == "local" \
and project_name in sync_module.get_enabled_projects()\
and active_site != remote_site:
overrides = get_site_local_overrides(os.getenv("AVALON_PROJECT"),
active_site)
for root_name, value in overrides.items():
if os.path.isdir(value):
try:
mapping["destination-path"] = [value]
mapping["source-path"] = [sync_settings["sites"]\
[remote_site]\
["root"]\
[root_name]]
except IndexError:
# missing corresponding destination path
log.debug("overrides".format(overrides))
log.error(
("invalid dirmap mapping, missing corresponding"
" destination directory."))
break
log.debug("local sync mapping:: {}".format(mapping))
return mapping
def uninstall():
pyblish.deregister_plugin_path(PUBLISH_PATH)
avalon.deregister_plugin_path(avalon.Loader, LOAD_PATH)
@ -326,3 +224,12 @@ def before_workfile_save(workfile_path):
workdir = os.path.dirname(workfile_path)
copy_workspace_mel(workdir)
class MayaDirmap(HostDirmap):
def on_enable_dirmap(self):
cmds.dirmap(en=True)
def dirmap_routine(self, source_path, destination_path):
cmds.dirmap(m=(source_path, destination_path))
cmds.dirmap(m=(destination_path, source_path))

11
openpype/hosts/maya/plugins/publish/extract_playblast.py
View file

@ -45,9 +45,12 @@ class ExtractPlayblast(openpype.api.Extractor):
# get cameras
camera = instance.data['review_camera']
override_viewport_options = (
self.capture_preset['Viewport Options']
['override_viewport_options']
)
preset = lib.load_capture_preset(data=self.capture_preset)
preset['camera'] = camera
preset['start_frame'] = start
preset['end_frame'] = end
@ -92,6 +95,12 @@ class ExtractPlayblast(openpype.api.Extractor):
self.log.info('using viewport preset: {}'.format(preset))
# Update preset with current panel setting
# if override_viewport_options is turned off
if not override_viewport_options:
panel_preset = capture.parse_active_view()
preset.update(panel_preset)
path = capture.capture(**preset)
self.log.debug("playblast path {}".format(path))

9
openpype/hosts/maya/plugins/publish/extract_thumbnail.py
View file

@ -32,6 +32,9 @@ class ExtractThumbnail(openpype.api.Extractor):
capture_preset = (
instance.context.data["project_settings"]['maya']['publish']['ExtractPlayblast']['capture_preset']
)
override_viewport_options = (
capture_preset['Viewport Options']['override_viewport_options']
)
try:
preset = lib.load_capture_preset(data=capture_preset)
@ -86,6 +89,12 @@ class ExtractThumbnail(openpype.api.Extractor):
# playblast and viewer
preset['viewer'] = False
# Update preset with current panel setting
# if override_viewport_options is turned off
if not override_viewport_options:
panel_preset = capture.parse_active_view()
preset.update(panel_preset)
path = capture.capture(**preset)
playblast = self._fix_playblast_output_path(path)

70
openpype/hosts/nuke/api/lib.py
View file

@ -24,6 +24,10 @@ from openpype.api import (
ApplicationManager
)
from openpype.tools.utils import host_tools
from openpype.lib.path_tools import HostDirmap
from openpype.settings import get_project_settings
from openpype.modules import ModulesManager
import nuke
from .utils import set_context_favorites
@ -1415,3 +1419,69 @@ def recreate_instance(origin_node, avalon_data=None):
dn.setInput(0, new_node)
return new_node
class NukeDirmap(HostDirmap):
def __init__(self, host_name, project_settings, sync_module, file_name):
"""
Args:
host_name (str): Nuke
project_settings (dict): settings of current project
sync_module (SyncServerModule): to limit reinitialization
file_name (str): full path of referenced file from workfiles
"""
self.host_name = host_name
self.project_settings = project_settings
self.file_name = file_name
self.sync_module = sync_module
self._mapping = None # cache mapping
def on_enable_dirmap(self):
pass
def dirmap_routine(self, source_path, destination_path):
log.debug("{}: {}->{}".format(self.file_name,
source_path, destination_path))
source_path = source_path.lower().replace(os.sep, '/')
destination_path = destination_path.lower().replace(os.sep, '/')
if platform.system().lower() == "windows":
self.file_name = self.file_name.lower().replace(
source_path, destination_path)
else:
self.file_name = self.file_name.replace(
source_path, destination_path)
class DirmapCache:
"""Caching class to get settings and sync_module easily and only once."""
_project_settings = None
_sync_module = None
@classmethod
def project_settings(cls):
if cls._project_settings is None:
cls._project_settings = get_project_settings(
os.getenv("AVALON_PROJECT"))
return cls._project_settings
@classmethod
def sync_module(cls):
if cls._sync_module is None:
cls._sync_module = ModulesManager().modules_by_name["sync_server"]
return cls._sync_module
def dirmap_file_name_filter(file_name):
"""Nuke callback function with single full path argument.
Checks project settings for potential mapping from source to dest.
"""
dirmap_processor = NukeDirmap("nuke",
DirmapCache.project_settings(),
DirmapCache.sync_module(),
file_name)
dirmap_processor.process_dirmap()
if os.path.exists(dirmap_processor.file_name):
return dirmap_processor.file_name
return file_name

4
openpype/hosts/nuke/startup/menu.py
View file

@ -6,10 +6,10 @@ from openpype.hosts.nuke.api.lib import (
import nuke
from openpype.api import Logger
from openpype.hosts.nuke.api.lib import dirmap_file_name_filter
log = Logger().get_logger(__name__)
# fix ffmpeg settings on script
nuke.addOnScriptLoad(on_script_load)
@ -20,4 +20,6 @@ nuke.addOnScriptSave(check_inventory_versions)
# # set apply all workfile settings on script load and save
nuke.addOnScriptLoad(WorkfileSettings().set_context_settings)
nuke.addFilenameFilter(dirmap_file_name_filter)
log.info('Automatic syncing of write file knob to script version')

84
openpype/hosts/webpublisher/plugins/publish/collect_batch_data.py Normal file
View file

@ -0,0 +1,84 @@
"""Loads batch context from json and continues in publish process.
Provides:
context -> Loaded batch file.
"""
import os
import pyblish.api
from avalon import io
from openpype.lib.plugin_tools import (
parse_json,
get_batch_asset_task_info
)
from openpype.lib.remote_publish import get_webpublish_conn
class CollectBatchData(pyblish.api.ContextPlugin):
"""Collect batch data from json stored in 'OPENPYPE_PUBLISH_DATA' env dir.
The directory must contain 'manifest.json' file where batch data should be
stored.
"""
# must be really early, context values are only in json file
order = pyblish.api.CollectorOrder - 0.495
label = "Collect batch data"
host = ["webpublisher"]
def process(self, context):
batch_dir = os.environ.get("OPENPYPE_PUBLISH_DATA")
assert batch_dir, (
"Missing `OPENPYPE_PUBLISH_DATA`")
assert os.path.exists(batch_dir), \
"Folder {} doesn't exist".format(batch_dir)
project_name = os.environ.get("AVALON_PROJECT")
if project_name is None:
raise AssertionError(
"Environment `AVALON_PROJECT` was not found."
"Could not set project `root` which may cause issues."
)
batch_data = parse_json(os.path.join(batch_dir, "manifest.json"))
context.data["batchDir"] = batch_dir
context.data["batchData"] = batch_data
asset_name, task_name, task_type = get_batch_asset_task_info(
batch_data["context"]
)
os.environ["AVALON_ASSET"] = asset_name
io.Session["AVALON_ASSET"] = asset_name
os.environ["AVALON_TASK"] = task_name
io.Session["AVALON_TASK"] = task_name
context.data["asset"] = asset_name
context.data["task"] = task_name
context.data["taskType"] = task_type
self._set_ctx_path(batch_data)
def _set_ctx_path(self, batch_data):
dbcon = get_webpublish_conn()
batch_id = batch_data["batch"]
ctx_path = batch_data["context"]["path"]
self.log.info("ctx_path: {}".format(ctx_path))
self.log.info("batch_id: {}".format(batch_id))
if ctx_path and batch_id:
self.log.info("Updating log record")
dbcon.update_one(
{
"batch_id": batch_id,
"status": "in_progress"
},
{
"$set": {
"path": ctx_path
}
}
)

7
openpype/hosts/webpublisher/plugins/publish/collect_fps.py
View file

@ -20,9 +20,8 @@ class CollectFPS(pyblish.api.InstancePlugin):
hosts = ["webpublisher"]
def process(self, instance):
fps = instance.context.data["fps"]
instance_fps = instance.data.get("fps")
if instance_fps is None:
instance.data["fps"] = instance.context.data["fps"]
instance.data.update({
"fps": fps
})
self.log.debug(f"instance.data: {pformat(instance.data)}")

76
openpype/hosts/webpublisher/plugins/publish/collect_published_files.py
View file

@ -1,21 +1,19 @@
"""Loads publishing context from json and continues in publish process.
"""Create instances from batch data and continues in publish process.
Requires:
anatomy -> context["anatomy"] *(pyblish.api.CollectorOrder - 0.11)
CollectBatchData
Provides:
context, instances -> All data from previous publishing process.
"""
import os
import json
import clique
import tempfile
import pyblish.api
from avalon import io
import pyblish.api
from openpype.lib import prepare_template_data
from openpype.lib.plugin_tools import parse_json, get_batch_asset_task_info
from openpype.lib.plugin_tools import parse_json
class CollectPublishedFiles(pyblish.api.ContextPlugin):
@ -28,28 +26,28 @@ class CollectPublishedFiles(pyblish.api.ContextPlugin):
order = pyblish.api.CollectorOrder - 0.490
label = "Collect rendered frames"
host = ["webpublisher"]
_context = None
targets = ["filespublish"]
# from Settings
task_type_to_family = {}
def _process_batch(self, dir_url):
task_subfolders = [
os.path.join(dir_url, o)
for o in os.listdir(dir_url)
if os.path.isdir(os.path.join(dir_url, o))]
def process(self, context):
batch_dir = context.data["batchDir"]
task_subfolders = []
for folder_name in os.listdir(batch_dir):
full_path = os.path.join(batch_dir, folder_name)
if os.path.isdir(full_path):
task_subfolders.append(full_path)
self.log.info("task_sub:: {}".format(task_subfolders))
asset_name = context.data["asset"]
task_name = context.data["task"]
task_type = context.data["taskType"]
for task_dir in task_subfolders:
task_data = parse_json(os.path.join(task_dir,
"manifest.json"))
self.log.info("task_data:: {}".format(task_data))
ctx = task_data["context"]
asset, task_name, task_type = get_batch_asset_task_info(ctx)
if task_name:
os.environ["AVALON_TASK"] = task_name
is_sequence = len(task_data["files"]) > 1
@ -60,26 +58,20 @@ class CollectPublishedFiles(pyblish.api.ContextPlugin):
is_sequence,
extension.replace(".", ''))
subset = self._get_subset_name(family, subset_template, task_name,
task_data["variant"])
subset = self._get_subset_name(
family, subset_template, task_name, task_data["variant"]
)
version = self._get_last_version(asset_name, subset) + 1
os.environ["AVALON_ASSET"] = asset
io.Session["AVALON_ASSET"] = asset
instance = self._context.create_instance(subset)
instance.data["asset"] = asset
instance = context.create_instance(subset)
instance.data["asset"] = asset_name
instance.data["subset"] = subset
instance.data["family"] = family
instance.data["families"] = families
instance.data["version"] = \
self._get_last_version(asset, subset) + 1
instance.data["version"] = version
instance.data["stagingDir"] = tempfile.mkdtemp()
instance.data["source"] = "webpublisher"
# to store logging info into DB openpype.webpublishes
instance.data["ctx_path"] = ctx["path"]
instance.data["batch_id"] = task_data["batch"]
# to convert from email provided into Ftrack username
instance.data["user_email"] = task_data["user"]
@ -230,23 +222,3 @@ class CollectPublishedFiles(pyblish.api.ContextPlugin):
return version[0].get("version") or 0
else:
return 0
def process(self, context):
self._context = context
batch_dir = os.environ.get("OPENPYPE_PUBLISH_DATA")
assert batch_dir, (
"Missing `OPENPYPE_PUBLISH_DATA`")
assert os.path.exists(batch_dir), \
"Folder {} doesn't exist".format(batch_dir)
project_name = os.environ.get("AVALON_PROJECT")
if project_name is None:
raise AssertionError(
"Environment `AVALON_PROJECT` was not found."
"Could not set project `root` which may cause issues."
)
self._process_batch(batch_dir)

38
openpype/hosts/webpublisher/plugins/publish/integrate_context_to_log.py
View file

@ -1,38 +0,0 @@
import os
import pyblish.api
from openpype.lib import OpenPypeMongoConnection
class IntegrateContextToLog(pyblish.api.ContextPlugin):
""" Adds context information to log document for displaying in front end"""
label = "Integrate Context to Log"
order = pyblish.api.IntegratorOrder - 0.1
hosts = ["webpublisher"]
def process(self, context):
self.log.info("Integrate Context to Log")
mongo_client = OpenPypeMongoConnection.get_mongo_client()
database_name = os.environ["OPENPYPE_DATABASE_NAME"]
dbcon = mongo_client[database_name]["webpublishes"]
for instance in context:
self.log.info("ctx_path: {}".format(instance.data.get("ctx_path")))
self.log.info("batch_id: {}".format(instance.data.get("batch_id")))
if instance.data.get("ctx_path") and instance.data.get("batch_id"):
self.log.info("Updating log record")
dbcon.update_one(
{
"batch_id": instance.data.get("batch_id"),
"status": "in_progress"
},
{"$set":
{
"path": instance.data.get("ctx_path")
}}
)
return

81
openpype/hosts/webpublisher/webserver_service/webpublish_routes.py
View file

@ -176,23 +176,48 @@ class TaskNode(Node):
class WebpublisherBatchPublishEndpoint(_RestApiEndpoint):
"""Triggers headless publishing of batch."""
async def post(self, request) -> Response:
# for postprocessing in host, currently only PS
host_map = {"photoshop": [".psd", ".psb"]}
# Validate existence of openpype executable
openpype_app = self.resource.executable
if not openpype_app or not os.path.exists(openpype_app):
msg = "Non existent OpenPype executable {}".format(openpype_app)
raise RuntimeError(msg)
# for postprocessing in host, currently only PS
output = {}
log.info("WebpublisherBatchPublishEndpoint called")
content = await request.json()
batch_path = os.path.join(self.resource.upload_dir,
content["batch"])
# Each filter have extensions which are checked on first task item
# - first filter with extensions that are on first task is used
# - filter defines command and can extend arguments dictionary
# This is used only if 'studio_processing' is enabled on batch
studio_processing_filters = [
# Photoshop filter
{
"extensions": [".psd", ".psb"],
"command": "remotepublishfromapp",
"arguments": {
# Command 'remotepublishfromapp' requires --host argument
"host": "photoshop",
# Make sure targets are set to None for cases that default
# would change
# - targets argument is not used in 'remotepublishfromapp'
"targets": None
}
}
]
add_args = {
"host": "webpublisher",
"project": content["project_name"],
"user": content["user"]
}
batch_path = os.path.join(self.resource.upload_dir, content["batch"])
# Default command and arguments
command = "remotepublish"
add_args = {
# All commands need 'project' and 'user'
"project": content["project_name"],
"user": content["user"],
"targets": ["filespublish"]
}
if content.get("studio_processing"):
log.info("Post processing called")
@ -208,32 +233,34 @@ class WebpublisherBatchPublishEndpoint(_RestApiEndpoint):
raise ValueError(
"Cannot parse batch meta in {} folder".format(task_data))
command = "remotepublishfromapp"
for host, extensions in host_map.items():
for ext in extensions:
for file_name in task_data["files"]:
if ext in file_name:
add_args["host"] = host
break
for process_filter in studio_processing_filters:
filter_extensions = process_filter.get("extensions") or []
for file_name in task_data["files"]:
file_ext = os.path.splitext(file_name)[-1].lower()
if file_ext in filter_extensions:
# Change command
command = process_filter["command"]
# Update arguments
add_args.update(
process_filter.get("arguments") or {}
)
break
if not add_args.get("host"):
raise ValueError(
"Couldn't discern host from {}".format(task_data["files"]))
openpype_app = self.resource.executable
args = [
openpype_app,
command,
batch_path
]
if not openpype_app or not os.path.exists(openpype_app):
msg = "Non existent OpenPype executable {}".format(openpype_app)
raise RuntimeError(msg)
for key, value in add_args.items():
args.append("--{}".format(key))
args.append(value)
# Skip key values where value is None
if value is not None:
args.append("--{}".format(key))
# Extend list into arguments (targets can be a list)
if isinstance(value, (tuple, list)):
args.extend(value)
else:
args.append(value)
log.info("args:: {}".format(args))

158
openpype/lib/path_tools.py
View file

@ -2,6 +2,8 @@ import json
import logging
import os
import re
import abc
import six
from .anatomy import Anatomy
@ -196,3 +198,159 @@ def get_project_basic_paths(project_name):
if isinstance(folder_structure, str):
folder_structure = json.loads(folder_structure)
return _list_path_items(folder_structure)
@six.add_metaclass(abc.ABCMeta)
class HostDirmap:
"""
Abstract class for running dirmap on a workfile in a host.
Dirmap is used to translate paths inside of host workfile from one
OS to another. (Eg. arstist created workfile on Win, different artists
opens same file on Linux.)
Expects methods to be implemented inside of host:
on_dirmap_enabled: run host code for enabling dirmap
do_dirmap: run host code to do actual remapping
"""
def __init__(self, host_name, project_settings, sync_module=None):
self.host_name = host_name
self.project_settings = project_settings
self.sync_module = sync_module # to limit reinit of Modules
self._mapping = None # cache mapping
@abc.abstractmethod
def on_enable_dirmap(self):
"""
Run host dependent operation for enabling dirmap if necessary.
"""
@abc.abstractmethod
def dirmap_routine(self, source_path, destination_path):
"""
Run host dependent remapping from source_path to destination_path
"""
def process_dirmap(self):
# type: (dict) -> None
"""Go through all paths in Settings and set them using `dirmap`.
If artists has Site Sync enabled, take dirmap mapping directly from
Local Settings when artist is syncing workfile locally.
Args:
project_settings (dict): Settings for current project.
"""
if not self._mapping:
self._mapping = self.get_mappings(self.project_settings)
if not self._mapping:
return
log.info("Processing directory mapping ...")
self.on_enable_dirmap()
log.info("mapping:: {}".format(self._mapping))
for k, sp in enumerate(self._mapping["source-path"]):
try:
print("{} -> {}".format(sp,
self._mapping["destination-path"][k]))
self.dirmap_routine(sp,
self._mapping["destination-path"][k])
except IndexError:
# missing corresponding destination path
log.error(("invalid dirmap mapping, missing corresponding"
" destination directory."))
break
except RuntimeError:
log.error("invalid path {} -> {}, mapping not registered".format( # noqa: E501
sp, self._mapping["destination-path"][k]
))
continue
def get_mappings(self, project_settings):
"""Get translation from source-path to destination-path.
It checks if Site Sync is enabled and user chose to use local
site, in that case configuration in Local Settings takes precedence
"""
local_mapping = self._get_local_sync_dirmap(project_settings)
dirmap_label = "{}-dirmap".format(self.host_name)
if not self.project_settings[self.host_name].get(dirmap_label) and \
not local_mapping:
return []
mapping = local_mapping or \
self.project_settings[self.host_name][dirmap_label]["paths"] or {}
enbled = self.project_settings[self.host_name][dirmap_label]["enabled"]
mapping_enabled = enbled or bool(local_mapping)
if not mapping or not mapping_enabled or \
not mapping.get("destination-path") or \
not mapping.get("source-path"):
return []
return mapping
def _get_local_sync_dirmap(self, project_settings):
"""
Returns dirmap if synch to local project is enabled.
Only valid mapping is from roots of remote site to local site set
in Local Settings.
Args:
project_settings (dict)
Returns:
dict : { "source-path": [XXX], "destination-path": [YYYY]}
"""
import json
mapping = {}
if not project_settings["global"]["sync_server"]["enabled"]:
log.debug("Site Sync not enabled")
return mapping
from openpype.settings.lib import get_site_local_overrides
if not self.sync_module:
from openpype.modules import ModulesManager
manager = ModulesManager()
self.sync_module = manager.modules_by_name["sync_server"]
project_name = os.getenv("AVALON_PROJECT")
active_site = self.sync_module.get_local_normalized_site(
self.sync_module.get_active_site(project_name))
remote_site = self.sync_module.get_local_normalized_site(
self.sync_module.get_remote_site(project_name))
log.debug("active {} - remote {}".format(active_site, remote_site))
if active_site == "local" \
and project_name in self.sync_module.get_enabled_projects()\
and active_site != remote_site:
sync_settings = self.sync_module.get_sync_project_setting(
os.getenv("AVALON_PROJECT"), exclude_locals=False,
cached=False)
active_overrides = get_site_local_overrides(
os.getenv("AVALON_PROJECT"), active_site)
remote_overrides = get_site_local_overrides(
os.getenv("AVALON_PROJECT"), remote_site)
log.debug("local overrides".format(active_overrides))
log.debug("remote overrides".format(remote_overrides))
for root_name, active_site_dir in active_overrides.items():
remote_site_dir = remote_overrides.get(root_name) or\
sync_settings["sites"][remote_site]["root"][root_name]
if os.path.isdir(active_site_dir):
if not mapping.get("destination-path"):
mapping["destination-path"] = []
mapping["destination-path"].append(active_site_dir)
if not mapping.get("source-path"):
mapping["source-path"] = []
mapping["source-path"].append(remote_site_dir)
log.debug("local sync mapping:: {}".format(mapping))
return mapping

37
openpype/modules/default_modules/sync_server/tray/delegates.py
View file

@ -4,6 +4,18 @@ from Qt import QtCore, QtWidgets, QtGui
from openpype.lib import PypeLogger
from . import lib
from openpype.tools.utils.constants import (
LOCAL_PROVIDER_ROLE,
REMOTE_PROVIDER_ROLE,
LOCAL_PROGRESS_ROLE,
REMOTE_PROGRESS_ROLE,
LOCAL_DATE_ROLE,
REMOTE_DATE_ROLE,
LOCAL_FAILED_ROLE,
REMOTE_FAILED_ROLE,
EDIT_ICON_ROLE
)
log = PypeLogger().get_logger("SyncServer")
@ -14,7 +26,7 @@ class PriorityDelegate(QtWidgets.QStyledItemDelegate):
if option.widget.selectionModel().isSelected(index) or \
option.state & QtWidgets.QStyle.State_MouseOver:
edit_icon = index.data(lib.EditIconRole)
edit_icon = index.data(EDIT_ICON_ROLE)
if not edit_icon:
return
@ -38,7 +50,7 @@ class PriorityDelegate(QtWidgets.QStyledItemDelegate):
editor = PriorityLineEdit(
parent,
option.widget.selectionModel().selectedRows())
editor.setFocus(True)
editor.setFocus()
return editor
def setModelData(self, editor, model, index):
@ -71,19 +83,30 @@ class ImageDelegate(QtWidgets.QStyledItemDelegate):
Prints icon of site and progress of synchronization
"""
def __init__(self, parent=None):
def __init__(self, parent=None, side=None):
super(ImageDelegate, self).__init__(parent)
self.icons = {}
self.side = side
def paint(self, painter, option, index):
super(ImageDelegate, self).paint(painter, option, index)
option = QtWidgets.QStyleOptionViewItem(option)
option.showDecorationSelected = True
provider = index.data(lib.ProviderRole)
value = index.data(lib.ProgressRole)
date_value = index.data(lib.DateRole)
is_failed = index.data(lib.FailedRole)
if not self.side:
log.warning("No side provided, delegate won't work")
return
if self.side == 'local':
provider = index.data(LOCAL_PROVIDER_ROLE)
value = index.data(LOCAL_PROGRESS_ROLE)
date_value = index.data(LOCAL_DATE_ROLE)
is_failed = index.data(LOCAL_FAILED_ROLE)
else:
provider = index.data(REMOTE_PROVIDER_ROLE)
value = index.data(REMOTE_PROGRESS_ROLE)
date_value = index.data(REMOTE_DATE_ROLE)
is_failed = index.data(REMOTE_FAILED_ROLE)
if not self.icons.get(provider):
resource_path = os.path.dirname(__file__)

15
openpype/modules/default_modules/sync_server/tray/lib.py
View file

@ -1,4 +1,3 @@
from Qt import QtCore
import attr
import abc
import six
@ -19,14 +18,6 @@ STATUS = {
DUMMY_PROJECT = "No project configured"
ProviderRole = QtCore.Qt.UserRole + 2
ProgressRole = QtCore.Qt.UserRole + 4
DateRole = QtCore.Qt.UserRole + 6
FailedRole = QtCore.Qt.UserRole + 8
HeaderNameRole = QtCore.Qt.UserRole + 10
FullItemRole = QtCore.Qt.UserRole + 12
EditIconRole = QtCore.Qt.UserRole + 14
@six.add_metaclass(abc.ABCMeta)
class AbstractColumnFilter:
@ -161,7 +152,7 @@ def translate_provider_for_icon(sync_server, project, site):
return sync_server.get_provider_for_site(site=site)
def get_item_by_id(model, object_id):
def get_value_from_id_by_role(model, object_id, role):
"""Return value from item with 'object_id' with 'role'."""
index = model.get_index(object_id)
item = model.data(index, FullItemRole)
return item
return model.data(index, role)

175
openpype/modules/default_modules/sync_server/tray/models.py
View file

@ -13,6 +13,23 @@ from openpype.api import get_local_site_id
from . import lib
from openpype.tools.utils.constants import (
LOCAL_PROVIDER_ROLE,
REMOTE_PROVIDER_ROLE,
LOCAL_PROGRESS_ROLE,
REMOTE_PROGRESS_ROLE,
HEADER_NAME_ROLE,
EDIT_ICON_ROLE,
LOCAL_DATE_ROLE,
REMOTE_DATE_ROLE,
LOCAL_FAILED_ROLE,
REMOTE_FAILED_ROLE,
STATUS_ROLE,
PATH_ROLE,
ERROR_ROLE,
TRIES_ROLE
)
log = PypeLogger().get_logger("SyncServer")
@ -68,10 +85,68 @@ class _SyncRepresentationModel(QtCore.QAbstractTableModel):
if orientation == Qt.Horizontal:
return self.COLUMN_LABELS[section][1]
if role == lib.HeaderNameRole:
if role == HEADER_NAME_ROLE:
if orientation == Qt.Horizontal:
return self.COLUMN_LABELS[section][0] # return name
def data(self, index, role):
item = self._data[index.row()]
header_value = self._header[index.column()]
if role == LOCAL_PROVIDER_ROLE:
return item.local_provider
if role == REMOTE_PROVIDER_ROLE:
return item.remote_provider
if role == LOCAL_PROGRESS_ROLE:
return item.local_progress
if role == REMOTE_PROGRESS_ROLE:
return item.remote_progress
if role == LOCAL_DATE_ROLE:
if item.created_dt:
return pretty_timestamp(item.created_dt)
if role == REMOTE_DATE_ROLE:
if item.sync_dt:
return pretty_timestamp(item.sync_dt)
if role == LOCAL_FAILED_ROLE:
return item.status == lib.STATUS[2] and \
item.local_progress < 1
if role == REMOTE_FAILED_ROLE:
return item.status == lib.STATUS[2] and \
item.remote_progress < 1
if role in (Qt.DisplayRole, Qt.EditRole):
# because of ImageDelegate
if header_value in ['remote_site', 'local_site']:
return ""
return attr.asdict(item)[self._header[index.column()]]
if role == EDIT_ICON_ROLE:
if self.can_edit and header_value in self.EDITABLE_COLUMNS:
return self.edit_icon
if role == PATH_ROLE:
return item.path
if role == ERROR_ROLE:
return item.error
if role == TRIES_ROLE:
return item.tries
if role == STATUS_ROLE:
return item.status
if role == Qt.UserRole:
return item._id
@property
def can_edit(self):
"""Returns true if some site is user local site, eg. could edit"""
@ -456,55 +531,6 @@ class SyncRepresentationSummaryModel(_SyncRepresentationModel):
self.timer.timeout.connect(self.tick)
self.timer.start(self.REFRESH_SEC)
def data(self, index, role):
item = self._data[index.row()]
if role == lib.FullItemRole:
return item
header_value = self._header[index.column()]
if role == lib.ProviderRole:
if header_value == 'local_site':
return item.local_provider
if header_value == 'remote_site':
return item.remote_provider
if role == lib.ProgressRole:
if header_value == 'local_site':
return item.local_progress
if header_value == 'remote_site':
return item.remote_progress
if role == lib.DateRole:
if header_value == 'local_site':
if item.created_dt:
return pretty_timestamp(item.created_dt)
if header_value == 'remote_site':
if item.sync_dt:
return pretty_timestamp(item.sync_dt)
if role == lib.FailedRole:
if header_value == 'local_site':
return item.status == lib.STATUS[2] and \
item.local_progress < 1
if header_value == 'remote_site':
return item.status == lib.STATUS[2] and \
item.remote_progress < 1
if role in (Qt.DisplayRole, Qt.EditRole):
# because of ImageDelegate
if header_value in ['remote_site', 'local_site']:
return ""
return attr.asdict(item)[self._header[index.column()]]
if role == lib.EditIconRole:
if self.can_edit and header_value in self.EDITABLE_COLUMNS:
return self.edit_icon
if role == Qt.UserRole:
return item._id
def add_page_records(self, local_site, remote_site, representations):
"""
Process all records from 'representation' and add them to storage.
@ -985,55 +1011,6 @@ class SyncRepresentationDetailModel(_SyncRepresentationModel):
self.timer.timeout.connect(self.tick)
self.timer.start(SyncRepresentationSummaryModel.REFRESH_SEC)
def data(self, index, role):
item = self._data[index.row()]
if role == lib.FullItemRole:
return item
header_value = self._header[index.column()]
if role == lib.ProviderRole:
if header_value == 'local_site':
return item.local_provider
if header_value == 'remote_site':
return item.remote_provider
if role == lib.ProgressRole:
if header_value == 'local_site':
return item.local_progress
if header_value == 'remote_site':
return item.remote_progress
if role == lib.DateRole:
if header_value == 'local_site':
if item.created_dt:
return pretty_timestamp(item.created_dt)
if header_value == 'remote_site':
if item.sync_dt:
return pretty_timestamp(item.sync_dt)
if role == lib.FailedRole:
if header_value == 'local_site':
return item.status == lib.STATUS[2] and \
item.local_progress < 1
if header_value == 'remote_site':
return item.status == lib.STATUS[2] and \
item.remote_progress < 1
if role in (Qt.DisplayRole, Qt.EditRole):
# because of ImageDelegate
if header_value in ['remote_site', 'local_site']:
return ""
return attr.asdict(item)[self._header[index.column()]]
if role == lib.EditIconRole:
if self.can_edit and header_value in self.EDITABLE_COLUMNS:
return self.edit_icon
if role == Qt.UserRole:
return item._id
def add_page_records(self, local_site, remote_site, representations):
"""
Process all records from 'representation' and add them to storage.

144
openpype/modules/default_modules/sync_server/tray/widgets.py
View file

@ -22,6 +22,20 @@ from .models import (
from . import lib
from . import delegates
from openpype.tools.utils.constants import (
LOCAL_PROGRESS_ROLE,
REMOTE_PROGRESS_ROLE,
HEADER_NAME_ROLE,
STATUS_ROLE,
PATH_ROLE,
LOCAL_SITE_NAME_ROLE,
REMOTE_SITE_NAME_ROLE,
LOCAL_DATE_ROLE,
REMOTE_DATE_ROLE,
ERROR_ROLE,
TRIES_ROLE
)
log = PypeLogger().get_logger("SyncServer")
@ -289,14 +303,19 @@ class _SyncRepresentationWidget(QtWidgets.QWidget):
if is_multi:
index = self.model.get_index(list(self._selected_ids)[0])
item = self.model.data(index, lib.FullItemRole)
local_progress = self.model.data(index, LOCAL_PROGRESS_ROLE)
remote_progress = self.model.data(index, REMOTE_PROGRESS_ROLE)
status = self.model.data(index, STATUS_ROLE)
else:
item = self.model.data(point_index, lib.FullItemRole)
local_progress = self.model.data(point_index, LOCAL_PROGRESS_ROLE)
remote_progress = self.model.data(point_index,
REMOTE_PROGRESS_ROLE)
status = self.model.data(point_index, STATUS_ROLE)
can_edit = self.model.can_edit
action_kwarg_map, actions_mapping, menu = self._prepare_menu(item,
is_multi,
can_edit)
action_kwarg_map, actions_mapping, menu = self._prepare_menu(
local_progress, remote_progress, is_multi, can_edit, status)
result = menu.exec_(QtGui.QCursor.pos())
if result:
@ -307,7 +326,8 @@ class _SyncRepresentationWidget(QtWidgets.QWidget):
self.model.refresh()
def _prepare_menu(self, item, is_multi, can_edit):
def _prepare_menu(self, local_progress, remote_progress,
is_multi, can_edit, status=None):
menu = QtWidgets.QMenu(self)
actions_mapping = {}
@ -316,11 +336,6 @@ class _SyncRepresentationWidget(QtWidgets.QWidget):
active_site = self.model.active_site
remote_site = self.model.remote_site
local_progress = item.local_progress
remote_progress = item.remote_progress
project = self.model.project
for site, progress in {active_site: local_progress,
remote_site: remote_progress}.items():
provider = self.sync_server.get_provider_for_site(site=site)
@ -360,12 +375,6 @@ class _SyncRepresentationWidget(QtWidgets.QWidget):
actions_mapping[action] = self._change_priority
menu.addAction(action)
# # temp for testing only !!!
# action = QtWidgets.QAction("Download")
# action_kwarg_map[action] = self._get_action_kwargs(active_site)
# actions_mapping[action] = self._add_site
# menu.addAction(action)
if not actions_mapping:
action = QtWidgets.QAction("< No action >")
actions_mapping[action] = None
@ -376,11 +385,15 @@ class _SyncRepresentationWidget(QtWidgets.QWidget):
def _pause(self, selected_ids=None):
log.debug("Pause {}".format(selected_ids))
for representation_id in selected_ids:
item = lib.get_item_by_id(self.model, representation_id)
if item.status not in [lib.STATUS[0], lib.STATUS[1]]:
status = lib.get_value_from_id_by_role(self.model,
representation_id,
STATUS_ROLE)
if status not in [lib.STATUS[0], lib.STATUS[1]]:
continue
for site_name in [self.model.active_site, self.model.remote_site]:
check_progress = self._get_progress(item, site_name)
check_progress = self._get_progress(self.model,
representation_id,
site_name)
if check_progress < 1:
self.sync_server.pause_representation(self.model.project,
representation_id,
@ -391,11 +404,15 @@ class _SyncRepresentationWidget(QtWidgets.QWidget):
def _unpause(self, selected_ids=None):
log.debug("UnPause {}".format(selected_ids))
for representation_id in selected_ids:
item = lib.get_item_by_id(self.model, representation_id)
if item.status not in lib.STATUS[3]:
status = lib.get_value_from_id_by_role(self.model,
representation_id,
STATUS_ROLE)
if status not in lib.STATUS[3]:
continue
for site_name in [self.model.active_site, self.model.remote_site]:
check_progress = self._get_progress(item, site_name)
check_progress = self._get_progress(self.model,
representation_id,
site_name)
if check_progress < 1:
self.sync_server.unpause_representation(
self.model.project,
@ -408,8 +425,11 @@ class _SyncRepresentationWidget(QtWidgets.QWidget):
def _add_site(self, selected_ids=None, site_name=None):
log.debug("Add site {}:{}".format(selected_ids, site_name))
for representation_id in selected_ids:
item = lib.get_item_by_id(self.model, representation_id)
if item.local_site == site_name or item.remote_site == site_name:
item_local_site = lib.get_value_from_id_by_role(
self.model, representation_id, LOCAL_SITE_NAME_ROLE)
item_remote_site = lib.get_value_from_id_by_role(
self.model, representation_id, REMOTE_SITE_NAME_ROLE)
if site_name in [item_local_site, item_remote_site]:
# site already exists skip
continue
@ -460,8 +480,8 @@ class _SyncRepresentationWidget(QtWidgets.QWidget):
"""
log.debug("Reset site {}:{}".format(selected_ids, site_name))
for representation_id in selected_ids:
item = lib.get_item_by_id(self.model, representation_id)
check_progress = self._get_progress(item, site_name, True)
check_progress = self._get_progress(self.model, representation_id,
site_name, True)
# do not reset if opposite side is not fully there
if check_progress != 1:
@ -482,11 +502,8 @@ class _SyncRepresentationWidget(QtWidgets.QWidget):
def _open_in_explorer(self, selected_ids=None, site_name=None):
log.debug("Open in Explorer {}:{}".format(selected_ids, site_name))
for selected_id in selected_ids:
item = lib.get_item_by_id(self.model, selected_id)
if not item:
return
fpath = item.path
fpath = lib.get_value_from_id_by_role(self.model, selected_id,
PATH_ROLE)
project = self.model.project
fpath = self.sync_server.get_local_file_path(project,
site_name,
@ -514,10 +531,17 @@ class _SyncRepresentationWidget(QtWidgets.QWidget):
self.model.is_editing = True
self.table_view.openPersistentEditor(real_index)
def _get_progress(self, item, site_name, opposite=False):
def _get_progress(self, model, representation_id,
site_name, opposite=False):
"""Returns progress value according to site (side)"""
progress = {'local': item.local_progress,
'remote': item.remote_progress}
local_progress = lib.get_value_from_id_by_role(model,
representation_id,
LOCAL_PROGRESS_ROLE)
remote_progress = lib.get_value_from_id_by_role(model,
representation_id,
REMOTE_PROGRESS_ROLE)
progress = {'local': local_progress,
'remote': remote_progress}
side = 'remote'
if site_name == self.model.active_site:
side = 'local'
@ -591,11 +615,11 @@ class SyncRepresentationSummaryWidget(_SyncRepresentationWidget):
table_view.viewport().setAttribute(QtCore.Qt.WA_Hover, True)
column = table_view.model().get_header_index("local_site")
delegate = delegates.ImageDelegate(self)
delegate = delegates.ImageDelegate(self, side="local")
table_view.setItemDelegateForColumn(column, delegate)
column = table_view.model().get_header_index("remote_site")
delegate = delegates.ImageDelegate(self)
delegate = delegates.ImageDelegate(self, side="remote")
table_view.setItemDelegateForColumn(column, delegate)
column = table_view.model().get_header_index("priority")
@ -631,19 +655,21 @@ class SyncRepresentationSummaryWidget(_SyncRepresentationWidget):
self.selection_model = self.table_view.selectionModel()
self.selection_model.selectionChanged.connect(self._selection_changed)
def _prepare_menu(self, item, is_multi, can_edit):
def _prepare_menu(self, local_progress, remote_progress,
is_multi, can_edit, status=None):
action_kwarg_map, actions_mapping, menu = \
super()._prepare_menu(item, is_multi, can_edit)
super()._prepare_menu(local_progress, remote_progress,
is_multi, can_edit)
if can_edit and (
item.status in [lib.STATUS[0], lib.STATUS[1]] or is_multi):
status in [lib.STATUS[0], lib.STATUS[1]] or is_multi):
action = QtWidgets.QAction("Pause in queue")
actions_mapping[action] = self._pause
# pause handles which site_name it will pause itself
action_kwarg_map[action] = {"selected_ids": self._selected_ids}
menu.addAction(action)
if can_edit and (item.status == lib.STATUS[3] or is_multi):
if can_edit and (status == lib.STATUS[3] or is_multi):
action = QtWidgets.QAction("Unpause in queue")
actions_mapping[action] = self._unpause
action_kwarg_map[action] = {"selected_ids": self._selected_ids}
@ -753,11 +779,11 @@ class SyncRepresentationDetailWidget(_SyncRepresentationWidget):
table_view.verticalHeader().hide()
column = model.get_header_index("local_site")
delegate = delegates.ImageDelegate(self)
delegate = delegates.ImageDelegate(self, side="local")
table_view.setItemDelegateForColumn(column, delegate)
column = model.get_header_index("remote_site")
delegate = delegates.ImageDelegate(self)
delegate = delegates.ImageDelegate(self, side="remote")
table_view.setItemDelegateForColumn(column, delegate)
if model.can_edit:
@ -815,12 +841,14 @@ class SyncRepresentationDetailWidget(_SyncRepresentationWidget):
detail_window.exec()
def _prepare_menu(self, item, is_multi, can_edit):
def _prepare_menu(self, local_progress, remote_progress,
is_multi, can_edit, status=None):
"""Adds view (and model) dependent actions to default ones"""
action_kwarg_map, actions_mapping, menu = \
super()._prepare_menu(item, is_multi, can_edit)
super()._prepare_menu(local_progress, remote_progress,
is_multi, can_edit, status)
if item.status == lib.STATUS[2] or is_multi:
if status == lib.STATUS[2] or is_multi:
action = QtWidgets.QAction("Open error detail")
actions_mapping[action] = self._show_detail
action_kwarg_map[action] = {"selected_ids": self._selected_ids}
@ -835,8 +863,8 @@ class SyncRepresentationDetailWidget(_SyncRepresentationWidget):
redo of upload/download
"""
for file_id in selected_ids:
item = lib.get_item_by_id(self.model, file_id)
check_progress = self._get_progress(item, site_name, True)
check_progress = self._get_progress(self.model, file_id,
site_name, True)
# do not reset if opposite side is not fully there
if check_progress != 1:
@ -895,20 +923,28 @@ class SyncRepresentationErrorWidget(QtWidgets.QWidget):
no_errors = True
for file_id in selected_ids:
item = lib.get_item_by_id(model, file_id)
if not item.created_dt or not item.sync_dt or not item.error:
created_dt = lib.get_value_from_id_by_role(model, file_id,
LOCAL_DATE_ROLE)
sync_dt = lib.get_value_from_id_by_role(model, file_id,
REMOTE_DATE_ROLE)
errors = lib.get_value_from_id_by_role(model, file_id,
ERROR_ROLE)
if not created_dt or not sync_dt or not errors:
continue
tries = lib.get_value_from_id_by_role(model, file_id,
TRIES_ROLE)
no_errors = False
dt = max(item.created_dt, item.sync_dt)
dt = max(created_dt, sync_dt)
txts = []
txts.append("{}: {}<br>".format("<b>Last update date</b>",
pretty_timestamp(dt)))
txts.append("{}: {}<br>".format("<b>Retries</b>",
str(item.tries)))
str(tries)))
txts.append("{}: {}<br>".format("<b>Error message</b>",
item.error))
errors))
text_area = QtWidgets.QTextEdit("\n\n".join(txts))
text_area.setReadOnly(True)
@ -1162,7 +1198,7 @@ class HorizontalHeader(QtWidgets.QHeaderView):
column_name = self.model.headerData(column_idx,
QtCore.Qt.Horizontal,
lib.HeaderNameRole)
HEADER_NAME_ROLE)
button = self.filter_buttons.get(column_name)
if not button:
continue

2
openpype/plugins/publish/extract_review.py
View file

@ -666,6 +666,8 @@ class ExtractReview(pyblish.api.InstancePlugin):
AssertionError: if more then one collection is obtained.
"""
start_frame = int(start_frame)
end_frame = int(end_frame)
collections = clique.assemble(files)[0]
assert len(collections) == 1, "Multiple collections found."
col = collections[0]

37
openpype/pype_commands.py
View file

@ -137,16 +137,13 @@ class PypeCommands:
SLEEP = 5 # seconds for another loop check for concurrently runs
WAIT_FOR = 300 # seconds to wait for conc. runs
from openpype import install, uninstall
from openpype.api import Logger
from openpype.lib import ApplicationManager
log = Logger.get_logger()
log.info("remotepublishphotoshop command")
install()
from openpype.lib import ApplicationManager
application_manager = ApplicationManager()
found_variant_key = find_variant_key(application_manager, host)
@ -220,10 +217,8 @@ class PypeCommands:
while launched_app.poll() is None:
time.sleep(0.5)
uninstall()
@staticmethod
def remotepublish(project, batch_path, host, user, targets=None):
def remotepublish(project, batch_path, user, targets=None):
"""Start headless publishing.
Used to publish rendered assets, workfiles etc.
@ -235,10 +230,9 @@ class PypeCommands:
per call of remotepublish
batch_path (str): Path batch folder. Contains subfolders with
resources (workfile, another subfolder 'renders' etc.)
targets (string): What module should be targeted
(to choose validator for example)
host (string)
user (string): email address for webpublisher
targets (list): Pyblish targets
(to choose validator for example)
Raises:
RuntimeError: When there is no path to process.
@ -246,21 +240,22 @@ class PypeCommands:
if not batch_path:
raise RuntimeError("No publish paths specified")
from openpype import install, uninstall
from openpype.api import Logger
# Register target and host
import pyblish.api
import pyblish.util
import avalon.api
from openpype.hosts.webpublisher import api as webpublisher
log = Logger.get_logger()
log = PypeLogger.get_logger()
log.info("remotepublish command")
install()
host_name = "webpublisher"
os.environ["OPENPYPE_PUBLISH_DATA"] = batch_path
os.environ["AVALON_PROJECT"] = project
os.environ["AVALON_APP"] = host_name
if host:
pyblish.api.register_host(host)
pyblish.api.register_host(host_name)
if targets:
if isinstance(targets, str):
@ -268,13 +263,6 @@ class PypeCommands:
for target in targets:
pyblish.api.register_target(target)
os.environ["OPENPYPE_PUBLISH_DATA"] = batch_path
os.environ["AVALON_PROJECT"] = project
os.environ["AVALON_APP"] = host
import avalon.api
from openpype.hosts.webpublisher import api as webpublisher
avalon.api.install(webpublisher)
log.info("Running publish ...")
@ -286,7 +274,6 @@ class PypeCommands:
publish_and_log(dbcon, _id, log)
log.info("Publish finished.")
uninstall()
@staticmethod
def extractenvironments(output_json_path, project, asset, task, app):

12
openpype/settings/defaults/project_settings/maya.json
View file

@ -8,16 +8,10 @@
"yetiRig": "ma"
},
"maya-dirmap": {
"enabled": true,
"enabled": false,
"paths": {
"source-path": [
"foo1",
"foo2"
],
"destination-path": [
"bar1",
"bar2"
]
"source-path": [],
"destination-path": []
}
},
"scriptsmenu": {

7
openpype/settings/defaults/project_settings/nuke.json
View file

@ -8,6 +8,13 @@
"build_workfile": "ctrl+alt+b"
}
},
"nuke-dirmap": {
"enabled": false,
"paths": {
"source-path": [],
"destination-path": []
}
},
"create": {
"CreateWriteRender": {
"fpath_template": "{work}/renders/nuke/{subset}/{subset}.{frame}.{ext}",

33
openpype/settings/entities/schemas/projects_schema/schema_project_nuke.json
View file

@ -46,6 +46,39 @@
}
]
},
{
"type": "dict",
"collapsible": true,
"checkbox_key": "enabled",
"key": "nuke-dirmap",
"label": "Nuke Directory Mapping",
"is_group": true,
"children": [
{
"type": "boolean",
"key": "enabled",
"label": "Enabled"
},
{
"type": "dict",
"key": "paths",
"children": [
{
"type": "list",
"object_type": "text",
"key": "source-path",
"label": "Source Path"
},
{
"type": "list",
"object_type": "text",
"key": "destination-path",
"label": "Destination Path"
}
]
}
]
},
{
"type": "dict",
"collapsible": true,

BIN
openpype/style/images/transparent.png
View file

Binary file not shown.
Side by side Swipe Overlay

Before

Width:  |  Height:  |  Size: 69 B

After

Width:  |  Height:  |  Size: 1.4 KiB

Before After
Before After

1597
openpype/style/pyqt5_resources.py
View file

File diff suppressed because it is too large Load diff

1770
openpype/style/pyside2_resources.py
View file

File diff suppressed because it is too large Load diff

6
openpype/tools/libraryloader/app.py
View file

@ -407,7 +407,8 @@ class LibraryLoaderWindow(QtWidgets.QDialog):
self.data["state"]["assetIds"] = asset_ids
# reset repre list
self._repres_widget.set_version_ids([])
if self._repres_widget:
self._repres_widget.set_version_ids([])
def _subsetschanged(self):
asset_ids = self.data["state"]["assetIds"]
@ -497,7 +498,8 @@ class LibraryLoaderWindow(QtWidgets.QDialog):
self._thumbnail_widget.set_thumbnail(thumbnail_docs)
version_ids = [doc["_id"] for doc in version_docs or []]
self._repres_widget.set_version_ids(version_ids)
if self._repres_widget:
self._repres_widget.set_version_ids(version_ids)
def _set_context(self, context, refresh=True):
"""Set the selection in the interface using a context.

91
openpype/tools/loader/model.py
View file

@ -15,6 +15,12 @@ from openpype.tools.utils.models import TreeModel, Item
from openpype.tools.utils import lib
from openpype.modules import ModulesManager
from openpype.tools.utils.constants import (
LOCAL_PROVIDER_ROLE,
REMOTE_PROVIDER_ROLE,
LOCAL_AVAILABILITY_ROLE,
REMOTE_AVAILABILITY_ROLE
)
def is_filtering_recursible():
@ -333,7 +339,6 @@ class SubsetsModel(TreeModel, BaseRepresentationModel):
repre_info = version_data.get("repre_info")
if repre_info:
item["repre_info"] = repre_info
item["repre_icon"] = version_data.get("repre_icon")
def _fetch(self):
asset_docs = self.dbcon.find(
@ -445,14 +450,16 @@ class SubsetsModel(TreeModel, BaseRepresentationModel):
for _subset_id, doc in last_versions_by_subset_id.items():
version_ids.add(doc["_id"])
site = self.active_site
query = self._repre_per_version_pipeline(list(version_ids), site)
query = self._repre_per_version_pipeline(list(version_ids),
self.active_site,
self.remote_site)
repre_info = {}
for doc in self.dbcon.aggregate(query):
if self._doc_fetching_stop:
return
doc["provider"] = self.active_provider
doc["active_provider"] = self.active_provider
doc["remote_provider"] = self.remote_provider
repre_info[doc["_id"]] = doc
self._doc_payload["repre_info_by_version_id"] = repre_info
@ -666,8 +673,8 @@ class SubsetsModel(TreeModel, BaseRepresentationModel):
if not index.isValid():
return
item = index.internalPointer()
if role == self.SortDescendingRole:
item = index.internalPointer()
if item.get("isGroup"):
# Ensure groups be on top when sorting by descending order
prefix = "2"
@ -683,7 +690,6 @@ class SubsetsModel(TreeModel, BaseRepresentationModel):
return prefix + order
if role == self.SortAscendingRole:
item = index.internalPointer()
if item.get("isGroup"):
# Ensure groups be on top when sorting by ascending order
prefix = "0"
@ -701,14 +707,12 @@ class SubsetsModel(TreeModel, BaseRepresentationModel):
if role == QtCore.Qt.DisplayRole:
if index.column() == self.columns_index["family"]:
# Show familyLabel instead of family
item = index.internalPointer()
return item.get("familyLabel", None)
elif role == QtCore.Qt.DecorationRole:
# Add icon to subset column
if index.column() == self.columns_index["subset"]:
item = index.internalPointer()
if item.get("isGroup") or item.get("isMerged"):
return item["icon"]
else:
@ -716,20 +720,32 @@ class SubsetsModel(TreeModel, BaseRepresentationModel):
# Add icon to family column
if index.column() == self.columns_index["family"]:
item = index.internalPointer()
return item.get("familyIcon", None)
if index.column() == self.columns_index.get("repre_info"):
item = index.internalPointer()
return item.get("repre_icon", None)
elif role == QtCore.Qt.ForegroundRole:
item = index.internalPointer()
version_doc = item.get("version_document")
if version_doc and version_doc.get("type") == "hero_version":
if not version_doc["is_from_latest"]:
return self.not_last_hero_brush
elif role == LOCAL_AVAILABILITY_ROLE:
if not item.get("isGroup"):
return item.get("repre_info_local")
else:
return None
elif role == REMOTE_AVAILABILITY_ROLE:
if not item.get("isGroup"):
return item.get("repre_info_remote")
else:
return None
elif role == LOCAL_PROVIDER_ROLE:
return self.active_provider
elif role == REMOTE_PROVIDER_ROLE:
return self.remote_provider
return super(SubsetsModel, self).data(index, role)
def flags(self, index):
@ -759,19 +775,25 @@ class SubsetsModel(TreeModel, BaseRepresentationModel):
return data
def _get_repre_dict(self, repre_info):
"""Returns icon and str representation of availability"""
"""Returns str representation of availability"""
data = {}
if repre_info:
repres_str = "{}/{}".format(
int(math.floor(float(repre_info['avail_repre']))),
int(math.floor(float(repre_info['avail_repre_local']))),
int(math.floor(float(repre_info['repre_count']))))
data["repre_info"] = repres_str
data["repre_icon"] = self.repre_icons.get(self.active_provider)
data["repre_info_local"] = repres_str
repres_str = "{}/{}".format(
int(math.floor(float(repre_info['avail_repre_remote']))),
int(math.floor(float(repre_info['repre_count']))))
data["repre_info_remote"] = repres_str
return data
def _repre_per_version_pipeline(self, version_ids, site):
def _repre_per_version_pipeline(self, version_ids,
active_site, remote_site):
query = [
{"$match": {"parent": {"$in": version_ids},
"type": "representation",
@ -780,7 +802,13 @@ class SubsetsModel(TreeModel, BaseRepresentationModel):
{'$addFields': {
'order_local': {
'$filter': {'input': '$files.sites', 'as': 'p',
'cond': {'$eq': ['$$p.name', site]}
'cond': {'$eq': ['$$p.name', active_site]}
}}
}},
{'$addFields': {
'order_remote': {
'$filter': {'input': '$files.sites', 'as': 'p',
'cond': {'$eq': ['$$p.name', remote_site]}
}}
}},
{'$addFields': {
@ -795,19 +823,32 @@ class SubsetsModel(TreeModel, BaseRepresentationModel):
]}
]}, 0]}
}},
{'$addFields': {
'progress_remote': {"$arrayElemAt": [{
'$cond': [{'$size': "$order_remote.progress"},
"$order_remote.progress",
# if exists created_dt count is as available
{'$cond': [
{'$size': "$order_remote.created_dt"},
[1],
[0]
]}
]}, 0]}
}},
{'$group': { # first group by repre
'_id': '$_id',
'parent': {'$first': '$parent'},
'files_count': {'$sum': 1},
'files_avail': {'$sum': "$progress_local"},
'avail_ratio': {'$first': {
'$divide': [{'$sum': "$progress_local"}, {'$sum': 1}]}}
'avail_ratio_local': {'$first': {
'$divide': [{'$sum': "$progress_local"}, {'$sum': 1}]}},
'avail_ratio_remote': {'$first': {
'$divide': [{'$sum': "$progress_remote"}, {'$sum': 1}]}}
}},
{'$group': { # second group by parent, eg version_id
'_id': '$parent',
'repre_count': {'$sum': 1}, # total representations
# fully available representation for site
'avail_repre': {'$sum': "$avail_ratio"}
'avail_repre_local': {'$sum': "$avail_ratio_local"},
'avail_repre_remote': {'$sum': "$avail_ratio_remote"},
}},
]
return query

62
openpype/tools/loader/widgets.py
View file

@ -31,6 +31,13 @@ from .model import (
)
from . import lib
from openpype.tools.utils.constants import (
LOCAL_PROVIDER_ROLE,
REMOTE_PROVIDER_ROLE,
LOCAL_AVAILABILITY_ROLE,
REMOTE_AVAILABILITY_ROLE
)
class OverlayFrame(QtWidgets.QFrame):
def __init__(self, label, parent):
@ -197,6 +204,10 @@ class SubsetWidget(QtWidgets.QWidget):
column = model.Columns.index("time")
view.setItemDelegateForColumn(column, time_delegate)
avail_delegate = AvailabilityDelegate(self.dbcon, view)
column = model.Columns.index("repre_info")
view.setItemDelegateForColumn(column, avail_delegate)
layout = QtWidgets.QVBoxLayout(self)
layout.setContentsMargins(0, 0, 0, 0)
layout.addLayout(top_bar_layout)
@ -1578,3 +1589,54 @@ def _load_subsets_by_loader(loader, subset_contexts, options,
))
return error_info
class AvailabilityDelegate(QtWidgets.QStyledItemDelegate):
"""
Prints icons and downloaded representation ration for both sides.
"""
def __init__(self, dbcon, parent=None):
super(AvailabilityDelegate, self).__init__(parent)
self.icons = tools_lib.get_repre_icons()
def paint(self, painter, option, index):
super(AvailabilityDelegate, self).paint(painter, option, index)
option = QtWidgets.QStyleOptionViewItem(option)
option.showDecorationSelected = True
provider_active = index.data(LOCAL_PROVIDER_ROLE)
provider_remote = index.data(REMOTE_PROVIDER_ROLE)
availability_active = index.data(LOCAL_AVAILABILITY_ROLE)
availability_remote = index.data(REMOTE_AVAILABILITY_ROLE)
if not availability_active or not availability_remote: # group lines
return
idx = 0
height = width = 24
for value, provider in [(availability_active, provider_active),
(availability_remote, provider_remote)]:
icon = self.icons.get(provider)
if not icon:
continue
pixmap = icon.pixmap(icon.actualSize(QtCore.QSize(height, width)))
padding = 10 + (70 * idx)
point = QtCore.QPoint(option.rect.x() + padding,
option.rect.y() +
(option.rect.height() - pixmap.height()) / 2)
painter.drawPixmap(point, pixmap)
text_rect = option.rect.translated(padding + width + 10, 0)
painter.drawText(
text_rect,
option.displayAlignment,
value
)
idx += 1
def displayText(self, value, locale):
pass

4
openpype/tools/pyblish_pype/control.py
View file

@ -190,7 +190,9 @@ class Controller(QtCore.QObject):
plugins = pyblish.api.discover()
targets = pyblish.logic.registered_targets() or ["default"]
targets = set(pyblish.logic.registered_targets())
targets.add("default")
targets = list(targets)
plugins_by_targets = pyblish.logic.plugins_by_targets(plugins, targets)
_plugins = []

19
openpype/tools/utils/constants.py
View file

@ -8,3 +8,22 @@ PROJECT_IS_ACTIVE_ROLE = QtCore.Qt.UserRole + 102
TASK_NAME_ROLE = QtCore.Qt.UserRole + 301
TASK_TYPE_ROLE = QtCore.Qt.UserRole + 302
TASK_ORDER_ROLE = QtCore.Qt.UserRole + 403
LOCAL_PROVIDER_ROLE = QtCore.Qt.UserRole + 500 # provider of active site
REMOTE_PROVIDER_ROLE = QtCore.Qt.UserRole + 501 # provider of remote site
LOCAL_PROGRESS_ROLE = QtCore.Qt.UserRole + 502 # percentage downld on active
REMOTE_PROGRESS_ROLE = QtCore.Qt.UserRole + 503 # percentage upload on remote
LOCAL_AVAILABILITY_ROLE = QtCore.Qt.UserRole + 504 # ratio of presence active
REMOTE_AVAILABILITY_ROLE = QtCore.Qt.UserRole + 505
LOCAL_DATE_ROLE = QtCore.Qt.UserRole + 506 # created_dt on active site
REMOTE_DATE_ROLE = QtCore.Qt.UserRole + 507
LOCAL_FAILED_ROLE = QtCore.Qt.UserRole + 508
REMOTE_FAILED_ROLE = QtCore.Qt.UserRole + 509
HEADER_NAME_ROLE = QtCore.Qt.UserRole + 510
EDIT_ICON_ROLE = QtCore.Qt.UserRole + 511
STATUS_ROLE = QtCore.Qt.UserRole + 512
PATH_ROLE = QtCore.Qt.UserRole + 513
LOCAL_SITE_NAME_ROLE = QtCore.Qt.UserRole + 514
REMOTE_SITE_NAME_ROLE = QtCore.Qt.UserRole + 515
ERROR_ROLE = QtCore.Qt.UserRole + 516
TRIES_ROLE = QtCore.Qt.UserRole + 517

2
openpype/vendor/python/common/capture.py vendored
View file

@ -116,6 +116,8 @@ def capture(camera=None,
if not cmds.objExists(camera):
raise RuntimeError("Camera does not exist: {0}".format(camera))
if width and height :
maintain_aspect_ratio = False
width = width or cmds.getAttr("defaultResolution.width")
height = height or cmds.getAttr("defaultResolution.height")
if maintain_aspect_ratio:

2
openpype/version.py
View file

@ -1,3 +1,3 @@
# -*- coding: utf-8 -*-
"""Package declaring Pype version."""
__version__ = "3.6.0-nightly.4"
__version__ = "3.6.0-nightly.5"

2
pyproject.toml
View file

@ -1,6 +1,6 @@
[tool.poetry]
name = "OpenPype"
version = "3.6.0-nightly.4" # OpenPype
version = "3.6.0-nightly.5" # OpenPype
description = "Open VFX and Animation pipeline with support."
authors = ["OpenPype Team <info@openpype.io>"]
license = "MIT License"