Frame13/ayon-core
1
0
Fork 1
mirror of https://github.com/ynput/ayon-core.git synced 2025-12-24 21:04:40 +01:00
Code Issues Projects Releases Packages Wiki Activity Actions

Merge branch 'develop' of github.com:pypeclub/OpenPype into bugfix/OP-3684_Refactored-integrate-doesnt-work-for-AE-multi-frame-publishes

Browse source
This commit is contained in:
Petr Kalis 2022-08-03 17:43:55 +02:00
commit 53cf211fcb
9 changed files with 252 additions and 100 deletions
Download patch file Download diff file Expand all files Collapse all files

24
CHANGELOG.md
View file

@ -1,6 +1,6 @@
# Changelog
## [3.12.3-nightly.1](https://github.com/pypeclub/OpenPype/tree/HEAD)
## [3.12.3-nightly.2](https://github.com/pypeclub/OpenPype/tree/HEAD)
[Full Changelog](https://github.com/pypeclub/OpenPype/compare/3.12.2...HEAD)
@ -11,21 +11,27 @@
**🚀 Enhancements**
- Kitsu: Shot&Sequence name with prefix over appends [\#3593](https://github.com/pypeclub/OpenPype/pull/3593)
- Ftrack: Update ftrack api to 2.3.3 [\#3588](https://github.com/pypeclub/OpenPype/pull/3588)
- Photoshop: implemented {layer} placeholder in subset template [\#3591](https://github.com/pypeclub/OpenPype/pull/3591)
- General: New Integrator small fixes [\#3583](https://github.com/pypeclub/OpenPype/pull/3583)
**🐛 Bug fixes**
- TrayPublisher: Fix wrong conflict merge [\#3600](https://github.com/pypeclub/OpenPype/pull/3600)
- Bugfix: Add OCIO as submodule to prepare for handling `maketx` color space conversion. [\#3590](https://github.com/pypeclub/OpenPype/pull/3590)
- Editorial publishing workflow improvements [\#3580](https://github.com/pypeclub/OpenPype/pull/3580)
- Nuke: render family integration consistency [\#3576](https://github.com/pypeclub/OpenPype/pull/3576)
- Ftrack: Handle missing published path in integrator [\#3570](https://github.com/pypeclub/OpenPype/pull/3570)
- Nuke: publish existing frames with slate with correct range [\#3555](https://github.com/pypeclub/OpenPype/pull/3555)
**🔀 Refactored code**
- General: Use query functions in general code [\#3596](https://github.com/pypeclub/OpenPype/pull/3596)
- General: Separate extraction of template data into more functions [\#3574](https://github.com/pypeclub/OpenPype/pull/3574)
- General: Lib cleanup [\#3571](https://github.com/pypeclub/OpenPype/pull/3571)
**Merged pull requests:**
- Enable write color sets on animation publish automatically [\#3582](https://github.com/pypeclub/OpenPype/pull/3582)
## [3.12.2](https://github.com/pypeclub/OpenPype/tree/3.12.2) (2022-07-27)
[Full Changelog](https://github.com/pypeclub/OpenPype/compare/CI/3.12.2-nightly.4...3.12.2)
@ -54,6 +60,7 @@
- NewPublisher: Python 2 compatible html escape [\#3559](https://github.com/pypeclub/OpenPype/pull/3559)
- Remove invalid submodules from `/vendor` [\#3557](https://github.com/pypeclub/OpenPype/pull/3557)
- General: Remove hosts filter on integrator plugins [\#3556](https://github.com/pypeclub/OpenPype/pull/3556)
- Nuke: publish existing frames with slate with correct range [\#3555](https://github.com/pypeclub/OpenPype/pull/3555)
- Settings: Clean default values of environments [\#3550](https://github.com/pypeclub/OpenPype/pull/3550)
- Module interfaces: Fix import error [\#3547](https://github.com/pypeclub/OpenPype/pull/3547)
- Workfiles tool: Show of tool and it's flags [\#3539](https://github.com/pypeclub/OpenPype/pull/3539)
@ -66,7 +73,6 @@
- TrayPublisher: Simple creation enhancements and fixes [\#3513](https://github.com/pypeclub/OpenPype/pull/3513)
- TrayPublisher: Make sure host name is filled [\#3504](https://github.com/pypeclub/OpenPype/pull/3504)
- NewPublisher: Groups work and enum multivalue [\#3501](https://github.com/pypeclub/OpenPype/pull/3501)
- Resolve: removed few bugs [\#3464](https://github.com/pypeclub/OpenPype/pull/3464)
**🔀 Refactored code**
@ -77,7 +83,6 @@
- General: Move load related functions into pipeline [\#3527](https://github.com/pypeclub/OpenPype/pull/3527)
- General: Get current context document functions [\#3522](https://github.com/pypeclub/OpenPype/pull/3522)
- Kitsu: Use query function from client [\#3496](https://github.com/pypeclub/OpenPype/pull/3496)
- Deadline: Use query functions [\#3466](https://github.com/pypeclub/OpenPype/pull/3466)
**Merged pull requests:**
@ -94,7 +99,6 @@
- NewPublisher: Added ability to use label of instance [\#3484](https://github.com/pypeclub/OpenPype/pull/3484)
- General: Creator Plugins have access to project [\#3476](https://github.com/pypeclub/OpenPype/pull/3476)
- General: Better arguments order in creator init [\#3475](https://github.com/pypeclub/OpenPype/pull/3475)
- Ftrack: Trigger custom ftrack events on project creation and preparation [\#3465](https://github.com/pypeclub/OpenPype/pull/3465)
**🐛 Bug fixes**
@ -105,14 +109,6 @@
- New Publisher: Disabled context change allows creation [\#3478](https://github.com/pypeclub/OpenPype/pull/3478)
- General: thumbnail extractor fix [\#3474](https://github.com/pypeclub/OpenPype/pull/3474)
- Kitsu: bugfix with sync-service ans publish plugins [\#3473](https://github.com/pypeclub/OpenPype/pull/3473)
- Flame: solved problem with multi-selected loading [\#3470](https://github.com/pypeclub/OpenPype/pull/3470)
- General: Fix query function in update logic [\#3468](https://github.com/pypeclub/OpenPype/pull/3468)
- General: Delete old versions is safer when ftrack is disabled [\#3462](https://github.com/pypeclub/OpenPype/pull/3462)
**🔀 Refactored code**
- Maya: Merge animation + pointcache extractor logic [\#3461](https://github.com/pypeclub/OpenPype/pull/3461)
- Maya: Re-use `maintained\_time` from lib [\#3460](https://github.com/pypeclub/OpenPype/pull/3460)
## [3.12.0](https://github.com/pypeclub/OpenPype/tree/3.12.0) (2022-06-28)

2
openpype/lib/__init__.py
View file

@ -115,6 +115,7 @@ from .transcoding import (
get_ffmpeg_codec_args,
get_ffmpeg_format_args,
convert_ffprobe_fps_value,
convert_ffprobe_fps_to_float,
)
from .avalon_context import (
CURRENT_DOC_SCHEMAS,
@ -287,6 +288,7 @@ __all__ = [
"get_ffmpeg_codec_args",
"get_ffmpeg_format_args",
"convert_ffprobe_fps_value",
"convert_ffprobe_fps_to_float",
"CURRENT_DOC_SCHEMAS",
"PROJECT_NAME_ALLOWED_SYMBOLS",

37
openpype/lib/transcoding.py
View file

@ -938,3 +938,40 @@ def convert_ffprobe_fps_value(str_value):
fps = int(fps)
return str(fps)
def convert_ffprobe_fps_to_float(value):
"""Convert string value of frame rate to float.
Copy of 'convert_ffprobe_fps_value' which raises exceptions on invalid
value, does not convert value to string and does not return "Unknown"
string.
Args:
value (str): Value to be converted.
Returns:
Float: Converted frame rate in float. If divisor in value is '0' then
'0.0' is returned.
Raises:
ValueError: Passed value is invalid for conversion.
"""
if not value:
raise ValueError("Got empty value.")
items = value.split("/")
if len(items) == 1:
return float(items[0])
if len(items) > 2:
raise ValueError((
"FPS expression contains multiple dividers \"{}\"."
).format(value))
dividend = float(items.pop(0))
divisor = float(items.pop(0))
if divisor == 0.0:
return 0.0
return dividend / divisor

43
openpype/modules/ftrack/plugins/publish/integrate_ftrack_api.py
View file

@ -87,6 +87,7 @@ class IntegrateFtrackApi(pyblish.api.InstancePlugin):
asset_versions_data_by_id = {}
used_asset_versions = []
# Iterate over components and publish
for data in component_list:
self.log.debug("data: {}".format(data))
@ -116,9 +117,6 @@ class IntegrateFtrackApi(pyblish.api.InstancePlugin):
asset_version_status_ids_by_name
)
# Component
self.create_component(session, asset_version_entity, data)
# Store asset version and components items that were
version_id = asset_version_entity["id"]
if version_id not in asset_versions_data_by_id:
@ -135,6 +133,8 @@ class IntegrateFtrackApi(pyblish.api.InstancePlugin):
if asset_version_entity not in used_asset_versions:
used_asset_versions.append(asset_version_entity)
self._create_components(session, asset_versions_data_by_id)
instance.data["ftrackIntegratedAssetVersionsData"] = (
asset_versions_data_by_id
)
@ -623,3 +623,40 @@ class IntegrateFtrackApi(pyblish.api.InstancePlugin):
session.rollback()
session._configure_locations()
six.reraise(tp, value, tb)
def _create_components(self, session, asset_versions_data_by_id):
for item in asset_versions_data_by_id.values():
asset_version_entity = item["asset_version"]
component_items = item["component_items"]
component_entities = session.query(
(
"select id, name from Component where version_id is \"{}\""
).format(asset_version_entity["id"])
).all()
existing_component_names = {
component["name"]
for component in component_entities
}
contain_review = "ftrackreview-mp4" in existing_component_names
thumbnail_component_item = None
for component_item in component_items:
component_data = component_item.get("component_data") or {}
component_name = component_data.get("name")
if component_name == "ftrackreview-mp4":
contain_review = True
elif component_name == "ftrackreview-image":
thumbnail_component_item = component_item
if contain_review and thumbnail_component_item:
thumbnail_component_item["component_data"]["name"] = (
"thumbnail"
)
# Component
for component_item in component_items:
self.create_component(
session, asset_version_entity, component_item
)

159
openpype/modules/ftrack/plugins/publish/integrate_ftrack_instances.py
View file

@ -3,7 +3,10 @@ import json
import copy
import pyblish.api
from openpype.lib import get_ffprobe_streams
from openpype.lib.transcoding import (
get_ffprobe_streams,
convert_ffprobe_fps_to_float,
)
from openpype.lib.profiles_filtering import filter_profiles
@ -79,11 +82,6 @@ class IntegrateFtrackInstance(pyblish.api.InstancePlugin):
).format(family))
return
# Prepare FPS
instance_fps = instance.data.get("fps")
if instance_fps is None:
instance_fps = instance.context.data["fps"]
status_name = self._get_asset_version_status_name(instance)
# Base of component item data
@ -168,10 +166,7 @@ class IntegrateFtrackInstance(pyblish.api.InstancePlugin):
# Add item to component list
component_list.append(thumbnail_item)
if (
not review_representations
and first_thumbnail_component is not None
):
if first_thumbnail_component is not None:
width = first_thumbnail_component_repre.get("width")
height = first_thumbnail_component_repre.get("height")
if not width or not height:
@ -253,20 +248,9 @@ class IntegrateFtrackInstance(pyblish.api.InstancePlugin):
first_thumbnail_component[
"asset_data"]["name"] = extended_asset_name
frame_start = repre.get("frameStartFtrack")
frame_end = repre.get("frameEndFtrack")
if frame_start is None or frame_end is None:
frame_start = instance.data["frameStart"]
frame_end = instance.data["frameEnd"]
# Frame end of uploaded video file should be duration in frames
# - frame start is always 0
# - frame end is duration in frames
duration = frame_end - frame_start + 1
fps = repre.get("fps")
if fps is None:
fps = instance_fps
component_meta = self._prepare_component_metadata(
instance, repre, repre_path, True
)
# Change location
review_item["component_path"] = repre_path
@ -275,11 +259,7 @@ class IntegrateFtrackInstance(pyblish.api.InstancePlugin):
# Default component name is "main".
"name": "ftrackreview-mp4",
"metadata": {
"ftr_meta": json.dumps({
"frameIn": 0,
"frameOut": int(duration),
"frameRate": float(fps)
})
"ftr_meta": json.dumps(component_meta)
}
}
@ -322,6 +302,13 @@ class IntegrateFtrackInstance(pyblish.api.InstancePlugin):
component_data = copy_src_item["component_data"]
component_name = component_data["name"]
component_data["name"] = component_name + "_src"
component_meta = self._prepare_component_metadata(
instance, repre, copy_src_item["component_path"], False
)
if component_meta:
component_data["metadata"] = {
"ftr_meta": json.dumps(component_meta)
}
component_list.append(copy_src_item)
# Add others representations as component
@ -339,9 +326,17 @@ class IntegrateFtrackInstance(pyblish.api.InstancePlugin):
):
other_item["asset_data"]["name"] = extended_asset_name
other_item["component_data"] = {
component_meta = self._prepare_component_metadata(
instance, repre, published_path, False
)
component_data = {
"name": repre["name"]
}
if component_meta:
component_data["metadata"] = {
"ftr_meta": json.dumps(component_meta)
}
other_item["component_data"] = component_data
other_item["component_location_name"] = unmanaged_location_name
other_item["component_path"] = published_path
component_list.append(other_item)
@ -424,3 +419,107 @@ class IntegrateFtrackInstance(pyblish.api.InstancePlugin):
return None
return matching_profile["status"] or None
def _prepare_component_metadata(
self, instance, repre, component_path, is_review
):
extension = os.path.splitext(component_path)[-1]
streams = []
try:
streams = get_ffprobe_streams(component_path)
except Exception:
self.log.debug((
"Failed to retrieve information about intput {}"
).format(component_path))
# Find video streams
video_streams = [
stream
for stream in streams
if stream["codec_type"] == "video"
]
# Skip if there are not video streams
# - exr is special case which can have issues with reading through
# ffmpegh but we want to set fps for it
if not video_streams and extension not in [".exr"]:
return {}
stream_width = None
stream_height = None
stream_fps = None
frame_out = None
for video_stream in video_streams:
tmp_width = video_stream.get("width")
tmp_height = video_stream.get("height")
if tmp_width and tmp_height:
stream_width = tmp_width
stream_height = tmp_height
input_framerate = video_stream.get("r_frame_rate")
duration = video_stream.get("duration")
if input_framerate is None or duration is None:
continue
try:
stream_fps = convert_ffprobe_fps_to_float(
input_framerate
)
except ValueError:
self.log.warning((
"Could not convert ffprobe fps to float \"{}\""
).format(input_framerate))
continue
stream_width = tmp_width
stream_height = tmp_height
self.log.debug("FPS from stream is {} and duration is {}".format(
input_framerate, duration
))
frame_out = float(duration) * stream_fps
break
# Prepare FPS
instance_fps = instance.data.get("fps")
if instance_fps is None:
instance_fps = instance.context.data["fps"]
if not is_review:
output = {}
fps = stream_fps or instance_fps
if fps:
output["frameRate"] = fps
if stream_width and stream_height:
output["width"] = int(stream_width)
output["height"] = int(stream_height)
return output
frame_start = repre.get("frameStartFtrack")
frame_end = repre.get("frameEndFtrack")
if frame_start is None or frame_end is None:
frame_start = instance.data["frameStart"]
frame_end = instance.data["frameEnd"]
fps = None
repre_fps = repre.get("fps")
if repre_fps is not None:
repre_fps = float(repre_fps)
fps = stream_fps or repre_fps or instance_fps
# Frame end of uploaded video file should be duration in frames
# - frame start is always 0
# - frame end is duration in frames
if not frame_out:
frame_out = frame_end - frame_start + 1
# Ftrack documentation says that it is required to have
# 'width' and 'height' in review component. But with those values
# review video does not play.
component_meta = {
"frameIn": 0,
"frameOut": frame_out,
"frameRate": float(fps)
}
return component_meta

1
openpype/plugins/publish/extract_review.py
View file

@ -360,6 +360,7 @@ class ExtractReview(pyblish.api.InstancePlugin):
os.unlink(f)
new_repre.update({
"fps": temp_data["fps"],
"name": "{}_{}".format(output_name, output_ext),
"outputName": output_name,
"outputDef": output_def,

82
openpype/plugins/publish/integrate.py
View file

@ -23,41 +23,6 @@ from openpype.pipeline.publish import KnownPublishError
log = logging.getLogger(__name__)
def assemble(files):
"""Convenience `clique.assemble` wrapper for files of a single collection.
Unlike `clique.assemble` this wrapper does not allow more than a single
Collection nor any remainder files. Errors will be raised when not only
a single collection is assembled.
Returns:
clique.Collection: A single sequence Collection
Raises:
ValueError: Error is raised when files do not result in a single
collected Collection.
"""
# todo: move this to lib?
# Get the sequence as a collection. The files must be of a single
# sequence and have no remainder outside of the collections.
patterns = [clique.PATTERNS["frames"]]
collections, remainder = clique.assemble(files,
minimum_items=1,
patterns=patterns)
if not collections:
raise ValueError("No collections found in files: "
"{}".format(files))
if remainder:
raise ValueError("Files found not detected as part"
" of a sequence: {}".format(remainder))
if len(collections) > 1:
raise ValueError("Files in sequence are not part of a"
" single sequence collection: "
"{}".format(collections))
return collections[0]
def get_instance_families(instance):
"""Get all families of the instance"""
# todo: move this to lib?
@ -576,8 +541,18 @@ class IntegrateAsset(pyblish.api.InstancePlugin):
if any(os.path.isabs(fname) for fname in files):
raise KnownPublishError("Given file names contain full paths")
src_collection = assemble(files)
src_collections, remainders = clique.assemble(files)
if len(files) < 2 or len(src_collections) != 1 or remainders:
raise KnownPublishError((
"Files of representation does not contain proper"
" sequence files.\nCollected collections: {}"
"\nCollected remainders: {}"
).format(
", ".join([str(col) for col in src_collections]),
", ".join([str(rem) for rem in remainders])
))
src_collection = src_collections[0]
destination_indexes = list(src_collection.indexes)
# Use last frame for minimum padding
# - that should cover both 'udim' and 'frame' minimum padding
@ -609,18 +584,27 @@ class IntegrateAsset(pyblish.api.InstancePlugin):
# a Frame or UDIM tile set for the template data. We use the first
# index of the destination for that because that could've shifted
# from the source indexes, etc.
first_index_padded = get_frame_padded(frame=destination_indexes[0],
padding=destination_padding)
if is_udim:
# UDIM representations handle ranges in a different manner
template_data["udim"] = first_index_padded
else:
template_data["frame"] = first_index_padded
first_index_padded = get_frame_padded(
frame=destination_indexes[0],
padding=destination_padding
)
# Construct destination collection from template
anatomy_filled = anatomy.format(template_data)
template_filled = anatomy_filled[template_name]["path"]
repre_context = template_filled.used_values
repre_context = None
dst_filepaths = []
for index in destination_indexes:
if is_udim:
template_data["udim"] = index
else:
template_data["frame"] = index
anatomy_filled = anatomy.format(template_data)
template_filled = anatomy_filled[template_name]["path"]
dst_filepaths.append(template_filled)
if repre_context is None:
self.log.debug(
"Template filled: {}".format(str(template_filled))
)
repre_context = template_filled.used_values
# Make sure context contains frame
# NOTE: Frame would not be available only if template does not
@ -628,12 +612,8 @@ class IntegrateAsset(pyblish.api.InstancePlugin):
if not is_udim:
repre_context["frame"] = first_index_padded
self.log.debug("Template filled: {}".format(str(template_filled)))
dst_collection = assemble([os.path.normpath(template_filled)])
# Update the destination indexes and padding
dst_collection.indexes.clear()
dst_collection.indexes.update(set(destination_indexes))
dst_collection = clique.assemble(dst_filepaths)[0][0]
dst_collection.padding = destination_padding
if len(src_collection.indexes) != len(dst_collection.indexes):
raise KnownPublishError((

2
openpype/version.py
View file

@ -1,3 +1,3 @@
# -*- coding: utf-8 -*-
"""Package declaring Pype version."""
__version__ = "3.12.3-nightly.1"
__version__ = "3.12.3-nightly.2"

2
pyproject.toml
View file

@ -1,6 +1,6 @@
[tool.poetry]
name = "OpenPype"
version = "3.12.3-nightly.1" # OpenPype
version = "3.12.3-nightly.2" # OpenPype
description = "Open VFX and Animation pipeline with support."
authors = ["OpenPype Team <info@openpype.io>"]
license = "MIT License"