mirror of
https://github.com/ynput/ayon-core.git
synced 2025-12-24 21:04:40 +01:00
* General: Connect to AYON server (base) (#3924) * implemented 'get_workfile_info' in entities * removed 'prepare_asset_update_data' which is not used * disable settings and project manager if in v4 mode * prepared conversion helper functions for v4 entities * prepared conversion functions for hero versions * fix hero versions * implemented get_archived_representations * fix get latest versions * return prepared changes * handle archived representation * raise exception on failed json conversion * map archived to active properly * make sure default fields are added * fix conversion of hero version entity * fix conversion of archived representations * fix some conversions of representations and versions * changed active behavior in queries * fixed hero versions * implemented basic thumbnail caching * added raw variants of crud methods * implemented methods to get and create thumbnail * fix from flat dict * implemented some basic folder conversion for updates * fix thumbnail updates for version * implemented v4 thumbnail integrator * simplified data mapping * 'get_thumbnail' function also expect entity type and entity id for which is the thumbnail received * implemented 'get_thumbnail' for server * fix how thumbnail id is received from entity * removed unnecessary method 'get_thumbnail_id_from_source' * implemented thumbnail resolver for v4 * removed unnecessary print * move create and delete project directly to server api * disable local settings action too on v4 * OP-3521 - added method to check and download updated addons from v4 server * OP-3521 - added more descriptive error message for missing source * OP-3521 - added default implementation of addon downloader to import * OP-3521 - added check for dependency package zips WIP - server doesn't contain required endpoint. Testing only with mockup data for now. * OP-3521 - fixed parsing of DependencyItem Added Server Url type and ServerAddonDownloader - v4 server doesn't know its own DNS for static files so it is sending unique name and url must be created during runtime. * OP-3521 - fixed creation of targed directories * change nev keys to look for and don't set them automatically * fix task type conversion * implemented base of loading v4 addons in v3 * Refactored argument name in Downloaders * Updated parsing to DependencyItem according to current schema * Implemented downloading of package from server * Updated resolving of failures Uses Enum items. * Introduced passing of authorization token Better to inject it than to have it from env var. * Remove weird parsing of server_url Not necessary, endpoints have same prefix. * Fix doubling asset version name in addons folder Zip file should already contain `addonName_addonVersion` as first subfolder * Fix doubling asset version name in addons folder Zip file should already contain `addonName_addonVersion` as first subfolder * Made server_endpoint optional Argument should be better for testing, but for calling from separate methods it would be better to encapsulate it. Removed unwanted temporary productionPackage value * Use existing method to pull addon info from Server to load v4 version of addon * Raise exception when server doesn't have any production dependency package * added ability to specify v3 alias of addon name * expect v3_alias as uppered constant * Re-implemented method to get addon info Previous implementation wouldn't work in Python2 hosts. Will be refactored in the future. * fix '__getattr__' * added ayon api to pyproject.toml and lock file * use ayon api in common connection * added mapping for label * use ayon_api in client codebase * separated clearing cache of url and username * bump ayon api version * rename env 'OP4_TEST' to 'USE_AYON_SERVER' * Move and renamend get_addons_info to get_addons_info_as_dict in addon_distribution Should be moved to ayon_api later * Replaced requests calls with ayon_api * Replaced OP4_TEST_ENABLED with AYON_SERVER_ENABLED fixed endpoints * Hound * Hound * OP-3521 - fix wrong key in get_representation_parents parents overloads parents * OP-3521 - changes for v4 of SiteSync addon * OP-3521 - fix names * OP-3521 - remove storing project_name It should be safer to go thorug self.dbcon apparently * OP-3521 - remove unwanted "context["folder"]" can be only in dummy test data * OP-3521 - move site sync loaders to addon * Use only project instead of self.project * OP-3521 - added missed get_progress_for_repre * base of settings conversion script * simplified ayon functions in start.py * added loading of settings from ayon server * added a note about colors * fix global and local settings functions * AvalonMongoDB is not using mongo connection on ayon server enabled * 'get_dynamic_modules_dirs' is not checking system settings for paths in setting * log viewer is disabled when ayon server is enabled * basic logic of enabling/disabled addons * don't use mongo logging if ayon server is enabled * update ayon api * bump ayon api again * use ayon_api to get addons info in modules/base * update ayon api * moved helper functions to get addons and dependencies dir to common functions * Initialization of AddonInfo is not crashing on unkonwn sources * renamed 'DependencyDownloader' to 'AyonServerDownloader' * renamed function 'default_addon_downloader' to 'get_default_addon_downloader' * Added ability to convert 'WebAddonSource' to 'ServerResourceSorce' * missing dependency package on server won't cause crash * data sent to downloaders don't contain ayon specific headers * modified addon distribution to not duplicate 'ayon_api' functionality * fix doubled function defintioin * unzip client file to addon destination * formatting - unify quotes * disable usage of mongo connection if in ayon mode * renamed window.py to login_window.py * added webpublisher settings conversion * added maya conversion function * reuse variable * reuse variable (similar to previous commit) * fix ayon addons loading * fix typo 'AyonSettingsCahe' -> 'AyonSettingsCache' * fix enabled state changes * fix rr_path in royal render conversion * avoid mongo calls in AYON state * implemented custom AYON start script * fix formatting (after black) * ayon_start cleanup * 'get_addons_dir' and 'get_dependencies_dir' store value to environment variable * add docstrings to local dir functions * addon info has full name * fix modules enabled states * removed unused 'run_disk_mapping_commands' * removed ayon logic from 'start.py' * fix warning message * renamed 'openpype_common' to 'ayon_common' * removed unused import * don't import igniter * removed startup validations of third parties * change what's shown in version info * fix which keys are applied from ayon values * fix method name * get applications from attribs * Implemented UI basics to be able change user or logout * merged server.py and credentials.py * add more metadata to urls * implemented change token * implemented change user ui functionality * implemented change user ui * modify window to handle username and token value * pass username to add server * fix show UI cases * added loggin action to tray * update ayon api * added missing dependency * convert applications to config in a right way * initial implementation of 'nuke' settings conversion * removed few nuke comments * implemented hiero conversion * added imageio conversion * added run ayon tray script * fix few settings conversions * Renamed class of source classes as they are not just for addons * implemented objec to track source transfer progress * Implemented distribution item with multiple sources * Implemented ayon distribution wrapper to care about multiple things during distribution * added 'cleanup' method for downlaoders * download gets tranfer progress object * Change UploadState enum * added missing imports * use AyonDistribution in ayon_start.py * removed unused functions * removed implemented TODOs * fix import * fix key used for Web source * removed temp development fix * formatting fix * keep information if source require distribution * handle 'require_distribution' attribute in distribution process * added path attribute to server source * added option to pass addons infor to ayon distribution * fix tests * fix formatting * Fix typo * Fix typo * remove '_try_convert_to_server_source' * renamed attributes and methods to match their content * it is possible to pass dependency package info to AyonDistribution * fix called methods in tests * added public properties for error message and error detail * Added filename to WebSourceInfo Useful for GDrive sharable links where target file name is unknown/unparsable, it should be provided explicitly. * unify source conversion by adding 'convert_source' function * Fix error message Co-authored-by: Roy Nieterau <roy_nieterau@hotmail.com> * added docstring for 'transfer_progress' * don't create metadata file on read * added few docstrings * add default folder fields to folder/task queries * fix generators * add dependencies when runnign from code * add sys paths from distribution to pythonpath env * fix missing applications * added missing conversions for maya renderers * fix formatting * update ayon api * fix hashes in lock file * Use better exception Co-authored-by: Ondřej Samohel <33513211+antirotor@users.noreply.github.com> * Use Python 3 syntax Co-authored-by: Ondřej Samohel <33513211+antirotor@users.noreply.github.com> * apply some of sugested changes in ayon_start * added some docstrings and suggested modifications * copy create env from develop * fix rendersettings conversion * change code by suggestions * added missing args to docstring * added missing docstrings * separated downloader and download factory * fix ayon settings * added some basic file docstring to ayon_settings * join else conditions * fix project settings conversion * fix created at conversion * fix workfile info query * fix publisher UI * added utils function 'get_ayon_appdirs' * fix 'get_all_current_info' * fix server url assignment when url is set * updated ayon api * added utils functions to create local site id for ayon * added helper functions to create global connection * create global connection in ayon start to start use site id * use ayon site id in ayon mode * formatting cleanup * added header docstring * fixes after ayon_api update * load addons from ynput appdirs * fix function call * added docstring * update ayon pyton api * fix settings access * use ayon_api to get root overrides in Anatomy * bumbayon version to 0.1.13 * nuke: fixing settings keys from settings * fix burnins definitions * change v4 to AYON in thumbnail integrate * fix one more v4 information * Fixes after rebase * fix extract burnin conversion * additional fix of extract burnin * SiteSync:added missed loaders or v3 compatibility (#4587) * Added site sync loaders for v3 compatibility * Fix get_progress_for_repre * use 'files.name' instead of 'files.baseName' * update ayon api to 0.1.14 * add common to include files * change arguments for hero version creation * skip shotgrid settings conversion if different ayon addon is used * added ayon icons * fix labels of application variants * added option to show login window always on top * login window on invalid credentials is always on top * update ayon api * update ayon api * add entityType to project and folders * AYON: Editorial hierarchy creation (#4699) * disable extract hierarchy avalon when ayon mode is enabled * implemented extract hierarchy to AYON --------- Co-authored-by: Petr Kalis <petr.kalis@gmail.com> Co-authored-by: Roy Nieterau <roy_nieterau@hotmail.com> Co-authored-by: Ondřej Samohel <33513211+antirotor@users.noreply.github.com> Co-authored-by: Jakub Jezek <jakubjezek001@gmail.com> * replace 'legacy_io' with context functions in load plugins * added 'get_global_context' to pipeline init * use context getters instead of legacy_io in publish plugins * use data on context instead of 'legacy_io' in submit publish job * skip query of asset docs in collect nuke reads * use context functions on other places * 'list_looks' expects project name * remove 'get_context_title' * don't pass AvalonMongoDB to prelaunch hooks * change how context is calculated in hiero * implemented function 'get_fps_for_current_context' for maya * initialize '_image_dir' and '_image_prefixes' in init * legacy creator is using 'get_current_project_name' * fill docstrings * use context functions in workfile builders * hound fixes * 'create_workspace_mel' can expect project settings * swapped order of arguments * use information from instance/context data * Use self.project_name in workfiles tool Co-authored-by: Roy Nieterau <roy_nieterau@hotmail.com> * Remove outdated todo Co-authored-by: Roy Nieterau <roy_nieterau@hotmail.com> * don't query project document in nuke lib * Fix access to context data * Use right function to get project name Co-authored-by: Roy Nieterau <roy_nieterau@hotmail.com> * fix submit max deadline and swap order of arguments * added 'get_context_label' to nuke * fix import * fix typo 'curent_context' -> 'current_context' * fix project_setting variable * fix submit publish job environments * use task from context * Removed unused import --------- Co-authored-by: Petr Kalis <petr.kalis@gmail.com> Co-authored-by: Roy Nieterau <roy_nieterau@hotmail.com> Co-authored-by: Ondřej Samohel <33513211+antirotor@users.noreply.github.com> Co-authored-by: Jakub Jezek <jakubjezek001@gmail.com> |
||
|---|---|---|
| .. | ||
| avalon_apps | ||
| clockify | ||
| deadline | ||
| example_addons | ||
| ftrack | ||
| job_queue | ||
| kitsu | ||
| log_viewer | ||
| muster | ||
| python_console_interpreter | ||
| royalrender | ||
| shotgrid | ||
| slack | ||
| sync_server | ||
| timers_manager | ||
| webserver | ||
| __init__.py | ||
| base.py | ||
| interfaces.py | ||
| launcher_action.py | ||
| project_manager_action.py | ||
| README.md | ||
| settings_action.py | ||
OpenPype modules/addons
OpenPype modules should contain separated logic of specific kind of implementation, such as Ftrack connection and its usage code, Deadline farm rendering or may contain only special plugins. Addons work the same way currently, there is no difference between module and addon functionality.
Modules concept
- modules and addons are dynamically imported to virtual python module
openpype_modulesfrom which it is possible to import them no matter where is the module located - modules or addons should never be imported directly, even if you know possible full import path
- it is because all of their content must be imported in specific order and should not be imported without defined functions as it may also break few implementation parts
TODOs
- add module/addon manifest
- definition of module (not 100% defined content e.g. minimum required OpenPype version etc.)
- defining a folder as a content of a module or an addon
Base class OpenPypeModule
- abstract class as base for each module
- implementation should contain module's api without GUI parts
- may implement
get_global_environmentsmethod which should return dictionary of environments that are globally applicable and value is the same for whole studio if launched at any workstation (except os specific paths) - abstract parts:
nameattribute - name of a moduleinitializemethod - method for own initialization of a module (should not override__init__)connect_with_modulesmethod - where module may look for it's interfaces implementations or check for other modules__init__should not be overridden andinitializeshould not do time consuming part but only prepare base data about module- also keep in mind that they may be initialized in headless mode
- connection with other modules is made with help of interfaces
climethod - add cli commands specific for the module- command line arguments are handled using
clickpython module climethod should expect single argument which is click group on which can be called any group specific methods (e.g.add_commandto add another click group as children seeExampleAddon)- it is possible to add trigger cli commands using
./openpype_console module <module_name> <command> *args
- command line arguments are handled using
Addon class OpenPypeAddOn
- inherits from
OpenPypeModulebut is enabled by default and doesn't have to implementinitializeandconnect_with_modulesmethods - that is because it is expected that addons don't need to have system settings and
enabledvalue on it (but it is possible...)
How to add addons/modules
- in System settings go to
modules/addon_paths(Modules/OpenPype AddOn Paths) where you have to add path to addon root folder - for openpype example addons use
{OPENPYPE_REPOS_ROOT}/openpype/modules/example_addons
Addon/module settings
- addons/modules may have defined custom settings definitions with default values
- it is based on settings type
dynamic_schemawhich hasname - that item defines that it can be replaced dynamically with any schemas from module or module which won't be saved to openpype core defaults
- they can't be added to any schema hierarchy
- item must not be in settings group (under overrides) or in dynamic item (e.g.
listofdict-modifiable) - addons may define it's dynamic schema items
- they can be defined with class which inherits from
BaseModuleSettingsDef - it is recommended to use pre implemented
JsonFilesSettingsDefwhich defined structure and use json files to define dynamic schemas, schemas and default values - check it's docstring and check for
example_addonin example addons - settings definition returns schemas by dynamic schemas names
Interfaces
- interface is class that has defined abstract methods to implement and may contain pre implemented helper methods
- module that inherit from an interface must implement those abstract methods otherwise won't be initialized
- it is easy to find which module object inherited from which interfaces with 100% chance they have implemented required methods
- interfaces can be defined in
interfaces.pyinside module directory - the file can't use relative imports or import anything from other parts of module itself at the header of file
- this is one of reasons why modules/addons can't be imported directly without using defined functions in OpenPype modules implementation
Base class OpenPypeInterface
- has nothing implemented
- has ABCMeta as metaclass
- is defined to be able find out classes which inherit from this base to be able tell this is an Interface
Global interfaces
- few interfaces are implemented for global usage
IPluginPaths
- module wants to add directory path/s to avalon or publish plugins
- module must implement
get_plugin_pathswhich must return dictionary with possible keys"publish","load","create"or"actions" - each key may contain list or string with a path to directory with plugins
ITrayModule
- module has more logic when used in a tray
- it is possible that module can be used only in the tray
- abstract methods
tray_init- initialization triggered afterinitializewhen used inTrayModulesManagerand beforeconnect_with_modulestray_menu- add actions to tray widget's menu that represent the moduletray_start- start of module's login in tray- module is initialized and connected with other modules
tray_exit- module's cleanup like stop and join threads etc.- order of calling is based on implementation this order is how it works with
TrayModulesManager - it is recommended to import and use GUI implementation only in these methods
- has attribute
tray_initialized(bool) which is set to False by default and is set byTrayModulesManagerto True aftertray_init - if module has logic only in tray or for both then should be checking for
tray_initializedattribute to decide how should handle situations
ITrayService
- inherits from
ITrayModuleand implementstray_menumethod for you - adds action to submenu "Services" in tray widget menu with icon and label
- abstract attribute
label - label shown in menu
- interface has pre implemented methods to change icon color
set_service_running- green iconset_service_failed- red iconset_service_idle- orange icon- these states must be set by module itself
set_service_runningis default state on initialization
ITrayAction
- inherits from
ITrayModuleand implementstray_menumethod for you - adds action to tray widget menu with label
- abstract attribute
label - label shown in menu
- abstract method
on_action_trigger - what should happen when an action is triggered
- NOTE: It is a good idea to implement logic in
on_action_triggerto the api method and trigger that method on callbacks. This gives ability to trigger that method outside tray
Modules interfaces
- modules may have defined their own interfaces to be able to recognize other modules that would want to use their features
Example:
-
Ftrack module has
IFtrackEventHandlerPathswhich helps to tell Ftrack module which other modules want to add paths to server/user event handlers -
Clockify module use
IFtrackEventHandlerPathsand returns paths to clockify ftrack synchronizers -
Clockify inherits from more interfaces. It's class definition looks like:
class ClockifyModule(
OpenPypeModule, # Says it's Pype module so ModulesManager will try to initialize.
ITrayModule, # Says has special implementation when used in tray.
IPluginPaths, # Says has plugin paths that want to register (paths to clockify actions for launcher).
IFtrackEventHandlerPaths, # Says has Ftrack actions/events for user/server.
ITimersManager # Listen to other modules with timer and can trigger changes in other module timers through `TimerManager` module.
):
ModulesManager
- collects module classes and tries to initialize them
- important attributes
modules- list of available attributesmodules_by_id- dictionary of modules mapped by their idsmodules_by_name- dictionary of modules mapped by their names- all these attributes contain all found modules even if are not enabled
- helper methods
collect_global_environmentsto collect all global environments from enabled modules with callingget_global_environmentson each of themcollect_plugin_pathscollects plugin paths from all enabled modules- output is always dictionary with all keys and values as an list
{
"publish": [],
"create": [],
"load": [],
"actions": [],
"inventory": []
}
TrayModulesManager
- inherits from
ModulesManager - has specific implementation for Pype Tray tool and handle
ITrayModulemethods