Import and deposit

NOTE

This documentation is a work in progress and is incomplete.

Please contact developers for more details.

• Import
  • Analysis watcher for automated import
  • Connection with ELLA-anno
• Deposit
• Default import gene panel

Options for import or deposit of variants and/or samples.

Import

[TODO]

• File: ella_config.yml (set by ELLA_CONFIG env variable)
• Key: import

Subkey	Explanation	Values
automatic_deposit_with_sample_id	[TODO]	True / False
preimport_script	[TODO]	[path]

Analysis watcher for automated import

ELLA can be configured with an analysis watcher for automated import of analyses by running a service of the backend image with entrypoint python /ella/src/vardb/watcher/analysis_watcher.py. This triggers a script which periodically checks for new analyses in ANALYSES_INCOMING and imports any new analyses to ANALYSES_PATH.

This script can optionally be provided with whitelist and/or blacklist arguments. These can be provided as command line arguments (--whitelist/--blacklist) or files (--whitelistfile/--blacklistfile), where command line arguments take precedence. File paths default to the environment variables ELLA_WATCHER_WHITELISTFILE/ELLA_WATCHER_BLACKLISTFILE.

If arguments are passed in --whitelist/--blacklist, they should be provided as space-separated Python regex (opens new window) patterns. If files are passed, patterns should be line-separated.

If a whitelist is provided (as either file or command line argument), any folder name in ANALYSES_INCOMING (see Setup environment) must match at least one of the whitelist patterns. If a blacklist is provided, and the folder name does not match any of the patterns in the blacklist, the analysis watcher will attempt to import it.

TIP

Changes in command line arguments require a restart of the analysis watcher to take effect, whereas changes in whitelist/blacklist files will be re-read before every check of ANALYSES_INCOMING, and therefore do not require a restart.

Connection with ELLA-anno

By running a service of the backend image with entrypoint python /ella/src/polling/polling.py, any jobs imported through the import UI can be processed through to ELLA-anno (opens new window).

Without this service running, no jobs will be processed from the import page.

Deposit

Processes that should be run when new analyses are deposited into ELLA, configured per user group. See usergroups.json (opens new window) for examples.

• File: usergroups.json (see user groups)
• Key: config.deposit.analysis

Subkey	Explanation	Values
pattern	What analysis name should be matched for this configuration block (e.g. search for user group specific parts in analysis names).	[regex]
postprocess	Processes that should be applied after analyses have been loaded into ELLA.	analysis_not_ready_warnings: Places analyses that includes variants that need verification or have warnings generated by the pipeline in the "Not ready" section on the OVERVIEW page. analysis_finalize_without_findings: Analyses that have no variants that need further work (only benign, no technical issues) are automatically finalized, without any user interaction. analysis_tag_all_classified: Adds an OVERVIEW comment automatically for the following conditions: ALL CLASSIFIED when all variants are already classified and still valid; NO VARIANTS when there are no non-filtered variants.
prefilter	Whether to prefilter this analysis (e.g. high-frequent variants), useful for limiting resource use for large gene panels. Note that these will not be available in the "FILTERED" variants list.	See Pre-filter.

NOTE

Only one user group and configuration can match a particular combination of analysis name and gene panel.

See also Annotation configuration.

Default import gene panel

Gene panel that should be pre-selected when importing data through the IMPORT function, configured per user group. See usergroups.json (opens new window) for examples.

• File: usergroups.json (see user groups)
• Key: default_import_genepanel
• Value: [gene panel name], [version] (latter is optional)