single_cell/parallel_integration

Description

Run multiple integration methods from OpenPipeline in parallel on the same
preprocessed h5mu input, then merge each method's annotations (leiden cluster
labels in .obs, integrated embeddings and UMAPs in .obsm, neighbor graphs in
.obsp and .uns) back into a single output h5mu.

Available integration methods: Harmony, scVI, Scanorama, BBKNN.

Type

nextflow_script

Links

Repository

Contributors

Launch

Run this component

Run the following command to execute this component with Nextflow:

cat > params.yaml <<'EOM'  
id: "run"  
modality: [ "rna" ]  
layer_log_normalized_counts: [ "log_normalized" ]  
obsm_embedding: [ "X_pca" ]  
obs_batch: [ "sample_id" ]  
obs_covariates: [ "sample_id" ]  
leiden_resolution: [ 1 ]  
harmony_theta: [ 2 ]  
scvi_early_stopping_monitor: [ "elbo_validation" ]  
scvi_early_stopping_patience: [ 45 ]  
scvi_early_stopping_min_delta: [ 0 ]  
scvi_reduce_lr_on_plateau: [ true ]  
scvi_lr_factor: [ 0.6 ]  
scvi_lr_patience: [ 30 ]  
scanorama_knn: [ 20 ]  
scanorama_batch_size: [ 5000 ]  
scanorama_sigma: [ 15 ]  
scanorama_approx: [ true ]  
scanorama_alpha: [ 0.1 ]  
bbknn_n_neighbors_within_batch: [ 3 ]  
bbknn_n_pcs: [ 50 ]  
harmony_obsm_integrated: [ "X_harmony_integrated" ]  
harmony_obsm_umap: [ "X_harmony_umap" ]  
harmony_obs_cluster: [ "harmony_integration_leiden" ]  
harmony_uns_neighbors: [ "harmony_integration_neighbors" ]  
harmony_obsp_neighbor_distances: [ "harmony_integration_neighbor_distances" ]  
harmony_obsp_neighbor_connectivities: [ "harmony_integration_neighbor_connectivities" ]  
scvi_obsm_integrated: [ "X_scvi_integrated" ]  
scvi_obsm_umap: [ "X_scvi_umap" ]  
scvi_obs_cluster: [ "scvi_integration_leiden" ]  
scvi_uns_neighbors: [ "scvi_integration_neighbors" ]  
scvi_obsp_neighbor_distances: [ "scvi_integration_neighbor_distances" ]  
scvi_obsp_neighbor_connectivities: [ "scvi_integration_neighbor_connectivities" ]  
scanorama_obsm_integrated: [ "X_scanorama_integrated" ]  
scanorama_obsm_umap: [ "X_scanorama_umap" ]  
scanorama_obs_cluster: [ "scanorama_integration_leiden" ]  
scanorama_uns_neighbors: [ "scanorama_integration_neighbors" ]  
scanorama_obsp_neighbor_distances: [ "scanorama_integration_neighbor_distances" ]  
scanorama_obsp_neighbor_connectivities: [ "scanorama_integration_neighbor_connectivities" ]  
bbknn_obsm_umap: [ "X_bbknn_umap" ]  
bbknn_obs_cluster: [ "bbknn_integration_leiden" ]  
bbknn_uns_neighbors: [ "bbknn_integration_neighbors" ]  
bbknn_obsp_neighbor_distances: [ "bbknn_integration_neighbor_distances" ]  
bbknn_obsp_neighbor_connectivities: [ "bbknn_integration_neighbor_connectivities" ]  
output: "$id.$key.output.h5mu"  
output_scvi_model: "$id.$key.output_scvi_model.scvi_model"  
publish_dir: "output/"  
EOM

nextflow run https://packages.viash-hub.com/vsh/openpipeline_composed.git \  
  -revision v0.2.1 \  
  -main-script target/nextflow/single_cell/parallel_integration/main.nf \  
  -params-file params.yaml

Inputs

Name	Type & Properties	Description
--id	string required	ID of the sample.
--input	file required	Input preprocessed h5mu file. Must contain a log-normalized layer and a PCA embedding (or the embedding specified via --obsm_embedding).
--modality	string	Which modality to process.
--layer_log_normalized_counts	string	Layer with the log-normalized counts. Read by the methods that operate on normalized expression: harmony, scanorama and bbknn.
--layer_raw_counts	string	Layer with the raw (integer) counts. Read by scVI, which requires raw counts rather than log-normalized data. If left unset, scVI reads the counts from the modality's .X.
--obsm_embedding	string	.obsm slot containing the embedding to use as input for methods that require a pre-computed dimensionality reduction (harmony, scanorama, bbknn).
--obs_batch	string	.obs column discriminating between batches. Read by scVI, scanorama and bbknn.
--obs_covariates	string multiple	.obs column(s) defining the covariates to regress out. Read by harmony.
--obs_categorical_covariates	string multiple	.obs column(s) with categorical covariates for scVI to correct for in addition to --obs_batch. Treated as nuisance factors, so do not use them for biological signal you want to keep. None by default.
--obs_numerical_covariates	string multiple	.obs column(s) with continuous (numerical) covariates for scVI to correct for in addition to --obs_batch. Treated as nuisance factors, so do not use them for biological signal you want to keep. None by default.
--var_input	string	.var column containing highly variable genes to subset to. Read by scVI. By default, scVI uses all genes.

Name

Type & Properties

--id

string

required

--input

file

required

--modality

string

--layer_log_normalized_counts

string

--layer_raw_counts

string

--obsm_embedding

string

--obs_batch

string

--obs_covariates

string

multiple

--obs_categorical_covariates

string

multiple

--obs_numerical_covariates

string

multiple

--var_input

string

Methods

Name	Type & Properties	Description
--integration_methods	string required multiple	Integration methods to run in parallel.

Name

Type & Properties

--integration_methods

string

required

multiple

Clustering options

Name	Type & Properties	Description
--leiden_resolution	double multiple	Leiden clustering resolution(s). Each method will produce one .obs cluster column per resolution, named "{method}_integration_leiden_{resolution}".

Name

Type & Properties

--leiden_resolution

double

multiple

Harmony options

Name	Type & Properties	Description
--harmony_theta	double multiple	Diversity clustering penalty. Higher values produce more diverse clusters.

Name

Type & Properties

--harmony_theta

double

multiple

scVI options

Name	Type & Properties	Description
--scvi_early_stopping	boolean	Whether scVI should perform early stopping on the validation set.
--scvi_early_stopping_monitor	string	Metric logged during validation set epoch.
--scvi_early_stopping_patience	integer	Number of validation epochs with no improvement after which training stops.
--scvi_early_stopping_min_delta	double	Minimum change in the monitored metric to qualify as improvement.
--scvi_max_epochs	integer	Number of training passes through the dataset.
--scvi_reduce_lr_on_plateau	boolean	Reduce learning rate when validation metric plateaus.
--scvi_lr_factor	double	Factor by which to reduce learning rate.
--scvi_lr_patience	double	Number of epochs with no improvement after which lr is reduced.

Name

Type & Properties

--scvi_early_stopping

boolean

--scvi_early_stopping_monitor

string

--scvi_early_stopping_patience

integer

--scvi_early_stopping_min_delta

double

--scvi_max_epochs

integer

--scvi_reduce_lr_on_plateau

boolean

--scvi_lr_factor

double

--scvi_lr_patience

double

Scanorama options

Name	Type & Properties	Description
--scanorama_knn	integer	Number of nearest neighbors used for matching.
--scanorama_batch_size	integer	Batch size used in the alignment vector computation.
--scanorama_sigma	double	Correction smoothing parameter on the Gaussian kernel.
--scanorama_approx	boolean	Use approximate nearest neighbors (annoy).
--scanorama_alpha	double	Alignment score minimum cutoff.

Name

Type & Properties

--scanorama_knn

integer

--scanorama_batch_size

integer

--scanorama_sigma

double

--scanorama_approx

boolean

--scanorama_alpha

double

BBKNN options

Name	Type & Properties	Description
--bbknn_n_neighbors_within_batch	integer	Top neighbours to report for each batch.
--bbknn_n_pcs	integer	Number of dimensions to use from the input embedding.
--bbknn_n_trim	integer	Trim each cell's neighbours to these many top connectivities. Set to 0 to skip; leave unset to use the BBKNN default (10 * n_neighbors_within_batch * n_batches).

Name

Type & Properties

--bbknn_n_neighbors_within_batch

integer

--bbknn_n_pcs

integer

--bbknn_n_trim

integer

Output slot naming

Name	Type & Properties	Description
--harmony_obsm_integrated	string	.obsm slot for the harmony-integrated embedding.
--harmony_obsm_umap	string	.obsm slot for the harmony UMAP embedding.
--harmony_obs_cluster	string	.obs prefix for the harmony leiden cluster columns.
--harmony_uns_neighbors	string	.uns slot for harmony neighbor graph metadata.
--harmony_obsp_neighbor_distances	string	.obsp slot for harmony neighbor distances.
--harmony_obsp_neighbor_connectivities	string	.obsp slot for harmony neighbor connectivities.
--scvi_obsm_integrated	string	.obsm slot for the scVI-integrated latent embedding.
--scvi_obsm_umap	string	.obsm slot for the scVI UMAP embedding.
--scvi_obs_cluster	string	.obs prefix for the scVI leiden cluster columns.
--scvi_uns_neighbors	string	.uns slot for scVI neighbor graph metadata.
--scvi_obsp_neighbor_distances	string	.obsp slot for scVI neighbor distances.
--scvi_obsp_neighbor_connectivities	string	.obsp slot for scVI neighbor connectivities.
--scanorama_obsm_integrated	string	.obsm slot for the scanorama-integrated embedding.
--scanorama_obsm_umap	string	.obsm slot for the scanorama UMAP embedding.
--scanorama_obs_cluster	string	.obs prefix for the scanorama leiden cluster columns.
--scanorama_uns_neighbors	string	.uns slot for scanorama neighbor graph metadata.
--scanorama_obsp_neighbor_distances	string	.obsp slot for scanorama neighbor distances.
--scanorama_obsp_neighbor_connectivities	string	.obsp slot for scanorama neighbor connectivities.
--bbknn_obsm_umap	string	.obsm slot for the BBKNN UMAP embedding.
--bbknn_obs_cluster	string	.obs prefix for the BBKNN leiden cluster columns.
--bbknn_uns_neighbors	string	.uns slot for BBKNN neighbor graph metadata.
--bbknn_obsp_neighbor_distances	string	.obsp slot for BBKNN neighbor distances.
--bbknn_obsp_neighbor_connectivities	string	.obsp slot for BBKNN neighbor connectivities.

Name

Type & Properties

--harmony_obsm_integrated

string

--harmony_obsm_umap

string

--harmony_obs_cluster

string

--harmony_uns_neighbors

string

--harmony_obsp_neighbor_distances

string

--harmony_obsp_neighbor_connectivities

string

--scvi_obsm_integrated

string

--scvi_obsm_umap

string

--scvi_obs_cluster

string

--scvi_uns_neighbors

string

--scvi_obsp_neighbor_distances

string

--scvi_obsp_neighbor_connectivities

string

--scanorama_obsm_integrated

string

--scanorama_obsm_umap

string

--scanorama_obs_cluster

string

--scanorama_uns_neighbors

string

--scanorama_obsp_neighbor_distances

string

--scanorama_obsp_neighbor_connectivities

string

--bbknn_obsm_umap

string

--bbknn_obs_cluster

string

--bbknn_uns_neighbors

string

--bbknn_obsp_neighbor_distances

string

--bbknn_obsp_neighbor_connectivities

string

Output

Name	Type & Properties	Description
--output -o	file required output	Output h5mu file — the input preprocessed file with each selected method's annotations merged into the target modality.
--output_compression	string	Compression format for the output h5mu file.
--output_scvi_model	file output	Output folder with the trained scVI model. Only produced when scVI is among the selected --integration_methods.

Name

Type & Properties

--output

-o

file

required

output

--output_compression

string

--output_scvi_model

file

output