Pipeline¶
The staged orchestrator and its result manifest.
rustscenic.pipeline ¶
End-to-end rustscenic stage orchestrator.
Public API: rustscenic.pipeline.run(rna, output_dir, *, fragments=None, peaks=None, tfs=None, motif_rankings=None, ...) -> PipelineResult
One call runs every rustscenic stage the user provides input for:
1. preproc (fragments + peaks) → cells × peaks AnnData
2. topics (cells × peaks AnnData) → cell-topic + topic-peak matrices
3. grn (RNA expression + TFs) → TF-target importances
4. regulons (grn) → top-N targets per TF
5. cistarget (regulons + motif DB) → motif-enriched regulons [optional]
6. enhancer (RNA + ATAC + TSS) → peak-gene links [optional]
7. eRegulon (GRN + motifs + links) → TF-enhancer-gene modules [optional]
8. aucell (RNA + regulons) → per-cell regulon activity
Outputs are written to output_dir as parquet / json / h5ad files so
downstream notebooks can pick up where the pipeline left off.
No new Python dependencies. Uses only numpy, pandas, pyarrow, scipy, plus the rustscenic Rust backend.
PipelineResult
dataclass
¶
PipelineResult(
output_dir: Path,
atac_matrix_path: Path | None = None,
cell_barcode_filter: dict[str, int] | None = None,
grn_path: Path | None = None,
regulons_path: Path | None = None,
candidate_regulons_path: Path | None = None,
pruned_regulons_path: Path | None = None,
aucell_path: Path | None = None,
topics_dir: Path | None = None,
cistarget_path: Path | None = None,
enhancer_links_path: Path | None = None,
eregulons_path: Path | None = None,
integrated_adata_path: Path | None = None,
elapsed: dict = dict(),
io_elapsed: dict = dict(),
memory: dict = dict(),
n_cells: int | None = None,
n_grn_edges: int | None = None,
n_regulons: int | None = None,
n_candidate_regulons: int | None = None,
n_pruned_regulons: int | None = None,
n_cistarget_rows: int | None = None,
n_enhancer_links: int | None = None,
n_eregulon_rows: int | None = None,
n_eregulons: int | None = None,
aucell_shape: list[int] | None = None,
regulon_source: str = "candidate_grn_activator_repressor",
grn_fit: dict = dict(),
grn_correlation: dict = dict(),
topics_fit: dict = dict(),
backend_execution: dict = dict(),
matrix_inputs: dict = dict(),
cistarget_rankings: dict = dict(),
region_cistarget_rankings: dict = dict(),
)
Artifacts and metadata from a pipeline run.
All file paths point inside output_dir. Stages that were skipped
because inputs weren't provided have None for their result path.
Attributes:
| Name | Type | Description |
|---|---|---|
output_dir |
Path
|
Directory containing the run's written artifacts and manifest. |
atac_matrix_path |
Path | None
|
Written cells-by-peaks AnnData file, when ATAC inputs were available. |
cell_barcode_filter |
dict[str, int] | None
|
Optional barcode-filter counts recovered from ATAC metadata. |
grn_path |
Path | None
|
Written TF-target importance table. |
regulons_path |
Path | None
|
Written active regulon collection used for scoring. |
candidate_regulons_path |
Path | None
|
Written GRN-derived candidate regulons before motif pruning. |
pruned_regulons_path |
Path | None
|
Written motif-pruned regulons, when annotation pruning was run. |
aucell_path |
Path | None
|
Written cells-by-regulons activity scores. |
topics_dir |
Path | None
|
Directory containing the fitted topic matrices, when ATAC was used. |
cistarget_path |
Path | None
|
Written motif enrichment table, when motif rankings were supplied. |
enhancer_links_path |
Path | None
|
Written peak-to-gene links, when linking inputs were available. |
eregulons_path |
Path | None
|
Written long eRegulon edge table, when assembly was run. |
integrated_adata_path |
Path | None
|
RNA AnnData file with AUCell scores attached, unless writing was disabled. |
elapsed |
dict
|
Recorded stage elapsed times in seconds. |
io_elapsed |
dict
|
Separately instrumented IO elapsed times in seconds. |
memory |
dict
|
Process peak-RSS snapshots observed after recorded stages. These are cumulative process peaks, not incremental allocations per stage. |
n_cells |
int | None
|
Number of RNA cells entering the run. |
n_grn_edges |
int | None
|
Number of inferred TF-target edges. |
n_regulons |
int | None
|
Number of active regulons used for AUCell scoring. |
n_candidate_regulons |
int | None
|
Number of GRN-derived candidate regulons before motif pruning. |
n_pruned_regulons |
int | None
|
Number of regulons retained by motif pruning, when performed. |
n_cistarget_rows |
int | None
|
Number of reported motif enrichment rows, when enrichment was run. |
n_enhancer_links |
int | None
|
Number of retained peak-to-gene links, when linking was run. |
n_eregulon_rows |
int | None
|
Number of rows in the assembled long eRegulon edge table. |
n_eregulons |
int | None
|
Number of assembled eRegulons. |
aucell_shape |
list[int] | None
|
Activity-matrix dimensions as |
regulon_source |
str
|
Label recording how the active regulon collection was selected. |
grn_fit |
dict
|
Recorded GRN fitting settings and fitted-tree summaries. |
grn_correlation |
dict
|
Recorded polarity and correlation settings and edge-count summaries. |
topics_fit |
dict
|
Recorded topic-model settings and assignment summaries. |
backend_execution |
dict
|
Per-stage backend symbol provenance or reasons stages were skipped. |
matrix_inputs |
dict
|
Recorded profiles of the RNA and shared-cell ATAC inputs. |
cistarget_rankings |
dict
|
Recorded gene-ranking input provenance and projection metadata. |
region_cistarget_rankings |
dict
|
Recorded region-ranking input provenance and projection metadata. |
run ¶
run(
rna: str | Path | Any,
output_dir: str | Path,
*,
adata_atac: Any | None = None,
fragments: str | Path | None = None,
peaks: str | Path | None = None,
tfs: str | Path | Iterable[str] | None = None,
motif_rankings: str | Path | DataFrame | None = None,
motif_annotations: str | Path | DataFrame | None = None,
region_motif_rankings: str | Path | DataFrame | None = None,
gene_coords: str | Path | DataFrame | None = None,
grn_n_estimators: int = 500,
grn_max_features: float = 0.1,
grn_early_stop_window: int = 25,
grn_early_stop_mode: str = "arboreto",
grn_target_block_size: int | None = None,
grn_top_targets: int = 50,
grn_regulon_polarities: str = "both",
grn_rho_threshold: float = 0.03,
grn_rho_mask_dropouts: bool = False,
aucell_top_frac: float = 0.05,
topics_n_topics: int = 30,
topics_n_passes: int = 3,
topics_method: str = "vb",
topics_n_iters: int = 200,
topics_n_threads: int = 1,
cistarget_top_frac: float = 0.05,
cistarget_auc_threshold: float = 0.05,
cistarget_nes_threshold: float | None = None,
enhancer_max_distance: int = 500000,
enhancer_min_abs_corr: float = 0.1,
eregulon_min_target_genes: int = 5,
eregulon_min_enhancer_links: int = 2,
write_integrated_adata: bool = True,
seed: int = 777,
verbose: bool = True,
) -> PipelineResult
Run the available rustscenic stages end-to-end.
The workflow runs only the stages the user supplies inputs for. At
minimum, rna is required (for GRN + AUCell). Providing
fragments and peaks enables preproc + topics. Providing
motif_rankings enables cistarget.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
rna
|
str | Path | Any
|
An AnnData, a path to an |
required |
output_dir
|
str | Path
|
Directory where all artifacts are written. Created if missing. |
required |
adata_atac
|
Any | None
|
Pre-built cells × peaks |
None
|
fragments
|
str | Path | None
|
Paths to a 10x-style |
None
|
peaks
|
str | Path | None
|
Paths to a 10x-style |
None
|
tfs
|
str | Path | Iterable[str] | None
|
Candidate transcription factor names. Path to a newline-separated
file, an iterable of strings, or |
None
|
motif_rankings
|
str | Path | DataFrame | None
|
Motif ranking DataFrame, or a path to a parquet / feather file with motifs as rows and genes as columns. If provided, cistarget runs to score candidate regulons for motif enrichment. |
None
|
motif_annotations
|
str | Path | DataFrame | None
|
Optional motif-to-TF annotation table. When provided alongside
|
None
|
region_motif_rankings
|
str | Path | DataFrame | None
|
Optional region-based motif ranking DataFrame, or path, with motifs as rows and peak / region IDs as columns. When supplied alongside ATAC inputs and gene coordinates, eRegulon assembly uses this exact region-cistarget path instead of the gene-cistarget bridge. File-backed parquet / feather rankings are projected to the peaks used by the current run so large region-ranking databases do not need to be loaded in full. |
None
|
gene_coords
|
str | Path | DataFrame | None
|
DataFrame with columns |
None
|
topics_method
|
str
|
|
'vb'
|
topics_n_iters
|
int
|
Gibbs sweeps (only used when |
200
|
topics_n_threads
|
int
|
Threads for the Gibbs sampler (only used when
|
1
|
grn_max_features
|
float
|
Fraction of candidate TFs sampled per split in GRN boosting.
|
0.1
|
grn_early_stop_mode
|
str
|
|
'arboreto'
|
grn_regulon_polarities
|
str
|
|
'both'
|
grn_target_block_size
|
int | None
|
Retained for API compatibility. Flat GRN scheduling ignores the value. |
None
|
grn_n_estimators
|
int
|
Maximum boosting trees fitted per target gene. Early stopping may
fit fewer; passed to |
500
|
grn_early_stop_window
|
int
|
Trailing-window size for the selected GRN stopping monitor. Zero disables early stopping. |
25
|
grn_top_targets
|
int
|
Maximum highest-importance targets per candidate regulon, applied
separately to each TF and polarity in signed mode. Candidate sets
require at least |
50
|
grn_rho_threshold
|
float
|
Absolute Pearson-correlation threshold for activating or repressing
GRN edges. Neutral edges are excluded from signed regulons.
Ignored for |
0.03
|
grn_rho_mask_dropouts
|
bool
|
Exclude cells where either gene is zero when scoring correlations. Ignored for unsigned regulons. |
False
|
aucell_top_frac
|
float
|
Fraction of the highest-ranked genes per cell used for AUCell recovery, greater than zero and at most one. |
0.05
|
topics_n_topics
|
int
|
Number of latent topics requested when ATAC input is available. |
30
|
topics_n_passes
|
int
|
Passes over the ATAC corpus for |
3
|
cistarget_top_frac
|
float
|
Ranking-universe fraction used for enrichment recovery AUC and for the target-recovery window during motif-annotation pruning. |
0.05
|
cistarget_auc_threshold
|
float
|
Minimum motif AUC for enrichment reporting and annotation-based motif filtering on the gene- and region-ranking paths. |
0.05
|
cistarget_nes_threshold
|
float | None
|
Optional minimum normalized enrichment score. |
None
|
enhancer_max_distance
|
int
|
Maximum peak-center to gene-TSS distance in base pairs for enhancer-to-gene linking when its inputs are available. |
500000
|
enhancer_min_abs_corr
|
float
|
Minimum absolute correlation retained for enhancer-to-gene links. |
0.1
|
eregulon_min_target_genes
|
int
|
Minimum surviving target genes required for an assembled eRegulon. |
5
|
eregulon_min_enhancer_links
|
int
|
Minimum peak-to-gene links required for an assembled eRegulon. |
2
|
write_integrated_adata
|
bool
|
Write |
True
|
seed
|
int
|
Random seed passed to GRN inference and the selected topic fitter. Hold the Gibbs thread count and other settings fixed as well. |
777
|
verbose
|
bool
|
Enable pipeline progress logging. Called stages may still emit their own messages or warnings when this is false. |
True
|
Returns:
| Type | Description |
|---|---|
PipelineResult - dataclass with paths to every artifact written.
|
|