sc-report

A section whose inputs are present but unreadable is reported at the top of the page and in provenance.json (sectionErrors); the rest is still rendered.

Citations

Every report prints this statement first in its Citations section, above the per-step methods, word for word (only the DOI is a link):

Citing Pennsieve. If you used Pennsieve to store, analyse, share or publish data reported in a publication, please cite: Goldblum Z, et al. Pennsieve: A Collaborative Platform for Translational Neuroscience and Beyond. Scientific Data 2025;12:1834. doi:10.1038/s41597-025-06075-5. Please also cite the datasets you used (their DOIs) and the methods listed below.

The paper is in processor/references.json (verified like the others), first in tables/citations.csv with role platform, and first in summary.json citations. It is not in this app's CITATION.cff: the app also runs outside Pennsieve.

A step's provenance.json may cite a bare DOI (as conventions.md describes) or an object {doi, pmid, text} (as sc-integrate and sc-markers write). Each DOI is looked up in processor/references.json, which scripts/verify_citations.py wrote from Crossref, PubMed, DataCite and the Discover API. The page shows the verified record, marked ✓. A DOI the table does not hold is shown with the step's own text, marked "as given by the step". st-compare (reused unchanged from the spatial suite) writes no citations, so its methods papers come from processor/citations.py. When an upstream app adds a reference, re-run scripts/verify_citations.py with its DOI added, so the report shows it verified.

Inputs, all placement: subfolder and none required: metadata, qc, integration, annotation, markers, compare, atac, regulome (see sample-input/README.md for which upstream output goes where). markers, compare and regulome may each receive several links, one <LABEL>/ folder each.

Every input path comes from the shared processor/safe_inputs.py (vendored unchanged from workflow-single-cell/shared/): InputDir.walk() follows symlinks only to where the platform's own links point (the entries of INPUT_DIR and of a real port folder, one hop: a link that an upstream app wrote into its own output is not trusted), and a folder's files are the walked files in it, never a fresh glob or path join. Untrusted content is handled as data:

• every file is read through the capped readers: JSON up to 20 MB, whole CSVs up to 256 MB; labels.csv, st-compare's de.csv and atac-prep's atac_qc.csv are read in chunks of a few columns from the checked file;
• per-cell tables (any CSV with an obs_id column, e.g. sc-qc's <sample_id>_qc.csv) are never shown;
• every value is HTML-escaped by Jinja2's autoescape, and the page carries a Content Security Policy (default-src 'none'; img-src data:; style-src 'unsafe-inline');
• a figure is inlined only through InputDir.inline_png: a PNG signature, at most 8 MB.

Outputs (two, so each in OUTPUT_DIR/<name>/):

Exit codes: 0 success; 1 none of the inputs found; 2 bad parameter or INPUT_DIR not a folder. provenance.json has the conventions' keys (app, version, tools, parameters, inputs, started, finished, samples, peak_rss_mb, cgroup_peak_mb, threads, citations) plus the folders found per port, sections, section errors and omitted figures.

drg-c2 (Discover 475), the whole of workflow A as it stood on 2026-10-31 18:48 UTC: sc-prep metadata, sc-qc qc, sc-integrate integration, sc-annotate labels and annotation, sc-markers (cell-type and cluster), st-compare (cell-type, condition: acute vs chronic). Each port was a real folder of symlinks into the upstream outputs, as the platform links them. Output in the workflow repository under data/work/drg-c2/report/{report,tables}/. The page (9.1 MB, 34 figures inlined) says:

• Samples: 12 samples, 20,415 nuclei (186–3,000 per sample; sc-prep capped at 3,000), human, Flex; acute (7) vs chronic (5) neck pain (sc-prep copied subject_experimental_group to condition, so no confounding caveat is raised for that copy).
• QC: 1,977 of 20,415 nuclei removed (9.7%; 4.6% to 16% per sample), mostly high mitochondrial fraction (1,252) and high top-20-gene share (593); ambient removal skipped (no GPU).
• Clusters: 15 clusters over 18,438 nuclei, Harmony.
• Cell types: 10 types; Unassigned 35%, Fibroblast 20%, Satellite glia 14%, Neuron 11%; median cell-type score 0.22 (markers mode). Largest difference in mean share: Nociceptor (NP MRGPRD-type) 5.9% in acute vs 3.3% in chronic (descriptive).
• Comparison: no cell type's share differs at FDR 0.05 (with 7 vs 5 samples the rank test cannot go below p = 0.0025 before correction); pseudobulk DE finds 12 significant gene × cell type results over 10 tested types.
• Markers: 500 (cell types) and 750 (clusters), with 6,777 and 12,871 pooled hits dropped as inconsistent across samples.
• Citations: 29, all rendered from the verified table; data credit for all 12 samples.

cpu: 1024, memory: 2048, computeTypes: [standard, lambda] (the image serves both; see processor/entry.py and processor/handler.py), timeout 10 min. Measured with docker run --memory 3g --cpus 1 --network none --read-only --tmpfs /tmp under amd64 emulation on an M-series Mac:

Memory does not grow with the number of cells beyond one 500,000-row chunk of three labels.csv columns. Image: 540 MB (python:3.12.16-slim-bookworm + pandas, matplotlib, jinja2, awslambdaric). The page's size is mostly its inlined figures: about 50–300 KB each at 150 dpi, at most MAX_FIGURES per section.

python3.12 -m venv .venv && .venv/bin/pip install -r requirements-dev.txt
make test PYTHON=.venv/bin/python           # synthetic fixture: both workflows, each alone, each port alone, none
make run PYTHON=.venv/bin/python            # Docker over data/fixture/linked/ -> data/output/
INPUT_HOST=/path/to/inputs make run         # Docker over your own folder (one subfolder per port)
make validate PYTHON=.venv/bin/python       # app.yml against the manifest schema (+ CITATION.cff with cffconvert)

tests/make_fixture.py writes contract-following upstream outputs for every port, with planted findings: a sample losing 45% to QC, a cluster in one sample only, neurons 5% → 15% between conditions, a significant composition test, a DOI the reference table does not hold, and st-compare's provenance without citations. The tests also cover:

• the flat layout (upstream folders with uninformative names) and the platform's real link layout (port folders whose entries are symlinks into sibling upstream outputs);
• links an upstream app wrote into its own output pointing outside it (a summary.json and a figure next to a run folder): never read, never inlined;
• the ATAC contract (passed false only), and the CSP meta tag;
• broken inputs: invalid JSON, a non-PNG .png, and HTML/script in a metadata value (escaped, never executed);
• a symlink escaping INPUT_DIR;
• MAX_FIGURES, bad parameters, no inputs, and the Lambda handler.

• Only as good as the upstream outputs. The report re-thresholds and cross-references; it takes every statistic as the steps wrote it. A caveat on the page restates an app's own limitation; it is not a new test.
• Descriptive composition. Shares per condition are means over samples; the test of a change is st-compare's, shown in the comparison section.
• Unknown formats are shown generically. sc-qc, sc-annotate, atac-prep and atac-regulome were being built at the same time as this app. Their sections show summary numbers, status, tables and figures, with dedicated sentences only where the contract fixes the format (QC counts are read from n_<cells|obs|nuclei>_before/after and n_removed_<reason> columns).
• Demo-dataset credit is recognised from sc-prep's sample paths and ids: drg-c2 (Discover 475, sub-UTD-CC…/sam-…-DRG?/), the DRG atlas (480, sub-…/sam-…/sam-…-<level>/) and the sympathetic ganglia multiome (465, sam-hSG-lib-<n>). Other data get no credit line: cite your data yourself.
• Static PNG figures, light theme, English only.
• Shared template with st-report. The two reports share the page's CSS, the methods/citations machinery and verify_citations.py. A shared package would be simpler once a third report exists; for now each is self-contained, and changes to one should be copied to the other.

Our code: Apache-2.0 (LICENSE). Bundled components and their licences: THIRD_PARTY.md (all permissive; no GPL Python packages, while the Debian base layer contains GPL system tools).

The report shows other people's work: each step's methods (cited per step on the page) and the data. Its own dependencies:

• Heumos et al. and the Single-cell Best Practices Consortium wrote the guide whose caveats the report restates.
• Matplotlib (Hunter) draws the composition figure; pandas and Jinja2 (Pallets) build the tables and the page.
• Demo data: Arendt-Tranholm A, Sankaranarayanan I and Price TJ shared the drg-c2 single-nucleus data (Discover 475, CC BY 4.0).
• Sankaranarayanan I and Price TJ shared the DRG atlas (Discover 480, CC BY 4.0).
• Yang L and Gereau R shared the sympathetic ganglia multiome atlas (Discover 465, CC BY 4.0).

scripts/verify_citations.py fetched every DOI the report can cite on 2026-10-31:

• Papers: Crossref for the title, the full author list in order, journal, year, volume, issue and pages; then PubMed (esearch by DOI, then esummary), where title, first author and year must match and the PMID is taken from the result.
• Datasets: DataCite (title, creators, year, licence) and the Discover API (DOI, contributors with ORCID, version, licence).

processor/references.json holds 60 papers and 3 datasets. All but the platform paper (59 papers and 3 datasets) are in CITATION.cff with author lists as Crossref and Discover give them, and in processor/references.json. The platform paper (printed first, not in CITATION.cff):

• Goldblum Z, Xu Z, Shi H, et al. (9 authors). Pennsieve: A Collaborative Platform for Translational Neuroscience and Beyond. Sci Data 12(1):1834 (2025, issued 2025-11-19). doi:10.1038/s41597-025-06075-5. PMID 41261152 (PubMed: title, first author and year matched on 2026-10-31).

The report's own:

• Heumos L, Schaar AC, Lance C, et al. Best practices for single-cell analysis across modalities. Nat Rev Genet 24(8):550-572 (2023). doi:10.1038/s41576-023-00586-w. PMID 37002403.
• Hunter JD. Matplotlib: A 2D Graphics Environment. Comput Sci Eng 9(3):90-95 (2007). doi:10.1109/MCSE.2007.55. (Crossref; not in PubMed)

The table also holds every DOI the other single-cell apps cite in their CITATION.cff or provenance.json (sc-prep, sc-qc, sc-integrate, sc-annotate, atac-prep, atac-regulome: e.g. Scrublet, CellBender, pipeComp, scANVI, CellTypist, SnapATAC2, ArchR, MACS, GENCODE 2023, JASPAR 2026, NumPy, scikit-image, the arXiv records of scikit-learn and PyTorch) and fastp 1.0 (Chen S, iMeta 4(5):e70078, 2025, doi:10.1002/imt2.70078, PMID 41112039), so they render verified. Pyro (Bingham et al., JMLR 2019) has no DOI; the report lists works by DOI only, so a citation without one (Pyro, if a step cited it so) is not shown.

Per step (shown when that step's provenance is among the inputs):

• sc-prep, shared: scanpy (PMID 29409532), anndata (doi:10.21105/joss.04371).
• sc-integrate: Stuart 2019 (PMID 31178118), Harmony (PMID 31740819), scVI (PMID 30504886), scvi-tools (PMID 35132262), Leiden (PMID 30914743), UMAP (doi:10.21105/joss.00861), scIB (PMID 34949812), Rousseeuw 1987 (doi:10.1016/0377-0427(87)90125-7).
• sc-markers: Wilcoxon 1945 (doi:10.2307/3001968), Mann & Whitney 1947 (doi:10.1214/aoms/1177730491), Benjamini & Hochberg 1995 (doi:10.1111/j.2517-6161.1995.tb02031.x), Pullin & McCarthy 2024 (PMID 38409056), Squair 2021 (PMID 34584091), Zimmerman 2021 (PMID 33531494), Zhang 2019 (PMID 31521605).
• st-compare: PyDESeq2 (PMID 37669147), DESeq2 (PMID 25516281), scCODA (PMID 34824236), Aitchison 1982 (doi:10.1111/j.2517-6161.1982.tb01195.x), SciPy (PMID 32015543).

Demo data:

• Arendt-Tranholm A, Sankaranarayanan I, Price TJ. Single nucleus sequencing of human C2 dorsal root ganglia tissue from fusion surgeries. Discover 475 v1 (2025). doi:10.26275/cxuf-ftav. CC BY 4.0. Described in Arendt-Tranholm A, et al. Transcriptome of the human C2 dorsal root ganglia in C1–2 arthrodesis surgery: insight for neck pain. Brain 149(4):1349-1364 (2026). doi:10.1093/brain/awaf370. PMID 41032656.
• Sankaranarayanan I, Price TJ. Single nucleus sequencing atlas of human dorsal root ganglion. Discover 480 v1 (2025). doi:10.26275/9qsp-i8dh. CC BY 4.0.
• Yang L, Gereau R. A single-cell multi-omic atlas of human sympathetic ganglia. Discover 465 v1 (2025). doi:10.26275/oqxx-si9h. CC BY 4.0.