AREE is an open, reproducible research resource that converts fragmented public omics datasets into harmonized, comparable resilience-biomarker evidence for Pacific oyster (Crassostrea gigas), and is deliberately extensible to other shellfish and aquaculture organisms.
It operationalizes Objective 1 of the AREE proposal:
Develop standardized open-access, user-friendly, reproducible bioinformatics pipelines for resilience biomarker discovery through systematic reanalysis, data integration, and meta-analysis.
AREE is an evidence engine, not a claims engine. It identifies associations and evidence convergence across studies and molecular layers — never confirmed mechanistic causation — and it never presents a statistically significant hit in a single study as a validated biomarker.
All datasets shipped in this repository are clearly labeled SIMULATED demo data. They exist to exercise the full pipeline end-to-end. Real public datasets can be added with minimal modification (see docs/adding_a_study.md).
- Study registry & intake (
registry/,src/intake/) — machine-readable dataset registration with controlled vocabularies for phenotypes and stressors, plus schema validation. - Standardized reanalysis workflows (
workflows/,modules/) — Nextflow DSL2 scaffolds for RNA-seq, methylation, proteomics, and metabolomics, in raw-reanalysis or processed-results modes. - Cross-study harmonization (
src/harmonize/) — a shared, assay-agnostic evidence schema plus identifier mapping with explicit confidence levels. - Meta-analysis & candidate prioritization (
src/meta_analysis/,src/prioritize/) — random-effects pooling, heterogeneity statistics, and a transparent (non-black-box) candidate score with hard tier gates. - User-facing outputs (
app/,docs/,src/reporting/) — a Streamlit interface, a Quarto documentation site, and per-candidate evidence cards.
Requires Python ≥ 3.9.
python -m venv .venv && source .venv/bin/activate
pip install -e ".[dev,app]"See docs/installation.md for details.
# 1. Validate a study registration file
aree validate-study registry/studies/GIGAS_HEAT01.yaml
# 2. Register every demo study
for f in registry/studies/GIGAS_*.yaml; do aree register-study "$f"; done
aree list-studies
# 3. Harmonize each study into the shared evidence table
for sid in GIGAS_HEAT01 GIGAS_OA02 GIGAS_PATH03 GIGAS_SAL04 GIGAS_LARV05 GIGAS_GROW06; do
aree harmonize --study "$sid"
done
# (or harmonize a single processed results table)
aree harmonize --study GIGAS_SAL04 \
--input data/demo/proteomics/GIGAS_SAL04_low_salinity_vs_control_protein_abundance_demo.tsv
# 4. Run a meta-analysis
aree meta-analyze --phenotype thermal_tolerance --feature-type gene
aree meta-analyze --feature-type gene
# 5. Generate biomarker evidence cards
aree build-evidence-cards --phenotype thermal_tolerance
# 6. Build the docs site / launch the interface
quarto render docs
streamlit run app/main.pyOr run the whole demo in one step:
make demoAREE/
├── README.md, LICENSE, CITATION.cff, CONTRIBUTING.md, CODE_OF_CONDUCT.md
├── pyproject.toml, Makefile
├── docs/ # Quarto site + narrative + how-to documentation
├── schemas/ # JSON Schema: study.schema.json, evidence.schema.json
├── registry/
│ ├── studies/ # per-study YAML registrations (+ templates)
│ ├── controlled_vocabularies/ # phenotype, stressor, tissue, life-stage, etc.
│ └── study_registry.csv # flat index (generated by `aree register-study`)
├── workflows/ # Nextflow DSL2 scaffolds (rnaseq/methylation/proteomics/metabolomics)
├── modules/ # reusable Nextflow process modules per assay
├── containers/ # container image strategy (documentation)
├── config/ # shared Nextflow config (base + demo)
├── data/
│ ├── demo/ # SIMULATED demo result tables per assay
│ ├── reference/ # genome/annotation metadata
│ └── mappings/ # identifier crosswalk + ambiguous-symbol map
├── src/
│ ├── common/ # shared paths, IO, vocabulary loaders
│ ├── intake/ # schema validation + registry ingestion
│ ├── harmonize/ # per-assay harmonizers -> shared evidence schema
│ ├── meta_analysis/ # random-effects pooling + heterogeneity
│ ├── prioritize/ # transparent scoring + tier gating
│ ├── reporting/ # evidence cards + provenance manifests
│ ├── validation/ # reusable validation checks
│ └── aree/ # the `aree` CLI
├── app/ # Streamlit interface
├── reports/ # generated outputs (gitignored)
├── tests/ # pytest suite
└── .github/workflows/ # CI
| Command | Purpose |
|---|---|
aree validate-study <file> |
Validate a study YAML against schema + controlled vocabularies |
aree register-study <file> [--update] |
Add (or update) a study in the registry |
aree list-studies |
List registered studies and their pipeline status |
aree harmonize --study <id> [--input <file>] |
Harmonize a study (or one processed table) into the evidence table |
aree meta-analyze [--phenotype <p>] [--feature-type <t>] |
Random-effects meta-analysis over the evidence table |
aree build-evidence-cards [--phenotype <p>] [--feature-type <t>] |
Generate per-candidate evidence cards |
Start with docs/index.qmd or render the site with
quarto render docs. Key pages:
- Why this resource matters
- Design document · Technical architecture
- Adding a study · Identifier mapping
- Interpreting meta-analysis · Interpreting candidate scores
- Governance & provenance · Roadmap
This is a functioning MVP. See docs/roadmap.md for a candid breakdown of what is complete and runnable (schemas, registry, harmonization for all four assay types, meta-analysis, prioritization, evidence cards, CLI, Streamlit app, Quarto docs, tests, CI) versus scaffolded but not yet production-ready (the Nextflow raw-data workflows, which are structurally complete but have not been executed against real sequencing data in this build) versus planned (real ortholog mapping, additional species, hosted deployment).
MIT — see LICENSE.
See CITATION.cff.