Pipeline
Every paper's position in the automated workflow. Each row is a stage; counts come from entities.db.schedule.
Discovered
stage='discovered'30min cronSeen by paper_discovery / local-folder ingest
→ paper_discovery.py
Triaged
stage='triaged'30min cronScored by paper_triage (regex + Haiku + composite)
→ paper_triage.py
Queued
stage='queued'manualsuggested_action ∈ {read, integrate}
→ /triage UI
Ingested
stage='ingested'on-demand + nightlyPDF/URL → pdftotext → entity + framings (Sonnet)
→ ingest_one_url.py
Refs extracted
stage='refs-extracted'at ingest + nightly backfillOpenAlex referenced_works + S2 fallback
→ enrich_citations_openalex.py + enrich_citations_s2.py
Graphified
stage='graphified'at ingestTyped extraction: claims, findings, hypotheses, concepts, relations
→ extract_typed.py (DeepSeek V4 Pro)
Merged into 7-way
stage='merged'—Cross-corpus graph updated (legacy graphify path)
→ — (deprecated; corpus.communities used instead)
Surfaced to multica
stage='multica-surfaced'30min cronAnton sees it in /pending
→ papers_to_multica.py
Next to triage
`discovered` → `triaged`
- The World Inside Neural Networks
- Probe-Based Data Attribution: Surfacing and Mitigating Undesirable Behaviors in LLM Post-Training
- Cognitive glues are shared models of relative scarcities: the economics of collective intelligence
- Anima Labs Phenomenology Pt1
- Interpreting Language Model Parameters
- Janus Information Flow Transformers 2025
- Towards a computational phenomenology of mental action: modelling meta-awareness and attentional control with deep parametric active inference
- On biological and artificial consciousness: A case for biological computationalism
Next to ingest
`triaged` → `ingested`
queue empty
Next refs extraction
`ingested` → `refs-extracted`
- The World Inside Neural Networks
- Anima Labs Phenomenology Pt1
- Interpreting Language Model Parameters
- Janus Information Flow Transformers 2025
- Towards a computational phenomenology of mental action: modelling meta-awareness and attentional control with deep parametric active inference
- Emergent Introspective Awareness in Large Language Models
- Covariance-based Sequence Pooling
- On biological and artificial consciousness: A case for biological computationalism
Community detection runs
Hybrid Leiden over Nomic vectors (cosine threshold) + edge boost from typed relations / citations. Wired into research_pipeline.sh --weekly (alongside label_communities.py). Singletons are filtered before write.
| When | Method | Scope | Nodes | Edges | Clusters | Members written | Modularity | Threshold | Boost |
|---|---|---|---|---|---|---|---|---|---|
| 5/19/2026, 11:31 | leiden_hybrid_concepts | concepts | 1500 | 23809 | 75 | 3004 | 0.360 | 0.72 | ×2.0 |
| 5/19/2026, 11:31 | leiden_hybrid_papers | papers | 97 | 3266 | 31 | 78 | 0.020 | 0.70 | ×2.0 |
| 5/19/2026, 2:17: | leiden_hybrid_concepts | concepts | 1500 | 2773 | 548 | 1112 | 0.860 | 0.78 | ×2.0 |
| 5/19/2026, 2:13: | leiden_hybrid_papers | papers | 91 | 2824 | 28 | 74 | 0.023 | 0.70 | ×2.0 |
Cross-corpus bridges
same_concept_as edges from external markdown corpora into corpus.entities. Built by cross_corpus_bridges.py via Nomic cosine ≥ 0.78. Top bridges become essay-candidate hunting ground.
| External corpus | Bridges | Avg cosine | Max cosine | Last built |
|---|---|---|---|---|
aboutblank_kb | 5,838 | 0.816 | 0.933 | 5/19/2026, 11:46 |
alexander | 553 | 0.786 | 0.929 | 5/19/2026, 4:05: |
research_notes | 40 | 0.786 | 0.841 | 5/19/2026, 11:33 |
zen | 6 | 0.758 | 0.768 | 5/19/2026, 11:33 |
Local reservoirs
Folders the pipeline can sweep on demand via ingest_local_folder.py.
synthesis/sources/papers/synthesis/papers/cleaned/alexander/papers/