skills/llm-wiki/phases/ingest.md

Phase: Ingest — Process a New Source

Run when user says "ingest this", "add this article", "process raw/", "ingest https://...".

Step 1 — Get the source

If URL provided:

firecrawl scrape "<url>" -o raw/<slug>.md

Slug = domain + path slugified. Example: ai-2027.com-main.md

If file already in raw/: proceed directly.

If text pasted: write it to raw/<slug>.md first.

Step 2 — Read existing index

# Check what's already in the wiki
Read wiki/index.md

Scan index for pages that might be related to this source. Note their paths.

Step 3 — Read the source

Read raw/<slug>.md

For large files (>200 lines), read in chunks. Use grep to find key sections first.

Step 4 — Discuss (optional but recommended)

For important sources, briefly discuss with the user:

• 2-3 most important takeaways
• Any surprises or contradictions with existing wiki content
• Which existing pages are most affected

Skip this for batch ingestion.

Step 5 — Write summary page

Write wiki/sources/<slug>.md using this structure:

---
title: "Source Title"
date: YYYY-MM-DD
type: source
tags: [tag1, tag2, tag3]
source-url: https://...
source-type: article | paper | video | transcript | book | podcast
ingested: YYYY-MM-DD
---

# Source Title

> One-sentence summary of the core claim or finding.

## Key Points

- Most important point, specific and concrete
- Second key point
- Third key point
(3-7 points, each a complete thought)

## Details

Expanded explanation of the most important ideas. 2-4 paragraphs.
Write as synthesis, not transcript. Future-you should be able to read
this instead of the original source.

## Key Quotes

> "The most memorable direct quote from the source."

## Implications

What this means for the field / for your work / for related topics.

## Questions Raised

- Open question this source raises
- Another unresolved question

## Related

- [[Concept Page Name]] — how it relates
- [[Entity Page Name]] — connection
- [[Another Source]] — agrees/disagrees because X

Step 6 — Update entity and concept pages

For each person, organization, concept, or project prominently mentioned:

If page exists: append new information under a dated section:

## [YYYY-MM-DD] New Finding from [[Source Title]]
- New data point or updated claim

If page doesn't exist: create it:

Entity page (wiki/entities/<slug>.md):

---
title: "Entity Name"
type: entity
entity-type: person | org | product | place
tags: []
first-mentioned: YYYY-MM-DD
source-count: 1
---

# Entity Name

> One-sentence description.

## Background
...

## Key Contributions / Role
...

## Appearances in Wiki
- [[Source where first mentioned]] — context

Concept page (wiki/concepts/<slug>.md):

---
title: "Concept Name"
type: concept
tags: []
first-mentioned: YYYY-MM-DD
source-count: 1
---

# Concept Name

> Definition in one sentence.

## Explanation
...

## How It Works
...

## Examples
...

## Connections
- [[Related Concept]] — relationship
- [[Opposing Concept]] — how they differ

## Sources
- [[Source Title]] — what it says about this concept

Limit: touch at most 15 pages per ingest to keep sessions focused.

Step 7 — Update wiki/index.md

Add a row in the appropriate section:

| [[sources/source-slug\|Source Title]] | YYYY-MM-DD | source | One-line summary |

Step 8 — Update wiki/overview.md

If the source significantly changes the high-level picture, revise the relevant section of wiki/overview.md. Add a note: > Updated YYYY-MM-DD after ingesting [[Source Title]]

Step 9 — Append to log

## [YYYY-MM-DD] ingest | Source Title

- Source: `raw/<slug>.md`
- Summary page: `wiki/sources/<slug>.md`
- Pages updated: [[Page A]], [[Page B]], [[Page C]]
- New pages created: [[New Entity]], [[New Concept]]
- Key takeaway: One sentence.

Quality Checklist

• Summary page written at wiki/sources/<slug>.md
• At least 2 existing pages updated with new information
• New entity/concept pages created for major new subjects
• [[WikiLinks]] used throughout (Obsidian-compatible)
• wiki/index.md updated
• wiki/log.md appended
• No raw source files modified