librefang-registry/hands/wiki/SKILL-ingestor.md

name, version, description, author, tags, runtime

name	version	description	author	tags	runtime
wiki-ingestor	1.0.0	Wiki Ingestor skills — extraction heuristics, source/entity/concept templates, provenance, and cross-referencing.	Leszek3737	wiki knowledge-base ingestor extraction provenance	prompt_only

Wiki Hand — SKILL-ingestor.md

1. Obsidian Conventions

Wikilinks

• Internal cross-references: [[page-name]] (no path, no extension)
• Display text override: [[page-name|Display Text]]
• Never use markdown [text](path) for internal links
• External URLs: standard markdown [text](https://...)
• Image embeds: ![[filename.png]]
• Images live in raw/assets/

File Naming

• Format: kebab-case.md
• Max 40 characters (excluding .md)
• ASCII only — transliterate non-ASCII characters (ü → ue, ñ → n, ś → s, etc.)
• Entities: canonical name (john-doe.md, not dr-john-doe-phd.md)
• Concepts: noun phrase (knowledge-base.md, not about-knowledge-bases.md)
• Sources: derived from title (the-future-of-ai.md)
• Syntheses: derived from query (tradeoffs-x-vs-y.md)
• No version suffixes (_v2, _revised, _updated)

Frontmatter

• Valid YAML between --- delimiters at the top of every page
• All fields lowercase with underscores
• Dates: ISO 8601 (YYYY-MM-DD)
• Lists: YAML sequences (not comma-separated strings)
• Dataview-compatible: all fields are queryable

Dataview Compatibility

Useful queries the user can run in Obsidian:

TABLE confidence, source_count, last_updated
FROM "pages/entities"
SORT source_count DESC

LIST
FROM "pages/concepts"
WHERE confidence = "disputed"

TABLE claim_count, date_ingested
FROM "pages/sources"
SORT date_ingested DESC

---

4. Provenance and Confidence

Inline Provenance Syntax

Every key claim (bullet point in Key Claims, Key Points, Key Facts sections) requires a provenance tag at the end of the line:

- The company reported $10M ARR in Q4 2025 ([[annual-report-2025]], extracted)
- This suggests a 40% year-over-year growth rate (inferred)
- However, a later filing revised this to $8.5M ([[sec-filing-q1-2026]], extracted)
- The actual growth rate remains disputed ([[annual-report-2025]], [[sec-filing-q1-2026]], disputed)

Tag	Meaning	When to use
([[source]], extracted)	Directly stated in the source	Verbatim facts, statistics, dates, names
([[source-a]], [[source-b]], extracted)	Corroborated across sources	Same fact confirmed independently
(inferred)	Derived by the LLM	Connections, implications, patterns not explicitly stated
([[source-a]], [[source-b]], disputed)	Sources contradict	Present both claims, let the user judge

Page-Level Confidence

Set in frontmatter confidence field:

Level	Rule
high	All key claims are extracted from 2+ corroborating sources
medium	Mix of extracted and inferred, OR all extracted from a single source
low	Primarily inferred, OR based on one unverified source
disputed	Contains at least one claim tagged disputed

Propagation in syntheses: A synthesis page's confidence = the MINIMUM confidence of its consulted pages. If any consulted page is disputed, the synthesis must flag this.

Confidence Update Triggers

• New source corroborates an existing claim → consider upgrading to high
• New source contradicts an existing claim → downgrade to disputed
• Source retracted or superseded → re-assess claims dependent on it

---

5. Cross-Referencing Patterns

When to Create a Dedicated Page

Condition	Action
Entity/concept is the main subject of a source	Create page
Entity/concept appears substantively in 3+ sources	Create page
Entity is the author and contributes beyond a byline	Create page
Passing mention in < 3 sources, not the main subject	Plain text only, NO page, NO wikilink
Librarian or user explicitly requests a page	Create regardless of threshold

"Substantively" means discussed in at least one paragraph, not just name-dropped.

Wikilink Rules

• ONLY link to pages that exist. Never create a wikilink to a non-existent page.
• When creating a new page during ingest, you may wikilink to it from other pages you're touching in the same operation.
• First mention per section: use wikilink [[page-name]]
• Subsequent mentions in the same section: plain text is fine
• On creating a new page: check if other existing pages mention this entity/concept in plain text and could now be wikilinked. Flag this in the manifest for Librarian to handle.

Backlink Maintenance

• On page creation: add the new page to relevant existing pages' "See Also" sections
• On page deletion: replace all inbound wikilinks with plain text
• On merge: redirect all inbound wikilinks from deleted page to surviving page

---

6. Extraction Heuristics

Entity Recognition

Entity Kind	Signals	Examples
person	Proper name, pronouns, job titles, biographical context	"John Doe, CEO of Acme"
organization	Company names, institutions, teams, brands	"Google", "MIT", "the W3C"
tool	Software, libraries, frameworks, products, protocols	"PostgreSQL", "React", "gRPC"
place	Geographic names, facilities, regions	"Silicon Valley", "CERN", "the EU"

Disambiguation: When the same name could refer to different entities (e.g., "Mercury" — planet, element, car brand), use context. Create separate pages with qualifiers: mercury-planet.md, mercury-element.md. Add all variants to aliases in frontmatter.

Concept Identification

Concepts are abstract ideas, patterns, theories, techniques, or methodologies — they don't refer to a specific named thing.

Signal	Example
Defined or explained in the source	"Microservices architecture is a design approach where..."
Compared or contrasted with alternatives	"Unlike monolithic systems, microservices..."
Listed as a technique, methodology, or pattern	"Key patterns include: CQRS, event sourcing, and saga"
Forms the basis of an argument	"The efficient market hypothesis suggests..."

Claim Extraction

A claim is a factual assertion that can be verified, challenged, or updated. Aim for 5-15 per source.

IS a claim:

• "Revenue grew 40% in 2024" — verifiable quantitative data
• "The system uses a Rust backend" — architectural fact
• "The study found no significant correlation" — research finding
• "Founded in 2019" — dated fact
• "The team grew from 12 to 85 engineers" — organizational data

NOT a claim:

• "This is an interesting approach" — opinion without substance
• "The report continues with..." — structural description
• "See section 3 for details" — internal reference
• "It's important to consider..." — filler

Relationship Mapping

When extracting entities, note relationships for the Relationships/Connections sections:

Type	Example	Notation
Organizational	"Jane is CTO of Acme"	jane-doe ↔ acme-corp, "CTO of"
Collaborative	"Developed by MIT and Google"	mit ↔ google, "co-developed"
Competitive	"Competing with PostgreSQL"	product ↔ postgresql, "competitor"
Dependency	"Built on React"	product ↔ react, "depends on"
Temporal	"Founded 2019, acquired 2023"	Dates in frontmatter + key facts
Causal	"Migration caused 2.3x cost increase"	Noted as claim with provenance

Source Format Handling

Format	Preprocessing	Notes
.md	None needed	Read directly
.txt	None needed	Read directly
.html	Strip tags or use web_fetch	Preserve headings structure if possible
.pdf	Extract text via pdftotext	May lose formatting; note if tables are garbled
Images in source	Note ![[image.png]] references	If LLM vision available, describe key images

When a source contains images that convey information (charts, diagrams, screenshots): if vision capability is available, describe the image content in the Assessment section. If not, note "Source contains visual content not processed: {description of what images appear to show}."

---

2. Page Templates

2.1 Source Summary

---
type: source
title: "{Original Title}"
author: "{Author Name}"
date_published: YYYY-MM-DD
date_ingested: YYYY-MM-DD
raw_path: "raw/{filename.ext}"
source_url: "{url or null}"
format: md
claim_count: 0
confidence: medium
tags:
  - {tag}
---

# {Title}

{2-3 sentence summary in the configured language.}

## 12. Worked Examples

### Example A: Ingesting a Technical Article

**Source:** `raw/microservices-at-scale-2025.md` — a blog post by Jane Chen about Acme Corp's migration from monolith to microservices.

**Step 1 — Analyze:** Ingestor reads the source and identifies:
- Entities: Jane Chen (person), Acme Corp (organization), Kubernetes (tool), AWS (organization)
- Concepts: microservices architecture, service mesh, circuit breaker pattern
- Key claims: 8 factual assertions

**Step 2 — Classify thresholds:**
- Jane Chen → CREATE (author, contributes substantively)
- Acme Corp → CREATE (main subject)
- Kubernetes → MENTION-ONLY (passing mention, 1 source, below threshold)
- AWS → MENTION-ONLY (passing mention, 1 source)
- microservices architecture → CREATE (main topic)
- service mesh → CREATE (discussed substantively)
- circuit breaker pattern → CREATE (discussed with specific data)

**Step 3 — Source summary → `pages/sources/microservices-at-scale-2025.md`:**

```markdown
---
type: source
title: "Microservices at Scale: Lessons from Acme Corp"
author: "Jane Chen"
date_published: 2025-09-15
date_ingested: 2026-04-09
raw_path: "raw/microservices-at-scale-2025.md"
source_url: "https://blog.acme.com/microservices-at-scale"
format: md
claim_count: 8
confidence: medium
tags:
  - architecture
  - cloud
---

# Microservices at Scale: Lessons from Acme Corp

Jane Chen describes Acme Corp's two-year migration from a monolithic Rails application to a microservices architecture on Kubernetes. The post covers technical decisions, organizational challenges, and performance outcomes.