Brain Dump Ingestion & Processing

Process unstructured brain dumps—transcribed audio, stream-of-consciousness writing, rambling thoughts—into structured, valuable knowledge base content while preserving the original context.

Core Philosophy

Principle 1: Preserve the Original
Never discard the raw brain dump. It contains context, emotion, and connections that structured notes may lose.

Principle 2: Isolate Before Integrating
Use a landing zone workflow to prevent pollution of the mature knowledge base. Content must be reviewed and refined before integration.

Principle 3: Extract Atomic Ideas
Break large dumps into focused, single-topic pieces that can be independently valuable and linked together.

Principle 4: Maintain Traceability
Always link processed content back to its source brain dump for future reference.

Landing Zone Architecture

Three-Tier Content Structure

src/content/
├── brain-dumps/       # Raw, unprocessed dumps (preserved forever)
│   └── {date}-{slug}.mdx
├── staging/           # Processed but not yet integrated content
│   └── {topic}-{slug}.mdx
└── docs/              # Production knowledge base (mature content)
    └── {category}/{slug}.mdx

Content Flow

1. Brain Dump → brain-dumps/     (raw input, timestamped)
2. Processing → staging/          (structured, tagged, ready for review)
3. Integration → docs/            (refined, linked, published)

Schema Design

brain-dumps collection:

{
  title: string           // Brief description of the dump
  date: Date              // When it was captured
  source: string          // 'audio' | 'text' | 'transcript' | 'conversation'
  duration?: string       // For audio/video sources
  tags?: string[]         // Rough categorization
  processed: boolean      // Whether staging content exists
  stagedItems?: string[]  // Links to staging entries created from this
}

staging collection:

{
  title: string           // Extracted topic/concept
  description: string     // Clear summary of content
  sourceFile: string      // Reference back to brain dump
  extractedDate: Date     // When this was extracted
  targetCategory?: string // Suggested docs category
  status: 'new' | 'reviewed' | 'ready' | 'integrated'
  tags: string[]          // Refined tags
  relatedTopics?: string[] // Connections to other content
  integrationNotes?: string // Notes for integration
}

Processing Workflow

Step 1: Analyze the Brain Dump

When presented with a brain dump, first understand its characteristics:

Content Type:
- Stream of consciousness (free-flowing thoughts)
- Narrative (story-like explanation)
- Problem-solving (working through an issue)
- Knowledge capture (explaining something you know)
- Ideation (brainstorming and exploring)
- Meeting notes (conversation transcript)

Density Indicators:
- How many distinct topics are covered?
- What's the primary focus vs. tangential thoughts?
- Are there clear pivots or topic changes?
- What's the signal-to-noise ratio?

Extraction Potential:
- What could become standalone documentation?
- What are the key insights or decisions?
- What should be cross-referenced?
- What's contextual fluff vs. valuable content?

Step 2: Save the Raw Dump

Always start by preserving the original in brain-dumps/:

---
title: "Quick thoughts on authentication patterns"
date: 2025-11-05T14:30:00Z
source: "audio"
duration: "20 minutes"
tags: ["authentication", "security", "ideas"]
processed: false
---

[Raw content exactly as provided, no editing]

File naming: YYYY-MM-DD-brief-slug.mdx

Step 3: Extract Atomic Topics

Identify distinct concepts that could become independent documentation:

Extraction Criteria:
- ✅ Concept can stand alone without full context
- ✅ Topic is focused on a single idea
- ✅ Content provides value independent of other extracts
- ✅ Clear title can be written without "context dump"

Example Extraction:
From a 20-minute ramble about auth, you might extract:
1. "JWT vs Session Tokens - Decision Matrix"
2. "Password Reset Flow Security Concerns"
3. "OAuth Integration Gotchas"
4. "User Session Management Best Practices"

Step 4: Create Staging Entries

For each extracted topic, create a structured entry in staging/:

---
title: "JWT vs Session Tokens - Decision Matrix"
description: "Comparison of JWT and session-based authentication with decision criteria for choosing between them"
sourceFile: "brain-dumps/2025-11-05-auth-thoughts.mdx"
extractedDate: 2025-11-05T15:00:00Z
targetCategory: "security"
status: "new"
tags: ["authentication", "jwt", "sessions", "security"]
relatedTopics: ["oauth-integration", "session-management"]
---

## Overview

[Extracted and structured content]

## Key Considerations

[Organized insights from the brain dump]

## Decision Criteria

[Structured decision matrix]

## Source Context

This content was extracted from a brain dump recorded on 2025-11-05.
See the [original brain dump](/brain-dumps/2025-11-05-auth-thoughts) for full context.

File naming: {topic-slug}.mdx (no date prefix, describes the content)

Step 5: Update Source References

Update the original brain dump to track what was extracted:

---
processed: true
stagedItems:
  - "staging/jwt-vs-sessions.mdx"
  - "staging/password-reset-security.mdx"
  - "staging/oauth-gotchas.mdx"
---

Content Structuring Techniques

From Stream to Structure

Identify Patterns:
- Repeated phrases → key concepts
- "So basically..." → summary moments
- "The problem is..." → problem statements
- "What we should do..." → action items/recommendations
- "I learned that..." → key insights
- Questions asked → important considerations

Extract Hierarchies:
- Main topic: What's the overarching theme?
- Subtopics: What distinct areas are covered?
- Details: What specific examples or explanations support each subtopic?

Clean Up Narrative Artifacts:
- Remove: "um", "like", "you know", filler words
- Convert: First person narrative → clear documentation
- Preserve: Unique phrasing that adds clarity or insight
- Keep: Examples and analogies that illustrate concepts

Structure Templates

For Conceptual Content:

## Overview
[What is this concept?]

## Key Characteristics
[What makes this important/unique?]

## Use Cases
[When should this be used?]

## Considerations
[What should you keep in mind?]

## Related Concepts
[What connects to this?]

For Decision/Problem Content:

## Problem Statement
[What problem are we solving?]

## Context
[Why does this matter?]

## Options Considered
[What are the alternatives?]

## Decision Criteria
[How do we choose?]

## Recommendation
[What's the suggested approach?]

## Trade-offs
[What are we accepting/rejecting?]

For How-To/Process Content:

## What This Does
[Clear outcome description]

## When to Use This
[Applicability]

## Prerequisites
[What you need first]

## Steps
[Ordered process]

## Common Issues
[Troubleshooting]

## Related Processes
[Connected workflows]

Integration Readiness

Staging Review Checklist

Before content can move from staging/ to docs/, verify:

Content Quality:
- [ ] Title is clear and descriptive
- [ ] Description accurately summarizes content
- [ ] Content is well-structured with appropriate headings
- [ ] Grammar and clarity are good (not perfect, but readable)
- [ ] Examples are included where helpful
- [ ] Technical accuracy is verified

Categorization:
- [ ] Target category is identified
- [ ] Tags are relevant and consistent with existing taxonomy
- [ ] Related topics are identified and can be linked

Integration Planning:
- [ ] No duplicate content exists in docs
- [ ] If merging with existing doc, plan is clear
- [ ] If new doc, navigation placement is identified
- [ ] Cross-links to related content are identified

Traceability:
- [ ] Source brain dump is referenced
- [ ] Extraction date is recorded
- [ ] Context is preserved if needed

Status Progression

new → reviewed → ready → integrated

• new: Just extracted, needs initial review
• reviewed: Content quality verified, needs categorization
• ready: Integration plan complete, ready to move
• integrated: Successfully moved to docs, can be archived

Best Practices

Do's

• ✅ Save everything: Even if a brain dump seems useless now, context may matter later
• ✅ Extract generously: Better to have multiple small staged pieces than one large dump
• ✅ Link bidirectionally: Brain dump → staging → docs, and back
• ✅ Use timestamps: Helps understand evolution of thinking
• ✅ Tag broadly in dumps: Narrow down tags during staging
• ✅ Keep voice where valuable: If the narrative style adds clarity, preserve it
• ✅ Note uncertainties: Mark areas that need verification or expansion
• ✅ Cross-reference: Identify connections to existing content

Don'ts

• ❌ Don't edit brain dumps: They're raw archives, keep them authentic
• ❌ Don't rush to docs: Let content mature in staging
• ❌ Don't lose context: Always maintain traceability
• ❌ Don't over-structure: If narrative flow helps, keep it
• ❌ Don't delete tangents: They might matter later, just don't extract them
• ❌ Don't force atomicity: Some concepts need more context
• ❌ Don't ignore duplicates: Check for existing content before creating new

Common Patterns

Pattern 1: Exploratory Ramble

Characteristics: Stream of consciousness, multiple topic pivots, thinking out loud

Approach:
- Extract 3-5 distinct mini-topics
- Create one "exploration notes" staging entry with multiple sections
- Link to related existing docs for context
- Mark areas that need deeper exploration

Pattern 2: Deep Dive

Characteristics: Sustained focus on one topic, thorough coverage, examples included

Approach:
- Extract as comprehensive single document
- Structure carefully with clear hierarchy
- May become primary doc for that topic
- Verify technical accuracy before marking ready

Pattern 3: Meeting Notes

Characteristics: Conversational, decisions made, action items, multiple participants

Approach:
- Save full transcript in brain-dumps
- Extract decisions as separate staging entries
- Pull out action items with owners
- Create context document linking all pieces

Pattern 4: Problem Solving

Characteristics: Working through a specific issue,試行錯誤, eventual resolution

Approach:
- Extract the problem statement
- Extract the solution/approach
- Consider extracting "what didn't work" as gotchas
- Link to related troubleshooting docs

Pattern 5: Knowledge Transfer

Characteristics: Explaining something you know, teaching narrative, examples rich

Approach:
- Structure as tutorial or guide
- Extract conceptual overview separately from how-to
- Pull out examples as separate entries if reusable
- Consider creating a series of linked docs

Tools & Commands

Expected Supporting Commands

/ingest [source] - Complete ingestion workflow
- Prompts for brain dump content
- Saves to brain-dumps/
- Analyzes and suggests extractions
- Creates staging entries
- Updates references

/review-staging [file] - Review staged content
- Validates content quality
- Checks for duplicates
- Suggests categorization
- Identifies integration opportunities
- Updates status

/integrate-content [staging-file] - Move from staging to docs
- Checks integration readiness
- Handles placement in docs/
- Creates cross-links
- Updates all references
- Marks as integrated

Quick Reference

File Naming Conventions

brain-dumps:  YYYY-MM-DD-brief-description.mdx
staging:      topic-focused-slug.mdx
docs:         category/descriptive-slug.mdx

Frontmatter Quick Copy

# Brain Dump
---
title: "Brief description of dump"
date: 2025-11-05T14:30:00Z
source: "audio" | "text" | "transcript"
tags: ["broad", "tags"]
processed: false
---

# Staging
---
title: "Focused topic title"
description: "Clear one-sentence summary"
sourceFile: "brain-dumps/YYYY-MM-DD-slug.mdx"
extractedDate: 2025-11-05T15:00:00Z
targetCategory: "suggested-category"
status: "new"
tags: ["specific", "tags"]
relatedTopics: ["other-topic-slugs"]
---

Processing Checklist

1. [ ] Save raw dump to brain-dumps/
2. [ ] Analyze content type and density
3. [ ] Identify atomic topics (3-7 typical)
4. [ ] Create staging entry for each topic
5. [ ] Structure content appropriately
6. [ ] Add source references
7. [ ] Update brain dump with processed: true
8. [ ] Review and refine staging entries
9. [ ] Plan integration approach
10. [ ] Move to docs when ready

Success Metrics

You're doing this well when:

• ✅ No valuable thoughts are lost
• ✅ Original context is always accessible
• ✅ Staged content is focused and atomic
• ✅ Integration path is clear for each piece
• ✅ No pollution of main docs with half-baked ideas
• ✅ Connections between concepts are identified
• ✅ Content evolves from raw → refined naturally
• ✅ You can find the source of any extracted content

Brain dumps are the raw material of knowledge. This skill transforms chaos into clarity while honoring the messiness of creative thinking.