Debo — Autonomous Personal AI Memory & Hybrid RAG Pipeline
Building a multimodal memory engine that embeds notes, browser clips, audio recordings, and conversations for sub-second semantic retrieval and context-aware conversational AI.
Hybrid RAG
BM25 + Cosine Vectors
< 150ms
Retrieval Latency
Multimodal
Voice, Web, Docs & Notes
Solo Build
Full-Stack & AI Ingestion
1. The Problem: Human Knowledge is Fragmented
Knowledge workers capture dozens of knowledge fragments daily — voice memos, bookmarks, markdown notes, code snippets, and research PDFs. Standard search engines require exact keyword recall, while conventional LLMs suffer from context window limits and total lack of personal episodic memory.
Debo was engineered as a second brain operating system: continuously chunking, embedding, indexing, and synthesizing semantic context during natural conversation.
2. Multimodal Ingestion & Chunking Architecture
// MULTIMODAL RAG PIPELINE
┌───────────────────────────────────────┐
│ Input: Voice / Notes / Web / Markdown │
└───────────────────────────────────────┘
│
▼
┌───────────────────────────────────────┐
│ Recursive Semantic Boundary Chunking │ ──► [Chunk Overlap: 15%]
└───────────────────────────────────────┘
│
▼
┌───────────────────────────────────────┐
│ Embedding: text-embedding-3 (1536 dim)│
└───────────────────────────────────────┘
│
▼
┌───────────────────────────────────────┐
│ Hybrid Store: Vectorize + PostgreSQL │
└───────────────────────────────────────┘
│
[Hybrid Search: BM25 + Cosine RRF]
│
▼
┌───────────────────────────────────────┐
│ Synthesized Prompt Context to Agent │
└───────────────────────────────────────┘- Semantic Chunking: Rather than arbitrary token slicing, chunks are split at paragraph, markdown heading, and speech pause boundaries to preserve atomic meaning.
- Reciprocal Rank Fusion (RRF): Combines BM25 lexical keyword matching with Cosine semantic similarity to prevent query drift and ensure exact keyword matches are never lost.
3. Eliminating Hallucinations & Latency
Every response synthesized by the conversational agent includes precise citation anchors referencing the exact source note, timestamp, and chunk ID.
Retrieval ranking automatically scales scores based on document recency, prioritizing recent meetings and notes while allowing explicit chronological queries.
4. Engineering Takeaways
Hybrid Retrieval is non-negotiable for personal memory: Vector embeddings alone struggle with acronyms, unique identifiers, and names. Implementing dual-index RRF ranking alongside semantic chunking transformed Debo into an ultra-reliable, production-grade second brain.