DriftSift — Diagnostic Intelligence Platform with MCP Server

What Is This

DriftSift is the MCP server interface to a system I stumbled upon called KPacks (Knowledge Packs). A KPack is a structured diagnostic library built from expert source material — dense, unstructured business wisdom (books, seminars, courses) processed through a multi-phase pipeline into standardized, actionable diagnostic modules. Each module ships with 7 asset types: checklists, rubrics, templates, worksheets, AI agents, playbooks, and experiments.

The pipeline that produces these: raw PDF intake → quality screening → “ore mining” (extracting insights) → cataloging → logic chain analysis → asset generation → module assembly. Think of it as refining raw intellectual ore into polished, reusable diagnostic tools.

DriftSift explores whether this system could diagnose real business problems and get business owners unstuck. The platform imports content from 6 kpack repositories (225+ modules across business operations, SaaS growth, leadership, deal-making, and AI operations) via a Ruby parser pipeline. Users connect Claude.ai or Claude Desktop via bearer token authentication and run complete diagnostic conversations — triage, interview, scoring, report, intervention — through natural language.

3,461 commits. 25 completed PRDs.

Why This Approach

The diagnostic format (interview → score → prescribe) is inherently conversational. An MCP server lets Claude act as the diagnostic facilitator — asking questions, processing responses, computing scores, and delivering assets — without building a custom chat UI. The user’s existing Claude subscription becomes the interface.

The TF-IDF router was built in plain Ruby rather than reaching for Elasticsearch or a vector database because the corpus is small (225 modules), the ranking signals are domain-specific (synonym expansion, negative signals), and the iteration cycle needed to be fast. 25 PRDs of refinement would have been painful with an external search dependency.

Key engineering decisions:

• TriageRouter — hand-rolled TF-IDF information retrieval in plain Ruby. A pure-Ruby router that scores free-text user input against 225+ diagnostic modules using TF-IDF/IDF-weighted keyword matching, stopword filtering, synonym expansion, multi-word phrase matching, negative signal suppression, domain diversity enforcement, and specificity-weighted scoring. Iteratively refined through 25 PRDs of production evaluation — each PRD corresponds to a triage precision problem discovered in real user sessions.

• MCP server as authenticated Rack middleware. 26 MCP tools mounted inside the Rails app, authenticated via query-param bearer token. Users sign up, get an API token, connect their Claude client, and run full diagnostics conversationally. This is a novel integration pattern — the AI assistant becomes the diagnostic interface, not just a chatbot.

• Kpack import pipeline. Structured Markdown files across 6 repositories are parsed into relational database records — modules, dimensions, questions, failure-fix mappings, score bands, and assets. The pipeline is idempotent. The schema is consistent: 5 dimensions × 20 points = 100, five named score bands, applied uniformly across all domains.

• Session observability. MCP tool invocations are logged to a database table with PII-safe allowlisting and Sentry breadcrumb enrichment. A ConciergeSnapshotService runs funnel analysis (stalled sessions, completion rates) to identify where users drop off.

What Would Break

• MCP protocol changes (it’s still evolving) would require tool interface updates across all 26 tools.
• The TF-IDF router doesn’t handle semantic similarity — it needs keyword overlap. A user describing their problem in novel language won’t match well.
• The kpack import pipeline is tightly coupled to the Markdown format. Structural changes in source kpacks require parser updates.

What I Learned

• MCP is a powerful integration pattern for domain-specific AI applications, but authentication and session management are still rough edges.
• Hand-rolled search is viable and preferable at small scale when the ranking signals are domain-specific. The key is having a tight evaluation loop — real queries producing bad results, diagnosed and fixed in the same day.
• Consistent schema design across knowledge domains (the 5×20 scoring model) is what made the cross-domain triage work. Without that constraint, the router would need per-domain ranking logic.

---

Repo: github.com/plentyofsaas/driftsift-app (private)

Live: driftsift.com

Status: Production on Spock VPS. Active development. MCP server live.