DriftSift — Diagnostic Intelligence Platform with MCP Server

What Is This

A Rails 8 application that runs structured business diagnostics against “knowledge packs” (kpacks) — curated diagnostic modules with interview questions, scoring models, failure-to-fix mappings, and prescriptive assets.

The platform imports content from 6 separate kpack repositories (225+ modules across business operations, SaaS growth, leadership, deal-making, and AI operations) via a Ruby parser pipeline.

The engineering centerpiece: a production MCP (Model Context Protocol) server built on the fast_mcp gem with 26 tools covering the full diagnostic workflow. Users connect Claude.ai or Claude Desktop via bearer token authentication and run complete diagnostic conversations — triage, interview, scoring, report, intervention — through natural language.

3,461 commits. 25 completed PRDs.

Why This Approach

The diagnostic format (interview → score → prescribe) is inherently conversational. An MCP server lets Claude act as the diagnostic facilitator — asking questions, processing responses, computing scores, and delivering assets — without building a custom chat UI. The user’s existing Claude subscription becomes the interface.

The TF-IDF router was built in plain Ruby rather than reaching for Elasticsearch or a vector database because the corpus is small (225 modules), the ranking signals are domain-specific (synonym expansion, negative signals), and the iteration cycle needed to be fast. 25 PRDs of refinement would have been painful with an external search dependency.

Key engineering decisions:

• TriageRouter — hand-rolled TF-IDF information retrieval in plain Ruby. A pure-Ruby router that scores free-text user input against 225+ diagnostic modules using TF-IDF/IDF-weighted keyword matching, stopword filtering, synonym expansion, multi-word phrase matching, negative signal suppression, domain diversity enforcement, and specificity-weighted scoring. Iteratively refined through 25 PRDs of production evaluation — each PRD corresponds to a triage precision problem discovered in real user sessions.

• MCP server as authenticated Rack middleware. 26 MCP tools mounted inside the Rails app, authenticated via query-param bearer token. Users sign up, get an API token, connect their Claude client, and run full diagnostics conversationally. This is a novel integration pattern — the AI assistant becomes the diagnostic interface, not just a chatbot.

• Kpack import pipeline. Structured Markdown files across 6 repositories are parsed into relational database records — modules, dimensions, questions, failure-fix mappings, score bands, and assets. The pipeline is idempotent. The schema is consistent: 5 dimensions × 20 points = 100, five named score bands, applied uniformly across all domains.

• Session observability. MCP tool invocations are logged to a database table with PII-safe allowlisting and Sentry breadcrumb enrichment. A ConciergeSnapshotService runs funnel analysis (stalled sessions, completion rates) to identify where users drop off.

What Would Break

• MCP protocol changes (it’s still evolving) would require tool interface updates across all 26 tools.
• The TF-IDF router doesn’t handle semantic similarity — it needs keyword overlap. A user describing their problem in novel language won’t match well.
• The kpack import pipeline is tightly coupled to the Markdown format. Structural changes in source kpacks require parser updates.

What I Learned

• MCP is a powerful integration pattern for domain-specific AI applications, but authentication and session management are still rough edges.
• Hand-rolled search is viable and preferable at small scale when the ranking signals are domain-specific. The key is having a tight evaluation loop — real queries producing bad results, diagnosed and fixed in the same day.
• Consistent schema design across knowledge domains (the 5×20 scoring model) is what made the cross-domain triage work. Without that constraint, the router would need per-domain ranking logic.

---

Repo: github.com/plentyofsaas/driftsift-app (private)

Status: Production on Spock VPS. Active development. MCP server live.