Never hallucinate
If the answer isn't in the business's own content, the assistant says so and hands off to a human. A hard constraint, not a tunable.
Back to all resources
Unicorn Nexus: Turning a Support Chatbot Into a Knowledge Intelligence Platform
unicorncraftstudios.comOverview
TL;DR
Most "AI chatbot for your website" products optimise for one thing: answering questions. I built Unicorn Nexus to answer a different one - what is your chatbot's inability to answer telling you about your business? Businesses upload their docs, embed an assistant ("Nova") on their site, and Nova answers strictly from what it was given, refusing to guess. That part is table stakes. The bet is the layer on top: a dashboard that mines every answered and unanswered question to show what's missing from the knowledge base - turning a support tool into a knowledge-improvement loop. Shipped solo across 8 phases. 53 logged decisions, 6 migrations, 2 production-shaped bugs caught by testing rather than by users, 1 credential incident handled correctly.
The two problems
Hallucination is a liability. Generic LLM chatbots invent prices, policies and facts with total confidence. For a support widget, a hallucinated refund policy isn't a curiosity - it's a legal exposure. Silence is invisible. Even a well-behaved bot saying "I don't know" fails quietly. Nobody reads a raw query log, so the business never learns that 40 people asked about a shipping policy that doesn't exist in their docs yet.
Role
- • Product
- • Architecture
- • Full-stack
- • Prompt Design
Duration
8 phases - solo build, ongoing
Team Members
Mayuresh Mule
The insight the whole product hangs on
Every unanswered question is a labelled data point about a content gap. Every answered question is a labelled data point about customer intent. So the chat log - the thing every competitor already has and ignores - is the asset. The product doesn't need a second data-collection surface; it needs to read the exhaust of the one it already has. Not "add more AI features," but "make the AI features the business already has legible and actionable."
1. The core answer loop
Where the money is saved and where the intelligence layer gets its data - in one pass, at answer time.
- 1
Question arrives
From the embedded widget on the tenant's own site.
~6 KB widget - 2
Embed + vector search
Searches this tenant's chunks only - isolation enforced by Postgres RLS.
pgvector - 3
Similarity floor
An obvious miss skips the LLM call entirely and costs nothing but a vector search.
Cost cap - 4
Grounded answer
Gemini Flash answers strictly from what was retrieved, or says it doesn't know.
Never guesses - 5
Logged + clustered
Every question is matched to a topic cluster in real time - no separate job.
One engine - 6
Gaps + Intents
The dashboard reads those clusters into two views the owner can act on.
The product
2. Product principles
These were written before the features, and they settled most arguments later.
The chat log is the product
Analytics, gap detection, clustering and drafting are all built by reading data the product already generates - no new instrumentation.
Free-tier-honest architecture
No feature assumes queues, cron, or vector index servers. Every phase works on Supabase and Vercel free tiers at real early-SaaS volumes.
Ship the honest version
A smaller feature with real data beats a bigger feature with placeholder data. This one cost four requested features.
3. Eight phases, solo
From an empty repo to a configurable multi-tenant SaaS.
- Phase 1-3
The core loop
Supabase Auth plus Postgres Row Level Security so tenant isolation is a database guarantee, a workspace per signup, ingestion (paste / PDF / URL → chunk → embed → pgvector), and grounded retrieval with a similarity floor.
- Phase 4
Distribution - the embeddable widget
One script tag, ~6 KB, zero dependencies, Shadow-DOM isolated so it can't break or be broken by the host site's CSS. It derives its own API origin from its script src, so the same file works on every deployment.
- Phase 5
The feedback loop - analytics
Query volume, response time, grouped content gaps, and the headline "I don't know" rate - computed client-side, because at free-tier volumes a server-side aggregation job solves a problem that doesn't exist yet.
- Phase 6
Brand, and building for AI search
A rebrand plus a marketing page written for GEO as well as SEO - structured data and quotable FAQ content, on the thesis that answer engines are now a real discovery surface that rewards different content.
- Phase 7
The strategic pivot - Knowledge Intelligence
An 11-feature wishlist collapsed into one clustering engine and two dashboard views. Detailed below - it's the call I'd most want to be judged on.
- Phase 7.5
UX maturity pass
Functional-but-generic dashboard to a premium responsive SaaS shell with three real breakpoints - done at the design-token and shell level, so ~15 pages inherited it for the cost of two files.
- Phase 7.6
Product identity
Renamed platform (Unicorn Nexus) and assistant (Nova) across every user-visible surface - while deliberately leaving the historical decision log unedited, old product name and all.
- Phase 8
Personalisation and configurability
Company settings and an assistant personality engine built purely as prompt engineering on the existing model, with grounding enforced in the function signature. Settings propagate live to embedded widgets - no re-pasting the snippet.
Click to expand
// The honest failure state is the feature: 'I don't know' plus a human handoff, logged as a gap.
4. The pivot: 11 features into 1 engine
I was handed an 11-feature wishlist and instructions to build "an AI Knowledge Intelligence Platform." Building all eleven would have produced eleven shallow, overlapping systems - several of which need real usage volume or infrastructure that doesn't exist yet, and would have had to be populated with fabricated numbers.
Asked for: 11 features
Wishlist- Knowledge Gap Intelligence, Customer Intent Analytics
- Knowledge Graph, Health Score, Duplicate Detection
- Freshness Tracking, Contradiction Detection
- AI Documentation Writer, Usage Analytics
- Relationship Explorer, Smart Recommendations
Built: 1 engine, 2 views
Shipped- One clustering engine - every question matched to a topic cluster at answer time, no separate job
- Knowledge Gaps - unanswered clusters ranked by volume × priority × recency, with one-click AI documentation drafting
- Customer Intents - every cluster with lazily AI-labelled topics, growth trend, and 👍/👎 satisfaction
- Labelling is milestone-gated, so LLM spend scales with signal, not traffic
Deferred: 4 features
Not built- Health Score - needs real usage data to mean anything
- Duplicate Detection - needs real content volume
- Contradiction Detection - O(n²) LLM comparisons, unaffordable on a free tier
- Relationship Explorer / Graph - sequenced behind the content volume that makes a graph legible
- Each was roadmapped with its reason written down, not silently dropped
Click to expand
// Ranked by a formula the owner can read - unanswered volume × priority × recency decay, not a black box.
Click to expand
// High volume plus low satisfaction is the real prioritisation signal for what to document next.
53
LOGGED DECISIONS
Dated, numbered product and architecture decisions, each with its trade-off, written contemporaneously.
8
SHIPPED PHASES
Auth and multi-tenancy through to per-tenant personalisation - solo, from an empty repo.
11 → 3
SCOPE CALL
An 11-feature wishlist collapsed into one engine and two views; four features deferred with reasons.
~6 KB
EMBED WIDGET
One script tag, zero dependencies, Shadow-DOM isolated, self-configuring from its own src.
5. Decisions worth defending
Seven calls, each with what it cost.
Tenant isolation via Postgres RLS, not app-code checks
Trade-off
More upfront schema design on every table.
Why
Tenant leakage is a security bug, not a UX bug - worth paying for with the database engine's guarantees rather than developer discipline.
Skip the LLM call below a similarity floor
Trade-off
A few borderline "maybe answerable" questions get the fallback.
Why
It caps cost on the highest-volume failure mode - obviously out-of-scope questions - for near-zero UX cost.
11 features → 1 engine, 2 views, 4 deferred
Trade-off
Much less breadth against the requested list.
Why
It avoided shipping fabricated data on features that need real volume first. The highest-leverage call in the project.
Cluster labelling is lazy and milestone-gated
Trade-off
Labels can lag a fast-growing cluster briefly.
Why
LLM spend stays proportional to signal rather than to raw traffic - the difference between a viable and a doomed free tier.
Personalisation is prompt-only, and required on the answer function
Trade-off
A more rigid function signature.
Why
It makes an ungrounded personalised answer structurally impossible instead of a policy someone has to remember.
Widget config is fetched on load, not baked into the snippet
Trade-off
One extra request per widget load.
Why
The literal difference between "settings changes need re-integration" and "settings changes just work" - a retention detail disguised as a technical one.
The rebrand didn't rewrite the historical decision log
Trade-off
The log carries an inconsistent product name across its own history.
Why
An accurate build history is worth more than a cosmetically consistent one - the same discipline a real postmortem needs.
6. When things went wrong
A portfolio that only shows things going right isn't credible. Three incidents, and what each fix actually was.
| Incident | Why it was dangerous | The response |
|---|---|---|
| Silent tracking failure | A migration wasn't applied in one environment, so every chat quietly failed to log. Nothing crashed; the bot worked perfectly while Analytics, Gaps and Intents stayed empty. | Made the failure loud: server-side logging, a tracked flag on the API response, and a /api/diagnostics health check with a dashboard banner. The fix is a monitoring feature, not a patch. |
| CORS bug that only fails in production | A public endpoint allowed only POST, left over from before any GET endpoint existed. It passed every same-origin local test and would have broken the next real cross-origin request. | Caught by testing the actual cross-origin path. "It works when I test it" and "it works" are different claims. |
Committed .env with live keys | A service-role key that bypasses all row-level security, plus an LLM key, sitting in git history. | Flagged immediately rather than quietly fixed, blast radius explained (full data access), key rotation recommended as the only real remedy, and shared git history left for the repo owner to decide on. |
7. What "working" will look like
No live production traffic yet, so instead of invented usage numbers: here's what is already instrumented and how each signal should be read once it has users.
| Signal | How to read it |
|---|---|
| "I don't know" rate | The headline quality metric. Trending down means the knowledge base is closing real gaps, not just growing. |
| Gaps resolved vs ignored | Whether the gap loop drives content work, or just generates a report nobody acts on. |
| Cluster growth × satisfaction | Topics that are both high-volume and poorly served - the actual prioritisation signal. |
tracked: true rate | An internal health metric born from the silent-tracking incident. If it regresses, every metric above becomes unreliable, so it gates trusting the rest. |
8. Roadmap - sequenced, not a backlog dump
Each phase waits on something specific. Nothing is built on top of a metric that doesn't exist yet.
- Phase 9 Planned
Leads
A widget three-action UI (Chat / Callback / WhatsApp) plus a leads pipeline. Needs a real data model with a status workflow, not a settings tweak.
- Phase 10 Planned
Hosted public chat page
Per-workspace page with location-based greetings. Needs subdomain routing and a geo-IP provider decision not yet made.
- Phase 11 Planned
Marketing copy rewrite
Sequenced after the visual identity, so it's written once against the final brand instead of twice.
- Phase 12 Planned
Document relationship pipeline
Duplicates, contradictions, graph - the Phase 7 cuts, now waiting on enough real content volume to make the signal meaningful instead of fabricated.
- Phase 13-15 Planned
Freshness, Health Score, Smart Recommendations
Each depends on an earlier phase's real data existing first.
What I'd do differently
Instrument for the intelligence layer from day one. The clustering engine retrofits meaning onto a query log that Phase 3 wasn't designed to feed it. It worked because the schema was additive-friendly, but designing the log with clustering in mind would have saved a migration. Write the "what we're explicitly not building and why" doc earlier. Collapsing the wishlist was right, but it happened reactively, mid-phase. Doing that scoping exercise as the first step of any large request makes the trade-off visible to a stakeholder before work starts, not after.