Build and Maintain an AI Knowledge Base

The WhatsApp and SMS AI Agent and AI agent rollout pipeline guides both assume a knowledge base already exists. This guide builds one from scratch: create the container, load documents, attach it to an agent, tune retrieval, and keep the content fresh. It also covers the dashboard companion page — Agents → Knowledge base — which exposes the same surface without code. The endpoint paths below are relative. Send them against https://api.orbit.devotel.io/api/v1.

What an Orbit knowledge base is

A knowledge base is a named collection of documents your AI agent can search when it needs grounded answers. It is the retrieval backbone behind the search_knowledge agent tool: on each turn, the agent’s runtime embeds the customer’s question, runs a similarity search over the chunks of every document chunk in the attached bases, and hands the top matches to the model as context. Without a knowledge base, the agent answers from its system prompt alone — which is fine for tone and routing logic, but wrong for anything factual that changes (refund policy, hours, account-specific procedures). Each knowledge base is workspace-scoped and tenant-owned: it belongs to your organization, any agent in the same organization can be attached to it, and no other tenant can see it. Permissions are the standard API scopes — knowledge:read for reads, knowledge:write (owner / admin / developer roles) for creates, uploads, and deletes.

1. Create the knowledge base

curl -X POST https://api.orbit.devotel.io/api/v1/knowledge-bases \
  -H "X-API-Key: dv_live_sk_..." \
  -H "Content-Type: application/json" \
  -d '{
    "name": "Support Docs",
    "description": "Refund, shipping, and account help articles"
  }'

The response includes the base’s id (e.g. kb_abc123) — keep it for every subsequent call. Optional fields:

description — a free-text label visible in the dashboard; no effect on retrieval.
refresh_schedule — one of manual (default, existing behavior), hourly, daily, or weekly. When set, external URL/RSS/Notion documents you load re-fetch on that cadence; manual leaves them scrape-once. See Step 6 below.
config — free-form metadata map, surfaced back on reads; operational notes belong here (e.g. a JIRA epic key, a team name).

2. Attach the base to an agent

Attach by updating the agent’s knowledge_base_ids array. The array is the contract: the agent’s search_knowledge tool can only search bases in this list, so the allowlist can never be bypassed by a maliciously or accidentally broad prompt.

curl -X PUT https://api.orbit.devotel.io/api/v1/agents/agent_abc123 \
  -H "X-API-Key: dv_live_sk_..." \
  -H "Content-Type: application/json" \
  -d '{ "knowledge_base_ids": ["kb_abc123"] }'

Multiple bases are the normal case — a support agent commonly reads from a Support Docs base plus a Billing Policies base, and the match quality from the strongest chunk wins regardless of which base it came from.

3. Load documents

Two upload paths, one endpoint: Text content (JSON body). Post the content directly; the base chunks and embeds it.

curl -X POST https://api.orbit.devotel.io/api/v1/knowledge-bases/kb_abc123/documents \
  -H "X-API-Key: dv_live_sk_..." \
  -H "Content-Type: application/json" \
  -d '{
    "name": "Refund Policy",
    "type": "markdown",
    "content": "# Refund Policy\n\nCustomers may request a refund within 30 days…"
  }'

Files (multipart form). Upload PDFs, Word documents, CSVs, images, and text formats; the content type is inferred from the filename extension if you omit the type field.

curl -X POST https://api.orbit.devotel.io/api/v1/knowledge-bases/kb_abc123/documents \
  -H "X-API-Key: dv_live_sk_..." \
  -F "file=@refund-policy.pdf" \
  -F "name=Refund Policy" \
  -F "type=pdf"

Supported type values: text, txt, markdown, md, html, pdf, csv, docx, json, py, xml, yaml, yml, js, ts, tsx, jsx, rtf, log, png, jpg, jpeg, gif, webp, image. Chunking. PDF/DOCX are extracted to plain text first, then split into overlapping chunks (1,000 characters per chunk, 150 characters of overlap to preserve context across the boundary). Image types (png, jpg, gif, webp, image) run through a vision OCR pass that transcribes the text before chunking. The chunk is the retrieval unit — a query matches one specific chunk, so tight topical documents retrieve better than one 300-page everything-PDF. Once loaded, manage the document lifecycle per KB:

GET /knowledge-bases/:id/documents — list every document with size and chunk count.
POST /knowledge-bases/:id/documents/:docId/retry — re-run an upload that failed partway (a stalled OCR pass, an interrupted extraction); the retry re-chunks the same source bytes.
DELETE /knowledge-bases/:id/documents/:docId — remove the document and its chunks from retrieval.
Versioning + moderation (GET /knowledge-bases/:id/documents/:docId/versions, POST .../rollback, POST .../approve, POST .../reject) — review and history for the draft→approved document lifecycle.

4. Tune retrieval

Retrieval tuning is where most “the agent ignores the KB” complaints end up. Tune the knobs, then re-benchmark — the deploy-time benchmark is the objective way to know a knob actually helped. Read the current config. A base that has never been tuned returns the neutral default (every knob a no-op, identical to the legacy raw-similarity ordering).

curl https://api.orbit.devotel.io/api/v1/knowledge-bases/kb_abc123/search-config \
  -H "X-API-Key: dv_live_sk_..."

Set a knob. Every field has a no-op default, so a PUT of {} cleanly resets to the legacy ordering and omitting a knob never silently changes that knob.

curl -X PUT https://api.orbit.devotel.io/api/v1/knowledge-bases/kb_abc123/search-config \
  -H "X-API-Key: dv_live_sk_..." \
  -H "Content-Type: application/json" \
  -d '{
    "min_score": 0.5,
    "recency_boost": 1.5,
    "recency_half_life_days": 30,
    "source_weights": { "Zendesk": 1.2, "Confluence": 0.8 },
    "include_categories": [],
    "exclude_categories": ["archive"]
  }'

What each knob does:

min_score (0–1, default 0) — drop chunks whose raw similarity score falls below the floor. Raise it when the agent retrieves drift-y near-misses and hallucinates them together.
recency_boost (0–5, default 0) — multiplier strength of the recency bump; fresher documents rank up to ×(1+boost).
recency_half_life_days (1–3650, default 30) — how quickly the recency boost decays.
source_weights (label → 0–20 multiplier) — boost or damp whole sources (e.g. boost the curated Zendesk articles, damp an old Confluence dump).
include_categories / exclude_categories — allow-list or deny-list of categories, matched against the document’s category metadata.

Detect drift with the benchmark. Retrieval drift shows up as a quality regression on the same golden query set even though nothing in your prompt changed — usually means fresh content hasn’t been added, stale content is outranking fresh, or the boost knobs are fighting each other. Run the benchmark, pin a baseline, re-run after every knob change:

POST /agents/:id/retrieval-benchmark/runs — score the agent’s recall on your pinned query set.
GET /agents/:id/retrieval-benchmark — read the latest persisted run.
POST /agents/:id/retrieval-benchmark/baseline — pin a run as the baseline for comparison.

The full rollout pipeline walks through how to wire this into a deploy: AI agent rollout pipeline.

5. Permissions and workspace scoping

Knowledge bases are scoped to your organization. An agent in the same organization searches any attached base; agents in other organizations cannot. In the dashboard, the Agents → Knowledge base page mirrors the same surface — anyone on the workspace with the dashboard’s agents read permission can browse it, and only owner / admin / developer roles can upload, edit, or delete. API access needs the knowledge scopes:

Every authenticated call to /knowledge-bases/* requires the base knowledge:read OR knowledge:write scope.
Mutations — create base, upload document, attach to a base, update search config, run refresh — additionally require knowledge:write, and the write guard also demands the owner, admin, or developer role.
The POST /knowledge-bases/:id/search endpoint accepts either standard auth or the internal agent-runtime token — that is how the search_knowledge tool reaches the base on the agent’s behalf without your customer’s API key appearing in the loop.

Workspace-level isolation is the tenant boundary: the same organization-scoped isolation that keeps your other Orbit resources private keeps your knowledge bases private.

6. Refresh, crawl sources, and detect gaps

A knowledge base that never gets fresh content drifts; Orbit ships three complementary ways to keep it current. External source documents. URL / RSS / Notion documents are re-fetched on the base’s refresh_schedule (manual, hourly, daily, weekly) once set at create time. POST /knowledge-bases/:id/refresh forces an immediate re-fetch of every external-source document in the base — run it when you know a source page changed. Sync connectors. Register managed connectors against Notion, Confluence, Google Drive, SharePoint, or Zendesk with POST /knowledge-bases/:id/connectors. The provider field accepts notion, confluence, google_drive, sharepoint, or zendesk. The connector defaults a connection_id to your organization id (the integrations-catalog convention) and a sync_schedule to daily; the scope object narrows the pull to specific spaces / folders / labels. The drift report (GET /knowledge-bases/:id/connector-drift-report) flags sources whose cached content diverged from the live source, and auto_refresh_on_drift: true makes drift detection queue the resync automatically.

curl -X POST https://api.orbit.devotel.io/api/v1/knowledge-bases/kb_abc123/connectors \
  -H "X-API-Key: dv_live_sk_..." \
  -H "Content-Type: application/json" \
  -d '{
    "provider": "zendesk",
    "display_name": "Help center",
    "sync_schedule": "daily",
    "auto_refresh_on_drift": true
  }'

Gap reports. Two surfaces flag missing content:

GET /agents/:id/kb-unanswered — clusters customer questions the agent struggled with (“I don’t have that info”-class phrases, escalations with a knowledge-gap keyword, reached-human-handoff) so you see “47 contacts asked about returns policy” and can add the article. POST /knowledge-bases/:id/gaps/draft pre-fills a draft document from a cluster; POST .../documents/:docId/approve makes it live.
GET /knowledge-bases/:id/staleness — per-document staleness bucket (fresh, stale, critical) with totals, so a slowly-rotting source flags before customers hit it.

Both reports land in the dashboard Knowledge base page — gap clusters pre-fill a one-click “Add to KB” draft, and staleness renders as a per-document badge.

Troubleshooting

Symptom	Fix
Agent never retrieves anything from the KB	Confirm `knowledge_base_ids` on the agent includes the base’s id; the `search_knowledge` tool can only search attached bases.
Retrieval returns stale docs ahead of fresh	Lower `recency_boost`, lengthen `recency_half_life_days`, or add the stale source to `exclude_categories`.
Upload `422` on a multipart form	Either the `name` or `type` multipart field is missing or a binary (PDF/DOCX/image) file is being sent as raw JSON — the base accepts binary via multipart only.
`Document content too large` on a JSON upload	JSON content is capped at 10 MB; split the file or switch to the multipart form, which accepts up to the same cap with file bytes.
`retryDocument` keeps failing	The retry re-reads the original source bytes; if the bytes are a broken upload (truncated file, zero-byte file), delete the document and upload again.
Connector drift report shows “diverged”	With `auto_refresh_on_drift: false` (the default) the report only flags; either POST a manual `/connectors/:connectorId/sync` or switch the connector to auto-refresh.
Staleness report shows a bucket of critical docs	`POST /knowledge-bases/:id/refresh` re-fetches every external-source document immediately — do that rather than deleting the whole base.
Retrieval benchmark dropped after a knob change	Reset by PUTting `{}` to the search-config endpoint, benchmark again, then re-apply knobs one at a time. The one-knob-at-a-time loop is what isolates the regression.

Worked example — a support base that stays fresh end-to-end

Create the base with a daily refresh schedule so URL-sourced docs keep re-fetching: POST /knowledge-bases with refresh_schedule: "daily".
Load the curated PDFs / help-center pages via POST /knowledge-bases/:id/documents, then a Zendesk connector with auto_refresh_on_drift: true so article edits propagate without a manual sync tick.
Attach the base to the agent: PUT /agents/:id with the base’s id in knowledge_base_ids.
Run the retrieval benchmark once before you trust the base in front of customers; pin that run as the baseline.
Each week, check GET /agents/:id/kb-unanswered for gap clusters and GET /knowledge-bases/:id/staleness for documents drifting into stale — add a draft from the gap cluster or force a refresh on the stale docs.
When you change a knob on the search-config, re-run the benchmark and compare to the pinned baseline before you let it reach production.

WhatsApp

Guides

Messaging Controls

Audience & Support Tools

Build and Maintain an AI Knowledge Base

Build and Maintain an AI Knowledge Base

What an Orbit knowledge base is

1. Create the knowledge base

2. Attach the base to an agent

3. Load documents

4. Tune retrieval

5. Permissions and workspace scoping

6. Refresh, crawl sources, and detect gaps

Troubleshooting

Worked example — a support base that stays fresh end-to-end

See also

​Build and Maintain an AI Knowledge Base

​What an Orbit knowledge base is

​1. Create the knowledge base

​2. Attach the base to an agent

​3. Load documents

​4. Tune retrieval

​5. Permissions and workspace scoping

​6. Refresh, crawl sources, and detect gaps

​Troubleshooting

​Worked example — a support base that stays fresh end-to-end

​See also

Build and Maintain an AI Knowledge Base

What an Orbit knowledge base is

1. Create the knowledge base

2. Attach the base to an agent

3. Load documents

4. Tune retrieval

5. Permissions and workspace scoping

6. Refresh, crawl sources, and detect gaps

Troubleshooting

Worked example — a support base that stays fresh end-to-end

See also