> ## Documentation Index
> Fetch the complete documentation index at: https://docs.orbit.devotel.io/llms.txt
> Use this file to discover all available pages before exploring further.

# Build and Maintain an AI Knowledge Base

> Create a knowledge base, load documents into it, attach it to an agent, tune retrieval with rerank controls and benchmarks, schedule refreshes, and use the staleness + gap reports to keep answers current.

# Build and Maintain an AI Knowledge Base

The [WhatsApp and SMS AI Agent](/guides/whatsapp-sms-ai-agent) and
[AI agent rollout pipeline](/guides/ai-agent-rollout-pipeline) guides both
assume a knowledge base already exists. This guide builds one from scratch:
create the container, load documents, attach it to an agent, tune retrieval,
and keep the content fresh. It also covers the dashboard companion page —
**Agents → Knowledge base** — which exposes the same surface without code.

The endpoint paths below are relative. Send them against
`https://api.orbit.devotel.io/api/v1`.

## What an Orbit knowledge base is

A knowledge base is a named collection of documents your AI agent can search
when it needs grounded answers. It is the retrieval backbone behind the
`search_knowledge` agent tool: on each turn, the agent's runtime embeds the
customer's question, runs a similarity search over the chunks of every
document chunk in the attached bases, and hands the top matches to the model
as context. Without a knowledge base, the agent answers from its system
prompt alone — which is fine for tone and routing logic, but wrong for
anything factual that changes (refund policy, hours, account-specific
procedures).

Each knowledge base is workspace-scoped and tenant-owned: it belongs to your
organization, any agent in the same organization can be attached to it, and
no other tenant can see it. Permissions are the standard API scopes —
`knowledge:read` for reads, `knowledge:write` (owner / admin / developer
roles) for creates, uploads, and deletes.

## 1. Create the knowledge base

```bash theme={null}
curl -X POST https://api.orbit.devotel.io/api/v1/knowledge-bases \
  -H "X-API-Key: dv_live_sk_..." \
  -H "Content-Type: application/json" \
  -d '{
    "name": "Support Docs",
    "description": "Refund, shipping, and account help articles"
  }'
```

The response includes the base's `id` (e.g. `kb_abc123`) — keep it for every
subsequent call. Optional fields:

* `description` — a free-text label visible in the dashboard; no effect on
  retrieval.
* `refresh_schedule` — one of `manual` (default, existing behavior),
  `hourly`, `daily`, or `weekly`. When set, external URL/RSS/Notion
  documents you load re-fetch on that cadence; `manual` leaves them
  scrape-once. See [Step 6](#6-refresh-crawl-sources-and-detect-gaps)
  below.
* `config` — free-form metadata map, surfaced back on reads; operational
  notes belong here (e.g. a JIRA epic key, a team name).

## 2. Attach the base to an agent

Attach by updating the agent's `knowledge_base_ids` array. The array is the
contract: the agent's `search_knowledge` tool can only search bases in this
list, so the allowlist can never be bypassed by a maliciously or
accidentally broad prompt.

```bash theme={null}
curl -X PUT https://api.orbit.devotel.io/api/v1/agents/agent_abc123 \
  -H "X-API-Key: dv_live_sk_..." \
  -H "Content-Type: application/json" \
  -d '{ "knowledge_base_ids": ["kb_abc123"] }'
```

Multiple bases are the normal case — a support agent commonly reads from a
`Support Docs` base plus a `Billing Policies` base, and the match
quality from the strongest chunk wins regardless of which base it came
from.

## 3. Load documents

Two upload paths, one endpoint:

**Text content (JSON body).** Post the content directly; the base chunks and
embeds it.

```bash theme={null}
curl -X POST https://api.orbit.devotel.io/api/v1/knowledge-bases/kb_abc123/documents \
  -H "X-API-Key: dv_live_sk_..." \
  -H "Content-Type: application/json" \
  -d '{
    "name": "Refund Policy",
    "type": "markdown",
    "content": "# Refund Policy\n\nCustomers may request a refund within 30 days…"
  }'
```

**Files (multipart form).** Upload PDFs, Word documents, CSVs, images, and
text formats; the content type is inferred from the filename extension if
you omit the `type` field.

```bash theme={null}
curl -X POST https://api.orbit.devotel.io/api/v1/knowledge-bases/kb_abc123/documents \
  -H "X-API-Key: dv_live_sk_..." \
  -F "file=@refund-policy.pdf" \
  -F "name=Refund Policy" \
  -F "type=pdf"
```

Supported `type` values: `text`, `txt`, `markdown`, `md`, `html`, `pdf`,
`csv`, `docx`, `json`, `py`, `xml`, `yaml`, `yml`, `js`, `ts`, `tsx`,
`jsx`, `rtf`, `log`, `png`, `jpg`, `jpeg`, `gif`, `webp`, `image`.

**Chunking.** PDF/DOCX are extracted to plain text first, then split into
overlapping chunks (1,000 characters per chunk, 150 characters of overlap to
preserve context across the boundary). Image types (`png`, `jpg`, `gif`,
`webp`, `image`) run through a vision OCR pass that transcribes the text
before chunking. The chunk is the retrieval unit — a query matches one
specific chunk, so tight topical documents retrieve better than one
300-page everything-PDF.

Once loaded, manage the document lifecycle per KB:

* `GET /knowledge-bases/:id/documents` — list every document with size and
  chunk count.
* `POST /knowledge-bases/:id/documents/:docId/retry` — re-run an upload that
  failed partway (a stalled OCR pass, an interrupted extraction); the retry
  re-chunks the same source bytes.
* `DELETE /knowledge-bases/:id/documents/:docId` — remove the document and
  its chunks from retrieval.
* Versioning + moderation
  (`GET /knowledge-bases/:id/documents/:docId/versions`,
  `POST .../rollback`, `POST .../approve`, `POST .../reject`) — review and
  history for the draft→approved document lifecycle.

## 4. Tune retrieval

Retrieval tuning is where most "the agent ignores the KB" complaints end
up. Tune the knobs, then re-benchmark — the deploy-time benchmark is the
objective way to know a knob actually helped.

**Read the current config.** A base that has never been tuned returns the
neutral default (every knob a no-op, identical to the legacy raw-similarity
ordering).

```bash theme={null}
curl https://api.orbit.devotel.io/api/v1/knowledge-bases/kb_abc123/search-config \
  -H "X-API-Key: dv_live_sk_..."
```

**Set a knob.** Every field has a no-op default, so a PUT of `{}` cleanly
resets to the legacy ordering and omitting a knob never silently changes
that knob.

```bash theme={null}
curl -X PUT https://api.orbit.devotel.io/api/v1/knowledge-bases/kb_abc123/search-config \
  -H "X-API-Key: dv_live_sk_..." \
  -H "Content-Type: application/json" \
  -d '{
    "min_score": 0.5,
    "recency_boost": 1.5,
    "recency_half_life_days": 30,
    "source_weights": { "Zendesk": 1.2, "Confluence": 0.8 },
    "include_categories": [],
    "exclude_categories": ["archive"]
  }'
```

What each knob does:

* `min_score` (0–1, default 0) — drop chunks whose raw similarity score
  falls below the floor. Raise it when the agent retrieves drift-y
  near-misses and hallucinates them together.
* `recency_boost` (0–5, default 0) — multiplier strength of the recency
  bump; fresher documents rank up to ×(1+boost).
* `recency_half_life_days` (1–3650, default 30) — how quickly the recency
  boost decays.
* `source_weights` (label → 0–20 multiplier) — boost or damp whole sources
  (e.g. boost the curated Zendesk articles, damp an old Confluence dump).
* `include_categories` / `exclude_categories` — allow-list or deny-list of
  categories, matched against the document's category metadata.

**Detect drift with the benchmark.** Retrieval drift shows up as a quality
regression on the same golden query set even though nothing in your prompt
changed — usually means fresh content hasn't been added, stale content is
outranking fresh, or the boost knobs are fighting each other. Run the
benchmark, pin a baseline, re-run after every knob change:

* `POST /agents/:id/retrieval-benchmark/runs` — score the agent's recall
  on your pinned query set.
* `GET /agents/:id/retrieval-benchmark` — read the latest persisted run.
* `POST /agents/:id/retrieval-benchmark/baseline` — pin a run as the
  baseline for comparison.

The full rollout pipeline walks through how to wire this into a deploy:
[AI agent rollout pipeline](/guides/ai-agent-rollout-pipeline#2-load-knowledge-sources).

## 5. Permissions and workspace scoping

Knowledge bases are scoped to your organization. An agent in the same
organization searches any attached base; agents in other organizations
cannot. In the dashboard, the **Agents → Knowledge base** page mirrors the
same surface — anyone on the workspace with the dashboard's agents read
permission can browse it, and only owner / admin / developer roles can
upload, edit, or delete.

API access needs the knowledge scopes:

* Every authenticated call to `/knowledge-bases/*` requires the base
  `knowledge:read` OR `knowledge:write` scope.
* Mutations — create base, upload document, attach to a base, update
  search config, run refresh — additionally require `knowledge:write`, and
  the write guard also demands the `owner`, `admin`, or `developer` role.
* The `POST /knowledge-bases/:id/search` endpoint accepts either standard
  auth or the internal agent-runtime token — that is how the `search_knowledge`
  tool reaches the base on the agent's behalf without your customer's API
  key appearing in the loop.

Workspace-level isolation is the tenant boundary: the same
organization-scoped isolation that keeps your other Orbit resources private
keeps your knowledge bases private.

## 6. Refresh, crawl sources, and detect gaps

A knowledge base that never gets fresh content drifts; Orbit ships three
complementary ways to keep it current.

**External source documents.** URL / RSS / Notion documents are re-fetched
on the base's `refresh_schedule` (`manual`, `hourly`, `daily`, `weekly`)
once set at create time. `POST /knowledge-bases/:id/refresh` forces an
immediate re-fetch of every external-source document in the base — run it
when you know a source page changed.

**Sync connectors.** Register managed connectors against Notion, Confluence,
Google Drive, SharePoint, or Zendesk with `POST /knowledge-bases/:id/connectors`.
The `provider` field accepts `notion`, `confluence`, `google_drive`,
`sharepoint`, or `zendesk`.
The connector defaults a `connection_id` to your organization id (the
integrations-catalog convention) and a `sync_schedule` to `daily`; the
scope object narrows the pull to specific spaces / folders / labels. The
drift report (`GET /knowledge-bases/:id/connector-drift-report`) flags
sources whose cached content diverged from the live source, and
`auto_refresh_on_drift: true` makes drift detection queue the resync
automatically.

```bash theme={null}
curl -X POST https://api.orbit.devotel.io/api/v1/knowledge-bases/kb_abc123/connectors \
  -H "X-API-Key: dv_live_sk_..." \
  -H "Content-Type: application/json" \
  -d '{
    "provider": "zendesk",
    "display_name": "Help center",
    "sync_schedule": "daily",
    "auto_refresh_on_drift": true
  }'
```

**Gap reports.** Two surfaces flag missing content:

* `GET /agents/:id/kb-unanswered` — clusters customer questions the agent
  struggled with ("I don't have that info"-class phrases, escalations with
  a knowledge-gap keyword, reached-human-handoff) so you see "47 contacts
  asked about returns policy" and can add the article. `POST
  /knowledge-bases/:id/gaps/draft` pre-fills a draft document from a
  cluster; `POST .../documents/:docId/approve` makes it live.
* `GET /knowledge-bases/:id/staleness` — per-document staleness bucket
  (`fresh`, `stale`, `critical`) with totals, so a slowly-rotting source
  flags before customers hit it.

Both reports land in the dashboard **Knowledge base** page — gap clusters
pre-fill a one-click "Add to KB" draft, and staleness renders as a
per-document badge.

## Troubleshooting

| Symptom                                          | Fix                                                                                                                                                                    |
| ------------------------------------------------ | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| Agent never retrieves anything from the KB       | Confirm `knowledge_base_ids` on the agent includes the base's id; the `search_knowledge` tool can only search attached bases.                                          |
| Retrieval returns stale docs ahead of fresh      | Lower `recency_boost`, lengthen `recency_half_life_days`, or add the stale source to `exclude_categories`.                                                             |
| Upload `422` on a multipart form                 | Either the `name` or `type` multipart field is missing or a binary (PDF/DOCX/image) file is being sent as raw JSON — the base accepts binary via multipart only.       |
| `Document content too large` on a JSON upload    | JSON content is capped at 10 MB; split the file or switch to the multipart form, which accepts up to the same cap with file bytes.                                     |
| `retryDocument` keeps failing                    | The retry re-reads the original source bytes; if the bytes are a broken upload (truncated file, zero-byte file), delete the document and upload again.                 |
| Connector drift report shows "diverged"          | With `auto_refresh_on_drift: false` (the default) the report only flags; either POST a manual `/connectors/:connectorId/sync` or switch the connector to auto-refresh. |
| Staleness report shows a bucket of critical docs | `POST /knowledge-bases/:id/refresh` re-fetches every external-source document immediately — do that rather than deleting the whole base.                               |
| Retrieval benchmark dropped after a knob change  | Reset by PUTting `{}` to the search-config endpoint, benchmark again, then re-apply knobs one at a time. The one-knob-at-a-time loop is what isolates the regression.  |

## Worked example — a support base that stays fresh end-to-end

1. Create the base with a daily refresh schedule so URL-sourced docs keep
   re-fetching: `POST /knowledge-bases` with
   `refresh_schedule: "daily"`.
2. Load the curated PDFs / help-center pages via
   `POST /knowledge-bases/:id/documents`, then a Zendesk connector with
   `auto_refresh_on_drift: true` so article edits propagate without a
   manual sync tick.
3. Attach the base to the agent: `PUT /agents/:id` with the base's id in
   `knowledge_base_ids`.
4. Run the retrieval benchmark once before you trust the base in front of
   customers; pin that run as the baseline.
5. Each week, check
   `GET /agents/:id/kb-unanswered` for gap clusters and
   `GET /knowledge-bases/:id/staleness` for documents drifting into stale —
   add a draft from the gap cluster or force a refresh on the stale docs.
6. When you change a knob on the search-config, re-run the benchmark and
   compare to the pinned baseline before you let it reach production.

## See also

* [AI agent rollout pipeline](/guides/ai-agent-rollout-pipeline) — deploy
  the KB into a canary + regression-gated production loop.
* [Build a WhatsApp and SMS AI Agent](/guides/whatsapp-sms-ai-agent) — the
  messaging form this KB usually grounds.
* [Creating Agents](/agents/creating-agents) — full agent configuration
  reference including `knowledge_base_ids`, tools, and escalation triggers.
* [Knowledge bases API reference](/api-reference/knowledge-bases) — endpoint
  shapes for the knowledge-base routes.