Skip to main content

The social-listening ingestion-to-queue model

Social listening brings the public conversation about your brand — X/Twitter posts, Reddit threads, Trustpilot or G2 reviews, public comments under your Instagram and Facebook posts — into the same Inbox workspace your team already uses for private channels. A mention arrives as an ordinary conversation, and the work your team already does for every other channel — assignment, triage, reply, SLA clocks, audit — applies to it unchanged. This page defines the ingestion-to-queue path the code ships today: the ingest endpoint a connector calls with a conversation-ready mention payload, the dedup and threading rules the payload carries, the queue filters anyone can read, and the role gates each surface enforces. It does not define sentiment scoring or analytics — the sentiment value ships from your provider and the queue merely displays it. What your provider labels as a sentiment score, a keyword match, or a follower-reach estimate is your provider’s call and yours to send; the queue renders it.

What social listening ingests, and what enters the queue

A social mention is a public post that names your brand. When your connector normalizes one of these and posts it to POST /inbox/social-mentions/ingest, the ingest writer makes it a conversation on the social_mention channel:
  • Standalone posts (X/Twitter, Reddit, reviews, news, forums, YouTube, app stores) group into one conversation per public author — a second post from the same author continues their thread.
  • Post comments (instagram_comment and facebook_comment) require a post_id and thread per parent post into a single replyable conversation, because the post is the context.
  • Duplicates collapse on the (channel, external_id) pair. Re-delivering a mention the connector already sent is a safe no-op returning deduped: true, and a genuinely new mention rings the org-admins’ notification bell.
The queue filter and the row chip read from the platform token you post (twitter, reddit, trustpilot, g2, google_reviews, facebook_public, facebook_comment, instagram_comment, youtube, app_store, play_store, news, forum, other). The rest of the mention payload the queue renders — sentiment (green/red/muted), matched_keyword, and reach — passes through untouched from the provider. post_id is required for the two comment platforms, with post_permalink, post_title, and parent_comment_id as optional context.

The mention payload

One POST to /inbox/social-mentions/ingest carries one normalized mention. Posting requires an API key with owner, admin, or super-admin scope; any operator can read the queue. A worked ingest call:
Re-send the exact same body and the (channel, external_id) dedup key collapses it — the response is a 200 with deduped: true and no second conversation:
A malformed body returns 422 VALIDATION_ERROR with per-field errors, and for instagram_comment / facebook_comment a missing post_id is rejected the same way. Fix the payload, then retry.

Platform tokens, per connector

Map each provider source to exactly one token — the token is what the queue filter and the row chip display, and what a connector implementer hard-codes per source: The two comment platforms are the only ones with extra structure — post_id is required — and they differ from facebook_public in threading per post rather than per author. Keep the mapping at the connector boundary; the queue never re-interprets it.

Connector setup cookbook

A connector is a small process you run — a poller against an API your provider exposes, or a webhook receiver your provider pushes into. Both shapes end in the same one-POST-per-mention call above. The account the connector authenticates with must hold owner, admin, or super-admin in your organization, because ingest writes on behalf of an automated caller. Run it as an authenticated operator: mint an API key under Settings → API keys with admin scope, store it in the connector’s environment, and send it as the X-API-Key header. Reading the queue (GET /inbox/social-mentions) is not gated to admins — any operator in the org can list it — but the two stubs below only write.

Stub 1: an X/Twitter poller

Poll the provider’s recent-search endpoint every five minutes, normalize each post, and hand it to the ingest endpoint. The provider’s post id is the external_id; keep a high-water mark only as a hint — dedup on the server side is what makes a re-poll safe.
Schedule the script every five minutes (cron, a k8s CronJob, a worker). If it runs twice over the same window, the re-sent ids all come back deduped: true and the queue stays clean.

Stub 2: a Trustpilot webhook

Trustpilot can push new reviews to a URL you expose. The receiver verifies the push is for your business, normalizes each review, and ingests it — same one-POST-per-mention shape, different ingress:
Whatever the ingress shape, answer the provider’s webhook quickly and treat each mention as an independent retryable unit — exactly what makes the next section workable.

Idempotent retry discipline

Build the connector to retry aggressively and the queue to stay correct:
  • Retry on any 5xx or network error, and stop retrying on 4xx. A 422 means the payload is malformed (or a comment platform is missing post_id) — retrying it unchanged will fail forever. Log the per-field errors the body returns, fix the mapping, and replay.
  • Keep external_id byte-stable. The dedup key is (channel, external_id). If your connector re-derives the id per run — stringifying a numeric id one day and an object the next — the same mention lands as duplicates. Capture the provider’s stable id once, at normalization time, and store nothing else.
  • Let the server own dedup. A client-side “have I already sent this” cache is a hint at best; after a restart it lies. The (channel, external_id) check runs on every ingest call, so an idempotent webhook re-delivery or an overlapping poll window is absorbed by the server, not by your state.
  • Replays are safe by construction. When you backfill from a provider export, send the rows in any order and any overlap joins the dedup path.

Routing cookbook

Routing rules live under POST /inbox/routing-rules and match on the conversation’s channel set, the message body keywords, and the analyzed sentiment label. The three examples below are the shapes a supervisor reaches for most often on the mentions queue — replace the <user-id> / <team-id> placeholders with real ids. Negative sentiment goes to a senior agent:
A listening keyword goes to the owning team:
Third example — scope to one network on the rail rather than in a rule. The routing-rule channel condition enumerates the private-channel set (sms, whatsapp, email, instagram, messenger, and so on) and has no social_mention entry, so a per-network split is a queue filter, not a routing condition: set the Social mentions rail’s platform filter (backed by GET /inbox/social-mentions?platform=…) to the networks the on-call team works, and let the sentiment / keyword rules above do assignment. Two things to watch:
  • Sentiment fails closed. A conversation whose latest message has not yet been analyzed has no sentiment label, so a sentiment condition never matches it. Mentions whose connector does not supply a sentiment value stay unassigned until a person or a keyword rule picks them up.
  • Keyword entries are either plain substrings or /regex/ patterns, case-insensitive. A regex between slashes is validated at save time (ReDoS-safe patterns only) — a bad pattern is rejected before saving, not at match time.
Before a rule goes live, dry-run it with POST /inbox/routing-rules/test against a sample conversation; the dry-run reports whether the rule’s conditions match and which action would fire.

SLA policy cookbook

Attach an SLA policy scoped to the mentions channel and every mention conversation gets first-response and resolution clocks on top of the queue view:
scope: "per_channel" requires channel_filter; sending org_wide with a channel filter is rejected at save time. business_hours_only: true stops the clocks outside your org’s published hours, so a mention that lands Friday night breaches Monday morning, not Saturday. When a clock breaches, the notify action alerts the named users; the other breach actions (reassign, webhook, Slack) work the same as on any other channel — see Inbox setup for the shared SLA workspace.

Instagram and Facebook comment automations

For instagram_comment and facebook_comment, a genuinely new comment can fire a published comment-received automation alongside the queue entry. Build the flow in Automation → Flow Studio: add a trigger node with the “comment received” source, scope it to specific post ids or leave it account-wide, optionally gate it on keywords in the comment text (case-insensitive), give it a non-empty private-reply body, and publish. The runtime sends one private DM to the commenter through Meta’s private-reply path, and Meta’s rule for comment replies is a seven-day window (not the 24-hour standard DM window) — the public thread stays in the Social mentions queue for a person to work. Meta accepts exactly one private reply per comment, so at most one matching published flow fires per comment; the first published flow whose post-id and keyword filters match wins. The automation is entirely tenant-owned: the reply body is yours, the flow is published from your org on your own connected social account, and the execution shows up in the Flows → Executions log like any other flow run. The full webhook-to-flow wiring, including HMAC signature verification, is in Meta comment-to-DM.

Tenant-owned controls and isolation

Every control here — the connector you attach, the routing condition you configure, the SLA policy you attach, the comment automations you publish — is an operator action on your own organization. Ingest writes audit entries with actor context, and mention conversations live in the same tenant_<id> schema as your other channels — no second data plane, no cross-org read. A mention nobody routes stays unassigned at the top of the queue until a human works it.

See also