Background Jobs (BullMQ)

Every part of VeroXM that has to happen later, retry on failure, or run on a schedule - delivering a webhook, purging a CDN, publishing content at a future time, translating an entry, aggregating analytics - runs on the same Redis-backed BullMQ job queue, not a cron script or a fire-and-forget HTTP call. That single piece of infrastructure is what makes VeroXM's background behavior reliable instead of best-effort.

Why a queue, and why it matters to you

Nothing is lost on a transient failure. A webhook receiver that is down for a minute, a CDN API that times out once, a database hiccup mid-job - none of these drop the work. The job stays in the queue and retries with backoff instead of silently failing the one time it happened to run.
Retries do not hammer the thing that just failed. Every queue backs off exponentially between attempts (2s, 4s, 8s ... depending on the queue), so a struggling downstream service gets breathing room instead of an immediate retry storm.
Bursts collapse instead of piling up. Several rapid content changes to the same entry do not each trigger their own CDN purge - see CDN purge below for how job coalescing turns a burst of saves into one purge call.
The work survives a request. The API responds to your save immediately; the webhook delivery, translation, or cache invalidation that save triggers happens after, off the request path, so a slow downstream integration never makes your save feel slow.

What actually runs on it

Queue	What it does	Retry policy
`webhook-delivery`	Delivers a signed HTTP POST for every subscribed `content.published` / `content.updated` / `content.deleted` / `approval.requested` event.	5 attempts, exponential backoff starting at 2s (2s/4s/8s/16s gaps)
`cdn-purge`	Calls your configured CDN provider's purge API after content changes.	3 attempts, 2s/4s backoff
`scheduled-publish`	Fires a delayed job for content scheduled to publish at a future time, re-verifying nothing changed since it was scheduled before actually publishing.	Delayed to the exact scheduled timestamp; re-checked at fire time
`content-translation`	AI-assisted translation of an entry into another locale.	3 attempts, exponential backoff starting at 5s
`content-consumption-aggregation`	A repeating scheduled job (not triggered by an event) that aggregates content-consumption signals into the ranking data personalization reads.	3 attempts, exponential backoff starting at 5s

Job coalescing: one purge, not one per save

The CDN-purge queue enqueues every job under a stable id - purge-<projectId> - with a short delay. BullMQ treats adding a job whose id is already waiting as a no-op, so three saves to the same project inside that window collapse into a single purge call instead of three:

Coalescing a burst of content changes

content.updated -> enqueue "purge-42" (5s delay)
content.updated -> enqueue "purge-42"  // already queued, no-op
content.updated -> enqueue "purge-42"  // already queued, no-op
                                        // 5s later: exactly one purge call runs

What "gives up" looks like

When a queue exhausts its attempts, the job does not silently vanish - the give-up is logged once, and the durable record of what happened lives outside the queue itself. Webhook attempts are each written as their own row in the delivery log (see Webhooks), visible from the dashboard, so a failing integration is something you can see and debug, not a job that disappeared into a queue.

This is why it's reliable, not just fast

The value of running this on BullMQ instead of a simple background task isn't speed - it's that a receiver being down for a minute, a CDN API timing out once, or the API process restarting mid-job does not lose the work. It retries, backs off, and leaves a record, the same way a production job queue is supposed to.

← Content Experiments CDN & Media Pipeline →