Operate
Scaling the worker
One process, or the server and workers split apart.
The worker takes jobs from a queue in Postgres: processing a build (compare and thumbnails), webhook deliveries, git checks and comments, email and Slack, and integration events. It also runs the periodic sweeps (retention, stalled uploads, overdue review requests, triage assignment, job pruning).
One process or many
Section titled “One process or many”circi all runs the HTTP server and the worker together. It's right for a small install, and a single container is enough.
circi serve # HTTP API and dashboardcirci worker # background jobsRun any number of each from the same image, against the same Postgres and storage. In compose that is two services with command: ['serve'] and command: ['worker']. A worker serves no HTTP, so the image's health check passes for it as long as the process runs.
Each job is claimed by one worker only, so several workers never run the same job, and a sweep takes a lock, so it's safe to run workers side by side. Run extra workers when the queue stays above zero: Instance › Health shows it under Background jobs, with each queue's waiting, running and failed counts.
Several servers can run behind one proxy: they share everything through Postgres and storage. A saved Instance setting reaches each process within about 30 seconds. See Instance settings.
How a worker spends its time
Section titled “How a worker spends its time”- A worker runs jobs in three parallel lanes. Two take builds first. The third, the light lane, never takes a build, so a webhook or an email doesn't wait behind image work. These counts are built in; there is no environment variable for them.
- A queued job wakes the worker through Postgres
LISTEN/NOTIFY. When the notification connection is down, or for a missed one, polling finds jobs (every 5 seconds with a listener, 1 second without). - A failed job is retried with a doubling wait.
JOB_RETRY_BASE_SECONDSThe first wait before a retry. It doubles each time.
JOB_RETRY_MAX_SECONDSThe longest wait between retries.
What to tune
Section titled “What to tune”| Goal | Setting |
|---|---|
| Use all cores for comparisons | CIRCI_DIFF_ENGINE=native (the default) |
| No native module, still off the main thread | CIRCI_DIFF_ENGINE=js-workers, and CIRCI_DIFF_THREADS to cap threads and memory |
| Fewer retries piling up | WEBHOOK_TIMEOUT_SECONDS, GITHUB_API_TIMEOUT_SECONDS, S3_*_TIMEOUT_SECONDS |
| Keep the job table small | JOB_RETENTION_DAYS |
WEBHOOK_TIMEOUT_SECONDS and JOB_RETENTION_DAYS are also limits in Instance settings: a variable that is set wins and locks the field, otherwise the saved value applies.