Settings from the administration panel

For whoever installs and runs Tabularia. Administration → Settings changes, while everything keeps running, the settings an administrator may own: limits, waits, retention, the assistant's budget, the caps of the data sources. The environment (.env, the Helm values) stays the starting value of every setting; the panel overrides it, and Back to the environment returns to it.

The settings are in boxes, one per component (Gateway, Kubernetes, Engine), with a search by name or variable. Each one shows a description in the reader's language, the value of the environment and its variable, the allowed range, and who changed it and when.

A change on its way: the setting found by its variable, the new value marked Unsaved next to the environment's, and Save in the bar at the bottom.

Three families

• Live (most of them): read by the code each time it is used. A change applies within ten seconds on every process: every gateway and orchestrator replica, the engine API, every worker process.
• Read when a process starts (marked with the restart icon ↻): saved at once, applied when the components that read them restart. The panel tells which components still run a previous value and restarts them (see Restarts).
• Never in the panel: addresses, credentials, keys, identities, namespaces, buckets, with one exception, the external ClickHouse's connection (see The ClickHouse connection); the rest: the switches tied to resources the Helm chart creates (scripting, Container node, cluster view, scaling), the audit log's retention, and anything that, wrong, would stop Tabularia from starting or anyone from signing in. They are the configuration of the installation and stay in the environment. Some that look harmless are left out on purpose: OIDC__SUPERUSER_GROUP (it makes administrators), OIDC__SCOPES, OIDC__VERIFY_TLS and SMTP__VERIFY_TLS (the security of a connection), the scripting base images, builder image, PyPI and CRAN addresses (code that runs in the builds), STORAGE__DIRECT_READ (it depends on the network between services), KUBERNETES__*_MAX_REPLICAS (the ceiling the installer puts on the panel itself), AUDIT__RETENTION_MINUTES and AUDIT__IP_RETENTION_DAYS (an administrator must not be able to shorten the trail of what administrators do), ENGINE__SETTINGS_PANEL (it governs the panel).

Database backups (on, schedule, number kept) are in Administration → Backups, and the processes per worker in Administration → Workers: not repeated here.

What can be changed

The panel accepts narrower ranges than the environment, so that a typing mistake cannot take the system down; a value outside them goes in the environment. The boxes of a feature the installation does not have are not shown (single sign-on, Kubernetes, scripting, the Container node) or greyed out (an engine that is not configured).

Gateway and Kubernetes

Sign-in and sessions

Single sign-on (when single sign-on is configured)

Orchestration and schedules

History kept

Webhooks and SIEM

AI assistant

Agents over MCP

dbt export

Previews and uploads

Database backups

Monitoring

OpenLineage

Gateway processes

Kubernetes API (on Kubernetes)

Python and R scripts (when scripting is on)

Image builds (when scripting is on)

Container nodes (when the Container node is on)

Engine

Workers and runs

Step cache

Dashboards

Embedded engines

Engine API

Import

Export

SFTP

S3 folders

HTTP APIs

Placeholders

ClickHouse connection (see The ClickHouse connection)

ClickHouse

BigQuery

The ClickHouse connection

The external ClickHouse can be configured, or moved to another server, from the panel instead of the environment. Administration → Settings → Engine → ClickHouse connection holds the server, its port and TLS, the working database, the user and its password, the AI assistant's user, the transport and how the server reaches the storage. The environment's CLICKHOUSE_EXTERNAL__* stay the starting values, as for every setting: an installation configured in its .env or Helm values keeps working unchanged, and Back to the environment returns to them.

• Test connection, at the bottom of the box, tries the values in the fields, even before they are saved: signing in, the working database, the AI user when it is set, and reading a small file from the storage (transport s3) or creating and dropping a table (transport push). Each step says what failed with the server's own words; the test saves nothing and leaves nothing on the server or in the bucket.
• The password is write-only. It is encrypted in the gateway as soon as it arrives, with the key gateway and engine already share (SECURITY__FERNET_KEY), and it stays encrypted in the gateway's database and in Valkey; only the engine's processes decrypt it, when they apply it. The panel shows only whether one is set, the audit only that it changed. An empty field keeps the current password.
• Applies after a restart of the engine API, the workers, the preview workers and the scheduler: a process keeps its connection for its life. After Save, the restart banner offers the rollout (see Restarts).
• The ClickHouse engine appears in the flows' engine picker once a server is set, and the ClickHouse box of tuning settings stops being greyed out.
• The network: the panel cannot change what the installation allows. On Kubernetes, if the Helm values set networkPolicy.egressCIDRs or networkPolicy.egressNamespaces, the engine's processes reach only those ranges and namespaces (plus the server named in externalServices.clickhouse.host): a server elsewhere must be added there first, and the test fails at the first step until it is.

Rules between settings

A change that breaks one of these is refused, and the error names the setting just changed:

• the tasks' soft time limit stays below the hard one, and the import read timeout below the soft one;
• the engine's hard time limit stays at least five minutes below ENGINE__RUN_STALE_TIMEOUT_SECONDS, the age after which the gateway declares a run lost (both directions: lowering the threshold below the engine's limit plus five minutes is refused too, and checked against the engine, so it is refused while the engine does not answer);
• an orchestration's heartbeat fits at least three times in the silence after which it is declared lost;
• the executions an orchestrator carries at once fit in its database pool (pool size, or the threads when it is automatic, plus the overflow), because each holds a connection while it asks the engine. The pool is sized when a process starts: raised together, the executions follow the pool only after the restart (each orchestrator caps them at the pool it has);
• single sign-on admitting only the members of the groups taken needs a list of them (or OIDC__SUPERUSER_GROUP), and the claim with the groups cannot be empty: either would shut everyone out, or take every group away at the next sign-in;
• going back to the installation's value is checked like a new value: back to the environment's, the lost-run threshold must still stay above the engine's hard limit, and the engine's hard limit under the threshold; a change that would drop another saved value (the pool back to automatic under 90 saved executions) is refused, naming it.

One rule the panel cannot check is between a setting and the Helm chart: a worker that stops (an upgrade, a restart from this panel, a drain, a scale-in, KEDA going down to its minimum) waits for its tasks for worker.terminationGracePeriodSeconds (3600 s by default), then Kubernetes kills it, tasks and all. Raising the tasks' hard limit above it lets those events cut the longer runs: raise the chart's value first. A cluster autoscaler may also cap the wait it gives a node's pods (ten minutes by default).

A saved value can stop being valid later, for instance after the environment changed under it. It is then dropped at every reading (the environment's applies, and the log says why); it does not block changes to other settings, and the panel marks it Not applied until it is changed or reset.

How a change travels

The gateway's settings. Saved in the gateway's database (table gateway_settings), with an audit entry settings.update (old and new value of each). Every gateway and orchestrator process reads the table again every ten seconds and lays the values on top of the environment; the process that saved applies them at once.

The engine's settings. Checked by the engine and saved in the gateway's database (table engine_settings) only once the engine accepted them, all or none, with an audit entry engine_settings.update. The engine keeps them in Valkey, every engine process reads them again every ten seconds, and the gateway sends them again every minute: a flushed Valkey gets them back. The upload limit is set once, in the Gateway box, and reaches the engine's own barrier the same way.

Saves go one at a time, across gateway replicas too (a lock in the database).

«Live» means from the next use: an orchestrator waiting for a step keeps the pause it started with until the next check, a script's or container's Job is followed at the pace it was started with, and a new connect timeout for ClickHouse applies to the connections opened from then on. A new time zone for the schedules computes again the next slot of every schedule at once.

Example: a smaller cap on CSV exports

The Viewer's and the editor's CSV downloads stop at 5,000,000 rows by default (EXPORT__MAX_ROWS). To bring that down to 2,000,000 without touching the deployment:

1. Administration → Settings, type EXPORT__MAX_ROWS in the search: the box Export (Engine) opens on Most rows in a CSV export, with the environment's value, its variable and the allowed range (0 to 100,000,000 rows).
2. Type 2000000. The field is marked Unsaved, with Undo the change, and the bar at the bottom counts «1 change to save».
3. Save. The engine checks the value and keeps it; within ten seconds every worker uses it, and the audit log has an engine_settings.update with the old and the new value. A larger export now stops and says so.

Back to the environment, on the setting, returns to 5,000,000.

Restarts

A setting marked ↻ is read only when a process starts: the gateway's threads and database pool, the workers' recycling and memory, the embedded engines' memory, the engine API's threads and preview wait, the scheduler's intervals. Saving it records, for each component that reads it, when it changed; the pods of that component started before then still run the previous value, and a banner on top of the page lists those components.

On Kubernetes (Helm settingsRollout.enabled, on by default) Restart them restarts those components, one pod at a time: the orchestrator evicts the oldest pod started before the request through the Kubernetes eviction API, waits until all the component's pods are ready again, evicts the next, and goes on component after component: run workers, preview workers, scheduler, engine API, gateway, orchestrator. The new pods read the panel's values when they start; nothing in the Deployments changes. The banner follows the restart and can stop it.

• A component with a single pod is unavailable for the few seconds its new pod takes to start. The restart waits for each component's last new pod to be ready before it moves to the next component.
• A worker pod finishes the run it has in hand before it stops (up to its grace period, worker.terminationGracePeriodSeconds, an hour by default). Worker pods go one at a time, and the pods not yet restarted keep taking new runs: with many workers and long runs the restart of the workers can take hours, and the components after them in the order wait. Restart the workers on their own, at a quiet hour, when runs are long.
• The scheduler (beat) is a single pod: for a second or two, while the evicted one stops and the new one starts, both are up. A periodic task due in that instant may be queued twice (the housekeeping tasks are harmless twice); a node drain does the same.
• Restarting the orchestrator stops the executions running on it: they are closed as failed (with the reason) and can be started again. The orchestrator restarts itself last; the next one finds nothing left and closes the request.
• The chart's PodDisruptionBudgets are honoured: when one says «not now» (a node being drained at the same time), the restart waits.
• A new pod that never becomes ready (a value that stops it from starting, such as a pool larger than Postgres accepts) stops the restart there, with the old pods still serving; the banner names the pod it waits for. Put the value back and restart again: the broken pod goes first.
• Pods that start for other reasons (a crash, the autoscaler) read the new values too.
• A pod stuck on its way out for more than 15 minutes (its node lost) no longer holds the restart.
• If the pods cannot be read (the Kubernetes API down, a missing right), the banner says so and offers no restart for those components.

Why eviction and not kubectl rollout restart: that command patches the Deployment, and the right to patch a Deployment is the right to change its image or its service account, which the gateway, facing the internet, must not hold. The chart grants the gateway and the orchestrator only to read the pods of the release's namespace, and the orchestrator alone, which runs the restart and answers nobody from outside, to evict them.

A worker or the scheduler that starts while Valkey holds no values yet (a new cluster, a Valkey restarted without persistence) waits up to 45 seconds for the gateway to send them before it fixes its start values; past that it starts with the environment's and says so in its log. The engine API cannot wait (the values come through it): one started on an empty Valkey keeps the environment's threads and preview wait until it restarts.

With settingsRollout.enabled: false, or on Docker Compose, the banner names what to restart (docker compose restart worker preview-worker, or the Deployments by label) and the administrator says when it is done.

Who

• Administrators change the values and start restarts.
• Observers see them, with masked names.
• Everyone else gets 403.
• API tokens cannot change them nor restart anything, whatever their owner.

Turning it off

ENGINE__SETTINGS_PANEL=false (Helm: engine.settingsPanel: false) keeps the environment the only source, for installations that manage configuration in Git: the panel shows the values without changing them, and values saved earlier stop applying within a minute (they stay in the database, and come back if the switch is turned on again).

Upgrading

Schema revision 0022 only adds two empty tables (gateway_settings, settings_rollouts) and a nullable column to engine_settings (the values that are not whole numbers, in JSON). Until someone saves a value, every setting keeps its environment value. During a rolling update the previous gateway keeps reading and writing whole numbers: for up to a minute it may send the engine a decimal setting rounded, until its pods are gone.

Edge cases tried

Each of these has a test (gateway tests/test_panel_overrides.py, tests/test_gateway_settings_api.py, tests/test_settings_rollout.py; engine tests/test_panel_settings.py):

• a value of the wrong type is refused, not converted: true for a number, "5" for a number, 1 for on/off, NaN or Infinity for a decimal;
• a text with spaces around is saved trimmed; a time zone that does not exist, a CPU or memory quantity Kubernetes would not accept, a text over its length are refused;
• a key outside the catalogue is refused (db.url, audit.retention_minutes, oidc.superuser_group), and so is one of a feature the installation does not have;
• a value written by hand in the database that is not JSON, out of range or outside the catalogue is dropped at reading, the others apply;
• the database unreachable: the values in force stay; at start, the environment's apply and the gateway starts anyway;
• the same value saved again changes nothing and writes no audit entry;
• a constraint the environment already breaks does not block unrelated changes;
• an engine older than the panel's decimal settings or the upload limit: the gateway sends it what it knows;
• a restart: one pod at a time, oldest first, the orchestrator itself last; a pod on its way out or a new one not ready is waited for; a disruption budget's «not now» is waited for; a new pod that never gets ready stops the restart; a stopped restart evicts nothing more; a component with no old pod closes the request.
• found in the bug hunt of 2026-10-23 and fixed, each with a test: NaN, Infinity and an integer beyond any decimal (10**400) for a decimal setting; a reset that broke the lost-run threshold against the engine's hard limit (both directions); a value saved while another thread laid the previous ones over the settings (kept the old ones for good); a heartbeat made shorter waiting out the longer one; executions raised with the pool beyond the pool the process has; settings read before the database answered at start (the first housekeeping would have used the environment's retention); a single-pod component closed before its replacement was ready; Kubernetes taken for Docker Compose when no service account token is mounted; the workers' pool resized from Administration → Workers going back to the environment's memory ceiling; a worker started on an empty Valkey freezing the environment's values; an engine refusal or a gateway refusal throwing away the other changes made with it in the page; «10.000» read as ten in French.