Tabularia Metadata Database

Tabularia Metadata Database An architecture diagram generated by Archify. audit_logs · append-only trail · Architecture component · actor_label survives audit_logs append-only trail actor_label survives banners · admin notices · Architecture component banners admin notices uploads · raw file + parquet · Architecture component uploads raw file + parquet pending_blob_deletions · deferred deletions · Architecture component · no foreign key pending_blob_deletions deferred deletions no foreign key users · local or SSO identity · Access control · unique (issuer, subject) users local or SSO identity unique (issuer, subject) user_groups · membership join · Access control · composite key user_groups membership join composite key groups · IdP groups, by name · Access control · unique name groups IdP groups, by name unique name permissions · capability per project · Access control · user_id XOR group_id permissions capability per project user_id XOR group_id projects · RBAC scope, self-nesting · Catalog and execution projects RBAC scope, self-nesting connections · credentials, Fernet · Catalog and execution · unique (project_id, name) connections credentials, Fernet unique (project_id, name) flow_versions · immutable history · Catalog and execution · unique (flow_id, version) flow_versions immutable history unique (flow_id, version) flows · definition + schedule · Catalog and execution · engine · notify_connection_id flows definition + schedule engine · notify_connection_id runs · one execution · the queue · Catalog and execution · claimed_by, heartbeat_at runs one execution · the queue claimed_by, heartbeat_at datasources · published parquet · Catalog and execution · snapshot_run_id guard datasources published parquet snapshot_run_id guard login_attempts · sign-in throttle · Architecture component · shared by replicas login_attempts sign-in throttle shared by replicas flow_presence · who has a flow open · Architecture component · shared by replicas flow_presence who has a flow open shared by replicas alembic_version · schema revision · Architecture component · Alembic, no model alembic_version schema revision Alembic, no model privacy_notice · the policy text · Architecture component privacy_notice the policy text disabled_engines · engine policy · Architecture component disabled_engines engine policy dbt_ai_texts · AI texts of dbt exports · Architecture component · per flow version dbt_ai_texts AI texts of dbt exports per flow version saved_views · a Viewer configuration · Architecture component · unique (project_id, name) saved_views a Viewer configuration unique (project_id, name) ai_models · models an admin enabled · AI assistant ai_models models an admin enabled ai_chats · a saved conversation · AI assistant ai_chats a saved conversation ai_chat_turns · question, answer, cost · AI assistant ai_chat_turns question, answer, cost ai_spend · daily spend per user · AI assistant · counter, no key ai_spend daily spend per user counter, no key data_contracts · one per datasource · Data contracts · no foreign keys by design · state behind the icon data_contracts one per datasource state behind the icon data_contract_versions · every saved document · Data contracts · no foreign keys by design · unique (datasource, version) data_contract_versions every saved document unique (datasource, version) data_contract_results · one check, one report · Data contracts · no foreign keys by design · notify queue data_contract_results one check, one report notify queue actor_id created_by owner_id user_id group_id user_id group_id project_id project_id project_id flow_id flow_id datasource_id project_id connection_id flow_id chat_id datasource_id Access control Catalog and execution AI assistant Data contracts · no foreign keys by design Legend Backend Database Security Message bus

Access control

  • • user_groups is a pure join table: its primary key is (user_id, group_id), so a membership cannot be duplicated.
  • • permissions grants one capability on one project to either a user or a group; both columns are nullable and exactly one is filled.
  • • groups.name is unique because SSO reconciles IdP groups by name, not by id.
  • • users carries a partial unique index on (oidc_issuer, oidc_subject): the SSO identity anchor, null for local accounts.

Project catalog

  • • projects nests through parent_id, a self-reference not drawn as an arrow; RBAC is checked against the project.
  • • flow_versions keeps immutable definition history, unique on (flow_id, version).
  • • connections and datasources are each unique on (project_id, name), so names only have to be unique inside a project.
  • • A datasource can come from a flow, from a database connection, or from neither: all three columns are nullable.
  • • dbt_ai_texts remembers what the AI wrote for a flow's dbt export, per flow version, so exporting the same version again gives the same files.

Execution

  • • runs.parent_run_id is a self-reference, not drawn: a parent run fans out into child runs.
  • • runs is also the orchestration queue: a PENDING row is claimed by one orchestrator (claimed_by) with a conditional update and kept alive by heartbeat_at.
  • • runs.datasource_id and datasources.snapshot_run_id point at each other, so snapshot_run_id is a plain integer with no foreign key to avoid a circular constraint.
  • • started_at includes queue time; engine_started_at records when the engine actually began, and drives the staleness clock.
  • • pending_blob_deletions deliberately has no foreign key: it must survive the rows whose objects it deletes.

Foreign keys listed, not drawn

  • • Ownership and actor columns all point at users.id: projects.owner_id, flows.owner_id, connections.owner_id, datasources.owner_id, saved_views.owner_id, runs.launched_by, flow_versions.created_by, ai_models.updated_by, privacy_notice.updated_by, disabled_engines.disabled_by.
  • • Scheduling actors: flows.run_scheduled_by, datasources.refresh_scheduled_by. Failure notices: flows.notify_connection_id points at connections.id.
  • • Publish target: runs.publish_project_id points at projects.id. saved_views.project_id points at projects.id like every catalog object.
  • • Self-references: projects.parent_id and runs.parent_run_id.
  • • All 35 foreign keys are nullable except the project_id columns, so deleting a user never cascades away history.

Replicas and contracts

  • • login_attempts and flow_presence moved out of process memory so the gateway can run in several copies; the scheduler sweeps the expired rows.
  • • alembic_version holds the schema's revision: a migration step moves it forward before a release, and a gateway refuses a database behind its code.
  • • data_contracts, data_contract_versions and data_contract_results carry no foreign key on purpose: SQLite does not enforce them in tests, and the rows are removed explicitly when the datasource goes.
  • • data_contract_results.notify is the outbox of the alerts: pending rows are sent by the scheduler, claimed one at a time.