[{"data":1,"prerenderedAt":3519},["ShallowReactive",2],{"doc:\u002Fdocs\u002Foverview":3,"docs-nav":3323},{"id":4,"title":5,"body":6,"description":13,"extension":3315,"meta":3316,"navigation":3318,"path":3319,"seo":3320,"stem":3321,"__hash__":3322},"readme\u002Fdocs\u002Foverview\u002FREADME.md","README",{"type":7,"value":8,"toc":3279},"minimark",[9,20,25,37,49,76,87,95,108,111,115,120,127,134,138,439,442,445,568,575,582,587,590,673,676,701,705,728,731,734,749,760,767,771,774,776,780,784,794,817,977,992,1029,1033,1040,1169,1188,1210,1214,1290,1358,1362,1365,1370,1578,1594,1598,1601,1611,1618,1621,1624,1638,1645,1653,1661,1665,1679,1807,1841,1847,1872,1924,1928,1935,2047,2051,2081,2084,2088,2134,2148,2154,2157,2160,2169,2194,2198,2209,2213,2227,2231,2244,2258,2262,2324,2328,2335,2387,2408,2412,2421,2425,2602,2606,2630,2637,2685,2704,2782,2789,2795,2868,2871,2932,2945,2973,3007,3011,3030,3056,3096,3107,3160,3171,3174,3178,3217,3221,3224,3252,3259,3262,3266,3275],[10,11,13,14],"p",{"align":12},"center","\n  ",[15,16],"img",{"src":17,"width":18,"height":18,"alt":19},"\u002Flogo.svg",96,"Tabularia logo",[21,22,24],"h1",{"id":23},"tabularia","Tabularia",[10,26,27],{},[28,29,33],"a",{"href":30,"rel":31},"https:\u002F\u002Fgithub.com\u002FJumpAsAService\u002FTabularia\u002Factions\u002Fworkflows\u002Fci.yml",[32],"nofollow",[15,34],{"alt":35,"src":36},"CI","https:\u002F\u002Fgithub.com\u002FJumpAsAService\u002FTabularia\u002Factions\u002Fworkflows\u002Fci.yml\u002Fbadge.svg",[10,38,39,43,44,48],{},[40,41,42],"strong",{},"Self-hosted, open-source visual data-preparation platform."," A visual flow editor\non pluggable engines: build a pipeline once, preview it on the whole dataset, and\ndecide ",[45,46,47],"em",{},"later"," where the compute runs.",[10,50,51,54,55,58,59,62,63,66,67,70,71,75],{},[40,52,53],{},"The same flow scales from a laptop to a warehouse without being rewritten."," Start\nwith nothing installed but Docker: transformations stream through memory in-process,\nno cluster, no external service, no per-query bill. When the data outgrows one\nmachine, point the flow at ",[40,56,57],{},"DuckDB"," (same process, spilling to disk), a ",[40,60,61],{},"managed\nClickHouse",", or ",[40,64,65],{},"Google BigQuery"," (serverless, nothing to operate) — the flow\ndefinition does not change by a single node. The parquet stays in your bucket and the\nwarehouse reads it ",[40,68,69],{},"in place",": nothing is copied, nothing streams back through the\nworkers. Storage follows the same curve, from MinIO on the same host to S3, Scaleway or\nGoogle Cloud Storage; deployment from one ",[72,73,74],"code",{},"docker compose up"," to Helm on Kubernetes.",[10,77,78,79,82,83,86],{},"Database sources and destinations, loops, charts, scheduling, cross-flow lineage, an\nAI assistant over your own catalog and an audit trail come in the box. On Kubernetes,\nyour team also ",[40,80,81],{},"writes Python and R in governed environments"," and ",[40,84,85],{},"orchestrates its\nown containers",", as steps of the same flows.",[88,89,90],"blockquote",{},[10,91,92],{},[45,93,94],{},"Tabularia takes its name from the Tabularium, the records office of ancient Rome —\nthe place where the state's tables were kept in order.",[10,96,97,98,101,102,107],{},"Current release: ",[40,99,100],{},"1.3.0 «Catone»"," — see ",[28,103,106],{"href":104,"rel":105},"https:\u002F\u002Fgithub.com\u002FJumpAsAService\u002FTabularia\u002Fblob\u002Fmain\u002FCHANGELOG.md",[32],"CHANGELOG.md",".",[109,110],"hr",{},[21,112,114],{"id":113},"part-i--business","Part I · Business",[116,117,119],"h2",{"id":118},"the-problem","The problem",[10,121,122,123,126],{},"Visual data-prep tools either cost thousands per seat, or quietly work on ",[40,124,125],{},"samples","\nof your data and hope for the best. And once a pipeline exists, moving it to bigger\niron usually means rebuilding it: the tool that was comfortable on a laptop does not\nspeak to the warehouse the company already pays for.",[10,128,129,130,133],{},"Tabularia is an open-source alternative you host yourself: a visual, drag-and-drop\npipeline builder that runs on the ",[40,131,132],{},"entire"," dataset, connects straight to your\ndatabases, keeps a full record of who did what — and lets the same flow run on five\nengines, from an in-process library to a serverless warehouse.",[116,135,137],{"id":136},"what-you-get","What you get",[139,140,141,148,158,171,253,263,273,297,303,313,327,344,358,378,399,405,419,425],"ul",{},[142,143,144,147],"li",{},[40,145,146],{},"Build pipelines visually — no code."," Drag transformations onto a canvas, connect\nthem, and preview the result at any step.",[142,149,150,153,154,157],{},[40,151,152],{},"No sampling, ever — unless you ask for it."," Previews, profiles and charts run on\nthe ",[45,155,156],{},"whole"," dataset through a streaming engine: what you see in the editor is what\nyou ship. Where every read is billed, an optional development sample keeps the\neditor cheap, per node and switchable, and never touches production runs.",[142,159,160,163,164,166,167,107],{},[40,161,162],{},"Compute where it makes sense."," The same flow runs on Polars, DuckDB, chDB, an\nexternal ClickHouse or BigQuery. Design in-process, then hand production to the\nwarehouse: the parquet stays in your bucket, the warehouse reads it ",[40,165,69],{},",\nnothing is copied and nothing streams through the workers. See\n",[28,168,170],{"href":169},"#run-it-on-your-engine--or-on-your-warehouse","Run it on your engine — or on your warehouse",[142,172,173,176,177,180,181,184,185],{},[40,174,175],{},"Explore without building a flow."," The ",[40,178,179],{},"Viewer"," is a read-only BI surface over any\ndatasource: filters, calculated fields, pivots and charts, computed on the whole\ndataset by the engine you choose. Any configuration can be ",[40,182,183],{},"saved as a view"," into a\nfolder, next to flows and datasources. A view stores the configuration and never the\nrows, so reopening it reads today's data — it is a lens, not a copy.",[139,186,187],{},[142,188,189,192,193,197,198,201,202,205,206,210,211,215,216,219,220,224,225,228,229,232,233,232,236,239,240,243,244,247,248,252],{},[40,190,191],{},"Connect to your databases."," Read directly from Postgres, MySQL, MariaDB,\nClickHouse, Trino, Oracle, Microsoft SQL Server and SharePoint (Excel and CSV); write results\nback to a database table, on all of them, or publish\nthem as a reusable datasource. SQL Server's driver is Microsoft's, under Microsoft's\nlicence, so it is not in the published images: you add it to yours\n(",[28,194,196],{"href":195},"\u002Fdocs\u002Fsqlserver","SQL Server","). Files (CSV\u002FExcel\u002FJSON\u002Fparquet) work too, uploaded,\non an ",[40,199,200],{},"SFTP server"," or in an ",[40,203,204],{},"S3 bucket",": a folder and a glob become one table,\nrefreshed on schedule, and a flow can drop its result there as a file (on SFTP the\nserver's host key is checked at every connection) (",[28,207,209],{"href":208},"\u002Fdocs\u002Fsftp","SFTP",",\n",[28,212,214],{"href":213},"\u002Fdocs\u002Fs3-datasource","S3",").\nAnd any ",[40,217,218],{},"HTTP API"," (REST\u002FJSON): page-number, offset, cursor and next-link pagination, OAuth2 client credentials too — the\ncredentials never leave the API's host (",[28,221,223],{"href":222},"\u002Fdocs\u002Fapi-datasource","HTTP APIs","). Inputs\nand outputs share ",[40,226,227],{},"one placeholder language",": dates that move by themselves\n(",[72,230,231],{},"sales_{yesterday:yyyyMMdd}.csv",", ",[72,234,235],{},"{last_refresh}",[72,237,238],{},"{run_id}",") in their own time\nzone, list variables and ",[40,241,242],{},"ranges"," (",[72,245,246],{},"day = today-7 .. today step 1d",": one file or one\ncall per day, a missing day skipped and listed), refreshed as a snapshot or appended\nby key (",[28,249,251],{"href":250},"\u002Fdocs\u002Fplaceholders","placeholders",").",[142,254,255,258,259,262],{},[40,256,257],{},"Ask your data."," An optional AI assistant answers questions in plain language over\nthe datasources you are allowed to read: it finds the table, reads the curated field\ndescriptions, writes and runs the query on your engine, and shows the result table the\nnumbers come from. Conversations are saved and can be reopened, so a question already\nanswered is never paid for twice, and ",[40,260,261],{},"every turn shows what it cost",". Any\nOpenAI-compatible endpoint works; an administrator decides which models may be used,\nand every query lands in the audit log.",[142,264,265,268,269,272],{},[40,266,267],{},"Nothing happens to your data that you cannot preview."," Every node shows its row\nand column count as the preview arrives — no extra query, it is what the preview\nalready returned — and edges can be removed with a click or the keyboard. Repeated\npreviews reuse a per-step cache that is written ",[45,270,271],{},"after"," the answer is on screen, so a\nsecond click is fast without the first one waiting for it.",[142,274,275,278,279,282,283,286,287,290,291,296],{},[40,276,277],{},"Data contracts: bad data does not get published."," A datasource can carry a\ncontract — columns, types and rules on the values — checked every time new data\narrives, ",[45,280,281],{},"before"," it is published. Each rule is either a ",[40,284,285],{},"warning"," (published,\nand reported) or an ",[40,288,289],{},"error"," (blocked: the datasource keeps serving the last good\ndata and the run says which rule broke). The state shows as an icon wherever the\ndatasource appears, down to the source nodes of the flows that read it; a change of\nstate can send an email; the contract exports to the open\n",[28,292,295],{"href":293,"rel":294},"https:\u002F\u002Fbitol-io.github.io\u002Fopen-data-contract-standard\u002F",[32],"ODCS"," format.",[142,298,299,302],{},[40,300,301],{},"Schedule and forget."," Put flows on a schedule (timezone-aware, DST-safe) and let\nthem refresh on their own; a load heatmap shows busy time-bands and collisions\nbefore they bite.",[142,304,305,308,309,252],{},[40,306,307],{},"Write R and Python in a governed environment."," Administrators define the\nenvironments — language version, pinned packages, the groups that may use them,\nvariables and secrets — and Tabularia builds the image. Every script runs as a\nKubernetes Job in its own namespace, with CPU and memory limits, network policies and\na timeout; scripts live in folders with their versions, Python is checked by Ruff as\nyou type, and every run keeps its log. An optional assistant writes and fixes the code\nbeside it without ever seeing a row of data (",[28,310,312],{"href":311},"\u002Fdocs\u002Fscripting","Python and R in flows",[142,314,315,318,319,232,323,252],{},[40,316,317],{},"Orchestrate your containers."," Run the images your team already ships — from Harbor,\nGitHub, GitLab, Docker Hub or your cloud's registry, limited to the repositories an\nadministrator allows each group — with the command, arguments, variables, CPU and\nmemory you set. A container gets its input as parquet and hands back its output, or\njust does its job elsewhere; retries, timeouts, logs and the outcome come with it\n(",[28,320,322],{"href":321},"\u002Fdocs\u002Fcontainer-node","The Container node",[28,324,326],{"href":325},"\u002Fdocs\u002Fcontainer-registries","Container registries",[142,328,329,332,333,336,337,340,341,252],{},[40,330,331],{},"Chain the work."," A ",[40,334,335],{},"flow group"," runs flows, datasource refreshes, saved scripts,\ncontainer images and ",[40,338,339],{},"Pushmail"," emails (datasources sent as Excel files) in steps,\ndrawn as a grid: the cells of a step run in parallel, the steps one after the other,\nand a failed step stops the chain. Build it by drag and drop, run it by hand or on a\nschedule, and watch every cell's state as it runs (",[28,342,339],{"href":343},"\u002Fdocs\u002Fpushmail",[142,345,346,349,350,353,354,357],{},[40,347,348],{},"Full history & governance."," Every run is recorded with downloadable results; an\n",[40,351,352],{},"audit log"," captures logins, flow runs, data exports and permission changes —\n\"who entered, who ran what, which data was downloaded\" — with a 24-hour access chart,\nand can be forwarded to ",[40,355,356],{},"Splunk"," as it happens, nothing skipped when Splunk is down.",[142,359,360,363,364,367,368,377],{},[40,361,362],{},"See how everything connects."," A dedicated ",[40,365,366],{},"Lineage"," view maps datasources and\nflows across the whole workspace: provenance, impact\u002Fblast-radius, staleness and\nbroken references at a glance. Optionally the same lineage is ",[40,369,370,371,376],{},"told to an external\ncatalog in the ",[28,372,375],{"href":373,"rel":374},"https:\u002F\u002Fopenlineage.io",[32],"OpenLineage"," standard"," (Marquez, DataHub,\nAstro…): every run becomes a job with the tables, queries, files and datasources\nit read and wrote, and a flow's dependencies can be exported without running it.",[142,379,380,383,384,387,388,391,392,232,395,398],{},[40,381,382],{},"Leave with your work."," Any flow exports as a ",[40,385,386],{},"dbt project",": one model per output,\nthe shared steps as ",[72,389,390],{},"ref()","-ed models, ",[72,393,394],{},"sources.yml",[72,396,397],{},"schema.yml"," with descriptions and\nthe data contract as dbt tests, seeds for the files it reads. Run it on your ClickHouse,\nwhich reads the source databases live, with dbt-duckdb, or natively in the source\ndatabase — as a complete project or as a folder for the data team's own project.\nEnd-to-end oracles run it with a real dbt and compare it with what Tabularia publishes.",[142,400,401,404],{},[40,402,403],{},"Multi-user from day one."," Password or SSO sign-in with any OIDC provider (Keycloak, Entra ID, Auth0, Okta…; verified against Keycloak)\nwith sessions you can see and close, users & groups, nested projects with inherited\nview \u002F edit \u002F manage \u002F connect permissions. Everyone works in the same place, safely.",[142,406,407,410,411,414,415,418],{},[40,408,409],{},"Connects to the rest of your stack."," API tokens and ",[40,412,413],{},"service accounts"," let\nschedulers and CI run flows and read results; ",[40,416,417],{},"webhooks"," call your systems when a\nrun finishes. Engine limits are tuned from the administration panel, without restarts.",[142,420,421,424],{},[40,422,423],{},"Yours to run."," Self-hosted, AGPL-3.0 open source, no per-seat fees, no data\nleaving your infrastructure. Point it at managed S3 \u002F Postgres \u002F Redis if you prefer.",[142,426,427,430,431,434,435,438],{},[40,428,429],{},"Speaks your language, looks the part."," UI in ",[40,432,433],{},"English, Italian, French, German,\nSpanish",", with ",[40,436,437],{},"Dark, Light, Imperium, Dracula and Monokai"," themes, selectable per user.",[116,440,170],{"id":441},"run-it-on-your-engine--or-on-your-warehouse",[10,443,444],{},"Every flow is stored as a declarative list of operations, decoupled from execution.\nThat is what makes the engine a deployment choice instead of a design constraint:",[446,447,448,470],"table",{},[449,450,451],"thead",{},[452,453,454,458,461,464,467],"tr",{},[455,456,457],"th",{},"Engine",[455,459,460],{},"Where the compute runs",[455,462,463],{},"What you need",[455,465,466],{},"How it is billed",[455,468,469],{},"Choose it when",[471,472,473,497,513,531,550],"tbody",{},[452,474,475,485,488,491,494],{},[476,477,478,481,482],"td",{},[40,479,480],{},"Polars"," ",[45,483,484],{},"(default)",[476,486,487],{},"In the worker process",[476,489,490],{},"Nothing",[476,492,493],{},"Your servers",[476,495,496],{},"Day-to-day work, anything that fits the worker's memory",[452,498,499,503,506,508,510],{},[476,500,501],{},[40,502,57],{},[476,504,505],{},"In the worker process, spilling to disk",[476,507,490],{},[476,509,493],{},[476,511,512],{},"Very large joins and aggregations on one machine",[452,514,515,520,523,526,528],{},[476,516,517],{},[40,518,519],{},"chDB",[476,521,522],{},"In the worker process (embedded ClickHouse)",[476,524,525],{},"The chDB build of the image",[476,527,493],{},[476,529,530],{},"Prototyping the ClickHouse dialect locally",[452,532,533,538,541,544,547],{},[476,534,535],{},[40,536,537],{},"ClickHouse (external)",[476,539,540],{},"On a managed or self-hosted ClickHouse server",[476,542,543],{},"A ClickHouse reachable from the workers",[476,545,546],{},"The server you rent",[476,548,549],{},"Big tables, sub-second aggregations, a warehouse you already run",[452,551,552,556,559,562,565],{},[476,553,554],{},[40,555,65],{},[476,557,558],{},"On BigQuery, serverless",[476,560,561],{},"A GCP project and a bucket on Google Cloud Storage",[476,563,564],{},"Per byte read, no idle cost",[476,566,567],{},"Data already on GCP, elastic scale, zero servers to operate",[10,569,570,571,574],{},"A flow carries ",[40,572,573],{},"two engines",": the development one, used by the editor and manual\nruns, and an optional production one, used by the scheduler and \"Run in production\".\nDesign on Polars with a development sample, schedule on ClickHouse or BigQuery: the\nflow does not change, and every run records the engine it actually ran on. An\nadministrator can restrict which engines the installation allows.",[10,576,577,578,107],{},"The engines agree on one data standard — nulls, casts, joins, aggregates, sort order,\npivots — locked by an oracle test suite that runs the same hand-computed expectations\non all five. The remaining differences are documented, not discovered in production:\n",[28,579,581],{"href":580},"\u002Fdocs\u002Fengines\u002Fengine-differences","docs\u002Fengines\u002Fengine-differences.md",[583,584,586],"h3",{"id":585},"how-far-it-scales-and-what-it-costs-to-get-there","How far it scales, and what it costs to get there",[10,588,589],{},"The flow is a declarative list of operations; the engine is a runtime decision. That\none property is what lets an installation grow without a migration:",[446,591,592,608],{},[449,593,594],{},[452,595,596,599,602,605],{},[455,597,598],{},"Stage",[455,600,601],{},"Compute",[455,603,604],{},"External services",[455,606,607],{},"Typical scale",[471,609,610,626,641,657],{},[452,611,612,617,620,623],{},[476,613,614],{},[40,615,616],{},"Laptop \u002F one server",[476,618,619],{},"Polars, streaming in the worker's memory",[476,621,622],{},"none",[476,624,625],{},"up to what the worker's RAM allows",[452,627,628,633,636,638],{},[476,629,630],{},[40,631,632],{},"One bigger machine",[476,634,635],{},"DuckDB, spilling to local disk",[476,637,622],{},[476,639,640],{},"joins and aggregates past RAM",[452,642,643,648,651,654],{},[476,644,645],{},[40,646,647],{},"Your warehouse",[476,649,650],{},"Managed or self-hosted ClickHouse",[476,652,653],{},"a ClickHouse the workers can reach",[476,655,656],{},"tens of millions of rows, sub-second aggregates",[452,658,659,664,667,670],{},[476,660,661],{},[40,662,663],{},"Serverless",[476,665,666],{},"BigQuery",[476,668,669],{},"a GCP project and a GCS bucket",[476,671,672],{},"elastic, no idle cost",[10,674,675],{},"Nothing above is a rewrite: it is a dropdown on the flow, and a flow can use one engine\nin the editor and another in production. Two mechanisms keep the growth honest:",[139,677,678,688],{},[142,679,680,683,684,687],{},[40,681,682],{},"A development sample, off in production."," Where every read is billed, an optional\nsample keeps the editor cheap. It is per source node, switchable, marked in the\ndefinition, and ",[40,685,686],{},"scheduled runs never sample"," — you cannot ship a sampled result by\naccident.",[142,689,690,693,694,696,697,700],{},[40,691,692],{},"A step cache with a ceiling."," Repeated previews of the same chain reuse the\nprevious step instead of recomputing it. The copy is made ",[45,695,271],{}," the preview has\nanswered, in a separate task, and steps above ",[72,698,699],{},"CACHE__MAX_STEP_ROWS"," are not cached at\nall: a big intermediate result never makes you wait for a copy you did not ask for.",[583,702,704],{"id":703},"managed-clickhouse","Managed ClickHouse",[10,706,707,708,711,712,715,716,719,720,723,724,727],{},"Set ",[72,709,710],{},"CLICKHOUSE_EXTERNAL__HOST"," and the engine appears in the picker. With the ",[72,713,714],{},"s3","\ntransport the server reads and writes the parquet ",[40,717,718],{},"directly on the object storage","\nthrough its ",[72,721,722],{},"s3()"," table function: nothing passes through the workers, and an\naggregation over 25 million rows answers in 0.6 s instead of 18 s, because the parquet\nis written with large row groups the server can skip. With the ",[72,725,726],{},"push"," transport\nthe worker stages the source in a table and streams the result back, which works with\nany server that cannot see the bucket. Validated on Scaleway's managed ClickHouse;\nClickHouse Cloud speaks the same protocol and reads S3-compatible buckets the same way\n(multi-replica services are not handled yet: query cancellation and the performance\npage read one replica).",[10,729,730],{},"Optionally, datasets above a size threshold are copied once into a MergeTree on the\nserver so the Viewer's full scans read local columns instead of the bucket.",[583,732,65],{"id":733},"google-bigquery",[10,735,736,737,740,741,744,745,748],{},"Point the object storage at Google Cloud Storage, give the engine a project and a\nservice account, and every flow can run on BigQuery. Each source parquet becomes a\n",[40,738,739],{},"temporary external table"," of the query (",[72,742,743],{},"gs:\u002F\u002Fbucket\u002Fkey","), so no dataset is\nloaded and no copy is made; the chain of operations is one GoogleSQL query; results\ncome back through the Storage Read API and are written as parquet like everywhere\nelse. Columns are checked with free dry runs; every job is capped by\n",[72,746,747],{},"BIGQUERY__MAXIMUM_BYTES_BILLED"," so a runaway query is refused before it runs.",[10,750,751,752,755,756,759],{},"The editor gets a ",[40,753,754],{},"native step-cache",": intermediate steps are materialised as\nexpiring tables in a ",[72,757,758],{},"tabularia_cache"," dataset by a background task, so the first click\non a node never waits and the next ones read a small native table. Measured on a\n25 M × 50 table with a 100 k development sample: 0.6–1.1 s per click with the cache,\n1.1–2.9 s without.",[10,761,762,763,766],{},"Know the cost model before pointing the Viewer at a wide table: BigQuery bills the\n",[40,764,765],{},"logical"," bytes of the columns a query touches, whatever the file compression. On\nthat same 25 M × 50 table, aggregations read 100 MB–1 GB per query (a fraction of a\ncent); sorting whole rows reads all 50 columns, about 19 GB, eleven cents per click.\nThe per-query cap and the development sample are the two guards; the free tier\ncovers 1 TB a month.",[116,768,770],{"id":769},"who-its-for","Who it's for",[10,772,773],{},"Data & analytics teams that want a self-hosted, governed, no-sampling data-prep tool\nwithout enterprise licensing — and anyone who needs an auditable, multi-user pipeline\nbuilder that can start on one machine and grow onto the warehouse they already have.",[109,775],{},[21,777,779],{"id":778},"part-ii--technical","Part II · Technical",[116,781,783],{"id":782},"architecture","Architecture",[10,785,786],{},[28,787,790],{"href":788,"rel":789},"https:\u002F\u002Ftabularia.org\u002Farchitecture\u002Fruntime-architecture.html",[32],[15,791],{"alt":792,"src":793},"Tabularia runtime architecture","\u002Farchitecture\u002Ftabularia-runtime.png",[10,795,796,802,803,808,809,816],{},[40,797,798],{},[28,799,801],{"href":788,"rel":800},[32],"Open the interactive diagram","\n(pan\u002Fzoom, light\u002Fdark theme, search, guided views, PNG\u002FSVG export). It is generated\nwith ",[28,804,807],{"href":805,"rel":806},"https:\u002F\u002Fgithub.com\u002Ftt-a1i\u002Farchify",[32],"Archify"," from\n",[28,810,813],{"href":811,"rel":812},"https:\u002F\u002Fgithub.com\u002FJumpAsAService\u002FTabularia\u002Fblob\u002Fmain\u002Fdocs\u002Farchitecture\u002Fruntime.architecture.json",[32],[72,814,815],{},"docs\u002Farchitecture\u002Fruntime.architecture.json",";\nevery node cites the source files that implement it, verified against the commit\npinned in the spec.",[446,818,819,829],{},[449,820,821],{},[452,822,823,826],{},[455,824,825],{},"Runtime component",[455,827,828],{},"Role",[471,830,831,839,851,867,879,887,895,911,919,939,953,965],{},[452,832,833,836],{},[476,834,835],{},"Web UI",[476,837,838],{},"Nuxt 3 app, calls the gateway only (JWT)",[452,840,841,844],{},[476,842,843],{},"Gateway",[476,845,846,847,850],{},"FastAPI control plane: auth, RBAC, audit; proxies to the engine after the permission check. Stateless between requests, so it runs in as many copies as you like (",[72,848,849],{},"APP__ROLE=api",")",[452,852,853,856],{},[476,854,855],{},"Orchestrator",[476,857,858,859,862,863,866],{},"the same image in another role (",[72,860,861],{},"APP__ROLE=orchestrator","): fires the schedules and runs the flows. Each run is a row in Postgres that any orchestrator claims with a conditional update, so copies never fire a schedule twice. ",[72,864,865],{},"APP__ROLE=all"," (the default) does both jobs in one process",[452,868,869,872],{},[476,870,871],{},"PostgreSQL",[476,873,874,875,850],{},"control-plane metadata (users, groups, projects, versioned flows, connections, runs, schedules, audit); its schema evolves through Alembic revisions (",[28,876,878],{"href":877},"\u002Fdocs\u002Fdatabase-migrations","database migrations",[452,880,881,884],{},[476,882,883],{},"Engine API",[476,885,886],{},"FastAPI on the private network: turns every preview and run into a Celery task",[452,888,889,892],{},[476,890,891],{},"Valkey",[476,893,894],{},"Celery broker, step-cache index, preview slots",[452,896,897,900],{},[476,898,899],{},"Run worker \u002F preview worker",[476,901,902,903,906,907,910],{},"Celery workers on two queues (",[72,904,905],{},"celery",": runs, DB ingest, export · ",[72,908,909],{},"preview",": interactive previews); in-process engines (Polars, DuckDB, chDB) and the clients of the external ones",[452,912,913,916],{},[476,914,915],{},"Celery beat",[476,917,918],{},"cache eviction and storage statistics",[452,920,921,924],{},[476,922,923],{},"Object storage",[476,925,926,927,930,931,934,935,938],{},"one bucket, all parquet: ",[72,928,929],{},"datasets\u002F"," snapshots, ",[72,932,933],{},"cache\u002F"," steps, ",[72,936,937],{},"out\u002F"," results. MinIO locally; any S3 (Scaleway, AWS…) or Google Cloud Storage through its S3 API in the cloud",[452,940,941,947],{},[476,942,943,944],{},"External engines ",[45,945,946],{},"(optional)",[476,948,949,950,952],{},"a managed ClickHouse reading and writing the parquet through ",[72,951,722],{},"; Google BigQuery reading it as temporary external tables over GCS",[452,954,955,958],{},[476,956,957],{},"External databases",[476,959,960,961,964],{},"sources streamed in Arrow batches into parquet snapshots (Postgres through ADBC, Oracle through python-oracledb in Thin mode, SQL Server through Microsoft's mssql-python, which ",[28,962,963],{"href":195},"you add to your image","; no other vendor client), whole row groups whatever the driver's batch; Output nodes can write tables back",[452,966,967,970],{},[476,968,969],{},"VictoriaMetrics + Grafana",[476,971,972,973,976],{},"scrape the engine ",[72,974,975],{},"\u002Fmetrics",", celery-exporter, cAdvisor and node-exporter",[10,978,979,980,983,984,987,988,991],{},"The ",[40,981,982],{},"gateway"," (control plane) is the only public ingress: it owns the metadata\nPostgres and enforces auth + RBAC on every call before proxying to the internal\n",[40,985,986],{},"engine"," (data plane), which stays stateless (Valkey + object storage only). Nothing\nthat outlives a request lives in a gateway process — the run queue, the login throttle\nand \"who has this flow open\" are tables — which is what lets the API tier scale out and\nthe ",[40,989,990],{},"orchestrator"," tier scale separately: one process at ~100 requests\u002Fs, three copies\nserved 500 concurrent users at a 0.12 s p95 on one laptop.",[10,993,994,995,1002,1003,1010,1011,232,1014,232,1017,210,1019,232,1022,82,1025,1028],{},"That metadata Postgres has a diagram of its own:\n",[40,996,997],{},[28,998,1001],{"href":999,"rel":1000},"https:\u002F\u002Ftabularia.org\u002Farchitecture\u002Fmetadata-schema.html",[32],"Open the metadata database schema","\n— the 28 tables and 35 foreign keys of release 1.1.0, read back from a live database\nrather than transcribed from the models, and generated with Archify from\n",[28,1004,1007],{"href":1005,"rel":1006},"https:\u002F\u002Fgithub.com\u002FJumpAsAService\u002FTabularia\u002Fblob\u002Fmain\u002Fdocs\u002Farchitecture\u002Fmetadata-schema.architecture.json",[32],[72,1008,1009],{},"docs\u002Farchitecture\u002Fmetadata-schema.architecture.json",".\nSix tables added since are not drawn yet: ",[72,1012,1013],{},"user_sessions",[72,1015,1016],{},"api_tokens",[72,1018,417],{},[72,1020,1021],{},"webhook_deliveries",[72,1023,1024],{},"engine_settings",[72,1026,1027],{},"siem_destinations"," (schema revisions\n0002–0005, now 34 tables and 40 foreign keys).",[116,1030,1032],{"id":1031},"declarative-ir--pluggable-engines","Declarative IR & pluggable engines",[10,1034,1035,1036,1039],{},"Flows are stored as a ",[40,1037,1038],{},"declarative IR"," — a JSON list of typed operations — fully\ndecoupled from execution. Adding or swapping an engine touches neither the routes, the\nworkers, nor saved flows. Five engines are registered:",[446,1041,1042,1054],{},[449,1043,1044],{},[452,1045,1046,1048,1051],{},[455,1047,457],{},[455,1049,1050],{},"id",[455,1052,1053],{},"Notes",[471,1055,1056,1074,1088,1106,1136],{},[452,1057,1058,1062,1067],{},[476,1059,1060],{},[40,1061,480],{},[476,1063,1064],{},[72,1065,1066],{},"polars",[476,1068,1069,1070,1073],{},"In-process, lazy, streaming. ",[40,1071,1072],{},"Default","; full operation coverage.",[452,1075,1076,1080,1085],{},[476,1077,1078],{},[40,1079,57],{},[476,1081,1082],{},[72,1083,1084],{},"duckdb",[476,1086,1087],{},"Out-of-core SQL (spills to disk) for very large joins\u002Faggregations. Full operation coverage.",[452,1089,1090,1095,1100],{},[476,1091,1092],{},[40,1093,1094],{},"chDB (ClickHouse)",[476,1096,1097],{},[72,1098,1099],{},"chdb",[476,1101,1102,1103,107],{},"Out-of-core SQL with the ClickHouse dialect, embedded. Full coverage except ",[72,1104,1105],{},"foreach",[452,1107,1108,1112,1117],{},[476,1109,1110],{},[40,1111,537],{},[476,1113,1114],{},[72,1115,1116],{},"clickhouse",[476,1118,1119,1122,1123,1126,1127,1129,1130,1132,1133,1135],{},[45,1120,1121],{},"Optional."," Same dialect and ops as chDB, executed on a ",[40,1124,1125],{},"remote ClickHouse server",". Enabled by ",[72,1128,710],{},". Transport ",[72,1131,714],{}," (the server reads\u002Fwrites parquet directly on the object storage) or ",[72,1134,726],{}," (staging table + streamed result).",[452,1137,1138,1142,1147],{},[476,1139,1140],{},[40,1141,666],{},[476,1143,1144],{},[72,1145,1146],{},"bigquery",[476,1148,1149,1151,1152,1155,1156,1159,1160,82,1162,1165,1166,107],{},[45,1150,1121],{}," GoogleSQL over ",[40,1153,1154],{},"temporary external tables"," on GCS; results through the Storage Read API. Enabled by ",[72,1157,1158],{},"BIGQUERY__PROJECT"," plus a service-account key. Full coverage except ",[72,1161,1105],{},[72,1163,1164],{},"median"," inside ",[72,1167,1168],{},"pivot",[10,1170,1171,1172,1175,1176,1179,1180,1183,1184,1187],{},"Each engine is a registry of per-operation implementations. DuckDB and chDB are\nguarded imports — absent packages simply mark the engine unavailable without breaking\nPolars; the external engines are listed but unavailable until configured. chDB is\n",[40,1173,1174],{},"fork-unsafe",", so it is imported ",[45,1177,1178],{},"lazily inside the Celery child"," (never in the\nprefork parent) to avoid inherited native-thread deadlocks. Users pick a ",[40,1181,1182],{},"preferred\nengine"," in settings (default for the Viewer and new flows); each flow persists the\nengine it was built with, so opening a non-preferred flow is regression-safe. Each\nflow also carries an optional ",[40,1185,1186],{},"production engine",": the editor (previews, manual\nruns) uses the development engine, while scheduled runs and \"Run in production\" use\nthe production engine. Every run records the engine it actually ran on.",[10,1189,1190,1191,1194,1195,1198,1199,1202,1203,1206,1207,107],{},"An administrator can narrow that choice for the whole installation. ",[40,1192,1193],{},"Admin → Engines","\ndecides which engines a flow may be built on. Disabling an engine stops it from being\n",[45,1196,1197],{},"chosen"," — at creation, when changing a flow's engine, and as a production engine — but\ndeliberately does ",[40,1200,1201],{},"not"," stop the flows already using it: a switch in an admin panel\nshould never halt a scheduled DAG the moment it is pressed. The panel reports how many\nflows still run on each engine — that list is the migration backlog. At least one\nengine always stays allowed, and the picker keeps ",[45,1204,1205],{},"not allowed here"," apart from ",[45,1208,1209],{},"not\nconfigured",[583,1211,1213],{"id":1212},"external-engines-in-detail","External engines in detail",[10,1215,1216,1218,1219,232,1221,1224,1225,232,1228,232,1231,232,1234,232,1237,1240,1241,1243,1244,1246,1247,1250,1251,1254,1255,1258,1259,1262,1263,1266,1267,1270,1271,1274,1275,1278,1279,1282,1283,1285,1286,1289],{},[40,1217,537],{}," — ",[72,1220,710],{},[72,1222,1223],{},"__PORT"," (8443 with TLS on\nmanaged services), ",[72,1226,1227],{},"__USERNAME",[72,1229,1230],{},"__PASSWORD",[72,1232,1233],{},"__DATABASE",[72,1235,1236],{},"__SECURE",[72,1238,1239],{},"__TRANSPORT","\n(",[72,1242,714],{}," | ",[72,1245,726],{},"), ",[72,1248,1249],{},"__S3_ENDPOINT"," (the storage endpoint ",[45,1252,1253],{},"as the server sees it",") and,\nrecommended, ",[72,1256,1257],{},"__S3_NAMED_COLLECTION",": a named collection holding the bucket credentials\non the server, so keys never travel inside a query nor land in ",[72,1260,1261],{},"query_log",". The engine\nraises ",[72,1264,1265],{},"max_threads"," on scan-heavy queries to hide object-storage latency, never on\n",[72,1268,1269],{},"LIMIT"," queries; ",[72,1272,1273],{},"__MATERIALIZE_MIN_ROWS"," turns on the MergeTree copy for the Viewer.\nInteractive previews carry a ",[72,1276,1277],{},"log_comment"," tag: when a newer preview supersedes one\nstill running, the server is told to ",[72,1280,1281],{},"KILL"," it. Preview timings and the server's own\n",[72,1284,1261],{}," are shown on the admin ",[40,1287,1288],{},"Performance"," page.",[10,1291,1292,1218,1294,232,1296,1299,1300,1303,1304,1307,1308,1310,1311,1314,1315,1317,1318,1321,1322,1325,1326,1329,1330,232,1333,1336,1337,1340,1341,1344,1345,232,1348,1344,1351,1354,1355,252],{},[40,1293,666],{},[72,1295,1158],{},[72,1297,1298],{},"BIGQUERY__CREDENTIALS_FILE"," (path of the\nservice-account JSON key inside the container) or ",[72,1301,1302],{},"BIGQUERY__CREDENTIALS_B64"," (the\nsame key, base64 on one line), ",[72,1305,1306],{},"BIGQUERY__LOCATION"," (empty = the bucket's region),\n",[72,1309,747],{}," (default 20 GiB) and ",[72,1312,1313],{},"BIGQUERY__CACHE_DATASET","\n(default ",[72,1316,758],{},", empty disables the step-cache). The service account needs\n",[45,1319,1320],{},"BigQuery Job User"," on the project, ",[45,1323,1324],{},"BigQuery Data Editor"," for the cache dataset, and\nread access on the bucket; the object storage must be Google Cloud Storage\n(",[72,1327,1328],{},"STORAGE__ENDPOINT=https:\u002F\u002Fstorage.googleapis.com"," with HMAC keys). Column names\nBigQuery cannot represent (",[72,1331,1332],{},"Ragione Sociale",[72,1334,1335],{},"Importo (€)",") are mapped to safe names\ninside the query and restored in the results; ",[72,1338,1339],{},"compute"," expressions are transpiled from\nthe common dialect with sqlglot (",[72,1342,1343],{},"%"," → ",[72,1346,1347],{},"MOD",[72,1349,1350],{},"strftime",[72,1352,1353],{},"FORMAT_DATE","). Jobs are\nlabelled with the preview tag so a superseded preview is cancelled on the service.\nVerified against a real project: the cross-engine oracle suite passes on BigQuery\n(",[72,1356,1357],{},"BIGQUERY_LIVE=1 pytest -k bigquery",[116,1359,1361],{"id":1360},"operations","Operations",[10,1363,1364],{},"~18 transforms plus source \u002F output \u002F control nodes, all engine-agnostic in the IR:",[10,1366,1367],{},[72,1368,1369],{},"select · drop · rename · cast · compute · sql · filter · sort · limit · unique · fill_null · drop_nulls · group_by · pivot · unpivot · join · union · foreach",[139,1371,1372,1383,1401,1478,1492,1502,1526,1538],{},[142,1373,1374,1378,1379,1382],{},[40,1375,1376],{},[72,1377,1105],{}," is a loop container: it iterates its body over a driver table with\n",[72,1380,1381],{},"{{placeholder}}"," substitution, appending results with bounded memory.",[142,1384,1385,1388,1389,1392,1393,1396,1397,1400],{},[40,1386,1387],{},"Upstream filters"," per source node: AND-ed ",[72,1390,1391],{},"{column, operator, value}"," conditions set in\nthe editor on the input datasource and saved with the flow. They become plain ",[72,1394,1395],{},"filter","\noperations injected right after the source is read, before anything else (development\nsample included), in ",[45,1398,1399],{},"every"," mode — editor previews, scheduled runs, \"Run in production\"\nand the dbt export — so a flow can read only the slice it needs from a large table.",[142,1402,1403,1406,1407,1410,1411,1414,1415,1418,1419,1414,1422,1425,1426,1414,1429,1414,1432,1414,1435,1438,1439,1414,1442,1445,1446,1448,1449,1452,1453,1456,1457,1414,1460,1463,1464,1467,1468,1470,1471,1474,1475,107],{},[40,1404,1405],{},"Cross-engine data standard."," The same flow must give the same data on every engine,\nso the engines follow one explicit semantics, SQL-like, locked by an oracle test suite\n(",[72,1408,1409],{},"backend\u002Ftests\u002Ftest_data_correctness.py",", hand-computed expectations run on Polars,\nDuckDB, chDB, external ClickHouse and BigQuery): comparisons with NULL are false\n(",[72,1412,1413],{},"ne","\u002F",[72,1416,1417],{},"not_in"," drop NULLs); aggregates ignore NULLs, ",[72,1420,1421],{},"count",[72,1423,1424],{},"n_unique"," never count NULL,\n",[72,1427,1428],{},"sum",[72,1430,1431],{},"mean",[72,1433,1434],{},"min",[72,1436,1437],{},"max"," of an all-NULL group are NULL, ",[72,1440,1441],{},"std",[72,1443,1444],{},"var"," are sample (n-1),\n",[72,1447,1164],{}," interpolates; sort puts NULLs ",[40,1450,1451],{},"last"," in both directions; failed casts give\nNULL (never an error), text is trimmed before parsing, text→int accepts integer literals\nonly, number→int truncates; joins never match NULL keys, ",[72,1454,1455],{},"on"," keys are one coalesced\ncolumn, ",[72,1458,1459],{},"left_on",[72,1461,1462],{},"right_on"," keep both key columns, a non-key homonym from the right\ngets ",[72,1465,1466],{},"_right","; ",[72,1469,1339],{}," overwrites an existing column in place; integer sums are\nint64; datetimes are ",[40,1472,1473],{},"naive UTC instants"," everywhere. The differences that remain\nbetween engines are listed in ",[28,1476,1477],{"href":580},"engine-differences",[142,1479,1480,1483,1484,1487,1488,1491],{},[40,1481,1482],{},"Development sampling"," per source node: \"first N rows\" or \"random p%\" set in the\neditor and saved with the flow. It only affects previews and editor runs: the gateway\nresolver injects it solely in development, never for scheduled runs or \"Run in\nproduction\", and strips any such marked operation from production launches as defence\nin depth. With ",[72,1485,1486],{},"PREVIEW__DEFAULT_SAMPLE_ROWS"," set, every source node without a sample\nof its own gets an ",[40,1489,1490],{},"automatic"," \"first N rows\" sample, shown as a badge on the node\nand switchable per node (\"off (all rows)\") — the right default where every read is\nbilled. Sample only the big table: joining two sampled sources loses most matches.",[142,1493,1494,1497,1498,1501],{},[40,1495,1496],{},"Field descriptions"," on datasources: a hand-curated ",[72,1499,1500],{},"{column: text}"," map, kept\nseparately from the inferred schema so it survives refreshes.",[142,1503,1504,1512,1513,1516,1517,1520,1521,1414,1523,1525],{},[40,1505,1506,1508,1509],{},[72,1507,1168],{}," \u002F ",[72,1510,1511],{},"unpivot"," follow one cross-engine standard: pivot columns are named by the\nvalue as text (",[72,1514,1515],{},"null"," for NULL, ",[72,1518,1519],{},"2024_web"," for multi-column keys, existing combinations\nonly, text-ordered), missing or all-NULL groups are NULL (",[72,1522,1421],{},[72,1524,1424],{}," → 0), and\nunpivot keeps only the index columns with the value cast to the common supertype.",[142,1527,1528,1533,1534,1537],{},[40,1529,1530],{},[72,1531,1532],{},"sql"," runs engine-native SQL against the node input (",[72,1535,1536],{},"FROM input","), with a\nguardrail floor that blocks filesystem \u002F URL \u002F executable access.",[142,1539,1540,1543,1544,1547,1548,1551,1552,1556,1557,1560,1561,82,1564,1567,1568,1570,1571,82,1574,1577],{},[40,1541,1542],{},"Nodes",": ",[72,1545,1546],{},"source"," (any datasource: an uploaded file, a database table or query, an\nSFTP or S3 folder, a SharePoint path, an HTTP API call), ",[72,1549,1550],{},"output"," (write to a DB table,\npublish a datasource — optionally with a copy on an external S3 bucket, see\n",[28,1553,1555],{"href":1554},"#publishing-to-an-external-bucket","Publishing to an external bucket"," —, drop a file on\nan SFTP server or an S3 bucket, or send it by email; append\u002Freplace + post-SQL for\ntables; names built with ",[28,1558,251],{"href":1559},"#dynamic-inputs-and-outputs","),\n",[72,1562,1563],{},"script",[72,1565,1566],{},"container"," (code and images run as Kubernetes Jobs). Refreshing\ndatasources and running other flows in order is the job of a ",[40,1569,335],{}," (steps of\nflows, refreshes, scripts, images and Pushmail emails); the ",[72,1572,1573],{},"refresh",[72,1575,1576],{},"runflow","\nnodes of flows saved before still run, but are no longer offered.",[10,1579,1580,1581,1584,1585,1587,1588,1590,1591,1593],{},"Every node's output is ",[40,1582,1583],{},"content-addressed and cached",": editing the last step of a\n10-step flow recomputes one step, not ten. The copy is written ",[45,1586,271],{}," the preview\nanswers, by a separate task, and only for steps up to ",[72,1589,699],{}," (1M by\ndefault): a step as large as the source is recomputed from the original parquet\ninstead of being copied. In-process engines and ClickHouse keep the steps as parquet\nunder ",[72,1592,933],{},"; BigQuery keeps them as expiring native tables. Cache entries evict by\nTTL.",[116,1595,1597],{"id":1596},"storage-layout","Storage layout",[10,1599,1600],{},"All parquet, on any S3-compatible object storage (MinIO by default) or on Google Cloud\nStorage through its S3 API — the two things GCS does differently (no batch delete,\nno chunked request checksums) are handled by the storage layer:",[1602,1603,1608],"pre",{"className":1604,"code":1606,"language":1607},[1605],"language-text","raw\u002F       ingested files, as uploaded\ndatasets\u002F  normalized parquet datasources\ncache\u002F     content-addressed step outputs\nout\u002F       run results, downloadable as CSV\u002FExcel\n","text",[72,1609,1606],{"__ignoreMap":1610},"",[10,1612,1613,1614,1617],{},"Parquet files are written with large row groups (",[72,1615,1616],{},"INGEST__PARQUET_ROW_GROUP_ROWS",",\ndefault 1 M) and, when a datasource declares sort keys, ordered on them: that is what\nlets an external engine skip most of a file on a filtered query instead of reading it.",[116,1619,1555],{"id":1620},"publishing-to-an-external-bucket",[10,1622,1623],{},"An Output node that publishes a datasource can also drop a copy of the same parquet\non a bucket Tabularia doesn't own — the one another team already reads from. It is\nthe integration path for the common case where nobody wants a data-prep tool writing\ninto their database, and it needs no orchestrator: the consumer polls a path.",[10,1625,1626,1627,1630,1631,1633,1634,1637],{},"By default the copy lands at ",[40,1628,1629],{},"the same key, always parquet, overwritten on every run",",\nso whoever reads it agrees on one path once and never has to discover a new filename.\nThe key may also carry ",[28,1632,251],{"href":1559}," — one object per\nday with ",[72,1635,1636],{},"exports\u002Fsales\u002F{today:yyyy\u002FMM\u002Fdd}.parquet",", resolved in the node's time zone.\nScheduled runs, «Run now» and runs started from the editor all write it.\nThe overwrite is a single object and becomes visible only when the upload completes:\na reader gets either the previous file or the new one in full, never a mixture.",[10,1639,1640,1641,1644],{},"It is ",[40,1642,1643],{},"best effort by design",". The datasource is the result; the copy is a delivery\ndownstream of it. A copy that fails — or cannot be made, like a key whose placeholder is\nwrong — leaves the run successful and the datasource published, and the error is\nrecorded on the run and returned by the API.",[88,1646,1647],{},[10,1648,1649,1652],{},[40,1650,1651],{},"No screen shows that error yet."," Until that is wired up, treat a successful run\nas saying nothing either way about the copy.",[88,1654,1655],{},[10,1656,1657,1660],{},[40,1658,1659],{},"Use a different bucket from the one Tabularia itself runs on."," A copy written\ninto that bucket is not checked against the objects already there: aim it at a key\nanother datasource is using and you overwrite that datasource's snapshot, outside the\npermission model.",[116,1662,1664],{"id":1663},"dynamic-inputs-and-outputs","Dynamic inputs and outputs",[10,1666,1667,1668,1671,1672,1675,1676,107],{},"Every name Tabularia builds — the folder and files a datasource reads, the call an API\ndatasource makes, the file, key or email an Output writes — speaks ",[40,1669,1670],{},"one placeholder\nlanguage",": dates that move by themselves, formatted the way the other side wants them,\nin a time zone of your choice. Nothing is evaluated as code; a wrong placeholder is\nrefused when you save, and every form previews the resolved value as you type\n(«today: ",[72,1673,1674],{},"sales_17102026.csv","»). The full reference is in\n",[28,1677,1678],{"href":250},"docs\u002Fplaceholders.md",[446,1680,1681,1691],{},[449,1682,1683],{},[452,1684,1685,1688],{},[455,1686,1687],{},"Write",[455,1689,1690],{},"Get (on 18 October 2026)",[471,1692,1693,1724,1746,1770,1786,1795],{},[452,1694,1695,1710],{},[476,1696,1697,1700,1701,1700,1704,1700,1707],{},[72,1698,1699],{},"{today}"," · ",[72,1702,1703],{},"{yesterday}",[72,1705,1706],{},"{today-7}",[72,1708,1709],{},"{today-1m}",[476,1711,1712,1700,1715,1700,1718,1700,1721],{},[72,1713,1714],{},"2026-10-18",[72,1716,1717],{},"2026-10-17",[72,1719,1720],{},"2026-10-11",[72,1722,1723],{},"2026-09-18",[452,1725,1726,1737],{},[476,1727,1728,1700,1731,1700,1734],{},[72,1729,1730],{},"{start_of_month}",[72,1732,1733],{},"{start_of_week}",[72,1735,1736],{},"{now-2h}",[476,1738,1739,1700,1742,1745],{},[72,1740,1741],{},"2026-10-01",[72,1743,1744],{},"2026-10-12"," · two hours ago, a date-time",[452,1747,1748,1759],{},[476,1749,1750,1700,1753,1700,1756],{},[72,1751,1752],{},"{yesterday:ddMMyyyy}",[72,1754,1755],{},"{today:yyyy\u002FMM\u002Fdd}",[72,1757,1758],{},"{now:yyyy-MM-dd'T'HH:mm}",[476,1760,1761,1700,1764,1700,1767],{},[72,1762,1763],{},"17102026",[72,1765,1766],{},"2026\u002F10\u002F18",[72,1768,1769],{},"2026-10-18T09:30",[452,1771,1772,1783],{},[476,1773,1774,1700,1777,1700,1780],{},[72,1775,1776],{},"{today:%Y%m%d}",[72,1778,1779],{},"{now:epoch}",[72,1781,1782],{},"{now:epoch_ms}",[476,1784,1785],{},"strftime, and seconds or milliseconds since 1970",[452,1787,1788,1792],{},[476,1789,1790],{},[72,1791,235],{},[476,1793,1794],{},"inputs: when the last successful refresh started (to fetch only what is new)",[452,1796,1797,1804],{},[476,1798,1799,1700,1801],{},[72,1800,238],{},[72,1802,1803],{},"{flow}",[476,1805,1806],{},"outputs: the run's number and the flow's name",[10,1808,1809,1812,1813,1816,1817,1820,1821,1824,1825,1828,1829,1832,1833,1836,1837,1840],{},[40,1810,1811],{},"Inputs."," A daily file on SFTP: folder ",[72,1814,1815],{},"\u002Fin\u002F{today:yyyy}\u002F{today:MM}",", files\n",[72,1818,1819],{},"sales_{yesterday:ddMMyyyy}.csv",". A ",[40,1822,1823],{},"list variable"," makes one read or call per value —\n",[72,1826,1827],{},"symbol = AAPL, MSFT, ENI.MI"," with the path ",[72,1830,1831],{},"\u002Fquotes\u002F{symbol}"," — and every row says which\nvalue it came from (",[72,1834,1835],{},"_param_symbol","). A ",[40,1838,1839],{},"range variable"," does the same over time:",[1602,1842,1845],{"className":1843,"code":1844,"language":1607,"meta":1610},[1605],"day = today-7 .. today step 1d          # the 7 days before today (today excluded)\nfiles: exports\u002F{day:yyyy\u002FMM\u002Fdd}\u002F*.parquet\n",[72,1846,1844],{"__ignoreMap":1610},[10,1848,1849,1850,82,1853,1856,1857,82,1860,1863,1864,1867,1868,1871],{},"reads one folder per day, stacks them with ",[72,1851,1852],{},"_file",[72,1854,1855],{},"_param_day",", skips and lists a\nday with no file, and fails only when the whole range has none. On an API the same range\nmakes one call per window, with ",[72,1858,1859],{},"{day}",[72,1861,1862],{},"{day_end}"," for APIs that take a from\u002Fto and\ncap how many days one call may cover (",[72,1865,1866],{},"step 7d"," over a year = 53 calls). Combined with\n",[40,1869,1870],{},"append by key"," — rows whose key already exists are replaced, new keys added — it is\nthe classic nightly incremental import: «every night re-read the last three days».",[10,1873,1874,1877,1878,1881,1882,1885,1886,1889,1890,1893,1894,1897,1898,1901,1902,1905,1906,1909,1910,1912,1913,232,1916,232,1919,82,1921,1923],{},[40,1875,1876],{},"Outputs."," An SFTP folder ",[72,1879,1880],{},"\u002Fout\u002F{today:yyyy}\u002F{today:MM}"," and file\n",[72,1883,1884],{},"report_{today:yyyyMMdd}_{run_id}"," write ",[72,1887,1888],{},"report_20261018_381.csv","; an S3 key\n",[72,1891,1892],{},"exports\u002F{flow}\u002F{now:yyyy-MM-dd'T'HHmm}.parquet"," in ",[72,1895,1896],{},"Europe\u002FRome","; an email with subject\n",[72,1899,1900],{},"Daily sales {yesterday:dd\u002FMM\u002Fyyyy}"," and attachment ",[72,1903,1904],{},"sales_{yesterday:yyyyMMdd}",". A\nresolved name can never leave its folder (",[72,1907,1908],{},"..",", a ",[72,1911,1414],{}," coming from a value, control\ncharacters and over-long names are refused), and outputs written before this language\nexisted keep their exact names: ",[72,1914,1915],{},"{date}",[72,1917,1918],{},"{datetime}",[72,1920,238],{},[72,1922,1803],{}," mean what\nthey always meant, and literal braces stored earlier were escaped by a migration.",[116,1925,1927],{"id":1926},"auth-rbac--audit","Auth, RBAC & audit",[10,1929,1930,1931,1934],{},"Administrators are granted in two ways: a personal flag, or membership of a group\nmarked as an ",[40,1932,1933],{},"administrators group"," — every member is an administrator for as long as\nthey belong to it, which also means the right can be granted from an identity provider\nthrough SSO group mapping. Both paths are audited, and no administrator can remove\ntheir own last route to administration: demoting or deactivating yourself, demoting or\ndeleting your only admin group, or leaving it, is refused rather than silently\nperformed.",[139,1936,1937,1951,1967,1977,1987,2001,2007],{},[142,1938,1939,1942,1943,1946,1947,1950],{},[40,1940,1941],{},"Sessions"," that can be closed: every sign-in (password or SSO) is a row the\ngateway checks on each request, in the same query as the user. The browser keeps it\nin an ",[72,1944,1945],{},"HttpOnly"," cookie no script can read; sign-out really ends it; people see and\nclose their own sessions, administrators anyone's from the Audit page; a password\nchange or a disabled account closes them; an optional idle timeout\n(",[72,1948,1949],{},"AUTH__SESSION_IDLE_MINUTES",") signs an unattended page out after a warning.",[142,1952,1953,243,1956,1960,1961,1963,1964,1966],{},[40,1954,1955],{},"An API for integrations",[28,1957,1959],{"href":1958},"\u002Fdocs\u002Fapi","docs\u002Fapi.md","): personal API tokens and\n",[40,1962,413],{}," for schedulers and CI, read-only or read-write, always\nexpiring, revocable, recorded in the audit log; ",[40,1965,417],{}," that call another\nsystem when a flow or a datasource refresh finishes, signed with HMAC and retried.",[142,1968,1969,243,1972,1976],{},[40,1970,1971],{},"Settings from the panel",[28,1973,1975],{"href":1974},"\u002Fdocs\u002Fengine-settings","docs\u002Fengine-settings.md","):\nadministrators change about a hundred settings of the gateway, the Kubernetes features\nand the engine (limits, waits, retention, the assistant's budget, the caps of the data\nsources) from Administration → Settings, in a box per component. Most apply within\nseconds on every process; the ones read only at start are marked, and the panel\nrestarts their pods one at a time on Kubernetes. Addresses, credentials and the\nswitches tied to the cluster stay in the environment; every change is checked and\naudited.",[142,1978,1979,243,1982,1986],{},[40,1980,1981],{},"Audit log to Splunk",[28,1983,1985],{"href":1984},"\u002Fdocs\u002Fsiem-splunk","docs\u002Fsiem-splunk.md","): every audit entry\nforwarded to the Splunk HTTP Event Collector within seconds, nothing skipped when\nSplunk is down, configured and tested from Administration → SIEM.",[142,1988,1989,1992,1993,1996,1997,2000],{},[40,1990,1991],{},"RBAC",": users & groups, nested projects, inherited ",[40,1994,1995],{},"view \u002F edit \u002F manage \u002F\nconnect"," permissions; access resolves to a set of ",[72,1998,1999],{},"readable_project_ids"," applied on\nevery query. Object-level permissions extend to connections and datasources on the\ndata plane.",[142,2002,2003,2006],{},[40,2004,2005],{},"Audit log",": append-only, with text snapshots that survive rename\u002Fdelete. Captures\nlogins, CRUD, flow runs, data exports, and permission changes; admin tab with a\n24-hour access-activity chart. Secrets (DB passwords) are encrypted at rest (Fernet).",[142,2008,2009,481,2012,2014,2015,232,2018,2021,2022,2025,2026,2029,2030,2033,2034,2040,2041,107],{},[40,2010,2011],{},"SSO \u002F OIDC",[45,2013,946],{},": sign in against Keycloak, Microsoft Entra ID (MSAL),\nAuth0 or Okta with the authorization-code flow (PKCE, ",[72,2016,2017],{},"state",[72,2019,2020],{},"nonce",", id_token\nvalidated against the IdP JWKS). Users are provisioned on first login and the IdP's\n",[72,2023,2024],{},"groups"," claim is reconciled onto Tabularia groups ",[40,2027,2028],{},"by name",". Off until\n",[72,2031,2032],{},"OIDC__ISSUER"," is set; local login always stays available as break-glass. See\n",[28,2035,2037],{"href":2036},"\u002Fdocs\u002Fdesign\u002Fsso-group-mapping",[72,2038,2039],{},"docs\u002Fdesign\u002Fsso-group-mapping.md"," and the\nrunnable Keycloak example in ",[28,2042,2044],{"href":2043},"\u002Fdocs\u002Fexamples\u002Fkeycloak",[72,2045,2046],{},"docs\u002Fexamples\u002Fkeycloak\u002F",[583,2048,2050],{"id":2049},"free-form-sql-contained","Free-form SQL, contained",[10,2052,2053,2054,2057,2058,2061,2062,2065,2066,1508,2069,2072,2073,2076,2077,2080],{},"A flow can carry a node holding hand-written SQL, and the assistant writes one for\nevery question. That node is bound to a single rule: ",[40,2055,2056],{},"it may read its own input and\nnothing else."," The query is parsed, and anything in a ",[72,2059,2060],{},"FROM"," or ",[72,2063,2064],{},"JOIN"," position that\nis not ",[72,2067,2068],{},"self",[72,2070,2071],{},"input"," (or a ",[72,2074,2075],{},"WITH"," defined in the same query) is refused — table\nfunctions, another database's tables, ",[72,2078,2079],{},"information_schema",", dictionary and settings\nlookups included. A query that cannot be parsed is refused rather than run. On DuckDB\nthe node executes in a locked in-memory sandbox with external access disabled; Polars\nonly ever sees the registered frame.",[10,2082,2083],{},"This matters because the engine holds credentials that can read the whole bucket: if a\nquery could name another table, the permission on a datasource would be decorative.",[116,2085,2087],{"id":2086},"ai-assistant","AI assistant",[10,2089,2090,2091,2096,2097,232,2100,2103,2104,210,2107,2110,2111,2114,2115,2118,2119,2122,2123,2125,2126,2129,2130,2133],{},"Optional, in the gateway, built with ",[28,2092,2095],{"href":2093,"rel":2094},"https:\u002F\u002Fai.pydantic.dev",[32],"pydantic-ai",", against the\nproviders an administrator adds from the panel: Anthropic through its own SDK, and any\nOpenAI-compatible endpoint (also from ",[72,2098,2099],{},"AI__BASE_URL",[72,2101,2102],{},"AI__SECRET_KEY","; verified on Scaleway\nGenerative APIs). The agent has three tools and nothing else: ",[72,2105,2106],{},"list_datasources",[72,2108,2109],{},"describe_datasource"," (columns, types and the hand-written ",[40,2112,2113],{},"field descriptions"," — the\nsemantic context that makes the query right — plus a few sample rows) and\n",[72,2116,2117],{},"query_datasource"," (one read-only ",[72,2120,2121],{},"SELECT … FROM self",", executed by the engine through\nthe same ",[72,2124,1532],{}," node and guardrails as the editor, rows capped by\n",[72,2127,2128],{},"AI__MAX_RESULT_ROWS","). Every tool re-checks the caller's permissions: a datasource\noutside the user's VIEW scope looks exactly like one that does not exist. Each query is\nan audit event (",[72,2131,2132],{},"ai.query",", with the SQL, the model and the row count).",[10,2135,2136,2139,2140,2143,2144,2147],{},[40,2137,2138],{},"Conversations are saved"," in the gateway's database, private to their author and\ndeleted with the account. They can be searched by title, reopened with their steps and\nresult tables, and continued — a question already answered costs nothing to read again.\nThe assistant names each conversation itself from the first question. Because the\nhistory lives on the server, the browser sends only a conversation id: it cannot forge\na history, and a turn is bounded by ",[72,2141,2142],{},"AI__MAX_COST_PER_TURN_USD"," and\n",[72,2145,2146],{},"AI__MAX_INPUT_TOKENS_PER_TURN",", not merely by a number of model round-trips.",[10,2149,2150,2153],{},[40,2151,2152],{},"What a call cost is on screen",", under each answer and for the conversation as a\nwhole. The figure comes from the price data shipped with pydantic-ai — there is no\nprice list of ours to go stale — and it is an estimate from the model's list price,\nnot your provider's invoice; a dash means that model could not be priced, which is not\nthe same as free.",[10,2155,2156],{},"While the assistant works, the page names the phase it is actually in — scanning the\ncatalog, reading the fields of a datasource, querying it, writing the answer — read\nfrom the same stream events the steps are read from.",[10,2158,2159],{},"Models are an administrator's decision (Admin → AI models): the provider's catalog is\nlisted, nothing is enabled by default, and embedding or audio models cannot be enabled\nfor chat.",[10,2161,2162,2165,2166,2168],{},[40,2163,2164],{},"People always know they are talking to an AI"," (EU AI Act, article 50). Before the\nfirst question a dialog says so, and it cannot be skipped. In both assistants every\nanswer is labelled ",[45,2167,2087],{}," with the model that wrote it, every question as the\nuser's, also for screen readers. A notice that stays under the input reads \"This is an\nAI: it can make mistakes. Always check the results\". A downloaded conversation opens\nwith the same notice and labels who wrote what.",[10,2170,2171,2172,243,2175,2178,2179,2182,2183,2143,2186,2189,2190,107],{},"On the external ClickHouse engine the assistant's queries can run as a ",[40,2173,2174],{},"dedicated\nread-only account",[72,2176,2177],{},"CLICKHOUSE_EXTERNAL__AI_USERNAME","): a second barrier behind the\nSQL node's allow-list, with no access to ",[72,2180,2181],{},"system",", to any table, or to ",[72,2184,2185],{},"url()",[72,2187,2188],{},"file()",". See ",[28,2191,2193],{"href":2192},"\u002Fdocs\u002Fengines\u002Fclickhouse-ai-user","docs\u002Fengines\u002Fclickhouse-ai-user.md",[583,2195,2197],{"id":2196},"in-the-flow-editor","In the flow editor",[10,2199,2200,2201,2204,2205,2208],{},"The editor's right panel has a second tab, ",[40,2202,2203],{},"Assistant",", that builds the flow on the\ncanvas with you: ask for \"the orders of last year, total per country\" and it puts the\ndatasource on the canvas, adds a filter and a group-by, and checks every step on a\npreview of the data — a step that fails or returns the wrong columns is fixed before it\ngoes on. It places the same operations you would, with the same parameters the node\npanel shows; it reads only the datasources you may read, and previews pass the checks\nthe editor's do. What it built comes back as a whole and is applied as ",[40,2206,2207],{},"one Undo\nstep","; it never saves, runs or schedules the flow. If you change the canvas while it\nworks, its version is not applied over yours without asking. The conversation is kept\nwith the flow and reopens with it; spending caps and the audit log count it like the\ndata assistant's.",[583,2210,2212],{"id":2211},"in-the-script-editor","In the script editor",[10,2214,2215,2216,2218,2219,2222,2223,107],{},"Beside the code of a Script node or a saved script, ",[40,2217,2203],{}," writes and fixes\nPython and R for the script's environment, and «Explain and fix the last error» starts\nfrom a failed run. It proposes whole files, applied only when you say so, as one change\nyou can undo. ",[40,2220,2221],{},"No data reaches the model",": the code, the environment and its\npackages, the names of the variables and of the inputs' columns, and only the error of\na failed run's log, never what the script printed. See\n",[28,2224,2226],{"href":2225},"\u002Fdocs\u002Fscripting#the-assistant","docs\u002Fscripting.md",[116,2228,2230],{"id":2229},"scheduling--timezone","Scheduling & timezone",[10,2232,2233,2234,243,2237,2240,2241,2243],{},"Cron-style schedules are evaluated in a ",[40,2235,2236],{},"deployment-wide timezone",[72,2238,2239],{},"APP__TIMEZONE",",\nDST-aware) and stored\u002Freturned in UTC; the frontend displays browser-local time. A\nschedule-load heatmap surfaces busy bands and collisions against a configurable\nworker capacity. Schedules fire from the ",[40,2242,990],{}," role: a due slot is taken with\na compare-and-swap on the time the process saw, so with several orchestrators exactly\none fires it; a flow whose last run failed on schedule sends an email, once, to the\naddresses set next to the schedule.",[10,2245,2246,2247,2249,2250,2253,2254,2257],{},"A ",[40,2248,335],{}," is queued and claimed like a flow's run. Its cells start together,\neach flow under its own run row, so the flow's schedule cannot start it twice while\nthe group runs it. A flow already running is never duplicated: its run counts for the\ncell when it runs in production and started after the previous step, otherwise the\ncell runs the flow once it ends. A saved script's cell takes the script's live slot the\nsame way; a container cell runs its image as a Container node with no edges; a\nPushmail cell sends one email with an ",[72,2251,2252],{},".xlsx"," per datasource, its recipients within the\nSMTP connection's allowed domains, checked again when it runs. Cells\ncount in ",[72,2255,2256],{},"ORCHESTRATOR__MAX_CONCURRENT"," like flow runs; within that, how many really\nrun at once is up to the workers and the queue.",[116,2259,2261],{"id":2260},"export-to-dbt","Export to dbt",[10,2263,2264,2265,2268,2269,62,2271,2274,2275,2278,2279,232,2281,2283,2284,2286,2287,243,2290,2293,2294,2297,2298,2301,2302,2305,2306,2309,2310,2313,2314,2319,2320,107],{},"A flow is not locked in the tool: from the Flows page, ",[45,2266,2267],{},"Export"," turns it into a dbt\nproject (administrators only — an admin group is the way to give it to the data team). One model per output node (",[72,2270,446],{},[72,2272,2273],{},"incremental"," in append for database\ntables written in append), the steps shared by several outputs as ",[72,2276,2277],{},"ephemeral"," models\nreferenced with ",[72,2280,390],{},[72,2282,394],{}," with the tables, their descriptions and columns,\n",[72,2285,397],{}," with every model's columns and descriptions and the ",[40,2288,2289],{},"data contract as dbt\ntests",[72,2291,2292],{},"not_null"," as a generic test on the column; a one-column ",[72,2295,2296],{},"unique"," — two NULLs are\na duplicate, as in Tabularia — ",[72,2299,2300],{},"accepted_values",", range, pattern, row count and expressions\nas singular tests, each with the contract's severity), a seed for every file the flow\nreads, a SQL-query datasource as an ephemeral model, and, from an upstream flow whose\noutput the flow reads, only the branch that publishes it. Three targets: ",[40,2303,2304],{},"ClickHouse","\n(dbt-clickhouse on your ClickHouse, which reads Postgres, MySQL and MariaDB live through\nits PostgreSQL and MySQL table engines), ",[40,2307,2308],{},"dbt-duckdb",", which attaches the source\ndatabases and runs the very SQL Tabularia runs, and the ",[40,2311,2312],{},"native warehouse","\n(dbt-postgres or dbt-clickhouse, with the SQL translated by sqlglot; MySQL and MariaDB\nsources are read by the other two targets). A dialog shapes it for the data team: a complete\nproject or a folder to copy into theirs, names and prefix, layers, their source names,\nschema, materializations, tests and emails. End-to-end oracles export every Sample flow,\nrun it with a real dbt and compare it with the datasource Tabularia publishes\n(",[28,2315,2318],{"href":2316,"rel":2317},"https:\u002F\u002Fgithub.com\u002FJumpAsAService\u002FTabularia\u002Fblob\u002Fmain\u002Fe2e\u002Fdbt_export\u002FREADME.md",[32],"e2e\u002Fdbt_export","). The details, the mapping and the limits are in\n",[28,2321,2323],{"href":2322},"\u002Fdocs\u002Fdbt-export","docs\u002Fdbt-export.md",[116,2325,2327],{"id":2326},"openlineage-optional","OpenLineage (optional)",[10,2329,2330],{"align":12},[15,2331],{"src":2332,"alt":2333,"width":2334},"\u002Fshots\u002Flineage-marquez.jpg","A Tabularia flow in Marquez: the datasources it read on the left, the flow job in the middle, the datasource it published on the right",900,[10,2336,707,2337,2340,2341,2344,2345,2348,2349,2352,2353,2356,2357,2359,2360,2365,2366,1700,2369,2372,2373,2375,2376,2379,2380,1508,2383,2386],{},[72,2338,2339],{},"OPENLINEAGE__URL"," (and ",[72,2342,2343],{},"OPENLINEAGE__API_KEY"," if the collector wants one) and\nevery run that closes is reported with the official OpenLineage client: a flow run is\na job named after its folder path, its output nodes are child jobs with a\n",[72,2346,2347],{},"ParentRunFacet",", a datasource refresh is a job of its own. Inputs are the datasources\nread — named as in Tabularia under their folder path (",[72,2350,2351],{},"tabularia:\u002F\u002F\u003Cnamespace>"," +\n",[72,2354,2355],{},"\u002FFolder\u002Fname","), with schema and snapshot version, the parquet they serve as a symlink\nand the Tabularia id in a ",[72,2358,23],{}," facet\n(",[28,2361,2364],{"href":2362,"rel":2363},"https:\u002F\u002Fgithub.com\u002FJumpAsAService\u002FTabularia\u002Fblob\u002Fmain\u002Fdocs\u002Fopenlineage\u002FTabulariaDatasetFacet.json",[32],"schema",") — database tables\n(",[72,2367,2368],{},"postgres:\u002F\u002Fhost:5432",[72,2370,2371],{},"db.schema.table",", the standard naming), the tables a SQL\nquery reads (extracted with sqlglot, the query itself in the ",[72,2374,1532],{}," facet) and\nSharePoint files; outputs are the datasources published, the tables written, the\nS3\u002FGCS objects and mirror copies. Emails are not datasets. Only metadata leaves: names,\nschemas, query texts, paths, counts and times — never rows, users or credentials.\nEvents leave after the commit that closes the run, through a bounded queue and a\nbackground thread: a collector that is down or slow never fails or delays a run.\nMeasured: closing a run costs 5–15 ms once, and under a storm of runs the gateway's\nthroughput is the same with lineage on or off, whether the collector is fast or\nanswers in 2 seconds.\n",[72,2377,2378],{},"OPENLINEAGE__FILE"," writes the same events as JSON Lines instead of, or\nbesides, the HTTP collector; the library's own ",[72,2381,2382],{},"OPENLINEAGE_URL",[72,2384,2385],{},"OPENLINEAGE_CONFIG","\nwork too, for composite and custom transports.",[10,2388,2389,2392,2393,2396,2397,2399,2400,2403,2404,107],{},[72,2390,2391],{},"GET \u002Fflows\u002F{id}\u002Fopenlineage"," returns the flow's dependencies as a static ",[72,2394,2395],{},"JobEvent","\nbuilt from its definition — to download (also from the Flows page, under ",[45,2398,2267],{},"), or\nto ",[72,2401,2402],{},"POST"," to the collector so that a catalog knows the flows before they ever run.\nThe full mapping, the dataset naming and the collector set-ups are in\n",[28,2405,2407],{"href":2406},"\u002Fdocs\u002Fopenlineage","docs\u002Fopenlineage.md",[116,2409,2411],{"id":2410},"monitoring","Monitoring",[10,2413,2414,2415,2417,2418,2420],{},"Ships in the box: ",[40,2416,969],{}," dashboards (task durations, cache hit\nrate, storage growth, per-container memory, preview latency by engine), plus cAdvisor,\nnode-exporter and a celery-exporter. Grafana is embedded in an admin-only Monitoring\ntab, and a ",[40,2419,1288],{}," page shows run durations, preview histograms per engine and\nthe external ClickHouse's slowest queries without any extra table.",[116,2422,2424],{"id":2423},"services-docker-compose","Services (docker-compose)",[446,2426,2427,2436],{},[449,2428,2429],{},[452,2430,2431,2434],{},[455,2432,2433],{},"Service",[455,2435,828],{},[471,2437,2438,2452,2476,2486,2499,2509,2523,2533,2543,2553,2566,2582],{},[452,2439,2440,2445],{},[476,2441,2442],{},[72,2443,2444],{},"frontend",[476,2446,2447,2448],{},"Nuxt 3 app (SSR) — ",[28,2449,2450],{"href":2450,"rel":2451},"http:\u002F\u002Flocalhost:3000",[32],[452,2453,2454,2458],{},[476,2455,2456],{},[72,2457,982],{},[476,2459,2460,2461,2464,2465,82,2468,2470,2471,2475],{},"FastAPI control plane (auth, RBAC, audit) in role ",[72,2462,2463],{},"all",": it also fires schedules and runs flows. Split it into ",[72,2466,2467],{},"api",[72,2469,990],{}," copies when you scale (see ",[28,2472,2474],{"href":2473},"\u002Fdocs\u002Fdeploy\u002Fkubernetes","docs\u002Fdeploy\u002Fkubernetes.md",") — :8000",[452,2477,2478,2483],{},[476,2479,2480],{},[72,2481,2482],{},"backend",[476,2484,2485],{},"FastAPI engine (preview)",[452,2487,2488,2496],{},[476,2489,2490,1508,2493],{},[72,2491,2492],{},"worker",[72,2494,2495],{},"preview-worker",[476,2497,2498],{},"Celery workers (runs, DB ingest \u002F preview)",[452,2500,2501,2506],{},[476,2502,2503],{},[72,2504,2505],{},"beat",[476,2507,2508],{},"Celery beat: the only clock of the periodic tasks; keep it at one copy",[452,2510,2511,2516],{},[476,2512,2513],{},[72,2514,2515],{},"keys-init",[476,2517,2518,2519,2522],{},"gives the ",[72,2520,2521],{},"keys"," volume (the encryption key) to the services, then exits",[452,2524,2525,2530],{},[476,2526,2527],{},[72,2528,2529],{},"postgres",[476,2531,2532],{},"control-plane metadata",[452,2534,2535,2540],{},[476,2536,2537],{},[72,2538,2539],{},"redis",[476,2541,2542],{},"Valkey — broker, cache index, counters",[452,2544,2545,2550],{},[476,2546,2547],{},[72,2548,2549],{},"minio",[476,2551,2552],{},"S3 object storage — console :9001, S3 API :9002",[452,2554,2555,2563],{},[476,2556,2557,1508,2560],{},[72,2558,2559],{},"victoriametrics",[72,2561,2562],{},"grafana",[476,2564,2565],{},"metrics + dashboards — :3001",[452,2567,2568,2579],{},[476,2569,2570,1508,2573,1508,2576],{},[72,2571,2572],{},"cadvisor",[72,2574,2575],{},"node-exporter",[72,2577,2578],{},"celery-exporter",[476,2580,2581],{},"metric exporters",[452,2583,2584,2595],{},[476,2585,2586,1508,2589,1508,2592],{},[72,2587,2588],{},"sampledb-postgres",[72,2590,2591],{},"sampledb-clickhouse",[72,2593,2594],{},"sampledb-init",[476,2596,2597,2598,2601],{},"the sample ERP and CRM (:5434, :8124) and the container that builds the ",[72,2599,2600],{},"Sample"," folder, then exits",[116,2603,2605],{"id":2604},"quickstart","Quickstart",[88,2607,2608],{},[10,2609,2610,2611,2615,2616,2623,2624,107],{},"Production on Kubernetes: see ",[28,2612,2613],{"href":2473},[72,2614,2474],{},"\nfor the components, the hardened backend image, ingress and storage sizing, and the\nHelm chart in ",[28,2617,2620],{"href":2618,"rel":2619},"https:\u002F\u002Fgithub.com\u002FJumpAsAService\u002FTabularia\u002Ftree\u002Fmain\u002Finfrastructure\u002Fhelm\u002Ftabularia",[32],[72,2621,2622],{},"infrastructure\u002Fhelm\u002Ftabularia",".\nUpgrading: ",[28,2625,2627],{"href":2626},"\u002Fdocs\u002Fdeploy\u002Frelease-checklist",[72,2628,2629],{},"docs\u002Fdeploy\u002Frelease-checklist.md",[10,2631,2632,2633,107],{},"Requires Docker and Docker Compose. Step by step, with the sample data and a first scheduled\nflow: ",[28,2634,2636],{"href":2635},"\u002Fdocs\u002Fgetting-started-docker-compose","getting started with Docker Compose",[1602,2638,2642],{"className":2639,"code":2640,"language":2641,"meta":1610,"style":1610},"language-bash shiki shiki-themes github-dark-dimmed github-light","git clone https:\u002F\u002Fgithub.com\u002FJumpAsAService\u002FTabularia.git\ncd Tabularia\u002Finfrastructure\ndocker compose up -d\n","bash",[72,2643,2644,2660,2670],{"__ignoreMap":1610},[2645,2646,2649,2653,2657],"span",{"class":2647,"line":2648},"line",1,[2645,2650,2652],{"class":2651},"sqRhv","git",[2645,2654,2656],{"class":2655},"sXfbr"," clone",[2645,2658,2659],{"class":2655}," https:\u002F\u002Fgithub.com\u002FJumpAsAService\u002FTabularia.git\n",[2645,2661,2663,2667],{"class":2647,"line":2662},2,[2645,2664,2666],{"class":2665},"swcJU","cd",[2645,2668,2669],{"class":2655}," Tabularia\u002Finfrastructure\n",[2645,2671,2673,2676,2679,2682],{"class":2647,"line":2672},3,[2645,2674,2675],{"class":2651},"docker",[2645,2677,2678],{"class":2655}," compose",[2645,2680,2681],{"class":2655}," up",[2645,2683,2684],{"class":2665}," -d\n",[10,2686,2687,2688,2691,2692,2695,2696,2699,2700,210,2702,252],{},"Nothing to configure: open ",[28,2689,2450],{"href":2450,"rel":2690},[32]," and create the administrator (the page\nasks for it only while the installation has no account). The sample data, a company\nwith an ERP and a CRM and eleven ready-made flows, appears a couple of minutes later.\nThe first ",[72,2693,2694],{},"docker compose up -d"," builds the images from the code: 2–3 minutes on a fast\nmachine, 5–15 on a laptop. Settings are optional: ",[72,2697,2698],{},"cp .env.example .env"," and edit.\nScript and Container nodes, saved scripts, container registries and the automatic\ndatabase backups need Kubernetes: on this stack they are shown greyed out (",[28,2701,312],{"href":311},[28,2703,322],{"href":321},[446,2705,2706,2718],{},[449,2707,2708],{},[452,2709,2710,2712,2715],{},[455,2711,2433],{},[455,2713,2714],{},"URL",[455,2716,2717],{},"Credentials",[471,2719,2720,2733,2746,2764],{},[452,2721,2722,2725,2730],{},[476,2723,2724],{},"App",[476,2726,2727],{},[28,2728,2450],{"href":2450,"rel":2729},[32],[476,2731,2732],{},"the administrator you create at the first visit",[452,2734,2735,2737,2743],{},[476,2736,843],{},[476,2738,2739],{},[28,2740,2741],{"href":2741,"rel":2742},"http:\u002F\u002Flocalhost:8000",[32],[476,2744,2745],{},"through the app's sign-in, or an API token",[452,2747,2748,2751,2757],{},[476,2749,2750],{},"Grafana",[476,2752,2753],{},[28,2754,2755],{"href":2755,"rel":2756},"http:\u002F\u002Flocalhost:3001",[32],[476,2758,2759,1508,2762],{},[72,2760,2761],{},"admin",[72,2763,2761],{},[452,2765,2766,2769,2775],{},[476,2767,2768],{},"MinIO",[476,2770,2771],{},[28,2772,2773],{"href":2773,"rel":2774},"http:\u002F\u002Flocalhost:9001",[32],[476,2776,2777,1508,2780],{},[72,2778,2779],{},"minioadmin",[72,2781,2779],{},[10,2783,2784,2785,2788],{},"Upload a CSV\u002FXLSX\u002FJSON\u002Fparquet file ",[40,2786,2787],{},"or"," connect a database, drag transformations\nfrom the sidebar, connect nodes, preview at any point, then run — or download any\nnode's data as CSV\u002FExcel.",[10,2790,2791,2794],{},[40,2792,2793],{},"Adding an external engine"," is configuration only. For a managed ClickHouse:",[1602,2796,2798],{"className":2639,"code":2797,"language":2641,"meta":1610,"style":1610},"CLICKHOUSE_EXTERNAL__HOST=xxxx.datawarehouse.it-mil.scw.eu\nCLICKHOUSE_EXTERNAL__PORT=8443\nCLICKHOUSE_EXTERNAL__SECURE=true\nCLICKHOUSE_EXTERNAL__USERNAME=...\nCLICKHOUSE_EXTERNAL__PASSWORD=...\nCLICKHOUSE_EXTERNAL__S3_ENDPOINT=https:\u002F\u002Fs3.it-mil.scw.cloud   # the bucket as the server sees it\n",[72,2799,2800,2812,2822,2832,2843,2853],{"__ignoreMap":1610},[2645,2801,2802,2805,2809],{"class":2647,"line":2648},[2645,2803,710],{"class":2804},"ssh_m",[2645,2806,2808],{"class":2807},"s6PUj","=",[2645,2810,2811],{"class":2655},"xxxx.datawarehouse.it-mil.scw.eu\n",[2645,2813,2814,2817,2819],{"class":2647,"line":2662},[2645,2815,2816],{"class":2804},"CLICKHOUSE_EXTERNAL__PORT",[2645,2818,2808],{"class":2807},[2645,2820,2821],{"class":2655},"8443\n",[2645,2823,2824,2827,2829],{"class":2647,"line":2672},[2645,2825,2826],{"class":2804},"CLICKHOUSE_EXTERNAL__SECURE",[2645,2828,2808],{"class":2807},[2645,2830,2831],{"class":2655},"true\n",[2645,2833,2835,2838,2840],{"class":2647,"line":2834},4,[2645,2836,2837],{"class":2804},"CLICKHOUSE_EXTERNAL__USERNAME",[2645,2839,2808],{"class":2807},[2645,2841,2842],{"class":2655},"...\n",[2645,2844,2846,2849,2851],{"class":2647,"line":2845},5,[2645,2847,2848],{"class":2804},"CLICKHOUSE_EXTERNAL__PASSWORD",[2645,2850,2808],{"class":2807},[2645,2852,2842],{"class":2655},[2645,2854,2856,2859,2861,2864],{"class":2647,"line":2855},6,[2645,2857,2858],{"class":2804},"CLICKHOUSE_EXTERNAL__S3_ENDPOINT",[2645,2860,2808],{"class":2807},[2645,2862,2863],{"class":2655},"https:\u002F\u002Fs3.it-mil.scw.cloud",[2645,2865,2867],{"class":2866},"sgHix","   # the bucket as the server sees it\n",[10,2869,2870],{},"For BigQuery, with the bucket on Google Cloud Storage:",[1602,2872,2874],{"className":2639,"code":2873,"language":2641,"meta":1610,"style":1610},"STORAGE__ENDPOINT=https:\u002F\u002Fstorage.googleapis.com               # HMAC keys in STORAGE__ACCESS_KEY \u002F SECRET_KEY\nSTORAGE__REGION=auto\nBIGQUERY__PROJECT=my-gcp-project\nBIGQUERY__CREDENTIALS_B64=...                                  # base64 -w0 service-account.json, one line\nPREVIEW__DEFAULT_SAMPLE_ROWS=100000                             # keep the editor cheap\n",[72,2875,2876,2889,2899,2908,2920],{"__ignoreMap":1610},[2645,2877,2878,2881,2883,2886],{"class":2647,"line":2648},[2645,2879,2880],{"class":2804},"STORAGE__ENDPOINT",[2645,2882,2808],{"class":2807},[2645,2884,2885],{"class":2655},"https:\u002F\u002Fstorage.googleapis.com",[2645,2887,2888],{"class":2866},"               # HMAC keys in STORAGE__ACCESS_KEY \u002F SECRET_KEY\n",[2645,2890,2891,2894,2896],{"class":2647,"line":2662},[2645,2892,2893],{"class":2804},"STORAGE__REGION",[2645,2895,2808],{"class":2807},[2645,2897,2898],{"class":2655},"auto\n",[2645,2900,2901,2903,2905],{"class":2647,"line":2672},[2645,2902,1158],{"class":2804},[2645,2904,2808],{"class":2807},[2645,2906,2907],{"class":2655},"my-gcp-project\n",[2645,2909,2910,2912,2914,2917],{"class":2647,"line":2834},[2645,2911,1302],{"class":2804},[2645,2913,2808],{"class":2807},[2645,2915,2916],{"class":2655},"...",[2645,2918,2919],{"class":2866},"                                  # base64 -w0 service-account.json, one line\n",[2645,2921,2922,2924,2926,2929],{"class":2647,"line":2845},[2645,2923,1486],{"class":2804},[2645,2925,2808],{"class":2807},[2645,2927,2928],{"class":2655},"100000",[2645,2930,2931],{"class":2866},"                             # keep the editor cheap\n",[10,2933,2934,2935,2937,2938,2944],{},"Recreate the workers (",[72,2936,2694],{},") and the engine shows up in the picker;\n",[28,2939,2941],{"href":2940},"\u002Fdocs\u002Fconfiguration",[72,2942,2943],{},"docs\u002Fconfiguration.md"," lists every variable, with its default.",[10,2946,2947,481,2950,2953,2954,2957,2958,2961,2962,2964,2965,2968,2969,2972],{},[40,2948,2949],{},"Secrets:",[72,2951,2952],{},"SECURITY__FERNET_KEY"," encrypts the connection passwords, the same value in\ngateway, engine and workers; they refuse to start with an invalid one, or with none\nunless a ",[72,2955,2956],{},"SECURITY__KEYS_DIR"," is shared, where it is created at the first start (the\nCompose stack of ",[72,2959,2960],{},"infrastructure\u002F"," does that, in its ",[72,2963,2521],{}," volume: back it up with the\ndata). On Kubernetes the chart sets the key. With\n",[72,2966,2967],{},"APP__ENV_NAME=production"," the gateway additionally refuses the dev-default secrets\n(",[72,2970,2971],{},"JWT__SECRET",", admin and DB passwords) until they are overridden. Storage, broker,\ndatabase, timezone and engines are all env-driven — pointing at managed services is a\nconfig change, not a code change.",[10,2974,2975,2978,2979,2986,2987,2990,2991,2994,2995,3000,3001,107],{},[40,2976,2977],{},"Database schema:"," the metadata schema is versioned with Alembic revisions in\n",[28,2980,2983],{"href":2981,"rel":2982},"https:\u002F\u002Fgithub.com\u002FJumpAsAService\u002FTabularia\u002Ftree\u002Fmain\u002Fgateway\u002Fapp\u002Fdb\u002Fmigrations\u002Fversions",[32],[72,2984,2985],{},"gateway\u002Fapp\u002Fdb\u002Fmigrations\u002Fversions\u002F",". The\ndevelopment compose migrates at startup (",[72,2988,2989],{},"DB__AUTO_MIGRATE=true","). Everywhere else the\nmigration is a step of its own, ",[72,2992,2993],{},"python -m app.db.migra",", run before the new version\nstarts (the Helm chart does it in an init container): it applies the revisions in one\ntransaction and checks the result against the models, and a gateway refuses to start on\na database behind its code. Revisions go forward only and keep the previous version\nworking during a rolling update; an installation from before Alembic is adopted by the\nfirst run. How to add a revision: ",[28,2996,2997],{"href":877},[72,2998,2999],{},"docs\u002Fdatabase-migrations.md",".\nUpgrading an installation on Kubernetes or minikube (backup, rollout, rollback, restore):\n",[28,3002,3004],{"href":3003},"\u002Fdocs\u002Fdeploy\u002Fupgrading",[72,3005,3006],{},"docs\u002Fdeploy\u002Fupgrading.md",[116,3008,3010],{"id":3009},"sample-data","Sample data",[10,3012,3013,3014,3017,3018,3022,3023,107],{},"A realistic ",[40,3015,3016],{},"cured-meats company"," dataset, on by default in the Compose stack — an ERP on Postgres and a CRM\non ClickHouse, generated with Faker\u002Fnumpy — plus an init container that builds a\nready-made ",[40,3019,3020],{},[72,3021,2600],{}," folder in Tabularia on top of it. The data ships built-in\nseasonality (monthly volume, product & channel seasonal mix, per-warehouse\nfulfilment anomalies) so pivots tell a story. Details in\n",[28,3024,3027],{"href":3025,"rel":3026},"https:\u002F\u002Fgithub.com\u002FJumpAsAService\u002FTabularia\u002Fblob\u002Fmain\u002Finfrastructure\u002Fsampledb\u002FREADME.md",[32],[72,3028,3029],{},"infrastructure\u002Fsampledb\u002FREADME.md",[10,3031,3032,3033,3035,3036,3039,3040,3043,3044,1543,3047,1243,3049,1243,3052,3055],{},"It is on in the Compose stack of ",[72,3034,2960],{},", at the ",[72,3037,3038],{},"small"," scale (",[72,3041,3042],{},"SAMPLES_SCALE","\nin ",[72,3045,3046],{},".env",[72,3048,3038],{},[72,3050,3051],{},"medium",[72,3053,3054],{},"robusta","). To leave it out:",[1602,3057,3059],{"className":2639,"code":3058,"language":2641,"meta":1610,"style":1610},"docker compose up -d --scale sampledb-init=0 --scale sampledb-postgres=0 --scale sampledb-clickhouse=0\n",[72,3060,3061],{"__ignoreMap":1610},[2645,3062,3063,3065,3067,3069,3072,3075,3078,3081,3083,3086,3088,3090,3093],{"class":2647,"line":2648},[2645,3064,2675],{"class":2651},[2645,3066,2678],{"class":2655},[2645,3068,2681],{"class":2655},[2645,3070,3071],{"class":2665}," -d",[2645,3073,3074],{"class":2665}," --scale",[2645,3076,3077],{"class":2655}," sampledb-init=",[2645,3079,3080],{"class":2665},"0",[2645,3082,3074],{"class":2665},[2645,3084,3085],{"class":2655}," sampledb-postgres=",[2645,3087,3080],{"class":2665},[2645,3089,3074],{"class":2665},[2645,3091,3092],{"class":2655}," sampledb-clickhouse=",[2645,3094,3095],{"class":2665},"0\n",[10,3097,979,3098,3100,3101,3103,3104,3106],{},[72,3099,2594],{}," container waits for the gateway and the two sample databases,\nfills them if they are empty (seconds at ",[72,3102,3038],{},", longer at ",[72,3105,3054],{},"), then logs in\nas the administrator and creates, idempotently by name:",[446,3108,3109,3119],{},[449,3110,3111],{},[452,3112,3113,3116],{},[455,3114,3115],{},"Where",[455,3117,3118],{},"What",[471,3120,3121,3130,3140,3150],{},[452,3122,3123,3127],{},[476,3124,3125],{},[72,3126,2600],{},[476,3128,3129],{},"the two connections (ERP, CRM) and four saved views (reports)",[452,3131,3132,3137],{},[476,3133,3134],{},[72,3135,3136],{},"Sample \u002F Flows",[476,3138,3139],{},"eleven flows — margin by category, cost to serve, RFM segments, CRM workload, leaderboards, private ⋈ public joins — three of them scheduled",[452,3141,3142,3147],{},[476,3143,3144],{},[72,3145,3146],{},"Sample \u002F Private tables",[476,3148,3149],{},"the nine company tables, and the tables the flows produce",[452,3151,3152,3157],{},[476,3153,3154],{},[72,3155,3156],{},"Sample \u002F Public tables",[476,3158,3159],{},"reference tables anyone could publish: a calendar with Italian holidays, the twenty regions",[10,3161,3162,3163,3166,3167,3170],{},"Some tables carry column descriptions and some deliberately do not, so the\ndifference an assistant-ready datasource makes is visible. The flows are run once\nso their output tables exist and the reports have something to show\n(",[72,3164,3165],{},"SAMPLES_RUN_FLOWS=false"," skips that). The folder is created with ",[40,3168,3169],{},"no permissions\non it",": who gets to see it is the administrator's call, like any other folder.",[10,3172,3173],{},"Restarting the stack does not duplicate anything: the container finds what it\nmade and stops.",[116,3175,3177],{"id":3176},"internationalization--themes","Internationalization & themes",[10,3179,3180,3181,232,3184,232,3187,232,3190,232,3193,3196,3197,232,3200,232,3203,210,3206,232,3209,3212,3213,3216],{},"UI copy lives in per-language i18n catalogs (",[72,3182,3183],{},"en",[72,3185,3186],{},"it",[72,3188,3189],{},"fr",[72,3191,3192],{},"de",[72,3194,3195],{},"es","), selected\nper user and persisted via cookie for SSR safety. Themes (",[72,3198,3199],{},"dark",[72,3201,3202],{},"light",[72,3204,3205],{},"imperium",[72,3207,3208],{},"dracula",[72,3210,3211],{},"monokai",") are CSS-variable palettes on ",[72,3214,3215],{},":root[data-theme]",", persisted in localStorage\nwith an anti-flash head script. Code and comments are English-only; end-user strings\nare translated.",[116,3218,3220],{"id":3219},"license","License",[10,3222,3223],{},"Copyright © 2025–2026 Leonardo Trivelli.",[10,3225,3226,3227,101,3230,3235,3236,3239,3240,3243,3244,3247,3248,3251],{},"Licensed under the ",[40,3228,3229],{},"GNU Affero General Public License v3.0",[28,3231,3234],{"href":3232,"rel":3233},"https:\u002F\u002Fgithub.com\u002FJumpAsAService\u002FTabularia\u002Fblob\u002Fmain\u002FLICENSE",[32],"LICENSE",".\nFor commercial licensing options, contact the author. The notices of the third-party software\nare in the app, under ",[40,3237,3238],{},"Open source licences"," in the settings menu. If you offer a changed\nversion to others over a network, the AGPL asks you to offer them its source: point\n",[72,3241,3242],{},"NUXT_PUBLIC_SOURCE_URL"," (Helm: ",[72,3245,3246],{},"frontend.publicSourceUrl",") at it, and ",[40,3249,3250],{},"Source code"," in the\nmenu leads there.",[10,3253,3254,3255,3258],{},"Tabularia is provided ",[40,3256,3257],{},"\"as is\", without warranty of any kind","; the authors are not liable\nfor any damage, including loss of data (AGPL-3.0, sections 15–16). Keep backups — and take one\nbefore every upgrade: database migrations only go forward.",[10,3260,3261],{},"\"BigQuery\" and \"Google Cloud\" are trademarks of Google LLC; \"ClickHouse\" is a\ntrademark of ClickHouse, Inc. Other product and company names mentioned here are\ntrademarks of their respective owners, named only to say what Tabularia works with;\nTabularia is not affiliated with, endorsed by or sponsored by any of them.",[583,3263,3265],{"id":3264},"third-party-services","Third-party services",[10,3267,3268,3269,3271,3272,3274],{},"The default docker-compose deployment includes ",[40,3270,2768],{}," (AGPL-3.0) and ",[40,3273,2750],{},"\n(AGPL-3.0) as separate, unmodified services accessed over standard APIs (S3, HTTP).\nThey are deployment choices, swappable via environment variables. All libraries used\nby Tabularia itself are under permissive licenses (MIT\u002FBSD\u002FApache-2.0\u002FISC).",[3276,3277,3278],"style",{},"html pre.shiki code .sqRhv, html code.shiki .sqRhv{--shiki-default:#F69D50;--shiki-light:#6F42C1}html pre.shiki code .sXfbr, html code.shiki .sXfbr{--shiki-default:#96D0FF;--shiki-light:#032F62}html pre.shiki code .swcJU, html code.shiki .swcJU{--shiki-default:#6CB6FF;--shiki-light:#005CC5}html .default .shiki span {color: var(--shiki-default);background: var(--shiki-default-bg);font-style: var(--shiki-default-font-style);font-weight: var(--shiki-default-font-weight);text-decoration: var(--shiki-default-text-decoration);}html .shiki span {color: var(--shiki-default);background: var(--shiki-default-bg);font-style: var(--shiki-default-font-style);font-weight: var(--shiki-default-font-weight);text-decoration: var(--shiki-default-text-decoration);}html .light .shiki span {color: var(--shiki-light);background: var(--shiki-light-bg);font-style: var(--shiki-light-font-style);font-weight: var(--shiki-light-font-weight);text-decoration: var(--shiki-light-text-decoration);}html.light .shiki span {color: var(--shiki-light);background: var(--shiki-light-bg);font-style: var(--shiki-light-font-style);font-weight: var(--shiki-light-font-weight);text-decoration: var(--shiki-light-text-decoration);}html pre.shiki code .ssh_m, html code.shiki .ssh_m{--shiki-default:#ADBAC7;--shiki-light:#24292E}html pre.shiki code .s6PUj, html code.shiki .s6PUj{--shiki-default:#F47067;--shiki-light:#D73A49}html pre.shiki code .sgHix, html code.shiki .sgHix{--shiki-default:#768390;--shiki-light:#6A737D}",{"title":1610,"searchDepth":2662,"depth":2672,"links":3280},[3281,3282,3283,3288,3289,3290,3293,3294,3295,3296,3297,3300,3304,3305,3306,3307,3308,3309,3310,3311,3312],{"id":118,"depth":2662,"text":119},{"id":136,"depth":2662,"text":137},{"id":441,"depth":2662,"text":170,"children":3284},[3285,3286,3287],{"id":585,"depth":2672,"text":586},{"id":703,"depth":2672,"text":704},{"id":733,"depth":2672,"text":65},{"id":769,"depth":2662,"text":770},{"id":782,"depth":2662,"text":783},{"id":1031,"depth":2662,"text":1032,"children":3291},[3292],{"id":1212,"depth":2672,"text":1213},{"id":1360,"depth":2662,"text":1361},{"id":1596,"depth":2662,"text":1597},{"id":1620,"depth":2662,"text":1555},{"id":1663,"depth":2662,"text":1664},{"id":1926,"depth":2662,"text":1927,"children":3298},[3299],{"id":2049,"depth":2672,"text":2050},{"id":2086,"depth":2662,"text":2087,"children":3301},[3302,3303],{"id":2196,"depth":2672,"text":2197},{"id":2211,"depth":2672,"text":2212},{"id":2229,"depth":2662,"text":2230},{"id":2260,"depth":2662,"text":2261},{"id":2326,"depth":2662,"text":2327},{"id":2410,"depth":2662,"text":2411},{"id":2423,"depth":2662,"text":2424},{"id":2604,"depth":2662,"text":2605},{"id":3009,"depth":2662,"text":3010},{"id":3176,"depth":2662,"text":3177},{"id":3219,"depth":2662,"text":3220,"children":3313},[3314],{"id":3264,"depth":2672,"text":3265},"md",{"source":3317},"README.md",true,"\u002Fdocs\u002Foverview",{"description":13},"docs\u002Foverview\u002FREADME","SMjcf85BVT1g-5a9SHgjV8sorcmB8SkSrKiGc8cYqP0",[3324,3341,3392,3420,3449,3470,3492,3511],{"title":3325,"icon":3326,"intro":3327,"items":3328},"Getting started","start","Install it on one machine, learn the few ideas everything else is built on, and run a first flow.",[3329,3331,3333,3336,3339],{"path":3330,"title":3325},"\u002Fdocs\u002Fgetting-started",{"path":2635,"title":3332},"Walkthrough with Docker Compose",{"path":3334,"title":3335},"\u002Fdocs\u002Fdeploy\u002Fminikube","Try it on your computer (minikube)",{"path":3337,"title":3338},"\u002Fdocs\u002Fconcepts","Concepts",{"path":3319,"title":3340},"Overview (README)",{"title":3342,"icon":3343,"intro":3344,"items":3345},"Capabilities","capabilities","What you can build: visual flows, analysis on any step, AI questions, pixel-perfect reports, dashboards, Python, R and SQL scripts, containers and the orchestration that ties them together.",[3346,3349,3352,3355,3358,3361,3364,3367,3369,3372,3375,3377,3380,3383,3385,3387,3390],{"path":3347,"title":3348},"\u002Fdocs\u002Fcapabilities","All capabilities",{"path":3350,"title":3351},"\u002Fdocs\u002Fcapabilities\u002Fflows","Low-code flows",{"path":3353,"title":3354},"\u002Fdocs\u002Fcapabilities\u002Fpreview-and-analysis","Preview and analysis",{"path":3356,"title":3357},"\u002Fdocs\u002Fcapabilities\u002Fask-your-data","Ask your data with AI",{"path":3359,"title":3360},"\u002Fdocs\u002Fcapabilities\u002Freports","Pixel-perfect reports",{"path":3362,"title":3363},"\u002Fdocs\u002Fcapabilities\u002Freport-functions","Report functions",{"path":3365,"title":3366},"\u002Fdocs\u002Fcapabilities\u002Fdashboards","Dashboards",{"path":311,"title":3368},"Python and R",{"path":3370,"title":3371},"\u002Fdocs\u002Fsql-scripts","SQL scripts",{"path":3373,"title":3374},"\u002Fdocs\u002Fcapabilities\u002Fnotebooks","marimo notebooks as apps",{"path":321,"title":3376},"Docker images in flows",{"path":3378,"title":3379},"\u002Fdocs\u002Fcapabilities\u002Fflow-groups","Flow groups: orchestration",{"path":3381,"title":3382},"\u002Fdocs\u002Fcapabilities\u002Fscheduling","Scheduling",{"path":343,"title":3384},"Pushmail: datasources, reports and dashboards by email",{"path":250,"title":3386},"Dynamic dates",{"path":3388,"title":3389},"\u002Fdocs\u002Fcapabilities\u002Fengines","Engines",{"path":580,"title":3391},"Engine differences",{"title":3393,"icon":3394,"intro":3395,"items":3396},"Sources","sources","Every way data comes in: files, databases, buckets, SFTP folders, HTTP APIs, SharePoint, Google Sheets, and the results of your own flows.",[3397,3400,3403,3406,3408,3410,3411,3414,3417],{"path":3398,"title":3399},"\u002Fdocs\u002Fsources","All sources",{"path":3401,"title":3402},"\u002Fdocs\u002Fsources\u002Ffiles","Uploaded files",{"path":3404,"title":3405},"\u002Fdocs\u002Fsources\u002Fdatabases","Database tables and queries",{"path":213,"title":3407},"S3 buckets",{"path":208,"title":3409},"SFTP folders",{"path":222,"title":223},{"path":3412,"title":3413},"\u002Fdocs\u002Fsources\u002Fsharepoint","Excel on SharePoint",{"path":3415,"title":3416},"\u002Fdocs\u002Fsources\u002Fgoogle-sheets","Google Sheets",{"path":3418,"title":3419},"\u002Fdocs\u002Fsources\u002Fflow-outputs","Results of flows and scripts",{"title":3421,"icon":3422,"intro":3423,"items":3424},"Connections","connections","The systems Tabularia reads from and writes to, how each connection is set up, and who may use it.",[3425,3428,3431,3433,3436,3438,3439,3441,3442,3445,3446],{"path":3426,"title":3427},"\u002Fdocs\u002Fconnections","All connections",{"path":3429,"title":3430},"\u002Fdocs\u002Fconnections\u002Fdatabases","Databases",{"path":195,"title":3432},"Microsoft SQL Server driver",{"path":3434,"title":3435},"\u002Fdocs\u002Fconnections\u002Fs3","S3 and compatible storage",{"path":208,"title":3437},"SFTP servers",{"path":222,"title":223},{"path":3412,"title":3440},"SharePoint",{"path":3415,"title":3416},{"path":3443,"title":3444},"\u002Fdocs\u002Fconnections\u002Fsmtp","SMTP email",{"path":325,"title":326},{"path":3447,"title":3448},"\u002Fdocs\u002Fconnections\u002Fdestinations","Writing data out",{"title":3450,"icon":3451,"intro":3452,"items":3453},"Integrations","integrations","How Tabularia fits your stack: AI providers, the REST API and webhooks, dbt, OpenLineage, single sign-on, SIEM and monitoring.",[3454,3457,3460,3462,3463,3464,3467,3468],{"path":3455,"title":3456},"\u002Fdocs\u002Fintegrations","All integrations",{"path":3458,"title":3459},"\u002Fdocs\u002Fintegrations\u002Fai","AI: where it is used",{"path":1958,"title":3461},"API, tokens and webhooks",{"path":2322,"title":2261},{"path":2406,"title":375},{"path":3465,"title":3466},"\u002Fdocs\u002Fsso-keycloak","Single sign-on with Keycloak",{"path":1984,"title":1981},{"path":3469,"title":2411},"\u002Fdocs\u002Fintegrations\u002Fmonitoring",{"title":3471,"icon":3472,"intro":3473,"items":3474},"Security & governance","governance","Who may see and do what, what is recorded, and how data quality and personal data are kept in check.",[3475,3478,3481,3483,3486,3488,3490],{"path":3476,"title":3477},"\u002Fdocs\u002Fgovernance","Overview",{"path":3479,"title":3480},"\u002Fdocs\u002Fgovernance\u002Fpermissions","Permissions",{"path":3482,"title":2005},"\u002Fdocs\u002Fgovernance\u002Faudit-log",{"path":3484,"title":3485},"\u002Fdocs\u002Fgovernance\u002Fdata-contracts","Data contracts",{"path":2036,"title":3487},"SSO and group mapping",{"path":2043,"title":3489},"Keycloak example on a laptop",{"path":2192,"title":3491},"A ClickHouse user for the AI assistant",{"title":3493,"icon":3494,"intro":3495,"items":3496},"Install & operate","operate","Run it in production on Kubernetes, configure it, upgrade it and keep it healthy.",[3497,3499,3502,3504,3506,3508,3509],{"path":2473,"title":3498},"Kubernetes and Helm",{"path":3500,"title":3501},"\u002Fdocs\u002Fdeploy\u002Fkind","A company-like cluster on one computer (kind)",{"path":3003,"title":3503},"Upgrading, backups and restore",{"path":2626,"title":3505},"Install and upgrade checklist",{"path":2940,"title":3507},"Configuration reference",{"path":1974,"title":1971},{"path":877,"title":3510},"Database migrations",{"title":3512,"icon":3513,"intro":3514,"items":3515},"Reference","reference","Release process and the remaining reference pages.",[3516],{"path":3517,"title":3518},"\u002Fdocs\u002Frelease-process","Release process",1793656371140]