[{"data":1,"prerenderedAt":1459},["ShallowReactive",2],{"doc:\u002Fdocs\u002Fcontainer-node":3,"docs-nav":1237},{"id":4,"title":5,"body":6,"description":16,"extension":1230,"meta":1231,"navigation":518,"path":1233,"seo":1234,"stem":1235,"__hash__":1236},"docs\u002Fdocs\u002Fcontainer-node.md","The Container node",{"type":7,"value":8,"toc":1217},"minimark",[9,13,17,29,38,43,85,97,101,247,269,279,286,303,307,319,323,420,430,436,440,443,470,473,476,652,702,716,720,726,874,880,893,897,931,945,950,968,972,1055,1059,1080,1084,1108,1112,1206,1213],[10,11,5],"h1",{"id":12},"the-container-node",[14,15,16],"p",{},"For whoever runs Tabularia on Kubernetes, and whoever writes flows that run images.",[14,18,19,20,24,25,28],{},"The ",[21,22,23],"strong",{},"Container node"," is a step of a flow that runs ",[21,26,27],{},"an image of yours",": a model\npackaged in an image, a Spark or dbt job your data team already ships, a company tool\nthat has run in a container for years. Tabularia runs it as a Kubernetes Job and\norchestrates it with the rest of the flow (order, schedule, retries, log, outcome); the\ncode and the image stay yours.",[14,30,31,32,37],{},"It is not the ",[33,34,36],"a",{"href":35},"\u002Fdocs\u002Fscripting","Script node",": there Tabularia builds the image from an\nenvironment an administrator defines, and you write the code in the editor. Here the\nimage comes ready from a registry.",[39,40,42],"h2",{"id":41},"before-you-start","Before you start",[44,45,46,60,67,74],"ul",{},[47,48,49,50,54,55,59],"li",{},"The installation runs on Kubernetes with ",[51,52,53],"code",{},"containers.enabled: true"," in the chart\n(",[33,56,58],{"href":57},"\u002Fdocs\u002Fcontainer-registries#turning-it-on","Container registries",").",[47,61,62,63,66],{},"An administrator has defined the registry, with the repositories whose images may run,\nand has given ",[21,64,65],{},"you"," (or one of your groups) access to it. The editor shows only the\nregistries and the images you may use.",[47,68,69,70,73],{},"The cluster's nodes can download from the registry: ",[21,71,72],{},"Try in the cluster",", in the\nregistry's page, checks it.",[47,75,76,77,80,81,84],{},"For a node with an input or an output, the ",[21,78,79],{},"scripts bucket"," is configured\n(Administration → Scripting environments): the data travels through it. When its\naddress is private, it is listed in ",[51,82,83],{},"networkPolicy.userCodePrivateCIDRs",".",[14,86,87,88,91,92,96],{},"The palette offers ",[21,89,90],{},"Container",", under ",[93,94,95],"em",{},"Control",", when at least one registry is granted\nto you.",[39,98,100],{"id":99},"the-node","The node",[102,103,104,116],"table",{},[105,106,107],"thead",{},[108,109,110,114],"tr",{},[111,112,113],"th",{},"Field",[111,115],{},[117,118,119,130,148,169,179,189,203,231],"tbody",{},[108,120,121,127],{},[122,123,124],"td",{},[21,125,126],{},"Registry",[122,128,129],{},"one of those granted to you",[108,131,132,137],{},[122,133,134],{},[21,135,136],{},"Image",[122,138,139,140,143,144,147],{},"a repository of the registry, ",[51,141,142],{},"team\u002Fforecast",". The exact names allowed are suggested; a pattern (",[51,145,146],{},"team\u002F*",") allows any name under it",[108,149,150,155],{},[122,151,152],{},[21,153,154],{},"Tag",[122,156,157,160,161,164,165,168],{},[51,158,159],{},"v3",", or a digest (",[51,162,163],{},"sha256:…","). Empty means ",[51,166,167],{},"latest",". The ↻ button, or the field itself, reads the tags from the registry",[108,170,171,176],{},[122,172,173],{},[21,174,175],{},"Command",[122,177,178],{},"the program to run, one item per line. Empty: the image's own entrypoint",[108,180,181,186],{},[122,182,183],{},[21,184,185],{},"Arguments",[122,187,188],{},"one per line. Empty: the image's own arguments",[108,190,191,196],{},[122,192,193],{},[21,194,195],{},"Variables",[122,197,198,199,202],{},"names in capitals, digits and ",[51,200,201],{},"_",", with a value",[108,204,205,214],{},[122,206,207,210,211],{},[21,208,209],{},"CPU",", ",[21,212,213],{},"Memory",[122,215,216,217,210,220,223,224,210,227,230],{},"what the container gets: ",[51,218,219],{},"500m",[51,221,222],{},"2","; ",[51,225,226],{},"512Mi",[51,228,229],{},"4Gi",". Both its request and its limit, within the installation's maximum",[108,232,233,244],{},[122,234,235,210,238,210,241],{},[21,236,237],{},"Timeout",[21,239,240],{},"Retries",[21,242,243],{},"Wait between",[122,245,246],{},"as on the Script node",[14,248,249,250,253,254,257,258,261,262,261,265,268],{},"Command and arguments are passed to the container as they are, one item per line: there\nis no shell, so no quotes, no ",[51,251,252],{},"$VARIABLE"," expansion, no ",[51,255,256],{},"&&",". Write\n",[51,259,260],{},"sh"," \u002F ",[51,263,264],{},"-c",[51,266,267],{},"your command line"," when you want a shell, if the image has one.",[14,270,271,274,275,278],{},[21,272,273],{},"What the image does."," Once the image and the tag are chosen, the panel reads the\nimage's own configuration from the registry and shows it: what it runs by default (its\nentrypoint and its command), the platforms it is built for, when it was built and its\nsize, its title, description and version when it declares them. ",[21,276,277],{},"Copy these into the\ncommand and the arguments"," starts from what the image runs (the entrypoint becomes the\ncommand, the image's command the arguments), to change it. The variables the image\ndeclares are offered with their default value: one click adds one to the node's\nvariables.",[14,280,281],{},[282,283],"img",{"alt":284,"src":285},"The flow editor with a Source, a Container node «Demand forecast» and an Output; the node's panel shows the registry Acme registry, the image analytics\u002Fdemand-forecast, the tag 1.4.0, what the image does by default and the variables it declares, and the arguments --days and 30","\u002Fmedia\u002Fguide\u002Fcontainers\u002Fcontainer-node-panel.png",[14,287,288],{},[93,289,290,291,294,295,298,299,302],{},"A Container node between a Source and an Output. Its panel names the registry, the image and the tag, reads from the registry what the image runs by default (",[51,292,293],{},"python \u002Fapp\u002Frun.py --days 7","), its platform, size and the variables it declares, and takes ",[51,296,297],{},"--days"," and ",[51,300,301],{},"30"," as arguments.",[39,304,306],{"id":305},"in-a-flow-group","In a flow group",[14,308,309,310,314,315,318],{},"A ",[33,311,313],{"href":312},"\u002Fdocs\u002Fcapabilities\u002Fflow-groups","flow group"," can run an image as a step of its own: ",[21,316,317],{},"Container (an image)"," in the\ngroup's catalog, or in the picker of an empty cell. The cell opens the same panel,\nwithout a flow's «Run»: registry, image, tag, command, arguments, variables, resources,\ntimeout and retries. The cell has no data in and none out; its failure stops the group.\nWhoever has no access to the registry or the image cannot open the group, as for a flow\nwith a Container node. A change of a cell's image, command or variables by someone else\nswitches off the group's schedule, set by another person.",[39,320,322],{"id":321},"what-the-container-finds","What the container finds",[102,324,325,334],{},[105,326,327],{},[108,328,329,332],{},[111,330,331],{},"Variable",[111,333],{},[117,335,336,350,360,385,404,412],{},[108,337,338,343],{},[122,339,340],{},[51,341,342],{},"TABULARIA_INPUT",[122,344,345,346,349],{},"an ",[51,347,348],{},"s3:\u002F\u002F"," address of a parquet file with the step before the node, when an edge enters it; empty otherwise",[108,351,352,357],{},[122,353,354],{},[51,355,356],{},"TABULARIA_OUTPUT",[122,358,359],{},"where to write a parquet file the nodes after it read, when an edge leaves it; empty otherwise",[108,361,362,382],{},[122,363,364,210,367,210,370,210,373,210,376,210,379],{},[51,365,366],{},"TABULARIA_RUN_ID",[51,368,369],{},"TABULARIA_FLOW_ID",[51,371,372],{},"TABULARIA_FLOW_NAME",[51,374,375],{},"TABULARIA_NODE_ID",[51,377,378],{},"TABULARIA_STEP_ID",[51,380,381],{},"TABULARIA_ATTEMPT",[122,383,384],{},"which run, flow, node and attempt this is",[108,386,387,401],{},[122,388,389,210,392,210,395,210,398],{},[51,390,391],{},"AWS_ACCESS_KEY_ID",[51,393,394],{},"AWS_SECRET_ACCESS_KEY",[51,396,397],{},"AWS_REGION",[51,399,400],{},"AWS_ENDPOINT_URL",[122,402,403],{},"the scripts bucket's credentials, only when the node has an input or an output",[108,405,406,409],{},[122,407,408],{},"the registry's variables",[122,410,411],{},"set by the administrators on the registry; secret ones through a Kubernetes Secret, never written in the Job",[108,413,414,417],{},[122,415,416],{},"the node's variables",[122,418,419],{},"as written in the node",[14,421,422,423,426,427,84],{},"A node may not redefine one of the registry's variables, nor use a name starting with\n",[51,424,425],{},"TABULARIA_"," or ",[51,428,429],{},"AWS_",[14,431,19,432,435],{},[51,433,434],{},"AWS_*"," variables are the ones most libraries read on their own (Polars, pandas\nwith s3fs, DuckDB, boto3, the AWS CLI), so reading and writing is usually one line.",[39,437,439],{"id":438},"input-and-output","Input and output",[14,441,442],{},"They are optional, and come from the node's edges, as on the Script node:",[44,444,445,455,464],{},[47,446,447,448,451,452,454],{},"an edge ",[21,449,450],{},"into"," the node: Tabularia runs the chain before it and writes the result\nto the scripts bucket, as parquet, at ",[51,453,342],{},";",[47,456,447,457,460,461,463],{},[21,458,459],{},"out of"," the node: the container must write a parquet file at\n",[51,462,356],{},"; Tabularia copies it into its storage and the nodes after it read\nit. A container that writes nothing there fails the node;",[47,465,466,469],{},[21,467,468],{},"neither",": the node is a step that runs something elsewhere (a training, a load into\nanother system, a report). The container gets no bucket credentials at all.",[14,471,472],{},"The step's files are deleted from the bucket when the node ends, whatever happened.",[14,474,475],{},"An image that does it, with Polars:",[477,478,483],"pre",{"className":479,"code":480,"language":481,"meta":482,"style":482},"language-python shiki shiki-themes github-dark-dimmed github-light","# run.py\nimport os\nimport sys\n\nimport polars as pl\n\nsource, target = os.environ.get(\"TABULARIA_INPUT\"), os.environ.get(\"TABULARIA_OUTPUT\")\ndays = int(sys.argv[sys.argv.index(\"--days\") + 1]) if \"--days\" in sys.argv else 7\nif source:\n    df = pl.scan_parquet(source).with_columns(pl.lit(days).alias(\"horizon_days\"))\n    if target:\n        df.sink_parquet(target)\n","python","",[51,484,485,494,505,513,520,534,539,564,612,620,637,646],{"__ignoreMap":482},[486,487,490],"span",{"class":488,"line":489},"line",1,[486,491,493],{"class":492},"sgHix","# run.py\n",[486,495,497,501],{"class":488,"line":496},2,[486,498,500],{"class":499},"s6PUj","import",[486,502,504],{"class":503},"ssh_m"," os\n",[486,506,508,510],{"class":488,"line":507},3,[486,509,500],{"class":499},[486,511,512],{"class":503}," sys\n",[486,514,516],{"class":488,"line":515},4,[486,517,519],{"emptyLinePlaceholder":518},true,"\n",[486,521,523,525,528,531],{"class":488,"line":522},5,[486,524,500],{"class":499},[486,526,527],{"class":503}," polars ",[486,529,530],{"class":499},"as",[486,532,533],{"class":503}," pl\n",[486,535,537],{"class":488,"line":536},6,[486,538,519],{"emptyLinePlaceholder":518},[486,540,542,545,548,551,555,558,561],{"class":488,"line":541},7,[486,543,544],{"class":503},"source, target ",[486,546,547],{"class":499},"=",[486,549,550],{"class":503}," os.environ.get(",[486,552,554],{"class":553},"sXfbr","\"TABULARIA_INPUT\"",[486,556,557],{"class":503},"), os.environ.get(",[486,559,560],{"class":553},"\"TABULARIA_OUTPUT\"",[486,562,563],{"class":503},")\n",[486,565,567,570,572,576,579,582,585,588,591,594,597,600,603,606,609],{"class":488,"line":566},8,[486,568,569],{"class":503},"days ",[486,571,547],{"class":499},[486,573,575],{"class":574},"swcJU"," int",[486,577,578],{"class":503},"(sys.argv[sys.argv.index(",[486,580,581],{"class":553},"\"--days\"",[486,583,584],{"class":503},") ",[486,586,587],{"class":499},"+",[486,589,590],{"class":574}," 1",[486,592,593],{"class":503},"]) ",[486,595,596],{"class":499},"if",[486,598,599],{"class":553}," \"--days\"",[486,601,602],{"class":499}," in",[486,604,605],{"class":503}," sys.argv ",[486,607,608],{"class":499},"else",[486,610,611],{"class":574}," 7\n",[486,613,615,617],{"class":488,"line":614},9,[486,616,596],{"class":499},[486,618,619],{"class":503}," source:\n",[486,621,623,626,628,631,634],{"class":488,"line":622},10,[486,624,625],{"class":503},"    df ",[486,627,547],{"class":499},[486,629,630],{"class":503}," pl.scan_parquet(source).with_columns(pl.lit(days).alias(",[486,632,633],{"class":553},"\"horizon_days\"",[486,635,636],{"class":503},"))\n",[486,638,640,643],{"class":488,"line":639},11,[486,641,642],{"class":499},"    if",[486,644,645],{"class":503}," target:\n",[486,647,649],{"class":488,"line":648},12,[486,650,651],{"class":503},"        df.sink_parquet(target)\n",[477,653,657],{"className":654,"code":655,"language":656,"meta":482,"style":482},"language-dockerfile shiki shiki-themes github-dark-dimmed github-light","FROM python:3.12-slim\nRUN pip install --no-cache-dir polars\nCOPY run.py \u002Fapp\u002Frun.py\nENTRYPOINT [\"python\", \"\u002Fapp\u002Frun.py\"]\n","dockerfile",[51,658,659,667,675,683],{"__ignoreMap":482},[486,660,661,664],{"class":488,"line":489},[486,662,663],{"class":499},"FROM",[486,665,666],{"class":503}," python:3.12-slim\n",[486,668,669,672],{"class":488,"line":496},[486,670,671],{"class":499},"RUN",[486,673,674],{"class":503}," pip install --no-cache-dir polars\n",[486,676,677,680],{"class":488,"line":507},[486,678,679],{"class":499},"COPY",[486,681,682],{"class":503}," run.py \u002Fapp\u002Frun.py\n",[486,684,685,688,691,694,696,699],{"class":488,"line":515},[486,686,687],{"class":499},"ENTRYPOINT",[486,689,690],{"class":503}," [",[486,692,693],{"class":553},"\"python\"",[486,695,210],{"class":503},[486,697,698],{"class":553},"\"\u002Fapp\u002Frun.py\"",[486,700,701],{"class":503},"]\n",[14,703,704,705,707,708,298,710,712,713,84],{},"With that image, the node's ",[21,706,185],{}," ",[51,709,297],{},[51,711,301],{}," run\n",[51,714,715],{},"python \u002Fapp\u002Frun.py --days 30",[39,717,719],{"id":718},"example-a-forecast-step-in-a-flow","Example: a forecast step in a flow",[14,721,722,723,59],{},"The image above, used in a flow that reads the sample «Margin by Category and Month»\n(",[21,724,725],{},"Sample \u002F Private tables",[727,728,729,767,778,791,817,841,851,865],"ol",{},[47,730,731,732],{},"Build the image and push it to your company's registry:",[477,733,737],{"className":734,"code":735,"language":736,"meta":482,"style":482},"language-bash shiki shiki-themes github-dark-dimmed github-light","docker build -t registry.acme.internal:5000\u002Fanalytics\u002Fdemand-forecast:1.4.0 .\ndocker push registry.acme.internal:5000\u002Fanalytics\u002Fdemand-forecast:1.4.0\n","bash",[51,738,739,757],{"__ignoreMap":482},[486,740,741,745,748,751,754],{"class":488,"line":489},[486,742,744],{"class":743},"sqRhv","docker",[486,746,747],{"class":553}," build",[486,749,750],{"class":574}," -t",[486,752,753],{"class":553}," registry.acme.internal:5000\u002Fanalytics\u002Fdemand-forecast:1.4.0",[486,755,756],{"class":553}," .\n",[486,758,759,761,764],{"class":488,"line":496},[486,760,744],{"class":743},[486,762,763],{"class":553}," push",[486,765,766],{"class":553}," registry.acme.internal:5000\u002Fanalytics\u002Fdemand-forecast:1.4.0\n",[47,768,769,770,773,774,59],{},"An administrator defines the registry, allows ",[51,771,772],{},"analytics\u002F*"," and gives your group\naccess to it (",[33,775,777],{"href":776},"\u002Fdocs\u002Fcontainer-registries#example-the-company-registry","the example in Container registries",[47,779,780,781,784,785,787,788,790],{},"In a flow, select the ",[21,782,783],{},"Source"," node and choose «Margin by Category and Month» as a\ncatalog datasource. Drag ",[21,786,90],{}," from the palette's ",[21,789,95],{}," group next to\nit and connect the Source to it.",[47,792,793,794,796,797,799,802,803,707,805,808,809,812,813,816],{},"In the node's panel: ",[21,795,126],{}," «Acme registry»; ",[21,798,136],{},[51,800,801],{},"analytics\u002Fdemand-forecast",", which the field suggests; ",[21,804,154],{},[51,806,807],{},"1.4.0",", from the list\nthe ↻ button reads. ",[21,810,811],{},"What the image does"," appears: it runs ",[51,814,815],{},"python \u002Fapp\u002Frun.py"," by\ndefault, for the platforms it was built for.",[47,818,819,821,822,824,825,827,828,830,831,707,833,210,836,707,838,84],{},[21,820,185],{},", one per line: ",[51,823,297],{},", then ",[51,826,301],{},". Leave ",[21,829,175],{}," empty: the image's\nentrypoint runs. ",[21,832,209],{},[51,834,835],{},"1",[21,837,213],{},[51,839,840],{},"1Gi",[47,842,843,844,847,848,84],{},"Drag ",[21,845,846],{},"Output (datasource \u002F database)"," after the node, connect it, name the\ndatasource «Demand horizon» in your folder, and ",[21,849,850],{},"Save",[47,852,853,856,857,860,861,864],{},[21,854,855],{},"Run"," in the node's panel: a node of the cluster pulls the image and runs it on the\nsources' sample. The panel says «Last run from here now: SUCCESS · 184 rows», ",[21,858,859],{},"Log","\nshows what the container printed, and the preview under the canvas has the input's\ncolumns and ",[51,862,863],{},"horizon_days"," = 30.",[47,866,867,869,870,873],{},[21,868,855],{}," in the toolbar runs the whole flow. In ",[21,871,872],{},"Runs",", the node's run shows the\nregistry and the image with its digest, the command, the attempts, the exit code and\nthe log.",[14,875,876],{},[282,877],{"alt":878,"src":879},"The same flow after Run in the node's panel: «Last run from here now: SUCCESS · 184 rows», the log with the container's two lines, and the preview under the canvas with the rows the container wrote","\u002Fmedia\u002Fguide\u002Fcontainers\u002Fcontainer-node-run-log.png",[14,881,882],{},[93,883,884,885,887,888,890,891,59],{},"Step 7: ",[21,886,855],{}," in the node's panel has run the image in the cluster. The panel says it succeeded with 184 rows, ",[21,889,859],{}," shows what the container printed, and the data view has the rows it wrote (scroll right for ",[51,892,863],{},[39,894,896],{"id":895},"a-run","A run",[727,898,899,905,911,922],{},[47,900,901,904],{},[21,902,903],{},"The image."," The tag is resolved to its digest when the run starts, and the Pod\nnames the digest: every attempt runs the same image, and the run page shows which.\nWhoever the run belongs to must still be allowed the registry and the image (for a\nschedule: the person who set it).",[47,906,907,910],{},[21,908,909],{},"The pull Secret."," A registry with credentials has its pull Secret written in the\ncontainers' namespace right before the Pod needs it.",[47,912,913,916,917,919,920,84],{},[21,914,915],{},"The Job",", in the containers' namespace. A failed attempt is run again as a new\nJob, as many times as ",[21,918,240],{}," says, after ",[21,921,243],{},[47,923,924,927,928,930],{},[21,925,926],{},"The outcome."," A failure stops the flow, as a failed Script node does: the steps\nafter it would read nothing, or stale data. A download the node cannot make (a\ncertificate it does not trust, credentials refused, an image gone) fails within\nabout twenty seconds, saying why. A container that goes over its ",[21,929,213],{}," is\nstopped by the cluster, and the run says so (exit code 137).",[14,932,933,934,936,937,940,941,944],{},"The run is a child of the flow's run, shown in ",[21,935,872],{}," with the node's name: the\nregistry and the image with its digest, the command, the attempts, the exit code, the\nresources, the variables it was given (the secret ones by name only) and the end of its\nlog. The log is read when the Job ends (the last 256 KB, ",[51,938,939],{},"CONTAINERS__LOG_MAX_BYTES",")\nand kept in Tabularia's database, not in the cluster: the Job goes once it is read.\nAfter 30 days (",[51,942,943],{},"ORCHESTRATOR__LOG_RETENTION_DAYS","; 0 keeps them) it is emptied, and the\nrun stays.",[14,946,947,949],{},[21,948,855],{},", in the node's panel, runs the node of the saved flow (saving it first) on the\nsources' development sample, without the nodes after it, in the trials' namespace.\nWhat it writes becomes the preview of the nodes after it, until the node changes.",[14,951,19,952,955,956,960,961,964,965,967],{},[21,953,954],{},"Lineage"," page draws the image (registry and repository) with the flows and\nflow groups that run it. With ",[33,957,959],{"href":958},"\u002Fdocs\u002Fopenlineage","OpenLineage"," on, a run of the node is a\nchild job of the flow's run (a cell's, of the group's run), reading the datasources\nwhose data reaches it, with the image — registry, repository, tag and digest — in the\n",[51,962,963],{},"tabularia_container"," job facet. An editor's ",[21,966,855],{}," is not reported.",[39,969,971],{"id":970},"where-it-runs","Where it runs",[102,973,974,982],{},[105,975,976],{},[108,977,978,980],{},[111,979],{},[111,981],{},[117,983,984,1002,1013,1021,1036],{},[108,985,986,989],{},[122,987,988],{},"namespace",[122,990,991,994,995,998,999,1001],{},[51,992,993],{},"containers.namespace"," for the flows' runs, ",[51,996,997],{},"containers.trialNamespace"," for the editor's ",[21,1000,855],{},", under a ResourceQuota",[108,1003,1004,1007],{},[122,1005,1006],{},"Pod Security",[122,1008,1009,1012],{},[51,1010,1011],{},"baseline",": the container may run as root, as most existing images do; privileged containers, host paths, host network and added capabilities are refused",[108,1014,1015,1018],{},[122,1016,1017],{},"service account",[122,1019,1020],{},"none: the Pod gets no token of the cluster's API",[108,1022,1023,1026],{},[122,1024,1025],{},"privileges",[122,1027,1028,1031,1032,1035],{},[51,1029,1030],{},"allowPrivilegeEscalation: false"," (no ",[51,1033,1034],{},"sudo"," inside)",[108,1037,1038,1041],{},[122,1039,1040],{},"network",[122,1042,1043,1044,1046,1047,1050,1051],{},"the internet and the other namespaces of the cluster, never the pods of Tabularia's namespace nor the cloud metadata addresses; private networks outside the cluster only when listed in ",[51,1045,83],{},"; nothing reaches into the container (",[51,1048,1049],{},"containers.blockTabularia",", on by default). The details, and what to list, in ",[33,1052,1054],{"href":1053},"\u002Fdocs\u002Fscripting#network-policies","Python and R in flows",[39,1056,1058],{"id":1057},"who-may-do-what","Who may do what",[44,1060,1061,1067,1070,1077],{},[47,1062,1063,1064,59],{},"An administrator decides which registries exist, which repositories of each may run,\nand who may use them (",[33,1065,58],{"href":1066},"\u002Fdocs\u002Fcontainer-registries#who-can-use-it",[47,1068,1069],{},"Someone with access may add a Container node, with an image granted to them, to a flow\nthey may edit, and run it.",[47,1071,1072,1073,1076],{},"Someone without access to a flow's registry or image ",[21,1074,1075],{},"cannot open the flow",": it is\nlisted greyed out, with the reason.",[47,1078,1079],{},"A schedule runs with the authority of whoever set it: if another person changes the\nnode's image, command, arguments or variables, the schedule is switched off, as for\nany change that widens what a schedule does.",[39,1081,1083],{"id":1082},"limits","Limits",[44,1085,1086,1092,1102],{},[47,1087,19,1088,1091],{},[21,1089,1090],{},"dbt export"," cannot carry a Container node: an output that reads what one\nwrites refuses the export; a node no exported output reads from is left out, with a\nnote.",[47,1093,1094,1097,1098,1101],{},[21,1095,1096],{},"Architectures."," The image must exist for the nodes' architecture (amd64, arm64):\na multi-architecture image works everywhere; an amd64-only image on an arm64 node\nends with ",[51,1099,1100],{},"exec format error"," in its log.",[47,1103,1104,1107],{},[21,1105,1106],{},"Registries with short-lived tokens"," (ECR) and the cluster's own identity are not\nsupported yet: a registry needs a user and a password or token, or none.",[39,1109,1111],{"id":1110},"settings","Settings",[102,1113,1114,1128],{},[105,1115,1116],{},[108,1117,1118,1120,1123,1126],{},[111,1119,331],{},[111,1121,1122],{},"Chart value",[111,1124,1125],{},"Default",[111,1127],{},[117,1129,1130,1150,1170,1190],{},[108,1131,1132,1137,1142,1147],{},[122,1133,1134],{},[51,1135,1136],{},"CONTAINERS__MAX_CPU",[122,1138,1139],{},[51,1140,1141],{},"containers.limits.cpu",[122,1143,1144],{},[51,1145,1146],{},"4",[122,1148,1149],{},"the most CPU a node may ask for",[108,1151,1152,1157,1162,1167],{},[122,1153,1154],{},[51,1155,1156],{},"CONTAINERS__MAX_MEMORY",[122,1158,1159],{},[51,1160,1161],{},"containers.limits.memory",[122,1163,1164],{},[51,1165,1166],{},"16Gi",[122,1168,1169],{},"the most memory",[108,1171,1172,1177,1182,1187],{},[122,1173,1174],{},[51,1175,1176],{},"CONTAINERS__MAX_TIMEOUT_SECONDS",[122,1178,1179],{},[51,1180,1181],{},"containers.limits.timeoutSeconds",[122,1183,1184],{},[51,1185,1186],{},"86400",[122,1188,1189],{},"the longest timeout",[108,1191,1192,1196,1198,1203],{},[122,1193,1194],{},[51,1195,939],{},[122,1197],{},[122,1199,1200],{},[51,1201,1202],{},"262144",[122,1204,1205],{},"the end of the log a run keeps",[14,1207,1208,1209,84],{},"All of them in ",[33,1210,1212],{"href":1211},"\u002Fdocs\u002Fconfiguration","the configuration reference",[1214,1215,1216],"style",{},"html pre.shiki code .sgHix, html code.shiki .sgHix{--shiki-default:#768390;--shiki-light:#6A737D}html pre.shiki code .s6PUj, html code.shiki .s6PUj{--shiki-default:#F47067;--shiki-light:#D73A49}html pre.shiki code .ssh_m, html code.shiki .ssh_m{--shiki-default:#ADBAC7;--shiki-light:#24292E}html pre.shiki code .sXfbr, html code.shiki .sXfbr{--shiki-default:#96D0FF;--shiki-light:#032F62}html pre.shiki code .swcJU, html code.shiki .swcJU{--shiki-default:#6CB6FF;--shiki-light:#005CC5}html .default .shiki span {color: var(--shiki-default);background: var(--shiki-default-bg);font-style: var(--shiki-default-font-style);font-weight: var(--shiki-default-font-weight);text-decoration: var(--shiki-default-text-decoration);}html .shiki span {color: var(--shiki-default);background: var(--shiki-default-bg);font-style: var(--shiki-default-font-style);font-weight: var(--shiki-default-font-weight);text-decoration: var(--shiki-default-text-decoration);}html .light .shiki span {color: var(--shiki-light);background: var(--shiki-light-bg);font-style: var(--shiki-light-font-style);font-weight: var(--shiki-light-font-weight);text-decoration: var(--shiki-light-text-decoration);}html.light .shiki span {color: var(--shiki-light);background: var(--shiki-light-bg);font-style: var(--shiki-light-font-style);font-weight: var(--shiki-light-font-weight);text-decoration: var(--shiki-light-text-decoration);}html pre.shiki code .sqRhv, html code.shiki .sqRhv{--shiki-default:#F69D50;--shiki-light:#6F42C1}",{"title":482,"searchDepth":496,"depth":507,"links":1218},[1219,1220,1221,1222,1223,1224,1225,1226,1227,1228,1229],{"id":41,"depth":496,"text":42},{"id":99,"depth":496,"text":100},{"id":305,"depth":496,"text":306},{"id":321,"depth":496,"text":322},{"id":438,"depth":496,"text":439},{"id":718,"depth":496,"text":719},{"id":895,"depth":496,"text":896},{"id":970,"depth":496,"text":971},{"id":1057,"depth":496,"text":1058},{"id":1082,"depth":496,"text":1083},{"id":1110,"depth":496,"text":1111},"md",{"source":1232},"docs\u002Fcontainer-node.md","\u002Fdocs\u002Fcontainer-node",{"title":5,"description":16},"docs\u002Fcontainer-node","o1PxUczz4tvAVSfhqfjUWfViZIFx1i8aAni7bVfNc6A",[1238,1257,1310,1342,1373,1400,1426,1451],{"title":1239,"icon":1240,"intro":1241,"items":1242},"Getting started","start","Install it on one machine, learn the few ideas everything else is built on, and run a first flow.",[1243,1245,1248,1251,1254],{"path":1244,"title":1239},"\u002Fdocs\u002Fgetting-started",{"path":1246,"title":1247},"\u002Fdocs\u002Fgetting-started-docker-compose","Walkthrough with Docker Compose",{"path":1249,"title":1250},"\u002Fdocs\u002Fdeploy\u002Fminikube","Try it on your computer (minikube)",{"path":1252,"title":1253},"\u002Fdocs\u002Fconcepts","Concepts",{"path":1255,"title":1256},"\u002Fdocs\u002Foverview","Overview (README)",{"title":1258,"icon":1259,"intro":1260,"items":1261},"Capabilities","capabilities","What you can build: visual flows, analysis on any step, AI questions, pixel-perfect reports, dashboards, Python, R and SQL scripts, containers and the orchestration that ties them together.",[1262,1265,1268,1271,1274,1277,1280,1283,1285,1288,1291,1293,1295,1298,1301,1304,1307],{"path":1263,"title":1264},"\u002Fdocs\u002Fcapabilities","All capabilities",{"path":1266,"title":1267},"\u002Fdocs\u002Fcapabilities\u002Fflows","Low-code flows",{"path":1269,"title":1270},"\u002Fdocs\u002Fcapabilities\u002Fpreview-and-analysis","Preview and analysis",{"path":1272,"title":1273},"\u002Fdocs\u002Fcapabilities\u002Fask-your-data","Ask your data with AI",{"path":1275,"title":1276},"\u002Fdocs\u002Fcapabilities\u002Freports","Pixel-perfect reports",{"path":1278,"title":1279},"\u002Fdocs\u002Fcapabilities\u002Freport-functions","Report functions",{"path":1281,"title":1282},"\u002Fdocs\u002Fcapabilities\u002Fdashboards","Dashboards",{"path":35,"title":1284},"Python and R",{"path":1286,"title":1287},"\u002Fdocs\u002Fsql-scripts","SQL scripts",{"path":1289,"title":1290},"\u002Fdocs\u002Fcapabilities\u002Fnotebooks","marimo notebooks as apps",{"path":1233,"title":1292},"Docker images in flows",{"path":312,"title":1294},"Flow groups: orchestration",{"path":1296,"title":1297},"\u002Fdocs\u002Fcapabilities\u002Fscheduling","Scheduling",{"path":1299,"title":1300},"\u002Fdocs\u002Fpushmail","Pushmail: datasources, reports and dashboards by email",{"path":1302,"title":1303},"\u002Fdocs\u002Fplaceholders","Dynamic dates",{"path":1305,"title":1306},"\u002Fdocs\u002Fcapabilities\u002Fengines","Engines",{"path":1308,"title":1309},"\u002Fdocs\u002Fengines\u002Fengine-differences","Engine differences",{"title":1311,"icon":1312,"intro":1313,"items":1314},"Sources","sources","Every way data comes in: files, databases, buckets, SFTP folders, HTTP APIs, SharePoint, Google Sheets, and the results of your own flows.",[1315,1318,1321,1324,1327,1330,1333,1336,1339],{"path":1316,"title":1317},"\u002Fdocs\u002Fsources","All sources",{"path":1319,"title":1320},"\u002Fdocs\u002Fsources\u002Ffiles","Uploaded files",{"path":1322,"title":1323},"\u002Fdocs\u002Fsources\u002Fdatabases","Database tables and queries",{"path":1325,"title":1326},"\u002Fdocs\u002Fs3-datasource","S3 buckets",{"path":1328,"title":1329},"\u002Fdocs\u002Fsftp","SFTP folders",{"path":1331,"title":1332},"\u002Fdocs\u002Fapi-datasource","HTTP APIs",{"path":1334,"title":1335},"\u002Fdocs\u002Fsources\u002Fsharepoint","Excel on SharePoint",{"path":1337,"title":1338},"\u002Fdocs\u002Fsources\u002Fgoogle-sheets","Google Sheets",{"path":1340,"title":1341},"\u002Fdocs\u002Fsources\u002Fflow-outputs","Results of flows and scripts",{"title":1343,"icon":1344,"intro":1345,"items":1346},"Connections","connections","The systems Tabularia reads from and writes to, how each connection is set up, and who may use it.",[1347,1350,1353,1356,1359,1361,1362,1364,1365,1368,1370],{"path":1348,"title":1349},"\u002Fdocs\u002Fconnections","All connections",{"path":1351,"title":1352},"\u002Fdocs\u002Fconnections\u002Fdatabases","Databases",{"path":1354,"title":1355},"\u002Fdocs\u002Fsqlserver","Microsoft SQL Server driver",{"path":1357,"title":1358},"\u002Fdocs\u002Fconnections\u002Fs3","S3 and compatible storage",{"path":1328,"title":1360},"SFTP servers",{"path":1331,"title":1332},{"path":1334,"title":1363},"SharePoint",{"path":1337,"title":1338},{"path":1366,"title":1367},"\u002Fdocs\u002Fconnections\u002Fsmtp","SMTP email",{"path":1369,"title":58},"\u002Fdocs\u002Fcontainer-registries",{"path":1371,"title":1372},"\u002Fdocs\u002Fconnections\u002Fdestinations","Writing data out",{"title":1374,"icon":1375,"intro":1376,"items":1377},"Integrations","integrations","How Tabularia fits your stack: AI providers, the REST API and webhooks, dbt, OpenLineage, single sign-on, SIEM and monitoring.",[1378,1381,1384,1387,1390,1391,1394,1397],{"path":1379,"title":1380},"\u002Fdocs\u002Fintegrations","All integrations",{"path":1382,"title":1383},"\u002Fdocs\u002Fintegrations\u002Fai","AI: where it is used",{"path":1385,"title":1386},"\u002Fdocs\u002Fapi","API, tokens and webhooks",{"path":1388,"title":1389},"\u002Fdocs\u002Fdbt-export","Export to dbt",{"path":958,"title":959},{"path":1392,"title":1393},"\u002Fdocs\u002Fsso-keycloak","Single sign-on with Keycloak",{"path":1395,"title":1396},"\u002Fdocs\u002Fsiem-splunk","Audit log to Splunk",{"path":1398,"title":1399},"\u002Fdocs\u002Fintegrations\u002Fmonitoring","Monitoring",{"title":1401,"icon":1402,"intro":1403,"items":1404},"Security & governance","governance","Who may see and do what, what is recorded, and how data quality and personal data are kept in check.",[1405,1408,1411,1414,1417,1420,1423],{"path":1406,"title":1407},"\u002Fdocs\u002Fgovernance","Overview",{"path":1409,"title":1410},"\u002Fdocs\u002Fgovernance\u002Fpermissions","Permissions",{"path":1412,"title":1413},"\u002Fdocs\u002Fgovernance\u002Faudit-log","Audit log",{"path":1415,"title":1416},"\u002Fdocs\u002Fgovernance\u002Fdata-contracts","Data contracts",{"path":1418,"title":1419},"\u002Fdocs\u002Fdesign\u002Fsso-group-mapping","SSO and group mapping",{"path":1421,"title":1422},"\u002Fdocs\u002Fexamples\u002Fkeycloak","Keycloak example on a laptop",{"path":1424,"title":1425},"\u002Fdocs\u002Fengines\u002Fclickhouse-ai-user","A ClickHouse user for the AI assistant",{"title":1427,"icon":1428,"intro":1429,"items":1430},"Install & operate","operate","Run it in production on Kubernetes, configure it, upgrade it and keep it healthy.",[1431,1434,1437,1440,1443,1445,1448],{"path":1432,"title":1433},"\u002Fdocs\u002Fdeploy\u002Fkubernetes","Kubernetes and Helm",{"path":1435,"title":1436},"\u002Fdocs\u002Fdeploy\u002Fkind","A company-like cluster on one computer (kind)",{"path":1438,"title":1439},"\u002Fdocs\u002Fdeploy\u002Fupgrading","Upgrading, backups and restore",{"path":1441,"title":1442},"\u002Fdocs\u002Fdeploy\u002Frelease-checklist","Install and upgrade checklist",{"path":1211,"title":1444},"Configuration reference",{"path":1446,"title":1447},"\u002Fdocs\u002Fengine-settings","Settings from the panel",{"path":1449,"title":1450},"\u002Fdocs\u002Fdatabase-migrations","Database migrations",{"title":1452,"icon":1453,"intro":1454,"items":1455},"Reference","reference","Release process and the remaining reference pages.",[1456],{"path":1457,"title":1458},"\u002Fdocs\u002Frelease-process","Release process",1793656371142]