Flow Features And Config Toggles

Breyta feature matrix for authoring and operating advanced flows.

Goal

Provide one checklist of core flow fields, step capabilities, and operational toggles.

Quick Answer

Use this page to verify that your flow covers invocations, interfaces, schedules, templates, waits, notifications, persistence, table resources, and bindings with canonical config shapes.
For the first draft proof, inline small one-off bodies when that is fastest. As
complexity grows, keep :flow orchestration-focused and move repeated or bulky
content into :functions, :templates, or packaged :steps.
Treat :persist as a common default for data-heavy steps, not a niche option.

Flow Definition Surface

For detailed overlap/version behavior choices, see Run Concurrency.

Interface And Schedule Choice By Target

Step Capabilities And Toggles

Orchestration Primitives (Non-Step)

Use flow/call-flow when one branch grows into a separate lifecycle (for example: ingress flow -> routing flow -> apply flow). Keep parent flow responsible for orchestration and move shaping into top-level :functions.

'(let [input (flow/input)
       normalized (flow/step :function :normalize {:ref :normalize :input input})
       routed (flow/call-flow :billing-route (:route-input normalized))
       result ^{:label "Apply routed action?"
                :yes "Call apply flow"
                :no "Skip apply"}
              (if (:should-apply routed)
                (flow/call-flow :billing-apply (:apply-input routed))
                {:skipped true})]
   {:routed routed
    :result result})

See construct metadata and polling details:

• Orchestration Constructs
• flow/poll

Use :fanout when the parent needs a bounded batch of sibling child workflows that can start now and collect later.
That async path is intentionally narrow:

• all items must be {:type :call-flow ...}
• effective concurrency must be at most 5; validation rejects larger values
• nested async child fanout is rejected beyond one level
• legacy non-child fanout items still work only on the sequential compatibility path

Example:

'(let [input (flow/input)
       fanout (flow/step :fanout :spawn-children
                         {:items (:items input)
                          :max-concurrency 5
                          :on-error :collect})]
   {:fanout fanout})

Loop pattern for bounded item processing with explicit persistence:

'(let [input (flow/input)
       items (:items input)
       batch (reduce (fn [acc item]
                       (let [item-result (flow/step :http :process-item
                                           {:connection :api
                                            :method :post
                                            :path "/process"
                                            :json {:id (:id item)}
                                            :persist {:type :blob}})]
                         (conj acc item-result)))
                     []
                     items)
       summary (flow/step :function :summarize
                 {:ref :summarize
                  :input batch})]
   summary)

Table-resource pattern for create-on-write persistence plus later bounded queries:

'(let [orders (flow/step :http :fetch-orders
                {:url "https://example.com/orders"
                 :persist {:type :table
                           :table "orders"
                           :rows-path [:body :items]
                           :write-mode :upsert
                           :key-fields [:order-id]
                           :columns [{:column :customer-id
                                      :semantic-type :reference
                                      :reference {:table "customers"
                                                  :remote-field :customer-id}}
                                     {:column :customer-name
                                      :semantic-type :text
                                      :computed {:type :lookup
                                                 :reference-column :customer-id
                                                 :field :name}}]}})
       open-orders (flow/step :table :query-open-orders
                     {:op :query
                      :table orders
                      :where [[:status := "open"]]
                      :sort [[:order-id :asc]]
                      :page {:mode :offset
                             :limit 25}})
       define-summary (flow/step :table :define-summary
                        {:op :set-column
                         :table orders
                         :column :order-summary
                         :definition {:semantic-type :text
                                      :computed {:type :expr
                                                 :expr {:op :concat
                                                        :args [{:field :customer-name}
                                                               " / "
                                                               {:field :status}]}}}})]
   {:orders orders
    :open-orders open-orders
    :define-summary define-summary})

For bounded tables, :set-column automatically backfills existing rows. :recompute remains available when you want to rerun derived/reference values later.

Dynamic enum columns are also part of the same authored-column surface:

'(flow/step :table :define-status-enum
   {:op :set-column
    :table orders
    :column :status
    :definition {:display-name "Status"
                 :enum {:options [{:id "open"
                                   :name "Open"
                                   :aliases ["OPEN" "Open"]}
                                  {:id "in-progress"
                                   :name "In progress"
                                   :aliases ["IN_PROGRESS" "In Progress"]}]}}})

Enum behavior:

• :enum implies type-hint "enum"
• writes, :update-cell, CSV import, and :recompute normalize incoming scalar values to stable ids
• matching accepts existing ids, names, and aliases
• unknown values dynamically grow the enum definition with a normalized id and a derived display name
• stored rows and query/export surfaces keep the normalized ids, while the web preview renders names

Display formatting remains render-only:

• column :format metadata and sparse :update-cell-format overrides can render relative-time, date, timestamp / date-time, and currency
• the web preview and Copy Markdown apply those formats to the currently visible page
• query/export surfaces keep canonical raw values

Use :materialize-join when the right product shape is a new derived table rather than more derived columns on an existing table:

'(let [rows (:rows (flow/input))
       joined (flow/step :table :materialize-orders
                {:op :materialize-join
                 :left {:rows rows}
                 :right {:table "customers"
                         :select [:customer-id :name]}
                 :join-type :left
                 :on [{:left-field :customer-id
                       :right-field :customer-id}]
                 :project {:keep-left :all
                           :right-fields [{:field :name :as :customer-name}]}
                 :into {:table "orders-enriched"
                        :write-mode :upsert
                        :key-fields [:order-id]}})
       joined-preview (flow/step :table :query-joined
                        {:op :query
                         :table joined
                         :page {:mode :offset
                                :limit 25}})]
   {:joined joined
    :joined-preview joined-preview})

Current :materialize-join boundary:

• flow/runtime-backed contract with matching CLI/API entrypoints
• equality joins only
• :left and :inner only
• same-workspace table sources only
• right-side cardinality 0..1
• destination table materialization only
• no resource-panel/manual UI replay surface

Current :materialize-join semantics:

• incremental materialization only via :append or :upsert
• no snapshot/replace mode yet
• joins read the current materialized row state of source tables
• run :recompute first if source-table computed/reference values must be refreshed before the join

Dedicated :materialize-join limits:

• inline :left {:rows ...} max: 1000
• table-source window max per side: 10000
• output row max: 10000
• join key max: 4
• projected right-field max: 64

Table-resource limits to design against:

• 500 table resources (families) per workspace
• 50_000 live rows per concrete table inside a family
• 200 columns per table
• 16 promoted/index fields per table
• 128 partitions per family
• 16 partitions touched per write
• 12 selected partitions per query/aggregate/export
• 24 selected partitions per preview/read/schema
• 256 max partition key bytes
• 64 KB per cell
• 256 KB max row payload
• 256 MB max table size
• 2 GB max workspace table DB size
• 1000 rows per write
• 1000 rows per query page
• 10000 max query scan window via page.offset + page.limit
• 200 max aggregate groups

Bounded aggregate reporting now also supports:

• order-by on group keys or metric aliases
• has-more truncation metadata
• metric-local conditional filters
• scalar arg-max / arg-min
• having on grouped results
• bounded collect-set
• date-bucket group specs for day, week, and month
• numeric-bin group specs
• percentile and median distribution metrics

Current recommendation when one logical dataset approaches bounded-table limits:

• keep 50_000 live rows per concrete table or partition as a real boundary
• use first-class :partitioning when the data naturally partitions by region, tenant, source, or a date bucket and most reads/writes stay within one partition or a small bounded subset
• keep the family root as the schema/metadata owner and select partition scope explicitly for query-like operations
• use separate explicit tables when the data truly represents different datasets or lifecycles, not just as a workaround for missing partition support
• if authors mainly rely on wide cross-partition scans, arbitrary joins, or general database behavior, move the workload to a dedicated :db step and an external database/query system instead of stretching bounded tables further

Security And Runtime Controls

High-Impact Limits

See full list: Limits And Recovery

CLI Commands That Surface These Features

Related

• Flow Definition
• Orchestration Constructs
• flow/poll
• Step Table
• Templates
• Webhooks And Secret Refs
• Waits, Signals, And Timeouts
• Notifications And Email