Step Files (:files)

Quick Answer

Use :files to:

• resolve an immutable source-tree-ref from a source spec such as :git
• fork a persistent changeset-ref from that source tree
• query either resource through :list, :read, and :search
• mutate only the changeset-ref through :write-file, :apply-edit, :replace, :replace-lines, :delete-file, and :move-file
• render the current logical view into a temporary local directory through :materialize
• capture filesystem edits from that directory back into the changeset through :capture
• compare the current logical view to the base source through :diff
• publish a changeset-ref to a branch through :publish
• open or reuse a pull request from that published branch through :open-change-request

Canonical import shape:

{:type :files :op :resolve-source :source {:type :git :repo "..." :ref "main"}}

Canonical writable flow:

1. :resolve-source -> source-tree-ref
2. :init-changeset -> changeset-ref
3. mutate the changeset
4. query the overlaid current view through the same read/list/search ops
5. :publish the changeset to a branch
6. :open-change-request from that published branch

Physical model:

• the runtime may clone/fetch into a temporary worker directory during import
• that temporary checkout is deleted after import
• the canonical source tree and changeset manifests live in Breyta's managed storage
• later :list, :read, :search, :materialize, and :diff operate on the persisted logical view; the runtime may use a disposable temporary local source cache for unchanged files, but that cache is not authoritative

Canonical Shape

Core fields:

:resolve-source

Connection note:

• git imports resolve :source.connection through the normal connection model
• for GitHub https://github.com/... sources, the resolved connection must currently be an :http-api connection so the runtime can use the GitHub REST/archive import path
• local repositories and non-GitHub git remotes use direct git import when no connection is supplied
• connection-backed non-GitHub imports require an explicit registered :source.provider

:init-changeset

This creates a new persisted changeset-ref. When the input is already a
changeset, the new one forks that overlay state instead of mutating the parent.

:list

:read

Exactly one of :path or :paths must be provided.

When :line-start or :line-end is present:

• use :path, not :paths
• the slice is bounded to at most 400 lines
• omitted :line-end reads from :line-start up to the bounded slice limit
• bounded read results always include :full-read-args
• bounded reads include :recommended-edit-op: :replace-lines for small edits, :write-file for full rewrites
• :replace-lines-args appears only when the slice is safe for bounded block edit

Full-file reads include copy-forward :write-file-args.

:search

Content hits include :read-args, which you can usually copy directly into a bounded follow-up :read.

:write-file

:apply-edit

:apply-edit remains supported, but direct :replace and :replace-lines ops are preferred for agent/tool use.

:replace

:replace-lines

:delete-file

:move-file

:diff

:materialize

This renders the current logical view into a materialized working area.
When the files service is enabled, the working area is service-owned and the
step returns a files-session://... handle plus lease fields instead of a raw
directory path. Without the files service, the fallback implementation returns a
temporary worker-local directory.

It is a one-way execution bridge only:

• persisted source-tree and changeset resources remain authoritative
• filesystem-side mutations do not write back into the logical resource state
• returned directories are temporary and disposable, not durable resource refs
• service-owned sessions are durable handoff handles, but callers must use the
  returned :lease-id and :generation for follow-up :capture calls
• deployed workers should point the materialization root and source-cache root
  at an explicit size-limited disk-backed mount such as
  /var/lib/flows-worker/files/..., instead of relying on generic process temp
  space inside the pod

To import changes made on disk back into the changeset, use :capture.

:capture

This is the reverse of :materialize — it reads files from a materialized
working area and writes any changes back into the changeset. Only files that
differ from the current changeset view are written; unchanged files are skipped.

Capture treats the directory as the desired state for the selected scope:

• changed files are written into the changeset
• new empty files are captured like any other file
• files missing from the directory are captured as deletes
• files larger than the capture size limit fail explicitly instead of being silently skipped

Typical pattern:

'(let [mat (flow/step :files :materialize {:op :materialize :source changeset})
       ;; Run an external tool against the materialized directory
       _ (flow/step :job :run-linter {:directory (:directory mat)})
       ;; Capture any fixes the tool made back into the changeset
       updated (flow/step :files :capture-fixes
                 {:op :capture
                  :changeset changeset
                  :directory (:directory mat)})]
   updated)

With files-service sessions, capture uses the returned handle and lease instead
of a directory:

'(let [mat (flow/step :files :materialize {:op :materialize :source changeset})
       updated (flow/step :files :capture-fixes
                 {:op :capture
                  :session-uri (:session-uri mat)
                  :lease-id (:lease-id mat)
                  :generation (:generation mat)})]
   updated)

If the session was materialized from a source-tree-ref rather than a
changeset-ref, include :changeset in the capture request so the service has
an explicit writeback target.

:heartbeat, :status, :release

These lifecycle ops only apply to files-service sessions.

• :heartbeat renews the lease expiration without capturing changes.
• :status returns the public session metadata.
• :release marks the session unusable and lets the files service reclaim the
  service-owned working directory.
• Expired sessions reject further heartbeat/capture/release attempts. The files
  service also reclaims expired service-owned roots on startup and before new
  materialization allocation so abandoned sessions do not keep consuming
  workspace disk quota indefinitely.

:publish

Current behavior:

• GitHub is the built-in provider; registered custom providers can handle their own publish path
• branch/commit publication happens through the provider API, not a temporary local git worktree
• the bound GitHub connection must be able to read repository metadata/branches and create blobs, trees, commits, refs, and pull requests for the target repository
• local validation or tests are not run on the flows worker before publishing
• by default :publish refuses provider/repo/branch/connection retargeting away from the changeset origin
• set :allow-target-override? true only for intentional retargeting
• :allow-target-override? is an authored runtime escape hatch, not an agent-facing affordance. Built-in :files tool calls do not let the model set it directly.

:open-change-request

Provide either:

• :published from :files :publish, or
• explicit :repository, :base-branch, and :branch

When :published is supplied, :open-change-request preserves that published
target by default and rejects conflicting repo, base-branch, or connection
overrides unless :allow-target-override? true is set.

Like :publish, :allow-target-override? on :open-change-request is intended
for authored flow logic only. If an agent needs an intentional retargeting
surface, package that behavior behind a flow-local packaged :steps definition.

Runtime Behavior

• :resolve-source currently imports git repositories only.
• :resolve-source uses the bound connection and disables ambient git credential helpers.
• GitHub sources with a bound :http-api connection use GitHub REST/archive reads.
• GitHub 401/403 can retry anonymously for public repositories.
• Anonymous retry does not bypass private repo auth.
• GitHub credential failures keep connection-id in error context.
• local repositories and non-GitHub git sources still use the direct git import path.
• :publish and :open-change-request resolve auth from the same bound :http-api connection rather than ambient worker credentials, so branch writes and pull-request creation should reflect the configured connection's repository access.
• :resolve-source returns an immutable source-tree-ref.
• repeated :resolve-source calls for the same repo, resolved commit, sparse-path selection, and retention window may reuse the existing stored source-tree-ref instead of reimporting file content.
• :init-changeset returns a persisted logical overlay over that source tree.
• :write-file, :apply-edit, :replace, :replace-lines, :delete-file, and :move-file only mutate the changeset resource.
• :list, :read, and :search work against the current logical view:
  • a plain source tree if :source is a source-tree-ref
  • the source tree plus changeset overlay if :source is a changeset-ref
• text files are stored as persisted content in Breyta storage.
• repeated :read and :search calls may use a local cache for unchanged source-tree files instead of re-fetching from storage.
• this cache is only an optimization; the stored source-tree and changeset resources are authoritative.
• binary files still appear in :list, can be moved/deleted through a changeset, and can match path searches.
• imported source trees persist binary file bytes as well as text file bytes so :materialize can render the full current logical view.
• :read rejects binary files instead of returning inline bytes.
• :read supports bounded single-file slices through :line-start / :line-end.
• bounded :read results include :line-start, :line-end, :total-lines, :truncated?, :recommended-edit-op, and :full-read-args.
• bounded :read results only include copy-forward :replace-lines-args when the reread slice is small enough for a safe bounded block edit.
• full-file single-path :read results include copy-forward :write-file-args and :recommended-edit-op set to :write-file.
• :search skips binary content reads but still matches binary paths by filename/path text.
• content search hits include :match-line and copy-forward :read-args for the matched file.
• :apply-edit only works on existing text files in the logical view.
• :apply-edit supports two deterministic edit modes:
  • :replace for exact substring replacement
  • :replace-lines for bounded line-window replacement with stale-slice protection
• direct :replace and :replace-lines ops are aliases for those same deterministic edit modes and are the preferred surface for agent/tool calls.
• :apply-edit :replace is best for short exact snippets you just reread, and fails when :edit.match is missing.
• :apply-edit :replace also fails on multiple matches unless :edit.all? true is set.
• when a multiline exact replace cannot be matched safely, the validation message explicitly guides callers toward a bounded reread plus :replace-lines.
• :apply-edit :replace-lines requires :edit.line-start, :edit.line-end, :edit.expected, and :edit.replace.
• :apply-edit :replace-lines is bounded to at most 400 lines and fails if the current line window no longer matches :edit.expected; reread the file slice and retry in that case.
• prefer :write-file after a full-file read when you are replacing most of a file or a whole function/body that no longer fits comfortably inside a bounded reread window.
• without the files service, :materialize writes the current logical view to a temporary local directory under the configured materialization root (or the process temp directory by default).
• deployed environments should set explicit size-limited disk-backed roots for
  both materialization and source-cache paths; in breyta-env the intended
  worker mount is /var/lib/flows-worker/files.
• :materialize is one-way by itself: if a later tool edits files in that directory, use :capture to sync those edits back into the changeset-ref.
• :publish targets the selected source provider and publishes the current logical diff through that provider. The built-in GitHub provider creates provider-side blobs, a tree, a commit, and a branch ref.
• the built-in GitHub provider resolves the base branch through GitHub repository branch/commit APIs before creating new Git database objects, which gives clearer errors for missing branches or inaccessible repositories.
• :publish uses regular file mode (100644); executable mode metadata is not preserved.
• :open-change-request targets the selected source provider. The built-in GitHub provider will reuse an existing open pull request for the published branch when GitHub rejects duplicate creation.
• success for this fixer path is bounded edit + publish + pull request; local validation on the flows worker is not part of the contract.
• source trees and changesets inherit bounded TTL/retention in the persisted manifest/content/resource path.
• structured resources such as tables remain table-native and are not flattened into :files automatically.

Result Shapes

:resolve-source

Returns a source-tree-ref previewing the imported tree:

{:type :resource-ref
 :uri "res://..."
 :content-type "application/vnd.breyta.source-tree+json"
 :preview {:source-type :git
           :repo "https://github.com/acme/repo.git"
           :ref "main"
           :commit-sha "abc123..."
           :file-count 42}}

:init-changeset, :write-file, :apply-edit, :delete-file, :move-file

These return the current changeset-ref:

{:type :resource-ref
 :uri "res://..."
 :content-type "application/vnd.breyta.files-changeset+json"
 :preview {:source-uri "res://...source-tree..."
           :change-count 3}}

:list

{:source "res://..."
 :count 3
 :limit 200
 :items [{:path "README.md"
          :size-bytes 121
          :content-type "text/markdown"
          :binary? false}
         {:path "assets/logo.bin"
          :size-bytes 2048
          :content-type "application/octet-stream"
          :binary? true}]}

:read

{:source "res://..."
 :count 1
 :items [{:path "src/app/core.clj"
          :size-bytes 84
          :content-type "text/plain"
          :content "(def persisted-value \"ready\")"
          :line-start 2
          :line-end 2
          :total-lines 2
          :truncated? true}]}

:search

{:source "res://..."
 :query "persisted-value"
 :count 1
 :limit 20
 :results [{:path "src/app/core.clj"
            :match :content
            :snippet "...persisted-value..."
            :size-bytes 84
            :content-type "text/plain"}]}

:diff

{:source "res://...changeset..."
 :base-source "res://...source-tree..."
 :count 2
 :changes [{:path "README.md"
            :change :modified
            :before-size-bytes 14
            :size-bytes 22
            :content-type "text/markdown"
            :binary? false}
           {:path "docs/notes.txt"
            :change :deleted
            :size-bytes 32
           :content-type "text/plain"
           :binary? false}]}

:materialize

{:source "res://...changeset..."
 :base-source "res://...source-tree..."
 :directory "/mounted/worker-tmp/breyta-files-materializations/ws-acme-abc123"
 :count 4
 :text-file-count 3
 :binary-file-count 1
 :size-bytes 1234
 :lifecycle {:storage :worker-local
             :authoritative? false
             :writeback? false}}

Service-backed materialization returns a stable session handle without exposing
a raw directory:

{:session-uri "files-session://ws-acme/..."
 :lease-id "..."
 :generation 1
 :expires-at "2026-04-26T08:00:00Z"
 :root-kind :service-owned
 :materialized? true
 :lifecycle {:storage :files-service
             :durable-handoff? true
             :raw-directory-exposed? false}}

:capture

{:changeset {:type :resource-ref
             :uri "res://..."
             :content-type "application/vnd.breyta.files-changeset+json"
             :preview {:source-uri "res://...source-tree..."
                       :change-count 3}}
 :captured-files 2
 :skipped-files 41
 :directory "/mounted/worker-tmp/breyta-files-materializations/ws-acme-abc123"}

Service-backed capture returns the renewed lease generation:

{:session-uri "files-session://ws-acme/..."
 :lease-id "..."
 :generation 2
 :changeset {:type :resource-ref
             :uri "res://...changeset..."}
 :capture-summary {:captured-files 2
                   :skipped-files 41}}

:publish

{:status "published"
 :provider :github
 :source "res://...source-tree..."
 :changeset "res://...changeset..."
 :repository "acme/repo"
 :base-branch "main"
 :branch "codex/security-fix/test"
 :base-commit-sha "abc123..."
 :tree-sha "def456..."
 :commit-sha "fedcba..."
 :ref-action :created
 :change-count 2
 :changes [{:path "docs/notes.txt" :change :deleted ...}
           {:path "src/app/core.clj" :change :modified ...}]}

When the changeset has no logical diff, :publish returns:

{:status "no_changes"
 :provider :github
 :repository "acme/repo"
 :base-branch "main"
 :branch "codex/security-fix/test"
 :change-count 0
 :changes []}

:open-change-request

{:status "created"
 :provider :github
 :repository "acme/repo"
 :base-branch "main"
 :branch "codex/security-fix/test"
 :number 42
 :url "https://github.com/acme/repo/pull/42"}

If the provider reports that the pull request already exists, the step reuses
the existing open pull request and returns :status "existing".

Canonical Examples

Resolve a repo, fork a changeset, mutate it, and query the overlaid view:

'(let [repo-tree (flow/step :files :resolve-repo
                   {:op :resolve-source
                    :source {:type :git
                             :repo "https://github.com/acme/service.git"
                             :ref "main"
                             :connection :github-api}})
       draft (flow/step :files :start-draft
               {:op :init-changeset
                :source repo-tree})
       draft (flow/step :files :update-readme
               {:op :write-file
                :changeset draft
                :path "README.md"
                :content "# Updated service\n"})
       draft (flow/step :files :edit-core
               {:op :replace
                :changeset draft
                :path "src/app/core.clj"
                :match "\"ready\""
                :replace "\"steady\""})
       draft (flow/step :files :tighten-guard
               {:op :replace-lines
                :changeset draft
                :path "src/app/core.clj"
                :line-start 10
                :line-end 14
                :expected "(when insecure?\n  (log/warn \"legacy path\")\n  true)"
                :replace "(when insecure?\n  (log/error \"legacy path\")\n  false)"})
       draft (flow/step :files :move-core
               {:op :move-file
                :changeset draft
                :from-path "src/app/core.clj"
                :to-path "src/app/main.clj"})
       draft (flow/step :files :delete-notes
               {:op :delete-file
                :changeset draft
                :path "docs/notes.txt"})
       files (flow/step :files :list-draft
               {:op :list
                :source draft})
       matches (flow/step :files :search-draft
                 {:op :search
                  :source draft
                  :query "persisted-value"})
       diff (flow/step :files :diff-draft
              {:op :diff
               :source draft})]
   {:files files
   :matches matches
    :diff diff
    :draft draft})

Publish a bounded draft and open or reuse a pull request:

'(let [repo-tree (flow/step :files :resolve-repo
                   {:op :resolve-source
                    :source {:type :git
                             :repo "https://github.com/acme/service.git"
                             :ref "main"
                             :connection :github-api}})
       draft (flow/step :files :start-draft
               {:op :init-changeset
                :source repo-tree})
       draft (flow/step :files :edit-core
               {:op :apply-edit
                :changeset draft
                :path "src/app/core.clj"
                :edit {:op :replace
                       :match "\"ready\""
                       :replace "\"steady\""}})
       published (flow/step :files :publish-draft
                   {:op :publish
                    :changeset draft
                    :connection :github-api
                    :repository "acme/service"
                    :base-branch "main"
                    :branch "codex/security-fix/test"
                    :commit-message "security: fix runtime issue"})
       pr (flow/step :files :open-pr
            {:op :open-change-request
             :connection :github-api
             :published published
             :change-title "Fix runtime issue"
             :change-body "Automated remediation from the security fixer flow."})]
   {:published published
    :pull-request pr})

Source Providers

:files routes provider-specific operations through a source provider
registry. Today GitHub is the built-in provider, so only GitHub is
auto-detected from the repository URL. Other forges must be set
explicitly via :provider after their provider has been registered:

;; Auto-detected as GitHub from the URL
{:op :resolve-source :source {:type :git :repo "https://github.com/acme/service.git"}}

;; Explicit provider after registering a GitLab provider
{:op :resolve-source :source {:type :git :repo "https://gitlab.com/acme/repo.git" :provider :gitlab}}

GitHub is the built-in provider. Custom providers can be registered for
GitLab, Bitbucket, Azure DevOps, or any other forge.

Provider-specific operations: :resolve-source, :publish, :open-change-request.
Provider-agnostic operations (work with any source): :list, :read, :search,
:write-file, :apply-edit, :replace, :replace-lines, :delete-file,
:move-file, :diff, :init-changeset, :materialize, :capture.

Connection And Installation Notes

:files uses connections for two purposes:

1. Source import (:resolve-source) — reads repository content
2. Publish/PR (:publish, :open-change-request) — writes branches and pull requests

For installable flows, declare the connection as a requirement:

;; Installer provides their own GitHub token:
{:requires [{:slot :github-api
             :type :http-api
             :label "GitHub API"
             :auth {:type :api-key}
             :base-url "https://api.github.com"}]}

;; Author provides the token (installer never configures GitHub):
{:requires [{:slot :github-api
             :type :http-api
             :provided-by :author
             :base-url "https://api.github.com"}]}

Then reference the slot in the step:

(flow/step :files :resolve-repo
  {:op :resolve-source
   :source {:type :git
            :repo "https://github.com/acme/service.git"
            :ref "main"
            :connection :github-api}})

The same connection is used for :publish and :open-change-request.

Source-tree resources (source-tree-ref, changeset-ref) are persisted in
Breyta's managed storage with bounded TTL. In installed flows, these are
internal to the flow's execution — the installer does not see or manage them.

Related

• Step Agent — exposes an agent-safe subset of :files as a built-in tool
• Packaged Steps
• Step Reference
• Installations — installable agent flows
• Flow Definition
• Persisted Results And Resource Refs