6cb8e9e904
* Add API changelog tooling: semantic OpenAPI diff with severity ranking
Answering "has the API changelog been updated?" currently means reading a
10k-line JSON diff by hand, so it falls to whoever is asked in Slack. This adds
the mechanical part of that job as two mage tasks.
`./bin/mage openapi-diff OLD.json NEW.json [--severity breaking]`
resources/openapi/openapi.json is generated from the Malli endpoint schemas, so
the spec at any git ref is what the API was at that ref. The task parses two
specs into {METHOD /path -> shape} maps, inlines $refs (component schemas change
too), and compares params, request body, and response schemas.
Findings are classified BREAKING / ADDITIVE / DOC_ONLY and sorted breaking-first.
A change is breaking when it requires MORE from the caller or provides LESS to
the caller; anything else is just a change. That rule is directional and it
inverts between request and response, so severity is decided from both schemas
rather than from how a change happens to render:
request breaking: newly required, type/enum narrowed, schema closed, field removed
additive: new OPTIONAL field or param, type/enum widened, made nullable
response breaking: field removed, field may now be null
additive: new field returned
Adding an optional parameter is not breaking; returning extra data is not
breaking. A public diff of two Stripe specs found 663 of 679 changes additive,
which is why filtering matters more than summarizing.
`./bin/mage openapi-staleness`
The self-healing CI job runs only on PRs labelled `openapi-self-healing`, so the
committed spec drifts; it is currently ~38 API commits behind. A stale spec makes
the diff under-report, so this reports drift and names the commits rather than
letting a clean diff read as "no changes". The label gate is deliberate (#67854
introduced it to stage the rollout after #67294 removed the spec entirely over
merge conflicts), so it is left alone here.
Response coverage is partial: ~199 response entries declare a schema and can be
compared, ~1,700 are description-only 2XX/4XX/5XX stubs and cannot be.
Tests cover both directions of every request and response rule, $ref resolution
including the SCIM schemas whose property is literally named "$ref", a guard
against the failure mode in mozilla-ai/otari#1315 (breaking markers dropped at
render time), and a signal-to-noise case asserting additive bulk cannot drown
the breaking few.
* Generate specs from git refs, so the diff is not bound to the committed blob
`./bin/mage openapi-diff --refs <old-ref> <new-ref>` checks each ref out into a
throwaway worktree, runs that ref's own `generate-openapi-spec`, and diffs the
two results. Nothing is committed and no checkout is touched.
This removes the staleness problem rather than reporting it. The committed
`openapi.json` only updates on PRs labelled `openapi-self-healing`, so it lags
source badly: `--committed origin/release-x.63.x origin/master` reports 0
findings, because both refs carry the same stale blob. Generating from source
reports 670 -> 714 operations and 10 removed endpoints for the same pair.
It needs no support from the ref being diffed beyond `generate-openapi-spec`
already existing, so it works on historical refs. `--committed` keeps the fast
path and now prints a warning that its result under-reports, since an empty diff
from a stale spec is indistinguishable from an API that did not change.
Validated against the real v63 -> master delta. The top removals are the three
`GET .../execute` endpoints that the hand-written 0.64 changelog documents as
replaced by `POST .../execute/values`, and it independently surfaces a query
param rename (`pinned_state` -> `pinned-state`) that has no entry yet.
That run also shows a precision limit worth stating: it reports 239 breaking
findings, but 217 are the identical `additionalProperties: false` line from one
PR (#82447, closing mu/defn argument schemas). The classification is right --
closing a schema does require more of the caller -- but as changelog input that
is one entry, not 217. The skill now tells the drafter to collapse findings by
text and treat a repeated finding as a single systematic change.
* Group identical findings so a systematic change reads as one entry
`--grouped` collapses findings with identical text across endpoints and sorts by
blast radius, because the per-endpoint view is the wrong shape for drafting a
changelog.
On the real v63 -> master delta it turns 239 breaking findings into 113 distinct
changes. The largest spans 217 endpoints and is a single upstream PR (#82447,
closing mu/defn argument schemas): one changelog entry, reported once with its
endpoints, instead of 217 identical lines to scroll past.
Grouping is a separate render mode rather than a transformation of the default.
The per-endpoint view answers "what changed on this route?", the grouped view
answers "what distinct changes happened?", and collapsing the former into the
latter would lose the ability to check a specific endpoint.
The long tail is where the individually-interesting entries live: the same run
surfaces a `pinned_state` -> `pinned-state` query param rename that no changelog
entry covers yet.
* Fix review findings: two defects let breaking changes pass silently
Eric's review found 13 issues. The two that mattered both caused false
negatives - the tool reporting nothing where a caller would break.
The depth guard counted nesting levels, not `$ref` hops. `seen` already breaks
cycles, so the counter was doing a different job than its name claimed: past 12
levels both specs collapsed to the literal "<deep>" and compared equal, hiding
every change in 52 of 637 operations. It now advances only on a `$ref` hop, and
52 falls to 17.
`response-lines` compared only top-level properties and delegated anything deeper
to `(widening? new old)`. Swapping those arguments inverts value-set semantics -
which is why the nullable case read correctly - but not object-property
semantics, so a removed nested response field read as additive. It now recurses
the way `body-lines` does.
Chasing the second one surfaced a third of the same shape: `widening?` never
compared array `items`, so a narrowing inside an array was invisible. Found by a
test written for the SCIM `$ref`-named-property case.
Also fixed:
- `committed-spec!` built a shell command by interpolation, so a ref containing
`$`, a quote, or a space produced a wrong file. Values now ride as positional
arguments and the shell never parses them.
- The `--grouped` header counted severity-filtered-out changes, inflating the
count above the body printed beneath it.
- The removed and added endpoint lists were severity-gated, so `--severity
breaking` hid the added half of an endpoint move - at exactly the severity the
skill tells a drafter to start with. Both lists now print at every severity.
- The staleness check watched 66 of the 158 files that contain `defendpoint`,
missing `src/metabase/*/api/*.clj` and all of `enterprise/backend/src`, so it
could report `OK: spec is current` with dozens of newer endpoint files on disk.
It also compared `%cI` strings across a history carrying both `+08:00` and `Z`,
and listed commits from a narrower pathspec than the date came from.
- `DOC_ONLY` was unreachable: a description-only change produced no findings and
was dropped. Descriptions are what the skill reads to spot changes it cannot
explain, so a reworded docstring now surfaces as doc-only.
- A misspelled `--severity` fell through to nil and silently printed everything.
It now exits. `--committed` without `--refs` was accepted and ignored; it now
exits too.
- Each `--refs` run left two multi-megabyte specs in /tmp.
- `brief` truncated its non-map branch without the "..." its map branch adds.
- The first commit message claimed a test covered the SCIM `$ref`-named property.
No such test existed; one exists now, along with tests for the depth guard, the
nested response removal, and the doc-only path.
One correction to that commit message stands: `babashka.json` would have avoided
the `:discouraged-namespace` ratchet entirely, since `mage/doctor.clj` already
uses it unsuppressed. Keeping cheshire for consistency with the three other mage
namespaces that use it; the budget moves 68 -> 69 against current master.
The `de-slop` skill the review could not find does exist, at
~/.claude/skills/de-slop - it is a personal skill, not in the repo.
* Fix second review: four more directions where a finding was inverted or absent
All nine findings addressed. The reviewer supplied a failing test for each of the
first five; all five reproduced on the previous head, and all now pass.
Three of the four major findings were the same mistake in different places: a
schema shape that the output path handed to `widening?` with swapped arguments,
which inverts value-set semantics but not object-property semantics.
- A response array of objects went to the leaf branch, so a removed element field
read as additive and an added one read as breaking. 24 of the 199 comparable
response schemas in the committed spec are top-level arrays. Arrays now recurse
through `items` in the output direction.
- `schema-props` unwraps a nullable `oneOf`, so a nullable OBJECT reached the
object branch, where only keys and requiredness were compared and the type
change was invisible. The committed spec has 726 nullable object properties.
Both `body-lines` and `response-lines` now compare type sets there, with the
severity test pointing the correct way for each direction.
- `response-lines` discarded the required sets, so a response field the API may
now omit produced nothing - the same loss of guarantee as one that may now be
null, which was already breaking. 174 of 637 operations declare `required` in a
response schema.
The fourth major finding was a shortcut in `widening?`: any empty old schema was
treated as "no constraint", so a field going from Malli `:any` to a concrete type
read as additive. `operations` stored an absent body as `{}`, which made "no body"
and "any value" indistinguishable; an absent body is now nil and `{}` falls
through to the type and enum checks. After ref resolution the committed spec has
1,182 property schemas that are `{}`.
The depth limit is gone rather than re-tuned. `seen` already terminates every
path against a finite component set, so the limit only ever replaced deep nodes
with the same literal on both sides - which compares equal and hides the change.
The MBQL clause schemas form chains long enough to hit it: 17 operations carried
`<deep>`, and now none do while 18 terminate on genuine cycles via `<recursive>`.
A self-diff of the committed spec costs 2.9s instead of 1.4s and still reports 0
findings.
Also fixed:
- `format-epoch` shelled out to `date -r`, which only BSD and macOS read as epoch
seconds; on GNU coreutils it means "the mtime of this file" and printed nothing,
silently, because the call was `:quiet?`. Now `java.time`.
- `openapi-staleness` claimed the spec was "current" from endpoint files alone.
661 of 763 component schemas live outside `.api` namespaces, so a shared-schema
change moves the generated spec invisibly. The OK message now says what it
actually checked and points at `--refs`.
- A `--refs` run that could neither generate nor read the second ref leaked the
first spec: materialization happened in the `let` bindings, outside the `try`
whose `finally` removes the directory.
- The skill told an agent that two identical committed specs prove the API did not
change, and that the commit touching `openapi.json` is the PR that changed the
API. Both are wrong for the same reason - the committed spec only moves on
`openapi-self-healing` PRs. It now says to re-run without `--committed` before
reporting no changes, and to search endpoint source for attribution.
* Drop the ratchet bump: babashka.json instead of cheshire, and fix stale docs
The `:discouraged-namespace` increase was avoidable and the rationale I gave for
it was backwards. `babashka.json` is not in the discouraged list, and 6 mage
namespaces already use it unsuppressed against 5 on cheshire - so the
"consistency" argument pointed the other way. Swapping to it removes both the
68 -> 69 bump and the `:clj-kondo/ignore`, and this PR no longer touches
.clj-kondo/ratchets.edn at all. `read-str` keywordizes by default, so the two
spec reads pass `{:key-fn identity}`; this namespace is string-key throughout.
Verified unchanged behavior: self-diff still 0 findings, the real 0.64 change
still 1 breaking.
Three docstring counts were wrong. The `api-source-pathspecs` docstring claimed
238 `/api/ee/` paths (actual 106 - it looks like 106 paths plus 132 operations,
double-counted), 66 files matching `src/metabase/**/api.clj` (actual 65), and 158
files containing `defendpoint` (actual 160). The argument the docstring makes is
unaffected - the narrow pathspec still misses most endpoint files - but the
numbers are now the measured ones.
Also removed documentation that described code this branch already deleted:
- A comment in `resolve-refs` said "Only a $ref hop advances depth" while the
docstring 17 lines above correctly says there is no depth limit. There is no
depth counter left; the comment described the previous commit's version.
- Two test comments and a test name referenced a `<deep>` placeholder guard that
no longer exists, and cited a "52 of 637" measurement of the mechanism that was
removed. The tests themselves are valid - they assert deep and long-chain
changes are detected - so only the prose changed.
- SKILL.md quoted "`--committed` reports 0 findings" for v63 vs master as a fixed
figure. That was true when written and is not now: the blobs have since
diverged and it reports 1. The underlying point stands, so it is stated
qualitatively rather than as a number that drifts.
- SKILL.md said `--refs` "works on any ref". The code only attempts that and has
a documented fallback for refs that will not build with the current toolchain;
the text now says so.
* Fix third review: object unions and nullable arrays hid breaking changes
- A union of two or more object variants was compared by its first
variant only, so removing or editing any other variant produced no
finding. Such unions now compare whole: widening only when every old
variant survives unchanged.
- Nullable arrays (`oneOf [array, null]`) skipped the array branch, so a
change to their items was ranked additive. Both array branches now
unwrap the null first.
- The render test never rendered. It now asserts breaking operations
print under the breaking headings of both renderers, and a new test
covers the --severity filter.
* Rank changes to unmodeled schema keywords breaking, and compare bounds, allOf, and tuples
The old widening check returned "safe" for any keyword it did not
recognize, so each review round found another way for a breaking change
to read as additive: allOf, minimum/minLength, map-of value schemas,
prefixItems, nullable enums, closed nullable objects.
`compatible?` replaces it. It takes a direction (:request or :response),
compares unions variant by variant, compares bounds, pattern, and format,
and compares allOf and prefixItems by position. A change to any keyword
it does not model ranks breaking. A false breaking finding costs one
look; a missed one ships undocumented.
Documentation keys (description, title, examples, default) are stripped
before comparison, so a reworded field description is doc-only instead
of an additive schema change.
Also:
- When two summaries are identical, the finding names where the schemas
differ (e.g. `items.enum: +"measure"`), so distinct changes no longer
merge in the grouped view.
- Generation deletes the checked-out spec before running, so a generator
that logs and swallows an error no longer passes the committed spec off
as fresh. A timeout falls back to the committed spec instead of
throwing.
- An empty or non-spec input file exits 1 instead of reporting 0 BREAKING.
- openapi-staleness lists commits after the spec commit, not including it.
* Close the gaps an independent check found in the compatible? rewrite
- Only a nullable union unwraps to its object. `string | object` and
similar unions are compared whole, so a change to the non-object
variant is no longer hidden.
- Keywords beside an object's properties (allOf, a map-of
additionalProperties, minProperties) are compared.
- If the field walk finds nothing but the stripped schemas differ, the
whole schema is ranked instead of falling through to doc-only.
- Inclusive and exclusive numeric bounds compare as one bound.
- A response allOf may gain members (narrowing output); a request allOf
may lose them.
- An enum or const without a type widening to its type is additive.
- requestBody.required and request content types are compared, and a
response that declares a schema for the first time is reported as
additive.
- A missing input file says so.
* Match operations across path-variable renames; count only visible severities in the grouped header
- Operations and path params are keyed by position, not variable name.
Renaming `{id}` to `{card-id}` leaves the URL a caller sends unchanged,
so it is now a doc-only "path variable renamed" finding rather than a
removed endpoint plus an added one. Routes that would merge under
position-keying (`/api/database/{id}/schemas` and
`/api/database/{virtual-db}/schemas`) keep their names. Reports still
print the real spec paths.
- At --severity breaking, the grouped header no longer counts the
new-endpoint list, which is additive. The list still prints.
- The skill's attribution section no longer opens with the
openapi.json git log that it then says names the wrong author.
* Rank a changed or dropped request default as breaking
`default` was stripped with the documentation keys, so a default that
flipped (say a query param from false to true) ranked doc-only and
--severity breaking hid it. The schema still validates the same
requests, but a caller who omits the field now gets different
behavior.
Request defaults are now compared: changed or dropped is breaking,
added where there was none is additive. Response defaults are ignored.
* State one definition of breaking that covers every rule
A change is breaking when an existing caller, sending exactly what it
sent before, can now fail or get a different result. The old wording
(requires more / provides less) did not cover changed defaults, and it
presented two conservative rules - removed request fields and changes
to unmodeled keywords - as if they followed from it. Those are now
stated as what the tool does when it cannot tell.
* Skill: point to docs-write, and attribute with all six API source pathspecs
de-slop is a personal skill the repo does not ship; docs-write carries
the Metabase style guide. The attribution git log used three of the six
pathspecs in api-source-pathspecs and missed enterprise api/ dirs and
routes.clj files (35 tracked files, e.g. sso/api/saml.clj).
973 lines
59 KiB
Clojure
973 lines
59 KiB
Clojure
{:min-bb-version "1.12.212"
|
|
;; we put path as bin, and everything is in the ./mage subdirectory,
|
|
;; so the namespaces are mage.cli, mage.format, etc.
|
|
;; dev/src is here for tasks whose logic is shared with JVM tests (e.g. dev.kondo-ratchet)
|
|
;; Let Mage use the same module-resolution rules as the Kondo hooks.
|
|
:paths ["mage/src" "mage/test" "bin/lint-migrations-file/src" "dev/src" ".clj-kondo/src"]
|
|
;; use Git version of dep temporarily until a new release > 0.16.4 is cut
|
|
:deps {dev.weavejester/cljfmt {:git/sha "075eae26e3c8dad433b88f05927bcd50e3571328"
|
|
:git/url "https://github.com/weavejester/cljfmt"} ;; run cljfmt from bb
|
|
io.github.paintparty/bling {:mvn/version "0.8.8"} ;; printing bells and whistles
|
|
medley/medley {:mvn/version "1.3.0"} ;; utilities
|
|
metosin/malli {:mvn/version "0.17.0"} ;; data validation
|
|
mvxcvi/puget {:mvn/version "1.3.4"} ;; pretty printing
|
|
org.babashka/json {:mvn/version "0.1.6"} ;; json parsing
|
|
table/table {:mvn/version "0.5.0"} ;; table printing
|
|
}
|
|
;; No :pods key on purpose: tasks lazily load only what they need.
|
|
:tasks
|
|
{:requires [[mage.cli :as cli]
|
|
[mage.color :as c]]
|
|
:init (do (defmacro task! "Binds the parsed command line arguments to `parsed`."
|
|
[& body]
|
|
;; Can't use ` in this file because it is strictly edn!
|
|
(list 'let
|
|
(vector 'parsed (list 'cli/parse! (list 'current-task)))
|
|
(cons 'do body))))
|
|
|
|
alias
|
|
{:doc "Installs 'mage' alias and autocomplete"
|
|
:examples [["./bin/mage alias" "Prints instructions to install the 'mage' alias and autocomplete"]
|
|
["./bin/mage alias install fish" "Installs mage alias for fish shell"]
|
|
["./bin/mage alias print zsh" "Prints shell script to put into .zshrc for zsh users"]]
|
|
:requires [[mage.alias :as alias]]
|
|
:arg-schema [:or
|
|
[:tuple [:enum {:name "shell-type"} "bash" "zsh" "fish"]]
|
|
[:tuple [:enum {:name "mode"} "install" "print"]
|
|
[:enum {:name "shell-type"} "bash" "zsh" "fish"]]]
|
|
:task (task! (alias/instructions parsed))}
|
|
|
|
cljfmt-staged
|
|
{:doc "Runs cljfmt on staged files"
|
|
:requires [[mage.format :as format]]
|
|
:examples [["./bin/mage cljfmt-staged" "Format staged files"]
|
|
["./bin/mage cljfmt-staged -c" "Check staged files"]]
|
|
:options [["-c" "--force-check" "Check staged files"]]
|
|
:task (task! (format/staged parsed))}
|
|
|
|
cljfmt-files
|
|
{:doc "Runs cljfmt on the given files/directories"
|
|
:requires [[mage.format :as format]]
|
|
:examples [["./bin/mage cljfmt-files src/metabase/events.clj" "Format events.clj"]
|
|
["./bin/mage cljfmt-files src" "Format all files in src"]
|
|
["./bin/mage cljfmt-files -c src" "Check all files in src"]]
|
|
:options [["-c" "--force-check" "Check staged files"]]
|
|
:arg-schema [:sequential [:string {:name "file-or-directory"
|
|
:description "Files or directories to format."}]]
|
|
:task (task! (format/files parsed))}
|
|
|
|
cljfmt-all
|
|
{:doc "Runs cljfmt on all (clojure) files"
|
|
:requires [[mage.format :as format]]
|
|
:examples [["./bin/mage cljfmt-all" "Format all files"]]
|
|
:options [["-c" "--force-check" "Check staged files"]]
|
|
:task (task! (format/all parsed))}
|
|
|
|
cljfmt-updated
|
|
{:doc "Runs cljfmt on all (Clojure) files relative to a git ref (default HEAD)"
|
|
:requires [[mage.format :as format]]
|
|
:examples [["./bin/mage cljfmt-updated" "Format updated files relative to HEAD"]
|
|
["./bin/mage cljfmt-updated master" "Format updated files relative to master"]]
|
|
:options [["-c" "--force-check" "Check staged files"]]
|
|
:arg-schema [:or [:tuple] [:tuple [:string {:name "git-ref" :description "Git ref to compare against"}]]]
|
|
:task (task! (format/updated parsed))}
|
|
|
|
kondo
|
|
{:doc "Runs Kondo against a file, directory, or everything we usually lint"
|
|
:examples [["./bin/mage kondo" "run Kondo against everything we usually lint"]
|
|
["./bin/mage kondo src/metabase/db.clj" "run Kondo against a file"]
|
|
["./bin/mage kondo src/metabase/db.clj src/metabase/config.clj" "run Kondo against 2 files"]
|
|
["./bin/mage kondo src/metabase/api/macros" "run Kondo against a directory"]]
|
|
:requires [[mage.kondo :as kondo]]
|
|
:task (task! (kondo/kondo (:arguments parsed)))}
|
|
|
|
kondo-updated
|
|
{:doc "Runs kondo against files changed compared to a git ref"
|
|
:examples [["./bin/mage kondo-updated" "run Kondo on files with changes relative to HEAD"]
|
|
["./bin/mage kondo-updated master" "run Kondo on files with changes relative to master"]]
|
|
:requires [[mage.kondo :as kondo]]
|
|
:task (task! (kondo/kondo-updated (:arguments parsed)))}
|
|
|
|
fix-unused-requires
|
|
{:doc "Removes unused requires flagged by clj-kondo"
|
|
:examples [["./bin/mage fix-unused-requires src/metabase/utils/validate.cljc" "Fix unused requires in a specific file"]
|
|
["./bin/mage fix-unused-requires src" "Fix unused requires in all files in src"]]
|
|
:requires [[mage.fix-unused-requires :as fix]]
|
|
:arg-schema [:sequential [:string {:name "file-or-directory"
|
|
:description "Files or directories to fix unused requires in"}]]
|
|
:task (task! (fix/fix-files parsed))}
|
|
|
|
load-namespaces
|
|
{:doc "Checks that every shipped namespace loads without :dev dependencies (and finds circular dependencies)"
|
|
:examples [["./bin/mage load-namespaces"]]
|
|
:requires [[mage.load-namespaces]]
|
|
:arg-schema [:tuple]
|
|
:task (task! (mage.load-namespaces/load-namespaces))}
|
|
|
|
codespell
|
|
{:doc "Check for misspellings in the source code."
|
|
:examples [["./bin/mage codespell"]]
|
|
:requires [[mage.codespell]]
|
|
:task (task! (mage.codespell/codespell (:arguments parsed)))}
|
|
|
|
codespell-interactive
|
|
{:doc "Check for misspellings in the source code, and interactively prompt to fix them."
|
|
:examples [["./bin/mage codespell-interactive"]]
|
|
:requires [[mage.codespell]]
|
|
:task (task! (mage.codespell/codespell-interactive (:arguments parsed)))}
|
|
|
|
start-maildev
|
|
{:doc "Start Maildev"
|
|
:examples [["./bin/mage start-maildev -h" "print help for start-maildev"]]
|
|
:requires [[mage.start-maildev :as start-maildev]]
|
|
:task (task! (start-maildev/start-maildev!))}
|
|
|
|
tail-d
|
|
{:doc "Tail the #d debug log in real time (see dev.debug). Asks the running REPL for the exact path; falls back to DEV_DEBUG_LOG or /tmp/<dir>_debug.log."
|
|
:examples [["./bin/mage tail-d" "tail the #d log, with bat syntax highlighting if available"]]
|
|
:requires [[mage.tail-d :as tail-d]]
|
|
:task (task! (tail-d/tail-d! parsed))}
|
|
|
|
start-python-runner
|
|
{:doc "Start python-runner container with internal Moto S3 server for Python transforms"
|
|
:examples [["./bin/mage start-python-runner" "start python-runner with internal S3"]]
|
|
:requires [[mage.start-python-runner :as start-python-runner]]
|
|
:task (task! (start-python-runner/start-python-runner!))}
|
|
|
|
start-db
|
|
{:doc "Start a db on a default port in docker"
|
|
:examples [["./bin/mage start-db postgres latest" "start the latest postgres db we support"]
|
|
["./bin/mage start-db mysql oldest" "start the oldest mysql db we support"]
|
|
["./bin/mage start-db postgres latest --port 5555" "start postgres on a custom port"]]
|
|
:requires [[mage.start-db :as start-db]]
|
|
:arg-schema [:tuple
|
|
[:enum :postgres :mysql :mariadb :mongo :clickhouse :sqlserver :oracle]
|
|
[:enum :oldest :latest]]
|
|
:options [["-p" "--port PORT" "Override the default host port for the container" :parse-fn Integer/parseInt]]
|
|
:db-info {:postgres {:ports {:oldest 5432 :latest 5433} :eol-url "https://endoflife.date/api/postgres.json"}
|
|
:mysql {:ports {:oldest 3308 :latest 3309} :eol-url "https://endoflife.date/api/mysql.json"}
|
|
:mariadb {:ports {:oldest 3306 :latest 3307} :eol-url "https://endoflife.date/api/mariadb.json"}
|
|
:mongo {:ports {:oldest 27017 :latest 27018} :eol-url "https://endoflife.date/api/mongodb.json"}
|
|
:sqlserver {:ports {:oldest 1433 :latest 1434} :eol-url "https://endoflife.date/api/mssqlserver.json"}
|
|
:oracle {:ports {:oldest 1521 :latest 1522} :eol-url "https://endoflife.date/api/oracle-database.json"}
|
|
;; some databases aren't on endoflife.date
|
|
:clickhouse {:ports {:oldest 8124 :latest 8123} :eol-url nil}}
|
|
:usage-fn start-db/usage
|
|
:task (task!
|
|
(let [[db version] (:arguments parsed)]
|
|
(start-db/start-db (-> (current-task) :db-info db) db version (:options parsed))))}
|
|
|
|
nrepl
|
|
{:doc "Starts the babashka nrepl: helpful for mage development"
|
|
:requires [[babashka.nrepl.server :as nrepl.server]
|
|
[mage.color :as c]]
|
|
:examples [["./bin/mage nrepl" "Starts the nrepl server"]
|
|
["./bin/mage nrepl --port 50506" "Starts the nrepl server"]
|
|
["./bin/mage nrepl -p 50506" "Starts the nrepl server"]]
|
|
:options [["-p" "--port PORT" "optionally provide a port to run the nrepl server on" :default 1667 :parse-fn Integer/parseInt]]
|
|
:task (task!
|
|
(let [{:keys [port]} (:options parsed)]
|
|
(spit ".nrepl-port" port)
|
|
(nrepl.server/start-server! {:port port})
|
|
(deref (promise))))}
|
|
|
|
lint-migrations
|
|
{:doc "Lint migrations files"
|
|
:requires [[lint-migrations-file]
|
|
[babashka.process :as p]]
|
|
:examples [["./bin/mage lint-migrations" "Lint all migrations and print report"]]
|
|
:task (do (cli/parse! (current-task))
|
|
(p/shell "bin/pre-lint-migrations-file.sh")
|
|
(lint-migrations-file/-main))}
|
|
|
|
new-migration
|
|
{:doc "Create a new directory-based migration file (v60+)"
|
|
:requires [[mage.new-migration :as new-migration]]
|
|
:examples [["./bin/mage new-migration 60 mq_indexes" "Create resources/migrations/060/20260905_mq_indexes.yaml"]]
|
|
:arg-schema [:tuple
|
|
[:int {:name "version" :description "Major version number (e.g., 60)"}]
|
|
[:string {:name "description" :description "Lowercase alphanumeric/underscore name (e.g., mq_indexes)"}]]
|
|
:task (task! (new-migration/new-migration parsed))}
|
|
|
|
jar-download
|
|
{:doc "Download (and optionally run) jar for a metabase version or branch"
|
|
:examples [["./bin/mage jar-download 50" "Download the latest enterprise version of release 50 to ~/path/to/my/metabase/jars"]
|
|
["./bin/mage jar-download 1.45.2 -r" "Download and run metabase_1.45.2.jar"]
|
|
["./bin/mage jar-download 1.45.2 -d ~/path/to/my/jars" "Download metabase_1.45.2.jar to ~/path/to/my/jars, deleting if it exists"]
|
|
["./bin/mage jar-download master --run --port 3001" "Download and run the latest master branch jar on port 3001"]
|
|
["JARS=~/my-stuff ./bin/mage jar-download 1.45.2" "Download metabase_1.45.2.jar to ~/my-stuff"]
|
|
["./bin/mage jar-download v1.56.0-beta -r" "Download and run metabase_1.56.0-beta.jar"]]
|
|
:requires [[mage.jar-download :as jar-download]]
|
|
:options [["-r" "--run" "run the new jar after downloading it"]
|
|
["-e" "--env-file FILE" "you can optionally provide a lein-env file to specify the environment to use when running the jar."]
|
|
["-d" "--delete" "delete the old jar if found, by default does not re-download it"]
|
|
["-p" "--port PORT" "you can optionally provide a port to run the jar on, if not, we use a sane port."]]
|
|
:arg-schema [:or
|
|
[:tuple [:string {:name "version"}]]
|
|
[:tuple [:string {:name "version"}] [:string {:name "path"}]]]
|
|
:usage-fn (fn [_]
|
|
(-> "Version can have the following forms:"
|
|
(str "\n -" (c/green "50") " for the latest enterprise version of release 50")
|
|
(str "\n -" (c/green "1.45.2") " for a specific version")
|
|
(str "\n -" (c/green "master") " for the latest jar created for master")
|
|
(str "\n -" (c/green "any-branch-with-a-pr") " for the latest uberjar created for that branch")
|
|
(str "\n -" (c/green "1.56.0-beta") " for a specific beta version")
|
|
(str "\n\nnote: If you get an error about 'Invalid or corrupt jarfile', run this command again with --delete.")))
|
|
:task (task! (jar-download/jar-download parsed))}
|
|
|
|
setup-stats-repl
|
|
{:doc "Connect to the stats repl"
|
|
:examples [["./bin/mage connect-stats-repl" "Connects to the stats repl"]]
|
|
:requires [[mage.stats-repl :as stats-repl]]
|
|
:task (stats-repl/connect (cli/parse! (current-task)))}
|
|
|
|
notify
|
|
{:doc "Send notification for a comment (that already exists)"
|
|
:requires [[mage.be-dev :refer [nrepl-eval]]]
|
|
:options [["-c" "--comment-id ID" "Notify about this comment id"]
|
|
["-p" "--port PORT" "Port to use for the task, defaults to value in .nrepl-port"]]
|
|
:examples [["./bin/mage notify -c 1234" "Notify about comment with id 1234"]
|
|
["./bin/mage notify --comment-id 5678 --port 59498" "Notify about comment with id 5678 on nrepl port 59498"]]
|
|
:task (let [opts (:options (cli/parse! (current-task)))
|
|
;; note: no quoting -> expect it to be int
|
|
code (format "(notify-comment-id! %d)" (parse-long (:comment-id opts)))]
|
|
(if (:port opts)
|
|
(nrepl-eval "metabase.comments.core" code (:port opts))
|
|
(nrepl-eval "metabase.comments.core" code)))}
|
|
|
|
run-tests
|
|
{:doc "Quickly run tests locally. Pass dirs, files, namespaces, or single tests as arguments."
|
|
:examples [["./bin/mage run-tests" "launch interactive test picker"]
|
|
["./bin/mage run-tests test/metabase/db" "run tests in test/metabase/db"]
|
|
["./bin/mage run-tests test/metabase/logger test/metabase/api_keys" "run tests in test/metabase/logger and test/metabase/api_keys"]
|
|
["./bin/mage run-tests test/metabase/config_test.clj" "run all tests in the metabase.config-test namespace"]
|
|
["./bin/mage run-tests metabase.config-test" "run all tests in the metabase.config-test namespace (by name)"]
|
|
["./bin/mage run-tests metabase.config-test/config-test" "run a single test by namespace/test-name"]]
|
|
;; TODO: add option to select files as well as dirs from fzf with styles
|
|
:options [["-s" "--selecting STYLE" "Style of test selection" :default "all"]
|
|
;; all, file, dir
|
|
["-p" "--port PORT" "Port to use for the task, defaults to value in .nrepl-port"]]
|
|
:arg-schema [:sequential
|
|
{:details "If none are given, you will be prompted to select directories recursively from test/ and enterprise/backend/test/"}
|
|
[:string {:name "file-dir-ns-or-test" :description "Files, directories (recursive), namespaces, or namespace/test-name to test"}]]
|
|
:requires [[mage.quick-test-runner :as qtr]]
|
|
:task (task! (qtr/go parsed))}
|
|
|
|
openapi-diff
|
|
{:doc "Semantic diff of two OpenAPI specs, classified breaking-first, for the API changelog"
|
|
:examples [["./bin/mage openapi-diff old.json new.json" "Full diff, breaking changes first"]
|
|
["./bin/mage openapi-diff old.json new.json --severity breaking" "Only what needs a changelog entry"]
|
|
["./bin/mage openapi-diff --refs release-x.63.x release-x.64.x --severity breaking" "Generate both specs from source, then diff"]
|
|
["./bin/mage openapi-diff --refs --committed release-x.63.x master" "Use each ref's committed spec (fast, may be stale)"]
|
|
["./bin/mage openapi-diff --refs --grouped release-x.63.x master --severity breaking" "One line per distinct change, widest blast radius first"]]
|
|
:arg-schema [:tuple
|
|
[:string {:name "old" :description "Path to the older openapi.json, or a git ref with --refs"}]
|
|
[:string {:name "new" :description "Path to the newer openapi.json, or a git ref with --refs"}]]
|
|
:options [["-s" "--severity LEVEL" "Only show findings at or above LEVEL: breaking or additive"]
|
|
["-r" "--refs" "Treat the arguments as git refs and generate each spec from its source"]
|
|
["-c" "--committed" "With --refs, read each ref's committed spec instead of generating (fast, may be stale)"]
|
|
["-g" "--grouped" "Collapse identical findings across endpoints: one line per distinct change"]]
|
|
:requires [[mage.openapi-diff]]
|
|
:task (task! (mage.openapi-diff/cli-diff parsed))}
|
|
|
|
openapi-staleness
|
|
{:doc "Is the committed OpenAPI spec current relative to the API source? (a stale spec under-reports)"
|
|
:examples [["./bin/mage openapi-staleness"]]
|
|
:arg-schema [:tuple]
|
|
:requires [[mage.openapi-diff]]
|
|
:task (task! (mage.openapi-diff/cli-staleness parsed))}
|
|
|
|
affected-modules
|
|
{:doc "Which backend modules have changes?"
|
|
:examples [["./bin/mage affected-modules master" "prints changes compared to `master`"]
|
|
["./bin/mage affected-modules"]]
|
|
:arg-schema [:or [:tuple] [:tuple [:string {:name "git-ref"}]]]
|
|
:requires [[mage.modules]]
|
|
:task (task! (mage.modules/cli-print-affected-modules (:arguments parsed)))}
|
|
|
|
modules-tree
|
|
{:doc "Print the module hierarchy, nesting enterprise extensions under their OSS module"
|
|
:examples [["./bin/mage modules-tree" "Print the full tree; * marks a custom :ns-prefix"]
|
|
["./bin/mage modules-tree --nested-only" "Print only roots with nested modules"]
|
|
["./bin/mage modules-tree --prefixes" "Show custom prefixes instead of *"]
|
|
["./bin/mage modules-tree --html --output modules.html" "Write the interactive explorer"]]
|
|
:options [["-n" "--nested-only" "Only print modules that have nested children"]
|
|
["-p" "--prefixes" "Print the :ns-prefix instead of the star"]
|
|
["-H" "--html" "Emit the interactive HTML explorer instead of the text tree"]
|
|
["-o" "--output FILE" "With --html, write to FILE instead of stdout"]
|
|
["-S" "--no-stats" "With --html, skip the slower per-module source metrics"]]
|
|
:requires [[mage.modules]]
|
|
:task (task! (mage.modules/cli-print-module-tree parsed))}
|
|
|
|
can-skip-driver-tests
|
|
{:doc "Can we skip driver tests for this branch?"
|
|
:examples [["./bin/mage can-skip-driver-tests master" "Returns zero (success) if we can skip driver tests based on changes related to `master`"]
|
|
["./bin/mage can-skip-driver-tests" "Returns zero (success) if we can skip driver tests based on changes? (defaults to master)"]]
|
|
:requires [[mage.modules]]
|
|
:task (task! (mage.modules/cli-can-skip-driver-tests (:arguments parsed)))}
|
|
|
|
fix-modules-config
|
|
{:doc "Regenerate .clj-kondo/config/modules/config.edn so it passes metabase.core.modules-test."
|
|
:examples [["./bin/mage fix-modules-config" "Fix the modules config (uses the running dev REPL if available, else a JVM)"]
|
|
["./bin/mage fix-modules-config --port 59498" "Use the dev REPL on a specific nREPL port"]]
|
|
;; Cold-JVM fallback invokes `clojure -X:dev dev.modules-config/fix-config!`.
|
|
:options [["-p" "--port PORT" "nREPL port to use, defaults to value in .nrepl-port"]]
|
|
:requires [[mage.modules]]
|
|
:task (task! (mage.modules/cli-fix-config (cli/parse! (current-task))))}
|
|
|
|
project-tests
|
|
{:doc "Run project-level checks."
|
|
:examples [["./bin/mage project-tests" "Run every suite"]
|
|
["./bin/mage project-tests backend" "Run backend checks"]
|
|
["./bin/mage project-tests migrations" "Run migration checks"]
|
|
["./bin/mage project-tests modules" "Run module checks"]
|
|
["./bin/mage project-tests ratchets" "Check the repository's kondo suppression ratchets"]]
|
|
:arg-schema [:or
|
|
[:tuple]
|
|
[:tuple [:enum {:name "suite"}
|
|
"backend"
|
|
"migrations"
|
|
"modules"
|
|
"ratchets"]]]
|
|
:requires [[mage.project-tests :as project-tests]]
|
|
:task (task! (if-let [suite (first (:arguments parsed))]
|
|
(project-tests/run! suite)
|
|
(project-tests/run!)))}
|
|
|
|
kondo-ratchets
|
|
{:doc "Validate the kondo and module ratchet files against the source tree (about half a minute, most of it a kondo run over the files with a discouraged-var or discouraged-namespace ignore)."
|
|
:examples [["./bin/mage kondo-ratchets" "Fail on an invalid ratchets file, an over-budget suppression, or a missing justification comment"]]
|
|
:requires [[dev.kondo-ratchet] [mage.kondo-ratchet]]
|
|
:task (task! (dev.kondo-ratchet/check {:attribute mage.kondo-ratchet/attribute-occurrences!}))}
|
|
|
|
kondo-ratchets-shrink
|
|
{:doc "Lower kondo and module budgets to match the source tree. This is the only Mage task that writes the ratchet files."
|
|
:examples [["./bin/mage kondo-ratchets-shrink" "Lower budgets to current counts without raising over-budget entries"]
|
|
["./bin/mage kondo-ratchets-shrink --seed :some-linter" "Set :some-linter's budget to its current count when introducing the linter"]
|
|
["./bin/mage kondo-ratchets-shrink --seed :discouraged-var/clojure.core/println" "Set one discouraged symbol's budget to its current count"]
|
|
["./bin/mage kondo-ratchets-shrink --seed :discouraged-var" "Set every discouraged-var symbol's budget to its current count"]]
|
|
:options [["-s" "--seed LINTER" "Set LINTER's budget to the actual count (may add or raise it); :discouraged-var/SYMBOL seeds one discouraged symbol, a bare :discouraged-var/:discouraged-namespace all of them"]]
|
|
:requires [[dev.kondo-ratchet] [mage.kondo-ratchet]]
|
|
:task (task! (dev.kondo-ratchet/fix! (assoc (:options parsed) :attribute mage.kondo-ratchet/attribute-occurrences!)))}
|
|
|
|
kondo-redundant-ignores
|
|
{:doc "Report inline kondo ignores that are no longer necessary (slow: full kondo run)."
|
|
:examples [["./bin/mage kondo-redundant-ignores" "Scan the backend tree for removable ignores"]
|
|
["./bin/mage kondo-redundant-ignores --fix" "Also remove them from source (restores + marks [kondo-keep] any that re-trigger)"]
|
|
["./bin/mage kondo-redundant-ignores --fix --audit" "Recheck [kondo-keep] sites too, unmarking any now truly redundant"]]
|
|
:options [["-f" "--fix" "Remove the candidate ignores from source"]
|
|
["-a" "--audit" "Include sites marked [kondo-keep] instead of skipping them"]]
|
|
:requires [[mage.kondo-ratchet]]
|
|
:task (task! (mage.kondo-ratchet/redundant-ignores parsed))}
|
|
|
|
kondo-insert-ignores
|
|
{:doc "Insert inline ignores at every site a linter flags -- for landing a new linter without a big-bang fix."
|
|
:examples [["./bin/mage kondo-insert-ignores :metabase/new-linter" "Insert ignores everywhere the linter fires, then seed its budget"]
|
|
["./bin/mage kondo-insert-ignores :equals-true test/metabase/pulse" "Only under the given paths"]]
|
|
:arg-schema [:sequential [:string {:name "linter-then-paths"}]]
|
|
:requires [[mage.kondo-ratchet]]
|
|
:task (task! (mage.kondo-ratchet/insert-ignores (:arguments parsed)))}
|
|
|
|
ls
|
|
{:doc "List all mage public and private tasks"
|
|
:examples [["./bin/mage ls" "List all tasks"]]
|
|
:requires [[mage.util :as u]]
|
|
:task (task! (u/print-tasks parsed))}
|
|
|
|
generate-cursive-resolve-as
|
|
{:doc "Generate XML configuration for Cursive's symbol resolution based on the current clj-kondo lint-as configuration."
|
|
:examples [["./bin/mage generate-cursive-resolve-as" "Generate resolve-as XML configuration"]
|
|
["./bin/mage generate-cursive-resolve-as > /Users/bgrabow/Library/Application\\ Support/JetBrains/IntelliJIdea2025.3/options/ClojureResolveSettings.xml"
|
|
"Generate resolve-as XML configuration and install it in your global editor configuration"]]
|
|
:requires [[mage.cursive-resolve-as :as cursive-resolve-as]]
|
|
:task (task! (cursive-resolve-as/do-it))}
|
|
|
|
analytics:setup
|
|
{:doc "Set up usage analytics development mode"
|
|
:examples [["./bin/mage analytics:setup" "Enable usage analytics dev mode and import content"]]
|
|
:requires [[mage.analytics :as analytics]]
|
|
:task (task! (analytics/setup!))}
|
|
|
|
analytics:export
|
|
{:doc "Export usage analytics content to YAML files"
|
|
:examples [["./bin/mage analytics:export" "Export to default location (resources/instance_analytics)"]
|
|
["./bin/mage analytics:export my-custom-dir" "Export to custom directory"]]
|
|
:arg-schema [:or
|
|
[:tuple]
|
|
[:tuple [:string {:description "Target directory for export"}]]]
|
|
:requires [[mage.analytics :as analytics]]
|
|
:task (task! (if-let [target-dir (first (:arguments parsed))]
|
|
(analytics/export! target-dir)
|
|
(analytics/export!)))}
|
|
|
|
analytics:teardown
|
|
{:doc "Tear down usage analytics development mode"
|
|
:examples [["./bin/mage analytics:teardown" "Remove usage analytics dev database and disable dev mode"]]
|
|
:requires [[mage.analytics :as analytics]]
|
|
:task (task! (analytics/teardown!))}
|
|
|
|
analytics:finalize
|
|
{:doc "Export usage analytics content and tear down development mode"
|
|
:examples [["./bin/mage analytics:finalize" "Export to default location and tear down"]]
|
|
:requires [[mage.analytics :as analytics]]
|
|
:task (task! (do
|
|
(analytics/export!)
|
|
(analytics/teardown!)))}
|
|
|
|
doctor
|
|
{:doc "Check your development environment for issues"
|
|
:examples [["./bin/mage doctor" "Run all health checks"]]
|
|
:requires [[mage.doctor :as doctor]]
|
|
:task (task! (doctor/doctor!))}
|
|
|
|
ci-report
|
|
{:doc "Generate a CI failure report for a PR, branch, or commit"
|
|
:examples [["./bin/mage ci-report" "Check current branch (PR or branch CI)"]
|
|
["./bin/mage ci-report 68292" "Generate report for PR #68292"]
|
|
["./bin/mage ci-report source-replacement" "Report for a branch by name"]
|
|
["./bin/mage ci-report abc123def" "Report for a specific commit SHA"]
|
|
["./bin/mage ci-report --detailed 68292" "Detailed report with full FAIL/ERROR blocks"]
|
|
["./bin/mage ci-report --attempt 1 68292"
|
|
"Report for attempt 1 of PR #68292 (use while a rerun attempt is in flight)"]
|
|
["./bin/mage ci-report --attempt=-1 68292"
|
|
"Report for one attempt prior to the latest (0 = latest, -1 = prior, etc.)"]
|
|
["./bin/mage ci-report --detailed > ci_report.md"
|
|
"Save detailed report to markdown file and print progress."]]
|
|
:options [["-d" "--detailed" "Include full FAIL/ERROR blocks with expected/actual diffs"]
|
|
["-a" "--attempt N" "Workflow run attempt number to fetch (default: latest). Negative N indexes back from latest (0 = latest, -1 = one prior). Useful when a rerun is in progress and you want results from an earlier attempt."
|
|
:parse-fn Integer/parseInt]]
|
|
:usage-fn (fn [_]
|
|
(str "Fetches CI check status and failed job logs from GitHub Actions.\n"
|
|
"Parses Trunk test reports to extract test failure details.\n\n"
|
|
"Accepts a PR number/URL, branch name, or commit SHA.\n"
|
|
"With no arguments, uses the current branch's PR if one exists,\n"
|
|
"otherwise falls back to the current branch's latest CI run.\n\n"
|
|
"Use --detailed to get full test failure output with expected/actual diffs.\n"
|
|
"Use --attempt N to fetch a specific workflow run attempt (e.g. attempt 1\n"
|
|
"while attempt 2 is rerunning). N may be 0 (latest) or negative\n"
|
|
"(e.g. -1 = one attempt prior to the latest).\n\n"
|
|
"Requires GitHub CLI (gh) to be installed and authenticated.\n"
|
|
"If you see 'gh: command not found' or auth errors, run: gh auth login"))
|
|
:requires [[mage.ci-report :as ci-report]]
|
|
:task (task! (ci-report/generate-report! parsed))}
|
|
|
|
;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
|
|
;; Private Tasks:
|
|
;; These tasks are more for internal tooling, e.g. for use from git hooks etc.
|
|
;; - hidden from `./bin/mage` listing and `bb tasks`, these all - start with a `-`
|
|
|
|
-ratchets-merge
|
|
{:doc "Internal three-way merge for ratchet files. Called by bin/merge-kondo-ratchets."
|
|
:arg-schema [:tuple
|
|
[:enum {:name "kind"} "kondo" "test" "modules"]
|
|
[:string {:name "base" :description "Ratchets file from the merge base"}]
|
|
[:string {:name "ours" :description "Git stage 2: ratchets file from the branch receiving the changes"}]
|
|
[:string {:name "theirs" :description "Git stage 3: ratchets file from the commits being applied"}]
|
|
[:string {:name "output" :description "Path to write the merged ratchets file"}]]
|
|
:requires [[dev.kondo-ratchet]]
|
|
:task (task!
|
|
(let [[kind base-path ours-path theirs-path output-path] (:arguments parsed)
|
|
modules? (= kind "modules")
|
|
read-stage (fn [path]
|
|
(if modules?
|
|
(binding [dev.kondo-ratchet/*module-ratchets-file* path]
|
|
(dev.kondo-ratchet/read-module-ratchets))
|
|
(binding [dev.kondo-ratchet/*ratchets-file* path]
|
|
(dev.kondo-ratchet/read-ratchets))))
|
|
[base ours theirs] (map read-stage [base-path ours-path theirs-path])]
|
|
(if modules?
|
|
(spit output-path
|
|
(dev.kondo-ratchet/render-module-ratchets
|
|
(dev.kondo-ratchet/merge-module-ratchets base ours theirs)))
|
|
(let [merged (dev.kondo-ratchet/merge-ratchets base ours theirs)
|
|
render (if (= kind "test") dev.kondo-ratchet/render-test dev.kondo-ratchet/render)]
|
|
(spit output-path (if (dev.kondo-ratchet/disabled? merged)
|
|
(slurp ours-path)
|
|
(render merged)))))
|
|
(println (str "merged ratchets into " output-path))))}
|
|
|
|
-kondo-ratchets-pr-body
|
|
{:doc "Render the body for an automated ratchet-shrink PR."
|
|
:arg-schema [:tuple
|
|
[:string {:name "before" :description "Ratchets file before shrinking"}]
|
|
[:string {:name "after" :description "Ratchets file after shrinking"}]
|
|
[:string {:name "modules-before" :description "Module ratchets before shrinking"}]
|
|
[:string {:name "modules-after" :description "Module ratchets after shrinking"}]
|
|
[:string {:name "test-before" :description "Test ratchets file before shrinking"}]
|
|
[:string {:name "test-after" :description "Test ratchets file after shrinking"}]
|
|
[:string {:name "workflow-url" :description "URL of the workflow run"}]]
|
|
:requires [[dev.kondo-ratchet]]
|
|
:task (task!
|
|
(let [[before after modules-before modules-after test-before test-after workflow-url]
|
|
(:arguments parsed)]
|
|
(print (dev.kondo-ratchet/shrink-pr-body-from-files
|
|
{:before before
|
|
:after after
|
|
:modules-before modules-before
|
|
:modules-after modules-after
|
|
:test-before test-before
|
|
:test-after test-after}
|
|
workflow-url))))}
|
|
|
|
-driver-decisions
|
|
{:doc "Determine which driver tests should run based on PR context. Outputs decisions for all drivers."
|
|
:examples [["./bin/mage -driver-decisions --git-ref=master --force-run=false --pr-labels= --skip=false" "Determine driver decisions for a PR"]
|
|
["./bin/mage -driver-decisions --pr-labels=ci:run-all-cloud-drivers" "Run all cloud drivers via label"]
|
|
["./bin/mage -driver-decisions --pr-labels=ci:run-mysql-mariadb" "Force-run a specific driver via label"]
|
|
["./bin/mage -driver-decisions --pr-labels=ci:run-all-drivers" "Force-run all drivers via label"]
|
|
["./bin/mage -driver-decisions --only-driver=bigquery" "Run only the BigQuery job, whatever the other rules say"]
|
|
["./bin/mage -driver-decisions --github-output-only" "Output only GITHUB_OUTPUT format (for CI)"]]
|
|
:requires [[clojure.string :as str]
|
|
[table.core :as table]
|
|
[mage.modules]]
|
|
:usage-fn
|
|
(fn [_current-task]
|
|
(let [driver-triggers (vec (sort mage.modules/default-modules-which-trigger-drivers))]
|
|
(str
|
|
"# How we decide to run a driver's tests. For each driver, we check these conditions in order (first match wins):\n"
|
|
(with-out-str
|
|
(table/table [["Priority" "Condition" "Result" "Reason"]
|
|
["0" "--only-driver=DRIVER was passed" "RUN/SKIP" "only that one driver runs"]
|
|
["1" "Global force-run" "RUN" "master/release branch or ci:run-all label"]
|
|
["2" "Global skip (no backend changes)" "SKIP" "workflow skip (no backend changes)"]
|
|
["3" "Driver is :h2 or :postgres" "RUN" "H2/Postgres always run"]
|
|
["4" "ci:run-all-drivers or ci:run-DRIVER" "RUN" "ci:run-all-drivers or ci:run-DRIVER label"]
|
|
["5" "Driver's own modules/drivers/* files changed" "RUN" "driver files changed"]
|
|
["6" "Cloud driver + ci:run-all-cloud-drivers" "RUN" "ci:run-all-cloud-drivers label"]
|
|
["7" "Cloud driver + query-processor updated" "RUN" "query-processor module updated"]
|
|
["8" "Cloud driver + driver deps affected" "RUN" "driver module affected by shared code changes"]
|
|
["9" "Cloud driver, no relevant changes" "SKIP" "no relevant changes for cloud driver"]
|
|
["10"
|
|
(str (pr-str driver-triggers) " module deps affected")
|
|
"RUN"
|
|
(str "any of " (pr-str driver-triggers) " affected by module changes")]
|
|
["11" "Self-hosted driver, not affected" "SKIP" "driver module not affected"]]))
|
|
"\nThis table decides only WHETHER a driver runs. Whether its failures GATE is a separate\n"
|
|
"question, answered at the end of the job by ci-conductor's quarantine rules.\n"
|
|
"\n\n## Labels of Interest\n\n"
|
|
"These PR labels influence driver test decisions:\n\n"
|
|
" ci:run-all-drivers Force-run ALL drivers\n"
|
|
" ci:run-DRIVER Force-run a specific driver, e.g. ci:run-mysql-mariadb\n"
|
|
" ci:run-all-cloud-drivers Run all cloud drivers (athena, bigquery, databricks, redshift, snowflake)\n"
|
|
"\nTo let a driver run but not gate, add a suite-quarantine rule in CI Conductor instead.\n"
|
|
"\n## All Drivers\n\n"
|
|
(str/join ", " (map name (sort mage.modules/all-drivers)))
|
|
"\n")))
|
|
:options [[nil "--git-ref REF" "The git ref to compute changed modules against"]
|
|
[nil "--force-run FORCE_RUN" "Run every driver regardless of the diff"]
|
|
[nil "--pr-labels PR_LABELS" "Comma delimited labels on the pr"]
|
|
[nil "--skip SKIP"]
|
|
[nil "--only-driver DRIVER" "Run only this driver's job, ignoring every other rule"]
|
|
[nil "--github-output-only" "Only output GITHUB_OUTPUT format (no human-readable output)"]]
|
|
|
|
:task (task! (mage.modules/-main parsed))}
|
|
|
|
-install-merge-drivers
|
|
{:doc "Installs custom git merge drivers for YAML migration files."
|
|
:requires [[mage.util :as u]
|
|
[clojure.string :as str]]
|
|
:task (task!
|
|
(let [git-dir (u/sh "git rev-parse --git-common-dir")
|
|
git-config-file (str git-dir "/config")
|
|
the-include (str "\n"
|
|
"[include]\n"
|
|
" path=../.gitconfig\n")]
|
|
(when-not (str/includes? (slurp git-config-file) the-include)
|
|
(println "Merge driver not installed, installing now.")
|
|
(spit git-config-file the-include :append true))))}
|
|
|
|
;; opposite of -install-merge-drivers, removes the include while it's causing issues.
|
|
-uninstall-merge-drivers
|
|
{:doc "Uninstalls custom git merge drivers for YAML migration files."
|
|
:requires [[clojure.string :as str]
|
|
[mage.util :as u]]
|
|
:task (task!
|
|
(let [git-dir (u/sh "git rev-parse --git-common-dir")
|
|
git-config-file (str git-dir "/config")
|
|
the-include (str "\n"
|
|
"[include]\n"
|
|
" path=../.gitconfig\n")]
|
|
(when (str/includes? (slurp git-config-file) the-include)
|
|
(println "Merge driver is installed, uninstalling now.")
|
|
(let [includeless (-> (slurp git-config-file) (str/replace the-include ""))]
|
|
(spit git-config-file includeless)))))}
|
|
|
|
-merge-yaml-migrations
|
|
{:doc "Custom git merge driver for YAML migration files."
|
|
:usage-fn (fn [_]
|
|
(str
|
|
"Treats each changeset as an atomic unit to avoid conflicts within changesets.\n"
|
|
"Preserves original formatting including blank lines and footer warnings"))
|
|
:arg-schema [:tuple
|
|
[:string {:name "base" :description "ancestor's version (base)"}]
|
|
[:string {:name "ours" :description "current version (ours)"}]
|
|
[:string {:name "theirs" :description "other branch's version (theirs)"}]
|
|
[:int {:name "conflict-marker-size" :description "conflict marker size" :min 0}]
|
|
[:string {:name "file-path" :description "file path"}]]
|
|
:requires [[mage.merge-yaml-migrations]]
|
|
:task (task! (mage.merge-yaml-migrations/-main parsed))}
|
|
|
|
-play
|
|
{:doc "Play sounds from mage.sound"
|
|
:examples [["./bin/mage -play success" "play success sound"] ["./bin/mage -play s" "play success sound"]
|
|
["./bin/mage -play error" "play error sound"] ["./bin/mage -play e" "play error sound"]
|
|
["./bin/mage -play ping" "play ping sound"] ["./bin/mage -play p" "play ping sound"]
|
|
["./bin/mage -play warning" "play warning sound"] ["./bin/mage -play w" "play warning sound"]]
|
|
:requires [[mage.sound :as sound]]
|
|
:arg-schema [:tuple [:enum "s" "success" "e" "error" "p" "ping" "w" "warning"]]
|
|
:task (task! (case (first (:arguments parsed))
|
|
("s" "success") (sound/success)
|
|
("e" "error") (sound/error)
|
|
("p" "ping") (sound/ping)
|
|
("w" "warning") (sound/warning)))}
|
|
|
|
-token-scan
|
|
{:doc "Security token scanner for detecting leaked API keys and secrets"
|
|
:examples [["./bin/mage -token-scan deps.edn bb.edn" "Scan specific files"]
|
|
["./bin/mage -token-scan -v deps.edn" "Scan with verbose output"]
|
|
["./bin/mage -token-scan -a" "Scan all files in the project"]]
|
|
:requires [[mage.token-scan :as token-scan]]
|
|
:options [["-a" "--all-files" "Scan all files in the project"]
|
|
["-v" "--verbose" "Verbose output"]]
|
|
:arg-schema [:sequential [:string {:description "Files to scan"}]]
|
|
:task (task! (token-scan/run-scan parsed))}
|
|
|
|
-repl
|
|
{:doc "Eval a string of clojure code in the backend dev server"
|
|
:requires [[babashka.process :as process]
|
|
[mage.color :as c]
|
|
[mage.be-dev :refer [nrepl-eval]]]
|
|
:examples [["./bin/mage -repl '(+ 1 1)'"
|
|
"Eval a form in the backend dev server in the user namespace"]
|
|
["./bin/mage -repl --namespace metabase.app-db.connection @application-db-counter"
|
|
"Eval a form in the backend dev server in the 'metabase.app-db.connection' namespace"]
|
|
["mage -repl --port 59498 --namespace metabase.app-db.connection '(read-string \"::hello\")'"
|
|
"Evaluate a keyword inside a namespace"]]
|
|
:options [["-n" "--namespace NAMESPACE" "Namespace to use for the task"]
|
|
["-p" "--port PORT" "Port to use for the task, defaults to value in .nrepl-port"]]
|
|
:task (let [parsed (cli/parse! (current-task))
|
|
code (first (:arguments parsed))
|
|
port (or (:port (:options parsed))
|
|
(slurp ".nrepl-port")
|
|
(throw (ex-info (c/red "No port specified, and no .nrepl-port file found.") {:babashka/exit 1})))
|
|
nns (or (:namespace (:options parsed)) "user")]
|
|
(nrepl-eval nns code port))}
|
|
|
|
-check-readable
|
|
{:doc "Check that code is readable parens a file"
|
|
:requires [[mage.readability-check :as readability]]
|
|
:arg-schema [:or
|
|
[:tuple [:string {:name "file"
|
|
:description "File to check"}]]
|
|
[:tuple
|
|
[:string {:name "file"
|
|
:description "File to check"}]
|
|
[:int {:name "line-number"
|
|
:description "A line in the file" :min 1}]]]
|
|
:examples [["./bin/mage -check-readable dev/src/dev.clj"
|
|
"Check file for readability"]
|
|
["./bin/mage -check-readable dev/src/dev.clj 300"
|
|
"Check top level form containing line 300 for readability"]]
|
|
:task (let [{:keys [arguments]} (cli/parse! (current-task))]
|
|
(apply readability/check arguments))}
|
|
|
|
;; Uncomment this if you want to play with the example calculator task:
|
|
#_#_-example-calculator
|
|
{;; `:doc` is a short docstring for the task, will be listed in `./bin/mage -h` and other help info.
|
|
;; Keep these short, they need to fit on one line in the output from `./bin/mage -h`.
|
|
:doc "The example calculator's task docstring"
|
|
;; examples get printed out when you pass -h or --help.
|
|
:examples [["./bin/mage -example-calculator 1 + 99" "evaluates to 100"]
|
|
["./bin/mage -example-calculator 100 - 99" "evaluates to 1"]]
|
|
;; The task is the actual code that runs when you run the task.
|
|
:task (task!
|
|
(println "Parsed input:") (u/pp parsed)
|
|
(let [{:keys [arguments data]} parsed
|
|
[a op b] arguments]
|
|
(println a (name op) b "=" (c/blue (({:+ + :- -} op) a b)))))
|
|
;; (optional) `:require` lazily libraries for just your task:
|
|
:requires [[mage.color :as c]
|
|
[mage.util :as u]]
|
|
;; (optional) `:options` are passed to [[clojure.tools.cli/parse-opts]].
|
|
;; See: https://clojure.github.io/tools.cli/index.html#clojure.tools.cli/parse-opts
|
|
:options [["-n" "--negate" "Negate the result of the calculation"]
|
|
["-a" "--also-add N" "Add N to the result of the calculation"]]
|
|
;; (optional) `:arg-schema` is a malli schema for the arguments passed to the task, after the options
|
|
:arg-schema [:tuple [:int {:name "number-a"}] [:enum {:name "operation"} :+ :-] [:int {:name "number-b"}]]
|
|
;; (optional) `:usage-fn` is a function called with the current-task map.
|
|
;; Returns a string containing extra-detailed usage information.
|
|
:usage-fn (fn [{:keys [doc] :as _task}]
|
|
(str "Optional extra usage information, for explicit callouts.\n"
|
|
"Reversed docstring: '" (c/green (apply str (reverse doc))) "'"))
|
|
;; (optional) Any other keys are completely allowed. Put things to lookup or that you want to be easy to change
|
|
;; here. These will be returned from `(current-task)`. See: start-db for an example of using `:ports` to define a
|
|
;; top-level map of ports. If something should be easily accessible in the task, put it here.
|
|
:data {:a [:b :c] :b [:d]}}
|
|
|
|
-test {:doc "run all mage tests"
|
|
:requires [[clojure.string :as str]
|
|
[clojure.test]
|
|
[mage.core-test] ;; loads the actual tests
|
|
[mage.util :as u]]
|
|
:arg-schema [:sequential [:string {:name "test-matching-pattern"}]]
|
|
:task (let [{:keys [arguments]} (task! parsed)
|
|
pattern (if (seq arguments)
|
|
(str "^mage\\..*(" (str/join "|" arguments) ").*")
|
|
"^mage\\..*")
|
|
_ (u/debug ["test regex matching pattern" pattern])
|
|
results (clojure.test/run-all-tests (re-pattern pattern))]
|
|
(if (zero? (+ (:fail results) (:error results)))
|
|
(println (c/green "All mage tests passed!"))
|
|
(do
|
|
(println (c/red "Some mage tests failed."))
|
|
(u/exit 1))))}
|
|
|
|
;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
|
|
;; Personal Tasks:
|
|
;; These tasks are for personal use, a place for half baked tasks.
|
|
;; Feel free to add your own tasks here, but don't expect them to be run by anyone else, or be listed in the
|
|
;; `./bin/mage` listing.
|
|
|
|
-escherize-pr-summary
|
|
{:doc "Print a day's PR summary for the metabase/metabase repo"
|
|
:requires [[mage.escherize.pr-summary :as eps]]
|
|
:examples [["./bin/mage -escherize-pr-summary | pbcopy" "Copy your daily"]]
|
|
:task (task! (eps/summarize-prs parsed))}
|
|
|
|
-escherize-fix-whitespace
|
|
{:doc "Removes all lines like '^<space>+$' from files in glob pattern."
|
|
:requires [[mage.escherize.fix-whitespace :as fix-whitespace]]
|
|
:examples [["mage -escherize-fix-whitespace '**/*.clj{,c,s}'" "Removes all lines like ' ' from all clj files"]]
|
|
:arg-schema [:sequential [:string {:name "glob-pattern" :description "Glob pattern to match files"}]]
|
|
:task (task! (fix-whitespace/run-clean parsed))}
|
|
|
|
-edpaget-dev-env
|
|
{:doc "Stand up (or tear down) an isolated dev environment for this worktree"
|
|
:examples [["./bin/mage -edpaget-dev-env --app-db postgres" "Postgres app-db with auto ports"]
|
|
["./bin/mage -edpaget-dev-env --app-db postgres --with mysql --with mongo" "Full stack"]
|
|
["./bin/mage -edpaget-dev-env" "Minimal: just port config (H2 app-db)"]
|
|
["./bin/mage -edpaget-dev-env --down" "Tear down containers for this worktree"]
|
|
["./bin/mage -edpaget-dev-env --status" "Show running containers and ports"]]
|
|
:requires [[mage.edpaget.dev-env :as dev-env]]
|
|
:options [[nil "--app-db DB" "App database: h2 (default), postgres, mysql" :default "h2"]
|
|
[nil "--with SERVICE" "Warehouse service (repeatable): postgres, mysql, mariadb, mongo, clickhouse, ldap, maildev"
|
|
:multi true :default [] :update-fn conj]
|
|
[nil "--down" "Tear down all containers for this worktree"]
|
|
[nil "--status" "Show current env status"]
|
|
[nil "--slot SLOT" "Override port slot (0-99)" :parse-fn Integer/parseInt]]
|
|
:task (task! (dev-env/dev-env! parsed))}
|
|
|
|
-johnswanson-dev-env
|
|
{:doc "Stand up (or tear down) an isolated dev environment for this worktree. Interactive prompts when flags omitted."
|
|
:examples [["./bin/mage -johnswanson-dev-env" "Interactive TUI to configure infra (saves config)"]
|
|
["./bin/mage -johnswanson-dev-env --run" "Configure + launch backend/frontend"]
|
|
["./bin/mage -johnswanson-dev-env --run --no-frontend" "Configure + backend only"]
|
|
["./bin/mage -johnswanson-dev-env --down" "Stop everything for this worktree"]
|
|
["./bin/mage -johnswanson-dev-env --status" "Show running containers and processes"]]
|
|
:requires [[mage.johnswanson.dev-env :as jds-dev-env]]
|
|
:options [["-e" "--edition EDITION" "ee or oss (default: prompt)"]
|
|
["-t" "--token TOKEN" "Token type: all-features, starter-cloud, pro-cloud, pro-self-hosted, none"]
|
|
[nil "--app-db DB" "App database: h2, postgres, mysql, mariadb (default: prompt)"]
|
|
[nil "--with SERVICE" "Warehouse service (repeatable): postgres, mysql, mariadb, mongo, clickhouse, ldap, maildev"
|
|
:multi true :default [] :update-fn conj]
|
|
[nil "--run" "Also launch backend + frontend in background"]
|
|
[nil "--no-frontend" "Skip frontend dev server (only with --run)"]
|
|
[nil "--down" "Stop everything for this worktree"]
|
|
[nil "--status" "Show current env status"]
|
|
[nil "--slot SLOT" "Override port slot (0-99)" :parse-fn Integer/parseInt]
|
|
[nil "--fresh" "Recreate containers from scratch"]]
|
|
:task (task! (jds-dev-env/dev-env! parsed))}
|
|
|
|
-bot-dev-env
|
|
{:doc "Stand up (or tear down) an isolated dev environment for this worktree. Interactive prompts when flags omitted."
|
|
:examples [["./bin/mage -bot-dev-env" "Interactive TUI to configure infra (saves config)"]
|
|
["./bin/mage -bot-dev-env --run" "Configure + launch backend/frontend"]
|
|
["./bin/mage -bot-dev-env --run --no-frontend" "Configure + backend only"]
|
|
["./bin/mage -bot-dev-env --down" "Stop everything for this worktree"]
|
|
["./bin/mage -bot-dev-env --status" "Show running containers and processes"]]
|
|
:requires [[mage.bot.dev-env :as jds-dev-env]]
|
|
:options [["-e" "--edition EDITION" "ee or oss (default: prompt)"]
|
|
["-t" "--token TOKEN" "Token type: all-features, starter-cloud, pro-cloud, pro-self-hosted, none"]
|
|
[nil "--app-db DB" "App database: h2, postgres, mysql, mariadb (default: prompt)"]
|
|
[nil "--with SERVICE" "Warehouse service (repeatable): postgres, mysql, mariadb, mongo, clickhouse, ldap, maildev"
|
|
:multi true :default nil :update-fn conj]
|
|
[nil "--bot BOT" "Bot name (enables bot-specific config: Docker containers, bot dirs, Playwright dirs)"]
|
|
[nil "--run" "Also launch backend + frontend in background"]
|
|
[nil "--no-frontend" "Skip frontend dev server (only with --run)"]
|
|
[nil "--down" "Stop everything for this worktree"]
|
|
[nil "--status" "Show current env status"]
|
|
[nil "--slot SLOT" "Override port slot (0-99)" :parse-fn Integer/parseInt]
|
|
[nil "--fresh" "Recreate containers from scratch"]
|
|
[nil "--worktree PATH" "Target a different worktree (default: current)"]]
|
|
:task (task! (jds-dev-env/dev-env! parsed))}
|
|
|
|
-bot-setup-worktree
|
|
{:doc "Run final configuration steps after a new worktree is created."
|
|
:examples [["./bin/mage -bot-setup-worktree --worktree $(pwd)" "Set up the current directory as a worktree"]]
|
|
:options [[nil "--worktree PATH" "Path to the worktree to set up (required)"]]
|
|
:requires [[mage.bot.setup-worktree :as setup-worktree]]
|
|
:task (task! (setup-worktree/setup-worktree! parsed))}
|
|
|
|
-bgrabow-ci-test-repl
|
|
{:doc "Start a REPL replicating a CI test job environment (Docker + env vars)"
|
|
:examples [["./bin/mage -bgrabow-ci-test-repl" "Interactive selection with fzf"]
|
|
["./bin/mage -bgrabow-ci-test-repl app-db/be-tests-mysql" "Start REPL for a specific job"]
|
|
["./bin/mage -bgrabow-ci-test-repl -n app-db/be-tests-mysql" "Dry run: print commands without executing"]]
|
|
:requires [[mage.bgrabow.ci-test-repl :as ci-test-repl]]
|
|
:options [["-n" "--dry-run" "Print docker, env, and clojure commands without executing"]]
|
|
:task (task!
|
|
(let [wj (first (:arguments parsed))
|
|
[workflow job] (when wj (clojure.string/split wj (re-pattern "/") 2))]
|
|
(ci-test-repl/ci-test-repl {:workflow workflow :job job :dry-run? (:dry-run (:options parsed))})))}
|
|
|
|
-bot-status
|
|
{:doc "Watch and display bot status"
|
|
:requires [[mage.bot.status :as status]]
|
|
:arg-schema [:? [:string {:name "file-path"}]]
|
|
:task (task! (status/run! parsed))}
|
|
|
|
-bot-generate-prompt
|
|
{:doc "Fill a template with placeholders and write an agent prompt"
|
|
:examples [["./bin/mage -bot-generate-prompt --template dev/bot/fixbot-agent.md --output .bot/fixbot/20260410/prompt.md --set ISSUE_ID=MB-123"
|
|
"Generate fixbot prompt"]]
|
|
:requires [[mage.bot.prompt :as bot-prompt]]
|
|
:options [[nil "--template FILE" "Template file path"]
|
|
[nil "--output FILE" "Output file path"]
|
|
[nil "--set KEY=VALUE" "Set a template variable (can be repeated)"
|
|
:multi true :default [] :update-fn conj]
|
|
[nil "--set-from-file KEY=PATH" "Set a template variable from a file's contents (can be repeated)"
|
|
:multi true :default [] :update-fn conj]]
|
|
:task (task! (bot-prompt/generate-prompt! parsed))}
|
|
|
|
;;; Unified autobot session management
|
|
|
|
-autobot-go
|
|
{:doc "Launch a bot in an autobot session with its own worktree, backend, and frontend (or remote PR env)"
|
|
:examples [["./bin/mage -autobot-go master --bot qabot --command /qabot"
|
|
"Launch qabot in an autobot session"]
|
|
["./bin/mage -autobot-go my-branch --bot fixbot --command '/fixbot MB-12345' --app-db mysql"
|
|
"Launch fixbot with MySQL"]
|
|
["./bin/mage -autobot-go uxw-291 --bot qabot --command /qabot --pr-env-url https://pr383713.coredev.metabase.com"
|
|
"Launch qabot against a remote PR preview environment"]]
|
|
:requires [[mage.bot.autobot :as autobot]]
|
|
:arg-schema [:tuple [:string {:name "branch-name"}]]
|
|
:options [[nil "--bot BOT" "Bot name (qabot, fixbot, uxbot, reprobot, cibot)"]
|
|
[nil "--command CMD" "Inner command to run (e.g., /qabot, '/fixbot MB-12345')"]
|
|
[nil "--app-db DB" "App database" :default "postgres"]
|
|
[nil "--base BASE" "Base branch/commit to branch from. If set and the branch does not exist, it will be created from this base."]
|
|
[nil "--pr-env-url URL" "PR preview environment URL (enables remote mode, no local dev-env)"]]
|
|
:task (task! (autobot/go! parsed))}
|
|
|
|
-bot-pr-env
|
|
{:doc "Configure a worktree for a remote PR preview environment (writes .bot/pr-env.env + caches session token)"
|
|
:examples [["./bin/mage -bot-pr-env --url https://pr383713.coredev.metabase.com --pr 383713"
|
|
"Set up this worktree to talk to the given PR preview environment"]]
|
|
:requires [[mage.bot.pr-env :as pr-env]]
|
|
:options [[nil "--url URL" "PR preview environment URL"]
|
|
[nil "--pr PR" "PR number"]]
|
|
:task (task! (pr-env/setup! parsed))}
|
|
|
|
-autobot-stop
|
|
{:doc "Stop an autobot session (kill tmux + dev env, keep worktree). No args = stop current session."
|
|
:examples [["./bin/mage -autobot-stop qabot-master" "Stop named session"]
|
|
["./bin/mage -autobot-stop" "Stop current session (from inside an autobot session)"]]
|
|
:requires [[mage.bot.autobot :as autobot]]
|
|
:arg-schema [:? [:string {:name "name-or-id"}]]
|
|
:task (task! (autobot/stop! parsed))}
|
|
|
|
-autobot-list
|
|
{:doc "List all autobot sessions with status"
|
|
:examples [["./bin/mage -autobot-list" "List all sessions"]]
|
|
:requires [[mage.bot.autobot :as autobot]]
|
|
:task (task! (autobot/list-all! parsed))}
|
|
|
|
-autobot-result
|
|
{:doc "Print the most recent result.md for a branch+bot combination"
|
|
:examples [["./bin/mage -autobot-result uxw-291-skip-ldap-for-non-ldap qabot"
|
|
"Print qabot's latest result.md for the given branch"]]
|
|
:requires [[mage.bot.autobot :as autobot]]
|
|
:arg-schema [:tuple [:string {:name "branch"}] [:string {:name "bot"}]]
|
|
:task (task! (autobot/result! parsed))}
|
|
|
|
-autobot-kill
|
|
{:doc "Tear down and remove an autobot session. No args = remove current session."
|
|
:examples [["./bin/mage -autobot-kill qabot-master" "Remove named session"]
|
|
["./bin/mage -autobot-kill" "Remove current session (from inside an autobot session)"]]
|
|
:requires [[mage.bot.autobot :as autobot]]
|
|
:arg-schema [:? [:string {:name "name-or-id"}]]
|
|
:task (task! (autobot/kill! parsed))}
|
|
|
|
;;; Shared bot tasks
|
|
|
|
-bot-fetch-issue
|
|
{:doc "Fetch a Linear issue by identifier (shared across bots)"
|
|
:examples [["./bin/mage -bot-fetch-issue MB-12345" "Fetch and display issue"]]
|
|
:requires [[mage.bot.linear :as bot-linear]]
|
|
:arg-schema [:tuple [:string {:name "issue-id"}]]
|
|
:task (task! (bot-linear/print-issue! parsed))}
|
|
|
|
-bot-server-info
|
|
{:doc "Print server configuration (ports, config file, users, API keys) for bot agents"
|
|
:examples [["./bin/mage -bot-server-info" "Show server info"]]
|
|
:requires [[mage.bot.server-info :as server-info]]
|
|
:task (task! (server-info/server-info! parsed))}
|
|
|
|
-bot-preflight
|
|
{:doc "Run prerequisite checks for a bot workflow"
|
|
:examples [["./bin/mage -bot-preflight --bot fixbot" "Run checks for fixbot (Linear required)"]
|
|
["./bin/mage -bot-preflight --bot qabot" "Run checks for qabot (Linear optional)"]]
|
|
:options [[nil "--bot BOT" "Bot name (fixbot, qabot, uxbot, reprobot, cibot)"]]
|
|
:requires [[mage.bot.preflight :as preflight]]
|
|
:task (task! (preflight/preflight! parsed))}
|
|
|
|
-bot-preflight-health
|
|
{:doc "Wait for Metabase backend to become healthy (retries for up to 5 minutes)"
|
|
:examples [["./bin/mage -bot-preflight-health" "Wait for backend health"]]
|
|
:requires [[mage.bot.preflight :as preflight]]
|
|
:task (task! (preflight/preflight-health! parsed))}
|
|
|
|
-bot-timestamp
|
|
{:doc "Print the current wall-clock time as YYYYMMDD-HHMMSS (no other output)"
|
|
:examples [["./bin/mage -bot-timestamp" "Print e.g. 20260413-143022"]]
|
|
:task (task! (println (.format (java.time.LocalDateTime/now)
|
|
(java.time.format.DateTimeFormatter/ofPattern "yyyyMMdd-HHmmss"))))}
|
|
|
|
-bot-repl-eval
|
|
{:doc "Evaluate Clojure code against the best-available REPL (nREPL first, socket REPL fallback)"
|
|
:examples [["./bin/mage -bot-repl-eval '(+ 1 2)'" "Auto-discover and eval"]
|
|
["./bin/mage -bot-repl-eval --type socket '(metabase.config/current-version)'" "Force socket REPL"]
|
|
["./bin/mage -bot-repl-eval --port 50666 '(+ 1 2)'" "Force a specific port"]]
|
|
:requires [[mage.bot.repl-eval :as repl-eval]]
|
|
:arg-schema [:? [:string {:name "code"}]]
|
|
:options [[nil "--code CODE" "Clojure code to eval (alternative to positional arg)"]
|
|
[nil "--port PORT" "Override discovery and use this port"]
|
|
[nil "--host HOST" "Override discovery and use this host (socket REPL only)"]
|
|
[nil "--type TYPE" "Force backend: 'nrepl' or 'socket'. Default: auto-detect."]
|
|
[nil "--refresh" "Ignore .bot/repl.env cache and re-run discovery"]]
|
|
:task (task! (repl-eval/repl-eval! parsed))}
|
|
|
|
-bot-api-call
|
|
{:doc "Make an API call to the local Metabase instance"
|
|
:examples [["./bin/mage -bot-api-call /api/health" "Health check"]
|
|
["./bin/mage -bot-api-call /api/card --api-key mb_admin_apikey" "List cards as admin"]
|
|
["./bin/mage -bot-api-call /api/card --method POST --api-key mb_admin_apikey --body '{\"dataset_query\":{}}'" "Create a card"]]
|
|
:requires [[mage.bot.api-call :as api-call]]
|
|
:arg-schema [:tuple [:string {:name "api-path"}]]
|
|
:options [[nil "--method METHOD" "HTTP method (GET, POST, PUT, DELETE)" :default "GET"]
|
|
[nil "--api-key KEY" "API key for x-api-key header"]
|
|
[nil "--body JSON" "JSON request body"]
|
|
[nil "--pretty BOOL" "Pretty-print JSON (default true)" :default "true"]
|
|
[nil "--raw" "Suppress diagnostic preamble on stderr (pure-body stdout for jq/python piping)"]]
|
|
:task (task! (api-call/api-call! parsed))}
|
|
|
|
-bot-md-to-pdf
|
|
{:doc "Convert a markdown file to PDF using md-to-pdf"
|
|
:examples [["./bin/mage -bot-md-to-pdf report.md" "Convert report.md to report.pdf"]
|
|
["./bin/mage -bot-md-to-pdf .bot/qabot/20260410/report.md" "Convert with full path"]]
|
|
:arg-schema [:tuple [:string {:name "file"}]]
|
|
:task (task!
|
|
(let [md-file (first (:arguments parsed))
|
|
dir (.getParent (java.io.File. ^String md-file))
|
|
fname (.getName (java.io.File. ^String md-file))]
|
|
(babashka.tasks/shell {:dir (or dir ".")} "npx" "-y" "md-to-pdf" fname)))}}}
|