Compare commits

...

262 Commits

Author SHA1 Message Date
Chris Truter
2db6e9b6f1 Give a measure or segment its parent table in the prompt
A measure's prompt carried its own name and description and nothing else, so a name like "Active" or
"Total" arrived with no way to tell what it counts. The table it is defined on is the context that
makes it interpretable, and it is the mirror of the measures and segments a table now sends.

`parent-table-by-entity` is a shared hydration on both via-parent projections: one batched select,
keyed by the child, capped at the same description length the table's own projection uses. A child has
one parent, so the whole cost is one description.

It is in the basis. Renaming or redescribing a table changes what every measure and segment on it is
told, so they should regenerate with it.
2026-09-11 02:10:59 -04:00
Chris Truter
b5c05a0708 Say why the query cannot own the measure/segment ordering
The comment led with app db collation, which is why ordering in SQL is risky rather than why it is
impractical. The reason it cannot be done there at all: the query fetches a chunk of tables at once,
so capping per table needs a window function, and which rows that cap keeps would then rest on tie
and NULL ordering that varies by dialect.
2026-09-09 09:57:40 -04:00
Chris Truter
403f6dadb3 Show a table's measures and segments in its generation prompt
A table's `:osi-context` projection carried only its name, display name, description and field
names. Measures and segments are the authored statements of what the table is used to compute and
how it is sliced — the strongest evidence available for the synonyms and example questions the model
is asked to write — and none of it reached the prompt.

Two batch hydrations add them as `{:name :description}` summaries, sorted by name, capped at 50 per
table with each description capped at 500 characters so one verbose child cannot crowd out the rest
of the table's context. Both are in the `:basis`: a new, renamed or redescribed measure is an
authored change that should schedule the table for regeneration, and both values are deterministic
and bounded, which is what the basis requires.

Every library table's basis changes, so the next run regenerates all of them. The template edit bumps
`generator-version` regardless.
2026-09-09 09:28:58 -04:00
Chris Truter
906aebf3d3 Improve OSI generation prompt evidence handling 2026-09-08 01:39:25 -04:00
Chris Truter
7735051b1b Keep rollout guidance in PR descriptions 2026-09-07 17:12:51 -04:00
Chris Truter
e091d8d3c2 Configure provider connections in structured usage tests 2026-09-07 16:46:25 -04:00
Chris Truter
4818565f36 Document the cluster upgrade requirement for OSI generation 2026-09-07 16:41:59 -04:00
Chris Truter
dde4a0092e Preserve usage across interrupted LLM streams 2026-09-07 16:41:59 -04:00
Chris Truter
651cec42ba Hint the TriggerKey in the generation scheduling test
`existing-triggers` hints it in production; without the same hint here the call
reflects, which the changed-tests Eastwood job fails on.
2026-09-07 16:41:59 -04:00
Chris Truter
2566752bc9 Leave the text block open when a Gemini stream is interrupted
Closing it emits :text-end, which presents a truncated answer as a complete one.
Provisional tool calls are still discarded and the last reported usage still flushed.
2026-09-07 16:41:59 -04:00
Chris Truter
efaf23cc89 Attribute consumer failures correctly in the vLLM stream guard
`io-guarded` wraps every IOException raised inside `.reduce`, including ones the
consumer throws. That relabels a downstream failure as a vLLM transport error and
tells `completion-safe-eduction` the source died, so it flushes as though the
stream had been interrupted. Only source failures are translated now.
2026-09-07 16:41:58 -04:00
Chris Truter
b5a9b92e8a Route the DeepSeek and vLLM adapters through completion-safe-eduction
Both landed on master while this branch was off it, so they never picked up the
wrapper the other ten adapters use. Without it a transport failure discards the
buffered text a tool stream had already made safe, and DeepSeek also drops its
last reported usage, undercounting billing and quotas.
2026-09-07 16:41:58 -04:00
Chris Truter
3d70c0cfa4 Address OSI generation and stream safety review findings 2026-09-07 16:41:58 -04:00
Chris Truter
16d23aa517 OSI generation engine: candidate selection, LLM generation, write-back, weekly task, run controls 2026-09-07 16:41:58 -04:00
Chris Truter
7d97df904c fix LLM provider readiness validation 2026-09-07 16:41:35 -04:00
Chris Truter
294fe63325 Configure LLM access for OSI metadata generation
Generation runs on its own model reference, defaulting to whatever Metabot runs on.
The reference names a connection in `llm-providers`, so pointing generation at a second
connection of the same provider type is all it takes to give it separate credentials and
separate spend — no new provider secrets, and nothing in this module knows it happened.

Registers `osi-generation` as its own usage source with a label in the AI analytics views,
so generation traffic is attributable apart from interactive Metabot usage.

`validate-model-ref!` becomes public and gains `normalize-model-ref`, shared by all three
model-reference settings. Normalizing before validating is what lets an admin clear one:
a form clears a string setting by sending "", which the old setters rejected as a missing
model name rather than treating as unset.

No LLM calls are made here.
2026-09-07 16:41:35 -04:00
Chris Truter
fa7e19d9a4 Preserve streaming hydration dependencies after restacking 2026-09-07 16:41:14 -04:00
Chris Truter
e420118c97 Let unexpected AI context failures abort reconciliation 2026-09-07 16:37:10 -04:00
Chris Truter
566300ec08 Nudge the index after the update commits, not before it runs
Toucan calls `before-update` hooks before opening its own transaction, so outside a
caller transaction `do-after-commit` ran the nudge immediately and the reconcile read
the row as it stood before the UPDATE. Nothing nudged again, leaving the slice stale
until the periodic full reconcile.

`after-update` runs inside that transaction, but `t2/changes` there is a lazy row whose
`contains?` answers for columns rather than changes, which would report every column as
changed. So a pipeline method publishes the real changes map around the update and the
hook reads it, keeping the `ai_context`-only filter: in steady state most generation
writes only restamp a basis, and each skipped nudge is an exclusive index lock not taken.
2026-09-07 16:37:10 -04:00
Chris Truter
64906c88c0 Record why a silent provider-side model change does not rebuild the index
`model-identity` cannot tell that a provider re-trained the model behind an
unchanged name. Detecting it would mean probing on every startup and re-embedding
the whole library on disagreement, where a false positive costs as much as a true
one — so the case is accepted rather than guarded.
2026-09-07 16:37:10 -04:00
Chris Truter
01174b002d Standardize the new entity-retrieval tests on ^:synchronized
Master retired `^:sequential` in #80064; these tests are new on this branch, so the
rename never reached them. `^:synchronized` is what now buys them isolation.
2026-09-07 16:37:10 -04:00
Chris Truter
6ab4126f78 Reference QUE2-804 from the nfc_path TODO 2026-09-07 16:37:09 -04:00
Chris Truter
7887841730 Correct three overstated claims in the entity-retrieval docs
field-path: no per-row rule can be right. BigQuery a.b.b and Mongo a.b both
arrive as {:name "b", :nfc_path ["a" "b"]} and want different answers, so a
BigQuery field sharing its parent's name collapses into that parent. Joining is
the safer guess -- appending would mangle every Mongo nested field rather than
only same-named BigQuery ones. Pinned in field-path-test so the limit is stated
rather than discovered.

Embeddings are not content-addressed. doc_id covers the entity and doc type as
well as the text, and insert-batch! has no cache, so the same text on two
entities is embedded twice. What is true, and what the comment meant, is that
both paths embed exactly the documents whose doc_id is not already stored --
batching changes the number of provider requests, not how much text is embedded.

hydration-query-chunk-size bounds one query and nothing else. hydrate hands each
batch fn the whole entity collection and holds its whole result map, and
raw-ai-context-rows materializes every chunk, so the constant never bounded
total hydration memory.
2026-09-07 16:37:09 -04:00
Chris Truter
cb0a754b4c Correct entity retrieval documentation contracts 2026-09-07 16:37:09 -04:00
Chris Truter
0f3dee138e Fix cross-driver field paths and stream field hydration 2026-09-07 16:37:09 -04:00
Chris Truter
2def4914d2 Name nested fields by nfc_path alone, and stream the hydration
field-path appended the leaf name to nfc_path, on the belief that nfc_path held
ancestors only. It does not. Both drivers that produce it write the whole chain
including the leaf: mongo.clj says so outright, and describe_table.clj builds
the field's name and its nfc-path from the same value. So a sql-jdbc JSON field
came out as "payload.user.id.payload -> user -> id" and a Mongo one as
"payload.user.id.id". Join nfc_path and stop appending.

Preferring nfc_path over the name also normalizes the two conventions, which
disagree: sql-jdbc stores the arrow-joined path as the field's name, Mongo
stores only the leaf. Mongo nested fields are now resolved rather than left
ambiguous, so that caveat is gone.

table-field-names claimed a memory bound it did not have: t2/select
materialized every field of every table in the chunk before the per-table cap
applied. keep-first-names is already a reducing fn, so a reducible-select folds
rows in as they arrive and the cap now bounds the peak, not just the result.

The hydration test grows a nested field, which is also the only coverage of the
nfc_path read transform: a path arriving as raw JSON rather than a vector would
silently produce a garbage name. The read-schema test grows a non-string item,
so restoring the item-type constraint would now fail it.
2026-09-07 16:37:09 -04:00
Chris Truter
451156caea Nudge the index from the osi_ai_context model again
Serdes import writes rows through Toucan like anything else and has nowhere
obvious to call a nudge from, so an imported synonym stayed unsearchable until
the ~15m full reconcile. Hanging the nudge off the model covers it, and every
future writer, instead of each write site having to remember.

The comment justifying the removal said a batch writer would otherwise cause a
per-row reconcile cascade. It would not: request-entity-sync! only adds to a
dirty set, and schedule! keeps at most one pending run, so N writes drain in one
run. Updates nudge only when ai_context changed, which is the one column an
index doc derives from, so a regenerate request or a basis stamp still queues
nothing.

The CRUD API's explicit nudges go away as redundant. before-delete rather than
after-delete because Toucan has no after-delete; the nudge defers to commit
either way, so it never fires for a write that rolls back.
2026-09-07 16:37:09 -04:00
Chris Truter
7b85662b6c Improve entity retrieval documentation 2026-09-07 16:36:39 -04:00
Chris Truter
2961a56e9d Relax the ai_context read schema, note the Mongo path gap
AiContextResponse asserted the write caps on responses, so a row that predates
a cap, arrived by serdes, or was written straight to the appdb would fail
validation and take the whole list down with it. Every other read path already
tolerates those rows: instructions truncate on read, and index docs skip values
they cannot use. Drop the caps and item types to match, and pin it with a test
that reads back an over-cap row.

field-path silently falls back to the bare leaf name for Mongo-style nesting, a
parent_id with no nfc_path, so those fields stay ambiguous. Say so: resolving
them means walking ancestors, which needs a whole table's fields in memory and
gives up the incremental cap.
2026-09-07 16:36:39 -04:00
Chris Truter
6a841f5c45 Fix review findings on the entity-retrieval spec layer
Regenerate no longer flips data_source. That column describes the content
sitting in the row, so flipping it on a request stopped the entity's approved
instructions reaching the agent while the content was still the approved
content, the opposite of what the endpoint documents. The generator flips it
when it writes, and reads the request from rewrite_requested_at.

An unusable ai_context degrades an entity to its name and description docs
rather than dropping it, which on a first build left it unfindable by its own
name. It no longer blocks reconciled_at either: the blob stays bad until
someone repairs the row, so waiting for it to heal pinned the staleness check
at critical forever. It reports a gauge instead. An unforeseen projection error
still blocks, since a retry might clear it.

Reject a :membership declaring neither :where nor :via-parent. It formats to
WHERE TRUE, so every row in the table registered as a library member, 467 of
them in the test appdb, with no error. member-entity's via-parent check now
requires the parent model to declare the projection too, matching
member-entities.

Field names handed to generation are path-qualified, so nested leaves sharing a
name stay distinct. table-field-names chunks its :in clause like the other
hydrations and caps incrementally rather than sorting the whole set.
2026-09-07 16:36:39 -04:00
Chris Truter
8724e1794c OSI foundation: entity-retrieval spec layer + osi_ai_context generation metadata and CRUD 2026-09-07 16:36:38 -04:00
Chris Truter
94d7dd520c Show what automated ratchet PRs shrink (#81807)
* Show what automated ratchet PRs shrink

* Tweak messaging

* Fix ratchet PR body test lint
2026-09-03 12:57:39 +02:00
Jakub Chodorowicz
e4cf63ff23 gate event apis on window instance (#81680) 2026-09-03 11:03:12 +02:00
Timofey Kachalov
c73e309b13 Adjust skill guidance telling agents not to offer a "deploy" step for data apps (#81791) 2026-09-03 11:45:58 +04:00
Timofey Kachalov
8196701636 Update data apps upsell (#81786) 2026-09-03 10:56:26 +04:00
github-automation-metabase
042ec24fe9 Tighten ratchets (#81862)
Co-authored-by: github-actions <github-actions@github.com>
2026-09-03 04:58:07 +00:00
Alexander Polyankin
ff335470e7 Route each module's app-DB queries through a db.clj namespace (#81824) 2026-09-03 04:52:56 +00:00
Fraser
75bce3e619 Move open-url out of visualizations into urls (#81842) 2026-09-03 16:14:55 +12:00
Fraser
9ddee1f434 Register the use-icon spec's own visualizations from viz-core (#81844) 2026-09-03 16:01:14 +12:00
Fraser
0337de20c1 Read the app slice instead of current-user in the redux hooks spec (#81841) 2026-09-03 15:46:09 +12:00
Fraser
c71cf2a379 Move useHelpLink from nav into common/hooks (#81843) 2026-09-03 15:45:42 +12:00
Fraser
06462d0005 Enforce shared tiers for urls (#81845) 2026-09-03 15:45:26 +12:00
Alexander Polyankin
e6aa23212c Write re-encrypted rows with plain SQL, not Toucan DML (#81829) 2026-09-02 23:42:56 +00:00
bryan
79041e2487 mcp-v2.00 prep: extract query guards + pulse helpers; additive shared-ns changes v2 needs (#81244)
* MCP v2 prep: extract query guards + pulse helpers; additive shared-ns changes v2 needs

Behavior-preserving groundwork for the v2 MCP surface, landed while v1 is
untouched and v2 is not yet present:

- NEW metabase.agent-api.query-guards (+ its test): query safety checks
  extracted so REST and MCP run identical guard logic.
- metabase.pulse: recipient-filtering and sensitive-metadata-stripping
  logic moved from private fns in api/pulse.clj into models/pulse.clj and
  exported via pulse.core / pulse.api, so REST and MCP share it. Pure move;
  existing REST pulse tests are the check (+ new direct tests).
- metabase.channel.urls: adds metric-path, transform-path, collection-path
  helpers (additive).
- metabase.metabot.scope: adds agent:{content,query,sql,delivery}:* scopes
  and collapses agent:viz:mcp-ui:{query,drill-through} into
  agent:viz:mcp-ui:* — a proven superset, backward-compatible with v1's
  gating (REPL-verified: old concrete scopes still match). Also regates
  agent:metric:* from SQL-generation to NLQ permission (scope_test).
- metabase.mcp.settings: adds mcp-v2-disabled-tools and
  mcp-query-handle-ttl-hours (additive; v1-safe; tests included).
- metabase.models.serialization.resolve{,.mp}: adds *numeric-ids-allowed?*
  dynvar; resolver miss messages no longer name a specific discovery tool
  (the calling surface owns the recovery sentence) — mp_test updated.
- metabase.server.streaming-response: optional :executor option threaded
  through do-f-async; default path (shared pool) unchanged. Needed by
  long-lived SSE keepalive streams so they can't exhaust the fixed pool.

Review question for this PR: does REST/v1 behavior stay identical?

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BjQcjQx7NPGr3Rpf6QaG4b

Review fixes (2026-08-27 deep review of #81244):
- native-marker? requires a truthy :native value (an explicit-null key from a
  JSON round-trip is not a marker); docstrings state the guard/agent-api
  sequencing explicitly (live v1 path untouched; call sites migrate at retirement).
- check-mcp-ui-native-query! accepts agent:sql:execute (v1's concrete scope)
  alongside agent:sql:run, and documents that scope claims arrive with the v2
  session rework — until then native-over-UI-credential fails closed.
- mcp-query-handle-ttl-hours getter no longer rethrows on an unparseable env
  var (falls back to the default instead of killing the GC task).
- *numeric-ids-allowed?* docstring rewritten in future tense naming its binder
  (the v2 query pipeline) and reader (the dialect rework PR).
- viz-scope comment corrected (v1 resources still register those strings until
  retirement; wildcard keeps them satisfied); mp.clj database-id tru args
  str-wrapped against MessageFormat locale grouping; streaming-response
  :executor docstring notes the Future.cancel(true) interruption requirement.

Follow-up (review round 2): the guards are now genuinely shared — agent-api's
private copies are deleted and its execution paths call query-guards directly.
Deliberate tightening: raw-scan native detection replaces normalize-then-inspect
(fail-closed on unparseable payloads). Full agent-api api-test green.

Re-review round: agent:sql:execute compat branch now tested; TTL getter's
catch path tested ("forever"); reject-native-query! no longer echoes the
payload in ex-data (matches the old live rejection shape); viz-scope comment
typo fixed; pure predicate tests ^:parallel; check-token-query-permissions!
docstring states the stage-0-fast-path/QP-backstop contract.

--- Close-out review response (slice 00's first adversarial round) ---

CRITICAL (native-query false-positive on legacy MBQL with a field/expression/tag named `native`)
FIXED. native-query? scanned the whole payload via tree-seq, so a legit query whose :expressions or
:template-tags map had a key named "native" was rejected (400) though it ran fine on master. It now
scans only the structural query-nesting edges (:stages/:joins, :query/:source-query), never
descending into caller-named :expressions/:template-tags; a malformed value off a structural edge is
still deep-scanned to stay fail-closed. Proven: a 17-case REPL matrix — every crafted false-positive
now false, every genuine native (legacy top/nested, MBQL5 stage, joins, string-keyed, malformed)
still true.

SECURITY (v1 MCP bearer authenticates a deactivated user) FIXED. resolve-access-token — the single
resolver shared by the v1 transport and the core session bearer bridge — now gates on
(t2/exists? :model/User :id user-id :is_active true). v1's transport dispatched straight on :user-id
with no active check; v2 was already refuse-only. Proven fails-when-broken: reverting the guard lets
a deactivated user's token resolve; restored, it is refused. v1 mcp.api-test (598 assertions) green.

IMPORTANT fixed: agent:delivery:write now granted by the metabot-other-tools bucket (was dead
config → alert_write would 403 everyone); metabot-nlq grants agent:content:read not :* (was
over-granting write to NLQ-only users); update-pulse-with-perm-checks! validates :cards against
CoercibleToCardRef (a bad string id was a 500, now a 400); check-mcp-ui-native-query!'s docstrings
now state it is dormant until the v2 session-rework slice wires it (was claiming to be live).

MINOR fixed: thrown-status test helper distinguishes no-throw from throw-without-status; kill-switch
check moved after the scope check (a scope-denied caller no longer learns the kill-switch state);
the query-handle-ttl default is one const, not two literals; mp_test gains positive assertions.

Re-verified (fresh JVM, run separately): query-guards + scope + settings + mp 328 assertions;
oauth + v1 mcp.api-test green. The one red in pulse-test is create-csv-xls-test = GDGT-3161, a
pre-existing EE-app-db bug (empty {} dataset_query -> download-perms 500) in the CREATE path (not
this fix's update path), reproducing on plain master — not introduced here. kondo 0/0; cljfmt clean;
modules-config regenerated (oauth-server :model-imports gains :model/User).

--- master parity sync (user-approved Option A) ---
Carries master 8c3ea6f40b (#80554 Remove Explorations dimension anchored blocks) + master search/appdb/index.clj (isolate-write! savepoint isolation). Stack forked before both; without them the dropped exploration_block.type migration is absent -> Liquibase changelog-integrity failure on real app-DBs (App-DB Part2 CI). Synced at base to restore parity. 41 files, all slice-untouched.

* Fix save-access-token! call arity after master merge

Master gave the test helper a 5th arg (client-id via
oauth-server.tu/with-oauth-client); the new
resolve-access-token-deactivated-user-test still called the 4-arg form.
Wrap it in with-oauth-client and pass client-id, matching every other
call site. Verified: full session-oauth-test namespace green locally
(10 tests, 27 assertions, 0 failures).

* Fix native-guard snake_case gap and make the tenant recipient filter work end-to-end

- native-map-edges gains :source_query/"source_query": legacy MBQL
  normalization canonicalizes the snake_case spelling too, so a payload
  like {"query" {"source_query" {"native" ...}}} reached the query
  processor as a native stage while native-query? said false — an
  MBQL-only-scoped caller could run raw SQL through /v1/execute,
  bypassing the scope split and the execute-sql kill switch. Regression
  tests cover both the string and keyword snake forms.

- The cross-tenant recipient filter compared :tenant_id on recipient
  maps that the :recipients hydration never populates, so a tenant
  caller lost every user recipient and a tenantless caller was filtered
  by nil = nil. Tenant membership is now looked up from the app DB
  inside the filter, and the filter only engages for a caller who
  belongs to a tenant — a tenantless user sees recipients unfiltered,
  as before tenants existed. The test now builds real users in real
  tenants (production-shaped recipient maps, ids only) instead of
  fabricating :tenant_id keys, and pins the tenantless-caller
  passthrough.

* Match native-SQL markers case- and separator-insensitively

`native-query?` matched edge names and markers case-exactly, but the QP
normalizer canonicalizes keys case-insensitively and treats `_` and `-`
alike. `{:query {:SOURCE_QUERY {:native "select 1"}}}` therefore scanned
as not-native and then executed as a native stage — REPL-confirmed, along
with `:Source_Query` and an uppercased `:NATIVE` marker.

That is a live bypass rather than a latent one: since
`+refuse-unscoped-native-sql` went live on /api/dataset, this scan is the
only gate between an unrestricted-stamped MCP UI credential and raw SQL.
The same scan backs `reject-native-query!` on the agent-api execution
paths, so those inherit the fix.

All key and value matching now goes through one `token` fn — namespace
kept, lowercased, `_` folded to `-`. Nodes are walked as `[token value]`
pairs rather than by fixed lookup, so a node carrying two spellings of
one edge has both scanned instead of one silently shadowing the other.

* Deep-scan a non-map stage element instead of throwing

`native-query?`'s `scan-map` iterated the node's entries without checking it was a map, so a
junk payload like `{:stages [1]}` threw "Don't know how to create ISeq from Integer" and
`/v2/query` answered 500 where shape validation should have answered 400
(`combined-query-rejects-malformed-payload-test`). A non-map where a stage belongs is now
deep-scanned like every other malformed sub-tree: never throws, and a marker after it still
trips the guard.

* Address round-3 review findings on the extraction slice

- The bearer resolver's is_active gate is defense in depth, not the primary control: deactivating
  through the model already revokes the user's tokens. The docstring now says so, and the test
  deactivates with a raw core_user UPDATE (the path the gate exists for) and asserts the token was
  not revoked, so the gate is what the test exercises.
- Pin that a tenant-scoped caller sees only same-tenant pulse recipients (tenantless users hidden).
- scope.clj comments described a wildcard collapse that is not in this diff; say what the slice
  does: adds scopes, removes none.
- Correct the guard test docstring: the /api/dataset gate is wired by the next slice.
- Pin the guard's deep-scan fallbacks and the streaming response's :executor option with tests.

* Widen the recipient merge-back to cover the tenant filter

Addresses review feedback on the tenant recipient filter: a filtered read
that is submitted back as a write silently deleted the recipients the
caller was never shown.

The `:channels` submitted to an update are authoritative --
`update-notification-channels!` deletes any recipient not present -- so
every recipient hidden on the read path has to be restored on the write
path. Master already did this for one read filter (sandboxed and
impersonated users, via `maybe-add-recipients`), but the tenant filter
added in this slice had no corresponding write-path compensation. A
tenant-scoped caller doing GET then PUT of the returned body therefore
dropped every other-tenant and tenantless recipient row.

Rather than add a second parallel mechanism, the tenant visibility rule
is now a single predicate (`recipient-visible-to-tenant-caller?`) used by
both sides: the read path filters with it, and the write path merges back
exactly what it excluded via `hidden-cross-tenant-recipients`.

Two supporting fixes:

- `maybe-add-recipients` now normalizes `:channel_type` the way
  `email-channel` does. The old `(= "email" ...)` string comparison only
  matched the REST shape, so a direct call with a hydrated pulse (keyword
  `:email`) skipped the merge entirely -- which also made the pre-existing
  sandbox protection REST-only.
- Merged recipients are de-duplicated by `(some-fn :id :email)`, the same
  key `update-pulse-with-perm-checks!` uses to diff recipients, so a
  caller matching both filters cannot get a recipient appended twice.

Verified in a REPL: the new endpoint-level round-trip test goes red
without the merge-back (two recipient rows destroyed) and green with it.
It asserts on `pulse_channel_recipient` rows rather than the API response,
since the response is filtered either way and cannot distinguish "hidden"
from "deleted". Pulse, pulse-channel and enterprise sandbox pulse suites
show no new failures (`create-csv-xls-test` fails identically on the
unmodified branch).

---------

Co-authored-by: Eric Normand <ericwnormand@gmail.com>
2026-09-02 23:18:07 +00:00
technomancy
9d216767d0 Remove be-tests-databricks-multi-catalog-ee job. (#81127)
In https://github.com/metabase/metabase/pull/80957 we trimmed this job
down so it no longer doubles the tests run. But it's still pointless
to have it as a separate job; it should be tested inline with the main
suite.

We still run some individual tests twice using the
`with-and-without-multi-level` macro when we need to cover both the
multi-catalog behavior and the non-multi-catalog behavior, but this is
only a handful of tests now instead of the entire suite or the entire
namespace.

Removal of `when-not (tx/db-test-env-var :databricks :multi-level-schema false)`
causes a lot of indentation changes here, so it's best to review with
whitespace diffs turned off.
2026-09-02 22:35:39 +00:00
Alexander Polyankin
a9a062731b Store setting values encrypted under the setting's name (#81751) 2026-09-02 17:26:10 -04:00
github-automation-metabase
a7090f30ba Auto-update documentation for master (#81820)
Update generated docs (master)

Co-authored-by: github-actions <github-actions@github.com>
2026-09-02 21:23:08 +00:00
Andrei Chernykh
8a7375b368 Split the SSO auth settings pages into card sections (#81750)
* Split the SSO auth settings pages into card sections

* Deflake modal escape close in data-model e2e specs

* Move docs changes to a separate PR

* Finish token-scale translation in the merged css modules

* Deflake dark mode toggle assertions in palette spec
2026-09-02 13:44:47 -07:00
Nathan Voxland
a55b6b6b8a Better handling of unparsable user settings (#78737) 2026-09-02 15:15:33 -05:00
Nathan Voxland
58819b5ea2 Pivot table download should respect currency formatting (#75969) 2026-09-02 14:23:06 -05:00
Tyler Plude
0c06e94821 [BOT-1838] Release JDBC savepoints when nested transactions OK (#77562)
* release JDBC savepoints when nested transactions succeed

* tighten inline comment

* pr feedback

* Release savepoints only when no later scope is still running
2026-09-02 12:09:44 -07:00
Andrei Simionescu
e34e6190c6 BOT-1927: gate audit-view chart queries behind a Run query button (#81215)
* BOT-1927: gate audit-view chart queries behind a Run query button

* Track the Run query click with useDisclosure instead of useState
2026-09-02 18:00:34 +00:00
github-automation-metabase
eeba22c5fe Tighten ratchets (#81805)
Co-authored-by: github-actions <github-actions@github.com>
2026-09-02 17:57:02 +00:00
Anna Tomka
a06e24c8bd GDGT-3104: Fix unreadable active tab count badge in dark theme (#81800)
Fix unreadable active tab count badge in dark theme (GDGT-3104)
2026-09-02 19:46:00 +02:00
Riley Thompson
af1c635d4e Fix sqlglot parse failure for ?::type placeholder casts (#81381)
* Fix sqlglot parse failure for ?::type placeholder casts (#81373)

With connection impersonation enabled, native queries are parsed by
sqlglot after template-variable substitution, when {{var}} has already
become a JDBC `?` placeholder. sqlglot's base tokenizer treats `?::` as
a single token (DuckDB's try-cast operator) in every dialect, and only
DuckDB's parser can consume it, so `{{var}}::date` -- a very common
pattern on Postgres/Redshift/Snowflake -- failed with "Unable to parse
native query".

Fix: before parsing in is_single_stmt_of_type, rewrite `?::` to `? ::`
(which sqlglot parses fine, emitting CAST(? AS type)). Occurrences are
located via the tokenizer's own QDCOLON tokens, so `?::` inside string
literals, dollar-quoted strings, and comments is left untouched, and
placeholder order is preserved in the re-emitted SQL. Multi-statement
rejection and the rest of the impersonation validation are unchanged.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XkBejATyUcaq2R74xFd5M1

* comments and tests

* Assert all placeholders survive; fix QDCOLON dialect guard (databricks, not duckdb)

Review feedback: the placeholder-cast test now asserts that every ? in the
input survives into the re-emitted SQL, not just that one does.

While adding duckdb coverage, it turned out the dialect guard in
_split_placeholder_casts named the wrong dialect: in sqlglot 28.6.0 the
?:: (QDCOLON) token is emitted by the *base* tokenizer but only
Databricks' parser consumes it (the expr?::type try-cast column
operator); duckdb's parser does not, so duckdb was still failing on
placeholder casts. The guard now skips databricks instead, duckdb gets
the same ?:: -> ? :: rewrite as everyone else, and tests cover both:
duckdb placeholder casts parse with placeholders preserved, and
databricks' native try-cast is left intact (re-emitted as TRY_CAST).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* clean up

---------

Co-authored-by: Claude <noreply@anthropic.com>
2026-09-02 13:42:38 -04:00
Chris Truter
30f9f25ba9 Refine kondo ratchet commands and prompts (#81776)
* Make ratchet shrinking explicit, and check it in one place

The ratchet test fixture ran `fix!` on every non-CI run, so `project-tests
ratchets` rewrote .clj-kondo/ratchets.edn in your working tree before
asserting against it. A check that edits what it checks leaves a diff nobody
asked for, in the file every PR conflicts over, and its only reachable
failure was a bug in the fixer that had just run.

Removing the auto-fix forces the dev/CI split to go too: a strict local test
that no longer tightens would fail on every branch that removes an ignore.
So over-budget is now the rule everywhere, and shrinking is only
`./bin/mage kondo-ratchets-shrink`, which the shrink workflow runs on master.

That left two implementations of nearly the same check. The babashka
`check-report` gains the justification finding — the one assertion the JVM
suite had that it lacked — so `./bin/mage kondo-ratchets` covers over-budget,
config over-budget, unjustified ignores, normalization and unknown linters on
its own. CI runs it instead of booting a JVM, and the five now-duplicated
in-repo assertions leave the test namespace, which drops from 6.2s to 0.8s
because it no longer walks the source tree.

Task names follow: check-kondo-ratchets (which nothing called) becomes
kondo-ratchets, fix-kondo-ratchets becomes kondo-ratchets-shrink, and the
merge task becomes private -kondo-ratchets-merge, since only the git merge
driver invokes it. bin/merge-kondo-ratchets keeps its name; installed merge
drivers point at it.

* Polish kondo ratchet guidance

* Prime the ratchet shrink workflow

Intentionally raise the :case-symbol-test budget from 1 to 2 without adding a suppression. The read-only check allows the unused budget; after this PR merges, the shrink workflow should restore it to 1 through a Tighten ratchets automation PR.

* Make the ratchet suite check the project

Keep the ratchet tooling unit tests in the backend suite, but make project-tests ratchets run the read-only repository check. Include it in the bare project-tests command so local project-level checks still validate the checked-in ratchet policy.

* List project-test suites once

* Address review findings on the ratchet check

check-report threw NPE on a ratchets map without :comment-exempt, while
tolerating a missing :ignore-counts or :config-counts. Default it, so the
three policy keys behave alike.

The run-project-ratchet-checks description and the step comment both still
described the old wiring. The check now runs alongside the backend checks,
which no longer include it, and "no JVM" is not quite true: resolving
clj-kondo on a cold cache shells out to the JDK prepare-backend installs.

* Warn about stale comment exemptions

The only check that a :comment-exempt entry is still needed was a local-only
test, which went vacuous when the dev and CI semantics were unified, so
nothing named a stale exemption any more. The check now reports one the way
it reports an :unlimited policy nobody uses: informational, never a failure,
since only the shrink workflow removes them.

* Say why the merge driver finds mage through its own path

The script runs against whatever repository holds the conflict, which is not
always the one it lives in -- the tests merge stages in a throwaway repo. A
relative bin/mage looks plausible and fails there, so record the reason.

* Stop wrapping two comments that did not need it

One fits on a single line once it says less. The other breaks at its sentence
boundary rather than mid-clause.

* Break the ratchets header on its sentences

Four sentences started mid-line and wrapped mid-clause, one of them trailing
"in the PR." onto a line of its own. Same words, one sentence per line.

* Keep comment exemptions until someone removes them

fix! dropped a :comment-exempt entry as soon as every ignore of its linter
carried a comment, or the last uncommented one was deleted -- so an exemption
could disappear as a side effect of unrelated work, and getting it back is a
change to defend in review.

An exemption records a decision about a linter, not a count, which is the
reason :unlimited policies are already kept at zero and only warned about.
Treat the two alike: the check and the shrinker name a stale exemption, and a
person deletes it.
2026-09-02 17:37:16 +00:00
Jakub Chodorowicz
89da67a9f3 Gdgt 3146 remap cypress maildev ports so they don't conflict with app maildev 2026-09-02 19:36:37 +02:00
Ben Grabow
ae10677d61 Use a single future for a batched feature check in tests (#81743)
Use a single future for a batched feature check when per-feature memoization is disabled
2026-09-02 11:31:48 -05:00
Ben Grabow
3bcaab2e3b Avoid has_any_column_privilege run on entire cluster (#81734)
* Avoid has_any_column_privilege run on entire cluster

* Ensure that privilege filter only lets through `true?` values
2026-09-02 11:29:54 -05:00
Nathan Voxland
2921ecdcf9 Fix image tile url tracking (#81371) 2026-09-02 11:23:22 -05:00
Kevin Kelleher
8819a72555 Prompt to reload after changing settings that update CSP (#80906)
* new domains settings page with reload on save

* review feedback

* better null safety
2026-09-02 15:05:07 +00:00
metamben
22e8ddba1f Return mongodb's collection in QP /native API response (#80918)
Fixes QUE2-822
2026-09-02 17:23:43 +03:00
metamben
8dbfef0fe6 Use the underlying nested field name (#81640)
Closes QUE2-834
Closes #81546
2026-09-02 17:22:55 +03:00
metamben
de7cedfeb6 Only set SQL Server instanceName when specified (#81595)
Closes QUE2-829
Closes 81270
2026-09-02 17:22:35 +03:00
Stas Gavrylov
5885d30ad0 GDGT-2486 Add elevation, radius, and spacing scale variables (#80481)
* GDGT-2486 Add elevation, radius, and spacing scale variables

* GDGT-2486 Remap scale tokens in files merged from master

* Address review

Review fixes for the GDGT-2486 scale migration:

- Revert double-shifted tokens from the merge-remap pass: it re-applied
  the old-to-new mapping to 22 files already remapped by the first pass,
  shifting spacing values one extra step (e.g. xl 24px -> xxl 32px).
- Fix codemod escapes where old key names kept their name but changed px:
  Modal default padding (lg->xl, 24px), MultiSelect radius (sm->xs, 6px),
  MetricSearchInput radius-sm, ThemeEditor PreviewPanel radius-md,
  ColumnInput theme.spacing.xs->xxs, ChartTypeSettings Space h props,
  TableRowForCacheableItem icon placeholder w (md->lg, 16px),
  ResizeHandle/ResizableBoxHandle grip h (xs->xxs, 4px).
- Pin Group/Stack default gap to "lg" (16px): Mantine's default is
  "md", which is 12px in the new scale (was 16px in the old scale).
- Allowlist the runtime-defined --mantine-spacing-xxs/-xxl and
  --mantine-radius-xxs CSS variables in find-never-defined-css-variables
  (fixes fe-lint CI).
- Add theme.unit.spec.ts covering the spacing/radius/shadow scales,
  dark-mode shadow set, whitelabel independence, and a guard asserting
  every scale-based component default resolves to a defined key.
- Derive Card/Paper story argTypes options from the real scale.
- Augment MantineThemeSizesOverride so the custom scale tokens appear in
  editor autocomplete for p/m/gap/radius/shadow props.
- Update the emotion-migrate and metabase-ui-component-from-figma skill
  token tables to the new scale (spacing/radius values, shadow tokens,
  snap tables).

* Update Loki Snapshots

* Address review: fix h/w style props missed by the scale codemod

The codemod did not remap height/width style props, so every bare h=/w=
scale-key value silently changed px with the new scale. Remap each to the
px-preserving key (xs 4px->xxs, md 16px->lg, lg 24px->xl, xl 32px->xxl):

- PaneHeader breadcrumb row h xl->xxl (32px, restores data studio
  pane header height)
- BrowseDatabases logo boxes h/w xl->xxl (32px)
- DatabaseConnectionStringField rows h lg->xl (24px)
- DatabaseEngineList logo Center h/w lg->xl (24px)
- Metabot chat chevron flexes h/w md->lg (16px)
- MetricsFilterPill h lg->xl (24px)
- DatasetNotebook divider h xs->xxs (4px)
- DataReference NodeList icon w md->lg (16px)
- transforms-python ResizableBoxHandle grip h xs->xxs (4px)

Audited: no remaining bare h/w/maw/mah/miw/mih old-key props whose px
changed (verified line-identical against master for every occurrence).

* Address review: fix bdrs and responsive-object size tokens missed by the codemod

Two more escape classes from the scale codemod, found by a full sweep of
every touched-token form (string style props, responsive objects, CSS
vars, theme accessors) with each line compared against master:

- bdrs style prop (border-radius): remapped px-preservingly at 69 sites
  in 58 files (xs 4px->xxs, sm 6px->xs, md 8px->sm). The codemod remapped
  radius= but not its style-prop alias bdrs.
- Responsive object values (e.g. p={{ base: "md", sm: "xl" }}): remapped
  the token VALUES at 9 sites in 6 files (base/sm/md breakpoint keys are
  untouched; only their spacing values shifted: md 16->lg, xl 32->xxl,
  xs 4->xxs).

Sweep result after fixes: zero remaining line-identical old-key token
references to spacing xs/md/lg/xl, radius xs/sm/md, or shadow xs/md in
any form. Intentional survivors unchanged: radius/shadow/bdrs "lg"
(disclosed broken-key-now-16px class), spacing/radius sm where the
px value did not change.

Verified: tsc, eslint (changed files), metabase/ui jest suite.

* Address review: fix remaining token escape forms (object props, braces props, style objects)

Final codemod escape classes from the exhaustive token sweep, all
remapped px-preservingly:

- Object-literal / default-parameter prop values (15 sites / 8 files):
  ItemList picker-item default containerProps pb xs->xxs (4px, the
  reported gap), FormFooter pt lg->xl, SlackSetup pt lg->xl,
  EmbeddingSecretKeyWidget mb xs->xxs, CreateDashboardForm labelProps
  mb xs->xxs x2, databases styles mb lg->xl, NotebookDataPicker
  menuDropdownProps mt xl->xxl, Banner story py md->lg
- Braces-wrapped string props: CloudSMTPConnectionCard p lg->xl (24px)
  and gap md->lg (16px), SettingsLdapForm mt xs->xxs,
  DashboardLeaveConfirmationModal mb lg->xl
- style-object CSS properties: DatasetNotebook borderRadius xs->xxs
- Ternary token values inside responsive objects: SummarizeStep gap
  md->lg (16px)

Sweep now returns zero real misses across every checked form: string
style props (incl. h/w/bdrs), braces props, responsive objects with
ternaries, object-literal/default-param props, style-object CSS props,
CSS var() references, theme.* accessors. Remaining grep hits are
untouched scales (lh, fz, size, breakpoints) or px-identical sm values.

* Address review: preserve Mantine spacing defaults exposed by Loki

* Update Loki Snapshots

* Address review: fix remaining scale regressions and visual guards

* Address review: make Loki update capture new references

* Update Loki Snapshots

* Address review: keep only targeted Loki references

* Address review: assign ticket component elevations

* Update Loki Snapshots

* Address review: tighten MonthPicker header spacing

* Address review: refine elevation visual stories

* Update Loki Snapshots

* Address review: keep renamed Menu Loki story selected

* Update Loki Snapshots

* Address review: preserve inherited Mantine defaults

* Address review: allow stock Mantine shadow tokens

* Address review: replace token audit test with ESLint rule

* Address review: drop unrelated whitespace-only dev/src changes

* Address review: use rem() in radius scale and rem docs

* Update Loki Snapshots

* GDGT-2486: Address PR comments

* GDGT-2486: Adjust negative tokens

* GDGT-2486: Address review findings

* Update Loki Snapshots

---------

Co-authored-by: Metabase Automation <github-automation@metabase.com>
2026-09-02 17:21:33 +03:00
Nicola Mometto
0b4bbf0427 Use a generic error message for the parameter-permission 403 (#81762) 2026-09-02 15:20:45 +01:00
lbrdnk
74e847104b Clean up the ai benchmark workflow (#80532)
* Clean up the ai-benchmark.yml

* Address review
2026-09-02 15:24:31 +02:00
Sébastien
bf5778bc50 Fix flaky embed parameters e2e by pre-accepting embedding terms (#81322)
Pre-accept embedding terms in embed parameters e2e setup
2026-09-02 14:53:32 +02:00
Kevin Kelleher
386fe3b12a Fix small Explorations UI issues (#81730)
fix comment badge line height, background color overflowing rounded borders
2026-09-02 08:37:41 -04:00
Andrei Simionescu
dc3f2134cb Move the Metabot model capability checks into the provider catalog (#81679)
* Move the Metabot model capability checks into the provider catalog
* Move the fast-mode proxy check into the Claude adapter
2026-09-02 12:10:51 +01:00
Jakub Chodorowicz
30be3958a2 Fix double-coercion in DOM distortions (#81672) 2026-09-02 11:18:06 +02:00
Andrei Simionescu
58ad1ba564 BOT-1033: Add box plot support to the metabot chart tools (#81198)
* BOT-1033: Add box plot support to the metabot chart tools

* BOT-1033: Narrow the chart-types docstring to the enums that derive from it
2026-09-02 10:03:13 +01:00
Sébastien
0da75ad4a3 Deflake the SQL default-parameters embed wizard test (#81759)
Wait for the preview query before asserting embed parameter results
2026-09-02 10:56:38 +02:00
Bartosz Rybacki
b3365881d0 Add brybacki to team and slack map (#81702) 2026-09-02 10:26:14 +02:00
Anna Tomka
663a04c231 chore: add annatomka as team member (#81713) 2026-09-02 10:03:21 +02:00
Kamil Mielnik
fc3927c030 Add core-frontend-gadget as codeowners of custom viz (#81333) 2026-09-02 14:27:49 +07:00
Sébastien
7a20107f87 Prefer the X-Metabase-Session header over the session cookie (#81165)
* Prefer the X-Metabase-Session header over the session cookie

* Don't fall back to the session cookie when the header is stale

* comment
2026-09-02 06:41:28 +02:00
Romeo Van Snick
577f242c41 Move the entities store into the metadata module and hide normalizr (#81290)
* Move the entities store into the metadata module and hide normalizr

* Record the moved schema file's import-time effect

* Describe what the metadata reducers do, not what they replaced

* Move the mirror's fixtures and slice-name helpers into the module (#81297)

* Move the mirror's fixtures and slice-name helpers into the module

* Build the QuestionSources fixture through the metadata store barrel

* Document why the metadata store is not a deletable mirror slice

* Trim the barrel and write-door comments
2026-09-02 10:46:02 +10:00
Nathan Voxland
de202b144a Create personal colection on user create (#78714) 2026-09-01 17:35:56 -05:00
dpsutton
9bf3b136af only derive site-url from authenticated superuser requests (#81130)
* only derive site-url from authenticated superuser requests

`maybe-set-site-url*` (global Ring middleware) derived `site-url` from request
headers whenever `(or (not (setup/has-user-setup)) superuser?)`. The left arm
needed no authenticated caller, so an unauthenticated request reaching the
instance before its first user was created could pin `site-url` from a forged
Host/Origin/X-Forwarded-Host header. Because the write is first-writer-wins and
setup never overwrites it, that attacker-chosen value survived setup and
poisoned every later auth/email link (notably password-reset URLs) for the life
of the instance.

Gate derivation on `superuser?` alone. `:is-superuser?` is already on the request
by the time this middleware runs (wrap-current-user-info is inner/earlier in the
onion), so an admin's own request still auto-derives `site-url` -- which happens
on the operator's next request after an interactive setup. An unauthenticated
pre-setup request now writes nothing; `site-url` simply stays unset (nil is
handled everywhere it's consumed, and a reset link built on nil is relative,
never attacker-controlled).

Deployments that provision headlessly, run multi-tenant, or never sign in as an
admin will otherwise sit at a nil site-url, so note in the setting's docs that
they should set MB_SITE_URL explicitly.

* Read Quartz job-data blobs through a class allow-list

Quartz persists job data, calendars, and BLOB triggers as serialized objects.
Metabase's job data is only ever EDN-shaped data (keywords, strings, numbers,
maps/sets/vectors) wrapped in Quartz's own JobDataMap/trigger classes, so when we
read it back we constrain deserialization to exactly those classes and decline
anything else.

Install a per-DB DriverDelegate (StdJDBCDelegate for H2/MySQL, PostgreSQLDelegate
for Postgres) that overrides getObjectFromBlob to read through a class allow-list
built from the plain-data classes job data consists of. This is scoped to
Quartz's own blob reads, so nothing else in the JVM is affected.

- metabase.util.deserialization-allowlist: a reusable, composable ObjectInputFilter
  allow-list. clojure-data-prefixes covers the EDN value space; callers add their
  own wrapper prefixes and read via read-object / read-object-bytes.
- metabase.task.secure-delegate: the allow-set (EDN + org.quartz.*), the blob-read
  bodies, and install!, wired into task.bootstrap. The gen-class delegates are
  AOT-compiled in the jar; from source they're compiled on demand so the class is
  loadable by name through the classloader Quartz uses. That runtime compile is
  fenced to source only (config/jar?): from a jar a missing class is a build error.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BjQcjQx7NPGr3Rpf6QaG4b

* Don't deserialize Java objects from H2 query results

Metabase reads scalar and collection values out of query results, never
serialized Java objects. Register a JavaObjectSerializer on the H2 driver so H2's
JAVA_OBJECT (de)serialization is declined rather than reconstructing arbitrary
classes from stored bytes. With that in place the ad hoc JAVA_OBJECT check in
read-column-thunk is redundant, so it's removed; read-column-thunk keeps its CLOB
handling. Ordinary values are unaffected.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BjQcjQx7NPGr3Rpf6QaG4b

* Strip internal keys from binning maps

Add :decode/api and :encode/serialize remove-internal-keys hooks to the
::binning schema so clients can't smuggle internal query-processor keys in
through a :binning map.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* Strip internal keys from more client-submitted query maps

Extend the :decode/api / :encode/serialize remove-internal-keys hooks to the
::parameter, ::template-tag, ::constraints, ::middleware-options, and ::page
schemas so clients can't smuggle internal query-processor keys in through them.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* Reformat strip-internal-keys-test doseq bindings

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* test improvements and fix reflection warnings

* Validate HoneySQL idents at the app-DB compile step

Honey SQL renders an ident as bare, unquoted SQL in several positions --
a form head it doesn't recognize, an ORDER BY direction, a DDL modifier,
the function name of a `:%fn.arg` var -- so an ident reaching any of them
is raw SQL text. Hold every ident to a plain-identifier shape, with an
explicit allow-list for the keywords Honey SQL renders as fixed SQL of its
own (operators, DDL modifiers, column types, ORDER BY directions).

Also close two gaps found along the way: symbols walked past the `:raw` and
`:inline` gates, since Honey SQL resolves symbol heads and clause keys the
same as keywords; and `safe-value?` trusted any value shape it didn't
recognize, where it now enumerates the types bound as `?` parameters.

Query idents that were kebab-case are renamed in the SQL only -- the
Clojure keys they feed, including the `:metabase-user-id` request key, are
mapped back at the read site and unchanged for their consumers.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013rQMWmi2o1WGSqK5DDZ99u

* Allow the Postgres operators our drivers register globally

`metabase.driver.postgres` registers the PG JSON, array and regex operators
with `honey.sql/register-op!`, which makes them global -- and the app DB can
be Postgres, so app-DB queries reach for them too: the semantic-search
migration compiles a `[:->> ...]`. Add them, and a test that holds the
allow-list to every non-namespaced operator Honey SQL knows, so an upgrade
or a new registration can't silently leave one out.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013rQMWmi2o1WGSqK5DDZ99u

* Snake_case the remaining app-DB query idents

Four more sites the sweeps turned up, all renamed in the SQL only:

- `request/session.clj` selects the current user under the same kebab
  aliases the session middleware did, so it gets the same rename-at-the-read
  -site treatment.
- Search scorer names double as SQL aliases, and three of them are
  kebab-case. They also key the weights in `search.config` and name the
  scores the API returns, so the conversion happens at the one seam that
  writes and reads the alias, leaving the scorer name alone. The alias gains
  a `_score` suffix because `:view-count` and `:data-layer` are named after
  the very columns they read.
- The model alias registry in `api/common.clj` feeds
  `column-with-model-alias`, which assembles `alias.column` idents, so its
  six kebab aliases are snake_cased along with the literal uses in the
  in-place search engine.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013rQMWmi2o1WGSqK5DDZ99u

* Constrain embedded card parameter values by the locked parameters

A dashboard's parameter value lookups receive the values of the other
parameters as chain-filter constraints, so the values offered for an
enabled parameter are limited to the rows the locked ones select. The card
lookups had no constraint argument to receive them: `card-param-values`
bound the JWT's params, used them only to check the searched parameter was
enabled and not pinned, then resolved the parameter to a Field and returned
that Field's whole value list.

Give the card lookups the same constraint argument and pass the JWT's
locked values from the embed layer. The argument is optional, so the
authenticated and public routes keep their current behaviour.

`param-type->op` and `param-type->default-options` move from
`parameters.dashboard` to `parameters.params`, so both the dashboard and
card constraint builders can reach them.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013rQMWmi2o1WGSqK5DDZ99u

* Snake_case the app-DB query idents the guard rejects

CI caught `queries.models.query`, and a grep for the same shape found four
more sites the earlier sweeps had missed -- they were confined to the
app_db, collections, search and permissions modules. The aliases move to
snake_case in the SQL; where the keys they feed are confined to their own
namespace the readers move with them, and where they have to match a shape
from elsewhere they are renamed back at the read site:

- `queries.models.query` aliases so the app-DB branch matches the keys
  `lib.metadata/card` returns, so it renames rather than propagates.
- `cache.task.refresh-cache-configs` feeds its aliases through a nested
  subquery whose outer `:select` names them, so both levels move, along
  with the readers and the test's expectations.

Also widen `safe-parameter?`: a `query_hash` read back from H2 arrives as a
`java.sql.Blob` rather than a byte array, and the enumeration rejected it.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013rQMWmi2o1WGSqK5DDZ99u

* Remove a scratch probe test

Committed by accident in the previous commit -- a throwaway namespace with
printlns, used while checking chain-filter behaviour.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013rQMWmi2o1WGSqK5DDZ99u

* Fix the locked-parameter constraints for previews and stale slugs

Three problems with the constraints the card parameter lookups build:

The preview routes passed the sanitized Card, whose `:parameters` has the
locked ones removed, so the constraints were always empty and a preview
offered wider value lists than the embed it previews. They now pass the
stored Card and take only `:embedding_params` from the token, which is what
the published routes already do.

`embedding_params` is keyed by slug and `:parameters` by id, and nothing
keeps them in sync; a locked slug naming no parameter produced no
constraint and widened the values rather than narrowing them. It is now
refused. `param-values-merged-params` refuses the same on dashboards, where
the slug otherwise became a nil key that the constraint builder dropped.

A locked value of `[]` reached the filter builder as a value rather than as
"no value", producing a `:=` clause with one argument.

The constraints also carry the parameter's own `:options` now, as the
dashboard builder does.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013rQMWmi2o1WGSqK5DDZ99u

* Allow a qualified argument in a :%fn.arg ident

`:%lower.metabase_field/name` renders `LOWER("metabase_field"."name")` --
Honey SQL splits the ident at the first `.` and formats what follows as
quoted entities, so a `/` there separates the parts of a qualified column
rather than landing in the SQL. Only the function name, which is rendered
unquoted, has to be a bare identifier.

The pattern rejected `/` anywhere after the `%`, which took the database
autocomplete endpoint with it.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013rQMWmi2o1WGSqK5DDZ99u

* Snake_case more app-DB query idents, and allow the FOR UPDATE modifiers

Found by widening the search: the earlier grep looked for the alias and its
`:select` on one line, so it missed every alias sitting on a continuation
line -- which is most of them.

- `activity_feed`: the recent-views queries alias `database-name`,
  `initial-sync-status`, `entity-coll-id` and `moderated-status`. The keys
  are read only within their own namespace, which builds the API response
  from them under different names, so the readers move too.
- `xrays.automagic-dashboards`: `num-fields` and `list-like?` -- the latter
  loses its `?`, which no snake_case spelling keeps.
- `:skip-locked`, `:nowait` and `:no-wait` are lock modifiers Honey SQL
  matches by name in a `:for` clause, not idents it renders, so they join
  the allow-list rather than being rewritten.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013rQMWmi2o1WGSqK5DDZ99u

* Move the query-analyzer's readers to the renamed table id

`query-analyzer/tables-for-native` aliases its table id, and that alias was
snake_cased without moving one of its two readers: `used-tables` still asked
for `:table-id` and so silently collected nothing. Unlike the write side,
which the guard rejects outright, a reader left behind reads nil.

The `tables-for-native` redefs in the tests carried the old shape too, so
they asserted against a result the function no longer produces.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013rQMWmi2o1WGSqK5DDZ99u

* Revert the HoneySQL ident guard and the renames it required

The guard held every ident in an app-DB query to a plain-identifier shape,
which turned out to have a long tail: each release of it found more query
aliases to rename, and a rename that missed a reader failed silently rather
than loudly, since the reader simply got nil.

This reverts the guard along with every alias rename made to satisfy it,
leaving the parameter-value security fixes, which do not depend on it.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013rQMWmi2o1WGSqK5DDZ99u

* Harden site-url derivation fix: fail-closed redirect check, discerning test

Follow-ups to gating site-url auto-derivation on authenticated superuser
requests.

- check-sso-redirect: fail closed when site-url is nil. Previously a nil
  site-url left our-host nil, so a redirect with a nil host but a non-nil
  scheme was non-relative yet matched (= nil nil) and passed -- an open
  redirect. Require a known host to match against: (and
  our-host (= (.getHost redirect) our-host)).

- misc-test: bind has-user-setup false in the non-superuser negative case.
  With has-user-setup true (the test-DB default) the old gate
  (or (not has-user-setup) superuser?) was already false for that request,
  so the test passed against the vulnerable code and distinguished nothing.

- misc: rewrite the SETTING SITE-URL block comment and the maybe-set-site-url
  docstring, which still described the removed behavior (site-url set on the
  initial setup request).

- system settings: drop the blank line in the site-url :doc (it made the new
  paragraph render as an indented code block in the generated env-var docs)
  and replace an em-dash with a comma.

* Read a card's parameters the way the rest of the code does

`card-param-constraints` read `:parameters` alone, but a native card can be
described entirely by its template tags and carry none -- which is how the
static embedding e2e fixture builds one. The locked slug then matched no
parameter and the lookup was refused. Use the same either/or source
`get-param-or-throw` uses a few lines above.

A parameter with no value constrains nothing whether or not it still
exists, so the existence check now runs after the empty-value guard rather
than before it.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013rQMWmi2o1WGSqK5DDZ99u

* Fix jar startup crash when the jar's directory contains a space

find-in-current-jar built a jar: URL by string-concatenating
get-jar-path, but get-jar-path returns the *decoded* filesystem path
(URL -> URI -> .getPath turns %20 back into a literal space). URL.toURI
then throws URISyntaxException on the raw space, killing startup while
loading bundled driver manifests.

Open the jar via the Path-based nio-fs helper instead — Paths don't
round-trip through URI encoding, so spaces are fine. This also stops
closing a filesystem out from under other consumers: the old code, on
FileSystemAlreadyExistsException, grabbed the provider's *cached*
filesystem and closed it in with-open; the Path-based overload creates
an independent uncached instance whose close affects only itself.

Verified against a jar in a spaced directory: the manifest URI is found
and slurpable, and a concurrently-open URI-cached filesystem for the
same jar stays open and registered.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013Fpja4n6bzdCy4QufM49Hb

---------

Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
Co-authored-by: Alexander Polyankin <alexander.polyankin@metabase.com>
2026-09-01 17:14:27 -05:00
Denis Berezin
84223a0d4d Restyle Number and Trend dashboard cards (UXW-5128) (#81131)
* Restyle Number and Trend dashboard cards (UXW-5128)

* Adjust sizes and tooltips

* Adjust sizes, fix icon

* fix

* Fixes after merge

* Update Loki Snapshots

* Fix multiple comparisons

* Add link to card title, Implement query builder view

* Undo scalar dashcard ellipsis menu styling changes

* Design review fixes

* Fix e2e

* Fix e2e

* Review fixes

* Update Loki Snapshots

* fix

* Self review fixes

* Self review fixes

* Add a "Show comparison value" display setting to the Trend visualization (#81148)

* Add Show comparison value setting to the Trend visualization

* Address review comments

* Standardize ellipsis menu styling across all dashboard cards (#81405)

* Standardize dashboard card ellipsis menus

* Update Loki Snapshots

* Update Loki Snapshots

* Refine dashcard menu button props

---------

Co-authored-by: Metabase Automation <github-automation@metabase.com>

* Fix review comments

* Update Loki Snapshots

* Add more tests

* Add keyboard navigation

* Fix borders for dark theme

* Update Loki Snapshots

---------

Co-authored-by: Metabase Automation <github-automation@metabase.com>
Co-authored-by: Anderson <anderson@metabase.com>
Co-authored-by: Anderson <ands.alves.nunes@gmail.com>
2026-09-02 01:11:13 +03:00
Chris Truter
e24150083f Report dynamic requires which linter cannot verify (#78082)
* Don't crash the modules require hook on dynamic requires

(requiring-resolve (:failing-test-var opts)) has no statically-known
namespace; unwrap-require now returns nil for such forms instead of
falling through, and both callers skip non-symbols. dev.clj has carried
a live :hook crash from this since the hook was added.

* Add a hook regression test for dynamic requiring-resolve

Covers the crash fix: dynamic arguments lint to nothing, static
cross-module violations still report. hooks.* test namespaces join the
outside-the-module-system prefix list.

* Fail closed on dynamic module requires

* Polish module require hook tests

* Assert module hook findings directly

* Document intentional dynamic module requires

* Update module linter ratchet
2026-09-01 21:51:21 +00:00
github-automation-metabase
70f9234106 Auto-update documentation for master (#81742)
Update generated docs (master)

Co-authored-by: github-actions <github-actions@github.com>
2026-09-01 21:35:27 +00:00
Nathan Voxland
540817adfe Improve document read/write performance (#76018) 2026-09-01 16:19:58 -05:00
Chris Truter
7f73a7464c Add merge tool for ratchet conflicts (#81341)
* Add a manual kondo ratchet conflict merger

* Split kondo ratchet merge tests

* Document the ratchet conflict merger

* Simplify unlimited ratchet merge policy

* Simplify ratchet merger for final policy schema

* Resolve ratchet conflicts to the stricter policy

Concurrent policy changes now always resolve: the smaller budget wins, a
finite budget beats :unlimited, and a removed entry beats any budget.
Comment exemptions merge on their own. Every stage is read through
read-ratchets and shape-checked before a disabled stage decides the result.

bin/merge-kondo-ratchets reads the index stages by number, so a file
deleted on one side and changed on the other exits nonzero with nothing
staged, and both-sides-add merges against an empty base. It runs from any
directory in the repository and stages only .clj-kondo/ratchets.edn.

mage.merge-kondo-ratchets-test drives the script against real conflicts
in temporary repositories.

* Point the clojure-write skill at the ratchet merge resolver

* Reject malformed stages before merging ratchets

The shape checks move out of read-ratchets into validate-policies, and merge-ratchets runs them on every stage, so a stage with :config-counts [] or :comment-exempt [] is an error rather than an empty set of policies that the merge would read as removals.

* Cover unreadable stages in the merger tests

An empty file, two forms, or a non-map stage now fails the reader instead of merging as no policies; the script leaves such a conflict for a human and stages nothing.

* Reject a non-map ratchet stage before merging

A nil stage passed every field check because dissoc, keys, and the defaults all accept nil, so a malformed stage could merge as an empty set of policies.

* Reject non-keyword linter names

The merge keys on (str linter) and renders the same way, so a string, nil or
symbol name was rewritten as a keyword, collided with one, or produced a file
that no longer read back -- and the script still exited 0 saying it had staged
the merge. Also describe the two sides by git stage: "current" and "incoming"
are inverted during a rebase.

* Run each refused merge once in the tests

leaves-unresolved? and output each ran the script, so every refusal ran it twice
and the "changed nothing" assertion covered a different run from the one whose
message was checked. Read the merged policies with the EDN reader while here.

* Compare the conflicted file, not just its status

A conflicted path reports UU, DU or UD however its contents are rewritten, so
comparing git status alone did not show that a refused run left the file alone.

* Tighten merge-tool docstrings and the ratchet conflict docs

* Compare the index stages of a refused merge too

Porcelain status reports a conflicted path as DU/UU whatever its stages hold, so it alone does not show
that a refused run left the conflict alone. The stages carry each side's blob.

* Spell out the unmerged status codes

UU/DU/UD as bare shorthand assumes the reader knows git's porcelain codes for a conflicted path.

* Clarify the unmerged-state snapshot comment
2026-09-01 21:08:21 +00:00
Nathan Voxland
85f1e975a5 Ensure CSV strings show as strings in excel (#81258) 2026-09-01 15:50:56 -05:00
Chris Truter
bc0200ece0 Enforce and maintain ratchet policies (#81319)
* Add trusted master ratchet shrinker

* Retry ratchet shrink after concurrent master pushes

* Test the check-kondo-ratchets report

* Apply ratchet policy to the check report

* Add severity-based inline ratchet policies

* Update ratchet policy documentation

* Finalize ratchet policy enforcement

* Move seeded linters to bounded ratchets

* Simplify unlimited ratchet policies

* Finish ratchet CI integration

* Document unlimited ratchet policy values

* Keep ignore counts as the policy map

* Validate ratchet policy keys against known linters

Policy keys in .clj-kondo/ratchets.edn must name a clj-kondo built-in from the
pinned version, a linter configured under .clj-kondo/, or an allowlisted
external diagnostic. The check and the fixer fail with one error listing every
unknown name instead of counting it. Config budgets must be non-negative
integers.

Drop :unresolved-require, a linter that never existed, from the semantic-search
ignore and the ratchets file.

* Keep unlimited ratchet policies with no ignores left

An :unlimited entry in :ignore-counts records a decision about the linter, so
the fixer no longer drops it when its count reaches zero. The check and the
fixer print one sorted warning naming such entries; it never fails CI. Bounded
zero budgets are still dropped, and :comment-exempt is unchanged.

* Deliver ratchet shrinking through an automation PR

* Validate seeds and read only tracked kondo configs

The known-linter set now comes from git-tracked `.edn` files under
`.clj-kondo`, so the dependency configs `mage kondo` copies in locally
(gitignored, absent from a clean checkout) no longer make validation
depend on the machine. `--seed` is checked against the same set before
anything is written, and a hand-written bounded `0` with no ignores
left is reported when the fixer drops it.

* Reopen the ratchet automation PR after a close

The "already carries this result" skip now requires an open PR, so a
branch left behind by a PR closed without merging is overwritten and
gets a fresh PR instead of silencing every later run. The compare step
reads PR state with the default token under `pull-requests: read`.

* Let the ratchet automation PR approve and merge itself

Copies the backport recipe: github-actions[bot] approves the PR with the default token (a user cannot approve their own PR, and the automation user owns it), then the automation user enables squash auto-merge. The commit identity is the docs generator's, which merges with that one approval. A ratchets-only change no longer flips backend_all, so the PR waits on the Project tests job alone.

* Describe the self-merging ratchet automation PR

* Expect the ratchet check namespace in the project-test suites

* Read the ratchets file as exactly one map

An empty file, a non-map, or a trailing second form used to read as an empty policy set, which the merger would have staged as the removal of every budget. Tracked kondo configs missing from the worktree (unstaged deletions, sparse checkouts) are skipped instead of failing the scan.

* Skip the uberjar for a ratchets-only change

* Keep fork PRs away from the ratchet automation steps

The head-branch lookup matched any repository's branch of that name, so a fork PR could have reached the approval and auto-merge steps; only an open PR from this repository counts now. Approval and auto-merge key on the open PR rather than on whether the branch was pushed, so a run that failed after opening the PR is repaired by the next one.

* Look up the ratchet automation PR by owner-qualified head

gh pr list caps at 30 results and filtered forks client-side, so enough fork PRs on the same branch name could hide the real automation PR and make the create step fail. The REST head=owner:branch filter is applied server-side and matches only this repository's branch.

* Stop checkout credentials from overriding the automation push token

actions/checkout persists GITHUB_TOKEN as an Authorization header, which takes precedence over the token embedded in the push URL, so the force-push to the automation branch would be rejected.

* Tolerate config budget and exemption reductions in CI

CI already let inline counts fall below their budgets so the shrinker could record them after merge, but a lowered config budget or a stale comment exemption still needed an exact match, so two PRs that each removed a suppression could fail the second merge. Config counts now fail CI only when they grow, stale exemptions never fail CI, and check-kondo-ratchets reports config growth alongside inline growth. Local runs stay exact.

* Say what CI rejects without listing the cases

* Keep actionable ratchets bounded

* Exclude the ratchets file with one pattern

Path filters are OR-ed, so a separate "!.clj-kondo/ratchets.edn" entry matched
every path that was not that file: backend_all became true for every PR, and
no_source_code_changes could never be true. The exclusion has to live inside the
pattern, as modules/drivers does above.

* Arm auto-merge against a known commit

The approve and merge steps keyed on the PR number alone, so nothing had checked
what the fixed automation branch name currently holds. Both paths now record the
sha they pushed or compared, and auto-merge matches it. Also guard the job to
this repository, as the workflow it replaces did.

* Report ratchet validation failures without a stack dump

check read and validated the file, then read it again. Now it parses once and
catches inside, so read-ratchets, validate-linters! and scan all report the same
one-line message. Also: CI=false was truthy in the tests, seeding a recorded
linter with no ignores said "nothing to seed" while dropping its policy, and
lowered-counts rebuilt the seeded set per entry.

* Keep the drift message off stdout

The drifted path prints its findings and then exits; folding it into fail! added
a summary line that check-reports-empty-unlimited-test pins. Split the throw from
the print so only messages the run has not already shown are printed.

* Tighten ratchet docstrings and the generated header

* Guard against standalone negations in the path filters

The patterns in a filter are OR-ed, so a "!..." entry matches every path the other entries don't
instead of subtracting from them, and the filter is then always on. The ratchets exclusion is written
inside one pattern for that reason; this keeps the next one from being written the other way.

* Reuse the automation PR on its ratchet file alone

The extra check that master had not touched ratchets.edn since the branch was cut only bites when master
moved to a different value that shrinks back to the same result, since any change to the result is already
caught by comparing the file. Dropping it means only a shallow fetch of the branch tip is needed.

* Force-push the automation PR when it conflicts

A PR whose ratchet file already matches is otherwise left alone, and a conflict is the one thing that
state cannot clear on its own. Mergeability is computed lazily, so UNKNOWN is polled past and, if it
never resolves, left for the next run.

* Ask once whether the automation PR conflicts

Polling for a definitive answer bought nothing: the next push to master runs this again within the hour,
so a mergeability check still in flight can read as not conflicted and be settled then.

* Say what the reuse comparison is against

git diff FETCH_HEAD -- <path> reads as a commit-to-commit diff. It is against the working tree, which is
the only reason comparing the branch tip answers whether the PR still delivers the right file.

* Fix the circular Filter type alias

Record<string, Filter> is expanded eagerly, so naming it inside Filter's own definition trips TS2456. An
inline index signature is deferred and holds the same shape.

* Prove a ratchets-only change skips the backend filters

* Run the path-filter specs when the filters change

* Use gh CLI to approve ratchet PRs
2026-09-01 20:31:30 +00:00
Ben Grabow
b4f84f1736 Skip some connection validity checks in test (#81725) 2026-09-01 15:22:41 -05:00
Nathan Voxland
52ba78451d Added map-tile-server-allowed-networks setting (#81243) 2026-09-01 15:10:53 -05:00
Ben Grabow
8e4e18f6d5 Don't hit DWH server when no parameters are being set (#81727) 2026-09-01 14:57:28 -05:00
Chris Truter
3c5d1111d7 Run project checks in a separate job (#81318) 2026-09-01 19:37:54 +00:00
Jeff Bruemmer
af549487df docs - ai connections (#81617) 2026-09-01 19:32:12 +00:00
github-automation-metabase
9e2fa97aa1 Auto-update documentation for master (#81619)
Update generated docs (master)

Co-authored-by: github-actions <github-actions@github.com>
2026-09-01 15:24:53 -04:00
github-automation-metabase
1d598c7af4 Update translations 2026-09-01 (#81668)
* Update translation sources 2026-09-01

* Update translations

---------

Co-authored-by: github-actions <github-actions@github.com>
2026-09-01 13:22:09 -06:00
Edward Paget
b244f3c7ca Give Quartz a dedicated app-db connection pool to prevent deadlocks (#72261)
* Reuse toucan2's current connection in Quartz to prevent deadlocks

When code inside a with-transaction or with-connection block triggers a
Quartz operation, Quartz would check out a new connection from the pool.
If the pool is saturated, this deadlocks the thread. Now the
ConnectionProvider checks for a bound toucan2 *current-connectable* and
reuses it via a non-closeable proxy that suppresses close/commit/rollback/
setAutoCommit, preventing interference with the outer transaction.

Closes UXW-307

* Address review comments

verify that asking for a connection while one is already bound via
t2.conn/*current-connectable* reuses the identical underlying connection
instead of checking out a new one. Plus fix an `if` that should be a
`when`.

* Revert Quartz connection-reuse proxy

Revert the non-closeable connection proxy that let Quartz reuse the
thread's bound toucan2 connection with close/commit/rollback/setAutoCommit
suppressed. Suppressing those methods breaks Quartz's data integrity:
JobStoreTX only releases its SELECT ... FOR UPDATE row locks on QRTZ_LOCKS
via commit/rollback, so the proxy either held the cluster lock for the
length of the outer transaction or, for autoCommit connections, defeated
the cluster mutex entirely, and made partial job-store writes
un-rollbackable.

The deadlock this was fixing (UXW-307) is addressed in the next commit by
giving Quartz its own connection pool instead.

* Give Quartz a dedicated app-db connection pool to prevent deadlocks

When code inside a with-transaction or with-connection block triggers a
Quartz operation, Quartz checks out a connection from the app-db pool. If
the pool is saturated, this deadlocks the thread (UXW-307). Fix it at the
root: Quartz now draws from its own small c3p0 pool, so job-store
operations never compete with application code for main-pool connections,
and Quartz keeps full control of its connections' transaction lifecycle.

- ApplicationDB now carries a quartz-data-source: a separate pool (max 5,
  min 1, configurable via MB_QUARTZ_MAX_CONNECTION_POOL_SIZE) when
  create-pool? is true, the raw data source otherwise. It inherits all the
  main pool's properties except sizing.
- mdb.connection/quartz-data-source exposes it behind the ApplicationDB
  read-write lock, re-exported via metabase.app-db.core.
- The Quartz ConnectionProvider draws from it; the tracing
  connection-interceptor hook is unchanged.
- Tests cover provider behavior (no reuse of the bound connection, not even
  wrapped), pool construction/sizing, lock-respecting acquisition, and a
  deterministic pool-of-1 deadlock regression test.

* Reject already-pooled data-sources for the dedicated Quartz pool

connection-pool-data-source returns an already-pooled data-source
as-is, so building the Quartz pool from one would silently hand Quartz
the main pool itself -- reintroducing the starvation deadlock the
dedicated pool exists to prevent. No current caller does this (the only
production :create-pool? true site uses the always-unpooled env-derived
data-source), but nothing stopped a future caller from doing it.

Fail fast instead: quartz-connection-pool-data-source now throws on
pooled input, and application-db documents that :create-pool? true
requires an unpooled data-source.

---------

Co-authored-by: John Swanson <john.swanson@metabase.com>
2026-09-01 12:09:11 -07:00
Chris Truter
3fd1f63cfc Refresh settings cache before reading LLM providers (#81714)
Refresh the settings cache before reading LLM provider connections

Provider connections all live in the one llm-providers setting, and settings
are cached in process, so an instance only learns about another instance's
edit when its cache next polls — once a minute.

Until then it answers a page load with a list that predates the edit, and it
decides whether an edit is allowed against that same list: a connection
another instance has just deleted still looks present, and the type it
belonged to still reports itself as connected, so a legitimate create is
refused.

Force the staleness check on the five endpoints that read the list. It costs
the one query that reads settings-last-updated, unless something has actually
changed.

This does not make the read-modify-write of the list atomic: two instances
editing different connections at the same moment still lose one of the two
edits. That needs a lock, and is left for its own change.
2026-09-01 20:08:03 +02:00
Egor Grushin
2459fe20ba Carry over picked schedule fields when the frequency changes (#80758)
* Carry over picked schedule fields when frequency changes (GDGT-3096)

* Drop redundant mid-frame test

* Hook normalizeScheduleValue to changeScheduleType

* TDD first

* reviewed

* reviewed

* reviewed

* reviewed

* reviewed

* reviewed

* reviewed

* reviewed

* reviewed
2026-09-01 18:52:23 +01:00
Nicola Mometto
0fec254d6d Compile the LIKE ESCAPE character as h2x/literal so driver inlining can't touch it (#81704)
With a driver bound, [:inline "!"] compiles through driver-specific
inline-value; MySQL emits the hex literal _utf8mb4 X'21', whose implicit
collation (utf8mb4's default, utf8mb4_0900_ai_ci) clashes with the
connection collation when MySQL aggregates collations for LIKE, failing
with 'Illegal mix of collations'. ::h2x/literal compiles through its own
registered formatter to a plain '!' on every dialect, which takes the
connection collation like the pattern parameter next to it.
2026-09-01 17:34:00 +00:00
Andrei Simionescu
ebad52172a Fix AdminSettingInput write race (#80737)
* Don't lose the last change to a setting when a save fails

* Drop a duplicated test title

* Drop the no-op catch on the write chain

* Replace the write queue with a pessimistic disabled state

* Extend the failed-save revert to select and radio inputs
2026-09-01 18:29:21 +01:00
Chris Truter
d7a519ab60 Stop timed-out mage commands and their descendants (#81463) 2026-09-01 19:20:43 +02:00
Riley Thompson
0e0e6ac82b Enforce single-statement impersonated queries on all sql-jdbc drivers (#81653)
* Enforce single-statement impersonated queries on all sql-jdbc drivers (SEC-1189)

Connection impersonation confines a non-admin's native queries to a restricted
DB role, which Metabase activates with a `SET ROLE` prefix. To stop a user from
appending `; SET ROLE <broader>` and climbing out of that role, impersonated
native queries must be a single statement. That guard
(`driver/validate-impersonated-query`) was only overridden for `:postgres` and
`:sqlserver`; every other impersonation-capable JDBC driver (MySQL, Snowflake,
ClickHouse, Starburst) inherited the no-op `:default` pass-through and ran the
stacked query verbatim. On MySQL a stacked `; SET ROLE ALL` activates every role
granted to the connection user, escaping the restricted role entirely.

Define `validate-impersonated-query` once at the `:sql-jdbc` floor so every
impersonation-supporting driver enforces the single-statement validation, and
remove the now-redundant per-driver `:postgres`/`:sqlserver` overrides.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XM9HwBRFsa3ptVAP7UWV14

* clean up

---------

Co-authored-by: Claude <noreply@anthropic.com>
2026-09-01 11:53:13 -04:00
Kevin Kelleher
4348d9373e Improve Card type usage (#81409)
* improve card type usage

* fix a few more Card type casts

* use same gate for visualize another way button as in getVisualizerInitialState so button doesn't silently do nothing
2026-09-01 15:49:36 +00:00
Michael Eby
3e094bd928 docs: reorganize line, bar, and area chart docs (#81108)
* docs: reorganize line, bar, and area chart docs

* Optimized 1 images, saved 199.4 KB (55.6% smaller)

* add content from learn bar chart and histograms articles

* edits from jeff

* remove learn article reference

---------

Co-authored-by: Metabase Automation <github-automation@metabase.com>
2026-09-01 10:54:36 -04:00
Ryan Laurie
880bb31835 Treat EoL dates as unsupported (#81639)
treat EoL dates as unsupported
2026-09-01 08:36:22 -06:00
Sébastien
698fd4f1ee Fix flaky terms acceptance in sdk iframe embed setup wizard (#80826) 2026-09-01 14:46:24 +02:00
Sébastien
4dda44859b Move embed wizard copy-event tracking test from Cypress to Jest (#80821) 2026-09-01 14:45:52 +02:00
Sébastien
bd7048574b Gate embedding-hub sandbox column-update test on stepper settling (#80808) 2026-09-01 14:45:18 +02:00
Sébastien
7c8d735d20 Deflake SDK target-collection data-picker test (#81558) 2026-09-01 14:44:45 +02:00
Alexander Polyankin
8b29969f1c Public settings cleanup (#81452)
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-09-01 08:19:25 -04:00
Nicola Mometto
90b3db6dec Escape LIKE metacharacters in user-supplied search strings (#81161)
* Escape LIKE metacharacters in user-supplied search patterns (SEC-847)

Name the LIKE escape character explicitly so the escaping also holds on app DBs without a default one

Add h2x/like-substring and h2x/like-prefix and use them at every search sink

* Update in-place search filter tests for escaped LIKE patterns
2026-09-01 12:34:22 +01:00
Chris Truter
39ad1d5167 Keep ratchets off release branches (#81033) 2026-09-01 11:58:19 +02:00
Chris Truter
ba06454a57 Stabilize transform API tests with explicit native-query permissions (#81574)
Grant native create-queries explicitly in transform API tests

The transform API tests grant :perms/transforms explicitly but rely on the ambient db-level create-queries default (:query-builder-and-native) for the shared dataset DB.
That default is fragile: the native level only exists db-level, so any table-level create-queries row written for the group granularizes it down to :query-builder, and re-coalescing can never bring the native level back.
A target table synced by an async transform run during another test's snapshot-restore window leaves exactly such a row, and full-database-permission-for-user takes the most restrictive value across the DB.
When that happens, every test creating or running a native-query transform as a non-superuser 403s for the rest of the run while the MBQL-source tests keep passing (seen on the SQL Server 2022 driver job).

Replace the bare transforms grant with with-transform-db-perms!, which also grants db-level create-queries :query-builder-and-native.
A db-level set deletes every row of that perm type - leaked table rows included - so the grant repairs any polluted state for the test and the scoped restore puts the prior rows back afterwards.
2026-09-01 09:39:30 +00:00
Andrei Simionescu
a1059fff57 BOT-2018: Support Anthropic fast mode for Metabot (#80511)
Admins can turn on fast mode for Metabot when the selected Anthropic model offers it. A rejected fast request falls back to standard speed and holds it for five minutes, so the speed and the cache prefix stay put.
2026-09-01 10:16:58 +01:00
Andrei Simionescu
3d5011e100 BOT-1668: hide the MCP server URL when the MCP server is disabled (#81195) 2026-09-01 10:08:38 +01:00
Timofey Kachalov
08b90a304c Escape and anchor the www. strip in CSP host allowlist (#81182) 2026-09-01 12:06:14 +04:00
Timofey Kachalov
49af71fd44 Validate embedding-app-origins-interactive before it reaches headers (#81166) 2026-09-01 12:05:48 +04:00
Nathan Voxland
e34951e8b7 Improve quoting and literal handling (#80254) 2026-09-01 00:35:52 -05:00
Fraser
8d618a8620 Viz loading slot and small moves (#81625) 2026-09-01 10:56:10 +12:00
YB
72edd1917c Bump mariadb-java-client to 2.7.15 (#81456)
bump mariadb-java-client to 2.7.15
2026-09-01 06:51:56 +08:00
YB
bb4b24c65f Bump httpclient5 to 5.6.4 (#81556)
bump httpclient5 to 5.6.4
2026-09-01 06:45:52 +08:00
Alexander Polyankin
baa2f30a5d Model the request scheme as https/http/unknown (#81592) 2026-08-31 18:22:12 -04:00
Alexander Polyankin
2742202af1 Restrict :param_fields in public/embed endpoints to public Field columns and enabled parameters (#81383) 2026-08-31 18:17:31 -04:00
Romeo Van Snick
2bd8150f2c Give the metadata mirror an enforced public API at metabase/metadata/store (#81272) 2026-09-01 09:12:14 +12:00
Ben Grabow
bd77074097 Migrate H2 -> SQLite sample db even when we're in new-install mode (#81596) 2026-08-31 15:58:10 -05:00
Chris Truter
a93af09ff5 Bump the remaining Hive artifacts to 4.2.1 (#81508)
4.2.1 is published for eight more of the Hive pins in the hive-like driver,
so move them along with hive-common rather than leaving the family split
across two patch versions.

hive-jdbc, hive-service and hive-hplsql stay at 4.2.0 -- no 4.2.1 was
published for those three.
2026-08-31 20:45:01 +00:00
Phoomparin Mano
c5e99eb320 Refresh MCP Apps UI credentials before expiry (#81041)
* fix broken mcp apps authentication due to credentials being cached

* add server-side tool calls to mint ui capability tokens

* rename MCP Apps credential metadata key

* rely on server tools for MCP Apps credentials

* show MCP Apps host errors

* add a test that we do not break existing chats

* fix transient MCP UI credential refresh failures

* combine credential refresh into a single tool

* refactor: extract ui credential constants

* refactor: simplify to a single refresh tool in frontend

* refactor: cleanup

* refactor: simplify unit tests

* refactor: use retry utility

* refactor: remove unused session-id

* chore: copy changes

* chore: rename method to redact-ui-credential

* remove v2 resource prefix

* add test for redacting secret in trace

* Harden MCP UI credential refresh after review (#81454)

* Harden MCP UI credential refresh lifecycle

* Harden MCP UI credential backend contracts

* Accept compatible MCP App resource URIs

* refactor: extract mcp auth hooks and utilities

---------

Co-authored-by: Chris Truter <crisptrutski@users.noreply.github.com>
2026-08-31 20:10:50 +00:00
Andrei Chernykh
4b8fca4eb0 Delete no-op resolutions entries (#81408) 2026-08-31 12:48:35 -07:00
Romeo Van Snick
0b035e3f44 Cut four of the common module's back-edges (#80052) 2026-09-01 07:40:16 +12:00
Chris Vandoodewaard
1b7226c27a Require job-level timeout-minutes in GitHub workflows (#80353)
* Add a lint check requiring job-level timeout-minutes in workflows

A job without `timeout-minutes` inherits GitHub's 6-hour default, so a hung
job burns six hours of runner time - and holds up everything queued behind it -
before anyone notices.

actionlint cannot enforce this: its rule set is compiled into the binary, and
`.github/actionlint.yaml` only accepts `self-hosted-runner`, `config-variables`,
and `paths`, the last of which suppresses findings rather than adding them. So
the rule lives in a small yq-based script instead.

It runs in the lint job next to actionlint, gated on the same workflow-file
filter and passed the same changed-file list, and emits `::error file=,line=::`
annotations. Jobs that call a reusable workflow are skipped, since GitHub
rejects `timeout-minutes` there.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* Fold the actionlint path filter into the workflows filter

The `actionlint` filter matched exactly the same paths as `workflows`, and the
lint job was the only consumer of it. Drop the duplicate and point the actionlint
steps at `workflows` / `workflows_files`.

Also give lint.yml's own snowplow job a 10 minute timeout, so the workflow
passes the check it adds.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* Stop yq shifting reported line numbers on files with header comments

yq slurps a leading comment block before parsing unless
--header-preprocess=false, so every line number it reported was short by the
size of that block. Five workflows start with a comment; on cache-warm.yml the
`warm` job was annotated at line 18 instead of 42.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* Update .github/file-paths.yaml

Co-authored-by: Ryan Laurie <30528226+iethree@users.noreply.github.com>

---------

Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
Co-authored-by: Ryan Laurie <30528226+iethree@users.noreply.github.com>
2026-08-31 19:20:45 +00:00
dpsutton
fd707ac9a3 Cache parse-approved-origins to stop per-request Invalid-URL log spam (#81126)
`parse-approved-origins` runs on essentially every API response (via
`approved-origin?` in the security-headers middleware, and again in the MCP
origin check) and re-parses the whole CORS-origins allowlist each time. Any
entry that fails to parse -- e.g. a malformed `http://localhost:3000:*` saved
into the SDK/MCP origins setting -- hits `log/errorf "Invalid URL"` on every
request, producing bursts of identical errors around each dashboard load.

The allowlist strings are slow-moving globals, so LRU-cache the parse. Callers
pass a couple of distinct strings (the merged SDK+MCP allowlist, and the
MCP-only one), so a small LRU bound keeps both cached without either evicting
the other, while remaining bounded.


Claude-Session: https://claude.ai/code/session_01BjQcjQx7NPGr3Rpf6QaG4b

Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-08-31 13:39:30 -05:00
Chris Truter
4f81af5691 Prevent missing search index tables from aborting caller transactions (#81573) 2026-08-31 20:33:20 +02:00
John Swanson
f2cef9e46c More clarity around revoking credentials when "base" entity is gone/deactivated (#80556)
* Fail closed when an OAuth token's client is gone

resolve-access-token looked up the access token, checked expiry, and read the
user id off the token row — but never consulted oauth_client. So an access
token kept authenticating even if the client had been deleted.

Require the token's client_id to still exist in oauth_client, using the
client_id already present in the fetched token data — one indexed lookup on the
bearer path, no schema change. A missing client now fails closed (401).

Tests:
- core-test: token resolves while its client exists, returns nil once the
  client row is deleted.
- session_oauth_test: register the referenced oauth_client so the bearer-bridge
  tests exercise the new client-existence check (previously they minted tokens
  against a client_id with no row).

* Revoke sessions and OAuth tokens on user deactivation

Add an `:event/user-credentials-revoked` event, emitted in the user
`before-update` hook when a user is deactivated, with two subscribers:

- session: delete the user's core_session rows
- oauth-server: mark the user's access and refresh tokens revoked_at

* Pin that an API key's lifecycle follows its synthetic user

An API key authenticates as its own synthetic user (api_key.user_id); the
creator_id column that records the admin who minted it is attribution only and
is never consulted at auth time. So deactivating that admin does not revoke the
key — which is the intended design: keys are org-level service credentials,
visible to and revocable by any admin from the API-key page, independent of who
created them. Revoke a key by deactivating its synthetic user
(DELETE /api/api-key/:id), not the human who minted it.

No behavior change. Add a regression test that pins this: a key survives its
creator's deactivation (200) but is revoked when its own synthetic user is
deactivated (401), so the two paths cannot be conflated by a future creator_id
join. The DELETE-endpoint revocation path is already covered by api-keys-work-e2e.

* Register the credentials-revoked topic in the events hierarchy

The events system validates topics against its own hierarchy in
metabase.events.hierarchy, not Clojure's global one, so the plain
derive calls left :event/user-credentials-revoked unregistered and
publish-event! threw a 500 whenever a user (or API key's synthetic
user) was deactivated. Use events/derive! like every other subscriber.
2026-08-31 11:29:45 -07:00
Chris Vandoodewaard
596e0c0f71 Fix for LTS tag causing minor release failure (#81038)
* filter lts versions when finding latest from tags

* always skip lts tag since it isn't valid for finding latest versions

* add guard for valid version to release-log

* add tests

* fix actionlint

* clean up/fix logic in release-log

* Keep returning the major version for rolling .x tags

getMajorVersionFromRef now recognizes v0.56.x / v0.51.2.x style tags in
addition to valid version strings, so only lts and non-version tags get
an empty string back.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
2026-08-31 18:28:27 +00:00
Kevin Kelleher
8c3ea6f40b Remove Explorations dimension anchored blocks (#80554)
* remove dimension anchored blocks from FE

* Remove dimension-anchored blocks from the backend

Research-plan groups are now always a metric sliced by dimensions; the
dimension-anchored shape (one dimension slicing several metrics) is gone:

- add_research_groups: drop anchor/dimension_id/metric_ids, require metric_id
- remove_from_research_plan: members lose metric_ids; dimension_ids required
- research-groups validation: metric-only, anchor dispatch removed
- blocks-tree: no :type on nodes; "By <dimension>" headings and the
  same-name disambiguation machinery deleted
- BlockSelection/ExplorationBlockNode schemas lose :type; the
  exploration_block.type column is no longer written (left in place,
  nullable)
- explorations.selmer prompt rewritten for the single group shape

Legacy dimension-anchored rows keep rendering (as metric-headed blocks);
pinned by the new blocks_test regression test.

* more FE cleanup, fix tests

* Drop exploration_block.type column

---------

Co-authored-by: John Swanson <john.swanson@metabase.com>
Co-authored-by: Nathan Voxland <nathan@metabase.com>
2026-08-31 17:14:01 +00:00
bryan
72987c24bc Validate LDAP server certificates and normalize synced attributes (#81039) 2026-08-31 10:43:37 -06:00
Oleksandr Yakushev
105262e022 Bump Toucan2 to 1.0.575 (#81449) 2026-08-31 19:22:45 +03:00
Andrei Chernykh
883ff5c9ac Remove unused webpack deps, fix rspack invalidate path (#81390) 2026-08-31 09:20:25 -07:00
Andrei Chernykh
fa7e1e598a Remove json-to-pretty-yaml (#81085) 2026-08-31 09:11:28 -07:00
Andrei Chernykh
94f0aec2e9 Replace @cypress/skip-test with a local helper (#80947)
* Replace @cypress/skip-test with a local e2e helper

* Document --spec requirement in the e2e-test skill

* Tidy skip-test helper comments and skill doc wording
2026-08-31 09:08:47 -07:00
Chris Truter
2da893377b Fix the cancellation race in process-userland-query-test (#81560)
* Wait for the query to reach the reduce step before cancelling in cancel-test

cancel-test gave the query a fixed 100ms head start before future-cancel and
assumed it would be sleeping inside *reduce* by then. Only an interrupt that
lands inside *run*'s try posts ::cancel to the canceled channel, so whenever
pre-pipeline setup took longer than 100ms the future died silently during
setup and the test timed out waiting for the cancel message.

That made the test flaky under CI load and deterministically red when run in
isolation: on a fresh JVM the first query's setup takes seconds, so the
granular-rerun step (which re-runs only the failed test) could never pass it,
pinning any PR that flaked onto it. Synchronize on a promise delivered from
*reduce* instead of sleeping.

* Block on a latch instead of sleeping in *reduce*

The 1000ms sleep was a second timing assumption: the cancel had to arrive
within it or the reduce would complete and save the execution. Await a latch
nobody opens instead — the interrupt wakes it just like the sleep, with no
window to miss. The 30s timeout only bounds the worker thread if cancellation
is genuinely broken.

* Use a never-delivered promise and drop the dead *result* call

A promise is the same CountDownLatch under the hood (interrupt propagates
identically from deref) and matches the started promise next to it. The
*result* call after the block was unreachable on the happy path — the
interrupt throws out of the deref — and pointless on the timeout path, where
the test failed 28 seconds earlier.

* Collapse the port/val channel assertions into one

A timeout or closed channel takes nil, which fails against ::cancel on its
own — no need to name the winning port. Also fixes the doubled word in the
testing string.

* fix cancellation test cleanup race

* Clarify cancellation test scope

* Strengthen cancellation metadata assertion

* Polish cancellation test formatting
2026-08-31 17:49:57 +02:00
Mihael Konjević
e815f913d1 Update Metabot data-source guidance for stale references and curation signals (#78248)
Clean up stale, contradictory, and unreachable guidance in Metabot prompts and skills

- Remove references to tools/params that never existed (`search_tables`, `search_transforms`, `get_report_details`, doc-search tool, `collection_id`/`database_id` on search, `update_dashboard` mutations, `is_library_member`, dead `source_tables` param, duplicate Slackbot tool)
- Align per-profile guidance with actual tool wiring: SQL-only profile has no notebook/metric/chart tools, Slackbot has no `read_resource`, `construct_notebook_query` schema differs by profile, SQL query-link behavior differs by tool
- Fix `read_resource`: pagination bug (server now returns a ready-to-use `next-page-uri` instead of broken `?page=N` guidance), curation-signal comparison, container/document listing
- Fix a Selmer `{% safe %}` vs `{% verbatim %}` bug that silently blanked all `{{#model_id}}` examples in the SQL profile prompt
- Document previously-missing required arguments across several tools (`create/edit/replace_sql_query`, `edit_chart`, time-clause `unit`, etc.)
- Fix `document_schema_collect`'s unreachable stop instruction and missing `database_id`
- Fix `list_available_fields` fallback for saved questions
- Add regression tests locking in the template-rendering and pagination fixes
2026-08-31 17:45:35 +02:00
Mahatthana (Kelvin) Nomsawadi
63057ee485 Key guest embed tokens per instance instead of one shared slot (#81351)
* key guest embed tokens per instance instead of one shared slot

Concurrent guest StaticQuestion/StaticDashboard embeds under one
MetabaseProvider wrote into a single Redux rawToken slot, so the
last-mounted instance's token won for all of them. Key by a stable
per-mount instance id instead.

* refactor: name the guest token key mountId

instanceId read as the Metabase instance, and collided with
EnsureSingleInstance's own instanceId. Also simplify the two reducer
cases the fix rewrites, which Immer lets us assign into directly, and
drop a !mountId guard that currentToken already covers.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CkZ5rUUDeeWN9GU2wuEUHk

* test: drop the multi-dashboard guest token spec

It passed with the whole fix reverted to master: asserting each endpoint
was called at least once is already true there, because every mount's
first render uses its own token prop and only the re-render collides. The
name also implied multiple StaticDashboards work, which they never have —
they share one global state.dashboard slot.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CkZ5rUUDeeWN9GU2wuEUHk

* test: cover a guest refresh that switches resource

Every existing refresh case reissues a token for the same resource, so
none of them notice which token the component ends up using — PR #81240
flipped that precedence and CI stayed green. Switching to a question that
takes different parameters is what makes the choice observable.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CkZ5rUUDeeWN9GU2wuEUHk

* fix: drop a guest mount's token when it unmounts

The iframe embed route re-keys its children when the token changes, so a
parent-side token update remounts the component with a fresh mountId.
Nothing removed the old entry, so Object.keys(...)[0] in
getOrRefreshGuestSession kept picking the dead mount and every request
was rewritten with its stale token. Once that token expired, the refresh
wrote back to the same dead key, so the map never recovered.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016stkr2R8EEPxp5gbyqDjdR

* test: cover a guest remount with a new token

Mounts a guest question, re-keys it with a second token the way the
iframe route does, and asserts the query goes out with the new token.
Fails without the unmount cleanup: no request reaches the new token's
url at all.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016stkr2R8EEPxp5gbyqDjdR

---------

Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-31 15:21:19 +00:00
github-automation-metabase
38cc09d133 Auto-update documentation for master (#81576)
Update generated docs (master)

Co-authored-by: github-actions <github-actions@github.com>
2026-08-31 11:03:40 -04:00
John Swanson
ba823a5488 Include every attribute remapping in the sandbox FieldValues fingerprint (#81040)
The per-user sandbox fingerprint (the advanced-FieldValues cache key, and
the sandbox component of data-access tokens) narrowed the hashed login
attributes to remappings whose dimension target resolved to a field of the
sandboxed table itself. But the query processor applies *every* attribute
remapping when it runs the sandboxed query, including ones targeting
columns of tables joined inside the sandbox card, or columns referenced by
name. Those remappings were silently dropped from the fingerprint, so two
users whose sandboxes differed only by such an attribute hashed to the
same field_values row and were served each other's cached values (and
could read each other's stored exploration results via the colliding
data-access token).

Fix: fingerprint every key in attribute_remappings, regardless of what
its target resolves to. This exactly mirrors the set of attributes the QP
consumes, keeps unrelated login-attribute changes from busting the cache,
and leaves hashes for own-table-only sandboxes (the common case)
unchanged. Existing colliding rows become unreachable under the new hash
and are reaped by the delete-expired-advanced-field-values sync job.

Fixes SEC-874.
2026-08-31 07:41:06 -07:00
Ben Grabow
b7d74aab6d Fix order-dependent transform permissions in tests (#81404)
* Small correctness and hardening fixes

A handful of independent, low-risk fixes bundled together — each too small to
ship on its own, each a behavior improvement that also closes a minor
hardening gap.

Reduce DB load / avoid unconditional work:
- Settings cache: a settings-last-updated cookie now triggers a cheap
  single-row staleness check and reloads only if the cache is genuinely
  behind, instead of unconditionally reloading the whole settings table.
  Exposes restore-cache-if-needed! with a :force-check? option as the shared
  "reload if actually stale" primitive the cookie path now uses.

Reduce log noise:
- CORS origin validation runs on every request's raw Origin header, so it now
  parses that client-supplied value with the silent try-parse-url rather than
  the variant that logs "Invalid URL" at ERROR — a malformed Origin no longer
  fills the operator's error log. Parsing behavior is otherwise identical, so
  no CORS allow/deny decision changes.

Correctness (case-insensitivity):
- forgot_password throttles on the lower-cased email, matching the
  case-insensitive account lookup, so User@x and user@x share one bucket
  rather than each getting a fresh allowance.

Add a missing throttle:
- PUT /api/user/:id/password throttles the old_password re-auth check, which
  previously had none, bringing it in line with the other credential
  endpoints.

* Set create queries permission explicitly for transforms tests

---------

Co-authored-by: dan sutton <dan@dpsutton.com>
2026-08-31 09:32:26 -05:00
Jeff Bruemmer
e8b8ad1a88 docs - script to generate page for supported AI providers (#81101) 2026-08-31 09:57:51 -04:00
Jeff Bruemmer
a1c217c51b docs - embedded ai chat updates (#80940)
* ai chat

* edits

* edits

* split

* revisions

Co-authored-by: Jeff Bruemmer <jeffbruemmer@gmail.com>

* revisions

* custom error image
2026-08-31 09:57:02 -04:00
Alexander Polyankin
629fd1fee0 Encrypt only settings that are secret or integrity-critical (#81506) 2026-08-31 09:37:14 -04:00
Chris Truter
f332a4e4b2 Stamp private ECR images with their Git commit (#81557) 2026-08-31 15:05:46 +02:00
Mihael Konjević
1c7ac88006 Respect sandboxing, impersonation, and routing in Metabot metadata (#80723)
* Withhold fingerprint statistics from row-restricted users in get_field_values/read_resource

Fingerprints were read from the shared Field row and refingerprinted on demand with no
restriction gating -- the same leak #436 fixed in llm/context.clj, left open here. Gate
both behind a fail-closed, per-database-batched row-restricted-table-ids check, resolving
each Field's owning table canonically rather than trusting caller-supplied metadata.

Also guard list_available_fields, viewing-context entity lookups, and typed-schemas
table/card selection against destination-database resources, and apply a missing
permission-filter-columns pass to card-details' FK targets.

* Widen LLM schema-context row-restriction handling to sandboxing and routing

row-restricted-user? only checked impersonation and failed open on a feature/license
lapse. Replace it with metabot's fail-closed row-restricted-table-ids, applied per table so
one restricted table doesn't blank out fingerprints for the whole response.

Also filter a table's own sandbox-restricted columns (previously only FK targets were),
constrain mentioned/FK-target tables to the requested database, add a database read-check,
and cap+log restricted-table field-values fetches per table.

* Cover destination-DB exclusion in typed schemas; quiet expected 404s in viewing context

Adds test coverage proving select-schema-cards/select-tables actually exclude
tables and cards backed by a destination (routed) database from typed-schema
generation -- the only guard on that path, previously unverified.

Also stops fetch-and-format logging every expected 404 (missing entity, or a
destination-db routing guard) at ERROR level; it now logs at DEBUG and still
falls back to the simple entity rendering, same as before.

* Quiet expected destination-DB 404s in get_metadata

safe-fetch was logging every fetch failure at error level, including
the routing-destination-database 404s from check-table-resource-database
/check-card-resource-database, which are an expected outcome rather
than a bug. Matches the debug-level treatment already given to the
same case in agent/user_context.clj.

* Style pass: trim design-rationale docstrings, shorten oversized fns, wrap long lines

- row-restricted-by-db: move the retry-ladder rationale out of the docstring and
  onto the try/catch lines it explains.
- permission-filter-columns: extract column-table-id/field-visible?/caller-joined?/
  referenced-field-visible?/column-visible? as top-level private fns instead of
  closures, bringing the function back under ~20 lines.
- build-schema-context: extract drop-or-enrich-fingerprints.
- Wrap a handful of lines pushed past 120 chars by the with-cache nesting and new
  test fixtures.

No behavior change; same 83 tests (315 assertions) still pass.

* Fail closed on unresolvable table IDs in row-restricted-table-ids

A table ID that couldn't be resolved to a Table row silently fell out
of the per-database restriction map and defaulted to unrestricted,
contradicting the fail-closed contract this function documents.

* Log restriction-probe failures in row-restricted-by-db

All three exception paths in row-restricted-by-db were silent, so an
impersonation/routing attribute failure left no trace of why every
fingerprint for a database fell back to the per-table restriction path.

* Handle unrecognized get_field_values data_source gracefully

mu/defn schemas aren't enforced in production, so an LLM-supplied
data_source could reach a case with no default clause and throw
uncaught, failing the whole tool call. Widen the case (and its schema)
to cover "question"/"report" like field-values already does, and give
both the permission-check case and field-values a graceful fallback
for anything else.

* Return an agent-facing 404 for a field-id with no resolvable table

check-column-table-perms! used api/check-404, which throws without
:agent-error?, so handle-agent-error rethrows it and fails the whole
tool call. This is reachable from ordinary stale result_metadata (a
dropped column, or serdes drift), not just a malicious input.

* Apply restricted-field-values cap after the eligibility filter

The per-table cap on restricted-table FieldValues fetches was applied
to raw columns before filtering to those that should have FieldValues,
so ineligible columns ahead of an eligible one in the raw list could
consume its slot in the budget and leave it without values. Filter
first, then cap -- grouping by the persisted Field's own table_id
while at it, consistent with the permission checks elsewhere in this
namespace.

* Move database read-check out of the extract-sources/generate-sql prelude

fetch-accessible-tables-with-columns required database read access
unconditionally, so /extract-sources hard-403'd even when a caller only
wanted its independent :card_ids behavior. generate-sql is the path
that needs the 403, so move the check there via build-schema-context
and leave get-tables-with-columns (extract-sources) to return an
empty result instead.

* Exclude destination-DB cards and fieldless models from answer-sources

get-metrics-and-models had no destination-database guard, so a card
backed by a destination (routed) database could still reach
list_available_data_sources -- the same gap already closed for
get_metadata. Also omit models whose :fields end up empty after
permission filtering (no view-data access to their table) instead of
listing them as data sources with nothing queryable.
2026-08-31 14:44:47 +02:00
Rodrigo López Dato
7d46650cad Don't use app DB pool to probe for initial connectivity (#81271)
Prevents app DB misconfigurations from timing out by failing immediately
2026-08-31 08:17:54 -03:00
YB
24f4aec505 Bump hive-common to 4.2.1 in the hive-like driver (#81460)
bump hive-common to 4.2.1 in the hive-like driver
2026-08-31 17:15:55 +08:00
YB
c839a31539 Pin reactor-core 3.8.7 in the Snowflake driver (#81462)
pin reactor-core 3.8.7 in the snowflake driver
2026-08-31 17:15:03 +08:00
YB
3866917fba Bump jsoup to 1.23.2 (#81457)
bump jsoup to 1.23.2 and pin it in the snowflake driver
2026-08-31 17:14:28 +08:00
YB
1866aaccd9 Bump sshd-core to 2.19.0 (#81461)
bump sshd-core to 2.19.0
2026-08-31 17:14:17 +08:00
YB
52fa5626a2 Bump httpclient5 to 5.6.3 (#81459)
bump httpclient5 to 5.6.3 and pin it in the clickhouse and databricks drivers
2026-08-31 17:14:05 +08:00
YB
ad47b4094b Bump Netty to 4.1.137.Final / 4.2.17.Final in driver deps (#81458)
bump netty to 4.1.137.final and 4.2.17.final in driver deps
2026-08-31 17:13:55 +08:00
Alexander Polyankin
d568f38f1a Encrypt plaintext values of encrypted-at-rest columns on startup (#81498) 2026-08-31 00:12:53 -04:00
Alexander Polyankin
0c54590106 Read settings without the cache while reconciling their encryption (#81491) 2026-08-30 23:02:41 -04:00
Alexander Polyankin
840faeb408 Reconcile settings encryption with the registry on every startup (#81478) 2026-08-30 21:38:46 -04:00
Alexander Polyankin
172bc22044 Name the source in strict-decrypt read failures (#81471) 2026-08-30 17:25:28 -04:00
Alexander Polyankin
c5878f2509 Encrypt legacy plaintext rows of :setter :none settings (#81464) 2026-08-30 15:25:34 -04:00
Ben Grabow
1e78c186c0 Improve log output in DWH GC job (#81447) 2026-08-29 04:05:59 +00:00
Alexander Polyankin
6b9bcaca8e Never encrypt the app DB on startup when there is existing data; add enable-encryption command (#81237) 2026-08-28 22:05:19 -04:00
John Swanson
418fbf1bf9 Treat API-key users consistently across user endpoints and attributes (#78976)
* Treat API-key users consistently across user endpoints and attributes

API keys are backed by rows in core_user with type = 'api-key'. Most
/api/user endpoints already restrict themselves to personal users, but
a few didn't, and user attributes were resolved for all user types.
Align everything on the rule that only personal users are managed via
/api/user and only personal users carry attributes:

- GET /api/user/:id and PUT /api/user/:id now 404 for non-personal
  users (API-key users and the internal user), matching the sibling
  endpoints (password, reactivate, password-reset-url, delete, modal)
  and the list endpoint's existing type = 'personal' filter. API keys
  are managed via /api/api-key instead.
- EE PUT /api/mt/user/:id/attributes now 404s for non-personal users.
- user/add-attributes only resolves attributes (tenant + jwt + login)
  for personal users; other types resolve to {}. Sandboxed queries run
  by a non-personal user therefore report a missing user attribute
  rather than reading attributes stored on the user row.

No migration: any pre-existing login_attributes on API-key rows are
left in place but no longer take effect. GET /api/user/current still
works for API-key authentication, with an unchanged response shape.

Scoped programmatic access as a first-class feature is tracked
separately in #72959.

* Provide a schema for `/api/user/current`

* Update kondo ratchet budgets

- :metabase/validate-defendpoint-has-response-schema 441 -> 440, since
  GET /api/user/current now has a response schema.
- :discouraged-var 100 -> 101 for the mt/with-temp ignore in the new
  sandboxing test: it creates a :model/User, which metadata providers
  do not support, matching the exception documented in the linter
  message and the file's existing ignores.

* CR resp - throw on missing `:type`
2026-08-28 15:41:39 -07:00
Ryan Laurie
7293988a96 dont refresh dynamic snowflake tables (#79846)
# Conflicts:
#	modules/drivers/snowflake/test/metabase/driver/snowflake_test.clj
2026-08-28 22:32:27 +00:00
Timofey Kachalov
e925a8df94 Temporarily disable Shoppy sample app tests (#81416) 2026-08-28 15:53:59 -06:00
Fraser
a0d5c5a0c7 Create the viz-core module (#80786) 2026-08-29 09:53:45 +12:00
Ryan Laurie
441cfa66ef Add Databricks Workspace Sharding (#81241)
* add env sharding for databricks

* follow the rules

* kick ci

* kick ci

* make the console output nicer
2026-08-28 15:15:46 -06:00
Timofey Kachalov
01add7e6e2 Fix per-table permission queries in db metadata endpoint (#81397) 2026-08-29 00:54:55 +04:00
appleby
e023152e90 Send finish-reason length for a Gemini MAX_TOKENS truncation (#80959)
* Send finish-reason length for a Gemini MAX_TOKENS truncation

Translate Gemini finishReason values to AI SDK v5 finish reasons and add them to the `:usage` chunk, as the other
adapters do. `MAX_TOKENS` now reports `:finish-reason "length"` instead of an in-stream error part, so the agent loop
stops cleanly and the client can offer to continue.

* Add missing image-related stop-reasons to gemini provider

The image related stop reasons should happen in practice since we only do text generation, but add them for
completeness.

https://docs.cloud.google.com/gemini-enterprise-agent-platform/reference/rest/v1/GenerateContentResponse#FinishReason

* Add stop-reasons from the Gemini Developer API as well

https://github.com/metabase/metabase/pull/80959#discussion_r3867216474
https://ai.google.dev/api/generate-content#FinishReason

* Review feedback: only add synthetic usage part for finish-reasons-without-error

* Review feedback: name the early-stop-without-error subset
2026-08-28 13:27:23 -07:00
Anderson
e9235e2983 Fix flaky remote-sync branch-switch e2e test (#80397)
* Deterministically gate remote-sync branch-switch test on its async boundaries

* Sign in before activating the token in the admin settings describe

* Wait for collection items responses before asserting synced content

* Reword test comments per writing conventions

* Trim comments and move timeouts to request/test-level options

* Remove test comments and restore waitForTask retries cap

* Fix H2 deadlock between the post-import search reindex and readers

* Additional cleanup

* Cover the in-task H2 reindex with a unit test

* Consolidate remote-sync e2e setup into a shared helper
2026-08-28 16:54:53 -03:00
appleby
5bf02e30d7 Support Anthropic partner models via the Google provider (#80133)
* Support Anthropic partner models via the Google provider

Anthropic models are served by `streamRawPredict` with an Anthropic Messages body, so route them through the new
`metabase.metabot.self.google.raw-predict` namespace, which wraps the existing claude adapter. Connecting probes the
model's own `streamRawPredict` route with an empty body: Google resolves the credential, project, location and model
from the URL before Anthropic sees the body, so an Anthropic-shaped 400 validates all four without spending a tokens.

* Title-case the Gemini model display names

To match the convention for all other providers and models.

* Translate mid-stream failures on the Google provider

`google-raw` handed back the SSE reducible unwrapped, so a failure while the agent loop consumed it escaped the
adapter untranslated and surfaced as a raw 500. Wrap it in `core/reducible-with-api-errors`, like every other adapter.

* Carry the probed model in :learned-config

We already reported `:probed-model` as sibling of `:learned-config`. Nest it inside so that it gets saved with the
learned config so we can use it for subsequent connection probes for the Google provider. This tends to be a much
better guess for what model we should use for the connection test rather than just using the first model in the fixed
models list, since there is no guarantee that model is actually present in the currently configured location.

* Probe a Google connection against the model it serves

The listing path re-probed whichever model the fixed catalog names first, so a connection at a location serving Claude
but not Gemini 3.x saved fine and then reported itself broken the moment its cache entry expired. Probe the model the
connection is known to serve instead — the one Metabot is pointed at, or the `:probed-model` its last successful probe
recorded — and fall back to the catalog's first model only when it has neither.

* Start the edit form on the model the connection was probed against

Reopening a connection while Metabot points elsewhere reset the Model picker to the type default and submitted that as
the probe target, so a Google project whose location serves Anthropic partner models but not Gemini had a no-op edit
rejected against credentials that work. Fall back to the connection's own probed-model, when the catalog still offers
it, before the type default.

* Ensure the obfuscated-value? regex matches values ending in newline

Enable "dotall" mode for the obfuscated-value? regex to support obfuscated values where one of the last two chars is a
newline. For example, the contents of JSON service account key used for Google cloud auth or similar.

https://docs.oracle.com/en/java/javase/21/docs/api/java.base/java/util/regex/Pattern.html#DOTALL

* Review feedback: validate-google-model!

* Review feedback: use proposed-model in vllm list-models

* Review feedback: ensure new-value is not an obfuscated-value? in set-single-provider-setting!

* Add comment about `model->family` dispatch

Co-authored-by: Chris Truter <crisptrutski@users.noreply.github.com>

---------

Co-authored-by: Chris Truter <crisptrutski@users.noreply.github.com>
2026-08-28 11:54:39 -07:00
John Swanson
cb7c71d1ba Allow embedding readable cards in documents without adhoc query-authoring perms (#80775)
Allow embedding readable cards in documents without query-authoring perms (UXW-5037)

Saving a document clones every embedded existing card into a document-owned
card row. The clone path ran check-run-permissions-for-query on the cloned
query, which for a native query demands create-queries=query-builder-and-native
on the whole database. A user who could view (and run) an existing native
question therefore could not save a document embedding it, losing their work
with a vague 403.

That check is an authoring gate, but the embedding user is not authoring a
query: the query text comes from an existing card row they passed a read
check on -- the client supplies only a card ID, never SQL. Running the clone
is still gated by the usual runtime permission checks (card read perms,
result-metadata view-data, block perms), which are the same checks that gate
running the source card, so cloning grants no new runtime capability. This
mirrors POST /api/card/:id/copy, which likewise clones a readable card
without an authoring check.

Split the documents-API card creation helper into create-card! (still
checked; used for client-supplied draft cards) and clone-card! (used when
cloning embedded cards and when copying a document's cards). Authoring a new
query -- via draft cards or by editing a cloned card's query through
PUT /api/card/:id -- still requires the appropriate query permissions.
2026-08-28 11:35:40 -07:00
Ben Grabow
2bfd318907 Dwh gc parallel (#81382)
* Bump timeout to 60 minutes

* Run redshift GC in parallel, with 1 worker per cluster node

* Run snowflake GC with N=8 parallelism

* Record dataset census before GC sweep

* Satisfy the linter
2026-08-28 13:27:49 -05:00
John Swanson
4c62275732 Hide all tenant groups/users from perms graph api (#74973)
* More consistently hide tenant users/groups

There were a few locations where we weren't correctly hiding the
existence of Tenant Groups such as "All Tenant Users" when the setting
was turned off. This also hides tenant *users* from the API surface.

* allow specifically showing tenant users

I realized that there was a possible regression in the last commit. We
were hiding tenant users across the board. This meant that a workflow
that was previously possible was not possible anymore:

- an admin turns off the tenant feature or disables tenants

- then the admin uses the API to reactivate some tenant users and turn
  them into internal users.

This is probably super niche anyway but it seems nicer to allow `GET
/api/user` to show these users, IF you pass the
`tenancy=external&status=all` flags.

Then you can reactivate the user and detach them from the tenant.

This is all API only so it's probably academic, but figured I'd keep the
door open.

* fix tests

* fix e2e tests

* Move tenant tests to enterprise tenant test namespaces

* Address review feedback on tenant-group hiding

- Rewrite hidden-tenant-group-ids to intersect the input IDs with all
  tenant-group PKs in memory instead of building an unbounded IN clause
- Return 404 from PUT/DELETE /api/permissions/group/:group-id for hidden
  tenant groups, matching GET /api/permissions/group/:id
- Assert GET /permissions/graph/group/:id succeeds when use-tenants is on
- Remove duplicate create-tenant-group-requires-use-tenants-test
2026-08-28 11:26:13 -07:00
John Swanson
2971a5b944 Strip client-supplied :card before hydrating in update-dashcards! (#73104)
`dashboard/update-dashcards!` calls `(params/dashcards->param-field-ids
(t2/hydrate new-dashcards :card))` to compute the field IDs referenced by
parameter mappings. `t2/hydrate` is a no-op when `:card` is already present
on a dashcard, so if the client `PUT`s `/api/dashboard/:id` with a dashcard
that includes a nested `:card` (the standard shape from `GET`), that object
is forwarded verbatim into `param-target->field-id`.

That function validates its `card` arg against `:metabase.queries.schema/card`
(added Oct 2025), which requires keywordized metadata (`:type/Category`,
`:source/table-defaults`, ...). After JSON round-trip through
`cy.request`, those values are strings, so the request 500s with
`Invalid input: {:result_metadata [{:semantic_type [...]}]}`.

`dissoc :card` before the hydrate call forces a fresh DB select with
properly-typed metadata, keeping the on-demand-DB field-value update path
unchanged.

Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-08-28 11:24:38 -07:00
dpsutton
439b8e2af5 Small correctness and hardening fixes (#81374)
A handful of independent, low-risk fixes bundled together — each too small to
ship on its own, each a behavior improvement that also closes a minor
hardening gap.

Reduce DB load / avoid unconditional work:
- Settings cache: a settings-last-updated cookie now triggers a cheap
  single-row staleness check and reloads only if the cache is genuinely
  behind, instead of unconditionally reloading the whole settings table.
  Exposes restore-cache-if-needed! with a :force-check? option as the shared
  "reload if actually stale" primitive the cookie path now uses.

Reduce log noise:
- CORS origin validation runs on every request's raw Origin header, so it now
  parses that client-supplied value with the silent try-parse-url rather than
  the variant that logs "Invalid URL" at ERROR — a malformed Origin no longer
  fills the operator's error log. Parsing behavior is otherwise identical, so
  no CORS allow/deny decision changes.

Correctness (case-insensitivity):
- forgot_password throttles on the lower-cased email, matching the
  case-insensitive account lookup, so User@x and user@x share one bucket
  rather than each getting a fresh allowance.

Add a missing throttle:
- PUT /api/user/:id/password throttles the old_password re-auth check, which
  previously had none, bringing it in line with the other credential
  endpoints.
2026-08-28 13:13:31 -05:00
Ben Grabow
8861e84d3f Cache clojure cli download (#81369) 2026-08-28 12:24:22 -05:00
Ben Grabow
0cad99631a Fix flaky query executions test (#81366)
* Flush other executions before test. Divert other executions during test.

* Flush after closing the batch ingress path.

* Remove extraneous changes

* Remove more extraneous changes
2026-08-28 12:04:12 -05:00
Nathan Voxland
393086c44a Don't allow reset password for deactivated accounts (#78699) 2026-08-28 11:32:14 -05:00
Ryan Laurie
7e3ab7a682 Cleanup data warehouses nightly (#79849)
* cleanup data warehouses nightly

* rename

* don't reinvent wheels

* simplify

* more

* cleanup

* updates

* Absorb failures at the redshift cluster level and at the driver level

* Extract old dataset name pattern

* Update ratchets

* Merge MB_REDSHIFT_TEST_HOSTS and MB_REDSHIFT_TEST_HOST when both are set

* Allow ad-hoc dispatch and report on outcomes of gc sweeps

* Don't run on forks

* Update ratchets

---------

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: Ben Grabow <ben.grabow@gmail.com>
2026-08-28 10:09:56 -05:00
Sébastien
e56d3de794 Wait for the content-translation dictionary in the SDK rerender spec (#81352) 2026-08-28 16:55:31 +02:00
github-automation-metabase
c06c3b8f16 Make isRenderedWithinViewport retry until element settles in viewport (#77401)
Make isRenderedWithinViewport retry until element settles in viewport

The command read getBoundingClientRect() once, so a caller asserting right
after a scroll-into-view action (focusing a recently moved dashboard filter)
could observe the element mid-scroll (top ~1647px vs 800px viewport) and fail.
Wrapping the geometry checks in .should() lets Cypress re-read the rect until
it lands within the viewport.
2026-08-28 16:19:42 +02:00
Nicola Mometto
190dcf1bd0 Compute longest common substring length with an iterative O(n*m) pass (#80985)
* Compute longest common substring length with an iterative O(n*m) pass

* Add perf edge-case tests for largest-common-subseq-length
2026-08-28 14:55:29 +01:00
Mihael Konjević
4fc305fa64 fix: recognize lib/*-stripped pMBQL in ->legacy-mbql, restore edit-chart's query-id (#78798)
* fix: recognize lib/*-stripped pMBQL in ->legacy-mbql, restore edit-chart's query-id

->legacy-mbql gated pMBQL->legacy conversion on :lib/type alone, so a query that
lost its :lib/* keys (e.g. round-tripped through the frontend's viewing context)
fell through unconverted. The raw pMBQL then got embedded in /question# links,
crashing the frontend with "Stage 0 does not exist". Detect pMBQL structurally
(:lib/type or :stages) and normalize against the query schema before converting.

Also add the query-id edit-chart's result was missing, so extract-charts can
capture edited charts into conversation state and chart->xml stops rendering
an empty query-id attribute.

* fix: preserve edit-chart's query in memory, don't fail turn on link conversion

edit-chart's result set :query-id without :query, so extract-charts (which
requires both to fire, then rebuilds :queries from :query alone) overwrote
the chart's real queries with [nil] on the next update-memory call.

->legacy-mbql could also throw instead of degrading: normalizing a
:lib/*-stripped query mints fresh :lib/uuids that don't match positional
aggregation/expression refs elsewhere in the query (e.g. an order-by on the
query's own aggregation), failing conversion inside the streaming transducer
and taking down the whole agent turn instead of just producing a bad link.
Falls back to the raw query on conversion failure now.

* fix: document ->legacy-mbql's conversion-failure fallback

Docstring claimed conversion always succeeds after normalize, but the
try/catch fallback added earlier can still return raw pMBQL when a
stripped query's positional refs don't resolve.

* test: mark edit-chart-result-survives-update-memory-test ^:parallel

Doesn't write shared state, same as the test above it.

* fix: thread edit-chart-tool's resolved query into edit-chart

edit-chart-tool falls back to current-queries-state when a chart's own
:queries is empty, but edit-chart only checked charts-state, so its
result carried :query-id with a nil :query in that case -- the same
[nil]-in-memory bug fixed earlier, via a different path. Pass the
caller's resolved query through as a fallback.

* fix: capture viewing-context charts' query-id, don't clobber on edit

chart-config->chart never set :query_id, so a chart seeded from viewing
context rendered query-id="" on edit and was never captured into :charts
state by extract-charts. Reuse the context-item id already shared with
:chart_id.

That capture also collided with edit-chart-tool's own memory write for
the same chart-id: extract-charts fully replaced the entry, dropping
:image_base_64/:timeline_events/:chart_config the tool had just written.
Merge onto the existing entry instead.

* fix: separate ->legacy-mbql's docstring from its bug history

Docstring for ->legacy-mbql read as the change's rationale (BOT-1604,
the :stages detection story) rather than the function's contract.
Trim the docstring to what a caller needs -- normalizes MBQL 5 to
legacy, passes through unchanged on failure -- and move the "why"
for the :stages guard down to an inline comment next to the check.
2026-08-28 15:10:46 +02:00
Stas Gavrylov
daf31c8bc3 Fix lint-eslint-pure warning noise in fresh worktrees (#81214)
Remove stale /* eslint-env node */ comments from frontend/build files

Flat config ignores eslint-env comments (ESLintEnvWarning per file, hard
errors as of ESLint v10), and they are redundant: eslint.config.mjs already
defines globals.node for frontend/build/**/*.js.
2026-08-28 16:08:27 +03:00
John Swanson
cfd6896c17 respect default time dimensions from list of dimensions (#79167)
* respect default time dimensions from list of dimensions

* Curate a default time dimension on the time-facet test fixture

Master's #79409 stopped auto-assigning a default dimension to new
metrics, so the time-facet tests' fixture card no longer gets a default
time dimension from dimension auto-sync. Since this branch makes the
curated default authoritative for the time-facet variant (no fallback to
the query's temporal breakout), curate the default explicitly on the
fixture.
2026-08-28 05:32:29 -07:00
YB
847666c6d1 [GDGT-2528] return an actionable error when an export target id does not exist (#80809)
return an actionable error when a serialization export target id does not exist
2026-08-28 20:27:13 +08:00
John Swanson
bb2b9786f3 Make register-listeners! idempotent, mirroring the queue-declaration fix (#80887)
The earlier fix (#80388) made *queue declarations* reload-safe by guarding
them with a load-time ownership claim and replace-on-reregister semantics,
but left the *listener* registry non-idempotent: register-listener! threw
"Listener already registered" whenever the channel was already present in
the root *listeners* atom.

Fix mirrors the queue side exactly: def-listener! now claims its channel's
declaration site at load time (claim-listener-declaration! — same-ns
re-claims are no-ops so reloads stay legal, cross-ns claims throw), and
activates through register-declared-listener!, which replaces any existing
registration instead of throwing. Ad-hoc batch-listen! keeps its strict
throw-on-duplicate contract (double-listen-throws-test et al.), since
runtime registrations have no load-time claim protecting them.
2026-08-28 05:01:03 -07:00
Romeo Van Snick
57ac36ff71 Match permissions URLs for databases without schemas (#81332) 2026-08-28 11:25:40 +00:00
Timofey Kachalov
8b42f49288 Normalize instance-origin matching when filtering data-app allowed_hosts (#81106) 2026-08-28 15:14:37 +04:00
YB
3ef8df323a Bump Bouncy Castle to 1.85.2 (#81287)
bump bouncy castle to 1.85.2
2026-08-28 18:44:37 +08:00
Alexander Polyankin
8b478a76ab Encrypt subscription/alert recipient details; enforce allow-list on send (#81227)
* Encrypt subscription/alert recipient details; enforce allow-list on send

Encrypt at rest the `details` column of `notification_recipient` and
`pulse_channel`, which stores external email recipients for alerts and
dashboard subscriptions, so plaintext values are rejected on the strict
read when an encryption key is configured. Register both columns in
`encrypted-string-columns` so `encrypt-db`/`decrypt-db` and key rotation
re-encrypt them, and add a v58 migration that encrypts existing rows and
decrypts them on rollback.

Also enforce `subscription-allowed-domains` at send time (previously only
on write): the notification send path calls `validate-email-handlers!`
before delivery and throws if any external recipient is not permitted by
the allow-list. The exception is re-thrown with a generic message so
recipient addresses are never written to the logs.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* Move the v58 pulse_channel encryption migration test into the v58 group

order-your-migration-tests-test requires migration tests to be grouped by
major version in ascending order; the new test had landed among the v48 tests.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
2026-08-28 12:19:26 +02:00
Sébastien
ed6247eb24 Fix collapsed visualization in the dashboard new-question query builder (#81163) 2026-08-28 10:10:07 +02:00
Chris Truter
53a31166db Justify the grandfathered kondo ignores (#78168)
* Justify the :metabase/discourage-millis-duration ignores

Claude-Session: https://claude.ai/code/session_01GParLfcfdFBwFrdzUWjGJN

* Justify the :reduce-without-init ignores

Claude-Session: https://claude.ai/code/session_01GParLfcfdFBwFrdzUWjGJN

* Justify the :def-fn ignores

Claude-Session: https://claude.ai/code/session_01GParLfcfdFBwFrdzUWjGJN

* Justify the :unquote-not-syntax-quoted ignores

Claude-Session: https://claude.ai/code/session_01GParLfcfdFBwFrdzUWjGJN

* Justify the :aliased-namespace-symbol ignores

Claude-Session: https://claude.ai/code/session_01GParLfcfdFBwFrdzUWjGJN

* Justify the :equals-true ignores

Claude-Session: https://claude.ai/code/session_01GParLfcfdFBwFrdzUWjGJN

* Justify the :case-symbol-test ignores

Claude-Session: https://claude.ai/code/session_01GParLfcfdFBwFrdzUWjGJN

* Justify the :metabase/prefer-with-dynamic-fn-redefs ignores

Claude-Session: https://claude.ai/code/session_01GParLfcfdFBwFrdzUWjGJN

* Justify the :syntax ignores

Claude-Session: https://claude.ai/code/session_01GParLfcfdFBwFrdzUWjGJN

* Justify the :unused-binding ignores

Claude-Session: https://claude.ai/code/session_01GParLfcfdFBwFrdzUWjGJN

* Justify the :unused-private-var ignores

Claude-Session: https://claude.ai/code/session_01GParLfcfdFBwFrdzUWjGJN

* Justify the :unresolved-symbol ignores

Claude-Session: https://claude.ai/code/session_01GParLfcfdFBwFrdzUWjGJN

* Justify the :unused-excluded-var ignores

Claude-Session: https://claude.ai/code/session_01GParLfcfdFBwFrdzUWjGJN

* Justify the :unresolved-namespace ignores

Claude-Session: https://claude.ai/code/session_01GParLfcfdFBwFrdzUWjGJN

* Justify the :unresolved-require ignores

Claude-Session: https://claude.ai/code/session_01GParLfcfdFBwFrdzUWjGJN

* Justify the :unused-value ignores

Claude-Session: https://claude.ai/code/session_01GParLfcfdFBwFrdzUWjGJN

* Justify the :unused-alias ignores

Claude-Session: https://claude.ai/code/session_01GParLfcfdFBwFrdzUWjGJN

* Justify the :metabase/namespace-name ignores

Claude-Session: https://claude.ai/code/session_01GParLfcfdFBwFrdzUWjGJN

* Justify the :missing-docstring ignores

Claude-Session: https://claude.ai/code/session_01GParLfcfdFBwFrdzUWjGJN

* Justify the :metabase/defsetting-namespace ignores

Claude-Session: https://claude.ai/code/session_01GParLfcfdFBwFrdzUWjGJN

* Justify the :metabase/validate-deftest ignores

Claude-Session: https://claude.ai/code/session_01GParLfcfdFBwFrdzUWjGJN

* Justify the :missing-protocol-method ignores

Claude-Session: https://claude.ai/code/session_01GParLfcfdFBwFrdzUWjGJN

* Justify the :discouraged-java-method ignores

Claude-Session: https://claude.ai/code/session_01GParLfcfdFBwFrdzUWjGJN

* Justify the :clojure-lsp/unused-public-var ignores

Claude-Session: https://claude.ai/code/session_01GParLfcfdFBwFrdzUWjGJN

* Justify the :metabase/modules ignores

Claude-Session: https://claude.ai/code/session_01GParLfcfdFBwFrdzUWjGJN

* Justify the :metabase/i-like-making-cams-eyes-bleed-with-horrifically-long-tests ignores

Claude-Session: https://claude.ai/code/session_01GParLfcfdFBwFrdzUWjGJN

* Justify the :metabase/test-helpers-use-non-thread-safe-functions ignores

Claude-Session: https://claude.ai/code/session_01GParLfcfdFBwFrdzUWjGJN

* Justify the bare :clj-kondo/ignore forms

Claude-Session: https://claude.ai/code/session_01GParLfcfdFBwFrdzUWjGJN

* Justify the :discouraged-namespace ignores

Claude-Session: https://claude.ai/code/session_01GParLfcfdFBwFrdzUWjGJN

* Justify the :deprecated-namespace ignores

Claude-Session: https://claude.ai/code/session_01GParLfcfdFBwFrdzUWjGJN

* Justify the :deprecated-var ignores

Claude-Session: https://claude.ai/code/session_01GParLfcfdFBwFrdzUWjGJN

* Justify the :discouraged-var ignores

Claude-Session: https://claude.ai/code/session_01GParLfcfdFBwFrdzUWjGJN

* Correct inaccurate justification comments from review

Reword the overclaims the review round caught: 'every render path'
and 'every test-query feature' scoped to what the tests cover, the
'mutates nothing' datasource comments acknowledge lazy pool init, the
'only channel' store comments name the pending general-cached-value
migration, the ASCII-makes-default-locale-safe clauses dropped
(Turkish casing changes ASCII I), and the accidentally-justified
import-vars ignore in metabase.test gets a real comment. Contested
claims move to the uncertain list rather than guessing.

Claude-Session: https://claude.ai/code/session_01GParLfcfdFBwFrdzUWjGJN

* Use Locale/US casing directly; drop the ignores it obsoletes

The review round was right that the circular-dep comments didn't
justify locale-sensitive casing: a Turkish default locale lowercases
ASCII I to a dotless i, mangling locale codes like "ID" and hiding
"Windows" from the os.name probe. Call .toLowerCase/.toUpperCase
with Locale/US at both sites, which removes the :discouraged-var
ignores entirely instead of excusing them. The logger comment now
states the real invariant: the notice reports logger reconfiguration
itself, so it can't go through log4j.

Claude-Session: https://claude.ai/code/session_01GParLfcfdFBwFrdzUWjGJN

* Correct remaining kondo justification comments

* Correct four justification comments from CodeRabbit review

* Correct the qp.store justification: the mbql5 shim reads the store too

* Complete kondo justification coverage

* Keep the named ignore forms while justifying them

The per-linter sweep was written when these sites still used the bare
`#_:clj-kondo/ignore`, so replaying it over the naming branch put the bare
form back under each new comment. Restore the linter vectors and rebuild
the budgets: :comment-exempt lands at 10, down from the 38 the mechanism
starts with.

* Justify the ignores master added under the exempted linters

The restack picked up new :deprecated-var and :clojure-lsp/unused-public-var
ignores from master. Left unjustified they would have re-exempted both linters,
undoing what this branch is for.

* Regenerate kondo ignore budgets after restacking onto master
2026-08-28 05:59:14 +00:00
Chris Truter
72d6cfe9cd Fail fast when a test loses its rollback boundary (#80994)
Fail the test when a rollback-only scope cannot roll back

Every top-level `with-temp` runs in a rollback-only transaction. If something
inside one commits -- DDL does that implicitly on H2 and MySQL, which also
destroys the savepoint -- the rows written up to that point are durable and
leak into every later test. Warning about it hides the leak until some
unrelated test counts rows and gets the wrong answer, and the six offending
sites found so far were caught exactly that way.

Throw instead when running tests. That covers every cause -- raw SQL DDL,
HoneySQL DDL, a savepoint lost to a deadlock -- where a guard on the SQL we
build would miss the raw statements `drop-table!` sends. Production keeps
warning and discarding what is still pending: there the transaction is a real
one that has already committed, and failing the request would not undo it.
2026-08-28 05:53:46 +00:00
technomancy
f9008f7884 Disable databricks JDBC driver's auto-retry. (#81231)
It sucks!

We will need to put our own error handling in place to do Actually
Good retries for 429s, but it's not clear yet where the best place for
that would be, because we're not getting stack traces from the 429s currently.
2026-08-28 00:05:28 -05:00
Romeo Van Snick
e6e11ee362 Write the metadata mirror from one listener, not 33 endpoint hookups (#81147)
* Write the metadata mirror from one listener, not 33 endpoint hookups

* Describe the hydration listener by what it does, not what it replaced

* Point the original_fields note at the listener, not onQueryStarted

* Pin the hydration listener to Api actions, and report a failed write
2026-08-28 13:16:41 +10:00
Romeo Van Snick
41cd2ab4f7 Name the chunks that lazy routes load (#80970) 2026-08-28 11:46:56 +10:00
Ben Grabow
f73eb6d00e Filter server-side when all target schemas are exactly named (#81250)
* Filter server-side when all target schemas are exactly named

* Blacklist metacharacters instead of whitelist word characters
2026-08-27 17:04:19 -07:00
Romeo Van Snick
21024ec984 Keep MCP node_modules out of the shared vendor chunk (#81157) 2026-08-28 09:40:04 +10:00
Fraser
a7815e9354 Give query_builder a store/ folder and an enforced public API (#79965) 2026-08-28 11:26:07 +12:00
Andrei Chernykh
e819ade161 bump dompurify to 3.4.14; removed from resolutions (#81228) 2026-08-27 15:19:20 -07:00
Nathan Voxland
0163ebfd7a Don't drop column order/visibility settings when one column no longer matches (#76088) 2026-08-27 16:18:30 -05:00
Cam Saul
27852dcc4c Resolve columns from model metadata saved in ~<=55 correctly (#75771)
* Resolve columns from model metadata saved in ~<=55 correctly

* Remove annoying :lib/options in stages and joins

* Test fix 🔧

* Code cleanup

* Comments

* Fix typo

* Revert pointless changes

* Try something different??

* Test fix 🔧

* Will disabling unneeded change fix e2e tests??

* Simplified fix 🔧

* PR Feedback

* cljfmt

* Test fix 🔧
2026-08-27 13:13:48 -07:00
Nathan Voxland
9eb4ce8524 Session Management (#80243) 2026-08-27 14:35:22 -05:00
github-automation-metabase
a9337e6046 Auto-update documentation for master (#81128)
Update generated docs (master)

Co-authored-by: github-actions <github-actions@github.com>
2026-08-27 15:33:00 -04:00
John Swanson
96faa6d511 re-check read permission when listing bookmarks (#80774)
SEC-669: re-check read permission when listing bookmarks

GET /api/bookmark (bookmarks-for-user) filtered only on the target's
archived column and never re-checked read permission, so a bookmark kept
leaking a card/dashboard/collection/document/exploration's name,
description, and type after the user lost read access without the item
being archived (moved to a private collection, or a revoked/narrowed
collection grant).

Add a visible_collection_ids CTE (permissions only; archived filtering is
left to the existing where-conditions) and filter each bookmark branch by
collection visibility via visible-collection-filter-clause. Visibility is
evaluated for the passed user-id via a UserScope, so the function is
correct for any user, not only the current request's user.
2026-08-27 12:28:20 -07:00
Uladzimir Havenchyk
dff2fd94ce chore: replace lodash.orderby with inline function (#81160)
* chore: replace lodash.orderby with inline function

* drop deps

* fix eslint
2026-08-27 22:15:20 +03:00
Nathan Voxland
a25cc1f7be Improve oracle parsing (#80043) 2026-08-27 14:12:55 -05:00
Nathan Voxland
5c63c45970 Better error handling on result_metadata.settings (#80909) 2026-08-27 14:12:10 -05:00
Andrei Chernykh
cb3d067c6e Bump js-yaml to 4.3.1 (#81229) 2026-08-27 12:10:32 -07:00
Andrei Chernykh
3ac0958c50 Replace @sinonjs/fake-timers with a local Date mock (#80930) 2026-08-27 11:46:36 -07:00
Egor Grushin
05300912fb Radio component redesign (#80372)
* Redesign Radio

* Add Radio stories to loki filter

* disallow radio labelPosition

* use StoryBoard in radio stories

* add new story's snapshots

* Radio.Card: border, leftSection, migrate call sites

- card border 0.5px border-neutral-strong, none when disabled
- children disallowed, leftSection slot for icons
- EditUserStrategyModal + DataSegregationStrategyPicker on label/description

* update loki snapshots

* Icon color prop instead of css module in DataSegregationStrategyPicker

* remove extra div

* make the indicator in the radio card optional

* fixed radio size, no size prop

* radio card body via mantine text/stack

* forwardRef radio card

* radio card border: highlight when selected

* update loki snapshots

* drop size="xs" from radio call sites

* fix card growing on select, interactive card stories, optional checkbox card indicator

* skip interactive card stories in loki, update card snapshots
2026-08-27 19:35:48 +01:00
Riley Thompson
3ecdab5ea7 Test for mongo multi column aggregation on joined models (#81079)
* add test

* use test helpers

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018bfeq38ZBHxqfwJAgRaRH4

---------

Co-authored-by: Claude <noreply@anthropic.com>
2026-08-27 18:34:19 +00:00
Ben Grabow
d18a2df374 Fix flaky snowflake key sharing (#81197) 2026-08-27 17:33:41 +00:00
Chris Vandoodewaard
32ff8a7c7c paginate PR results for backport Slack reminder (#81216)
paginate PR results
2026-08-27 13:16:27 -04:00
Chris Truter
0be2972872 Make rollback-only app DB transactions actually roll back (#80065)
* Key cached session tokens on the app db

with-empty-h2-app-db! swaps in a different application database, and this cache was keyed on the username
alone. A token minted against the previous database names a session that does not exist in the new one, so
the request 401s, and the token minted inside leaks back out when the original is restored. The retry in
client-fn cleared the whole cache and tried again, which papered over it often enough to look like flake:
"Failed to authenticate X after three tries".

Key it on the app db as well, like every neighbouring cache does via memoize-for-application-db.

databases-list-can-upload-test and dismiss-spinner-test go from failing to passing; both were in the CI
failure list, the former across 12 jobs.

* Fix rollback-only app DB transaction semantics

* Reject rollback-only combinations that cannot roll back

- :nested-transaction-rule :ignore skips the savepoint :rollback-only needs, so the pair is rejected at any
  depth rather than silently committing
- a throw out of the transaction body now rolls the connection back explicitly, since restoring autocommit
  commits whatever is still open -- a requested rollback that failed was committing the work it had to discard
- the unsupported-options error names the options, and drops the "application database" framing that
  misattributed failures on other java.sql.Connections

* Restore transaction state even when the savepoint rollback throws

The callback accumulators are discarded unconditionally at the top of rollback!, so leaving the state
restore behind a throwing .rollback left the tree half-unwound: an outer scope that caught the error and
committed would run its before-commit callbacks against the failed nested scope's state.

* Create test-data Databases outside the caller's transaction

Their ids are memoized for the whole app db, so the Database has to outlive whatever scope first asked for
one. Created inside a test's rollback-only with-temp it is rolled back while the memo keeps its id, and every
later lookup hands out an id for a row that never existed.

do-outside-transaction is the same detachment run-after-commit-callbacks! already did privately.

* Don't cache a personal collection id created inside a transaction

The cache's premise is that a Personal Collection cannot be deleted, so its id never changes. A rollback
breaks that: the collection goes away and the cached id points at a row that never existed, which reads as
a 403 on every later request that resolves the user's personal collection.

* Skip the test-database cache in a transaction rather than detaching

Detaching made the Database durable but took a second connection while the caller's transaction was still
open. When that transaction holds cluster lock rows -- data_permissions and search index setup both take
them -- the second connection waits on them until the lock acquisition times out.

Bypassing the cache keeps the work on the caller's connection, and matches how the personal collection id
cache handles the same problem. do-outside-transaction goes with it, having no other callers.

* Stop the outer commit when a requested rollback failed

The full rollback only runs when this call turned autocommit off, so for a nested scope or a caller-managed
connection a failed rollback! just propagated. Code between there and the outermost scope can catch it, and
the commit branch would then commit the writes the caller asked to discard.

Mark the tree instead, and refuse to commit it. Throwing rather than quietly rolling back: the caller asked
to commit and cannot, and dropping the outer work silently trades one bug for a worse one.

* Tolerate a savepoint an outermost rollback-only can no longer roll back to

A statement in the body can commit the transaction under us -- DDL does that implicitly on H2 and MySQL --
which invalidates the savepoint. Those writes are durable by then, so failing the scope reports a problem
nobody can act on while leaving the data exactly where it was.

At the outermost scope, discard whatever is still pending and log where it happened. Nested scopes still
poison the tree and throw, since there the enclosing work is pending and must not be committed.

* Note the option of rejecting DDL inside a savepoint-bearing transaction

* Never commit a transaction whose rollback failed

Four review passes flagged the same hole from different angles. Restoring autocommit commits whatever is
still pending, and after a rollback we were relying on to discard those writes, that is exactly the work
we were told to throw away. Track it and leave autocommit off instead, letting the pool roll the
connection back on check-in.

Two narrower routes to the same place:

- rollback! now marks the tree itself, so an exception-driven rollback that fails poisons it too. Before,
  only a requested rollback did, and the error path could still reach a commit.
- the mark is rechecked after before-commit callbacks drain. A callback can catch a nested scope whose
  rollback failed, which lands after the earlier check has already passed.

Two tests changed meaning as a result: a nested scope whose rollback throws now stops the outer scope
committing, so it no longer completes and its own after-commit callbacks no longer run. That is the point
-- its writes are still there.

Also adds the regression test for personal collection id caching, models autocommit honestly in the mock
rather than pinning it true, and records what the in-transaction test-Database lookup gives up.

* Build the test-data Database before with-temp opens its transaction

Created inside a rollback-only scope it never survives, so it is rebuilt for every test, and the ids
memoized in metabase.test.data.impl outlive it. Cards then get inserted against table ids for rows that no
longer exist: FK_REPORT_CARD_REF_TABLE_ID on a fresh app db.

Materialise it before the transaction opens, where it can commit. The table and field lookup maps get the
same in-transaction bypass the Database id already had, for the case where a dataset is first asked for
inside a scope anyway -- neither is enough on its own.

Reproduce with a throwaway app db, which is what CI has and a local one does not:

  MB_DB_TYPE=h2 MB_DB_FILE=/tmp/fresh-$$ ./bin/test-agent :only '[metabase.queries.models.card-test]'

* Only pre-warm the dataset for the default driver

db-id resolves through the driver under test, so in a driver-test namespace the pre-warm reached for that
warehouse's dataset -- work a with-temp there may never need, against a warehouse that may not be up.
Driver failures went from 2 to 12 in the push that added it.

The FK problem it fixes is on the app-db side, for the default dataset, so confine it there. Best effort
even then: a dataset that cannot be built must not take an unrelated with-temp down with it.

* Evict temporary users from personal collection cache

* Evict the personal collection cache on the rollback, not on the user

Clearing the temp User's own entry misses the case that actually bites: a committed user's Personal
Collection is created lazily on its first permission check, which often lands inside somebody else's
with-temp. Nothing evicts it when that scope rolls back, and every later request for that user 403s.

Key off the with-temp boundary instead, and drop whatever it may have cached. Costs a re-lookup per user
afterwards; correct whichever user and whichever scope created the entry.

agent-api + activity-feed went from 21 failures to none.

* Skip the dataset pre-warm where the app db is deliberately empty

with-empty-h2-app-db! rebinds the app db, and the memo key includes its identifier, so the pre-warm missed
the cache and created a test-data Database inside a scope whose whole contract is emptiness.
databases-list-can-upload-test snapshots an empty Database set and saw two.

Bind *skip-dataset-prewarm?* there instead. Also drops the earlier :h2 restriction, which came from a bad
reading of two CI runs sampled at different completion stages: it could never have helped here, since those
helpers are H2, and it took the pre-warm away from driver tests that needed it.

Reproduce with a throwaway app db, which is what CI has and a local one does not:

  MB_DB_TYPE=h2 ./bin/test-agent :only '[metabase.queries.models.card-test]'

* Register the cache eviction with a unique key

metabase.test.redefs already holds `:around :default` on do-with-temp*, and methodical overwrites methods
sharing a key rather than composing them. Whichever namespace loaded last would win, silently dropping
either this eviction or the rollback-only transaction wrapper. It happened to work; it should not depend on
load order.

Also notes what a failed connection-level discard leaves behind. Only a caller-supplied connection is
exposed: on one we own, the same flag stops autocommit being restored, so the writes are never committed.
It logs rather than throws because throwing from that catch would mask the exception being unwound, and
master never attempted the discard at all, so a caller-supplied connection is no worse off than before.
Signalling it properly would mean telling the caller some other way.

* Note that session tokens are cached without regard for transactions

The user ids directly above skip the memo inside a transaction, since users created in a `with-temp` are
discarded with it. The token cache has no equivalent guard, so a session rolled back with its scope leaves a
token behind and the request fails with "Failed to authenticate X after three tries". The retry in
`client-fn` hides it more often than not.

* Note that a rollback can leave a stale personal collection id cached

"A Personal Collection cannot be deleted" does not hold for a transaction that rolls back. Tests evict on
the with-temp boundary; production has no equivalent and keeps the id for the rest of the TTL. Accepting
that for now -- it predates honoring :rollback-only, and closing it properly wants an after-rollback hook
alongside do-before-commit/do-after-commit.

* Follow the created-globally rename in the token cache TODO

* Build the query for export inside the guard, not beside it

`:query_json` already returns nil for a card whose query cannot be exported, but `lib/query` was called in the
argument position, outside `try-export-query`. A stored `dataset_query` with no stages builds a query that fails
its own output schema, so one such Card takes down every response that walks a set of Cards -- the typed-schemas
endpoint among them.

metabase.search.appdb.scoring-test alongside metabase.typed-schemas-rest.api-test goes from 13 failures to 0.

* Don't run a test request inside a rollback-only transaction

`user-request` takes a temporary Session when it is handed a User or an id rather than a `:rasta`-style keyword,
and the handler runs in-process on this thread. Now that the with-temp rolls back for real, every app db write
the request made is discarded when it returns -- the warehouse table writes survive on their own connection and
the API still answers 200, so the test sees a successful call that changed nothing.

Insert the Session and delete it in a `finally` instead. A caller already inside a transaction still gets the
same behaviour it always had; what goes away is the extra rollback scope around the request itself.

metabase-enterprise.action-v2.models.undo-test goes from 25 failures to 0, passing all 134 assertions, which
matches master.

* Mint the test users' sessions before anything opens a transaction

`authenticate!` ran about 150 times in a single `pulse.api.pulse-test` run, nearly always inside a transaction.
A session minted inside a `with-temp` is rolled back with the scope, the next request 401s, and the retry in
`client-fn` cleared the whole cache -- every user's token, not just the dead one -- so everyone re-authenticated
in-transaction and the cycle repeated. Each of those inserts bumps the user's `last_login` through the
`:model/Session` after-insert hook, which is what breaks a test that reads the creator before making its request.

Mint the four test users' sessions at initialization, where nothing has opened a transaction yet, so they are
durable. On a 401 forget only that user's token, so one temporary user's rolled-back session cannot evict them.

pulse.api.pulse-test goes from one failure to none, matching master. publish-refresh-error-event-sends-admin-email-test
passes too, where it fails standalone on master: that email only goes to admins with a `last_login`, and
crowberto's was rolled back with every login.

* Ask for the FieldValues the chain filter test needs

The full FieldValues row for `categories.name` is created lazily, and `update-field-values-for-table!` only
refreshes rows that already exist -- so a run that creates it inside a `with-temp` discards it and nothing
makes another. `select-one-pk` then returns nil and the test dies on "Not something with an ID".

Same shape as the rest of this branch: a durable row created on demand, now rolled back with the scope that
created it. Ask for the row instead of assuming an earlier test left one behind.

* Run the metric leaf queries one at a time inside a transaction

A transaction owns a single connection and `future` hands that binding to every thread it starts, so the
leaves' app db writes -- used-card stats behind a cluster lock, query rows -- interleave on one session.
Savepoints are per session: a thread rolling back to its own destroys the ones its siblings are holding, the
next release fails with "savepoint does not exist", and the transaction aborts, taking the rest of the test
with it. That is why api-arithmetic-test loses a different test on each CI run.

Only tests get here. A request runs in no transaction, where each future takes its own pooled connection and
the parallelism is worth having.

On a throwaway Postgres the namespace errors on two tests per run before this and passes three runs out of
three after.

* Commit a lock row that would otherwise roll back with the caller

A lock row is written once and read forever after, and acquisition leans on that: rival first-time inserters
block on the winner's uncommitted unique-index entry and retry once it commits. Inside a `with-temp` that
commit never comes, so the row is gone again and the next acquisition replays the race -- on MariaDB until the
lock-wait timeout expires.

Insert it on a connection of our own when the caller was already in a transaction, which is the only case
where the row would not survive. Nothing in production takes a cluster lock from inside another transaction,
so this leaves that path alone -- worth keeping that way, since lock names are dynamic in places and rows that
used to roll back with a failed transaction would otherwise persist.

Reasoning rather than measurement; the TODO on the function says so and says what to do if it does not help.

* Bound the out-of-band insert on its own connection

`set-query-timeout!` returns false on MySQL running MariaDB Connector/J, and the fallback sets
`innodb_lock_wait_timeout` on the session -- but `acquire-lock-row!` sets it on the caller's connection, not
the one the insert opens. Blocked by a rival's uncommitted row, the insert would then wait out the server
default instead of the second it was given, on exactly the databases this is meant to help.

Give it the same treatment on its own connection. Also note in the docstring that the second connection is
only taken until the row exists, which is what bounds the cost of holding two at once.

* Resolve the venues id before counting queries

The cache is fine; the query the test counts is its own `(mt/id :venues)`. Inside a transaction the test-data
id lookup deliberately skips its memo, so the call lands in the window that is supposed to see none.

* Keep the pre-warm out of app dbs that are meant to be empty

`with-empty-h2-app-db!` already opts out of the dataset pre-warm; the serialization test dbs and the
schema-migration ones never did. A `with-temp` inside them materialised the test-data Database, and an extract
then saw a Database no assertion expects -- serialization.v2.load-test/database-test wanted one and found two.

* Do not insert a lock row out of band outside tests

The claim in "Commit a lock row that would otherwise roll back with the caller" was wrong: production does take
cluster locks inside another transaction. `load-from-h2!` inserts Databases, and so takes the batch-permissions
locks, inside one big load transaction. Our fresh connection then blocked on a next-key gap lock the caller's
own SELECT ... FOR UPDATE was holding, and a thread cannot outwait itself -- it timed out, retried, and failed.
That is what broke dump-to-h2, load-from-h2 and rotate-encryption-key on MariaDB.

Gate the out-of-band insert on `config/is-test?`, which is the only place a transaction may never commit, so
production goes back to exactly what it did before. Keep a fallback for the same shape inside tests, where
`load-from-h2!` can block on itself just as readily: on a lock-wait timeout, insert on the caller's connection.

* Stop three card tests leaving rows behind

`updating-model-query-does-not-shift-metadata-overrides-test` had no cleanup at all, and
`create-card-with-type-and-dataset-test` opened a plain `t2/with-transaction` where its siblings pass
`{:rollback-only true}` -- so both committed cards with a `{}` dataset_query that every later test walking the
card table then had to survive. That leak is older than this branch; honoring the rollback is what makes the
second fix work.

`get-series-for-card-permission-test` read crowberto's personal collection with `select-one-pk`, and a nil
collection_id puts the card in the root collection, which rasta can read -- so the test quietly asserted 403
against a card that was never private. Ask for the collection instead.

* Truncate the heartbeat fixture times to milliseconds

Postgres stores microseconds, so a timestamp ending .7369995 comes back a millisecond above its own
millis-truncation and the assertion misses by one.

* Tolerate a card whose query will not parse when finding related entities

Nothing at the schema level stops a stored `dataset_query` being empty, and such a card simply has no
context-bearing forms. Without the guard one of them takes down the related-entities computation, and with it
every x-ray, for everyone else. Same treatment `card-details` already gives an unexportable query.

* Give the end-to-end schema test a Database of its own

The test asserts exactly which questions, metrics and models fall inside the test-data Database's scope, which
holds only as long as no other namespace commits a Card against that Database. Two such leaks have been fixed
on this branch already and there is at least one more; the assertion cannot be made safe by chasing them,
because any test added later can break it again.

Run it against a copy of the Database instead, so nothing else can write there.

A throwaway namespace that commits one Card on the test-data Database ahead of it reproduces the CI failure
exactly (core_test.clj:147) and passes with this change.

* Recognise the MySQL lock-wait timeout in the in-band fallback

`lock-wait-timed-out?` knew about `SQLTimeoutException` and our own `::acquisition-timeout` marker, but not
ER_LOCK_WAIT_TIMEOUT itself. The server raises that on its own schedule, so on MySQL and MariaDB -- the
databases the fallback exists for -- it could arrive as a plain `SQLException` and the fallback would not fire.
Check the code through the cause chain.

* Take the revisions with the cards

`with-model-cleanup` deletes rows directly, which skips the Card hook that would have removed the revisions
this test's POST and two PUTs create. Name Revision in the cleanup as well.

* Truncate once, and align the chain filter's temp values

The second `truncate-to` could not do anything the first had not.

* Create the test users before the related tests race for them

`user->id` skips its memo inside a transaction and creates the users there instead, so on this branch they roll
back with the scope and the next test makes them again. These tests are parallel and every one of them takes a
`with-temp` Card whose creator defaults to rasta, so several threads insert the same user at once and all but
one time out on the row lock. The namespace goes from three errors to none.

Also covers the guard in `related/definition`: a Card whose stored query will not parse contributes no
context-bearing forms, and a genuinely similar Card still outranks it.

* Cover the card whose query will not build

The guard in `card-details` had no test: a Card with a stage-less stored query answers without `:query_json`,
and the Card beside it in the same response is unaffected.

* Cover the batch path and the lock-wait code

The first version of the card test called `get-table-details` once per card, which cannot see the failure it
claims to cover -- the bug was that one bad card killed the whole `cards-details` seq the others were in. Go
through that seq instead. Removing the guard turns the test red at the same malli output error CI was showing,
which the per-card version did not.

Also cover `lock-wait-timed-out?`: MySQL's ER_LOCK_WAIT_TIMEOUT on its own and inside a cause chain, and any
other error still propagating rather than being taken for a timeout.

* Create the temp search index before the transaction, not inside it

`with-temp-index-table` runs CREATE TABLE and DROP TABLE on the ambient connection, and on H2 and MySQL that
commits the transaction around it. Six tests opened it inside a `with-temp`, so the rollback had nothing left
to take back and every row they created leaked -- a Transform, some Cards and Collections, a published Table, a
Document -- to any later test that walks the app db.

That one leaked Transform is the whole of the serialization failures: a collection-scoped export includes every
Transform in the instance, so all four tests came back with one entity more than they expected.

Take the index table before the `with-temp` instead. The nested scope reuses it, so nothing else moves.

`metabase-enterprise.search.scoring-test` also had no fixtures at all, and fails on its own: the first lazy
`initialize :db` fires mid-test and recreates the H2 database underneath a test already writing index
bookkeeping.

The 133 namespaces that run before serialization.api-test in the Enterprise job reproduce the CI failures
exactly -- 5011 assertions, 5 failures -- and pass with this, at 4714 assertions.

* Say the cluster lock comments plainly

Lead with the point rather than building up to it, and drop the clause chains. No behaviour change.

The TODO above `clear-cached-session-tokens!` was stale: it asked for the `created-globally` treatment, which
landed earlier on this branch. Point at what is actually left instead -- initialization can still first happen
inside a `with-temp`, and then the sessions roll back with it.

* Keep the non-index assertions where the app db cannot index

`with-temp-index-table` is a `when`, so wrapping a whole test in it skips the whole test on MySQL and MariaDB.
Two of the tests moved in the last commit also assert things that need no index, and those assertions went
quiet there. Add a sibling that runs the body either way, and use it for those two.

`search.appdb.scoring-test` had no initialization fixture, the same gap its enterprise counterpart had: the
first lazy `initialize` can recreate the H2 db underneath a test already writing index bookkeeping.

* Sort the fixtures require

* Download through the client that retries a 401

`downloaded-card-filenames-test` passed a raw cached token straight to `client-full-response`, so it skipped
the retry every other test gets from `client-fn`. A session rolled back with the scope that made it then came
back here as a bare 401, which is what the BigQuery job has been failing on.

`user-http-request-full-response` still returns the headers the test reads.

* Improve rollback test isolation and comments

* Do not poison the transaction tree when the savepoint is already gone

MySQL and MariaDB roll back the whole transaction to break a deadlock, discarding every savepoint with it. The
ROLLBACK TO SAVEPOINT that follows fails with "SAVEPOINT does not exist", and marking the tree rollback-required
there is backwards: the server has already discarded the writes, so nothing is pending for an enclosing scope to
commit by mistake. A rollback-only scope whose DDL committed implicitly already gets this treatment.

The refusal message named a rollback-only transaction, but the ordinary error path sets the flag too.

Also validate :nested-transaction-rule with a throw. It is a public option, and `assert` compiles away under
`*assert*` false. Sorting the unsupported keys keeps the message stable across runs.

* Keep the test request's session out of the login hooks

:model/Session's after-insert hook publishes :event/user-login, which stamps User.last_login, and
:event/user-joined for a User that has never logged in. Deleting the Session afterwards undoes neither, so every
user-http-request left both behind for whatever ran next.

Write the row instead. created_at has to come along, since the model supplied it.

* Ask whether the app db can hold the index, not whether any engine can

supports-index? is true for any active engine. With semantic search on MySQL or MariaDB, the macro that exists to
run the body without an app-db index took the app-db branch and tried to create the table anyway.

* Say something when a stored query is dropped

Both guards swallowed every exception silently, so a Card whose query failed to build for an unrelated reason
looked exactly like one that has no query. try-export-query, right next to the Metabot guard, already warns.

Related entities logs at debug: it runs for every candidate Card while ranking.

* Commit the out-of-band lock row rather than assume autocommit

The docstring said the row was committed, but nothing committed it -- the insert relied on the pooled connection
arriving in autocommit, which the pool does not promise. The caller's SELECT cannot find the row otherwise.

* Let a failed dataset pre-warm be heard

Now that the pre-warm runs for every driver, a warehouse that is down fails it on every top-level with-temp. At
debug that is invisible on a normal run and resurfaces much later as a Database that does not exist. Warn once
per message.

Also note that mt/id costs a query inside a transaction, which is what a with-call-count window has to plan for.

* Polish prewarm comments

* Mark the tree rollback-required even when the savepoint vanished

A vanished savepoint does not mean nothing is pending. DDL committing implicitly, or a server breaking a deadlock,
ends the transaction and discards the writes so far -- but anything written afterwards sits in a fresh transaction
and is still pending. Skipping the flag there let an enclosing scope swallow the rollback error and commit exactly
those writes.

Say so at the point of the catch, since the reasoning does not survive a quick read.

* Gate every app-db index helper on app-db support

with-temp-index-table and with-appdb-search-if-available-otherwise-legacy still asked supports-index?, which is
true for any active engine. With semantic search on MySQL or MariaDB they ran app-db DDL the app DB cannot take,
including from scopes nested inside the fallback body that exists to avoid exactly that.

Cover the borrowed session too: it must leave last_login empty, and delete itself afterwards.

* Cover the out-of-band lock row against a connection in a transaction

Hands the dedicated connection over with autocommit off and checks the row survives the caller's rollback. Without
the explicit commit the insert dies with the connection, and acquisition then fails on the row it just wrote.

Runs on MySQL and PostgreSQL only; H2 takes an in-process lock and writes no row.

* Pin the vanished-savepoint case against the outer commit

A savepoint the server already discarded says nothing about what was written after it. The existing rollback tests
throw generic exceptions, so nothing stopped the SQLState 3B001 / MySQL 1305 exemption from coming back.

* Pin the engine in the otherwise-legacy fallback

Where the app DB cannot hold an index, the fallback took whatever default engine was active. Semantic can be
active there, and the bodies are written for an index the app DB does not have. Run them under legacy search, as
the macro's name says. with-legacy-search moves above its first use.

* Polish rollback and search comments

* Stop two remapping scopes deadlocking on MySQL and MariaDB

A remapping writes a Dimension for a Field the whole suite shares, and holds it until its scope ends. Two threads
doing that at once deadlock on the unique index over dimension.field_id: each holds the record lock the other's
uniqueness check wants, while asking for the insert-intention gap in front of it. MySQL and MariaDB pick a victim
and roll it back, which drops its savepoints, so the rollback that follows fails on a savepoint that is gone.

Serialise remapping scopes against each other. Tests that never remap keep running in parallel alongside them.

The helper already carried a TODO saying it was not thread-safe.

Reproduced on MariaDB 12.3 and confirmed on master, so this predates the branch. custom-values-test failed nearly
every run before, and three runs in a row are clean after. Verified on H2 as well.

* Serialise every temporary Dimension, not just remapping scopes

Remapping is not the only way a test writes a Dimension for a shared Field: card-test, card/metadata-test,
field-values-test and data-studio's table-test all create them directly, in parallel, on the Orders and Products
fields. Those still raced with the remapping scopes.

Move the lock to `do-with-temp*` for :model/Dimension so every site is covered, including ones added later.
`locking` is reentrant, so a `with-temp` holding several Dimensions still works.

* Take the search index out of six more with-temp transactions

Same fix as the earlier pass over the search tests, for the ones that pass a warehouse driver: their index scope
sits inside `with-temp`, so `drop-table!` runs DDL on the ambient connection. H2 and MySQL commit the transaction
to do it, which makes the temporary Transforms durable and destroys the savepoint. The rollback that follows can
no longer take them back, and it discards toucan's compensating deletes along with everything else still pending,
so the Transforms outlive the test.

Three tests later in the same namespace count transforms and saw the strays: root-collection-items-include-
transforms-test expected one and found four.

These were missed the first time because `mt/normal-drivers-with-feature :transforms/table` skips them on H2, so
only a driver run reaches them.

Reproduced with a MariaDB warehouse: transform-test went 3 failures to 0, and the four namespaces that failed
together now match master at 2532 assertions. H2 is unchanged at 109.

* Let the failed-rollback warning reach the console

The shared Console appender only accepts FATAL, so the warning naming a rollback-only scope that could not roll
back went to logs/test-log.json and nowhere a CI run gets read. It took hours to find six tests leaking rows
because of it; the warning had been firing the whole time.

Give that logger an appender that accepts WARN. The message carries the mb-test key, so it names the test that
broke its own isolation rather than the later one that trips over the rows.

* Hoist the search index scope above with-temp in two document search tests

Both tests opened `with-appdb-search-and-legacy-search` inside `mt/with-temp`.
That scope creates and drops the index table on the ambient connection, and on H2
and MySQL the DDL commits the transaction `with-temp` meant to roll back, so the
temp Documents leaked into later tests.

Found by the throw this branch adds: both showed up as errors at connection.clj:293
in the app DB test jobs.

* Serialize the reduced-fields card test

It holds a rollback-only savepoint across two `with-temp` Cards and an API call. Run in parallel, something else
commits the transaction that savepoint lives in, so both Cards are discarded and the downstream query reports its
source Card as missing.

Only MySQL surfaces it, and only under a full suite: the test passes on its own, and six local configurations could
not reproduce it. Confirmed by running the MySQL and MariaDB driver jobs with this single change -- all four green,
against four runs where they failed.

* Materialize the test users' Personal Collections before with-temp opens

`user->personal-collection` is a get-or-create and sits on the permissions path every API request takes. A
namespace that does not initialize `:test-users-personal-collections` creates Lucky's the first time a request
calculates permissions -- which happens inside a `with-temp` -- so the transaction rolls it back. Every later
request tries to create it again, and the parallel ones time out on each other's COLLECTION table lock:

  GET /api/database expected a status code of 200, got 500.
  Timeout trying to lock table; INSERT INTO COLLECTION (..., "Lucky Pigeon's Personal Collection", ...)

That is what failed `databases-list-include-saved-questions-test` on the Mongo, Redshift and Athena jobs -- the
empty database list was the 500, not a visibility problem.

Prewarm them next to the dataset, under the same skip flag, so helpers that need an empty app DB are unaffected.
This also closes the related hole where `:test-users` itself first initializes inside `with-temp`, since that
component initializes it.

Confirmed on the Mongo driver jobs: MongoDB Latest went from failing to green.

* Update the test-user session note now that initialization is prewarmed
2026-08-27 19:05:06 +02:00
bryan
13dde0bba3 Reject unneeded connection properties in SQL-JDBC additional-options (#81097) 2026-08-27 10:29:18 -06:00
Nathan Voxland
c24f084ae6 Improve SVG handling (#80249) 2026-08-27 11:25:08 -05:00
Andrei Chernykh
e1fbadec0b Remove unused raf dependency (#80931) 2026-08-27 09:17:21 -07:00
Andrei Chernykh
ad48fed046 Remove the direct reselect usage and dependency (#81034) 2026-08-27 09:16:17 -07:00
Andrei Chernykh
7c08b5d95c Remove cypress-each (#81086) 2026-08-27 09:15:55 -07:00
Andrei Chernykh
25ed6aac2b Swap merge-refs for Mantine's mergeRefs (#81104) 2026-08-27 09:15:15 -07:00
Andrei Chernykh
5c169f142a Replace react-innertext with a local helper (#81105) 2026-08-27 09:14:44 -07:00
Chris Truter
42cf92479a Ratchet config-level kondo suppressions too (#78217)
* Ratchet config-level kondo suppressions too

:config-counts in ratchets.edn tracks, per linter, the :off switches and
:exclude entries in .clj-kondo/config.edn (top-level and per ns-group).
Only weakening counts: adding discouragements or turning a linter on is
free. Lowering works like the inline budgets; raising is a hand edit.

* Count every excluded item and :config-in-comment in the config ratchet

Map-shaped :exclude values with vector entries counted as one suppression
({compojure.core [GET DELETE POST PUT]} is 4, not 1), and the
:config-in-comment :off switch wasn't aggregated at all, so both could
grow without tripping CI. Reseeded.

* Count :config-in-call suppressions, and self-heal on config.edn

`:config-in-call` is a scoped `:linters` map like `:config-in-ns`, so a
linter switched off inside one was invisible to the ratchet. Both scopes
now fold into the same count.

The self-heal workflow only watched Clojure sources, which was right while
budgets came from inline ignores alone. Now that `.clj-kondo/config.edn`
feeds `:config-counts`, removing a suppression there can leave a budget
stale with no way to trigger the fix.

* Count scoped :off suppressions at any nesting depth

Linter scopes nest more than one level: :discouraged-java-method keys by
class and then by method. Counting only the first level meant an :off one
map further down waived a warning the ratchet never saw. No budget moves --
the repo's only two-deep entry is a :level :error discouragement.

* Regenerate kondo ignore budgets after restacking onto master
2026-08-27 17:40:39 +02:00
Kevin Kelleher
fc487f06c9 Explorations summary document (#78848)
Co-authored-by: Nathan Voxland <nathan@metabase.com>
2026-08-27 11:32:54 -04:00
github-automation-metabase
6d2c060ef1 Fix flaky document-comment split test (#77391)
* Fix flaky document-comment split test by using realClick on hover-revealed button

The test clicked the hover-revealed "new thread" comment button of the
second paragraph via Cypress's synthetic .click(). After realHover() moved
the cursor to reveal the button, the synthetic click could act on the button
while its hover transition was unwinding, so the click did not reliably
trigger the route navigation that opens the comments sidebar — leaving
getSidebar() to time out.

Switch to .realClick(), which moves the real mouse onto the button (keeping
it visible/stable) and dispatches a real click that reliably fires the link.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* Wait for document save to persist before opening comments sidebar

The split-paragraph test opened the comments sidebar immediately after the
Save button disappeared, but a successful save schedules a deferred
navigation back to /document/:id. Under network throttling that navigation
resolved after the sidebar route had opened, clobbering it, so the sidebar
never mounted and getSidebar timed out. Wait on the PUT request and the
"Document saved" toast so the post-save navigation settles first.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-08-27 17:28:43 +02:00
Nicola Mometto
f8ce01fcca Check the email_verified claim in the generic OIDC provider (#80269)
Google SSO already requires the id-token email to be verified; the
generic OIDC provider (and slack-connect, which derives from it) did
not look at the claim at all.

Tokens with email_verified present and not true are now rejected. An
absent claim is still accepted: it is optional per OIDC Core and some
IdPs never send it.
2026-08-27 16:16:12 +01:00
Merlin-S3NS
e433af3f90 Fix: Support Google Cloud Universe Domain in BigQuery driver (#68607)
* Fix: Support Google Cloud Universe Domain in BigQuery driver

* feat(bigquery): Add universe domain to connection properties UI

* feat(bigquery): improve universe domain helper text

* Update bigquery_cloud_sdk.clj

Indeed we are out of sync as the impersonating was removed from master branch.

* removed empty comment

* Fixing formatting
2026-08-27 10:57:23 -04:00
Nicola Mometto
57f6f37864 Match SSO group names by exact string value when syncing memberships (#80615)
Ignore non-string SSO group names when syncing group memberships
2026-08-27 15:45:51 +01:00
github-automation-metabase
2ef0e4d264 Fix flaky dashboard bulk-move empty-state test (#78457)
Wait for the question move PUT to settle before navigating to the
affected dashboard, so its empty state is asserted only after the
dashcard has actually been removed.
2026-08-27 16:44:22 +02:00
Nemanja Glumac
c090eb292a Delete redundant e2e spec (#81174) 2026-08-27 16:03:30 +02:00
Kevin Kelleher
9e329ce6fe Ensure highlighted bars are included in brush drill filter (#80025)
* ensure highlighted bars are included in brush drill filter

* fix e2e

* fix e2e again, December 2026 label was not appearing maybe due to a screen size flake
2026-08-27 13:53:32 +00:00
github-automation-metabase
0d36304bf5 Stabilize scatter tooltip test against ExplicitSize resize flake (#78458)
The 'adding non-series columns to the tooltip' scatter test hovered a
bubble immediately after render. The ExplicitSize throttle re-renders the
chart once its container is measured, moving bubbles out from under the
realHover() before the tooltip fires, so [data-testid=echarts-tooltip]
was intermittently never found (4000ms timeout).

Apply the same view-toggle stabilization the sibling tooltip test already
uses (extracted as stabilizeScatterChart) before each hover, forcing a
re-render against an already-measured container.

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-08-27 15:29:26 +02:00
Chris Truter
dd1cb0f0ba Require justifications for linter ignores (#77820)
* Require a justification comment above inline ignores, ratcheted per linter

An ignore of a linter outside :comment-exempt needs a substantive ;;
comment on the line above or trailing on the same line (bare ;; dividers
don't count, and no attempt is made to check the comment is about the
ignore). ~18% of ignores have one today, so 35 linters start exempt.
The set only shrinks: once a linter's last uncommented ignore gains a
comment, the fixer drops its exemption and CI holds the line.

* Locate trailing justification comments in masked source

A semicolon inside a string or character literal on the ignore's line
counted as a trailing comment, so an unjustified ignore could pass.
The masker now keeps the ; that starts a comment (and blanks the char
after a backslash), so justified? finds only real comment starts.

* Share one tree scan across the ratchet tests

Each ratchet test walked and slurped the whole source tree; a delay
runs the scan once for all three.

* Invalidate the shared tree scan per test run

The namespace-level delay cached the scan for the JVM's lifetime, so a
long-lived REPL compared fresh ratchet-file contents with stale counts.
The :once fixture now resets the cache; a directly-run test var scans
fresh.

* Scope the shared tree scan to the fixture's lifetime

The atom cache survived a run, so a later directly-invoked test var
(which skips :once fixtures) reused a stale scan. The cache now holds a
delay only while the fixture is live; without it every call scans fresh.

* Require canonical syntax for kondo ignores

Keep ignore detection deliberately lexical and fail closed: every real marker must use the key-first map form, otherwise the ratchet rejects it with an actionable error. This avoids maintaining a partial Clojure reader for the two attr maps that previously placed the ignore behind another key.

Justification comments must be substantive and either directly above the ignore or trailing on its line. Authenticate comment openers in masked source so strings and code-line comments cannot pose as justifications.

* Harden kondo ignore scanning edge cases

* Use reader delimiters for kondo marker boundaries

* Match Clojure reader whitespace in kondo markers

* Reserve clj-kondo namespaced map syntax

* Harden kondo scanner diagnostics

* Rebuild the budgets and exemptions on the named-linter base

Rebased onto the ignore-naming branch, so every budget is recomputed from
the tree and :comment-exempt is reseeded with exactly the linters that
still have an uncommented ignore: 38, up from 35. The three extra are
linters that were hidden inside bare `#_:clj-kondo/ignore` forms until
they were spelled out; :all itself drops out, having no ignores left.

* Point at :comment-exempt after inserting ignores for a new linter

kondo-insert-ignores writes comment-less ignores, which the justification
ratchet rejects for non-exempt linters.

* Leave ignores preceded by a # for a human, and gate the exemption advice

A `#` immediately before the matched form is ambiguous: in `#^{...}` it is
legacy metadata that removal must take with it, in `x#^{...}` it closes a
syntax-quote gensym that removal must leave alone. Telling them apart means
reading Clojure, which this scanner deliberately does not do -- so report
the site under :skipped, as it already does for prefixless maps and
unbalanced braces, and let a human excise it.

Separately, kondo-insert-ignores no longer advises a :comment-exempt entry
unconditionally. An inserted ignore lands under whatever comment already sat
above the flagged form, so a linter whose sites are all justified would get
an exemption that no-stale-exemptions-test then rejects.

* Align the scan-test expectations

The occurrence vector's :justified? column and the counts map both drifted
out of alignment as linters were added to the fixture.

* Regenerate kondo ignore budgets after restacking onto master
2026-08-27 15:16:59 +02:00
Chris Truter
ec3b24fadd Make every kondo ignore name the linters it suppresses (#79531)
* Make every kondo ignore name the linters it suppresses

The vector-less `#_:clj-kondo/ignore` suppresses every linter on the next
form, so a site added to silence one warning quietly absorbs any other that
appears there later. Replace all 50 with explicit linter vectors.

The budgets absorb what the bare forms were hiding: :discouraged-var
106 -> 129, :unresolved-namespace 3 -> 9, :type-mismatch 2 -> 6,
:unresolved-symbol 4 -> 9, plus first entries for :consistent-alias,
:duplicate-require and :redundant-do. :all drops out entirely, so a new
bare ignore now fails the ratchet with no budget to land in.

* Use the mdb.env alias in app-db-as-data-warehouse

Var-quotes resolve namespace aliases, private vars included, so the
fully-qualified name was never needed. Dropping it retires the
:aliased-namespace-symbol ignore and the unused-alias warning the
workaround caused.

* Regenerate kondo ignore budgets after restacking onto master
2026-08-27 14:23:09 +02:00
Jakub Chodorowicz
7eba2629f2 harden host event listener access across membrane (#81036) 2026-08-27 14:02:06 +02:00
Jakub Chodorowicz
c433278e06 Gdgt 2472 loader (#80586) 2026-08-27 11:53:29 +02:00
Oleh K
11be7cb3e7 Restyle NavLink with primary and secondary variants (#80640)
* Add primary/secondary NavLink variants and sibling components (GDGT-2469)

Restyle the design-system NavLink from Figma:

- Add `primary` (brand-fill selected) and `secondary` (subtle gray selected)
  variants, with Default/Hover/Pressed/Selected states.
- Add the `icon-{primary,secondary,disabled,brand}-inverse` color tokens.
- Add `NavLinkButton` and `NavLinkBadge` sibling components for the right
  section; both invert automatically on a selected `primary` item (no prop),
  the button also has its own hover/press and stops click propagation.
- Give NavLink a `leftIcon` prop so it renders and colors the leading icon
  itself instead of styling the section slot.
- Add a two-variant state-matrix Storybook showcase (Loki-covered).

* Migrate NavLink call sites to primary/secondary variants (GDGT-2469)

- Admin nav: `admin-nav` -> `secondary`
- Entity picker: current level -> `primary`, other levels -> `secondary`;
  search results -> `primary`
- Remove the now-unused `default`/`mb-light`/`admin-nav` variant CSS

* Revert NavLink `leftIcon` prop back to `leftSection` (GDGT-2469)

Drop the polymorphic NavLink wrapper and colour the leading icon via the
`.NavLinkSection[data-position="left"]` CSS overrides again.

* Update Loki Snapshots

* Cleanup

* Update Loki Snapshots

* Add Loki reference for the NavLink Overview story (GDGT-2469)

* Update Loki Snapshots

* Address NavLink review comments (GDGT-2469)

- Migrate the remaining `default` NavLink (embedding SimpleDataPicker) to
  `primary`, restoring its selected styling
- Render NavLinkButton as a `<div role="button">` (valid inside a NavLink's
  `<a>`) with keyboard activation, instead of a `<button>`

* Update Loki Snapshots

---------

Co-authored-by: Metabase Automation <github-automation@metabase.com>
2026-08-27 11:42:14 +02:00
Oleksandr Yakushev
cad622a164 [test] Extract and reuse deferred str mechanism for testing macro (#79932) 2026-08-27 10:50:01 +03:00
Ben Grabow
92fdc53695 Only setup tracking db once per jvm (#80960)
* Only setup tracking db once per jvm

* kick ci
2026-08-27 04:53:59 +00:00
Alexander Polyankin
c3dddcfcc6 Encrypt public_uuid at rest, resolved via an indexed prefix (#80929) 2026-08-26 21:08:08 -04:00
Tyler Plude
31e8463f0f [BOT-2019] Harden Metabot SQL editor access (#80544)
* harden sql access

* remove hardcoded sql capability from document context

* make SQL permission denials terminal and visible

* make read-check denials terminal and gate document_schema_collect
2026-08-26 18:06:39 -07:00
Romeo Van Snick
afa13457ae Reset transforms-enabled before the transform permission tests (#80996)
* Reset transforms-enabled before the transform permission tests

* Clear transforms-enabled rather than setting it false
2026-08-27 10:55:52 +10:00
Denis Berezin
7692d3f479 UXW-5117 Fix Timeline event pills displayed on a wrong date when merged (#80567)
* UXW-5117 Implement timeline events stacks

* Improve performance

* Fix edge cases, improve performance, fix issues

* Review fixes

* Place chips at real data point

* Self review
2026-08-27 03:05:06 +03:00
Romeo Van Snick
0a601f07c6 Keep entity identity stable when a write changes nothing (#80976)
* Keep entity identity stable when a write changes nothing

* Skip null field writes in the tables reducer
2026-08-27 09:23:39 +10:00
Romeo Van Snick
cf3200aa96 Load the template tag's field from the API, not the metadata mirror (#80974) 2026-08-27 09:22:55 +10:00
Romeo Van Snick
f5b9b156e8 Move the data reference panes off the metadata mirror (#80973) 2026-08-27 09:22:03 +10:00
Romeo Van Snick
518f75d5f5 Read the picked table's database from the RTK cache, not the metadata mirror (#80972) 2026-08-27 09:21:43 +10:00
Kevin Kelleher
cc1d5a93d1 Preserve timelines modal stack on close (#81037)
turn flat timelines routes into a tree
2026-08-26 19:03:55 -04:00
Andrei Chernykh
3a824943f0 Remove mockdate dependency (#80941) 2026-08-26 15:17:25 -07:00
Ben Grabow
3d7eed5d4a Split postgres driver tests (#81111) 2026-08-26 16:36:55 -05:00
technomancy
cb7114ff8a Stop duplicating databricks test jobs for multi-catalog (#80957)
Stop duplicating databricks test jobs.

In https://github.com/metabase/metabase/pull/57148/changes we added
support for databricks multi-catalog feature. In this PR the existing
databricks job was duplicated, so every databricks test ran twice. The
rationale for this (and more broadly the rationale and background for
the entire feature) was never explained.

The code which looks at the multi-catalog flag looks to be tested in
the metabase.driver.databricks-test test. So running the entire driver
suite all over again just with that one flag set differently looks to
be very wasteful.

We have been putting a lot of load on databricks recently with the
extreme levels of backporting we're doing, which has led to periods
during which 99% of databricks jobs time out. The fact that we're
running twice as many tests as we need to is surely a contributing
factor to this.

It would be even better if the multi-catalog functionality were tested
inside the main test suite instead of spun off into a separate job,
but I want to focus on getting it stable first.

Also we split databricks into 3 jobs, but since the bottleneck is in
the DB itself, this was counter-productive. I've brought it back to 2
jobs like the other cloud databases.
2026-08-26 13:59:16 -07:00
Krishn
af1df1432c Document background task and run statuses (#80578) 2026-08-26 16:54:52 -04:00
Nemanja Glumac
d35b91c92c Run CI only for the base of a stack of PRs (#81098) 2026-08-26 22:13:51 +02:00
Andrei Chernykh
bceebfaee4 Remove dead React 17 compatibility (#81080)
Remove dead React 17 compat
2026-08-26 12:21:23 -07:00
Alexander Polyankin
55d7df2c9c Improve encrypted app-db column handling (#80785)
* Reject plaintext values in encrypted app-db columns via a strict encrypted read

Make the canonical encrypted read path strict: when MB_ENCRYPTION_SECRET_KEY is set,
a value that is not encrypted is rejected instead of trusted, so a value written
directly via SQL cannot stand in for a properly encrypted one. This relies on the
existing aes256-cbc-hmac-sha512 authentication of encrypted values.

- encryption/maybe-decrypt is now strict; the previous lenient behaviour is
  maybe-decrypt-accepting-plaintext. Every existing lenient caller (settings,
  secrets, key rotation, the encryption-check sentinel, migrations) uses the
  accepting-plaintext variant explicitly.
- models.interface: encrypted-json-out / transform-encrypted-json are the only
  (strict) variants. The explorations encrypted-text/EDN transforms are strict too.
- app-db.encryption: the rotation list (encrypted-columns) now also includes the
  exploration_query_result columns, so key rotation and encrypt-db cover them.
- A v63 clojure migration (EncryptEncryptedJsonColumns) encrypts any plaintext
  values left in the encrypted-json columns (e.g. the v58 credentials SQL backfill)
  so strict reads succeed after upgrade.

Secrets and settings keep the lenient path for now and will be handled in follow-up PRs.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* Use explicit arities for maybe-decrypt / maybe-decrypt-accepting-plaintext

Replace the [& args] + argument-sniffing with plain [v] / [secret-key v] arities, which
is unambiguous (the value itself can be a byte array).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* Make secrets strict; move credentials migration to v58; split decrypt by type

- Narrow the re-encryption migration to auth_identity.credentials only (the sole
  column the v58 SQL backfill wrote as plaintext) and move it to v58 so it backports
  to every affected release. Other encrypted columns are always written encrypted and
  covered by encrypt-db, so they need no migration.
- transform-secret-value is now strict (secrets reject plaintext at rest); encrypt-db's
  secret loop reads raw + lenient so the enable-encryption-later flow still works.
- Split the decrypt helpers by value type like maybe-encrypt/maybe-encrypt-bytes:
  maybe-decrypt / maybe-decrypt-bytes (strict) and their -accepting-plaintext lenient
  counterparts, with String/bytes type hints and explicit arities.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* Drop the -accepting-plaintext decrypt variants; use strict + explicit try/catch

The lenient variants silently returned plaintext, which conflates "legacy plaintext
(fine)" with "corrupt/invalid" — and only the caller knows which. Notably the
encryption-check sentinel treated a plaintext UUID as :valid when it should be :invalid.

Each caller now uses the strict maybe-decrypt / maybe-decrypt-bytes and handles the
throw explicitly: the sentinel and the startup check treat a decrypt failure as
invalid; key rotation and the secret loop tolerate plaintext (guarding with
possibly-encrypted-*?); settings and migration reads fall back to the raw value.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* Remove now-unused metabase.test require in encryption-test

Left over after dropping the lenient-decrypt tests.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* Reintroduce -accepting-plaintext decrypt variants for plaintext-tolerant callers

Bring back maybe-decrypt-accepting-plaintext / maybe-decrypt-bytes-accepting-plaintext so the
plaintext-tolerant callers (settings reads, migration reads, key rotation and the secret loop)
call them directly instead of wrapping the strict maybe-decrypt in try/catch.

The encryption-check sentinel and the startup key-check stay strict, but now express it as an
explicit possibly-encrypted-string? guard + lenient decrypt so a plaintext UUID still reads as
invalid (rather than being trusted) without a try/catch. The core app-db column transforms
(models.interface, exploration) remain strict.

Also apply review nit: maybe-blob->bytes uses cond->.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* encrypt-db: also re-encrypt secret-value BYTES columns (stored_result.result_data)

stored_result.result_data (like secret.value) is a mi/transform-secret-value byte column, read
strictly via maybe-decrypt-bytes. do-encryption only re-encrypted secret.value, so a result_data
written as plaintext before a key was set stayed plaintext, and once a key is set the strict read
throws (500 on GET /api/exploration/query/:id).

List both byte columns and loop over them. Rename encrypted-columns -> encrypted-string-columns and
add encrypted-bytes-columns, splitting the two lists by wire type (string vs bytes).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* Address review: stream re-encryption; fail instead of double-encrypting undecryptable values

- `maybe-decrypt-accepting-plaintext` / `maybe-decrypt-bytes-accepting-plaintext` now throw when a value looks
  encrypted but cannot be decrypted with the current key, instead of logging and returning it unchanged. Returning it
  let the key-rotation path re-encrypt it into `encrypt_new(encrypt_old(x))`, permanently unrecoverable. They still
  return genuinely-plaintext values as-is.
- Re-encryption streams rows via `reducible-select` (the string path and a new bytes helper) so a large column such as
  `stored_result.result_data` is not held in memory all at once, and catches the strict decrypt to clear (clearable
  columns) or abort rather than double-encrypting.
- Callers that must tolerate the throw handle it: the `encryption-check` sentinel and `check-encryption` treat a
  decrypt failure as a wrong key; the Setting after-select degrades gracefully on read; `encrypted-json-out` (migration
  reads) returns the raw value on failure.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* Fix rotate-encryption-key test for the lenient Setting read

The Setting after-select degrades gracefully on an undecryptable value (returns the raw ciphertext) rather than
throwing, so reading a wrong-key setting yields ciphertext, not a thrown exception. Assert not= plaintext instead of
thrown? for that read; the strict Database/User columns still throw.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* Encrypt api_key hashes at rest (#80902)

---------

Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
2026-08-26 20:43:07 +02:00
github-automation-metabase
a135d86c9a Update translations 2026-08-26 (#81077)
* Update translation sources 2026-08-26

* Update translations

---------

Co-authored-by: github-actions <github-actions@github.com>
2026-08-26 18:41:31 +00:00
Andrei Simionescu
1f807106ab BOT-1013: Correct identifier casing when Metabot fixes SQL (#79674)
Validation transpiled with `identify=True` which quotes every identifier and
freezes its case, so a bare `subtotal` on Snowflake stopped resolving to
`SUBTOTAL`. Keep the LLM's own quoting instead, and only quote reserved words,
folded first so quoting can't change what they match.
2026-08-26 18:21:30 +00:00
Ben Grabow
245862781f Ignore flaky sync failures caused by table deletion race conditions (#81073) 2026-08-26 18:19:24 +00:00
4295 changed files with 1347555 additions and 1256052 deletions

View File

@@ -81,7 +81,7 @@ Everything else in the module is **internal** and not allowed to be referenced f
`dev/src/dev/modularization_help.clj` — `potemkin-ns!` helper that generates `.core` re-export boilerplate from a target namespace.
`test/metabase/core/modules_test.clj` — runs on config changes; validates structure, team assignments, API and model boundaries.
`dev/test/metabase/core/modules_test.clj` — runs on config changes; validates structure, team assignments, API and model boundaries.
### API Design Inside a Module
@@ -119,7 +119,7 @@ When the linter complains, the answer is **always refactoring**, never suppressi
- `dev/src/dev/module_score.clj` — quality scoring
- `dev/src/dev/modularization_help.clj` — Potemkin re-export helper
- `dev/src/dev/model_boundary_config.clj` + `dev/src/dev/model_tracking.clj` — model-level enforcement
- `test/metabase/core/modules_test.clj` — config validation tests
- `dev/test/metabase/core/modules_test.clj` — config validation tests
- `src/metabase/core/init.clj` — OSS module init aggregation
- `enterprise/backend/src/metabase_enterprise/core/init.clj` — EE module init aggregation
- `src/metabase/models/resolution.clj` — `:model/X` keyword → namespace map
@@ -198,6 +198,6 @@ Use the `clojure-eval` skill (preferred) or `clj-nrepl-eval` to drive the dev to
(dev.module-score/scores (dev.module-score/deps) (dev.module-score/config))
```
For tests of the config itself, use `./bin/test-agent :only '[metabase.core.modules-test]'`. After editing Clojure files, run `clj-paren-repair`. After editing `config.edn`, re-run the modules-test.
For tests of the config itself, use `./bin/mage project-tests modules`. After editing Clojure files, run `clj-paren-repair`. After editing `config.edn`, re-run the modules-test.
**Update your agent memory** as you discover idioms: which modules use `:friends` and why, which models are most-imported (suggesting they belong in their own module), which settings are accessed cross-module via `(setting/get …)` instead of `:uses` deps, and which refactors the team is actively running.

View File

@@ -304,12 +304,13 @@ Tag response Malli `:map` schemas with `{:closed true}` so the generated OpenAPI
**Linter Suppressions:**
The right shape is a **reader-discard map** placed directly above the form being silenced, listing the rule keywords explicitly:
The right shape is a **reader-discard map** placed directly above the form being silenced, listing the
rule keywords explicitly and including the required explanation:
```clojure
#_{:clj-kondo/ignore [:metabase/validate-defendpoint-route-uses-kebab-case
:metabase/validate-defendpoint-has-response-schema]}
(api.macros/defendpoint :post "/api_key" ...)
;; Loaded by name from the plugin registry.
#_{:clj-kondo/ignore [:clojure-lsp/unused-public-var]}
(defn plugin-entrypoint ...)
```
For a single require entry, hang it as metadata on that entry:
@@ -320,7 +321,12 @@ For a single require entry, hang it as metadata on that entry:
[metabase.query-processor.store :as qp.store])
```
Common Metabase-specific rules you'll see: `:metabase/modules`, `:metabase/validate-defendpoint-route-uses-kebab-case`, `:metabase/validate-defendpoint-has-response-schema`, `:metabase/prefer-with-dynamic-fn-redefs`, `:deprecated-namespace`, `:discouraged-namespace`. Don't use the keyword form (`#_:clj-kondo/ignore`); always the map form so it's grep-able by which rule(s) are suppressed.
Use the map form, not the keyword form (`#_:clj-kondo/ignore`), so searches can identify the suppressed
rules. Keep `:clj-kondo/ignore` as the map's first key; the ratchet scanner rejects other orderings.
Fix the warning when possible. Adding a suppression is a last resort and requires approval. If one is
necessary, add an explanatory `;;` comment directly above it or at the end of the same line. CI enforces
these requirements with `./bin/mage kondo-ratchets`.
**Configurable Options:**

View File

@@ -111,7 +111,12 @@ liability it usually is.
### Miscellaneous
- [ ] Example data is bird-themed when possible
- [ ] Kondo linter suppressions use proper format (not `#_:clj-kondo/ignore` keyword form)
- [ ] Kondo suppressions use a reader-discard map, not `#_:clj-kondo/ignore`, with
`:clj-kondo/ignore` as the first key
- [ ] Each new suppression is approved and has a `;;` comment explaining why the warning cannot be fixed
- [ ] Any ratchet budget increase is explained in the PR
- [ ] PRs that remove ignores leave ratchet budgets unchanged; the shrink workflow records reductions on
`master`
## Pattern matching table

View File

@@ -123,5 +123,12 @@ breakage keeps happening, *that* is the signal a comment was warranted.
- Spaces on a line with nothing after it is not allowed
- After changing module boundaries (adding/removing/renaming a `src` namespace, a cross-module `require`
or `:model/X` reference, or a new module), run `./bin/mage fix-modules-config` to regenerate
`.clj-kondo/config/modules/config.edn` and keep `metabase.core.modules-test` green. No-op when nothing
drifted; see "Module Boundaries" in the project `CLAUDE.md`.
`.clj-kondo/config/modules/config.edn`, then run `./bin/mage project-tests modules`. The fixer is a no-op when
nothing drifted; see "Module Boundaries" in the project `CLAUDE.md`.
- Fix kondo warnings instead of suppressing them. Add a suppression only as a last resort, with approval
and a `;;` comment explaining why it is necessary.
- Do not lower budgets in `.clj-kondo/ratchets.edn` after removing ignores. The shrink workflow records
reductions on `master`.
- If `.clj-kondo/ratchets.edn` conflicts during a merge, rebase, or restack, run
`./bin/merge-kondo-ratchets`. It preserves one-sided changes, chooses the policy that allows fewer
suppressions when both sides change the same entry, and stops when the conflict needs a human decision.

View File

@@ -37,14 +37,16 @@ First check if snapshots exist:
ls e2e/snapshots/default.sql 2>/dev/null && echo "snapshots exist" || echo "snapshots missing"
```
Always pass the spec path via `--spec`. The runner feeds its arguments to Cypress's CLI parser (`cypress.cli.parseRunArguments`), which silently drops bare positional paths and then runs the entire suite. A leading `run` argument is harmless, but the path itself must follow `--spec`. Sanity-check that the first `Running:` line says `(1 of 1)`.
If snapshots exist, skip regeneration for speed:
```bash
MB_EDITION=oss CYPRESS_VIDEO=false CYPRESS_RETRIES=0 CYPRESS_GUI=false GENERATE_SNAPSHOTS=false bun test-cypress $ARGUMENTS
MB_EDITION=oss CYPRESS_VIDEO=false CYPRESS_RETRIES=0 CYPRESS_GUI=false GENERATE_SNAPSHOTS=false bun test-cypress --spec path/to/spec.cy.spec.js
```
If snapshots are missing (first run), let the runner generate them:
```bash
MB_EDITION=oss CYPRESS_VIDEO=false CYPRESS_RETRIES=0 CYPRESS_GUI=false bun test-cypress $ARGUMENTS
MB_EDITION=oss CYPRESS_VIDEO=false CYPRESS_RETRIES=0 CYPRESS_GUI=false bun test-cypress --spec path/to/spec.cy.spec.js
```
When running enterprise tests, replace `MB_EDITION=oss` with `MB_EDITION=ee` in the commands above.
@@ -67,9 +69,11 @@ GREP="test name here" MB_EDITION=oss bun test-cypress --spec path/to/spec.js
When the user asks to verify a test is not flaky:
```bash
MB_EDITION=oss bin/e2e-stress-test $ARGUMENTS
MB_EDITION=oss bin/e2e-stress-test --spec path/to/spec.cy.spec.js
```
(The stress script forwards its arguments to `bun test-cypress`, so the same `--spec` rule applies.)
Set `E2E_STRESS_RUNS=N` for more iterations (default: 5). Stops on first failure and prints screenshot paths. Read those screenshots to analyze the failure.
## Analyzing failures

View File

@@ -171,7 +171,7 @@ Create `ComponentName.module.css` alongside the component file:
/* Use design system tokens — NEVER raw color/spacing values */
.root {
border: 1px solid var(--mb-color-border-neutral);
border-radius: var(--mantine-radius-md);
border-radius: var(--mantine-radius-sm);
background-color: var(--mb-color-background_page-primary);
}
@@ -246,7 +246,8 @@ Remove the old styled file entirely. Remove all imports of it from other files.
**Spacing** (`p`, `px`, `py`, `pt`, `pb`, `pl`, `pr`, `m`, `mx`, `my`, `mt`, `mb`, `ml`, `mr`):
- `"xs"` = 4px, `"sm"` = 8px, `"md"` = 16px, `"lg"` = 24px, `"xl"` = 32px
- `"xxxs"` = 0.125rem, `"xxs"` = 0.25rem, `"xs"` = 0.375rem, `"sm"` = 0.5rem, `"md"` = 0.75rem, `"lg"` = 1rem, `"xl"` = 1.5rem, `"xxl"` = 2rem, `"xxxl"` = 2.5rem
- Use numeric `0` for zero spacing.
- Custom: `rem(48)` for non-standard values (import `rem` from `metabase/ui`)
**Dimensions** (`w`, `h`, `maw`, `mah`, `miw`, `mih`):
@@ -289,11 +290,16 @@ Remove the old styled file entirely. Remove all imports of it from other files.
**Mantine spacing** — `var(--mantine-spacing-<size>)`:
- `--mantine-spacing-xs` (4px), `--mantine-spacing-sm` (8px), `--mantine-spacing-md` (16px), `--mantine-spacing-lg` (24px), `--mantine-spacing-xl` (32px)
- `--mantine-spacing-xxxs` (0.125rem), `--mantine-spacing-xxs` (0.25rem), `--mantine-spacing-xs` (0.375rem), `--mantine-spacing-sm` (0.5rem), `--mantine-spacing-md` (0.75rem), `--mantine-spacing-lg` (1rem), `--mantine-spacing-xl` (1.5rem), `--mantine-spacing-xxl` (2rem), `--mantine-spacing-xxxl` (2.5rem)
**Mantine radius** — `var(--mantine-radius-<size>)`:
- `--mantine-radius-xs` (4px), `--mantine-radius-sm` (6px), `--mantine-radius-md` (8px), `--mantine-radius-xl` (40px)
- `--mantine-radius-xxxs` (0.125rem), `--mantine-radius-xxs` (0.25rem), `--mantine-radius-xs` (0.375rem), `--mantine-radius-sm` (0.5rem), `--mantine-radius-md` (0.75rem), `--mantine-radius-lg` (1rem), `--mantine-radius-xl` (1.5rem)
**Mantine shadows (elevation)** — `var(--mantine-shadow-<size>)`:
- `--mantine-shadow-xs`, `--mantine-shadow-xs_outline`, `--mantine-shadow-sm`, `--mantine-shadow-sm_outline`, `--mantine-shadow-lg_outline`
- Light and dark modes use different shadow values (defined in `frontend/src/metabase/ui/theme.ts`)
**Mantine font sizes** — `var(--mantine-font-size-<size>)`:
@@ -313,24 +319,36 @@ When migrating, **never carry over hardcoded `rem`/`px` values** from the origin
**Spacing** — snap to the nearest Mantine spacing token:
| Hardcoded value | Nearest token | Style prop | CSS variable |
| --------------------------------- | ------------- | ------------ | ------------------------------ |
| `0.125rem` (2px) | `2px` | `2` (number) | `2px` (keep literal, no token) |
| `0.2rem` (3.2px), `0.25rem` (4px) | **xs** (4px) | `"xs"` | `var(--mantine-spacing-xs)` |
| `0.5rem` (8px) | **sm** (8px) | `"sm"` | `var(--mantine-spacing-sm)` |
| `0.75rem` (12px) | between sm/md | `rem(12)` | `0.75rem` (no exact token) |
| `1rem` (16px) | **md** (16px) | `"md"` | `var(--mantine-spacing-md)` |
| `1.5rem` (24px) | **lg** (24px) | `"lg"` | `var(--mantine-spacing-lg)` |
| `2rem` (32px) | **xl** (32px) | `"xl"` | `var(--mantine-spacing-xl)` |
| Hardcoded value | Nearest token | Style prop | CSS variable |
| --------------------------------- | ------------- | ---------- | ------------------------------- |
| `0` (0px) | — | `0` | `0` |
| `0.125rem` (2px) | **xxxs** | `"xxxs"` | `var(--mantine-spacing-xxxs)` |
| `0.2rem` (3.2px), `0.25rem` (4px) | **xxs** | `"xxs"` | `var(--mantine-spacing-xxs)` |
| `0.375rem` (6px) | **xs** | `"xs"` | `var(--mantine-spacing-xs)` |
| `0.5rem` (8px) | **sm** | `"sm"` | `var(--mantine-spacing-sm)` |
| `0.75rem` (12px) | **md** | `"md"` | `var(--mantine-spacing-md)` |
| `1rem` (16px) | **lg** | `"lg"` | `var(--mantine-spacing-lg)` |
| `1.5rem` (24px) | **xl** | `"xl"` | `var(--mantine-spacing-xl)` |
| `2rem` (32px) | **xxl** | `"xxl"` | `var(--mantine-spacing-xxl)` |
| `2.5rem` (40px) | **xxxl** | `"xxxl"` | `var(--mantine-spacing-xxxl)` |
Values between steps: keep the literal (e.g. `rem(10)`), don't round to a token.
**Border radius** — snap to the nearest Mantine radius token:
| Hardcoded value | Nearest token | CSS variable |
| ---------------- | ----------------------------- | -------------------------- |
| `0.25rem` (4px) | **xs** (4px) | `var(--mantine-radius-xs)` |
| `0.375rem` (6px) | **sm** (6px) | `var(--mantine-radius-sm)` |
| `0.5rem` (8px) | **md** (8px) | `var(--mantine-radius-md)` |
| `1rem`+ (16px+) | **xl** (40px) or keep literal | `var(--mantine-radius-xl)` |
| Hardcoded value | Nearest token | CSS variable |
| ---------------- | ------------- | ----------------------------- |
| `0` (0px) | — | `0` |
| `0.125rem` (2px) | **xxxs** | `var(--mantine-radius-xxxs)` |
| `0.25rem` (4px) | **xxs** | `var(--mantine-radius-xxs)` |
| `0.375rem` (6px) | **xs** | `var(--mantine-radius-xs)` |
| `0.5rem` (8px) | **sm** | `var(--mantine-radius-sm)` |
| `0.75rem` (12px) | **md** | `var(--mantine-radius-md)` |
| `1rem` (16px) | **lg** | `var(--mantine-radius-lg)` |
| `1.5rem` (24px) | **xl** | `var(--mantine-radius-xl)` |
Fully-round pills/badges (radius ≥ half the element height): keep a literal
such as `9999px`.
**Font sizes** — snap to the nearest Mantine font size token:

View File

@@ -205,7 +205,7 @@ Per `docs/developers-guide/frontend.md` ("Colors"): every color must be a **sema
The same discipline applies to **dimensional** values — radius, spacing/padding/gap, elevation/shadow, and typography (font size, line-height, weight). Prefer the codebase's existing scale variable over a literal: `--mantine-radius-*`, `--mantine-spacing-*`, `--mantine-shadow-*`, and the Mantine font tokens.
But Figma's scales don't always line up with the codebase's. When a Figma value **has no matching step** in the corresponding scale — e.g. Figma `radius/md = 12px` while `--mantine-radius-md` is `8px`, or a Figma `xxxs` spacing step with no Mantine equivalent — **don't silently bake in a literal.** Flag it to the user the same way you flag primitive colors: name the property, the Figma value, and the nearest codebase token, so it can be resolved with design / the scale owners (align the value, or add a scale step). Only fall back to a literal if the user confirms it's an accepted one-off, and leave a short comment in the code explaining why.
But Figma's scales don't always line up with the codebase's. When a Figma value **has no matching step** in the corresponding scale — e.g. Figma `radius/md = 16px` while `--mantine-radius-md` is `0.75rem`, or a Figma spacing step that has no Mantine equivalent — **don't silently bake in a literal.** Flag it to the user the same way you flag primitive colors: name the property, the Figma value, and the nearest codebase token, so it can be resolved with design / the scale owners (align the value, or add a scale step). Only fall back to a literal if the user confirms it's an accepted one-off, and leave a short comment in the code explaining why. The codebase scales are defined in `frontend/src/metabase/ui/theme.ts` (`SPACING_SCALE`, `RADIUS_SCALE`, `LIGHT_SHADOWS`/`DARK_SHADOWS`).
### Mantine implementation notes (reference)

View File

@@ -191,6 +191,8 @@
metabase.channel.render.pdf.font/load-fonts!
metabase.warehouses.models.database/assert-not-h2!
metabase.driver.sql-jdbc.execute/execute-prepared-statement!
;; only inspects a details map and throws; the `!` is for the throw, not for a mutation
metabase.driver/validate-db-details!
metabase.mcp.usage/record-mcp-session!
metabase.mcp.usage/record-mcp-session-end!
metabase.mcp.usage/record-mcp-tool-call!
@@ -496,6 +498,7 @@
metabase.notification.test-util/with-card-notification clojure.core/let
metabase.notification.test-util/with-system-event-notification! clojure.core/let
metabase.notification.test-util/with-temp-notification clojure.core/let
metabase.oauth-server.test-util/with-oauth-client clojure.core/fn
metabase.osi.ai-context.api-test/with-test-entry clojure.core/let
metabase.premium-features.core/defenterprise-schema clj-kondo.lint-as/def-catch-all
metabase.premium-features.defenterprise/defenterprise-schema clj-kondo.lint-as/def-catch-all

View File

@@ -104,6 +104,7 @@
model-persistence
models
parameters
public-sharing
queries
query-processor
search
@@ -141,7 +142,7 @@
metabase.activity-feed.core
metabase.activity-feed.init}
:uses #{api
app-db
batch-processing
collections
config
events
@@ -347,6 +348,7 @@
:api #{metabase.app-db.cluster-lock ; TODO FIXME
metabase.app-db.core
metabase.app-db.init
metabase.app-db.setting
metabase.app-db.setup
metabase.app-db.transient-error}
:uses #{auth-provider
@@ -355,10 +357,7 @@
connection-pool
task
util}
:model-imports #{:model/Field
:model/QueryCache
:model/Secret
:model/Setting}}
:model-imports #{:model/Field}}
appearance
{:team "UX West"
@@ -440,6 +439,7 @@
:uses #{api
app-db
collections
permissions
queries
util}
:model-exports #{:model/CardBookmark :model/CollectionBookmark :model/DashboardBookmark}
@@ -579,6 +579,8 @@
auth-identity
classloader
config
llm
metabot
models
query-processor
search
@@ -590,6 +592,7 @@
{:team "UX West"
:api #{metabase.collections.core
metabase.collections.curation
metabase.collections.init
metabase.collections.models.collection
metabase.collections.models.collection.root
metabase.collections.schema}
@@ -628,6 +631,7 @@
:uses #{api
app-db
collections
documents
eid-translation
events
lib-be
@@ -656,6 +660,7 @@
comments
{:team "UX West"
:api #{metabase.comments.api
metabase.comments.core
metabase.comments.init}
:uses #{analytics
api
@@ -696,7 +701,7 @@
models
util}
:model-exports #{:model/ModerationReview}
:model-imports #{:model/Card :model/Dashboard}}
:model-imports #{:model/Card :model/Dashboard :model/User}}
contextual-interestingness
{:team "UX West"
@@ -999,9 +1004,14 @@
entity-retrieval
{:team "Metabot"
:api #{metabase.entity-retrieval.core
metabase.entity-retrieval.mirror}
:uses #{premium-features}
:model-imports #{:model/OsiAiContext}}
metabase.entity-retrieval.mirror
metabase.entity-retrieval.spec}
:uses #{collections premium-features util}
:model-imports #{:model/Card
:model/Measure
:model/OsiAiContext
:model/Segment
:model/Table}}
events
{:team "DevEx"
@@ -1032,7 +1042,9 @@
api
app-db
collections
comments
contextual-interestingness
documents
events
interestingness
lib
@@ -1057,6 +1069,7 @@
:model-imports #{:model/Card
:model/Collection
:model/Comment
:model/Document
:model/Segment
:model/StoredResult
:model/StoredResultUse
@@ -1350,6 +1363,7 @@
{:team "Graphy"
:api #{metabase.measures.api}
:uses #{api
entity-retrieval
events
lib
lib-be
@@ -1368,6 +1382,7 @@
metabase.metabot.config
metabase.metabot.core
metabase.metabot.init
metabase.metabot.metadata-perms
metabase.metabot.self
metabase.metabot.settings
metabase.metabot.tools.entity-details}
@@ -1448,6 +1463,7 @@
:api #{metabase.metrics.api
metabase.metrics.core}
:uses #{api
app-db
events
lib
lib-be
@@ -1510,6 +1526,7 @@
:model/Dashboard
:model/DashboardTab
:model/Database
:model/Document
:model/Field
:model/Measure
:model/NativeQuerySnippet
@@ -1600,6 +1617,7 @@
:uses #{api
api-scope
appearance
events
mcp
models
request
@@ -1607,7 +1625,8 @@
system
task
tracing
util}}
util}
:model-imports #{:model/User}}
osi
{:team "Metabot"
@@ -1765,7 +1784,11 @@
metabase.public-sharing.validation}
:uses #{api
settings
util}}
util}
:model-imports #{:model/Action
:model/Card
:model/Dashboard
:model/Document}}
public-sharing-rest
{:team "UX West"
@@ -1786,7 +1809,8 @@
request
tiles
util}
:model-imports #{:model/Card
:model-imports #{:model/Action
:model/Card
:model/Dashboard
:model/DashboardCard
:model/Document}}
@@ -1851,6 +1875,7 @@
dashboards
documents
embedding
entity-retrieval
events
graph
lib
@@ -1920,6 +1945,8 @@
:model/Collection
:model/Dashboard
:model/Database
:model/StoredResult
:model/StoredResultUse
:model/Table}}
query-permissions
@@ -1961,6 +1988,7 @@
metabase.query-processor.pipeline
metabase.query-processor.pivot
metabase.query-processor.preprocess
metabase.query-processor.result-serialization
metabase.query-processor.schema
metabase.query-processor.setup
metabase.query-processor.store
@@ -2140,14 +2168,14 @@
secrets
{:team "Graphy"
:api #{metabase.secrets.core}
:uses #{api driver models premium-features util}
:model-exports #{:model/Secret}}
:uses #{api driver models premium-features util}}
segments
{:team "Graphy"
:api #{metabase.segments.api
metabase.segments.schema}
:uses #{api
entity-retrieval
events
lib
lib-be
@@ -2187,7 +2215,6 @@
request
session
settings
setup
sso
system
tracing
@@ -2222,8 +2249,7 @@
settings
{:team "DevEx"
:api #{metabase.settings.core
metabase.settings.init}
:api #{metabase.settings.core metabase.settings.init}
:uses #{api
app-db
config
@@ -2232,7 +2258,8 @@
premium-features
util
enterprise/advanced-permissions}
:model-exports #{:model/Setting}}
:model-exports #{:model/Setting}
:model-imports #{:model/User}}
settings-rest
{:team "DevEx"
@@ -2408,6 +2435,7 @@
metabase.task.core}
:uses #{app-db
classloader
config
tracing
util}}
@@ -2473,6 +2501,7 @@
lib
lib-be
parameters
premium-features
query-processor
settings
util}
@@ -2717,6 +2746,7 @@
:model-imports #{:model/Dashboard
:model/Database
:model/PermissionsGroupMembership
:model/PulseChannelRecipient
:model/Tenant}}
users-rest
@@ -2776,7 +2806,10 @@
query-permissions
util}
:model-exports #{:model/ViewLog}
:model-imports #{:model/Card :model/Dashboard :model/Table}}
:model-imports #{:model/Card
:model/Dashboard
:model/Document
:model/Table}}
warehouse-schema
{:team "UX West"
@@ -2795,6 +2828,7 @@
audit-app
collections
driver
entity-retrieval
lib
lib-be
models
@@ -3161,7 +3195,8 @@
:uses #{api
content-verification
util
enterprise/api}}
enterprise/api}
:model-imports #{:model/Card :model/Dashboard}}
enterprise/core
{:team "DevEx"
@@ -3364,7 +3399,6 @@
:api #{metabase-enterprise.entity-retrieval.init}
:uses #{analytics-interface
app-db
collections
entity-retrieval
health-inspector
premium-features
@@ -3524,6 +3558,23 @@
util}
:model-imports #{:model/AuthIdentity :model/User}}
enterprise/osi-generation
{:team "Metabot"
:uses #{analytics-interface
api
app-db
entity-retrieval
llm
metabot
osi
premium-features
settings
task
util
enterprise/semantic-search}
:api #{metabase-enterprise.osi-generation.api metabase-enterprise.osi-generation.init}
:model-imports #{:model/OsiAiContext}}
enterprise/permission-debug
{:team "UX West"
:api #{metabase-enterprise.permission-debug.api}
@@ -3801,12 +3852,14 @@
:api #{metabase-enterprise.support-access-grants.api
metabase-enterprise.support-access-grants.init}
:uses #{api
app-db
auth-identity
events
models
request
settings
system
task
util}
:model-imports #{:model/AuthIdentity :model/Session :model/User}}

View File

@@ -58,6 +58,7 @@
metabase.app-db.core/finish-db-setup!
metabase.app-db.core/migrate!
metabase.app-db.core/setup-db!
metabase.app-db.encryption-test-util/do-with-encrypted-app-db!
metabase.app-db.schema-migrations-test.impl/run-migrations-in-range!
metabase.app-db.setup/migrate!
metabase.app-db.setup/setup-db!

View File

@@ -1,47 +1,80 @@
;; Per-linter budgets for inline `:clj-kondo/ignore` forms.
;; metabase.core.kondo-ratchet-test fails when the budgets drift from the actual counts.
;; `./bin/mage fix-kondo-ratchets` lowers budgets to match the tree; local test runs do it
;; automatically. Raising a budget, or adding one for a new linter (`--seed`), is a hand edit
;; to defend in your PR.
;; Budgets for kondo suppressions: inline `:clj-kondo/ignore` forms per linter (:ignore-counts), and
;; config-level waivers in .clj-kondo/config.edn (:config-counts -- :off switches and :exclude entries).
;; Each :ignore-counts value is a non-negative integer budget, or :unlimited for no ceiling.
;; Checks fail when a count exceeds its numeric budget; unused budget is allowed.
;; Any ignore outside :comment-exempt needs an explanatory comment directly above or trailing on its line.
;; `./bin/mage kondo-ratchets-shrink` lowers budgets unless `--seed` explicitly adds or raises one.
;; The workflow runs it on master, so feature branches do not need to record reductions.
;; Raising or adding a budget (`--seed` for inline ignores, a manual edit for config) or widening the
;; exemptions must be explained in the PR.
;; :all is the vector-less ignore form, which suppresses every linter on the next form.
{:ignore-counts {:aliased-namespace-symbol 3
:all 48
:case-symbol-test 1
:clojure-lsp/unused-public-var 9
:def-fn 1
:deprecated-namespace 95
:deprecated-var 84
:discouraged-java-method 3
:discouraged-namespace 81
:discouraged-var 111
:equals-true 1
:main-without-gen-class 1
:metabase/defsetting-namespace 2
:metabase/disallow-hardcoded-driver-names-in-tests 34
:metabase/discourage-millis-duration 1
:metabase/i-like-making-cams-eyes-bleed-with-horrifically-long-tests 13
:metabase/modules 7
:metabase/namespace-name 4
:metabase/prefer-with-dynamic-fn-redefs 28
:metabase/test-helpers-use-non-thread-safe-functions 17
:metabase/validate-defendpoint-has-response-schema 440
:metabase/validate-defendpoint-query-params-use-kebab-case 50
:metabase/validate-defendpoint-route-uses-kebab-case 44
:metabase/validate-deftest 10
:missing-docstring 10
:missing-protocol-method 3
:non-arg-vec-return-type-hint 1
:reduce-without-init 3
:syntax 2
:uninitialized-var 1
:unquote-not-syntax-quoted 1
:unresolved-namespace 3
:unresolved-require 1
:unresolved-symbol 4
:unsorted-required-namespaces 1
:unused-alias 1
:unused-binding 9
:unused-excluded-var 2
:unused-namespace 1
:unused-private-var 9
:unused-value 1}}
{:ignore-counts {:aliased-namespace-symbol 3
:case-symbol-test 1
:clojure-lsp/unused-public-var 9
:consistent-alias 2
:def-fn 1
:deprecated-namespace 95
:deprecated-var 86
:discouraged-java-method 4
:discouraged-namespace 71
:discouraged-var 130
:duplicate-require 1
:equals-true 1
:main-without-gen-class 1
:metabase/defsetting-namespace 2
:metabase/disallow-hardcoded-driver-names-in-tests 34
:metabase/discourage-millis-duration 1
:metabase/i-like-making-cams-eyes-bleed-with-horrifically-long-tests 13
:metabase/modules 23
:metabase/namespace-name 4
:metabase/prefer-with-dynamic-fn-redefs 28
:metabase/test-helpers-use-non-thread-safe-functions 18
:metabase/validate-defendpoint-has-response-schema 439
:metabase/validate-defendpoint-query-params-use-kebab-case 50
:metabase/validate-defendpoint-route-uses-kebab-case 44
:metabase/validate-deftest 10
:missing-docstring 12
:missing-protocol-method 3
:non-arg-vec-return-type-hint 1
:reduce-without-init 3
:redundant-do 1
:syntax 2
:type-mismatch 8
:uninitialized-var 1
:unquote-not-syntax-quoted 1
:unresolved-namespace 3
:unresolved-symbol 11
:unsorted-required-namespaces :unlimited
:unused-alias 2
:unused-binding 9
:unused-excluded-var 2
:unused-namespace 2
:unused-private-var 8
:unused-value 1}
:config-counts {:constant-condition 1
:deprecated-namespace 7
:deprecated-var 16
:discouraged-namespace 14
:discouraged-var 14
:inline-def 1
:main-without-gen-class 1
:metabase/defsetting-must-specify-export 1
:metabase/defsetting-namespace 1
:metabase/prefer-with-dynamic-fn-redefs 3
:missing-docstring 1
:private-call 1
:redundant-ignore 1
:redundant-primitive-coercion 1
:refer-all 1
:type-mismatch 1
:unexpected-recur 1
:unresolved-symbol 19
:unresolved-var 1
:unused-referred-var 4}
:comment-exempt #{:deprecated-namespace
:discouraged-namespace
:discouraged-var
:metabase/modules
:metabase/validate-defendpoint-has-response-schema
:metabase/validate-defendpoint-query-params-use-kebab-case
:metabase/validate-defendpoint-route-uses-kebab-case}}

View File

@@ -20,6 +20,23 @@
:message error
:type :metabase/modules))))))))
(defn- lint-dynamic-require* [node current-ns config]
(when (and (not (modules/ignored-namespace? config current-ns))
(modules/module current-ns))
(hooks/reg-finding! (assoc (meta node)
:message "Module dependency cannot be statically determined from a dynamic require"
:type :metabase/modules))))
(defn- quoted-require? [node]
(or (= (hooks/tag node) :quote)
(and (= (hooks/tag node) :list)
(let [[x] (:children node)]
(and (hooks/token-node? x)
(= (hooks/sexpr x) 'quote))))))
(defn- keyword-node? [node]
(keyword? (hooks/sexpr node)))
(defn- unwrap-require [node]
(case (hooks/tag node)
;; some.namespace of some.namespace/some-var
@@ -42,7 +59,11 @@
;; something like [some.namespace :as whatever]
:vector
(recur (first (:children node)))))
(recur (first (:children node)))
;; anything else -- e.g. a runtime expression like `(requiring-resolve (:failing-test-var opts))` --
;; has no statically-known namespace
nil))
(comment
(unwrap-require (-> "(requiring-resolve 'metabase.notification.payload.execute/execute-dashboard)"
@@ -51,20 +72,24 @@
second)))
(defn- lint-require* [node current-ns config]
(let [[_require & args] (:children node)
required-namespaces (->> (keep unwrap-require args)
;; `require` runs through `cc/load-libs` which allows
;; #{:as :reload :reload-all :require :use :verbose :refer :as-alias}
;; as args. If you use a REPL tool that applies clj-kondo linting (e.g., Clojure MCP)
;; you may wish to `:reload` or `:reload-all` a namespace. So we shouldn't assume all
;; args are keywords:
(remove keyword?))]
(lint* node current-ns config required-namespaces)))
(let [[_require & args] (:children node)
;; Keywords such as `:reload` and `:reload-all` are options, not namespace dependencies.
namespace-nodes (remove keyword-node? args)
static-nodes (filter quoted-require? namespace-nodes)
required-namespaces (->> (keep unwrap-require static-nodes)
(filter simple-symbol?))]
(lint* node current-ns config required-namespaces)
(when (or (not= (count static-nodes) (count namespace-nodes))
(not= (count required-namespaces) (count static-nodes)))
(lint-dynamic-require* node current-ns config))))
(defn- lint-requiring-resolve* [node current-ns config]
(let [[_requiring-resolve symb-node] (:children node)
required-namespace (unwrap-require symb-node)]
(lint* node current-ns config [required-namespace])))
(if (and (quoted-require? symb-node)
(simple-symbol? required-namespace))
(lint* node current-ns config [required-namespace])
(lint-dynamic-require* node current-ns config))))
(defn lint-require [x]
(lint-require* (:node x) (:ns x) (modules/config x))

View File

@@ -171,6 +171,7 @@
(str/starts-with? (name ns-symb) prefix))
["build-drivers."
"build."
"hooks." ; clj-kondo hook tests, outside the module system
"i18n." ; bin/i18n
"lint-migrations-file-test"
"load-namespaces." ; bin/load-namespaces

View File

@@ -286,6 +286,7 @@
(update context :node update-node)))
(comment
(require '[clojure.pprint])
(defn- defsetting* [form]
(hooks/sexpr
(:node
@@ -293,7 +294,6 @@
{:node
(hooks/parse-string
(with-out-str
#_{:clj-kondo/ignore [:unresolved-namespace]}
(clojure.pprint/pprint
form)))}))))

View File

@@ -0,0 +1,65 @@
(ns hooks.clojure.core.require-test
(:require
[clj-kondo.hooks-api :as hooks]
[clj-kondo.impl.utils]
[clojure.test :refer :all]
[hooks.clojure.core.require]))
(def ^:private config
'{:linters {:metabase/modules {:level :warning}}
:metabase/modules {search {:uses #{}}
task {:api #{metabase.task.core}}}})
(defn- lint [hook form current-ns]
(binding [clj-kondo.impl.utils/*ctx* {:config config
:ignores (atom nil)
:findings (atom [])
:namespaces (atom {})}]
(hook {:node (hooks/parse-string (pr-str form))
:ns current-ns
:config config})
@(:findings clj-kondo.impl.utils/*ctx*)))
(defn- lint-require [form current-ns]
(lint hooks.clojure.core.require/lint-require form current-ns))
(defn- lint-requiring-resolve [form current-ns]
(lint hooks.clojure.core.require/lint-requiring-resolve form current-ns))
(def ^:private dynamic-require-finding
{:message "Module dependency cannot be statically determined from a dynamic require"
:type :metabase/modules})
(def ^:private cross-module-finding
{:message "Module task should not be used in the search module. [:metabase/modules search :uses]"
:type :metabase/modules})
(deftest ^:parallel dynamic-requiring-resolve-test
(testing "outside the module system"
(is (= [] (lint-requiring-resolve '(requiring-resolve (:failing-test-var opts)) 'dev))))
(testing "inside a module"
(are [form] (=? [dynamic-require-finding]
(lint-requiring-resolve form 'metabase.search.core))
'(requiring-resolve (:failing-test-var opts))
'(requiring-resolve @some-atom)
'(requiring-resolve some-local)
'(requiring-resolve "not-a-symbol"))))
(deftest ^:parallel dynamic-require-test
(testing "outside the module system"
(is (= [] (lint-require '(require some-local) 'dev))))
(testing "inside a module"
(is (=? [dynamic-require-finding]
(lint-require '(require some-local) 'metabase.search.core)))))
(deftest ^:parallel static-requiring-resolve-still-linted-test
(is (=? [cross-module-finding]
(lint-requiring-resolve '(requiring-resolve 'metabase.task.impl/go)
'metabase.search.core))
"a statically-known cross-module violation is still reported"))
(deftest ^:parallel static-require-still-linted-test
(is (=? [cross-module-finding]
(lint-require '(require 'metabase.task.impl :reload)
'metabase.search.core))
"a statically-known cross-module violation is still reported while keyword flags are ignored"))

View File

@@ -16,7 +16,7 @@ skip =
frontend/src/metabase/querying/expressions/tokenizer/lezer.js,
frontend/src/metabase/query_builder/components/NativeQueryEditor/CodeMirrorEditor/language.ts,
frontend/src/metabase/static-viz/constants/char-sizes.ts,
frontend/src/metabase/visualizations/lib/mapping_codes.ts,
frontend/src/metabase/viz-core/lib/mapping_codes.ts,
frontend/*/mocks/*,
modules/drivers/mongo/src/metabase/driver/mongo/operators.clj,
src/metabase/analyze/classifiers/name.clj,

13
.github/CODEOWNERS vendored
View File

@@ -42,6 +42,7 @@ docs/api.json # auto-generated
docs/configuring-metabase/environment-variables.md # auto-generated
docs/configuring-metabase/config-template.md # auto-generated
docs/installation-and-operation/commands.md # auto-generated
docs/ai/providers.md # auto-generated
docs/embedding/sdk/api # auto-generated
docs/embedding/eajs/snippets # auto-generated
@@ -54,3 +55,15 @@ src/metabase/analytics/prometheus.clj @metabase/cloud-ops
# @metabase/metabot-frontend
enterprise/frontend/src/metabase-enterprise/metabot @metabase/metabot-frontend
frontend/src/metabase/metabot @metabase/metabot-frontend
# @metabase/core-frontend-gadget
bin/custom-viz @metabase/core-frontend-gadget
docs/developers-guide/custom-visualizations.md @metabase/core-frontend-gadget
enterprise/frontend/src/custom-viz @metabase/core-frontend-gadget
enterprise/frontend/src/metabase-enterprise/api/custom-viz-plugin.ts @metabase/core-frontend-gadget
enterprise/frontend/src/metabase-enterprise/custom_viz @metabase/core-frontend-gadget
frontend/src/metabase-types/api/custom-viz-plugin.ts @metabase/core-frontend-gadget
frontend/src/metabase/plugins/oss/custom-viz.ts @metabase/core-frontend-gadget
frontend/src/metabase/visualizations/custom-visualizations @metabase/core-frontend-gadget
frontend/test/__support__/custom-viz-fixtures @metabase/core-frontend-gadget
.github/workflows/release-custom-viz.yml @metabase/devexp @metabase/core-frontend-gadget

View File

@@ -8,6 +8,9 @@ paths:
# GitHub added parallel steps (`parallel`, `background`, `wait`, `wait-all`,
# `cancel`) in June 2026, and actionlint 1.7.12 does not know the schema yet:
# https://github.blog/changelog/2026-06-25-actions-steps-can-now-be-run-in-parallel/
.github/workflows/backend.yml:
ignore:
- 'step must run script with "run" section or run action with "uses" section'
.github/workflows/lint.yml:
ignore:
- 'step must run script with "run" section or run action with "uses" section'

View File

@@ -25,6 +25,8 @@ runs:
COMMIT: ${{ inputs.backport-commit }}
shell: bash
run: | # bash
source "${{ github.action_path }}/kondo-ratchets.sh"
git config --global user.email "metabase-bot@metabase.com"
git config --global user.name "Metabase bot"
@@ -34,7 +36,7 @@ runs:
git checkout ${TARGET_BRANCH}
git fetch --all
git checkout -b ${BACKPORT_BRANCH}
git cherry-pick ${COMMIT} || true
cherry_pick_backport "${COMMIT}"
CONFLICT_LIST=$(git ls-files -u)
@@ -46,10 +48,7 @@ runs:
echo "has-conflicts=true" >> "$GITHUB_OUTPUT"
# Add a shell script for resolving conflicts
echo "git reset HEAD~1" > ./backport.sh
echo "rm ./backport.sh" >> ./backport.sh
echo "git cherry-pick ${COMMIT}" >> ./backport.sh
echo "echo 'Resolve conflicts and force push this branch.\n\nTo backport translations run: bin/i18n/merge-translations <release-branch>'" >> ./backport.sh
write_backport_script "${COMMIT}"
chmod +x ./backport.sh
git add ./backport.sh
git commit -m "Add backport resolution script"

View File

@@ -0,0 +1,375 @@
#!/usr/bin/env bash
set -euo pipefail
action_dir=$(cd "$(dirname "$0")" && pwd)
cut_release_workflow="$action_dir/../../workflows/cut-release-branch.yml"
# shellcheck source=.github/actions/create-backport/kondo-ratchets.sh
source "$action_dir/kondo-ratchets.sh"
# Any editor launch fails the run, so a continuation that waits on an editor cannot pass.
export GIT_EDITOR=false
test_root=
trap '[ -z "$test_root" ] || rm -rf -- "$test_root"' EXIT
trap 'exit 129' HUP
trap 'exit 130' INT
trap 'exit 143' TERM
test_root=$(mktemp -d)
active_ratchets='{:ignore-counts {:example 1}}'
disabled_ratchets=';; Kondo ignore ratchets apply only to master; this release branch opts out.
;; The test and fixer recognize this explicit opt-out.
{:disabled true}'
# Coverage:
#
# Target state Incoming change Other conflict Expected result
# ----------------- -------------------- --------------- ------------------------------------------
# active ratchets unrelated file none leave ratchets unchanged
# active ratchets ratchets none install the release opt-out
# disabled ratchets ratchets none keep the opt-out; finish with an empty commit
# divergent ratchets ratchets none resolve ratchets and finish the cherry-pick
# divergent ratchets ratchets (on a pty) none finish without opening an editor
# active ratchets unrelated file app.txt leave the conflict for manual resolution
# divergent ratchets ratchets + app.txt app.txt resolve only ratchets
# release cut ratchets none cut and backport commit the same opt-out
# n/a cut-release workflow n/a the workflow calls the shared writer
# any invalid commit n/a propagate the cherry-pick failure
# divergent ratchets generated backport.sh app.txt resolve only ratchets through the manual script
#
# Known gaps: an older target with no ratchets file, and an incoming commit that
# deletes the ratchets file.
test_unrelated_backport_preserves_target_ratchets() (
local repo="$test_root/unrelated"
init_repo "$repo"
local commit
commit=$(feature_commit "$repo" change_app)
git -C "$repo" checkout -q release
cd "$repo"
cherry_pick_backport "$commit"
assert_eq feature "$(cat app.txt)" "the unrelated change was not cherry-picked"
assert_ratchets "$active_ratchets"
assert_cherry_pick_completed
assert_worktree_clean
)
test_clean_ratchet_backport_installs_release_opt_out() (
local repo="$test_root/clean-ratchet"
init_repo "$repo"
local commit
commit=$(feature_commit "$repo" change_ratchets)
git -C "$repo" checkout -q release
cd "$repo"
cherry_pick_backport "$commit"
assert_ratchets_disabled
assert_cherry_pick_completed
assert_worktree_clean
)
test_disabled_release_absorbs_ratchet_only_backport() (
local repo="$test_root/disabled-release"
init_repo "$repo"
local commit
commit=$(feature_commit "$repo" change_ratchets)
git -C "$repo" checkout -q release
commit_release_change "$repo" disable_release_ratchets "Disable release ratchets"
cd "$repo"
cherry_pick_backport "$commit"
assert_ratchets_disabled
assert_cherry_pick_completed
assert_empty_commit
assert_worktree_clean
)
test_ratchet_only_conflict_is_resolved() (
local repo="$test_root/ratchet-conflict"
init_repo "$repo"
local commit
commit=$(conflicting_ratchet_commit "$repo")
cd "$repo"
cherry_pick_backport "$commit"
assert_ratchets_disabled
assert_cherry_pick_completed
assert_worktree_clean
)
test_continuation_never_opens_an_editor() (
local repo="$test_root/editor"
init_repo "$repo"
local commit
commit=$(conflicting_ratchet_commit "$repo")
cd "$repo"
# Git only opens an editor for the continuation when stdin is a terminal, so run it on a pty. The
# exported GIT_EDITOR=false turns any editor launch into a failure. The single quotes are deliberate:
# $1 and $2 are for the inner bash, not this shell.
# shellcheck disable=SC2016
python3 -c 'import os, pty, sys; sys.exit(os.waitstatus_to_exitcode(pty.spawn(sys.argv[1:])))' \
bash -c 'source "$1/kondo-ratchets.sh" && cherry_pick_backport "$2"' bash "$action_dir" "$commit" >/dev/null ||
fail "the cherry-pick continuation opened an editor"
assert_ratchets_disabled
assert_cherry_pick_completed
)
test_non_ratchet_conflict_remains_manual() (
local repo="$test_root/non-ratchet-conflict"
init_repo "$repo"
local commit
commit=$(feature_commit "$repo" change_app)
git -C "$repo" checkout -q release
commit_release_change "$repo" change_release_app "Change release app"
cd "$repo"
cherry_pick_backport "$commit"
assert_ratchets "$active_ratchets"
assert_only_conflicts app.txt
assert_cherry_pick_pending
)
test_mixed_conflict_resolves_only_ratchets() (
local repo="$test_root/mixed-conflict"
init_repo "$repo"
local commit
commit=$(feature_commit "$repo" change_app_and_ratchets)
git -C "$repo" checkout -q release
commit_release_change "$repo" change_release_app_and_ratchets "Change release app and ratchets"
cd "$repo"
cherry_pick_backport "$commit"
assert_ratchets_disabled
assert_only_conflicts app.txt
assert_cherry_pick_pending
)
test_release_cut_and_backport_commit_the_same_opt_out() (
local cut_repo="$test_root/release-cut"
init_repo "$cut_repo"
cd "$cut_repo"
git checkout -qb release-x.99.x
write_disabled_ratchets
git add --sparse .clj-kondo/ratchets.edn
git commit -q --allow-empty --no-verify -m "Cut release-x.99.x branch"
git show HEAD:.clj-kondo/ratchets.edn > "$test_root/release-cut.edn"
local backport_repo="$test_root/release-cut-backport"
init_repo "$backport_repo"
local commit
commit=$(feature_commit "$backport_repo" change_ratchets)
git -C "$backport_repo" checkout -q release
cd "$backport_repo"
cherry_pick_backport "$commit"
git show HEAD:.clj-kondo/ratchets.edn > "$test_root/backport.edn"
cmp -s "$test_root/release-cut.edn" "$test_root/backport.edn" ||
fail "the release cut and the backport committed different opt-out files"
assert_eq "$disabled_ratchets" "$(cat "$test_root/release-cut.edn")" \
"the release cut did not install the expected opt-out"
)
test_cut_release_workflow_uses_the_shared_writer() (
grep -Fq 'source .github/actions/create-backport/kondo-ratchets.sh' "$cut_release_workflow" ||
fail "the cut-release workflow does not source kondo-ratchets.sh"
grep -Fq 'write_disabled_ratchets' "$cut_release_workflow" ||
fail "the cut-release workflow does not call write_disabled_ratchets"
! grep -Fq ':disabled' "$cut_release_workflow" ||
fail "the cut-release workflow carries its own copy of the opt-out"
)
test_invalid_commit_failure_is_propagated() (
local repo="$test_root/invalid"
init_repo "$repo"
cd "$repo"
if cherry_pick_backport deadbeef; then
fail "an invalid commit succeeded"
fi
)
test_generated_script_resolves_ratchets_end_to_end() (
local repo="$test_root/generated-script"
init_repo "$repo"
local commit
commit=$(feature_commit "$repo" change_app_and_ratchets)
git -C "$repo" checkout -q release
commit_release_change "$repo" change_release_app_and_ratchets "Change release app and ratchets"
cd "$repo"
cherry_pick_backport "$commit"
git cherry-pick --abort
local release_commit
release_commit=$(git rev-parse HEAD)
write_backport_script "$commit"
chmod +x backport.sh
git add backport.sh
git commit -qm "Add manual backport script"
bash -n backport.sh
grep -Fq 'write_disabled_ratchets ()' backport.sh || fail "the generated script omitted write_disabled_ratchets"
./backport.sh
assert_ratchets_disabled
assert_only_conflicts app.txt
assert_cherry_pick_pending
assert_eq "$release_commit" "$(git rev-parse HEAD)" "the generated script did not return HEAD to the release commit"
[ ! -e backport.sh ] || fail "the generated script did not remove itself"
! git ls-files --error-unmatch backport.sh >/dev/null 2>&1 || fail "the generated script is still tracked"
! git status --porcelain | grep -q backport.sh || fail "the generated script left a status entry"
)
# Test harness
fail() {
echo "FAIL: $*" >&2
exit 1
}
assert_eq() {
local expected=$1
local actual=$2
local message=$3
[ "$expected" = "$actual" ] || fail "$message (expected '$expected', got '$actual')"
}
assert_ratchets() {
assert_eq "$1" "$(cat .clj-kondo/ratchets.edn)" "unexpected ratchet state"
}
assert_ratchets_disabled() {
assert_ratchets "$disabled_ratchets"
}
assert_cherry_pick_completed() {
! git rev-parse -q --verify CHERRY_PICK_HEAD >/dev/null || fail "the cherry-pick is still pending"
}
assert_cherry_pick_pending() {
git rev-parse -q --verify CHERRY_PICK_HEAD >/dev/null || fail "the cherry-pick state was lost"
}
assert_only_conflicts() {
local expected
local actual
expected=$(printf '%s\n' "$@" | sort)
actual=$(git diff --name-only --diff-filter=U | sort)
assert_eq "$expected" "$actual" "unexpected unresolved files"
}
assert_empty_commit() {
git diff --quiet HEAD^ HEAD || fail "the backport commit was not empty"
}
assert_worktree_clean() {
[ -z "$(git status --porcelain)" ] || fail "the backport left worktree changes"
}
init_repo() {
local repo=$1
git init -q -b release "$repo"
git -C "$repo" config user.email test@example.com
git -C "$repo" config user.name "Backport test"
printf 'base\n' > "$repo/app.txt"
mkdir -p "$repo/.clj-kondo"
printf '%s\n' "$active_ratchets" > "$repo/.clj-kondo/ratchets.edn"
git -C "$repo" add .
git -C "$repo" commit -qm "Base"
}
feature_commit() {
local repo=$1
shift
git -C "$repo" checkout -qb feature
"$@" "$repo"
git -C "$repo" add .
git -C "$repo" commit -qm "Feature"
git -C "$repo" rev-parse HEAD
}
# Sets up a release branch whose ratchets conflict with the feature commit, and prints that commit.
conflicting_ratchet_commit() {
local repo=$1
local commit
commit=$(feature_commit "$repo" change_ratchets)
git -C "$repo" checkout -q release
commit_release_change "$repo" change_release_ratchets "Change release ratchets"
echo "$commit"
}
change_app() {
printf 'feature\n' > "$1/app.txt"
}
change_ratchets() {
printf '{:ignore-counts {:example 2}}\n' > "$1/.clj-kondo/ratchets.edn"
}
change_app_and_ratchets() {
change_app "$1"
change_ratchets "$1"
}
change_release_app() {
printf 'release\n' > "$1/app.txt"
}
change_release_ratchets() {
printf '{:ignore-counts {:release 1}}\n' > "$1/.clj-kondo/ratchets.edn"
}
change_release_app_and_ratchets() {
change_release_app "$1"
change_release_ratchets "$1"
}
# Setup writes the opt-out through the shared writer; the only copy is the expected text at the top.
disable_release_ratchets() {
(cd "$1" && write_disabled_ratchets)
}
commit_release_change() {
local repo=$1
local change=$2
local message=$3
"$change" "$repo"
git -C "$repo" add .
git -C "$repo" commit -qm "$message"
}
run_test() {
local description=$1
local test=${description// /_}
test=${test//-/_}
test="test_$test"
declare -F "$test" >/dev/null || fail "missing test function: $test"
printf 'TEST: %s\n' "$description"
"$test"
printf 'PASS: %s\n' "$description"
}
run_test "unrelated backport preserves target ratchets"
run_test "clean ratchet backport installs release opt-out"
run_test "disabled release absorbs ratchet-only backport"
run_test "ratchet-only conflict is resolved"
run_test "continuation never opens an editor"
run_test "non-ratchet conflict remains manual"
run_test "mixed conflict resolves only ratchets"
run_test "release cut and backport commit the same opt-out"
run_test "cut-release workflow uses the shared writer"
run_test "invalid commit failure is propagated"
run_test "generated script resolves ratchets end-to-end"

View File

@@ -0,0 +1,74 @@
#!/usr/bin/env bash
set -euo pipefail
# The only place that knows what a release branch's opt-out looks like. The cut-release workflow, the
# backport helper, and the tests all go through this function.
write_disabled_ratchets() {
mkdir -p .clj-kondo
printf '%s\n' \
';; Kondo ignore ratchets apply only to master; this release branch opts out.' \
';; The test and fixer recognize this explicit opt-out.' \
'{:disabled true}' \
> .clj-kondo/ratchets.edn
}
disable_ratchets() {
local commit=$1
local file=.clj-kondo/ratchets.edn
[ -n "$(git diff-tree --root --no-commit-id --name-only -r "$commit" -- "$file")" ] || return 0
echo "$file: disabling kondo ratchets on the release branch"
write_disabled_ratchets
git add -- "$file"
if ! git rev-parse -q --verify CHERRY_PICK_HEAD >/dev/null; then
if ! git diff --cached --quiet HEAD; then
git commit --amend --no-edit
fi
return 0
fi
if [ -z "$(git ls-files -u)" ]; then
echo "$file was the only conflict, finishing the cherry-pick"
if git diff --cached --quiet HEAD; then
echo "nothing left to apply, committing the backport empty"
git commit --allow-empty --no-edit
else
# GIT_EDITOR outranks core.editor, VISUAL, and EDITOR, so the continuation can never wait on
# an editor; the repository's own editor configuration is left alone.
GIT_EDITOR=true git cherry-pick --continue
fi
fi
}
cherry_pick_backport() {
local commit=$1
local status=0
git cherry-pick "$commit" || status=$?
if [ "$status" -ne 0 ] && ! git rev-parse -q --verify CHERRY_PICK_HEAD >/dev/null; then
echo "cherry-pick failed without entering a resolvable conflict state" >&2
return "$status"
fi
disable_ratchets "$commit"
if [ "$status" -ne 0 ] &&
git rev-parse -q --verify CHERRY_PICK_HEAD >/dev/null &&
[ -z "$(git ls-files -u)" ]; then
echo "cherry-pick stopped without a file conflict to resolve" >&2
return "$status"
fi
}
write_backport_script() {
local commit=$1
{
echo "#!/usr/bin/env bash"
echo "set -euo pipefail"
echo "git reset HEAD~1"
echo "rm ./backport.sh"
declare -f write_disabled_ratchets disable_ratchets cherry_pick_backport
printf 'cherry_pick_backport %q\n' "$commit"
printf '%s\n' "printf 'Resolve conflicts and force push this branch.\\n\\nTo backport translations run: bin/i18n/merge-translations <release-branch>\\n'"
} > ./backport.sh
}

View File

@@ -93,8 +93,8 @@ runs:
if ${{ inputs.maildev }}; then
echo -e "${Y}checking maildev container..." &&
while ! nc -z localhost 1080; do sleep 1; done &&
while ! nc -z localhost 1025; do sleep 1; done &&
while ! nc -z localhost 1180; do sleep 1; done &&
while ! nc -z localhost 1125; do sleep 1; done &&
echo -e "${G}Maildev is up and running!"
echo -e "${Y}checking maildev-ssl container..." &&

View File

@@ -5,9 +5,6 @@ inputs:
java-version:
required: true
default: "25"
clojure-version:
required: true
default: "1.12.0.1488"
runs:
using: "composite"
@@ -17,12 +14,40 @@ runs:
with:
java-version: ${{ inputs.java-version }}
distribution: "temurin"
- name: Cache spec
id: cache-spec
shell: bash
run: .github/scripts/cache-keys.sh | tee -a "$GITHUB_OUTPUT"
- name: Restore Clojure CLI
uses: actions/cache/restore@v5
with:
path: ${{ steps.cache-spec.outputs.clojure-path }}
key: ${{ steps.cache-spec.outputs.clojure-key }}
- name: Install Clojure CLI
shell: bash
env:
CLOJURE_PREFIX: ${{ steps.cache-spec.outputs.clojure-path }}
CLOJURE_VERSION: ${{ steps.cache-spec.outputs.clojure-version }}
run: |
set -uo pipefail
echo "$CLOJURE_PREFIX/bin" >> "$GITHUB_PATH"
export PATH="$CLOJURE_PREFIX/bin:$PATH"
# Whether the prefix already holds the right install is decided by running it, not by the
# cache-hit flag. The installer bakes the prefix into the clj/clojure shims, so a tree restored
# under a different $HOME points at a directory that is not there; and on a runner that is not
# ephemeral, a previous version's prefix outlives the cache entry that stopped matching it.
if clojure --version | grep -qF "$CLOJURE_VERSION"; then
clojure --version
exit 0
fi
cd "$RUNNER_TEMP"
for attempt in 1 2 3; do
curl --retry 3 --retry-delay 5 --fail -O https://download.clojure.org/install/linux-install-${{ inputs.clojure-version }}.sh \
&& sudo bash ./linux-install-${{ inputs.clojure-version }}.sh \
curl --retry 3 --retry-delay 5 --fail -O "https://download.clojure.org/install/linux-install-$CLOJURE_VERSION.sh" \
&& bash "./linux-install-$CLOJURE_VERSION.sh" --prefix "$CLOJURE_PREFIX" \
&& break
echo "::warning::Clojure install attempt $attempt failed, retrying in 10s..."
sleep 10
@@ -39,11 +64,6 @@ runs:
version: "0.10.2" # keep in sync with mise.toml
github-token: ${{ github.token }}
- name: Cache spec
id: cache-spec
shell: bash
run: .github/scripts/cache-keys.sh | tee -a "$GITHUB_OUTPUT"
- name: Restore M2 cache
uses: actions/cache/restore@v5
with:

View File

@@ -7,9 +7,8 @@ default: &default
- "locales/**"
- "bin/**/!(*.md)"
actionlint:
- added|modified: ".github/workflows/*.yml"
- added|modified: ".github/workflows/*.yaml"
workflows:
- added|modified: ".github/workflows/*.{yml,yaml}"
shared_sources: &shared_sources
- "src/**/*.cljc"
@@ -65,6 +64,9 @@ frontend_specs: &frontend_specs
- "frontend/**/*.unit.*"
- "enterprise/frontend/**/*.unit.*"
- ".github/**/*.unit.spec.*"
# file-paths.unit.spec.ts asserts what this file's own patterns match, so a change to it has to
# run the specs that guard it.
- ".github/file-paths.yaml"
- "jest.config.js"
- "jest.tz.unit.conf.json"
@@ -113,11 +115,15 @@ backend_sources: &backend_sources
- "src/**"
- "modules/drivers/{*,*/{*,!(test)/**}}"
- "resources/**"
- ".clj-kondo/**"
# The ratchets file drives project_ratchet_checks alone, so the automation PR that lowers the budgets
# runs the Project tests job by itself. Patterns are OR-ed, so a separate "!..." entry would match
# every other path instead of excluding this one.
- ".clj-kondo/{!(ratchets.edn),*/**}"
backend_specs: &backend_specs
- *shared_specs
- "test/**"
- "dev/test/**"
- "enterprise/backend/test/**"
- "test_modules/**"
- "modules/drivers/*/test/**"
@@ -130,6 +136,45 @@ backend_all: &backend_all
- *backend_sources
- *backend_specs
# Project checks have narrower triggers than the backend matrix.
# Changes to their shared CI entry points exercise every suite; otherwise each
# suite follows only the files that can affect its result.
project_tests_infra: &project_tests_infra
- ".github/actions/prepare-backend/**"
- ".github/file-paths.yaml"
- ".github/workflows/backend.yml"
- ".github/workflows/run-tests.yml"
- "bb.edn"
- "bin/mage"
- "bin/mage.bb"
- "mage/src/mage/color.clj"
- "mage/src/mage/project_tests.clj"
- "mage/src/mage/shell.clj"
- "mage/src/mage/util.clj"
project_backend_checks:
- *project_tests_infra
- "{src,test,enterprise,dev,bin,mage}/**/*.{clj,cljc,cljs}"
- "modules/**/*.clj"
- "modules/**/*.cljc"
- "modules/**/*.cljs"
- "**deps.edn"
- ".clj-kondo/config.edn"
- ".clj-kondo/config/modules/config.edn"
- ".github/team.json"
- "dev/resources/dev/modules_config_test/**"
# Only the ratchet check reads the budgets, so a diff that touches nothing else runs that suite alone.
project_ratchet_checks:
- ".clj-kondo/ratchets.edn"
project_migration_checks:
- *project_tests_infra
- "bin/lint-migrations-file/**"
- "resources/migrations/**/*.{yaml,yml}"
- "src/metabase/app_db/custom_migrations.clj"
- "src/metabase/app_db/custom_migrations/**"
sources: &sources
- *shared_sources
- *frontend_sources

View File

@@ -50,7 +50,11 @@ digest() {
}
shopt -s nullglob
DEPS_FILES=(deps.edn modules/drivers/deps.edn modules/drivers/*/deps.edn)
DEPS_FILES=(deps.edn
bin/lint-migrations-file/deps.edn
bin/load-namespaces/deps.edn
modules/drivers/deps.edn
modules/drivers/*/deps.edn)
shopt -u nullglob
DEPS_HASH="$(digest "${DEPS_FILES[@]}")"
@@ -70,6 +74,10 @@ if [ "$CYPRESS_MATCHES" != "1" ]; then
fi
CYPRESS_VERSION="$CYPRESS_VERSIONS"
# The Clojure CLI version CI installs. It lives here rather than in prepare-backend so that the version
# and the key that identifies its cache entry cannot drift apart.
CLOJURE_VERSION="1.12.0.1488"
# An incremental lint cache is only ever an accelerator, so it keys on the commit and always falls back
# to the newest entry on the prefix. The exact key never pre-exists, which keeps the entry current.
ESLINT_SHA="${GITHUB_SHA:-$(git rev-parse HEAD)}"
@@ -96,6 +104,15 @@ spec() {
echo "bun-store-key=bun-store-$OS-$LOCK_HASH"
echo "bun-store-restore-key=bun-store-$OS-"
# No restore-key: an entry from another version installs a working `clojure` of the wrong version, and
# the freshness check in prepare-backend - running the binary - would accept it.
#
# The installer bakes this prefix into the clj/clojure shims, so the tree only works when restored to
# the same absolute path it was built at; prepare-backend reinstalls when it is not.
echo "clojure-path=$HOME/.clojure-cli"
echo "clojure-key=clojure-cli-$OS-$CLOJURE_VERSION"
echo "clojure-version=$CLOJURE_VERSION"
echo "cypress-path=$HOME/.cache/Cypress"
echo "cypress-key=cypress-$OS-$CYPRESS_VERSION"
echo "cypress-restore-key=cypress-$OS-"

71
.github/scripts/check-job-timeouts.sh vendored Executable file
View File

@@ -0,0 +1,71 @@
#!/usr/bin/env bash
# Fails when a job in .github/workflows/ does not set `timeout-minutes`.
#
# Without it a job inherits GitHub's 6-hour default, so a hung job burns six hours of runner time -
# and holds up everything queued behind it - before anyone notices. actionlint cannot check this: its
# rule set is compiled in and .github/actionlint.yaml only takes `self-hosted-runner`,
# `config-variables`, and `paths` (which suppresses findings rather than adding them). Hence this
# script.
#
# The rule is per job: a `timeout-minutes` on a step does not satisfy it, and steps are not required
# to carry one of their own.
#
# Jobs that call a reusable workflow (`uses:` at the job level) are skipped - GitHub rejects
# `timeout-minutes` there. The called workflow's own jobs are checked when it lives in this repo.
#
# .github/scripts/check-job-timeouts.sh [workflow-file ...]
#
# With no arguments it checks every workflow; CI passes the files that changed, like actionlint does.
set -euo pipefail
# Arguments are used as given, so the paths in the annotations match what the caller passed. Without
# them the full scan has to run from the repo root.
if [ $# -gt 0 ]; then
workflows=("$@")
else
cd "$(git rev-parse --show-toplevel)"
shopt -s nullglob
workflows=(.github/workflows/*.yml .github/workflows/*.yaml)
if [ ${#workflows[@]} -eq 0 ]; then
echo "::error::check-job-timeouts.sh: no workflow files found under .github/workflows/" >&2
exit 1
fi
fi
status=0
for file in "${workflows[@]}"; do
# `awk NF` drops the blank line yq emits when a file has nothing to report.
missing="$(
# --header-preprocess=false: yq otherwise slurps any leading comment block before parsing, which
# shifts every line number it reports by the size of that block.
yq --header-preprocess=false '
.jobs
| to_entries
| .[]
| select(((.value | has("timeout-minutes")) or (.value | has("uses"))) | not)
| [(.key | line), .key]
| @tsv
' "$file" | awk NF
)"
[ -n "$missing" ] || continue
while IFS=$'\t' read -r line job; do
echo "::error file=${file},line=${line}::Job \"${job}\" does not set a job-level timeout-minutes" >&2
done <<< "$missing"
status=1
done
if [ "$status" -ne 0 ]; then
echo >&2
echo "Every job must set an explicit timeout-minutes; without one it inherits GitHub's 6-hour default." >&2
echo "Pick a value with headroom over the job's normal runtime." >&2
exit 1
fi
echo "All jobs in the ${#workflows[@]} workflow file(s) checked set timeout-minutes."

View File

@@ -1,6 +1,5 @@
#!/usr/bin/env bash
# Fails when a `clojure` invocation under .github/ uses an alias combination that cache-warm does not
# pre-resolve.
# Fails when a Clojure classpath used by CI is not pre-resolved by cache-warm.
#
# tools.deps selects dependency versions globally per classpath, so two alias combinations can resolve
# the same transitive library to different versions. A combination the warm job never resolved is
@@ -30,10 +29,18 @@ EOF
# :deps - tools.deps built-in used by write-poms.sh; contributes no dependencies of its own.
# :outdated - scheduled dependency-bump workflow only; a cold resolve there costs nothing anyone waits on.
# Alias combinations the warm job pre-resolves.
declared="$(sed -n '/combos=(/,/^ *)/p' "$WARM" | grep -oE '":[^"]*"' | tr -d '"' | sort -u)"
# Dependency roots and alias combinations the warm job pre-resolves.
declared="$({
sed -n '/^[[:space:]]*combos=(/,/^ *)/p' "$WARM" \
| grep -oE '":[^"]*"' \
| tr -d '"' \
| sed 's#^#.|#'
sed -n '/^[[:space:]]*nested_combos=(/,/^ *)/p' "$WARM" \
| grep -oE '"[^"]+\|:[^"]+"' \
| tr -d '"'
} | sort -u)"
if [ -z "$declared" ]; then
echo "::error::check-preresolve-aliases.sh: found no combos=( ... ) list in $WARM" >&2
echo "::error::check-preresolve-aliases.sh: found no pre-resolved classpaths in $WARM" >&2
exit 1
fi
@@ -43,27 +50,60 @@ unknown="$(grep -rhoE '\$\{\{[^}]*\}\}' --include='*' \
--exclude='*.png' --exclude='*.jpg' .github/ 2>/dev/null \
| sort -u | grep -v 'edition' || true)"
scan() { # scan <edition>: alias strings used by clojure invocations, with expressions substituted
scan() { # scan <edition>: dependency root and aliases used by .github Clojure invocations
grep -rhE 'clojure[[:space:]]+(-P[[:space:]]+)?-[XMATP]?:' \
--include='*' --exclude='*.png' --exclude='*.jpg' .github/ 2>/dev/null \
| sed "s/\\\${{[^}]*edition[^}]*}}/$1/g" \
| grep -oE 'clojure[[:space:]]+(-P[[:space:]]+)?-[XMATP]?:[A-Za-z0-9:_.-]+' \
| grep -oE ':[A-Za-z0-9:_.-]+' \
| sed 's/:$//'
| grep -oE 'clojure[[:space:]]+(-P[[:space:]]+)?-[XMATP]?:[A-Za-z0-9:_./-]+' \
| grep -oE ':[A-Za-z0-9:_./-]+' \
| sed -e 's/:$//' -e 's#^#.|#'
}
found="$( { scan ee; scan oss; } | sort -u )"
scan_project_tests() { # scan_project_tests <function> <dependency-root>
# Mage shell commands are vectors split across lines, so flatten each function body before
# extracting its alias.
sed -n "/(defn- $1 /,/^(defn/p" mage/src/mage/project_tests.clj \
| tr '\n' ' ' \
| grep -oE '"clojure"[[:space:]]+("-P"[[:space:]]+)?"-[XMATP]?:[A-Za-z0-9:_./-]+' \
| grep -oE ':[A-Za-z0-9:_./-]+' \
| sed -e 's/:$//' -e "s#^#$2|#"
}
covered="$(printf '%s\n%s\n' "$declared" "$WAIVED" | sort -u)"
# Check each scan separately. One scan still finding commands would otherwise conceal that another has
# stopped matching, which silently drops everything it used to cover.
require_scan() { # require_scan <what> <result>
[ -n "$2" ] && return
echo "::error::check-preresolve-aliases.sh: found no Clojure invocation in $1; the scan pattern no longer matches" >&2
exit 1
}
workflow_checks=$( { scan ee; scan oss; } || true)
clojure_checks=$(scan_project_tests run-clojure-checks! . || true)
migration_checks=$(scan_project_tests run-migration-checks! bin/lint-migrations-file || true)
require_scan ".github/" "$workflow_checks"
require_scan "run-clojure-checks! (mage/src/mage/project_tests.clj)" "$clojure_checks"
require_scan "run-migration-checks! (mage/src/mage/project_tests.clj)" "$migration_checks"
scanned="$(printf '%s\n' "$workflow_checks" "$clojure_checks" "$migration_checks" | sort -u)"
found="$( {
printf '%s\n' "$scanned"
# build-scripts.yml runs this classpath from its own dependency root.
printf '%s\n' 'bin/load-namespaces|:test'
} | sort -u )"
covered="$( {
printf '%s\n' "$declared"
printf '%s\n' "$WAIVED" | sed 's#^#.|#'
} | sort -u)"
missing="$(comm -23 <(printf '%s\n' "$found") <(printf '%s\n' "$covered") || true)"
if [ -n "$missing" ]; then
echo "::error::Alias combinations used under .github/ that cache-warm does not pre-resolve:" >&2
echo "::error::Clojure classpaths used by CI that cache-warm does not pre-resolve:" >&2
printf '%s\n' "$missing" | sed 's/^/ /' >&2
echo >&2
echo "Add each to the combos list in $WARM, or waive it in this script with a reason." >&2
echo "Add each to the matching root or nested combo list in $WARM, or waive it here with a reason." >&2
echo "Leaving it uncovered means those jobs re-fetch their version deltas from Maven on every run." >&2
exit 1
fi
echo "All $(printf '%s\n' "$found" | grep -c .) clojure alias combinations under .github/ are pre-resolved."
echo "All $(printf '%s\n' "$found" | grep -c .) Clojure classpaths used by CI are pre-resolved."

57
.github/scripts/file-paths.unit.spec.ts vendored Normal file
View File

@@ -0,0 +1,57 @@
import { readFileSync } from "fs";
import { join } from "path";
import { load } from "js-yaml";
import { isMatch } from "micromatch";
const FILTERS = join(__dirname, "..", "file-paths.yaml");
type Filter = string | Filter[] | { [changeType: string]: Filter };
// A filter is a list of patterns, and an entry may name the change types it applies to as
// { "added|modified": pattern }. Either way the patterns are what matters here.
const patterns = (value: Filter): string[] => {
if (typeof value === "string") return [value];
if (Array.isArray(value)) return value.flatMap(patterns);
return Object.values(value).flatMap(patterns);
};
describe("file-paths.yaml", () => {
const filters = load(readFileSync(FILTERS, "utf8")) as Record<string, Filter>;
// dorny/paths-filter matches with picomatch, which micromatch wraps, so this asks the same
// question CI asks of a changed path.
const matches = (name: string, path: string) =>
patterns(filters[name]).some((pattern) =>
isMatch(path, pattern, { dot: true }),
);
// dorny/paths-filter ORs the patterns in a filter, so a "!..." entry does not subtract from the
// other entries - it matches every path they don't, turning the filter permanently on. Excluding
// a path means writing the exclusion inside one pattern, the way backend_sources keeps
// ratchets.edn out with ".clj-kondo/{!(ratchets.edn),*/**}".
it("never excludes a path with a standalone negation", () => {
const negated = Object.entries(filters).flatMap(([name, filter]) =>
patterns(filter)
.filter((pattern) => pattern.startsWith("!"))
.map((pattern) => `${name}: ${pattern}`),
);
expect(negated).toEqual([]);
});
it.each(["backend_sources", "backend_all"])(
"%s skips a ratchets-only change and still follows the rest of .clj-kondo",
(name) => {
expect(matches(name, ".clj-kondo/ratchets.edn")).toBe(false);
expect(matches(name, ".clj-kondo/config.edn")).toBe(true);
expect(matches(name, ".clj-kondo/config/modules/config.edn")).toBe(true);
},
);
it("runs the ratchet check on the ratchets file", () => {
expect(matches("project_ratchet_checks", ".clj-kondo/ratchets.edn")).toBe(
true,
);
});
});

4
.github/team.json vendored
View File

@@ -110,7 +110,9 @@
"uladzimirdev",
"egorgrushin",
"metatyb",
"olehk-mb"
"olehk-mb",
"annatomka",
"brybacki"
]
},
{

View File

@@ -1,5 +1,5 @@
name: AI Benchmark
run-name: "dispatch evals · ${{ github.event_name == 'pull_request' && format('smoke PR#{0}', github.event.pull_request.number) || (inputs.ref != '' && inputs.ref || (inputs.pr != '' && format('PR#{0}', inputs.pr) || 'master')) }}"
run-name: "Evals · ${{ github.event_name == 'pull_request' && 'label' || 'manual' }} · ${{ github.event.pull_request.number || inputs.pr || inputs.ref || github.sha }}"
on:
pull_request:
@@ -33,23 +33,59 @@ concurrency:
cancel-in-progress: true
jobs:
unsupported:
prepare:
if: >-
github.event_name == 'pull_request'
github.event.pull_request.head.repo.full_name == github.repository
&& github.event_name == 'pull_request'
&& github.event.label.name == 'evals/smoke'
&& github.event.pull_request.head.repo.full_name != github.repository
|| github.event_name == 'workflow_dispatch'
runs-on: ${{ vars.SLIM_RUNNER_KEY }}
timeout-minutes: 5
outputs:
pr: ${{ steps.prepare.outputs.pr }}
ref: ${{ steps.prepare.outputs.ref }}
scope: ${{ steps.prepare.outputs.scope }}
sample: ${{ steps.prepare.outputs.sample }}
evals_branch: ${{ steps.prepare.outputs.evals_branch }}
steps:
- run: |
echo "::error::evals/smoke only runs on same-repo PRs; this PR's head is on a fork"
exit 1
- name: Prepare
id: prepare
env:
EVENT_NAME: ${{ github.event_name }}
LABEL_PR: ${{ github.event.pull_request.number }}
INPUT_PR: ${{ inputs.pr }}
INPUT_REF: ${{ inputs.ref }}
INPUT_SCOPE: ${{ inputs.scope }}
INPUT_SAMPLE: ${{ inputs.sample }}
INPUT_EVALS_BRANCH: ${{ inputs.evals_branch }}
run: |
set -euo pipefail
# The two paths are mutually exclusive, and `on:` admits no other trigger.
if [[ "$EVENT_NAME" == 'pull_request' ]]; then
PR="$LABEL_PR"; REF=''; SCOPE='smoke'
SAMPLE='5'; EVALS_BRANCH='main'
else
PR="$INPUT_PR"; REF="$INPUT_REF"; SCOPE="$INPUT_SCOPE"
SAMPLE="${INPUT_SAMPLE:-5}"; EVALS_BRANCH="$INPUT_EVALS_BRANCH"
fi
# None of these is ever multi-line; a newline would forge extra output entries.
for v in PR REF SCOPE SAMPLE EVALS_BRANCH; do
if [[ "${!v}" == *$'\n'* ]]; then
echo "::error::${v} must be a single line"
exit 1
fi
done
{
echo "pr=$PR"
echo "ref=$REF"
echo "scope=$SCOPE"
echo "sample=$SAMPLE"
echo "evals_branch=$EVALS_BRANCH"
} >> "$GITHUB_OUTPUT"
announce:
if: >-
github.event_name == 'pull_request'
&& github.event.label.name == 'evals/smoke'
&& github.event.pull_request.head.repo.full_name == github.repository
needs: prepare
if: github.event_name == 'pull_request'
runs-on: ${{ vars.SLIM_RUNNER_KEY }}
timeout-minutes: 5
outputs:
@@ -75,11 +111,7 @@ jobs:
core.setOutput('comment_id', String(data.id));
resolve:
# Collapse the dispatch inputs into the one full sha everything downstream needs: it keys the
# uberjar artifact, tags the ECR image, and tells eval-run which commit was measured. `pr` was
# not consulted at all before, so `pr: 12345` built the tip of whatever branch the run was
# launched from and then reported the result against the PR.
if: github.event_name == 'workflow_dispatch'
needs: prepare
runs-on: ${{ vars.DEFAULT_RUNNER_KEY }}
timeout-minutes: 5
permissions:
@@ -92,8 +124,8 @@ jobs:
id: sha
env:
GH_TOKEN: ${{ github.token }}
PR: ${{ inputs.pr }}
REF: ${{ inputs.ref }}
PR: ${{ needs.prepare.outputs.pr }}
REF: ${{ needs.prepare.outputs.ref }}
DEFAULT_SHA: ${{ github.sha }}
run: |
set -euo pipefail
@@ -119,75 +151,53 @@ jobs:
echo "sha=${SHA}" >> "$GITHUB_OUTPUT"
image:
needs: [resolve]
# `resolve` is workflow_dispatch-only, so tolerate it being skipped. The label guard is not
# optional: the trigger is `pull_request: [labeled]` with no filter of its own, so without it
# any label on any same-repo PR builds ee+oss and pushes to ECR — and since announce/smoke/
# finalize all skip in that case, a failure shows up as a bare red check with nothing
# explaining it.
if: >-
!cancelled()
&& (needs.resolve.result == 'success' || needs.resolve.result == 'skipped')
&& (github.event_name == 'workflow_dispatch'
|| (github.event_name == 'pull_request'
&& github.event.label.name == 'evals/smoke'
&& github.event.pull_request.head.repo.full_name == github.repository))
needs: resolve
uses: ./.github/workflows/build-private-ee-docker-image.yml
secrets:
PR_ENV_IAM_ROLE: ${{ secrets.PR_ENV_IAM_ROLE }}
PR_ENV_AWS_ACCOUNT_ID: ${{ secrets.PR_ENV_AWS_ACCOUNT_ID }}
with:
# Empty on the label path on purpose — the image workflow then falls back to the PR head,
# which is the artifact run-tests has already built and cached under that sha.
commit: ${{ needs.resolve.outputs.sha }}
smoke:
needs: [announce, image]
# `!cancelled()` is load-bearing, not decoration: on the label path `resolve` is skipped by
# design, and `image` only runs at all because it opts back in with its own `!cancelled()`.
# GitHub propagates that skip *transitively*, so a plain boolean `if` here — carrying the
# implicit `success()` — skips smoke even though image succeeded and the image is in ECR.
# Opting out of the implicit check means the needs' results have to be asserted by hand.
dispatch:
needs: [prepare, announce, resolve, image]
if: >-
!cancelled()
&& needs.announce.result == 'success'
&& needs.prepare.result == 'success'
&& (needs.announce.result == 'success' || needs.announce.result == 'skipped')
&& needs.resolve.result == 'success'
&& needs.image.result == 'success'
&& github.event_name == 'pull_request'
&& github.event.label.name == 'evals/smoke'
&& github.event.pull_request.head.repo.full_name == github.repository
runs-on: ${{ vars.SLIM_RUNNER_KEY }}
timeout-minutes: 5
env:
COMMENT_ID: ${{ needs.announce.outputs.comment_id }}
runs-on: ${{ vars.SLIM_RUNNER_KEY }}
steps:
- name: Dispatch smoke eval-run
- name: Dispatch eval-run
env:
GH_TOKEN: ${{ secrets.METABASE_AUTOMATION_USER_TOKEN }}
PR: ${{ github.event.pull_request.number }}
HEAD_SHA: ${{ github.event.pull_request.head.sha }}
EVALS_BRANCH: main
EVALS_BRANCH: ${{ needs.prepare.outputs.evals_branch }}
PR: ${{ needs.prepare.outputs.pr }}
SHA: ${{ needs.resolve.outputs.sha }}
SCOPE: ${{ needs.prepare.outputs.scope }}
SAMPLE: ${{ needs.prepare.outputs.sample }}
COMMENT_ID: ${{ needs.announce.outputs.comment_id }}
run: |
gh api repos/metabase/evals/actions/workflows/eval-run.yml/dispatches \
-f ref="$EVALS_BRANCH" \
-f "inputs[pr]=$PR" \
-f "inputs[ref]=$HEAD_SHA" \
-f "inputs[scope]=smoke" \
-f "inputs[ref]=$SHA" \
-f "inputs[scope]=$SCOPE" \
-f "inputs[surfaces]=all" \
-f "inputs[sample]=5" \
-f "inputs[sample]=$SAMPLE" \
-f "inputs[evals_branch]=$EVALS_BRANCH" \
-f "inputs[comment_id]=$COMMENT_ID"
finalize:
# If the build or dispatch fell over, eval-run never took over the comment — leave a
# failure verdict instead of a comment stuck at "building the image…".
needs: [announce, image, smoke]
if: always() && needs.announce.result == 'success' && needs.smoke.result != 'success'
needs: [announce, resolve, image, dispatch]
if: always() && needs.announce.result == 'success' && needs.dispatch.result != 'success'
runs-on: ${{ vars.SLIM_RUNNER_KEY }}
timeout-minutes: 5
env:
COMMENT_ID: ${{ needs.announce.outputs.comment_id }}
SMOKE_RESULT: ${{ needs.smoke.result }}
IMAGE_RESULT: ${{ needs.image.result }}
NEEDS_JSON: ${{ toJSON(needs) }}
steps:
- name: Comment (build/dispatch failed)
uses: actions/github-script@v9
@@ -195,11 +205,11 @@ jobs:
github-token: ${{ secrets.METABASE_AUTOMATION_USER_TOKEN }}
script: |
const runUrl = `${context.serverUrl}/${context.repo.owner}/${context.repo.repo}/actions/runs/${context.runId}`;
// A concurrency-cancelled retrigger lands here too; don't stamp it as an infra failure.
const cancelled = [process.env.SMOKE_RESULT, process.env.IMAGE_RESULT].includes('cancelled');
const cancelled = Object.values(JSON.parse(process.env.NEEDS_JSON))
.some(j => j.result === 'cancelled');
const body = cancelled
? `🤖 **Evals smoke** — ⚪ cancelled before evals started ([logs](${runUrl}))`
: `🤖 **Evals smoke** — ❌ build or dispatch failed before evals started ([logs](${runUrl}))`;
? `🤖 **Evals** — ⚪ cancelled before evals started ([logs](${runUrl}))`
: `🤖 **Evals** — ❌ build or dispatch failed before evals started ([logs](${runUrl}))`;
// The replacement run deletes this comment on its way in, so a concurrency cancel can
// land here with nothing to patch. Losing the stamp is fine; failing the job isn't.
await github.rest.issues.updateComment({
@@ -214,30 +224,3 @@ jobs:
issue_number: context.payload.pull_request.number,
name: 'evals/smoke',
}).catch(() => {});
dispatch:
needs: [resolve, image]
# Deliberately no `always()`: benchmarking without the image this run was supposed to produce
# would measure a stale build or nothing at all. A failed `image` fails the run, which is the
# signal here — unlike the PR path there is no comment for `finalize` to stamp.
if: github.event_name == 'workflow_dispatch'
runs-on: ${{ vars.SLIM_RUNNER_KEY }}
timeout-minutes: 5
steps:
- name: Dispatch eval-run
env:
GH_TOKEN: ${{ secrets.METABASE_AUTOMATION_USER_TOKEN }}
EVALS_BRANCH: ${{ github.event.inputs.evals_branch }}
PR: ${{ inputs.pr }}
# The sha the image was actually built and tagged with, not the raw `ref` input.
REF: ${{ needs.resolve.outputs.sha }}
SCOPE: ${{ inputs.scope }}
SAMPLE: ${{ inputs.sample }}
run: |
gh api repos/metabase/evals/actions/workflows/eval-run.yml/dispatches \
-f ref="$EVALS_BRANCH" \
-f "inputs[pr]=$PR" \
-f "inputs[ref]=$REF" \
-f "inputs[scope]=$SCOPE" \
-f "inputs[sample]=${SAMPLE:-5}" \
-f "inputs[evals_branch]=$EVALS_BRANCH"

View File

@@ -332,7 +332,6 @@ jobs:
build-static-viz: true
test-args: >-
:only '"test"'
:exclude-tags [:mb/driver-tests]
:partition/total 2
:partition/index 0
exclude-tag: ':mb/driver-tests :mb/transforms-python-test'

View File

@@ -10,6 +10,18 @@ on:
description: Run the full suite regardless of the diff (master / release branches).
type: boolean
default: false
run-project-backend-checks:
description: Run project-level backend checks.
type: boolean
default: false
run-project-migration-checks:
description: Run project-level migration checks.
type: boolean
default: false
run-project-ratchet-checks:
description: Run the ratchet check for a diff that touches nothing but the ratchets file.
type: boolean
default: false
# Run the backend test suite on demand, against the ref you dispatch on, optionally under a
# profiler. Everything else about the jobs -- matrix legs, tag filters, quarantine gating -- is
@@ -46,6 +58,95 @@ env:
ONLY_TESTS: ${{ inputs.tests }}
jobs:
project-checks:
name: 'Project tests: Suites'
if: >-
${{ inputs.force-run ||
inputs.run-project-backend-checks ||
inputs.run-project-migration-checks ||
inputs.run-project-ratchet-checks }}
runs-on: ${{ vars.DEFAULT_RUNNER_KEY }}
timeout-minutes: 15
steps:
- uses: actions/checkout@v6
- name: Prepare back-end environment
uses: ./.github/actions/prepare-backend
# Bootstrap the shared Mage runtime before concurrent commands try to install it themselves.
- name: Prepare Mage
run: ./bin/mage
- parallel:
- name: Run backend checks
if: ${{ inputs.force-run || inputs.run-project-backend-checks }}
run: ./bin/mage project-tests backend
- name: Run migration checks
if: ${{ inputs.force-run || inputs.run-project-migration-checks }}
run: ./bin/mage project-tests migrations
# Babashka, not a JVM test run, so it is cheap enough to run alongside the backend checks.
# It still needs the JDK prepare-backend installs, to resolve clj-kondo on a cold cache.
- name: Run ratchet checks
if: ${{ inputs.force-run || inputs.run-project-backend-checks || inputs.run-project-ratchet-checks }}
run: ./bin/mage kondo-ratchets
project-tests:
name: Project tests
needs: project-checks
if: ${{ always() && !cancelled() }}
runs-on: ${{ vars.SLIM_RUNNER_KEY }}
timeout-minutes: 5
steps:
- name: Check Project tests status
uses: actions/github-script@v9
env:
needs: ${{ toJson(needs) }}
with:
script: | # js
const needs = JSON.parse(process.env.needs);
const jobs = Object.entries(needs).map(
([jobName, jobValues]) => ({
name: jobName,
result: jobValues.result
}));
if (jobs.every(job => job.result === 'skipped' || job.result === 'success')) {
console.log('');
console.log(' |>>> |>>> |>>>');
console.log(' | | |');
console.log(' _ _|_ _ _ _|_ _ _ _|_ _');
console.log(' | |_| |_| |______| |_| |_| |______| |_| |_| |');
console.log(' | | | | | |');
console.log(' | [] [] | | [] [] | | [] [] |');
console.log(' | |______| |______| |');
console.log(' | ___ |');
console.log(' |___________________| |___________________|');
console.log(' |___|');
console.log('');
console.log('Everything is up to code and structurally sound.');
process.exit(0);
}
console.log('');
console.log(' .-===========-.____');
console.log(' /_______________\\ \\____');
console.log(' | .-----------. | \\___');
console.log(' | | x x | | \\__');
console.log(' | | ~ | | _\\_');
console.log(" | `-----------' | /___\\");
console.log(' | o o |');
console.log(" `-----------------'");
console.log(' /=============\\');
console.log(' / O O O O O O O \\');
console.log(' \\_______________/');
console.log('');
console.log('One or more project checks are off the blueprint. Hard hats required.');
jobs.forEach((job) => {
if (job.result !== 'success' && job.result !== 'skipped') {
console.log(`${job.name} - ${job.result}`);
}
});
process.exit(1);
be-linter-clj-kondo:
if: ${{ !inputs.skip }}
runs-on: ${{ vars.DEFAULT_RUNNER_KEY }}
@@ -80,7 +181,7 @@ jobs:
- uses: actions/checkout@v6
- name: Prepare back-end environment
uses: ./.github/actions/prepare-backend
- run: clojure -X:dev:ee:ee-dev:drivers:drivers-dev:test:eastwood/test
- run: clojure -X:dev:dev/test:ee:ee-dev:drivers:drivers-dev:test:eastwood/test
name: Run Eastwood linter on our backend tests
be-linter-eastwood-test-changed:
@@ -108,13 +209,14 @@ jobs:
# last synced don't widen our scope. Falls back to BASE_SHA if older than fetched depth.
git fetch --no-tags --depth=200 origin "$BASE_SHA"
DIFF_BASE=$(git merge-base "$BASE_SHA" "$HEAD_SHA" 2>/dev/null || echo "$BASE_SHA")
# Test roots we lint: core, EE, and each driver module's test tree.
# Test roots we lint: core, EE, dev/test, and each driver module's test tree.
# :eastwood/test derives matching :source-paths from the classpath, so any
# namespace we name here is reachable. Include renames (R) so a test that
# was just renamed still gets linted under its new name.
FILES=$(git diff --name-only --diff-filter=AMR "$DIFF_BASE" "$HEAD_SHA" \
-- 'test/**/*.clj' 'test/**/*.cljc' \
'enterprise/backend/test/**/*.clj' 'enterprise/backend/test/**/*.cljc' \
'dev/test/**/*.clj' 'dev/test/**/*.cljc' \
'modules/drivers/**/test/**/*.clj' 'modules/drivers/**/test/**/*.cljc')
if [ -z "$FILES" ]; then
@@ -130,7 +232,14 @@ jobs:
exit 1
fi
NS=$(sed -E 's#^(test/|enterprise/backend/test/|modules/drivers/[^/]+/test/)##; s#\.cljc?$##; s#/#.#g; s#_#-#g' <<<"$FILES" \
# Convert each test path to a Clojure namespace: strip its known test root and
# extension, then translate path separators and underscores. Keep these roots
# aligned with FILES above and the Eastwood classpath aliases below.
NS=$(sed -E \
-e 's#^(test/|enterprise/backend/test/|dev/test/|modules/drivers/[^/]+/test/)##' \
-e 's#\.cljc?$##' \
-e 's#/#.#g' \
-e 's#_#-#g' <<<"$FILES" \
| sort -u \
| paste -sd' ' -)
echo "Changed test namespaces: $NS"
@@ -139,7 +248,7 @@ jobs:
if: steps.changed.outputs.namespaces != ''
env:
NAMESPACES: ${{ steps.changed.outputs.namespaces }}
run: clojure -X:dev:ee:ee-dev:drivers:drivers-dev:test:eastwood/test :namespaces "[$NAMESPACES]"
run: clojure -X:dev:dev/test:ee:ee-dev:drivers:drivers-dev:test:eastwood/test :namespaces "[$NAMESPACES]"
be-tests:
if: ${{ !inputs.skip }}
@@ -368,6 +477,7 @@ jobs:
be-tests-result:
needs:
- project-tests
- be-linter-eastwood
- be-linter-eastwood-test
- be-linter-eastwood-test-changed

View File

@@ -136,5 +136,7 @@ jobs:
context: bin/docker/
platforms: linux/amd64
network: host
build-args: |
GIT_COMMIT_SHA=${{ steps.meta.outputs.tag }}
tags: ${{ secrets.PR_ENV_AWS_ACCOUNT_ID }}.dkr.ecr.us-east-1.amazonaws.com/metabase-enterprise:${{ steps.meta.outputs.tag }}
push: true

View File

@@ -79,6 +79,7 @@ jobs:
test-build-scripts:
runs-on: ${{ vars.DEFAULT_RUNNER_KEY }}
timeout-minutes: 40
steps:
- uses: actions/checkout@v6
- name: Prepare back-end environment
@@ -88,10 +89,6 @@ jobs:
- name: Run build and release script tests
run: clojure -X:dev:drivers:build:build-dev:build-test:ci
timeout-minutes: 15
- name: Run Liquibase migrations linter tests
run: clojure -M:test
working-directory: bin/lint-migrations-file
timeout-minutes: 15
- name: Run load-namespaces tests
run: clojure -M:test
working-directory: bin/load-namespaces

View File

@@ -1,4 +1,4 @@
# Writes the three large dependency caches. Every other job restores them and none of them writes.
# Writes the shared dependency caches. Every other job restores them and none of them writes.
#
# A single writer rather than the usual let-every-job-save arrangement, because a pull request here fans
# out to roughly 200 jobs, ~75 of which install Cypress. Letting each of those save would have them all
@@ -13,7 +13,7 @@
# at it, and master's copy backs up every ref that has none of its own.
#
# Keys are content-derived (see .github/scripts/cache-keys.sh), so a push whose dependencies already have
# an entry does nothing but three lookups. There is no cron: nothing here is keyed by date.
# an entry does nothing but four lookups. There is no cron: nothing here is keyed by date.
#
#
# PRINCIPLES:
@@ -69,6 +69,13 @@ jobs:
path: ${{ steps.cache-spec.outputs.bun-store-path }}
key: ${{ steps.cache-spec.outputs.bun-store-key }}
lookup-only: true
- name: Look up Clojure CLI
id: clojure-lookup
uses: actions/cache/restore@v5
with:
path: ${{ steps.cache-spec.outputs.clojure-path }}
key: ${{ steps.cache-spec.outputs.clojure-key }}
lookup-only: true
- name: Look up Cypress binary
id: cypress-lookup
uses: actions/cache/restore@v5
@@ -77,10 +84,19 @@ jobs:
key: ${{ steps.cache-spec.outputs.cypress-key }}
lookup-only: true
# Also runs when only the Clojure CLI entry is missing: installing it is what there is to save,
# and the pre-resolve below needs it either way.
- name: Prepare backend
if: steps.m2-lookup.outputs.cache-hit != 'true'
if: steps.m2-lookup.outputs.cache-hit != 'true' || steps.clojure-lookup.outputs.cache-hit != 'true'
uses: ./.github/actions/prepare-backend
- name: Save Clojure CLI
if: steps.clojure-lookup.outputs.cache-hit != 'true'
uses: actions/cache/save@v5
with:
path: ${{ steps.cache-spec.outputs.clojure-path }}
key: ${{ steps.cache-spec.outputs.clojure-key }}
# Pre-resolve every Clojure dependency classpath CI actually uses, so the saved cache is a SUPERSET
# of what each job resolves at runtime.
#
@@ -102,7 +118,9 @@ jobs:
combos=(
":build:dev:ee:ee-dev:drivers:drivers-dev:cljs" # build + uberjar (combined classpath)
":cljs" # shadow-cljs (build:cljs); resolves :cljs alone -> covers asm:9.2
":dev:ee:ee-dev:drivers:drivers-dev:test:eastwood" # backend.yml (eastwood + eastwood/test)
":dev:ee:ee-dev:drivers:drivers-dev:test:eastwood" # backend.yml (eastwood)
":dev:dev/test:ee:ee-dev:drivers:drivers-dev:test:eastwood/test" # backend.yml (eastwood tests)
":dev:dev/test:ee:ee-dev:drivers:drivers-dev:test:ci" # run-tests.yml (project tests)
":dev:ci:ee:ee-dev:drivers:drivers-dev:test" # test-driver.yml (ee) -> covers data.json:2.4.0
":dev:ci:oss:oss-dev:drivers:drivers-dev:test" # test-driver.yml (oss)
":dev:ci:test:ee:ee-dev" # backend.yml matrix (ee)
@@ -124,6 +142,20 @@ jobs:
echo "::endgroup::"
done
# These commands use nested tools.deps projects; resolving the same alias from the
# repository root does not warm their dependencies.
nested_combos=(
"bin/lint-migrations-file|:test" # project migration checks
"bin/load-namespaces|:test" # build-scripts.yml
)
for spec in "${nested_combos[@]}"; do
project=${spec%%|*}
combo=${spec#*|}
echo "::group::${project}: clojure -P -M${combo}"
(cd "$project" && clojure -P -M"${combo}")
echo "::endgroup::"
done
echo "::group::./bin/mage"
./bin/mage
echo "::endgroup::"

View File

@@ -59,6 +59,12 @@ jobs:
# Pushing the new branch triggers set-milestone.yml, which does the
# full milestone backfill and then dispatches release-notes-preview
# to upload the markdown to #writing.
#
# Kondo ignore ratchets apply only to master. The cut commit replaces the
# budgets with an explicit opt-out so future budget changes are not applied
# to the release branch. Keeping the opt-out in the canonical file avoids
# introducing a separate marker that could drift out of sync, and the
# backport helper writes the same opt-out so both paths stay identical.
- name: Cut the branch
id: cut
run: |
@@ -66,6 +72,9 @@ jobs:
git config user.name github-actions
git config user.email github-actions@github.com
git checkout -b "$BRANCH"
source .github/actions/create-backport/kondo-ratchets.sh
write_disabled_ratchets
git add --sparse .clj-kondo/ratchets.edn
git commit --allow-empty --no-verify -m "Cut $BRANCH branch"
git push -u origin "$BRANCH"
echo "sha=$(git rev-parse HEAD)" >> "$GITHUB_OUTPUT"

View File

@@ -75,6 +75,14 @@ jobs:
clojure -M:ee:doc command-documentation
continue-on-error: false
- name: Generate AI provider documentation
id: generate_ai_provider_docs
run: |
set -e # Exit on any error
echo "Generating AI provider documentation..."
clojure -M:ee:doc ai-providers-documentation
continue-on-error: false
- name: Generate usage analytics documentation
run: ./bin/generate-usage-analytics-docs.bb
@@ -164,6 +172,7 @@ jobs:
- \`clojure -M:ee:doc config-template\`
- \`clojure -M:ee:doc api-documentation\`
- \`clojure -M:ee:doc command-documentation\`
- \`clojure -M:ee:doc ai-providers-documentation\`
- \`./bin/generate-usage-analytics-docs.bb\`
- \`bun run embedding-sdk:docs:generate\`
- \`bun run embedding-eajs:docs:generate\`

View File

@@ -329,34 +329,45 @@ jobs:
:only-tags [:mb/driver-tests]
:exclude-tags [:mb/transforms-python-test]
:fail-fast? true
:partition/total 3
:partition/total 2
:partition/index 0
- name: Driver Tests Part 2
test-args: >-
:only-tags [:mb/driver-tests]
:exclude-tags [:mb/transforms-python-test]
:fail-fast? true
:partition/total 3
:partition/total 2
:partition/index 1
- name: Driver Tests Part 3
test-args: >-
:only-tags [:mb/driver-tests]
:exclude-tags [:mb/transforms-python-test]
:fail-fast? true
:partition/total 3
:partition/index 2
env:
CI: 'true'
DRIVERS: databricks
MB_DATABRICKS_TEST_HOST: ${{ secrets.MB_DATABRICKS_JDBC_TEST_HOST }}
MB_DATABRICKS_TEST_HTTP_PATH: ${{ secrets.MB_DATABRICKS_JDBC_TEST_HTTP_PATH }}
MB_DATABRICKS_TEST_TOKEN: ${{ secrets.MB_DATABRICKS_JDBC_TEST_TOKEN }}
MB_DATABRICKS_TEST_CATALOG: 'metabase_ci'
MB_DATABRICKS_TEST_CATALOG_ROUTING: 'metabase_routing_catalog'
MB_DATABRICKS_TEST_CLIENT_ID: ${{ secrets.MB_DATABRICKS_JDBC_TEST_CLIENT_ID }}
MB_DATABRICKS_TEST_OAUTH_SECRET: ${{ secrets.MB_DATABRICKS_JDBC_TEST_OAUTH_SECRET }}
name: Databricks
steps:
- name: Select Databricks workspace
shell: bash
env:
TOTAL_SHARDS: '2'
HOST_0: ${{ secrets.MB_DATABRICKS_JDBC_TEST_HOST }}
TOKEN_0: ${{ secrets.MB_DATABRICKS_JDBC_TEST_TOKEN }}
HTTP_PATH_0: ${{ secrets.MB_DATABRICKS_JDBC_TEST_HTTP_PATH }}
HOST_1: ${{ secrets.MB_DATABRICKS_JDBC_TEST_HOST_2 }}
TOKEN_1: ${{ secrets.MB_DATABRICKS_JDBC_TEST_TOKEN_2 }}
HTTP_PATH_1: ${{ secrets.MB_DATABRICKS_JDBC_TEST_HTTP_PATH_2 }}
run: | # sh
shard=$(( GITHUB_RUN_NUMBER % TOTAL_SHARDS ))
echo "Using Databricks workspace shard $(( shard + 1 )) of $TOTAL_SHARDS"
for name in HOST HTTP_PATH TOKEN; do
var="${name}_${shard}"
if [ -z "${!var}" ]; then
echo "::error::Secret $var is empty - is the shard $shard Databricks workspace configured?"
exit 1
fi
echo "MB_DATABRICKS_TEST_${name}=${!var}" >> "$GITHUB_ENV"
done
- uses: actions/checkout@v6
- name: Test Databricks driver
id: test-driver
@@ -367,59 +378,6 @@ jobs:
test-args: ${{ matrix.job.test-args }}
quarantine-engine: ${{ vars.QUARANTINE_ENGINE }}
be-tests-databricks-multi-catalog-ee:
needs: determine-driver-skips
if: ${{ needs.determine-driver-skips.outputs.databricks-should-run == 'true' }}
runs-on: ${{ vars.DEFAULT_RUNNER_KEY }}
timeout-minutes: 50
strategy:
fail-fast: false
matrix:
job:
- name: Driver Tests Part 1
test-args: >-
:only-tags [:mb/driver-tests]
:exclude-tags [:mb/transforms-python-test]
:fail-fast? true
:partition/total 3
:partition/index 0
- name: Driver Tests Part 2
test-args: >-
:only-tags [:mb/driver-tests]
:exclude-tags [:mb/transforms-python-test]
:fail-fast? true
:partition/total 3
:partition/index 1
- name: Driver Tests Part 3
test-args: >-
:only-tags [:mb/driver-tests]
:exclude-tags [:mb/transforms-python-test]
:fail-fast? true
:partition/total 3
:partition/index 2
env:
CI: 'true'
DRIVERS: databricks
MB_DATABRICKS_TEST_MULTI_LEVEL_SCHEMA: true
MB_DATABRICKS_TEST_HOST: ${{ secrets.MB_DATABRICKS_JDBC_TEST_HOST }}
MB_DATABRICKS_TEST_HTTP_PATH: ${{ secrets.MB_DATABRICKS_JDBC_TEST_HTTP_PATH }}
MB_DATABRICKS_TEST_TOKEN: ${{ secrets.MB_DATABRICKS_JDBC_TEST_TOKEN }}
MB_DATABRICKS_TEST_CATALOG: 'metabase_ci'
MB_DATABRICKS_TEST_CATALOG_ROUTING: 'metabase_routing_catalog'
MB_DATABRICKS_TEST_CLIENT_ID: ${{ secrets.MB_DATABRICKS_JDBC_TEST_CLIENT_ID }}
MB_DATABRICKS_TEST_OAUTH_SECRET: ${{ secrets.MB_DATABRICKS_JDBC_TEST_OAUTH_SECRET }}
name: Databricks (Multi-Catalog)
steps:
- uses: actions/checkout@v6
- name: Test Databricks driver with multi-catalog
id: test-driver
uses: ./.github/actions/test-driver
with:
junit-name: 'be-tests-databricks-multi-catalog-ee'
artifact-suffix: ${{ matrix.job.name }}
test-args: ${{ matrix.job.test-args }}
quarantine-engine: ${{ vars.QUARANTINE_ENGINE }}
be-tests-h2:
needs: determine-driver-skips
if: ${{ needs.determine-driver-skips.outputs.h2-should-run == 'true' }}
@@ -814,9 +772,20 @@ jobs:
enable-ssl-tests: 'true'
enable-aws-iam-tests: 'true'
job:
- name: Driver Tests
# Split because the unpartitioned Driver Tests job was running ~40 minutes of tests against
# `timeout-minutes: 50` (checkout and prepare-backend take the rest), and Postgres Latest was
# timing out.
- name: Driver Tests Part 1
test-args: >-
:only-tags [:mb/driver-tests]
:partition/total 2
:partition/index 0
exclude-tag: ':mb/transforms-python-test'
- name: Driver Tests Part 2
test-args: >-
:only-tags [:mb/driver-tests]
:partition/total 2
:partition/index 1
exclude-tag: ':mb/transforms-python-test'
- name: Transforms Python Tests
test-args: >-
@@ -1309,7 +1278,6 @@ jobs:
- be-tests-bigquery-cloud-sdk-ee
- be-tests-clickhouse-ee
- be-tests-databricks-ee
- be-tests-databricks-multi-catalog-ee
- be-tests-druid-jdbc-ee
- be-tests-h2
- be-tests-h2-oss

View File

@@ -73,6 +73,21 @@ jobs:
id: cypress-prep
uses: ./.github/actions/prepare-cypress
# esbuild resolves every import in the spec bundle's graph, even the ones tree shaking then drops.
# The cljs imports resolve into target/cljs_dev, so the compiled CLJS has to exist before Cypress runs.
# run-tests.yml compiles it in parallel with the uberjar build, so downloading is ~10x faster than compiling here.
- name: Download pre-built CLJS
id: prebuilt-cljs
uses: actions/download-artifact@v8
continue-on-error: true
with:
name: e2e-cljs-${{ github.event.pull_request.head.sha || github.sha }}
path: target/cljs_dev
- name: Compile CLJS
if: steps.prebuilt-cljs.outcome != 'success'
run: bun run build-pure:cljs
- name: Run Metabase
run: node e2e/runner/run_cypress_ci.js start
@@ -246,6 +261,13 @@ jobs:
id: cypress-prep
uses: ./.github/actions/prepare-cypress
# esbuild resolves every import in the spec bundle's graph, even the ones tree shaking then drops.
# The cljs imports resolve into target/cljs_dev, so the compiled CLJS has to exist before Cypress runs.
# The checked-out tree is the previous commit, which the current run's CLJS artifact was not built from,
# so compile from source instead of downloading.
- name: Compile CLJS
run: bun run build-pure:cljs
- name: Run Metabase
run: node e2e/runner/run_cypress_ci.js start

View File

@@ -203,7 +203,9 @@ jobs:
fail-fast: false
matrix:
environment: ["production", "development"]
# TEMPORARILY DISABLED: re-enable by removing the `false &&` below.
if: |
false &&
!cancelled() && needs.get-sample-apps-data.result == 'success' &&
(github.ref == 'refs/heads/master' || github.base_ref == 'master')
runs-on: ${{ vars.DEFAULT_RUNNER_KEY }}
@@ -270,7 +272,9 @@ jobs:
# They use the shoppy production environment and include performance measurement tests.
e2e-shoppy-synthetic-tests:
needs: [get-sample-apps-data]
# TEMPORARILY DISABLED: re-enable by removing the `false &&` below.
if: |
false &&
!cancelled() && needs.get-sample-apps-data.result == 'success' &&
github.event_name == 'push' &&
github.ref == 'refs/heads/master'
@@ -419,6 +423,21 @@ jobs:
id: cypress-prep
uses: ./.github/actions/prepare-cypress
# esbuild resolves every import in the spec bundle's graph, even the ones tree shaking then drops.
# The cljs imports resolve into target/cljs_dev, so the compiled CLJS has to exist before Cypress runs.
# run-tests.yml compiles it in parallel with the uberjar build, so downloading is ~10x faster than compiling here.
- name: Download pre-built CLJS
id: prebuilt-cljs
uses: actions/download-artifact@v8
continue-on-error: true
with:
name: e2e-cljs-${{ github.event.pull_request.head.sha || github.sha }}
path: target/cljs_dev
- name: Compile CLJS
if: steps.prebuilt-cljs.outcome != 'success'
run: bun run build-pure:cljs
- name: Run Metabase
run: node e2e/runner/run_cypress_ci.js start

View File

@@ -1,87 +0,0 @@
name: Kondo Ratchets Self-Heal
# Stale budgets in .clj-kondo/ratchets.edn (above the actual ignore counts) fail CI. On PRs
# labelled kondo-ratchets-self-healing, this commits the lowered budgets to the branch;
# unlabelled PRs rely on local test runs to tighten the file.
#
# Two jobs so the automation token never shares an environment with PR-controlled code: the
# ratchet-down job runs ./bin/mage from the PR branch and only exports the fixed file as an
# artifact; the push job starts from a clean runner, executes nothing from the PR, and pushes.
permissions:
contents: read
on:
pull_request:
types: [opened, synchronize, reopened, labeled, unlabeled]
paths:
- "**.clj"
- "**.cljc"
- "**.cljs"
concurrency:
group: ${{ github.workflow }}-${{ github.event.pull_request.number }}
cancel-in-progress: true
jobs:
ratchet-down:
runs-on: ${{ vars.DEFAULT_RUNNER_KEY }}
timeout-minutes: 10
# fork PRs don't get the automation token, so we couldn't push to them anyway
if: >-
contains(github.event.pull_request.labels.*.name, 'kondo-ratchets-self-healing')
&& github.event.pull_request.head.repo.full_name == github.repository
outputs:
changed: ${{ steps.diff.outputs.changed }}
steps:
- uses: actions/checkout@v6
with:
ref: ${{ github.event.pull_request.head.ref }}
persist-credentials: false
- name: Prepare back-end environment
uses: ./.github/actions/prepare-backend
- name: Lower ignore budgets to match the source tree
run: ./bin/mage fix-kondo-ratchets
- name: Check for changes
id: diff
run: |
if git diff --quiet .clj-kondo/ratchets.edn; then
echo "budgets already tight"
echo "changed=false" >> "$GITHUB_OUTPUT"
else
echo "changed=true" >> "$GITHUB_OUTPUT"
fi
- uses: actions/upload-artifact@v7
if: steps.diff.outputs.changed == 'true'
with:
name: ratchets-edn
path: .clj-kondo/ratchets.edn
# dot-directories are excluded by default, which would ship an empty artifact
include-hidden-files: true
push:
needs: ratchet-down
if: needs.ratchet-down.outputs.changed == 'true'
runs-on: ${{ vars.DEFAULT_RUNNER_KEY }}
timeout-minutes: 5
steps:
- uses: actions/checkout@v6
with:
ref: ${{ github.event.pull_request.head.ref }}
persist-credentials: false
- uses: actions/download-artifact@v8
with:
name: ratchets-edn
path: .clj-kondo/
- name: Commit to the PR branch
env:
PR_BRANCH: ${{ github.event.pull_request.head.ref }}
REPO: ${{ github.repository }}
# METABASE_AUTOMATION_USER_TOKEN so the self-healing commit triggers CI
# (a GITHUB_TOKEN push would not)
AUTOMATION_TOKEN: ${{ secrets.METABASE_AUTOMATION_USER_TOKEN }}
run: |
git config --global user.name "Metabase Automation"
git config --global user.email "github-automation@metabase.com"
git add .clj-kondo/ratchets.edn
git commit -m "Ratchet down kondo ignore budgets" --no-verify
git push "https://x-access-token:${AUTOMATION_TOKEN}@github.com/${REPO}.git" "HEAD:$PR_BRANCH"

View File

@@ -0,0 +1,166 @@
name: Shrink ratchets
# Lowers the budgets in .clj-kondo/ratchets.edn to match master and delivers the result through one
# automation PR, because the rulesets require a pull request for every change to master. The PR is
# approved by github-actions[bot] and merges itself once the ratchet check passes. A merge of that PR
# touches only ratchets.edn, which matches `.clj-kondo/**` below, so it triggers one more run that finds
# nothing to lower and stops: a single no-op, not a loop.
on:
push:
branches: [master]
paths:
- "**.clj"
- "**.cljc"
- "**.cljs"
- ".clj-kondo/**"
- "bb.edn"
workflow_dispatch:
permissions:
# checkout and the fetch of the automation branch
contents: read
# the compare step reads whether the automation PR is still open, and the approve step reviews it as
# github-actions[bot]; pushes and the PR itself use the automation user's token
pull-requests: write
# When master is pushed again, the newest run cancels the older one, checks out the latest master it can
# see, and force-updates the automation branch. The PR therefore contains one commit on top of master.
concurrency:
group: kondo-ratchets-shrink
cancel-in-progress: true
env:
AUTOMATION_BRANCH: automation/shrink-ratchets
RATCHETS_FILE: .clj-kondo/ratchets.edn
jobs:
shrink:
name: Shrink ratchets
# A fork with Actions enabled has no automation token, so this would only ever fail at the push.
if: github.repository == 'metabase/metabase'
runs-on: ${{ vars.DEFAULT_RUNNER_KEY }}
timeout-minutes: 10
steps:
# `ref: master` checks out the branch head at checkout time rather than the pushed commit, so a run
# that queued behind another one starts from the newest master it can see.
- uses: actions/checkout@v6
with:
ref: master
# The persisted GITHUB_TOKEN header would override the automation token in the push URL and
# get the force-push rejected. Nothing here needs it: the repo is public, so the fetch of the
# automation branch is anonymous, and every gh call carries its own token.
persist-credentials: false
- name: Set up Java for cold dependency resolution
uses: actions/setup-java@v5.2.0
with:
distribution: temurin
java-version: 25
- name: Shrink ignore limits
run: |
cp "$RATCHETS_FILE" "$RUNNER_TEMP/ratchets-before.edn"
./bin/mage kondo-ratchets-shrink
# Two independent outputs: `push` says the branch needs a new commit, `pr` names the open automation
# PR if there is one. Approval and auto-merge key on `pr` alone, so a run that failed after opening
# the PR is repaired by the next run instead of leaving the PR stranded.
- name: Compare with master and the automation PR
id: compare
env:
GH_TOKEN: ${{ github.token }}
run: |
# The REST `head=owner:branch` filter is applied server-side and only matches this repository's
# branch, so a fork carrying a branch of the same name can never hand its PR number to the
# approval and auto-merge steps, however many such forks exist. Only an open PR counts; a branch
# left behind by a PR closed without merging is overwritten below and gets a new PR, so a closed
# one never stops the budgets from being lowered.
open_pr=$(gh api "repos/${GITHUB_REPOSITORY}/pulls?state=open&base=master&head=${GITHUB_REPOSITORY_OWNER}:${AUTOMATION_BRANCH}" \
--jq '.[0].number // empty')
echo "pr=$open_pr" >> "$GITHUB_OUTPUT"
if git diff --quiet -- "$RATCHETS_FILE"; then
echo "Ratchet budgets already match master $(git rev-parse --short HEAD)"
[ -z "$open_pr" ] || echo "The open automation PR #$open_pr is re-approved below"
echo "push=false" >> "$GITHUB_OUTPUT"
exit 0
fi
# The ratchet file is the whole payload, so an open PR that already carries this result is left
# alone. Rebasing it onto every master push would restart its CI and churn the diff without
# changing what it delivers. Compare the open PR with the working tree, which contains the
# shrunk ratchets file produced above, to determine whether the PR would deliver the same file.
#
# A conflicted PR is the exception: only a fresh commit clears it. Mergeability is computed
# lazily, so a pending UNKNOWN reads as not conflicted and the next run settles it.
if [ -n "$open_pr" ] \
&& git fetch --quiet --depth=1 origin "refs/heads/$AUTOMATION_BRANCH" 2>/dev/null \
&& git diff --quiet FETCH_HEAD -- "$RATCHETS_FILE" \
&& [ "$(gh pr view "$open_pr" --repo "$GITHUB_REPOSITORY" \
--json mergeable --jq .mergeable)" != CONFLICTING ]; then
echo "The automation PR #$open_pr already carries this result"
echo "push=false" >> "$GITHUB_OUTPUT"
echo "head=$(git rev-parse FETCH_HEAD)" >> "$GITHUB_OUTPUT"
exit 0
fi
git diff --stat -- "$RATCHETS_FILE"
echo "push=true" >> "$GITHUB_OUTPUT"
- name: Update the automation PR
id: deliver
if: steps.compare.outputs.push == 'true'
# The automation user's token pushes the branch and owns the PR: a branch pushed with
# GITHUB_TOKEN starts no pull_request workflows, so the PR would never get its checks. The commit
# identity is the docs generator's, which merges with one approval under the unattributed-changes
# rule. Nothing raises a budget.
env:
GH_TOKEN: ${{ secrets.METABASE_AUTOMATION_USER_TOKEN }}
OPEN_PR: ${{ steps.compare.outputs.pr }}
run: |
./bin/mage -kondo-ratchets-pr-body \
"$RUNNER_TEMP/ratchets-before.edn" \
"$RATCHETS_FILE" \
"${GITHUB_SERVER_URL}/${GITHUB_REPOSITORY}/actions/runs/${GITHUB_RUN_ID}" \
> "$RUNNER_TEMP/ratchets-pr-body.md"
git config user.name github-actions
git config user.email github-actions@github.com
git add -- "$RATCHETS_FILE"
git commit --quiet -m "Tighten ratchets"
git push --force --quiet \
"https://x-access-token:${GH_TOKEN}@github.com/${GITHUB_REPOSITORY}.git" \
"HEAD:refs/heads/$AUTOMATION_BRANCH"
echo "head=$(git rev-parse HEAD)" >> "$GITHUB_OUTPUT"
if [ -n "$OPEN_PR" ]; then
gh pr edit "$OPEN_PR" --body-file "$RUNNER_TEMP/ratchets-pr-body.md"
echo "Updated the automation PR #$OPEN_PR"
echo "pr=$OPEN_PR" >> "$GITHUB_OUTPUT"
exit 0
fi
url=$(gh pr create \
--base master \
--head "$AUTOMATION_BRANCH" \
--title "Tighten ratchets" \
--label "no-backport" \
--body-file "$RUNNER_TEMP/ratchets-pr-body.md")
echo "Opened the automation PR $url"
echo "pr=${url##*/}" >> "$GITHUB_OUTPUT"
- name: Approve the automation PR
if: (steps.deliver.outputs.pr || steps.compare.outputs.pr) != ''
# GITHUB_TOKEN reviews as github-actions[bot], a different identity from the automation user that
# owns the PR, so the approval counts; a user cannot approve their own PR. Reviews survive a push,
# so approving an already-open PR again is harmless and keeps a reopened one covered.
env:
GH_TOKEN: ${{ github.token }}
PR_NUMBER: ${{ steps.deliver.outputs.pr || steps.compare.outputs.pr }}
run: gh pr review --approve "$PR_NUMBER"
# Armed only against the commit this run pushed or compared: the branch name is fixed, and anyone
# with push access could otherwise collect this run's approval for their own commit.
- name: Enable auto-merge
if: (steps.deliver.outputs.pr || steps.compare.outputs.pr) != ''
&& (steps.deliver.outputs.head || steps.compare.outputs.head) != ''
env:
GH_TOKEN: ${{ secrets.METABASE_AUTOMATION_USER_TOKEN }}
PR_NUMBER: ${{ steps.deliver.outputs.pr || steps.compare.outputs.pr }}
HEAD_SHA: ${{ steps.deliver.outputs.head || steps.compare.outputs.head }}
run: gh pr merge --squash --auto --match-head-commit "$HEAD_SHA" "$PR_NUMBER"

View File

@@ -16,12 +16,12 @@ jobs:
runs-on: ${{ vars.SLIM_RUNNER_KEY }}
timeout-minutes: 10
outputs:
actionlint: ${{ steps.changes.outputs.actionlint }}
actionlint_files: ${{ steps.changes.outputs.actionlint_files }}
codeql: ${{ steps.changes.outputs.codeql }}
i18n: ${{ steps.changes.outputs.i18n }}
openapi: ${{ steps.changes.outputs.openapi }}
snowplow: ${{ steps.changes.outputs.snowplow }}
workflows: ${{ steps.changes.outputs.workflows }}
workflows_files: ${{ steps.changes.outputs.workflows_files }}
yaml: ${{ steps.changes.outputs.yaml }}
steps:
- uses: actions/checkout@v6
@@ -52,7 +52,7 @@ jobs:
uses: ./.github/actions/prepare-backend
- name: Setup actionlint
if: needs.files-changed.outputs.actionlint == 'true'
if: needs.files-changed.outputs.workflows == 'true'
id: download
shell: bash
run: |
@@ -81,8 +81,12 @@ jobs:
# Lint Github Actions
- name: Run actionlint
if: needs.files-changed.outputs.actionlint == 'true'
run: ${{ steps.download.outputs.executable }} --color --verbose ${{ needs.files-changed.outputs.actionlint_files }}
if: needs.files-changed.outputs.workflows == 'true'
run: ${{ steps.download.outputs.executable }} --color --verbose ${{ needs.files-changed.outputs.workflows_files }}
- name: Check every workflow job sets a timeout
if: needs.files-changed.outputs.workflows == 'true'
run: .github/scripts/check-job-timeouts.sh ${{ needs.files-changed.outputs.workflows_files }}
# Generate and Lint OpenAPI spec
- name: Generate OpenAPI spec
@@ -123,6 +127,7 @@ jobs:
needs: files-changed
if: needs.files-changed.outputs.snowplow == 'true'
runs-on: ${{ vars.DEFAULT_RUNNER_KEY }}
timeout-minutes: 10
steps:
- name: Checkout
uses: actions/checkout@v6

View File

@@ -16,3 +16,7 @@ jobs:
run: ./bin/mage -h
- name: Run mage tests
run: ./bin/mage -test
- name: Lint create-backport scripts
run: shellcheck .github/actions/create-backport/{kondo-ratchets,create-backport-test}.sh
- name: Run create-backport tests
run: ./.github/actions/create-backport/create-backport-test.sh

View File

@@ -45,40 +45,36 @@ jobs:
- name: Install tsx
run: npm i -g tsx
- name: Get version number
id: get-version
uses: actions/github-script@v9
env:
VERSION_INPUT: ${{ inputs.version }}
with:
script: | # js
const {
getMajorVersionFromRef,
getMajorVersion,
} = require('${{ github.workspace }}/release/dist/index.cjs');
const { getMajorVersionFromRef } = require('${{ github.workspace }}/release/dist/index.cjs');
console.log({ context });
// if the version is explicitly set, use that
if (context.payload?.inputs?.version) {
core.exportVariable('VERSION', context.payload.inputs.version);
return;
}
const version = process.env.VERSION_INPUT || getMajorVersionFromRef(context.payload.ref);
// otherwise, get the version from the release branch name
const ref = context.payload.ref;
const majorVersion = getMajorVersionFromRef(ref);
console.log({ ref, majorVersion });
core.exportVariable('VERSION', majorVersion);
console.log({ version });
core.exportVariable('VERSION', version);
core.setOutput('version', version);
- name: generate release Log
if: ${{ steps.get-version.outputs.version != '' }}
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
run: cd release && tsx ./src/release-log-run.ts "$VERSION" > "v$VERSION.html"
- name: upload release log to the web
if: ${{ steps.get-version.outputs.version != '' }}
run: |
aws s3 cp \
"release/v$VERSION.html" \
"s3://${{ vars.AWS_S3_STATIC_BUCKET }}/release-log/v$VERSION.html"
- name: Create cloudfront invalidation
if: ${{ steps.get-version.outputs.version != '' }}
run: |
aws cloudfront create-invalidation \
--distribution-id ${{ vars.AWS_CLOUDFRONT_STATIC_ID }} \

View File

@@ -1,63 +0,0 @@
name: Reset Cloud Test Dataset
on:
workflow_dispatch:
inputs:
driver:
description: 'Driver to reset dataset for'
required: true
default: 'snowflake'
type: choice
options:
- snowflake
- bigquery-cloud-sdk
- redshift
- databricks
dataset-name:
description: 'Dataset name to drop (e.g., test-data)'
required: true
default: 'test-data'
type: string
jobs:
reset-dataset:
runs-on: ${{ vars.DEFAULT_RUNNER_KEY }}
timeout-minutes: 10
env:
CI: 'true'
DRIVERS: ${{ inputs.driver }}
# Snowflake
MB_SNOWFLAKE_TEST_USER: METABASE CI
MB_SNOWFLAKE_TEST_ACCOUNT: ${{ secrets.MB_SNOWFLAKE_TEST_ACCOUNT }}
MB_SNOWFLAKE_TEST_PASSWORD: ${{ secrets.MB_SNOWFLAKE_TEST_PASSWORD }}
MB_SNOWFLAKE_TEST_PRIVATE_KEY: ${{ secrets.MB_SNOWFLAKE_TEST_PRIVATE_KEY }}
MB_SNOWFLAKE_TEST_WAREHOUSE: ${{ secrets.MB_SNOWFLAKE_TEST_WAREHOUSE }}
# BigQuery
MB_BIGQUERY_TEST_PROJECT_ID: ${{ secrets.BIGQUERY_TEST_PROJECT_ID }}
MB_BIGQUERY_TEST_CLIENT_ID: ${{ secrets.MB_BIGQUERY_TEST_CLIENT_ID }}
MB_BIGQUERY_TEST_CLIENT_SECRET: ${{ secrets.MB_BIGQUERY_TEST_CLIENT_SECRET }}
MB_BIGQUERY_TEST_ACCESS_TOKEN: ${{ secrets.MB_BIGQUERY_TEST_ACCESS_TOKEN }}
MB_BIGQUERY_TEST_REFRESH_TOKEN: ${{ secrets.MB_BIGQUERY_TEST_REFRESH_TOKEN }}
MB_BIGQUERY_CLOUD_SDK_TEST_SERVICE_ACCOUNT_JSON: ${{ secrets.MB_BIGQUERY_CLOUD_SDK_TEST_SERVICE_ACCOUNT_JSON }}
MB_BIGQUERY_CLOUD_SDK_TEST_SERVICE_ACCOUNT_JSON_ROUTING: ${{ secrets.MB_BIGQUERY_CLOUD_SDK_TEST_SERVICE_ACCOUNT_JSON_ROUTING }}
# Redshift
MB_REDSHIFT_TEST_USER: metabase_ci
MB_REDSHIFT_TEST_DB: testdb
MB_REDSHIFT_TEST_HOST: ${{ secrets.MB_REDSHIFT_TEST_HOST }}
MB_REDSHIFT_TEST_PASSWORD: ${{ secrets.MB_REDSHIFT_TEST_PASSWORD }}
# Databricks
MB_DATABRICKS_TEST_HOST: ${{ secrets.MB_DATABRICKS_JDBC_TEST_HOST }}
MB_DATABRICKS_TEST_HTTP_PATH: ${{ secrets.MB_DATABRICKS_JDBC_TEST_HTTP_PATH }}
MB_DATABRICKS_TEST_TOKEN: ${{ secrets.MB_DATABRICKS_JDBC_TEST_TOKEN }}
MB_DATABRICKS_TEST_CATALOG: metabase_ci
MB_DATABRICKS_TEST_CLIENT_ID: ${{ secrets.MB_DATABRICKS_JDBC_TEST_CLIENT_ID }}
MB_DATABRICKS_TEST_OAUTH_SECRET: ${{ secrets.MB_DATABRICKS_JDBC_TEST_OAUTH_SECRET }}
steps:
- uses: actions/checkout@v6
- uses: ./.github/actions/prepare-backend
- name: Drop dataset
run: >-
clojure -X:dev:drivers:drivers-dev:test
metabase.test.data.impl/drop-dataset!
:driver '"${{ inputs.driver }}"'
:dataset-name '"${{ inputs.dataset-name }}"'

View File

@@ -20,6 +20,10 @@ permissions:
jobs:
# `force-run` and `force-skip` are global overrides; `defer` means this
# job has no opinion and every downstream gate behaves as it always has.
#
# Only the lowest unmerged PR of a stack runs CI: it is the one whose base
# ref is the stack base. Every PR above it is force-skipped and gets tested
# once it reaches the bottom of the stack.
should-run:
name: Decide which jobs should run
runs-on: ${{ vars.SLIM_RUNNER_KEY }}
@@ -35,6 +39,7 @@ jobs:
${{ (github.ref_name == 'master' || startsWith(github.ref_name, 'release-x.')) && 'force-run'
|| contains(github.event.pull_request.labels.*.name, 'ci:run-all') && 'force-run'
|| contains(github.event.pull_request.labels.*.name, 'ci:skip') && 'force-skip'
|| (github.event.pull_request.stack != null && github.event.pull_request.stack.base.ref != github.event.pull_request.base.ref) && 'force-skip'
|| 'defer' }}
run: echo "verdict=$VERDICT" | tee -a "$GITHUB_OUTPUT"
@@ -60,8 +65,13 @@ jobs:
embedding_sdk_host_sample_apps: ${{ steps.changes.outputs.embedding_sdk_host_sample_apps }}
embedding_sdk_ci: ${{ steps.changes.outputs.embedding_sdk_ci }}
custom_viz_all: ${{ steps.changes.outputs.custom_viz_all }}
project_backend_checks: ${{ steps.changes.outputs.project_backend_checks }}
project_migration_checks: ${{ steps.changes.outputs.project_migration_checks }}
project_ratchet_checks: ${{ steps.changes.outputs.project_ratchet_checks }}
# A ratchets-only change (the shrink automation PR) needs the ratchet check, not an uberjar.
no_source_code_changes: >-
${{ steps.changes.outputs.documentation == 'true'
${{ (steps.changes.outputs.documentation == 'true'
|| steps.changes.outputs.project_ratchet_checks == 'true')
&& steps.changes.outputs.backend_all == 'false'
&& steps.changes.outputs.frontend_all == 'false'
&& steps.changes.outputs.e2e_all == 'false' }}
@@ -210,6 +220,9 @@ jobs:
needs.files-changed.outputs.backend_all != 'true' &&
needs.static-viz-files-changed.outputs.static_viz != 'true' }}
force-run: ${{ needs.should-run.outputs.verdict == 'force-run' }}
run-project-backend-checks: ${{ needs.files-changed.outputs.project_backend_checks == 'true' }}
run-project-migration-checks: ${{ needs.files-changed.outputs.project_migration_checks == 'true' }}
run-project-ratchet-checks: ${{ needs.files-changed.outputs.project_ratchet_checks == 'true' }}
semantic-search-tests:
if: |

View File

@@ -0,0 +1,81 @@
name: Cleanup DWH Test Data
on:
schedule:
- cron: '0 7 * * *'
workflow_dispatch:
inputs:
drivers:
description: 'Drivers to sweep (comma-separated)'
required: false
default: 'snowflake,bigquery-cloud-sdk,redshift'
type: string
temp-data-hours:
description: 'TTL in hours for per-run garbage (minimum 2)'
required: false
default: '2'
type: string
fixture-hours:
description: 'TTL in hours for datasets runs share (minimum 24)'
required: false
default: '96'
type: string
# `inputs.*` is empty on the scheduled run, so each falls back to the nightly value.
env:
GC_TEMP_HOURS: ${{ inputs.temp-data-hours || 2 }} # for temp data
GC_FIXTURE_HOURS: ${{ inputs.fixture-hours || 96 }} # for reusable data
GC_DRIVERS: ${{ inputs.drivers || 'snowflake,bigquery-cloud-sdk,redshift' }}
concurrency:
group: test-cleanup-dwh-data
cancel-in-progress: false
permissions:
contents: read
jobs:
gc:
name: Sweep orphaned test data
if: ${{ github.repository == 'metabase/metabase' }}
runs-on: ${{ vars.DEFAULT_RUNNER_KEY }}
timeout-minutes: 60
env:
CI: 'true'
# Snowflake
MB_SNOWFLAKE_TEST_USER: METABASE CI
MB_SNOWFLAKE_TEST_ACCOUNT: ${{ secrets.MB_SNOWFLAKE_TEST_ACCOUNT }}
MB_SNOWFLAKE_TEST_PASSWORD: ${{ secrets.MB_SNOWFLAKE_TEST_PASSWORD }}
MB_SNOWFLAKE_TEST_PRIVATE_KEY: ${{ secrets.MB_SNOWFLAKE_TEST_PRIVATE_KEY }}
MB_SNOWFLAKE_TEST_WAREHOUSE: ${{ secrets.MB_SNOWFLAKE_TEST_WAREHOUSE }}
# BigQuery
MB_BIGQUERY_TEST_PROJECT_ID: ${{ secrets.BIGQUERY_TEST_PROJECT_ID }}
MB_BIGQUERY_CLOUD_SDK_TEST_SERVICE_ACCOUNT_JSON: ${{ secrets.MB_BIGQUERY_CLOUD_SDK_TEST_SERVICE_ACCOUNT_JSON }}
MB_BIGQUERY_CLOUD_SDK_TEST_SERVICE_ACCOUNT_JSON_ROUTING: ${{ secrets.MB_BIGQUERY_CLOUD_SDK_TEST_SERVICE_ACCOUNT_JSON_ROUTING }}
# Redshift
MB_REDSHIFT_TEST_USER: metabase_ci
MB_REDSHIFT_TEST_DB: testdb
MB_REDSHIFT_TEST_DB_ROUTING: dev
MB_REDSHIFT_TEST_HOSTS: ${{ secrets.MB_REDSHIFT_TEST_HOSTS }}
MB_REDSHIFT_TEST_HOST: ${{ secrets.MB_REDSHIFT_TEST_HOST }}
MB_REDSHIFT_TEST_PASSWORD: ${{ secrets.MB_REDSHIFT_TEST_PASSWORD }}
steps:
- uses: actions/checkout@v6
- uses: ./.github/actions/prepare-backend
- name: Sweep orphaned test data
run: |
clojure -X:dev:drivers:drivers-dev:test metabase.test.data.gc/gc-orphans! \
:drivers "\"${{ env.GC_DRIVERS }}\"" \
:temp-data-hours "${{ env.GC_TEMP_HOURS }}" \
:fixture-hours "${{ env.GC_FIXTURE_HOURS }}" \
:report-dir '"target/gc-report"'
# The report is most wanted on the runs that fail, so upload it regardless of the sweep's outcome.
- name: Upload sweep report
if: ${{ !cancelled() }}
continue-on-error: true
uses: actions/upload-artifact@v7
with:
name: dwh-gc-report-${{ github.run_id }}-attempt-${{ github.run_attempt }}
path: target/gc-report
if-no-files-found: warn

Binary file not shown.

After

Width:  |  Height:  |  Size: 26 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 40 KiB

After

Width:  |  Height:  |  Size: 40 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 40 KiB

After

Width:  |  Height:  |  Size: 40 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 7.2 KiB

After

Width:  |  Height:  |  Size: 8.0 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 67 KiB

After

Width:  |  Height:  |  Size: 68 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 67 KiB

After

Width:  |  Height:  |  Size: 68 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 60 KiB

After

Width:  |  Height:  |  Size: 61 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 60 KiB

After

Width:  |  Height:  |  Size: 61 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 66 KiB

After

Width:  |  Height:  |  Size: 66 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 66 KiB

After

Width:  |  Height:  |  Size: 67 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 59 KiB

After

Width:  |  Height:  |  Size: 60 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 59 KiB

After

Width:  |  Height:  |  Size: 60 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 66 KiB

After

Width:  |  Height:  |  Size: 66 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 66 KiB

After

Width:  |  Height:  |  Size: 66 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 59 KiB

After

Width:  |  Height:  |  Size: 60 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 59 KiB

After

Width:  |  Height:  |  Size: 60 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 29 KiB

After

Width:  |  Height:  |  Size: 29 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 26 KiB

After

Width:  |  Height:  |  Size: 26 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 29 KiB

After

Width:  |  Height:  |  Size: 29 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 26 KiB

After

Width:  |  Height:  |  Size: 27 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 24 KiB

After

Width:  |  Height:  |  Size: 25 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 26 KiB

After

Width:  |  Height:  |  Size: 26 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 23 KiB

After

Width:  |  Height:  |  Size: 24 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 31 KiB

After

Width:  |  Height:  |  Size: 32 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 25 KiB

After

Width:  |  Height:  |  Size: 26 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 25 KiB

After

Width:  |  Height:  |  Size: 25 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 24 KiB

After

Width:  |  Height:  |  Size: 25 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 32 KiB

After

Width:  |  Height:  |  Size: 32 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 26 KiB

After

Width:  |  Height:  |  Size: 27 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 26 KiB

After

Width:  |  Height:  |  Size: 27 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 83 KiB

After

Width:  |  Height:  |  Size: 84 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 76 KiB

After

Width:  |  Height:  |  Size: 78 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 72 KiB

After

Width:  |  Height:  |  Size: 73 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 74 KiB

After

Width:  |  Height:  |  Size: 76 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 89 KiB

After

Width:  |  Height:  |  Size: 90 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 78 KiB

After

Width:  |  Height:  |  Size: 79 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 85 KiB

After

Width:  |  Height:  |  Size: 87 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 71 KiB

After

Width:  |  Height:  |  Size: 72 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 76 KiB

After

Width:  |  Height:  |  Size: 78 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 74 KiB

After

Width:  |  Height:  |  Size: 75 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 75 KiB

After

Width:  |  Height:  |  Size: 76 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 76 KiB

After

Width:  |  Height:  |  Size: 78 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 75 KiB

After

Width:  |  Height:  |  Size: 75 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 71 KiB

After

Width:  |  Height:  |  Size: 72 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 71 KiB

After

Width:  |  Height:  |  Size: 72 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 74 KiB

After

Width:  |  Height:  |  Size: 75 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 82 KiB

After

Width:  |  Height:  |  Size: 83 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 75 KiB

After

Width:  |  Height:  |  Size: 76 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 72 KiB

After

Width:  |  Height:  |  Size: 73 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 74 KiB

After

Width:  |  Height:  |  Size: 74 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 88 KiB

After

Width:  |  Height:  |  Size: 89 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 78 KiB

After

Width:  |  Height:  |  Size: 79 KiB

Some files were not shown because too many files have changed in this diff Show More