* Fix SDK entity navigation dropping click behavior filters on GUI questions
* Give ad-hoc question targets their own click behavior type
* Cover the GUI click behavior target with an e2e test
* Fix the internal-navigation dashcard setup clobbering itself
* GDGT: migrate TabRow to CSS modules to fix gradient theme issue
* Update Loki Snapshots
* increase scroll gradient size
* Update Loki Snapshots
* Add dark theme story for narrow dashboard with many tabs
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
* Add placeholder Loki reference for dark theme narrow tabs story
Copy of the light theme snapshot so the loki-update workflow detects a
diff and replaces it with the CI-generated image.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
* Update Loki Snapshots
* move left/right styles into existing classes
---------
Co-authored-by: Metabase Automation <github-automation@metabase.com>
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
* Require app-DB value slots to hold bound parameters
HoneySQL resolves a map in a value slot toward SQL structure rather than
data, so a value can reach the database as syntax instead of as a bound
parameter. `[:param ...]` is the only form that always binds.
Adds `metabase.app-db.value-guard`:
- `auto-param` rewrites inline `[:param* v]` markers into HoneySQL's
`[:param :kN]` plus the params map, so callers do not keep a separate
map in sync.
- `long*`/`longs` coerce ids, which covers the common lookup without
param noise and rejects anything that is not a number.
- `assert-values-wrapped!` walks the compiled query at the Toucan
compile step and throws when a value slot holds something that could
become SQL.
The walk is clause-aware: it descends only into `:where`/`:having`/`:set`/
`:values`/`:join-by` and knows which argument of each operator is a value,
so column and operator positions are left alone. An operator that is not
classified fails closed.
This is an allowlist and lives apart from `honeysql-guard`, which is a
blocklist of known shapes and is backported to older versions.
Two forms needed handling beyond plain scalars and params: h2x wrappers,
which are checked by their payload rather than trusted by their wrapper,
and HoneySQL's `:%fn.col` call shorthand, which is a closed enumerated set
rather than a prefix test so that a constructed keyword cannot pass.
`strict-modules` is empty for now. A module joins it once its call sites
carry params or coercions, at which point plain scalars are rejected there
too.
* Accept java.time values in value slots
Toucan's before-insert hooks add created_at/updated_at as ZonedDateTime,
which clojure.core/inst? does not cover, so every insert was rejected.
All java.time.temporal.Temporal values bind as parameters.
* Check nested values, report the offending leaf, defer installation
Three changes, all from measuring the check against a real module.
A dev-authored marker on a subquery asserts that its SQL is dev-authored.
It says nothing about the values inside it, which are just as reachable
from a request, so the check now descends into a marked subquery instead
of passing it wholesale. `honeysql-guard` also descends here, but with a
blocklist, so a nested unwrapped value passes it today.
When a nested value was rejected the error named the enclosing subquery
rather than the value, which pointed debugging at the wrong place. The
nested check now throws on the leaf.
The check is no longer installed at the Toucan compile step. Against the
`collections` module it errored 91 of 148 tests, and none of those was an
unparameterized user value -- they were correlated subqueries, a UNION
used as an operand, and SQL function calls in `:set`, all of which are
real SQL that happens to sit in a value-bearing clause. Installing it
globally is therefore not the right first step; per-namespace opt-in is,
and that opt-in has to key off the calling namespace rather than the
model, since every Toucan model keyword is `:model/Something` and carries
no module information.
* Adopt the value check in the content-translation queries
Gives the check one real caller instead of leaving it unused. `locale`
reaches `translations-for-locale` from a request query parameter, so it is
coerced where it enters the query and the query is checked before it runs.
Adoption is an ordinary function call at the site it protects -- no compile
hook, no ambient state -- so the next namespace adopts independently and
nothing changes for namespaces that have not.
Adds `str*` alongside the id coercions. It asserts a value is already a
string, which rejects the maps and keyword vectors HoneySQL would compile as
SQL while leaving a legitimate string untouched.
`checked` deliberately does not demand bound parameters. A coercion returns
a plain scalar, indistinguishable at runtime from an unwrapped one, so that
stronger property cannot be asserted from the compiled form.
Also records a blocker found while writing this: `[:param ...]` needs its
values in a separate `:params` map, and `honeysql-guard` rejects any unmarked
nested map, so that map never reaches HoneySQL. `[:param]` is therefore
unusable through Toucan until `honeysql-guard` learns about `:params`, which
is a change to a backported namespace and is left alone here.
* Scope the value check to namespaces that have adopted it
Runs the check inside Toucan's compile step again, but only for queries
issued from a namespace listed in `enforcing-namespace-prefixes`. A query
from anywhere else compiles exactly as before, so a namespace adopts on its
own schedule and nothing has to be switched on everywhere at once.
The alternative was a `checked` call at each query site, which only works as
long as everyone remembers to write it. Putting it in the pipeline means an
adopting namespace cannot forget.
Scoping by namespace rather than by model because a model is shared: opting
in `:model/Card` would opt in collections, search, dashboards and
serialization together, which is the big-bang this is trying to avoid. A
`db.clj` is the unit a team actually cleans up, and a prefix entry covers a
whole module once it is ready.
`enforcing-caller?` reads the call stack rather than an ambient binding. The
check runs several frames below the caller inside Toucan, so a dynamic var
would have to be threaded through Toucan internals and would not survive a
thread hand-off; the stack already carries the answer. Measured at ~4us per
query against a DB round trip of hundreds, and only paid at the compile step.
`enforcing-namespace-prefixes` is public so a clj-kondo hook can read the
same set and check adopted namespaces at author time.
Tests prove both directions: a query from the adopting namespace is rejected,
and the same query from a non-adopting namespace is not. A bare keyword in a
value slot is the discriminator, since `honeysql-guard` permits it and this
check does not. `collections` stays at the baseline 1958 assertions, 0 errors.
* Bind values inline with [:auto/param ...]
Makes the design doc's ergonomic half real. A caller writes the value where
it belongs and `bound` lifts it into the params map HoneySQL wants, so the
two never have to be kept in sync:
(bound [query {:where [:= :locale [:auto/param locale]]}]
(t2/select :model/ContentTranslation query))
Two things had to be fixed for this to reach the database.
`honeysql-guard` rejected the `:params` map as an unmarked nested map. It
now treats `:params` like `:set` and `:values` -- keys that hold values
rather than nested queries -- so its contents are still checked. Nothing in
`:params` can become SQL regardless of shape, since HoneySQL binds it as a
parameter, so checking it catches a value JDBC would later refuse rather
than an injection.
HoneySQL also takes params as a format option rather than a query clause, so
`bound` passes them through `toucan2.honeysql2/*options*`.
The marker is `:auto/param` rather than a bare keyword so it cannot collide
with a HoneySQL operator, and an unresolved one now throws: HoneySQL would
otherwise compile `[:auto/param v]` into a `PARAM(?)` function call, so
forgetting `bound` would produce wrong SQL instead of an error.
* Lift [:auto/param ...] in the pipeline instead of at the call site
A caller now writes the marker inline and nothing else:
(t2/select :model/ContentTranslation
{:where [:= :locale [:auto/param locale]]
:order-by [:msgid]})
The compile step lifts every marker into the params map and hands it to
HoneySQL, so the query is an ordinary Toucan query and the wrapper macro is
gone. One less thing to remember, and a query that forgets the wrapper is no
longer possible to write.
The method is `:around` rather than `:before` because it has to replace the
query it passes on, not only inspect it.
An unresolved marker still throws. It now means the query reached the
database by some route other than the compile step, since anything through
the pipeline gets lifted.
* Pin what an unlifted marker does, and correct why a hostile value is refused
HoneySQL does not recognise `:auto/param`, so a marker handed straight to
`sql/format` compiles as a call to a function named PARAM over its argument.
The database then refuses the statement -- `Function "PARAM" not found` --
so it is broken rather than dangerous, but it is worth pinning, since the
marker only does its job on the path that lifts it.
Also corrects a test comment that credited JDBC with refusing a hostile
non-scalar. Through Toucan it never reaches the driver: `honeysql-guard`
rejects the unmarked map at the compile step first.
* Check every clause that can hold a value, not a list of five
The walk inspected `:where`, `:having`, `:set`, `:values` and `:join-by`.
HoneySQL has about 92 clauses, so that missed real ones -- notably join ON
conditions, which are the most used clause family in the codebase (~650
sites across `:join`, `:left-join` and friends), plus `:do-update-set` and
`:on-duplicate-key-update`.
Worse than the omissions was the direction. Unknown *operators* failed
closed, but an unlisted *clause* was skipped silently, so a clause nobody
had classified went unchecked. The walk now lists the clauses that hold only
structure -- `:select`, `:from`, `:order-by`, the DDL -- and checks
everything else, so a clause added to HoneySQL later is inspected rather
than ignored.
A join clause is a flat sequence alternating target and ON condition, and
both are keyword-headed vectors. The operator table tells them apart: a
condition starts with a known operator, anything else is a table.
Also drops `long*`, `longs` and `str*`. They were `clojure.core/long` and
`str` with docstrings -- an alias layer with no added behaviour, and nothing
used them once the marker existed. `[:auto/param]` is what the walk can
actually see; a coercion returns a plain scalar it cannot distinguish from
an uncoerced one. GHY-4475 wants amending to match.
* Load the check from app-db core, and move test scaffolding out of src
Removing the `keep-me` no-op exposed a real gap: nothing loaded this
namespace, so neither the marker lift nor the check ran in a running system.
It was masked while the adopting namespace still required it. `app-db.core`
now requires it the way it requires `honeysql-guard`, and `keep-me` is a
reference inside `comment` rather than a defn, matching the convention.
The test-only query that proved the check is scoped to its caller no longer
lives in `src`. The test namespace adds itself to the enforcing set instead,
which works now that the munged class prefixes are derived on each call
rather than captured once at load; two copies of that set could drift.
Adds a test that an unmarked string is compared as a literal. A string was
never the danger -- HoneySQL binds it -- and saying so keeps the marker's
purpose clear: it is for values that are not strings, which HoneySQL would
otherwise compile as structure.
Records that the marker belongs in a query map rather than a kv-arg. Toucan
builds kv-args into the where clause after the lift runs, so a marker passed
that way ends up inside a PARAM call instead of a comparison.
* Reduce to the value binding, and drop the walk
Review found the walk failing open in every position it did not explicitly
classify -- a map reaching `check-clause!` returned nil rather than throwing,
which left `{:where [:and [:= :a 1] {:raw "1=1 OR true"}]}` compiling to
`WHERE (a = ?) AND (1=1 OR true)`. The left-hand side of every comparison,
arguments past the declared indexes, `:join-by` conditions, and
`:limit`/`:offset`/`:fetch` were all unchecked for the same reason. A check
that advertises a guarantee it does not provide is worse than no check, so
the walk is gone rather than patched. It can come back built fail-closed.
What remains is the part that worked: a `[:auto/param v]` marker is lifted
at the compile step into HoneySQL's `[:param :kN]` plus its params map, so a
value written inline binds as `?`.
Four bugs in that half are fixed. `postwalk` descended into a marker's own
payload, so a nested marker bound a `[:param]` vector instead of the value;
it now uses `prewalk` and takes the payload whole. Toucan re-enters `compile`
with the `[sql & args]` vector it produced, so the method ran twice per
query; it now only walks a map. Param keys were sequential and guessable,
letting a `[:param :p1]` from request data bind another slot's value; keys
are now random. The options binding replaced `:params` wholesale, dropping an
enclosing compile's params; it merges.
`honeysql_guard.clj` is reverted. `:params` is never a key in a query map --
it reaches HoneySQL as a format option -- so that change was dead code in a
file backported to v58, v61 and v63, and the comment claiming `[:param]`
needed it was wrong.
* Reject a malformed marker, and do not treat a map entry as one
A marker-headed vector of the wrong arity now throws. HoneySQL does not
recognise the leftover form and compiles it into a call to a function named
PARAM, so `[:auto/param v extra]` previously failed at the database with an
error pointing nowhere near the typo.
A `MapEntry` is a two-element vector whose first element is its key, so the
map `{:auto/param v}` would otherwise look like a marker and the map holding
it would be rewritten away. That map means "a column named :auto/param", so
reaching it takes misuse rather than ordinary use -- but excluding the type
costs one check, so the walk does it rather than leaving the shape to chance.
Testing `vector?` instead of `sequential?` does not help here, since
`MapEntry` implements `IPersistentVector`.
This also drops support for a marker written as a seq. Nothing writes one,
and the looser test was widened for surface area rather than a caller.
Adds tests for the write path and for a query with no markers, neither of
which the unit tests reached: they exercise `auto-param` directly, so the
pipeline method's own branches were untested.
* Accept one marker shape, and never let one reach SQL
`[:auto/param value]` is the one supported form. Anything else headed by the
marker -- a seq, the wrong arity -- now throws instead of being left to
compile, because HoneySQL does not recognise the keyword and turns whatever
is left into a call to a function named PARAM that fails at the database
with nothing pointing back to the query.
Two assertions hold the line. Nothing marker-shaped may survive the rewrite,
and every value lifted out has to come back as a bound argument: a marker in
a slot HoneySQL formats as an identifier is rewritten but never consumed, so
`{:from [[:auto/param :t]]}` silently produced `FROM param AS p3lg...`,
dropping the value and putting the generated key in the SQL text. That now
throws.
A marked nil is left to HoneySQL rather than bound. It turns a literal nil in
a comparison into `IS NULL`, while a bound parameter gets `= ?`, which no row
satisfies -- so wrapping a nullable value silently returned nothing.
A `MapEntry` is still excluded from the marker test, so a column named
`:auto/param` is left alone rather than rewritten away.
* Look for the marked value among the arguments, not for a count
`assert-every-value-bound!` compared how many values were lifted to how many
arguments came back, which any arguments satisfy -- not specifically the ones
that were lifted. A query carrying unmarked literals has arguments to spare,
so a dropped marker hid behind them:
{:from [[:auto/param :t]] :where [:and [:= :a 1] [:= :b 2]]}
;; => FROM param AS p16vl... WHERE (a = ?) AND (b = ?) -- two args, one
;; param, count satisfied, value silently gone
It now looks for each marked value among the arguments. A sequential value is
spread across one argument per element, so its elements are looked for rather
than the collection; anything else, including the map this exists to
neutralize, binds whole.
Tests cover the shapes where the number of arguments and the number of marked
values disagree: a collection, the same value marked twice, and marked values
sitting alongside unmarked ones.
* Drop the bound-value assertion, and say what is not checked
`assert-every-value-bound!` rejected ordinary queries. It inferred from a
value's Clojure type whether HoneySQL would spread it across several
arguments, but that is a decision HoneySQL makes per operator: `:in` spreads
a collection, `=` binds one whole. So a vector compared with `=`, and a set
in an `:in`, were each checked the wrong way and threw. A collection holding
`nil` or `false` threw as well, because the check used a set as a predicate
and a set returns nil for those members even when it contains them.
It was also unsound in the direction it existed for. Comparing against the
argument list loses which argument came from which value, so a dropped
marker passed whenever an unrelated literal elsewhere happened to equal it:
`{:from [[:auto/param "de"]] :where [:= :msgid "de"]}` was accepted, with the
value gone and the generated key left in the SQL as an identifier.
No post-hoc check can be sound here -- HoneySQL gives no signal for a param
it did not consume -- so the namespace docstring now states the limitation
instead of a check pretending to cover it. A marker still may not survive the
rewrite, and a malformed one still throws; both of those are about the marker
itself rather than about what HoneySQL did with it.
* Lint a converted namespace's values for a missing marker
Marking a value does nothing to keep it marked: a later edit adding an
unmarked value slot to a converted namespace passes silently. This adds the
author-time check that holds the line, extending the hook that already
enforces `db.clj` confinement rather than adding a second one -- the two are
halves of the same rule, and that hook already knows which namespaces are
`db.clj`.
It reports a symbol reaching a value slot without a marker, following the
clause shapes a query map uses. Only a symbol: a literal cannot carry a
request value, and a value built inside the function is out of reach of a
check that does not follow it across a call. Only a `db.clj`, which is what
makes it decidable -- the values there are named in the parameter vector a
few lines up.
Off by default, enabled per namespace. Run globally it reports 269 sites, of
which 196 are ids that Malli already constrains to an integer and that
therefore cannot become SQL. Warning about those would bury the 73 that
matter -- strings and enums like `perm-type`, `setting-key`, `namespace`. A
namespace is added to `:config-in-ns` once its own values are marked, so the
list grows with the migration rather than ahead of it.
`metabase.content-translation.db` is the first entry. Removing its marker now
fails the lint.
* Bind a marked kv-arg, which is how queries are actually written
`(t2/select :model/X :locale [:auto/param v])` did not work, and kv-args are
how nearly every query in the codebase is written: 1125 call sites across
`*/db.clj` against 3 that pass a query map. So the marker covered the style
almost nobody uses.
Toucan builds a kv-arg into the where clause before the compile step sees it,
taking the marker keyword for the operator and folding the column in, so
`:locale [:auto/param v]` arrives as `[:auto/param :locale v]`. That arity
was previously rejected as malformed. It is now recognised and rewritten to
`[:= :locale [:param k]]`, which is the comparison Toucan would have built.
A marked kv-arg holding nil still compares as `IS NULL` rather than binding,
matching an unmarked one.
The lint reads the same style now. With it enabled across `src` it reports
1006 sites rather than the 269 it saw when it only walked query maps, of
which 769 are id-shaped and 237 are not -- close to the 78% the project's
estimate predicted, and the 237 are the ones that need a marker rather than
a coercion.
* Pair kv-args from the model, and do not lint written values
Several query fns take an argument before the model -- (t2/select-one-fn
:value :model/Setting :key k) -- so pairing the kv-args from a fixed offset
misaligned them and the values went unreported. They are now paired from the
model keyword, which is where the pairs actually start.
Insert calls are no longer linted for unmarked values. A value written into a
row is not a where-clause value an attacker can turn into SQL structure, and
a define-before-insert hook may read it before the query compiles, so a
marker there breaks the write rather than protecting anything.
* Reject a marked operator form, and check every argument of one
A kv-arg value may itself be an operator -- `:locale [:in [...]]` -- and
Toucan folds that into `[:auto/param :locale [:in [...]]]`, which the kv
rewrite turned into `[:= :locale [:param k]]`. That binds the operator form
as a single value and changes the comparison, so `IN (?, ?)` became `= ?`
against a list. It now throws and points at the working form, which is the
marker inside the operator rather than around it: `[:in [:auto/param vs]]`.
The linter examined an operator's arguments only when there were exactly
two, so `[:between :col lo hi]` and any other arity went unchecked -- a false
negative, which is the expensive direction for this lint. It now takes every
argument after the column, and `between` joins the operator set.
`read-call?` no longer assumes the call head is Named. Kondo always passes a
resolved var so this was unreachable, but the failure would have been a
linter crash rather than one bad file.
Also drops a duplicated paragraph that described the key generation as
gensymed when it uses ThreadLocalRandom, and replaces the ns docstring's
description of a misplaced marker with the statement it actually produces.
* Lint the query shape as well as its values, under one finding
A query written as `:column value` pairs is now reported alongside a value
that reaches a slot unmarked, under a single `:metabase/unsafe-app-db-query`
finding. A namespace opts into both at once, and the pair can be flipped on
globally when every `db.clj` has been converted.
Toucan builds kv-args into the where clause itself, so the query's shape is
not on the page -- there is nothing for a reader, this linter, or the `.sql`
extraction that follows this project to read. Every call has a map form:
`{:where [...]}` for a select, and for `update!` a conditions map, since its
spec takes `conditions-map-or-query?` rather than only pairs.
Checking the shape rather than the pairs' values also removes the reason the
marker had to understand a kv-arg at all. The value check now walks query
maps only.
* Redesign the LDAP settings page and its group mapping card
* Keep the mapping editor fields aligned when a name is rejected
* Merge the LDAP group mapping e2e flows into one
* Address the LDAP page review feedback
* Drop comments that restate the code
* Lock the LDAP attributes card and group the locked cards
`handle-circular-refs-test` has been failing intermittently since #82447.
The last assertion wraps `(umd/describe :metabase.lib.schema/query)` in a
500ms `with-timeout`. That timeout exists as a smoke test against runaway ref
expansion -- before `max-depth` was added to `metabase.util.malli.describe`,
describing this schema took literal minutes. It was never meant to be a
performance budget.
#82447 required closed map schemas in `mu/defn`, which enlarged the schemas
`describe` walks to its `max-depth` of 3. Since expansion cost is roughly
exponential in branching factor, this made the call ~4.5x slower. Measured on
one machine, isolated runs:
before #82447: 89ms cold, 33-50ms warm
after #82447: 232ms cold, 158-193ms warm
Headroom against the 500ms budget dropped from ~14x to ~2.8x. The test is
`^:parallel` and `with-timeout` runs its body in a `future`, so it competes for
the thread pool with every other parallel test; under CI contention ~175ms
crosses 500ms. That is why it presents as flaky rather than consistently red,
and why it showed up on the Postgres 14 and Java 25 suites -- those runners are
slower/more contended, not semantically different.
The describe output itself is unchanged (identical 187-char string), so only
speed regressed, not correctness.
Raising the timeout to 5000ms keeps the runaway-recursion guard meaningful (that
failure mode is minutes, not milliseconds) while surviving pool contention.
Not treating the 4.5x as a perf bug to chase: `umd/describe`'s only production
caller is the error-explanation path in `metabase.api.macros`, reached only
after validation has already failed, and it describes the nested sub-schema
rather than the full query schema. Validation itself is unaffected by #82447
(sub-microsecond before and after).
* Build the public question from the store
* Pass the expression editor its database, not the Metadata object
* Build the FieldValuesWidget test metadata with createMockMetadata
* Name the endpoints the public question's parameters come from
Refuse to save a query handle carrying bound parameters
A handle minted by execute_sql re-attaches the :parameters it ran with, but
:parameters is runtime-only and serialize-query strips it on the way into
dataset_query. Saving such a handle persisted the query without its filter and
without complaining, so the card returned rows the agent's own run excluded.
Fail closed in resolve-query-handle-for-save!, the chokepoint shared by
question_write, metric_write and transform_write, with a teaching error naming
the concrete alternative.
Closes GHY-4547
* Send the SAML SessionIndex in single logout requests
Metabase's LogoutRequest identified the session to end by NameID alone, which
is the user's email. Auth0 issues a distinct user ID per connection, so one
email can own several sessions and Auth0 cannot tell which to end:
client_error: No active session(s) found matching LogoutRequest
SAML Profiles 4.4.4.1 makes SessionIndex the element that scopes a
LogoutRequest to a session. Record the SessionIndex the IdP sends in its login
assertion on the session it creates, and send it back at logout.
SessionIndex is per session, not per user, so it lives on core_session rather
than auth_identity, which has a unique constraint on (user_id, provider) and
would let concurrent logins overwrite each other.
A hardcoded SessionIndex was tried once and broke Azure AD (#41600). Only an
index the IdP itself issued is ever sent; sessions without one (every session
predating this column, and every non-SAML session) produce exactly the request
they produce today. The existing saml-logout test, which asserts on the full
signed logout URL for a session with no SessionIndex, passes unchanged.
Requires saml20-clj 4.3.0, which parses SessionIndex out of login assertions
and can emit it in a LogoutRequest.
Fixes#77837
* Send the IdP's NameID, not the user's email, in logout requests
An IdP matches a LogoutRequest on NameID plus SessionIndex. We sent the user's
email as the NameID, but the IdP's NameID is not necessarily an email: Auth0
sends an opaque `auth0|<id>`. A LogoutRequest naming an identifier the IdP never
issued matches no session, so Auth0 answered:
No active session(s) found matching LogoutRequest
even once we started sending the correct SessionIndex.
Record the NameID and its Format from the login assertion alongside the
SessionIndex, and send those at logout. Sessions predating these columns have a
NULL NameID and fall back to email, so logout keeps working across an upgrade.
Verified end to end against a live Auth0 tenant: the LogoutRequest now carries
NameID `auth0|<id>` with Format `unspecified` and the assertion's SessionIndex,
Auth0 returns StatusCode Success, and the session row is deleted.
* Manage Unsynced Content modal
* add actions
* Show dependies when attempting to remote sync
* adjust spacing of dependency rows
* handling snippet and dashboard dependencies
* reshaping API response for FE consumption
* save from modal again
* cleanup
* PR Feedback, Design Feedback
* handle instance analytics case
* fixing function string
* Mike's feedback
* fix getIcon hook
A file shared with no text, or a message that only mentions the bot, reached the thread history as an empty user
message, and Anthropic rejects the whole request for it. Name the attachments instead, or leave the message out.
* Warm the Kondo cache so cache-reading hooks run in CI
`hooks/ns-analysis` reads only `.clj-kondo/.cache`, and Kondo writes that cache after every file in a run has
been analysed. `mage kondo` deleted the cache and linted in one pass, so a hook that asks about another
namespace's vars found nothing and skipped its check. `:metabase/prefer-with-dynamic-fn-redefs` therefore never
fired in CI while warning locally against a warm cache.
Lint once and discard the findings before the real pass. `kondo-redundant-ignores` and `kondo-insert-ignores`
get the same warm-up: on a cold cache the former reports needed suppressions as redundant.
Turning the linter on exposes 327 flagged `with-redefs` forms. 316 become `mt/with-dynamic-fn-redefs`. Nine keep
`with-redefs` where the redefined function runs on a thread a dynamic binding does not reach, or on a value the
proxy cannot wrap. Two `.cljc` sites are covered by a namespace entry in config.edn instead, since the macro is
JVM only.
Delegating replacements capture the original with `mt/original-fn`. A bare symbol or `@#'the-var` resolves to
the proxy once a var has been patched, which recurses on the second test in a JVM.
Inline ignore budget 28 to 37, config suppressions 3 to 4.
* Stop counting prefer-with-dynamic-fn-redefs ignores
Genuine cases are common: Quartz workers, pool callbacks, queue delivery workers and values the proxy cannot
wrap all need a root swap. A numeric ceiling adds churn to every PR that meets one. Each ignore still needs a
comment explaining why.
* Harden the Kondo warm-up and keep a primitive fn on with-redefs
`next-page-size` in the BigQuery driver takes and returns primitive longs, so compiled callers invoke it
through `IFn$LLLLL`. The dynamic-redefs proxy is a plain variadic fn, so the conversion threw
ClassCastException, and since patching is permanent every later BigQuery query in the JVM would too.
The warm pass now also lints `dev/src`, which defines vars that `dev/test` redefines. That exposes six sites
in the kondo ratchet tests, now converted. A warm pass that exits with anything but 0, 2 or 3 fails the
task rather than leaving a partial cache. `kondo-updated` warms when no cache exists, and the ratchet tools
warm the roots they lint.
* Only skip kondo-updated's warm pass after a finished one
A cache directory can be partial, left by an editor or an interrupted run, and `kondo-updated` then skipped
warming and missed cache-dependent findings. `warm-cache!` now writes a marker when it finishes, deleting it
first so an interrupted pass can't leave an old one, and `kondo-updated` checks for the marker instead.
* Only let a full warm pass mark the Kondo cache warm
A warm over a narrower set of roots leaves the cache incomplete, so it must not write the marker
`kondo-updated` trusts, nor clear one that a full pass wrote.
* Point the Athena TODO at the macro the test now uses
* Say why kondo-updated accepts a stale cache
Snowplow events are POSTed asynchronously and then ingested by
snowplow-micro before they can be read back via `micro/good`. The
1000ms retry budget in the Cypress helper was too tight for the event
to land on loaded CI runners or under network throttling, producing
spurious "Snowplow retry timeout" failures. Bump the budget to 5000ms;
this only affects how long a failing assertion waits, since matching
events still return early.
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* Drop the unused metadataFields prop from the reference detail pages
* List the reference databases from the fetched response
* Fetch the reference segment list where it is rendered
* Add relation selectors to the metadata store
* Read the data selector's entities from the store
* Take the data selector's entities as plain objects in its consumers
* Format the data selector spec
* Inline the table schema lookup
* Name the data selector's lookups prop entityLookups
* useRenderedDataVisibility
* static viz, other adjustments
* remove dead hook
* Split panels, simplification
* handling line series with a range very close to chart boundary, ignore goal line
* return true on traversal if intersection is found
* Move security-center endpoints into enterprise security_center
Moves security-center.ts out of the shared metabase/api module into
enterprise/frontend/src/metabase-enterprise/security_center/api.ts. All
of its endpoints hit /api/ee/ URLs and every consumer lives in the EE
security_center module, so it was an OSS api file with zero OSS
consumers.
The file now uses EnterpriseApi.injectEndpoints, matching the existing
in-module EE precedents (monitor/ai-auditing/metabot-analytics/api.ts,
table-editing/api/index.ts). The security-advisory tag type moves from
the OSS TAG_TYPES to ENTERPRISE_TAG_TYPES alongside it.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* Cast test store state for EnterpriseApi selectors
The listSecurityAdvisories endpoint selector is now typed against the
enterprise tag union, while renderWithProviders builds a store typed
with the OSS State. The runtime state shape is unchanged, so the spec
casts through a helper with a justification comment.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* Use relative imports for EnterpriseApi inside the enterprise module
security_center/api.ts was reaching its own module through the
metabase-enterprise alias; same-module imports are relative per the
import-style convention.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* Import EnterpriseApi through the metabase-enterprise alias
../api/api reaches the sibling api module, not security_center's own
files, so the same-module-imports-are-relative convention does not
apply here. The alias form matches the other in-module EE endpoint
owners (metabot-analytics, table-editing) and is the specifier the
no-base-api-access rule tracks EnterpriseApi by.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* Rewrap the cast comment in the promo card spec
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* Verify security-center mutations invalidate the advisory cache
* Simplify security-center tests and flush query updates
* Match the security-center api spec titles to the folder's should idiom
---------
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
* Use affected-test plans to narrow frontend unit tests
* Pass affected test plans by artifact and reject Jest runner failures
* Extract unit test selection preparation from workflow YAML
* Follow the shared Jest config files in the frontend path filters
* Drop the unused import/no-commonjs directive from the filter spec
* Leave heap retention specs out of the planner's unit list
* Check unit test results in a script and let suite hook failures reach quarantine
* Warn when a narrowed unit test run matches none of Jest's test paths
* Reword the workflow cast comment in the test plan spec
* Serve the locale catalogues as rspack chunks
The catalogues were fetched from `app/locales/<locale>.json`, a path outside
everything rspack does for an asset. They were unhashed, so `cacheable?` never
granted them far-future headers, and uncompressed, so the server gzipped each
one per request.
Importing them instead puts them through the normal pipeline: rspack hashes each
catalogue, emits it as its own chunk under `app/dist`, writes the `.br` and `.gz`
siblings, and resolves the URL at runtime. A hashed file there is already
cacheable, so no new rule is needed.
Production swaps `devtool` for SourceMapDevToolPlugin, which takes an exclude. A
source map for a catalogue maps JSON onto itself and would add about 29 MB.
* Format the specs the locale change touched
* Keep the catalogues out of the SDK bundle
A context module bundles its whole directory wherever it resolves, so importing
the catalogues added all 36 to the SDK's reachable graph: 66 files to 102, and
its total gzipped size up 275%.
The SDK should not carry them regardless of size. It renders in a customer's
page against whichever instance it is pointed at, so its catalogue has to come
from that instance rather than from whenever the SDK was built.
`locale-catalog` now names how a catalogue is loaded, the way `ee-plugins` and
`sdk-ee-plugins` name their implementations. The app imports from its own
bundle; the SDK bundle config aliases the fetch instead, which also keeps the
context module out of it.
* Let the SDK cache the catalogues it fetches
A catalogue served from `app/locales/<locale>.json` carries no hash, so
`cacheable?` never matched it and the SDK re-downloaded one on every load.
The SDK does not need the app's chunk hash to fix that, and it cannot have it:
the two bundles are separate compilations with their own chunk ids, filename
templates and runtime wrappers, so the same catalogue hashes differently in
each. What it needs is any token that changes when the catalogues do, and it
already has one. The instance serves the SDK bundle, so whatever build produced
it produced the catalogues too, and `EnvironmentPlugin` already compiles that
build's version and commit into it.
The SDK now names its build in the query, and a catalogue request that carries
one is cacheable. Without one it is not, so a caller that omits it cannot pin a
stale catalogue.
* Move the locale loading out of metabase/api
Nothing here calls an API endpoint. The catalogue is a static asset, loaded by
import in the app and by a plain fetch in the SDK, so it sits alongside
`utils/i18n`, which applies what these return.
* Swap the locale loader by path, not by alias
The loader was reached through a `locale-catalog` alias, which had to be
declared in the rspack aliases, tsconfig and jest config before the SDK bundle
could override it: four places to say one thing.
The importer now uses a plain relative import, which every resolver already
understands, and the SDK config keys its override on the resolved path. One
config knows a swap happens, and it is the one doing the swapping.
* Let the SDK load catalogues as chunks too
The SDK fetched catalogues from the instance over HTTP because a context
import pulled all 36 into its bundle. They were never in its initial
payload: an import() context always emits async chunks. What tripped CI
was the gate itself, which is `embedding-sdk-chunked/total`, the reachable
set. All 36 counted, 8.5 MB gzip against a 3.1 MB bundle, so 275%.
The SDK can use the same loader. `sdk-public-path.ts` points
`__webpack_public_path__` at the instance, and the package injects the
bundle as a script tag, so a host app's bundler never sees the import().
Chunks land under `app/embedding-sdk/chunks/`, which the backend already
serves with far-future immutable headers, so the build token and its
`cacheable?` clause are not needed either.
Measure the catalogues out of the reachable totals instead. One loads per
session and none for English, so counting all 36 overstates by 35x, and
their size tracks the translation process rather than any code change.
Drops the loader, its spec, the build-time alias and the backend clause.
The SDK's chunkFilename takes `[name]` so the gate can recognise them.
* Fold the loader into its only caller
Nothing overrides `loadLocaleCatalog` through the build any more, so it does
not need to be its own module.
Drop the SourceMapDevToolPlugin too. It existed to keep source maps off the
locale chunks, which only cost disk in a dev build, and it made `devtool`
conditional to work around the two not being able to run together.
Clear the comments that explained the change rather than the code: the specs
said what the catalogue used to do, and two references pointed at the file
this commit removes.
* Point the locale test hooks at the chunk
The catalogue arrives as a hashed rspack chunk, so the specs that stubbed or
waited on `GET /app/locales/<locale>.json` were watching a request that no
longer happens.
Five e2e specs used the intercept only as a synchronisation point, so they
now wait on the chunk instead. `content-translations` stubbed an empty
catalogue to keep the UI in English while it checks that content is
translated, and a chunk cannot carry a JSON body, so it blocks the chunk. Its
Arabic block asserts English chrome and `AR-` content only, nothing that
needs the catalogue applied.
That leans on LocaleProvider catching a failed load, so cover it: a rejected
catalogue still renders the children.
Also fix the import order the module move disturbed in three consumers.
* Spy on loadLocalization instead of wrapping the module
* Drop a tombstone comment
* Keep the locale set when its catalogue fails to load
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01NUuaopJ7ZtMVvdwwxF6qyx
* Wait on the language landing, not on the catalogue request
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01NUuaopJ7ZtMVvdwwxF6qyx
* Keep the locales directory when the translations step clears it
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01NUuaopJ7ZtMVvdwwxF6qyx
* Assert a real Dutch string after the first language switch
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01NUuaopJ7ZtMVvdwwxF6qyx
* Re-render the children once the catalogue is applied
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01NUuaopJ7ZtMVvdwwxF6qyx
---------
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
* Consolidate the module viewers into one explorer
`./bin/mage modules-tree --html` writes a single page with three views
of the module config: the tree, a coupling ranking, and a dependency
graph around focused modules. `dev.module-explorer/open!` builds the
same page in a dev REPL with namespace-level edges, so the graph can
show which API namespaces each consumer uses.
The graph replaces dev.module-viz and its Jetty server. Source paths,
stats and teams go through the shared resolver, so nested and renamed
modules report correctly. module-team moves into hooks.common.modules.
* Label module counts as used by and uses
* Refine module explorer semantics and state
* Move MetabotPromptButton into metabot behind a querying plugin slot
* Stop querying reading metabot permissions and sidebar state
* Test the data reference auto-open prop instead of metabot state
* Pass metabot's editor and error UI in as props instead of a plugin slot
* Fix editor specs for the metabot permissions hook and the barrel mock
Only the upload layer's own 4xx errors reach the model, and everything else gets a generic line. CSV validation errors had no status code, so they throw as 422s now and the upload endpoint answers 422 instead of 500 for a malformed file.
* Break requiring-resolve cycles
* Place moved code where its consumers can already reach it
Avoids widening module :api sets for three of the extracted namespaces.
dashboard->resolved-params moves to metabase.parameters.params.
template-tag-parameters and parameter-template-tag? move to metabase.query-processor.card.
Quartz's ConnectionProvider moves to metabase.app-db.quartz; task.bootstrap keeps the connection interceptor.
metabase.app-db.connection, metabase.dashboards.params and metabase.queries.template-tags leave :api.
* Restore comments lost when code moved
The c3p0 deadlock explanation goes back above the `locking` call in prometheus.clj.
The driver-implementation caveats go back on the requires that replaced their requiring-resolve calls.
The PulseChannel-hooks note and the placement rationales for the moved parameter functions come back too.
* Point mcp v2 projections at the moved dashboard->resolved-params
Master's mcp v2 merge landed a caller while the function was moving to metabase.parameters.params.
---------
Co-authored-by: Chris Truter <chris@metabase.com>
A card's `database_id` is copied from its source card's row, so a dependent rewritten before its
source kept the old database id. The cascade now rewrites cards in dependency order, sources first.
* merge containerize-uberjar into uberjar workflow
* big refactor
* move things around, add inputs to fetch-artifact action
* update conditional logic
* clean up comments
cleanup whitespace
* simplify conditional logic
* guard conditional matrix
* review cleanup
* add job names
* cleanup whitespace
* update get-docker-info to resolve repo/tag inputs
* trim trailing whitespace
* add ecr auth if required
* add repo/tag inputs to containerize from uberjar
* use uberjar workflow to push container to ecr
* cleanup pr-env
* add proper label handling to pr-env
* clean up support for ecr
* simplify containerize
* move jar healthcheck into build step
* move containerize back out of uberjar
* review fixes
* a few more fixes
* code review fixes
* cleanup whitespace
* rename workflow to match scheme
* add verification on release, use DOCKERHUB_OWNER var
* fix conditional
* remove whitespace
* end with newline
* add extra guard for labels not pr env related
* Bump tiptap to 3.31.3
* Dedupe prosemirror-transform
* Fix document e2e failures after tiptap bump
* Drop PublicDocument unit spec
* Remove dark-mode band around document title
* Remove the dispatch-type hierarchy
Its only consumer was the `[:dispatch-type/fn :dispatch-type/*]` method on the ClojureScript
`=?-diff`, which the `:default` method now covers with an `fn?` check. Nothing else passed
`:hierarchy #'u/dispatch-type-hierarchy`, so `:dispatch-type/*` was never reachable as a parent
anywhere else.
`dispatch-type-keyword` still returns `:dispatch-type/*` for unmapped types.
* Rename :dispatch-type/* to :dispatch-type/unknown
The `*` read as a wildcard because the hierarchy made it one. With the hierarchy gone it is just
the value `dispatch-type-keyword` returns for a type it cannot classify, and the old name invited
`(defmethod foo :dispatch-type/*)` as a catch-all that would silently match only unclassified values.
* Tighten the dispatch-type docstring and comment
State the contract and drop the rest: the docstring repeated the `;; we should add more mappings here as
needed` comment sitting three lines below it, and said twice that the keywords have no hierarchy.
* Fix flaky persist-refresh admin email test by giving it its own logged-in admin
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* Warm babashka classpath once so parallel modules-consistency tests stop racing
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* Whitelist the babashka classpath warm-up for the parallel-test linter
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix: let EditableDashboard ask for the staged data picker
The picker picks itself: staged above 100 data sources, simple below, and
the simple one only offers tables and models. So passing entityTypes:
["question"] to EditableDashboard did nothing on a small instance, with
no way to opt out — SdkQuestion could already ask for the staged picker,
EditableDashboard could not.
Drop dataPickerProps from the shared dashboard props too. Only the
editable dashboard can reach the query builder, so Static and Interactive
had been inheriting a prop that could never do anything.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011ftDCPjFUZxKwoPda82Nc1
* docs: describe dataPicker in the dashboard guide
The guide's data picker paragraph still described dataPickerProps as
entity types only, and the note added for dataPicker called it the "full"
picker, which is not one of the type's values and is a term the guide
never introduces.
Fold both keys into the intro paragraph, name the pickers as the type
does, and leave only the entityTypes constraint after the snippet. The
defaults also go on the dataPickerProps JSDoc, so they reach the props
reference the guide links to.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TacHQDkaEDmdu5jYJUPs1X
---------
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Records one query_execution row per action execution under new context values action-execute / public-action-execute, so runs appear in the Usage Analytics Query log beside reads.
* Call build-scripts.yml from run-tests.yml
Make build-scripts.yml a workflow_call workflow and invoke it from
run-tests.yml, rather than duplicating its jobs there. The jobs
themselves are unchanged; only the triggers move.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* add proper conditions for build-scripts workflow
---------
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
* Stop the Data Studio index redirect clobbering a live navigation
`/data-studio` is an index route that only knows where to send you once the
`hasSeenGuide` user key-value request lands. The layout renders the nav around
it in the meantime, so a click can start a navigation while the redirect is
still deciding. Because a `route.lazy` destination resolves its module before
the router commits the location, the index route is still mounted in that
window -- and its `<Navigate>` replaced the navigation the user had asked for,
dropping them on the guide instead.
`useIsNavigating` already exists for exactly this window (see
`useDashboardUrlQuery`), so sit it out: whatever is under way unmounts us.
That is what quarantined the transforms permissions e2e test (GDGT-3169): it
clicked "Data transformation" straight after visiting `/data-studio` and then
looked for the enable-transforms page on the guide. The e2e helper now lands on
the redirect target before touching the nav, which also covers the same latent
race in the schema-viewer entry-point test.
* Trim comments to what a reader needs
* Let the Data Studio index redirect settle in the library entry test
* Cleanup
* GDGT-3169: Address PR comments
* Add store selectors for a table's field ids and foreign keys
* Read the query builder's source table from the store
* Read TableInfo from the store
* Link a connected table with the table URL helpers
* Build the connected table link from a dataset query
* refactor: rename the embedding hub checklist to setup guide (#79514)
* refactor: rename the embedding hub checklist to setup guide
"Embedding hub" originally meant the whole embedding section; what
shipped under that name in Sept 2025 was only the onboarding checklist,
which users already see as "Setup guide" -- it is the nav label and the
route. Renaming the code to match frees the name for the section the
embedding settings reorganization is building, and stops "embedding hub"
meaning two things.
Directory: frontend/src/metabase/embedding/embedding-hub -> setup-guide,
and the admin wrapper alongside it. Symbols: EmbeddingHub -> SetupGuide,
EmbeddingHubXrayPickerModal -> SetupGuideXrayPickerModal,
EmbeddingHubAdminSettingsPage -> SetupGuideAdminSettingsPage.
Deliberately unrenamed: useGetEmbeddingHubSteps, EmbeddingHubStepId and
the rest of the checklist payload types. Those name the
/api/ee/embedding-hub/checklist contract, and the backend keeps that name
on purpose -- renaming three persisted booleans buys nothing user-visible
and costs a migration.
Frontend-only. No persisted state, no API contract, no route change.
* refactor: finish the setup guide rename in tests and backend
EMB-2208 also covers the spec file names and the two backend comments.
The e2e spec and the SDK embed-setup unit spec move to setup-guide names
and their describe wording follows.
The backend keeps embedding-hub in the setting names, the API path and
the namespace, so both files get a note saying the names track the
Setup guide checklist rather than the embedding hub section.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01NnWkk2XpAZurnyELCZFtSE
* refactor: rename setup guide UI types and hooks
The UI model for the checklist still carried the old word:
EmbeddingHubStep, EmbeddingHubStepId, EmbeddingHubAction,
EmbeddingHubModalToTrigger, the two step hooks, and the home page
component. None of these name the API contract. They are declared in
setup-guide/types, independent of api/embedding-hub.ts, and
EmbeddingHubStepId lists seven step ids where the API's own
CheckListApiStep lists twelve.
api/embedding-hub.ts and everything in it keeps the old name, as does
the embedding-hub-checklist cache tag: those do name
/api/ee/embedding-hub/checklist.
Comments follow. The two above the embedding-hub-*-embed-snippet-created
settings now say the setting names stay while the thing they track is
the Setup guide.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01NnWkk2XpAZurnyELCZFtSE
* refactor: reword two missed setup guide mentions
An it() title in the SDK embed-setup spec and a comment above the
checklist keys in the API both still said "embedding hub" where they
meant the Setup guide.
The docstrings that name the API itself are left alone, as is the
:embedding-hub token feature.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01NnWkk2XpAZurnyELCZFtSE
* refactor: rename the checklist API client to setup guide
The client module named the endpoint rather than the contract. Nothing
in it is frozen except the URL, which is a literal string: the RTK cache
tag lives only in the frontend store, and the exported types and hook are
ours. So api/embedding-hub.ts becomes api/setup-guide.ts, its types and
query hook follow, and the embedding-hub-checklist tag becomes
setup-guide-checklist.
The surviving url: "/api/ee/embedding-hub/checklist" gets a comment
saying it is the one thing that has to keep the name.
The backend is untouched apart from comments. Its namespace keeps the old
name, so the symbols inside it stay consistent with it -- the seam is
between the two modules, not inside either.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01NnWkk2XpAZurnyELCZFtSE
* docs: name the setup guide in two comments
Shortening these to "the checklist" dropped which checklist it is, to buy
consistency with the surrounding names. Losing the information is the
worse trade.
In collection.ts there is no longer a mismatch to buy off anyway, since
the tag beside it is now setup-guide-checklist. In api.clj the comment
does disagree with its own namespace, and that is fine: the namespace
docstring already explains the seam.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01NnWkk2XpAZurnyELCZFtSE
* refactor: rename the checklist schema and builder
Both are private to the namespace and referenced nowhere else, so
nothing about them is frozen. The frontend type for the same payload is
already SetupGuideChecklistResponse, and two names for one response
across the seam is worse than a symbol disagreeing with the namespace
around it -- that disagreement is explained by the ns docstring.
The namespace, the route, the settings and the token feature keep the
old name, as does the endpoint docstring, which is published in the API
docs.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01NnWkk2XpAZurnyELCZFtSE
* refactor: say Setup guide in the two setting descriptions
They said "for tracking in the embedding hub" directly under a comment
saying these track the Setup guide and not the hub section, so the file
contradicted itself and a reader could not tell which line was stale.
Nothing renders these descriptions -- no admin settings widget reads
them, only the settings API payload carries them -- so they are internal
documentation, and the same accuracy argument applies as to any comment.
The setting names still stay.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01NnWkk2XpAZurnyELCZFtSE
* refactor: say Setup guide in the endpoint docstring
It described the response as the embedding hub checklist while the schema
right above it is SetupGuideChecklistResponse. The path in the route
docstring and the namespace still name the hub, because those are bound
to the URL.
This string is published in resources/openapi/openapi.json and
docs/api.json. Add the openapi-self-healing label to the PR and CI
regenerates the spec; docs/api.json has no per-PR check and picks it up
on its next regeneration.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01NnWkk2XpAZurnyELCZFtSE
* refactor: lowercase setup guide in prose
The codebase capitalises "Setup guide" only where it is the literal UI
label, as in EmbeddingNav, and writes it lowercase in running prose --
"Go to setup guide", "Return to the setup guide to continue". The
backend comments and descriptions added here did the opposite, and
api.clj disagreed with itself between one docstring and the next.
Sentence-initial mentions keep their capital, since that is ordinary
sentence case rather than the name.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01NnWkk2XpAZurnyELCZFtSE
---------
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
* style: format the setup guide route imports
oxfmt --check failed fe-lint on the integration branch; the two
setup-guide lazy imports were wrapped in the pre-oxfmt style.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BGNfTR7qNNvEQczJQDomoi
* feat: add the embedding hub shell and its Get started tab (#80346)
* feat: add the embedding hub shell and its Get started tab
The hub is a top-level section at /embedding, built on AreaLayout like
Monitor and Data Studio rather than on the admin sidebar, with its own nav
and an app-switcher entry. /admin/embedding stays exactly as it is; removing
it is EMB-1526 and comes last.
Get started replaces the vertical stepper with a grid of numbered cards in
two sections plus a useful-links row. Each Fine-tune step is locked by the
feature it needs rather than by one stand-in for "Pro", since an instance can
license SSO without modular embedding. A feature-locked card names no
prerequisite: "Set up SSO to unlock" is a promise the step is one action
away, which is only true above the paywall.
One tab today. The remaining six arrive with their own issues, each adding a
route, an export and a nav entry.
Two shared changes ride in front of it, each its own concern:
* `BaseLink` resolves an explicit `..` against the current route, which the
hub needs because it mounts admin components under a second base path.
* `AreaContent` grows a `fullWidth` prop, mirroring what the admin settings
layout already does, for the tabs that are editors rather than settings.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01QHCSv6oQRLUUGo34V1eEDp
* docs: correct the embedding hub module tier note
The comment said the hub could drop to feature tier once its components
were lifted out of admin. It cannot: the Permissions and Tenancy tabs
mount admin permissions and admin People a second time on purpose, and
admin keeps its own copy of both, so the hub imports admin-tier code
permanently.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01NnWkk2XpAZurnyELCZFtSE
* fix: finish the setup guide rename in the Get started page
The rename landed on the base branch, so it never reached files that
only exist here: the Get started spec still imported EmbeddingHubChecklist
from metabase/api/embedding-hub, which no longer resolves, and renaming
useGetEmbeddingHubSteps moved it past useSetupGuideModals alphabetically
and broke the sorted import blocks.
Both edits were made during a rebase, where pre-commit hooks do not run,
which is why lint-staged did not catch either one.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01NnWkk2XpAZurnyELCZFtSE
* feat: let AreaTab render an action as well as a link
The embedding hub's "New embed" control opens a modal rather than
navigating, so it was a Mantine Button with its own height and padding
and did not collapse with the navbar. It now goes through AreaTab, which
makes it the same size as every other rail item by construction.
Omitting `to` renders an UnstyledButton instead of a link. The two forms
are spelled out separately because their polymorphic component types do
not unify.
* refactor: make the UpsellBanner body text optional
The embedding hub's banner is a title and a button with no body copy.
Rendering an empty Text left a stray paragraph, and the large variant
top-aligned the row, so the gem and the button sat above the single
title line.
Without children the body is skipped and the row centres. Every existing
caller passes children, so none of them change.
* fix: return to the setup guide the user actually came from
The guide has three hosts, and every route out of it was hardcoded to
the admin one, so a user who started in the embedding hub was deposited
in admin settings on the way back.
The wizards now navigate relatively, resolving to whichever host mounted
them. The connect-a-database round trip leaves the app entirely, so it
carries the origin instead: the return param holds the path rather than
a bare "true", and the modal navigates there. Links already in flight
carry the old literal and fall back to admin as before.
The home-page stepper returns home; the admin guide keeps the default.
* feat: serve the setup guide checklist without a licence, and add the hub's two steps
The checklist sat behind the :embedding premium gate while the hub that
renders it is admin-only, so an unlicensed admin reached the page and
got a 402. Every step then read as incomplete -- including ones the
instance had genuinely done, which is worse than showing nothing. The
gate is gone; the endpoint checks superuser itself, which the gate had
been providing as a side effect.
The hub's own two steps join the response. Completion belongs to the
instance rather than to whichever host is asking, so both hosts read the
same source: configure-ai moves off a pair of frontend settings reads,
and create-custom-theme becomes answerable at all.
Telling a seeded theme from a user's needs a marker: the names are
stored already translated, and seeding is lazy so the ids are not
ordered. Existing rows are deliberately left false rather than
backfilled -- themes are recent, so the instances affected are few and
already embedding-active, and no retroactive rule is reliable.
* refactor: give the hub its own path, step list and unlicensed layout
Get started was the index route, so /embedding was a prefix of every
future tab's path and the wizards were siblings rather than children.
Both forced special cases in tab selection. It now lives at
/embedding/get-started with the wizards nested under it and /embedding
redirecting there, so tab selection is a plain segment match.
The hub also owns its steps outright rather than borrowing actions from
the shared hook. It already owned the copy, and the half-and-half
arrangement is what forced the action lookup and the omitted-step
handling; both are gone, and so is the URL parameter the hook only had
so the hub could pass its own wizard paths.
Without modular embedding every Fine-tune step is locked and AI is the
one advanced step still reachable, so it moves up into the first section
and the grid fixes two columns. Numbering follows position in the list
rather than arithmetic across three groups, which is what let the order
vary at all.
Also drop the helpers, the fullWidth chain and the SetupGuide prop this
branch introduced but nothing calls yet -- each arrives with the tab that
first needs it -- and order each file from its entry point down.
* refactor: address review findings on the hub shell
Restore the two setup-guide e2e tests the branch had deleted; nothing in
it justified their removal.
Drop the AvailableInEmbedding guard from the hub and Data Studio: it sat
inside a guard built from a selector that already returns false in an
iframe, so no state rendered differently. Add the hub selector spec its
Data Studio and Monitor siblings already had.
Take getIsEmbeddingHubApp as a tenth input selector rather than repeating
the path check inline; the instantiation-depth limit that comment warned
about no longer applies.
Order BaseLink's relative check last so the change is a pure append, and
say what actually goes wrong rather than describing it abstractly.
* fix: keep the checklist from 400ing without a database
has-user-added-database? ended in a when-let, so an instance with
neither a user database nor a sample database got nil rather than
false. The response schema is closed and requires a boolean, so the
whole checklist 400d. Every existing test called (mt/id) first, which
creates a database and hid it.
has-configured-ai? gets the same treatment: it ands two settings, and
a nil from either would fail the schema the same way.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JzD6eCBAWFddH31oQg9woe
* test: cover unlicensed checklist access and the hub's two steps
The 402-without-:embedding assertion described the gate this branch
removed, so it was red. It is replaced by the behaviour that gate
change was for: a 200 with real completion on an unlicensed instance,
paid steps reporting false rather than erroring, and non-admins
rejected now that check-superuser carries the access control the
premium gate used to provide as a side effect.
Adds coverage for create-custom-theme (a seeded theme does not count)
and configure-ai (credentials and embedded Metabot, both halves).
Leaves data-isolation-strategy-test without an (mt/id) call - that is
what caught the nil add-data, so it is the regression guard.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JzD6eCBAWFddH31oQg9woe
* test: add e2e coverage for the hub's Get started tab
The tab shipped on unit tests alone. These five cover what needs a
real browser or a real backend: the redirect from /embedding, both
step orderings, the wizard's relative back link, feature locking, and
completion arriving from the checklist endpoint unlicensed.
Its own filename rather than sharing the security tab's spec - the two
cover different surfaces, and separate files mean no conflict when the
stack restacks.
Both blocks are @EE. The unlicensed case is an EE build with no token,
not an OSS build: the checklist is served from /api/ee, so on OSS the
route would not exist and every step would read incomplete for a
reason this page is not responsible for.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JzD6eCBAWFddH31oQg9woe
* chore: record the embedding-hub module's new dependencies
Regenerated by ./bin/mage fix-modules-config. Both entries come from
the two checklist steps this branch adds: metabot for the AI settings
has-configured-ai? reads, and :model/EmbeddingTheme for the is_default
lookup behind has-created-custom-theme?.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JzD6eCBAWFddH31oQg9woe
* fix: account for embedding_theme.is_default in the serdes spec
serialization-complete-spec-test requires every column on a serialized
model to be declared, and is_default was new. It goes in :skip rather
than :copy: the flag records whether this instance seeded the row, not
anything about the theme itself, so it does not travel. An imported
theme lands as false - the admin's own - which is what importing one
is, and it ticks the create-custom-theme checklist step accordingly.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JzD6eCBAWFddH31oQg9woe
* refactor: serve the setup guide checklist from OSS
A community build never mounts /api/ee/*, so the checklist endpoint did
not exist there at all -- the four steps its UI actually shows without a
licence (connect a database, create a dashboard, get an embed snippet,
configure AI) had no backend to report them.
Move the namespace to metabase.embedding-hub.api and the route to
/api/embedding-hub/checklist. Two things had to change to load without
enterprise code:
- The JWT and SAML settings ship only with enterprise code, so a missing
require would fail at load. has-configured-sso? becomes a defenterprise
whose OSS body returns false; the real implementation stays in
metabase-enterprise.embedding-hub.api behind :feature :embedding, the
same token the route was gated on before.
- :model/Tenant, :model/ConnectionImpersonation and :model/DatabaseRouter
resolve to enterprise namespaces a community build does not ship.
Guarding each query with the :embedding feature short-circuits before
Toucan resolves the model, the way has-configured-sandboxes? already
did with :sandboxes.
Every key stays in the response and keeps its type, so the frontend needs
only the new URL. Enterprise behaviour is unchanged.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01F5HK8wzZwmAsF9PkgFMfLu
* docs: spell out "embedding hub" in setup guide comments
A bare "hub" reads ambiguously next to the setup guide, which the same
files also talk about.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01F5HK8wzZwmAsF9PkgFMfLu
* refactor: keep the moved checklist code closer to its original
The move added a few things it did not need to: boolean wraps around
expressions the :type :boolean settings already returned as booleans, a
nested cond where a flat one had been, and a named helper where the file
already inlines its feature checks. Undo all of it, so the diff against
the original shows only what moving the namespace forced.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01F5HK8wzZwmAsF9PkgFMfLu
* refactor: address review feedback on the embedding hub
Frontend: object args for useSetupGuideModals, {isLocked, reason} for
locked steps, edition-aware embed snippet copy, no complete badge on
locked cards, margins instead of nested stacks, and trimmed comments.
Backend: gate enterprise-only models on ee-available? rather than the
:embedding feature, and move the checklist api test to OSS.
* refactor: keep the original order in useGetCurrentApp
The hub lives at /embedding, so it never overlaps /admin -- moving the
admin branch to the end changed nothing at runtime and only widened the
diff. Append the hub case instead.
* refactor: rename the upsell utm constants
Renames UPSELL_CAMPAIGN/UPSELL_LOCATION to UTM_CAMPAIGN/UTM_CONTENT and
points SetupGuide at the "setup-guide" campaign.
Also restores the `boolean` wrapper on has-user-added-database?, since the
trailing when-let yields nil on an instance with no sample database and the
checklist response schema requires a boolean.
* fix: unbreak the embedding hub module boundary
The OSS embedding-hub api namespace now requires metabase.config.core, so
the module needs `config` in its :uses set — regenerated with
`./bin/mage fix-modules-config`.
Also reformats admin/routes.tsx to satisfy oxfmt.
* test: drop unit tests the e2e spec already covers
The Get started page's e2e spec walks the real step order, the upsell banner
and a checklist round-trip against a live backend, so the unit tests asserting
the same things were restating a weaker version of it. What is left is what
e2e cannot reach: mixed token shapes, checklist states that need a mocked
response, and the locked-card click gate.
routes.unit.spec.tsx stubbed the guard, the layout, the page and the whole
setup-guide module, which left it asserting that React Router matches a path.
Rewritten to read the route tree as data instead -- no mocks, no rendering,
and it still covers sso-setup, which no e2e visits.
Also drops two tests that asserted a code path the page does not have: it
builds its Fine-tune steps from a local list and never calls
useGetSetupGuideSteps, so no card can be omitted.
* test: drop the hub root redirect test
One line of route config that the Get started e2e spec already asserts
against a real browser. Removing it takes the Navigate lookup, the type cast
and the ReactNode plumbing with it, leaving the path list -- which is the part
nothing else covers.
* refactor: give the New embed action its own nav component
New embed is not a tab -- it renders in the nav's lower slot, outside the
tab list, is never selected and navigates nowhere. Reusing AreaTab for it
meant making `to` optional and branching the render, which put a link/button
split into a component seven other tabs depend on.
AreaNavButton is the button form as its own sibling, so AreaTab goes back
untouched.
* test: name only what the AI checklist test arranges
The block sets the provider key and nothing else -- embedded Metabot being
on is the default, not part of the arrangement. The block below it is what
pins that flag's contribution.
* style: match the New embed button to the design
It carries its own fill at rest rather than lighting up only on hover the
way a tab does, and its label and icon take the brand colour. Centred in
both the expanded and collapsed states.
* style: build the New embed button on the design system Button
The Figma spec is the design system Button -- subtle variant, MD, an `add`
leftSection -- so use it rather than hand-rolling the chrome. The tinted
resting fill comes from Mantine's `light` variant; our own `subtle` renders
transparent, which is a divergence from the Figma token worth raising with
whoever owns the design system.
* test: drop the AreaNavButton spec
It pinned that the component renders a button rather than a link, which was
a real decision while the chrome was hand-rolled. Building on the design
system Button makes it structurally true, so the spec now covers Mantine.
The layout spec already covers the only branch left, the collapsed label.
* test: sharpen the Get started page assertions
The per-feature test framed licensing one feature as a customer shape; it is
only how the gating is shown, since Pro carries everything bar the cloud-only
features. The prerequisite comment now says why an unlicensed step names none:
it is locked regardless. The AI modal is asserted by its title rather than by
there being any dialog at all.
* test: walk both setup wizards to the end
Each wizard now has a test that seeds its steps, reaches the last one and
takes the action that closes it out, asserting the hub is what it lands on.
They end differently: SSO navigates through the anchor itself, permissions
through a navigate("..") handler.
Also corrects the header comment. Both blocks are @EE because the steps are
enterprise-only, not because of where the checklist endpoint used to live --
it moved to OSS, so the old reason no longer holds.
* feat: name the area in the sidenav
The design gives the embedding hub a title beside its logo. AreaLayout
had no notion of one, so it takes an optional title and passes it
through; Monitor and Data Studio are unchanged.
It hides with the navbar, like the header controls do, so a collapsed
rail stays icon-only.
* docs: promise only the basics on the OSS first-embed blurb
AI is not part of a simple embed on OSS, so the pro line oversells what
the steps below it cover.
* perf: load the embedding hub in its own chunk
The hub pages were reachable from the eagerly imported route tree, so
they shipped in the main bundle for every user, embedding or not. Name
each page in an import() and register it with lazy, as monitor and
transforms routes do.
Resolve every loader in the route spec: an import() path and its export
are no longer type-checked, and a typo would first show as a blank page.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01ErtCFepL9GnTTBE3QwSmiH
* test: name the hub spec for the hub, not one tab
Every tab's browser test belongs in one spec, one describe each, so the
file name cannot go stale as tabs land. Its header pointed at a path that
does not exist; the older onboarding checklist is in
onboarding/embedding-homepage.cy.spec.ts.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01ErtCFepL9GnTTBE3QwSmiH
* fix: import the user selectors from metabase/current-user
getUserIsAdmin and useUserKeyValue moved into the current-user module
on master, so the old paths no longer resolve and the frontend build
fails outright.
* test: mock the LLM endpoints the AI card opens
The AI card opens AIProviderConfigurationModal, which fetches
/api/llm/providers and /api/llm/provider-types. Neither was mocked, so
"opens the provider modal while unconfigured" failed on the unmocked-route
guard -- deterministically, on every branch carrying this spec.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BdJ7UHzej4V16G2W6AP3hQ
* style: sort the LLM mock imports
sort-imports orders members by codepoint, so ProviderTypes precedes
Providers. Caught by fe-lint, not by the pre-commit hook.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BdJ7UHzej4V16G2W6AP3hQ
* fix: restore connect-database card copy on the get started page
The card had inherited the "Create a dashboard" card's description
from a copy-paste. Restore the wording used by the old setup guide
checklist while the design catches up.
* refactor: replace NumberedCard render closures with a HubCard union
The section arrays only needed to carry which card to render and its
data; wrapping each in a (position) => ReactNode closure just to defer
numbering added a layer with no benefit over switching on a plain
union at render time.
---------
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
* feat: add the embedding hub's Security tab (#80347)
* feat: add the embedding hub's Security tab
The method toggles become rows in one card rather than a card each, in the
design's order: Modular embedding, Modular embedding SDK, Guest embeds,
Full-app embedding. Below the paywall the card lists guest embeds alone --
the others need `embedding_simple`, whose runtime only registers when the
token carries it, so their toggles would be switches that turn nothing on.
The card's heading appears only when it lists more than one method.
SameSite is paid-only for the same reason: it governs the session cookie and
guest embeds never set one, so on a guest-only instance it offers a decision
that cannot affect anything the admin owns.
Published guest embeds are listed when any exist, whatever the toggle says.
That needs the two listing endpoints to answer with embedding switched off,
so they drop `check-embedding-enabled` and keep only their settings
permission check. Publishing and every serving path stay gated -- an admin
who has just turned guest embeds off is exactly the one who needs to audit
what is still live.
Admin's own Security page and its siblings keep their current exports
alongside the new ones; EMB-1526 removes that section and them with it.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01QHCSv6oQRLUUGo34V1eEDp
* fix: keep the CORS description literal for i18n extraction
babel-plugin-ttag extracts translatable strings at build time, so a jt
template's static text has to be literal in the source. Hoisting the
sentence into a variable and interpolating it made the extractor
hard-fail, which broke verify-i18n-files on this branch and on every
branch stacked above it.
Branch the whole jt expression instead, so each arm carries its own
literal sentence and the HoverCard stays the only interpolation. The
edition-aware copy is unchanged, and the resulting msgid matches what the
base branch already emits, so existing translations still apply.
* refactor: use the shared UpsellBanner on the Security page
Follows the Get started page off the hub's fork of UpsellBanner and onto
the shared component, wired the same way the embedding settings page
already wires it.
* test: scroll the published guest embeds card into view
The hub clips its content and scrolls it internally, so on a CI-sized
viewport the card sits below the fold. Cypress treats an element clipped
by a non-scrollable `overflow: hidden` ancestor as hidden, so findByText
resolved the heading but `be.visible` failed.
Both assertions were identical, so they move into one helper.
* feat: present modular, SDK and guest embeds as one switch
Separate switches add complexity without adding security, since whoever
can embed also controls them, and guest is a property of modular rather
than a method beside it. Full-app keeps its own switch.
The merge is presentational: the switch reads on when any of the three
settings is on and writes all three at once, which is what the backend
flag will do once EMB-2257 replaces them with one. A setting pinned to an
env var cannot be written, so one pinned setting locks the row rather
than letting a partial write recreate the mixed state.
* test: cover the Security page's conditional cards
The spec only exercised the published-embeds card. SameSite and the full-app
origins list are conditional too, and their conditions are the ones a reader
is most likely to get wrong.
* feat: rewrite the embedding legalese modal copy
One switch now covers modular embedding, the SDK and guest embeds, so
the modal no longer needs a per-setting headline. The list becomes one
paragraph, and the actions read Cancel / Agree.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JVkbXYuZi14Cpnv22AGGJu
* feat: name the method each Security setting belongs to
One page now holds settings from several embedding methods, so the
secret key and the authorized origins say which method they apply to.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JVkbXYuZi14Cpnv22AGGJu
* feat: reword the SameSite cookie description
Drop the method list, split the copy into two paragraphs, and render it
in the secondary colour the other rows inherit from SettingHeader --
the nested Text was resetting the colour to primary.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JVkbXYuZi14Cpnv22AGGJu
* feat: reword the embedding methods rows
The paid row leads with what modular embedding and the SDK let you
build. OSS has one method, so its row says Enable embedding rather
than naming guest embeds.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JVkbXYuZi14Cpnv22AGGJu
* test: match the renamed authorized origins title
The widget title now names the method, so the exact-text query no
longer matched.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JVkbXYuZi14Cpnv22AGGJu
* docs: source the merged switch rationale
The reason was real but read as an assertion. Name where it came from:
Alessio's call in the Embedding Settings Reorganization thread, carried
by EMB-2257.
Trim the OSS paragraph to its first sentence. Which methods the token
feature gates is visible in the code right below it.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01ErtCFepL9GnTTBE3QwSmiH
* perf: load the hub's Security tab in its own chunk
Follows the lazy route registration the Get started tab now uses, so the
Security page does not pull the admin settings widgets into the main
bundle. Resolve every loader in the spec: an import() path and its export
are no longer type-checked.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01ErtCFepL9GnTTBE3QwSmiH
* refactor: drop the EmbeddingSecurityWidgets wrapper
It had one consumer. The admin Security page never used it -- it renders
the two widgets itself -- so the extraction shared nothing, and living in
that page's file made the two look like one component renamed. The hub
page composes the widgets directly instead.
Restores the admin page's spec, which had been repointed at the wrapper
and so left the shipped page uncovered. The widget behaviour it asserted
is already covered by the hub page's own spec.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01ErtCFepL9GnTTBE3QwSmiH
* refactor: compose the methods card the way peer settings do
Pass the heading to SettingsSection instead of rendering a Title inside
it, and drop the Stack the section already renders. Label the row's
switch, which every other embedding toggle does.
Say where the merged-switch decision came from rather than asserting it,
and mark mergedSettingKeys as temporary at both ends -- EMB-2257 replaces
the fan-out with one setting and deletes the prop.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01ErtCFepL9GnTTBE3QwSmiH
* refactor: title the published embeds card as its source page does
SettingTitle, matching the admin Guest embeds page this card came from,
which also gives the section its anchor id.
Drops embeddingHubAppearance -- no appearance route exists.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01ErtCFepL9GnTTBE3QwSmiH
* fix: write every merged setting from the terms modal
The modal intercepts the first enable, and it wrote only the setting it
was handed. So the very first time an admin turned the merged switch on
-- the path every new instance takes, since the terms start unseen --
enable-embedding-sdk and enable-embedding-static stayed off while the
switch read on.
The existing test missed it by pinning show-simple-embed-terms to false,
which routes around the modal. Covered now at that seam, and end to end
through a real click and a fresh load.
The browser test has to start from embedding off, which the default
snapshot does not: it enables sdk and static, and the merged switch reads
on when any of the three is on. Publishing a dashboard is itself gated on
guest embeds, so the sibling test enables them before it publishes.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01ErtCFepL9GnTTBE3QwSmiH
* fix: check embedding and embedding_simple features independently
Modular embedding and full-app embedding are separate token
features. Gating full-app on embedding_simple hid it for accounts
that only have the embedding feature.
* test: cover embedding_simple and embedding features independently
The methods card now checks the two token features separately, so
the test setup needs to mock embedding alongside embedding_simple
and cover the full-app-only case the merge previously collapsed.
---------
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
* feat: consolidate the three embedding method settings into one flag (#80348)
* feat: back the merged embedding switch with one setting
Modular embedding, the SDK for React and guest embeds become one
setting, `enable-embedding-modular`; full-app keeps its own. The
previous commit merged them in the UI only, writing all three at once,
which left the gates reading something the switch did not say.
There is no migration. Env beats the app db when a setting resolves, and
a Liquibase changeset cannot see env, so a migration would mishandle
env-pinned instances -- and writing env-derived values into the db would
freeze them, breaking the operator's env var as the live control.
Instead the getter falls back: until the new setting is set it reads as
the OR of the three it replaces, taken through normal resolution so
their env vars count. Nothing is rewritten, so nothing is lost going
back.
The fallback keys off the raw value's *source*, not its value, because
an unset boolean reads as its `false` default. Without that, "an admin
turned it off" and "nobody set it" are the same state, and turning
embedding off bounces back on from a legacy value.
Analytics use `simple_event` rather than a new `embed_share` version:
its event field is a closed enum, and Snowplow is on its way out in
favour of Metaplow, so versioning it is throwaway work.
The three old settings are deprecated rather than removed. Retiring
their env vars is a breaking change, documented separately.
* chore: mark the replaced embedding settings deprecated in 0.65.0
The embedding hub is expected to ship in 65. Tracked in the Notion
release-projects entry; if it slips, these three move with it.
* fix: report embedding telemetry from the setting that is now written
Nothing writes the three replaced settings after the cutover, so on any
instance using the merged toggle they stay false forever while embedding
is on. Left alone, every field derived from them would report embedding
as disabled and adoption would look like it collapsed.
Both call sites now read the setting that is actually written. The field
names stay as they are so existing reports keep resolving, which does
mean three fields that used to differ now always agree -- there is no
distinction left to report.
internal-stats reads through `setting/get` rather than
`get-value-of-type`, which bypasses custom getters and would have missed
the fallback to the replaced settings. Its output feeds metering-stats,
so this one is a billing signal, not only analytics.
* test: point the embedding tests at the merged setting
The three settings the modular flag replaces still exist, so these kept
passing through the fallback getter rather than through the gate they mean
to exercise.
The migration test keeps the legacy keys: it asserts on rows a historical
changeset wrote, not on current behaviour.
* refactor: read the merged embedding setting everywhere
Frontend and e2e still keyed off the three settings the modular flag replaces.
Repointing them is what makes the flag the single source of truth rather than
one more value alongside them.
The legacy keys stay in the settings types: the backend still defines them.
The legalese modal loses its setting argument. It now writes the merged flag
and clears both terms settings -- either one left on would re-open it. Turning
guest embeds on now goes through the modal, which it did not before; that
follows from guest being folded into modular.
* fix: keep the embedding terms modal paid-only
Consolidation pointed the OSS row at the same setting the modal guards,
so an OSS admin enabling guest embeds would have been asked to accept
terms that do not apply: the copy is about modular embedding and the SDK,
and about shared accounts as unfair use of a paid seat, while guest
embeds serve anonymous viewers over a signed JWT and OSS has no seats.
Gate the terms path on the embedding_simple token feature. Paid still
sees the modal on off -> on, OSS never does.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01ErtCFepL9GnTTBE3QwSmiH
* test: start the embed modal card test from embedding off
The merged setting reads as the OR of the settings it replaces, which
the pro snapshot leaves on, so the card the test looks for never
rendered.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JVkbXYuZi14Cpnv22AGGJu
* fix: stop reading the deprecated embedding settings
Marking the three replaced settings deprecated made every remaining
call site a lint failure. some-embedding-enabled? reads the merged
setting, which answers the same until an admin writes it. The env-var
sync tests exercise the deprecated settings on purpose, so they keep
them behind an ignore, and the budget records the ten new ones.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JVkbXYuZi14Cpnv22AGGJu
* test: drive the guest embed tests from the static flag
These exercise the /api/embed endpoints, which guest embedding gates.
Pointing them at the merged enable-embedding-modular made them assert
against a flag wider than the behaviour under test.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011eWbw5TiosJfLk9uLwKLgH
* fix: collapse the merged embedding flag in the e2e setup
Renaming the three settings to one left each call site written twice.
In embed-parameters the two calls disagreed -- simple true, static false
became modular true then modular false -- so the whole spec ran with
embedding off.
The two auth-switch tests now share one setup by design, since one flag
governs both options. Their titles said otherwise.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011eWbw5TiosJfLk9uLwKLgH
* feat: consolidate the two embedding terms settings into one flag
The merged toggle governs modular embedding and the SDK, so the terms
behind it are one gate, not two. The switch was OR-ing the pair inline
while the wizard and theme editor still wrote only the simple one, which
re-opened the legalese modal over terms the admin had accepted.
show-modular-embed-terms reads as that same OR until an admin answers it,
so an upgrade cannot drop an acceptance, and every accept path now writes
the one setting.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011eWbw5TiosJfLk9uLwKLgH
* docs: explain the merged embedding stats read and e2e setup
Note why the stats namespace reads through setting/get (to run the
fallback getter), and what the merged e2e specs start their embedding
state from.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011eWbw5TiosJfLk9uLwKLgH
* test: give the embedding settings specs one enable option
The helper carried isEmbeddingSdkEnabled and isEmbeddingSimpleEnabled,
OR-ing them into the single setting that now backs the switch. They also
doubled as the embedding_sdk and embedding_simple token features, which
stay separate, so the two uses are split rather than merged: one
isEmbeddingEnabled for the setting, token features passed on their own.
Restores envSettingKeys over a boolean, since the card still houses a
second row whose setting can be pinned to an env var, and drops terms
fixtures from specs that cannot reach the modal.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011eWbw5TiosJfLk9uLwKLgH
* test: cover the merged legalese behaviour
One flag means a state that could not happen before: embedding on with
the terms never accepted. An OSS admin turns guest embeds on, sees no
legalese because none is shown below the paywall, and upgrades.
The admin toggle prompts on off to on only, so it stays quiet there by
design; the embed flow asks for the terms instead. Both halves are
covered, along with the other states the flow section renders.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011eWbw5TiosJfLk9uLwKLgH
* test: drive the embed setup helper from one enable flag
The helper OR-ed simpleEmbeddingEnabled and guestEmbeddingEnabled into
the single setting the switch now backs -- two options that cannot be set
independently once there is one flag. They collapse into one
modularEmbeddingEnabled, which drives the setting and the token feature.
The two guest AppearanceStep tests kept modular enabled via the simple
option anyway, so dropping the guest option leaves their behaviour (dim
state follows the static terms) unchanged.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011eWbw5TiosJfLk9uLwKLgH
* test: point the embed API tests at the merged flag
check-embedding-enabled now gates on enable-embedding-modular, but this
namespace still set enable-embedding-static at four sites, so the whole
embed API suite 400'd with "Embedding is not enabled." in every driver
job. Its siblings were all repointed; this file was missed.
The merged getter's legacy fallback does not cover it: it only fires
when get-raw-value-source is :default, and that checks the db value
first, so any earlier test writing the merged flag kills the fallback
for the rest of the JVM.
* fix: add modular embedding to the embed_share enum
The merged setter emits modular_embedding_enabled/_disabled, which was
not in the embed_share iglu enum. The comment called that acceptable
because Snowplow is being retired, but public-sharing-embed-flow's
afterEach asserts zero bad rows, so it failed CI.
Adds schema 1-0-3 with the two values, the same way simple_embedding_*
was added in 1-0-2, and bumps the version the tracker sends. 1-0-3 also
drops the stale self.version 1-0-1 inherited from 1-0-2.
* test: add the merged terms setting to the serdes baseline
show-modular-embed-terms is :export? true, so it appears in a fresh
export but not in the checked-in baseline, failing
baseline-comparison-test on every app-db and the EE backend suite.
enable-embedding-modular needs no entry -- it is :export? false.
* fix: track the modular embedding toggle as a simple_event
embed_share's `event` is a closed enum, so modular_embedding_* needed a
new schema version. Cutting 1-0-3 was a worse trade than it looked:
schema->version holds one entry per event type and the four legacy
toggles all fire embed_share, so pointing that entry at an unpublished
1-0-3 makes the URI unresolvable and drops every embed_share event in
prod, not just the modular ones.
simple_event takes a free-form event string, needs no schema at all, and
track-event! already fans out to Metaplow, which is where embedding
analytics are headed anyway. This is what EMB-2257 specified. The context
the enum-typed event carried moves into event_detail, the way the SDK
bundle already sends it.
Reverts the 1-0-3 schema added in a5cc6db545d.
* refactor: name the legalese gate for what it decides
The flag reads as show/do-not-show the legalese, so the toggle guard and
the fallback test labels say that, and the labels spell out that
unaccepted is true and accepted is false.
* test: enable preview embed tests through the merged flag
The local helper turned on enable-embedding-static, which only reaches
the endpoint guard through the legacy fallback. An earlier test in the
namespace leaves enable-embedding-modular explicitly set -- with-temporary-setting-values
captures through the getter and restores through the setter -- so the
fallback is gone by the time these run and they 400.
* chore: budget the vector-less kondo ignores this branch adds
The embedding settings sync tests read deprecated settings on purpose,
and use the vector-less ignore form at ten new call sites. Master's
ratchet schema has no :all entry, so seed one at the actual count.
* test: justify the vector-less kondo ignores in the settings tests
Master's kondo-ratchet-test now requires an explanatory comment on any
ignore whose linter is outside :comment-exempt, and :all is not in that
set. Each of the ten sites says why it reads a deprecated setting.
* test: use the map form for the deprecated-var ignores
The ten ignores in the settings tests were the vector-less
#_:clj-kondo/ignore form, which suppresses every linter on the next
form. Naming the rule means the suppression says what it suppresses.
:all drops out of the ratchets entirely (that budget was all this
branch's) and :deprecated-var goes 86 -> 96. Numbers regenerated on
this branch with ./bin/mage fix-kondo-ratchets --seed :deprecated-var.
* test: accept the merged terms in the get-code spec
show-modular-embed-terms falls back to the legacy sdk and simple
settings until an admin accepts it, so clearing only the simple one
left the terms still owed and the Agree CTA still reachable.
* refactor: drop the replaced embedding settings from the types
enable-embedding-sdk, -simple and -static and the two legacy terms
settings have no frontend reader left now that everything goes through
enable-embedding-modular and show-modular-embed-terms.
---------
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix: point the merged-in EMB-2309 test at the modular flag
Master's new test used simpleEmbeddingEnabled, which the three-way
embedding flag consolidation renamed to modularEmbeddingEnabled. The
merge took both sides cleanly, so nothing conflicted.
* feat: add the embedding hub's Authentication tab (#79509)
* feat: add the embedding hub's Authentication tab
A JWT-only view of the authentication settings. Admin's authentication page
does not change -- this is a second view onto the same settings, so the admin
JWT form is mounted as-is rather than forked, which brings user provisioning
along with it.
Below it, a banner links out to the other SSO methods. After the admin
embedding section goes, this is the hub's only route to SAML, OIDC, Google
and LDAP, so the link is functional rather than decoration.
Below the paywall the tab shows an upsell. `UpsellSSO` could not be reused:
it is the narrow sidebar card from admin authentication and hardcodes a
242px max width, so it renders squished as a page-level upsell. The new one
follows the shape the shipped embedding upsells already use, which is also
why `UpsellCardContent` gains an `image-panel` variant -- a single bordered,
tinted panel, for artwork that carries its own framing or none.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01QHCSv6oQRLUUGo34V1eEDp
* fix: reword the upsell trial copy
Design's wording for the shared upsell card: "Get a 14 day trial of this
and other Pro features." The theme upsell spec asserted the old string,
so its trial-copy test could not pass and its negative assertion had
quietly stopped matching anything.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BdJ7UHzej4V16G2W6AP3hQ
* feat: give the Authentication tab its three states
The tab showed the JWT form unconditionally. Per the design and
Alessio's answers it now has three: the form once JWT is configured, a
card pointing at admin when only SAML is, and an empty state offering
to set JWT up when neither is.
Only JWT and SAML count as configured SSO, matching has-configured-sso?
on the backend. They are the only methods an embed can drive --
/auth/sso resolves to one or the other, and the SDK ships a jwt and a
saml auth path and nothing else. OIDC, Google and LDAP sign people into
Metabase itself and stay linked out.
Configure JWT goes to the guide's SSO wizard rather than dropping the
admin into a bare form: it is the flow Get started already uses and it
covers the endpoint and the test too. It is mounted under this tab, so
the wizard's back link names Authentication and finishing it returns
here.
The page also waits on the settings query the JWT form gates on, so the
banner no longer renders beside the form's spinner, and the banner is
dropped from the SAML state where the card already links to the same
page.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BdJ7UHzej4V16G2W6AP3hQ
* refactor: show the plain JWT form when nothing is configured
The tab offered an empty state that sent the admin into the setup
guide's stepped SSO wizard, mounted under this tab. Alessio's call:
show the standard JWT page here and leave the stepped flow to the setup
guide, which already owns it -- he is also redesigning the JWT page, so
a bespoke entry point here would be short-lived.
The SAML-only state is unchanged: it still shows its card and its link
to admin, because JWT cannot be configured from this tab in that case.
Removes the empty-state card, the nested authentication/sso-setup route,
its URL helper, and the back-link label prop that only existed to name
this tab from inside the wizard.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BdJ7UHzej4V16G2W6AP3hQ
* fix: drop the JWT form's own heading in the hub
The hub's Authentication page already renders a page title, and mounting
the admin JWT form inside it added a second one -- two h1s stacked, the
lower reading just "JWT", which the design does not have.
The form now takes an optional `title`, defaulting to the same "JWT" it
always used, so the admin route at settingsRoutes.tsx renders unchanged.
The hub passes `null`; SettingsPageWrapper already skips the heading box
when the title is falsy.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BdJ7UHzej4V16G2W6AP3hQ
* refactor: give the hub its own upsells module
data-studio and monitor each keep their upsell pages in an `upsells/`
directory with a BaseUpsellPage that assembles DottedBackground,
LineDecorator and UpsellCardContent. The hub had no such module, so its
SSO upsell hand-assembled the same four imports from admin/upsells.
Mirrors that structure: BaseUpsellPage holds the assembly, and
AuthenticationUpsellPage is left with the token gate and the copy. The
card is aligned to the page heading the hub renders above it, where the
other two centre theirs.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BdJ7UHzej4V16G2W6AP3hQ
* refactor: trim the authentication page's comments
Cuts the JWT/SAML rationale down to the fact that matters -- embedding
supports only those two -- and the loading note to why a second
subscriber is not a second request.
Drops `?? false` on the `jwt-configured` read. It is optional in
metabase-types/api/settings.ts where its SAML peer is not, but the only
use site treats undefined and false alike, and nowhere else in the
codebase coalesces it.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BdJ7UHzej4V16G2W6AP3hQ
* refactor: drop the image-panel upsell variant
The variant existed so the SSO illustration could sit in a single
bordered panel rather than the nested double frame `image-card` draws.
That is a change to a component every upsell in the product renders, for
one card, so it waits until design confirms the frame is wrong.
The SSO upsell falls back to `image-card` in the meantime.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BdJ7UHzej4V16G2W6AP3hQ
* chore: drop the unused embeddingHubAppearance url
Nothing calls it and `/embedding/appearance` has no route. It belongs to
the Appearance tab, which lands with EMB-1532.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BdJ7UHzej4V16G2W6AP3hQ
* refactor: stop parameterising the SSO upsell's source
One call site, one value. The data-studio upsell pages hardcode their
campaign and location the same way.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BdJ7UHzej4V16G2W6AP3hQ
* test: cover the Authentication tab end to end
Three cases the unit tests cannot reach: that the mounted admin JWT form
actually writes its settings from the hub, that a SAML-only instance gets
the Admin card instead, and that an unlicensed one gets the upsell with
nothing configurable under it.
The heading assertion is here rather than in a unit test because the
duplicate title only shows once the real form renders inside the page.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BdJ7UHzej4V16G2W6AP3hQ
* test: tighten the Authentication tab's e2e assertions
Matches the JWT field by its full label rather than a loose regex. The
asterisk is part of it: Mantine renders the required marker inside the
label element, and getByLabelText matches the label's textContent, which
includes aria-hidden nodes.
Also asserts the upsell's own copy on an unlicensed instance, rather than
only that the form is absent.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BdJ7UHzej4V16G2W6AP3hQ
* fix: lazy-load the JWT form in the hub through the route slot
#80268 (merged from master) moved SettingsSAMLForm/JWTForm/OIDCForm from
eager PLUGIN_AUTH_PROVIDERS components to lazy route slots, dropping the
eager JWT assignment this page's direct render relied on. Loading the same
route slot with React.lazy keeps the hub on the auth-jwt chunk #80268 split
off instead of re-adding an eager import.
* fix: add a background wash to the SSO upsell illustration
Matches the illustration background the Localization and Tenants
upsells use, so the four embedding hub upsell tabs read consistently.
* test: trim duplicate copy assertions from the Authentication e2e spec
Card copy and the SAML-configured link's href are already unit-tested
next to the component. What is left is what needs a real browser --
confirming the upsell (or the SAML card) renders in place of the JWT
form.
---------
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix: enable modular embedding in new param-values test
Master added card-param-values-native-card-without-parameters-test
enabling only enable-embedding-static. This branch consolidated the
three embedding method settings into one flag, so that setting alone
no longer enables the static embed endpoint and the request 400s with
"Embedding is not enabled". Use the file's own
with-embedding-enabled-and-new-secret-key! macro, as every other test
here does.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016amK37aKSDk7sgJsT3AVJQ
* fix: carry the design scale renames onto the stack
GDGT-2486 (#80481) renamed the spacing and radius scales in place,
mechanically remapping every call site that already existed on master.
These files were authored on the stack before that landed, so their
tokens are stale -- still-valid names that now resolve to different
sizes.
Spacing xs->xxs, md->lg, lg->xl, xl->xxl; radius sm->xs, in both the
component props and the CSS module custom properties. Spacing sm is
unchanged at 8px.
Verified against the mapping #80481 applied across its own 1339 files.
CI cannot catch this, so it is a deliberate pass rather than a fix for
a failing check.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01YFXLWvx9oSucyyp5Ezu3Hv
* feat: add the embedding hub's Permissions tab (#79510)
* feat: add the embedding hub's Permissions tab
The admin permissions editor, mounted a second time under the hub. Admin
permissions does not change, and the tab shows whatever the admin editor
shows -- five tabs when tenants is on -- rather than a narrowed copy, so
there is no forked component to keep in sync.
Making it mountable elsewhere needs a base path the URL builders can read.
That is a module-level value rather than a React context, because
`getDatabaseFocusPermissionsUrl` and `getGroupFocusPermissionsUrl` are
called from redux selectors and thunks, which have no component to read a
context from. Only one editor renders at a time, and the write happens
during render, so the first child render already builds correct URLs; admin
wraps its own routes too, so the value is restored whenever admin's editor
mounts after the hub's.
Three navigations inside the editor built their target from a literal
/admin/permissions rather than that base path, so picking a collection or
switching between the databases and groups views threw a hub user into
admin. They go through the base path now, like every other permission URL.
The tab carries the hub's page title, which is load-bearing: without it the
editor's tab row starts at the top of the area, where "Permissions help"
sits under the app switcher.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01QHCSv6oQRLUUGo34V1eEDp
* fix: restore the permissions base path from admin's own routes
The hub's Permissions tab set a module-level base path during render and
relied on its unmount effect alone to put admin's back. Navigating from the
tab to /admin/permissions runs admin's render before that passive cleanup,
so admin's breadcrumb selectors could build hub links on their first pass.
Admin now wraps its permissions routes in `PermissionsBasePath` as well, so
the default is set during admin's own render. That is what the tab's commit
message already claimed.
Also drops the tab's explicit `<Outlet />`: `PermissionsBasePath` defaults
its children to one, and adds the spec the component never had -- the
set-during-render contract is what the admin wrapper rests on.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EMyCTcwtcQsGggkCL9QmGp
* fix: make the embedding hub's Permissions tab full-width
The 800px content cap and AreaContent's own padding were both still
applying, so the permissions editor never reached the design's full
width. Adds a fullWidth escape hatch to AreaContent and wires it
through for the permissions route only.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UFKUvQLEMmyQ43m5VPtLMa
* fix: make the permissions editor's selection color configurable
Admin's purple (navbar-admin*, protected from whitelabeling) was
hardcoded into AdminTreeNode's selected row and EntityNameLink,
leaking into the embedding hub's blue-branded mount of the same
editor. Both now read from a context, defaulting to admin's purple;
the hub overrides it via PermissionsBasePath's new selectionColors
prop.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UFKUvQLEMmyQ43m5VPtLMa
* fix: derive every permissions accent color from one value
Three separate tokens (background_surface-brand-strong/-subtle, plus
navbar-admin's own triplet) made the override fragile, and missed the
"You've made changes" bar entirely -- it stayed on accent7 regardless.
Collapses the context to a single ColorName; every consumer (tree
selection, entity link, edit bar) derives its own shade from it via
color()/alpha() instead of picking a second unrelated token. EditBar's
admin boolean becomes an accentColor prop so permissions can plug into
the same lever.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UFKUvQLEMmyQ43m5VPtLMa
* fix: move the permissions accent color to CSS Modules, off fabricated tokens
The prior fix introduced custom CSS variables (--permissions-tree-node-*)
that read as new design-system colors, and extended two emotion files
that are deprecated in favor of CSS Modules. Replaces both:
- The context now carries a plain isHub boolean (utils/is-hub.tsx)
instead of a color value, mirroring base-path.ts's own module-singleton
shape where that pattern is actually needed (redux selectors/thunks
with no component tree) -- isHub's consumers are all components, so it
stays a React context.
- AdminTreeNode moves out of FilterableTree.styled.tsx into a plain
component + AdminTreeNode.module.css, selecting between colors the
design system already owns (navbar-admin/accent7 vs core-brand/brand)
rather than a fabricated bridge variable. Selectors use `li.<class>` --
a bare class ties in specificity with TreeNodeRoot's own emotion rule
and the winner would depend on injection order.
- EntityNameLink and EditBar keep their original, byte-identical emotion
styling for admin; the hub override is a plain inline style / an
existing `admin` boolean, never a new token.
- EditBar itself is migrated off emotion entirely (EditBar.module.css),
since it was being touched anyway and the pattern was already
deprecated.
- EmbeddingHubLayout's full-width routing moves from an inline pathname
check into a `fullWidth` field on each tab's config, so adding another
full-width tab is a one-line change.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UFKUvQLEMmyQ43m5VPtLMa
* test: cover navigation through every level of the hub's permissions editor
Clicks through Data (Groups and Databases views), Collections, and
Application, drilling one level into each, then asserts the URL and nav
never leave the embedding hub.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UFKUvQLEMmyQ43m5VPtLMa
* refactor: rename permissions is-hub to isEmbeddingHub and rework tree colors
Full name makes the flag's purpose explicit at call sites. Selection
colors move off ad-hoc emotion overrides onto AdminTreeNode.module.css,
using the data-css-specificity-hack convention and core-white instead
of mantine's color token.
* fix: route tenant collection navigation through the hub's base path
Both pages hardcoded navigate("/admin/permissions/...") in their
item-select handler, so selecting a shared or tenant collection while
inside the embedding hub kicked the user out into full admin. Use
getPermissionsBasePath() like every other permissions sub-page, and
restore the e2e test's item-click coverage for both tabs.
* test: assert permissions editor path by prefix, not substring
cy.url().should("include", ...) matches anywhere in the URL. Anchor on
pathname with a leading-^ regex instead, which also makes the paired
not.include("/admin/permissions") checks redundant -- an anchored
/embedding/permissions match already rules that out.
* refactor: simplify permissions color override and drop redundant comment
Use the raw CSS var string directly in PermissionsTable's inline style
instead of the color() helper -- no compile-time benefit was needed here
that outweighs the extra import. Also trims a comment on
PermissionsBasePath's isEmbeddingHub prop that only restated its name.
* refactor: derive the hub color switch from the base path, drop the context
isEmbeddingHub tracked its own state alongside base-path.ts's, always set
together by the same caller -- a second variable with no way to drift
independently. Collapse it into a plain derived check instead of a
context/provider pair.
Also drops PermissionsBasePath's now-redundant admin wrapper: reaching
admin permissions with the hub's base path still set requires a hard
navigation (typed URL, or this app's back button), which reinitializes
the module and needs no explicit reset. Only the hub's own mount/unmount
still needs to touch the singleton.
* test: cover the new table-under-database permissions route
getAdminPermissionsRoutes() gained a route for tables directly under a
database, without a schema segment.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01W8T4ZvTbdRbF85B98maGNh
* fix: carry the design scale renames onto the stack
GDGT-2486 (#80481) renamed the spacing scale in place, mechanically
remapping every call site that already existed on master. This page was
authored on the stack before that landed, so its token is stale -- a
still-valid name that now resolves to a different size.
Spacing md->lg.
Verified against the mapping #80481 applied across its own 1339 files.
CI cannot catch this, so it is a deliberate pass rather than a fix for
a failing check.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01YFXLWvx9oSucyyp5Ezu3Hv
* fix: align permissions chrome with the hub app switcher
The hub places its app switcher 3.5rem from the right edge and its page
title 2.5rem from the left, but the permissions toolbar's help button
ended 1.75rem from the right and the save banner sat at 2rem on both
sides, so neither lined up on the hub's full-width permissions page.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01NoNBWFMfL68mPmofjNq66x
---------
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
* feat: add the embedding hub's Tenancy tab (#79511)
* feat: restyle the tenants upsell for the embedding hub
New copy, a new illustration and a left-aligned card, from the embedding
hub's design. `UpsellTenants` is shared: admin People renders the same card
above its people list, so this changes that surface too. That is intended --
the two should not diverge -- and it is split out here rather than buried in
the Tenancy tab commit, which is what actually needs it.
The old artwork had this card as its only reference, so it goes with it.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EMyCTcwtcQsGggkCL9QmGp
* feat: add the embedding hub's Tenancy tab
A second view of the tenant surfaces. Tenants live under admin People and
stay there, so the route fragment is mounted a second time rather than
copied. What that costs is not the routes -- it is the roughly thirty
hardcoded /admin/people/tenants strings inside them, which now come from a
`useTenantUrls` context that admin and the hub each provide.
A context rather than the module-level value the Permissions tab uses: these
builders are only ever called from components, while
`getGroupFocusPermissionsUrl` is also called from redux selectors and thunks
that have no component to read a context from.
The tab renders conditionally rather than reusing `createTenantsRouteGuard`:
that guard redirects to /admin/people when tenants are licensed but
`use-tenants` is still off, which is exactly the state a hub user reaches by
clicking a tab that is always visible. So: no licence shows the upsell,
licensed-but-off shows the enable card, both on mounts the fragment.
The enable card opens the user-strategy modal in place rather than routing to
it, because that modal route hangs off the tenants listing, which this branch
of the tab does not render -- the URL would match and nothing would mount.
Two width interactions look like the hub's fault and are not: the listing
centres itself with an auto inline margin, which beats align-items on a flex
item and collapses it, so it gets a child with an explicit width; and its
empty state capped at 700px, narrower than every sibling tab, so the cap is
now the hub's column width.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EMyCTcwtcQsGggkCL9QmGp
* fix: remove tenant/user URL helpers left dead by createTenantUrls
useTenantUrls and useUserUrls replaced every caller of these; nothing
references them any more.
* fix: fix tenancy page UTM campaign, dead route fallback, and a typo
The docs link used a different UTM campaign than the rest of the hub
(embedding_hub vs embedding-hub), so its clicks were not attributed
with the others. The routes.tsx fallback for PLUGIN_TENANTS.tenantsRoutes
could never render, since EmbeddingHubTenancyPage only mounts an Outlet
once tenants are licensed and enabled. Also fixes a missing "you" in the
enable-tenancy copy, and tags the new unlicensed e2e block @EE to match
its siblings in the file.
* refactor: trim comments to their one-line why
No behavior change; shortens comments that had drifted into restating
the code or arguing a design decision instead of stating the reason.
* fix: add breathing room around the tenants upsell diagram
The diagram frame filled the full width of its bordered box with no
margin, unlike the design. Shrinks the frame ~15% and recenters it
within the same canvas so the artwork carries its own padding, the
same way the shape geometry already did vertically.
* fix: revert tenancy oss e2e block to match its true siblings
An earlier commit tagged this block @EE to match the file's older
unlicensed-SSO convention, but the tab's actual siblings (appearance,
localization, nav) all use a plain "oss" describe with no tag. Matches
those instead.
* test: enable tenancy through the UI instead of the API in e2e
The test only asserted the tenant tabs appear once use-tenants is
true, setting it via H.updateSetting. Drives the enable card and the
Pick a user strategy modal instead, so at least one path through the
actual UI is exercised.
* refactor: rebuild tenant URLs as a module singleton, not a context
createTenantUrls returned a whole object of URL builders bound to a
basePath threaded through React context, so tracing what a given call
produces meant following the provider tree. Every real consumer of
these URLs is a component, same as the permissions editor's, and that
one already solves this with a module-level base path plus plain
exported functions (admin/permissions/utils/base-path.ts). Matches
that shape here: setTenantsBasePath/getTenantsBasePath/
resetTenantsBasePath, called imperatively by whichever host mounts
the tenant routes.
This also removes the two describe/hook layers (TenantUrlsProvider,
useUserUrls) that existed only to have somewhere to call a hook from.
* refactor: keep tenant/user URL functions at their original names
newTenant, editTenant, editUser, deactivateUser, etc. now read the
shared base path directly instead of being deleted and rebuilt behind
a hook -- same signatures every other call site already expects.
* refactor: revert tenant-aware components to plain function calls
These only became components or gained a hook call because a hook
needed a component to run in. Reading the new singleton is a plain
import, so UnsubscribeUserMenuItem goes back to a bare element, and
ReactivateUserButton/TenantsListingEmptyState/
TenantsSummaryOnboardingStep/EditUserStrategySettingsButton go back
to their original arrow-function bodies.
EmbeddingHubTenancyPage sets the base path itself now, the same way
EmbeddingHubPermissionsBasePath does for the permissions editor.
* test: assert on real content after enabling tenancy in e2e
Replaces the toast assertion (flaky) with the tenants listing's own
empty-state copy, and adds a check on the OSS upsell's description
alongside its title, so the test confirms the actual tenants-specific
content rendered, not just any upsell.
* refactor: consolidate tenant URL builders
Reuse the permissions editor's existing setPermissionsBasePath/
getPermissionsBasePath instead of a second, parallel permissionsPath
field on the tenant URL module. Use namespace imports (Urls/TenantUrls)
consistently instead of importing individual functions, matching the
convention already used throughout the urls files. Inline the two
single-use tenantGroupUrl/tenantPeopleUrl helpers at their call sites
now that getTenantsBasePath is exported directly. Preserve the
declaration order of the pre-existing tenant URL functions and the
Urls import alias rather than reshuffling them.
* fix: keep tenants upsell centered in admin
The embedding hub's restyle of UpsellTenants (left-aligned, no
horizontal padding) was applied unconditionally, so admin People's
existing render of the same card lost its centered layout. Add an
align prop, defaulting to admin's original centered layout, and have
the hub opt into the left-aligned one.
* refactor: rename tenant-urls to base-path, matching its sibling
Drops the tautological spec — the round-trip it tested has no
equivalent for the permissions editor's base-path.ts, and the real
behavior is covered by EmbeddingHubTenancyPage's own spec and e2e.
Aligns file name, accessor order, and comment placement to that
sibling, and un-exports ADMIN_TENANTS_BASE_PATH now nothing outside
the module reads it.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0146A1MwgsMWTE4xdRyfdLpY
* refactor: use named imports for getTenantsBasePath
The Urls/TenantUrls namespace alias is reserved for actual urls.ts
files; common/tenants isn't one.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0146A1MwgsMWTE4xdRyfdLpY
* fix: rename UpsellTenants's location prop to source
Avoids confusion with router location; matches
UpsellEmbeddingTheme/UpsellBigCard/UpsellPill's existing convention
of taking a source prop and mapping it to the tracking location
internally.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0146A1MwgsMWTE4xdRyfdLpY
* refactor: simplify EmbeddingHubTenancyPage from self-review
Reuses SettingsPageWrapper instead of a hand-rolled Stack/Title shell,
matching the Authentication and Security tabs. Renames the local
`root` to `tenancyBasePath`, marks the docs-link icon aria-hidden so
its accessible name is just "Documentation", and simplifies the
route-guard comment. Also asserts on the upsell's description text,
not just its title.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0146A1MwgsMWTE4xdRyfdLpY
* fix: match tab boundaries in TenancyTabs, not raw string prefixes
pathname.startsWith(tab.to) has no segment boundary, so a tenant
whose id starts with "groups" or "people" (:tenantId is a free-form
path param) would light up the wrong tab. Reuses EmbeddingHubLayout's
isUnder, which already solves this for the hub's own top-level tabs.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0146A1MwgsMWTE4xdRyfdLpY
* fix: extend timeout for the post-apply tenants-listing assertion
Enabling tenancy through the modal chains two network round trips
before the empty state can render: the use-tenants setting update's
own refetch, then the tenants list query once the listing mounts.
That's tight against Cypress's 4s default under CI load — reproduced
twice in a row in CI, not flaky-random.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0146A1MwgsMWTE4xdRyfdLpY
* fix: match the tenants-listing empty state text with a regex
The prior timeout bump was the wrong diagnosis: the empty state's
copy is interpolated with three inline anchors (jt`...`), so no
single DOM node's text content ever equals the plain string exactly
-- Cypress's default string matcher requires a whole-node match, not
a substring. The CI screenshot showed the text rendered correctly;
the assertion just couldn't find it. A regex matches the substring
instead, which is why it kept failing even at 10s.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0146A1MwgsMWTE4xdRyfdLpY
* fix: align the Tenancy tab's title-to-card gap with its siblings
Dropped the gap="xl" override on SettingsPageWrapper so it uses the
same default gap as the other embedding hub tabs, and added the same
background wash the other upsell illustrations use.
* test: trim duplicate copy assertions from the Tenancy oss e2e test
Card copy is already unit-tested next to the component. What is left is
what needs a real browser -- confirming the upsell renders in place of
the real tenants surfaces.
* fix: keep the reset-password link under the tenants path
UserSuccessModal is mounted both by admin People and by the tenants
routes the hub's Tenancy tab renders, so the hardcoded /admin/people
link sent a hub user into admin. Urls.resetPassword already branches
on isInternalUser via getTenantsBasePath.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014H8Ma3D2dsLvaz4PkpsxTS
* test: count the new table-under-database permissions loader
* fix: carry the design scale renames onto the stack
GDGT-2486 (#80481) renamed the spacing scale in place and dropped the
shadow scale's md key, mechanically remapping every call site that
already existed on master. This page was authored on the stack before
that landed, so its tokens are stale.
Spacing md->lg, lg->xl, xl->xxl, and shadow md->sm -- the shadow scale
has no md key any more, so that card was rendering no shadow at all.
Verified against the mapping #80481 applied across its own 1339 files.
CI cannot catch this, so it is a deliberate pass rather than a fix for
a failing check.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01YFXLWvx9oSucyyp5Ezu3Hv
---------
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
* feat: add the embedding hub's Appearance tab (#79512)
* feat: add the embedding hub's Appearance tab
The embedding theme editor plus the appearance settings that show up inside
an embed, on one tab -- there is no separate Themes tab, so the standalone
nav item goes away and its upsell moves here.
Both listing and editor now take their base path from the host so their links
resolve under whichever section mounted them. The listing's own "Themes"
heading is behind `showHeading`, which admin keeps and the hub turns off:
the hub page is titled "Appearance" and puts the grid under a "Themes"
section of its own, and two headings would stack.
The two halves are technically gated on different features -- themes on
`embedding_simple`, the appearance settings on `whitelabel` -- but they
arrive together in practice, so the tab is gated as a whole rather than
building a partial state nobody will see.
`EmbeddedAppearanceSettings` is a deliberate subset: loading message, the
Metabase links toggle and the two empty-state illustrations. Logo, favicon,
login and landing illustrations, instance colours, fonts and application
name stay in admin, since they are not what an embed shows.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EMyCTcwtcQsGggkCL9QmGp
* docs: trim self-review comments in the Appearance tab
Cut the multi-paragraph JSDoc/inline comments down to their factual
core, and drop soft pointers to "the design" in favor of stating the
actual reason.
* fix: lowercase "full-app embedding" in the Appearance banner copy
The phrase sits mid-sentence, not as a heading, so it shouldn't be
capitalized like the Full-app embedding settings card title.
* fix: nest the theme editor route under appearance/theme
New and edit theme URLs were /embedding/appearance/new and
/embedding/appearance/:themeId, colliding with the tab's own path
segment convention. Nest them under a theme segment instead, and make
the editor full-width like the rest of its side-by-side layout instead
of capped to the tab's default 800px.
* fix: match the full-app appearance banner to the design spec
Was background-info with a border; the Figma spec calls for
background-brand with no border.
* fix: link the Get started theme card to the hub's Appearance tab
The Appearance tab now exists, so the card no longer needs to fall
back to the admin theme listing.
* fix: update routes spec for the appearance/theme route nesting
The spec still expected embedding/appearance/:themeId from before the
theme editor route moved under embedding/appearance/theme/:themeId.
* fix: use SettingsPageWrapper for the Appearance tab's title
Was a bespoke Stack + Title with a hardcoded 2.5rem gap, which put the
tab out of step with the other embedding hub tabs' title-to-card
spacing.
* test: count the new table-under-database permissions loader
* fix: load the hub theme editor lazily, behind its token gate
The theme editor route mounted its page eagerly via `element=` with a
static import, instead of the `lazy=` pattern every sibling route
uses. That pulled the EAJS runtime into the main bundle rather than a
lazy chunk, and skipped the `embedding_simple` token gate the
Appearance page applies before showing the editor, so a deep link to
/embedding/appearance/theme/:themeId opened it on plans that cannot
use it.
Add EmbeddingHubThemeEditorPage as a lazy-loaded wrapper that checks
the gate and redirects to the Appearance tab when it's absent, and
route :themeId through it via `lazy=` like the rest of the hub.
Co-Authored-By: Claude Sonnet 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01YT26JiAJ25NK11W8PJLymT
* fix: carry the design scale renames onto the stack
GDGT-2486 (#80481) renamed the spacing scale in place, mechanically
remapping every call site that already existed on master. This page was
authored on the stack before that landed, so its tokens are stale --
still-valid names that now resolve to different sizes.
Spacing xs->xxs, md->lg.
Verified against the mapping #80481 applied across its own 1339 files.
CI cannot catch this, so it is a deliberate pass rather than a fix for
a failing check.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01YFXLWvx9oSucyyp5Ezu3Hv
* fix: full-bleed the embedding hub's upsell pages
Review on #79513 flagged the Localization upsell's dotted background as
cropped. Root cause: BaseUpsellPage rendered inside the hub's 800px
capped column, so DottedBackground had no room to fade before its edge,
and LineDecorator's oversized bleed lines got clipped mid-fade.
Fixes it by making a gated tab's whole content area full-width (same
mechanism Permissions already uses) and moving BaseUpsellPage's title
inside the dotted field, so the field owns the whole tab instead of
sitting under a separately-rendered heading. Title and card now share
one fit-content, centered box so their edges always line up, with an
explicit 160px gap between them matching the design.
Authentication, Tenancy and Appearance each get their own hub-local
upsell page through BaseUpsellPage now, instead of reusing admin's
UpsellTenants/UpsellEmbeddingTheme -- those stay untouched, since they
still serve admin surfaces directly and get removed once EMB-1526 drops
the admin embedding routes.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015WHaGA2f7AsXGCdQmBzsHw
* fix: redirect the bare appearance/theme path back to Appearance
Visiting /embedding/appearance/theme with no theme ID matched the
theme route segment but had nothing to render, leaving a blank page
(flagged in review on #79512). Adds an index redirect back to the
Appearance tab, matching the pattern the hub root already uses.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015WHaGA2f7AsXGCdQmBzsHw
* fix: flow the full-app appearance banner's link inline
The Admin link sat in its own nowrap Group next to the sentence, so on
narrow screens the sentence wrapped but the link got pushed onto its
own line at the far edge instead of flowing with the text (flagged in
review on #79512). Renders the whole banner as one Text with the link
inline, so it wraps as a normal sentence.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015WHaGA2f7AsXGCdQmBzsHw
* test: cover the hub's theme basePath and the editor guard
Three gaps on the Appearance tab. Clicking an existing theme card was
only proven for the New theme button, so a regression sending card
clicks to the admin route would have passed. The theme editor's token
guard had no test at any level, and its route table redirects the bare
`theme` path regardless of plan, so the deep link needs a themeId to
exercise the guard at all. The branding section asserted only the
loading message, not the two illustration widgets beside it.
Also renames the "oss" block to "unlicensed": it runs on the EE jar
with no token, which is the unlicensed state, not an OSS build.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DZe6AQywXThKuntLQtLt1B
---------
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix: use the embedding db helpers in the snowplow payload
The master merge kept two t2/count calls in the modular-embedding event
while dropping the toucan2 require, so the namespace failed to compile.
* test: use the merged embedding flag in the collection browser spec
#80881 landed on master using enable-embedding-sdk, which this branch
folds into enable-embedding-modular.
* feat: add the embedding hub's Localization tab (#79513)
* feat: add the embedding hub's Localization tab
Content translation renders only from the admin embedding section today --
it is not on the admin Localization page -- so this is the one tab that is a
genuine move rather than a second view. The instance-wide localization
settings stay in admin and are not surfaced. Admin's copy goes away with the
rest of the admin embedding section, in EMB-1526.
Below the paywall it shows an upsell, which keeps the hub's tab set the same
seven on every edition. Its copy and illustration are placeholders: no Figma
frame exists yet, so the artwork is a generated placeholder captioned as one
rather than another upsell's, which would make a design that does not exist
look settled. Both are drop-in replacements.
The two specs that need the finished nav -- seven tabs in design order, and
the gem on each paid tab -- land here, because this is the first branch where
all seven exist. They get a nav describe of their own rather than riding
along in a tab's.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EMyCTcwtcQsGggkCL9QmGp
* fix: correct stale comments on the Localization tab
The page comment claimed admin already loses content translation, but
this PR only adds the hub tab -- the admin route is removed separately
by EMB-1526. The upsell comment pointed at a gitignored plan file that
no reader of the merged code can ever open.
* refactor: reuse BaseUpsellPage for the Localization upsell
Was a hand-duplicated copy of the same DottedBackground + LineDecorator
+ UpsellCardContent shape AuthenticationUpsellPage already builds via
BaseUpsellPage, which is also why its title-to-card gap didn't match
its sibling tabs. Also finalizes the description copy, which was a
placeholder.
* fix: replace the Localization upsell's placeholder illustration
Real artwork in place of the generated placeholder box.
* fix: correct backwards duplicate/link wording in Localization's doc comment
The comment described 'duplicates the admin embedding section's UI'
and 'linking to it' the wrong way round: rendering the same settings
component in two places is reuse, and what the other tabs do instead
is genuinely link out to admin.
* test: move nav-completeness and copy assertions from e2e to unit specs
Two e2e tests --reaches all seven tabs from the nav, shows every tab
with a gem on the paid ones-- duplicated coverage routes.unit.spec.tsx
and EmbeddingHubLayout.unit.spec.tsx already had (route existence,
page resolution, nav link order) or could cheaply get (the gem
indicator, now covered by a new EmbeddingHubLayout test). Dropped both
and moved the Localization tab's body copy assertion into its own
unit spec, alongside the title check it already had.
* test: cover the licensed Localization tab in e2e
The tab branches on the content_translation token feature, but only the
upsell half was covered. Add the pro case asserting the dictionary
configuration mounts, and rename the sibling block from "oss" to
"unlicensed" — it runs on the EE jar with no token, which is the
unlicensed state, not an OSS build. Matches the naming the get started
and authentication blocks already use in this file.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DZe6AQywXThKuntLQtLt1B
* fix: keep the permissions base path across hub tabs
Both the Permissions tab and the Tenancy tab declare the hub as the
permissions editor's URL host, and both reset that on unmount. React
renders the incoming tree before cleaning up the outgoing one, so
navigating between them reset a value the other had just claimed, and
the tenants listing's permissions link fell back to the admin route.
Comparing the stored value could not fix it: both hosts store the same
string. Track which host claimed last with a useId, and let only that
one release.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01QdtGmjnS93RHcNfu5mMUA7
* style: wrap the permissions base path setter signature
oxfmt requires it under the line limit.
---------
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
* feat: update the JWT signing key flow to the approved design (#81068)
* feat: add the embedding hub's Localization tab
Content translation renders only from the admin embedding section today --
it is not on the admin Localization page -- so this is the one tab that is a
genuine move rather than a second view. The instance-wide localization
settings stay in admin and are not surfaced. Admin's copy goes away with the
rest of the admin embedding section, in EMB-1526.
Below the paywall it shows an upsell, which keeps the hub's tab set the same
seven on every edition. Its copy and illustration are placeholders: no Figma
frame exists yet, so the artwork is a generated placeholder captioned as one
rather than another upsell's, which would make a design that does not exist
look settled. Both are drop-in replacements.
The two specs that need the finished nav -- seven tabs in design order, and
the gem on each paid tab -- land here, because this is the first branch where
all seven exist. They get a nav describe of their own rather than riding
along in a tab's.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EMyCTcwtcQsGggkCL9QmGp
* fix: correct stale comments on the Localization tab
The page comment claimed admin already loses content translation, but
this PR only adds the hub tab -- the admin route is removed separately
by EMB-1526. The upsell comment pointed at a gitignored plan file that
no reader of the merged code can ever open.
* refactor: reuse BaseUpsellPage for the Localization upsell
Was a hand-duplicated copy of the same DottedBackground + LineDecorator
+ UpsellCardContent shape AuthenticationUpsellPage already builds via
BaseUpsellPage, which is also why its title-to-card gap didn't match
its sibling tabs. Also finalizes the description copy, which was a
placeholder.
* fix: replace the Localization upsell's placeholder illustration
Real artwork in place of the generated placeholder box.
* fix: correct backwards duplicate/link wording in Localization's doc comment
The comment described 'duplicates the admin embedding section's UI'
and 'linking to it' the wrong way round: rendering the same settings
component in two places is reuse, and what the other tabs do instead
is genuinely link out to admin.
* test: move nav-completeness and copy assertions from e2e to unit specs
Two e2e tests --reaches all seven tabs from the nav, shows every tab
with a gem on the paid ones-- duplicated coverage routes.unit.spec.tsx
and EmbeddingHubLayout.unit.spec.tsx already had (route existence,
page resolution, nav link order) or could cheaply get (the gem
indicator, now covered by a new EmbeddingHubLayout test). Dropped both
and moved the Localization tab's body copy assertion into its own
unit spec, alongside the title check it already had.
* test: cover the licensed Localization tab in e2e
The tab branches on the content_translation token feature, but only the
upsell half was covered. Add the pro case asserting the dictionary
configuration mounts, and rename the sibling block from "oss" to
"unlicensed" — it runs on the EE jar with no token, which is the
unlicensed state, not an OSS build. Matches the naming the get started
and authentication blocks already use in this file.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DZe6AQywXThKuntLQtLt1B
* fix: keep the permissions base path across hub tabs
Both the Permissions tab and the Tenancy tab declare the hub as the
permissions editor's URL host, and both reset that on unmount. React
renders the incoming tree before cleaning up the outgoing one, so
navigating between them reset a value the other had just claimed, and
the tenants listing's permissions link fell back to the admin route.
Comparing the stored value could not fix it: both hosts store the same
string. Track which host claimed last with a useId, and let only that
one release.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01QdtGmjnS93RHcNfu5mMUA7
* style: wrap the permissions base path setter signature
oxfmt requires it under the line limit.
* feat: update the JWT signing key flow to the approved design
Replace the single setup modal with the approved flow: a read-only
generated key shown in "Create a secret key", and a confirm step before
regenerating an existing key.
- read-only key box on one line, icon-only copy button
- "Regenerate key" is now a secondary button
- regenerating asks "Delete key and generate a new one?" first, then
shows the new key in "Store your new key"
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CSPdZEZk6NgAeB9C21YWDA
* test: cover the JWT secret key create and regenerate flows
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CSPdZEZk6NgAeB9C21YWDA
* fix: always show a freshly generated JWT signing key
Reading the token from the RTK Query cache surfaced the key generated
the last time the modal was open, so a user could copy or save a key
that was about to be replaced. Hold it in local state instead.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CSPdZEZk6NgAeB9C21YWDA
* refactor: reuse ConfirmModal and CopyTextField props in FormSecretKey modals
RegenerateKeyConfirmModal duplicated ConfirmModal's title/message/
confirm/cancel layout instead of using it, and SecretKeyModal hand-rolled
the CopyButton/Tooltip/ActionIcon markup that getCopyTextFieldProps
already provides. Also aligns SecretKeyModal's body spacing, text color,
and cancel button styling with ConfirmModal's conventions.
* fix: prevent dismissing FormSecretKey modals via X, outside click, or escape
The generated key can't be shown again, and the regenerate confirmation
is destructive, so both must be dismissed through an explicit button
rather than accidentally closed.
* fix: don't autofocus the secret key modal's readonly input
Mantine's modal focus trap was landing on the readonly key field by
default, which triggers its focus border color, not any override on
the field itself. Autofocus the confirm button instead, and exclude
the input from focus entirely since it's disabled until the token
loads.
* style: drop border/padding overrides on the secret key input
Mantine's own Input[data-variant="default"] rule already sets a
neutral border color at the same specificity, so the custom border
was a coin-flip fight against the design system rather than a real
override; the default padding already accounts for the copy button's
rightSection too. Keep the CSS specificity hack for the properties
that do need overriding (radius, background, monospace font).
* test: use full-string queries for the new secret key assertions
Only the lines this diff added; existing regex queries elsewhere in
the file are left as-is.
* refactor: rename SecretKeyModal.tsx back to SetupKeyModal.tsx
Keeps the reviewer-facing diff on this branch to a plain modification
instead of a delete+add across an unrelated PR. The component itself
stays named SecretKeyModal.
* refactor: name the has-secret-key check instead of checking value directly
value is a raw secret-key string; using its truthiness inline four
times reads as cryptic. hasSecretKey names what's actually being
checked.
* test: assert the copy button writes the token to the clipboard
The old test only checked the button rendered with no visible label,
which didn't verify copying actually works.
* fix: stop naming an embed type in the shared enable CTA
Enabling embedding via SSO or Guest now flips the same toggle
(enable-embedding-modular), so wording like "enable modular
embedding"/"enable guest embeds" implied a type-scoped action that
no longer exists. Reworded the CTA sentences and toasts to generic
"embedding"; the two terms flags and their legalese stay separate
since SSO and Guest/AGPL are gated to different editions and never
shown to the same admin.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01QdRnBCJB84JkuZwLrdhsJp
* test: follow the new secret key modal flow in the JWT spec
The set-up modal's confirm button is now "Create", and regenerating
goes through a confirmation modal ("Delete key and generate a new
one?") before the store-new-key modal, so the spec's single "Done"
click no longer matched either step.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01W8T4ZvTbdRbF85B98maGNh
* test: expect the generic wording in the guest enable CTA
146acfbb6ea reworded the shared CTA to "enable embedding" when the
simple-embed feature is unavailable, which is the case for the guest
card in both the EE-without-token and OSS/Starter specs.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01W8T4ZvTbdRbF85B98maGNh
* test: use the renamed Create button in the hub's JWT spec
The hub's copy of the admin JWT form goes through the same secret key
modal, whose confirm button this branch renamed from Done to Create.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01W8T4ZvTbdRbF85B98maGNh
* fix: import useMetadataToasts from its new home
Master moved the hook out of metabase/metadata/hooks into
metabase/common/hooks; this file was the last caller left behind.
* fix: close the secret key modal when generation fails
The modal turns off its close button, escape and click-outside so an unsaved
key cannot be dismissed by accident. On a failed `random_token` request no key
ever arrives, so the confirm button stays disabled and the modal has no exit at
all. Closing from the catch keeps the guard for the case it was written for.
Reported by @lorem--ipsum on #81068.
* fix: carry the design scale renames onto the stack
GDGT-2486 (#80481) renamed the spacing and radius scales in place,
mechanically remapping every call site that already existed on master.
This modal was rewritten on the stack before that landed, so the tokens
on its new markup are stale -- still-valid names that now resolve to
different sizes.
Spacing md->lg and lg->xl on the modal, radius md->sm on the secret key
field's CSS module.
Verified against the mapping #80481 applied across its own 1339 files.
CI cannot catch this, so it is a deliberate pass rather than a fix for
a failing check.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01YFXLWvx9oSucyyp5Ezu3Hv
* fix: keep the guest CTA a whole sentence per edition
Interpolating a translated noun into a translated sentence leaves
translators with "To continue, enable %s and agree to…" and no way to
inflect the surrounding words. Give each edition its own complete
sentence instead.
The "agree to the usage conditions" branch loses its edition split
altogether: show-static-embed-terms is force-false on Pro, so
termsAccepted is always true there and the branch never matches.
Updates the three negative assertions in the OSS/starter spec that
still named the old copy, and which no longer guarded anything.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01QdtGmjnS93RHcNfu5mMUA7
* refactor: leave the modular section comment alone
The file's only change was a reworded doc comment, with no code under
it -- and the new wording invented "SSO-specific terms", which do not
exist: the SSO radio gates on show-modular-embed-terms, the Guest radio
on show-static-embed-terms.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01QdtGmjnS93RHcNfu5mMUA7
* fix: let the keyboard reach the generated secret key
tabIndex={-1} took the field out of the tab order, so the key could not
be read or selected without a mouse. It was not what kept autofocus off
the input: useFocusTrap returns the [data-autofocus] element first, and
that is already the confirm button.
Records why the modal has no escape hatches, and what a hung request
costs, next to the props that cause it.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01QdtGmjnS93RHcNfu5mMUA7
* test: cover that the secret key modals resist Escape
All three modals disable the close button, outside clicks and Escape so
a shown key cannot be lost, and nothing asserted it -- the PR asked
reviewers to check it by hand instead.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01QdtGmjnS93RHcNfu5mMUA7
* style: drop redundant rules on the secret key modals
ConfirmModal already renders its confirm button feedback-negative, and
flex-wrap: nowrap is the default for the flex container the rule itself
declares. Trims the comment above the specificity hack to what the
rules under it actually do.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01QdtGmjnS93RHcNfu5mMUA7
---------
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
* Remove the embedding admin settings and relocate its components (#81018)
* feat: delete the admin embedding route section, redirect to the hub
Deletes the /admin/embedding route subtree now that the embedding hub
replaces it, and adds redirects from every old path to its /embedding
equivalent so no bookmarked or linked URL breaks. Removes the
"Embedding" entry from the admin sidebar/paths list, and the three
route-chrome files that back the deleted section directly
(AdminEmbeddingApp, EmbeddingNav, the admin-side setup-guide page) --
none of them are imported anywhere else.
Sweeps every hardcoded /admin/embedding/... reference this forces:
the SSO wizard's troubleshooting copy, the setup-guide default return
paths, the related-settings links, and the entire e2e suite (15 spec
files). Two e2e specs that only asserted on the deleted admin sidebar
are removed; their edition-specific content checks (OSS/starter
upsell + CORS visibility) are ported into the hub's own security e2e
spec, updated to match its actual (redesigned) upsell banner. Adds
dedicated e2e coverage for the redirect map itself, including the two
backward-compat chained-redirect cases.
Moves the version-pinning card to admin General, the one piece with
no hub-tab home -- and fixes a dropped `embedding_simple` gate in the
process: the original render path required embedding_simple AND
embedding_sdk AND is-hosted?, and the first port of this card only
carried the latter two.
Part 1 of 2 for this PR -- part 2 relocates the components this
section leaves stranded under admin/, once this part isolates the
routing diff.
* refactor: relocate stranded embedding admin components
Moves the components left homeless by commit 1's admin/embedding route
deletion: the theme editor/listing cluster to embedding/themes/, and
the hub-only settings widgets to embedding/settings/.
Along the way, moving the settings widgets surfaced that shared/embedding
cannot import feature/admin (shared is the architectural foundation and
never imports up). SettingHeader, SettingsSection, and AdminSettingInput
had to come out of admin/ too, into a new shared settings-components/
module, so the widgets are self-sufficient. All ~114 consumers across
admin/, enterprise/, and transforms/ were repointed to the new module.
The rest of the old EmbeddingSettings/ cluster (EmbeddingSettings.tsx,
GuestEmbedsSettings, the EmbeddingSecuritySettings wrapper,
EmbeddingSdkSettings, NewEmbedButton, SharedCombinedEmbeddingSettings)
had zero remaining consumers once commit 1 landed, so it's deleted
rather than left as dead weight.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EMyCTcwtcQsGggkCL9QmGp
* fix: carry the theme editor's app-tier boundary override to its new path
ThemeEditor already had its own app-tier module-boundaries.mjs entry,
predating this PR: it previews the live embed through the app-tier
embedding-iframe-sdk runtime, which has real module-load side effects
(custom element registration, a window.metabaseConfig watcher) and
can't be split into a shared-tier reusable piece.
The previous commit moved the folder but left that entry's pattern
pointing at the old admin/embedding path, so it silently stopped
matching and the files fell through to the general shared/embedding
classification -- caught by a boundaries/element-types lint failure.
Updates the pattern to follow the move, no suppression needed.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EMyCTcwtcQsGggkCL9QmGp
* fix: update EmbeddingHubTenancyPage's SettingsSection import path
SettingsSection moved to metabase/settings-components in this branch's
relocation commit; this import was missed.
* fix: correct theme-editor redirect target to include the /theme segment
/admin/embedding/themes/:themeId redirected to /embedding/appearance/:themeId,
but the theme editor is actually mounted at /embedding/appearance/theme/:themeId
(routes.tsx nests it under a "theme" path segment). The old target matched no
route.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01QdRnBCJB84JkuZwLrdhsJp
* fix: update EmbeddingHubAppearancePage's SettingsSection import path
The move of SettingsSection to metabase/settings-components missed this
call site, so the Appearance tab failed to resolve the module at runtime.
* refactor: drop the unmounted InteractiveEmbeddingSettingsCard
Removing the embedding admin settings page deleted the only render site
for PLUGIN_ADMIN_SETTINGS.InteractiveEmbeddingSettingsCard, leaving the
slot written but never read. The origins widget it wrapped is still
mounted, by the hub's Security tab.
* fix: return setup wizards to the guide host they came from
The permissions and SSO wizards left the guide with a relative "..",
which resolves against the route, not the host, so a user who entered
from the embedding home page landed in the hub. Carry the host in the
existing returnToEmbeddingSetupGuide param instead, so it also survives
a reload mid-wizard.
* fix: remount a redirect whose target is itself a redirect
Without a key, React reuses the RedirectRoute instance across a
sibling redirect chain, so the first target stays frozen in state and
/admin/embedding/modular never reaches /embedding/security.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01W8T4ZvTbdRbF85B98maGNh
* fix: repoint DatabaseCachingEditor at the relocated SettingsSection
Master added this importer after the module move, so it still pointed
at metabase/admin/components/SettingsSection, which this branch
deletes.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01W8T4ZvTbdRbF85B98maGNh
* fix: redirect retired embedding paths to the hub
Six legacy redirects still pointed at /admin/embedding and
/admin/embedding/guest, which this branch turns into redirects
themselves. Chained that way the second RedirectRoute reused the
first's instance and froze on the first target, so they were made to
work by keying the shared redirect() helper -- changing behaviour for
every redirect in the app to serve six call sites in one file.
Point the six straight at /embedding/security instead and restore
router/redirect.tsx and its spec. No redirect in the tree targets
another redirect now, and the helper's own spec passes unkeyed.
Cover the whole table rather than a spec per page: admin/routes.unit.spec.tsx
gains the 14 path/target pairs read off the real route tree, plus the
no-chaining invariant. The four legacy paths that had no e2e case at all
join the redirects spec's table, which now covers all 14.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01YT26JiAJ25NK11W8PJLymT
* fix: default the theme editor base path to the hub
The editor's basePath still fell back to /admin/embedding/themes, a
route this branch deletes, while its ThemeListing sibling had already
been moved onto the hub path. Only the hub mounts the editor and it
passes basePath explicitly, so nothing was broken -- but cancel and
delete would have navigated through a redirect.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01YT26JiAJ25NK11W8PJLymT
* docs: trim the module boundary comments
The theme-editor note grew from five lines to ten while restating the
move it documents; the settings-components and embedding-hub ones
listed ticket history the code does not need. Keep the reason each
carve-out exists and drop the narrative around it.
Every line touched here is a comment this branch itself added.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01YT26JiAJ25NK11W8PJLymT
* refactor: drop the -setup suffix from the wizard paths
get-started/permissions-setup and get-started/sso-setup stutter under a
parent segment that already says setup. Rename them to
get-started/permissions and get-started/sso.
Safe to rename rather than redirect: the hub reaches master only when
embedding-hub-main merges, and that PR is still open, so no released
Metabase has ever served these URLs.
Sharing a final segment with the /embedding/permissions tab is fine --
different parent -- and EmbeddingHubLayout's "does not light up
Permissions on the permissions wizard" test covers exactly that.
The two hub routes come from an earlier PR in this stack; they are
touched here because this branch's redirects point at them.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01YT26JiAJ25NK11W8PJLymT
* test: restore the OSS and starter nav checks, fix the upsell
The deleted embedding-admin-settings-{oss,starter} specs each checked
which links their edition's sidebar offered. That sidebar is gone; the
hub shows every tab to every edition and marks the unlicensed ones with
a gem, so assert that instead, once per edition as before.
The OSS upsell test asserted a button and so could never pass. OSS
leaves PLUGIN_ADMIN_SETTINGS.useUpsellFlow returning no trigger, which
sends UpsellCta down its url branch to an ExternalLink; paid builds wire
the trigger up and get an UnstyledButton. Assert the link.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01YT26JiAJ25NK11W8PJLymT
* test: assert against the hub layout, not admin
The embedding pages now render inside the embedding hub, so the specs
look for embedding-hub-main and the "Embedding hub" nav instead of
admin-layout-content and admin-layout-sidebar.
* test: visit the content localization page
The content translation settings moved off /embedding/security onto
their own /embedding/localization page.
* test: follow the theme editor to its hub route
The editor moved from /admin/embedding/themes/:id to
/embedding/appearance/theme/:id, and it renders full width rather than
inside the admin sidebar layout.
* test: start New embed from the hub nav
The button now lives in the hub nav rather than in the SDK and guest
embed setting cards, so the specs share one clickNewEmbedButton helper.
The guest-only entry point is gone with those cards, and with it the
assertion that it forces guest auth.
* test: drop the modular embedding settings spec
It asserted on the Tenants link inside the settings card, which the
embedding admin settings removal took with it.
* fix: drop the modular embedding gate on version pinning
Version pinning is about the SDK package, so the SDK feature and a
hosted instance are the only conditions it needs.
* docs: say why theme-editor is app tier
Name the import chain that forces it, and why settings-components is
separate from the data-only settings module.
* docs: correct the no-literal-metabase-strings reasons
The strings are exempt because only admins see them, not because they
are legal copy.
* docs: note where the theme upsell coverage went
The component no longer gates itself, so its upsell tests live in
theme-upsell.cy.spec.ts.
* fix: import AdminSettingInput from its relocated path
Master added the fast-mode input to this section after the relocation
commit was written, so the new import still pointed at the old widgets
directory this branch empties.
* fix: redirect the setup guide straight to the hub
/admin/embedding/setup-guide pointed at /embedding, which is itself a
redirect to /embedding/get-started. Every other retired path lands on a
real page, so send this one there directly too.
* refactor: require basePath on the theme editor and listing
Both defaulted basePath for a second host that no longer exists, and the
move to the hub left each default pointing at the other one's path: the
editor would have returned to /embedding/appearance/theme and the listing
would have created themes at /embedding/appearance/new, neither of which
routes anywhere. The hub always passes basePath, so drop the defaults.
* test: assert every hub tab from one configuration
The per-edition nav checks only asserted the four gated tabs, so a gem
appearing on an ungated one went unnoticed. List all seven with their
expected gem state instead, shared by the OSS and Starter blocks.
* test: cut the admin redirect spec to two browser cases
admin/routes.unit.spec.tsx already asserts the whole retired-path table
off the route tree. Keep only what reading the tree cannot show -- that a
redirect fires in a browser and carries its params -- instead of a restore
per path.
* docs: state where the New embed button lives now
The comment contrasted the hub nav against settings cards that no longer
exist, which goes stale as soon as nobody remembers them.
* fix: point the theme editor page at the relocated module
This branch moves the theme editor components from
metabase/admin/embedding/components/ThemeEditor to
metabase/embedding/themes/components/ThemeEditor. The hub's lazy
theme editor page still imported the old path, which the rebase
onto this branch left broken since git has no way to track an
import string across a rename.
Co-Authored-By: Claude Sonnet 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01YT26JiAJ25NK11W8PJLymT
* fix: drop the stray unused ThemeEditor import in routes.tsx
The ThemeEditor relocation commit's routes.tsx hunk updated a
top-level `import { EmbeddingThemeEditorApp } from ...` line to its
new path. That line had already been deleted when the theme editor
route switched to a `lazy=` loader, so rebasing the relocation
commit onto that fix reintroduced it as an unused import, since git
applied the patch by fuzzy context match rather than conflicting.
Co-Authored-By: Claude Sonnet 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01YT26JiAJ25NK11W8PJLymT
* fix: align embedding e2e specs with the hub pages
Repointing these specs from the admin embedding pages to the hub was a
URL and test-id swap, so assertions that the hub renders differently
were left behind:
- The Get started card reads "Configure data permissions and tenants";
only the permissions page kept "and enable tenants".
- The Security page's upsell passes an onClick, so UpsellCta renders a
button rather than an anchor and the upgrade URL is not in the DOM.
- Published guest embeds render only when embeds exist, so unpublishing
removes the section instead of showing empty states.
- The tenants links are gone: tenancy off opens the user-strategy modal
in place and tenancy on renders a tab to the hub's own tenancy page,
both covered by EmbeddingHubTenancyPage.unit.spec.tsx.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016amK37aKSDk7sgJsT3AVJQ
* fix: match the hub's checklist card DOM in setup-guide e2e
The migration to the hub kept the admin card's DOM contract. A hub card
is an anchor, a button or a div depending on whether the step links,
opens a modal or is locked, so closest("button") only matched a third of
them; its done badge is a check icon labelled "Step N complete" with no
"Done" text; a locked card carries aria-disabled and a hover tooltip
rather than a lock icon; and the SSO card is titled "Set up SSO".
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016amK37aKSDk7sgJsT3AVJQ
* fix: drop the stale settings wait in content translation e2e
The dictionary upload helper waited on GET /api/setting after visiting
the localization page. The admin page fetched it on load and the hub's
Localization tab does not, so the wait timed out and took the five
upload-form tests with it. The findByTestId assertion that follows
already retries.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016amK37aKSDk7sgJsT3AVJQ
* test: assert the OSS upsell renders a link, not a button
The hub's Security page takes its click handler from
PLUGIN_ADMIN_SETTINGS.useUpsellFlow, whose OSS stub returns
triggerUpsellFlow: undefined. UpsellBanner then falls back to
useUpgradeAction, which also returns onClick: undefined off a hosted
instance, so UpsellCta matches its url branch and renders an anchor.
An OSS bundle can therefore never render this CTA as a button. The
assertion passed locally only because build-hot bundles the enterprise
plugins regardless of MB_EDITION; CI's oss-subset job builds the real
OSS bundle and got a link.
Restores the href assertion the migration dropped, with the hub's
campaign and location.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01YFXLWvx9oSucyyp5Ezu3Hv
* test: await the throttled recents before asserting the embed
The spec aliases the throttled /api/activity/recents response as
@getRecents but never waits on it. That worked while visitNewEmbedPage
clicked a card on /admin/embedding, which only rendered once that page
had loaded its settings -- several seconds in which the throttled
response made progress.
The hub's "New embed" button is in the nav and is clickable as soon as
the layout mounts, so the wizard now opens immediately and the response
is still in flight when the assertions run. CI shows neither dashboard
title present, not the example dashboard, so this is the wizard still
resolving rather than a fallback.
Awaiting the alias restores what the test name already promises.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01YFXLWvx9oSucyyp5Ezu3Hv
* fix: order the embedding imports as import/order expects
eslint import/order wants metabase/embedding/themes before
metabase/embedding-hub/upsells. Fixes the fe-lint CI failure.
* fix: restore the Box import on the JWT form
The master merge's import resolution dropped Box while the group-schema
block still renders it.
* fix: repoint DomainsSettingsPage at the relocated modules
Master added this page against the pre-move paths for SettingsSection,
SettingHeader and isSettingSetFromEnvVar.
* test: restore the csp-img and custom-viz mock settings
setup() indexes both off generalSettings; master's copy of the mock
dropped them.
* style: order the relocated settings-components imports
import/order after repointing them at the new module.
* fix: drop the settings left behind on the general page
csp-img-enabled and custom-viz-enabled render on the domains page now,
so their reads here were dead, along with the imports they needed.
* style: drop the blank line eslint left in the import block
* refactor: query the hub content by its main landmark
The hub's content area is the main content of the page, so mark it up as
one instead of reaching for it by test id. Drops embedding-hub-main in
favour of component="main" and repoints all 45 call sites.
Kept on the outer Box rather than AreaContent, which the monitor layout
also renders -- tagging it there would give that page a main landmark
too. MonitorContent still uses monitor-main on its equivalent Box.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0198kPp8K99BRJ15z73m5MY4
* test: assert the checklist step number explicitly
The label carries the step's position, so matching /^Step \d+ complete$/
proved only that some badge was there. Every one of these sites runs
under a pro token, so the numbers come from the licensed order.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0198kPp8K99BRJ15z73m5MY4
* Revert "refactor: query the hub content by its main landmark"
This reverts commit 667b15013d.
* refactor: give settings-components a public API
The module is new in this PR, so every consumer's import line is already
in the diff -- the cheapest moment to add the barrel its shared-tier
neighbours settings and segments already enforce.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0198kPp8K99BRJ15z73m5MY4
* refactor: exempt the relocated modules in the eslint config
Leaving admin/ drops these files out of the exemption that turns the
metabase-string and color-literal rules off, and embedding-hub is
already on that list. Two files go back to matching their originals
byte for byte -- an inline comment forced a JSX reflow.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0198kPp8K99BRJ15z73m5MY4
* fix: validate the setup guide return path
The parameter went unchecked into a link target, so a crafted url
rendered a working off-instance link on an admin page and an empty
one rendered a dead link whose onClick still marked the step done.
Uses the same origin check as the login redirect parameter.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0198kPp8K99BRJ15z73m5MY4
* fix: build the return-to link with URLSearchParams
The old concatenation appended a literal "?" whether or not the step's
own path already carried a query string.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0198kPp8K99BRJ15z73m5MY4
* refactor: drop the code the admin removal stranded
The deleted pages were the only callers of UpsellSdkLink, the related
settings helpers and their section wrapper; the theme listing's own
gate was the only caller of UpsellEmbeddingTheme. ColorRow had none.
RelatedSettingCard stays -- the tenants onboarding step uses it.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0198kPp8K99BRJ15z73m5MY4
* refactor: drop showHeading from the theme listing
The admin route that mounted the listing as a whole page is gone, so
its one remaining caller always passes false and the heading branch
never renders. The Appearance page titles the section itself.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0198kPp8K99BRJ15z73m5MY4
* test: trim dead setup and restore the sdk plugin stub
Three setup parameters and two mock settings had no caller once the
csp-img and custom-viz reads moved to the domains page. The plugin
stub was assigned without ever being put back.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0198kPp8K99BRJ15z73m5MY4
* test: tighten the hub tab and locked step assertions
assertHubTabs compared one hardcoded absent tab; counting the links
makes the caller's list exhaustive instead. The locked production
step now asserts its tooltip, which replaced the old inline reason.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0198kPp8K99BRJ15z73m5MY4
* docs: correct three stale comments
The widgets folder is no longer a sibling of AdminSettingInput, the
EMB-2229 sentence promised a tier change that branch does not make,
and the smoketest assertion needed the reason it expects nothing.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0198kPp8K99BRJ15z73m5MY4
* test: keep the no-fetch assertion for the appearance upsell
The theme listing spec asserted it before its token gate moved to the
page; assert it where the gate lives now.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0198kPp8K99BRJ15z73m5MY4
---------
Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
* Move the embedding hub module from app tier to feature tier (#81019)
* refactor: move what the hub shares with admin into common
After the tier flip admin and the embedding hub are both feature tier,
and a feature module may not import another, so anything both read has
to surface into a shared module.
base-path.ts is that: six admin files, three enterprise files and two
hub files read it, so it moves to common along with its spec. The
public-links table moves for the same reason -- admin's Public sharing
page and EmbeddedResources both render it, and EmbeddedResources itself
moves to embedding/settings since only the hub's Security page uses it.
EmbeddingHubPermissionsBasePath hardcodes the hub's path and is read by
the hub alone, so it moves into the hub rather than into common. Every
other file here is a one to three line import repoint.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014XchaaoRd76S9WU3NzhqKW
* refactor: flip the embedding hub to feature tier
The hub's only cross-tier imports are routes.tsx reaching admin's
permissions routes and EmbeddingHubThemeEditorPage mounting the app-tier
theme editor. Both go into an app-tier embedding-hub-routes element
ahead of the feature one, the same carve-out admin-routes used, so the
rest of the module can flip.
Drops admin-routes itself, now dead. #80146 created it solely so
admin/routes.tsx could lazy-mount that same theme editor, replacing a
feature/admin -> app/theme-editor whitelist entry; #81018 then moved the
editor to embedding/themes and left admin with plain redirects, so
nothing in admin/routes.tsx reaches app tier any more. This ticket is
where that edge resurfaces on the hub.
Verified against both rule sets, which are built separately:
module-boundaries stays at 32 problems with no admin/routes finding, and
lint-eslint-pure, which loads enforcedRules, exits 0.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014XchaaoRd76S9WU3NzhqKW
* docs: drop the stale tenants base path contrast
The comment said this module lives in common, unlike the permissions
editor's equivalent, because it is read from both OSS and EE call sites.
Both halves are now false: this PR moves that equivalent into common
too, and it is read from enterprise as well. The contrast distinguishes
nothing, so it goes rather than getting reworded.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014XchaaoRd76S9WU3NzhqKW
* fix: keep the theme editor page off the pages barrel
The page sits in the app-tier carve-out so it can import
EmbeddingThemeEditorApp, but pages/index.ts is feature-tier and
re-exported it. The boundary lint does not follow re-exports, so a
feature-tier file could reach the app tier through the barrel: a direct
import errors, the barrel import passed silently.
routes.tsx now imports the file directly, which is legal because
routes.tsx is itself in the carve-out.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MgJHnFajYcKBARP8J33ZmR
---------
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
* docs: name the embedding hub as the seeding trigger
The hook moved behind EmbeddingHubLayout, so it now fires on first visit
to the hub rather than to the themes listing page.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PzJNGUUd5hZjQjdSjHC6mV
* refactor: move the embedding hub's Toucan 2 calls into a db namespace
Master added a kondo rule keeping Toucan 2 query calls in module db
namespaces (#82099). The hub's OSS module predates it, so merging master
into the integration branch left 17 warnings in api.clj and a red
Clj-Kondo job.
Extracts the queries verbatim into metabase.embedding-hub.db, mirroring
metabase.embedding-rest.db.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PzJNGUUd5hZjQjdSjHC6mV
* fix: refetch the setup guide checklist after a single setting write
The checklist query is invalidated by updateSettings but not by
updateSetting, so a step written through useAdminSetting -- the AI one --
stayed grey until the page was reloaded.
Tagging the singular mutation rather than the AI page fixes every step
written that way. It costs nothing on instances that never open the hub:
RTK Query only refetches subscribed queries, and updateSetting is already
the pessimistic admin path that refetches session-properties on each call.
The per-click writes go through updateUserSetting, which is untouched.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PzJNGUUd5hZjQjdSjHC6mV
* fix: return to the setup guide after creating a theme from it
The editor always went back to the Appearance listing, so the setup guide's
theme step was a one-way trip. It now takes an optional returnPath, and the
hub page sets it from a ?return=get-started marker on the link.
The step links straight into a new theme rather than the listing, so the
return lands where the user actually came from. Only the one known marker
value is honored.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PzJNGUUd5hZjQjdSjHC6mV
* fix: satisfy the frontend linter on the hub specs
The theme editor's spec imports the page, which is app-tier because it
mounts EmbeddingThemeEditorApp, so the spec is tagged the same way its
routes.unit.spec.tsx sibling already is.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix: reorder the apps in the app switcher
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix: drop the tenants docs icon inside the embedding hub
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix: restyle the New embed nav button
Primary rather than secondary, so it does not get lost on the page. And
collapsed, where it sets the nav rail's width, it takes the same 32px box
as the tabs beside it instead of a taller, wider button.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0188APErQre5KQuN6WdmJedA
* fix: state the modular embedding terms for guest embeds
Enabling guest embeds writes the same enable-embedding-modular setting
the SSO path does, so a Pro admin ended up with modular embedding on
without ever being shown its terms.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix: turn tenancy on straight from the embedding hub
The button said "Enable multi-tenancy" and then asked which strategy to
use, so the choice was already made twice. It now writes the setting and
the page renders the listing, which is what the hub can do and admin
cannot: admin's own entry point still opens the modal.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0188APErQre5KQuN6WdmJedA
* fix: drop the tenant headings the hub tab already names
Under the hub's Tenancy tab each sub-tab repeated its own name as a page
heading, so the tab bar and the heading said the same word. The listings
take a showTitle prop to leave it out; admin passes nothing and keeps its
headings.
Which host mounted the tenant routes is now one isEmbeddingHubTenancy
helper, mirroring the permissions editor's isEmbeddingHubPermissions,
rather than the base-path comparison the docs button had inlined.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0188APErQre5KQuN6WdmJedA
* fix: drop the tenancy invalidation the settings API does
updateSetting already invalidates setup-guide-checklist on every write and
permissions-group when the key is use-tenants, so the hub was asking for
both a second time.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0188APErQre5KQuN6WdmJedA
* docs: point the tenancy hub check at its permissions twin
Two files now share the name is-embedding-hub.ts, and nothing said the
second was deliberate rather than a duplicate to collapse.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0188APErQre5KQuN6WdmJedA
* refactor: ask for the tenancy host where the permissions one does
The listings took a showTitle prop fed by the tenant wrappers, while the
permissions editor's components call their own check inline. Follow that,
so the two hub-aware subsystems read the same, and the wrappers go back to
forwarding nothing.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0188APErQre5KQuN6WdmJedA
* fix: validate the setup-guide return path before navigating
The param is read straight from the URL and handed to navigate, while the
sibling that reads the same param checks it against the site origin first.
Router normalization happens to make an off-origin value land same-origin
today, which is not something to rely on.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0188APErQre5KQuN6WdmJedA
* style: format the JWT spec after the merge resolution
The merge commit skipped the hooks so master's files would not be
reformatted wholesale, which also skipped the two files resolved by hand.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0188APErQre5KQuN6WdmJedA
* test: drop the strategy modal step from the tenancy spec
The hub's Enable multi-tenancy button writes use-tenants directly and
lands on the listing; only admin's own entry point still opens the
strategy modal.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0188APErQre5KQuN6WdmJedA
* test: treat an already-enabled terms card as accepted
A section that mounted on stale settings stays on screen with its
disabled Enabled button, so keying the early return on the card's
presence made the helper look for an accept button that was gone.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0188APErQre5KQuN6WdmJedA
* test: assert the embed preview loader before accepting terms
On Pro, the Guest radio now renders the modular embedding terms card, so
the helper's terms round-trip runs between opening the modal and the
loader assertion. The preview mounts at modal open and its iframe
finishes inside that window, so the loader had already unmounted by the
time the assertion started.
Asserting the loader immediately after the modal opens restores master's
timing while keeping the terms click.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix: call useUpsellFlow only when the upsell renders
The hook sets window.name in a useEffect, which renames the Cypress AUT
frame. Any page that mounted it broke cy.press even on Pro, where no
upsell is shown -- embed-parameters.cy.spec.ts failed with "Could not
find AUT frame". Both hub pages already gate the banner on the token
feature, so moving the hook into the banner keeps it out of the Pro path.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01QyreUcTwdriuiQ3ngskSpF
* fix: match the designs in the embedding hub layout
Per design review on #80336: the theme editor is full screen again, with
no nav sidebar and no app switcher, and the sidebar's Permissions entry
uses the eye icon the designs call for.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01QyreUcTwdriuiQ3ngskSpF
* fix: keep the secret key input controlled when cleared
Clearing the field saves an empty value, and the backend then reports the
setting as present-but-nil, so /api/session/properties sends null and the
input's value went from a string to null. React flagged the controlled to
uncontrolled switch on every clear.
The two other reads of the setting behave the same either way; they use
the normalized value so nothing in the component sees the raw null.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017H2PEghttWPv6wFmLQfEdu
* chore: regenerate modules config after master merge
The merge dropped queries_rest's metabase.embedding.validation require:
this branch removed the check-embedding-enabled gate on the embeddable
cards endpoint, and master removed check-allowed-to-change-embedding.
With no call sites left, queries.rest no longer uses embedding.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013SnFSpp92VDRya282ZbCWu
---------
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
* disable explorations
* fix e2e
* Keep the explorations API tests running while the routes are unmounted
The explorations routes are intentionally left out of the application's
route map, so every `mt/user-http-request` against `/api/exploration/...`
in `metabase.explorations.api-test` and
`metabase-enterprise.sandbox.explorations-test` returned 404.
Let the test HTTP handler be overridden per test: `metabase.server.test-handler`
gains a public `make-test-handler` that takes the API routes to serve, a
bindable `*test-handler*`, and `do-with-api-routes`. A new
`metabase.explorations.test-util` mounts `/api/exploration` in front of the
application's routes (wrapped with `+auth` as the route map used to), and both
namespaces install it as an `:each` fixture.
Also assert that the application itself still 404s on `/api/exploration`, and
regenerate the modules config, which no longer lists
`metabase.explorations.api` as an exported API namespace.
* Keep the explorations Metabot profile unregistered while explorations are disabled
The agent endpoint accepted profile_id "explorations" even though
nothing on the branch mounts the feature, so the profile's tools and
prompt were still reachable over the API. Define the profile without
registering it, and reject an unregistered profile with a 400 before a
turn is persisted or the title job calls the LLM. Tests that exercise
the profile register it through a new test-util helper.
* Close the listing surfaces that still handed out residue exploration rows
Explorations are disabled, but an instance upgraded past this version,
used to create explorations, then downgraded keeps its exploration
tables. Four listing surfaces still handed those rows to clients. Not
anymore!
- Search: "exploration" joins the excluded-models set and leaves the
models search order, so it drops out of the searchable model set.
Both engines filter stale index rows at query time, a models filter
naming it is rejected, and it is no longer an available model.
Ingestion and the shared API model map are untouched.
- Collection items: dropped from the default model list and the models
parameter allowlist, so it is neither listed nor requestable.
- Bookmarks: the exploration arm leaves the bookmark union and the type
leaves the bookmarkable models, so listing omits it and create/delete
no longer match a route (404, like any unknown model).
- Comments: dropped from the API type map, from which the request
schema derives, so exploration targets are rejected.
The disabled-state tests now live together in
metabase.explorations.disabled-test: the routes 404 and Metabot profile
400 tests move there from the explorations and metabot API tests, and
one test per closed surface joins them. Tests that asserted explorations
were listed by these surfaces are removed; the comment-context gate test
runs the registered gate directly, since the comments API no longer
serves exploration targets; the exploration model's search test keeps
only its spec-registration check.
Frontend callers that still send the model (the collection content
all-models constant) get a 400 from the items endpoints until that
constant drops it.
* Stop the collection page from requesting the exploration model
* review feedback
---------
Co-authored-by: John Swanson <john.swanson@metabase.com>
* LDAP and Password auto log out when mfa enabled
* Fix lint
* Update core.clj
* Update gate_test.clj
* Update session_test.clj
* Require MFA enrollment on login when it is `:required` but user is not enrolled
* Tidying up, fixing unit tests
* Update session.clj
* Update session_test.clj
* Associate MFA Auth Identity with session
* Check for present mfa auth
* Uxw 4972 login introduce 2fa enrollment on login fe (#79485)
* updating admin setting
* MFA enrollment on login
* grace period setting
* Recovery Codes
* ..
* .
* ..
* auto grace period, clear button
* starting e2e coverage for enforcement
* Propagate MFA id to new session after getting recovery codes
* SSO mfa e2e tests, FE polish
* Update session.clj
* Update core.clj
* Revert "Update core.clj"
This reverts commit 5474ba3ea4.
* Tests passing aside from my new ones for password reset
* Fix `apply-mfa-gate` logic for SSO logins while MFA is required
* Fix MFA session checks post BE SQLi hardening
One of the `:raw` HoneySQL clauses was missing the `^:allow-raw-sql`
annotation. Such illegal SQL clauses throw 500s in dev mode, but in
prod and test modes they are replaced with `TRUE` or a similar dummy
clause. So the MFA enforcement conditions in the session middleware
were reduced as follows:
- intended: `mfa_auth_identity_id IS NOT NULL OR (auth_identity.provider <> 'password' AND auth_identity.provider <> 'ldap')`
- sanitized: `mfa_auth_identity_id IS NOT NULL OR (TRUE AND TRUE)`
which is equivalent to `TRUE`, nerfing the check completely!
This commit brings back the enforcement.
* more e2e
* comment cleanup
* UX adjustment
* disable required options for non enrolled admins
* confirmation modal
* PR Feedback
* Derive all the new MFA-related audit events from `::audit-log/user-event`
* FE lint and unit tests
* happy e2e
* auth card changes
* Addressing BE review comments; thanks Mike
* BE lint and formatter fixes
* fix up ::provider/supports-mfa usages to use the custom auth-identity hierarchy
* type check fixes
* Keep up with the security hardening: `:closed` input schemas
* Kevin feedback
* Propagate MFA information to the new session after changing your password (#82123)
* Propagate session id into user info
* Update api_test.clj
Add tests
Revert "Update api_test.clj"
This reverts commit 9bd4fce7400ca2fcf9e2760746ea0b6f0c67e65c.
* Fix lints
* Update db.clj
* Update api_test.clj
* Update src/metabase/users_rest/api.clj
Co-authored-by: Braden Shepherdson <braden@metabase.com>
* Rename query
* Update api_test.clj
---------
Co-authored-by: Braden Shepherdson <braden@metabase.com>
* enable the saml container in CI
* typo
* BE unit test fixes
* Add a response schema for our new endpoint `/api/session/mfa/enroll`
There's a ratchet making sure the number of ignores for `defendpoint`s
without response schemas does not increase, so a new endpoint is forced
to include one.
* fix OSS tests
* Revert "fix OSS tests"
This reverts commit 0515db4409.
* Move mfa-enforcement setting into OSS so it functions in OSS tests
It cannot be set to values other than `:off` in OSS, and there are no
MFA providers in OSS either, so it cannot be used.
* fix some dangling references
* green CI
---------
Co-authored-by: Ethan McCue <ethan@mccue.dev>
Co-authored-by: Nick Fitzpatrick <nick@metabase.com>
Co-authored-by: Nick Fitzpatrick <nickfitz.582@gmail.com>
The bot used to GET whatever url_private a file_share event carried, with the
workspace bot token attached, so a remote file could point it at any address the
registering app chose. It now refuses anything that isn't an https Slack file
host, turns remote files away before download, follows at most one redirect and
only to another Slack file host, and holds the size limit as the bytes arrive
rather than trusting the size on the event.
* Enable the feature the unknown-parameter test needs to reach its check
`resolve-parameters` refuses any `parameters` at all without `dashboard-subscription-filters`, and
that refusal runs before the unknown-id check. The teaching-error test supplied a bad id without
enabling the feature, so whenever it ran without ambient token state the call failed on the feature
and never reached the check the test exists to pin — both assertions red, which is what CI hit.
Wrapped in `mt/with-premium-features`, matching the two sibling parameter tests, which already pin
their own token state rather than inheriting it.
Verified fails-when-broken: swapping the feature set to `#{}` fails both assertions.
Claude-Session: https://claude.ai/code/session_018PNCQxp9kCypqE3dLQS9va
* [mcp-v2 12] definitions: cover the write-scope readback degradation
Both tools echo the created/updated row, which carries its name and
definition — so a token holding only `agent:content:write` would learn
the content of anything it may touch. `v2.write/readback` degrades that
echo to an ack, and all four call sites here relied on it untested.
`readback-requires-read-scope-test` drives create and update for both
tools with the write scope alone, asserting the payload is exactly
`{id, note}` and the note names the missing read scope. It then reads the
row from the DB to show the write itself still landed, and repeats the
update with both scopes to show the full row comes back — so the
assertion cannot pass merely because the tool returned something thin.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AbCo3ybRitdVSQXYcG97QR
* [mcp-v2 12] definitions: reject a blank name
Both tools declare `name` as `[:string {:min 1}]`, which " " satisfies,
so a whitespace-only name was accepted and stored — laxer than the REST
endpoints whose checks these tools otherwise inherit, which take
`ms/NonBlankString`. The JSON schema the MCP manifest is built from
cannot express non-blank, so the check belongs in the handler, beside
the `str/blank?` check `revision_message` already gets.
`check-name!` runs in all four branches immediately after the method-args
check, before any id resolution — so a blank name on update reads as an
argument error rather than "not found".
`blank-name-test` was written first and watched fail: the create half
stored `{"id":1,"name":" ", ...}`, and the update half returned the
not-found message because the lookup preceded any name check.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AbCo3ybRitdVSQXYcG97QR
* [mcp-v2 12] definitions: cover clearing a description
`clear` is the only way to unset a property — a null cannot carry that
meaning, since strict clients fill every unset property with null and the
registry strips nulls at the boundary. `dispatch-write`'s expansion is
unit-tested in `write_test`, but nothing checked that a cleared property
actually reaches the column through these two tools.
`clear-description-test` creates each entity with a description, clears
it, and asserts both that the column is null and that the key is absent
from the echo — the concise projection drops nils rather than echoing
null back.
It also pins where a non-clearable property is refused: `clear`'s schema
is `[:sequential [:enum "description"]]`, so that is rejected as an
invalid argument before `expand-clear` runs, making its own "can't be
cleared" message unreachable for these two tools.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AbCo3ybRitdVSQXYcG97QR
* MCP v2 get_parameter_values: reject date constraints chain filtering can't parse
`check-constraints!` promised to reject any constraint chain filtering would
silently drop, but covered only the three field-shaped drop paths. The fourth is
in the value: `chain-filter/add-filter` routes a string value on a temporal field
through `params.dates/date-string->filter` and catches a parse failure into
`nil`, dropping that filter and running the query unnarrowed.
Proven before fixing — a `_DATE_` constraint of "sometime last spring" against a
checkins dashboard was accepted and returned all 100 venues with
`has_more_values false`: a completely unfiltered list handed back as a filtered
one. That is the failure this tool exists to prevent, and a date string an LLM
would plausibly produce is the likeliest way to hit it.
The check runs `date-string->filter` itself, so it can't disagree with
`add-filter` about what parses — the same "probe with the real thing" property
the reachability check gets from `filterable-field-ids`. Temporal-ness is tested
as `effective_type` (falling back to `base_type`) against `:type/Temporal`,
matching `lib.types.isa/temporal?` on the column `instance->metadata` builds. It
is scoped exactly to `add-filter`'s branch — bare strings only, per field-id, so
a multi-field constraint is judged per field the way the fetch judges it. The
error names the grammar that does parse rather than only refusing.
Also fixes an adjacent misdiagnosis in the same fn: when the target parameter has
no field-ids at all, chain filtering falls back to `filter-values-from-field-refs`,
which ignores constraints outright. That was reported as "no join path to this
parameter's table", which sends the reader looking for a missing FK. It now names
the real cause.
parameters-test 32 tests / 89 assertions; :module parameters 151t / 385a;
fix-modules-config unchanged; kondo 0/0; cljfmt clean.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XJcw6ZamZBNhJ8SQxgJB13
* MCP v2 get_parameter_values: cover constraints and query applied together
The tool's most involved path had no test. `constraints-test` passes constraints
with no query and `query-search-test` passes a query with no constraints, so
nothing reached `chain-filter-search` with a non-empty constraint set — the one
route that also pins `chain-filter/*allow-implicit-uuid-field-remapping*` false.
The test asserts the two narrowings compose rather than merely co-exist: a
category that matches the query but is excluded by the constraint must be absent.
Searching "African" alone finds it; searching "African" under price=4 returns
nothing, which fails if the constraint is dropped once a query is present.
No behaviour change — the path was already correct. This closes the coverage hole
so it stays that way.
parameters-test 33 tests / 92 assertions; kondo 0/0; cljfmt clean.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XJcw6ZamZBNhJ8SQxgJB13
* MCP v2 get_parameter_values: blame the search when a query matches nothing
An empty result carried one sentence regardless of how it was reached: "its
source may be empty, filtered to nothing for you, or a free-text filter with no
value list." Every cause it offers is a property of the parameter, so after a
search that simply missed, the agent is told the filter may be broken when the
actual recovery is to search for something else — and none of the three listed
causes is even true.
`steering-line` now takes the caller's `query` and, when there was one, names it
and points at the recovery that fits: search for less, or drop `query` to list
everything. The generic sentence still covers the no-query case, where those
three causes really are the candidates.
Threading `query` into `values-content` is the only structural change; the
slicing and the other three branches are untouched.
Covered by no-match-for-query-blames-the-query-test, which asserts on both
targets that the line quotes the search and drops the causes that can't explain
it.
parameters-test 34 tests / 98 assertions; kondo 0/0; cljfmt clean.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XJcw6ZamZBNhJ8SQxgJB13
* Fix archived document hits being dropped from archived searches
validate-and-enrich-documents re-fetched document hits with a hardcoded
`:archived false`, so any document matched by an `archived: true` search
failed that lookup and was dropped by the `keep` — while `:total` still
counted it. The result was an empty `data` alongside a nonzero total that
no offset could ever reach.
The predicate now takes the archived state the search asked for:
`validate-and-enrich-documents` gains an `archived?` first argument and
selects `:archived (boolean archived?)`, with the call site in `search`
passing `(boolean archived)`.
Reachable only since the v2 search tool started exposing `archived`
through this pipeline; documents are archivable (`:archived true` in the
document search spec), so both `{archived: true, type: ["document"]}` and
an unscoped `{archived: true}` lost every document hit.
Test proven red before the fix (parameter added, hardcoded `false` kept):
archived hit dropped (expected [id], actual ()), and an active hit wrongly
retained for an archived search. The new test pins both directions.
Verified: metabot.tools.search-test + mcp.v2.tools.search-test +
mcp.v2.common-test 72 tests / 270 assertions / 0 failures; kondo 0/0;
cljfmt clean.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CF3ZXRaq6wy8sFR9L8DPKa
* Test that only a superuser's search reaches the engine for transforms
This pipeline dropped remove-unreadable-transforms and relies instead on
the transform search spec's `:visibility :superuser`, which the engine
applies when it narrows a context to its applicable models. That holds
only if the pipeline hands the engine the caller's real superuser status,
and nothing pinned that.
Captures the context passed to ranked-results and asserts `:models`
contains "transform" for an admin and never for a non-superuser, with
"dashboard" asserted present in both directions so neither can pass on an
empty set.
Proven to be a real guard by mutation: with `:is-superuser?` hardcoded to
true in the per-query context, the non-superuser assertion failed
(expected (not (contains? models "transform")), actual (not (not true)))
while the admin assertions stayed green.
Deliberately narrower than an end-to-end search: getting a transform to
appear in the index needs search.ingestion/*force-sync* plus an explicit
ingest! built from private search.engine/search.ingestion vars (see
metabase.transforms.search-test), which would couple this test to search
internals. An end-to-end attempt was discarded because its positive
control failed — the admin did not find their own transform — leaving the
non-superuser assertions vacuous. The visibility rule itself is covered by
metabase.search.filter-test; what is new here is the wiring.
Verified: metabot.tools.search-test + mcp.v2.tools.search-test +
mcp.v2.common-test 74 tests / 279 assertions / 0 failures; kondo 0/0;
cljfmt clean.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CF3ZXRaq6wy8sFR9L8DPKa
* Stop a caller-supplied collection-id escaping a metabot's confinement
An embedded metabot, and the nlq profile, may only search inside its own
collection. That is a containment boundary, not a default. The shared
search fn had been changed to let an explicit collection-id take
precedence over it:
collection-id (or collection-id (when confined? (:collection_id metabot)))
so a caller passing both would search wherever it asked, outside the bot's
collection. Inert today — no caller sets :collection-id, and the v1 tool
schemas are closed — but the v2 search tool fills that key straight from a
request filter, so the escape is one caller away.
Precedence inverted: the confined collection wins, and a caller-supplied
collection-id applies only when the metabot is unconfined.
Test proven red first: confined-collection-is-not-overridable-test failed
on exactly the escape case (expected the bot's collection, got the
caller's) while both controls passed. It pins all three cases — confined
with no collection-id, confined against an explicit one, and an
unconfined metabot still honouring an explicit one, so the fix cannot
over-restrict.
Trade-off: a confined caller asking for a sub-collection inside its
confinement now gets the whole confined subtree instead of the narrower
request. Intersecting would need a descendant check; no caller passes both
today, so it is not built.
Verified: search suites 75 tests / 282 assertions / 0 failures; the other
caller, metabot.agent.core-test, 23 tests / 140 assertions / 0 failures;
kondo 0/0; cljfmt clean.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CF3ZXRaq6wy8sFR9L8DPKa
* Build narrowing disclosures from the final type set
When `type` is omitted, each of created_by / collection_id / archived can
narrow the search to the types that support it, and each narrowing is
disclosed. Every disclosure built its "narrowed the search to ..." list as
`(set/difference effective-types (set bad))` — the full engine surface
minus only its OWN exclusions — so with two filters narrowing at once the
later message advertised types the earlier one had already removed.
With {created_by: "me", archived: true} and no type, the archived
disclosure claimed the search covered `collection` and `segment` when the
final type set excluded both. The namespace's stated contract is that
narrowing is disclosed; here it was disclosed inaccurately.
The accumulator now records {:excluded :label :because} per filter instead
of a prebuilt sentence, and the sentences are formatted after the fold
from the final narrowed set — one correct type list everywhere, each
message keeping its own reason for what it excluded.
Test proven red first: the archived disclosure failed with
expected: "action, dashboard, document, measure, metric, model, question"
actual: "action, collection, dashboard, ... question, segment"
The created_by disclosure passed, because that filter's exclusions are a
superset and its private subtraction coincided with the final set — which
is why no single-filter test could have caught this.
Single-filter wording is unchanged, so the existing
omitted-type-narrows-and-discloses-filter-incompatibility-test passes as
written.
`:label` rather than `:filter` for the key: as `:filter` it shadowed
clojure.core/filter in the formatting fn. Kondo did not flag it.
Verified: mcp.v2.tools.search-test + mcp.v2.common-test +
metabot.tools.search-test 76 tests / 288 assertions / 0 failures;
kondo 0/0; cljfmt clean.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CF3ZXRaq6wy8sFR9L8DPKa
* Steer an empty list page instead of returning a bare envelope
truncation-line only fires on arithmetic truncation — (offset + limit) <
total — so a page that came back empty carried no steering line at all.
An offset at or past the end produced
{"data":[],"returned":0,"total":37}
which reads as "nothing matches" when the truth is "nothing *here*", and
gave the caller no hint that paging back would help. The same silence hit
an empty first page whose every match was dropped after the count (a stale
index hit, a row gone unreadable).
list-content now emits an empty-page line whenever data is empty and total
is positive, splitting the two cases: a positive offset is told the offset
and total and steered back with a smaller `offset`; offset 0 is told the
matches are no longer readable or have been removed, and is NOT steered to
an offset, which could not help. A floored total stays "at least N".
Deliberately silent when total is 0 or unknown — that envelope already
says what happened, and inventing a line there would be noise.
truncation-line itself is untouched, so the parameters tool's own
steering-line wrapper (which handles its zero-returned case before ever
reaching truncation-line) is unaffected.
Test proven red first: the five positive assertions failed against the
bare JSON payload, while all three negative controls (total 0, unknown
total, non-empty page) passed before and after.
Verified: common + v2 search + v2 parameters + metabot search 107 tests /
377 assertions / 0 failures; kondo 0/0; cljfmt clean.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CF3ZXRaq6wy8sFR9L8DPKa
* Disclose the types a collection-scoped search silently drops
The namespace's contract is that the engine's silent narrowing is never
relied on. Two types slipped through it when `type` was omitted and
`collection_id` was set:
transform — its spec has `:collection-id false`, so
search-context->applicable-models drops the model outright from any
collection-scoped search. Always, on every edition.
table — the ::collection-hierarchy where-clause excludes tables when the
Library feature is absent, so OSS silently searched no tables.
Named explicitly, each was already a teaching error. With `type` omitted
both were dropped in silence, and the disclosure the caller did get named
them as in scope: "collection_id narrowed the search to ..., table,
transform — database, measure, segment don't live in collections."
Both are now narrowed and disclosed alongside the collectionless types,
each with its own reason, since they are excluded by different parts of
the engine and only one is edition-dependent.
The transform half was not in the original review note; it surfaced from
the pre-fix disclosure text while fixing the table half.
Tests proven red first: 6 assertions across the two affected deftests.
One pre-existing assertion legitimately changed — the collection_id
sub-test of omitted-type-narrows-and-discloses-filter-incompatibility-test
asserted exactly 1 disclosure and now sees 2. It is also now pinned to
#{:library}, so the count no longer depends on whichever token the suite
happens to run under.
Verified: v2 search + common + parameters + metabot search 108 tests /
386 assertions / 0 failures; kondo 0/0; cljfmt clean.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CF3ZXRaq6wy8sFR9L8DPKa
* Sign the MCP session capability payload so a client cannot self-assert it
`supports-mcp-ui?` is what `:required-extensions` gates on, and it read a
payload the CLIENT echoes back in its own session id. Unsigned, that is a
self-assertion: any caller could mint `<uuid>.{"v":1,"ui":true}` and claim
MCP Apps support it never advertised at `initialize` — which is what let a
narrow bearer reach `refresh_ui_credential` and mint a UI credential.
Session ids are now `<uuid>.<payload>.<signature>`. The signature covers
the uuid as well as the payload: every UI-capable session carries the
identical payload, so a payload-only signature would be a reusable token
that could be lifted onto any correlator. It is domain-separated from the
UI-credential HMAC so neither can be replayed as the other.
An unsigned or tampered id remains a VALID session. The id authenticates
nothing — it is a stateless correlator, and any well-formed one works for
its presenter, which `well-formed-but-never-issued-session-id-is-accepted-for-its-presenter-test`
pins. Only the claim inside it is disbelieved, so the gate fails closed
without breaking that property.
`supports-mcp-ui?` also stops returning true for a bare UUID. That branch
made the signature pointless — a caller could omit the payload entirely
and be believed anyway. Such ids predate capability hints, so `unknown`
is the honest reading.
Behavioural, and shared with v1: a client holding a session minted before
this change loses MCP Apps tools until it re-initializes. Bounded by
session lifetime, and it fails closed.
* fix(mcp): ignore database id for get_fields
Allow agents to carry database_id through the browse hierarchy without rejecting get_fields requests.\n\nRefs GHY-4396
* Stop stamping the MCP UI credential as an unrestricted session
The credential was stamped `::scope/unrestricted`, so the route allowlist
was the only thing between it and full session privilege. Those are two
different questions — which routes the credential may pass through, and
what privilege it arrives with — and collapsing them into one value meant
any way of reaching a route off the list (a routing change, a new alias,
a mistake in the set) served the request as the user.
It now carries `::scope/mcp-ui`, a second sentinel alongside
`::unrestricted`. It satisfies no endpoint's declared `:scope`, and
`ensure-scopes-checked` refuses it where none is declared, so the same
mistake fails closed. Being a keyword it can never be requested, granted,
or named on a consent screen. `:token-scopes-checked` is what lets the
allowlisted routes serve it, since they declare no scope of their own.
Deliberately NOT done by annotating those routes with the sentinel:
`/api/user/current`, `/api/session/properties` and the `/api/dataset`
routes live in `users-rest`, `session` and `query-processor`, and marking
them would push MCP vocabulary into three unrelated namespaces —
deepening the coupling GHY-4400 exists to remove, to buy locality in a
set currently frozen at eight entries.
The allowlist's docstring now says what it is: an inventory of what the
embedded app happens to call, because the credential is installed as
app-wide auth on the client, rather than a decision about what it should
reach.
* [mcp-v2 12] definitions: pin the table-move test's permission precondition
`table-move-permission-test` passed a solo namespace run but failed CI's
full-suite partition. `with-no-data-perms-for-all-users!` resets only the
All Users group, so the temp group's access to `checkins` was left at
whatever default a freshly created group inherits — ambient state that
differs between a namespace run and a full partition. When the analyst
could author on `checkins`, the retarget succeeded and the assertion
failed as "expected a tool error, got success".
Block the database for the group and grant `venues` back, so `checkins`
is denied by construction rather than by the absence of a grant, and
assert both halves of the precondition up front so a recurrence fails at
the setup with a clear message instead of as a confusing success.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AbCo3ybRitdVSQXYcG97QR
* Stop answering JSON-RPC notifications
A message carrying no `id` is a notification (JSON-RPC 2.0 §4.1) and must
not be answered. A known method sent without one — `{"jsonrpc": "2.0",
"method": "ping"}` — executed and then replied with `"id": null`: a
response invented for something the client is not listening on, in the
shape §5 reserves for a message that could not be parsed at all.
Notifications still execute. A notification is a request the client is
not waiting for, not one to ignore, so the dispatch is unchanged and only
the result is dropped. The malformed case keeps its null-id reply, which
is what §5 actually asks for; conflating the two is what produced the
bug.
Keyed on absence of the `id` key rather than a nil value, so a request
that explicitly sends `"id": null` is still treated as malformed rather
than silently swallowed.
* Pin the superuser flag instead of reading it off a shared test user
transform-visibility-is-superuser-only-test failed on the MySQL 9 EE and
MariaDB EE app-db shards, deterministically on both. The failure was the
POSITIVE control: the admin case saw models #{"dashboard"}, so transform
was absent for everyone and the non-superuser assertions were proving
nothing.
Cause: the test took the superuser flag from the ambient test user, and
`mt/with-test-user` resolves it through `request/with-current-user`, which
SELECTs the shared user row:
(t2/select-one [:model/User [:is_superuser :is-superuser?] ...] :id ...)
That makes the assertion depend on the state of a row other tests in the
same parallel shard also touch. This branch has already been bitten by the
same class of pollution once — recents-total-is-a-floor-test, fixed by
moving off the shared :crowberto user.
`api/*is-superuser?*` is now bound explicitly. That is the exact var
`search.filter/visible-to?` reads, and pinning it is what the assertion is
actually about: that this pipeline propagates the caller's superuser
status into the search context, not where the status came from. A sanity
assertion on the admin branch ("dashboard" must be present) keeps a future
empty/short model set from passing as a green positive control.
This isolates the variable rather than hiding it: if transform were being
dropped for some reason other than the superuser flag, the test would
still fail on those shards with the flag pinned true.
Still catches the regression it guards: with `:is-superuser?` hardcoded to
true in the per-query context, the non-superuser assertion fails with
expected (not (contains? models "transform")), actual (not (not true)).
Verified: metabot search + v2 search + v2 common 79 tests / 308 assertions
/ 0 failures; kondo 0/0; cljfmt clean.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CF3ZXRaq6wy8sFR9L8DPKa
* Run the transform visibility test on both search engines
The previous fix was wrong. transform-visibility-is-superuser-only-test
kept failing on the MySQL 9 EE and MariaDB EE shards after it, with the
positive control still red:
"a superuser's search reaches the engine with transforms in scope"
expected: (contains? models "transform")
actual: (not (contains? #{"dashboard"} "transform"))
The superuser flag was never the cause. Pinning it did isolate the
variable, which is how this became findable: with `:is-superuser?` bound
true the admin case still saw no transform, so the gate had to be
elsewhere.
Two engines gate transforms by different code. MySQL and MariaDB cannot
hold the appdb search index, so they fall back to in-place, whose
applicable-models has a rule the appdb path has no equivalent of:
(empty? enabled-types) (disj "transform")
`enabled-transform-source-types` comes from `transforms-enabled`, whose
getter falls back to a cloud-token check when the setting is unset — so
under `with-premium-features` it was false, the source-type set was empty,
and in-place dropped transform for everyone. H2 uses the appdb path, which
has no such gate, so the test passed locally and was meaningless on the
two shards that ran the other engine.
Both inputs are now pinned: `transforms-enabled` is set explicitly, and
the body runs under `with-appdb-search-and-legacy-search` so the shard's
engine choice can no longer decide whether this test means anything. The
assertion count doubling from 4 to 8 confirms both passes execute.
Diagnosis proven locally rather than by another CI round-trip: with
`transforms-enabled` flipped to false, exactly 1 of 8 assertions fails —
the superuser case, on the in-place pass only — reproducing the CI failure
byte for byte on H2.
Verified: metabot search + v2 search + v2 common 79 tests / 312 assertions
/ 0 failures; kondo 0/0; cljfmt clean.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CF3ZXRaq6wy8sFR9L8DPKa
* fix(mcp): include dashboard parameter wiring
* Cover the query_handle source, which no live test reached
Both handle tests were deferred behind the unregistered execute_sql tool, so
nothing exercised resolve-source's query_handle branch: a regression there —
notably resolving through the native-rejecting resolve-query-handle! instead of
resolve-query-handle-for-save!, which would make every execute_sql handle
unsaveable — passed the suite untouched.
Mint handles straight into the store instead, the way question_test already
does. The deferred tests stay as the ledger for the whole-tool path; these pin
the branch in the meantime.
Four cases: an MBQL handle creates a transform whose target follows the
handle's database, a native handle stores as a native transform without
re-demanding agent:sql:run (minting it already passed that gate — the
exemption is deliberate, and is the half of the contract the mutation above
would silently break), a handle swaps an existing transform's source, and an
unknown handle teaches the recovery rather than surfacing an internal error.
Verified by mutation: pointing the branch at resolve-query-handle! fails 2 of
the 4; disabling the branch outright fails all 4.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014Lzc5ERUWqWRx9JbDxK5Lc
* Mint a cursor only for a projection with a provable unique key
`tiebreaker-specs` fell back to the full projected tuple whenever no source-table
PK was projected, and treated that tuple as a total order. For an unaggregated
stage it isn't: the tuple repeats, and the strictly-past-the-boundary keyset then
skips every row tied with the boundary.
A `fields`-narrowed projection of non-unique columns hit this. PRODUCTS holds 200
rows across 4 categories; paging a CATEGORY-only projection at row_limit 5 served
20 rows and then reported itself complete — no truncation flag, no cursor, 180
rows silently gone.
Split the two meanings the function conflated. `unique-key-specs` now returns nil
when neither uniqueness proof holds, so a missing proof is distinguishable from an
order that is already total, and both callers refuse instead of inventing one.
The second proof has to look back past the last stage rather than just testing it:
`next-page-query` carries an aggregated query's keyset in an appended stage, so
from page 2 on the last stage is that pass-through — unaggregated and PK-less.
Inspecting only the last stage would close the gap and kill every aggregated chain
at page 2 instead.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JL8iPKC7nSLarfcnYDkaCS
* fix(mcp): address v2 protocol review
Reject unsupported DELETE without dropping session ownership. Enforce JSON-RPC 2.0 envelopes and notification silence. Keep pre-dispatch tool failures on the JSON-RPC error channel.
* Carry the target's database through a source-database swap
The target was only recomputed when the call named one, so an update that
swapped the source onto another database left the stored target pointing at the
old one. The target records the database it writes and that database follows the
query, so the row — and the echo built from it — named a database the transform
does not write. Feeding that echo back then hit resolve-target's target/query
database check and was refused, dead-ending the read-modify-write the tool is
built around. Only the JSON blob was stale; before-update recomputes the
target_db_id column, so runs went to the right place.
Patch the database onto the existing target instead of rebuilding it through
resolve-target: a target this tool can't author (incremental) still gets its
database corrected rather than becoming uneditable by a source-only edit. The
caller's own `target` still wins, being later in the cond->.
The test drives it across a real second database via with-temp-copy-of-db and
was watched failing first: stale database in both echo and stored row, plus the
round-trip rejection as the third symptom.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014Lzc5ERUWqWRx9JbDxK5Lc
* Supersede the previous page's keyset instead of stacking beside it
`lib/filter` appends — `add-filter-to-stage` skips only exact duplicates — so
every cursor page ANDed a fresh boundary predicate onto the stage while leaving
its predecessors in place. Paging ORDERS two rows at a time, the stored handle
grew 668 -> 2692 bytes over nine pages and kept going, one dead predicate per
page, in the payload and in the emitted WHERE alike. With the full-tuple key each
of those clauses is quadratic in the projection width.
Stamp each minted clause and drop stamped ones off the target stage before adding
the current page's. Dropping them cannot change results: every row the new
predicate admits lies strictly past a boundary row that itself satisfied the old
one, so the new predicate implies the old. Filters the caller wrote carry no stamp
and are left alone.
The stamp rides in the clause's options map, which no schema declares. It survives
serialization, the base64/JSON round trip through the handle store, and
rehydration today; should Lib ever strip undeclared option keys the predicates
would accumulate again exactly as before — same results, larger query — so this is
an optimization and not something correctness rests on.
Stored handle now flat at 716 bytes across nine pages.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JL8iPKC7nSLarfcnYDkaCS
* Finish the sentence that teaches the source shapes
`accepted-shapes` is appended to five source-shape teaching errors — it is what
an agent reads to work out how to retry — and it promised "either" and then
listed one dialect. The missing half was the name-based dialect, which
definition->query genuinely resolves via portable-query?, so the message
advertised only one of the two shapes the tool accepts. Restored in the wording
metric.clj already uses for the same pair.
Nothing held the sentence to the code, because only the numeric-id dialect had a
test. Cover the portable branch, and assert the sentence names both dialects so
the half-written form can't come back.
Verified by mutation: disabling the portable branch fails the new
portable-definition test and nothing else.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014Lzc5ERUWqWRx9JbDxK5Lc
* Refuse tag ids that name no tag, instead of dropping them
update-transform-tags! filters unknown ids out of the list it is given and
writes what's left, so `tag_ids` — documented as "replaces the current list" —
replaced it with a shorter one and reported success. An agent constructing ids
from a stale read has no reason to look twice, and jobs select the transforms
they run by tag, so the tag that failed to attach is noticed when a schedule
doesn't fire. Checked in the tool rather than in the shared write, which REST
also calls: this is the same restatement check-target-free! makes of REST's
target-conflict 403.
Cover collection_id and tag_ids while here — neither had a test, which is why
the silent drop had gone unremarked. The transform-folder namespace rule is
pinned too, since that refusal comes from a layer below this tool.
Adds a :model/TransformTag reference, so the generated module config gains it.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014Lzc5ERUWqWRx9JbDxK5Lc
* Pin the cursor's reliance on the QP to re-type a stringified boundary
Every execute_query run carries `js-int-to-string?`, which renders integers past
2^53-1 as strings. The cursor boundary is read straight out of the served row, so
a key column holding such values mints a predicate that compares a numeric column
against a string literal — verified, it really does emit
`[">" [field type/BigInteger] "9007199254740995"]`.
That compiles correctly, on every driver, because the query processor re-types it:
`wrap-value-literals` gives the bare literal the column's effective type and
`auto-parse-filter-values` parses it back to a bigint, both above the driver layer
(preprocess.clj:106-107).
Nothing covered that, and it is invisible from the cursor code, which reads a row
value and hands it to `lib/>` with no indication that its type is the query
processor's problem. Add a fixture whose values straddle the boundary and a test
that pages across it, plus a comment naming the dependency, so a change to the
middleware order or to the boundary read fails loudly here rather than in a
warehouse.
No production behavior change.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JL8iPKC7nSLarfcnYDkaCS
* Mint a handle from the question, not from the page it was read on
A cursor page's stored query embeds the keyset boundary it resumed from — that is
how paging works — and the handle returned alongside was minted straight from it.
So a caller who paged to page 3 and then saved got a card reading "orders WHERE
id > 4171": a scroll position persisted as a question, which is nobody's question.
Strip the minted keyset off the last stage before storing, in `mint-handle!` so
every mint site is covered. Only predicates this namespace stamped come off; a
filter the caller wrote is part of the question and stays. Paging is untouched —
the cursor carries its own boundary — and it is a no-op for a query that never
paged.
Re-running a cursor page's handle now serves the question from the top rather than
that page again. The tool description and the namespace docstring promised
"exactly the query that ran" and are corrected: a handle is what a later tool
saves or visualizes, so the page boundary was never the useful part of it.
One artifact stays: a query carrying its own :limit has it spent down across pages,
so a deep page's handle asks for the rows still owed rather than the count the
caller wrote. Recovering that needs state the handle store deliberately does not
keep; `without-page-boundary` says so.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JL8iPKC7nSLarfcnYDkaCS
* Register the :document projection where both its tools can see it
document.clj projects through :document but did not require the namespace
that registered it, so the registration was reachable only via api.clj's
alphabetical require order. Loading the document tool on its own — from a
test, or any future consumer — left the projection unregistered, and
projections/project throws a 500 that the error sanitizer collapses to a
bare "Internal error".
Move the registration to projections.clj, which both tools already require,
matching the :question block's rule for a projection two tools share. The
key vector goes with it; content.clj's select-keys pre-narrow is dropped
rather than duplicated, since project re-applies select-keys on every path
and ::document retains the full map anyway.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015AFrL3LpdWBaQizx5QWaF5
* Return dimensions the way the metric and measure endpoints do
The `dimensions` include documents itself as returning the same
dimensions/dimension_mappings pair GET /api/metric/:id and
GET /api/measure/:id return. Nothing compared the two, and it drifted three
ways:
- It skipped the wire encoding, so the section shipped the internal
kebab-case shape (display-name, effective-type, and an internal
:lib/source) where every other field in the tool, and the REST API, uses
snake_case REST property names.
- It kept orphaned dimensions, which both endpoints drop unless asked for
them. An agent grouping by one would query a column that no longer exists.
- compute-dimensions reconciled every computed pair for an uninitialized
metric, while seeding takes own-table and explicitly-joined columns only.
A metric nobody has opened in the UI reported every FK-reachable column,
and since this read never persists, it kept reporting them.
Encode through ->api-dimensions/->api-dimension-mappings, drop orphans, and
give compute-dimensions the same three-way metric branch sync-dimensions!
has, sharing one definition of the seed filter with it.
The parity test this needed also replaces the curated-metric fixture, which
was built from get_content's own output and so agreed with whatever shape
the tool emitted.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015AFrL3LpdWBaQizx5QWaF5
* Don't record a document view on a get_content read
`documents/get-document` publishes `:event/document-read` unless told not
to, and that handler bumps view_count, stamps last_viewed_at, writes a
view_log row, and pushes the document onto the caller's recently-viewed
list. get_content declares readOnlyHint and idempotentHint, and the
dimensions include already goes out of its way to compute without
persisting; a document read was quietly doing four writes and filling a
user's recents with the agent's reads.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015AFrL3LpdWBaQizx5QWaF5
* Register the :metric projection once, where both its tools can see it
content.clj and metric.clj each registered :metric with identical key
vectors. register-projection! overwrites silently, so which one won was
decided by api.clj's require order, and the two would have drifted apart the
first time either was edited.
Deleting either copy on its own just moves the problem: whichever tool keeps
the registration leaves the other reading a projection registered by a
namespace it does not require. Register it in projections.clj instead, which
both tools already require — the same place :question and :document live,
and for the same reason.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015AFrL3LpdWBaQizx5QWaF5
* Drop the TODO comments above the two restored document tests
Both tests were deferred while get_content did not exist and came back with
this slice; only the notes telling a reader they were still disabled were
left behind.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015AFrL3LpdWBaQizx5QWaF5
* Stop documenting a per-type scope gate that isn't there
The type->spec docstring said each entry carried "the extra runtime :scope
the type needs on top of the tool's base agent:resource:read", and the ns
docstring said each item was scope-checked. No entry has a :scope key, the
handler never reads one, and agent:resource:read is not a scope that exists.
The behaviour is right — GHY-4225 folded the per-entity read scopes into
agent:content:read, and the tool gates on that once at the registry
boundary, the same way search does. Only the comments were wrong, and a
comment describing a security control that isn't implemented is one a reader
will trust. Say what actually happens, and say that the absence of a
per-type scope is deliberate.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015AFrL3LpdWBaQizx5QWaF5
* Cover the read paths get_content had no tests for
Four gaps, all coverage rather than defects — the behaviour was already
correct in each case:
- entity_id resolution had no test at all, though it is half the id schema.
Covers collection, question, snippet and document by entity_id, plus the
two teaching errors: alerts are numeric-only, and a string that is neither
shape is rejected.
- the definition include was only exercised for question. measure, segment
and transform each build theirs behind a catch that returns nil, and the
section is omitted when it does, so a broken exporter was indistinguishable
from a type with nothing to export.
- reads-formerly-gated-types named transforms but tested only alert, snippet
and document, so no transform read ran under a real agent:content:read
token.
- the transform target table had no test.
On that last one: the permission gate's false branch turns out to be
unreachable. Reading a transform requires superuser or data analyst,
superusers read every table, and table-permission-for-user hands every data
analyst manage-table-metadata :yes unconditionally, which alone satisfies
can-read? on a Table. Rather than force it with redefs, the test covers the
reachable paths — present on a detailed read, absent on a concise one, absent
when the target has never been created — and records why the gate is dead
code kept as defence in depth.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015AFrL3LpdWBaQizx5QWaF5
* Say what the SQL kill switch actually gates, and stop telling callers to pass a cursor alone
Two docs that contradicted the code.
check-execute-sql-enabled! claimed "every v2 path that runs or stores raw SQL
consults this one gate." It does not: run_saved_question executes a stored native
card without it, as does execute_query over a :source-card naming one. The gate
covers the paths where the agent AUTHORS the SQL — execute_sql, and question_write
/ transform_write storing agent-authored native — which is what the setting's own
description scopes it to, and what the card's read check guards on the other side.
Say that instead, so nobody infers the stronger rule and treats those two paths as
holes.
The cursor argument told callers to "pass it alone" while the tool description
told them row_limit alongside keeps the page size — and the cursor test passes
both. The exclusivity it was reaching for is already stated in the same sentence
and covers only query | query_handle | cursor, so an agent obeying the arg would
have silently dropped back to 100-row pages.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JL8iPKC7nSLarfcnYDkaCS
* Move the v2 redaction queries into mcp/db.clj, guarding the id splice
`redaction.clj` was the one namespace this branch adds that ran its own
app-DB queries, against the policy that a module's queries live in its
`db.clj`. Both move across unchanged in behaviour: the `t2/hydrate` call
becomes `mcp.db/hydrate-notification`, and the batched tenant lookup
becomes `mcp.db/user-id->tenant-id`. `lookup-recipient-tenant-ids` keeps
only the part that is actually redaction logic — deriving the ids of the
recipients whose `:user` hydration came back nil.
The empty-collection case moves with the query rather than staying at the
call site: `[:in ()]` is not valid SQL, and that is a fact about the query,
not about recipients.
`user-id->tenant-id` refuses anything that is not a collection of
integers. The ids are spliced into an `[:in ...]` clause, where HoneySQL
binds a number as a parameter but renders a string, a raw form, or a
nested map as SQL — so a non-integer element arriving there is an
injection vector rather than a type error. It is refused before any SQL is
built, and the offending values ride `ex-data` rather than the message so
nothing caller-controlled reaches a log line.
The check is `int?`, not `Long`: `core_user.id` is a 32-bit column and
comes back from JDBC as an `Integer`, so a Long-only check would satisfy
any test written with literals and then refuse every real call. `int?`
still admits only boxed integer types, which is the whole of what the
guard needs. It throws rather than using `assert`, matching the mint-time
guard in `session.clj` — `*assert*` can be compiled out, and a guard that
disappears in some builds is not a guard.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FwF5gxSyq2T5FsnyhViozq
* update tools descriptions
* update mcp v2 skills (#81915)
update skills
* Name the right threat in the user-id splice guard
The guard's docstring said HoneySQL renders a string in an `[:in ...]`
clause as SQL. It does not — strings and numbers are both bound as
parameters. The shapes HoneySQL reads as syntax rather than as a value are
vectors (function calls, so `[:raw "1) OR 1=1 --"]` renders verbatim and
`[:inline x]` inlines a literal), maps (subqueries), and keywords and
symbols (identifiers). Those are the injection vectors; a string id is a
plain bug, refused all the same because a silent no-match is a worse answer
than a throw.
The check itself was already correct and is unchanged. Only the reason
given for it was wrong, which matters for a guard people will edit code
against — the docstring is what tells the next reader which shapes have to
stay refused.
Covers `[:inline ...]` and a bare symbol in the refusal test, the two real
shapes the first pass missed.
Deliberately not wrapping the ids in `[:lift ...]` on top of this. It is
unreachable behind an `int?` check, and it would trade the throw for
driver-dependent silence: `[:in [[:lift "1"]]]` binds the string and H2
coerces it, returning a match, where Postgres would not. For this lookup
that is the wrong direction — a recipient whose tenant silently reads as
nil slips past the tenant filter.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FwF5gxSyq2T5FsnyhViozq
* mcp-v2.02 bookmark_content (#81246)
* MCP v2 tool: bookmark_content
Adds the v2 bookmark_content tool — per-user favorites, keyed by (type, id).
Extraction: bookmarks/api.clj factors bookmark!/un-bookmark!/bookmarked? out
of the REST endpoints so REST and MCP share them. The POST endpoint now
creates via app-db/select-or-insert! (concurrency-safe: a racing duplicate
returns the existing row instead of tripping the unique constraint); its 400
on already-bookmarked is unchanged. REST tests extended, existing pass.
Unlike REST, both tool directions are idempotent (see ns docstring for why),
both read-check the item, and the name echo is gated on the read scope so a
write-only token can't use it as a name oracle.
Tests: 9 tool tests + bookmarks REST suite green.
Review question for this PR: one tool — is bookmark_content's contract right?
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BjQcjQx7NPGr3Rpf6QaG4b
Also lands (first consumer, per the common.clj slimming in the core PR):
- NEW v2/resolve.clj — entity-id / numeric-id resolution + the read-check
pairing with the not-found collapse (+ resolve_test, moved from common_test).
- NEW v2/write.clj — readback (the scope-gated write echo).
CI guard: this slice's mcp -> bookmarks edge put `bookmarks` into the
modules-that-trigger-driver-tests set (mage's module-graph ratchet, 44 -> 45).
Bookmark logic cannot affect driver behavior — the edge is a consumer edge —
so `bookmarks` joins driver-affecting-overrides rather than bumping the
ratchet. Full mage suite green (82 tests).
Review nit (round 1): fetch-item's flavor-mismatch teaching error printed
"is a null" for a card type outside question/model/metric; guard the nil case
with a type-agnostic message.
--- Round-2 /review-pr response ---
IMPORTANT (per-user-test could not fail on its named assertion) FIXED. The test called the tool
bare, so a tool-level error was discarded and the assertions passed regardless. It now routes
both calls through tool-result (throws on tool error) and has :lucky bookmark-then-un-bookmark
its own row, so the "one user's bookmark is invisible to another's un-bookmark" claim has a live
counterfactual. Proven fail-when-broken (stubbing un-bookmark! to a no-op turns the test red).
MINOR fixed: the id schema now requires a positive int (matching REST's ms/PositiveInt); the
degraded write-scope readback now surfaces :bookmarked (via a new backward-compatible ack-keys
arg on v2.write/readback, so a write-only caller still learns the resulting state).
MINOR deferred (pre-existing REST behavior, out of a tool-slice's scope, confirmed structurally):
archived content is bookmarkable but filtered from listings; the Trash collection is bookmarkable;
readback's :url key has no producer on this branch (lands with collection_write). Each is shared
with the existing REST bookmark endpoints; changing it means deciding REST semantics.
Fresh-JVM: bookmark-test 9 tests / 43 assertions / 0 failures; kondo 0/0; modules unchanged.
* Route resolve.clj through mcp.db; correct the db.clj docstring
Three fixes from review of the mcpv2-01-core merge resolution.
resolve.clj called t2/select-one directly, so the toucan2 kondo rule this
same change adds shipped already violated by a file it introduces -- and at
the highest-value site, where an agent-supplied id enters a query. Adds
mcp.db/select-one-by-id, which asserts the id is already an integer (entity
ids are resolved before they reach it, so a string there is a bug rather
than a lookup), and routes resolve.clj through it. The rule now flags only
v2/redaction.clj, which predates this branch.
bookmarks/db.clj's ns docstring still claimed every function was "a direct
Toucan 2 call with no additional logic" after the generic dispatch tier
moved in, which is the one part of that docstring a reader relies on.
insert-bookmark!'s unique-constraint rationale moves from its docstring to
an inline comment on the select-or-insert! call it explains.
Tests: bookmarks.api-test + mcp.v2.tools.bookmark-test + mcp.v2.resolve-test,
30 tests / 150 assertions, green.
* readback: refuse an empty read-scopes instead of skipping the gate
`(remove f [])` and `(remove f nil)` both yield `()`, so an empty or nil
`read-scopes` made `missing` empty and returned the full ungated row -- a
gate that did not run was indistinguishable from one that ran and passed.
Not reachable from `bookmark_content`, which passes a literal one-element
vector. It matters because this namespace is shared write-path machinery for
the `_write` tools still to land, and the mistakes a future caller actually
makes -- omitting the argument, or computing `read-scopes` from a per-type
map that misses -- both landed on silently ungated rather than an error.
There is no legitimate empty case, so it throws.
Adds write_test.clj, which the namespace did not have: full-row grant,
degradation (exact key set, so `:name` cannot leak), empty and nil
`read-scopes` refusal, the unrestricted sentinel and nil `token-scopes`,
note pluralization, and an ack-key absent from the row. Verified the
refusal test bites -- removing the guard fails it twice (`[]` and nil) and
leaves the other five green.
Also drops `readback`'s 3-arity, which had no callers, and the ns
docstring's reference to which slice lands next.
Tests: write-test + tools.bookmark-test + resolve-test + bookmarks.api-test,
36 tests / 161 assertions, green.
* Clean up comments in resolve.clj
Removed unnecessary comments for clarity.
* Drop the stray toucan-in-mcp kondo rule from this branch
The rule entered here through the mcpv2-01-core merge resolution rather than
from either parent, and it now lives on mcpv2-00b-shared-infra next to the
mcp/db.clj boundary it governs. Keeping a second copy here only produced seven
false positives on MCP test namespaces, whose toucan2.core requires are fixture
setup rather than the agent-id boundary the rule protects.
Every src/metabase/mcp namespace that touches toucan2.core is already exempt
(db.clj and models.*), so the rule guards nothing on this branch. It comes back
with the shared infra when the stack restacks.
* Fold the dead per-model inserts into one unknown-model failure
The generic (model, id, user-id) tier left three sibling functions dispatching
three different ways. insert-bookmark! reaches the bookmark models through a map
and select-or-insert!, so the five per-model insert fns it replaced had no
callers left anywhere; delete them.
An unrecognized model string failed differently in each of the three: a case
with no default threw "No matching clause", while insert-bookmark! destructured
a missing key into [nil nil] and failed further down inside Toucan with a nil
modelable. Route all three through one ex-info. Nothing reaches this today --
both callers validate against a schema first -- but this branch promotes
metabase.bookmarks.db to module API, so callers that validate nothing else can
now arrive.
Also guard the REST contract the insert swap sits under: a duplicate POST to
/api/bookmark/:model/:id still answers 400, which nothing covered before.
---------
Co-authored-by: Eric Normand <ericwnormand@gmail.com>
* mcp-v2.00b: land common.clj at its final size in 01 (#81975)
* Extract the v2 shared machinery into 01
The decomposition rule was "shared machinery lands with its first consumer."
The cost is that the first consumer becomes a dependency for every slice
after it, so a slice needing two functions inherits everything beneath the
slice that happened to add them.
`get_content` is the worked example: it calls `common/response-format` and
`common/select-fields`, both of which first appear in 08-search, so it sits
behind eight slices to reach two functions. Slices 03, 06 and 07 contribute
nothing it uses -- 03 adds zero defs to common.clj, 07 zero, 06 one. They
are in its path because 08 was stacked on them chronologically.
Brings the three shared namespaces to their final size here instead:
- common.clj 172 -> 342: the fields/response-format machinery (with its
private helpers) and the list envelope. `select-fields` needs
`projections/catalog`, which resolves from a runtime registry and whose
namespace already ships in 01; `LevenshteinDistance` is already on the
classpath via metabot.
- resolve.clj (118) and write.clj (85), which 02 created and 03 finalised.
Taken at their reviewed state, including routing the read through
`mcp.db/select-one-by-id` and refusing an empty `read-scopes` in
`readback` rather than silently skipping the gate.
- mcp/db.clj gains `select-one-by-id`, which resolve.clj needs.
Module config regenerated: `channel` (list envelope urls) and
`eid-translation` (id resolution) join the mcp module's :uses.
Slices 02, 03, 06, 07 and 08 still define this code and will conflict with
identical content when they merge; each needs the duplicate dropped. In
exchange 16 of the 21 slices stop depending on each other and can be
reviewed in any order.
Tests: v2 common/registry/api/redaction + transport, 75 tests /
445 assertions, green.
* Move the toucan-in-mcp kondo rule here, with the code it governs
The rule discourages toucan2.core outside metabase.mcp.db, and this branch is
what introduces resolve.clj routing its read through mcp.db/select-one-by-id.
Landing the rule with the slice that first followed it left a window where the
convention held in code but nothing enforced it.
Exemptions unchanged: metabase.mcp.db is the seam itself, and
metabase.mcp.models.* define the models it sits on top of. Warning rather than
error while v2/redaction.clj still violates it.
* Bring resolve_test and write_test along with their namespaces
This branch was shipping resolve.clj and write.clj with no tests -- including
readback's empty-read-scopes guard, which exists precisely because that
function had none and its fail-open path went unnoticed.
Tests: resolve-test + write-test + common-test, 17 tests / 51 assertions.
* Scope the toucan rule to source, and land the redaction fix with it
CI failed on six warnings, all in test namespaces. The pattern matched
namespace names beginning `metabase.mcp.`, which includes
`metabase.mcp.api-test` and its siblings. Tests call Toucan directly on
purpose -- fixtures, row assertions, counts -- and none of that is an
agent-supplied trust boundary, so the rule's reason for existing does not
apply to them. Excluded using the same `-test`/`test-util` convention the
file's own `test-namespaces` group uses rather than inventing another.
That left one real violation, v2/redaction.clj, and the fix for it already
existed on 02 (8791186195: the redaction queries moved into mcp/db.clj
behind an id-splice guard, with db_test.clj). Cherry-picked here, since a
rule that arrives already-violated teaches the next reader that violating it
is normal.
clj-kondo over src/metabase/mcp and test/metabase/mcp: 0 errors, 0 warnings.
Tests: db-test + v2 redaction/resolve/write/common, 25 tests / 82 assertions.
* Drop the toucan-in-mcp kondo rule; it belongs in the closing PR
Landing the rule at the bottom of the stack means every later slice that
calls Toucan directly fails the Clj-Kondo job when it merges into 01 --
warnings are fatal there. That is ~13 slices, each adding one new
tools/X.clj with a median of 2 t2/ call-sites.
The rule's value is stopping *new* violations, and during the merge window
nobody is writing new MCP namespaces; the code is already written. Enforcing
now converts clean merges into unscheduled refactors.
The redaction fix cherry-picked alongside it stays -- routing those queries
through mcp/db.clj behind an id-splice guard is worth having regardless of
whether a linter enforces the pattern.
Rule text and its exemptions are recorded on GHY-4416 for the closing PR,
which lands it after the burn-down with :level :error rather than :warning.
* Land common.clj's tests alongside the functions
The hoist moved the functions into 01 without their tests, leaving
select-fields, response-format, frontend-url and the list envelope with
no CI signal until slice 08 lands.
Cherry-picks the five deftests and the fields-test projection fixture
from mcpv2-08-tool-search; common_test.clj is now byte-identical to that
branch's version. 10 tests, 67 assertions, 0 failures.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0156QMhf6CLY92wtY3AfKepV
* Close three coverage blind spots in common_test
truncation-line had no direct test at all — it was only ever reached
through list-content, which never passes :param or :total-floor?. Both
are on the path parameters.clj takes on later slices, so the new
truncation-line-test pins the narrow-with sentence and the "at least N"
floor rendering, plus the two nil cases.
The unknown-path assertion in select-fields-test matched only
#"Unknown field path.*Nearest", so any three catalog entries would have
passed and the Levenshtein ranking — the sole use of the
LevenshteinDistance import — went unasserted. It now pins the leading
two suggestions ("nmae" -> name, collection); the third slot is left
open because collection_path and parameters.name are close enough that
pinning it would test the tie-break, not the ranking.
paths->tree's reduce across distinct top-level keys was unexercised —
the existing merge cases both walk one branch. Added the ordinary
fields: ["name","parameters.type"] shape.
Tests for the three bugs filed against this code (GHY-4420 the
total-floor? boundary, GHY-4421, GHY-4422) are deliberately left out:
they would assert today's buggy behavior, and each belongs with its
fix. The floor case here stays strictly truncated for that reason.
11 tests, 72 assertions, 0 failures.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0156QMhf6CLY92wtY3AfKepV
* Restore list-content's :empty-hint option ahead of its callers
The hoisted list-content computes its own empty-page sentence and takes
no :empty-hint. Six branches further up the stack (10-duplicate,
16-browse, 19-visualize, 20-oauth, 21-retire-v1, dashboard-write) carry
the design-C list-content that does, and the one caller is
list_databases in tools/browse.clj, which explains that browsing needs
query-builder or table-metadata permission on some database. Restacking
those onto this branch would hand that :empty-hint to a function that
ignores unknown map keys: no compile error, no failing test, the
sentence just disappears. Accepting the option here means the slices
land on a list-content that already honors it.
The two sentences are kept mutually exclusive by construction rather
than by coincidence of the current call site:
(or (empty-page-line opts) (when (= 0 total) empty-hint))
empty-page-line says "you paged past the end, N exist, page back";
:empty-hint says "there are genuinely zero results, and here is why".
Gating on (= 0 total) is not redundant with the or — without it, a
static hint could print over a paging situation and tell someone with
37 databases that none are visible to them.
The new list-content-empty-hint-test exists so that an incorrect
conflict resolution fails loudly instead of silently. It asserts the
verbatim browse hint text at total 0 (red if :empty-hint support is
dropped again), and that the hint does not displace the empty-page line
at total 37 / offset 100, nor the truncation line on a non-empty page
(red if the guard is "simplified" to a bare or). Both testing strings
name the browse.clj restack they guard.
12 tests, 82 assertions, 0 failures.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0156QMhf6CLY92wtY3AfKepV
* Gate list-content's :empty-hint on the offset, not just the total
`:empty-hint` is documented as the domain reason a genuinely empty result
set is empty, and the gate was meant to keep it away from a caller who
merely paged past the end. It only checked the total, so it did not.
`empty-page-line` requires `(pos? total)`, so at a total of 0 it declines
for every offset — including a nonzero one. The `or` then fell through to
the hint, and `list_databases` (which forwards the caller's raw offset
through `paged-list-content`) would answer "No databases are visible to
you" for someone who had actually paged past the end of an empty list.
The sentence stays true at a zero total, so this is a contract violation
rather than a false statement, but it is not what the comment claims.
Add the offset to the gate and pin it with a regression test. Verified
the test fails against the previous gate and passes against this one.
---------
Co-authored-by: Eric Normand <ericwnormand@gmail.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
* mcp-v2.09a prep: extract the documents Markdown round-trip layer (#81432)
* MCP v2 prep: extract the documents Markdown round-trip layer
Part 9a of the MCP v2 decomposition of #77844. This lands the documents
Markdown layer that the document_write tool (a later slice, 9b) is the first
consumer of, split off so it reviews as a serialization extraction with no MCP
surface. v1 and v2 are untouched; this slice adds no tool.
- NEW metabase.documents.markdown (+ its 929-line test): the CommonMark <->
ProseMirror AST boundary — parse/serialize/splice, {% card %}/{% entity %}
token scanning, escaping, a nesting-depth bound. Ported verbatim (byte-
identical to the source), so its injection / permission-leak / stack-overflow
hardening is preserved intact.
- metabase.documents.models.document: gains the write orchestration
(create-document!, update-document!, get-document, clone-cards-in-document!
and their card-creation helpers). The unrelated explorations content-gating
block (can-read?/can-write? overrides, the content-visibility cache) is
deliberately NOT ported — it is a different feature's drift in the same file
and would change document permission semantics. Two further exploration-
Summary threads (a stored_result_use.card_id column from an unlanded
migration; serdes snapshot-embed hardening) are likewise excluded.
- metabase.documents.api.document: card-creation logic moves out of the REST
layer into the model (519 -> 358 lines); the endpoints become thin wrappers.
The unchanged REST document api-test is the check (88 tests / 518 assertions).
- metabase.documents.prose-mirror / documents.core: prose-mirror gains
smart-link-model->db-model; the facade exports the new parse/serialize/splice
+ model fns.
- metabase.comments.core / comments.models.comment: the comment facade gains
child-target-ids-for-document / comments-for-document / register-context-gate!
(notify-comment-id! kept — a grep confirmed zero callers, and dropping a
public re-export is a one-way door).
Fresh-JVM: markdown-test 285 assertions; REST document api-test 518; :module
comments 154; all 0 failures. kondo 0/0; cljfmt clean; modules-config regenerated
(documents module gains :model/Database). The one :module documents error
(collection_test Duplicate key) is a pre-existing deterministic bug in a file
this slice does not touch, reproducing identically in isolation.
* Address review findings in the documents markdown layer
- splice gets the same StackOverflowError -> 400 backstop parse has: the
re-parse of the edited region runs through flexmark, whose inline parser
can blow the stack on a long delimiter run in the replacement text.
- A codeBlock :language holding a backtick (which a tilde fence's info
string legitimately parses into) now re-serializes behind a tilde fence
instead of silently breaking the fence line, which let fenced
{% card %} text re-parse as a real embed; a newline in :language
collapses to a space.
- inline-lines collects per-line chunks and joins once: appending to the
accumulated string was quadratic in a paragraph's inline node count
(40k nodes: 1.9s -> 156ms, and it scaled 4x per doubling before).
- attr-pos-long / format-number range-check before (long n), so an
integral double past long range is a teaching error, not a raw
IllegalArgumentException.
- update-document!'s :cards schema matches the REST DocumentUpdateOptions
it backs (:int keys, ignored without :document) instead of narrowing it.
- comments-for-document keeps a deleted parent that still has replies,
content scrubbed to {}, matching GET /api/comment.
- create-card! back to private (no caller outside the namespace); stale
smart-link docstrings and a broken namespace link cleaned up.
- Tests for all of the above, plus previously uncovered branches:
container content-model violations (incl. via splice), splice span
bounds and the insertion/separator-only arms, token-grammar teaching
errors, and the entity-token code-mark/unknown-model guards.
* mcp-v2.03 collection_write (#81247)
* MCP v2 tool: bookmark_content
Adds the v2 bookmark_content tool — per-user favorites, keyed by (type, id).
Extraction: bookmarks/api.clj factors bookmark!/un-bookmark!/bookmarked? out
of the REST endpoints so REST and MCP share them. The POST endpoint now
creates via app-db/select-or-insert! (concurrency-safe: a racing duplicate
returns the existing row instead of tripping the unique constraint); its 400
on already-bookmarked is unchanged. REST tests extended, existing pass.
Unlike REST, both tool directions are idempotent (see ns docstring for why),
both read-check the item, and the name echo is gated on the read scope so a
write-only token can't use it as a name oracle.
Tests: 9 tool tests + bookmarks REST suite green.
Review question for this PR: one tool — is bookmark_content's contract right?
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BjQcjQx7NPGr3Rpf6QaG4b
Also lands (first consumer, per the common.clj slimming in the core PR):
- NEW v2/resolve.clj — entity-id / numeric-id resolution + the read-check
pairing with the not-found collapse (+ resolve_test, moved from common_test).
- NEW v2/write.clj — readback (the scope-gated write echo).
CI guard: this slice's mcp -> bookmarks edge put `bookmarks` into the
modules-that-trigger-driver-tests set (mage's module-graph ratchet, 44 -> 45).
Bookmark logic cannot affect driver behavior — the edge is a consumer edge —
so `bookmarks` joins driver-affecting-overrides rather than bumping the
ratchet. Full mage suite green (82 tests).
Review nit (round 1): fetch-item's flavor-mismatch teaching error printed
"is a null" for a card type outside question/model/metric; guard the nil case
with a type-agnostic message.
--- Round-2 /review-pr response ---
IMPORTANT (per-user-test could not fail on its named assertion) FIXED. The test called the tool
bare, so a tool-level error was discarded and the assertions passed regardless. It now routes
both calls through tool-result (throws on tool error) and has :lucky bookmark-then-un-bookmark
its own row, so the "one user's bookmark is invisible to another's un-bookmark" claim has a live
counterfactual. Proven fail-when-broken (stubbing un-bookmark! to a no-op turns the test red).
MINOR fixed: the id schema now requires a positive int (matching REST's ms/PositiveInt); the
degraded write-scope readback now surfaces :bookmarked (via a new backward-compatible ack-keys
arg on v2.write/readback, so a write-only caller still learns the resulting state).
MINOR deferred (pre-existing REST behavior, out of a tool-slice's scope, confirmed structurally):
archived content is bookmarkable but filtered from listings; the Trash collection is bookmarkable;
readback's :url key has no producer on this branch (lands with collection_write). Each is shared
with the existing REST bookmark endpoints; changing it means deciding REST semantics.
Fresh-JVM: bookmark-test 9 tests / 43 assertions / 0 failures; kondo 0/0; modules unchanged.
* MCP v2 tool: collection_write
Adds the v2 collection_write tool — create/update/move/archive collections.
Extraction: collections/update.clj lifts update-collection! out of
PUT /api/collection/:id — the four privates (move-collection!,
archive-collection!, move-or-archive-collection-if-needed!,
maybe-send-archived-notifications!) move verbatim (docstrings de-REST-ified,
notification alias renamed); the endpoint becomes a thin delegator through
the collections.core facade, mirroring the existing create-collection!
pattern. Verified move: bodies diff clean modulo whitespace/docstrings.
The foundation's REST-test changes for this file are all the BROWSE
extraction's repoints, so master's collections_rest api-test runs UNCHANGED
against this branch — that unchanged suite is the extraction check
(fresh-JVM: 202 tests, 941 assertions, green, incl. 23 tool tests).
Deferred to the content tool PR: echo-matches-concise-read-test (the write
echo vs get_content cross-check) — it requires tools/content.
Review question for this PR: one tool — is collection_write's contract right?
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BjQcjQx7NPGr3Rpf6QaG4b
Also lands (first consumer, per the common.clj slimming in the core PR):
- v2/resolve.clj grows collection resolution (resolve-collection-id{,-or-personal}).
- v2/write.clj grows `_write` method dispatch (dispatch-write + expand-clear).
- common.clj grows frontend-url. Tests moved alongside (resolve_test, write_test).
Review nits (round 1): fix the self-contradictory `name` field description
("Create only (editable on update)" -> "Required on create; editable on
update"); drop structuredContent from the write echo to match the stack's
text-only convention (no concrete programmatic consumer, no outputSchema).
CI kondo fix: the update-collection! extraction moved the only `tru` caller in
collections_rest/api.clj into collections/update.clj, leaving a dead
`[metabase.util.i18n :refer [tru]]` require — an unused-require warning, which
`mage kondo` (CI's check) exits nonzero on. Removed it.
--- Round-2 /review-pr response ---
IMPORTANT (archived:true + parent_id silently discarded the move) FIXED. The dispatch tested
archived first and returned, so a caller passing both got a trashed collection and no move, with
no signal — defeating the tool's "never believes an ignored field took effect" invariant. It is
now a teaching error naming the fix (move first then archive, or pass parent_id on a later
archived:false restore). Regression test added.
MINOR fixed: the four (is (tool-error ...)) assertions now match the specific expected message,
not any error text; the :name destructure was renamed (coll-name) so it no longer shadows
clojure.core/name.
Fresh-JVM: collection-test 24 tests / 65 assertions / 0 failures; kondo 0/0; modules unchanged.
--- Close-out M5 ---
MINOR (bare tool-error assertion): authority-level-update-test now asserts the specific "paid feature" message (parity with the create-side twin), so a wrong error text fails.
* Route resolve.clj through mcp.db; correct the db.clj docstring
Three fixes from review of the mcpv2-01-core merge resolution.
resolve.clj called t2/select-one directly, so the toucan2 kondo rule this
same change adds shipped already violated by a file it introduces -- and at
the highest-value site, where an agent-supplied id enters a query. Adds
mcp.db/select-one-by-id, which asserts the id is already an integer (entity
ids are resolved before they reach it, so a string there is a bug rather
than a lookup), and routes resolve.clj through it. The rule now flags only
v2/redaction.clj, which predates this branch.
bookmarks/db.clj's ns docstring still claimed every function was "a direct
Toucan 2 call with no additional logic" after the generic dispatch tier
moved in, which is the one part of that docstring a reader relies on.
insert-bookmark!'s unique-constraint rationale moves from its docstring to
an inline comment on the select-or-insert! call it explains.
Tests: bookmarks.api-test + mcp.v2.tools.bookmark-test + mcp.v2.resolve-test,
30 tests / 150 assertions, green.
* readback: refuse an empty read-scopes instead of skipping the gate
`(remove f [])` and `(remove f nil)` both yield `()`, so an empty or nil
`read-scopes` made `missing` empty and returned the full ungated row -- a
gate that did not run was indistinguishable from one that ran and passed.
Not reachable from `bookmark_content`, which passes a literal one-element
vector. It matters because this namespace is shared write-path machinery for
the `_write` tools still to land, and the mistakes a future caller actually
makes -- omitting the argument, or computing `read-scopes` from a per-type
map that misses -- both landed on silently ungated rather than an error.
There is no legitimate empty case, so it throws.
Adds write_test.clj, which the namespace did not have: full-row grant,
degradation (exact key set, so `:name` cannot leak), empty and nil
`read-scopes` refusal, the unrestricted sentinel and nil `token-scopes`,
note pluralization, and an ack-key absent from the row. Verified the
refusal test bites -- removing the guard fails it twice (`[]` and nil) and
leaves the other five green.
Also drops `readback`'s 3-arity, which had no callers, and the ns
docstring's reference to which slice lands next.
Tests: write-test + tools.bookmark-test + resolve-test + bookmarks.api-test,
36 tests / 161 assertions, green.
* Clean up comments in resolve.clj
Removed unnecessary comments for clarity.
* Drop the stray toucan-in-mcp kondo rule from this branch
The rule entered here through the mcpv2-01-core merge resolution rather than
from either parent, and it now lives on mcpv2-00b-shared-infra next to the
mcp/db.clj boundary it governs. Keeping a second copy here only produced seven
false positives on MCP test namespaces, whose toucan2.core requires are fixture
setup rather than the agent-id boundary the rule protects.
Every src/metabase/mcp namespace that touches toucan2.core is already exempt
(db.clj and models.*), so the rule guards nothing on this branch. It comes back
with the shared infra when the stack restacks.
* Route collections/update.clj's app-DB queries through collections/db.clj
The collections module keeps its app-DB queries in db.clj; update.clj still
reached for toucan2 directly. `update!` and the final re-read map onto the
existing `update-collection!` and `collection`; the archived-card sweep and the
move's parent lookup needed new fns, mirroring the query shapes
collections-rest/db.clj settled on.
`t2/hydrate` stays in update.clj — it hydrates the value api/write-check
returns rather than fetching one, matching how measures, segments, and
dashboards handle hydrate after the same refactor.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FZzZhTfVjp6ZGe9fBfK997
* Collapse parent_id's not-found and no-access answers
`resolve-collection-id` probed existence with no read check and let the
downstream write-check throw, so a token holding only agent:content:write could
tell a nonexistent collection id ("not found") from one it merely cannot write
("You don't have permissions"), and enumerate collection ids across the
permission boundary. POST /api/collection answers 403 either way. The `id`
argument already collapsed both through `resolve-and-read`; `parent_id` now
takes the same path, which keeps the bogus-id guard the probe was there for.
Two existing perms tests asserted the old behavior by pointing at another
user's personal collection; they now use a readable-but-not-writable collection,
so they still prove the write check fires without depending on the leak. The
unreadable-parent case is what the new test covers.
Also repairs this namespace's tool-result/tool-error helpers, stale since
c521e4c71c changed call-tool to return {:result}/{:error}. That commit updated
bookmark_test.clj but reached collection_test.clj through a merge with no
textual conflict, leaving every error assertion in the namespace failing.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FZzZhTfVjp6ZGe9fBfK997
* Drop two collections-rest db helpers left without callers
`cards-in-collection` and `collection-location-columns` each had exactly one
caller, inside the two private fns this branch moved to
`metabase.collections.update`. The collections module cannot require
collections-rest, so update.clj carries its own copies; these are dead.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FZzZhTfVjp6ZGe9fBfK997
* Refuse a parent_id move that would leave a collection in the trash
Archiving sets only archived/archived_directly/archive_operation_id, so a
trashed collection keeps its pre-trash :location. An update passing parent_id
without archived therefore skipped the archive branch, moved the collection
against that stale location, and echoed the new parent while the collection sat
in the trash — nothing appeared under the new parent.
`archived: true` + parent_id was already rejected; this is the same
contradiction from the other side, so it gets the same teaching error, pointing
at the `archived: false` + parent_id restore path that already works.
The check needs the collection's current state, so it can't join the pure
arg checks in check-method-args!. It takes the collection resolve-existing
already returned rather than fetching its own.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FZzZhTfVjp6ZGe9fBfK997
* Pin the write-echo degradation and the collection-id not-found collapse
Both properties were load-bearing and unasserted. The scope-gating test drove a
write-only token but checked only that :id was an int, which holds whether or
not the readback gate runs — dropping the gate, or widening its ack-keys to
leak :name, failed nothing. resolve-collection-id's main branch had no coverage
at all; the file tested only the nil/"root"/"trash" sentinels.
Each test was verified by mutation: removing the readback call reddens the
degradation test on both methods, and reverting the id collapse to
resolve-id-or-404 reddens the not-found test on the message-equality assertion
that is the actual security property.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FZzZhTfVjp6ZGe9fBfK997
* Close collection_write's namespace argument to the one namespace it documents
Any non-blank string was accepted and passed straight through to the create.
assert-valid-namespace only checks a collection against its parent, so a
root-level create with a typo passed: namespace "snippet" produced a collection
in a hierarchy nothing surfaces, and the echo confirmed the typo back — the one
thing the tool's contract says must never happen.
[:enum "snippets"] mirrors authority_level's [:enum "official"] two entries
below, and puts the allowed value in the tool schema where most clients will
reject a typo before it reaches us. transforms is left out deliberately: those
collections are created by the transforms machinery and carry its lifecycle.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FZzZhTfVjp6ZGe9fBfK997
---------
Co-authored-by: Eric Normand <ericwnormand@gmail.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
* [mcp-v2 09b] document_write tool
Second half of split slice 09 (DEC-0001). 09a landed the documents Markdown
round-trip layer; this adds the thin MCP tool that drives it.
Review question: does the write tool enforce document-level write permission
before it reaches the docstring-only-guarded model fns, and does it refuse to
launder a foreign card into a document the caller cannot write?
Surface:
- src/metabase/mcp/v2/tools/document.clj — the document_write handler.
create! runs api/create-check before documents/create-document!.
update! runs read-check (resolve-and-read-with) -> check-not-archived ->
api/write-check -> validate-collection-move-permissions, all before
documents/update-document!. Card-embed and smart-link resolution are
permission-checked (mi/can-read?); a foreign readable card is cloned, never
referenced; replace_all is bounded by max-replace-all-work.
- :document projection registered here (first mcp consumer); content/11 drops
its duplicate when it lands.
Inherited 09a-review finding (closed): the documents.core model fns enforce
permission by docstring only. 09b is their first direct caller, so the tool
carries the checks. permission-denied-write-leaves-no-db-trace-test asserts DB
state (no Document row, no cloned Card) on a refused write, not just :isError;
verified fails-when-broken by removing api/write-check.
Deferred tests (restore with slice-11/content): create-clones-card-test,
write-echo-and-read-name-the-body-alike-test.
Gates: document-test 80 assertions, :module mcp 5280 assertions, kondo 0,
cljfmt clean, modules config regenerated (mcp :uses comments+documents,
comments :api gains comments.core).
--- Correctness fix (round-1 review, critical) ---
apply-edit/replace-all spliced new_str into Markdown source as-is. splice re-parses the
edited region, so a replacement like "*" or a leading "#" reopened the block as a list or
heading and could drop a character (the "*" consumed as a list marker) — a text-only edit
silently corrupting structure and losing data. Reproduced: body "x x x x x" + edit x->*
replace_all yielded a bulletList with "* * * *" (4 not 5).
Fix: new public documents/escape-text (markdown.clj) escapes a raw string to its literal
form for any position — inline emphasis/link/code chars per line, plus a per-line leading
block-start escape. apply-edit escapes new_str before splicing; over-escaping a char not in
a special position is inert (CommonMark renders \# as #). splice itself is unchanged (its
markdown-source contract is load-bearing for heading/card-embed inserts — verified 09a
markdown-test stays green). New edit-replacement-is-literal-text-not-markdown-test locks it,
proven fails-when-broken.
* Resolve user smart links under the mention picker's visibility rule
`smart-link-readable?` let any non-sandboxed caller resolve any `:model/User` row, and the stored
label falls back to `:common_name` — the email address when a user has no name. A document author
could therefore enumerate names and emails across tenants, deactivated accounts and API-key
accounts included, by writing `{% entity model="user" %}` tokens and reading the labels back.
User rows are now selected the way `GET /api/comment/mentions` and `GET /api/user/recipients`
select them: a superuser sees everyone; anyone else sees active personal accounts in their own
tenant, narrowed by the `user-visibility` setting. Tests cover a deactivated user and
`user-visibility :none`. Modules config regenerated (mcp now uses users).
* Don't escape an edit that lands inside code
`apply-edit` escaped `new_str` unconditionally so a replacement like `*` or a leading `#` could not
reopen the block as a list or heading. That reasoning holds in prose and inverts inside code:
backslashes are literal in a code span or fenced block, so the escaping stored characters the caller
never wrote. `{old_str "ls", new_str "my_var"}` against "run `ls -la` now" stored
"run `my\_var -la` now" — verified by driving apply-edit before the fix. Snake_case identifiers in
code are the common case.
Escaping is now decided per match: `in-code-context?` counts unescaped backticks and fence lines
before the offset, and `replace-all` prices each occurrence against the serialization it is splicing,
so one match in prose and another in a code span each round-trip correctly.
Claude-Session: https://claude.ai/code/session_018PNCQxp9kCypqE3dLQS9va
* Enable the feature the user-visibility assertion depends on
The smart-link test narrowed `user-visibility` to `:none` and asserted that another user's name stops
resolving. But `user-visibility` is gated on `:email-restrict-recipients`, so in an OSS build the
setting reads `:all` whatever is stored: the narrowing never happened and the assertion pinned the
default instead of the rule. It passed locally and failed every OSS App DB CI job.
Wrapped in `mt/with-premium-features`, matching how the other `user-visibility` tests set it up.
Claude-Session: https://claude.ai/code/session_018PNCQxp9kCypqE3dLQS9va
* Fix document_write Markdown table handling
Reject GFM tables in full document bodies and edit replacements before persistence. Advertise the limitation in the tool description.
* fix(mcp): parse document edits as Markdown
* Budget document_write's edit work per call, not per edit
`max-replace-all-work` was checked once inside `replace-all`, so every entry
in `edits` got its own fresh allowance. `edits` is an unbounded list, which
makes that no bound at all: 100 edits each priced just under the ceiling buy
100x the work in one request.
The ceiling now covers the call. `apply-edit` takes and returns
`{:ast :spent}`, pricing each edit against the total the earlier ones have
already spent. It has to be spent down incrementally rather than totalled up
front — an edit's match count is only knowable against the text its
predecessors produced.
Plain edits are priced too. A single splice still re-serializes the whole
document, so `edit-work` counts splices (one for an edit, one per occurrence
for a `replace_all`) and both kinds draw on the same budget.
`replace-all` no longer serializes to price itself. It takes the
serialization and matches `apply-edit` already computed, which drops a full
redundant serialization per `replace_all` and makes its docstring's "one
serialization rather than one per match" true.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016fD5YPdm6TNR2WfU3k7MMB
* Follow collection_write's contract changes into document_write
Slice 03 changed two shared contracts under this branch. Both merged clean
and both were invisible to a targeted kondo run, which cannot resolve vars
across namespaces.
`v2.write/readback` gained an `ack-keys` parameter — caller-supplied keys
that survive when the response degrades for a token without the read scope.
`document_write` has none: nothing it echoes back is caller-supplied rather
than read-gated, so it passes nil, as collection_write does.
`registry/call-tool` now returns `{:error …}` when the registry refuses
before dispatch and `{:result mcp-content}` after the handler runs, so
`:isError` and `:content` moved under `:result`. The test helpers still read
them at the top level, which made every refusal look like a success and hid
both budget tests behind a vacuous pass. `error-text` reads either shape from
one place: a schema violation refused at the boundary carries `:error`, a
handler's teaching error carries `:result {:isError true}`.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016fD5YPdm6TNR2WfU3k7MMB
* Record a recent view when a document is created
`command palette > should display collection names for documents in recents`
was failing on this branch (2 of 2 runs), passing 5/5 on master with a
byte-identical spec.
The documents extraction swapped `get-document` for `read-check` +
`hydrate-document` on the create path, so `:event/document-read` stopped
firing. That event's handler is the only thing that writes the `RecentViews`
row (documents/view_log.clj), and the command palette surfaces a just-created
document only through recents — the spec POSTs a document and never visits it.
Creating is arguably not reading, but recents rides on the same event, and
UXW-1786 expects a new document to be findable. Restored `get-document` and
dropped `post-document-does-not-record-view-test`, which pinned the new
behaviour; it and the e2e spec cannot both hold. The POST/PUT symmetry that
motivated it does not apply: PUT never recorded a view on master, POST always
did.
Added `post-document-records-a-recent-view-test` beside the GET/PUT view tests
so the expectation is asserted at the API level — the Cypress spec reports an
empty palette without saying why.
Decoupling recents from `:event/document-read` would let view counts stop
treating creation as consumption; that is a separate change.
* Make the session-signature tests capable of failing
The two signature assertions in `mcp-ui-capability-claim-must-be-signed-test`
passed for the wrong reason, so they proved nothing about signing.
`#"\\."` matches a literal backslash followed by any character, not an escaped
dot, so the split was a no-op: `uuid` received the whole session id and
`payload`/`sig` were nil. The reconstructed ids were malformed and rejected
before signature verification ever ran. Replacing `session-payload-signature`
with a constant left all six assertions green.
Fixing the split alone was not enough. The tampering case re-encoded the SAME
payload onto an already `ui true` id, so it rebuilt byte-identical and was
believed for the right reason. It now mints a `ui false` id and swaps in a
`ui true` payload under the original's signature, which is the escalation the
signing exists to stop.
Guards added so this cannot silently rot again: the splits assert they reached
a signature, the lifted signature asserts it differs from the original, and a
control asserts the unmodified id claims nothing.
Verified by mutation: with `session-payload-signature` stubbed to a constant
the test now fails 3 assertions, where it previously passed clean.
* Pin the transform search gate without writing a global setting
transform-visibility-is-superuser-only-test wrapped its body in
(mt/with-temporary-setting-values [transforms-enabled true]
(mt/with-premium-features #{:transforms-basic} ...))
which left `transforms-enabled` explicitly set for the rest of the JVM, and
took metabase.metabot.tools.transforms-test and metabase.revisions.api-test
down with it on every backend shard.
`with-temporary-setting-values` does not restore a setting to *unset*. It
captures `(setting/get setting-k)` on entry and writes that value back in its
`finally` (test/metabase/test/util.clj:552-609). `transforms-enabled` is unset
here, so the capture runs its getter's `cloud-enabled-transforms?` fallback
((and (is-hosted?) (has-feature? :transforms-basic))) and yields a computed
boolean rather than nil. Restoring that boolean persists a real Setting row,
so `explicit-transforms-enabled` stops returning nil and the fallback every
other transform test relies on is dead for the remainder of the run.
The capture also happened before any premium features were stubbed -- the
setting form was the outer one -- so it read ambient state, not
#{:transforms-basic}.
Downstream that lands on the permission check: `transform-readable?`
(src/metabase/transforms/models/transform.clj:35-42) gates on
`check-feature-enabled` *ahead* of the `api/*is-superuser?*` bypass, and that
gate begins with `(setting/get :transforms-enabled)`
(src/metabase/premium_features/token_check.clj:715-730). Once the setting is
pinned false, even admin callers are denied, which is why the revisions test
saw 403 on its superuser case and the metabot transform tools returned
"You don't have permissions to do that.".
Pin the gate at its actual seam instead: redefine
`transforms.gating/enabled-source-types`, which is the single place
search/impl.clj:364 reads it from. That covers what the test needs -- the
in-place engine drops transform for everyone when the enabled set is empty --
without touching persisted state, so there is nothing to restore and nothing
to leak.
Adding `:hosting` to the feature set would also have made the leaked value
harmless, but it only changes *which* value leaks; the setting would still
come back explicitly set. The underlying defect in
`do-with-temporary-setting-value!` (capturing the computed value rather than
the raw stored one) is left for a separate change -- it has ~229 call sites
and wants its own audit.
* Move the document comment queries into comments.db
Master added a clj-kondo hook requiring Toucan2 query calls to live in
`metabase[-enterprise].<module>.db`. This branch had added two `t2/select`
calls to `comments/models/comment.clj`. Neither side fails alone; merging the
new rule with the existing calls turned CI's Clj-Kondo job red with two
warnings.
`document-comments` and `document-child-target-counts` now live in
`comments/db.clj` alongside the module's other queries. The model functions
keep their logic — the deleted-comment scrub, `:creator` hydration, the
`select-keys` projection — and delegate only the query.
Full-tree kondo: 0 errors, 0 warnings; no other file violates the new rule.
:module comments 174 assertions, 0 failures / 0 errors. :module documents
1408 assertions with only the two pre-existing `collection-test` errors, which
predate this change.
* Move document_write's Toucan calls into mcp.db
#82099 added `:metabase/t2-query-namespace`, which confines application
database query calls to `metabase[-enterprise].<module>.db`. It arrived with
the master merge and made this file the only violator in the tree — three
warnings, and CI requires zero.
`select-by-ids` covers the two `:in` lookups and mirrors the existing
`select-one-by-id`, which already takes a dynamic model.
`select-users-where` takes an assembled HoneySQL clause, which is looser than
the "direct Toucan 2 call with no additional logic" the namespace documents.
The clause is built from the `user-visibility` setting, group membership and
the superuser flag — it decides which users the caller may resolve at all, so
it is a permission decision. Moving it into a db namespace would hide it and
pull those dependencies in. #066c63b88ee, the comments move on this branch,
set the precedent: the logic-bearing fn stays, only the query moves.
`select-by-ids` deliberately has no `{:pre [(seq ids)]}` despite
`select-one-by-id` guarding its argument. An AssertionError is not an
Exception, so it would bypass the `catch Exception` in `smart-link-rows` and
turn a logged warning into a propagated failure — a behavior change this
relocation should not smuggle in. Callers guard the empty case instead.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016fD5YPdm6TNR2WfU3k7MMB
* mcp-v2.04 prep: portable-dialect execution rework in the agent lib (#81248)
* MCP v2 tool: bookmark_content
Adds the v2 bookmark_content tool — per-user favorites, keyed by (type, id).
Extraction: bookmarks/api.clj factors bookmark!/un-bookmark!/bookmarked? out
of the REST endpoints so REST and MCP share them. The POST endpoint now
creates via app-db/select-or-insert! (concurrency-safe: a racing duplicate
returns the existing row instead of tripping the unique constraint); its 400
on already-bookmarked is unchanged. REST tests extended, existing pass.
Unlike REST, both tool directions are idempotent (see ns docstring for why),
both read-check the item, and the name echo is gated on the read scope so a
write-only token can't use it as a name oracle.
Tests: 9 tool tests + bookmarks REST suite green.
Review question for this PR: one tool — is bookmark_content's contract right?
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BjQcjQx7NPGr3Rpf6QaG4b
Also lands (first consumer, per the common.clj slimming in the core PR):
- NEW v2/resolve.clj — entity-id / numeric-id resolution + the read-check
pairing with the not-found collapse (+ resolve_test, moved from common_test).
- NEW v2/write.clj — readback (the scope-gated write echo).
CI guard: this slice's mcp -> bookmarks edge put `bookmarks` into the
modules-that-trigger-driver-tests set (mage's module-graph ratchet, 44 -> 45).
Bookmark logic cannot affect driver behavior — the edge is a consumer edge —
so `bookmarks` joins driver-affecting-overrides rather than bumping the
ratchet. Full mage suite green (82 tests).
Review nit (round 1): fetch-item's flavor-mismatch teaching error printed
"is a null" for a card type outside question/model/metric; guard the nil case
with a type-agnostic message.
--- Round-2 /review-pr response ---
IMPORTANT (per-user-test could not fail on its named assertion) FIXED. The test called the tool
bare, so a tool-level error was discarded and the assertions passed regardless. It now routes
both calls through tool-result (throws on tool error) and has :lucky bookmark-then-un-bookmark
its own row, so the "one user's bookmark is invisible to another's un-bookmark" claim has a live
counterfactual. Proven fail-when-broken (stubbing un-bookmark! to a no-op turns the test red).
MINOR fixed: the id schema now requires a positive int (matching REST's ms/PositiveInt); the
degraded write-scope readback now surfaces :bookmarked (via a new backward-compatible ack-keys
arg on v2.write/readback, so a write-only caller still learns the resulting state).
MINOR deferred (pre-existing REST behavior, out of a tool-slice's scope, confirmed structurally):
archived content is bookmarkable but filtered from listings; the Trash collection is bookmarkable;
readback's :url key has no producer on this branch (lands with collection_write). Each is shared
with the existing REST bookmark endpoints; changing it means deciding REST semantics.
Fresh-JVM: bookmark-test 9 tests / 43 assertions / 0 failures; kondo 0/0; modules unchanged.
* MCP v2 tool: collection_write
Adds the v2 collection_write tool — create/update/move/archive collections.
Extraction: collections/update.clj lifts update-collection! out of
PUT /api/collection/:id — the four privates (move-collection!,
archive-collection!, move-or-archive-collection-if-needed!,
maybe-send-archived-notifications!) move verbatim (docstrings de-REST-ified,
notification alias renamed); the endpoint becomes a thin delegator through
the collections.core facade, mirroring the existing create-collection!
pattern. Verified move: bodies diff clean modulo whitespace/docstrings.
The foundation's REST-test changes for this file are all the BROWSE
extraction's repoints, so master's collections_rest api-test runs UNCHANGED
against this branch — that unchanged suite is the extraction check
(fresh-JVM: 202 tests, 941 assertions, green, incl. 23 tool tests).
Deferred to the content tool PR: echo-matches-concise-read-test (the write
echo vs get_content cross-check) — it requires tools/content.
Review question for this PR: one tool — is collection_write's contract right?
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BjQcjQx7NPGr3Rpf6QaG4b
Also lands (first consumer, per the common.clj slimming in the core PR):
- v2/resolve.clj grows collection resolution (resolve-collection-id{,-or-personal}).
- v2/write.clj grows `_write` method dispatch (dispatch-write + expand-clear).
- common.clj grows frontend-url. Tests moved alongside (resolve_test, write_test).
Review nits (round 1): fix the self-contradictory `name` field description
("Create only (editable on update)" -> "Required on create; editable on
update"); drop structuredContent from the write echo to match the stack's
text-only convention (no concrete programmatic consumer, no outputSchema).
CI kondo fix: the update-collection! extraction moved the only `tru` caller in
collections_rest/api.clj into collections/update.clj, leaving a dead
`[metabase.util.i18n :refer [tru]]` require — an unused-require warning, which
`mage kondo` (CI's check) exits nonzero on. Removed it.
--- Round-2 /review-pr response ---
IMPORTANT (archived:true + parent_id silently discarded the move) FIXED. The dispatch tested
archived first and returned, so a caller passing both got a trashed collection and no move, with
no signal — defeating the tool's "never believes an ignored field took effect" invariant. It is
now a teaching error naming the fix (move first then archive, or pass parent_id on a later
archived:false restore). Regression test added.
MINOR fixed: the four (is (tool-error ...)) assertions now match the specific expected message,
not any error text; the :name destructure was renamed (coll-name) so it no longer shadows
clojure.core/name.
Fresh-JVM: collection-test 24 tests / 65 assertions / 0 failures; kondo 0/0; modules unchanged.
--- Close-out M5 ---
MINOR (bare tool-error assertion): authority-level-update-test now asserts the specific "paid feature" message (parity with the create-side twin), so a wrong error text fails.
* MCP v2 prep: portable-dialect execution rework in the agent lib
The shared machinery behind every portable-MBQL write path — metabot's
construct_notebook_query today, and the v2 MCP tools (metric_write,
question_write, execute_query) as they land on the stack:
- metabot/tools/construct: execute-representations-query reworked (new
signature; numeric-id source detection; recovery-hint wrapping around
resolution failures).
- NEW metabot/tools/recovery_hints: the standard 'what to call next'
sentences appended to dialect teaching errors.
- agent-lib representations{,.repair}: the corresponding repair/resolve
pipeline changes (resolve.clj needed nothing beyond master's).
Master's post-fork pMBQL→'MBQL 5' doc/message renames are re-applied on
top of the pulled files (wholesale pulls from the foundation would have
reverted them).
Behavior check: master's agent-lib suites (repair_test, resolve_test) and
metabot tool tests run UNCHANGED and green in a fresh JVM.
Review question for this PR: does construct_notebook_query (the existing
metabot consumer) behave identically where not deliberately extended?
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BjQcjQx7NPGr3Rpf6QaG4b
Review fixes (adversarial review round 1):
- F1: the agent-api /v2/construct-query endpoint and the slackbot query tool
called execute-representations-query without :recovery-hint, so both v1
surfaces relayed bare pipeline errors where slice 03 had baked guidance into
the throw sites. Wired recovery-hints/recovery-hint into both (as the ns
docstring already claims every v1 entry point does). Proven: a URI-in-
source-table error now carries the "use the portable FK" sentence on the
agent-api path.
- F4: node-table-fks never collected a numeric field ref's target table, so it
was never permission-checked; the no-fk-path error then named it from the
un-sandboxed metadata provider — a table-name oracle for any probed id once a
numeric binder lands. Fall back to the bare id for the numeric case (matching
the vector case's own-name behavior). Dark today (no binder on this slice),
fixed before it can go live.
Test fix: the F1 change gave slackbot's execute-representations-query call a
second (:recovery-hint) arg, but slackbot_query_test stubbed the fn with 1-arg
lambdas — the extra arg broke the stubs (nil results). Made the 4 stubs
variadic ([external-query & _]) so they accept the opts map; production change
unchanged. Suite green (53 assertions).
--- Round-2 /review-pr response ---
CRITICAL (numeric field refs bypass the pre-repair permission check) FIXED. A bare numeric field
id in a query node was never resolved to its table, so node-table-fks skipped it and query-check
never gated the field's table. node-table-fks now emits the source-field as a tagged numeric ref;
resolve-table-fk resolves it to the field's table via the permission-aware metadata provider and
checks that table. Two regression tests: one asserts the FK-target table is permission-checked,
one asserts a forbidden FK-target table blocks the call.
CRITICAL (export-source-field-portable writes an un-permission-checked FK path) REFUTED, and the
refutation is closed by the fix above. The exported source-field is the FK-BEARING column, which
by definition lives on the source table — the same table node-table-fks now collects and
query-check gates. Verified in a fresh JVM: the exported field resolves to a column on the source
table, not the FK-target; denying perms on the target raises 403 before export runs. The reviewer
conflated the FK-target table with the FK-bearing column.
IMPORTANT fixed: the recovery_hints ns docstring no longer asserts wiring that does not exist (it
now states the v2 query slice must pass it, and that until then it has no production caller); the
:no-fk-path message and the :unknown-table key collision are corrected. MINOR fixes applied.
Fresh-JVM: construct-representations-test + repair-test 228 tests / 616 assertions / 0 failures;
kondo 0/0; modules unchanged.
* Read-check the numeric source-card unconditionally
The numeric `source-card:` branch guarded its read check on
`api/*current-user-id*` being bound, so a caller with no current user got the
card's database id back without any permission check. Its portable sibling,
`tools.u/get-card-by-entity-id`, read-checks unconditionally — two branches of
one function disagreeing on whether permissions apply.
The guard was not there to avoid a crash: `read-check` -> `mi/can-read?`
resolves Card perms against the current-user permission set and denies when
there is no user, so removing the guard fails closed.
Not reachable in production yet — nothing binds `*numeric-ids-allowed?*` to
true until the v2 query slice lands — but this branch is where the dialect is
authored, so it is where the check belongs.
Adds the numeric `source-card` coverage the other numeric branches already
had: database-id resolution, that the card row reaches `read-check`, a denied
card surfacing 403, the same with `*current-user-id*` nil as the regression
guard, and an unknown id surfacing `:unknown-card`. Also covers two branches
that had none: `:unknown-table-id` for a numeric `source-table` that resolves
to nothing (kept distinct from the portable form's `:unknown-table`, since the
two want different recovery vocabulary), and `with-recovery-hint` passing an
error through untouched when there is no hint to add.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Nng1SV562GGqn1Qf8dQmpM
* Route numeric source-card through the read-checked content store
The numeric-id dialect accepts a bare numeric `source-card`, but
`import-mbql` matched only `portable-id?`, which requires a string. A
numeric id therefore never reached the ContentStore -- and on agent
paths the store is the `read-checked` one, so that lookup is where the
permission check lives. The only remaining check was the explicit
`read-check` in `resolve-database-id-from-first-stage`, which by
construction sees `stages[0]` only, and which a stage carrying both
`source-table` and `source-card` skips entirely (the `cond` takes the
`source-table` branch).
Downstream, the repair pass's cross-stage column inference runs for any
stage with a `source-card` and reports the card's column names when a
referenced column is missing, so an unchecked card surfaced its columns
to a caller without access to it.
Fix at the resolution chokepoint rather than at the first stage:
`import-mbql` now has a numeric `source-card` branch, gated on
`*numeric-ids-allowed?*` so the portable-only surface is untouched.
Because that walk recurses, this covers every stage and every join
instead of just the one the database-id lookup happens to inspect. The
new `import-card-by-id` mirrors `import-card-by-entity-id`, including
its cross-database guard, and returns the id it was handed -- the
routing, not the translation, is the point.
Also drop the `(when api/*current-user-id* ...)` guard on the numeric
card's `read-check` in `construct.clj`. The entity_id branch checks
unconditionally and the numeric table branch fails closed; only this one
failed open for an unauthenticated caller.
Verified against a real appdb with an unprivileged user: both shapes now
raise 403 with no column names in the message or ex-data, where before
they resolved and reported them. The new tests fail on the unfixed tree
and pass here.
Test evidence: resolve.mp-test 41 tests / 136 assertions;
construct-representations + repair + resolve 249 tests / 707 assertions;
metabot tools 28 tests / 386 assertions; new
construct-numeric-permissions-test 4 tests / 11 assertions. All green.
* Require numeric ids to denote something the query may reference
Three findings from the slice-04 review, all the same bug: the numeric
dialect treats an LLM-authored integer as a resource reference and acts
on it directly. The portable dialect never had to state these rules --
it resolves *names* through a metadata provider that is already scoped
to the query's database and filtered to visible rows, so "in this
database" and "a column the warehouse exposes" fall out of resolution.
A bare id skips all of it.
`visible-table` / `visible-field` put the contract back: everything the
numeric dialect accepts is looked up through the query's own provider,
and the active/visibility rules are re-applied on top, because
`metadata-spec->honey-sql` deliberately drops its filter for by-id
lookups (`active-only?` is `(not (or id-set name-set))`). That default
is right for trusted callers asking for a specific row and wrong for an
agent-authored id.
1. Cross-database tables. `resolve-table-fk` returned a `pos-int?`
verbatim, so a join could pull a table from another database into a
query. `api/query-check` does not close this: it asks whether the
caller may query table N, never whether N belongs to this query's
database, so it passes for anyone with access to the other database.
Traced on the unfixed tree, the check ran against the foreign table
(id 9, db_id 2) inside a query targeting db 1 and passed; the query
failed later only because repair could not find the table in the
wrong provider. The refusal was incidental, not a boundary.
`resolve/mp.clj` has cross-database guards for cards, measures and
segments and none for tables, because the portable path never needed
one.
2. Sensitive / inactive / retired fields, and inactive or hidden tables,
were reachable by id and came back as result columns. The portable
dialect cannot name them at all -- they are absent from the by-name
fetch.
3. Table-existence oracle. The numeric branch of
`resolve-database-id-from-first-stage` distinguished "no such table"
from "table you cannot see" before any permission check had run,
which enumerates table ids instance-wide. It now read-checks and
reports both the same way. This lookup cannot go through the provider
-- it is what decides which provider to build -- so the check lives
there.
`assert-numeric-ids-are-referenceable!` raises the agent-facing errors
ahead of the permission sweep; `resolve-table-fk` independently refuses
the same ids, so the sweep fails closed even if a future caller skips
the assertion. Both new error keys already have recovery hints on both
surfaces. No-ops entirely on the portable-only surface.
Test evidence: 9 tests / 21 assertions in the new ns, all green, and 11
assertions across all three findings fail on the unfixed tree. Existing
suites unchanged: construct-representations + repair + resolve +
resolve.mp, 290 tests / 843 assertions, 0 failures.
Also corrects two wrong premises the tests caught: a query whose only
source is a foreign table is not a violation (the provider is simply
built for that database), and
`with-non-admin-groups-no-root-collection-perms` does not gate warehouse
tables, so the oracle test revokes data permissions instead.
* Read-check numeric metric, segment, and measure refs too
Follow-up to the numeric `source-card` fix, from review feedback: the
same gap existed for every other content reference, not just the source.
`import-mbql` guarded its content branches with `portable-id?`, which is
string-only. A numeric `["metric" {} 42]`, `["segment" {} 42]`, or
`["measure" {} 42]` therefore matched no branch at all -- it was never
resolved, so it never reached `import-fk`, so the read-checked content
store never saw it. Same shape as the `source-card` leak, across seven
more branches.
Rather than give each branch a numeric twin -- fourteen branches, and a
later one easy to forget -- the guard itself widens: `content-ref?` is
"a portable entity_id, or a bare id on the numeric-id surface", and
every content branch shares it. `import-fk` then dispatches numerics per
model, so `import-card-by-id` gains `import-measure-by-id` and
`import-segment-by-id` siblings, each carrying the same cross-database
guard as its portable counterpart. Metric refs resolve as Cards, which
is what they are.
Also from review:
- `:unknown-card-id` for a numeric card miss, matching `export-card-by-id`
and the `:unknown-table-id` precedent -- a numeric miss and a portable
miss want different recovery advice. Updates the one test that asserted
the old key.
- `resolve-database-id-from-first-stage`'s docstring said `:unknown-table`
where the code throws `:unknown-table-id`.
- Dropped a `^clojure.lang.ExceptionInfo` hint on `with-recovery-hint`
that propagates nowhere.
- `numeric-field-id-error!` no longer claims lib rejects bare numeric ids,
which stopped being true once the numeric surface landed.
The read-check on the numeric `source-table` lookup moved into
`metabot.db/readable-active-table-database-id`, keeping Toucan calls in
the db namespace where this branch put them.
Test evidence: 308 tests / 883 assertions across
construct-numeric-permissions, construct-representations, repair,
resolve, and resolve.mp -- all green.
* Correct a test comment about the PathSchema stack overflow
The comment called it a nested-quantifier backtracking problem. It is
not: `path-char-rx` excludes `/`, so segment boundaries are forced and
there is nothing to backtrack over — a 218-character failing input
matches in 0 ms. java.util.regex simply recurses once per quantifier
iteration on a *successful* match, so depth grows linearly with segment
count (~2000 segments overflows a default stack on its own). Atop the
pipeline's own frames the threshold drops far enough to hit it here.
Same conclusion either way — pre-existing, dev/test only, unrelated to
this branch — but the stated mechanism was wrong and would have sent the
next reader after a bug that isn't there.
* Say why the visibility lookups swallow every exception
Review noted the catch-all was undocumented. It is deliberate: this pass
decides only what a numeric id may name, so failing closed is the safe
direction and a provider outage is diagnosed elsewhere. Say so, rather
than leaving a bare `(catch Exception _ nil)` to read as an oversight.
* Drop a redundant do in a test helper
`try` already has an implicit body; kondo flagged it.
* Build the test fixtures with Lib instead of mt/mbql-query
`mt/mbql-query` is deprecated since 0.61.0, and clj-kondo exits 2 on
warnings — so the two call sites this ns added were failing the
Clj-Kondo CI job outright, not merely producing noise. Its own docstring
says to use Lib for new tests; done.
Same queries, same result columns: the victim card keeps its named
`SecretMargin` expression and its aggregation, which is what makes a
column-name leak unmistakable in the assertions.
Full-repo kondo now reports 0 errors, 0 warnings (was 2 warnings, exit 2).
* Close four numeric-id permission gaps found in review
All four are in the numeric-id handling this branch added.
**Two-element field refs escaped collection.** `node-table-fks` checked the
portable form in slots 2 *and* 1 but the numeric form in slot 2 only. Repair
inserts the options map later, so an LLM-authored `["field", 42]` still carries
its target in slot 1 when the collection pass runs — that shape reached
resolution unchecked. Both forms now read both slots.
**Field visibility ignored the parent chain.** A JSON column unfolds into child
fields carrying their own visibility, so an ordinary-looking child of a
`:sensitive` parent was referenceable by id — reaching inside a column the
warehouse marked unqueryable. `visible-field` now walks `:parent-id` and
requires every ancestor to be visible too, bounded so a corrupt chain cannot
spin.
**Unreadable and missing content were distinguishable.** A denial surfaced
`read-check`'s 403 while an absent id produced a 400 `:unknown-…`. Content ids
are sequential and easy to guess, so the status code alone told a caller whether
a hidden card, measure, or segment exists — the same oracle already closed for
the numeric table lookup. `maybe-read-check` now collapses a 403 to `nil`, which
every resolver already translates into its not-found error, so one change covers
all six store methods; non-403 exceptions still propagate, since those are real
errors rather than permission answers. `resolve-database-id-from-first-stage`
read-checks outside the store, so it got the same collapse.
**The table lookup denied users it should have allowed.** `mi/can-read?` for a
Table is polymorphic over the instance, and its published-collection branch reads
`:is_published` and `:collection_id`.
`readable-active-table-database-id` selected only `:id`/`:db_id`, so that branch
evaluated against absent keys and a user whose access came through a published
collection was told the table does not exist. It selects the whole row now. This
one was a fail-closed regression introduced by the previous commit's oracle fix.
Tests: four new cases, each verified to fail on the pre-fix tree. Existing tests
that asserted the 403 shape are updated to the collapsed one — their intent (the
check runs, fails closed, does not depend on a bound user, never leaks an
entity_id) is unchanged and still asserted.
319 tests / 935 assertions green across construct-numeric-permissions,
construct-representations, repair, resolve, resolve.mp, and content-store.
Full-repo kondo: 0 errors, 0 warnings.
* Close the query-check oracle and cover scoped content refs
Second review round. Both blocking findings are the same root cause as the
one the previous commit fixed — `mi/can-read?` is strictly weaker than
`mi/can-query?` for a Table, since only `can-read?` carries the
`manage-table-metadata :yes` disjunct — reached by two routes that fix did
not cover.
**The table oracle was still open one line later.** The previous commit
closed it at `readable-active-table-database-id`, which gates on
`can-read?`. `check-source-table-query-permissions!` runs immediately after
and gates on `api/query-check` → `can-query?`. So a metadata-only user
passed every earlier gate and got a bare 403 there, while a nonexistent id
got a 400 — confirming the table exists, is active, and is in this database.
`query-check-table!` now collapses a denial on a *numeric* reference into the
same `:unknown-table-id` an absent id produces. The portable path keeps its
403: its references are unguessable names, so there is nothing to enumerate
and the accurate status is worth more.
**Numeric measure and segment refs never reached a query-check at all.**
`node-table-fks` matched maps and `["field" …]` vectors, so `["measure" {} 44]`
and `["segment" {} 43]` fell through both branches — the tables they are
scoped to were never collected, and `assert-numeric-ids-are-referenceable!`
iterates the same collection, so it skipped them too. They do get
`api/read-check` through the store, but `can-read?` on both models delegates
to the Table's `can-read?` and lands on the same weaker gate. Adds a
`::scoped-content` tag mirroring the existing `::field-id` one, resolved to
the owning table through the read-checked store. Metric refs are Cards and
are deliberately not covered: a card pulls in whatever tables its own query
names, which this pre-flight sweep cannot see, and the QP re-checks them at
execution.
Also from the same review:
- `snippet-id` had been widened to `content-ref?`, but no resolver implements
`NativeQuerySnippet` — a numeric one fell through `import-fk`'s model tests
to `not-implemented!` and surfaced as a bare 501 with no `:agent-error?`,
never reaching the store. Narrowed back to `portable-id?`; `content-ref?`'s
docstring no longer claims snippets.
- `active-table-database-id` was dead: the pre-oracle-fix version its checked
sibling superseded within this branch, with the shorter name that reads as
the obvious choice. Deleted. The one test "stubbing" it was stubbing the
dead var and hitting the real app DB, passing only because id 999 happened
not to exist — retargeted to the function actually under test.
- `*numeric-ids-allowed?*`'s docstring described its readers in the future
tense; this PR is what added them. Rewritten to state the contract instead
of the rollout.
- `import-card-by-id`'s docstring still said a denial becomes "a 403", which
the previous commit's collapse made false.
- `visible-field`'s docstring now says the parent walk checks each ancestor's
visibility but not its `:table-id`, and why that is sound.
Tests: two new cases, both mutation-checked. The segment one deliberately puts
the segment on a different table than the stage's `source-table`, so it cannot
be satisfied by the source table's own check — the first version could, and
passed with the fix reverted.
321 tests / 943 assertions green. Full-repo kondo: 0 errors, 0 warnings.
* Keep the export fallback suppressed for a refused card
The 403 collapse in the previous commits removed a signal an internal
caller depended on. `llm-shape/export-query-for-llm` documents that "a
permission-refused export renders nothing at all rather than the
fallback", and implemented that by catching the 403. Once the content
store turned a denial into `nil`, that 403 stopped arriving, so a card
the caller may not read fell through to `query-edn-fallback` and printed
the raw query map.
No entity_id leaked — the export still refuses to emit one, so the
fallback showed `:source-card 1`, the numeric id the caller already
supplied. But the caller's contract changed, and the intent of that catch
was to render nothing.
The two requirements are not in conflict. Agent-facing, a denial and a
miss must be indistinguishable: same status, same error key, so a
guessable id cannot be used to probe for hidden content. Internally,
callers still need to tell them apart to decide whether falling back is
appropriate. So the collapse keeps the uniform error and records the
refusal out of band, in `*last-lookup-refused?*`, which says only that
*some* lookup was refused — never which row, so it cannot itself become
an oracle.
Caught by `llm-shape-test/transform-query->text-source-card-permission-test`,
which I had not run: it lives outside the affected-set I had been
checking. The second review round flagged this mechanism and I recorded
it as out of scope; it was not.
175 tests / 768 assertions green across llm-shape, content-store,
construct-numeric-permissions, construct-representations, and
resolve.mp. Full-repo kondo: 0 errors, 0 warnings.
* Collapse the read-check denial for numeric lookups only
The collapse was too broad. It lived in `maybe-read-check`, which every
content lookup passes through, so it applied to entity_id lookups as well
as numeric ones — and the two surfaces want opposite things.
Numeric ids are sequential and trivially guessable, so a distinguishable
denial is an existence oracle and the 403 has to become the same
not-found a missing id produces. A 21-character NanoID is not guessable,
so there is nothing to enumerate, and callers there depend on the accurate
status: `POST /api/agent/v2/construct-query` returns 403 for a metric
whose card the caller cannot read, and `llm-shape/export-query-for-llm`
suppresses its EDN fallback on one.
Collapsing both meant the entity_id path started answering 400
`:unknown-card` where it had answered 403 — caught by CI in
`agent-api.api-test/construct-query-permission-checks-test`, which asserts
the 403 for exactly that case.
`maybe-read-check` now takes `collapse-denial?`: true for the three
`-by-id` methods, false for the three `-by-entity-id` ones. The refusal is
still recorded in `*last-lookup-refused?*` either way, so callers that need
to know a denial happened without depending on the status keep working.
This is the second time the chokepoint's breadth cut the wrong way — it is
what made one edit cover all six store methods, and also what swept in a
surface that did not want the change. The split keeps the single
chokepoint and makes the difference explicit at it.
462 tests / 1698 assertions green across construct-numeric-permissions,
construct-representations, repair, resolve, resolve.mp, content-store,
llm-shape, and agent-api. Full-repo kondo: 0 errors, 0 warnings.
---------
Co-authored-by: Eric Normand <ericwnormand@gmail.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
* mcp-v2.05 metric_write (#81249)
* MCP v2 tool: bookmark_content
Adds the v2 bookmark_content tool — per-user favorites, keyed by (type, id).
Extraction: bookmarks/api.clj factors bookmark!/un-bookmark!/bookmarked? out
of the REST endpoints so REST and MCP share them. The POST endpoint now
creates via app-db/select-or-insert! (concurrency-safe: a racing duplicate
returns the existing row instead of tripping the unique constraint); its 400
on already-bookmarked is unchanged. REST tests extended, existing pass.
Unlike REST, both tool directions are idempotent (see ns docstring for why),
both read-check the item, and the name echo is gated on the read scope so a
write-only token can't use it as a name oracle.
Tests: 9 tool tests + bookmarks REST suite green.
Review question for this PR: one tool — is bookmark_content's contract right?
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BjQcjQx7NPGr3Rpf6QaG4b
Also lands (first consumer, per the common.clj slimming in the core PR):
- NEW v2/resolve.clj — entity-id / numeric-id resolution + the read-check
pairing with the not-found collapse (+ resolve_test, moved from common_test).
- NEW v2/write.clj — readback (the scope-gated write echo).
CI guard: this slice's mcp -> bookmarks edge put `bookmarks` into the
modules-that-trigger-driver-tests set (mage's module-graph ratchet, 44 -> 45).
Bookmark logic cannot affect driver behavior — the edge is a consumer edge —
so `bookmarks` joins driver-affecting-overrides rather than bumping the
ratchet. Full mage suite green (82 tests).
Review nit (round 1): fetch-item's flavor-mismatch teaching error printed
"is a null" for a card type outside question/model/metric; guard the nil case
with a type-agnostic message.
--- Round-2 /review-pr response ---
IMPORTANT (per-user-test could not fail on its named assertion) FIXED. The test called the tool
bare, so a tool-level error was discarded and the assertions passed regardless. It now routes
both calls through tool-result (throws on tool error) and has :lucky bookmark-then-un-bookmark
its own row, so the "one user's bookmark is invisible to another's un-bookmark" claim has a live
counterfactual. Proven fail-when-broken (stubbing un-bookmark! to a no-op turns the test red).
MINOR fixed: the id schema now requires a positive int (matching REST's ms/PositiveInt); the
degraded write-scope readback now surfaces :bookmarked (via a new backward-compatible ack-keys
arg on v2.write/readback, so a write-only caller still learns the resulting state).
MINOR deferred (pre-existing REST behavior, out of a tool-slice's scope, confirmed structurally):
archived content is bookmarkable but filtered from listings; the Trash collection is bookmarkable;
readback's :url key has no producer on this branch (lands with collection_write). Each is shared
with the existing REST bookmark endpoints; changing it means deciding REST semantics.
Fresh-JVM: bookmark-test 9 tests / 43 assertions / 0 failures; kondo 0/0; modules unchanged.
* MCP v2 tool: collection_write
Adds the v2 collection_write tool — create/update/move/archive collections.
Extraction: collections/update.clj lifts update-collection! out of
PUT /api/collection/:id — the four privates (move-collection!,
archive-collection!, move-or-archive-collection-if-needed!,
maybe-send-archived-notifications!) move verbatim (docstrings de-REST-ified,
notification alias renamed); the endpoint becomes a thin delegator through
the collections.core facade, mirroring the existing create-collection!
pattern. Verified move: bodies diff clean modulo whitespace/docstrings.
The foundation's REST-test changes for this file are all the BROWSE
extraction's repoints, so master's collections_rest api-test runs UNCHANGED
against this branch — that unchanged suite is the extraction check
(fresh-JVM: 202 tests, 941 assertions, green, incl. 23 tool tests).
Deferred to the content tool PR: echo-matches-concise-read-test (the write
echo vs get_content cross-check) — it requires tools/content.
Review question for this PR: one tool — is collection_write's contract right?
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BjQcjQx7NPGr3Rpf6QaG4b
Also lands (first consumer, per the common.clj slimming in the core PR):
- v2/resolve.clj grows collection resolution (resolve-collection-id{,-or-personal}).
- v2/write.clj grows `_write` method dispatch (dispatch-write + expand-clear).
- common.clj grows frontend-url. Tests moved alongside (resolve_test, write_test).
Review nits (round 1): fix the self-contradictory `name` field description
("Create only (editable on update)" -> "Required on create; editable on
update"); drop structuredContent from the write echo to match the stack's
text-only convention (no concrete programmatic consumer, no outputSchema).
CI kondo fix: the update-collection! extraction moved the only `tru` caller in
collections_rest/api.clj into collections/update.clj, leaving a dead
`[metabase.util.i18n :refer [tru]]` require — an unused-require warning, which
`mage kondo` (CI's check) exits nonzero on. Removed it.
--- Round-2 /review-pr response ---
IMPORTANT (archived:true + parent_id silently discarded the move) FIXED. The dispatch tested
archived first and returned, so a caller passing both got a trashed collection and no move, with
no signal — defeating the tool's "never believes an ignored field took effect" invariant. It is
now a teaching error naming the fix (move first then archive, or pass parent_id on a later
archived:false restore). Regression test added.
MINOR fixed: the four (is (tool-error ...)) assertions now match the specific expected message,
not any error text; the :name destructure was renamed (coll-name) so it no longer shadows
clojure.core/name.
Fresh-JVM: collection-test 24 tests / 65 assertions / 0 failures; kondo 0/0; modules unchanged.
--- Close-out M5 ---
MINOR (bare tool-error assertion): authority-level-update-test now asserts the specific "paid feature" message (parity with the create-side twin), so a wrong error text fails.
* MCP v2 tool: metric_write
Adds the v2 metric_write tool — create/update metrics from an MBQL
definition (either dialect) or a query_handle.
Extraction: queries/card_write_checks.clj lifts the pre-write check stack
(check-card-can-be-saved!, check-no-save-cycle!, actual-collection-id,
check-allowed-to-create-card!, check-allowed-to-update-card!) out of
queries_rest/api/card.clj (−98; taken wholesale — zero master drift) so
REST and MCP run identical checks. queries/core.clj facade HAND-MERGED:
master's post-fork parameter-template-tag? export kept; foundation's
infer-metadata-with-model-overrides export skipped (no consumer here yet).
Projection ownership fix (the WORKLOG's load-order landmine, hit live):
the :metric key projection moves from content.clj's load-time side effect
into this tool — the type's owner — so metric_write works without
tools/content loaded. The content PR drops its copy.
Deferred tests (see WORKLOG ledger): the execute_query handle flow → query
PR; the get_content echo-parity and definition round-trip → content PR.
card_test's param-fields-hydrated-test is a post-fork master addition and
stays (foundation's deletion is fork lag, not intent).
Fresh-JVM: metric tool suite + agent-lib suites + full card REST suite
green (1155+ assertions across the adjudication runs).
Review question for this PR: one tool — is metric_write's contract right?
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BjQcjQx7NPGr3Rpf6QaG4b
Also lands (first consumer, per the common.clj slimming in the core PR):
- NEW v2/queries.clj — the representations-pipeline entry point, portable-dialect
resolution, and the full query-handle store (mint/encode/decode/resolve, the
save-path variant this tool consumes; the execute tools add the minting call
sites in the query PR). Handle tests move alongside as queries_test.
Review finding (round 1, both reviewers): check-metric-shape! rejected only a
PURE-native query (lib/native-only-query?); a native stage nested with an MBQL
stage slipped through to the generic can't-be-saved message (or an incidental
editable? rejection that would open if :native-permissions were ever populated
server-side). Tightened to the whole-tree query-guards/native-query? — the same
guard question_write uses — so any native stage is refused with the metric-
specific message. Regression test added.
--- Round-2 /review-pr response ---
CRITICAL (update! never refreshes result_metadata after a definition swap) REFUTED. Verified in a
fresh JVM: creating a metric then updating it to a different definition (count -> sum with a month
breakout) refreshes result_metadata from a 1-column to a 2-column set. queries/update-card!'s hook
recomputes result_metadata whenever dataset_query changes, so the extraction did not lose the
recompute step. Pinned with metric-write-lifecycle-test asserting the before/after metadata differ.
IMPORTANT fixed: destructiveHint is now true (archived:true trashes the metric; matches the sibling
collection_write tool); the tool no longer emits structuredContent against the stack's text-only
convention; an id on a create call is now rejected rather than silently ignored; collection_position
requires a positive int (matching the downstream contract).
MINOR: applicable fixes applied; the remaining two (a duplicated frontend-url, a kondo allowlist
churn) deferred as cosmetic.
Fresh-JVM: metric-test 25 tests / 79 assertions / 0 failures; kondo 0/0; modules unchanged.
* MCP v2 prep: portable-dialect execution rework in the agent lib
The shared machinery behind every portable-MBQL write path — metabot's
construct_notebook_query today, and the v2 MCP tools (metric_write,
question_write, execute_query) as they land on the stack:
- metabot/tools/construct: execute-representations-query reworked (new
signature; numeric-id source detection; recovery-hint wrapping around
resolution failures).
- NEW metabot/tools/recovery_hints: the standard 'what to call next'
sentences appended to dialect teaching errors.
- agent-lib representations{,.repair}: the corresponding repair/resolve
pipeline changes (resolve.clj needed nothing beyond master's).
Master's post-fork pMBQL→'MBQL 5' doc/message renames are re-applied on
top of the pulled files (wholesale pulls from the foundation would have
reverted them).
Behavior check: master's agent-lib suites (repair_test, resolve_test) and
metabot tool tests run UNCHANGED and green in a fresh JVM.
Review question for this PR: does construct_notebook_query (the existing
metabot consumer) behave identically where not deliberately extended?
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BjQcjQx7NPGr3Rpf6QaG4b
Review fixes (adversarial review round 1):
- F1: the agent-api /v2/construct-query endpoint and the slackbot query tool
called execute-representations-query without :recovery-hint, so both v1
surfaces relayed bare pipeline errors where slice 03 had baked guidance into
the throw sites. Wired recovery-hints/recovery-hint into both (as the ns
docstring already claims every v1 entry point does). Proven: a URI-in-
source-table error now carries the "use the portable FK" sentence on the
agent-api path.
- F4: node-table-fks never collected a numeric field ref's target table, so it
was never permission-checked; the no-fk-path error then named it from the
un-sandboxed metadata provider — a table-name oracle for any probed id once a
numeric binder lands. Fall back to the bare id for the numeric case (matching
the vector case's own-name behavior). Dark today (no binder on this slice),
fixed before it can go live.
Test fix: the F1 change gave slackbot's execute-representations-query call a
second (:recovery-hint) arg, but slackbot_query_test stubbed the fn with 1-arg
lambdas — the extra arg broke the stubs (nil results). Made the 4 stubs
variadic ([external-query & _]) so they accept the opts map; production change
unchanged. Suite green (53 assertions).
--- Round-2 /review-pr response ---
CRITICAL (numeric field refs bypass the pre-repair permission check) FIXED. A bare numeric field
id in a query node was never resolved to its table, so node-table-fks skipped it and query-check
never gated the field's table. node-table-fks now emits the source-field as a tagged numeric ref;
resolve-table-fk resolves it to the field's table via the permission-aware metadata provider and
checks that table. Two regression tests: one asserts the FK-target table is permission-checked,
one asserts a forbidden FK-target table blocks the call.
CRITICAL (export-source-field-portable writes an un-permission-checked FK path) REFUTED, and the
refutation is closed by the fix above. The exported source-field is the FK-BEARING column, which
by definition lives on the source table — the same table node-table-fks now collects and
query-check gates. Verified in a fresh JVM: the exported field resolves to a column on the source
table, not the FK-target; denying perms on the target raises 403 before export runs. The reviewer
conflated the FK-target table with the FK-bearing column.
IMPORTANT fixed: the recovery_hints ns docstring no longer asserts wiring that does not exist (it
now states the v2 query slice must pass it, and that until then it has no production caller); the
:no-fk-path message and the :unknown-table key collision are corrected. MINOR fixes applied.
Fresh-JVM: construct-representations-test + repair-test 228 tests / 616 assertions / 0 failures;
kondo 0/0; modules unchanged.
* metric_write update: hydrate moderation reviews like PUT /api/card/:id does
`update-card!` un-verifies a verified card whose query changes by reading `:moderation_reviews`
off the card-before. `update!` passed the bare write-check row, so a verified metric kept its
Verified badge after a definition swap through the tool, unlike through REST. The card-before is
now hydrated the way the REST handler hydrates it.
* Pin the create-side permission stack
`check-allowed-to-create-card!` is the only thing stopping a caller saving a metric into a collection
they cannot curate, or over a query they have no data permission to run — `create-card!` performs no
check of its own. Every create test in this namespace runs as an admin, for whom those checks are
vacuous, so deleting the line left the suite green.
Verified fails-when-broken: removing the call fails both branches of this test.
Claude-Session: https://claude.ai/code/session_018PNCQxp9kCypqE3dLQS9va
* Confine this branch's Toucan calls to db namespaces
New policy: t2 is called only from db.clj namespaces. The two source files
this branch adds each held a couple of direct Toucan calls; both move out.
metabase.queries.db takes the Dashboard collection_id lookup and the
:in_dashboards hydration out of card-write-checks. The local
dashboard-id->collection-id alias was pure indirection once the query moved,
so both sites now call the db fn directly.
metabase.mcp.v2.db takes the Card read-back and the moderation-review
hydration out of the metric tool. The comment explaining why the card-before
is hydrated stays at the call site, where the reasoning belongs.
Nothing outside this branch is touched: resolve.clj and redaction.clj came in
on mcpv2-01-core, and metric_test.clj's Toucan calls are read-back assertions
that deliberately bypass the production helpers.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017tqiaJmaEXSfaWGUZ4LQLv
* Teach the agent about an empty metric definition
`definition: {}` came back as the sanitized "Internal error", which is exactly
what normalize-definition's docstring promises never to happen for a malformed
definition.
The arg schema is an open map, so `{}` validates. normalize-query's first cond
branch short-circuits an empty query to `{}` before its strict validation runs,
so nothing throws there either. `{}` is truthy, so create!'s "pass the metric's
query" fallback never fires. can-save? is the one that finally objects, by way
of a mu/defn input-schema violation the tool can only report as an internal
error.
Guard the normalized result instead. Both create! and update! reach it through
resolve-definition, so one check covers both.
Strict MCP clients are the ones that hit this: they fill every declared
property, so "no definition" arrives as `{}` rather than an absent key.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017tqiaJmaEXSfaWGUZ4LQLv
* Let a metric_write caller choose the display
The tool hard-coded :display :scalar on create and never wrote it on update,
with no display argument at all. But the shape gate deliberately accepts one
grouping, so `count by month` saved as a metric page rendering a single number,
unfixable through MCP. agent-api's create_metric and update_metric both took an
optional display, so this was a regression rather than a standing limitation.
Accept an optional display on both methods, defaulting to scalar. The accepted
values are enumerated so an LLM-invented display is an argument error instead of
junk on the card.
The enum is a local copy of agent-api's ::card-display rather than a require:
that namespace is the v1 REST surface this tool replaces and GHY-4250 removes,
and pinning v2 to it for seventeen strings buys a dependency we would have to
undo. No canonical list exists to share instead — four other namespaces already
keep their own.
Echo the written display back. It is a detailed-projection key, but the echo is
the only place a caller sees the scalar they never asked for.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017tqiaJmaEXSfaWGUZ4LQLv
* Fold mcp.v2.db into mcp.db to satisfy the app-DB query linter
CI's `./bin/mage kondo` (full tree) flagged src/metabase/mcp/v2/db.clj:12:
"Application database query calls like `t2/select-one` must live in
metabase[-enterprise].<module>.db namespaces". The rule matches
`metabase.<module>.db`, and `metabase.mcp.v2.db` is a segment deeper, so
the module's db namespace already existed one level up as
`metabase.mcp.db` and this was a second one the rule cannot see.
Moved `hydrate-moderation-reviews` into metabase.mcp.db beside the
existing `hydrate-notification`, repointed metric.clj's two call sites
(`card` folds into the existing `select-one-by-id`), and deleted the
duplicate namespace.
Missed locally because the previous run linted only the files the merge
touched; this file came in with the branch and was never in that set.
Reproduced and fixed against the full-tree run CI uses: 0 errors,
0 warnings. Metric suite 30 tests, 95 assertions green. Modules config
unchanged.
---------
Co-authored-by: Eric Normand <ericwnormand@gmail.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
* Follow the base's contracts after the merge
Three changes the base made to shared namespaces that `question_write` had to
be adapted to; each merged textually clean and broke silently.
`v2.write/readback` dropped its 3-arity in favour of a required `ack-keys`, so
every write threw an ArityException. Passes nil, like collection_write.
`registry/call-tool` now returns `{:result …}`/`{:error …}` rather than raw MCP
content. The test namespace read `:isError`/`:content` off the top level, so its
error-path assertions failed and its success-path assertions passed vacuously. A
local `call-tool` unwraps `:result` and renders a pre-dispatch registry refusal
into the same content shape, which is what the assertions here are written
against.
The `:metabase/t2-query-namespace` rule confines app-db calls to a module's `.db`
namespace, and this file's three `t2/select-one` calls were the tree's only
violations. They go through `mcp.db/select-one-by-id`.
Verified: full `./bin/mage kondo` clean, and the twelve MCP v2 namespaces run
199 tests / 835 assertions green.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0147UgqNGEjk9nV4jRnt9oS2
* Fix duplicate_content gate laundering and copy-vs-create semantics
`copy-question!` dissoc'd `:document_id` when building the copy. That column
is not placement -- it is the Card read gate. `mi/can-read? :model/Card`
conjoins `parent-document-permits?`, which short-circuits to `true` on a nil
`document_id`, so the copy was adjudicated by collection permissions alone
and disclosed, permanently and to every reader of the destination, material
the Document's content gate exists to withhold. `POST /api/card/:id/copy`
preserves the column and `create-card!`'s `data-keys` allowlist carries it,
so this tool was the only path that laundered the gate away.
Refuse a document-scoped source with a teaching error rather than carrying
the column: `copy-document!` re-parents such cards precisely because their
`collection_id` tracks the document rather than standing alone, so a copy
that kept `document_id` would claim membership in a document whose body
never references it. Neither outcome is the "standalone question in
collection-id" the caller asked for, so name the operation that works.
Also drop `check-allowed-to-create-card!` in favour of `api/create-check` +
`create-card!`, matching REST `/copy` and `clone-card!` (UXW-5037). Copying
is not authoring -- the query comes from a row the caller already passed a
read check on -- and the authoring stack refused a user who could read and
run a native card but lacked native query-building perms. Destination
create-permission is still enforced.
Tool description: drop the per-type create scopes GHY-4225 removed, stop
scoping the `uncopied` report to deep copies (`card->decision` discards
before it consults `deep-copy?`, so shallow copies drop cards too), and
state the document-scoped refusal.
`is_deep_copy` now rejects only an explicit `true` on a non-dashboard;
`false` is the documented default and the strict inputSchema forces clients
to send it. `readback` gets explicit ack-keys `[:type :uncopied_count]` so a
write-only token learns a copy was partial -- `:uncopied` itself must not
survive, since `redact` only redacts *unreadable* cards and an
archived-but-readable one keeps its real name.
Tests: pin the document-gated refusal through `mi/can-read?` as a user who
cannot read the parent document; pin the native-card copy; pin shallow-copy
`uncopied`; pin the degraded key set. Fix two existing tests -- the
"nothing was written" block counted the source collection the copy never
targets, and the destination-permission test asserted a bare `:isError`.
* Check run permission before inferring a card's result metadata
`question_write`'s update path ran the REST check stack before attaching
`:result_metadata`, so `check-update-result-metadata-data-perms` saw a patch
that did not contain the key and never fired. On an update carrying only
`column_metadata`, `check-allowed-to-modify-query` did not fire either — it is
gated on `:dataset_query` changing — so nothing checked data permissions at all
before inference ran over the stored query.
A caller with curate access to the card and no data permission on its query
could therefore annotate its columns, and the inference teaching error named the
query's real columns to them one guess at a time. The write succeeded, storing
the fully inferred metadata.
The create path never had this hole: `check-allowed-to-create-card!` runs the
run-permission check unconditionally before inference. This gives update the
same check, gated on the inference actually happening. It also covers what the
REST endpoint checks on a supplied `result_metadata`, because the overrides only
annotate columns the inference produced — they cannot name a table the query
does not.
`check-allowed-to-run-query!` is the pair `check-allowed-to-create-card!`
already inlined, extracted so both paths call the same thing.
Verified fails-when-broken: removing the call fails both new tests.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0147UgqNGEjk9nV4jRnt9oS2
* Derive protected-resource scopes from the resource path
The bare `/.well-known/oauth-protected-resource` endpoint advertised the v1
canonical resource with the v2 scope set: 6 scopes instead of 22. A client that
probes the bare path -- the behavior BOT-1617 added the endpoint to serve --
read 6, requested those 6, and received a token authorizing nothing on the
resource it had asked about. Empty `tools/list`, and no in-product way to widen
the grant afterwards. Worse, the wrong pairing became durable: the authorization
code grant copies the code's scope onto the access and refresh tokens without
re-consulting the client's registered ceiling, so recovery meant re-consenting
from scratch.
The cause was `protected-resource-metadata` taking the scope set as a separate
`scopes-fn` parameter. Three call sites passed a closure over their own path;
the fourth passed `mcp-resource-scopes` bare, which hit a 0-arity overload
hardcoding the v2 path. Both spellings type-check, and nothing at the call site
says "v2" -- the sibling accessor `mcp-canonical-path` returns the v1 path, so
the two read as interchangeable ways to name the MCP resource.
Rather than correct the fourth call site, drop the parameter: the function now
derives scopes from its own `resource-path`, so the resource and its scope set
cannot disagree. Slice 21 retires v1 and collapses this function; with the
parameter gone it cannot reintroduce the mismatch by overlooking an endpoint.
Delete the 0-arity overload, a delegating arity hiding a non-nil default. Its
only other caller was `default-grant-scopes`, where the union made it a provable
no-op -- `v2-scopes` is a subset of `all-scopes`, verified against a live
instance.
Also on the metadata/narrowing path:
- Delete the `:extra-scopes` plumbing. `registered-opt-in-scopes` read the key
off every registered tool, but `register-tool!`'s closed allowlist rejects it,
so the function provably returned the empty set. Since `register-tool!` runs
at namespace load from a top-level `deftool`, and those namespaces load from
the route table at boot, the first tool to use the documented key would have
taken down the entire v2 MCP surface rather than just that tool. This also
removes the only `mcp.core -> mcp.v2.registry` require.
- Narrow to the union of what every named resource accepts. RFC 8707 permits
several resource indicators and the token must work against each, but the
check asked only whether v2 was named *at all* and then applied that single
answer to the whole request. Naming a v1 alias alongside v2 discarded every
v1-only scope, order-independently -- contradicting the comment two lines
above it and silently shrinking a grant the user then approved.
- Canonicalize resource URIs via `.getAuthority` rather than `.getHost`.
`getHost` is nil for hosts `java.net.URI` considers non-conformant, notably
any containing an underscore -- routine for Docker Compose and internal k8s
service names, and accepted by Metabase's own `u/url?` as a Site URL. On such
an instance no indicator ever matched and narrowing was disabled instance-wide
and silently. Userinfo stripping, host case folding and default-port elision
are preserved.
- Normalize a bare-string `resources` argument. The endpoint schema admits one,
and `keep` over a String iterates characters, none of which canonicalize --
which would skip narrowing entirely. Latent today only because oidc-provider
vectorizes upstream; a security control should not depend on a coercion one
layer away.
- Answer RFC 6749 `invalid_scope` when narrowing removes every requested scope.
`narrow-scope-to-resource` returns nil both for "nothing was requested" and
"nothing survived", and the call site treated them alike -- rendering a consent
screen listing no permissions and minting a zero-scope token, a handshake that
looks successful and yields an empty tool list with no diagnostic.
Tests: assert, for every protected-resource endpoint including the bare one,
that `scopes_supported` equals the scope set for the `:resource` the response
itself advertises -- one invariant covering all four, keyed off the response so
the two cannot drift. The previous tests could not catch this: the bare-path
test omitted `scopes_supported` entirely and the other asserted only
`sequential?`, both of which stay green with the feature deleted. Verified by
reintroducing the defect and confirming the new assertions go red.
Deferred as follow-ups: BOT-2124 (resource indicators are not enforced as a
token audience), BOT-2125 (v2 sub-paths escape narrowing under prefix routing),
BOT-2126 (observability when narrowing declines to apply).
* Make the v2 UI tools' handle resolvable by the iframe
`visualize_query` and `render_drill_through` return `{query_handle}`, but the
iframe bundle that ships on this branch reads `structuredContent.query` and
short-circuits on `if (query)`. With `query` absent the pending tool result was
never stored, `deserializedCard` stayed null, `isReady` never became true, and
the loading spinner never cleared. Both tools were non-functional.
The slice dropped three coordinated pieces that make the handle resolvable.
Ported from `mcp-v2-foundation`:
- `GET /api/embed-mcp/queries/:handle`, returning `{:query :prompt}` behind
`check-session-header!` with an `ms/UUIDString` route param.
- `mcp-ui-parameterized-request-surface` + `on-mcp-ui-request-surface?`.
`mcp-ui-request-surface` matches `[method uri]` by exact string and cannot
express a path parameter, so without this the route 401s at the credential
middleware before the router ever sees it. The regex is UUID-pinned and
anchored with `re-matches`, keeping the entry as narrow as a literal one.
- `fetchQueryByHandle` plus a union `{query?, query_handle?, prompt?}` reader in
`useMcpApp`, guarding out-of-order resolutions with `pendingQueryHandleRef`.
One bundle serves v1 and v2, so both payload shapes stay supported.
Chose foundation's GET form over the POST variant on
`emb-2195-mcp-apps-are-stuck-on-loading-spinner-due-to-claude-sandbox`
(`79b792254cb`). The POST form needs no middleware change, but foundation's ships
the resolver, typed error messages, and the out-of-order guard already written.
`onAuthenticated` now receives the credential rather than reading it from state:
`setUiCredential` has not committed when the callback fires, so resolving from
the closure would use the previous (empty) credential and 401.
`queryError` is threaded to the route so a handle that cannot resolve renders an
error instead of spinning forever.
Docstring corrections. Three sites asserted the resolution endpoint existed in
the present tense; that is now true, and they name the route. The claim that the
store read is "keyed on `(mcp session, user)`" was false: `find-handle-row`
joins `core_session` and filters on `user_id` alone, and the looseness is
deliberate so a client rotating its MCP session keeps its handles. Verified live
against a running instance: cross-user resolution 404s, cross-session for the
same user 200s. The real native-SQL boundary is `check-mcp-ui-native-query!` on
`/api/dataset` via `+refuse-unscoped-native-sql`, which the file never named.
Fixed a dangling wikilink to a var that does not exist.
Dropped `:idempotentHint true` from `visualize_query`; the fresh-`query` path
inserts a handle row per call. `render_drill_through` keeps it.
Removed the `display` argument and output key. It was enum-validated and echoed
but no renderer read it, and `mcp_query_handle` has no column for it, so on the
documented preferred path (mint by `query`, re-visualize by `query_handle`) the
chart type was lost regardless. Rather than advertise a knob that does nothing,
it comes back end to end in GHY-4451.
Reverted the `all-scopes` rewrite. It was set-identical to the base
(`OLD-ONLY ()`, `NEW-ONLY ()`, 22 = 22) and deleted a correct sentence, while
the PR body claimed a scope-advertisement fix that never existed.
Tests. `visualize_test.clj` positively pinned the broken shape, asserting
`(= #{:query_handle} (set (keys body)))` — one side of a two-sided contract, and
green while the widget spun. It now asserts the absence of query data instead,
and two new tests walk the whole loop: mint a handle, call the tool, resolve the
returned handle over the credential-authenticated route. Deleting the endpoint
fails 9 tests; emptying the allowlist entry fails 6. The frontend spec covered
only v1's inline shape, so three tests now cover the handle path; two of them
fail against the previous hook.
Follow-ups filed: GHY-4450 (tools return success for handles the iframe will be
403'd on), GHY-4451 (restore `display` end to end), GHY-4452 (error-message,
schema and coverage cleanups), GHY-4453 (`prompt` dropped on the handle branch).
* Collapse an unreadable dashboard into the same not-found as a missing one
`resolve-dashboard!` read the row with no read check and threw the collapsed
not-found only when the row was absent. A dashboard that existed in a collection
the caller could not curate was refused later, by the write check, with a 403 —
distinguishable from the 404 a nonexistent id gave. A token scoped to one
collection could enumerate every dashboard id on the instance by watching one
against the other.
Resolving through `v2.resolve/resolve-and-read` gives the argument the read
check the rest of the module's id arguments have, so both cases read alike. The
write check still decides whether a card may be saved into the dashboard. This
also subsumes the hand-rolled existence check: `resolve-and-read` turns a
numeric id matching no row into the same not-found, which is what kept a bad id
from reaching the DB as a raw FK violation.
`update-move-into-dashboard-requires-destination-write-test` revoked every
permission on the destination, so under the read check it would now pass on
not-found and stop proving what it was written to prove — that the write
requirement on the move still runs. It now grants read and withholds write, so
the write check is again what refuses.
Verified fails-when-broken: restoring the unchecked read fails the new test.
Closes GHY-4443.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0147UgqNGEjk9nV4jRnt9oS2
* Clear the five kondo warnings failing CI on this stack
`./bin/mage kondo` over the whole repo exits 2 on five warnings, so the
backend-tests / Clj-Kondo job fails. Both files are byte-identical to this
slice's parent commit -- the warnings arrive with earlier slices in the stack
and fail the same way there. Fixing here so slice 20 can go green.
- `browse_test.clj`: the `deep` collection is created for its side effect only,
since the assertions look it up by name rather than through the binding.
Renamed `_deep`.
- `question_test.clj`: `check-semantic-type!` is flagged as destructive inside a
`^:parallel` test purely on the `!` suffix. It parses a keyword, throws on a
bad one and returns it -- no state touched -- so the suffix means "may throw",
not "mutates". Added to the `:parallel/safe` whitelist the linter's own
message points at.
* Gate a native query_handle like the other two query sources
The `query_handle` source stored whatever query the handle resolved to without
consulting either SQL gate, so a token holding only `agent:content:write` could
save a card containing raw SQL, on create and on update, with the
`mcp-execute-sql-enabled` kill switch turned off. A later `run_saved_question`
then executes it — `execute_sql` rebuilt without its scope or its switch, which
is what `check-execute-sql-enabled!` exists to prevent.
The exemption rested on the premise that minting a handle already spent those
gates. It doesn't. `construct_native_query` mints under `agent:sql:construct`
and never consults the kill switch, and `/api/embed-mcp/drills` mints from an
uninspected caller-supplied string. Nor does the handle stay with the credential
that minted it: resolution joins `core_session` on `user_id` alone, the MCP
session id being recorded for telemetry, so any credential of that user can
spend a handle minted by any other. There is no independent handle TTL either —
handles outlive the request by the backing session's lifetime.
Gating on the resolved query's shape, as the inline `query` source already does,
covers all of it without knowing which argument carried the SQL. MBQL handles
are unaffected.
The docstring asserted the premise this commit refutes, so it now records why a
handle is not proof the gates were spent.
`create-question-via-native-query-handle-test` pinned the vulnerable behavior as
intended and is inverted. The update path had no native-handle coverage at all;
it gets both halves, plus the kill switch on the handle route.
Verified fails-when-broken: removing the guard fails 9 assertions across the two
tests.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0147UgqNGEjk9nV4jRnt9oS2
* Pin the invariants two advertised-scope tests only appeared to check
Both tests passed regardless of their input, so they read as coverage of the
advertised scope sets without providing any. Rewritten to assert on the sources
rather than on values the construction already guarantees, and verified by
injecting the fault each is meant to catch.
- `advertised-scopes-are-distinct-test` counted duplicates in the *output* of
functions that build through a `sorted-set`, which is distinct by
construction whatever goes in. It now checks every endpoint path and the v2
surface literal, where a scope declared twice upstream is silently swallowed.
Its docstring also still described unioning "the default grant with the opt-in
scopes", which no longer exist.
- `default-grant-covers-everything-advertised-test` asserted that
`default-grant-scopes` covers the sets it is defined as the union of. It now
iterates `mcp-endpoint-paths`, which covers the per-path arities a client
actually meets -- including the v1 aliases, inside the ceiling today only
because `all-scopes` also feeds `supported-scopes`.
Add `mb-full-client-can-still-authorize-test`. Dropping `mb:full` from the
advertised sets is safe only because oidc-provider validates a requested scope
against the client's own registered `:scopes` and never against the provider's
`:scopes-supported`. Nothing pinned that, so un-advertising a scope would start
rejecting the first-party clients that legitimately hold it, silently.
Soften two docstrings that claimed more than the code delivers:
- `supported-scopes` implied explicit registration gates `mb:full`. Dynamic
registration is unauthenticated and passes a client-supplied `scope` through
unchecked, so omitting the scope here narrows who finds it, not who may ask.
- `narrow-scope-to-resource` read as though naming a resource bound the token to
it. It shapes the consent screen and the stored grant only; `resolve-access-token`
never reads `:resource`, and a client that omits the indicator is not narrowed
at all. Both gaps are inherited; BOT-2124 tracks closing them.
* alert_write: close scope, error-message, and delivery-notice gaps
Six fixes from review of the slice.
Scope: clearing `send_once` turns a single scheduled run into an
unbounded series, the same commitment a new schedule makes, so
`execute-scope-trigger` now gates it on `agent:query:run`. Only an
explicit `false` counts — nested nulls survive the boundary's stripping,
so `send_once: null` stays an omission.
Errors: `fetch-alert` now checks write alongside read, collapsing
unreadable, unwritable, and nonexistent into the one not-found that
GHY-4217 needs. That lets the blanket 403 catch in `update!` go, so
every other 403 — a disallowed email domain, a channel template the
caller can't write — reports its real cause instead of claiming the
alert does not exist. `can-write?` mirrors `can-update?` for the bodies
this tool sends: it never reassigns `creator_id` and refuses `card_id`.
Delivery: `notify-creator-of-delivery-change!` becomes
`notify-caller-of-delivery!` and runs on create too. The notification
API's own "you were added" mail removes the current user, so an
agent-created alert pointed at a third party told the token's owner
nothing.
Subscriptions: `:subscriptions` is a `:multi-row?` nested spec, so
writing one back deletes the rest. Passing `schedule` to an alert with
several is now refused, mirroring the multi-channel handler guard.
Validation: the extracted `update-notification!` validates
`::NotificationApiUpdateInput` rather than relying on callers, which the
one non-endpoint caller did not do.
Reverts the `util/cron.clj` `(or hour 0)` change. `check-schedule!`
requires an hour for weekly and monthly, so it was unreachable here,
while the shared util's other callers — pulse sends, v1 create_alert,
database sync schedules — would have had stored null-hour rows move from
hourly to midnight with no test pinning either behavior.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VyVfcPqrmdSnV4YPXDMcUK
* Check the update's new state before probing the warehouse
`update!` ran `check-target-move!` — which opens a warehouse connection to
resolve `target-table-exists?` — before any permission check on the database
that probe connects to.
The tool's own `api/write-check` covers the transform as stored. The caller's
`definition` then names its own `:database`, and `resolve-source` runs no
permission check of its own, so `source-db` becomes any database id the caller
writes down; `resolve-target` stamps it onto the target, and the probe follows
it. The first and only authorization touching the new database lives inside
`transforms/update-transform!`, on the line after.
So a data analyst holding transforms permission on the transform's own
database — enough to pass `write-check` — could ask "does table X exist in
database B?" for a database they hold nothing on, one call per guess, and
drive outbound connections there. Two further gates were skipped along with
the permission check: `check-database-feature` rejects the audit database, the
sample database, and any database with DB routing enabled, and
`throw-if-db-routing-enabled!`'s own docstring says to call it before any
driver operation.
Run the new state's gates first, which is the ordering the create path already
states in a comment ("Before the target check, which opens a warehouse
connection — a bad argument shouldn't pay for that"). REST's `PUT
/api/transform/:id` has the same order and never connects.
The regression test asserts the ordering directly rather than the outcome: it
redefines `driver/table-exists?` to record whether it ran, since the tool
converts a throwing probe into a generic "Internal error" that an
outcome-based assertion passes on. Watched failing on `probed` with the two
forms swapped, and only that test fails.
Verified in a clean JVM on this branch: transform-test 23 tests / 83
assertions; transforms.crud-test + transforms.models.transform-test 10 tests /
41 assertions; kondo 0/0; cljfmt no-op; fix-modules-config unchanged.
* Normalize a retyped model's display, and let a model be a list
Retyping a question to a model left its `display` alone, where
`PUT /api/card/:id` forces table. A model's display is not inert: the query
processor runs the pivot QP when it is `:pivot`, so a pivot question retyped
through this tool kept returning pivoted data — REST makes that unreachable.
A chart display also drives subscription rendering, dashcard sizing, and the
search display filter.
The earlier review dismissed this as cosmetic because "the model layer renders
as table regardless." It doesn't: nothing normalizes a model's display anywhere
in the render path. The invariant holds only because the UI's own convert action
sets table, and REST's rule catches the rest.
REST leaves an explicit `list` alone, and writing that carve-out showed the tool
could not express `list` at all — so an agent could neither build a list-view
model nor preserve one it was updating, even though the model editor offers
exactly table and list. `list` joins the display enum, and the carve-out honors
it.
Retype-to-model had no MCP coverage at all before this.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0147UgqNGEjk9nV4jRnt9oS2
* Close the three pre-merge findings from the stack review, and the OSS test failure
Adversarial review over slices 02..21 raised 21 findings against `subscription_write`;
three were called pre-merge, and the PR review found a fourth that reddens the OSS job.
The rest are deferred to a follow-up issue.
**`update!` did its work before anything authorized it.** `mi/can-read?` on a non-alert
Pulse is `(or superuser? creator? recipient?)`, so being on the recipient list alone
passed `fetch-subscription`'s gate — with no permission on the dashboard or its
collection. Everything between that gate and `update-pulse-with-perm-checks!`'s
`write-check` then ran for such a caller: `resolve-parameters` answers an unknown id by
printing the dashboard's complete parameter-id set, `classify` probes user existence and
active status, and the channel-topology errors name the channel count and type. The
write check's own 403 also escaped uncaught, so a recipient could tell subscriptions they
are a target of from ids they know nothing about — the existence oracle the sibling
`alert_write` closes with a named GHY-4217 comment one commit below on this stack.
Gating `fetch-subscription` on `write-check` closes all four: the disclosures never run,
and `resolve-and-read-with` collapses the 403 into the same not-found everyone else gets.
Purely additive — `update-pulse-with-perm-checks!` runs `(api/write-check
(retrieve-pulse id))`, the identical check on the identical object, so no caller who
previously succeeded is now refused.
**A schedule field the schedule type never reads was dropped silently.** `check-schedule!`
enforced only the missing-required direction, so `{schedule_type: "hourly", schedule_hour:
9}` reported success and delivered the dashboard 24 times a day with `schedule_hour` NULL.
`alert_write` refuses the same shapes, and its docstring claims the two tools are kept in
step. Adds `check-ignored-schedule-fields!` over this tool's vocabulary (no
`:schedule_minute`), matching the writer exactly: `schedule_hour` for all but hourly,
`schedule_day` for weekly and monthly, `schedule_frame` for monthly. A filled-in null
stays an omission, and monthly's more specific `mid`/`schedule_day` error still wins.
**Repointing a Slack subscription erased the channel's other stored details.** The email
branch carries `(:details existing)` forward with a comment explaining why; the slack
branch replaced the column outright, so an edit dropped `include_pdf`/`attachment_only`
set in the app and the `channel_id` the sender prefers over the display name. The next
send dropped the PDF, and a channel rename sent it nowhere. Now merges onto the stored
details, and `slack-details` carries the cache entry's id as `:channel_id` so a repoint
overwrites the stale one rather than leaving it to misroute.
**The two EE-only parameter tests could not pass on OSS.** Every `define-premium-feature`
getter is `(and config/ee-available? (has-feature? ...))`, which `with-premium-features`
cannot flip, so both failed on the feature refusal while reporting the feature as set.
Wrapped in `mt/when-ee-evailable`, the pattern `redaction_test` already uses for this.
Each fix is pinned by a test that fails when the fix is reverted, verified by reverting
each one in turn: the write-check test's byte-identical baseline comparison catches a 403
that merely contains "not found", and `update-requires-write-permission-test` is tightened
from `(some? (tool-error ...))` — a tautology once it returns — to the collapsed message,
which is why this defect shipped green.
Verification: 44 tests, 116 assertions on EE and 113 on OSS (the guarded bodies compile
away), both fresh-JVM; `:module mcp` 565 tests / 6096 assertions; kondo and cljfmt clean.
Before this change the OSS run was 2 failures and 1 error.
* fix: add_action must write a virtual_card
The frontend recognizes an action dashcard through
`visualization_settings.virtual_card.display` (`isActionDashCard` in
frontend/src/metabase/actions/utils.ts). `add_action` hand-built its
visualization settings and omitted the wrapper every other virtual op
gets from `virtual-card-settings`, so an MCP-created action dashcard
rendered as an empty card instead of a button, and neither the
ActionSidebar nor DashCardActionsPanel recognized it.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KPHtStCj5NibqNCV4e1dpp
* fix: add_parameter requires a name
`add_parameter` reused `parameter-fields` wholesale, where `:name` is
optional because `update_parameter` patches properties individually. A
nameless parameter gets no slug, which half-breaks embedding, public
links and URL parameter sync, and `dashboard->resolved-params` requires
a non-blank name — so the tool errored on read-back *after* the write
had committed, leaving the parameter behind.
`:name` now lives on each op rather than in the shared field list:
required on add, optional on update.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KPHtStCj5NibqNCV4e1dpp
* fix: move_parameter refuses both index and dashcard_id
The op documents "pass exactly one of `index` or `dashcard_id`", but the
`cond` took the `dashcard_id` branch first, so a caller sending both got
the inline placement and no reorder with no error. `add_link` already
refuses the analogous both/neither case; this makes move_parameter
consistent with it.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KPHtStCj5NibqNCV4e1dpp
* fix: move must not autoplace a dashcard against itself
`placement` took its siblings from the whole tab before the moved row
had been removed, so the card being moved counted as an obstacle to
itself and the search could never return the slot it already held. A
`move` carrying only a dashcard_id — which the schema allows, and which
reads as "leave it where it is" — always displaced the card: a
full-width size_y 4 card alone on a tab jumped from row 0 to row 4.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KPHtStCj5NibqNCV4e1dpp
* Omit content_markdown when a document body has no Markdown form
get_content filled `content_markdown` with `prose-mirror/ast->text` output
when `documents/serialize` threw. That text is a flat space-join of text
nodes: no headings, no `{% card %}` embeds, no `:::` layout containers.
`content_markdown` is the key `document_write` accepts as a whole-body
rewrite, and both tool descriptions route a read-modify-write through it.
An agent reading a document with an unrenderable block, appending a line,
and writing it back replaced the stored AST with a single paragraph and
orphaned every anchored comment thread -- and the tool reported success.
`document_write`'s own `body-projection` faces the identical call and does
the opposite, with the rule in its docstring: "old_str is matched against
this exact text, so text that isn't the serialization is worse than none."
It emits `content_markdown_unavailable`. This mirrors it, including the
`Throwable` catch, so the two document paths agree.
Proven in a REPL against a body holding a heading, a `mysteryBlock` and a
card embed: `serialize` throws, `ast->text` yields "Q3 revenue regional
breakdown", and re-parsing that string returns 1 paragraph in place of 3
blocks -- card embed and heading gone.
Both tool descriptions corrected: `document_write` no longer tells the
agent to read with get_content and rewrite via content_markdown, and
get_content's round-trip promise now names the exception.
The existing fallback test asserted the degraded value was correct; it now
asserts the marker. Reverting the fix fails 5 assertions across the two
tests.
* Route the search tool's app-DB queries through mcp/db.clj
The Clj-Kondo check went red on the merge with three warnings, all in
tools/search.clj:
Application database query calls like `t2/select` must live in
metabase[-enterprise].<module>.db namespaces
The rule arrived with the base merge, and the module had already been routed
through metabase.mcp.db; search.clj was the last file still querying directly.
Its three reads move there as collections-for-read-check,
collection-id->name+location and snippets-by-archived-state, each a direct
Toucan 2 call with no added logic, per that namespace's own contract. The
column scoping the snippet query depends on travels with it: only
id/name/description plus collection_id for the read check, never :content,
which holds the SQL body. search.clj no longer requires toucan2.
snippet-rows-does-not-load-content-test stubs `t2/select` through with-redefs
to assert that column list. It still passes unchanged — with-redefs swaps the
var, and mcp.db resolves it at call time — so the guard survives the move
rather than needing to be rewritten around it.
Verified: targeted kondo 0/0 and a full `./bin/mage kondo` 0/0; cljfmt clean;
v2 search + v2 common + metabot search 81 tests / 329 assertions / 0 failures.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CF3ZXRaq6wy8sFR9L8DPKa
* Fix four correctness defects in the query slice found by adversarial review
Keyset paging and the SQL hint each had a way to return the wrong rows, or no
rows, without saying so. All four are proven in a REPL against the Sample
Database, with a regression test each.
1. `with-total-order` and `next-page-query` dropped the query's `:parameters`.
Both rebuild the query through `lib/prepare-for-serialization`, which strips
`:parameters` as runtime-only — but on an MBQL query they are a real filter
that the QP's parameter middleware ANDs on. `execute_query` runs every query
through `with-total-order` before executing, so whenever the rewrite fired
the QP ran the query unfiltered: a filter for quantity = 2 returned
quantities [2 3 2 6 5], with no error and no truncation flag, and the minted
handle described a different question than the one asked. `execute_sql`
already worked around the identical strip; the MBQL path did not.
2. A caller could forge the private `::keyset` marker. The marker means "this
namespace minted this predicate", and both `without-page-boundary` and
`supersede-previous-keyset` strip clauses carrying it. But a tool's `:query`
is an open [:map] and a namespaced option key rides through JSON decode and
serialization intact, so a caller could stamp its own filter and have it
silently dropped from the saved handle and from every later page — a narrow
visible first page over a wide underlying read. Caller-supplied queries are
now stripped where they enter. The cursor path deliberately keeps the marker:
superseding the previous page's predicate depends on it, and stripping there
stacks a dead filter per page instead.
3. `strip-sql-noise` threw StackOverflowError on an unbalanced quote. The
literal-blanking pattern `'(?:[^']|'')*'` is an alternation under a
quantifier, which java.util.regex compiles to one stack frame per character
consumed. A single unterminated quote made it backtrack across the whole
query. It runs on the `validate_only` path with no execution to gate it, and
StackOverflowError is an Error, so it escaped the handler that turns tool
failures into teaching errors and reached the transport uncaught. Replaced
with the equivalent unrolled loop `'[^']*(?:''[^']*)*'`, verified to produce
identical output on every literal shape and to handle 50k characters.
4. The cursor silently dropped every row whose key column was NULL. A keyset
predicate is all comparisons, and under three-valued logic every term is NULL
— never true — for a row whose key is NULL, so those rows were never served
and the response still reported truncated: false. Ordering ORDERS by DISCOUNT
served 1,915 of 18,760 rows and called the result complete. Nullable keys now
refuse to mint, joining the remap and lossy-value guards; MBQL cannot express
NULLS FIRST/LAST, so refusing is the only driver-independent answer.
The refusal keys on declared nullability AND an observed `fingerprint nil%`
above zero. Declared nullability alone is too blunt to gate on — in a typical
warehouse only primary keys are NOT NULL, so it would have cost the cursor
for nearly every non-PK-ordered query. The fingerprint is a sync-time sample,
so a column that gains its first NULL after the last sync stays cursorable
until the next one; that residual is the price of keeping paging usable.
`:module mcp` 658 tests / 6506 assertions green; kondo and cljfmt clean.
* Refuse a constraints value that names a column
`get_parameter_values` accepted `[:map-of :keyword :any]` for `constraints`, and
chain filtering compiles those values into MBQL. A value shaped like a clause was
therefore compiled as one: `{"_PRICE_": [["field", 49, nil]]}` became a reference
to that column, so the server evaluated `WHERE VENUES.PRICE = VENUES.CATEGORY_ID`
and returned the matching rows.
The caller needs only read access to a dashboard with two filters on one table.
It needs no query permission — `*param-values-query*` runs these lookups elevated
precisely so a caller who can only read a dashboard can still see its filter
values — and the reference reached columns marked `visibility_type: sensitive`,
which exist to be unreachable. A name-shaped reference works without a field id.
Reported by galdre on #81245 with a proof; his repro is the test.
`::constraint-value` now admits only scalars and sequences of scalars, so no
clause shape validates and nothing caller-supplied is ever compiled. Refusing at
the boundary rather than blacklisting known clause forms: a blacklist needs
updating every time MBQL grows a shape.
The ns docstring claimed no caller-supplied query material reaches the warehouse.
That is true again, but only because of this schema, so it now says so.
Verified by reproduction: before the fix both field-reference shapes returned
success where the test expects a refusal; after, all four attack shapes are
refused. The test keeps anti-vacuity controls — the unconstrained call returns
the full list and a literal constraint narrows it — so it cannot pass by failing
to reach the warehouse. Namespace: 86 assertions, 0 failures.
* Pin the constraints refusal against a caller who cannot query
The test I first wrote proved the schema rejects a clause-shaped constraint, but
ran under `with-fixtures`, which grants full data permissions. That misses the
point of the finding: the danger is not the shape, it is that a caller holding
NO query permission could read data through it.
Rebuilt on galdre's repro. The caller now has collection read and
`:perms/view-data :unrestricted` but `:perms/create-queries :no`, and the test
asserts up front that it cannot run a query against VENUES at all — so the
refusals below are about privilege, not validation tidiness. Adds his
sensitive-column case: a reference to a column marked `visibility_type:
sensitive`, which exists to be unreachable.
Verified by mutation: with the schema returned to `:any` this fails 3 assertions
(both field-reference shapes plus the sensitive column); the earlier weaker
version caught only 2. Anti-vacuity controls keep their teeth — the unconstrained
call returns all 100 values, a literal constraint narrows it, and a literal still
works after the fix, so the refusal is shape-based rather than blanket.
* Stop get_content handing a caller data-derived values it cannot reach
Two paths returned values computed from, or describing, rows and cards the
caller was denied. Both were reported in review of #81616.
`include: ["fields"]` returned stored column fingerprints verbatim. A
fingerprint is computed over the whole table: `:min` and `:max` are
individual cell values and `:distinct-count` counts every row. A caller
sandboxed to 10 of 100 rows received statistics over all 100, and a caller
with no data permission on the table at all received the same numbers.
Columns whose table is row-restricted now lose `:fingerprint`, via
`metabot.metadata-perms/row-restricted-table-ids` -- which covers
sandboxing, impersonation and database routing, and fails closed. Nothing
else in the payload is data-derived: names, types, semantic types and field
ids are schema, not rows.
`include: ["layout"]` returned `:visualization_settings` and
`:parameter_mappings` for dashcards whose card had just been redacted as
unreadable. `redact-dashcard` reduced the card to nothing, which is the
whole redaction for the projection, but `dashboard-layout` reads the same
dashcard directly -- so the caller still learned the card's author-written
title, the warehouse columns it renders with their labels, its click
targets, and the field ids its filters are wired to. The redaction now
strips those alongside the card. Layout keys (sizing, container styling)
describe the dashboard, not the card, and survive.
`:parameter_mappings` is emptied rather than removed: it is a required key
on a DashboardCard, and the parameters summary runs
`dashboard->resolved-params` over the same row, which fails its schema when
the key is absent.
The layout case is parity with REST rather than a regression --
`dashboards-rest/api.clj`'s `hide-unreadable-card` leaves the same settings
behind -- but this file already chose to be stricter than REST twice (the
link-entity strip in this very function, and the transform target-table
gate), so it follows that standard.
Tests: the fingerprint rule needs a live sandbox, so it lives in
`metabase-enterprise.mcp.v2.redaction-sandbox-test` alongside the other
sandbox-only redaction; it asserts an admin still sees fingerprints first,
so it cannot pass vacuously, and checks the table really is row-restricted
before trusting the result. The layout rule holds in OSS and is pinned in
`metabase.mcp.v2.redaction-test`. Reverting either fix fails its test.
`:module mcp` 5892 assertions, 0 failures.
* Cover archived reads and not-found parity beyond `question`
Two gaps Eric raised in review of #81616.
`get-content-reads-archived-content-test` pins the read-path behavior for
trashed content, which was projected but never exercised. Archived content
reads rather than 404ing -- an agent asked to restore something has to be
able to read it first -- and reports `archived: true` so it is not mistaken
for live. A live entity in the same test asserts the flag is not constant.
`get-content-not-found-parity-across-types-test` extends the existing oracle
check, which covered only `question`, to model, dashboard, document and
collection. Each type has its own fetch fn, so parity on one proves nothing
about the others; the test asserts the unreadable and nonexistent messages
differ only by the id.
Snippet is deliberately absent from that loop: snippets live in the
`snippets` collection namespace rather than under the restricted collection,
so `with-non-admin-groups-no-collection-perms` cannot make one unreadable.
Its parity needs a snippets-namespace setup of its own -- noted in the test.
Found while writing these, not fixed here: `:archived` is in the question,
collection and document projections but not the dashboard one, so a trashed
dashboard reads back with no archived flag. Filed to GHY-4471 rather than
widening this PR.
`content-test` 39 tests / 213 assertions, 0 failures.
* Name the scope get_parameter_values actually requires
scope-gate-test said the tool requires agent:resource:read. It declares
agent:content:read, and the test's own assertions say so — agent:sql:run is
refused, agent:content:read succeeds. agent:resource:read gates a resource
(catalog://metabase/fields), not this tool.
* fix(mcp): name budget-omitted tables from source rows
`budget-omitted` read `:id`/`:name` off the already-projected payload, but a
`fields` selection that excludes them leaves nothing to read: every omitted
entry serialized as `{"id": null, "name": null}`, so the caller could not tell
which tables were dropped and "request in a separate call" was unactionable.
`assemble-tables` no longer builds `:omitted` — its `:tables` is always a
prefix of `payloads`, which is index-aligned with `rows`, so `get-fields`
names the dropped tables from the source rows instead.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015QkpS1yAdHikDjE1hQaShC
* fix(mcp): withhold fingerprints from row-restricted users
`get_fields`' detailed projection returned each field's `:fingerprint`, which
is computed at sync time across every row of the table. A user whose row
access is narrowed by sandboxing, connection impersonation, or database
routing got correctly narrowed `:values` but a whole-table fingerprint:
`:min`/`:max`/`:earliest`/`:latest` are literal values from rows they cannot
see, and `:global :distinct-count` is whole-table cardinality for every column
type. There is no safe subset, so the whole map is withheld, following
`metabot.tools.field-stats/field-statistics`.
The restriction lookup is batched over the request's tables and fails closed.
The `fields` projection path is covered by the same `detailed?` branch, so a
`fields: ["fields.fingerprint"]` selection cannot route around it.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015QkpS1yAdHikDjE1hQaShC
* fix(mcp): require query permission to inline field values
`get_fields` gated only on `mi/can-read?` on the table, but then inlined
cached FieldValues, whose own check bottoms out in `can-read?` as well. A
caller holding `manage-table-metadata` alone therefore got values that
`/api/field/:id/values` refuses with a 403 — REST gates `summary` on
`read-check` and `values` on `query-check`, and that split has to hold here
too: values are rows of the table, not metadata about it.
`attach-inline-values` now primes the table perms cache once per request and
restricts the batched fetch to tables the caller can query. Tables it can only
read keep their full field metadata, without `:values`. The gate is per table,
so a mixed batch resolves each one on its own.
`response_format` now documents that values and fingerprints are
permission-dependent and absent rather than empty when withheld.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015QkpS1yAdHikDjE1hQaShC
* test(mcp): drive browse_data through the registry
Every browse_data test called the handler directly, so the scope gate, the
closed args schema, and top-level nil-stripping went unexercised — unlike
browse_collection, which already dispatches through `registry/call-tool`.
Covers scope denial and grant (including tools/list visibility), the closed
map's teaching errors for unknown keys, wrong types, out-of-range limits and
bad enum values, the strict-client contract that an all-nulls call equals the
minimal one, the interaction between stripping and contains?-based per-action
validation, and nested nils being caught by the schema rather than reaching
the handler.
No behavior changes: every mechanism already worked. Each assertion group was
verified non-vacuous by perturbing it and confirming it goes red.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015QkpS1yAdHikDjE1hQaShC
* Build the sandbox test's card query with Lib
mt/mbql-query is deprecated since 0.61.0, and the :deprecated-var warning
fails the Clj-Kondo CI job.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015AFrL3LpdWBaQizx5QWaF5
* Move get_content's app-db queries into metabase.mcp.db
The module-boundary linter requires t2 calls to live in a <module>.db
namespace, and its warnings fail the Clj-Kondo CI job.
table-by-id deliberately omits select-one-by-id's integer precondition: a
nil table_id would raise an AssertionError, which escapes the catch
Exception that degrades one definition to nil, failing the whole batch.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015AFrL3LpdWBaQizx5QWaF5
* test(mcp): waive the mbql-query deprecation in the sandbox test ns
The full-repo kondo run reported three `#'metabase.test.data/mbql-query is
deprecated` warnings and exited non-zero. `:deprecated-var` needs
whole-project analysis to resolve the potemkin import, so the per-file lints
the file was checked with reported 0/0.
Uses the ns attr-map waiver rather than inline ignores: `:deprecated-var` is
budgeted at 86 in `.clj-kondo/ratchets.edn`, and three more would trade a red
lint for a red `kondo-ratchet-test`. The attr-map is counted by neither ratchet
and is the settled convention for this deprecation in EE test namespaces.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015QkpS1yAdHikDjE1hQaShC
* [mcp-v2 12] definitions: adapt to the base's shared contracts
The merge from mcpv2-01-core was textually clean but semantically broken
in three ways, none of which appeared as a conflict.
- `v2.write/readback` dropped its 3-arity for a single 4-arity taking
`ack-keys`. All four call sites passed three args, an ArityException at
runtime. Now pass `nil` explicitly, matching metric/collection/question.
- `registry/call-tool` now answers `{:result …}` / `{:error …}` rather
than bare MCP content. The test harness read the old shape, so every
refusal looked like a success and every result read nil — 101 failures.
`call-tool!` now normalizes both into the `{:isError true}` MCP shape,
as `content_test/call-content` already does, leaving the helpers alone.
- `metabase.mcp.db` arrived with the `:metabase/t2-query-namespace` rule
confining Toucan 2 calls to it. Both `t2/select-one :model/Table` calls
become `mcp.db/table-by-id`, whose contract already covers the
nil-tolerant column-value case these callers have.
Also: with the base's dynamic-require hook failing closed, `api-fn` trips
`:metabase/modules`. Added the sanctioned ignore and raised the budget to
24 — the dynamic resolve is deliberate, keeping `mcp` a consumer of the
segments/measures *models* rather than the *modules*.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AbCo3ybRitdVSQXYcG97QR
* kick ci
* Pass ack-keys to readback in alert_write
readback takes [token-scopes read-scopes row ack-keys]. The call site passed
three arguments, which fails to compile against the integration branch.
nil matches every other _write tool: ack-keys exempts caller-supplied keys from
the read gate on the degraded path, and alert_write returns only server-derived
state.
* Move alert_write's app-db queries into mcp.db
recipients/classify called t2/exists? and check-writable! called t2/select-one
directly, which the module-boundary linter rejects outside metabase.mcp.db.
notification-by-payload-type already existed for the select-one.
active-user-exists? is new, alongside the existing subscription-pulse-exists?.
* Unwrap the call-tool envelope in alert_test
registry/call-tool returns {:result ...}/{:error ...}. The helpers read :isError
off the bare response, so every expected-error assertion saw a nil :isError and
reported "got success".
Same shape as bookmark_test.clj on the integration branch.
* Revert slice-17 fixes that rode in on the merge
The merge resolution fixed two pre-existing defects in slice 17's files to get
this branch's lint green: `v2.write/readback` called with 3 args where it takes
4 (alert.clj, subscription.clj), and six `:metabase/t2-query-namespace`
warnings. Both belong on `mcpv2-17-query`, and carrying them here guarantees a
conflict when 17 fixes them itself.
Restores the five files to `origin/mcpv2-17-query` byte-for-byte and drops the
four `mcp.db` fns that existed only to serve them. The six browse fns 16 needs
stay.
Lint is now red on this branch with 17's own 2 errors and 6 warnings, in files
identical to 17's. It goes green when 17 does.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015QkpS1yAdHikDjE1hQaShC
* Give readback its fourth arg, move Toucan calls into mcp.db, and teach the tests call-tool's new shape
Three defects surfaced while merging 17 into 16. 16 patched them locally to get
its own CI green; they belong here.
readback takes [token-scopes read-scopes row ack-keys], but alert_write and
subscription_write passed three, so both threw an ArityException on every create
and update. Ten of the other eleven call sites pass nil for ack-keys; only
bookmark passes a value, because bookmarked is caller-supplied and has to
survive scope degradation. Neither of these returns anything the caller handed
in, so nil is right.
The t2-query-namespace hook wants Toucan query calls in the module's db
namespace. notification-by-payload-type and subscription-pulse-exists? already
existed and already fit; nullable-fields-with-fingerprints, active-user-exists?,
dashboard-parameters and delete-query-handles-created-before! are new. query.clj
and recipients.clj no longer need toucan2.core at all.
call-tool now returns {:error {:code :message}} on a pre-dispatch rejection and
{:result mcp-content} once the handler has run, rather than MCP content
directly. Six test namespaces still read the old shape, which cost 135 failures
in CI: :content came back nil and :isError never fired, so a tool error could
pass as success. The helpers take the whole outcome now and share dispatch-error?
and response-text, mirroring browse_test on 16 so the two agree. scope-gating-test
asserts the denial arrives as {:error {:message ...}}, pinning the behavior change
instead of papering over it.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016GJCEEY8W7fXmtMHUBS3Mc
* Fix subscription_write against the integration branch
readback takes four args; the call site passed three.
Two app-db queries moved into mcp.db, which the module-boundary linter requires:
subscription-pulse-exists? already existed, dashboard-parameters is new.
Test helpers unwrap the {:result ...}/{:error ...} envelope registry/call-tool
now returns, matching bookmark_test.clj.
* Drop the duplicate active-user-exists?
Slice 15 added one next to notification-by-payload-type; this branch already had
an identical definition. The merge kept both.
* Drop the duplicate active-user-exists?
Slice 15 added one next to notification-by-payload-type; this branch already had
an identical definition. The merge kept both.
* Unwrap the envelope in the surplus-schedule-field test
This case reads the response directly rather than through tool-error, so each
schedule type is asserted independently. It still read the pre-envelope shape,
so :isError was always nil and the refusal looked like a success.
* Fix dashboard_write against the integration branch
Four app-db queries moved into mcp.db, which the module-boundary linter requires:
dashboard.clj reuses the generic select-by-ids/select-one-by-id, subscription.clj
reuses subscription-pulse-exists? and a new dashboard-parameters.
Test helpers unwrap the {:result ...}/{:error ...} envelope registry/call-tool
now returns.
browse.clj still carries app-db calls here; slice 16 has already moved them and
merges first.
* Restore two mcp.db fns dropped in the integration merge
delete-query-handles-created-before! (called by the query-handle GC task) and
nullable-fields-with-fingerprints (called by v2/query.clj) are added by this
slice. Resolving db.clj wholesale toward the integration branch removed them,
so every namespace loading them failed with No such var.
* Stop import-mbql recursing forever on a numeric :source-card
content-ref? matches a portable entity_id or a bare numeric id. import-fk of a
numeric id returns a numeric id, so recurring on the whole match re-entered this
branch on its own output until the stack blew.
The :source-table branch escapes only because it rewrites the value to card__N
and stops matching. Recur on the rest of the map instead, and set :source-card
from the resolved value.
* Stop import-mbql recursing forever on a numeric :source-card
content-ref? matches a portable entity_id or a bare numeric id. import-fk of a
numeric id returns a numeric id, so recurring on the whole match re-entered this
branch on its own output until the stack blew.
The :source-table branch escapes only because it rewrites the value to card__N
and stops matching. Recur on the rest of the map instead, and set :source-card
from the resolved value.
* Stop import-mbql recursing forever on a numeric :source-card
content-ref? matches a portable entity_id or a bare numeric id. import-fk of a
numeric id returns a numeric id, so recurring on the whole match re-entered this
branch on its own output until the stack blew.
* Stop import-mbql recursing forever on a numeric :source-card
content-ref? matches a portable entity_id or a bare numeric id. import-fk of a
numeric id returns a numeric id, so recurring on the whole match re-entered this
branch on its own output until the stack blew.
* Stop import-mbql recursing forever on a numeric :source-card
content-ref? matches a portable entity_id or a bare numeric id. import-fk of a
numeric id returns a numeric id, so recurring on the whole match re-entered this
branch on its own output until the stack blew.
* Stop import-mbql recursing forever on a numeric :source-card
content-ref? matches a portable entity_id or a bare numeric id. import-fk of a
numeric id returns a numeric id, so recurring on the whole match re-entered this
branch on its own output until the stack blew.
* Pass ack-keys to readback in dashboard_write
readback takes [token-scopes read-scopes row ack-keys]; this call site passed
three. nil matches every other _write tool: dashboard_write returns
server-derived entity state, so nothing is exempt from the read gate.
* Unwrap the call-tool envelope in duplicate_test
registry/call-tool returns {:result ...}/{:error ...}; the helpers read :isError
off the bare response, so a refusal read as a success.
* Unwrap the call-tool envelope in visualize_test
registry/call-tool returns {:result ...}/{:error ...}. The helpers read :isError
and :structuredContent off the bare response, so every error assertion saw nil
and the suite reported 10 failures / 14 errors against working code.
Adds dispatch-error?/response-text matching the other v2 tool suites, and
routes the two direct (:isError (call! ...)) reads through it.
* Migrate legacy MCP OAuth clients and tokens onto the v2 scope set
Slice 21 repoints /api/metabase-mcp at the v2 tool surface, which gates every
tool on one of six coarse scopes. Every MCP client connected to a shipped
x.60-x.63 release holds a token carrying the 17 legacy per-entity agent scopes
instead. No legacy scope satisfies any v2 tool scope and registry/list-tools
filters silently, so those clients handshake successfully and then get HTTP 200
with an empty tools list - no error, nothing logged.
It does not self-heal: the refresh grant copies the original scope forward and
can only narrow, and rotation issues a fresh refresh token each time, so a
refreshing client stays broken indefinitely. Manual re-auth fails too, because
/oauth/authorize validates the requested scopes against the client's registered
snapshot before narrowing, returning a 400 invalid_request rendered raw in the
user's browser tab.
Two data-only custom migrations fix both halves:
- WidenDynamicOAuthClientScopesForMcpV2 unions the six v2 scopes into every
dynamically registered client's snapshot. This raises a ceiling, not a grant:
no token gains anything, and the user still consents per-authorization.
- RevokeLegacyMcpOAuthTokens stamps revoked_at on access AND refresh tokens
that satisfy no v2 tool scope and carry a scope outside the six. Both tables
must be revoked or the client refreshes instead of re-authenticating.
Afterwards a client's next request gets the existing invalid_token 401 with the
discovery challenge, re-authenticates against a snapshot that now validates, and
lands with the six scopes - the RFC 6750 path every MCP client already handles.
A 403 insufficient_scope step-up was rejected deliberately: step-up is currently
broken in both Claude Code and Codex, so it would leave clients stuck.
The scope strings are literals rather than a read of mcp.paths/v2-surface-scopes
so the migrations' behaviour is frozen against later edits to that vector. Both
are idempotent and safe to re-run.
Tests: a row-level migration test over the changelog, an end-to-end transport
test that characterizes the empty-tool-list bug and then pins the fixed
behaviour, and an authorize test proving the 400 becomes a consent page.
* Re-apply the browsable-database filter to FK neighbour tables
related_tables filtered only on mi/can-read?, which evaluates per-table perms
against :db_id and has no notion of a router destination: a destination reads as
affirmatively readable because the Database read check redirects to its router.
An FK from a still-router table into a table whose :db_id was reassigned carried
the destination's column names into the response by the neighbour path, which
the directly-requested path already refuses. browsable-db-ids is already computed
for that path; thread it through.
* Make the destination-permission tests reach the create-check
Both tests asserting the destination create-permission refusal wrapped the
destination in `with-non-admin-groups-no-collection-perms`, which revokes
*read* as well as curate. `destination-collection-id` resolves the argument
through `resolve-collection-id`, whose documented behaviour is to collapse
"doesn't exist" and "exists but not readable" into one not-found error so the
argument never reports the existence of a collection the caller cannot see --
so the call was refused there, before `api/create-check` ever ran, and both
tests failed on the tightened `#"(?i)permission"` assertion introduced with
the `check-allowed-to-create-card!` removal.
Grant read on the destination so it is readable but not curatable, which is
the state the create-check exists to refuse. Matches the sibling case in
`collection_test.clj`, which grants read for the same reason and notes that a
not-found collapse there would mean the destination check had stopped running.
* Move app-db queries out of dashboards/write and pulse/broken-subscriptions
The app-db linter requires t2/... calls to live in <module>.db namespaces.
Slice 18's write.clj and broken_subscriptions.clj ran queries inline, which
Kondo flagged once this branch merged with mcpv2-01-core (where both the
linter and the .db namespaces live).
Each moved call is a direct Toucan 2 call in the module's .db namespace with
no added logic; call sites keep identical semantics. Reused existing helpers
where they already covered the query (dashboard, update-dashboard!, card,
card-queries, dashcards-by-ids, insert-dashboard!, insert-dashboard-tabs!).
Also regenerates .clj-kondo/config/modules/config.edn, restoring pulse and
query-permissions to dashboards-rest :uses along with the model imports the
generator derives.
* Ship only the scope-widening half of the MCP v2 OAuth migration
Widening a dynamic client's registered scope snapshot grants nothing: no token
gains a scope, and the user consents to the six on the new screen. It is safe to
run on upgrade.
Revoking the legacy-shaped tokens is the half that forces the re-authentication,
and it logs every connected MCP client out once. That is a release decision, not
a migration detail, so RevokeLegacyMcpOAuthTokens stays written and tested but is
not wired into a changeset. The test pins that no token is revoked yet, so wiring
it up later is a visible change rather than a silent one.
* Revert "Ship only the scope-widening half of the MCP v2 OAuth migration"
This reverts commit 88fec4366f.
* Pass token-scopes to issue-ui-credential in callback_api_test
This slice retires the 2-arity that minted claimless, gate-exempt :legacy
credentials. Four call sites here still used it and threw ArityException.
* Drop the duplicate check-semantic-type! kondo entry
Resolving .clj-kondo/config.edn by unioning both sides of the conflict added an
entry the file already carried further down, which made the EDN unparseable:
Duplicate key: metabase.mcp.v2.tools.question/check-semantic-type!
A broken config takes down Clj-Kondo and kondo-ratchet-test together.
* Take the integration branch's browse_test
Resolving this file's conflict earlier kept a comment from this branch, and with
it the whole pre-slice-16 file. assemble-tables no longer returns :omitted, so 6
failures and 23 errors followed. This branch adds no test the integration branch
lacks.
* Drop opt-in-scopes with the plumbing it called
Slice 20 removed registered-opt-in-scopes from the v2 registry; keeping this
wrapper left a call to a function that no longer exists, and nothing else
referenced it. The v2.registry require goes with it.
* Fix the last v1-retirement fallout: arity, unused require, module config
visualize_test still called the 2-arity issue-ui-credential this slice retires.
usage_test required metabase.mcp.v2.api without using it, and marked a test
^:parallel that calls register-tool! — which mutates the shared registry.
Retiring v1 removes namespaces, so the module boundaries moved; regenerated
.clj-kondo/config/modules/config.edn.
* Point usage_test at v2's behaviour on the retired v1 alias
The bare /api/mcp alias now serves v2. Three assertions still described v1:
- a direct call-tool result read :isError off the bare response, but v2 returns
{:result ...}/{:error ...}
- an unknown tool is a registry rejection ({:error ...}) rather than a handler
result carrying :isError
- DELETE tore the session down and stamped ended_at on v1; v2 answers 405, so
nothing on that path ends the row
The DELETE case is a real gap, not just a test change: record-mcp-session-end!
is now callerless. Filed as GHY-4492 rather than wired up here, since choosing
where a v2 session ends is a design decision.
* Fix unregistered card-type schema keyword in check-card-can-be-saved!
`::queries.schema/card-type` was never registered; the schema is
`::card.type`. Malli threw `invalid-schema` at validation time, which
surfaced as an HTTP 500 where callers expected a 400/403.
The typo predates this merge, but nothing reached it: the REST card
endpoint still carried its own inline copy of this check, so the shared
fn had no live caller. Dropping that duplicate during the merge routed
the endpoint here and exposed it.
* GHY-4400 Give the MCP UI credential a purpose-built API (#82047)
* GHY-4400 Give the MCP UI credential a purpose-built API
`refresh_ui_credential` was a scope-escalation primitive. A token granting
only `agent:query:run` is refused `/api/user/current`, but the UI credential
minted from it was served the full profile — email, login and JWT attributes,
SSO source, group ids. Reproduced against the base commit; `/api/session/
properties` was worse, handing an admin's credential 400 settings keys against
140 for an anonymous caller, including SMTP, Slack, SAML, LDAP and embedding
secret configuration.
Both endpoints were reachable because `mcp-ui-request-surface` was an inventory
of what the embedded app happened to call rather than a decision about what the
credential should reach, and because the middleware stamped
`:token-scopes-checked true` unconditionally — which silently defeated the
`::scope/mcp-ui` stamp it sat next to. The comment claiming this "fails closed"
was wrong; it does now.
- Add `GET /api/embed-mcp/bootstrap`, returning the user and settings the
iframe needs. The user projection is a closed schema, so widening it is an
edit someone makes on purpose. Settings come back at non-admin visibility
whoever holds the credential.
- Replace the allowlist with `[method uri] -> required scope`. Query routes
and drills cost `agent:query:run` — the scope that already bought the iframe,
since the v2 shell resources declare it. `:token-scopes-checked` is set only
when the credential's signed claim covers the route, so a route added without
a scope decision is refused rather than served.
- Drop `/api/user/current` and `/api/session/properties` from the surface, and
seed the RTK caches from bootstrap instead, mirroring what the SSO path in
`embedding-sdk-ee/auth/auth.ts` already does.
The credential still reaches endpoints its minting bearer cannot: all seven
declare no `:scope`, so `ensure-scopes-checked` 403s a scoped bearer on each.
That delta is what the credential is for; what it is not is a place for routes
to accumulate. `request-surface-is-a-reviewed-allowlist-test` pins the set.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_019r4TtdH2RQy1eGbQ5odnjr
* Record why the seeded caches are safe, and how they could stop being
Verified rather than assumed, against RTK 2.12.0: neither `initiate()` on a
freshly upserted entry issues a request. The SSO path in
`embedding-sdk-ee/auth/auth.ts` runs the same sequence, but its endpoints stay
reachable, so a spurious refetch there would succeed silently — it corroborates
nothing.
The subscription those calls register is also the one way this design breaks. An
invalidation of `current-user` or `session-properties` refetches, and both now
fail — the second one fails open, answering anonymously with public-only
settings rather than erroring.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_019r4TtdH2RQy1eGbQ5odnjr
* Build the scope-gate test's query without a deprecated macro
`mt/mbql-query` has been deprecated since 0.61.0, and `mage kondo` exits
non-zero on warnings, so the one call reddened CI. Per-file linting cannot see
it: `:deprecated-var` needs the defining namespace's analysis cache, so the file
reads clean in isolation and only the whole-tree run reports it.
The test needs nothing from that macro beyond a valid non-native query, so
spelling out the map it expands to keeps the scope gate as the only thing
separating the two statuses.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_019r4TtdH2RQy1eGbQ5odnjr
* Test the MCP iframe against the bootstrap projection, not a full user
Nothing on the frontend would have failed if the projection dropped a field the
render tree needs. The hook spec mocked the API module and seeded `{id: 1}`, so
it stayed green whatever the payload became, and the one MCP spec that mounts a
real `SdkQuestion` seeds a full `createMockUser()`.
- `createMockMcpAppsBootstrapUser` is an object literal typed as
`McpAppsBootstrapUser` rather than cast, so the build breaks if the type loses
a field.
- `createMockMcpAppsBootstrapSettings` picks the keys the backend's
`bootstrap-setting-visibilities` actually allows, derived from the setting
registry rather than guessed. The existing SDK state helpers re-expand
whatever they are given through `createMockSettings`, which would put the
admin-only keys back and make the exercise vacuous.
- `McpQuestionView.unit.spec.tsx` mounts the real question view with the store
seeded only from those fixtures, and asserts a chart actually renders.
What this does and does not catch, measured rather than assumed. Removing
`token-features` from the seeded settings turns the new spec red. Removing
`locale`, `permissions` or `is_superuser` from the user does NOT: every
consumer on the reachable path reads them optionally and defaults, so nothing
observable changes. Those three have compile-time coverage only. Runtime
coverage for them needs assertions on behavior they drive — a non-`en` locale
changing rendered text, a drill menu gated on `can_create_queries` — which is a
different test than this one.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_019r4TtdH2RQy1eGbQ5odnjr
* Stop asserting the UI credential reaches /api/user/current
v1's copy of the surface test still expected the pre-GHY-4400 behavior. It is
the twin of the v2 test already updated on this branch, and the only remaining
place asserting the old contract.
Its credential is minted through v1's `refresh_ui_credential`, so it is
`:legacy` — which makes this the one test pinning what a legacy credential can
still reach, coverage lost when `bootstrap-accepts-ui-credential-test` moved to
the scoped arity.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_019r4TtdH2RQy1eGbQ5odnjr
---------
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
* Sort the malli.util require in collections.db
Clj-Kondo and Cljfmt were both red on this file: kondo reported
`Unsorted namespace: malli.util`, cljfmt reported incorrect formatting. Same
defect from two angles — `c8d1ef7b47c`'s merge resolution placed the branch's
`[metabase.app-db.core]` above master's `[malli.util]`.
Full-tree kondo is clean afterwards, and no other file carries the same
problem.
This does not address the rest of that merge: `collections_rest/api.clj` still
has master's `/root/items` refactor reverted, which is what is failing ~164
backend assertions and 20+ e2e groups.
* Finish master's kebab-case rename on /root/items (GHY-4495)
Master's #79863 renamed the collection-items query params from snake_case to
kebab-case — `pinned_state` -> `pinned-state`, `include_library` ->
`include-library`, and the rest — updating the frontend and tests with it. Merge
`c8d1ef7b47c` applied that rename only PARTIALLY: `/root/items` kept the old
names while its own sibling `/root/items/metadata` had already been renamed, and
the test file was left similarly half-converted.
The endpoint schemas are `:closed true`, so an unrecognised param is ignored
rather than refused. Every filter silently stopped applying: `pinned-state=
is_not_pinned` returned pinned items, `include-available-models=true` returned
no `available_models`, and `collection-type` did not filter. The frontend was
already sending kebab-case, which is why this broke collection browsing broadly
rather than in one place.
Completing the rename is the whole fix:
- collections_rest/api.clj — the 17 occurrences in `/root/items`
- collections_rest/api_test.clj — 7 call sites the merge left snake_case
- collections/children.clj — docstrings still naming the old spelling
collections-rest: 186 tests, 950 assertions, 0 failures / 0 errors.
documents.api.collection-test: 0 failures. Full-tree kondo 0/0, cljfmt clean.
No remaining snake_case call sites anywhere in src/test/enterprise/e2e/frontend.
Note for GHY-4495: master's `children_query.clj` did NOT need restoring and
`collections/children.clj` did NOT need deleting. The branch's extraction (slice
16, for `browse_collection`) is fine as it stands; the failure was the rename,
not the refactor.
* Read ProseMirror node attrs by string key (GHY-4496)
Master changed ProseMirror node `attrs` from keyword keys to string keys and
made it a schema-enforced invariant: `::node.attrs` is a `string-keyed-object`
whose docstring states "every key, declared or not, is a string, so the map
never mixes keyword and string keys". The `mcpv2-01-core` merge (c8d1ef7b47)
took master's string-keyed `prose_mirror.clj` but kept the branch's
keyword-keyed consumers, so every read silently missed.
Silent, not loud: these produce wrong values rather than exceptions, and CI
was quiet about all of it.
- `update-cards-in-ast` never matched, so a card created inside a document kept
its negative placeholder id instead of being rewritten to the real one.
- `markdown/serialize` dropped a code block's language and rendered every
heading as h1, because `(:level attrs)` / `(:language attrs)` read nil off a
stored AST and fell through to their defaults.
- `changed_blocks` came back empty from document_write, since block identity is
keyed on `_id`.
- `get_content` reported `card_id` nil for every card embed, and comment
anchoring resolved no ancestors.
Every AST reaching this code from the app db is string-keyed: `:model/Document`
runs `prose-mirror/normalize-document` as an `:out` transform, and
`string-keyed-object` stringifies attrs keys on decode. Verified by round-tripping
a real Document row -- wrote `{:level 2}`, read back `{"level" 2}`.
Two sites are deliberately asymmetric: `markdown.clj`'s `segment->nodes` and
`container-node` read their own keyword-keyed parse structures and emit
string-keyed prose-mirror nodes. Converting both sides there would have broken
the parse path.
The test files hand hand-written ASTs straight to `parse`/`serialize`, bypassing
normalization, so their fixtures encoded a shape production never produces.
Converting them turned up two further defects in this commit that the source
sweep alone missed: the smartLink fallback serialized an empty label, and
`ast-id->type`/`changed-blocks` still read `:_id`.
Behaviour is unchanged wherever it was already correct -- the diff is 209
insertions against 209 deletions, all key style.
Verified: metabase.documents.markdown-test (352 assertions), plus
prose-mirror-test, search-test, mcp.v2.tools.content-test,
mcp.v2.tools.document-test, comments.render-test, search.ingestion-test --
864 assertions, 0 failures. `metabase.documents.api.document-test` improves
from 58 failures to 9; the remaining 9 (`copy-document-test`) fail identically
on the unpatched branch and are unrelated to attrs.
kondo 0 errors 0 warnings; cljfmt clean.
* Point McpQuestionView's mock import at __support__/state (GHY-4497)
The merge (c8d1ef7b47) left this spec importing `metabase/redux/store/mocks`,
a path master no longer has, failing `frontend-tests / fe-type-check` with
TS2307. Both `createMockEmbedState` and `createMockState` live under
`__support__/state` -- the barrel re-exports `./embed` and `./state` -- which is
where the rest of the tree imports them from (see
`embedding-sdk-bundle/test/server-mocks/sdk-init.ts`).
`bun run type-check-pure` goes from 66 errors to 65: this one, and nothing else
introduced. The other 65 are a missing ClojureScript build in a fresh worktree
(`cljs/*` TS2307) plus pre-existing @types/node mismatches (`Timeout` vs
`number`, `ImportMeta.dirname`), all present before this change and none in
files it touches.
The audit's second item for this ticket -- regenerating
`.clj-kondo/config/modules/config.edn` -- turned out to be a no-op:
`./bin/mage fix-modules-config` reports "config.edn already up to date", and
`metabase.core.modules-test` passes with 4492 assertions. GHY-4495 restored the
`metabase.pulse` `:api` entry the audit found dropped, so the drift was gone by
the time the ordering allowed this to run.
* Declare the MCP keyset marker in the clause options schema
MCP v2's keyset paging stamps a private marker on the filter clauses it mints,
so the next page can supersede its predecessor instead of stacking a dead
predicate beside it. Master's #82196 made `lib.schema.common/::options`
`:closed true` and enumerated every namespaced option key the platform stamps —
Mongo's `join-local`, SQL Server's `optimized-bucketing?`, the add-alias-info and
add-remaps keys. It could not have included this one: MCP v2 lives on this
branch.
So `lib/filter` began rejecting the stamped clause:
Invalid input: [nil #:metabase.mcp.v2.query{:keyset ["disallowed key, got: true"]}]
`next-page-cursor!` treats any failure to build a cursor as "this query cannot be
paged" — correct, since refusing to page is always safe — so the rejection
surfaced as cursors silently never being minted rather than as an error. That is
why it read as a guard refusal: `metabase.mcp.v2.query-test` failed 1 assertion
and errored 8 more with "Query handle not found", all downstream of a nil cursor.
Bisected: passes at `eea6bb9f914` (slice 17's own merge) and `a8024807cbc`,
fails at `c8d1ef7b47c` (the master merge). The merge changed none of
`mcp/v2/query.clj`, `mcp/v2/queries.clj` or `agent_api/query_guards.clj` — it
broke them through Lib.
Registering the key is the step #82196 performed for every other option-stamping
module, not a workaround. `{:optional true}`, so no existing query is affected.
`mcp/v2/query.clj`'s logic is unchanged: the GHY-4142 refusal contract — no
cursor unless gap-free paging is provable — is intact.
Also here:
- `internal-keys-fixture-covers-the-schemas-test` derives the declared key set
from the schema and requires a fixture entry for each, so it broke by design
until the fixture gained one. Adding it also gets the marker covered by the two
stripping tests, which is worth having.
- The `keyset-marker` docstring said the marker "rides in a clause's options map,
which no schema declares" and that undeclared keys would merely be stripped.
Both are now false — they are rejected — so it says what actually holds.
metabase.mcp.v2.query-test 62 assertions, metabase.lib.serialize-test 31, both
0 failures / 0 errors. Full-tree kondo 0/0. `:module query-processor`'s 8
failures are pre-existing, reproduced at a clean checkout of the same HEAD.
* GHY-4498 Accept a numeric id sent as a JSON string (#82294)
Accept a numeric id sent as a JSON string (GHY-4498)
Claude Desktop serializes every value of an `anyOf [integer, string]` param
as a string, so `run_saved_question` received `id: "16211"` and rejected it —
while the advertised schema's string branch says that value is valid. The
model cannot force the JSON type, so the numeric id was unreachable from
those clients and only the entity_id worked.
`normalize-id` coerces strings matching `^[1-9][0-9]*$` to integers, at three
chokepoints that between them cover every id param: `resolve-id-or-404` (the
read tools, and every collection_id/parent_id through `resolve-collection-id`),
`dispatch-write` (the `id` of all ten `_write` tools), and `get_content`'s
per-item fetch, which needs its own because alerts and subscriptions have no
entity_id column and reject a non-integer id before any resolution runs.
The `"root"`/`"trash"` sentinels are matched before coercion, and digit runs
too large for a long stay strings and fail validation like any other non-id.
Schemas are unchanged: the string branch already admitted these values.
The teaching error now names a retry that works — reading "pass a numeric id"
the obvious next move is to resend the same number, which fails identically.
Claude-Session: https://claude.ai/code/session_01EWoDYrBS4e4dAWaDeWLpVV
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
* Send alert_write's update only the fields the write schema accepts
Master's merge brought a new `notification/schema.clj` and closed every
notification schema, including the `hydrated-notification-schema` behind
`::NotificationApiUpdateInput`. `alert_write`'s update path was passing back the
whole hydrated notification from `get-notification`:
(notification.api/update-notification! id (assoc existing :payload payload))
An open map tolerated the read-only hydration keys. A closed one rejects them:
:creator ["disallowed key"], :id, :created_at, :updated_at
:payload {:id, :created_at, :updated_at, :disable_links, :card ["disallowed key"]}
`->mcp-error-content` sanitizes any non-caller-facing exception to a bare
"Internal error" and logs the original at a level the test runner suppresses, so
every alert_write update — including simply pausing an alert with
`{:active false}` — failed with no usable message. 7 failures / 24 errors.
The body is now built from the write shape rather than trimmed down from the read
shape, so a future hydration key cannot reintroduce this. `:subscriptions` and
`:handlers` still carry their existing rows when untouched, because the update
spec deletes whatever a body omits.
Bisected: passes at `a8024807cbc`, fails at `c8d1ef7b47c` (the master merge).
metabase.mcp.v2.tools.alert-test: 35 tests, 141 assertions, 0 failures / 0 errors.
kondo 0/0, cljfmt clean.
Not fixed here: `mcp.v2.tools.dashboard-test`'s 2 failures / 4 errors are a
DIFFERENT cause and unchanged by this. `metabase.dashboards.db/update-dashboard!`
types its `changes` as `(mut/select-keys ::dashboards.schema/dashboard.update
[:parameters])`, so any update touching `:description` or `:cache_ttl` is refused
— a dashboards-module bug, not a notification one.
* Say in bookmark_content's description that nothing reads a bookmark back
Nine of ten fleet workers, told to confirm a bookmark they had just written,
spent three or more calls establishing that no read path exists: grepping all
22 tool schemas for "bookmark" (it appears only inside `bookmark_content`),
checking `get_content`'s `include` enum, checking `search` for a `bookmarks:`
mode symmetric with its existing `recent:`, and reading the generated fields
catalog. Each concluded correctly that the write's own response is the only
report of bookmark state, but only after spending the calls to prove a
negative.
The absence is a real gap, not a defect to paper over: `search` and
`get_content` would each need the row hydrated with a per-user bookmark flag,
and `metabase.mcp.v2.projections` is by contract pure -- "no app-DB access and
no read of the current user" -- so a `bookmarked` field belongs in the fetch
layer, not the projection. That is a larger change than this one.
What is cheap now is telling the agent, so it stops paying to discover it.
* Say in search's `recent` description that it tracks UI views only
Five of ten fleet workers read content with `get_content`, then called
`search(recent: true)` expecting to see it, and got an empty list. Several
could not tell "these reads are not recorded" from "the recents index has not
caught up yet", and spent a call each establishing which.
Recents come from the view log, which the REST read paths write and the MCP
tools do not. That is the existing behaviour and this commit does not change
it -- an agent's tool calls are arguably not "views" in the sense the sidebar
means. It was simply never written down where the agent would look.
* Match master's response schema on the document copy endpoint
`POST /api/document/:id/copy` was 400ing on its own response:
"Invalid response" {:creator ["should be spelled :creator_id, received: #Instance :model/User ..."]}
The endpoint declared `:- ::documents.schema/document` and returns
`copy-document!`'s result, which is `get-document` — hydrated with `:creator`,
`:can_write`, `:can_delete`, `:can_restore`, `:is_remote_synced`. That was fine
while `::document` was open. The master merge brought its closed version, which
describes the app-DB row only ("every column of `:document`"), so a hydrated
document no longer satisfies it.
Master's own copy endpoint declares `:- [:map [:id ::documents.schema/document.id]]`
— an OPEN map that asserts an id is present and permits the rest. Its test asserts
on `:name`, `:collection_id` and the document AST, so master returns the whole
document too; the schema is permissive rather than descriptive. Adopting that line
verbatim leaves our declaration identical to master's, so there is nothing to
reconcile at the eventual merge.
Considered and rejected: adding a `::document.hydrated` schema. It worked, but put
new surface in a shared module to solve a branch-local problem, and diverged from
master where matching it costs nothing.
metabase.documents.api.document-test: 100 tests, 546 assertions, 0 failures /
0 errors — was 9 failures / 1 error. kondo 0/0, cljfmt clean.
* Let update-dashboard! take the schema its name promises
`dashboards.db/update-dashboard!` typed its `changes` as
`(mut/select-keys ::dashboards.schema/dashboard.update [:parameters])`, so every
attribute except `:parameters` was refused:
Invalid input: {:description ["disallowed key, got: nil"],
:cache_ttl ["disallowed key, got: nil"]}
`::dashboard.update` describes itself as "what an update (or insert) of a
Dashboard accepts", and `insert-dashboard!` directly above takes the whole thing.
Update accepting strictly fewer fields than insert was the anomaly; the narrowing
fit when the only caller wrote nothing else.
This branch added such a caller. `metabase.dashboards.write/update-dashboard!`
(new in `c0ddd5fcd4a`, GHY-4147 — the file does not exist on master) selects
`:name`, `:description`, `:archived`, `:collection_id`, `:cache_ttl`, `:width`,
`:position` and more before delegating here, so every dashboard_write touching an
attribute failed. Not merge damage: `dashboards/db.clj` is identical to master,
and the incompatibility is our new caller meeting master's narrower type.
mcp.v2.tools.dashboard-test: 2 failures / 4 errors -> 1 failure / 0 errors.
:module dashboards 114 assertions, 0 failures. :module dashboards-rest 1020
assertions with one pre-existing failure —
`dashboard-card-query-export-format-no-self-service-test`, an EE
download-permissions 403 — verified by running it with this change reverted.
kondo 0/0, cljfmt clean.
The remaining dashboard-test failure is GHY-4500, a separate bug: our copy of
`check-parameter-mapping-permissions` in `dashboards/write.clj` kept the narrow
`::parameter-mapping` schema that #82196 widened on the REST side, while still
assoc'ing `:dashcard-id`/`:card-id` onto every mapping.
* Declare the mapping keys the permission check is handed (GHY-4500)
Every `dashboard_write` that added or changed a parameter mapping —
`wire_parameter` by `target_tag`, `target_field` or raw `target`, and
`patch_dashcard` with `parameter_mappings` — returned "Internal error". Filters
could not be built through the MCP at all. `validate_only: true` succeeded on the
same ops, since it never reaches the check.
#82196 closed `::parameters.schema/parameter-mapping` to
`{parameter_id, target, card_id}`, and widened the REST copy of
`check-parameter-mapping-permissions` to also allow `:dashcard-id`/`:card-id`.
This branch has a second copy of that function in `dashboards/write.clj` — a file
that does not exist on master — which kept the narrow schema while lines 167/174/
183 still assoc those two keys onto every mapping before calling it:
Invalid input: [{:dashcard-id ["disallowed key, got: 177"],
:card-id ["disallowed key, got: 166"]}]
`mu/defn` throws `::invalid-input`, which carries no `:status-code`, so the MCP
sanitizer reported it as "Internal error" — see GHY-4502.
The schema now matches the REST copy verbatim, since this function is a fork of
it and the two drifting is what caused this.
Dev and test only: `mu.fn/instrument-ns?` skips prod, so schema checks do not run
there. That still covers every developer machine and CI, and the frontend was
never affected because it goes through the REST copy.
mcp.v2.tools.dashboard-test: 26 tests, 65 assertions, 0 failures / 0 errors — was
1 failure. :module dashboards green. kondo 0/0, cljfmt clean.
* Say in search's description that an empty result can be a cold index
A fleet worker searched five times for content that provably existed and got
`{"data":[],"returned":0,"total":0}` every time. Its instance's active search
index held one row where healthy peers held sixty-six -- indexing had not
finished when the worker started. Nothing in the response distinguished that
from a genuine no-match, so the worker concluded the content was absent and
abandoned search for manual browsing, turning a two-call job into about eleven.
The honest fix is for the envelope to carry the index's status, but nothing
readiness-related is exported from `metabase.search.core` today, so the tool
cannot ask. Until it can, this at least tells the agent that an empty result
has two causes and names the tools that do not depend on the index.
* Advertise the v2 scope set on every MCP path now that v1 is retired
`mcp-resource-scopes` branched on the path: `/api/metabase-mcp/v2` got the six
scopes the v2 tool registry gates on, and the other aliases got
`all-scopes` — twenty, including the fourteen per-entity agent-API scopes
(`agent:question:create`, `agent:sql:execute`, …) that v1's tools gated on.
That was right while those paths still served v1: narrowing them to the v2 set
would have told a client to request scopes no v1 tool accepted, leaving it with
an empty `tools/list`.
Retiring v1 (#81708) pointed every path at the same v2 handler and inverted the
bug. The canonical `/api/metabase-mcp` — the path the docs advertise, the one
`.well-known/oauth-protected-resource` names, and the one clients probe — served
v2 tools while advertising fourteen scopes nothing gates on, putting them on a
consent screen for capabilities that no longer exist. The old docstring
anticipated this and said the fn "collapses to the v2 answer" when v1 retires;
this is that collapse.
/api/metabase-mcp 20 scopes -> 6
/api/mcp 20 scopes -> 6
/api/metabase-mcp/v2 6 scopes -> 6 (unchanged)
`path` stays in the signature: RFC 9728 metadata is per-resource, and a future
surface may diverge again.
The tests changed because they pinned the branch itself — "a v1 alias keeps the
scopes v1's tools gate on", and an assertion literally computing
`(= (not= path "/api/metabase-mcp/v2") …)`. They now assert every path answers
the same set, which is the property that will keep holding.
No user-facing path changes: `docs/ai/mcp.md` and `llms-install.md` already
advertise `/api/metabase-mcp`, and `/api/metabase-mcp/v2` stays routed for
client configs written while the surface lived there.
Verified: 726 assertions across oauth-server.core-test, oauth-server.api-test,
mcp.v2.api-test and mcp.transport-test, 0 failures. kondo 0/0, cljfmt clean.
* Name the function when a server-side schema check fails (GHY-4502)
Any exception without a 4xx `:status-code` or an explicit MCP error code became
the two words "Internal error". The real exception was logged server-side, which
an agent cannot read and which on a dev box goes to somebody else's terminal.
That cost real time three times over on this branch alone — alert_write's
hydrated body, dashboard_write's parameter mappings, and the document copy
endpoint were all `mu/defn` schema failures reported as "Internal error", each
needing instrumentation past the sanitizer to see. The ticket records an agent
spending nine failing calls over two turns retrying variations, learning nothing,
because every answer was the same two words.
A `mu/defn` failure now names the function:
Server-side schema check failed in `check-parameter-mapping-permissions`:
[{:dashcard-id ["disallowed key, got: 177"]}]. This is a bug in Metabase, not
something to retry — report it.
The message says it is a bug rather than a caller mistake, so the agent reports
it instead of retrying variations of a call that cannot work.
Deliberately narrower than the ticket asked, on one point. The humanization is
included only for `::invalid-input`, where it describes the caller's OWN argument
echoed back. `me/humanize` embeds offending values ("disallowed key, got: 177"),
so for `::invalid-output` — which describes SERVER-produced data — echoing it
could hand the caller a row they have no right to read. Those name the function
and stop. `:value` is never touched in either case: it holds the whole argument
or return value.
Everything else still redacts: the existing SQL, JDBC and NPE cases in
`error-redaction-test` are unchanged.
The correlation-id half of the ticket is not done here.
mcp.v2.common-test: 13 tests, 91 assertions, 0 failures. :module mcp 3487
assertions with one pre-existing error in `query-test`
(`unknown-caller-key-never-reaches-the-qp-test`, a query-validation teaching
error) — verified by running it with this change reverted. kondo 0/0.
* Roll back the dashboard row when a create's second save fails (GHY-4501)
`dashboard_write` `create` with ops writes the dashboard row, then applies the
ops in a second save. When that second save threw, the row stayed. The tool's own
description tells the agent to "find it by name and finish it with update", so in
the field the agent read the error, called `create` again, and again — leaving a
pile of empty dashboards.
There was already a mitigation: the ops are compiled against a blank dashboard
before anything is written, so a bad OP cannot orphan a row
(`create-with-a-bad-op-writes-nothing-test`). It cannot catch a failure that is
not in the ops. The reported case was a `wire_parameter` create failing inside
`check-parameter-mapping-permissions` — a schema rejection the dry run never
reaches, because it does not perform the permission check.
The two writes now share a transaction, so anything throwing after the row exists
takes the row with it, whatever the reason.
Fixing GHY-4500 removed the reported trigger, but not the window: any post-create
failure had the same effect. The new test forces one directly rather than relying
on a bug to produce it, so the guarantee is pinned independently of whatever
happens to throw next.
mcp.v2.tools.dashboard-test: 27 tests, 67 assertions, 0 failures / 0 errors.
mcp.v2.dashboard-ops-test green. kondo 0/0, cljfmt clean.
* Retire the /api/metabase-mcp/v2 alias everywhere
`/api/metabase-mcp/v2` is where the v2 surface lived while it was behind a flag.
The switchover repointed the canonical path and `/api/mcp` at it, so the alias
had nothing left to do.
Removing it from `endpoint-paths` alone was not safe: the path kept serving
traffic while dropping out of everything derived from that set. In particular
`narrow-scope-to-resources` would no longer match an RFC 8707 indicator naming
it, and that path fails OPEN — an unmatched indicator narrows nothing, so a
client naming `/v2` would have kept `mb:full` where it previously lost it.
Carried through all five places:
- `mcp/paths.clj` — out of `endpoint-paths`; the docstring now names the three
things derived from the set and says what a drifted path costs.
- `mcp/core.clj` — `mcp-v2-path` deleted. It still read `mcp.paths/v2-path`,
which was already gone, so `metabase.mcp.core` would not load at all.
- `api_routes/routes.clj` — the `/v2` sub-route dropped; `/metabase-mcp` is now
a single handler.
- `oauth_server/api/metadata.clj` — the `/v2` RFC 9728 document removed. It
advertised `:resource` as the `/v2` URL, telling clients to send an indicator
nothing would recognise.
- tests — repointed at the canonical path, as the migration comments said the
switchover would ("The final switchover PR repoints this at metabase-mcp").
`oauth_server/core_test` needed real edits rather than a find-and-replace:
`/v2` was one of three paths in the union tests, so swapping the string would
have left them comparing a path against itself and quietly asserting nothing.
mcp.v2.api-test 134 assertions, transport-test 218, oauth core+api 338 — all 0
failures / 0 errors. :module mcp 3485 assertions, one pre-existing error in
query-test. Full-tree kondo 0/0. No reference to the retired path survives
outside one historical note in the paths docstring.
* Apply cljfmt to mcp.v2.tools.search
Whitespace only; the Cljfmt CI job was red on this file.
* Fix MCP Apps dropping bound template-tag values when rendering a handle
`execute_sql` mints a query_handle whose payload is deliberately wider than a dataset
query: the values bound to {{tag}} placeholders ride alongside it under `parameters`,
because `prepare-for-serialization` strips `:parameters` from the query as runtime-only.
`getMcpDeserializedCard` parsed that whole payload into `dataset_query`, so the bound
values ended up nested where nothing reads them, and the run assembled its parameters
from the card instead. Visualizing a handle re-ran the SQL with every {{tag}} unbound.
Split the payload instead: the dataset query becomes the card, and the bound values
become `initialSqlParameters`, keyed by template-tag name. That is the form the SDK
matches against the question's tag-derived parameters (by slug). Putting them on the
card would not work — they carry no `id`, and a card-level `parameters` array
short-circuits the very tag-derived list they need to be matched against.
* Restore the response schema on GET /collection/root/items
The merge from master kept the branch's endpoint body, which predates master adding
`:- ::ItemsResponse`, along with two suppressions that the branch no longer needs: the
kebab-case one went stale when the query params were renamed, and the response-schema
one put the tree one over budget on two ratchets.
* Name the required scope in the scope-denial error
The refusal read "Insufficient scope to call tool: bookmark_content" — the tool
the caller already named, and nothing it could act on. A fleet worker holding a
read-only token recorded that this contradicts the server's own `initialize`
instructions, which promise "a failed call always names its fix", and that it
had no way to learn which scope to request.
Both values are in hand at the branch: it fires on
`(not (mcp.scope/matches? token-scopes (:scope tool)))`, so the required scope
and the held set are bound right there. The message now carries them:
Insufficient scope to call tool: bookmark_content.
Requires agent:content:write; your token holds agent:content:read.
`:scope` may be a set of alternatives ([[mcp.scope/matches?]] accepts either),
which reads as "Requires one of a, b"; `token-scopes` may carry the
`::api.scope/unrestricted` keyword, which is not a scope a caller can request,
so only strings are listed back.
The tests changed because twelve of them pinned the old string exactly. They
now match an anchored prefix, so the tool name each one cares about is still
asserted while the scope detail is free to change; `registry-test` asserts the
full new shape, including the held-scope clause.
Verified: `:module mcp`, 3525 assertions, 0 failures. The one remaining error
(`query-test/unknown-caller-key-never-reaches-the-qp-test`) reproduces
identically on the unpatched tree — it is the closed-options-schema breakage
from the master merge, unrelated to this change.
* Return collection_path from get_content, as search already does
`get_content` returned a bare `collection_id`, so an agent asked where an item
lives had to spend a second call resolving the collection. Two fleet workers hit
this on the same task; one made the extra call and then noted it was redundant,
because `search`'s own rows had carried `collection_path` all along.
The path walk moves to `v2.resolve` so both tools share one implementation
rather than forking the permission handling: ancestors the caller cannot read
are omitted rather than failing the path, matching
[[collection/effective-ancestors]]'s breadcrumb semantics (A > B > C with B
unreadable reads "A/C"). `search` keeps its own batched version — it walks whole
result pages and its two lookups are per page, not per row.
`collection_path` joins `question-enrichment-keys`, the set of projection keys
computed at read time rather than selected as Card columns, so the column-select
paths (`list_models`, browse rows) drop it as they already do for
`query_summary` and `template_tags`.
Verified against a live instance: card 20 returns
`"collection_path":"Examples"`, matching what `search` reports for the same
card, and a root item returns nil rather than an empty string.
* Move the GHY-4313 unknown-key test onto the path the whitelist protects
Master closed `:metabase.lib.schema/query` in #82196, so a fresh `query` carrying an
unknown top-level key is now rejected upstream rather than reaching the QP input
whitelist. The test asserting the key is dropped ran on that path, so it errored on the
rejection before exercising the guard it exists to prove.
The guard is `select-keys` against `query-passthrough-keys`, not the schema, and the
distinction matters: the closed schema covers only the fresh path. A handle's stored
query is checked shallowly (non-empty stages, positive last-stage limit) and the
`/drills` callback stores a caller-supplied query verbatim, so an unknown key really
does reach the execution boundary there. Move the test to the handle path, where the
whitelist is load-bearing, and add a second test recording the fresh path's rejection.
Verified by mutation: replacing the `select-keys` with the raw query turns the handle
test red on `(not (contains? captured :evil))`.
* Test the MCP App parameter fix against the payload execute_sql actually mints
The spec asserted against a legacy `dataset_query` shape that was invented, not observed.
The real serialized query is MBQL 5 — `stages` rather than `dataset_query`, and
`template-tags` as an array rather than a map — captured from `prepare-for-serialization`
in the REPL. The extraction happens to be shape-agnostic, since it reads only
`parameters[].target`, so the tests passed against a fiction and proved nothing about
what the embed receives.
Rebuild them on the real payload and add the case that decides it: drive
`getCardUiParameters` and `getParameterValuesForQuestion` — the helpers
`loadQuestionSdk` uses — and assert the tag-derived parameter resolves to slug `cat` and
takes the bound value. Splitting the payload cleanly is worthless if the halves don't
rejoin there.
The value arrives array-wrapped because `normalizeParameterValue` wraps every string- and
number-typed parameter on the way to the request; that is the shape a filter widget
produces too.
Verified by mutation: making the parameter extraction return {} turns both value-carrying
tests red.
* Cover the MCP App parameter chain through to the request payload
Extends the binding test to the full path `loadQuestionSdk` walks: build the Question
from the card, apply template-tag parameters, derive values from initialSqlParameters,
set them, then assemble request parameters as `runQuestionQuery` does. Asserts the QP
receives the bound value against the target the tag declares.
This is the level the bug lived at, so it is the level worth guarding: the earlier tests
only prove the payload splits, and a clean split is worthless if the halves do not
rejoin.
* Correct the keyset-marker stripping claim in resolve-input
The comment said caller-supplied queries are stripped in "the `:query` branch above and
the `/drills` callback". `strip-caller-keyset-markers` has exactly one call site, the
`:query` branch — `POST /api/embed-mcp/drills` stores its caller-supplied query verbatim.
Documenting a guard that isn't there is worse than documenting the gap, so say what the
code does and why the gap is tolerated: reaching it needs the scoped UI credential, and
the effect is confined to the caller's own narrowed-question reporting.
* Fix Duplicate key crash in document collection-items tests
`#{normal-card-id dash-id in-document-card-id}` is a runtime set, and
`PersistentHashSet/createWithCheck` throws on duplicates rather than deduping. Card and
Dashboard ids come from separate sequences, so in a fresh app DB where this namespace is
the first to create both, each gets id 1 and the literal blows up before any assertion
runs. It survived full-suite runs, where thousands of preceding rows keep the sequences
apart, and failed only under `:only`.
Match on (id, model) pairs instead. The vectors differ by model string, so no duplicate is
possible, and it also closes the correctness hole the bare id set had — an unrelated
Dashboard sharing a Card's id would have been pulled into the filtered set.
Applied to the latent third instance in the trash-items test too, which spans Document,
Card, and Dashboard sequences and was one id alignment from the same crash.
metabase.documents.api.collection-test: 11 tests, 66 assertions, 0 failures (was 64 pass,
2 errors).
* GHY-4510 Withhold query_summary when a source card is unreadable
get_content on a card the caller can read rendered `query_summary` with
`lib/describe-query` over `application-database-metadata-provider`, which applies no
per-user read check. For a wrapper whose query sources a restricted card, that resolved
and printed the restricted card's display name — the one read surface that leaked it,
while the direct read 403s, the run path refuses via `check-card-read-perms`, and search
filters it out.
Gate the summary on the caller being able to read every card the query reads, at any
depth (`all-source-card-ids-recursive`, so a transitively nested source counts too).
Fails closed: a missing source card, or a query that can't be walked, withholds the
summary rather than rendering from metadata we cannot account for.
question, model, and metric all resolve through `fetch-card`, so one gate covers the
three types the report named.
The test drives the report's attack — restricted card in a revoked collection, readable
wrapper sourcing it — and asserts the name appears nowhere in the response, not merely
that `query_summary` is nil. Verified failing first: `query_summary` came back as the
restricted card's name verbatim.
metabase.mcp.v2.tools.content-test: 41 tests, 224 assertions, 0 failures.
* Placeholder the table and field ids in execute_query's examples
The description stated `ORDERS = table 5` and `CREATED_AT = field 42` as fact.
On a stock Sample Database ORDERS is table 2, and table 5 is FEEDBACK — a real
table — so an agent copying the example queries the wrong table and gets a
plausible answer with no error anywhere in the trace.
A fleet worker did exactly that: `execute_query {stages:[{source-table: 5,
limit: 5}]}` returned FEEDBACK's columns (ID, ACCOUNT_ID, EMAIL,
DATE_RECEIVED, RATING, ...). Nineteen of twenty-five workers on one case
flagged the discrepancy and looked the id up instead; the ones who did not
would have had no signal.
Correcting the numbers would not fix it. The ids are per-instance, and the
fleet saw 5 resolve to FEEDBACK on most instances and REVIEWS on another from
the same seed data — so any literal id in this docstring is wrong somewhere.
`<TABLE_ID>` and `<FIELD_ID>` cannot be pasted.
The same description already said "never invent or guess ids — discover ids
first", which its own examples contradicted.
* Decouple the MCP App parameter test from getParameterValuesForQuestion
CI type-checks the PR merged with master, and master changed that helper's signature from
{card, metadata, queryParams} to {card, queryParams, buildQuestion}. The spec compiled on
the branch and failed on the merge — a semantic conflict, clean to merge and broken to
type-check.
Call `getParameterValuesByIdFromQueryParams` against `question.parameters()` instead. That
is precisely what the helper delegates to on both sides — the old form passed
`getCardUiParameters(card, metadata)`, the new one passes `buildQuestion(card).parameters()`,
and `Question.parameters()` is `getCardUiParameters` over the same card — so the test
asserts the same semantics while no longer depending on a signature in flux. Both
primitives it now uses are byte-identical on master.
8 tests green; type-check, eslint, and oxfmt clean.
* GHY-4511 Make a question's visualization_settings readable (#82355)
* GHY-4511 Make a question's visualization_settings readable
A chart's settings could not be read back: not in concise, not in detailed,
not through `fields`, and `question_write` did not echo them either. An agent
could write a goal line or a series binding and had no way to check it.
- `get_content` gains a `settings` include for question/model, returning the
stored `visualization_settings` verbatim ({} when nothing is stored).
- `visualization_settings` joins the question detailed projection, which also
makes it a valid `fields` path. It stays one whole-blob path: its own keys
contain literal dots, so nothing deeper could ever be addressed.
- `question_write` echoes `visualization_settings` in its response.
- A `fields` path with no stored value now answers null instead of dropping
out, so an unset `cache_ttl` no longer looks like an unreadable field. A
path through a missing parent still drops — a null-valued shell would claim
a parent that isn't there.
- The visualization-settings skill's "To confirm" section now describes what
ships.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TUQL4KAPYRe7DJbwRF4dTy
* Name the include after the property it returns
Review: `include: ["settings"]` returned `visualization_settings`, so the
section name and the property name had to be translated by anyone reading
either. `visualization_settings` is also the name every other surface uses —
the REST API, question_write's args, the card model, the skill — so it is the
one a model reaches for unprompted.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TUQL4KAPYRe7DJbwRF4dTy
---------
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
* GHY-4512 Read a link card's entity back behind a read check (#82357)
`add_link` with `entity` saved correctly all along. `get_content`'s `layout` include stripped `:entity` from every link card on the way out, so an entity link read back as `link: {}` — indistinguishable from an empty card, and impossible to round-trip through `patch_dashcard`. The strip was there to stop the `name`/`description` cached beside the reference at write time from reaching a caller no read check stands behind.
`dashboard-layout` now hydrates `:dashcard/linkcard-info`, the same batched hydration the REST dashboard endpoint runs, and returns the bare `{model, id}` reference — the shape `patch_dashcard` takes back verbatim — when the caller can read the target, and `{restricted: true}` when they cannot. The cached name and description still never leave the server.
Claude-Session: https://claude.ai/code/session_019aHyUCgLuS6QzkcRcZDcdd
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
* GHY-4516 Let collection_write create transform folders (#82356)
browse_collection browses the transforms namespace and transform_write
files a transform into a collection_id, but collection_write's namespace
enum was "snippets" only — so the folders those two tools talk about
could not be made through MCP at all.
Update needed no change: namespace is already create-only, a move stays
inside the row's own namespace, and parent_id "root" lands at the root of
the transforms tree rather than the content root. Tests pin that.
Claude-Session: https://claude.ai/code/session_015dqtq17qCpVgMeRvu9uPWt
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
* GHY-4517 Honor `name` on measure clauses (#82358)
The measures middleware substitutes a measure's underlying aggregation for
the `:measure` clause, carrying `:lib/uuid` and `:display-name` across but
dropping `:name`. A caller-supplied `{"name": "revenue"}` passed validation
and was then silently discarded, so the output column came back as the
underlying aggregation's name (`sum`, `count`) and a next-stage
`["field", {}, "revenue"]` failed with a raw database error.
The metrics middleware already preserves `:name`; a regression test now
covers that too. Also documents in the query-dialect skill what a metric's
or measure's output column is called.
Claude-Session: https://claude.ai/code/session_01Egi1Dqi8XGihQzQ5BmQYat
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
* Keep the MCP Apps iframe working when an endpoint declares a scope (#82361)
* Let enforce-scope defer to the MCP Apps UI route gate
An MCP Apps UI credential is not an OAuth token, and an endpoint's declared `:scope` is not
the vocabulary that confines it. `metabase.mcp.ui-surface/request-surface` is — a handful of
routes, each priced in the MCP scope the minting session had to hold — and it is strictly
narrower than any endpoint scope could be.
Those two mechanisms currently refuse each other. The credential carries `::mcp-ui`, which
satisfies no declared `:scope` by design, so declaring one on any route of that surface 403s
the iframe while every other caller is unaffected: an ordinary session carries
`::unrestricted` and passes. It merges clean and fails silently — the iframe stops rendering.
`ensure-scopes-checked` already honours `:token-scopes-checked` as "an authorization decision
already happened upstream". `enforce-scope` did not, and that asymmetry is the bug. The MCP UI
stamp is set only when the surface's own scope gate passes, so honouring it defers to a
route-specific decision rather than waiving one.
Conditioned on `::mcp-ui`, not on the stamp alone: `enforce-scope` sets the stamp itself on
success, so trusting it unconditionally would make every per-endpoint `:scope` a no-op
underneath a namespace-level `enforce-scope`. The test asserts that non-leak directly.
* Guard every MCP UI surface route against the endpoint scope layer
Drives each route on `request-surface` with a real UI credential through the real stack and
asserts the response is not a scope rejection — neither `unsupported_scope` from
`enforce-scope` nor `scope_not_permitted` from `ensure-scopes-checked`.
Asserting on the error codes rather than on a status or a scope name is what makes this
independent of whichever scope a future branch declares, and of what the handler then does
with a deliberately thin probe body: a 400 is a pass, because the request reached the handler.
A coverage assertion pins the probe set to the surface, so adding a route without a probe
fails rather than silently skipping the check.
Verified by mutation, both directions: declaring a `:scope` on POST /api/dataset turns this
red on exactly that route with the carve-out removed, and green with it in place.
* drop v2 ping tool and update llm-install (#82362)
* update llm-install
* drop v2 ping
* Drop the require comments and redundant keep-me around test_echo
* GHY-4518 Summarize a native question by its SQL, and name its database (#82363)
`query_summary` delegates to `lib/describe-query`, whose native-stage display
name is the placeholder "Native query" — enough in the UI, where the SQL renders
beside it, and nothing at all to a caller holding only the row. An audit of an
all-native dashboard had to guess each card's purpose from its description.
Native cards now summarize as the head of their own query text: whitespace
collapsed to one line, capped at 300 characters. The full query stays behind
`include: ["definition"]`, so the summary can stay a summary. A non-string
native query — Mongo stores a map — keeps the generic Lib description rather
than rendering a map.
Both branches stay behind the `source-cards-readable?` gate from GHY-4510: a
native query names a Card in a `{{#123}}` template tag as readily as an MBQL
stage names one in `:source-card`, and `all-source-card-ids` covers both.
The row also carries `database_name` now, read off the metadata provider it
already builds. Resolving `database_id` to a name previously cost a second
`search` call. It is flat rather than nested to match the vocabulary `search`
rows already use, and it joins `question-enrichment-keys` so the column-select
paths drop it instead of selecting a column that does not exist.
Claude-Session: https://claude.ai/code/session_01GXJcdTASE4jxhmE4fzRNkw
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
* GHY-4519 Answer a date parameter with its range, not hundreds of dates (#82366)
`get_parameter_values` returned every distinct date in the column for a
`date/*` parameter — 721 on one dashboard, 100 per page — when what a caller
needs is the span and the grammar it can filter with.
A column-backed date parameter now answers:
{"kind": "date", "min": "2016-04-30", "max": "2020-04-19",
"distinct_dates": 18760,
"accepts": ["YYYY-MM-DD", "YYYY-MM-DD~YYYY-MM-DD", "past30days", "thisyear"]}
The range is aggregated rather than folded out of the fetched values, because
listing stops at `result-limit` and values come back ascending: the list's last
value is the 1000th-earliest, not the column's max. On orders.created_at that
gap is fourteen months — the list ends at 2019-02-22 against a real max of
2020-04-19. `chain-filter-range` builds the same source table, joins and
constraint filters the values query builds, aggregated to one row, so
constraints narrow the range exactly as they narrow a list.
Excluded: a date parameter with a static-list or card source, whose dates are a
curated set to pick from rather than a span to write a value inside.
`query` on a date parameter is now a teaching error. A range is two dates and a
grammar, so there is no list to search, and applying it silently would hand back
the whole span as though it had been narrowed.
Claude-Session: https://claude.ai/code/session_01GR7MMvsg2kH4dE24YyX1kS
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
* Address review nits in useMcpApp
- Drop the unjustified cast and FIXME on window.metabaseConfig; the global
Window.metabaseConfig declaration already types instanceUrl.
- Remove the redundant void operator on the async IIFE.
- Remove the redundant JSDoc on McpAppState.queryError.
* Address review: drop the unreachable v1 payload path, trim comments
Review feedback from sanex3339 on #81245.
Two of the comments were questions — whether the iframe can still receive a v1 response, and
whether the v1 field needs describing, given FE and BE ship from one instance. The answer is
that it cannot, and more strongly than the questions assume: v1 was deleted from this branch
in 84c334c6d2, and both v2 UI tools put `{query_handle}` in `structuredContent` and nothing
else (their output schemas declare only that key). So the inline `query` field, the branch
that read it, and the comments explaining both are dead. Removed.
That removal surfaced why the dead path survived: 8 tests drove the v1 inline shape, so the
suite exercised a payload the backend cannot produce. The fixtures now carry handles — the
real shape — and resolve through the mocked callback API. One test asserted the dead path
outright ("still renders v1's inline query") and is gone with it.
Also replaced the `// Unjustified type cast. FIXME` on `structuredContent` with a runtime
narrowing that checks each field instead of asserting the whole shape, and shortened the
comments flagged across the four other files. Kept the two that carry a hazard rather than a
restatement: the RTK subscription warning in `useMcpUserAndSettingsFetch`, and why
`McpQuestionView`'s spec must not use `mockSettings`.
20 suites, 68 tests green; type-check, eslint, oxfmt clean.
---------
Co-authored-by: Eric Normand <ericwnormand@gmail.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-authored-by: Aleksandr Lesnenko <alxnddr@gmail.com>
Co-authored-by: Aleksandr Lesnenko <alxnddr@users.noreply.github.com>
Audit a permission refusal only when someone reaches for something themselves,
the model naming an ID or an ID inside a query the client sent us. Refusals hit
while rendering a query one of our own tools wrote stay quiet, so the log keeps
only the ones that are a real access attempt.
Show a query only if you could run it, using the query processor's own check
rather than read access to the database, and check every table and column the
rendering would name so column sandboxes are caught too.
Co-authored-by: Chris Truter <crisptrutski@users.noreply.github.com>
Fixes https://github.com/metabase/metabase/issues/65493
If you have a `details.schema` property set in your connection (this was
removed in v42 in favor of schema filters), you can't change it in the
UI in the current Metabase version. If you wanted to specify a schema
through the "Additional JDBC connection string options", the old
buried schema property would take precedence anyway.
This makes it so the deprecated field is removed when the schema is
set in additional-options.
I've verified this using a real connection and querying `SELECT current_schema()`
as suggested in the bug report.
The test read `:future-ref` straight off the registry entry as soon as the
fetch signalled it had started. But `detached-fetch!` registers the entry
before the fetch begins and only stores the future just after submitting it,
so the thunk can deliver `started` before the `reset!` lands. When it does,
`stalled-future` is nil and `future-cancelled?` throws an NPE.
Under 300 spinning threads a probe caught the window 1-4 times per 20000
fetches; inserting a 5ms sleep between the `future` and the `reset!` makes
the old test fail every run.
Poll for the future instead. Only the test is affected: the sweep acts on
entries older than *fetch-max-age-ms* (an hour), so it can never observe one
whose `:future-ref` is still unset.
Claude-Session: https://claude.ai/code/session_01J1yFPpYiWH2ik8LB9rs7BT
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
* multiple jvms per runner for bigquery
* trigger CI
* add failing test
* lower timeout
* remove debug stuff
* clean up comments
---------
Co-authored-by: Claude <noreply@anthropic.com>
* BigQuery: use calendar days for datetimeDiff day unit
Fixes#82193
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
* Avoid :raw for date part in BigQuery day diff
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
* clean up
* add timezone to test
---------
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
* add saved question extraction prototype
* fix additional failure cases
* refactor: reconciler
* revert me: manual skill file edit for source swapping
* set data permissions to view for databases referenced in the query
* rename data app collection and group
* prevent agents from removing savedQuestionSourceId
* docs: clarify data app query location
* chore: cleanup query-sync canonical module
* refactor: split query tests into different files
* Swap query source for data app query hooks and apply dynamic filters on top of it (#79584)
* code improvements and bug-fixes for data apps permissions
* Add data apps model imports
* address reviews on data apps permissions
* run sync queries before vite build
* Add data app resource link (#79878)
add button to view data app resources
* Show permission error when users cannot access a data app (#79874)
show permission error page when user do not have permission to view data apps
* Add data app group link in overflow menu (#79873)
Add data app group link
* Store and reconcile data app entity ids in data_app.yaml (#79833)
* Store data app resource entity IDs
* make entity ids in data_app.yaml authoritative
* add top-level value after path key
* simplify data_app.yaml manifest parser
* only allow changing data app collection if empty
* consolidate backend test helpers and remove redundant tests
* chore: restore README changes
* ci: fix failing tests
* Add support of actions to Data App reconciler (#79842)
Add support of actions to Data App reconciler
* Fix bugs/inconsistency and add more e2e for data apps reconciler/syncing (#80002)
* Update semantic schema skill about preventing segments and measures in dynamic parts of a query in Data Apps (#80288)
* Grant view data access on a table-by-table basis in data app groups (#80189)
* grant view data on table instead of database level
* refactor table scan in reconciler
* update e2e tests for permission grants
* Grant data app view permissions per table
* Skip unchanged data app permissions
* add e2e data app test for sandboxing
* reset host app sources after runs
* Preserve sandboxing for data app users (#80406)
* add e2e data app test for sandboxing
* improve e2e tests for data apps
* Remove sandbox E2E from table permissions change
* preserve existing sandboxing from sandboxed groups in data apps
* assert that we grant non-sandboxed tables
* Explain data app sandbox exclusion
* Add data app sandbox E2E coverage
* Exercise sandbox preservation in data app E2E
* Disable saved question sources in data apps (#80139)
* Disable saved question sources in data apps
* Preserve question-only schema scope
* Remove the saved question schema parameter
* Copy table-sourced metrics into data app collection and do not codegen card-sourced metrics (#80086)
* copy table-sourced metrics into the data app collections
* copy table-sourced metrics into the data app collections
* Address metric sync review feedback
* Accept keyword metric display in query response
* Fix data app metric reconciliation ordering
* Remove old lockfile format
* exclude card-sourced metrics from schema generator
* do not use card terms in metrics to avoid ambiguity with card-sourced metrics
* fix model imports
* Fix data app metric reconciliation
* refactor: cleanup reconciler unit tests
* Fix duplicated metric client helper
* add safe metrics path
* Handle unsupported metric dependencies
* Fix data app CI test regressions
* Fix "Re-adding removed query fails to adopt deleted card" (#80341)
* Clean-up data apps permissions endpoints (#80825)
* Remove entity ids from data_app.yaml (#81317)
Remove data app entity ids from data_app.yaml
* Adjust data apps permissions code (#81093)
* Remove the data app table permission grants (#81584)
* Add permissions_group.is_data_app_group and hide data app permission groups from UI (#81705)
* Add data app user management and access warnings (#81839)
* Address data app user access review feedback
* additional e2e test cases for bugs in manage users page
* fix failing frontend type check on unit test
* fix migration id collision for data apps
* combine migration changes into one file
* fix data app metric sync after trashing copied metrics
* fix cli runtime dependency packaging for embedding sdk
* avoid loading data app bundle during resource setup
* prevent redundant data app collection permission updates
* allow admins to remove data app members after token expires
* update docs and add e2e test for missing resource collection case
* isolate data app sync using transactions
* do not leave user on an empty page when removing the last member on the page
* reserve sandbox-host slug in data app config
* ignore generated data app action fixtures
* move queries into db.clj
* preserve legacy permissions when adding data app members
* regenerate the kondo modules config
The data-apps module reaches for :model/Permissions and :model/DataPermissions,
neither of which was recorded in :model-imports, so metabase.core.modules-test
fails on the current tree.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011nNKkhQjjiUSM7qwLReyfP
* reject data app queries whose metrics read another card
Sync copies a referenced metric but rewrites nothing inside the copy, and
copies nothing that copy in turn reads, so those references still point at
the originals. The app publishes successfully and then fails for every
viewer without access to the originals' collections.
Checking while resolving the query, rather than at codegen, also catches a
metric edited into that shape after its schema was generated.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011nNKkhQjjiUSM7qwLReyfP
* exclude metrics that read a saved card anywhere from generated schemas
source-card-id only reads stage 0, so a table-sourced metric that joins a
saved question passed the codegen filter. The CLI checks the whole query and
rejects such a metric, aborting sync-resources for any app that uses it.
Checking the whole query here keeps codegen consistent with what sync
supports, and removes the database round trip that asked which of the
referenced cards were metrics -- the step that let a saved-question join
through in the first place.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011nNKkhQjjiUSM7qwLReyfP
* resolve copied resources' table dependencies through the server
Numeric source-table entries miss implicit joins: a copied model querying
Orders and reading Products.Category through a foreign key names Products
nowhere, so it was left out of the app's permission warnings.
Run the same metadata-based lookup the query endpoint uses, through a new
endpoint the CLI calls for copied models, actions and metrics, and drop the
JSON walk it replaces.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011nNKkhQjjiUSM7qwLReyfP
* cover the create-queries grant the database defaults leave on app groups
An app group that predates the first non-audit database is given
create-queries :query-builder-and-native by set-new-database-permissions!,
alongside the :blocked view-data it special-cases for app groups. A user with
data access through another group would gain native query authoring just by
joining the app.
The reconciler already resets it, but nothing failed when :perms/create-queries
was dropped from its list. The pairing cannot be produced through
set-database-permission!, which cascades view-data to :unrestricted for any
create-queries value other than :no, so the test writes the row the way the
defaults insert it.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011nNKkhQjjiUSM7qwLReyfP
* prevent models based on saved questions from being copied
* return to a valid member page after overlapping removals
* do not resolve query objects when either enabled flag is false
* prevent shorthand properties and spreads from overriding generated ids
* prevent adding duplicate members and retry only members that are failed to add
* define schema for data app query endpoint
* reconcile only the current data app group during resource setup
* expand the schema fingerprint
* add data app group field in permission group schemas
* prevent new databases from granting query creation to data app groups
* throw validation errors when invalid query spec fields are passed
* do not fail validation on columns metadata
* add enabled metadata to query definition schema
* accept more data app specific query metadata and binning options
---------
Co-authored-by: Timofey Kachalov <timofei@metabase.com>
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
* test(ui): add PasswordInput Storybook story
* test(ui): add PasswordInput to Loki as baseline for restyle
* feat(ui): restyle PasswordInput from Figma and add input-background token
* test: cover PasswordInput visibility toggle in unit and e2e tests
* fix(PasswordInput): use highlight border color in focused error mode by overriding Input styles
* Update Loki Snapshots
* test(ui): add Loki reference for PasswordInput Overview story
* fix(PasswordInput): address review feedback
- keep border width constant and use an inset ring for focus/error to avoid layout shift
- tie right padding to the toggle button size so text clears it in lg
- restrict size prop to sm | md | lg
- order color keys alphabetically
- explicit visibility assertion in the e2e spec
* Update Loki Snapshots
---------
Co-authored-by: Metabase Automation <github-automation@metabase.com>
* Use JetBrains Mono as the code font
Ship JetBrains Mono (regular + bold) and point the monospace font family
at it, replacing the hardcoded Monaco/Menlo stacks scattered across the
codebase.
- Add the woff2 files, @font-face declarations, and the type, fallback
map, docs and backend test entries that keep the admin font picker in
sync with the fonts directory.
- Route every monospace call site through
--mb-default-monospace-font-family, and define that variable in the
embedding SDK's variable set, where it was previously missing. With
the variable always defined, the per-call-site `, monospace` fallback
is redundant and is dropped.
- Inline the variable at the CodeMirror theme, which was the only
consumer of monospaceFontFamily, and delete typography.ts.
- Set the Code component to 13px/22px per the design spec.
* Update Loki Snapshots
---------
Co-authored-by: Metabase Automation <github-automation@metabase.com>
Bedrock takes an AWS access key pair and Google Gemini Enterprise a service
account key file, so the AI settings page can't keep calling every provider
connection an API key.
The tests pass ids like 789 for entities they do not create. format-entity looks the
id up, and the test asserts on the fallback rendering that a 404 produces. When a
parallel test happens to hold that id, read-check gives a 403 instead, which renders
nothing, and the test fails.
Use Integer/MAX_VALUE, which no fixture allocates.
Claude-Session: https://claude.ai/code/session_01GCoSa7L7KrXh42rDRYR3Nk
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
* Read a card's UI parameters off its question
* Drop a tombstone comment
* Pin the agreement between getCardUiParameters and Question.parameters
* Type the unsaved card in the agreement test
* Format the agreement test
* Move the application permissions selector slot into current-user
PLUGIN_APPLICATION_PERMISSIONS.selectors was only read by current-user, so the slot now lives there as PLUGIN_APPLICATION_PERMISSIONS_SELECTORS with its OSS defaults, and the EE application permissions plugin registers into it from metabase/current-user. getRoutes and tabs stay in metabase/plugins for the admin permissions editor.
* Translate viz nouns and formatting operators on demand
`noun` was the one field on a visualization definition still translated at
import. `get noun()` matches what ObjectDetail and TreemapChart already do,
and the registry stores a definition by reference, so the getter survives.
The table-formatting operator names were four module-level objects. They are
functions now. `ALL_OPERATOR_NAMES` becomes `getAllOperatorNames()`, which
its three readers call.
25 of the 124 sites that translate before a catalogue can be applied.
* Translate settings titles on demand
`title` in a settings definition follows the same rule as the rest of the
object. 152 settings already use `get title()`; these 23 were the laggards.
The group-managers confirmation held two frozen strings, so it becomes a
function rather than an object with getters.
49 of 124.
* Lift translated containers into functions
Where a whole array or object is translations, the function goes on the
container so consumers keep a plain array or object: PieChart's ring titles,
the whitelabel illustration options and descriptions, PivotTable's partitions
(which held JSX built at import), the model-caching options, the cloud
migration messages, and the geo filter placeholders.
Where only a field or two of a mixed config is translated, and call sites read
it by property, the getter goes on the field instead. That covers the
collection configs, the permission option records and the loading messages,
none of which are spread anywhere.
Four files carried a whole-file disable rather than one per site, hiding 20
sites behind 4 directives. The real total is 140, not 124.
118 of 140.
* Clear the last module-scope translations
Validation schemas become functions, matching getOidcFormSchema and the other
lazy schemas already here. Module-local error strings become functions called
at their use sites. Three exported constants follow: NULL_DISPLAY_VALUE,
BEFORE_UNLOAD_UNSAVED_MESSAGE and NO_X_AXIS_VALUES_ERROR_MESSAGE, across 37
files.
The two Mantine `aria-label` defaults get a getter. Mantine merges
`defaultProps` by spread on each render, so the getter is read every time.
`ttag/no-module-declaration` now passes across all five trees it covers with
no opt-outs left, and `--report-unused-disable-directives` confirms none were
stranded.
* Stop spreads from flattening lazy translations
A spread or a rest destructure reads every getter it copies. Where an object
holds `get title()` so the string translates when it is read, spreading that
object at module scope translates it at import instead, which is the bug the
getters were meant to avoid.
Adds `mergeLazily` / `omitLazily`, which copy property descriptors instead,
and uses them where the flattening happened at import:
- `viz-core/lib/settings/utils.ts` flattened twice, once in the rest
destructure and once in `...def`, so every setting built through
`fieldSetting` / `metricSetting` / `dimensionSetting` froze at import.
- `palette/shortcuts/index.ts` spread five shortcut maps into one.
- `PaletteShortcutsModal` spread each shortcut again while grouping, so that
now happens per render.
Measured with a test that imports every source module with `ttag` mocked:
40 modules and 370 translation calls at import, down to 35 and 256.
* Build parameter, admin and graph settings when read
* Build option lists when they are rendered
* Guard import-time translation with a test
* Make the instance analytics message a function so the plugin registry can replace it
* Resolve admin path names when the navbar renders them
* Give the enterprise caching tabs a lazy name too
* Skip generated cli-snippets from the side-effect scan
* Key the null slice by its untranslated label
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01NUuaopJ7ZtMVvdwwxF6qyx
* Fail the import-time guard on unlisted import errors
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01NUuaopJ7ZtMVvdwwxF6qyx
* Read lazy translations once per render, not per lookup
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01NUuaopJ7ZtMVvdwwxF6qyx
* Let jest transform the MCP ext-apps package
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01NUuaopJ7ZtMVvdwwxF6qyx
* Skip the SDK cli scripts in the import-time guard
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01NUuaopJ7ZtMVvdwwxF6qyx
---------
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
* Namespace the table name BigQuery's compiled native form carries as :qp/table-name
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
* Update the BigQuery compile tests for :qp/table-name
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
* Update the remaining BigQuery compile tests for :qp/table-name
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
---------
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
Refuse to create new tables in static datasets with Snowflake.
Table creation will always succeed on non-static datasets, but if it's
static then the table being created has to be a table that's in the
dataset definition.
Ensure all tests which alter the dataset by creating new tables
declare a non-static dataset. The transform-related tests switch over
to using transforms-dataset/transforms-test which most of them are
already using.
Upload tests have their dataset defined in a fixture so we can do it
all in one place instead of editing 60 different deftests.
Update E2E test timings with averaged data
Collected timing data from 7 days of test runs.
All specs with timing data were included (simple averages).
🤖 Generated automatically by update-e2e-timings workflow
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: GitHub Actions <actions@github.com>
* Update map tile server settings with suggested copy from DOC-301
And also include a link to the relevant page in the metabase docs.
* Add a note about not including private keys to the docstring for map-tile-server-url
so that the warning also shows up in the env var docs.
* BigQuery: use PERCENTILE_CONT for median and percentile
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
* update docs
* update doc
---------
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
Fixes UXW-4714.
The static viz for tables uses a fixed width, but it was too big for the
PDFs, causing tables with long text in columns to lay out badly.
This PR disables the fixed width of `780px` on tables for PDFs, so that
the tables now lay out correctly.
* Fix collisions on metabase.notification.payload.core-test namespace name. Lint for future collisions.
* Include .clj-kondo/{src,test} in backend check triggers
* Expect the new project test namespace
* Symbolically reference project test suite namespaces
* Include resources, test_resources, test_config dirs in project checks
* Return [namespace relative-path] pairs.
Skip local/src
* Catch read-file errors and explain why some files may not have ns forms
* Sort paths that collide
* Copy the transform tables in dump-to-h2 and load-from-h2
None of the nine transform_* models were in copy/entities, but metabase_table is,
and it carries a transform_id FK to transform. A table row reached the target
referencing a transform row that never arrived, and the deferred FK check
rejected it at commit.
transform_run_cancelation also goes in entities-without-autoinc-ids, since its
primary key is run_id rather than an autoincrement id.
Adds copied-tables-have-copied-fk-parents-test, which migrates a throwaway H2
database and fails if any copied table's FK parent is not copied.
* Simplify the copy foreign-key regression test
* Clarify the foreign-key coverage test names
* Make defendpoint reject request schemas that reach an open map, a keyword-keyed map-of or :any
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
* Close the MBQL 5 query and column schemas and declare the keys the query processor adds
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
* Type the shared schemas requests reach and drop ms/Map and ms/JSONString
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
* Close every map schema a request can reach
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
* Regenerate the modules config
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
* Keep action parameters string-keyed and type lens params and permission graph ids
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
* Declare the sandbox coercion keys a column's options can carry
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
* Declare the alias-info and Mongo keys a column can carry and strip internal keys from legacy column metadata
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
* Carry the Mongo column keys in the serialize test fixture
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
* Declare the keys the frontend and query processor put on columns, parameters and card embeds
- column metadata: :remapping, a typed :lib/options, the nest-breakouts option key
- query parameters accept the rest of a stored filter declaration
(sectionId, isMultiSelect, values source config etc.)
- static-list values may be one-tuples; raw-value template tags may
carry a nil widget-type
- card embed attrs declare chart_href, child_target_id and host_data;
explore filter refs accept the legacy field_ref the chart column has
- notification card echoes a metric's query_description
- serialization deep-sort orders mixed keyword/string keys
- viz-settings column refs are normalized before matching; a Field
hydrated with its table can be fingerprinted; broken segment
definitions read back as nil
- test fixtures: real fingerprints, snippet tag type, http channel
details, comment content, no min_duration_ms on duration strategies
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
* Wire the defendpoint check, keep every map single-keyed, and apply the review findings
- defendpoint now calls closed-schemas/check! on every request schema;
the walker no longer lets :any leak through collections, treats
map?/:some as :any, skips :=/:enum/:fn values and never skips the
default entry
- no map mixes keyword and string keys: ms/string-keyed-object gives an
opaque JSON object string-keyed declared entries; ProseMirror attrs
(documents, comments, explorations embeds), CSV upload parts, search
weights and testing query specs use it and their readers use string
keys; the serdes deep-sort comparator is back to master
- the Metabot context is fully typed from the frontend types instead of
an opaque default; a draft transform query without a database is
accepted
- permission graphs, settings PUT and SCIM PATCH reject a body whose
map-of entries the decoder dropped; ad-hoc query endpoints drop a
client-sent cache strategy; action parameter values keep collections
out without nil-ing them; SCIM meta.resourceType is optional; sandbox
user attributes are LoginAttributes; a metric filter clause carries its
offset options; explore filter refs are the legacy field_ref the
chart column has; result_metadata written by older releases and
hasVariableTemplateTagTarget on parameters are declared; a bare
map-of on a request path is guarded against scalars
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
* Close the endpoints tests define and let a column's target and name field be an id reference
The check runs on every defendpoint, the ones test namespaces define included. A column's
:target and :name-field are hydrated Fields on the backend but id references in the
metadata the frontend hands over; the closed column schema nil'd the reference, which
lost the search field of a model's primary key.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
* Hand an action's query only the parameter values, and fix two fixtures the closed schemas caught
A query action's stored parameters carry the frontend's widget settings; the query gets
the value keys. The Metabot viewing-context tests use ids no card can have, and the
Mongo cache test spells the strategy key the schema does.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
* Declare the table name BigQuery's compiled native form carries on a native stage
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
* Keep a BigQuery RECORD's sub-field sync metadata off its result column, and fix a binning fixture
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
---------
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
* Move Snyk scans into the Lint Security workflow
Fold snyk.yml's three jobs into lint.security.yml as snyk-generate-poms,
snyk-frontend-projects, and snyk-maven-projects, and delete snyk.yml.
Snyk now runs on the existing daily 11:00 UTC cron instead of its own
05:00 cron and dependency-file push trigger. The notify job reports Snyk
failures to Slack alongside CodeQL and Trivy, and Snyk's upload-sarif is
bumped from v3 to v4 to match the rest of the workflow.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
* add timeouts
---------
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
* GDGT-3132 fix misaligned undo toast
* migrate UndoListing to from styled components to CSS module
* absolute px -> rem, compact -> compact-md
* fix inconsistent toast height, use line-height instead of measuring the DOM to handle single vs multi line layouts
* strip comments
* GDGT-3195: Fix inefficiencies in formatValue
formatValue is fine per-cell but pathological in bulk; the "Copy results"
button made this visible, with a 100x2000 grid spending ~4.5s in this path.
- Drop a dead dayjs strict parse. The format literal 'T' is not valid dayjs
escaping (dayjs wants [T]), so isValid() returned false for every value.
It sat above the string/number/boolean branches, so every non-date cell
paid for it.
- Only JSON.parse the value when a markdown_template will consume it,
instead of on every call.
- Memoize the per-column type predicates (isURL/isEmail/isTime/isDate/
isNumber/isCoordinate/isBoolean) in a WeakMap keyed on the column. Each
fans out into recursive type-hierarchy lookups that cross into CLJS, and
they only depend on the column. Entries snapshot the type fields they
were computed from, since the custom-viz API hands third-party code
mutable column objects.
- Skip new URL() construction for values with no ":" in getUrlProtocol.
- Cache Intl.NumberFormat by its constructor options, bounded at 100
entries with oldest-first eviction.
Also make the value-formatting signatures honest: formatUrl, getLinkText,
getLinkUrl and getRemappedValue take unknown, which is what they actually
receive. getUrlProtocol keeps its string signature, with getLinkUrl
narrowing before the call. This removes both `value as string` casts.
Benchmarked with a synthetic 100x2000 copy path (medians of 5):
full table 4512ms -> 440ms, text 1231 -> 12, currency 637 -> 20,
integer 535 -> 15, boolean 343 -> 5, email 101 -> 4.
Claude-Session: https://claude.ai/code/session_01KUcocQj2bJPwVBi4dXKyEP
* Cleanup
* Revert the createPlainCellFormatter change
Hoisting the options spread out of the per-cell closure is not worth the
churn. Measured in isolation with a realistic 22-key columnSettings, the
spread costs 0.023us per cell against 0.002us to reuse a hoisted
reference, so it saves ~4ms per 200k cells — a rounding error next to the
dead dayjs parse at ~13.5us per cell, and nothing at all for columns with
click behaviour, where both shapes spread per cell anyway.
* Address review feedback
Guard the markdown_template JSON.parse with a typeof check instead of
carrying the placeholder cast comment.
Cover the two changes the review called out as untested:
- numberFormatterForOptions returns the same formatter for equivalent
options, a separate one for different options, and evicts the oldest
entry once the cache is full. The eviction test resets the module so
the 100-entry boundary does not depend on what earlier tests cached.
- formatUrl passes strings and numbers through untouched and renders
booleans and objects rather than returning them raw.
Both were checked by mutation: breaking the coercion and stubbing the
cache lookup to always miss fails exactly those assertions.
* Move the DataReference library pane slot into querying
DataReference is the slot's only reader and querying owns its props type, so the slot lives next to it as PLUGIN_DATA_REFERENCE.LibraryPane.
The enterprise library plugin registers into it at boot, and plugins/oss/library.ts no longer imports querying's DataReference types.
* Move the verified search filter slot into search
SearchSidebar is the only reader of PLUGIN_CONTENT_VERIFICATION.VerifiedFilter, so the field becomes PLUGIN_SEARCH_FILTERS in metabase/search/plugin, next to the other search filter components. The EE content verification plugin registers the filter there from initializePlugin, and the rest of PLUGIN_CONTENT_VERIFICATION stays in plugins without the search import.
* Declare the search filters slot with definePluginSlot
* Import definePluginSlot from the plugins entry point
* Describe the verified filter default without naming its consumer
* Cover enterprise feature gates and plugin reset behaviour
* Remove redundant plugin registration tests
* Move the useGetIcon plugin slot into hooks/use-icon
The hook is the slot's only reader, so the slot and its OSS default live next to it as PLUGIN_ICON.
The enterprise collections plugin registers into it at boot, and plugins/oss/collections.ts no longer imports metabase/hooks/use-icon.
* Declare the entity icon slot with definePluginSlot
* Import definePluginSlot from the plugins entry point
* Cover enterprise feature gates and plugin reset behaviour
* Remove redundant plugin registration tests
* Trigger the stats workflows from Run tests, gated on the uberjar artifact
Both workflows have recorded nothing since 2026-09-03. PR #81238 took the push
trigger off `uberjar.yml`, so master merges reach it only through
`workflow_call`. A reusable workflow reached that way raises no `workflow_run`
event, and both watchers went quiet. 143 master commits went unmeasured.
They now watch "Run tests", which is what builds the jar on a master merge.
Its conclusion is not a usable gate, because it is `failure` on master more
often than `success`, from jobs that have nothing to do with the jar. The
uberjar artifact is the right gate: a new job looks it up without downloading
it, and a docs-only merge that skipped the build finds nothing and stops.
Calling both from `run-tests.yml` would have been lower latency, but a stats
failure would then turn the master run red, and `rerun-workflows.yml` watches
that run and would re-run master CI over it. Keeping them in their own run
keeps a stats problem to itself.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01171Lkmug15JzrcN2aMBLMk
* Let a PR run the size benchmark against its own uberjar
A `workflow_run` workflow always runs the copy on the default branch, so a
change here cannot be tried out before it merges. This matches the trigger
Bundle Load Stats already carries. A called run measures and reports; the
upload step stays gated on the `workflow_run` path, so it writes no row.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01171Lkmug15JzrcN2aMBLMk
* Trim the trigger comments in the stats workflows
---------
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
* Give the visualizations a question builder instead of the metadata object
* Let the visualizations read the question builder themselves
* Read the store when a chart brush builds its question
* Drop the single-use question helper
* Give the map and object-detail views the question builder too
* Drop the last v1 metadata reads from the visualization tree
* Send the Field columns a parameter widget reads in param_fields
* Stop re-resolving a parameter widget's fields through the store
* Keep the empty default a parameter widget's fields prop had
* Read the driver-dependent Field columns in param_fields expectations
* Add a harness for measuring bundle load time
* Report the second load and make the benchmark safe to run in CI
* Run the load benchmark across network and CPU conditions
* Sign the load benchmark in to a real instance
* Track page load times on every master merge
* Backfill load times for recent master commits
* Rewrite the load-time backfill in TypeScript with a tested core
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* Launch the benchmark jar the way we ship it, and lint the .ts scripts
* Measure the whole load, not just DOMContentLoaded
* Do not let the performance marks gate a reading
* Drop the backfill workflow
The goal is a global trend, which the per-merge job builds on its own after a
few days. Backfilling only bought a readable chart sooner, at the cost of a
second workflow and the largest script in the PR.
The commit date stamp goes with it. It existed so a backfilled row could claim
its commit's date rather than today's, and the per-merge job never set it.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012b67ubbXJVMuQrpuEksbeh
* Let a PR run the load benchmark against its own uberjar
A `workflow_run` workflow always runs the copy on the default branch, so a
change here cannot be tried out before it merges. The `workflow_call` trigger
lets CI hand the workflow a commit and reuse the uberjar it already built. A
called run measures and reports; it does not write rows to Stats.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01171Lkmug15JzrcN2aMBLMk
* Report each state from one load, not a median per reading
Every reading was medianed on its own, so a row described a load that never
happened. It could also put the readings out of order and print a page-ready
that precedes the TTFB beside it.
`measure.js` now picks the run whose DOMContentLoaded is the median of its
series and reports that run whole. The second visit happens once per browser
profile, so it stays the run itself, and its page-ready now comes from that same
load rather than from a median that included the series' first, uncached run.
`Cold spread %` still comes from all the cold runs. A spread is a property of the
series rather than of one load.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01171Lkmug15JzrcN2aMBLMk
* Read the largest paint with an observer, and cut the timeout
The first CI run reported a zero largest paint in all four conditions, and would
have done so forever. These entries never reach the performance timeline, so
`getEntriesByType` cannot see them. Only a `PerformanceObserver` can, and it has
to exist before the paint. Paint entries are buffered, which is why
first-contentful-paint needed none of this.
The browser keeps revising the value until the user interacts, and the reading is
taken once the page reports ready. It is a lower bound rather than the final
figure, which is what makes it comparable between commits.
That same run took 4.4 minutes, of which the measurement was 164 seconds. The
timeout drops from 25 minutes to 10.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01171Lkmug15JzrcN2aMBLMk
* Move the bench to frontend/test and write it in TypeScript
---------
Co-authored-by: Fraser <frasermcintosh1@gmail.com>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
* Take a metadata provider in the detail-view query helpers
* Select the detail-view queries instead of building them
* Take the provider for the table's own database
* Guard the table query selector's memoisation
* Type the table query guard against the API table
* for EE/white_label - do not use hardcoded metabase in filenames
* test coverage includes test for special characters in application-name
* also use gibberish names in tests
Closes QUE2-820
Closes#80557
There are two changes:
1. handle `nil` nodes gracefully during traversal, and
2. don't add `nil` nodes to the graph in the first place.
Both of these changes alone fix the reported issue, but both make sense on their own.
* Stabilize collection type-filter options via an items-metadata endpoint, default the type checkboxes to unchecked, and hide the filters for collections of 10 or fewer items
* Show every type in the filter popover, disabling the ones without matching items
* Use kebab-case params on the metadata endpoint and adapt toolbar tests to the visibility threshold
* Scope the items metadata to the models the list shows, name its count total_items, and align root permissions with the items list
* Assert the restricted root metadata count against the items list instead of a fixture-dependent number
* Code review improvements
* Cleaned up kebob usage
* Adapt the branch to master's show-exploration-documents param and re-record kondo budgets
* Cover the items-metadata endpoint's exploration-document scoping
---------
Co-authored-by: Nathan Voxland <nathan@metabase.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Every provider behind Metabot now streams reasoning to the chain-of-thought UI,
with one predicate per adapter deciding the settings gate, the request directive,
stream forwarding and replay together.
* BOT-1895: Shared groundwork. Drop the pre-registry saved-model fallback from
Azure and Google; make tool_calls outrank reasoning in delta classification,
which previously lost a tool call's only id/name chunk; consolidate
reasoning-part coalescing into one helper.
* BOT-2027: Bedrock. Request thinking per model via the shared builders. The
openai-family gate answers false — probes across all four GPT models confirm
Bedrock never emits reasoning summaries.
* BOT-2028: Azure. Gate derived from the admin-typed deployment name, matched
case-insensitively, which also fixes a latent temperature 400 on cased Claude
deployments. Negative paths verified live; the positive path rests on docs.
* BOT-2025: Z.AI. Forward reasoning_content and send the thinking directive
explicitly — GLM was already thinking invisibly on every chat. Thinking-only
models get reasoning_effort and a 2048 floor on forced tool calls, flagged on
the supported-models row so the sets cannot diverge.
* BOT-2024: OpenRouter. Forward flat delta.reasoning and classify all 26 models
by live probe; the gpt-5.5/5.6 family streams under the server default, where
an explicit enable verifiably suppresses it. Forced tool calls floored on
mandatory-reasoning models.
* BOT-2026: Google Gemini. One catalog drives context windows, directive and
gate. Thought summaries on chat, thinkingLevel LOW on structured (no off
switch), and a 2048 floor — probed, a real prompt at 512 spent the whole
budget thinking and returned no tool call.
* BOT-2030: Mistral. Net-new cost: the default sends no thinking and only
"high"/"none" are accepted. Flattens vector content chunks into the shared
xf's events, and adds the opt-in reasoning replay hook Mistral's docs require.
* BOT-2046: Moonshot. kimi-k3 always thinks — explicit effort, mandatory in-turn
replay, 2048 floor; kimi-k2.6 is optional and disabled under a forced tool
choice, which it rejects while thinking.
Co-authored-by: Paolo Rascuna <1936871+prascuna@users.noreply.github.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* Take a metadata provider in the shared query helpers
* Select the segment query instead of building it
* Take the provider for the segment definition's database
* Guard the segment query selector's memoisation
* Guard the segment query assertion against a vacuous pass
* Add a jest project for heap retention specs
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012b67ubbXJVMuQrpuEksbeh
* Scope the parsed x value cache to one getXValues pass
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012b67ubbXJVMuQrpuEksbeh
* Assert heap retention on the settled pass, not the first touch
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012b67ubbXJVMuQrpuEksbeh
* Key the log timestamp cache on the log object
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012b67ubbXJVMuQrpuEksbeh
* Key the schema viewer flow graph on the response object
The memo key serialized every table and field of the schema, so each distinct
schema held a megabyte-scale string forever. RTK Query returns a stable
reference per cache entry, so a WeakMap keeps the identity that useGraphSync
depends on without retaining the schema.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012b67ubbXJVMuQrpuEksbeh
* Key pie.rows on the dataset instead of a JSON copy of it
The memo key serialized every row and column, so each dataset the user viewed
held roughly its own size in cache key for the life of the tab. A WeakMap on
the dataset with a short settings subkey keeps the same hit rate.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012b67ubbXJVMuQrpuEksbeh
* Ban new uses of underscore's memoize
The 17 remaining call sites are allowlisted. The list may only shrink.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012b67ubbXJVMuQrpuEksbeh
* Narrow the memoize rule to module scope
A cache built inside a function, a component render, a class field or a
constructor is released with whatever owns it. Only a module scope cache lives
for the life of the tab, so the rule now flags just those.
This drops the allowlist from 17 files to 5, and each remaining entry carries
the reason its set of keys stays small.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012b67ubbXJVMuQrpuEksbeh
* Remove the memoize allowlist
Three of the five module scope caches are gone:
- getScrollBarSize takes no arguments, so it is a lazy value, not a cache.
- getScheduleExplanation returns a string that no caller compares by identity.
- cronToBuilderValue moves to the one caller that needs a stable identity.
Schedule takes its value as a useMemo dependency, so StrategyForm now holds
that memo itself. A user can write any number of cron strings, so a module
cache for them was unbounded.
The remaining two carry an eslint-disable with the reason their set of keys
stays small.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012b67ubbXJVMuQrpuEksbeh
* Follow renderer_utils into the viz-core module
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012b67ubbXJVMuQrpuEksbeh
* Cover metabase/utils/memoize in the module scope rule
That memoize roots its WeakMap on the function it wraps, so it only releases
once that function does. At module scope the function never does, and the
nested Maps hold every argument ever passed. No call site does this today, but
the rule could not have caught one.
The rule now covers both sources, so it drops "underscore" from its name.
memoizeClass stays allowed: it keys on the instance.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* Drop re-reselect from the dashboard selectors
createCachedSelector keyed getQuestionByCard and
getDashcardParameterMappingOptions on the card id, a primitive, in a
FlatObjectCache. That cache is a plain object with no eviction, so a session
kept one Question per card id it ever saw, and each Question pins the whole
Metadata snapshot it was built with.
reselect v5 defaults to weakMapMemoize, which keys on the card and the metadata
objects themselves. Entries are released when their card leaves the store or
the metadata is replaced, and identity still holds for every card in the
working set, which is what connect and useSelector compare on.
Measured over 300 cards, with the states dropped: 6.2 MB retained before,
0.0 MB after. Per entry, 21462 B before and 51 B after.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* Drop re-reselect from the lockfile
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* Back metabase/utils/memoize with reselect's weakMapMemoize
The hand-rolled implementation had two problems. It allocated two arrays on
every call, including cache hits: 225 ms against 20 ms for weakMapMemoize over
two million hits. And its WeakMap was keyed on the wrapped function rather than
the wrapper, so a cache built around a module-level helper outlived the wrapper
that owned it. A plain Map would have been safer.
weakMapMemoize matches on the same terms, identity for objects and value for
primitives, so no call site changes. It also weak-keys object arguments, which
the old one held strongly.
definition-cache passed weakMapMemoize as the memoize and argsMemoize options.
reselect v5 already defaults to both, so those are dropped.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* Route all memoization through metabase/utils/memoize
underscore's memoize is gone from the app. Its 16 call sites now use the house
helper, which is reselect's weakMapMemoize. Three of them were in metabase-lib,
which cannot import metabase/utils, and all three took no arguments, so they
became plain lazy fields instead.
Two rules keep it that way:
- no-direct-memoize-import bans memoize from underscore and weakMapMemoize or
lruMemoize from reselect, in any scope. Only metabase/utils/memoize may reach
for an implementation.
- no-module-level-memoize keeps its job, now that the house helper is the only
one left to police.
measureTextWidthSafely took a style object that was rebuilt on every render, so
identity keying would never have hit. Its third argument is now the font family
string, which is the only field any caller ever set.
useMemoizedCallback returns the callable shape rather than casting to T, since
the memoized function also carries clearCache and resultsCount.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* Build memoizeClass on the house memoize
It kept a module-level WeakMap and walked a nested Map trie keyed on the
instance, the method, the argument count and then each argument, assembling
that path into an array on every call.
Keying weakMapMemoize on the instance and then the arguments gives the same
per-instance caching. One cache per decorated method replaces the shared trie,
so the method no longer needs to be part of the key, and neither does the
argument count. The file is down to 61 lines from 128.
A leak spec covers the property this rests on: an instance's results are
released with the instance.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* Measure memoizeClass retention under load
2,000 instances per pass, 20 calls each, across a method with no arguments and
one taking a primitive and an object. Nothing is retained once the instances
are dropped.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* Delete memoizeClass
It decorated a prototype method by rewriting the property descriptor, and put
the memoization 600 lines away from the method it applied to. Its one consumer
memoized a method whose body uses no `this`, so the same caching comes from
assigning a memoized arrow to a class field. The field is created per instance,
so the cache is released with the instance, and the memoization is now visible
at the definition.
metabase/utils/memoize is 14 lines, from 128.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* Run the heap specs from their own jest config
They were a project inside jest.config.js, and a bare `jest` runs every
project, which is what CI does. So the sharded unit run picked them up, without
--expose-gc, and they failed as designed.
They now live in jest.memory.conf.js, the way the timezone specs live in their
own config. The shared base moves to jest.base.conf.js so neither file has to
duplicate it.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* Reduce the heap specs to one collection assertion each
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012b67ubbXJVMuQrpuEksbeh
* Follow the store mocks into test support
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012b67ubbXJVMuQrpuEksbeh
---------
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
A settings manager could change a connection's base URL without entering its
masked secret again. The update merged the stored secret back in, and
credential verification sent it to the new host before anything was persisted.
A network policy cannot tell a legitimate provider from a public host the
caller controls.
Changing a connection's effective base URL now requires freshly submitted
credentials for every sensitive field that would otherwise carry a secret to
the new destination. Environment-owned credentials must be repointed through
deployment configuration. Trusted provisioning such as config.yml is
unchanged.
Generic writes to the llm-providers setting run the same field validators, so
PUT /api/setting cannot bypass the connection API's checks.
* Enforce a network policy on LLM provider base URLs
A settings manager could point any provider's base URL at an internal
host; the model listing behind the admin Connect button then fetched
GET /v1/models from it with bare clj-http, and a 4xx body preview came
back in the error. Semi-blind SSRF.
New llm-allowed-networks setting, same shape as warehouse-allowed-networks:
external-only on Cloud, allow-all self-hosted (vLLM/Ollama on localhost
or a private network is the normal case there). Admin-only, so the
settings-manager class can't loosen it.
Every base-url registry field validates against it on write, so the
API rejects a forbidden URL before verify-credentials! makes the
request. core/request and llm.anthropic/chat-completion check again at
request time and resolve DNS through the policy resolver, which closes
the rebinding gap. The AI proxy URL is operator config and is exempt.
e2e pins MB_LLM_ALLOWED_NETWORKS=allow-all next to the warehouse pin,
since the mock LLM server is on localhost.
* Cover the embedding and Google token sinks too
Semantic search posts embeddings to llm-openai-api-base-url and to the
settings-manager-writable ee-embedding-service-base-url with bare
clj-http, so the same writer class reached the same kind of sink. Those
now run the llm-allowed-networks check and DNS resolver; the operator's
ai-service-base-url is exempt, and ee-embedding-service-base-url is
validated on write.
A Google service account key carries its own token_uri, and the
credential library posts to it to mint an access token before any
request we guard. Keys are now refused unless token_uri is one of
Google's OAuth hosts.
* Use var links in LLM policy docstrings
* Keep SSRF rejections out of the embedder breaker
* Normalize connection-time LLM policy errors
* Link LLM timeout settings in request docs
* Fix reflection warnings in LLM URL validation
* Use explicit policies for managed AI endpoints
* Address final Copilot review feedback
* Reject non-global special-purpose addresses
* Align network policy terminology
* Make external-only the LLM network policy everywhere
llm-allowed-networks is now environment-only (:internal, no setter)
and defaults to external-only on every instance. A settings manager is
who it defends against, and on Cloud a customer admin loosening it
would be reaching for our own infrastructure, so nobody sets it through
the API. Self-hosted operators with a vLLM server on a private network
opt in with MB_LLM_ALLOWED_NETWORKS; a value that is not one of the
three policies fails closed with one warning.
The rejection message says what to do: self-hosted, it names the
setting and which value fits; on Cloud, that private networks are out
of reach. The vLLM field's placeholder and help text no longer
advertise a private host as the default setup.
The managed endpoints' :allow-private is now a floor rather than a
replacement, so an explicit allow-all keeps an AI service on localhost
reachable. An embedding service URL supplied by the environment gets
the same floor, since the setter cannot vet it.
* Enforce the LLM network policy at connection time only
The request-time check resolved the host with a blocking lookup that
the connection's DNS resolver then repeated, and in the embedding path
it ran ahead of the breaker's fast-fail. Requests now run only the
I/O-free syntax check up front; the resolver decides on the addresses
the connection actually opens, and its refusal maps to the same 400
the set-time check gives. The set-time check still resolves, so a bad
URL is refused when it is entered.
One helper, llm-request-opts, replaces the three hand-copied
sequences; the refusal is thrown from one constructor. It carries
:status 400 beside :status-code because the semantic-search dead-letter
queue files errors by :status and was retrying a refused endpoint on
the fast schedule.
A base URL with a username or password is refused outright, and the
refusal names only the host, since the message reaches Metabot chat
users and the API's 400 body. A connection-time refusal is logged; it
is the only trace a host that rebinds leaves.
* Read llm-allowed-networks from the environment alone
get-value-of-type falls back to the application database, so a value
stored there before the setter went away kept working. The getter now
reads only the environment variable, and the setting carries a :doc
string so the generated environment-variable docs keep it now that it
has no setter. On Cloud the default applies; the operator's environment
is still honored, which the e2e runner's pin relies on.
* Disable redirects for LLM requests
* Normalize embedding service base URLs
* Normalize configured embedding URLs on read
* Clarify LLM network policy documentation
* Revert "Clarify LLM network policy documentation"
This reverts commit 8f2cff7c4963efba6756c117927ca56492cd49c5.
* Polish new LLM security comments
* Clarify embedding API key fallback
* Treat unallocated IPv6 space as non-global
* Grant private-network policy floors to environment-supplied URLs only
* Refuse IPv6 space IANA has not allocated
2000::/3 is the global-unicast space, but everything in it from 2d00:: up
is reserved: no ISP routes it, so a name resolving there is either a
mistake or an internal network squatting on it. `3000::1` passed as public
before.
* Pin Google's token endpoints by whole URI
The transport posts to whatever `token_uri` says, so checking only the
scheme and host let through another path, port, or query string on
accounts.google.com.
* Validate connections written as the llm-providers setting
The connection endpoints check a base URL against the network policy, but
the list they write is a setting in its own right: PUT /api/setting or a
config.yml entry saved a loopback URL straight past them. Run the field
validators in the setter too, on the connections a write changes.
* Hold the token-cache invalidation to the LLM network policy
/api/premium-features/token/refresh POSTs to ai-service-base-url with the
premium token in the URL path. That setting is admin-writable, so the
request needs the same syntax check, redirect ban and DNS policy as any
other LLM request; only an environment-supplied URL earns the private
floor.
* Check the target host when a JVM proxy is in the way
clj-http's route planner honours -Dhttps.proxyHost, so the connection is
opened to the proxy and the :dns-resolver is handed the proxy's hostname:
the target is resolved by the proxy, out of reach, and a private egress
proxy is itself refused under external-only. Treat an operator-configured
proxy as the trusted boundary it is and check the target host up front
instead.
* Judge a NAT64 address by the IPv4 it reaches
64:ff9b::/96 is globally reachable, but its low 32 bits are the IPv4 a
gateway translates to, so 64:ff9b::7f00:1 passed external-only and reached
127.0.0.1. RFC 6052 forbids the prefix carrying a non-global IPv4; nothing
on this side of the gateway enforces that.
* Validate only the connection fields a write changes
Comparing whole connections meant rotating an API key re-ran the base URL
check, so a private URL stored before the check existed made its own
connection unwritable.
* Name a network policy that would change the outcome
A refusal under a deployment endpoint's allow-private floor advised
setting MB_LLM_ALLOWED_NETWORKS=allow-private, which was already in force.
The resolver records the policy it refused under; use that for the message
and the log line.
* Refuse an unresolvable host behind a JVM proxy
Everywhere else a host that will not resolve is allowed, because the
connection then fails on its own. Behind a proxy there is no connection to
fail: the proxy resolves the target, so a split-horizon or cluster-internal
name went out with nothing checked at all.
* Keep proxy-only DNS working
* Document and test the LLM proxy enforcement boundary
* Match JVM proxy detection to Apache routing and fix tile fixture
* Build query builder questions from the store's builder
* Build document questions from the store's builder
* Build SDK and document questions from the store's builder
* Build queries from a provider, not the raw Metadata object
* Build questions from cards through the metadata store
* Type the dashcard menu against the question dashcards its callers pass
* Name the guard the callers actually use
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HNvsg9cPuNQ6BHiDwa1bif
---------
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
The illustrations switch only appeared next to a custom icon, so turning illustrations off
and then removing the icon left no way to switch them back on. It now appears whenever
illustrations are off as well.
Clackbot answered every failure with "Something went wrong", so a user denied by AI
permissions or an admin with a misconfigured provider had nothing to act on. Both the
streamed error part and the thrown-exception path now look the failure up by error
code and reply with copy that names the problem, and usage-limit errors pass their
server-authored message through. Anything unrecognised keeps the generic line, so raw
provider errors stay out of shared Slack channels.
* Serve the metabase-lib provider from the metadata store
* Serve metric metadata providers from the store too
* Serve providers for metrics and for per-item database lookups
* Take a provider lookup in the transforms query helpers
* Resolve SDK query sources from the RTK cache, not the mirror
* Delete the unused isSourceEmpty helper and the dead tableId prop chain
* Canonicalise getMetadata's options so both call shapes share one Metadata
* Build v1 Questions through the metadata store (#81666)
* Use explicit hierarchy for action keywords
The :table.row/* -> :table.row/common and :data-grid.row/* -> :data-grid.row/common
relationships lived in Clojure's global hierarchy, sharing one mutable namespace with model
keywords, hook traits and every other subsystem. Move them into metabase.actions.hierarchy,
mirroring metabase.events.hierarchy: a defonce var holding a hierarchy map, mutated through
derive!.
Four multimethods consume those parents -- action-arg-map-schema, normalize-action-arg-map,
validate-inputs! and default-mapping -- and now dispatch on the new hierarchy. Driver-level
action methods are untouched: they dispatch on [driver action] through the driver hierarchy
and only ever name leaf action keywords.
The enterprise action_v2 module derives through metabase.actions.core, which re-exports
derive!.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* State the real reason driver action dispatch is unaffected
The hierarchy docstring said driver-level action methods are safe because they only ever name leaf action
keywords. That describes an incidental convention -- `[:postgres :model.row/create]` happens to be the only
such method -- and implies a fragile dependency that does not exist.
`driver/hierarchy` is its own `(make-hierarchy)`, and `isa?` against an explicit hierarchy never consults the
global one for keyword parents, so the derives this namespace moves had no effect on `prepare-query*` before
the move either. Say that instead, and note it at `prepare-query*` too, where someone adding a driver method
will actually be reading.
---------
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-authored-by: Chris Truter <chris@metabase.com>
* Use explicit hierarchy for auth providers
Provider relationships (:provider/saml -> ::provider/provider, etc.) lived in Clojure's
global hierarchy, sharing one mutable namespace with model keywords, hook traits and every
other subsystem. Move them into metabase.auth-identity.hierarchy, mirroring
metabase.events.hierarchy: a defonce var holding a hierarchy map, mutated through derive!
and underive!, read through isa? and descendants.
The three provider multimethods (validate, authenticate, login!) now dispatch on that
hierarchy via methodical's :hierarchy option.
Callers outside the module go through metabase.auth-identity.core, which re-exports the four
operations; namespaces inside the module use metabase.auth-identity.hierarchy directly, since
core requires them.
derive! is whitelisted as :parallel/safe because each test registers its own distinct
:provider/test-* keyword, so the additions never overlap. The plain `derive` calls it replaces
were equally global but escaped the linter by not ending in `!`.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* Register test providers at namespace level, not inside parallel tests
Each of these eight ^:parallel tests registered its provider by calling derive! and
methodical/defmethod in the test body. Both mutate process-wide state, so the tests were never
actually parallel-safe -- only derive! was visible to the linter, because defmethod does not
end in `!`.
Hoist both registrations above their deftest, alongside the test providers already registered
at the top of the namespace. The deftests are now pure assertions over providers that exist by
the time the namespace has loaded, which is what ^:parallel claims. No linter exemption for
metabase.auth-identity.core/derive! is needed, so .clj-kondo/ratchets.edn is untouched.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-authored-by: Chris Truter <chris@metabase.com>
Domain entity types derived from :DomainEntity/* in Clojure's global hierarchy, sharing one
mutable namespace with model keywords, hook traits and every other subsystem. Specs are loaded
from YAML files, so adding a spec file was writing into that shared space.
Move them into metabase.xrays.domain-entities.hierarchy, mirroring metabase.events.hierarchy:
a defonce var holding a hierarchy map, mutated through derive!. A leaf namespace rather than
an in-file var because both specs (which registers types and validates them) and core (which
ranks candidate entities by ancestor count) read it.
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Key-value namespaces registered by defnamespace derived from ::registered-namespace in
Clojure's global hierarchy, sharing one mutable namespace with model keywords, hook traits and
every other subsystem. Namespaces are loaded from EDN files on disk, so an operator adding a
file was writing into that shared space.
Move them into a hierarchy var private to this namespace, mutated through derive! -- the same
shape as metabase.events.hierarchy, kept in-file because defnamespace is the only writer and
known-namespaces the only reader.
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
* Fail the mage suite when a test namespace is not required
`mage -test` runs `run-all-tests`, which only sees namespaces that are already
loaded, and loading happens solely through this namespace's `:require`. A test
file missing from it never runs, and nothing says so.
`mage.quick-test-runner-test` and `mage.start-db-test` were missing that way.
They are required by `mage/test/mage/runtests.clj`, but CI runs `mage -test`,
which does not read that file, so their four tests had not been running.
* Drop the dead mage test runner and a test that cannot fail
Nothing referenced mage/test/mage/runtests.clj. It required three namespaces
directly, which made mage.quick-test-runner-test and mage.start-db-test look
wired up while `mage -test` never loaded either.
mage.start-db-test asserted a string literal, so it passed unconditionally.
Wiring it in would have added a green test covering nothing. start-db still
deserves real tests; this makes their absence visible.
* Fix EditableDashboard collapsing when entering edit mode without a sized parent
* Also fix EditableDashboard collapsing while sharing
* Revert "Also fix EditableDashboard collapsing while sharing"
This reverts commit be2f86c738.
GDGT-2028: Fix wrong column headers in alerts with a remapped foreign key
The header titles and the header row were built separately and zipped by
index. Only the header row dropped the remap target column, so every title
after it shifted by one. Both now derive from `visible-columns`.
An H2 dump made by an OSS build kept the metabot_permissions rows that migration 061 seeds into the fresh target instead of copying the source's own rows, because the table only appears in cmd.copy/entities when the Enterprise backend is on the classpath. The seed references the four magic groups at ids 1 to 4, which on the source are often different groups or no groups at all, so restoring that dump into Cloud either failed on a foreign key or handed Metabot permissions to whoever happened to sit at ids 3 and 4.
The table is now copied on every edition. The load side also skips rows whose group is missing from the dump, so the dumps customers have already sent restore once Cloud picks this up, and magic groups left without rows get the 061 seed re-run by magic_group_type so a restored instance matches a fresh one. Grants that landed on a real custom group cannot be told apart from a deliberate grant and stay a data fix, tracked in CLO-5999.
* Start subprocesses with ProcessBuilder
Runtime.exec's three-arg form has been deprecated since JDK 18, and it forced the command and environment
through String arrays with array type hints to match. ProcessBuilder takes both as collections, so the
call site is one line and the array juggling is gone. `:env` still replaces the parent environment rather
than adding to it, which is what the docstring promises: that is `.clear` then `.putAll`, not a bare
`.putAll`. read-lines now hints the BufferedReader the namespace already imports.
* Spend one timeout budget across a command's three waits
sh* waited for the exit code, then stdout, then stderr, giving each a fresh timeout-ms. A command that
dribbled output could take three times its budget, and the exception still claimed the single figure, so
"Timed out after 200 ms." could be thrown 600 ms in. The three waits now measure against one deadline
taken when the process starts.
* Measure the backend health-check wait with nanoTime
check-backend-health! polls for five minutes against a wall-clock start, so a clock adjustment mid-poll
could cut the wait short or stretch it. The interval literal is now written as a duration too.
* Kill the command however the wait ends, not only on timeout
Three fixes around sh*'s failure handling.
The catch was for ExceptionInfo carrying :timed-out?, so any other way out left the command running: a
reader future failing on the pipe, or the calling thread being interrupted. It now catches Throwable and
asks whether the process is still alive, which is the question that actually decides whether to kill.
:or does not fire on an explicit nil, so a caller computing :timeout-ms from an env var and handing us nil
got a NullPointerException building the deadline rather than the default.
The test's 200 ms budget left the shell to fork twice and write both pids with no margin. On a loaded
runner the file could come up a line short, failing the descendant assertion for the wrong reason. A
second is still far below the bound the test asserts.
* Stop waiting on a command that orphaned our output pipe
sh* could sit for as long as an orphan lived, whatever its timeout said: 62 s against a 500 ms budget in
the case that prompted this. with-open closed the BufferedReader, and that close takes the same lock a
blocked readLine holds, so it waited for EOF -- which a process the command left holding our stdout never
delivers. Closing the pipe underneath ends the read at once, so the streams are now closed directly.
Closing the pipe means a cut-off read reports EOF rather than an error, which would make truncation
invisible. Two things guard against that. The readers get a floor of their own once the exit code is in,
so a command that finished a moment inside its budget is not robbed of the output it had already written.
And if the pipe is still held when that floor runs out, sh* says so and throws, because half a command's
output returned as though it were all of it is worse than a failure.
The orphan itself is still running afterwards. Reaching it needs process-group handling, which is a
larger change and no mage command leaks one today.
* Rewrite the comments and docstrings in mage.shell
The docstrings now state the caller's contract and nothing else: what sh* returns, what it throws, what
the timeout actually covers now that one budget spans the whole command. Rationale about how the body
works moved down to the lines it explains -- why the descendants are listed before anything is signalled,
why the deadline is read off a monotonic clock, why the pipes are closed rather than the readers.
* Tighten the comments once more, and drop the duration glosses
A second pass over the comment wording, and a correction: a DST change does not move the epoch
milliseconds `currentTimeMillis` reports, so it was never one of the things nanoTime protects the deadline
from. NTP corrections and manual clock changes are.
The `; 5 seconds` style glosses are gone throughout. Writing the constant as `(* 5 1000)` already says
which unit it is in, and the name carries the rest.
* Serialize mage shell output and fix test namespace loading
`println` writes the text and its newline separately, so the stdout and
stderr readers could land halves of two lines on one line of the terminal.
Lock on the destination, which also keeps two commands streaming at once
from splitting each other.
`mage -test` runs `run-all-tests`, which only sees namespaces that are
already loaded, and nothing required mage.bot.pr-env-test or
mage.bot.repl-eval-test, so eight tests never ran. The `(comment ...)`
block meant to hold these requires open only referenced an alias that
existed for its own benefit; kondo does not flag alias-less requires, so
it was protecting nothing.
A Bedrock connection with no access key pair now signs with whatever the AWS
SDK default credentials chain resolves (environment, IRSA, EKS Pod Identity,
ECS task role, EC2 instance profile), and the SDK refreshes those on its own,
so a pod that already has a role needs no long-lived keys. Explicit keys still
override the chain, and half a pair or a lone session token is rejected.
On Metabase Cloud the chain would resolve our own identity rather than the
customer's, so a keyless connection is rejected there, and a token status the
license service could not confirm counts as hosted.
The streamGenerateContent xf walked each event's parts with a plain
`reduce`, which strips the `reduced` wrapper a cancelled consumer
returns. The stop never reached the outer reduction, so every event
that followed was still translated and pushed downstream — Google's
response was read to the end, and billed, after the client had gone.
The other adapters propagate the stop; only this one dropped it.
Use u/reduce-preserving-reduced so the early stop survives, and pin it
with a test that cancels on the first text delta: before the fix the
consumer kept receiving text from later events.
Co-authored-by: Paolo Rascuna <1936871+prascuna@users.noreply.github.com>
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
* Only serve tenant attributes and member_count to superusers
* Scope pulse recipients and the Slack directory by tenancy
* Update cross-tenant recipient test for non-superuser scoping
* GDGT-2467: Restrict Paper padding/radius to curated token options
* Update Loki Snapshots
* GDGT-2467: Update entity picker width e2e expectation for Paper padding
* GDGT-2467: Keep Paper default padding at 0, add no-padding option to stories
* GDGT-2467: Keep all padding options in Paper stories, only curate radius
* Update Loki Snapshots
* GDGT-2467: Type PAPER_RADIUS_OPTIONS against RadiusScaleKey
---------
Co-authored-by: Metabase Automation <github-automation@metabase.com>
* Resolve the under-test transform's target live in cycle detection
* Split the change-target overwrite e2e test into the two cases it conflated
Retargeting onto the transform's own source table is a self-cycle (400), not a
target conflict, so exercise the conflict case with a table produced by another
transform and add a dedicated self-cycle case.
* Make the transform under test win target collisions in get-transform-cycle
* remove live-target-table-id
* Make fix-sql-params regex linear-time
The tag-body class excluded only `}`, so `\s*[^}]+\s*` had three overlapping
quantifiers and backtracked cubically on an unclosed `{ {` followed by
whitespace. Exclude `{` from the body and drop the redundant \s* so every
position matches in linear time.
* Use :standardsql dialect in test so no kondo ignore is needed
Previously it was just picking a table at random and using that row,
but in redshift it turns out that table's row doesn't actually exist,
causing a bunch of failures. Let's look specifically at the people
test-data table instead of just using whatever.
Sweep orphaned search_index__* tables at the start of reindex! instead of at activation, so a rebuild
that dies partway is cleaned up by the next run instead of leaving its table behind. The sweep is best
effort and never aborts the reindex.
* add test
* Cast DATE inputs before timezone conversion on Oracle and SQL Server
`convertTimezone` on a `DATE` column failed on both drivers:
- Oracle: `FROM_TZ` only accepts a `TIMESTAMP` and refuses to implicitly
promote a `DATE` (ORA-00932: inconsistent datatypes: expected TIMESTAMP got
DATE). Cast date-typed inputs to `TIMESTAMP` first.
- SQL Server: `AT TIME ZONE` rejects a `date` argument (error 8116: Argument
data type date is invalid for argument 1 of AT TIME ZONE function). Cast
date-typed inputs to `datetime2` first.
Fixes `convert-timezone-date-column-test` for both drivers (#27186).
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01RYP35BadwdEL9LAC9zHWQZ
* Derive expected DATE column values from the results timezone in test
`convert-timezone-date-column-test` hardcoded the raw `d` column as
midnight `+07:00`, which is only right for drivers that support
`:set-timezone`. SQL Server can't set a session timezone, so its results
timezone falls back to the database timezone (UTC in CI) and the column
formats as `Z`. Compute the expected values the same way `format-rows`
does so the assertion holds on every driver with `:convert-timezone`.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01RYP35BadwdEL9LAC9zHWQZ
---------
Co-authored-by: Claude <noreply@anthropic.com>
We want to drop as much usage of legacy-mbql as we can. This function
is never used internally, but it's exported in the driver-api. We
should mark it as deprecated since everything else in this namespace
is deprecated, so we can remove it when the time comes.
Need to get this in today so it's included in the 64 branch.
* Replace Bigquery/Snowflake dataset tracking table; track by name instead.
The current system of using a tracking table was based on the
assumption that all datasets would need to be tracked by access
time. In fact, datasets fall into one of two buckets, either they are
static and immutable, or they are temporary one-offs. Static datasets
change *very* rarely and do not necessitate automated tracking. If we
neglect to clean them up, we could go for decades before the
accumulated datasets pose a real problem. But non-static datasets are
used once and accumulate *very* rapidly, needing to be cleaned up.
All datasets defined using `defdataset` are considered static.
Rather than using a database to track which ones must be cleaned up,
the logic can be much simpler; use the date embedded in the dataset's
name. It turns out it's even simpler because the logic to track
datasets based on their name has existed this entire time since 2023
in `metabase.driver.sql.test-util.unique-prefix`; we just stopped
using it in Snowflake/Bigquery in favor of the overly-complex tracking
table which A) leaked like mad and B) mistakenly marking static
datasets as old and causing them to get deleted while they were still
in use.
Redshift already uses the `unique-prefix` namespace generating its
schemas, so it only needs a few minor changes to adapt to the changes
in the gc job and the `unique-prefix` functions.
This also simplifies the gc nightly task; rather than having two
separate settings, we only need one setting for how many hours old a
non-static dataset has to be before cleaning it up. Also adds a
`dry-run?` option for manual testing.
* Address review feedback.
* Update workflow yaml to refer to single :hours parameter
* Remove 256-row limit from bigquery/snowflake
* Fix snowflake dataset :name in report data
* Get rid of with-write-stmt helper.
* Accept tracked? arg for manually GCing tracked datasets.
* Keep the data model in sync when a change is undone
* Extract shared TableSection update handlers into a hook
* Undo a custom field order with a single request
* Guard the optimistic table patch by table id
* Reproduce the dropped refetch scenario in the middleware test
* Link the upstream RTK issue from the dropped-refetch middleware and test
* Accept "all" as a collection id in the browser public API
Adds the public api plumbing
* refactor: own the ItemsTable container query inside ItemsTable
The CSS container was defined in CollectionContent.module.css, which meant it only worked on ItemsTable when it was rendered inside CollectionItemsTable. The virtual "all collections" root renders ItemsTable on its own, so the container moves onto ItemsTable.
* refactor: split the collection breadcrumbs into a view and its data
We'll need to customize what breadcrumbs will be rendered for the virtual root.
This commit extracts the view that we'll use directly.
* Clear the browsed collection while the next one loads
RTK Query keeps serving the previous collection from `data` while the next one
loads, and keeps serving it once the argument becomes skipToken. A caller that
reads `can_write` during the switch therefore offers to create content in the
collection the user just left.
`currentData` drops to undefined on any argument change, so it covers both
cases and the manual clearing from 397a3e82e04 goes away.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Gdru2a8afSo6y24e99W6hy
* Don't let the popped stack entry override the clicked breadcrumb
Clicking a collection breadcrumb from inside a question landed on the
collection the browser was mounted on, not on the one the user clicked.
Opening a question, a metric or a model pushes a navigation stack entry, so the
back button keeps working. That entry carried an onPop that reset the view to
the mounted collection. The breadcrumb handler sets the view to the clicked
crumb and then pops the stack, so onPop ran last and overwrote the click.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Gdru2a8afSo6y24e99W6hy
* Render the virtual "all collections" root in the collection browser
collectionId="all" now lists everything the current user can reach, as flat
siblings, in the same order the entity picker uses: "Our analytics", the shared
tenant collections, "Our data" and the personal collection.
useAllCollectionsItems builds those rows:
- A 403 on GET /api/collection/root means the user cannot read "Our analytics"
itself, so the hook falls back to /api/collection/root/items, which has no
read check on the root. The models=collection argument is required: without
it the endpoint returns no models at all.
- "Our analytics" needs a numeric id for the table, so it gets 0. Real
collection ids start at 1, and the caller maps 0 back to "root".
- The shared tenant collections come from the collection tree, gated on the
use-tenants setting and on the tenants plugin. shallow=true keeps the
backend from building a nested tree the caller never reads.
- The backend excludes the tenant collection from /api/collection/root/items,
so the hook adds it from the current user.
CollectionBrowser renders the rows with ItemsTable directly, without the
archive column: archiving resolves its permissions from a parent collection,
and the virtual root has none.
The breadcrumbs put a static "All collections" crumb in front of
CollectionBreadcrumbsView. The personal and the namespaced collections lose
their "Our analytics" crumb, because the API reports them as its children
while the virtual root shows them as siblings.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Gdru2a8afSo6y24e99W6hy
* Key the SDK breadcrumbs by type and id
Breadcrumb ids come from separate id spaces, so a collection and a dashboard
can both be 5. On an instance where collection 5 holds dashboard 5 the trail is
[all-collections "all", collection 5, dashboard 5], and two children of the
same list share the key 5. React warns about the duplicate key in the host
application's console.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Gdru2a8afSo6y24e99W6hy
* fix sdk package type
* e2e tests
* Unit test the virtual "all collections" root
* "all collections" -> "all-collections"
Co-authored-by: Mahatthana (Kelvin) Nomsawadi <me@bboykelvin.dev>
* pageSize support
* Assert the error message, not the error container
The virtual root error tests only checked that the sdk-error-container
testid rendered. That proves an error appeared, not which one, so any
other failure rendering into the same container would keep them green.
They now look for "Failed to load collections".
Verified by mutation: changing that string in CollectionBrowser turns
all four tests red, where the testid version stayed green.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01C9MrpBnd3VEiXVvYf15Qqd
* change root id from 0 to -1 and wrap the mapping
* import already existing defer
* not.toHaveLength(0) -> toBeGreaterThan(0)
* clarify that the root is 403 in the test
* `SdkBrowseCollectionId` -> `SdkBrowserCollectionId`
* docs(sdk): list the accepted collectionId values as bullets
The CollectionBrowser `collectionId` doc described five accepted values in
one running paragraph, and adding "all" pushed it past readable. Split it
into one bullet per value, and document the entity ID string, which the
paragraph never mentioned.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UTu6CuVJnRjzS1HrmrJQT3
* else if
* rename the virtual root identifiers to all mode root
* refactor root status into union
* clarify when the root could be empty
* fix eslint
---------
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-authored-by: Mahatthana (Kelvin) Nomsawadi <me@bboykelvin.dev>
* Warn when startup or a migration encrypts data that was stored unencrypted
The settings backfill reports that settings saved by an older version are being
converted, the startup sweep reports each column whose plaintext values it
encrypted, and the migrations that encrypt existing rows report how many they
touched, so an operator can see when and why data changed at rest.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
* Find legacy unencrypted values first and hand them to a handler before encrypting them
`legacy-unencrypted-value?` decides what counts as legacy plaintext, the startup
sweep and the encrypting migrations filter their rows with it, and
`handle-legacy-unencrypted-values!` is called with the count before anything is
encrypted, so it can turn from a warning into something else later. The
migrations keep private mirrors of both, as they do for the encrypted JSON
helpers, and read their rows eagerly: reducing a reducible query closes the
transaction's connection.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
* Call the handler's location parameter title
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
* Make the encryption migrations no-ops and move the app-db module's queries to db.clj
The Encrypt* custom migrations did the same work `encrypt-plaintext-columns!` does
on every startup, so their forward bodies are gone; the reversible ones keep their
decrypting rollback for downgrades. The handler now takes only a `table.column`
location. The encryption and setting namespaces run their Toucan 2 queries through
the new `metabase.app-db.db`, like the other modules.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
* Drop the decrypting rollbacks of the no-op encryption migrations
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
* Make a fresh install come out fully encrypted from the migrations alone
The encryption-touching data migrations that only rewrote existing rows are no-ops
now that the startup sweep covers their columns, and MigrateMetricsToV2 goes with
them. Two v58 changesets left rows a fresh install would then see as legacy: the
SQL copy of password authentication into auth_identity is retired from the
changelog and redone in Clojure with the credentials encrypted, and sample content
is created by CreateSampleContentV3 once setting.value_with_aad exists so the
example-dashboard-id row is written whole. The settings backfill skips the
encryption-check sentinel, which the encryption tooling rewrites itself. The
fresh-install test asserts that setup logs no app-db warning and that the sweep
afterwards finds nothing.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
* Keep the sample-content migration tests in version order
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
* Stop inserting the plaintext encryption-check marker on new databases
The v53 changeset that inserted it is retired from the changelog: the encryption
tooling writes the sentinel itself, whole, and reads a missing row the same way, so
the settings backfill no longer needs to special-case it.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
* Delete the plaintext encryption-check marker instead of retiring its changeset
The v53 changeset inserted the marker on every new database, encrypted ones
included, so it never said anything about the actual state. It stays in the
changelog as a checksum-agnostic no-op, since a rollback to a release that still
has it must not re-run the insert, and a new changeset deletes the marker rows: with
no row the state is unknown and judged from the content, which the encryption
tooling already does.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
* Leave the v53 encryption-check changeset untouched
Its checksum is recorded on every existing database and validCheckSum: ANY does
not lift the check, so it keeps inserting the plaintext marker on a new database
and the v58 changeset deletes the row right after.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
* Keep the data migrations, retire the Encrypt* changesets, and never change a row's encryption state
The migrations that rewrite encrypted JSON columns are back with their original
bodies, but `encrypted-json-in` now writes a row back the way it was stored:
encrypted only when the stored value decrypts with the current key, so a
migration never turns plaintext into ciphertext or the reverse. The Encrypt*
migrations are removed outright, with a changeset retiring their changelog
records, since encrypting legacy plaintext is the startup sweep's job.
CreateSampleContentV2 is untouched again and a small migration fills its
setting's value_with_aad from the strictly decrypted value; the LLM providers
migration writes both columns too.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
* Address the review: decide by decrypting, stream the sweep, and write NULL rather than an empty SET
The settings backfill now treats a value as encrypted only if it decrypts, so a
plaintext value that merely looks like ciphertext is filled in instead of being
skipped forever. update-setting-values! takes the columns to set as a map, so a
value that encrypts to nothing is written as NULL as before instead of compiling to
an empty SET. The startup sweep streams each column again, warning once before the
first row it encrypts. The unused bytes predicate is gone, the string one is
private, and the two changelog-retirement changesets are one, using the changelog
table property.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
* Keep the SQL copy of password authentication; the startup sweep encrypts the credentials
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
* Give the internal user no password
It is inactive and can never log in, and a password would be copied into
auth_identity bare by the v58 SQL changesets, leaving a fresh install with one
row for the startup sweep to encrypt.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
---------
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
* add deterministic data_sensitivity rules classifier
* fill data_sensitivity rule table to full enum breadth
* add data_sensitivity scan step and opt-in setting
* register classify-data-sensitivity as the last analyze step
* document PUBLIC as scanned-or-reviewed for data_sensitivity; add end-to-end tests for the data_sensitivity scan
* label non-Text base types and integer/band variants in the data sensitivity classifier
* add reset-data-sensitivity! to clear classifier-written labels
* Add test for cumulative sum of a literal number (#27807)
Adds a QP test verifying that a custom aggregation expression of
CumulativeSum(1) on the Products table, broken out by Created At by
month and ordered ascending, runs successfully instead of failing
with 'Field 1: Field does not exist'.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011kKRbqUbfSKNViTEzS82ss
* clean up
* undo
---------
Co-authored-by: Claude <noreply@anthropic.com>
* Metadata for plain-`:field` expressions should propagate Field ID and Table ID
* Fix 🔧
* Add tests 🔧
* Test fix 🔧
* Test fix 🔧
* Test fixes 🔧
* Test fix 🔧
* add nullable data_sensitivity column to Field and its user-settings mirror
* Expose data_sensitivity on PUT /api/field/:id
* add tests for data_sensitivity survives sync, reactivation, and base-type changes
* add serdes round-trip tests for data_sensitivity on Field and FieldUserSettings
* Expose data_sensitivity in Lib column metadata
* tweak wording
* Keep on-demand FieldValues fetches alive past request cancellation
On-demand FieldValues are fetched on the request thread, so a canceled HTTP
request threw the warehouse scan away: every retry started the scan over and
nothing ever landed in the app DB, which is what makes these requests feel
like they take forever.
Run the fetch on a detached thread instead. It runs to completion and persists
what it fetched even when the caller stops waiting, and concurrent callers for
the same field (and sandboxing/constraint context) share the one in-flight run
rather than each kicking off their own scan.
Fixes GHY-2937 (#67843)
* Log detached FieldValues fetch failures
When every caller has walked away — the case detached-fetch! exists for —
nobody derefs the promise, so a thrown exception was dropped without a trace.
* Don't wipe cached FieldValues when the warehouse scan fails
`distinct-values` catches and logs warehouse errors, then returns nil — but
`create-or-update-full-field-values!` destructured that nil into empty values
and handed it to `persist-field-values!`, which reads empty as "this field has
no values" and calls `clear-field-values-for-field!`. So any failed or
timed-out scan deleted the values the field already had cached, and a field
slow enough to hit the QP timeout could never keep a cache at all.
Distinguish the two cases: nil means the scan failed, so leave whatever is
cached alone and return ::fv-fetch-failed. A scan that genuinely comes back
empty still clears, as before. The sync path already made this distinction
(see fetch-distinct-per-field's :failed-fields).
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XwGt3aocjPgJeGXTuEjujp
* Guarantee detached FieldValues fetches always complete
A registry entry left behind with its promise undelivered is worse than a
leak: every later caller for that key parks on it forever, holding a request
thread each. Two paths could do that.
The `log/warnf` added for failed fetches sits inside the catch, so an appender
that throws skips both the registry cleanup and the delivery. And the registry
entry is claimed before the work is submitted, so a rejected submit — which is
what `future`'s pool does once the JVM starts shutting down — orphans it.
Route both through `complete-fetch!`, which still drops the entry before
delivering so a woken caller starts a fresh fetch rather than re-attaching to
a finished one.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XwGt3aocjPgJeGXTuEjujp
* Sweep and cancel stalled detached FieldValues fetches
Nothing bounded the in-flight registry except every code path remembering to
clean up after itself, which is a property that keeps not holding — the
logging call has now stranded entries twice.
Give each entry a timer and its future, and sweep on the way into
detached-fetch!, which is the only moment the registry can grow. The sweep
cancels the work and *completes* the entry rather than just forgetting it:
callers park on the promise, so dropping one undelivered leaves them blocked
forever, each holding a request thread.
Only a fetch wedged outside the query processor can reach the max age — the
warehouse query is already capped by *query-timeout-ms* (20 minutes in prod).
Removing an entry is now conditional on it still being the registered one, so
a swept fetch that later finishes can't evict its own replacement.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XwGt3aocjPgJeGXTuEjujp
* Filter stalled fetches in the doseq rather than up front
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XwGt3aocjPgJeGXTuEjujp
* Cap the in-flight FieldValues registry at 100 entries
Callers park on their fetch, so the number in flight is today limited by the
request thread pool — but that is an accident of how these endpoints are
served, not something this namespace can guarantee, and it stops holding the
moment anything frees a caller early.
Past the cap, detached-fetch! stops detaching and runs the work on the calling
thread. That degrades to what happened before detaching existed — no
single-flighting, no surviving cancellation — rather than failing the request,
so a ceiling that should never be reached can't take a filter dropdown down
with it. Capacity is checked inside the swap, so the ceiling holds exactly
under concurrent claims.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XwGt3aocjPgJeGXTuEjujp
* Refuse FieldValues fetches past the in-flight cap
Reaching the cap means something is wrong — callers park on their fetch, so
the number in flight is bounded by the request thread pool unless that stops
holding. Absorb it by running the work inline and the ceiling stops being a
signal; throw a 503 and it says what it means.
Raised to 200 now that hitting it fails the request rather than degrading it:
an instance has to have raised MB_JETTY_MAXTHREADS well past its default of 50
before this is reachable at all.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XwGt3aocjPgJeGXTuEjujp
* Pin down that joining an in-flight fetch bypasses the cap
The behavior was already right — claim-fetch!'s `contains?` short-circuits
before the count check — but it reads as incidental, and a caller that only
attaches to existing work adds no entry, so the ceiling has no reason to
refuse it. Say so, and test it.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XwGt3aocjPgJeGXTuEjujp
* Bound how long a caller waits on a detached FieldValues fetch
sweep-stalled-fetches! only runs when a request arrives, which is exactly what
the case it guards against prevents: callers parked on a wedged fetch are
request threads, so enough of them and nothing reaches the sweep to cancel
anything. The watchdog was only reachable while the system was healthy.
Bound the waiter's deref by the same max age. The expensive thing here is the
parked request threads rather than the registry entry, and giving those back
lets requests flow again — at which point the next one sweeps the corpse. No
scheduler needed.
fetch-max-age-ms stays a backstop rather than a request-latency bound: a value
nearer a sensible HTTP timeout would return the thread far sooner, but
answering before the values are ready changes what the endpoint promises, and
that wants deciding on its own.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XwGt3aocjPgJeGXTuEjujp
* Report scan failures from POST /api/field/:id/rescan_values
The endpoint discarded the result and always answered {:status :success}. That
was already misleading, and the cache-wipe fix made it worse: a failed scan now
leaves the existing FieldValues untouched, so there is no observable trace of
it at all — the caller is told the scan succeeded and sees the values it had
before.
Return a 500 when the scan failed. The button already toasts on error, so this
is the difference between "Scan triggered!" and a visible failure.
Unlike the table and database endpoints, this one runs the scan synchronously,
so it is in a position to know.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XwGt3aocjPgJeGXTuEjujp
* Serve warm advanced FieldValues without a detached fetch
The advanced branch wrapped the whole select-or-insert in detached-fetch!, so a
cache hit — one SELECT — paid a thread handoff and a registry sweep for a plain
read. field-id->field-values-for-current-user walks advanced fields serially,
so a sandboxed user with N filters paid N of them.
Look the row up on the calling thread and detach only on a miss, which is the
only case that scans the warehouse. This is what the full-values path at
get-or-create-full-field-values! already did.
The miss path gains one extra SELECT, since select-or-insert! still does its
own read before inserting. That is noise next to a warehouse scan, and keeping
select-or-insert! there is what makes two callers racing a genuine miss safe —
detached-fetch! only single-flights within one JVM.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XwGt3aocjPgJeGXTuEjujp
* Stop the FieldValues tests mutating JVM-global state
Two tests could only be trusted because of a test-runner setting. The cap test
stuffed 200 fake entries into the shared in-flight registry, so anything else
asking for field values during that window would have been refused with a 503;
three tests swapped distinct-values with `with-redefs`, which rebinds the root
for the whole process. Under the current runner nothing else is running, but
that is a property of `:multithread? :vars`, not of the tests — flip it and the
failures land in other people's namespaces.
Make the two knobs dynamic so the cap test can bind the ceiling to 0 and drive
everything through the public API instead: one real fetch held in flight, a new
key refused, and a joiner admitted. That tests more than the fake entries did.
Use the thread-local `mt/with-dynamic-fn-redefs` where the work stays on
threads that inherit bindings. Where it delegates to the original, capture it
with `mt/original-fn`: `patch-vars!` installs its proxy as the root for the
life of the JVM, so a bare symbol captured after some earlier test patched the
same var would have recursed into the proxy.
The rescan test keeps `with-redefs` — `mt/user-http-request` is served on a
Jetty worker that inherits nothing, so only a root rebind is visible there.
Commented so nobody "fixes" it.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XwGt3aocjPgJeGXTuEjujp
---------
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
The three system prompts, the two usage limit settings and the quota message no
longer read their environment variables, so they are set through the admin page,
the API or a config file like every other setting. Name, icon and
show-illustrations keep theirs, which matches the MB_APPLICATION_* whitelabel
settings that self-hosted deployments set once at startup.
Demoting a setting used to drop it from the config file template as well, because
the template generator shared the environment variable docs filter. That filter
now sits with the docs alone, which puts use-tenants back in the template.
transforms-setup-complete is derived and never read back, so it is marked
read-only and the template stops advertising a key that does nothing.
An `:error-code` means the throw site wrote this as an API response rather than
hitting an unexpected failure, so the client gets the message and the error data
and nothing else. The Metabot token-limit 402 and the transforms trial-limit 402
both returned a full backend trace before this.
Validation errors, real 500s and the older `:error_code` spelling are untouched.
* MCP v2 prep: extract query guards + pulse helpers; additive shared-ns changes v2 needs
Behavior-preserving groundwork for the v2 MCP surface, landed while v1 is
untouched and v2 is not yet present:
- NEW metabase.agent-api.query-guards (+ its test): query safety checks
extracted so REST and MCP run identical guard logic.
- metabase.pulse: recipient-filtering and sensitive-metadata-stripping
logic moved from private fns in api/pulse.clj into models/pulse.clj and
exported via pulse.core / pulse.api, so REST and MCP share it. Pure move;
existing REST pulse tests are the check (+ new direct tests).
- metabase.channel.urls: adds metric-path, transform-path, collection-path
helpers (additive).
- metabase.metabot.scope: adds agent:{content,query,sql,delivery}:* scopes
and collapses agent:viz:mcp-ui:{query,drill-through} into
agent:viz:mcp-ui:* — a proven superset, backward-compatible with v1's
gating (REPL-verified: old concrete scopes still match). Also regates
agent:metric:* from SQL-generation to NLQ permission (scope_test).
- metabase.mcp.settings: adds mcp-v2-disabled-tools and
mcp-query-handle-ttl-hours (additive; v1-safe; tests included).
- metabase.models.serialization.resolve{,.mp}: adds *numeric-ids-allowed?*
dynvar; resolver miss messages no longer name a specific discovery tool
(the calling surface owns the recovery sentence) — mp_test updated.
- metabase.server.streaming-response: optional :executor option threaded
through do-f-async; default path (shared pool) unchanged. Needed by
long-lived SSE keepalive streams so they can't exhaust the fixed pool.
Review question for this PR: does REST/v1 behavior stay identical?
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BjQcjQx7NPGr3Rpf6QaG4b
Review fixes (2026-08-27 deep review of #81244):
- native-marker? requires a truthy :native value (an explicit-null key from a
JSON round-trip is not a marker); docstrings state the guard/agent-api
sequencing explicitly (live v1 path untouched; call sites migrate at retirement).
- check-mcp-ui-native-query! accepts agent:sql:execute (v1's concrete scope)
alongside agent:sql:run, and documents that scope claims arrive with the v2
session rework — until then native-over-UI-credential fails closed.
- mcp-query-handle-ttl-hours getter no longer rethrows on an unparseable env
var (falls back to the default instead of killing the GC task).
- *numeric-ids-allowed?* docstring rewritten in future tense naming its binder
(the v2 query pipeline) and reader (the dialect rework PR).
- viz-scope comment corrected (v1 resources still register those strings until
retirement; wildcard keeps them satisfied); mp.clj database-id tru args
str-wrapped against MessageFormat locale grouping; streaming-response
:executor docstring notes the Future.cancel(true) interruption requirement.
Follow-up (review round 2): the guards are now genuinely shared — agent-api's
private copies are deleted and its execution paths call query-guards directly.
Deliberate tightening: raw-scan native detection replaces normalize-then-inspect
(fail-closed on unparseable payloads). Full agent-api api-test green.
Re-review round: agent:sql:execute compat branch now tested; TTL getter's
catch path tested ("forever"); reject-native-query! no longer echoes the
payload in ex-data (matches the old live rejection shape); viz-scope comment
typo fixed; pure predicate tests ^:parallel; check-token-query-permissions!
docstring states the stage-0-fast-path/QP-backstop contract.
--- Close-out review response (slice 00's first adversarial round) ---
CRITICAL (native-query false-positive on legacy MBQL with a field/expression/tag named `native`)
FIXED. native-query? scanned the whole payload via tree-seq, so a legit query whose :expressions or
:template-tags map had a key named "native" was rejected (400) though it ran fine on master. It now
scans only the structural query-nesting edges (:stages/:joins, :query/:source-query), never
descending into caller-named :expressions/:template-tags; a malformed value off a structural edge is
still deep-scanned to stay fail-closed. Proven: a 17-case REPL matrix — every crafted false-positive
now false, every genuine native (legacy top/nested, MBQL5 stage, joins, string-keyed, malformed)
still true.
SECURITY (v1 MCP bearer authenticates a deactivated user) FIXED. resolve-access-token — the single
resolver shared by the v1 transport and the core session bearer bridge — now gates on
(t2/exists? :model/User :id user-id :is_active true). v1's transport dispatched straight on :user-id
with no active check; v2 was already refuse-only. Proven fails-when-broken: reverting the guard lets
a deactivated user's token resolve; restored, it is refused. v1 mcp.api-test (598 assertions) green.
IMPORTANT fixed: agent:delivery:write now granted by the metabot-other-tools bucket (was dead
config → alert_write would 403 everyone); metabot-nlq grants agent:content:read not :* (was
over-granting write to NLQ-only users); update-pulse-with-perm-checks! validates :cards against
CoercibleToCardRef (a bad string id was a 500, now a 400); check-mcp-ui-native-query!'s docstrings
now state it is dormant until the v2 session-rework slice wires it (was claiming to be live).
MINOR fixed: thrown-status test helper distinguishes no-throw from throw-without-status; kill-switch
check moved after the scope check (a scope-denied caller no longer learns the kill-switch state);
the query-handle-ttl default is one const, not two literals; mp_test gains positive assertions.
Re-verified (fresh JVM, run separately): query-guards + scope + settings + mp 328 assertions;
oauth + v1 mcp.api-test green. The one red in pulse-test is create-csv-xls-test = GDGT-3161, a
pre-existing EE-app-db bug (empty {} dataset_query -> download-perms 500) in the CREATE path (not
this fix's update path), reproducing on plain master — not introduced here. kondo 0/0; cljfmt clean;
modules-config regenerated (oauth-server :model-imports gains :model/User).
--- master parity sync (user-approved Option A) ---
Carries master 8c3ea6f40b (#80554 Remove Explorations dimension anchored blocks) + master search/appdb/index.clj (isolate-write! savepoint isolation). Stack forked before both; without them the dropped exploration_block.type migration is absent -> Liquibase changelog-integrity failure on real app-DBs (App-DB Part2 CI). Synced at base to restore parity. 41 files, all slice-untouched.
* Fix save-access-token! call arity after master merge
Master gave the test helper a 5th arg (client-id via
oauth-server.tu/with-oauth-client); the new
resolve-access-token-deactivated-user-test still called the 4-arg form.
Wrap it in with-oauth-client and pass client-id, matching every other
call site. Verified: full session-oauth-test namespace green locally
(10 tests, 27 assertions, 0 failures).
* Fix native-guard snake_case gap and make the tenant recipient filter work end-to-end
- native-map-edges gains :source_query/"source_query": legacy MBQL
normalization canonicalizes the snake_case spelling too, so a payload
like {"query" {"source_query" {"native" ...}}} reached the query
processor as a native stage while native-query? said false — an
MBQL-only-scoped caller could run raw SQL through /v1/execute,
bypassing the scope split and the execute-sql kill switch. Regression
tests cover both the string and keyword snake forms.
- The cross-tenant recipient filter compared :tenant_id on recipient
maps that the :recipients hydration never populates, so a tenant
caller lost every user recipient and a tenantless caller was filtered
by nil = nil. Tenant membership is now looked up from the app DB
inside the filter, and the filter only engages for a caller who
belongs to a tenant — a tenantless user sees recipients unfiltered,
as before tenants existed. The test now builds real users in real
tenants (production-shaped recipient maps, ids only) instead of
fabricating :tenant_id keys, and pins the tenantless-caller
passthrough.
* Match native-SQL markers case- and separator-insensitively
`native-query?` matched edge names and markers case-exactly, but the QP
normalizer canonicalizes keys case-insensitively and treats `_` and `-`
alike. `{:query {:SOURCE_QUERY {:native "select 1"}}}` therefore scanned
as not-native and then executed as a native stage — REPL-confirmed, along
with `:Source_Query` and an uppercased `:NATIVE` marker.
That is a live bypass rather than a latent one: since
`+refuse-unscoped-native-sql` went live on /api/dataset, this scan is the
only gate between an unrestricted-stamped MCP UI credential and raw SQL.
The same scan backs `reject-native-query!` on the agent-api execution
paths, so those inherit the fix.
All key and value matching now goes through one `token` fn — namespace
kept, lowercased, `_` folded to `-`. Nodes are walked as `[token value]`
pairs rather than by fixed lookup, so a node carrying two spellings of
one edge has both scanned instead of one silently shadowing the other.
* Deep-scan a non-map stage element instead of throwing
`native-query?`'s `scan-map` iterated the node's entries without checking it was a map, so a
junk payload like `{:stages [1]}` threw "Don't know how to create ISeq from Integer" and
`/v2/query` answered 500 where shape validation should have answered 400
(`combined-query-rejects-malformed-payload-test`). A non-map where a stage belongs is now
deep-scanned like every other malformed sub-tree: never throws, and a marker after it still
trips the guard.
* Address round-3 review findings on the extraction slice
- The bearer resolver's is_active gate is defense in depth, not the primary control: deactivating
through the model already revokes the user's tokens. The docstring now says so, and the test
deactivates with a raw core_user UPDATE (the path the gate exists for) and asserts the token was
not revoked, so the gate is what the test exercises.
- Pin that a tenant-scoped caller sees only same-tenant pulse recipients (tenantless users hidden).
- scope.clj comments described a wildcard collapse that is not in this diff; say what the slice
does: adds scopes, removes none.
- Correct the guard test docstring: the /api/dataset gate is wired by the next slice.
- Pin the guard's deep-scan fallbacks and the streaming response's :executor option with tests.
* Widen the recipient merge-back to cover the tenant filter
Addresses review feedback on the tenant recipient filter: a filtered read
that is submitted back as a write silently deleted the recipients the
caller was never shown.
The `:channels` submitted to an update are authoritative --
`update-notification-channels!` deletes any recipient not present -- so
every recipient hidden on the read path has to be restored on the write
path. Master already did this for one read filter (sandboxed and
impersonated users, via `maybe-add-recipients`), but the tenant filter
added in this slice had no corresponding write-path compensation. A
tenant-scoped caller doing GET then PUT of the returned body therefore
dropped every other-tenant and tenantless recipient row.
Rather than add a second parallel mechanism, the tenant visibility rule
is now a single predicate (`recipient-visible-to-tenant-caller?`) used by
both sides: the read path filters with it, and the write path merges back
exactly what it excluded via `hidden-cross-tenant-recipients`.
Two supporting fixes:
- `maybe-add-recipients` now normalizes `:channel_type` the way
`email-channel` does. The old `(= "email" ...)` string comparison only
matched the REST shape, so a direct call with a hydrated pulse (keyword
`:email`) skipped the merge entirely -- which also made the pre-existing
sandbox protection REST-only.
- Merged recipients are de-duplicated by `(some-fn :id :email)`, the same
key `update-pulse-with-perm-checks!` uses to diff recipients, so a
caller matching both filters cannot get a recipient appended twice.
Verified in a REPL: the new endpoint-level round-trip test goes red
without the merge-back (two recipient rows destroyed) and green with it.
It asserts on `pulse_channel_recipient` rows rather than the API response,
since the response is filtered either way and cannot distinguish "hidden"
from "deleted". Pulse, pulse-channel and enterprise sandbox pulse suites
show no new failures (`create-csv-xls-test` fails identically on the
unmodified branch).
---------
Co-authored-by: Eric Normand <ericwnormand@gmail.com>
In https://github.com/metabase/metabase/pull/80957 we trimmed this job
down so it no longer doubles the tests run. But it's still pointless
to have it as a separate job; it should be tested inline with the main
suite.
We still run some individual tests twice using the
`with-and-without-multi-level` macro when we need to cover both the
multi-catalog behavior and the non-multi-catalog behavior, but this is
only a handful of tests now instead of the entire suite or the entire
namespace.
Removal of `when-not (tx/db-test-env-var :databricks :multi-level-schema false)`
causes a lot of indentation changes here, so it's best to review with
whitespace diffs turned off.
* Split the SSO auth settings pages into card sections
* Deflake modal escape close in data-model e2e specs
* Move docs changes to a separate PR
* Finish token-scale translation in the merged css modules
* Deflake dark mode toggle assertions in palette spec
* release JDBC savepoints when nested transactions succeed
* tighten inline comment
* pr feedback
* Release savepoints only when no later scope is still running
* Fix sqlglot parse failure for ?::type placeholder casts (#81373)
With connection impersonation enabled, native queries are parsed by
sqlglot after template-variable substitution, when {{var}} has already
become a JDBC `?` placeholder. sqlglot's base tokenizer treats `?::` as
a single token (DuckDB's try-cast operator) in every dialect, and only
DuckDB's parser can consume it, so `{{var}}::date` -- a very common
pattern on Postgres/Redshift/Snowflake -- failed with "Unable to parse
native query".
Fix: before parsing in is_single_stmt_of_type, rewrite `?::` to `? ::`
(which sqlglot parses fine, emitting CAST(? AS type)). Occurrences are
located via the tokenizer's own QDCOLON tokens, so `?::` inside string
literals, dollar-quoted strings, and comments is left untouched, and
placeholder order is preserved in the re-emitted SQL. Multi-statement
rejection and the rest of the impersonation validation are unchanged.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XkBejATyUcaq2R74xFd5M1
* comments and tests
* Assert all placeholders survive; fix QDCOLON dialect guard (databricks, not duckdb)
Review feedback: the placeholder-cast test now asserts that every ? in the
input survives into the re-emitted SQL, not just that one does.
While adding duckdb coverage, it turned out the dialect guard in
_split_placeholder_casts named the wrong dialect: in sqlglot 28.6.0 the
?:: (QDCOLON) token is emitted by the *base* tokenizer but only
Databricks' parser consumes it (the expr?::type try-cast column
operator); duckdb's parser does not, so duckdb was still failing on
placeholder casts. The guard now skips databricks instead, duckdb gets
the same ?:: -> ? :: rewrite as everyone else, and tests cover both:
duckdb placeholder casts parse with placeholders preserved, and
databricks' native try-cast is left intact (re-emitted as TRY_CAST).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* clean up
---------
Co-authored-by: Claude <noreply@anthropic.com>
* Make ratchet shrinking explicit, and check it in one place
The ratchet test fixture ran `fix!` on every non-CI run, so `project-tests
ratchets` rewrote .clj-kondo/ratchets.edn in your working tree before
asserting against it. A check that edits what it checks leaves a diff nobody
asked for, in the file every PR conflicts over, and its only reachable
failure was a bug in the fixer that had just run.
Removing the auto-fix forces the dev/CI split to go too: a strict local test
that no longer tightens would fail on every branch that removes an ignore.
So over-budget is now the rule everywhere, and shrinking is only
`./bin/mage kondo-ratchets-shrink`, which the shrink workflow runs on master.
That left two implementations of nearly the same check. The babashka
`check-report` gains the justification finding — the one assertion the JVM
suite had that it lacked — so `./bin/mage kondo-ratchets` covers over-budget,
config over-budget, unjustified ignores, normalization and unknown linters on
its own. CI runs it instead of booting a JVM, and the five now-duplicated
in-repo assertions leave the test namespace, which drops from 6.2s to 0.8s
because it no longer walks the source tree.
Task names follow: check-kondo-ratchets (which nothing called) becomes
kondo-ratchets, fix-kondo-ratchets becomes kondo-ratchets-shrink, and the
merge task becomes private -kondo-ratchets-merge, since only the git merge
driver invokes it. bin/merge-kondo-ratchets keeps its name; installed merge
drivers point at it.
* Polish kondo ratchet guidance
* Prime the ratchet shrink workflow
Intentionally raise the :case-symbol-test budget from 1 to 2 without adding a suppression. The read-only check allows the unused budget; after this PR merges, the shrink workflow should restore it to 1 through a Tighten ratchets automation PR.
* Make the ratchet suite check the project
Keep the ratchet tooling unit tests in the backend suite, but make project-tests ratchets run the read-only repository check. Include it in the bare project-tests command so local project-level checks still validate the checked-in ratchet policy.
* List project-test suites once
* Address review findings on the ratchet check
check-report threw NPE on a ratchets map without :comment-exempt, while
tolerating a missing :ignore-counts or :config-counts. Default it, so the
three policy keys behave alike.
The run-project-ratchet-checks description and the step comment both still
described the old wiring. The check now runs alongside the backend checks,
which no longer include it, and "no JVM" is not quite true: resolving
clj-kondo on a cold cache shells out to the JDK prepare-backend installs.
* Warn about stale comment exemptions
The only check that a :comment-exempt entry is still needed was a local-only
test, which went vacuous when the dev and CI semantics were unified, so
nothing named a stale exemption any more. The check now reports one the way
it reports an :unlimited policy nobody uses: informational, never a failure,
since only the shrink workflow removes them.
* Say why the merge driver finds mage through its own path
The script runs against whatever repository holds the conflict, which is not
always the one it lives in -- the tests merge stages in a throwaway repo. A
relative bin/mage looks plausible and fails there, so record the reason.
* Stop wrapping two comments that did not need it
One fits on a single line once it says less. The other breaks at its sentence
boundary rather than mid-clause.
* Break the ratchets header on its sentences
Four sentences started mid-line and wrapped mid-clause, one of them trailing
"in the PR." onto a line of its own. Same words, one sentence per line.
* Keep comment exemptions until someone removes them
fix! dropped a :comment-exempt entry as soon as every ignore of its linter
carried a comment, or the last uncommented one was deleted -- so an exemption
could disappear as a side effect of unrelated work, and getting it back is a
change to defend in review.
An exemption records a decision about a linter, not a count, which is the
reason :unlimited policies are already kept at zero and only warned about.
Treat the two alike: the check and the shrinker name a stale exemption, and a
person deletes it.
* GDGT-2486 Add elevation, radius, and spacing scale variables
* GDGT-2486 Remap scale tokens in files merged from master
* Address review
Review fixes for the GDGT-2486 scale migration:
- Revert double-shifted tokens from the merge-remap pass: it re-applied
the old-to-new mapping to 22 files already remapped by the first pass,
shifting spacing values one extra step (e.g. xl 24px -> xxl 32px).
- Fix codemod escapes where old key names kept their name but changed px:
Modal default padding (lg->xl, 24px), MultiSelect radius (sm->xs, 6px),
MetricSearchInput radius-sm, ThemeEditor PreviewPanel radius-md,
ColumnInput theme.spacing.xs->xxs, ChartTypeSettings Space h props,
TableRowForCacheableItem icon placeholder w (md->lg, 16px),
ResizeHandle/ResizableBoxHandle grip h (xs->xxs, 4px).
- Pin Group/Stack default gap to "lg" (16px): Mantine's default is
"md", which is 12px in the new scale (was 16px in the old scale).
- Allowlist the runtime-defined --mantine-spacing-xxs/-xxl and
--mantine-radius-xxs CSS variables in find-never-defined-css-variables
(fixes fe-lint CI).
- Add theme.unit.spec.ts covering the spacing/radius/shadow scales,
dark-mode shadow set, whitelabel independence, and a guard asserting
every scale-based component default resolves to a defined key.
- Derive Card/Paper story argTypes options from the real scale.
- Augment MantineThemeSizesOverride so the custom scale tokens appear in
editor autocomplete for p/m/gap/radius/shadow props.
- Update the emotion-migrate and metabase-ui-component-from-figma skill
token tables to the new scale (spacing/radius values, shadow tokens,
snap tables).
* Update Loki Snapshots
* Address review: fix h/w style props missed by the scale codemod
The codemod did not remap height/width style props, so every bare h=/w=
scale-key value silently changed px with the new scale. Remap each to the
px-preserving key (xs 4px->xxs, md 16px->lg, lg 24px->xl, xl 32px->xxl):
- PaneHeader breadcrumb row h xl->xxl (32px, restores data studio
pane header height)
- BrowseDatabases logo boxes h/w xl->xxl (32px)
- DatabaseConnectionStringField rows h lg->xl (24px)
- DatabaseEngineList logo Center h/w lg->xl (24px)
- Metabot chat chevron flexes h/w md->lg (16px)
- MetricsFilterPill h lg->xl (24px)
- DatasetNotebook divider h xs->xxs (4px)
- DataReference NodeList icon w md->lg (16px)
- transforms-python ResizableBoxHandle grip h xs->xxs (4px)
Audited: no remaining bare h/w/maw/mah/miw/mih old-key props whose px
changed (verified line-identical against master for every occurrence).
* Address review: fix bdrs and responsive-object size tokens missed by the codemod
Two more escape classes from the scale codemod, found by a full sweep of
every touched-token form (string style props, responsive objects, CSS
vars, theme accessors) with each line compared against master:
- bdrs style prop (border-radius): remapped px-preservingly at 69 sites
in 58 files (xs 4px->xxs, sm 6px->xs, md 8px->sm). The codemod remapped
radius= but not its style-prop alias bdrs.
- Responsive object values (e.g. p={{ base: "md", sm: "xl" }}): remapped
the token VALUES at 9 sites in 6 files (base/sm/md breakpoint keys are
untouched; only their spacing values shifted: md 16->lg, xl 32->xxl,
xs 4->xxs).
Sweep result after fixes: zero remaining line-identical old-key token
references to spacing xs/md/lg/xl, radius xs/sm/md, or shadow xs/md in
any form. Intentional survivors unchanged: radius/shadow/bdrs "lg"
(disclosed broken-key-now-16px class), spacing/radius sm where the
px value did not change.
Verified: tsc, eslint (changed files), metabase/ui jest suite.
* Address review: fix remaining token escape forms (object props, braces props, style objects)
Final codemod escape classes from the exhaustive token sweep, all
remapped px-preservingly:
- Object-literal / default-parameter prop values (15 sites / 8 files):
ItemList picker-item default containerProps pb xs->xxs (4px, the
reported gap), FormFooter pt lg->xl, SlackSetup pt lg->xl,
EmbeddingSecretKeyWidget mb xs->xxs, CreateDashboardForm labelProps
mb xs->xxs x2, databases styles mb lg->xl, NotebookDataPicker
menuDropdownProps mt xl->xxl, Banner story py md->lg
- Braces-wrapped string props: CloudSMTPConnectionCard p lg->xl (24px)
and gap md->lg (16px), SettingsLdapForm mt xs->xxs,
DashboardLeaveConfirmationModal mb lg->xl
- style-object CSS properties: DatasetNotebook borderRadius xs->xxs
- Ternary token values inside responsive objects: SummarizeStep gap
md->lg (16px)
Sweep now returns zero real misses across every checked form: string
style props (incl. h/w/bdrs), braces props, responsive objects with
ternaries, object-literal/default-param props, style-object CSS props,
CSS var() references, theme.* accessors. Remaining grep hits are
untouched scales (lh, fz, size, breakpoints) or px-identical sm values.
* Address review: preserve Mantine spacing defaults exposed by Loki
* Update Loki Snapshots
* Address review: fix remaining scale regressions and visual guards
* Address review: make Loki update capture new references
* Update Loki Snapshots
* Address review: keep only targeted Loki references
* Address review: assign ticket component elevations
* Update Loki Snapshots
* Address review: tighten MonthPicker header spacing
* Address review: refine elevation visual stories
* Update Loki Snapshots
* Address review: keep renamed Menu Loki story selected
* Update Loki Snapshots
* Address review: preserve inherited Mantine defaults
* Address review: allow stock Mantine shadow tokens
* Address review: replace token audit test with ESLint rule
* Address review: drop unrelated whitespace-only dev/src changes
* Address review: use rem() in radius scale and rem docs
* Update Loki Snapshots
* GDGT-2486: Address PR comments
* GDGT-2486: Adjust negative tokens
* GDGT-2486: Address review findings
* Update Loki Snapshots
---------
Co-authored-by: Metabase Automation <github-automation@metabase.com>
* Move the entities store into the metadata module and hide normalizr
* Record the moved schema file's import-time effect
* Describe what the metadata reducers do, not what they replaced
* Move the mirror's fixtures and slice-name helpers into the module (#81297)
* Move the mirror's fixtures and slice-name helpers into the module
* Build the QuestionSources fixture through the metadata store barrel
* Document why the metadata store is not a deletable mirror slice
* Trim the barrel and write-door comments
* only derive site-url from authenticated superuser requests
`maybe-set-site-url*` (global Ring middleware) derived `site-url` from request
headers whenever `(or (not (setup/has-user-setup)) superuser?)`. The left arm
needed no authenticated caller, so an unauthenticated request reaching the
instance before its first user was created could pin `site-url` from a forged
Host/Origin/X-Forwarded-Host header. Because the write is first-writer-wins and
setup never overwrites it, that attacker-chosen value survived setup and
poisoned every later auth/email link (notably password-reset URLs) for the life
of the instance.
Gate derivation on `superuser?` alone. `:is-superuser?` is already on the request
by the time this middleware runs (wrap-current-user-info is inner/earlier in the
onion), so an admin's own request still auto-derives `site-url` -- which happens
on the operator's next request after an interactive setup. An unauthenticated
pre-setup request now writes nothing; `site-url` simply stays unset (nil is
handled everywhere it's consumed, and a reset link built on nil is relative,
never attacker-controlled).
Deployments that provision headlessly, run multi-tenant, or never sign in as an
admin will otherwise sit at a nil site-url, so note in the setting's docs that
they should set MB_SITE_URL explicitly.
* Read Quartz job-data blobs through a class allow-list
Quartz persists job data, calendars, and BLOB triggers as serialized objects.
Metabase's job data is only ever EDN-shaped data (keywords, strings, numbers,
maps/sets/vectors) wrapped in Quartz's own JobDataMap/trigger classes, so when we
read it back we constrain deserialization to exactly those classes and decline
anything else.
Install a per-DB DriverDelegate (StdJDBCDelegate for H2/MySQL, PostgreSQLDelegate
for Postgres) that overrides getObjectFromBlob to read through a class allow-list
built from the plain-data classes job data consists of. This is scoped to
Quartz's own blob reads, so nothing else in the JVM is affected.
- metabase.util.deserialization-allowlist: a reusable, composable ObjectInputFilter
allow-list. clojure-data-prefixes covers the EDN value space; callers add their
own wrapper prefixes and read via read-object / read-object-bytes.
- metabase.task.secure-delegate: the allow-set (EDN + org.quartz.*), the blob-read
bodies, and install!, wired into task.bootstrap. The gen-class delegates are
AOT-compiled in the jar; from source they're compiled on demand so the class is
loadable by name through the classloader Quartz uses. That runtime compile is
fenced to source only (config/jar?): from a jar a missing class is a build error.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BjQcjQx7NPGr3Rpf6QaG4b
* Don't deserialize Java objects from H2 query results
Metabase reads scalar and collection values out of query results, never
serialized Java objects. Register a JavaObjectSerializer on the H2 driver so H2's
JAVA_OBJECT (de)serialization is declined rather than reconstructing arbitrary
classes from stored bytes. With that in place the ad hoc JAVA_OBJECT check in
read-column-thunk is redundant, so it's removed; read-column-thunk keeps its CLOB
handling. Ordinary values are unaffected.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BjQcjQx7NPGr3Rpf6QaG4b
* Strip internal keys from binning maps
Add :decode/api and :encode/serialize remove-internal-keys hooks to the
::binning schema so clients can't smuggle internal query-processor keys in
through a :binning map.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* Strip internal keys from more client-submitted query maps
Extend the :decode/api / :encode/serialize remove-internal-keys hooks to the
::parameter, ::template-tag, ::constraints, ::middleware-options, and ::page
schemas so clients can't smuggle internal query-processor keys in through them.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* Reformat strip-internal-keys-test doseq bindings
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* test improvements and fix reflection warnings
* Validate HoneySQL idents at the app-DB compile step
Honey SQL renders an ident as bare, unquoted SQL in several positions --
a form head it doesn't recognize, an ORDER BY direction, a DDL modifier,
the function name of a `:%fn.arg` var -- so an ident reaching any of them
is raw SQL text. Hold every ident to a plain-identifier shape, with an
explicit allow-list for the keywords Honey SQL renders as fixed SQL of its
own (operators, DDL modifiers, column types, ORDER BY directions).
Also close two gaps found along the way: symbols walked past the `:raw` and
`:inline` gates, since Honey SQL resolves symbol heads and clause keys the
same as keywords; and `safe-value?` trusted any value shape it didn't
recognize, where it now enumerates the types bound as `?` parameters.
Query idents that were kebab-case are renamed in the SQL only -- the
Clojure keys they feed, including the `:metabase-user-id` request key, are
mapped back at the read site and unchanged for their consumers.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013rQMWmi2o1WGSqK5DDZ99u
* Allow the Postgres operators our drivers register globally
`metabase.driver.postgres` registers the PG JSON, array and regex operators
with `honey.sql/register-op!`, which makes them global -- and the app DB can
be Postgres, so app-DB queries reach for them too: the semantic-search
migration compiles a `[:->> ...]`. Add them, and a test that holds the
allow-list to every non-namespaced operator Honey SQL knows, so an upgrade
or a new registration can't silently leave one out.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013rQMWmi2o1WGSqK5DDZ99u
* Snake_case the remaining app-DB query idents
Four more sites the sweeps turned up, all renamed in the SQL only:
- `request/session.clj` selects the current user under the same kebab
aliases the session middleware did, so it gets the same rename-at-the-read
-site treatment.
- Search scorer names double as SQL aliases, and three of them are
kebab-case. They also key the weights in `search.config` and name the
scores the API returns, so the conversion happens at the one seam that
writes and reads the alias, leaving the scorer name alone. The alias gains
a `_score` suffix because `:view-count` and `:data-layer` are named after
the very columns they read.
- The model alias registry in `api/common.clj` feeds
`column-with-model-alias`, which assembles `alias.column` idents, so its
six kebab aliases are snake_cased along with the literal uses in the
in-place search engine.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013rQMWmi2o1WGSqK5DDZ99u
* Constrain embedded card parameter values by the locked parameters
A dashboard's parameter value lookups receive the values of the other
parameters as chain-filter constraints, so the values offered for an
enabled parameter are limited to the rows the locked ones select. The card
lookups had no constraint argument to receive them: `card-param-values`
bound the JWT's params, used them only to check the searched parameter was
enabled and not pinned, then resolved the parameter to a Field and returned
that Field's whole value list.
Give the card lookups the same constraint argument and pass the JWT's
locked values from the embed layer. The argument is optional, so the
authenticated and public routes keep their current behaviour.
`param-type->op` and `param-type->default-options` move from
`parameters.dashboard` to `parameters.params`, so both the dashboard and
card constraint builders can reach them.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013rQMWmi2o1WGSqK5DDZ99u
* Snake_case the app-DB query idents the guard rejects
CI caught `queries.models.query`, and a grep for the same shape found four
more sites the earlier sweeps had missed -- they were confined to the
app_db, collections, search and permissions modules. The aliases move to
snake_case in the SQL; where the keys they feed are confined to their own
namespace the readers move with them, and where they have to match a shape
from elsewhere they are renamed back at the read site:
- `queries.models.query` aliases so the app-DB branch matches the keys
`lib.metadata/card` returns, so it renames rather than propagates.
- `cache.task.refresh-cache-configs` feeds its aliases through a nested
subquery whose outer `:select` names them, so both levels move, along
with the readers and the test's expectations.
Also widen `safe-parameter?`: a `query_hash` read back from H2 arrives as a
`java.sql.Blob` rather than a byte array, and the enumeration rejected it.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013rQMWmi2o1WGSqK5DDZ99u
* Remove a scratch probe test
Committed by accident in the previous commit -- a throwaway namespace with
printlns, used while checking chain-filter behaviour.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013rQMWmi2o1WGSqK5DDZ99u
* Fix the locked-parameter constraints for previews and stale slugs
Three problems with the constraints the card parameter lookups build:
The preview routes passed the sanitized Card, whose `:parameters` has the
locked ones removed, so the constraints were always empty and a preview
offered wider value lists than the embed it previews. They now pass the
stored Card and take only `:embedding_params` from the token, which is what
the published routes already do.
`embedding_params` is keyed by slug and `:parameters` by id, and nothing
keeps them in sync; a locked slug naming no parameter produced no
constraint and widened the values rather than narrowing them. It is now
refused. `param-values-merged-params` refuses the same on dashboards, where
the slug otherwise became a nil key that the constraint builder dropped.
A locked value of `[]` reached the filter builder as a value rather than as
"no value", producing a `:=` clause with one argument.
The constraints also carry the parameter's own `:options` now, as the
dashboard builder does.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013rQMWmi2o1WGSqK5DDZ99u
* Allow a qualified argument in a :%fn.arg ident
`:%lower.metabase_field/name` renders `LOWER("metabase_field"."name")` --
Honey SQL splits the ident at the first `.` and formats what follows as
quoted entities, so a `/` there separates the parts of a qualified column
rather than landing in the SQL. Only the function name, which is rendered
unquoted, has to be a bare identifier.
The pattern rejected `/` anywhere after the `%`, which took the database
autocomplete endpoint with it.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013rQMWmi2o1WGSqK5DDZ99u
* Snake_case more app-DB query idents, and allow the FOR UPDATE modifiers
Found by widening the search: the earlier grep looked for the alias and its
`:select` on one line, so it missed every alias sitting on a continuation
line -- which is most of them.
- `activity_feed`: the recent-views queries alias `database-name`,
`initial-sync-status`, `entity-coll-id` and `moderated-status`. The keys
are read only within their own namespace, which builds the API response
from them under different names, so the readers move too.
- `xrays.automagic-dashboards`: `num-fields` and `list-like?` -- the latter
loses its `?`, which no snake_case spelling keeps.
- `:skip-locked`, `:nowait` and `:no-wait` are lock modifiers Honey SQL
matches by name in a `:for` clause, not idents it renders, so they join
the allow-list rather than being rewritten.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013rQMWmi2o1WGSqK5DDZ99u
* Move the query-analyzer's readers to the renamed table id
`query-analyzer/tables-for-native` aliases its table id, and that alias was
snake_cased without moving one of its two readers: `used-tables` still asked
for `:table-id` and so silently collected nothing. Unlike the write side,
which the guard rejects outright, a reader left behind reads nil.
The `tables-for-native` redefs in the tests carried the old shape too, so
they asserted against a result the function no longer produces.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013rQMWmi2o1WGSqK5DDZ99u
* Revert the HoneySQL ident guard and the renames it required
The guard held every ident in an app-DB query to a plain-identifier shape,
which turned out to have a long tail: each release of it found more query
aliases to rename, and a rename that missed a reader failed silently rather
than loudly, since the reader simply got nil.
This reverts the guard along with every alias rename made to satisfy it,
leaving the parameter-value security fixes, which do not depend on it.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013rQMWmi2o1WGSqK5DDZ99u
* Harden site-url derivation fix: fail-closed redirect check, discerning test
Follow-ups to gating site-url auto-derivation on authenticated superuser
requests.
- check-sso-redirect: fail closed when site-url is nil. Previously a nil
site-url left our-host nil, so a redirect with a nil host but a non-nil
scheme was non-relative yet matched (= nil nil) and passed -- an open
redirect. Require a known host to match against: (and
our-host (= (.getHost redirect) our-host)).
- misc-test: bind has-user-setup false in the non-superuser negative case.
With has-user-setup true (the test-DB default) the old gate
(or (not has-user-setup) superuser?) was already false for that request,
so the test passed against the vulnerable code and distinguished nothing.
- misc: rewrite the SETTING SITE-URL block comment and the maybe-set-site-url
docstring, which still described the removed behavior (site-url set on the
initial setup request).
- system settings: drop the blank line in the site-url :doc (it made the new
paragraph render as an indented code block in the generated env-var docs)
and replace an em-dash with a comma.
* Read a card's parameters the way the rest of the code does
`card-param-constraints` read `:parameters` alone, but a native card can be
described entirely by its template tags and carry none -- which is how the
static embedding e2e fixture builds one. The locked slug then matched no
parameter and the lookup was refused. Use the same either/or source
`get-param-or-throw` uses a few lines above.
A parameter with no value constrains nothing whether or not it still
exists, so the existence check now runs after the empty-value guard rather
than before it.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013rQMWmi2o1WGSqK5DDZ99u
* Fix jar startup crash when the jar's directory contains a space
find-in-current-jar built a jar: URL by string-concatenating
get-jar-path, but get-jar-path returns the *decoded* filesystem path
(URL -> URI -> .getPath turns %20 back into a literal space). URL.toURI
then throws URISyntaxException on the raw space, killing startup while
loading bundled driver manifests.
Open the jar via the Path-based nio-fs helper instead — Paths don't
round-trip through URI encoding, so spaces are fine. This also stops
closing a filesystem out from under other consumers: the old code, on
FileSystemAlreadyExistsException, grabbed the provider's *cached*
filesystem and closed it in with-open; the Path-based overload creates
an independent uncached instance whose close affects only itself.
Verified against a jar in a spaced directory: the manifest URI is found
and slurpable, and a concurrently-open URI-cached filesystem for the
same jar stays open and registered.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013Fpja4n6bzdCy4QufM49Hb
---------
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
Co-authored-by: Alexander Polyankin <alexander.polyankin@metabase.com>
* Don't crash the modules require hook on dynamic requires
(requiring-resolve (:failing-test-var opts)) has no statically-known
namespace; unwrap-require now returns nil for such forms instead of
falling through, and both callers skip non-symbols. dev.clj has carried
a live :hook crash from this since the hook was added.
* Add a hook regression test for dynamic requiring-resolve
Covers the crash fix: dynamic arguments lint to nothing, static
cross-module violations still report. hooks.* test namespaces join the
outside-the-module-system prefix list.
* Fail closed on dynamic module requires
* Polish module require hook tests
* Assert module hook findings directly
* Document intentional dynamic module requires
* Update module linter ratchet
* Add a manual kondo ratchet conflict merger
* Split kondo ratchet merge tests
* Document the ratchet conflict merger
* Simplify unlimited ratchet merge policy
* Simplify ratchet merger for final policy schema
* Resolve ratchet conflicts to the stricter policy
Concurrent policy changes now always resolve: the smaller budget wins, a
finite budget beats :unlimited, and a removed entry beats any budget.
Comment exemptions merge on their own. Every stage is read through
read-ratchets and shape-checked before a disabled stage decides the result.
bin/merge-kondo-ratchets reads the index stages by number, so a file
deleted on one side and changed on the other exits nonzero with nothing
staged, and both-sides-add merges against an empty base. It runs from any
directory in the repository and stages only .clj-kondo/ratchets.edn.
mage.merge-kondo-ratchets-test drives the script against real conflicts
in temporary repositories.
* Point the clojure-write skill at the ratchet merge resolver
* Reject malformed stages before merging ratchets
The shape checks move out of read-ratchets into validate-policies, and merge-ratchets runs them on every stage, so a stage with :config-counts [] or :comment-exempt [] is an error rather than an empty set of policies that the merge would read as removals.
* Cover unreadable stages in the merger tests
An empty file, two forms, or a non-map stage now fails the reader instead of merging as no policies; the script leaves such a conflict for a human and stages nothing.
* Reject a non-map ratchet stage before merging
A nil stage passed every field check because dissoc, keys, and the defaults all accept nil, so a malformed stage could merge as an empty set of policies.
* Reject non-keyword linter names
The merge keys on (str linter) and renders the same way, so a string, nil or
symbol name was rewritten as a keyword, collided with one, or produced a file
that no longer read back -- and the script still exited 0 saying it had staged
the merge. Also describe the two sides by git stage: "current" and "incoming"
are inverted during a rebase.
* Run each refused merge once in the tests
leaves-unresolved? and output each ran the script, so every refusal ran it twice
and the "changed nothing" assertion covered a different run from the one whose
message was checked. Read the merged policies with the EDN reader while here.
* Compare the conflicted file, not just its status
A conflicted path reports UU, DU or UD however its contents are rewritten, so
comparing git status alone did not show that a refused run left the file alone.
* Tighten merge-tool docstrings and the ratchet conflict docs
* Compare the index stages of a refused merge too
Porcelain status reports a conflicted path as DU/UU whatever its stages hold, so it alone does not show
that a refused run left the conflict alone. The stages carry each side's blob.
* Spell out the unmerged status codes
UU/DU/UD as bare shorthand assumes the reader knows git's porcelain codes for a conflicted path.
* Clarify the unmerged-state snapshot comment
* Add trusted master ratchet shrinker
* Retry ratchet shrink after concurrent master pushes
* Test the check-kondo-ratchets report
* Apply ratchet policy to the check report
* Add severity-based inline ratchet policies
* Update ratchet policy documentation
* Finalize ratchet policy enforcement
* Move seeded linters to bounded ratchets
* Simplify unlimited ratchet policies
* Finish ratchet CI integration
* Document unlimited ratchet policy values
* Keep ignore counts as the policy map
* Validate ratchet policy keys against known linters
Policy keys in .clj-kondo/ratchets.edn must name a clj-kondo built-in from the
pinned version, a linter configured under .clj-kondo/, or an allowlisted
external diagnostic. The check and the fixer fail with one error listing every
unknown name instead of counting it. Config budgets must be non-negative
integers.
Drop :unresolved-require, a linter that never existed, from the semantic-search
ignore and the ratchets file.
* Keep unlimited ratchet policies with no ignores left
An :unlimited entry in :ignore-counts records a decision about the linter, so
the fixer no longer drops it when its count reaches zero. The check and the
fixer print one sorted warning naming such entries; it never fails CI. Bounded
zero budgets are still dropped, and :comment-exempt is unchanged.
* Deliver ratchet shrinking through an automation PR
* Validate seeds and read only tracked kondo configs
The known-linter set now comes from git-tracked `.edn` files under
`.clj-kondo`, so the dependency configs `mage kondo` copies in locally
(gitignored, absent from a clean checkout) no longer make validation
depend on the machine. `--seed` is checked against the same set before
anything is written, and a hand-written bounded `0` with no ignores
left is reported when the fixer drops it.
* Reopen the ratchet automation PR after a close
The "already carries this result" skip now requires an open PR, so a
branch left behind by a PR closed without merging is overwritten and
gets a fresh PR instead of silencing every later run. The compare step
reads PR state with the default token under `pull-requests: read`.
* Let the ratchet automation PR approve and merge itself
Copies the backport recipe: github-actions[bot] approves the PR with the default token (a user cannot approve their own PR, and the automation user owns it), then the automation user enables squash auto-merge. The commit identity is the docs generator's, which merges with that one approval. A ratchets-only change no longer flips backend_all, so the PR waits on the Project tests job alone.
* Describe the self-merging ratchet automation PR
* Expect the ratchet check namespace in the project-test suites
* Read the ratchets file as exactly one map
An empty file, a non-map, or a trailing second form used to read as an empty policy set, which the merger would have staged as the removal of every budget. Tracked kondo configs missing from the worktree (unstaged deletions, sparse checkouts) are skipped instead of failing the scan.
* Skip the uberjar for a ratchets-only change
* Keep fork PRs away from the ratchet automation steps
The head-branch lookup matched any repository's branch of that name, so a fork PR could have reached the approval and auto-merge steps; only an open PR from this repository counts now. Approval and auto-merge key on the open PR rather than on whether the branch was pushed, so a run that failed after opening the PR is repaired by the next one.
* Look up the ratchet automation PR by owner-qualified head
gh pr list caps at 30 results and filtered forks client-side, so enough fork PRs on the same branch name could hide the real automation PR and make the create step fail. The REST head=owner:branch filter is applied server-side and matches only this repository's branch.
* Stop checkout credentials from overriding the automation push token
actions/checkout persists GITHUB_TOKEN as an Authorization header, which takes precedence over the token embedded in the push URL, so the force-push to the automation branch would be rejected.
* Tolerate config budget and exemption reductions in CI
CI already let inline counts fall below their budgets so the shrinker could record them after merge, but a lowered config budget or a stale comment exemption still needed an exact match, so two PRs that each removed a suppression could fail the second merge. Config counts now fail CI only when they grow, stale exemptions never fail CI, and check-kondo-ratchets reports config growth alongside inline growth. Local runs stay exact.
* Say what CI rejects without listing the cases
* Keep actionable ratchets bounded
* Exclude the ratchets file with one pattern
Path filters are OR-ed, so a separate "!.clj-kondo/ratchets.edn" entry matched
every path that was not that file: backend_all became true for every PR, and
no_source_code_changes could never be true. The exclusion has to live inside the
pattern, as modules/drivers does above.
* Arm auto-merge against a known commit
The approve and merge steps keyed on the PR number alone, so nothing had checked
what the fixed automation branch name currently holds. Both paths now record the
sha they pushed or compared, and auto-merge matches it. Also guard the job to
this repository, as the workflow it replaces did.
* Report ratchet validation failures without a stack dump
check read and validated the file, then read it again. Now it parses once and
catches inside, so read-ratchets, validate-linters! and scan all report the same
one-line message. Also: CI=false was truthy in the tests, seeding a recorded
linter with no ignores said "nothing to seed" while dropping its policy, and
lowered-counts rebuilt the seeded set per entry.
* Keep the drift message off stdout
The drifted path prints its findings and then exits; folding it into fail! added
a summary line that check-reports-empty-unlimited-test pins. Split the throw from
the print so only messages the run has not already shown are printed.
* Tighten ratchet docstrings and the generated header
* Guard against standalone negations in the path filters
The patterns in a filter are OR-ed, so a "!..." entry matches every path the other entries don't
instead of subtracting from them, and the filter is then always on. The ratchets exclusion is written
inside one pattern for that reason; this keeps the next one from being written the other way.
* Reuse the automation PR on its ratchet file alone
The extra check that master had not touched ratchets.edn since the branch was cut only bites when master
moved to a different value that shrinks back to the same result, since any change to the result is already
caught by comparing the file. Dropping it means only a shallow fetch of the branch tip is needed.
* Force-push the automation PR when it conflicts
A PR whose ratchet file already matches is otherwise left alone, and a conflict is the one thing that
state cannot clear on its own. Mergeability is computed lazily, so UNKNOWN is polled past and, if it
never resolves, left for the next run.
* Ask once whether the automation PR conflicts
Polling for a definitive answer bought nothing: the next push to master runs this again within the hour,
so a mergeability check still in flight can read as not conflicted and be settled then.
* Say what the reuse comparison is against
git diff FETCH_HEAD -- <path> reads as a commit-to-commit diff. It is against the working tree, which is
the only reason comparing the branch tip answers whether the PR still delivers the right file.
* Fix the circular Filter type alias
Record<string, Filter> is expanded eagerly, so naming it inside Filter's own definition trips TS2456. An
inline index signature is deferred and holds the same shape.
* Prove a ratchets-only change skips the backend filters
* Run the path-filter specs when the filters change
* Use gh CLI to approve ratchet PRs
* Reuse toucan2's current connection in Quartz to prevent deadlocks
When code inside a with-transaction or with-connection block triggers a
Quartz operation, Quartz would check out a new connection from the pool.
If the pool is saturated, this deadlocks the thread. Now the
ConnectionProvider checks for a bound toucan2 *current-connectable* and
reuses it via a non-closeable proxy that suppresses close/commit/rollback/
setAutoCommit, preventing interference with the outer transaction.
Closes UXW-307
* Address review comments
verify that asking for a connection while one is already bound via
t2.conn/*current-connectable* reuses the identical underlying connection
instead of checking out a new one. Plus fix an `if` that should be a
`when`.
* Revert Quartz connection-reuse proxy
Revert the non-closeable connection proxy that let Quartz reuse the
thread's bound toucan2 connection with close/commit/rollback/setAutoCommit
suppressed. Suppressing those methods breaks Quartz's data integrity:
JobStoreTX only releases its SELECT ... FOR UPDATE row locks on QRTZ_LOCKS
via commit/rollback, so the proxy either held the cluster lock for the
length of the outer transaction or, for autoCommit connections, defeated
the cluster mutex entirely, and made partial job-store writes
un-rollbackable.
The deadlock this was fixing (UXW-307) is addressed in the next commit by
giving Quartz its own connection pool instead.
* Give Quartz a dedicated app-db connection pool to prevent deadlocks
When code inside a with-transaction or with-connection block triggers a
Quartz operation, Quartz checks out a connection from the app-db pool. If
the pool is saturated, this deadlocks the thread (UXW-307). Fix it at the
root: Quartz now draws from its own small c3p0 pool, so job-store
operations never compete with application code for main-pool connections,
and Quartz keeps full control of its connections' transaction lifecycle.
- ApplicationDB now carries a quartz-data-source: a separate pool (max 5,
min 1, configurable via MB_QUARTZ_MAX_CONNECTION_POOL_SIZE) when
create-pool? is true, the raw data source otherwise. It inherits all the
main pool's properties except sizing.
- mdb.connection/quartz-data-source exposes it behind the ApplicationDB
read-write lock, re-exported via metabase.app-db.core.
- The Quartz ConnectionProvider draws from it; the tracing
connection-interceptor hook is unchanged.
- Tests cover provider behavior (no reuse of the bound connection, not even
wrapped), pool construction/sizing, lock-respecting acquisition, and a
deterministic pool-of-1 deadlock regression test.
* Reject already-pooled data-sources for the dedicated Quartz pool
connection-pool-data-source returns an already-pooled data-source
as-is, so building the Quartz pool from one would silently hand Quartz
the main pool itself -- reintroducing the starvation deadlock the
dedicated pool exists to prevent. No current caller does this (the only
production :create-pool? true site uses the always-unpooled env-derived
data-source), but nothing stopped a future caller from doing it.
Fail fast instead: quartz-connection-pool-data-source now throws on
pooled input, and application-db documents that :create-pool? true
requires an unpooled data-source.
---------
Co-authored-by: John Swanson <john.swanson@metabase.com>
Refresh the settings cache before reading LLM provider connections
Provider connections all live in the one llm-providers setting, and settings
are cached in process, so an instance only learns about another instance's
edit when its cache next polls — once a minute.
Until then it answers a page load with a list that predates the edit, and it
decides whether an edit is allowed against that same list: a connection
another instance has just deleted still looks present, and the type it
belonged to still reports itself as connected, so a legitimate create is
refused.
Force the staleness check on the five endpoints that read the list. It costs
the one query that reads settings-last-updated, unless something has actually
changed.
This does not make the read-modify-write of the list atomic: two instances
editing different connections at the same moment still lose one of the two
edits. That needs a lock, and is left for its own change.
With a driver bound, [:inline "!"] compiles through driver-specific
inline-value; MySQL emits the hex literal _utf8mb4 X'21', whose implicit
collation (utf8mb4's default, utf8mb4_0900_ai_ci) clashes with the
connection collation when MySQL aggregates collations for LIKE, failing
with 'Illegal mix of collations'. ::h2x/literal compiles through its own
registered formatter to a plain '!' on every dialect, which takes the
connection collation like the pattern parameter next to it.
* Don't lose the last change to a setting when a save fails
* Drop a duplicated test title
* Drop the no-op catch on the write chain
* Replace the write queue with a pessimistic disabled state
* Extend the failed-save revert to select and radio inputs
* Enforce single-statement impersonated queries on all sql-jdbc drivers (SEC-1189)
Connection impersonation confines a non-admin's native queries to a restricted
DB role, which Metabase activates with a `SET ROLE` prefix. To stop a user from
appending `; SET ROLE <broader>` and climbing out of that role, impersonated
native queries must be a single statement. That guard
(`driver/validate-impersonated-query`) was only overridden for `:postgres` and
`:sqlserver`; every other impersonation-capable JDBC driver (MySQL, Snowflake,
ClickHouse, Starburst) inherited the no-op `:default` pass-through and ran the
stacked query verbatim. On MySQL a stacked `; SET ROLE ALL` activates every role
granted to the connection user, escaping the restricted role entirely.
Define `validate-impersonated-query` once at the `:sql-jdbc` floor so every
impersonation-supporting driver enforces the single-statement validation, and
remove the now-redundant per-driver `:postgres`/`:sqlserver` overrides.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XM9HwBRFsa3ptVAP7UWV14
* clean up
---------
Co-authored-by: Claude <noreply@anthropic.com>
* improve card type usage
* fix a few more Card type casts
* use same gate for visualize another way button as in getVisualizerInitialState so button doesn't silently do nothing
* Escape LIKE metacharacters in user-supplied search patterns (SEC-847)
Name the LIKE escape character explicitly so the escaping also holds on app DBs without a default one
Add h2x/like-substring and h2x/like-prefix and use them at every search sink
* Update in-place search filter tests for escaped LIKE patterns
Grant native create-queries explicitly in transform API tests
The transform API tests grant :perms/transforms explicitly but rely on the ambient db-level create-queries default (:query-builder-and-native) for the shared dataset DB.
That default is fragile: the native level only exists db-level, so any table-level create-queries row written for the group granularizes it down to :query-builder, and re-coalescing can never bring the native level back.
A target table synced by an async transform run during another test's snapshot-restore window leaves exactly such a row, and full-database-permission-for-user takes the most restrictive value across the DB.
When that happens, every test creating or running a native-query transform as a non-superuser 403s for the rest of the run while the MBQL-source tests keep passing (seen on the SQL Server 2022 driver job).
Replace the bare transforms grant with with-transform-db-perms!, which also grants db-level create-queries :query-builder-and-native.
A db-level set deletes every row of that perm type - leaked table rows included - so the grant repairs any polluted state for the test and the scoped restore puts the prior rows back afterwards.
Admins can turn on fast mode for Metabot when the selected Anthropic model offers it. A rejected fast request falls back to standard speed and holds it for five minutes, so the speed and the cache prefix stay put.
4.2.1 is published for eight more of the Hive pins in the hive-like driver,
so move them along with hive-common rather than leaving the family split
across two patch versions.
hive-jdbc, hive-service and hive-hplsql stay at 4.2.0 -- no 4.2.1 was
published for those three.
* Add a lint check requiring job-level timeout-minutes in workflows
A job without `timeout-minutes` inherits GitHub's 6-hour default, so a hung
job burns six hours of runner time - and holds up everything queued behind it -
before anyone notices.
actionlint cannot enforce this: its rule set is compiled into the binary, and
`.github/actionlint.yaml` only accepts `self-hosted-runner`, `config-variables`,
and `paths`, the last of which suppresses findings rather than adding them. So
the rule lives in a small yq-based script instead.
It runs in the lint job next to actionlint, gated on the same workflow-file
filter and passed the same changed-file list, and emits `::error file=,line=::`
annotations. Jobs that call a reusable workflow are skipped, since GitHub
rejects `timeout-minutes` there.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
* Fold the actionlint path filter into the workflows filter
The `actionlint` filter matched exactly the same paths as `workflows`, and the
lint job was the only consumer of it. Drop the duplicate and point the actionlint
steps at `workflows` / `workflows_files`.
Also give lint.yml's own snowplow job a 10 minute timeout, so the workflow
passes the check it adds.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
* Stop yq shifting reported line numbers on files with header comments
yq slurps a leading comment block before parsing unless
--header-preprocess=false, so every line number it reported was short by the
size of that block. Five workflows start with a comment; on cache-warm.yml the
`warm` job was annotated at line 18 instead of 42.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
* Update .github/file-paths.yaml
Co-authored-by: Ryan Laurie <30528226+iethree@users.noreply.github.com>
---------
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
Co-authored-by: Ryan Laurie <30528226+iethree@users.noreply.github.com>
`parse-approved-origins` runs on essentially every API response (via
`approved-origin?` in the security-headers middleware, and again in the MCP
origin check) and re-parses the whole CORS-origins allowlist each time. Any
entry that fails to parse -- e.g. a malformed `http://localhost:3000:*` saved
into the SDK/MCP origins setting -- hits `log/errorf "Invalid URL"` on every
request, producing bursts of identical errors around each dashboard load.
The allowlist strings are slow-moving globals, so LRU-cache the parse. Callers
pass a couple of distinct strings (the merged SDK+MCP allowlist, and the
MCP-only one), so a small LRU bound keeps both cached without either evicting
the other, while remaining bounded.
Claude-Session: https://claude.ai/code/session_01BjQcjQx7NPGr3Rpf6QaG4b
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
* Fail closed when an OAuth token's client is gone
resolve-access-token looked up the access token, checked expiry, and read the
user id off the token row — but never consulted oauth_client. So an access
token kept authenticating even if the client had been deleted.
Require the token's client_id to still exist in oauth_client, using the
client_id already present in the fetched token data — one indexed lookup on the
bearer path, no schema change. A missing client now fails closed (401).
Tests:
- core-test: token resolves while its client exists, returns nil once the
client row is deleted.
- session_oauth_test: register the referenced oauth_client so the bearer-bridge
tests exercise the new client-existence check (previously they minted tokens
against a client_id with no row).
* Revoke sessions and OAuth tokens on user deactivation
Add an `:event/user-credentials-revoked` event, emitted in the user
`before-update` hook when a user is deactivated, with two subscribers:
- session: delete the user's core_session rows
- oauth-server: mark the user's access and refresh tokens revoked_at
* Pin that an API key's lifecycle follows its synthetic user
An API key authenticates as its own synthetic user (api_key.user_id); the
creator_id column that records the admin who minted it is attribution only and
is never consulted at auth time. So deactivating that admin does not revoke the
key — which is the intended design: keys are org-level service credentials,
visible to and revocable by any admin from the API-key page, independent of who
created them. Revoke a key by deactivating its synthetic user
(DELETE /api/api-key/:id), not the human who minted it.
No behavior change. Add a regression test that pins this: a key survives its
creator's deactivation (200) but is revoked when its own synthetic user is
deactivated (401), so the two paths cannot be conflated by a future creator_id
join. The DELETE-endpoint revocation path is already covered by api-keys-work-e2e.
* Register the credentials-revoked topic in the events hierarchy
The events system validates topics against its own hierarchy in
metabase.events.hierarchy, not Clojure's global one, so the plain
derive calls left :event/user-credentials-revoked unregistered and
publish-event! threw a 500 whenever a user (or API key's synthetic
user) was deactivated. Use events/derive! like every other subscriber.
* filter lts versions when finding latest from tags
* always skip lts tag since it isn't valid for finding latest versions
* add guard for valid version to release-log
* add tests
* fix actionlint
* clean up/fix logic in release-log
* Keep returning the major version for rolling .x tags
getMajorVersionFromRef now recognizes v0.56.x / v0.51.2.x style tags in
addition to valid version strings, so only lts and non-version tags get
an empty string back.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
* remove dimension anchored blocks from FE
* Remove dimension-anchored blocks from the backend
Research-plan groups are now always a metric sliced by dimensions; the
dimension-anchored shape (one dimension slicing several metrics) is gone:
- add_research_groups: drop anchor/dimension_id/metric_ids, require metric_id
- remove_from_research_plan: members lose metric_ids; dimension_ids required
- research-groups validation: metric-only, anchor dispatch removed
- blocks-tree: no :type on nodes; "By <dimension>" headings and the
same-name disambiguation machinery deleted
- BlockSelection/ExplorationBlockNode schemas lose :type; the
exploration_block.type column is no longer written (left in place,
nullable)
- explorations.selmer prompt rewritten for the single group shape
Legacy dimension-anchored rows keep rendering (as metric-headed blocks);
pinned by the new blocks_test regression test.
* more FE cleanup, fix tests
* Drop exploration_block.type column
---------
Co-authored-by: John Swanson <john.swanson@metabase.com>
Co-authored-by: Nathan Voxland <nathan@metabase.com>
* Replace @cypress/skip-test with a local e2e helper
* Document --spec requirement in the e2e-test skill
* Tidy skip-test helper comments and skill doc wording
* Wait for the query to reach the reduce step before cancelling in cancel-test
cancel-test gave the query a fixed 100ms head start before future-cancel and
assumed it would be sleeping inside *reduce* by then. Only an interrupt that
lands inside *run*'s try posts ::cancel to the canceled channel, so whenever
pre-pipeline setup took longer than 100ms the future died silently during
setup and the test timed out waiting for the cancel message.
That made the test flaky under CI load and deterministically red when run in
isolation: on a fresh JVM the first query's setup takes seconds, so the
granular-rerun step (which re-runs only the failed test) could never pass it,
pinning any PR that flaked onto it. Synchronize on a promise delivered from
*reduce* instead of sleeping.
* Block on a latch instead of sleeping in *reduce*
The 1000ms sleep was a second timing assumption: the cancel had to arrive
within it or the reduce would complete and save the execution. Await a latch
nobody opens instead — the interrupt wakes it just like the sleep, with no
window to miss. The 30s timeout only bounds the worker thread if cancellation
is genuinely broken.
* Use a never-delivered promise and drop the dead *result* call
A promise is the same CountDownLatch under the hood (interrupt propagates
identically from deref) and matches the started promise next to it. The
*result* call after the block was unreachable on the happy path — the
interrupt throws out of the deref — and pointless on the timeout path, where
the test failed 28 seconds earlier.
* Collapse the port/val channel assertions into one
A timeout or closed channel takes nil, which fails against ::cancel on its
own — no need to name the winning port. Also fixes the doubled word in the
testing string.
* fix cancellation test cleanup race
* Clarify cancellation test scope
* Strengthen cancellation metadata assertion
* Polish cancellation test formatting
Clean up stale, contradictory, and unreachable guidance in Metabot prompts and skills
- Remove references to tools/params that never existed (`search_tables`, `search_transforms`, `get_report_details`, doc-search tool, `collection_id`/`database_id` on search, `update_dashboard` mutations, `is_library_member`, dead `source_tables` param, duplicate Slackbot tool)
- Align per-profile guidance with actual tool wiring: SQL-only profile has no notebook/metric/chart tools, Slackbot has no `read_resource`, `construct_notebook_query` schema differs by profile, SQL query-link behavior differs by tool
- Fix `read_resource`: pagination bug (server now returns a ready-to-use `next-page-uri` instead of broken `?page=N` guidance), curation-signal comparison, container/document listing
- Fix a Selmer `{% safe %}` vs `{% verbatim %}` bug that silently blanked all `{{#model_id}}` examples in the SQL profile prompt
- Document previously-missing required arguments across several tools (`create/edit/replace_sql_query`, `edit_chart`, time-clause `unit`, etc.)
- Fix `document_schema_collect`'s unreachable stop instruction and missing `database_id`
- Fix `list_available_fields` fallback for saved questions
- Add regression tests locking in the template-rendering and pagination fixes
* key guest embed tokens per instance instead of one shared slot
Concurrent guest StaticQuestion/StaticDashboard embeds under one
MetabaseProvider wrote into a single Redux rawToken slot, so the
last-mounted instance's token won for all of them. Key by a stable
per-mount instance id instead.
* refactor: name the guest token key mountId
instanceId read as the Metabase instance, and collided with
EnsureSingleInstance's own instanceId. Also simplify the two reducer
cases the fix rewrites, which Immer lets us assign into directly, and
drop a !mountId guard that currentToken already covers.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CkZ5rUUDeeWN9GU2wuEUHk
* test: drop the multi-dashboard guest token spec
It passed with the whole fix reverted to master: asserting each endpoint
was called at least once is already true there, because every mount's
first render uses its own token prop and only the re-render collides. The
name also implied multiple StaticDashboards work, which they never have —
they share one global state.dashboard slot.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CkZ5rUUDeeWN9GU2wuEUHk
* test: cover a guest refresh that switches resource
Every existing refresh case reissues a token for the same resource, so
none of them notice which token the component ends up using — PR #81240
flipped that precedence and CI stayed green. Switching to a question that
takes different parameters is what makes the choice observable.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CkZ5rUUDeeWN9GU2wuEUHk
* fix: drop a guest mount's token when it unmounts
The iframe embed route re-keys its children when the token changes, so a
parent-side token update remounts the component with a fresh mountId.
Nothing removed the old entry, so Object.keys(...)[0] in
getOrRefreshGuestSession kept picking the dead mount and every request
was rewritten with its stale token. Once that token expired, the refresh
wrote back to the same dead key, so the map never recovered.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016stkr2R8EEPxp5gbyqDjdR
* test: cover a guest remount with a new token
Mounts a guest question, re-keys it with a second token the way the
iframe route does, and asserts the query goes out with the new token.
Fails without the unmount cleanup: no request reaches the new token's
url at all.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016stkr2R8EEPxp5gbyqDjdR
---------
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
The per-user sandbox fingerprint (the advanced-FieldValues cache key, and
the sandbox component of data-access tokens) narrowed the hashed login
attributes to remappings whose dimension target resolved to a field of the
sandboxed table itself. But the query processor applies *every* attribute
remapping when it runs the sandboxed query, including ones targeting
columns of tables joined inside the sandbox card, or columns referenced by
name. Those remappings were silently dropped from the fingerprint, so two
users whose sandboxes differed only by such an attribute hashed to the
same field_values row and were served each other's cached values (and
could read each other's stored exploration results via the colliding
data-access token).
Fix: fingerprint every key in attribute_remappings, regardless of what
its target resolves to. This exactly mirrors the set of attributes the QP
consumes, keeps unrelated login-attribute changes from busting the cache,
and leaves hashes for own-table-only sandboxes (the common case)
unchanged. Existing colliding rows become unreachable under the new hash
and are reaped by the delete-expired-advanced-field-values sync job.
Fixes SEC-874.
* Small correctness and hardening fixes
A handful of independent, low-risk fixes bundled together — each too small to
ship on its own, each a behavior improvement that also closes a minor
hardening gap.
Reduce DB load / avoid unconditional work:
- Settings cache: a settings-last-updated cookie now triggers a cheap
single-row staleness check and reloads only if the cache is genuinely
behind, instead of unconditionally reloading the whole settings table.
Exposes restore-cache-if-needed! with a :force-check? option as the shared
"reload if actually stale" primitive the cookie path now uses.
Reduce log noise:
- CORS origin validation runs on every request's raw Origin header, so it now
parses that client-supplied value with the silent try-parse-url rather than
the variant that logs "Invalid URL" at ERROR — a malformed Origin no longer
fills the operator's error log. Parsing behavior is otherwise identical, so
no CORS allow/deny decision changes.
Correctness (case-insensitivity):
- forgot_password throttles on the lower-cased email, matching the
case-insensitive account lookup, so User@x and user@x share one bucket
rather than each getting a fresh allowance.
Add a missing throttle:
- PUT /api/user/:id/password throttles the old_password re-auth check, which
previously had none, bringing it in line with the other credential
endpoints.
* Set create queries permission explicitly for transforms tests
---------
Co-authored-by: dan sutton <dan@dpsutton.com>
* Withhold fingerprint statistics from row-restricted users in get_field_values/read_resource
Fingerprints were read from the shared Field row and refingerprinted on demand with no
restriction gating -- the same leak #436 fixed in llm/context.clj, left open here. Gate
both behind a fail-closed, per-database-batched row-restricted-table-ids check, resolving
each Field's owning table canonically rather than trusting caller-supplied metadata.
Also guard list_available_fields, viewing-context entity lookups, and typed-schemas
table/card selection against destination-database resources, and apply a missing
permission-filter-columns pass to card-details' FK targets.
* Widen LLM schema-context row-restriction handling to sandboxing and routing
row-restricted-user? only checked impersonation and failed open on a feature/license
lapse. Replace it with metabot's fail-closed row-restricted-table-ids, applied per table so
one restricted table doesn't blank out fingerprints for the whole response.
Also filter a table's own sandbox-restricted columns (previously only FK targets were),
constrain mentioned/FK-target tables to the requested database, add a database read-check,
and cap+log restricted-table field-values fetches per table.
* Cover destination-DB exclusion in typed schemas; quiet expected 404s in viewing context
Adds test coverage proving select-schema-cards/select-tables actually exclude
tables and cards backed by a destination (routed) database from typed-schema
generation -- the only guard on that path, previously unverified.
Also stops fetch-and-format logging every expected 404 (missing entity, or a
destination-db routing guard) at ERROR level; it now logs at DEBUG and still
falls back to the simple entity rendering, same as before.
* Quiet expected destination-DB 404s in get_metadata
safe-fetch was logging every fetch failure at error level, including
the routing-destination-database 404s from check-table-resource-database
/check-card-resource-database, which are an expected outcome rather
than a bug. Matches the debug-level treatment already given to the
same case in agent/user_context.clj.
* Style pass: trim design-rationale docstrings, shorten oversized fns, wrap long lines
- row-restricted-by-db: move the retry-ladder rationale out of the docstring and
onto the try/catch lines it explains.
- permission-filter-columns: extract column-table-id/field-visible?/caller-joined?/
referenced-field-visible?/column-visible? as top-level private fns instead of
closures, bringing the function back under ~20 lines.
- build-schema-context: extract drop-or-enrich-fingerprints.
- Wrap a handful of lines pushed past 120 chars by the with-cache nesting and new
test fixtures.
No behavior change; same 83 tests (315 assertions) still pass.
* Fail closed on unresolvable table IDs in row-restricted-table-ids
A table ID that couldn't be resolved to a Table row silently fell out
of the per-database restriction map and defaulted to unrestricted,
contradicting the fail-closed contract this function documents.
* Log restriction-probe failures in row-restricted-by-db
All three exception paths in row-restricted-by-db were silent, so an
impersonation/routing attribute failure left no trace of why every
fingerprint for a database fell back to the per-table restriction path.
* Handle unrecognized get_field_values data_source gracefully
mu/defn schemas aren't enforced in production, so an LLM-supplied
data_source could reach a case with no default clause and throw
uncaught, failing the whole tool call. Widen the case (and its schema)
to cover "question"/"report" like field-values already does, and give
both the permission-check case and field-values a graceful fallback
for anything else.
* Return an agent-facing 404 for a field-id with no resolvable table
check-column-table-perms! used api/check-404, which throws without
:agent-error?, so handle-agent-error rethrows it and fails the whole
tool call. This is reachable from ordinary stale result_metadata (a
dropped column, or serdes drift), not just a malicious input.
* Apply restricted-field-values cap after the eligibility filter
The per-table cap on restricted-table FieldValues fetches was applied
to raw columns before filtering to those that should have FieldValues,
so ineligible columns ahead of an eligible one in the raw list could
consume its slot in the budget and leave it without values. Filter
first, then cap -- grouping by the persisted Field's own table_id
while at it, consistent with the permission checks elsewhere in this
namespace.
* Move database read-check out of the extract-sources/generate-sql prelude
fetch-accessible-tables-with-columns required database read access
unconditionally, so /extract-sources hard-403'd even when a caller only
wanted its independent :card_ids behavior. generate-sql is the path
that needs the 403, so move the check there via build-schema-context
and leave get-tables-with-columns (extract-sources) to return an
empty result instead.
* Exclude destination-DB cards and fieldless models from answer-sources
get-metrics-and-models had no destination-database guard, so a card
backed by a destination (routed) database could still reach
list_available_data_sources -- the same gap already closed for
get_metadata. Also omit models whose :fields end up empty after
permission filtering (no view-data access to their table) instead of
listing them as data sources with nothing queryable.
* Treat API-key users consistently across user endpoints and attributes
API keys are backed by rows in core_user with type = 'api-key'. Most
/api/user endpoints already restrict themselves to personal users, but
a few didn't, and user attributes were resolved for all user types.
Align everything on the rule that only personal users are managed via
/api/user and only personal users carry attributes:
- GET /api/user/:id and PUT /api/user/:id now 404 for non-personal
users (API-key users and the internal user), matching the sibling
endpoints (password, reactivate, password-reset-url, delete, modal)
and the list endpoint's existing type = 'personal' filter. API keys
are managed via /api/api-key instead.
- EE PUT /api/mt/user/:id/attributes now 404s for non-personal users.
- user/add-attributes only resolves attributes (tenant + jwt + login)
for personal users; other types resolve to {}. Sandboxed queries run
by a non-personal user therefore report a missing user attribute
rather than reading attributes stored on the user row.
No migration: any pre-existing login_attributes on API-key rows are
left in place but no longer take effect. GET /api/user/current still
works for API-key authentication, with an unchanged response shape.
Scoped programmatic access as a first-class feature is tracked
separately in #72959.
* Provide a schema for `/api/user/current`
* Update kondo ratchet budgets
- :metabase/validate-defendpoint-has-response-schema 441 -> 440, since
GET /api/user/current now has a response schema.
- :discouraged-var 100 -> 101 for the mt/with-temp ignore in the new
sandboxing test: it creates a :model/User, which metadata providers
do not support, matching the exception documented in the linter
message and the file's existing ignores.
* CR resp - throw on missing `:type`
* Send finish-reason length for a Gemini MAX_TOKENS truncation
Translate Gemini finishReason values to AI SDK v5 finish reasons and add them to the `:usage` chunk, as the other
adapters do. `MAX_TOKENS` now reports `:finish-reason "length"` instead of an in-stream error part, so the agent loop
stops cleanly and the client can offer to continue.
* Add missing image-related stop-reasons to gemini provider
The image related stop reasons should happen in practice since we only do text generation, but add them for
completeness.
https://docs.cloud.google.com/gemini-enterprise-agent-platform/reference/rest/v1/GenerateContentResponse#FinishReason
* Add stop-reasons from the Gemini Developer API as well
https://github.com/metabase/metabase/pull/80959#discussion_r3867216474https://ai.google.dev/api/generate-content#FinishReason
* Review feedback: only add synthetic usage part for finish-reasons-without-error
* Review feedback: name the early-stop-without-error subset
* Deterministically gate remote-sync branch-switch test on its async boundaries
* Sign in before activating the token in the admin settings describe
* Wait for collection items responses before asserting synced content
* Reword test comments per writing conventions
* Trim comments and move timeouts to request/test-level options
* Remove test comments and restore waitForTask retries cap
* Fix H2 deadlock between the post-import search reindex and readers
* Additional cleanup
* Cover the in-task H2 reindex with a unit test
* Consolidate remote-sync e2e setup into a shared helper
* Support Anthropic partner models via the Google provider
Anthropic models are served by `streamRawPredict` with an Anthropic Messages body, so route them through the new
`metabase.metabot.self.google.raw-predict` namespace, which wraps the existing claude adapter. Connecting probes the
model's own `streamRawPredict` route with an empty body: Google resolves the credential, project, location and model
from the URL before Anthropic sees the body, so an Anthropic-shaped 400 validates all four without spending a tokens.
* Title-case the Gemini model display names
To match the convention for all other providers and models.
* Translate mid-stream failures on the Google provider
`google-raw` handed back the SSE reducible unwrapped, so a failure while the agent loop consumed it escaped the
adapter untranslated and surfaced as a raw 500. Wrap it in `core/reducible-with-api-errors`, like every other adapter.
* Carry the probed model in :learned-config
We already reported `:probed-model` as sibling of `:learned-config`. Nest it inside so that it gets saved with the
learned config so we can use it for subsequent connection probes for the Google provider. This tends to be a much
better guess for what model we should use for the connection test rather than just using the first model in the fixed
models list, since there is no guarantee that model is actually present in the currently configured location.
* Probe a Google connection against the model it serves
The listing path re-probed whichever model the fixed catalog names first, so a connection at a location serving Claude
but not Gemini 3.x saved fine and then reported itself broken the moment its cache entry expired. Probe the model the
connection is known to serve instead — the one Metabot is pointed at, or the `:probed-model` its last successful probe
recorded — and fall back to the catalog's first model only when it has neither.
* Start the edit form on the model the connection was probed against
Reopening a connection while Metabot points elsewhere reset the Model picker to the type default and submitted that as
the probe target, so a Google project whose location serves Anthropic partner models but not Gemini had a no-op edit
rejected against credentials that work. Fall back to the connection's own probed-model, when the catalog still offers
it, before the type default.
* Ensure the obfuscated-value? regex matches values ending in newline
Enable "dotall" mode for the obfuscated-value? regex to support obfuscated values where one of the last two chars is a
newline. For example, the contents of JSON service account key used for Google cloud auth or similar.
https://docs.oracle.com/en/java/javase/21/docs/api/java.base/java/util/regex/Pattern.html#DOTALL
* Review feedback: validate-google-model!
* Review feedback: use proposed-model in vllm list-models
* Review feedback: ensure new-value is not an obfuscated-value? in set-single-provider-setting!
* Add comment about `model->family` dispatch
Co-authored-by: Chris Truter <crisptrutski@users.noreply.github.com>
---------
Co-authored-by: Chris Truter <crisptrutski@users.noreply.github.com>
Allow embedding readable cards in documents without query-authoring perms (UXW-5037)
Saving a document clones every embedded existing card into a document-owned
card row. The clone path ran check-run-permissions-for-query on the cloned
query, which for a native query demands create-queries=query-builder-and-native
on the whole database. A user who could view (and run) an existing native
question therefore could not save a document embedding it, losing their work
with a vague 403.
That check is an authoring gate, but the embedding user is not authoring a
query: the query text comes from an existing card row they passed a read
check on -- the client supplies only a card ID, never SQL. Running the clone
is still gated by the usual runtime permission checks (card read perms,
result-metadata view-data, block perms), which are the same checks that gate
running the source card, so cloning grants no new runtime capability. This
mirrors POST /api/card/:id/copy, which likewise clones a readable card
without an authoring check.
Split the documents-API card creation helper into create-card! (still
checked; used for client-supplied draft cards) and clone-card! (used when
cloning embedded cards and when copying a document's cards). Authoring a new
query -- via draft cards or by editing a cloned card's query through
PUT /api/card/:id -- still requires the appropriate query permissions.
* Bump timeout to 60 minutes
* Run redshift GC in parallel, with 1 worker per cluster node
* Run snowflake GC with N=8 parallelism
* Record dataset census before GC sweep
* Satisfy the linter
* More consistently hide tenant users/groups
There were a few locations where we weren't correctly hiding the
existence of Tenant Groups such as "All Tenant Users" when the setting
was turned off. This also hides tenant *users* from the API surface.
* allow specifically showing tenant users
I realized that there was a possible regression in the last commit. We
were hiding tenant users across the board. This meant that a workflow
that was previously possible was not possible anymore:
- an admin turns off the tenant feature or disables tenants
- then the admin uses the API to reactivate some tenant users and turn
them into internal users.
This is probably super niche anyway but it seems nicer to allow `GET
/api/user` to show these users, IF you pass the
`tenancy=external&status=all` flags.
Then you can reactivate the user and detach them from the tenant.
This is all API only so it's probably academic, but figured I'd keep the
door open.
* fix tests
* fix e2e tests
* Move tenant tests to enterprise tenant test namespaces
* Address review feedback on tenant-group hiding
- Rewrite hidden-tenant-group-ids to intersect the input IDs with all
tenant-group PKs in memory instead of building an unbounded IN clause
- Return 404 from PUT/DELETE /api/permissions/group/:group-id for hidden
tenant groups, matching GET /api/permissions/group/:id
- Assert GET /permissions/graph/group/:id succeeds when use-tenants is on
- Remove duplicate create-tenant-group-requires-use-tenants-test
`dashboard/update-dashcards!` calls `(params/dashcards->param-field-ids
(t2/hydrate new-dashcards :card))` to compute the field IDs referenced by
parameter mappings. `t2/hydrate` is a no-op when `:card` is already present
on a dashcard, so if the client `PUT`s `/api/dashboard/:id` with a dashcard
that includes a nested `:card` (the standard shape from `GET`), that object
is forwarded verbatim into `param-target->field-id`.
That function validates its `card` arg against `:metabase.queries.schema/card`
(added Oct 2025), which requires keywordized metadata (`:type/Category`,
`:source/table-defaults`, ...). After JSON round-trip through
`cy.request`, those values are strings, so the request 500s with
`Invalid input: {:result_metadata [{:semantic_type [...]}]}`.
`dissoc :card` before the hydrate call forces a fresh DB select with
properly-typed metadata, keeping the on-demand-DB field-value update path
unchanged.
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
A handful of independent, low-risk fixes bundled together — each too small to
ship on its own, each a behavior improvement that also closes a minor
hardening gap.
Reduce DB load / avoid unconditional work:
- Settings cache: a settings-last-updated cookie now triggers a cheap
single-row staleness check and reloads only if the cache is genuinely
behind, instead of unconditionally reloading the whole settings table.
Exposes restore-cache-if-needed! with a :force-check? option as the shared
"reload if actually stale" primitive the cookie path now uses.
Reduce log noise:
- CORS origin validation runs on every request's raw Origin header, so it now
parses that client-supplied value with the silent try-parse-url rather than
the variant that logs "Invalid URL" at ERROR — a malformed Origin no longer
fills the operator's error log. Parsing behavior is otherwise identical, so
no CORS allow/deny decision changes.
Correctness (case-insensitivity):
- forgot_password throttles on the lower-cased email, matching the
case-insensitive account lookup, so User@x and user@x share one bucket
rather than each getting a fresh allowance.
Add a missing throttle:
- PUT /api/user/:id/password throttles the old_password re-auth check, which
previously had none, bringing it in line with the other credential
endpoints.
* Flush other executions before test. Divert other executions during test.
* Flush after closing the batch ingress path.
* Remove extraneous changes
* Remove more extraneous changes
* cleanup data warehouses nightly
* rename
* don't reinvent wheels
* simplify
* more
* cleanup
* updates
* Absorb failures at the redshift cluster level and at the driver level
* Extract old dataset name pattern
* Update ratchets
* Merge MB_REDSHIFT_TEST_HOSTS and MB_REDSHIFT_TEST_HOST when both are set
* Allow ad-hoc dispatch and report on outcomes of gc sweeps
* Don't run on forks
* Update ratchets
---------
Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: Ben Grabow <ben.grabow@gmail.com>
Make isRenderedWithinViewport retry until element settles in viewport
The command read getBoundingClientRect() once, so a caller asserting right
after a scroll-into-view action (focusing a recently moved dashboard filter)
could observe the element mid-scroll (top ~1647px vs 800px viewport) and fail.
Wrapping the geometry checks in .should() lets Cypress re-read the rect until
it lands within the viewport.
* fix: recognize lib/*-stripped pMBQL in ->legacy-mbql, restore edit-chart's query-id
->legacy-mbql gated pMBQL->legacy conversion on :lib/type alone, so a query that
lost its :lib/* keys (e.g. round-tripped through the frontend's viewing context)
fell through unconverted. The raw pMBQL then got embedded in /question# links,
crashing the frontend with "Stage 0 does not exist". Detect pMBQL structurally
(:lib/type or :stages) and normalize against the query schema before converting.
Also add the query-id edit-chart's result was missing, so extract-charts can
capture edited charts into conversation state and chart->xml stops rendering
an empty query-id attribute.
* fix: preserve edit-chart's query in memory, don't fail turn on link conversion
edit-chart's result set :query-id without :query, so extract-charts (which
requires both to fire, then rebuilds :queries from :query alone) overwrote
the chart's real queries with [nil] on the next update-memory call.
->legacy-mbql could also throw instead of degrading: normalizing a
:lib/*-stripped query mints fresh :lib/uuids that don't match positional
aggregation/expression refs elsewhere in the query (e.g. an order-by on the
query's own aggregation), failing conversion inside the streaming transducer
and taking down the whole agent turn instead of just producing a bad link.
Falls back to the raw query on conversion failure now.
* fix: document ->legacy-mbql's conversion-failure fallback
Docstring claimed conversion always succeeds after normalize, but the
try/catch fallback added earlier can still return raw pMBQL when a
stripped query's positional refs don't resolve.
* test: mark edit-chart-result-survives-update-memory-test ^:parallel
Doesn't write shared state, same as the test above it.
* fix: thread edit-chart-tool's resolved query into edit-chart
edit-chart-tool falls back to current-queries-state when a chart's own
:queries is empty, but edit-chart only checked charts-state, so its
result carried :query-id with a nil :query in that case -- the same
[nil]-in-memory bug fixed earlier, via a different path. Pass the
caller's resolved query through as a fallback.
* fix: capture viewing-context charts' query-id, don't clobber on edit
chart-config->chart never set :query_id, so a chart seeded from viewing
context rendered query-id="" on edit and was never captured into :charts
state by extract-charts. Reuse the context-item id already shared with
:chart_id.
That capture also collided with edit-chart-tool's own memory write for
the same chart-id: extract-charts fully replaced the entry, dropping
:image_base_64/:timeline_events/:chart_config the tool had just written.
Merge onto the existing entry instead.
* fix: separate ->legacy-mbql's docstring from its bug history
Docstring for ->legacy-mbql read as the change's rationale (BOT-1604,
the :stages detection story) rather than the function's contract.
Trim the docstring to what a caller needs -- normalizes MBQL 5 to
legacy, passes through unchanged on failure -- and move the "why"
for the :stages guard down to an inline comment next to the check.
Remove stale /* eslint-env node */ comments from frontend/build files
Flat config ignores eslint-env comments (ESLintEnvWarning per file, hard
errors as of ESLint v10), and they are redundant: eslint.config.mjs already
defines globals.node for frontend/build/**/*.js.
* respect default time dimensions from list of dimensions
* Curate a default time dimension on the time-facet test fixture
Master's #79409 stopped auto-assigning a default dimension to new
metrics, so the time-facet tests' fixture card no longer gets a default
time dimension from dimension auto-sync. Since this branch makes the
curated default authoritative for the time-facet variant (no fallback to
the query's temporal breakout), curate the default explicitly on the
fixture.
The earlier fix (#80388) made *queue declarations* reload-safe by guarding
them with a load-time ownership claim and replace-on-reregister semantics,
but left the *listener* registry non-idempotent: register-listener! threw
"Listener already registered" whenever the channel was already present in
the root *listeners* atom.
Fix mirrors the queue side exactly: def-listener! now claims its channel's
declaration site at load time (claim-listener-declaration! — same-ns
re-claims are no-ops so reloads stay legal, cross-ns claims throw), and
activates through register-declared-listener!, which replaces any existing
registration instead of throwing. Ad-hoc batch-listen! keeps its strict
throw-on-duplicate contract (double-listen-throws-test et al.), since
runtime registrations have no load-time claim protecting them.
* Encrypt subscription/alert recipient details; enforce allow-list on send
Encrypt at rest the `details` column of `notification_recipient` and
`pulse_channel`, which stores external email recipients for alerts and
dashboard subscriptions, so plaintext values are rejected on the strict
read when an encryption key is configured. Register both columns in
`encrypted-string-columns` so `encrypt-db`/`decrypt-db` and key rotation
re-encrypt them, and add a v58 migration that encrypts existing rows and
decrypts them on rollback.
Also enforce `subscription-allowed-domains` at send time (previously only
on write): the notification send path calls `validate-email-handlers!`
before delivery and throws if any external recipient is not permitted by
the allow-list. The exception is re-thrown with a generic message so
recipient addresses are never written to the logs.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* Move the v58 pulse_channel encryption migration test into the v58 group
order-your-migration-tests-test requires migration tests to be grouped by
major version in ascending order; the new test had landed among the v48 tests.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
Fail the test when a rollback-only scope cannot roll back
Every top-level `with-temp` runs in a rollback-only transaction. If something
inside one commits -- DDL does that implicitly on H2 and MySQL, which also
destroys the savepoint -- the rows written up to that point are durable and
leak into every later test. Warning about it hides the leak until some
unrelated test counts rows and gets the wrong answer, and the six offending
sites found so far were caught exactly that way.
Throw instead when running tests. That covers every cause -- raw SQL DDL,
HoneySQL DDL, a savepoint lost to a deadlock -- where a guard on the SQL we
build would miss the raw statements `drop-table!` sends. Production keeps
warning and discarding what is still pending: there the transaction is a real
one that has already committed, and failing the request would not undo it.
It sucks!
We will need to put our own error handling in place to do Actually
Good retries for 429s, but it's not clear yet where the best place for
that would be, because we're not getting stack traces from the 429s currently.
* Write the metadata mirror from one listener, not 33 endpoint hookups
* Describe the hydration listener by what it does, not what it replaced
* Point the original_fields note at the listener, not onQueryStarted
* Pin the hydration listener to Api actions, and report a failed write
SEC-669: re-check read permission when listing bookmarks
GET /api/bookmark (bookmarks-for-user) filtered only on the target's
archived column and never re-checked read permission, so a bookmark kept
leaking a card/dashboard/collection/document/exploration's name,
description, and type after the user lost read access without the item
being archived (moved to a private collection, or a revoked/narrowed
collection grant).
Add a visible_collection_ids CTE (permissions only; archived filtering is
left to the existing where-conditions) and filter each bookmark branch by
collection visibility via visible-collection-filter-clause. Visibility is
evaluated for the passed user-id via a UserScope, so the function is
correct for any user, not only the current request's user.
* Redesign Radio
* Add Radio stories to loki filter
* disallow radio labelPosition
* use StoryBoard in radio stories
* add new story's snapshots
* Radio.Card: border, leftSection, migrate call sites
- card border 0.5px border-neutral-strong, none when disabled
- children disallowed, leftSection slot for icons
- EditUserStrategyModal + DataSegregationStrategyPicker on label/description
* update loki snapshots
* Icon color prop instead of css module in DataSegregationStrategyPicker
* remove extra div
* make the indicator in the radio card optional
* fixed radio size, no size prop
* radio card body via mantine text/stack
* forwardRef radio card
* radio card border: highlight when selected
* update loki snapshots
* drop size="xs" from radio call sites
* fix card growing on select, interactive card stories, optional checkbox card indicator
* skip interactive card stories in loki, update card snapshots
* Key cached session tokens on the app db
with-empty-h2-app-db! swaps in a different application database, and this cache was keyed on the username
alone. A token minted against the previous database names a session that does not exist in the new one, so
the request 401s, and the token minted inside leaks back out when the original is restored. The retry in
client-fn cleared the whole cache and tried again, which papered over it often enough to look like flake:
"Failed to authenticate X after three tries".
Key it on the app db as well, like every neighbouring cache does via memoize-for-application-db.
databases-list-can-upload-test and dismiss-spinner-test go from failing to passing; both were in the CI
failure list, the former across 12 jobs.
* Fix rollback-only app DB transaction semantics
* Reject rollback-only combinations that cannot roll back
- :nested-transaction-rule :ignore skips the savepoint :rollback-only needs, so the pair is rejected at any
depth rather than silently committing
- a throw out of the transaction body now rolls the connection back explicitly, since restoring autocommit
commits whatever is still open -- a requested rollback that failed was committing the work it had to discard
- the unsupported-options error names the options, and drops the "application database" framing that
misattributed failures on other java.sql.Connections
* Restore transaction state even when the savepoint rollback throws
The callback accumulators are discarded unconditionally at the top of rollback!, so leaving the state
restore behind a throwing .rollback left the tree half-unwound: an outer scope that caught the error and
committed would run its before-commit callbacks against the failed nested scope's state.
* Create test-data Databases outside the caller's transaction
Their ids are memoized for the whole app db, so the Database has to outlive whatever scope first asked for
one. Created inside a test's rollback-only with-temp it is rolled back while the memo keeps its id, and every
later lookup hands out an id for a row that never existed.
do-outside-transaction is the same detachment run-after-commit-callbacks! already did privately.
* Don't cache a personal collection id created inside a transaction
The cache's premise is that a Personal Collection cannot be deleted, so its id never changes. A rollback
breaks that: the collection goes away and the cached id points at a row that never existed, which reads as
a 403 on every later request that resolves the user's personal collection.
* Skip the test-database cache in a transaction rather than detaching
Detaching made the Database durable but took a second connection while the caller's transaction was still
open. When that transaction holds cluster lock rows -- data_permissions and search index setup both take
them -- the second connection waits on them until the lock acquisition times out.
Bypassing the cache keeps the work on the caller's connection, and matches how the personal collection id
cache handles the same problem. do-outside-transaction goes with it, having no other callers.
* Stop the outer commit when a requested rollback failed
The full rollback only runs when this call turned autocommit off, so for a nested scope or a caller-managed
connection a failed rollback! just propagated. Code between there and the outermost scope can catch it, and
the commit branch would then commit the writes the caller asked to discard.
Mark the tree instead, and refuse to commit it. Throwing rather than quietly rolling back: the caller asked
to commit and cannot, and dropping the outer work silently trades one bug for a worse one.
* Tolerate a savepoint an outermost rollback-only can no longer roll back to
A statement in the body can commit the transaction under us -- DDL does that implicitly on H2 and MySQL --
which invalidates the savepoint. Those writes are durable by then, so failing the scope reports a problem
nobody can act on while leaving the data exactly where it was.
At the outermost scope, discard whatever is still pending and log where it happened. Nested scopes still
poison the tree and throw, since there the enclosing work is pending and must not be committed.
* Note the option of rejecting DDL inside a savepoint-bearing transaction
* Never commit a transaction whose rollback failed
Four review passes flagged the same hole from different angles. Restoring autocommit commits whatever is
still pending, and after a rollback we were relying on to discard those writes, that is exactly the work
we were told to throw away. Track it and leave autocommit off instead, letting the pool roll the
connection back on check-in.
Two narrower routes to the same place:
- rollback! now marks the tree itself, so an exception-driven rollback that fails poisons it too. Before,
only a requested rollback did, and the error path could still reach a commit.
- the mark is rechecked after before-commit callbacks drain. A callback can catch a nested scope whose
rollback failed, which lands after the earlier check has already passed.
Two tests changed meaning as a result: a nested scope whose rollback throws now stops the outer scope
committing, so it no longer completes and its own after-commit callbacks no longer run. That is the point
-- its writes are still there.
Also adds the regression test for personal collection id caching, models autocommit honestly in the mock
rather than pinning it true, and records what the in-transaction test-Database lookup gives up.
* Build the test-data Database before with-temp opens its transaction
Created inside a rollback-only scope it never survives, so it is rebuilt for every test, and the ids
memoized in metabase.test.data.impl outlive it. Cards then get inserted against table ids for rows that no
longer exist: FK_REPORT_CARD_REF_TABLE_ID on a fresh app db.
Materialise it before the transaction opens, where it can commit. The table and field lookup maps get the
same in-transaction bypass the Database id already had, for the case where a dataset is first asked for
inside a scope anyway -- neither is enough on its own.
Reproduce with a throwaway app db, which is what CI has and a local one does not:
MB_DB_TYPE=h2 MB_DB_FILE=/tmp/fresh-$$ ./bin/test-agent :only '[metabase.queries.models.card-test]'
* Only pre-warm the dataset for the default driver
db-id resolves through the driver under test, so in a driver-test namespace the pre-warm reached for that
warehouse's dataset -- work a with-temp there may never need, against a warehouse that may not be up.
Driver failures went from 2 to 12 in the push that added it.
The FK problem it fixes is on the app-db side, for the default dataset, so confine it there. Best effort
even then: a dataset that cannot be built must not take an unrelated with-temp down with it.
* Evict temporary users from personal collection cache
* Evict the personal collection cache on the rollback, not on the user
Clearing the temp User's own entry misses the case that actually bites: a committed user's Personal
Collection is created lazily on its first permission check, which often lands inside somebody else's
with-temp. Nothing evicts it when that scope rolls back, and every later request for that user 403s.
Key off the with-temp boundary instead, and drop whatever it may have cached. Costs a re-lookup per user
afterwards; correct whichever user and whichever scope created the entry.
agent-api + activity-feed went from 21 failures to none.
* Skip the dataset pre-warm where the app db is deliberately empty
with-empty-h2-app-db! rebinds the app db, and the memo key includes its identifier, so the pre-warm missed
the cache and created a test-data Database inside a scope whose whole contract is emptiness.
databases-list-can-upload-test snapshots an empty Database set and saw two.
Bind *skip-dataset-prewarm?* there instead. Also drops the earlier :h2 restriction, which came from a bad
reading of two CI runs sampled at different completion stages: it could never have helped here, since those
helpers are H2, and it took the pre-warm away from driver tests that needed it.
Reproduce with a throwaway app db, which is what CI has and a local one does not:
MB_DB_TYPE=h2 ./bin/test-agent :only '[metabase.queries.models.card-test]'
* Register the cache eviction with a unique key
metabase.test.redefs already holds `:around :default` on do-with-temp*, and methodical overwrites methods
sharing a key rather than composing them. Whichever namespace loaded last would win, silently dropping
either this eviction or the rollback-only transaction wrapper. It happened to work; it should not depend on
load order.
Also notes what a failed connection-level discard leaves behind. Only a caller-supplied connection is
exposed: on one we own, the same flag stops autocommit being restored, so the writes are never committed.
It logs rather than throws because throwing from that catch would mask the exception being unwound, and
master never attempted the discard at all, so a caller-supplied connection is no worse off than before.
Signalling it properly would mean telling the caller some other way.
* Note that session tokens are cached without regard for transactions
The user ids directly above skip the memo inside a transaction, since users created in a `with-temp` are
discarded with it. The token cache has no equivalent guard, so a session rolled back with its scope leaves a
token behind and the request fails with "Failed to authenticate X after three tries". The retry in
`client-fn` hides it more often than not.
* Note that a rollback can leave a stale personal collection id cached
"A Personal Collection cannot be deleted" does not hold for a transaction that rolls back. Tests evict on
the with-temp boundary; production has no equivalent and keeps the id for the rest of the TTL. Accepting
that for now -- it predates honoring :rollback-only, and closing it properly wants an after-rollback hook
alongside do-before-commit/do-after-commit.
* Follow the created-globally rename in the token cache TODO
* Build the query for export inside the guard, not beside it
`:query_json` already returns nil for a card whose query cannot be exported, but `lib/query` was called in the
argument position, outside `try-export-query`. A stored `dataset_query` with no stages builds a query that fails
its own output schema, so one such Card takes down every response that walks a set of Cards -- the typed-schemas
endpoint among them.
metabase.search.appdb.scoring-test alongside metabase.typed-schemas-rest.api-test goes from 13 failures to 0.
* Don't run a test request inside a rollback-only transaction
`user-request` takes a temporary Session when it is handed a User or an id rather than a `:rasta`-style keyword,
and the handler runs in-process on this thread. Now that the with-temp rolls back for real, every app db write
the request made is discarded when it returns -- the warehouse table writes survive on their own connection and
the API still answers 200, so the test sees a successful call that changed nothing.
Insert the Session and delete it in a `finally` instead. A caller already inside a transaction still gets the
same behaviour it always had; what goes away is the extra rollback scope around the request itself.
metabase-enterprise.action-v2.models.undo-test goes from 25 failures to 0, passing all 134 assertions, which
matches master.
* Mint the test users' sessions before anything opens a transaction
`authenticate!` ran about 150 times in a single `pulse.api.pulse-test` run, nearly always inside a transaction.
A session minted inside a `with-temp` is rolled back with the scope, the next request 401s, and the retry in
`client-fn` cleared the whole cache -- every user's token, not just the dead one -- so everyone re-authenticated
in-transaction and the cycle repeated. Each of those inserts bumps the user's `last_login` through the
`:model/Session` after-insert hook, which is what breaks a test that reads the creator before making its request.
Mint the four test users' sessions at initialization, where nothing has opened a transaction yet, so they are
durable. On a 401 forget only that user's token, so one temporary user's rolled-back session cannot evict them.
pulse.api.pulse-test goes from one failure to none, matching master. publish-refresh-error-event-sends-admin-email-test
passes too, where it fails standalone on master: that email only goes to admins with a `last_login`, and
crowberto's was rolled back with every login.
* Ask for the FieldValues the chain filter test needs
The full FieldValues row for `categories.name` is created lazily, and `update-field-values-for-table!` only
refreshes rows that already exist -- so a run that creates it inside a `with-temp` discards it and nothing
makes another. `select-one-pk` then returns nil and the test dies on "Not something with an ID".
Same shape as the rest of this branch: a durable row created on demand, now rolled back with the scope that
created it. Ask for the row instead of assuming an earlier test left one behind.
* Run the metric leaf queries one at a time inside a transaction
A transaction owns a single connection and `future` hands that binding to every thread it starts, so the
leaves' app db writes -- used-card stats behind a cluster lock, query rows -- interleave on one session.
Savepoints are per session: a thread rolling back to its own destroys the ones its siblings are holding, the
next release fails with "savepoint does not exist", and the transaction aborts, taking the rest of the test
with it. That is why api-arithmetic-test loses a different test on each CI run.
Only tests get here. A request runs in no transaction, where each future takes its own pooled connection and
the parallelism is worth having.
On a throwaway Postgres the namespace errors on two tests per run before this and passes three runs out of
three after.
* Commit a lock row that would otherwise roll back with the caller
A lock row is written once and read forever after, and acquisition leans on that: rival first-time inserters
block on the winner's uncommitted unique-index entry and retry once it commits. Inside a `with-temp` that
commit never comes, so the row is gone again and the next acquisition replays the race -- on MariaDB until the
lock-wait timeout expires.
Insert it on a connection of our own when the caller was already in a transaction, which is the only case
where the row would not survive. Nothing in production takes a cluster lock from inside another transaction,
so this leaves that path alone -- worth keeping that way, since lock names are dynamic in places and rows that
used to roll back with a failed transaction would otherwise persist.
Reasoning rather than measurement; the TODO on the function says so and says what to do if it does not help.
* Bound the out-of-band insert on its own connection
`set-query-timeout!` returns false on MySQL running MariaDB Connector/J, and the fallback sets
`innodb_lock_wait_timeout` on the session -- but `acquire-lock-row!` sets it on the caller's connection, not
the one the insert opens. Blocked by a rival's uncommitted row, the insert would then wait out the server
default instead of the second it was given, on exactly the databases this is meant to help.
Give it the same treatment on its own connection. Also note in the docstring that the second connection is
only taken until the row exists, which is what bounds the cost of holding two at once.
* Resolve the venues id before counting queries
The cache is fine; the query the test counts is its own `(mt/id :venues)`. Inside a transaction the test-data
id lookup deliberately skips its memo, so the call lands in the window that is supposed to see none.
* Keep the pre-warm out of app dbs that are meant to be empty
`with-empty-h2-app-db!` already opts out of the dataset pre-warm; the serialization test dbs and the
schema-migration ones never did. A `with-temp` inside them materialised the test-data Database, and an extract
then saw a Database no assertion expects -- serialization.v2.load-test/database-test wanted one and found two.
* Do not insert a lock row out of band outside tests
The claim in "Commit a lock row that would otherwise roll back with the caller" was wrong: production does take
cluster locks inside another transaction. `load-from-h2!` inserts Databases, and so takes the batch-permissions
locks, inside one big load transaction. Our fresh connection then blocked on a next-key gap lock the caller's
own SELECT ... FOR UPDATE was holding, and a thread cannot outwait itself -- it timed out, retried, and failed.
That is what broke dump-to-h2, load-from-h2 and rotate-encryption-key on MariaDB.
Gate the out-of-band insert on `config/is-test?`, which is the only place a transaction may never commit, so
production goes back to exactly what it did before. Keep a fallback for the same shape inside tests, where
`load-from-h2!` can block on itself just as readily: on a lock-wait timeout, insert on the caller's connection.
* Stop three card tests leaving rows behind
`updating-model-query-does-not-shift-metadata-overrides-test` had no cleanup at all, and
`create-card-with-type-and-dataset-test` opened a plain `t2/with-transaction` where its siblings pass
`{:rollback-only true}` -- so both committed cards with a `{}` dataset_query that every later test walking the
card table then had to survive. That leak is older than this branch; honoring the rollback is what makes the
second fix work.
`get-series-for-card-permission-test` read crowberto's personal collection with `select-one-pk`, and a nil
collection_id puts the card in the root collection, which rasta can read -- so the test quietly asserted 403
against a card that was never private. Ask for the collection instead.
* Truncate the heartbeat fixture times to milliseconds
Postgres stores microseconds, so a timestamp ending .7369995 comes back a millisecond above its own
millis-truncation and the assertion misses by one.
* Tolerate a card whose query will not parse when finding related entities
Nothing at the schema level stops a stored `dataset_query` being empty, and such a card simply has no
context-bearing forms. Without the guard one of them takes down the related-entities computation, and with it
every x-ray, for everyone else. Same treatment `card-details` already gives an unexportable query.
* Give the end-to-end schema test a Database of its own
The test asserts exactly which questions, metrics and models fall inside the test-data Database's scope, which
holds only as long as no other namespace commits a Card against that Database. Two such leaks have been fixed
on this branch already and there is at least one more; the assertion cannot be made safe by chasing them,
because any test added later can break it again.
Run it against a copy of the Database instead, so nothing else can write there.
A throwaway namespace that commits one Card on the test-data Database ahead of it reproduces the CI failure
exactly (core_test.clj:147) and passes with this change.
* Recognise the MySQL lock-wait timeout in the in-band fallback
`lock-wait-timed-out?` knew about `SQLTimeoutException` and our own `::acquisition-timeout` marker, but not
ER_LOCK_WAIT_TIMEOUT itself. The server raises that on its own schedule, so on MySQL and MariaDB -- the
databases the fallback exists for -- it could arrive as a plain `SQLException` and the fallback would not fire.
Check the code through the cause chain.
* Take the revisions with the cards
`with-model-cleanup` deletes rows directly, which skips the Card hook that would have removed the revisions
this test's POST and two PUTs create. Name Revision in the cleanup as well.
* Truncate once, and align the chain filter's temp values
The second `truncate-to` could not do anything the first had not.
* Create the test users before the related tests race for them
`user->id` skips its memo inside a transaction and creates the users there instead, so on this branch they roll
back with the scope and the next test makes them again. These tests are parallel and every one of them takes a
`with-temp` Card whose creator defaults to rasta, so several threads insert the same user at once and all but
one time out on the row lock. The namespace goes from three errors to none.
Also covers the guard in `related/definition`: a Card whose stored query will not parse contributes no
context-bearing forms, and a genuinely similar Card still outranks it.
* Cover the card whose query will not build
The guard in `card-details` had no test: a Card with a stage-less stored query answers without `:query_json`,
and the Card beside it in the same response is unaffected.
* Cover the batch path and the lock-wait code
The first version of the card test called `get-table-details` once per card, which cannot see the failure it
claims to cover -- the bug was that one bad card killed the whole `cards-details` seq the others were in. Go
through that seq instead. Removing the guard turns the test red at the same malli output error CI was showing,
which the per-card version did not.
Also cover `lock-wait-timed-out?`: MySQL's ER_LOCK_WAIT_TIMEOUT on its own and inside a cause chain, and any
other error still propagating rather than being taken for a timeout.
* Create the temp search index before the transaction, not inside it
`with-temp-index-table` runs CREATE TABLE and DROP TABLE on the ambient connection, and on H2 and MySQL that
commits the transaction around it. Six tests opened it inside a `with-temp`, so the rollback had nothing left
to take back and every row they created leaked -- a Transform, some Cards and Collections, a published Table, a
Document -- to any later test that walks the app db.
That one leaked Transform is the whole of the serialization failures: a collection-scoped export includes every
Transform in the instance, so all four tests came back with one entity more than they expected.
Take the index table before the `with-temp` instead. The nested scope reuses it, so nothing else moves.
`metabase-enterprise.search.scoring-test` also had no fixtures at all, and fails on its own: the first lazy
`initialize :db` fires mid-test and recreates the H2 database underneath a test already writing index
bookkeeping.
The 133 namespaces that run before serialization.api-test in the Enterprise job reproduce the CI failures
exactly -- 5011 assertions, 5 failures -- and pass with this, at 4714 assertions.
* Say the cluster lock comments plainly
Lead with the point rather than building up to it, and drop the clause chains. No behaviour change.
The TODO above `clear-cached-session-tokens!` was stale: it asked for the `created-globally` treatment, which
landed earlier on this branch. Point at what is actually left instead -- initialization can still first happen
inside a `with-temp`, and then the sessions roll back with it.
* Keep the non-index assertions where the app db cannot index
`with-temp-index-table` is a `when`, so wrapping a whole test in it skips the whole test on MySQL and MariaDB.
Two of the tests moved in the last commit also assert things that need no index, and those assertions went
quiet there. Add a sibling that runs the body either way, and use it for those two.
`search.appdb.scoring-test` had no initialization fixture, the same gap its enterprise counterpart had: the
first lazy `initialize` can recreate the H2 db underneath a test already writing index bookkeeping.
* Sort the fixtures require
* Download through the client that retries a 401
`downloaded-card-filenames-test` passed a raw cached token straight to `client-full-response`, so it skipped
the retry every other test gets from `client-fn`. A session rolled back with the scope that made it then came
back here as a bare 401, which is what the BigQuery job has been failing on.
`user-http-request-full-response` still returns the headers the test reads.
* Improve rollback test isolation and comments
* Do not poison the transaction tree when the savepoint is already gone
MySQL and MariaDB roll back the whole transaction to break a deadlock, discarding every savepoint with it. The
ROLLBACK TO SAVEPOINT that follows fails with "SAVEPOINT does not exist", and marking the tree rollback-required
there is backwards: the server has already discarded the writes, so nothing is pending for an enclosing scope to
commit by mistake. A rollback-only scope whose DDL committed implicitly already gets this treatment.
The refusal message named a rollback-only transaction, but the ordinary error path sets the flag too.
Also validate :nested-transaction-rule with a throw. It is a public option, and `assert` compiles away under
`*assert*` false. Sorting the unsupported keys keeps the message stable across runs.
* Keep the test request's session out of the login hooks
:model/Session's after-insert hook publishes :event/user-login, which stamps User.last_login, and
:event/user-joined for a User that has never logged in. Deleting the Session afterwards undoes neither, so every
user-http-request left both behind for whatever ran next.
Write the row instead. created_at has to come along, since the model supplied it.
* Ask whether the app db can hold the index, not whether any engine can
supports-index? is true for any active engine. With semantic search on MySQL or MariaDB, the macro that exists to
run the body without an app-db index took the app-db branch and tried to create the table anyway.
* Say something when a stored query is dropped
Both guards swallowed every exception silently, so a Card whose query failed to build for an unrelated reason
looked exactly like one that has no query. try-export-query, right next to the Metabot guard, already warns.
Related entities logs at debug: it runs for every candidate Card while ranking.
* Commit the out-of-band lock row rather than assume autocommit
The docstring said the row was committed, but nothing committed it -- the insert relied on the pooled connection
arriving in autocommit, which the pool does not promise. The caller's SELECT cannot find the row otherwise.
* Let a failed dataset pre-warm be heard
Now that the pre-warm runs for every driver, a warehouse that is down fails it on every top-level with-temp. At
debug that is invisible on a normal run and resurfaces much later as a Database that does not exist. Warn once
per message.
Also note that mt/id costs a query inside a transaction, which is what a with-call-count window has to plan for.
* Polish prewarm comments
* Mark the tree rollback-required even when the savepoint vanished
A vanished savepoint does not mean nothing is pending. DDL committing implicitly, or a server breaking a deadlock,
ends the transaction and discards the writes so far -- but anything written afterwards sits in a fresh transaction
and is still pending. Skipping the flag there let an enclosing scope swallow the rollback error and commit exactly
those writes.
Say so at the point of the catch, since the reasoning does not survive a quick read.
* Gate every app-db index helper on app-db support
with-temp-index-table and with-appdb-search-if-available-otherwise-legacy still asked supports-index?, which is
true for any active engine. With semantic search on MySQL or MariaDB they ran app-db DDL the app DB cannot take,
including from scopes nested inside the fallback body that exists to avoid exactly that.
Cover the borrowed session too: it must leave last_login empty, and delete itself afterwards.
* Cover the out-of-band lock row against a connection in a transaction
Hands the dedicated connection over with autocommit off and checks the row survives the caller's rollback. Without
the explicit commit the insert dies with the connection, and acquisition then fails on the row it just wrote.
Runs on MySQL and PostgreSQL only; H2 takes an in-process lock and writes no row.
* Pin the vanished-savepoint case against the outer commit
A savepoint the server already discarded says nothing about what was written after it. The existing rollback tests
throw generic exceptions, so nothing stopped the SQLState 3B001 / MySQL 1305 exemption from coming back.
* Pin the engine in the otherwise-legacy fallback
Where the app DB cannot hold an index, the fallback took whatever default engine was active. Semantic can be
active there, and the bodies are written for an index the app DB does not have. Run them under legacy search, as
the macro's name says. with-legacy-search moves above its first use.
* Polish rollback and search comments
* Stop two remapping scopes deadlocking on MySQL and MariaDB
A remapping writes a Dimension for a Field the whole suite shares, and holds it until its scope ends. Two threads
doing that at once deadlock on the unique index over dimension.field_id: each holds the record lock the other's
uniqueness check wants, while asking for the insert-intention gap in front of it. MySQL and MariaDB pick a victim
and roll it back, which drops its savepoints, so the rollback that follows fails on a savepoint that is gone.
Serialise remapping scopes against each other. Tests that never remap keep running in parallel alongside them.
The helper already carried a TODO saying it was not thread-safe.
Reproduced on MariaDB 12.3 and confirmed on master, so this predates the branch. custom-values-test failed nearly
every run before, and three runs in a row are clean after. Verified on H2 as well.
* Serialise every temporary Dimension, not just remapping scopes
Remapping is not the only way a test writes a Dimension for a shared Field: card-test, card/metadata-test,
field-values-test and data-studio's table-test all create them directly, in parallel, on the Orders and Products
fields. Those still raced with the remapping scopes.
Move the lock to `do-with-temp*` for :model/Dimension so every site is covered, including ones added later.
`locking` is reentrant, so a `with-temp` holding several Dimensions still works.
* Take the search index out of six more with-temp transactions
Same fix as the earlier pass over the search tests, for the ones that pass a warehouse driver: their index scope
sits inside `with-temp`, so `drop-table!` runs DDL on the ambient connection. H2 and MySQL commit the transaction
to do it, which makes the temporary Transforms durable and destroys the savepoint. The rollback that follows can
no longer take them back, and it discards toucan's compensating deletes along with everything else still pending,
so the Transforms outlive the test.
Three tests later in the same namespace count transforms and saw the strays: root-collection-items-include-
transforms-test expected one and found four.
These were missed the first time because `mt/normal-drivers-with-feature :transforms/table` skips them on H2, so
only a driver run reaches them.
Reproduced with a MariaDB warehouse: transform-test went 3 failures to 0, and the four namespaces that failed
together now match master at 2532 assertions. H2 is unchanged at 109.
* Let the failed-rollback warning reach the console
The shared Console appender only accepts FATAL, so the warning naming a rollback-only scope that could not roll
back went to logs/test-log.json and nowhere a CI run gets read. It took hours to find six tests leaking rows
because of it; the warning had been firing the whole time.
Give that logger an appender that accepts WARN. The message carries the mb-test key, so it names the test that
broke its own isolation rather than the later one that trips over the rows.
* Hoist the search index scope above with-temp in two document search tests
Both tests opened `with-appdb-search-and-legacy-search` inside `mt/with-temp`.
That scope creates and drops the index table on the ambient connection, and on H2
and MySQL the DDL commits the transaction `with-temp` meant to roll back, so the
temp Documents leaked into later tests.
Found by the throw this branch adds: both showed up as errors at connection.clj:293
in the app DB test jobs.
* Serialize the reduced-fields card test
It holds a rollback-only savepoint across two `with-temp` Cards and an API call. Run in parallel, something else
commits the transaction that savepoint lives in, so both Cards are discarded and the downstream query reports its
source Card as missing.
Only MySQL surfaces it, and only under a full suite: the test passes on its own, and six local configurations could
not reproduce it. Confirmed by running the MySQL and MariaDB driver jobs with this single change -- all four green,
against four runs where they failed.
* Materialize the test users' Personal Collections before with-temp opens
`user->personal-collection` is a get-or-create and sits on the permissions path every API request takes. A
namespace that does not initialize `:test-users-personal-collections` creates Lucky's the first time a request
calculates permissions -- which happens inside a `with-temp` -- so the transaction rolls it back. Every later
request tries to create it again, and the parallel ones time out on each other's COLLECTION table lock:
GET /api/database expected a status code of 200, got 500.
Timeout trying to lock table; INSERT INTO COLLECTION (..., "Lucky Pigeon's Personal Collection", ...)
That is what failed `databases-list-include-saved-questions-test` on the Mongo, Redshift and Athena jobs -- the
empty database list was the 500, not a visibility problem.
Prewarm them next to the dataset, under the same skip flag, so helpers that need an empty app DB are unaffected.
This also closes the related hole where `:test-users` itself first initializes inside `with-temp`, since that
component initializes it.
Confirmed on the Mongo driver jobs: MongoDB Latest went from failing to green.
* Update the test-user session note now that initialization is prewarmed
* Ratchet config-level kondo suppressions too
:config-counts in ratchets.edn tracks, per linter, the :off switches and
:exclude entries in .clj-kondo/config.edn (top-level and per ns-group).
Only weakening counts: adding discouragements or turning a linter on is
free. Lowering works like the inline budgets; raising is a hand edit.
* Count every excluded item and :config-in-comment in the config ratchet
Map-shaped :exclude values with vector entries counted as one suppression
({compojure.core [GET DELETE POST PUT]} is 4, not 1), and the
:config-in-comment :off switch wasn't aggregated at all, so both could
grow without tripping CI. Reseeded.
* Count :config-in-call suppressions, and self-heal on config.edn
`:config-in-call` is a scoped `:linters` map like `:config-in-ns`, so a
linter switched off inside one was invisible to the ratchet. Both scopes
now fold into the same count.
The self-heal workflow only watched Clojure sources, which was right while
budgets came from inline ignores alone. Now that `.clj-kondo/config.edn`
feeds `:config-counts`, removing a suppression there can leave a budget
stale with no way to trigger the fix.
* Count scoped :off suppressions at any nesting depth
Linter scopes nest more than one level: :discouraged-java-method keys by
class and then by method. Counting only the first level meant an :off one
map further down waived a warning the ratchet never saw. No budget moves --
the repo's only two-deep entry is a :level :error discouragement.
* Regenerate kondo ignore budgets after restacking onto master
* Fix flaky document-comment split test by using realClick on hover-revealed button
The test clicked the hover-revealed "new thread" comment button of the
second paragraph via Cypress's synthetic .click(). After realHover() moved
the cursor to reveal the button, the synthetic click could act on the button
while its hover transition was unwinding, so the click did not reliably
trigger the route navigation that opens the comments sidebar — leaving
getSidebar() to time out.
Switch to .realClick(), which moves the real mouse onto the button (keeping
it visible/stable) and dispatches a real click that reliably fires the link.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* Wait for document save to persist before opening comments sidebar
The split-paragraph test opened the comments sidebar immediately after the
Save button disappeared, but a successful save schedules a deferred
navigation back to /document/:id. Under network throttling that navigation
resolved after the sidebar route had opened, clobbering it, so the sidebar
never mounted and getSidebar timed out. Wait on the PUT request and the
"Document saved" toast so the post-save navigation settles first.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Google SSO already requires the id-token email to be verified; the
generic OIDC provider (and slack-connect, which derives from it) did
not look at the claim at all.
Tokens with email_verified present and not true are now rejected. An
absent claim is still accepted: it is optional per OIDC Core and some
IdPs never send it.
* Fix: Support Google Cloud Universe Domain in BigQuery driver
* feat(bigquery): Add universe domain to connection properties UI
* feat(bigquery): improve universe domain helper text
* Update bigquery_cloud_sdk.clj
Indeed we are out of sync as the impersonating was removed from master branch.
* removed empty comment
* Fixing formatting
Wait for the question move PUT to settle before navigating to the
affected dashboard, so its empty state is asserted only after the
dashcard has actually been removed.
* ensure highlighted bars are included in brush drill filter
* fix e2e
* fix e2e again, December 2026 label was not appearing maybe due to a screen size flake
The 'adding non-series columns to the tooltip' scatter test hovered a
bubble immediately after render. The ExplicitSize throttle re-renders the
chart once its container is measured, moving bubbles out from under the
realHover() before the tooltip fires, so [data-testid=echarts-tooltip]
was intermittently never found (4000ms timeout).
Apply the same view-toggle stabilization the sibling tooltip test already
uses (extracted as stabilizeScatterChart) before each hover, forcing a
re-render against an already-measured container.
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* Require a justification comment above inline ignores, ratcheted per linter
An ignore of a linter outside :comment-exempt needs a substantive ;;
comment on the line above or trailing on the same line (bare ;; dividers
don't count, and no attempt is made to check the comment is about the
ignore). ~18% of ignores have one today, so 35 linters start exempt.
The set only shrinks: once a linter's last uncommented ignore gains a
comment, the fixer drops its exemption and CI holds the line.
* Locate trailing justification comments in masked source
A semicolon inside a string or character literal on the ignore's line
counted as a trailing comment, so an unjustified ignore could pass.
The masker now keeps the ; that starts a comment (and blanks the char
after a backslash), so justified? finds only real comment starts.
* Share one tree scan across the ratchet tests
Each ratchet test walked and slurped the whole source tree; a delay
runs the scan once for all three.
* Invalidate the shared tree scan per test run
The namespace-level delay cached the scan for the JVM's lifetime, so a
long-lived REPL compared fresh ratchet-file contents with stale counts.
The :once fixture now resets the cache; a directly-run test var scans
fresh.
* Scope the shared tree scan to the fixture's lifetime
The atom cache survived a run, so a later directly-invoked test var
(which skips :once fixtures) reused a stale scan. The cache now holds a
delay only while the fixture is live; without it every call scans fresh.
* Require canonical syntax for kondo ignores
Keep ignore detection deliberately lexical and fail closed: every real marker must use the key-first map form, otherwise the ratchet rejects it with an actionable error. This avoids maintaining a partial Clojure reader for the two attr maps that previously placed the ignore behind another key.
Justification comments must be substantive and either directly above the ignore or trailing on its line. Authenticate comment openers in masked source so strings and code-line comments cannot pose as justifications.
* Harden kondo ignore scanning edge cases
* Use reader delimiters for kondo marker boundaries
* Match Clojure reader whitespace in kondo markers
* Reserve clj-kondo namespaced map syntax
* Harden kondo scanner diagnostics
* Rebuild the budgets and exemptions on the named-linter base
Rebased onto the ignore-naming branch, so every budget is recomputed from
the tree and :comment-exempt is reseeded with exactly the linters that
still have an uncommented ignore: 38, up from 35. The three extra are
linters that were hidden inside bare `#_:clj-kondo/ignore` forms until
they were spelled out; :all itself drops out, having no ignores left.
* Point at :comment-exempt after inserting ignores for a new linter
kondo-insert-ignores writes comment-less ignores, which the justification
ratchet rejects for non-exempt linters.
* Leave ignores preceded by a # for a human, and gate the exemption advice
A `#` immediately before the matched form is ambiguous: in `#^{...}` it is
legacy metadata that removal must take with it, in `x#^{...}` it closes a
syntax-quote gensym that removal must leave alone. Telling them apart means
reading Clojure, which this scanner deliberately does not do -- so report
the site under :skipped, as it already does for prefixless maps and
unbalanced braces, and let a human excise it.
Separately, kondo-insert-ignores no longer advises a :comment-exempt entry
unconditionally. An inserted ignore lands under whatever comment already sat
above the flagged form, so a linter whose sites are all justified would get
an exemption that no-stale-exemptions-test then rejects.
* Align the scan-test expectations
The occurrence vector's :justified? column and the counts map both drifted
out of alignment as linters were added to the fixture.
* Regenerate kondo ignore budgets after restacking onto master
* Make every kondo ignore name the linters it suppresses
The vector-less `#_:clj-kondo/ignore` suppresses every linter on the next
form, so a site added to silence one warning quietly absorbs any other that
appears there later. Replace all 50 with explicit linter vectors.
The budgets absorb what the bare forms were hiding: :discouraged-var
106 -> 129, :unresolved-namespace 3 -> 9, :type-mismatch 2 -> 6,
:unresolved-symbol 4 -> 9, plus first entries for :consistent-alias,
:duplicate-require and :redundant-do. :all drops out entirely, so a new
bare ignore now fails the ratchet with no budget to land in.
* Use the mdb.env alias in app-db-as-data-warehouse
Var-quotes resolve namespace aliases, private vars included, so the
fully-qualified name was never needed. Dropping it retires the
:aliased-namespace-symbol ignore and the unused-alias warning the
workaround caused.
* Regenerate kondo ignore budgets after restacking onto master
* Add primary/secondary NavLink variants and sibling components (GDGT-2469)
Restyle the design-system NavLink from Figma:
- Add `primary` (brand-fill selected) and `secondary` (subtle gray selected)
variants, with Default/Hover/Pressed/Selected states.
- Add the `icon-{primary,secondary,disabled,brand}-inverse` color tokens.
- Add `NavLinkButton` and `NavLinkBadge` sibling components for the right
section; both invert automatically on a selected `primary` item (no prop),
the button also has its own hover/press and stops click propagation.
- Give NavLink a `leftIcon` prop so it renders and colors the leading icon
itself instead of styling the section slot.
- Add a two-variant state-matrix Storybook showcase (Loki-covered).
* Migrate NavLink call sites to primary/secondary variants (GDGT-2469)
- Admin nav: `admin-nav` -> `secondary`
- Entity picker: current level -> `primary`, other levels -> `secondary`;
search results -> `primary`
- Remove the now-unused `default`/`mb-light`/`admin-nav` variant CSS
* Revert NavLink `leftIcon` prop back to `leftSection` (GDGT-2469)
Drop the polymorphic NavLink wrapper and colour the leading icon via the
`.NavLinkSection[data-position="left"]` CSS overrides again.
* Update Loki Snapshots
* Cleanup
* Update Loki Snapshots
* Add Loki reference for the NavLink Overview story (GDGT-2469)
* Update Loki Snapshots
* Address NavLink review comments (GDGT-2469)
- Migrate the remaining `default` NavLink (embedding SimpleDataPicker) to
`primary`, restoring its selected styling
- Render NavLinkButton as a `<div role="button">` (valid inside a NavLink's
`<a>`) with keyboard activation, instead of a `<button>`
* Update Loki Snapshots
---------
Co-authored-by: Metabase Automation <github-automation@metabase.com>
Stop duplicating databricks test jobs.
In https://github.com/metabase/metabase/pull/57148/changes we added
support for databricks multi-catalog feature. In this PR the existing
databricks job was duplicated, so every databricks test ran twice. The
rationale for this (and more broadly the rationale and background for
the entire feature) was never explained.
The code which looks at the multi-catalog flag looks to be tested in
the metabase.driver.databricks-test test. So running the entire driver
suite all over again just with that one flag set differently looks to
be very wasteful.
We have been putting a lot of load on databricks recently with the
extreme levels of backporting we're doing, which has led to periods
during which 99% of databricks jobs time out. The fact that we're
running twice as many tests as we need to is surely a contributing
factor to this.
It would be even better if the multi-catalog functionality were tested
inside the main test suite instead of spun off into a separate job,
but I want to focus on getting it stable first.
Also we split databricks into 3 jobs, but since the bottleneck is in
the DB itself, this was counter-productive. I've brought it back to 2
jobs like the other cloud databases.
* Reject plaintext values in encrypted app-db columns via a strict encrypted read
Make the canonical encrypted read path strict: when MB_ENCRYPTION_SECRET_KEY is set,
a value that is not encrypted is rejected instead of trusted, so a value written
directly via SQL cannot stand in for a properly encrypted one. This relies on the
existing aes256-cbc-hmac-sha512 authentication of encrypted values.
- encryption/maybe-decrypt is now strict; the previous lenient behaviour is
maybe-decrypt-accepting-plaintext. Every existing lenient caller (settings,
secrets, key rotation, the encryption-check sentinel, migrations) uses the
accepting-plaintext variant explicitly.
- models.interface: encrypted-json-out / transform-encrypted-json are the only
(strict) variants. The explorations encrypted-text/EDN transforms are strict too.
- app-db.encryption: the rotation list (encrypted-columns) now also includes the
exploration_query_result columns, so key rotation and encrypt-db cover them.
- A v63 clojure migration (EncryptEncryptedJsonColumns) encrypts any plaintext
values left in the encrypted-json columns (e.g. the v58 credentials SQL backfill)
so strict reads succeed after upgrade.
Secrets and settings keep the lenient path for now and will be handled in follow-up PRs.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* Use explicit arities for maybe-decrypt / maybe-decrypt-accepting-plaintext
Replace the [& args] + argument-sniffing with plain [v] / [secret-key v] arities, which
is unambiguous (the value itself can be a byte array).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* Make secrets strict; move credentials migration to v58; split decrypt by type
- Narrow the re-encryption migration to auth_identity.credentials only (the sole
column the v58 SQL backfill wrote as plaintext) and move it to v58 so it backports
to every affected release. Other encrypted columns are always written encrypted and
covered by encrypt-db, so they need no migration.
- transform-secret-value is now strict (secrets reject plaintext at rest); encrypt-db's
secret loop reads raw + lenient so the enable-encryption-later flow still works.
- Split the decrypt helpers by value type like maybe-encrypt/maybe-encrypt-bytes:
maybe-decrypt / maybe-decrypt-bytes (strict) and their -accepting-plaintext lenient
counterparts, with String/bytes type hints and explicit arities.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* Drop the -accepting-plaintext decrypt variants; use strict + explicit try/catch
The lenient variants silently returned plaintext, which conflates "legacy plaintext
(fine)" with "corrupt/invalid" — and only the caller knows which. Notably the
encryption-check sentinel treated a plaintext UUID as :valid when it should be :invalid.
Each caller now uses the strict maybe-decrypt / maybe-decrypt-bytes and handles the
throw explicitly: the sentinel and the startup check treat a decrypt failure as
invalid; key rotation and the secret loop tolerate plaintext (guarding with
possibly-encrypted-*?); settings and migration reads fall back to the raw value.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* Remove now-unused metabase.test require in encryption-test
Left over after dropping the lenient-decrypt tests.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* Reintroduce -accepting-plaintext decrypt variants for plaintext-tolerant callers
Bring back maybe-decrypt-accepting-plaintext / maybe-decrypt-bytes-accepting-plaintext so the
plaintext-tolerant callers (settings reads, migration reads, key rotation and the secret loop)
call them directly instead of wrapping the strict maybe-decrypt in try/catch.
The encryption-check sentinel and the startup key-check stay strict, but now express it as an
explicit possibly-encrypted-string? guard + lenient decrypt so a plaintext UUID still reads as
invalid (rather than being trusted) without a try/catch. The core app-db column transforms
(models.interface, exploration) remain strict.
Also apply review nit: maybe-blob->bytes uses cond->.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* encrypt-db: also re-encrypt secret-value BYTES columns (stored_result.result_data)
stored_result.result_data (like secret.value) is a mi/transform-secret-value byte column, read
strictly via maybe-decrypt-bytes. do-encryption only re-encrypted secret.value, so a result_data
written as plaintext before a key was set stayed plaintext, and once a key is set the strict read
throws (500 on GET /api/exploration/query/:id).
List both byte columns and loop over them. Rename encrypted-columns -> encrypted-string-columns and
add encrypted-bytes-columns, splitting the two lists by wire type (string vs bytes).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* Address review: stream re-encryption; fail instead of double-encrypting undecryptable values
- `maybe-decrypt-accepting-plaintext` / `maybe-decrypt-bytes-accepting-plaintext` now throw when a value looks
encrypted but cannot be decrypted with the current key, instead of logging and returning it unchanged. Returning it
let the key-rotation path re-encrypt it into `encrypt_new(encrypt_old(x))`, permanently unrecoverable. They still
return genuinely-plaintext values as-is.
- Re-encryption streams rows via `reducible-select` (the string path and a new bytes helper) so a large column such as
`stored_result.result_data` is not held in memory all at once, and catches the strict decrypt to clear (clearable
columns) or abort rather than double-encrypting.
- Callers that must tolerate the throw handle it: the `encryption-check` sentinel and `check-encryption` treat a
decrypt failure as a wrong key; the Setting after-select degrades gracefully on read; `encrypted-json-out` (migration
reads) returns the raw value on failure.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* Fix rotate-encryption-key test for the lenient Setting read
The Setting after-select degrades gracefully on an undecryptable value (returns the raw ciphertext) rather than
throwing, so reading a wrong-key setting yields ciphertext, not a thrown exception. Assert not= plaintext instead of
thrown? for that read; the strict Database/User columns still throw.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* Encrypt api_key hashes at rest (#80902)
---------
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
Validation transpiled with `identify=True` which quotes every identifier and
freezes its case, so a bare `subtotal` on Snowflake stopped resolving to
`SUBTOTAL`. Keep the LLM's own quoting instead, and only quote reserved words,
folded first so quoting can't change what they match.
* Quote ClickHouse impersonation role as a single identifier
Stop splitting the role attribute on commas so the whole value is one
role identifier. This removes multi-role activation via a comma-separated
attribute.
* Escape backslashes when quoting the ClickHouse role identifier
A trailing backslash would otherwise escape the closing quote of the
role identifier. Escape backslashes before doubling interior quotes.
* Delete e2e tests tagged @skip
Removes 53 skipped blocks (86 test cases), the oldest of them skipped since
2020, plus the fixtures and helpers they left unreferenced. Three spec files
held nothing but skipped tests and are deleted outright.
The @skip tag itself still exists in the workflows and the matrix builder.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* Drop the @skip grep tag from the e2e workflows
No spec carries the tag anymore, so the exclusion is dead weight. With
`-@skip` gone the tagged-run expression no longer needs format() to append
it, and collapses to the input or the default tag set.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
* BOT-2010: Post Metabot's Slack answer as a markdown block
Channel answers went in an `mrkdwn` `section`: no table syntax, and a
3000-char cap that cut them short. They now go in a `markdown` block at 12000.
- `final-text-blocks` emits `{:type "markdown"}`, and the answer is elided to
`markdown-text-limit` (12000) rather than the section limit
- the prompt switches bold and strikethrough to `**text**` and `~~text~~`;
`*one*` renders italic and `~one~` does not render at all, so the old
dialect would have silently lost both. Link syntax is untouched --
`<url|label>` does render inside a `markdown` block (measured)
- the test harness models Slack's real rejection for each block type
(`invalid_blocks` for `section`, bare `msg_too_long` for `markdown`), and
charges the markdown budget across the payload as Slack documents it.
Without this the answer would no longer be checked by anything and the
limit assertions would pass vacuously
Verified against the live API: a `markdown` block takes exactly 12000 chars,
Slack expands one into several as it renders -- a heading becomes a `header`,
a rule a `divider`, a table a `table` -- and that expansion counts against
the 50-block ceiling, which is why the prompt still bans headings and rules.
Co-authored-by: Paolo Rascuna <1936871+prascuna@users.noreply.github.com>
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
* Restrict markdown link URI schemes in rendered notifications
Only http/https/mailto links and images survive markdown rendering for
email/Slack notifications; other schemes and unparseable URIs are
dropped instead of being passed through (or, for malformed URIs when a
site URL is set, throwing mid-render).
* Diff pulse recipients by id or email in the subscription-permission check
Key the added-recipients diff the same way pulse-channel keys
recipients (user recipients by :id, external ones by :email) so
external-address changes are covered by the check too.
* Allow inline base64 images in notification markdown
The app's markdown renderer accepts data:image/...;base64 URIs
(DATA_IMAGE_URI_PATTERN); let notification rendering accept the same
pattern so text-card images look the same in email.
* Cross-link DATA_IMAGE_URI_PATTERN to its backend counterpart
* Share the data-image URI pattern between backend and frontend
Lift the pattern into metabase.util.markdown.image (cljc), compile it
into the frontend bundle via shadow-cljs, and re-export it as
DATA_IMAGE_URI_PATTERN so both renderers accept the same inline images
by construction.
* Apply resolve-uri to Slack images too
* Render a dropped link-ref as its text alone instead of restoring brackets
* Compute the action model's write permission from the API, not the metadata mirror
* Read the embedding picker's source card from the API, not the metadata mirror
Clicking the database row in the TablePicker before its expand handler is
wired drops the click, so the schema fetch never fires and cy.wait("@getSchema")
times out with "No request ever occurred." Gate on the expand toggle's
aria-expanded state (collapsed -> click -> expanded) so the schema request
reliably fires before selecting the database — same pattern already proven in
the sibling publish/unpublish test.
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
A transform's :query source is auth-checked against its source database but leaves out some entities named by the query. This checks those too by leveraging on QP's check-query-permissions*.
The shared verifyTablePreview helper opened the column-description
hovercard with a single one-shot realHover(). That hovercard opens on a
delayed openDelay timer, so a lone realHover can miss it: after
cy.wait("@dataset") the preview grid re-renders on a microtask and
Chrome v133+ headless hit-tests CDP mouse events differently, either of
which drops the hover and cancels the open timer. hovercard() then only
retries the query, never re-hovers, so the card stays permanently
absent for that attempt.
Replace realHover() with the re-queried synthetic mouseenter+mouseover
dispatch already proven in data-model-shared-1's local verifyTablePreview,
so the events land on the post-render DOM node.
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* BOT-522: Seed the Slack agent loop with persisted conversation state
The Slack loop wrote `metabot_message.state` on every turn but always started
`run-agent-loop` with `:state {}`, so it never read back what it had persisted —
unlike the in-app path, which seeds from `conversation-state`.
This makes sure we feed the state back into the agent loop, filtering out errored messages.
Co-authored-by: Paolo Rascuna <1936871+prascuna@users.noreply.github.com>
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
This feature was not following the bug-reporting-enabled gate on the BE previously, and we now enforce using the session user when making a non-anonymous report.
As this is a endpoint that has been in the wild for some time, we'll first report any non-compliance first while overriding with the desired behaviour of only using the session user instead of breaking the API by returning an error
* Read the permissions tree's databases from the API, not the metadata mirror
* Record each edit's database in permissions state instead of holding subscriptions
* Take each database field from the endpoint that returns it
Two e2e failures traced to the same cause: no single endpoint returns
everything the permissions tree reads.
GET /api/database is the only endpoint that hydrates router_user_attribute
(warehouses_rest/api.clj:361 and :501). Reading the database from
getDatabaseMetadata therefore dropped it, and the "(Database routing enabled)"
breadcrumb stopped rendering.
GET /api/database/:id/metadata is the only endpoint that returns tables. The
group page's database list passes listDatabases entries, which have none, so
hasPermissionValueInSubgraph found no children and the blocked-child
confirmation never opened.
The mirror had merged both, and held them for the session. getPermissionsDatabase
now merges them explicitly, taking router_user_attribute from the list and tables
from the metadata response. The tables are kept in permissions state rather than
read back from the request's cache entry, which is dropped as soon as the admin
navigates away from that database.
databases.unit.spec.ts pins the merge with fixtures shaped like the two real
responses, which is what the previous specs could not catch: they seeded one
complete database, making the two sources indistinguishable.
like cards, the actions table has a top level column for database
id. But we can use the canonical value on the action's query and let the
db value be for bulk lookups and as an index. But we make sure to keep
it in lockstep as we go.
* Replace legacy button CSS with Mantine buttons
* Do not click the disabled map update button in the e2e test
* Polish admin header and add-row wrapping
* Use static view-section paddings instead of clamp
* Refuse analytics-dev DB connection/creation unless analytics-dev-mode is on
An :is-audit-dev database is a handle onto the app-db; it carries no real
connection details of its own. Reusing the app-db datasource for it is only
ever valid while analytics dev mode is enabled. Previously the connection
router only *skipped* the datasource reuse when the mode was off, silently
falling through to build a credential-less pool against the wrong host
(surfacing in CI as `FATAL: role "runner" does not exist`).
Make the invariant explicit and fail loudly at every site:
- db->pooled-connection-spec throws for an :is-audit-dev DB when the mode is off
- can-connect? throws likewise instead of reporting the DB as connectable
- create-analytics-dev-database! refuses to create the DB when the mode is off
Tests create/connect these DBs, so they now run under analytics-dev-mode=true.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BjQcjQx7NPGr3Rpf6QaG4b
* Update is-audit-dev routing test for the throw-on-invariant-violation behavior
The connection layer now throws for an :is-audit-dev database reached with
analytics-dev-mode off, rather than silently declining to route it to the app
DB. Update the test's dev-mode-off case to assert the throw.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BjQcjQx7NPGr3Rpf6QaG4b
---------
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
* Deflake dependency broken list spec
visitBrokenDependencies did a bare cy.visit with no wait, so tests
immediately queried dynamically-loaded list items with only Cypress's
default 4s timeout - flaky when the breaking-dependencies API is slow.
Mirror the sibling visitUnreferencedEntities helper: intercept the
/api/ee/dependencies/graph/breaking request, wait on it, and assert the
list is visible before the test interacts with it.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* Wait for the test's own broken entities before asserting
The prior deflake waited for the page's breaking-dependencies request, but
the underlying data still raced: setup gated on a bare count
(nodes.length >= 3), which the instance's pre-existing Usage Analytics
breaking dependencies satisfy before the test's transform-table break is
analyzed. The page then rendered only Usage Analytics rows and the test's
"Test Transform Table" / question / model were absent.
Gate setup on the specific expected entities by name instead of a count,
and align the breaking helper to include personal collections (matching its
sibling waitForUnreferencedEntities) so the personal-collection question is
visible to the check.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* checkbox redesign
* small fix
* fix disabled checkbox visual size
* disallow checkbox labelPosition
* add new loki tests
* update loki snapshots
* fixup! update loki snapshots
* fix e2e specs relying on checkbox Icon-check/Icon-dash classes
* checkbox card border
add border-highlight token, use it for selected card
* disallow checkbox size
single 16px size; xs/sm were both 16px and md was unused
* disallow checkbox card children
* drop redundant role from checkbox icon
* checkbox card body via mantine text/stack
* forwardRef checkbox card
* drop border on disabled checkbox card
* drop disabled checkbox border ring
alpha border over alpha bg stacked into a visible ring
* hide unchecked card indicator icon
mantine hides indicator icon via color:transparent; .icon color override defeated it
* update colors loki snapshot
The java.time classes grouped under ::->absolute-datetime and ::->time were derived in
Clojure's global hierarchy, sharing one mutable namespace with model keywords, hook traits and
every other subsystem -- and registering concrete JDK classes there is especially easy to
collide with.
Move them into a hierarchy var private to this namespace, mutated through derive! -- the same
shape as metabase.events.hierarchy, kept in-file because add-type-info is private and is the
only reader.
The prefer-method calls against Object still apply: class-to-class isa? goes through Java
assignability rather than any keyword hierarchy, so Object continues to match every temporal
class regardless of which hierarchy is in use.
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
* better development against audit db
* Bump :discouraged-namespace kondo ratchet to 81
connection.clj now requires metabase.audit-app.core with a
:discouraged-namespace ignore (audit-db dev support), so the budget
needs +1.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BjQcjQx7NPGr3Rpf6QaG4b
---------
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
* Add hierarchy visualization utilities
* Normalize hierarchy visualization label newlines
* Preserve ordered hierarchy traversal in visualizations
* Format hierarchy utility docstrings
* Move hierarchy visualization utilities to dev
* Move hierarchy visualization tests alongside the dev namespace
Also sort nodes unreachable from a root, so malformed hierarchies still
render in a stable order, and mark the hierarchy tests parallel.
* Keep hierarchy visualization tests discoverable by the test runner
The runner skips `dev` directories and non-`metabase.*` namespaces, so a
`dev.*` test never runs. Follow `metabase.core.modules-test` and
`metabase.core.kondo-ratchet-test`, which test dev namespaces from a
`metabase.*` test namespace, and say so in the namespace docstring.
* List hierarchy nodes without drawing a tree
`nodes` returns every tag as a flat vector, leaves-to-roots by default so a node
precedes its parents, or alphabetically on request. Ordered hierarchies break ties
by derivation order and the rest fall back to `pr-str`, both of which `hierarchy->graph`
already worked out, so there is no separate code path for the two kinds.
A test pins the topological order to `ordered-hierarchy/sorted-tags` so the dev output
cannot drift from the order the runtime uses.
* Fix ordered hierarchy ordering and validation
* Format hierarchy utility docstrings
* Mark ordered hierarchy tests parallel
* Return nil from first-common-ancestor when tags share no ancestor
Passing a tag with no ancestors as the second argument made the lookup call nil as a
predicate, so disconnected roots and tags outside the hierarchy threw an NPE instead of
returning nil.
* Reject children declared after a tag's first occurrence in any basis
The check only ran for nested vectors, so a tag introduced in one basis could pick up
children in a later one. Hoisting it into derive-basis applies the same rule to every
top-level basis.
* Ratchet down kondo ignore budgets
---------
Co-authored-by: Metabase Automation <github-automation@metabase.com>
The notification create/update endpoints let a handler attach an existing
ChannelTemplate by id, or carry an inline template that creates, overwrites,
or deletes one, but didn't apply the permission check used when writing a
ChannelTemplate directly. Gate any such template write on
mi/can-write? :model/ChannelTemplate, mirroring the existing inline-channel
check. Seeding and the pulse->notification migration go through the model
layer directly, so they're unaffected.
POST /api/action and PUT /api/action/:id now require the current user to
have ad-hoc native query permission on the target database when the
submitted dataset_query is native.
* Load the enterprise admin pages through plugin route slots
* Load the tenant collection pages through plugin route slots
* Resolve every plugin route slot in a test
* Name the chunks the plugin route slots load
Check revert perms against the table of the restored definition
For Segment and Measure reverts, table_id is re-derived from the
definition on update, so check write perms against the source table of
the restored revision's definition when it has one, falling back to the
revision's stored table_id otherwise.
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
* Default undeclared endpoint params to an empty map schema
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* Ignore unparsed request bodies instead of declaring them
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* Pass multipart params as the request body of multipart endpoints
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* Declare query and route params read by dashboard and database endpoints
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* Ignore the kebab-case linter for newly declared snake_case query params
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
* Anchor remote-sync conflict-modal test on the async conflict lifecycle
* Reword test comments per writing conventions
* Trim comments and move timeouts to request/test-level options
* Use navigation helper when navigating to the transforms page
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
* Fail fast on terminal sync task statuses
---------
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
* Fix detached-DOM flake in upload table deletion e2e test
Re-query the upload-tables-table container on each assertion instead of
caching it via within(). The table re-renders when getUploadTables
resolves, detaching the node within() captured, so a retrying query
against the stale subtree got stuck on an intermediate render.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* Appease the linter
---------
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-authored-by: Nemanja <31325167+nemanjaglumac@users.noreply.github.com>
* Key cached session tokens on the app db
with-empty-h2-app-db! swaps in a different application database, and this cache was keyed on the username
alone. A token minted against the previous database names a session that does not exist in the new one, so
the request 401s, and the token minted inside leaks back out when the original is restored. The retry in
client-fn cleared the whole cache and tried again, which papered over it often enough to look like flake:
"Failed to authenticate X after three tries".
Key it on the app db as well, like every neighbouring cache does via memoize-for-application-db.
databases-list-can-upload-test and dismiss-spinner-test go from failing to passing; both were in the CI
failure list, the former across 12 jobs.
* Track globally-created test users per application database
with-empty-h2-app-db! swaps in an empty app db, where the users another database created do not exist.
`created-globally?` was a boolean, so it stayed true across the swap and user->id kept handing back ids from
the previous database. Authenticating then inserted a Session for a user that was not there:
Failed to authenticate :crowberto after three tries: Referential integrity constraint violation:
FK_SESSION_REF_USER_ID: PUBLIC.CORE_SESSION FOREIGN KEY(USER_ID) REFERENCES PUBLIC.CORE_USER(ID) (1)
Track the set of app dbs that have them instead. The memo beside it was already keyed this way via
memoize-for-application-db; only the flag was not.
databases-list-can-upload-test failed 8/8 before this and 0/8 after; the whole namespace now passes.
* Say what the token cache holds
Update E2E test timings with averaged data
Collected timing data from 7 days of test runs.
All specs with timing data were included (simple averages).
🤖 Generated automatically by update-e2e-timings workflow
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: GitHub Actions <actions@github.com>
Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: iethree <30528226+iethree@users.noreply.github.com>
Reject impersonation role attribute equal to the driver default-role sentinel
connection-impersonation-role now rejects a role attribute that (case-insensitively)
equals the driver's default-database-role sentinel (e.g. NONE / DEFAULT), which would
otherwise emit SET ROLE NONE and drop the connection to the base warehouse credential.
* Resolve card UI parameter fields from param_fields for saved cards
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* Resolve parameters against the query for questions opened from a dashboard
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
* Remove unused data from embed and public endpoints
Public and static-embed card/dashboard/document endpoints now return a
minimal placeholder dataset_query (a blank native query against the same
database) instead of selecting individual query keys. The frontend only
needs the database id here. Parameter combining and :param_fields
hydration in public-card now run before the card is trimmed since both
read template tags from the full query.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* Add tests for embed card and dashboard dataset_query
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* Assert strictly on dataset_query in public and embed tests
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* Handle native queries in Table viz
* Fix column shadowing in HeaderCellWithColumnInfo
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* Use param_fields for public question parameters
Public and static-embed card payloads carry param_fields instead of the
real query, so PublicOrEmbeddedQuestion now builds UI parameters from
them via getCardUiParametersFromParamFields, mirroring the saved-mode
dashboard path. The query-based helpers are renamed to the FromQuery /
FromParamFields scheme on both the card and dashboard sides, and the
ParameterWithTarget casts are dropped now that getParameterTargetField
accepts an optional target.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* Use param_fields for embed setup card parameters
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* Use param_fields for guest embed SDK question parameters
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* Route card UI parameters by param_fields presence
Restores getCardUiParameters as the entry point: it delegates to
getCardUiParametersFromParamFields when the card carries param_fields
(public and static-embed payloads) and to the query-based variant
otherwise. The SDK SQL parameter surfaces use it since they serve both
guest and authenticated contexts. Also reverts the dashboard helper
renames back to getSaved/getUnsavedDashboardUiParameters.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* Fix preview embed card param values
The preview-embed card values and remapping endpoints get their card via
card-for-unsigned-token -> public-card, which now replaces the query with
a blank native placeholder, so field-filter parameters could no longer be
resolved and the response schema rejected the nil result. Bind
qp.perms/*param-values-query* around the card fetch, the escape hatch
remove-card-non-public-columns already honors, matching the public
sharing values endpoints.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* Add preview embed test for remapped parameter fields
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* fix
* vscode fix
* Backfill template tag parameters in param_fields UI parameters
GET /api/card/:id returns parameters as stored, unlike the public and
embed endpoints which combine them with template tags, so
getCardUiParametersFromParamFields now derives parameters via
getParametersFromCard like the query-based variant always did. Guest
embed payloads are unaffected: their parameters arrive already combined
and the blank query derives nothing.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* Use getCardUiParameters dispatcher in SDK load-question
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* Guard applyTemplateTagParameters by param_fields presence
Cards whose payloads carry param_fields have parameters already combined
with template tags by the backend, so re-applying from the query would
wipe them. Keying the guard on the card payload keeps it in sync with
how the card was loaded, replacing the isGuestEmbed flag checks.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* Use getCardUiParameters everywhere and build test queries via Lib
Question.parameters and the query builder call sites go back to the
getCardUiParameters entry point, which now routes by param_fields
presence, leaving those files unchanged relative to master. The
template tag fallback test builds its query with
Lib.createTestJsNativeQuery instead of an inline legacy map.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* Make getCardUiParameters the only entry point
The FromQuery and FromParamFields variants are private to cards.ts now;
all consumers call getCardUiParameters, which routes by param_fields
presence.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* Build preview embed remapped-fields test query via lib
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* Restore null fallback in useAvailableParameters
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* Fix
* Preserve query type in stripped public and embed card queries
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* Add placeholder aggregations to stripped MBQL queries
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* Keep parameter template tags in stripped native queries and revert FE workarounds
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* Export parameter-template-tag? via import-vars
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* Add test for :param_fields hydration on GET /api/card/:id
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
* Clear All tenant users metabot permissions when switching to group-level mode
Switching metabot permissions to group-level only wiped the All Users magic group's rows.
Under tenants the simple mode also writes to All tenant users, and since resolution takes the
most permissive value across a user's groups, a leftover :yes there overrode every group-level :no.
POST /advanced now clears both magic groups; DELETE /advanced keeps both instead of dropping the
tenant toggle. The confirmation dialog names both groups when tenants are on.
Fixes#80394
* Backfill the tenant permission cleanup and make both mode-switch dialogs tenant-aware
Instances that switched to group-level mode before the fix still carry the seeded all-yes rows
for All tenant users, so a migration clears them wherever metabot-advanced-permissions is true.
The disable dialog now says both magic groups keep their access under tenants, and both dialogs
use the "All internal users" name the permissions UI shows when tenants are on.
* Explain why the tenant-permission migration only reads the setting row
* Move the tenant-permission migration to 063 for the single backport
* Move the tenant-permission migration to 061 so it backports to every affected release
* Scope metabot permission resolution to the groups the current mode shows
Resolution took the most permissive value across every group a user belongs to regardless of
mode, so any row on a group the admin UI hides -- the All Users and All tenant users rows in
group-level mode, the seeded data-analyst rows and any API-written group in simple mode --
silently overrode every visible :no. The write-site fixes (deleting rows on POST /advanced, the
backfill migration gated on the setting row) only covered the paths that went through the
endpoint; PUT /api/setting, serdes import and PUT / on a hidden group all bypassed them.
Filter in the resolver instead: simple mode only counts Administrators, All Users and All tenant
users; group-level mode counts everything but those two default groups. Hidden rows stay in the
table and come back into force on switching modes, so POST /advanced no longer deletes them and
DELETE /advanced genuinely keeps them, which also restores the simple-mode values after a round
trip instead of leaving the default groups at all-no. With leftover rows inert the migration is
no longer needed.
* Leave the mode-switch dialog copy as it was
* Refuse the metabot permission mode switch while an env var forces the setting
The setter writes the row regardless, but the env var keeps winning on read, so DELETE /advanced would
delete every group's rows and leave the instance in group-level mode with nobody but admins allowed in.
* Flip the metabot permission mode before dropping the group-level rows
Deleting first meant a failed setting write left group-level mode in place with its permissions already gone. In the new order a failed delete only leaves rows that simple mode ignores.
* Say in the docstrings that resolution only sees the active mode's groups
* Clear the default-group rows again when switching to group-level mode
Preserving them so they returned on a switch back was a silent widening: the 061 migration seeds every default group all-yes, so a round trip through group-level mode handed everyone AI access again, including the tenant rows that caused this bug.
Both mode-switch dialogs promise removal, and nothing in the docs describes a mode remembering its old configuration, so the modes now delete the rows the other one owns, as before. Resolution stays scoped to the active mode's groups: that is what makes rows created by other write paths, and the rows already sitting on instances in the field, harmless without a migration.
* Switch the metabot permission mode in one transaction
Ordering the two writes only chose which half-applied switch an admin got: the mode changed while the rows did not, or the rows went while the mode stayed. The setting write goes last inside the transaction, since it updates the settings cache in place and a rollback would leave the new value there.
* Reload the settings cache when a mode switch rolls back
The setting write updates the cache in memory, which the rollback does not undo, so a commit failure would leave this node serving a mode the database does not have.
* Cover the settings cache in the mode-switch rollback test
The redefined setter now writes the setting for real before throwing, so the test fails if the rollback leaves the new mode in the cache.
* Extract the mode's group list to one definition
Both the resolution query and the mode-switch endpoints had their own copy of the
Administrators / All Users / All tenant users list, and read as opposites: one
selected with :in, the other deleted with :not-in. They now share
visible-groups-clause, with hidden-groups-clause defined as its complement.
* Improve Metabot permissions documentation
* Address Metabot permissions review feedback
* initial deepseek support
* [BOT-1899] Port DeepSeek onto the LLM provider connection registry (#80100)
Wire DeepSeek into the LLM provider connection registry
DeepSeek arrived as an adapter reading its own credential settings, which is how
providers worked before the connection list. Make it a connection type like the
rest:
- `deepseek` joins `metabase.llm.provider`'s registry with the api-key and
base-url fields the admin connection form renders itself. No key prefix:
DeepSeek keys carry the same `sk-` an OpenAI key does, so matching on it would
claim OpenAI keys and reject nothing.
- `llm-deepseek-api-key` / `-api-base-url` become the type's single-provider
environment variables, reading and writing through the connection they
configure like every other provider's.
- The adapter takes `:credentials` off the request instead of reading settings,
so two DeepSeek connections can hold different keys.
- The frontend mirrors the new type name and gets a DeepSeek mark.
The app-db migration that folds the old per-provider settings into the
connection list is a frozen snapshot of the registry as it stood then, and stays
untouched: no released version wrote a DeepSeek setting, so there is nothing to
adopt, and a rollback drops the connection because the versions it rolls back to
cannot serve DeepSeek either. A test pins that.
Regenerated the settings docs, and listed DeepSeek among the providers you can
bring a key for.
* revert docs changes
* review feedback
---------
Co-authored-by: Maksym Yakubych <maksym@metabase.com>
Co-authored-by: Maksym Yakubych <undrfined@gmail.com>
* Fix flaky native database source selectDatabase detach
The database picker popover opens automatically and re-mounts its rows as the database list finishes loading, which could detach the target row while Cypress waited for it to become actionable ("elements ... disappeared from the page"). Anchor the click in selectDatabase behind a visibility assertion so Cypress re-queries the popover chain on detach and clicks a freshly attached, settled row.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* Force-click the native database picker row to avoid detach
The database picker popover opens automatically and, as GET /api/database
resolves, its onQueryStarted hook re-writes shared metadata. That hands the
selector a fresh databases array reference and re-mounts the AccordionList rows.
The previous visibility-anchored click still entered Cypress's actionability
wait, during which the target row detached ("elements ... disappeared from the
page"). Click the row with { force: true } so Cypress fires the moment the row
is found and visible, skipping the actionability wait window in which the
re-mount can detach it.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-authored-by: Uladzimir Havenchyk <125459446+uladzimirdev@users.noreply.github.com>
* BOT-1606: Truncate over-long Metabot answers instead of going silent
`channel/final-text-blocks` put the whole answer into a single Slack `section`
block. Slack caps a section's `text.text` at 3000 characters, so any longer
answer came back as `invalid_blocks` ("must be less than 3001 characters") and
the thread simply went quiet -- no answer, no error, nothing to retry.
Elide the answer at the limit, and when that actually cuts it, follow it with a
`context` block saying so and linking to the full answer in Metabase. The notice
is deliberately kept out of the message's `:text`, which `thread->history`
replays back to the model as the assistant's own words.
The answer stays a single mrkdwn `section` block, as before. Answers that already
fit are byte-for-byte unchanged: `u.str/elide` returns its input untouched.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* BOT-1606: Address review comments on the truncation notice
Works through appleby's review nits on #80105:
- Docstring summary for `truncation-notice` is one sentence (continuation
aligned to 3 spaces, matching the rest of the file).
- The notice now names the limit, so a reader can judge how much to trim:
"This answer was longer than 3000 characters."
- Adopted the suggested "...and try again." ending.
- Added a debug log on the truncation path, recording the answer length
against the limit.
- Moved the shared `oversized-answer` fixture into `slackbot.test-util`;
`channel_test` and `api_test` had identical copies.
- `slack-section-text-limit` now derives from the production constant
rather than restating it, which makes the drift test redundant.
`log/debugf` evaluates its args only once the level is enabled, so the
debug log was verified through the REPL with log capture on rather than
through the suite, which cannot reach it.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Ew4fCZUQC2fu4MZ7JamMLm
* BOT-1606: Cap the visualization title so Slack accepts the message
The truncation shipped so far only guarded the answer block. The error in
BOT-1606 names `/blocks/1`, which is the visualization title block, not the
answer: `format-viz-title` embeds a `/question#<base64-query>` link, and for
a large query that base64 alone runs past Slack's 3000-character section
limit, so `chat.postMessage` rejects the whole message.
Reproduced with a native query of the size Metabot routinely writes: a
7,119-character SQL string yields a 9,770-character link and a 9,820-character
title block, and Slack returns the reported error verbatim.
A cut URL is a broken URL, so drop the link rather than truncate it and keep
the title; with no title left, `viz-output->blocks` already falls back to
"Query results". `section-text-limit` becomes public for this -- streaming
already required the channel namespace, so no new dependency.
Both this and the answer truncation log at `info` rather than `debug`:
`metabase.slackbot` inherits INFO from log4j2.xml, so a debug line there is
invisible on a default instance. Link drops get their own counter rather than
reusing `responses-truncated`, which counts responses; a long answer and a
long query link are different problems and worth telling apart.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Ew4fCZUQC2fu4MZ7JamMLm
* BOT-1606: Fall back to plain text when Slack rejects the blocks
`send-channel-response` logged a failed `chat.postMessage` and stopped
there, so a user whose message Slack refused saw the "is doing science..."
status and then nothing. `send-dm-response` already retries with a
plain-text reply; do the same here, with the same wording.
A plain-text reply carries no blocks for Slack to reject, so it survives
whatever made the block message fail -- including causes we have not
anticipated and so cannot cap the way the answer and viz title are capped.
Add `responses-undeliverable`, incremented from both paths when the
fallback fails too. It counts only total delivery failure: a response the
fallback rescues is deliberately not counted, so the metric answers "how
often did a user get nothing at all", which is the alertable question.
Left the error-cleanup fallback in `send-dm-response` uncounted -- that
path is already reporting a thrown exception, and its fallback carries an
error notice rather than an answer.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Ew4fCZUQC2fu4MZ7JamMLm
* BOT-1606: Cap alt_text, and stop miscounting title-only elisions
Follow-ups from a Clojure review of the branch.
Slack caps an image block's `alt_text` at 2000 characters, not the 3000 it
allows a section block, and `viz-output->blocks` passed the raw title
straight through. A title of 2001-3000 characters therefore cleared
`format-viz-title` untouched -- comfortably inside the section limit -- and
then failed `chat.postMessage` with `invalid_blocks`: the same silence this
branch exists to fix, reachable through the function it hardened. Cut the
title with `limit-chars` rather than `elide`, since alt text is plain and an
ellipsis would only eat into the budget.
`format-viz-title` also reported a dropped link whenever the rendered title
was over the limit, including when there was no link at all -- logging
`link_length=0` while claiming a drop, and incrementing `viz-links-dropped`
for something that never happened. Both cases still elide the title; only the
reporting now branches.
Docstrings: several first lines ran on into the next line, which
https://guide.clojure.style/#docstring-summary asks not to happen. Also
`format-viz-title` claimed to return nil only when given neither a title nor
a link, though it also returns nil for a link too long to keep -- as the
branch's own test asserts -- and `slack-section-text-limit` restated the
sentence from the var it aliases.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Ew4fCZUQC2fu4MZ7JamMLm
---------
Co-authored-by: Paolo Rascuna <1936871+prascuna@users.noreply.github.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
* Fix flaky alerts toggle e2e: re-query preview iframe for Alerts click
The 'toggles alerts for question when email is set up' spec cached the
embed preview iframe body via within() and chained the Alerts button
visibility assertion straight into the click. Toggling the sidebar
options re-renders the embedded question, so the cached body and the
found ActionIcon detach mid-click ('the page updated while this command
was executing'). Re-query the iframe body fresh and split the visibility
assertion from the click so Cypress retries against a stable element.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* Give re-render-dependent alerts lookups a longer timeout
Iteration 1 removed the mid-click detach by re-querying the preview
iframe body, and 49/50 burn-in runs passed. The residual failure is a
different flake: toggling a sidebar option re-renders (and re-runs the
query of) the embedded question inside the already-loaded iframe, and
that round-trip occasionally exceeds Cypress's default 4s command
timeout, so findByRole for the "Alerts" button times out with only a
"group" role present in the iframe body. Give the Alerts button and
alert-modal lookups that depend on a re-render a 15s budget so Cypress
waits the slow re-fetch+render out instead of asserting mid-render.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-authored-by: Sébastien <sebastien@metabase.com>
The "should update existing sandboxes when changing column selection"
test raced in two ways:
- It waited on @updatePermissionsGraph with the default 5s request
timeout, but the "Next" click first runs several sequential metadata
fetches (list tables, list databases, permissions-graph revision)
before issuing that PUT, so under load the request can appear later
than 5s and the wait times out. Give it a 15s timeout.
- It asserted H.undoToast().should("not.exist") with no positive anchor,
which can pass against an unrendered page. Anchor it on the positive
success signal ("Select data" step marked complete) first.
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-authored-by: Sébastien <sebastien@metabase.com>
* Add connection and read timeouts to python-runner requests
* Tighten timeout docstrings
* Cancel the runner on a client timeout, retry transient log-poll timeouts
Review follow-ups on the python-runner timeouts:
- a client-side timeout on /execute now fires /cancel, so the runner
stops working on a run nobody is waiting for
- a read timeout while polling /logs is retried instead of ending the
poll loop for the rest of the run
- use u/seconds->ms instead of inline math
PUT /api/notification/:id accepted a different id from what was in the URL which diverges from normal PUT behaviour. This fix aligns it with how we've handled PUT updates in this codebase by only handling allowlisted params for the update.
* Deflake SDK question data-picker target-collection test
The "change target collection to a different entity id" test blindly
pressed Escape then clicked "Pick your starting data" to open the data
picker. In the embedding SDK the DataStep always renders the data picker
and auto-opens it when no source table is selected, and that open state
settles asynchronously after the collection re-render. When the popover
auto-opened after the blind Escape, the follow-up click toggled it back
closed, so H.popover() never appeared and the Orders lookup timed out.
Wait until the popover is actually gone after Escape before reopening it
from a known-closed state, and use visibility anchors instead of exist,
removing the toggle race.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* Deflake SDK question data-picker target-collection test
The embedding data picker uses isInitiallyOpen={!table}; with questionId="new"
no source table is ever selected, so the picker auto-opens on every re-render and
stays open. The prior approach toggled it (esc to close, click to reopen), which
raced the async auto-open: sometimes the click toggled the just-opened popover
shut, sometimes esc failed to close it at all. Interact with the already-open
popover directly to remove the toggle race.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Update E2E test timings with averaged data
Collected timing data from 7 days of test runs.
All specs with timing data were included (simple averages).
🤖 Generated automatically by update-e2e-timings workflow
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: GitHub Actions <actions@github.com>
Reaction types registered by defreaction derived from :metabot/registered-action in Clojure's
global hierarchy, sharing one mutable namespace with model keywords, hook traits and every
other subsystem. Move them into a hierarchy var private to this namespace, mutated through
derive! -- the same shape as metabase.events.hierarchy, kept in-file because defreaction is
the only writer and known-reaction-types the only reader.
This also narrows the schema-rebuild watch. It was attached to #'clojure.core/global-hierarchy,
so it woke on every derive anywhere in the application and compared descendant sets to decide
whether anything relevant had changed; now it only fires for reaction registrations.
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
The legacy-cookie alias :search.engine/fulltext -> :search.engine/appdb lived in Clojure's
global hierarchy, sharing one mutable namespace with model keywords, hook traits and every
other subsystem. Move it into metabase.search.hierarchy, mirroring metabase.events.hierarchy:
a defonce var holding a hierarchy map, mutated through derive!.
All thirteen engine multimethods now dispatch on that hierarchy, and known-engine? walks it
rather than the global one.
The hierarchy has a single writer inside the search module, so it exposes only derive! and
ancestors -- the operations that are actually used.
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
The model persistence test clicked the global and per-database persistence
toggles without waiting for their requests to complete, then immediately
POSTed to persist a card. The card-persist endpoint returned 400
("Persisting models not enabled for database") when it raced ahead of the
per-database setting, and the per-database toggle could fail to render when
the database page booted from a stale global setting.
Intercept and wait on both persist-enable requests before proceeding.
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The "unsubscribe from an alert when the user has not created it" test
clicked the Unsubscribe button immediately after a bare findByText for
the confirm modal's title. That query resolves the instant the title
mounts, while the Mantine modal is still animating in and its title
transiently covers the button's click point, so cy.click() intermittently
failed with "covered by another element".
Anchor on the title being visible first so the click is deferred until
the modal's entrance transition has settled.
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Re-running mq init in a REPL (dev server startup plus test init, or a
retried :mq test-init step) threw for the exploration queues:
```
Queue :queue/exploration-plan is already registered with different config.
```
This is because of an attempt to catch cases where we were declaring
the same queue multiple times with different config. However, this was
subtly broken in two ways:
- first, there are two phases to queue registration: declaration, and
registration. Declaration just means "write down what queue X's config
is." Registration means actually executing that. Our check ran on
*registration*, not on declaration. So if there were two different
calls to `def-queue!` for the same queue, the first one would declare
its `def-queue*`, then the second would overwrite it, and there would
be no meaningful check left.
- second, doing registration twice would always throw, because some
queue configs contained functions, and declaring the function twice
resulted in non-equal results.
Instead, we want to just check that we're not *declaring* the same queue
multiple times in different namespaces. We allow declaring the same
queue twice in the same namespace for REPL convenience - technically a
loophole, but this is similar to a check we have for `defsetting`s and
it doesn't seem to have been a problem.
* Pin exploration auto-binning to an explicit bin width (UXW-4391)
Exploration charts fan one metric/dim query out into several
segment-filtered variants that share an x-axis. With {:strategy :default}
binning, the QP re-derives the bin width per variant from that variant's
own filters, so a segment that range-filters the binned column (e.g.
"Highly rated products", rating >= 4) got 0.125-wide bins while its
siblings got 0.75-wide bins.
Fix: resolve the width the QP would use for the unsegmented query
(card-level filters included, per-segment filter excluded) and pin it on
the breakout as {:strategy :bin-width}. The binning middleware never
renegotiates an explicit bin width, and bin edges snap to multiples of
the width, so every variant lands on the same bin grid. Card-level range
filters on the dim still narrow the bins, since they're shared by all
variants.
- move filters->field-map + extract-bounds from the binning middleware
into metabase.lib.binning.util (middleware delegates; extract-bounds
now returns nil and the middleware keeps the qp.error-type throw)
- add metabase.lib.binning/default-bin-width (exported via lib.core):
the width :default binning resolves for a column given the query's
own filters, nicer-breakout fixed point included
- pin it in metabase.explorations.query-plan.mbql/apply-default-bucket
* Address CR
- default-bin-width now keys its filter lookup by the column ref's own
id-or-name (like the QP binning middleware) instead of (or (:id column)
(:name column)) with a regression test for the bug this fixed.
- Build the segment :definition test fixtures through lib instead of the
mt/mbql-query macro
* Hide the empty dashcard menu on visualizer cards when downloads are not permitted
* Address review comments
* Type the dashcard menu's underlying-question items as an array and simplify code
* add copy value click action
* move details above filter
* move copy into it's own section
* fix import
* e2e adjustments
* Adding unit test for column settings
* Don't show copy action on PK columns
h2x/cast already guards cast targets: a plain type name is emitted raw,
anything else is quoted as an identifier. The MySQL and Postgres
json-query implementations bypassed that route and spliced the stored
field's database-type into SQL unvalidated (CONVERT(.., <raw>) and
::<raw> respectively).
Expose the plain-type-name predicate as h2x/raw-type-name? and use it:
Postgres quotes a non-plain type as an identifier (same rule as
h2x/cast); MySQL rejects it with an invalid-query error, since
CONVERT's target type cannot be a quoted identifier.
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Tile endpoints resolve latField/lonField legacy refs via resolve-field;
a ref with :source-field (implicit join) now fails fast with a 400
instead of producing a broken query. The check lives in resolve-field,
which every tile endpoint (ad-hoc, card, dashcard, public, embed,
preview-embed) reaches through tiles-query.
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Internal query-processor and permission keys (`:qp/...`, `:query-permissions/...`,
`:impersonation/...`, `:persisted-info/...`, `:source-query/...`) are added to a
query during preprocessing and drive permission enforcement, so a client must
not be able to supply them. Previously five separate stripper middlewares
removed them one key at a time.
Replace those with a single declarative rule: a qualified map key outside the
`:lib` namespace is an internal key and gets stripped when a query is
deserialized from JSON. The rule is a Malli `:decode/api` (and
`:encode/serialize`) transformer on the query/stage/join/clause-options schemas,
run via `lib.serialize/prepare-after-deserialization` at the JSON boundaries
(REST API request decode and app-DB read) and once as a preprocessing
middleware, `qp.perms/remove-internal-keys`, gated by `qp.pipeline/*pivot?*` so
pivot sub-query keys survive their re-run.
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
* Improve API schemas
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* Keywordize nested parameter option map keys
`::parameter-with-value` only keywordizes its top-level keys (via
`normalize-map-no-kebab-case`), so on the query-string parameter path -- where
the whole value arrives as a JSON string with string keys at every level -- the
nested `parameter.options` and `dimension.options` maps kept string keys. The
api decode's `strip-extra-keys` pass then dropped those undeclared string keys,
silently emptying `:options` (a case-insensitive filter would run case
sensitively, a relative date filter would lose `:include-current`, a
stage-targeted dimension would lose `:stage-number`).
Give both option schemas a `:decode/normalize` of `lib.schema.common/normalize-map`
so their keys are keywordized before the strip pass, matching how the rest of
the lib schemas decode map keys.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* Address agent-api / xrays review feedback
- /v2/query continuation-token branch now normalizes-and-validates the embedded
query like the query_handle branch does, instead of trusting the unsigned
token's shape; the total-row budget is read off that validated query.
- reject-native-query! normalizes the payload to MBQL 5 (best-effort) and checks
for a native stage with lib/any-native-stage?, so native detection no longer
depends on how the JSON was decoded (keyword vs string keys); /v1/execute and
the query_handle path decode with plain json/decode and let normalization do
the keywordizing. Malformed payloads still fall through to the existing shape
checks for their clean 400s.
- Renamed validate-query -> normalize-and-validate-query (it normalizes and
strips, not just validates) and led its docstring with what it returns.
- xrays cell-query route params decode declaratively via a ::cell-query schema
(:decode/api base64-decodes + validates ::root.cell-query) instead of
base64-decoding once in the schema and again in the handler.
- Tiles reject test asserts a clean 400 (status-code) rather than any Exception.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* Update sandbox dimension-target test for keywordized option keys
The dimension-target `:options` map now keywordizes its keys (e.g. `"stage-number"`
-> `:stage-number`), so the attribute-remappings round-trip test expects the
normalized keyword form on the way back out.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
---------
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
* add permalink to quarantine entry, update compare function to return QuarantineEntry type
* print search url to make tests easy to find
* update tests
* remove a few comments
* simplify search url
Disable "Send now" when the alert has no delivery target (#79881)
Extract alertHasValidTarget from alertIsValid: the handler and channel
checks without the subscriptions one. Send now is a one-off send that
ignores the schedule, so it gates on the target only, while Done keeps
the full check.
Fix disabled Done button in question alert modal (#79914)
bg is a style prop, lands inline on the root and beats the :disabled
rule in Button.module.css, so the button kept its brand background while
the label switched to the dark text-disabled token. color feeds
--button-bg instead, which the disabled rule overrides.
* Run full query-permission checks when compiling a native form (SEC-822)
POST /api/dataset/native compiled and returned the query after checking only
database-level native perms, so a referenced source card or card template tag
was resolved without a read check. Preprocess the query once (with inline
parameter compilation enabled), run the same check-query-permissions* checks
the run path uses (surfacing a permission failure as a 403), then compile the
preprocessed query. Also trim the response to :query and :params.
* Validate link-card entity ids on write for dashboard cards (SEC-836, SEC-832)
DashboardCard validated the link-card entity id only when building the read
query; nothing checked it on write, so a non-integer id could be stored and
would then break every hydration of the dashboard. Add a write-side check in
define-before-insert and a new define-before-update, and wrap the notification
link-card renderer in try/catch so one bad card degrades to a missing part
instead of failing the whole subscription. Adds regression tests.
* Require positive-integer entity ids in dependency backfill (SEC-830)
replace-dependencies! placed to_entity_id directly in the insert, and the
dashboard click_behavior extraction forwarded targetId with no type check. Add
an explicit pos-int? check at the insert where to-add is built (a runtime
check, not an mu/defn schema, which is stripped from prod builds), and filter
targetId to positive integers during extraction so bad ids drop rather than
making the backfill job throw. Adds regression tests.
* Validate click-behavior target ids on write for dashboard cards (SEC-830)
Review follow-up: the entity ids the dependency backfill consumes come from
dashcard click_behavior targetIds, so validate them at the write boundary as
well, keeping the deep guard in replace-dependencies! as defense in depth.
Covers both the top-level click_behavior and per-column column_settings ones,
on insert and update. Adds regression tests.
The 'converting question to a native query' e2e test asserted on
[data-testid=cell-data] immediately after clicking 'Convert this
question to a native query' (and after 'Explore results'), without
waiting for the POST /api/dataset run to settle. Under load the
assertion raced a stale/mid-flight results table and timed out.
Mirror the working convertToSql() helper: intercept POST /api/dataset
and cy.wait for it before each cell-data assertion.
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* Move plan util from common into lib utils
* Move waitUntilNextFramePainted from common into lib utils
* Move export format types from common into metabase-types
* Move plan helpers into the settings module
Add a copy button to the query builder's view footer, next to download.
It copies whatever is on screen: tabular results (tables, pivot grids,
or a chart toggled to the raw data view) as tab-separated text plus a
text/html table flavor so spreadsheets paste a grid, and any other
chart as a png rendered through the same pipeline as the png download.
Copies are always formatted, matching the rendered table: remapped
columns, column visibility and order, renamed titles, click-behavior
link text, and pivot grids with their headers and totals. Raw data
stays a download concern. Copying is capped at the rows loaded in the
browser, with a toast saying when only the first slice was copied.
The clipboard write starts inside the click's user activation so Safari
accepts it, cell click metadata is built only for columns whose text
needs it so wide tables serialize in linear time, and the selected-cell
copy now shares the quoting TSV serializer.
Cover Database connection details on serialization import
Pins the two checks in load-one! for Database: an H2 engine is refused, and
low-level JDBC keys are dropped from details, write_data_details and
admin_details.
* Harden document card-creation permission checks
Route document create/update/copy card creation through the same checks
POST /api/card applies: create-check the target collection, read-check
each card copied by document_id FK, and query-permission-check the fields
that parameter targets name.
* Guard document AST entity ids before app-DB lookups
smartLink/cardEmbed reference ids come from free-form document JSON and
may be any value. Add a shared node-entity-id accessor that yields the id
only when it is a positive integer, and use it in the raw-entity readers
(serdes export transform, export-time dependencies, and descendants) so a
malformed reference is skipped rather than passed to a Toucan query.
* Add access and type checks to comment notifications
Mentioned users are now resolved to active users whose permissions cover
reading the commented entity before being added to the notification
recipients, matching how other delivery surfaces scope their audiences.
The TipTap text-node renderer now coerces non-string :text values to
strings so they always compile as escaped text rather than as hiccup
elements.
* Scope the comment mentions listing to visible users
GET /api/comment/mentions now applies the same audience rules as
GET /api/user/recipients: non-superusers only see users in their own
tenant, further narrowed by the user-visibility setting (:none -> self,
:group -> shared-group members, :all -> everyone in the tenant). This
also resolves the endpoint's long-standing TODO about tenant handling.
same-groups-user-ids moves from the user REST namespace into the user
model so both endpoints share one implementation.
* Move JoinConditionOperator from metabase-lib into metabase-types
* Move font family and DeepPartial types into metabase-types
* Generate mock document ids without the lib utils uuid helper
* Import JoinConditionOperator from the api barrel in metabase-lib
* Drop lib-utils re-export shims and inline docs at the moved declarations
* init vllm support
* tmp improvements
* cleanups; bump max tokens floor for vllm; env shadowed cred field guards
* fix openrouter temperature
* vllm reasoning
* fixes
* cleanups
* make vllm transport errors non-retryable
* fix vllm connect diagnostics; add recorded stream fixtures
* [BOT-1930] Rebuild vLLM support on the provider connection registry (#80103)
* Rebuild vLLM support on the provider connection registry
vLLM becomes a provider type in `metabase.llm.provider` rather than a set of
`llm-vllm-*` settings read by the adapter: a required base URL and an optional
API key, no default model, and an `MB_LLM_VLLM_API_BASE_URL` group so the
environment can configure one connection. The base URL is the credential — a
server started without `--api-key` takes no key — so it alone brings a usable
connection into existence, and the admin form renders from the descriptor
instead of a bespoke component.
Two things the adapter used to reach out for now travel with the connection:
- The connect-time preflight, which exercises tool calling and structured
output against the model that will be used, runs from `verify-credentials!`
under a new `probe?` flag. Only writes probe; a model listing must not spend
a generation on the operator's server.
- Whether the served model streams reasoning, which only the probe can answer,
is reported back as `:learned-config` and stored on the connection instead of
in a global setting, so two vLLM connections no longer overwrite each other's
verdict. The request path reads it from the credentials it is handed.
A vLLM connect has no default model to select, so it adopts the model its probe
exercised, and an edit re-probes against the model the connection is actually
serving Metabot.
* Document connecting a vLLM server
The provider list in the AI settings docs is hand-maintained. vLLM is the one
provider an admin can misconfigure outside Metabase, so it gets the server flags
the connect-time check looks for alongside its entry.
* fix kondo
* revert docs
---------
Co-authored-by: Tyler Plude <tsplude@gmail.com>
* review comments
---------
Co-authored-by: Maksym Yakubych <maksym@metabase.com>
Co-authored-by: Maksym Yakubych <undrfined@gmail.com>
* upload all pass/failures for backend tests on rerun
* also for frontend tests
* add some tests
* cleanup whitespace
* copilot review
* a few more fixes from local review
* determine rerun from github context instead of input
* Use content-based cache keys for m2, bun, cypress, eslint.
Eagerly populate cache on push to master/release for m2, bun, cypress.
Read-write eslint cache from every branch.
* Handle multiline outputs
* Remove unneeded M2 restore on prepare-frontend
* Handle potentially multiple cyprss version matches. Update commentary.
* Validate list of alias combinations in use across the repo
* Check preresolve aliases in parallel
* Enable Qwen 3.8 Max via OpenRouter
* Downgrade tool_choice "required" to "auto" for qwen3.8-max
Alibaba returns a 400 request error if you pass tool_choice "required" and unlike moonshot / Kimi K3, does not allow
you to disable thinking mode for Qwen 3.8.
* Add coerce-stringified-scalars to transform tool call args
Qwen 3.8 sometimes sends integer args as strings, like `{:limit "15"}`. Use the tool's declared schema to
automatically transform any simple numeric or boolean scalars.
* Test: transform revision diffs should not include past source bodies
A transform's revision history diffs two full snapshots, so every past
:source (native SQL / Python body) and :source_database_id shows up in
the diff - even for versions that read from a database the current
viewer has no query access to. Add a test asserting the diff only
describes that the source changed, without reproducing the old source.
* Guard transform revision history per snapshot, not per current row
* extra guards in metabot tool
* GDGT-2932: Refactor MCP/CLI analytics to shared-layout + nested routes
Replaces the tab-query-param pages with a shared layout (header, tab
navigation, filter bar, and a single "has any data" gate) rendering
usage/events sub-routes through an Outlet, mirroring the
dependency-diagnostics pattern. New URLs:
- /monitor/ai-auditing/mcp/{usage,events}
- /monitor/ai-auditing/cli/{usage,calls}
with an index redirect from the bare section path to /usage.
Shared filter/pagination state is passed to the active leaf route via a
typed React context (following MonitorContent's existing
useMonitorSidebarContext precedent) rather than react-router's untyped
useOutletContext.
Also fixes two bugs found in cross-review: tab links were dropping the
current query string (clearing filters on every tab switch, since
LinkTab doesn't inherit location.search), and the index redirect was
gated behind the "has data" query so a bare /mcp or /cli with zero rows
never canonicalized to /usage. Bundles a parity fix for useCliHasData
(now surfaces query errors, matching useMcpHasData) so the CLI usage
view doesn't spin forever on a failed count query.
* Fix MonitorHeaderTabs to use aria-current instead of the ARIA tabs pattern
MonitorHeaderTabs rendered its options with role="tab" (via Mantine
Tabs.Tab + a Link component), but each "tab" is a real, bookmarkable
route rather than in-place content switching — the ARIA tabs pattern
implies activating a tab doesn't navigate, which isn't true here. It
also never rendered a matching tabpanel/aria-controls pairing, since
the actual content lives in a react-router <Outlet/> outside the Tabs
tree entirely.
Rewritten as plain nav links with aria-current="page" on the active
one, matching the existing AreaTab component's approach for the same
kind of route-driven navigation. Visual pill styling is preserved via
a small CSS module. Updates the two dependency-diagnostics/tasks/mcp/
cli consumers' tests and e2e specs that asserted the old role="tab".
* GDGT-2932: Review
* GDGT-2932: Keep error count queries mounted across analytics routes
* GDGT-2932: Cleanup
* Fix task tabs E2E selector
* GDGT-2932: Address PR comments
* GDGT-2932: Cleanup
* GDGT-2932: Remove LinkTab. Introduce shared AiAnalyticsSectionLayout. Fix comments
* Move the xyflow stylesheet into the graph components
* Give the schema viewer one loader for its route and its prefetch
* Load the dependency graph from the plugin slot lazily
* Deflake metabot error boundary test by loading the chat module at suite setup
* Preload the chat module in the shared metabot test setup
* Use the shared metabot test setup in the error boundary spec
* Clean up leftover JS-era config, deps, and docs after the TypeScript migration
* Address review: keep .jsx in resolve.extensions, drop .js from loader tests, includes-based e2e validator filter
* Keep @babel/node for a separate deps cleanup round
* Keep js/jsx in rspack loader tests
* Fix flaky dashboard parameter-removal e2e test
The "should be able to remove parameter (metabase#17933)" test asserted on
the full dashboard URL query string with cy.location("search").should("eq",
...). The order in which parameter slugs are serialized into the URL is not
deterministic, so a correct-values-but-different-order string
(?text_starts_with=G&text_ends_with= vs ?text_ends_with=&text_starts_with=G)
could never match and the retry just spun until timeout. Empty/stale-URL
timing races produced the other observed failures.
Replace the four exact-string assertions with an order-independent helper that
parses the query string via URLSearchParams and deep-equals the expected
param map. It still retries through cy .should(callback), so async URL updates
settle before the assertion resolves.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* Wait for parameter-remove button to enable before clicking
The parameter-removal step in the dashboard parameters spec clicked the
sidebar's Remove control via cy.findByText("Remove"). That query resolves to
the button's inner label <span>, not the <button> element. A span is never
"disabled", so Cypress's actionability check did not wait: it clicked the span
while the underlying button was still disabled.
The Remove button stays disabled until the dashboard's subscriptions have
loaded (the sidebar warns before archiving any subscription bound to the
filter). When that GET /api/pulse request was slow, the click landed on the
disabled button as a silent no-op, the parameter was never removed, and the
subsequent URL assertion timed out waiting for the slug to disappear.
Target the button element by role and assert it is enabled before clicking so
the click waits for the subscriptions request to settle.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* Fix linting error
---------
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-authored-by: Nemanja <31325167+nemanjaglumac@users.noreply.github.com>
The test opened the info sidebar right after an ad-hoc edit-mode save that
only awaited the PUT /api/dashboard request. The trailing query_metadata
reload re-rendered the header, dropping the 'More info' click so the
sidesheet never opened. Use the H.saveDashboard() helper, which waits for the
full save-and-settle sequence before proceeding.
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* Fix getTable intercept in segment_created tracking e2e test
The segment editor's DataStep loads a table via fetchTableMetadataAndForeignKeys, which calls GET /api/table/:id/query_metadata. The test intercepted the bare GET /api/table/:id endpoint instead, so cy.wait("@getTable") always timed out because that request never fires in this flow. Point the intercept at the query_metadata endpoint the editor actually calls.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* Gate segment filter click on enabled Add-filters button
The segment editor renders a disabled "Add filters to narrow your answer"
button until onChange builds the query from fully-loaded table + FK metadata.
Waiting on the getTable response alone isn't enough — the trailing foreign-key
metadata loads race the click, so the click lands on the still-disabled button
and the filter popover never opens. Assert the button is enabled first.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* add sample db
* delete cross version 57 since we don't support it anymore
* update cross version tests to use postgres
* fix modal selector to work on v1.60
* fix modal selector for v60 and v63 compatibility
* add better logging
* use qa image
* use herestring to handle large output properly
* use data from WA image
* fix flake in nested model assertion
* cleanup the README
* code review fixes
* Validate imported queries against this instance's MBQL schema (GHY-4241)
An export decides the shape of the queries it carries, so an import can hand us
a query shape this version has no representation for. `lib/normalize` does not
always reject one: it can quietly produce something the schema rejects, which
is then stored and breaks the card on read.
After `import-mbql` normalizes an ingested query, validate the result against
`::lib.schema/query` and refuse the import if it fails. Only full queries are
checked - bare refs and the MBQL fragments inside visualization settings have no
standalone schema to validate against.
Normalization failures stay non-fatal. They fire today on content this instance
produced itself (pivot column refs in visualization settings raise
`:malli.core/invalid-schema`), so promoting them would reject legitimate
imports. Validating the successfully-normalized result catches the real problem
without that fallout.
`MB_SERIALIZATION_SKIP_SCHEMA_VALIDATION` opts out. `metabase.models.serialization`
cannot read an EE setting, so the flag reaches it through a dynamic var bound by
`load-metabase!` - the one choke point every import path funnels through.
Also validate the bundled instance analytics content against the same schema in
CI. That content re-imports on every EE boot and a failure there propagates out
of `init!` to `System/exit`, so a bundle regeneration that broke it would
crash-loop rather than degrade.
* Fix import validation documentation
* Refactor documentation for import validation settings
* update docs
* Report the schema refusal as a 400 and stop duplicating its errors (GHY-4241)
The serdes import API reads `:status` off ex-data itself rather than going
through the API layer's exception handling, so `:status-code` left it nil and
the refusal surfaced as a 500 for what is a client input error.
`u/strip-error` already renders ex-data into the message, so the truncated
`Schema errors:` tail was a second, shorter copy of `:schema-errors`.
* Drop the unreachable status code from the schema refusal (GHY-4241)
`load-one!` rewraps every exception from the load block in a fresh ex-info
carrying `path-error-data`, and the import API reads `:status` off the
outermost exception only. Neither `:status` nor `:status-code` attached here
was ever visible, so claiming one in a comment was worse than saying nothing.
* Name both causes in the refusal, and pin bundle ingest errors (GHY-4241)
`mu/defn` schemas are not instrumented in prod, so an app DB can hold MBQL the
QP tolerates but `::lib.schema/query` rejects - a stale `:join-alias`, a field
filter with no `:widget-type`. Such content exports intact and is then refused
on import, so "most likely exported by a newer Metabase" was the wrong
diagnosis for a whole class of refusals. Name both causes, and point at
continue_on_error as the per-entity escape rather than only the global one.
The bundle test proved every query imports but not that every file parses.
`ingest-list` collects unreadable files in an errors atom rather than failing,
so a malformed bundled YAML would drop out of the queries under test while
still crashing EE boot. Assert the atom is empty too.
* Move refusal rationale out of docstrings, harden a test helper (GHY-4241)
Docstrings state the contract; the reasoning behind the check, the message
wording and the sentinel moves to comments beside the code it explains.
`*skip-schema-validation?*` no longer names who binds it, which would go stale
silently. `validate-imported-query!` now says legacy MBQL 4 queries are left
alone as well, which was true but unstated.
`cause-chain-messages` used `map ex-message`, so an exception with no message
would put a nil in the vector and NPE the callers' `re-find` - failing the
test for a reason unrelated to what it asserts.
* kick ci
* Simplify the import schema check's control flow (GHY-4241)
`validate-imported-query!` returns its argument and no longer consults the
opt-out, so it is a plain step in `import-mbql`'s pipeline and the caller
decides whether to run it. That removes the sentinel `normalize-imported`
needed to keep the refusal outside its `catch`, along with the comment
explaining the sentinel.
`some->` replaces `->`, which makes `normalize-imported`'s nil guard
redundant.
* update comment
* Route semantic embeddings through the provider SPI
Summary
- Register the existing ai-service, OpenAI, and Ollama implementations through the shared provider registry.
- Route semantic embedding, readiness, resolution, and preparation through the SPI while preserving the existing public adapter.
- Preserve the compatibility client model-dimensions contract by normalizing it to vector-dimensions at the provider boundary.
- Keep semantic-specific batching, metrics, and settings policy in semantic search.
Scope
This migrates consumers to the SPI. Persisted embedding-space compatibility and in-process provider selection are intentionally handled by later stack layers.
Testing
- Built-in provider behavior and existing semantic embedding adapter tests.
- Compatibility-client dimension normalization and conflict handling.
* Describe the embeddings client contract in SPI terms
The batch entry point delegates to the provider registry rather than a multimethod.
* Port complexity-test fake embedder to the provider SPI
get-embeddings-batch is a plain fn now, so the old defmethod no longer compiles
and broke every job that loads this namespace. Register a capturing provider
instead and assert on the descriptor it receives.
* Induce the dimension mismatch from the model, not the mock vector
The SPI rejects a vector whose length doesn't match the resolved model, so a dim-1 mock embedding
no longer reaches pgvector. Shrink the configured model instead, which is what an upgrade does.
* Persist immutable embedding-space identity
Summary
- Persist embedding-space IDs in semantic and library-entity index metadata.
- Backfill semantic metadata through a preserving schema migration and rebuild legacy entity vectors once when identity is unknown.
- Include immutable space identity in compatibility checks and physical table names.
- Enforce the PostgreSQL 63-byte identifier limit using UTF-8 byte length.
Scope
This establishes durable compatibility. Synchronization between entity compatibility checks and rebuilds follows in the next stack layer.
Testing
- Semantic migration preservation and backfill.
- Entity-index rebuild and compatibility behavior.
- Embedding-space table naming and non-ASCII identifier limits.
* Resolve the model in the index-cleanup test helper
record-new-index-table! now persists embedding_space_id into a NOT NULL column,
so create-test-index must build a resolved model that carries one.
* Track the index-status rename after rebase onto master
* Resolve the model-v2 variant in the reconcile tests
A bare assoc keeps the v1 embedding-space-id on a renamed model, so the rebuild wrote a meta row
whose space no longer matched what the provider resolves and nothing repopulated.
* Serialize entity index queries with rebuilds
Summary
- Coordinate library searches and reconciles with shared and exclusive PostgreSQL advisory locks.
- Keep compatibility rechecks and vector queries on the same locked connection so they cannot straddle a rebuild.
- Use non-blocking search lock acquisition so waiting searches cannot exhaust the pgvector pool.
- Preflight compatibility before embedding and return no results while a reconcile owns the index.
Scope
This PR only addresses library entity-index concurrency and degradation behavior.
Testing
- Search call ordering and locked connection reuse.
- Incompatible-index preflight and reconcile-contention degradation.
* cljfmt entity-retrieval core-test
* Follow the search-unfiltered rename in the entity-retrieval tests
Rebase fallout: the parent branch renamed `search` to `search-unfiltered`, and these call sites came across from
before the rename.
* Enable the in-process embedding provider
Summary
- Allow plugin-defined providers in semantic settings and safely encode provider names in physical index identifiers.
- Enable the bundled in-process Arctic provider with dimension readiness, query prefixing, and token metrics.
- Document the optional plugin and its glibc 2.34+ Linux or Apple Silicon macOS runtime requirement.
- Build the embedder jar in CI and run one artifact-only smoke path through the provider SPI on the OSS classpath.
Scope
This intentionally supports one pinned Arctic model. Custom model sources, MiniLM, and library overrides remain out of scope.
Testing
- Settings, prefix, token-accounting, readiness, and derived table-name regressions.
- Jar-manifest discovery, platform failure modes, batching, no-network behavior, and real ONNX inference.
* Re-hash qualified index table names over the 63-byte limit
qualify-index wraps the model table name with the metadata qualifier but did not
re-check Postgres's 63-byte identifier limit. A longer provider-encoded name plus
the qualifier could exceed it, so Postgres truncated the name on CREATE while
table-exists? still looked it up by the full name -- reporting the index table as
missing. Re-apply hash-identifier-if-exceeds-pg-limit to the qualified name.
* Hash only the table component of a qualified index name
Postgres's 63-byte limit is per identifier. Hashing the whole schema-qualified
name dropped the schema, so in app-db mode a long semantic_search.<table> became
a bare index_<sha1> created outside the isolated schema. Split off the schema and
hash only the table component.
* Cover in-process MiniLM complexity scoring
* Activate the in-process embedding provider on demand
* Stub lazy provider activation in token usage test
* Update Kondo ratchets for embedder changes
* Rely on bootstrap for in-process provider registration
* Cover the bundled complexity-score model in the artifact smoke test
The artifact job only exercised the retrieval model, so a broken or missing MiniLM
bundle -- which uses a different pooling and the three-input INT8 graph -- would have
shipped unnoticed.
* Document both bundled embedder models
* Register the in-process provider in standalone CLI mode
Provider registration happens in metabase.core.core/init!, which `--mode complexity-score`
skips entirely. A synonym axis configured for the in-process provider therefore failed with
:provider-not-registered on the CLI while working on the server.
The activation moves into metabase.embeddings.startup so both entry points run the same
routine instead of drifting apart.
* Assert the descriptor the in-process synonym embedder sends
The test claimed the embedder passes the MiniLM descriptor to the provider SPI but only
checked that it was a function, so an Arctic descriptor would have passed too.
* Pin the CLI's provider registration with a test
Nothing caught the standalone CLI skipping provider registration the first time, so assert
appdb mode registers before it scores.
* Honour :snowplow? false in the ai-service adapter
Routing single embeddings through the batch SPI dropped the option, so the health and
circuit-recovery probes started reporting their synthetic embeddings as organic token usage.
* Make the dimension-mismatch test reach pgvector
Shrinking the model with assoc kept the old embedding-space id, so resolution failed and the
query never ran -- the same path search-degrades-when-configured-model-cannot-be-resolved-test
already covers. Use a self-consistent model, stub the compatibility preflight aside, and assert
the warning that only the 22000 branch emits.
* Use dynamic redefs for the mock index's table suffix
Pre-existing on master: a plain defn redefined on one thread, so nothing here needs the root
swap. Same mock-index value either way.
* Document the tiny bundled models as a test tier
* Fix index-health-check false degraded report
row->index's :embedding-model now carries :embedding-space-id and
:embedding-spi-version, which get-configured-embedding-model doesn't
return, so the equality check in index-health-check always failed and
reported every healthy index as degraded. Compare only the keys
settings actually configure.
* Give v1 Arctic-embed models their own query prefix
model-family-query-prefixes matched every snowflake-arctic-embed
model name and prefixed queries with "query: ", which is only correct
from v1.5 onward. v1 models (including the bundled xs test-tier
model) expect the longer CQE-style instruction instead.
* Carry the plugin-name pin into embeddings.startup
The provider registration this branch moved into
embeddings.startup/ensure-in-process-provider! re-duplicated the
manifest name as its own private literal. Make it the one public
constant both the startup logic and the tests reference, instead of
reintroducing a third copy.
* Correct the Arctic Embed query-prefix cutover to v2.0
The generation split is at v2.0, not v1.5. m-v1.5 was getting "query: "
when it wants the longer instruction, and m-long matched neither pattern
so it got no prefix at all.
Replace the two mutually-exclusive patterns with an ordered first-match-wins
table. That drops the exclusivity constraint the previous shape needed and
covers every model in the family.
* Stop the query-prefix setting claiming `query: ` family-wide
The description predates the generation split and tells admins the whole
snowflake-arctic-embed family expects `query: `, which is wrong for the
bundled model.
* Anchor the Arctic Embed patterns to the model-name segment
Unanchored, they matched any name merely containing the family name, and
a false positive there is unsuppressable: `ee-embedding-query-prefix` can
replace a matched prefix but never remove one.
* Compare embedding spaces in the semantic health check
Settings can stay identical while a provider upgrade resolves the same
model into a new space, stranding the index. The space id is what
actually decides whether the stored vectors are still usable, so compare
that rather than provider/model/dimensions.
Resolution reaches the provider, so it can fail outright when the plugin
is absent or the platform is unsupported. That is its own incident, not
a model mismatch.
* Document the in-process semantic search model
* Add in-process semantic search integration test
* Drop incompatible legacy entity indexes
* Address embedder review findings
* Quote legacy index identifiers separately
* Internal load-namespaces tool to check for compilation errors
* Ignore load-namespaces in module checks
* lint load-namespaces
* Reject positional args
* Test the new utility
* Guard against nonexistent root.
Make the code more testable.
* Load CodeMirror editors lazily so they leave the initial bundle
* Inline the single-use dynamic imports into the lazy calls
* Share the codemirror split chunk with the embedding SDK build
* Split the snippet, Python and public document pages by route instead of wrappers
* Dedupe modules shared by async chunks instead of naming CodeMirror
* Drop the duplicate metrics viewer route that master already added
* Keep barrel imports and mark the expression language side-effect free
* Move the action editor boundary to the new action-creator module
* Drop the codemirror cache group the SDK config no longer needs
* Drop props types exported for wrappers that route splitting replaced
* Delay the code fallback so a fast chunk load never flashes uncoloured code
* Render nothing while the code editor chunk loads
* Delay the whole expression widget instead of showing a spinner inside it
* Open the editors from routes and modals instead of loading states
* Fold the Metabot preview chunk into the message's own loading state
* Fold the query editor chunk into the transform pages' own loading state
* Prefetch the expression widget from the notebook so its popovers open ready
* Await the run error message now that the code editor loads lazily
* Hold the custom column popover shut until the expression widget arrives
* Wait for the expression editor to hold focus before typing into it
* Type into the page directly when asserting the read-only native editor
* Read the editor value only once its document stops changing
* Fetch the routes-only plugin pages when their route is matched
* Fetch the remaining plugin route factories when their route is matched
* Load the AI settings pages as one chunk
* Say why these route files register no prefetch
* Derive the AI settings prefetch paths from the route paths
* Prefetch the model migration page like its siblings
* Count the python transform pages master already made lazy
* Give each AI settings page its own chunk
* Move useEntityData out of the tiptap SmartLink node module
* Load the Metabot chat panel lazily
* Load the public document page lazily
* Recover from a failed fetch of the Metabot chat panel
* Fetch the public document chunk at startup
* Fetch the data studio, model and snippet pages when their route is matched
* Fetch the monitor and timeline pages when their route is matched
* Name the chunks the route loaders build
* Fetch the AI auditing and library pages when their route is matched
* GDGT-2977 Style full markdown output in shared Markdown component
- Markdown.module.css covers every CommonMark+GFM element as :where() base layer
- compact prop for line-clamped previews
- disallowHeading renders headings as paragraphs instead of unwrapping to text
- drop duplicated rules from AIMarkdown, delete AdvisoryCard.module.css
- kitchen sink fixture + stories + spec failing on unstyled elements
* GDGT-2977 Trim markdown specs to our own behavior
- drop tests that only exercise react-markdown/remark-gfm
- cover urlTransform (metabase://, data:image, unsafe protocols)
- assert headings render as p when disallowed
- merge AdvisoryCard markdown cases
* GDGT-2977 Fix markdown spec asserting against mantine style tag
- container.firstElementChild was the injected <style>, so every
querySelector check passed vacuously
- render into a host element and select the markdown root from it
- javascript: urls collapse href to empty, which drops the link role
* addressing the PR's comments
* GDGT-2977 Match skeleton caption tooltip to legend caption
compact + lineClamp, per PR review
* fix small regression
* Let conversation titles run on their own model
Every LLM call read the one `llm-metabot-provider` setting, so naming a
conversation — a short, high-volume call — burned the same model the chat runs
on. Add an `llm-title-model` setting whose getter resolves rather than defaults:
the stored value, else the fastest model of the connection Metabot uses, else
that model itself. Provider types now carry a `:mini-model` alongside
`:default-model`, taken from each adapter's own supported-models list so the
fallback names a model the picker actually offers; Azure and the managed
provider have none and fall through.
Move model selection out of the AI providers card into its own Models section,
so per-use-case choices have somewhere to live. Deleting a connection clears a
title model pinned to it, reading the raw stored value so a merely-derived one
is left alone.
* Name a mini model for every provider type that has one
Mistral and Z.AI were the only types with no `:mini-model`, so conversation
titles on those connections fell through to the model Metabot chats on. Both
adapters whitelist exactly one model today, so their mini model is that same
model — the fall-through by another name, but spelled out where a real small
model goes when one is added.
Two tests hold the line. The registry map fails when a type gains or loses a
`:mini-model`, since a missing one reads as nil and quietly sends titles to
the full-size model. The listability test fails when a type names a model its
adapter will not offer: `list-models` returns the whitelist intersected with
the provider's catalog, so an id the whitelist omits never reaches the admin
picker, which renders it as a blank selection.
That second test caught Anthropic naming `claude-haiku-4-5` while the Claude
whitelist carried only `claude-haiku-4-5-20251001`, leaving the new title
picker blank on the commonest setup there is. The whitelist was the odd one
out — the adapter already defaults to the undated alias — so it loses the date.
* Sell the title model as the mini model it already was
Review asked whether each feature should get its own picker or whether one
cheap model should serve every quick task; the answer was one model. The
setting already resolved through the registry's `:mini-model` and nothing but
titles read it, so this is a rename — `llm-title-model` becomes
`llm-mini-model` — plus copy that presents naming conversations as the first
of the quick tasks rather than the point.
The derivation tests now pin the stored value to nil instead of assuming a
clean database; a crashed run had left a stored value behind, which the old
tests read as the derived one.
* Return Haiku's dated id, which is what the Anthropic catalog serves
Dropping the date assumed /v1/models would list the bare alias. Review
testing with a live key showed it does not: the catalog carries only
claude-haiku-4-5-20251001, so the undated whitelist key filtered Haiku out
of the picker, and the derived mini-model reference matched no option —
titles kept running on Haiku while the picker sat blank.
The alias stays where it always worked: inference-call defaults. The
whitelist and the registry go back to the catalog's id, which is also the
whitelist's convention — the 2025 models are dated, the 2026 models are not,
tracking what the catalog returns for each.
* Move a pinned mini model with the deployment its connection now serves
Editing an Azure connection's deployment already moves the Metabot
selection, because the reference bakes the deployment name in. A mini model
pinned to that connection went stale the same way, so it moves the same
way. A derived mini model is left unstored — it follows the Metabot
selection on its own.
* use json_query as the cache key to prevent cardinality cache collisions
* fix e2e, persist default settings that depend on cardinality
* review comments
* Display percentages instead of values on stacked bar charts
Adds a Stack value format setting that lets stacked bar chart segments display their percentage of the stack total.
Addresses #67702
* Handle percetanges for negative values
* Show decimals for fractional stack percentages
* Show percentage totals for stacked bars
* Improve condition readability
---------
Co-authored-by: Abhijnya K G <53806883+Abhijnya002@users.noreply.github.com>
* Invalidate the cached models list when credentials change
A credential write can change which models the provider serves, e.g. a different Bedrock region, Google location, or
Azure resource.
* Support multiple LLM provider connections at once
Metabot could only ever be connected to one AI provider, because each
provider's credentials lived in its own hardcoded settings
(llm-anthropic-api-key, llm-azure-api-key/-base-url, llm-bedrock-*, ...)
and llm-metabot-provider named the single provider in use. Adding a
provider meant editing a `case` in six namespaces plus three constant
maps on the frontend, and two keys for the same provider were impossible.
Replace that with a list of provider *connections* stored in one
`llm-providers` JSON setting, fronted by a CRUD API rather than edited as
a raw blob -- the same shape the `oidc-providers` setting already uses.
Provider types are now a pure-data registry describing each type's
credential fields, so the admin UI renders a connection form from the
server descriptor instead of a bespoke component per provider, and the
model picker lists models from every connection, grouped by connection.
Connection keys are slugs defaulting to the provider type, so existing
`llm-metabot-provider` values keep resolving unchanged. `metabase`
becomes an ordinary connection, which collapses the three-segment
`metabase/provider/model` special case into the generic rule. The
per-provider env vars keep working and each synthesizes one read-only
connection; MB_LLM_PROVIDERS puts the whole list under env control.
* Fix managed provider handling in the connection list
Five issues from reviewing the new admin UI, all around the Metabase-managed
provider:
- The managed provider's models were fetched from the proxy, so instances
that cannot reach it showed "Couldn't load models". The proxy serves one
benchmarked model rather than a listable catalog, so the registry now
carries that catalog directly and nothing is fetched. It doubles as the
allow-list `llm-metabot-provider` is validated against, replacing the
separate copy in metabot.settings.
- The managed provider had an Edit action that opened an empty modal (or one
offering to re-Connect something already connected). There is nothing to
edit, so managed connections now only offer Remove.
- It could be added twice. Provider types can declare themselves singletons;
the API rejects a second connection and the FE drops already-connected
singletons from the Add dropdown.
- The empty state repeated the section description verbatim.
- Cards showed the raw type slug under the name ("Metabase" / "metabase").
The provider type's label is now shown only when it differs from the
connection's name, i.e. when the admin renamed it.
Also drops the startup step that materialized a `metabase` connection on any
proxy-configured instance, which made the managed provider look connected when
nobody had connected it. It is now materialized only where a `metabase/...`
model reference is actually written: the legacy metabot-v3 sync, and the
migration of an instance already pointed at it.
* Address review comments on the provider e2e helpers and managed setup spec
- Drop the doc comments added to e2e-metabot-helpers.ts.
- Move the mock LLM server's base URL into the connection's own config rather
than the legacy `llm-anthropic-api-base-url` setting. Both worked, because
the adapters fall back to that setting when the connection carries no base
URL, but relying on the fallback left the connection describing one endpoint
while requests went to another.
- Assert that the add-on purchase really does precede the connection POST,
which the test claimed in its name but never checked, and add the missing
case: a failed purchase must not create the connection.
* Stop repeating the model family in the model picker
Providers already carry the family in the model's display name, so prefixing
it with the group produced labels like "Opus · Claude Opus 4.8" and
"Anthropic · Anthropic: Claude Haiku 4.5". Show the display name on its own.
The group still comes back from the API: the backend orders each provider's
models by it, so dropping it here keeps the grouping-driven order without
showing the label twice.
* Select a default model on connect, and simplify the ad-hoc connect view
- Connecting a provider when Metabot had nothing usable to run on now points
`llm-metabot-provider` at that connection's default model, so the first
provider you add leaves the instance working instead of connected with no
model selected. An existing selection that still resolves is left alone, so
adding a second provider does not silently switch Metabot over to it. The
"did it have a usable model" check has to run before the new connection is
saved: the setting's default names the `anthropic` connection, which a
freshly saved Anthropic connection would otherwise satisfy retroactively.
- The ad-hoc "connect to a model" modal no longer renders the admin list with
an "Add a provider" button that opens a second modal. It is one connect form,
and once a provider exists it is the model picker. Extracted
ProviderConnectionForm so the admin modal and this view share it.
- Show a skeleton while the connection and provider-type lists load.
- Replace the add-on ordering assertion, which compared indexes into the call
history, with a purchase that only resolves once the test releases it: the
connection must not be created while it is in flight.
* Fix the managed-provider-limit spec for the single-form connect view
The ad-hoc connect modal no longer offers an "Add a provider" button that
opens a nested modal, so assert on the provider picker the form now renders
directly.
* Show the model picker after connecting from the ad-hoc modal
Connecting a provider in the ad-hoc view called onClose straight away, so the
modal shut before the picker it was supposed to hand over to could render.
Stay open and show the picker instead, except for a provider that serves a
single fixed model (the managed one), where there is nothing to pick.
* Split the AI provider list and setup views apart
The one component branched on an `isModal` prop to be either the admin list
(cards, add/edit/remove, model picker) or the ad-hoc connect flow, which meant
neither was composable and the modal branch read as an afterthought. Split it
into AIProviderList and AIProviderSetup, each owning its own data and state,
sharing ProviderConnectionForm, LlmModelPicker and ProviderListSkeleton.
Call sites pick the one they mean instead of passing a flag.
Also use `useDisclosure` for the add-provider modal. `editing` and `deleting`
stay on `useState` because they carry the row being acted on rather than a flag.
* Cancel the managed AI add-on from the backend, not a plugin hook
`PLUGIN_METABOT.useOnProviderRemoved` only ran when the managed connection was
removed through that one component, so removing it any other way — the API
directly, or the fallback path in the delete handler — left the customer paying
for an add-on they no longer used.
Move the cancellation into `DELETE /api/llm/providers/:key` behind a
`defenterprise`, which is where the invariant actually holds: removing the
managed connection cancels its add-on for every caller. The EE side decides
which product type to cancel from the same premium features the frontend was
reading, and keeps the rule that an instance on legacy tiered pricing with no
metered offer to move to is left alone. Extracted the Store call the
cloud-add-ons endpoints already make into `cloud-add-ons.core` so both share it.
The plugin hook and its EE implementation are gone.
Also clear fetch-mock state between AI provider spec setups. The file grew a
second `describe`, and its routes and call history leaked into whichever suite
jest packed next into the same worker.
* Address review comments on the model picker and provider API shape
- Snake_case the two kebab-case keys the provider-type endpoint put on the
wire (docs-url, default-model), matching display_name alongside them. The
internal registry keeps its idiomatic kebab-case keys.
- Drop the model picker's own success toast: useAdminSetting's updateSetting
already toasts both outcomes, so picking a model fired two toasts and the
error path only looked unhandled.
- Show SetByEnvVar under the picker when llm-metabot-provider comes from an
env var, so the disabled Select says why.
* Remove the unused GET /api/llm/list-models endpoint
Nothing called it: no frontend, SDK, or e2e reference, only its own tests. It
listed models for whichever connection Metabot was pointed at, which
GET /api/llm/models now covers for every connection.
It is in the generated OpenAPI spec, so this is a breaking change for anyone
calling it directly. The spec is regenerated by the openapi-self-healing CI job
rather than by hand — a local run rewrites far more of the file than this change
touches.
* Pick a provider type from an icon grid instead of a dropdown
Replace ProviderTypeSelect with a ProviderTypePicker card grid, giving each
provider type an icon the backend names in the provider-type registry.
Connections now also report the env vars that configured them, so the UI can
name them rather than just saying the connection is env-managed.
* Use real provider logos, drop the stepper, lead with Metabase
- Drop the two-step "Provider / Configure" indicator. The modal swaps content
in place rather than paging, so it rendered as a lone empty circle next to a
floating label and conveyed no progress.
- Ship brand logos for the five bring-your-own-key providers, mirroring how
driver logos already work: a file per provider under
app/assets/img/llm-providers, looked up by provider type, falling back to the
registry's icon when there is no logo (the managed provider keeps `metabot`).
Sourced from svgl. The three monochrome marks are rewritten to currentColor so
they survive dark mode; Azure and AWS keep their brand colour. Bedrock uses the
AWS mark for want of a Bedrock-specific one.
- Offer the Metabase-managed provider first when the instance can actually use
it, since it is the option to reach for before bringing a key.
- Fix the blank managed connect screen. MetabaseAIProviderSetup decided between
"connect" and "already connected" from `llm-metabot-configured?`, which under
multi-provider is true as soon as *any* connection works — so adding Metabase
alongside an existing Anthropic connection rendered the connected-state card
for a subscription that was never bought, which draws nothing. The caller now
passes whether the managed provider itself is connected.
The provider-types order assertions became environment-dependent once ordering
keys off proxy availability (this worktree's .env sets MB_LLM_PROXY_BASE_URL), so
they now assert the set of types plus the lead position explicitly.
* Drop the provider picker heading that repeated the modal title
* Right-align the managed provider footer to match the other setup screens
* Focus the confirm button so Enter confirms instead of dismissing the modal
* Tuck the optional connection fields behind Advanced settings
The connect form led with Display name and API base URL, so the one field
that actually matters — the API key — was buried among inputs that already
have working defaults.
Show only the fields a provider type marks required, and collapse the rest
plus Display name into an Advanced settings disclosure at the bottom. The
split comes from the field descriptors rather than a hardcoded list, so
Azure (whose base URL is required) keeps both inputs up front.
Editing a connection that already customized an optional field opens the
section, so a non-default value is never hidden.
* Let the registry say which connection fields are advanced
Inferring it from `required` was wrong for Bedrock. `required` means
validation fails without a value, which is a different question from whether
an admin has to think about the field. Region has a default, so it is not
required — but it decides which models are reachable and where inference
runs, so burying it hands EU customers a silent us-east-1 default.
Add an explicit `advanced?` to the field descriptors and split on that. Only
the base URLs and Bedrock's session token carry it; region now stays up
front where it belongs.
* Cover submitting the provider connection form with the enter key
* Treat the single-provider env vars as a supported way to configure a provider
They were named and documented as a legacy compatibility shim — `legacy-env-connections`,
`migrate-legacy-provider-connections!`, and a doc line promising they "keep working".
That undersells them: configuring one provider by environment variable is the simpler
option when you do not want to hand-write the `llm-providers` JSON, and it is resolved
from the environment on every read, so editing a variable is picked up rather than being
frozen into a copy at first startup.
Rename to say what they are, and reword the docs to position the two mechanisms as
alternatives rather than old and new. The one-shot at startup only ever adopted
credentials stored in the app DB, so name it for that instead of for migrating away from
something we are keeping.
* Type the LLM provider logo lookup and move it out of the icon component
The logo map lived inline in ProviderTypeIcon as a Record<string, string>, so a new
provider type silently fell back to the generic icon instead of failing the build.
Give provider types a LlmProviderTypeName union and key the map on it, so adding a type
without deciding on a logo is a compile error. The map moves to metabot/constants.ts and
the lookup to metabot/utils/llm-provider.ts, matching how databases/ splits ENGINE_LOGO
from getEngineLogo.
* Focus the first credential field once a provider is picked
* Show the Metabase logo for the managed provider
* Only badge a provider connection when something is actually wrong
The green check was driven by `usable`, which only asks whether the required fields are
non-blank. The connect form already refuses to submit until they are, so every connection
made through the UI got a check — including one whose API key the provider rejects, right
above a "Couldn't load models" error saying so.
Drop the check and keep the warning, which still carries a real signal for connections
that arrive incomplete from MB_LLM_* or MB_LLM_PROVIDERS, and label it so it says what is
wrong rather than just showing a red triangle.
* Lighten the provider list to divider-separated rows
Each connection was a bordered Card, but the list already sits inside SettingsSection,
which is itself a bordered, padded card — so every row read as a card inside a card.
Follow DatabaseInfoSection instead: one container, rows separated by a divider. Also drop
the icon from 40 to 32, since the tile no longer has to anchor a card of its own.
* Tighten the provider list spacing and align the add button with the rows
The list and its add button sat in the same lg-gap stack as the model picker, so a control
that belongs to the list was pushed 24px away from it, as far as the next setting. Group
them in their own sm stack and keep lg for the gap to the picker.
The subtle button also carries horizontal padding, so its + sat indented from the provider
logos below it. Drop the padding, matching the Advanced settings toggle on the connect
form, so both start on the same edge.
Tighten the section header gap from lg to md via the stackProps SettingsSection already
exposes for this.
* Fetch provider models once and report failures on the provider that failed
A model-loading failure was only visible under the model picker, which reads as a problem
with the picker rather than with Anthropic. It also meant the failure was invisible
anywhere a picker was not rendered.
/api/llm/models already returns every connection in one response with a per-connection
error, and RTK Query already caches it under the llm-models tag, so the fetch is shared by
construction. Wrap it in useLlmConnectionModels, which hands back the grouped select
options and the errors keyed by connection. Any future picker goes through that hook and
gets the same cache entry and the same keying, rather than re-deriving either.
The provider row now renders its own error and the picker no longer lists them; the picker
keeps the request-level error, which is a different failure.
* Pick the provider from a pasted API key
Paste a key anywhere on the provider grid and it jumps straight to that
provider's form with the key filled in, ready to connect.
The registry already declared a `:prefix` per credential field — the backend
validates against it — so this just exposes it and reuses it for recognition.
Anthropic, OpenAI and OpenRouter carry one; nothing else is guessable.
Longest prefix wins, since OpenAI's `sk-` also matches an Anthropic
`sk-ant-` key. Text that matches nothing is left alone.
* Leave the paste-to-detect shortcut unannounced
Drop the hint under the provider grid. The shortcut stays; it is just for
people who find it. `canRecognizeApiKeys` existed only to decide whether to
render that line, so it goes too rather than sitting unused.
* Gate the managed provider usage panel on the features that fill it
The panel was shown whenever the metabot plugin was enabled, which is a
weaker condition than having usage and pricing to report. Move the check
behind a hasMetabaseManagedProviderDetails plugin hook so the enterprise
side decides from the premium features, and OSS answers false.
* Register the Mistral and Z.AI provider types in the connection registry
master added both providers to the single-provider settings architecture this branch
replaces, so rebasing dropped them from the admin UI: the adapters, the model listers and
the MB_LLM_MISTRAL_*/MB_LLM_ZAI_* settings all survived, but nothing offered either type
as something an admin could connect.
Registry entries restore that, and adding them to `single-provider-settings` is what keeps
an instance already configured through those env vars working — the same entry drives both
the env-resolved connection and the app-DB adoption at startup.
* Resolve provider connections in the LLM call tests
`parse-provider-model` stopped deriving a provider from the model string's first segment
and started resolving it against a configured connection, but this file still called it
with nothing configured, so every test that reached an adapter failed on "No LLM provider
connection named". Configuring the default connections is the fixture those tests were
always implying.
The invalid-format cases now assert the error the new resolution actually raises: an
unparseable string and an unknown connection are the same failure, because a name that is
not a connection is all "unknown provider" can mean once the registry is the source.
* Move the onboarding AI step onto the provider connection list
The setup wizard's AI step was built against the single-provider form this branch removed,
so it still imported `AIProviderConfigurationForm` and the provider enum and no longer
compiled. It now renders `AIProviderSetup`, the same connect-then-pick-a-model view the
ad-hoc modal uses, so onboarding and the admin page stay one flow rather than two.
Preselecting `metabase` goes away with the dropdown it was written for. The backend already
sorts the managed provider first in the type picker, which is what the preselection was for,
and picking it in a grid of logos is not a step the wizard needs to skip on the admin's
behalf.
* Add the Mistral logo and make a missing provider logo fail the build
LlmProviderTypeName is written by hand and had drifted: the registry gained mistral and
zai, the union did not, so the exhaustive LLM_PROVIDER_LOGOS record was exhaustive over a
stale set and both types fell through to the generic icon without a type error.
Add them to the union, which does make the record fail to compile until each is decided.
Mistral gets its mark from svgl; svgl has no Z.AI logo, so zai is an explicit null and
keeps the fallback icon until we have a real asset.
Nothing links the registry to the union, so add a backend test over the registry's type
names that names both frontend files to update.
* Thread per-connection credentials through the Mistral and Z.AI call paths
Both adapters took :credentials in list-models but ignored them in mistral-raw/zai-raw,
which read llm-mistral-api-key / llm-zai-api-key directly. So a connection's models listed
under its own key while every actual chat call went out under the global setting: a second
Mistral connection would silently answer with the first one's key, and a connection with no
global setting configured would fail outright.
Match the openrouter/claude shape — credentials first, configured settings for whatever
they do not carry — for both the key and the base URL, and cover the call path, which no
adapter had a test for.
* Give Z.AI its logo
Taken from z.ai's own logo.svg, since svgl carries no Z.AI mark. Stripped the Illustrator
export's ~10.5KB of unused .st0-.st194 declarations — only two of them were referenced —
and kept the geometry and colours exactly as shipped.
Every registered type now has a mark, so the fallback is only reachable when the backend
serves a type this union has not caught up with. Test it that way instead.
* Let instances that are only offered managed AI reach the purchase flow
The managed provider tile is enabled by `type-available?`, which asks whether
the LLM proxy URL is readable. That setting was gated on `metabase-ai-managed` or
`metabot-v3` — features granted only by the AI add-on products — while
`offer-metabase-ai-managed` comes from the starter and pro-cloud plans. Every
cloud instance eligible to buy the add-on therefore saw the tile as unavailable
and had no way to reach the purchase flow that grants the feature.
* Cover per-connection credentials on the remaining providers' call paths
Every provider tested credentials through list-models and none through the call path,
which is why Mistral and Z.AI could list models under a connection's own key while sending
every chat request under the global setting.
Claude, OpenAI and OpenRouter resolve credentials inline like Mistral did, so they get the
same explicit/blank pair. Azure and Bedrock instead forward :credentials into a shared
request helper the list-models tests already exercise, so what is worth pinning there is
the forwarding: with no settings saved, the request only goes out if the passed-in
credentials arrive.
* Pin every provider type's default model
The removed settings-put-connect-*-defaults-model tests asserted this per provider; the
connection API replaced them with one test that a first connection gets a model selected,
and provider-types-test only spot-checked anthropic and azure. A typo in mistral's or
zai's :default-model would have shipped silently.
* Alias metabase.llm.test-util as llm.tu, not lct
The letters do not spell anything in the namespace, so at a use site like
lct/default-connections there is no way to tell which namespace it came from. The repo
convention for a module's test utilities is <module>.tu — lib.tu, notification.tu.
Add it to kondo's :consistent-alias map too; it was absent, which is why the alias could
be picked freely in the first place.
* Make the whole provider header toggle the usage details
Only the chevron was clickable, so the icon and name next to it did nothing — unlike
Accordion.Control, which is what makes the entire header hit the toggle everywhere else.
Promote the header itself to the control and leave the chevron as the affordance, with
the options menu as a sibling rather than nested inside it. SetByEnvVar moves below the
header for the same reason: it carries a docs link, and a link inside a button is invalid.
Not Mantine's Accordion, because expandability is per-row here — only the managed provider
has a panel, and wrapping the rest would give them a control that opens nothing.
* Let the provider modal play its closing animation
The modal hardcoded opened and the parent mounted it conditionally, so closing unmounted
it outright and Mantine never got to run the exit transition. Opening animated because
mounting still runs the enter one.
Drive it with opened instead and keep it mounted, the way ConfirmModal a few lines below
already does. Clearing the edited connection moves to onExitTransitionEnd so the title
does not flip to "Add a provider" on the way out.
* Fix the edit modal never closing
Openness was derived from isAdding || editing != null, but closing only cleared isAdding
and left editing to be cleared after the exit transition — which could not start, because
editing still held the modal open. Editing a provider trapped it.
Track openness in its own disclosure so closing is one flag, and let editing carry only
which connection to show, cleared after the transition. Adding worked before, which is why
the existing test missed this; cover closing from edit too.
* Revert the provider modal close-animation work
Reverts 7e78358857d and 1a7c37d6412. The missing close animation is not this component's:
the Modal theme attaches enter-only CSS keyframes (popInFromBottom, fadeIn) to every modal
in the app and animation.module.css defines no exit counterpart, so driving this one with
opened bought nothing but state to keep in sync — and the first attempt at it trapped the
edit modal open.
Back to mounting it conditionally. Keeps the fetch-mock teardown from the reverted commit,
which is unrelated hygiene: without it this spec's routes leak into
MetabotAgentDataSourcePills and fail it in a combined run.
* Serve a request from its own connection's credentials, nothing else
Every adapter fell back to the single-provider `llm-*-api-key` setting when
the credentials it was handed came up blank. That made sense while the two
systems coexisted, but the env vars are themselves connections now, so the
fallback could only ever fire by lending one connection another's key — a
connection whose key an admin cleared would silently keep working on someone
else's, while `connection-usable?` reported it as broken.
Resolution now fills each field's registry default into the config, so
`:credentials` carries everything the adapter needs, including the base URL
it used to read from the settings. The adapters take credentials and nothing
else, and throw when they are missing.
Tests drive the adapters the way production does — by passing credentials —
rather than by setting a global and calling with none.
* Finish removing the settings fallback: Mistral, Z.AI, and the docs
Mistral and Z.AI landed alongside the previous commit and were copied from
the old shape, so both still reached for their `llm-*-api-key` setting when
the credentials came up blank. A half-applied invariant is worse than either
state — the next adapter gets copied from whichever one you happen to open.
The docstrings across all seven still advertised the fallback I had already
deleted, including the canonical `LLMRequestOpts` contract. Code that lies
about itself in the one place people go to read the rule is how the rule
comes back.
* Drop the editorializing line from the credentials doc
The entry should say what the field is, not argue for it.
* Reject runtime writes to the per-provider credential settings
These configure a connection only when set by an environment variable, which
llm.provider/connections resolves on every read. Writing one to the app DB landed somewhere
nothing reads: the settings API returned 200 and the request kept using whatever the
connection held. The e2e specs are the proof — one pointed llm-anthropic-api-base-url at a
mock server and would have talked to the real Anthropic instead.
Mark all sixteen :setter :none, so a write raises "you cannot set X; it is a read-only
setting" instead. Tests keep working because with-temporary-setting-values passes
:bypass-read-only?; connections are managed through /api/llm/providers.
Give each a :doc, since env-var-dox drops a read-only setting from the generated docs unless
it has one, and format-config-name then labels them environment-variable-only.
The custom setters and normalize-llm-base-url are unreachable now, so they go; base URLs on
connections were never normalized, only the setting path was.
* Disable the managed-AI Connect button instead of hiding it
An admin who had not ticked the Terms box saw no Connect button at all, so
the modal gave no sign that connecting was possible or what unlocked it.
Show it disabled: the checkbox is now visibly the thing standing in the way.
The gate moves out of `connectAction`, which keeps meaning "there is nothing
to do here" — already connected, or not an admin — and stays null in those
cases so the button remains hidden.
* Let a connection name the model it serves, so Azure is selectable
Azure's listing endpoint returns the regional catalog rather than the customer's
deployments, so list-models always came back empty and the picker had nothing to offer: you
could connect Azure and never pick a model. Master collected the deployment alongside the
credentials; the connection API dropped that and nothing replaced it.
Give the registry a :model-field naming the config key holding the model, and Azure a
required Deployment field. /api/llm/models still makes the call — it is what verifies the
credentials — but reports the configured deployment instead of the empty list. The connect
form needs no frontend change: it is descriptor-driven, so the field renders itself.
Two things this pulled in. config-complete? now demands the deployment, which is right for
"can this connection serve a request" but wrong for the adapter's auth check, so that
gets credentials-complete?, which ignores the model field. And the environment path needs
to express the deployment too, or an env-configured Azure connection reports unusable, so
llm-azure-model joins the single-provider settings.
Also give the unsupported-model error a 400: the deployment is typed by hand, so a bad
prefix is a bad request, not an outage.
* Import the provider logos instead of pointing at asset paths
The filename map plus a path-building helper came from ENGINE_LOGO, but that is the older
pattern: 57 places import an SVG as a module, and DatabaseHostnameWithProviderField's
ProviderIcon does exactly this — vendor logos colocated in the component directory, imported
and mapped by provider name.
So colocate them and import them. The bundler resolves the reference, which means a renamed
or missing file fails the build rather than 404ing at runtime, and the path string stops
being something to keep in sync by hand. The map moves next to the imports, which drops
metabot/utils/llm-provider.ts and the constants entry entirely.
The per-file src assertions go: jest maps every SVG to one stub, so what is left to check
is which of the three marks renders — the Metabase logo, a vendor image, or the fallback.
* Judge the onboarding AI step on a usable connection, not the first listed
connections[0] answered three questions it could not answer well. A listed connection is
not necessarily one that can serve a request — one configured by env var with a field
missing reports usable false — so onboarding would call itself connected, hide the skip
button, and report that provider to analytics.
Ask for the first usable connection instead. The skip button also waited on the list being
empty, which is equally true while the request is in flight, so it flashed on every load;
gate it on isLoading as well.
The completed-step title inferred skipping from a missing label, but the label comes from
the provider-types request, so a connected provider with that request still in flight read
as skipped. Pass whether a provider connected separately from what it is called.
* Match the loading skeleton to the list it stands in for, and colocate it
It still described the card-per-connection layout: two 4rem blocks with rounded corners and
a gap between them. The list has been divider-separated rows for a while now, so the
skeleton was a size and a shape that no longer appear anywhere.
Rebuild it from the row it mimics — PROVIDER_ICON_SIZE plus the row's py="sm" — with a
divider between and the button that follows, and move it into AIProviderList.tsx beside
those constants, so the next layout change has it in view rather than a file away.
Cover it too: nothing asserted the skeleton rendered at all, which is how it drifted.
* Keep the Metabase mark on the managed provider when whitelabelled
LogoIcon resolves through PLUGIN_LOGO_ICON_COMPONENTS, which the whitelabel plugin fills
with the customer's logo. That is right nearly everywhere, and wrong here: this row is
Metabase's own hosted AI service, so a whitelabelled instance was labelling it "Metabase"
— the registry string is not rebranded — beside someone else's logo.
Use DefaultLogoIcon, as WelcomePage and SdkUsageProblemBanner already do where the mark has
to be Metabase's regardless of branding.
* Stop standing in for an unknown provider with a real one
Gemini is a provider we could plausibly add, and the day we do, every test asserting it is
unknown starts asserting the opposite of what it means. Use a name nobody will ship.
Same for the Azure family placeholder, which picked gemini for a family Azure does not
serve. The genuine google/gemini-2.5-flash model references stay.
* Match on the secondary button instead of nesting a && inside a ternary
isEditing ? (onCancel && <Cancel/>) : <Back/> reads as two branches but is three, and the
one that renders nothing is the one you cannot see. The rest of this component already
picks its shape with match, so this now does too — and onCancel is narrowed by the pattern
rather than guarded separately.
* Pick the Azure model provider from a dropdown again
The rewrite collapsed master's Model provider select and Deployment name
input into one text field where the admin types `openai/` as a prefix. Split
them back apart and compose the model from the two, so the wire family is
picked rather than spelled.
`:model-field` becomes `:model-fields`, a vector composed with `/`, which is
the shape the Azure adapter already parses back out. A required field the
registry gives a `:default` now counts as filled — the form shows it
pre-selected, and resolution supplies it — so the family select does not
block Connect on a connection that simply accepted it.
* Drop two explanatory comments
Both restated what the line below already says.
* Shrink the loading skeleton to the row's content height
The skeleton painted the full 3rem row, padding included, so it read as two
heavy slabs butted against the divider. Wrap it in the row's own py="sm" and
size the bar to the provider icon instead: the row still occupies the same
height, so nothing shifts when the list arrives.
* Name the provider on the closed model picker
A model name alone does not say who serves it, so the picker gave no way to tell which
provider was selected without opening it. With several connections — two Anthropic keys,
say — that is the part you most need to see.
The option label carries the provider, since that is what a closed Select renders. The
dropdown keeps rendering the model alone through renderOption, because the group heading
above it already names the provider.
* Only treat a usable connection as one worth moving on from
Setup moved past the provider picker as soon as any connection was listed,
while the step around it counted only usable ones. A connection that exists
but cannot serve a request landed between them: the picker was replaced by a
model dropdown with nothing in it, the step still offered to skip, and there
was no way to connect anything.
That is reachable, not theoretical — the single-provider env vars synthesize
a connection as soon as one credential field is set, so MB_LLM_AZURE_API_KEY
on its own produces one Azure needs three fields to use.
* Serve the disconnect test's stream from the connection under test
The fake OpenRouter server was reached through the per-provider base-URL
setting, which no longer feeds a request now that credentials come from the
connection: the stream went to openrouter.ai and the turn finalized as an
error rather than an abort.
Disabling clj-http decompression on the LLM request also had to move from
http/post to the fn the adapters actually call, so the aborted stream no
longer races a ZLIB EOF against the cancel.
* Stop shipping an icon name the frontend never reads
The frontend keys its logos on the provider type and fails to compile for a
type it has no logo for, so the registry icon was only ever reachable for a
type the server knows and the client does not — which cannot happen when the
two ship as one artifact. Seven of the eight entries said "ai" anyway, and
the eighth said "metabot" for the managed provider, which is drawn from the
Metabase logo component before the icon is consulted.
Drop it from the registry, the response, and the type, and let
ProviderTypeIcon own the generic fallback it was already defaulting to.
* Point the setup wizard's AI e2e tests at the connection API
All three drove the single-provider flow this branch replaced: a Provider Select that is
now a grid of buttons, and PUT /api/metabot/settings, which no longer exists. The Anthropic
one also aimed the mock server via llm-anthropic-api-base-url, which stopped being writable
when those settings became read-only — it now types the base URL into the connection's own
advanced settings, which is where a connection reads it from.
The managed test additionally stubs GET /api/llm/provider-types and /api/llm/providers: the
managed type is only connectable once the LLM proxy is configured, and that has no backing
in e2e, so it is patched the same way the token features already are.
* Match the required API key label the way the unit specs do
Mantine appends a required marker to a required field's label, so the setup
spec's exact-string lookup found nothing once the API key field came from the
provider registry with :required? true. The unit specs already match on a
regex; the e2e spec was the last one on the bare string.
Also name do-with-entitlements! for the thread-unsafe settings it binds, which
Kondo flags.
* Edit the stored connection list rather than the environment-resolved one
The environment winning on read is right, but writes rebuilt the list from
connections, where an env-configured connection replaces the stored one with
the same key. Saving anything else then dropped the shadowed credentials from
llm-providers for good, so taking the env var back off did not restore them.
Editing the stored list also lets PUT report a shadowed key as env-configured
instead of 404ing, and stops the startup managed-connection write from landing
in the app DB when the whole list comes from MB_LLM_PROVIDERS -- which would
have pointed Metabot at a connection that never resolves.
* Configure an Azure deployment from the environment the way the form does
MB_LLM_AZURE_MODEL landed under a :model config key that nothing reads --
azure declares :model-fields [:model-family :deployment-name] -- so an
environment-configured Azure connection composed no model and counted as
incomplete, which hid the Metabot UI instance-wide behind
llm-metabot-configured?. Replace it with one variable per config key.
The same completeness rule dropped Azure from the startup adoption entirely,
because the credentials that existed before this branch never named a
deployment: they carried it in llm-metabot-provider. An upgraded instance was
left pointing at a connection that did not exist, with Metabot failing on every
request. A deployment names what to call rather than what authenticates the
call, so stop requiring it, and recover the pair from the model reference
during adoption so the connection can still offer it in the picker.
* Add a provider alongside the managed one instead of cancelling it
Running out of included tokens is a reason to bring your own key, not to end the
subscription. "Use a different AI provider" was wired straight to the add-on
cancellation: it charged ahead with no confirmation, offered no provider to
switch to, and left the metabase connection and llm-metabot-provider pointing at
each other afterwards, so nothing on screen changed.
Let the button open its own modal again. AIProviderSetup short-circuits to the
model picker once any connection exists, which is exactly wrong here, so let a
caller start on the connection form; the managed connection is left alone and
the model picker follows the save. Removing the connection outright is already
the list's Remove action, which cancels the add-on server-side.
* Let the single-colour provider marks follow the theme
The marks were loaded through <img>, and an SVG in an <img> is its own document:
currentColor resolves against that document's initial color, black, with no way
to reach the color the tile sets. Anthropic, OpenAI and OpenRouter therefore
painted black on a near-black tile in dark mode, and Bedrock's navy "aws"
wordmark left only the orange smile visible.
Inline those four instead, so currentColor picks up the tile's text colour --
dark on light, light on dark, which is how each brand publishes its monochrome
mark. Mistral, Z.AI and Azure keep their own colours and stay images. Bedrock's
wordmark becomes currentColor, reproducing AWS's own light and dark variants.
OpenRouter pointed at a clipPath the file never defined; harmless while it was
sandboxed in an <img>, but an id to collide with once inlined.
* Optimize the provider logos and normalize them to one size
The files came from seven different sources and showed it: viewBoxes ranged from
24x24 to 512x512, some carried fixed width/height, and each baked in its own
arbitrary padding. Rendered into the same 20px tile the marks came out anywhere
from 12px to 20px tall, so the row read as a jumble.
Run everything through SVGO (viewBox kept, dimensions and editor cruft dropped),
then re-cut each viewBox to a square centred on the mark's real ink extent. Every
mark now fills the tile's width, with padding controlled by the tile alone rather
than by whoever exported the file. Z.AI is inset 12%: its mark is itself a filled
rounded square, so at an equal bounding box it reads far heavier than the open
marks.
* Say when the provider lists fail to load
Both components defaulted the two queries to empty arrays and never read
their error, so a failed request was indistinguishable from having nothing
configured. The provider-types failure was the worse of the two: it rendered
an empty picker grid — no providers to choose, no explanation, no way on.
The connections failure is quieter but wronger, since an admin with several
providers would be told to connect their first one.
Report the error where the list or the picker would have gone, using the
same getErrorMessage the rest of this feature already uses.
* Tighten the provider section's spacing and hang the icon off the name
Sloan flagged three gaps as feeling high even where they match other sections: drop the
section's header gap from md to sm, and the gap out to the model picker from lg to md. The
row-to-button gap was already the tightest of the three and stays.
The icon was centred against the whole text block, so it sat slightly off the name and
drifted further as an error or env-var line appeared beneath it. Give the name the icon's
height and centre it there, so the two line up on one row however many follow.
* Collapse the add button's box, which was most of the spacing
Trimming gaps barely moved this because the space around "Add another provider" was not a
gap. Metabase's Button defaults to size md, so it is 42px tall; p={0} removes the padding
but not the height, leaving its ~20px label centred with about 11px of dead space above and
below it. That is the two gaps flagged around the button, and no gap value could reach them.
Let it take its label's height once it is an inline link. Rows go from py="sm" to "xs" as
well, since a 32px icon does not need 8px above and below it.
* Give the add button its height back
Collapsing it to its label bought vertical space but cost the button its shape. Keep the
tighter rows from the same change.
* Draw every provider logo on one 24x24 grid
Making each viewBox square left seven different squares -- 24, 88, 300.23, 256,
259.53, 511, 30.72 -- so the files only matched once a consumer sized them. Bake
each mark's scale and centring into its path data instead, so every file is
literally width=24 height=24 viewBox="0 0 24 24" with the mark centred on that
grid and its longest side spanning it.
Z.AI keeps its 12% inset, for the same reason as before: its mark is a filled
rounded square and reads heavier than the open marks at an equal box.
* Sit the row actions on the icon band and cut the gap to the add button
* Fit a provider's name and type inside the icon's height
* Give each provider row a little more room
* Inline the remaining provider logos instead of loading them by URL
Mistral, Z.AI and Azure rendered as broken images. Their files are valid; the
copies the bundler emitted were not. BannerPlugin is configured with no test, so
it prepends the licence header to every emitted asset -- and re-applies it on
each watch rebuild, so the served .svg started with six stacked comment blocks
before its <svg>, which no browser will parse as an image. Editing these files
repeatedly is what re-emitted them and made it visible.
Inline all seven the way the other four already were: SVGR turns them into
modules, so nothing reaches the asset pipeline and one import style covers the
set. Azure's gradients were on ids a/b/c, which are too generic to share a
document with, so they are namespaced now.
The banner covers every other emitted asset too, so any .svg re-emitted during a
dev session hits this; that is worth fixing in the build config separately.
* Call the managed provider Metabase AI service and tighten the add-provider modal
Drops the redundant 'Selected provider' subtitle and 'About Metabase AI service'
heading, puts the free-token and pricing lines in their own callout, and swaps the
row's ellipsis button for a neutral ActionIcon.
* Pin the base URL the credential-adoption tests read
* Mock the provider endpoints the onboarding AI modal calls
* Give the explorations e2e helper a provider connection instead of read-only settings
* Make Moonshot AI a provider type you connect to
* Regenerate the settings docs
* Answer the review on the model listing endpoint
- Drop the per-provider :group heuristics. The picker groups by connection
name and never reads the field, so it was computed on every admin page load
for nothing.
- Key the model cache on a hash of the connection's config rather than on the
config itself, so a credential is not held in a process-wide cache, and a
rotated one gets its own entry instead of the old list.
- Take the model a connection composes from its own config into account when
Metabot's selection has to move: Azure has no default model, so it was never
a fallback, and editing its deployment left the selection naming one that no
longer exists.
* Migrate the per-provider credential settings instead of adopting them at startup
The startup adoption left the settings it copied from behind, frozen: nothing
writes to them any more and they are read-only, so an instance rolled back to
code that still reads them would run on whatever was last stored there rather
than on what is configured. A reversible migration fixes both ends — it clears
the settings on the way up, and writes the connection list back into them (and
rewrites a model reference older code cannot resolve) on the way down.
Also pulls the copy of `trimmed-string` the two LLM namespaces shared into
`metabase.util`.
* Wait for the model list before opening the picker in its spec
* Make Google Gemini a provider type you connect to
Carries the Gemini Enterprise Agent Platform provider over to the connection
list: a registry entry with the same form every other provider gets, credentials
resolved from the connection rather than from the `llm-google-*` settings (which
become environment-only like the rest), and the connection naming the model it
serves — the platform has no listing endpoint worth trusting, so connecting
validates the model with a free countTokens probe, the way Azure validates its
deployment.
* Cover the Google credentials in the settings migration
* Name the Gemini models Metabot works best with
* Use Moonshot AI's own mark, not Kimi's
The file was Kimi's rounded-square K — Kimi is Moonshot's chatbot, not the
company we connect to. Icon sets ship the two separately and it is an easy
one to pick wrong.
Replaced with the MoonshotAI mark, kept in the same shape as the other
single-colour logos here so it takes the theme's text colour.
* Give Google the form the upstream PR built for it
Three field kinds the registry could not express before, so the connection form
can offer what the settings page did:
- a fixed model list, replacing the free-text field and the paragraph of advice
about which IDs work;
- an authentication method to pick, rendered as a segmented control;
- a file to upload for the service account key, read into the config the way
every other credential is typed into it.
A field can now hang off another one's value (`show_when`), which is what makes
the credential follow the method — and a field the form hides is cleared on
save, so switching methods drops the credential it replaced.
* Let Gemini's models come from the shared picker
A connection only names its own model when nothing else can name it — Azure,
whose deployments its listing endpoint does not return, and which serves one of
them. Gemini's models are a known list, so pinning one per connection put a
second model picker inside the form and hid the other model from the one below
it. The list becomes the type's fixed catalog instead, the way the managed
provider's is, and both models show up per connection in the model picker.
A fixed catalog no longer skips the provider call: that is what verifies the
credentials, so it is made against the catalog's first model and the catalog is
what comes back. The managed provider still answers from the registry alone —
there is nothing to call, and instances that cannot reach the proxy would fail.
* Answer the migration review notes
Rename the frozen table's :credentials to :required-keys — they are the keys a
provider needs to count as configured, and "credentials" made the optional ones
sound like something else.
Stop the downgrade encrypting every value it writes. write-setting! ran
maybe-encrypt unconditionally, so base URLs, regions and the model reference —
settings declared :encryption :no — came out of a rollback holding ciphertext.
Reads decrypt opportunistically so nothing broke, but a setting should hold what
its declaration says it holds. Only the :sensitive? settings are encrypted now,
and a test pins the raw storage both ways.
The Google row is also renamed to Google Gemini Enterprise alongside the
registry, to keep the adopted connection's display name in step.
* Close the review's holes in the provider registry and its API
Redact the Google service account key. secret-field-keys covered only
:password fields, and the key is a :file upload, so GET /api/llm/providers
returned the whole credential — private key included — in plaintext. Fields can
now be marked :secret? regardless of input type, which also makes the masked
echo-back flow cover the key on edit.
Require what Google actually requires. Every field was individually optional
because either credential will do, so an empty config counted as complete —
usable: true on a connection that can serve nothing. Provider types now declare
alternative credential groups (:required-any): a service account key on its own,
or an OAuth token together with the project ID it bills to. Completeness and
validation both honor the groups.
Treat blank credentials as absent. The form clears the field it hides when the
authentication method changes, but the empty string it sent was stored and
passed to the adapter, which fed "" to the service-account parser — the
"no JSON input found" from the review. Blank values are dropped on save, and
the adapter trims what reaches it so an env-supplied blank cannot shadow the
other credential or the project ID a key carries.
Reserve the metabase connection key. Any type could claim it, splitting
managed-model-ref? from managed-type? — a bring-your-own-key connection whose
requests would be tagged as proxied in metabot_message and confuse the managed
free-tier limit. Only the managed type may hold the key now, a squatter blocks
managed connection with a clear message instead of colliding, and the unused
proxied-model-ref? is gone.
Also: per-field :validate hooks in the registry (Google's project ID and
location get the setting-specific validation the old defsettings had), select
values are checked against their options, with-entitlements takes a kwargs map,
and the cond-> in self.clj is realigned.
* Pick the Google model in the connection form
Connecting a Google provider validates the credentials with a countTokens probe
that needs a model, but the form collected none, so the probe ran against a
hardcoded default the chosen location may not serve — a confusing rejection the
admin had no control over, and no guarantee the model they wanted works at all.
Provider types now ship their fixed catalog and alternative credential groups to
the frontend. A type with a catalog gets a model picker in its connection form,
the probe runs against the picked model, and Metabot starts on it. The Connect
button also honors the credential groups, so it no longer offers to connect a
Google provider with nothing filled in.
* Quiet the kondo warnings CI caught
u/lower-case-en over str/lower-case in the reserved-key check, and the
validate-config! additions split into their own deftests instead of growing the
original past the one-concern-per-test lint.
* Report the provider Metabot landed on, not the first usable connection
The setup step resolved its connected provider with find(usable) over the
connection list, which is insertion order. Connect two providers during setup
and pick the second one's model, and the step still reported the first — in its
title and in what submitAiConfig records. Resolve the connection from the
llm-metabot-provider reference instead, the way the pre-connection-list code
compared against the saved value.
* Spell the field-level secret flag :sensitive?
Same word as the defsetting option it mirrors, so a grep for :sensitive?
surfaces the registry's secret fields alongside the settings.
* Give the LLM settings migration its own test file, and a managed round trip
Twice now a reviewer could not find the migration's coverage, because the tests
sat in the shared custom_migrations_test and referenced the migration only by
its changeset id. They live in a file mirroring the migration's path now, with a
docstring naming the change class that connects the two.
The round trip crisptrutski asked about also surfaced what he predicted: the
managed connection is not in the migration's provider table, so every rollback
of a managed-AI instance logged a false 'no equivalent setting: metabase'
warning. It holds no credentials and its model reference survives the rollback,
so it is dropped silently now, and a new test walks the managed round trip.
* Let the environment shadow connections field by field, and settings write through
Answers crisptrutski's review of the environment and settings story.
The MB_LLM_* variables now shadow a stored connection's config per field, the
way an env var shadows any other setting, instead of swapping the whole
connection out. A lone MB_LLM_ANTHROPIC_API_BASE_URL reaches the stored
connection's base URL rather than doing nothing; everything the environment
does not supply stays editable, and env_fields tells the form exactly which
inputs to disable. A standalone env connection still only comes into existence
when a credential variable is set — which is why Azure's base URL is no longer
marked as one: alone it used to synthesize an unusable connection that shadowed
the stored credentials wholesale. A metabase/ reference pinned by
MB_LLM_METABOT_PROVIDER synthesizes the managed connection it names, so that
reference resolves without an app-DB row.
The per-provider settings are a view over the connection list rather than
read-only. :setter :none broke config.yml provisioning outright — the
provisioner calls setting/set! on every entry and nothing catches the throw, so
the instance failed to start — and orphaned generate-sql and semantic search,
which still read llm-anthropic-api-key and llm-openai-api-key directly after
the migration deleted the rows underneath them. Getters resolve the connection
field with the environment and registry defaults layered in; setters write
through to the connection, creating it on first write and keeping the old
setters' field validation. Both docs files are regenerated: the config
template carries the settings again.
Also from the review: deleting the managed connection requires superuser — it
cancels the Store subscription, and everything else that can cancel add-ons is
superuser-only — and the confirmation dialog says so; the automatic
llm-metabot-provider writes (first connection, edit follow-up, delete
fallback) are skipped when the env var pins the selection, restoring what
check-not-env-shadowed! used to guarantee; the edit form starts the model
picker on the model the connection is actually serving and a changed pick
follows through to the selection for fixed-catalog types; and redact masks
every field of a provider type the registry does not recognize, instead of
none of them.
* Reconcile the restack onto bot-1875-gemini-models-fe
Mechanical conflict resolutions during the rebase left a few files at the wrong
side of the divergence; this restores the reviewed merge result: the shared
tool-function conversion in the Claude and OpenAI adapters, the settings
endpoints stripped from the relocated metabot API slice, and
gemini-3.7-flash — new on the Gemini branch — joining the fixed catalog.
* Restore the stop-reason propagation the restack dropped from the adapters
* Strip trailing slashes off base URLs when a connection resolves
* Slug-proof the reserved key, follow the effective config, and seed the models cache
* Count a blank-config connection as dropped on rollback
* Warn that deleting the anthropic or openai connection turns off the feature reading it
* Regenerate the environment variable docs against current master
* Drop the env-var doc section master removed in the backend hardening
* Update src/metabase/llm/settings.clj
Co-authored-by: Chris Truter <crisptrutski@users.noreply.github.com>
* Mark the dev rollback subqueries so the app-DB honeysql guard allows them
* Gate the settings-access delete test on EE, where advanced permissions exist
* Wait for the AI setup buttons to render before clicking them
---------
Co-authored-by: Mike Appleby <86076+appleby@users.noreply.github.com>
Co-authored-by: Chris Truter <crisptrutski@users.noreply.github.com>
* Add a versioned embedding provider SPI
Summary
- Add a runtime registry for embedding providers with readiness, model resolution, preparation, and batch embedding.
- Define immutable embedding-space descriptors and legacy deterministic identities.
- Validate provider results, dimensions, counts, and finite numeric vector components.
- Make SPI version 1 a compile-time constant; provider artifacts must embed the literal version they were built against.
Scope
This defines the neutral provider contract only. It does not migrate semantic-search consumers or ship a provider plugin.
Testing
- Registration and version compatibility.
- Identity stability and revision round trips.
- Registry snapshot consistency and malformed provider output.
* Rename shadowing local in provider/prepare!
The local bound from :prepare! shadowed the prepare! var (eastwood local-shadows-var).
* Add the Arctic embedder plugin
Summary
- Build a separate runtime plugin containing the pinned Snowflake Arctic embedding model and DJL/ONNX dependencies.
- Bundle arm64 and AVX2 exports, lazy-load native inference, and cap local batches.
- Register literal embedding SPI version 1 and derive immutable space identity from the exact model/runtime bundle.
- Fail readiness closed for unsupported platforms, musl, and glibc older than 2.34.
- Disable DJL network telemetry and revision-address bundle URLs so model upgrades cannot reuse stale cached content.
Scope
This PR ships and registers the provider but does not yet select it from semantic-search settings. Consumer integration follows upstack.
Testing
- Model bundle and plugin uberjar build tests.
- Catalog, classloader, platform-readiness, identity, cache-key, and batching coverage.
- Artifact-only real inference with zero outbound network attempts.
* Follow prune-provided-libs! rename in the embedder plugin build
* Merge nested let in embedder-plugin artifact test
Kondo flagged the inner let as redundant; fold both binding vectors into one.
* Deref the now-delayed core provided-libs in the embedder build
* Bundle MiniLM for in-process complexity scoring
* Activate the embedder plugin in its artifact smoke test
* Activate lazy embedding providers on demand
* Use plugin initializer in artifact smoke test
* Make lazy provider activation best effort
* Log lazy embedding provider activation failures
* Declare the embeddings plugin dependency
* Keep embeddings provider internal
* Activate the in-process embedder during bootstrap
* Fix embedder validation and optional plugin startup
* Declare the embedder plugin as a module
The deps graph only scanned src, enterprise/backend/src, and modules/drivers, so the plugin's
namespaces belonged to no module and anyone linting them got six unresolved-dependency
warnings. Scanning modules/embedder/src too lets the generator record what the plugin actually
takes from the host.
* Move embedder model caching into the weekly cache generator
Review feedback on #78310:
- Drop the redundant persist-credentials: false checkout options.
- Stop writing the model cache from the CI job. The weekly cache-generator
workflow is the single cache writer; the build job now restores only.
The downloader is content-addressed (SHA-256 verified, refetches deltas),
so a stale weekly cache still protects against HuggingFace rate limits
without affecting correctness.
* Bundle snowflake-arctic-embed-xs instead of Arctic-L
The bundled models are a test tier: they exercise the packaging, DJL
runtime, tokenizer, and pooling machinery without shipping the 1.1 GB
Arctic-L artifacts. xs keeps the same family, query-prefix scheme, and
cls pooling as Arctic-L at 23 MB per architecture. A release-tier build
with the full model comes later, when the plugin is published for
customers or offline use.
At ~90 MB of pinned downloads there is nothing worth caching, so the
cache-generator wiring is reverted and the CI job downloads directly.
* Fail loudly when a catalog entry lacks this architecture's export
resolved-model computed the embedding-space-id from
get-in spec [:architectures arch :sha256] without checking it was
present. A model missing this architecture's export would silently
bake a nil into the identity hash instead of failing, producing a
syntactically valid but bogus embedding-space-id.
* Pin the embedder plugin manifest name to one constant
"Metabase In-Process Embedder" was duplicated in core.clj (the
registered?/load-plugin! lookup key) and the artifact test. A rename
in modules/embedder/resources/metabase/embedder/metabase-plugin.yaml
without updating both call sites would silently disable the plugin,
since registered? just returns false for a name it doesn't recognize.
* Downgrade the hashed-table-name log line to debug
Hashing fires deterministically for every identifier derived from a
long provider/model pairing (e.g. "in-process" + the bundled xs
model), so it was logging a warning on every lookup rather than once
at creation.
* Cover the missing-architecture catalog guard
The error path added alongside the guard had no test, so the earlier
silent nil-hashing behaviour could come back unnoticed.
* Assert the reason on the missing-architecture guard
The message alone leaves `:reason :architecture-not-bundled` untested,
which is the part callers actually branch on.
* Drop the hashed-table-name log line
It fires from a pure name-derivation helper on every lookup. The table
creation it hints at is already logged at info by the caller, and the
parameters behind the hash are in index_metadata.
* Extract the missing-architecture check out of the smoke test
The guard's assertions pushed the smoke test to 155 lines, past the
150-line deftest limit, so Clj-Kondo failed. Pull it into a helper the
way the platform-support checks already are.
* Update README with new image URL and bug reporting info
Updated the image URL for the Metabase product screenshot and added a bug reporting guide.
* put metabase-product-screenshot-updated.png to the docs
* fix link in the readme
---------
Co-authored-by: Timur Mashara <timur@metabase.com>
* Use Breadcrumb component in Detail View (GDGT-2596)
Detail View had its own Breadcrumb, duplicating the shared one introduced
in #75114. Point TableNav, ModelNav and CollectionBreadcrumb at
metabase/common/components/Breadcrumb and delete the local copy.
The shared component hardcoded showTooltip={false}, which would have lost
the full-text tooltip Detail View shows on truncated crumbs, so make it a
prop. It defaults to false, leaving every other consumer unchanged.
Crumbs now take the shared component's sizing (0.875em, bold) instead of
inheriting 14px regular; the separator is sized to match so the row stays
consistent. Colour and icons are unchanged.
* Use URL helpers for the Detail View breadcrumb links
TableNav built the database and schema URLs by hand. Extract databaseSlug
from browseDatabase so browseSchemaBySlug can reuse it, then build both
links from the helpers. Both now carry the database slug rather than only
the id.
* Style official collections in the Detail View breadcrumbs
Collection crumbs hardcoded icon="folder", so an official collection
rendered like a regular one. Use CollectionBadge, which resolves the icon
from the collection's authority level, the same way the model page does.
Drop the text-secondary override too: it made Detail View darker than
every other breadcrumb in the app bar.
* Align Detail View breadcrumbs with the rest of the app bar
The detail view nav carried a -13px left margin, putting its first crumb
13px closer to the logo than the collection breadcrumbs shown on every
other page. Drop it so both start at the same offset.
* Match the Detail View breadcrumb separator to the app bar
Its color, font size and line height all differed from the separator the
collection breadcrumbs render. Use the same values, and Box rather than
Text so it inherits the line height instead of setting its own.
* Update the database breadcrumb href expectation
The database crumb now goes through Urls.browseDatabase, which appends
the slug, so the link is /browse/databases/1-sample-database rather than
/browse/databases/1.
* Make query builder breadcrumb text the same size (GDGT-2609)
The database and schema crumbs rendered smaller than the rest of the
breadcrumb. HeadBreadcrumbs spread {...rest} after className, so a caller
forwarding className -- even as undefined -- wiped the computed Container
and headVariant classes, and those crumbs fell back to Breadcrumb's
0.875em. Merge the incoming className instead of letting it clobber.
Size the head-variant dividers and the leading icon to match the crumbs
too, so the whole row reads as one line of text.
* Address review comment
Keep the props spread last and exclude className from it, rather than
reordering the spread ahead of className.
* Stub the evals trigger
* use ref arg
* dummy to trigger build
* Clean up
* Use appropriate token
* Simplify pr message
* Use .evals_ref for default ref
* Comment cleanup
* Adjust ref
* Use evals branch
* Use general automation token
* readme for .evals_branch
* Use main of evals for label trigger
* Revert "dummy to trigger build"
This reverts commit e6502c6ff076a47d94eb829b516fc45b37cd7680.
* whitespace
* Address review remarks
* Use ai benchmark naming so we can trigger from pr
* Address review comments
* Remove now redundant branch pinning
* Review remarks
* Enable passing commit to uberjar from other workflows
* Unify ai-benchmark and pr-env build logic
* linter
* checkout action
* separate build step
* Use wip branch
* adjust job output
* Make image building manually triggerable
* tmp comment
* Address review remarks on the build extraction
- ai-benchmark's `image` regained the label guard `uberjar` used to carry; without
it any label on any same-repo PR built ee+oss and pushed to ECR.
- New `resolve` job turns the dispatch `pr`/`ref` inputs into one full sha, and
feeds it to both the image build and the eval-run dispatch. `pr` was ignored
before, so `pr: N` benchmarked the launch branch while reporting against the PR;
`ref: master` looked for an artifact named after a branch.
- prepare-uberjar-artifact takes a `commit` so the upload key matches what
check-existing-artifacts/fetch-artifact look up. Without it a build for an
explicitly passed commit uploaded under the event sha and then failed to
retrieve itself.
- uberjar's build checkout honours an explicit commit, but stays on the event ref
otherwise so PR builds keep compiling the merge ref.
- Concurrency group includes the calling workflow, so a re-label on one caller no
longer evicts the other's queued run.
- ECR existence check targets the pushed registry explicitly and no longer hides
real errors behind "not found".
- Scoped `secrets: inherit` down to the two secrets dockerize needs, added
read-only default permissions, dropped the dead `jar xf`, restored the comment
explaining `finalize`, and reset EVALS_BRANCH to main.
* Point smoke evals back at the evals-docker-registry branch
Temporary while metabase/evals#32 is unmerged — the ECR image logic is only on
that branch. TODO comment retained so this cannot merge as-is.
* Align ai-benchmark pr/ref precedence with evals eval-run
eval-run.yml resolves both inputs with `ref` overriding the PR head; this
resolved `pr` first, so the same two inputs meant opposite things either side of
the dispatch. Flip precedence and the run-name to match, and point the `ref`
description at eval-run's containment rule instead of claiming pr wins.
* Cleanup the uberjar worfklow
* Smoke should be triggered on !cancelled()
* Reset pointer to evals branch
* Give List, Map, ObjectDetail and PivotTable their own definition file
* Let a visualization register a loader for its component
* Load every chart component on demand
* Load the chart chunk while the query is still running
* Delete the echarts lazy boundary the chart split made redundant
* Dedupe modules shared by the SDK's async chunks (#79748)
The chunked SDK entry's splitChunks filter only matched chunks whose
name starts with "embedding-sdk-chunk", so the unnamed chunks created by
import() were excluded from splitChunks entirely. With every chart now
loaded through import(), each echarts-based chart chunk bundled its own
copy of echarts, zrender and the shared viz code, growing the reachable
SDK total from 3.3MB to 6.2MB gzipped.
Include async chunks in the filter and add an asyncCommons cache group
(minChunks: 2) so shared modules move into a common async chunk. The
vendors group keeps its own name-based filter so async modules are not
merged into the eagerly loaded vendor chunks. Initial payload is
unchanged; the legacy monolith and the bootstrap stay excluded.
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
* Lower the size floor for the SDK's shared async chunks
* Load chart components before a story renders
* Retry a chart chunk that fails to download
* Keep one loading state while a chart chunk loads
* Delete the leaflet and pivot-table lazy boundaries the chart split made redundant
* Say why onInit is late now that the renderer is not lazy
* Register the charts a test renders before it runs
---------
Co-authored-by: Nicolò Pretto <info@npretto.com>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
* Adapt the collection bulk action bar to pinned, unpinned, and mixed selections
* Align the bulk action bar per-state actions with the Figma spec
* Show the bulk action overflow for unpinned selections and fix its trigger styling
* Move bulk action logic into UnarchivedBulkActions and address review feedback
* Center the bulk action overflow trigger icon via a label class
* Make BulkActionButton code nicer
* Address review comments
* Don't test MySQL/MariaDB AppDB on anything but latest outside release/master.
* generate matrix up front
# Conflicts:
# .github/workflows/app-db.yml
* bugfix
* parameterize it all
* make things more uniform
---------
Co-authored-by: Ryan Laurie <iethree@gmail.com>
* Use an isolated executor pool for test isolation
* Fix another test
* Fix some other likely flaky tests
* Consistently use the mock executor in the tests that need isolated behavior.
* Check via .getDataset instead of .listDatasets when all datasets are exactly named
* Use perf/every?
* Handle some corner case differences between .listDatasets and .getDataset method behaviors
The `latest_qe` CTE joined `query_execution` to its per-card max
`started_at`, which returns every execution tied at that timestamp
rather than one. A card whose two most recent runs landed in the same
clock tick therefore fanned out into two identical rows on
`/monitor/errors` — same card_id, duplicate row keys in the table, and
an inflated `total_count`. Rank the tied executions and keep one.
Also give the ORDER BY a unique tiebreaker: no sort column is unique,
so tied cards were ordered arbitrarily and could repeat on one page of
LIMIT/OFFSET results while going missing from another.
* Update SelectButton styling to match Select (#GDGT-2705)
SelectButton diverged from the new Mantine Select: bolder text, a smaller
lighter chevron, a taller radius and, on long values, wrapping to two
centered lines instead of truncating.
Match the md Select input on every measurable property — 40px height, 4px
radius, regular weight, left aligned, 16px chevron in text-primary, and
Select's disabled tokens — and truncate long values with an ellipsis. The
clear affordance now sits alongside the chevron rather than replacing it,
matching Select's layout.
Migrate the component from Emotion to CSS modules while here.
Also fixes three Select issues surfaced by comparing the two side by side:
- long values cut off mid-glyph instead of ellipsizing
- with a clear button, the value ran underneath it, because the right-hand
padding was pinned to a flat 40px while Mantine widens that section to
60px (md) / 41px (xs)
- the disabled right section painted an opaque background over the field's
translucent disabled fill, showing a square-cornered box on any
non-white surface
* Remove comment
* Address review comments
- use !! instead of a null check, onClear is never null
- drop size={16} on both icons, it is Icon's default
- drop overflow: hidden from .SelectInput; the input already computes
overflow: clip, so the declaration never applied and the ellipsis works
without it
- cover sm, lg and xl in the clear-button padding fix, not just md and xs
* Stop hydrating the collections entity slice
* Delete the unread collections entity slice
* Drop the orphaned createMockEntitiesState import
* Wait for the collection query in the TableCollection link test
* Await the collection tree in the SavedEntityPicker tests
* security center: don't let a failed matching query override the version check
an advisory whose matching query names a table the instance doesn't have yet
fails at parse time, and we mapped that to :error. the notification task treats
:error the same as :active, so instances that the version range had already
ruled out got notified anyway. that's how SC-2026-015 reached 1.58.
affected means in-range AND matched, so an undeterminable query result shouldn't
outrank a version check that already said no. same as SQL giving FALSE AND NULL
= FALSE. in-range instances with a query we genuinely couldn't run still report
:error.
* security center: split evaluate-advisory!-test to stay under the kondo length limit
* Read database and field display data from RTK Query, not state.entities
* Read table and database breadcrumb data from RTK Query, not state.entities
* Delete the legacy useTableQuery and useDatabaseQuery entity hooks
* Delete the legacy useDatabaseListQuery entity hook
* Keep FileUploadStatus on the mirror-backed table hook
* Add Gemini Enterprise Agent Platform provider settings
* Invalidate the cached models list when credentials change
A credential write can change which models the provider serves, e.g. a different Bedrock region, Google location, or
Azure resource.
* Give the FileInput placeholder and clear button a visible color
The theme resolves these to transparent making them invisible withouth the override.
* Review feedback: whitelist recommended models
* Fix useUpdateMetabotSettingsMutation import
* Add gemini-3.7-flash to list of recommended models
* Add llm-google-* settings for the Gemini Enterprise Agent Platform
The location, project ID and model become path segments of every request, so each is validated as URL-safe.
* Wire the google provider into Metabot provider settings
* Add Google Gemini provider via streamGenerateContent API
Add new provider supporting Gemini models via the Gemini Enterprise Agent Platform
- New ns `metabase.metabot.self.google`. Handles generic provider concerns like
- contructing appropriate endpoint urls for the given model and location
- validating caller-provided credentials via a countToken call
- credentials handling
- validating config
- remapping upstream errors with improved error messages
- toplevel request entrypoint
- New ns `metabase.metabot.self.google.stream-generate-content`
- handles wire format and aisdk chunk tranlation for Google's streamGenerateContent API
* Wire Google credentials into the admin API
Optional credential settings that are set by an env var no longer block connect or disconnect.
* Skip the provider setting write when shadowed by an env var
* Drop the unused llm-bedrock-configured? setting
Nothing read it: the frontend never referenced it, and its last backend caller went away when
configured-provider-credentials switched to provider-credentials-complete? for the bedrock branch.
* Extract the shared tool->function conversion into schema/tool-function
The mu/defn doc-stripping + filter-schema-by-features + JSON Schema transform boilerplate was copy-pasted across the
claude, openai, chat-completions, and google adapters.
* Review feedback: allow settings to be cleared even if env shadowed
There was a mismatch where check-not-env-shadowed! allowed nil values but set-setting-unless-env-shadowed! did
not. Update set-setting-unless-env-shadowed! to preform nil writes so that a disconnect fully clears the appdb
settings even if some are env shadowed.
Change `get_transform_python_library_details` to read from `:source` instead
of `:content` to match what the API sends. The code is wrapped in a Markdown
fence longer than any backtick run inside it, prefixed with a line telling
the model to treat it as data, and cut off at 100k characters with a
truncation marker when the library is bigger than that.
* Rename dashboard-click-drill and its unit spec to TypeScript
* Convert dashboard-click-drill to TypeScript
* Address review feedback: discriminate click-behavior types and replace non-null assertions with checkNotNull
* Deduplicate base intersections in click-behavior union types
* Keep remove-table button in the row when table names are long
The table selector laid its label button and remove button out in a Group
that wraps by default. The label button is a flex item, so it kept the
default min-width: auto and refused to shrink below the width of its
longest line. A long database or schema name made it claim the whole row
and pushed the remove button onto a line of its own, at the left edge.
Stop the group wrapping, let the button shrink, and truncate the label
lines instead. The lines use max-width rather than width so a short line
keeps the left alignment the Stack gives it -- Button centers its label,
so a full-width line would be centered.
Also space the two label lines apart slightly.
* Space the remove button with margin instead of padding
Padding right sat inside the ActionIcon, so its hover background ran past
the icon and the glyph sat off centre. Margin keeps the same gap from the
row edge with a symmetric 28x28 hit area.
* Use Ellipsified for the table selector label lines
Manual truncation clipped the text with no way to read the full name.
Ellipsified brings the overflow styles and a tooltip that appears only
when the text is actually cut.
Dropping the Stack's align=start matters here: with flex-start each line
sized to its max-content width, so it never overflowed itself and the
truncation check never fired -- the button label clipped the text
instead, without an ellipsis. Stretched lines re-expose the button's
centered text-align, hence ta=left.
* Drop no-op layout props from the table selector
The button root class duplicated miw={0}, which already lands as an
inline min-width. The Stack's w/miw/justify did nothing: it stretches to
the button label by default, the label already carries min-width 0, and
its height is content-driven. ActionIcon has min-width equal to its
natural size, so it cannot shrink and flex="0 0 auto" changed nothing.
Verified by removing each and re-measuring at 800/1440/2560px.
* Drop redundant truncation styles and the empty-state Ellipsified
Mantine's button label already sets overflow hidden, and Ellipsified
carries its own min-width 0, so both declarations restated what was
already there. The css module is back to its state before this branch.
"Select a table..." is a fixed short string that never truncates, so it
goes back to a plain Box.
* Add Data Studio guide page and reorganize sidebar navigation (#76707)
* adjustments
* transform tabs, e2e adjustments
* oss test
* redirect first timers to the guide page and clean things up
* PR Feedback
* OSS copy, re-adding tests
---------
Co-authored-by: Nick Fitzpatrick <nickfitz.582@gmail.com>
Notebook-built transform sources now reach Metabot as the same portable JSON that `construct_notebook_query` consumes, with tables and fields by name, while native SQL still passes through verbatim. Card entity IDs inside an exported query now come from the permission-checked content store, so a source card the reader can't open renders no query at all instead of falling back to an EDN dump.
* Invert visualizer click-object formatting via a Visualization prop
Visualization no longer imports formatVisualizerClickObject from the
visualizer module; DashCardVisualization supplies a transformClickObject
callback instead, so the dashboard side owns the visualizer coupling.
* Give the transforms and Data Studio pages their own chunks
* Give the monitor and account pages their own chunks
* Load the transform pages as one chunk
* Give the chart-rendering feature pages their own chunks
* Load the metric pages as one chunk
* Fetch the Data Studio Library metric pages when their route is matched
* Load the metric pages and their Data Studio wrappers as one chunk
* Say why the metric wrappers share the core chunk
* Split the app shell's dashboard selectors into their own module
* Declare the dashboard card actions beside their type constants
* Keep the dashboard reducer off the parameter and data-fetching actions
* Split the tab thunks off the tab reducer
* Fetch the embed setup modals when one is opened
* Give the dashboard its own chunk
* Keep the query builder URL sync from replacing a pending navigation
* Keep the table picker's automatic selection off a pending navigation
* Read the pending navigation from the router, not from a render
* Wait for the code-split embed modal before probing for its terms card
* Drop the note restating what import type already says
* Outrank the grid placeholder style the dashboard chunk now loads later
* Show column descriptions in the Detail visualization via an info icon tooltip
* Reveal the Detail column description icon on hover instead of always
* Reserve the description icon's width so hovering never shifts the Detail table
* Address review comments
* Remove delay showing info icon
* Add a shift-pressed hook and shift+click selection for rows and pinned cards
* Add a Select entry to the collection item overflow menu
* Cover shift+click and the Select menu entry with an e2e test
* Keep shift+click on row actions from selecting the row
* Simplify collection item selection callbacks
* Address collection selection review feedback
* Scope pinned card queries now that pinned items are listed as rows
A pinned item renders both as a card and as a table row, so unscoped link
queries by item name now match twice. Scope them to the pinned section and
wait for the pinned query before asserting.
* Address review comments
Drop the quarantine-conflict flag: a driver marked `skip` in ci-test-config
whose own `modules/drivers/*` files changed now simply runs, instead of
spinning up a runner that fails immediately and demands a `ci:run-<driver>`
label. Changes to another driver's files, or to shared code, leave the skip
in place.
Rename the decision logic's "quarantine" vocabulary to "skip", matching the
config's own `status: skip`. "Quarantine" now refers only to ci-conductor's
suite-level gating rules.
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
* Make useListSelect key-safe and pinned cards selectable
* Cover shared pinned and table selection with tests
Adds a CollectionContent selection harness and end-to-end coverage for
selecting the same item from its pinned card and its table row, and points
the pinned item menu helper at the card element so it still resolves once
the card becomes a selection target rather than a link.
* Swap the pinned card icon for a checkbox on hover in select mode
* Reset pinned card hover state outside select mode
* Target projected metric dataset requests in e2e
* Simplify pinned selection and select-all handling
* Simplify pinned selection inputs
* Share one selection entry between pinned cards and table rows
Pinned items now appear in the collection contents list (#79679), so the
pinned grid and the table render the same logical item twice. Both read and
write the single useListSelect entry keyed by id plus model, and select-all
covers the listed page once instead of composing a separate pinned set.
* Simplify selected pinned card styling
* consolidate driver CI labels under ci:run-<driver>
break-quarantine-<driver> and ci:run-<driver> were two spellings of the
same override: ci:run-* is checked at Priority 3, ahead of the quarantine
check, so it already forces a quarantined driver to run.
Drop the break-quarantine-* family and point the quarantine-conflict hint,
the mage help, and the workflow error messages at ci:run-<driver>.
* rename ci:all-cloud-drivers to ci:run-all-cloud-drivers
Every other driver force-run label is spelled ci:run-*.
Co-authored-by: Chris Vandoodewaard <chris.vandoodewaard@metabase.com>
* Stop the alert run-status icon from shrinking
The warning icon sits in a flex row next to the run's error message, and
with flex-shrink at its default of 1 a long message squeezed it down to a
few pixels wide while its height stayed at 16px.
* Use FixedSizeIcon for the alert run-status icon
Ten Metabot settings were marked :doc false, excluding them from
environment-variables.md. Remove the flag and write real descriptions
so MB_METABOT_NAME, MB_METABOT_ICON, MB_METABOT_SHOW_ILLUSTRATIONS,
the three system prompt settings, and the four EE usage-limit/
permissions settings show up in the generated docs.
* Give the admin section its own chunks
* Load the admin pages as two chunks
* Fetch the tenant people and group pages when their route is matched
* Fetch the permissions pages when their route is matched
* Export AdminApp and DataPermissionsPage by name
* Say why the tenant wrappers get their own chunk
* Show pinned items in the collection contents list
* Update e2e specs for pinned items appearing in the contents list
* Simplify the collection empty state offset to a plain vh value
* Simplify bookmark state, skip the pinned query in trash, and drop the dead toolbar prop
* Use a real pointer click for the pinned item menu so hover cannot be raced away
* fix: run source-card field-type inference before cross-stage inference (BOT-1604)
The repair pipeline mini-resolves earlier stages to type a later stage's
cross-stage field refs, but ran that pass before stamping base-type onto a
source-card stage's own bare-name field refs. The mini-resolve of a
source-card-sourced prefix then always failed schema validation and was
silently skipped, leaving cross-stage refs untyped and the query permanently
unrunnable for any multi-stage construct_notebook_query built on a saved
question/model.
* docs: fix stale pass numbering in repair pipeline docstring (BOT-1604)
Number the new source-card field-type pass 4.5 and keep the pre-existing
cross-stage pass as 5, instead of bumping cross-stage to 5.5 - this leaves
the other in-file references to "Pass 5" correct and matches the file's
existing fractional-pass-number convention.
- metric-details surfaces the metric's FK-less join dimensions (which the
consumer-framed dimension list silently drops), each carrying the exact
portable join clause to paste into `joins:` and an alias-qualified field
reference. Fail closed: an entry is emitted only when its join clause exports
and targets a real table; join-key columns are excluded.
- the :no-fk-path repair error is now actionable, pointing the agent at the
metric's /dimensions resource for the join to add.
Also fix a pre-existing option-key typo in resources.clj
(:with-queryable-dimensions / :with-field-values -> the ?-suffixed forms the
readers expect) so the plain metric resource correctly skips dimensions and
field-value fetches.
* docs - host the model button image in the repo
The troubleshooting guide linked the model button screenshot from
https://www.metabase.com/learn/images/models/model-icon.png, which now
404s with no redirect. Learn images were moved out of the marketing repo
into a CDN bucket, and that key space has been renamed several times with
old keys hard-deleted rather than redirected.
This broke the nightly Update Docs job in metabase/docs.metabase.github.io
on Aug 8-10. It only goes green today because the link gate tolerates one
broken link and the other one got fixed upstream.
Commit the image here instead of chasing the CDN. It matches how every
other image in docs/ is referenced, it gets versioned along with the
release branch so older doc snapshots keep the image they shipped with,
and it is the only form our link checkers actually validate: lychee runs
--offline (external URLs skipped) and the docs site's htmlproofer runs
--disable-external.
The source file was a WebP served with a .png extension, so it has been
transcoded to a real PNG - pngcrush in optimize-images.yml would have
failed on the original bytes.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* Optimized 1 images, saved 1.8 KB (1.7% smaller)
---------
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-authored-by: Metabase Automation <github-automation@metabase.com>
fix(ui): align Alert icon with text when there is no title
Icon box was locked to the 24px title row, so title-less alerts (e.g.
incremental transform warning) hung the icon above a single line of text.
Without a title, icon and close button now center on the first message line.
* BOT-1953: Suppress the Metabot sidebar on the conversation route
The full-page Metabot surface disables the app-bar button and the Cmd+E
shortcut so the chat cannot open on top of itself. That check compared the
pathname against /question/ask exactly, but #77549 added a second route
rendering the same MetabotAsk, /metabot/conversation/:convoId, which
MetabotAsk itself redirects to after the first message. The sidebar came
back from there on.
useIsFullPageMetabot matches either route and drives both suppression
sites. useIsAskPage stays strict for MetabotAsk's redirect, which must not
re-fire once it is already on the conversation route. The route path now
has one spelling, Urls.CONVERSATION_BASE_PATH, so a rename cannot silently
turn the suppression off again.
The conversation matcher pins the route shape rather than a prefix, with an
optional trailing slash because react-router matches the route either way.
Cmd+E had no coverage because a note in the test harness claimed tinykeys
shortcuts do not fire under jsdom. They do, bound to Control rather than
Meta since jsdom reports an empty navigator.platform.
* BOT-1953: Match the Metabot route predicates with react-router
useIsAskPage compared the pathname to /question/ask by string equality,
so /question/ask/, which react-router serves as the same page, left the
app-bar button and Cmd/Ctrl+E active on a full-page Metabot surface.
Both predicates now go through matchPath, the matcher the router itself
uses: trailing slash and case tolerated, extra segments and an empty
conversation id rejected.
* BOT-1953: Keep the urls module to plain URL builders
The two route predicates move into the metabot hooks that use them, which
now call matchPath directly; urls/ goes back to exporting only functions
that build strings. Their url-level specs go with them, since the metabot
ui and app-bar button specs already pin the behavior.
* remove dead workflows and support code
# Conflicts:
# .github/workflows/build-for-release.yml
# .github/workflows/main.yml
# .github/workflows/presto-kerberos-integration-test.yml
* remove dead clojure files
* Stop autopinning models and metrics on creation
* Fix e2e assertions that expected new models to be pinned
* Remove negative-case pin tests per review
* Add lazyModalRoute for modals under a code-split page
* Split documents into its own chunk
* Prefetch the documents chunk from links into it
* Prefetch the comments sidesheet with the document page
* Fix two Metabot tool failures surfaced by the reliability data
get_transform_details passed the transform's query map straight into the XML
formatter. clojure.data.xml reads a map in second position as an attribute map
and then throws on MBQL 5's namespaced :lib/type key, so every query-source
transform failed and only Python ones (no :query key) got through. That is 72
errors in 75 calls over 90 days, across 15+ instances. The query now renders as
SQL text, matching how the line below already handles :target.
create_sql_query has two implementations sharing one tool name, and neither used
the shared agent-error contract. The :sql profile's code-edit variant had no
error handling at all, so the :agent-error? exceptions validate-database-access
raises escaped uncaught and were tracked as tool failures. That variant errored
on 26% of calls against 0.04% for the :internal one. Both now delegate to
handle-agent-error, as the other tools do: the intended validation message goes
back to the model, and anything unexpected is rethrown so real defects stay
visible in the telemetry rather than being reported as successes.
Neither tool had a test; both now do.
* Address review: transform rendering and 403 classification
Transform source queries now render verbatim (data.xml escaped <, >, &,
which breaks old_string matching against the real query), python bodies
render, and the source type attribute is no longer a keyword literal.
Notebook-built queries fall back to EDN with the attached metadata
provider stripped, shared with transform->xml, whose string guard always
dropped the normalized (map) query.
validate-database-access tags the plain 403 from read-check so both
create_sql_query variants classify permission failures as agent errors,
and the pre-classification ERROR logs are gone; run-tool already logs
both paths at the right levels.
* Tolerate string queries in transform-query->text
Legacy transforms stored raw SQL strings under :query, and orphaned sources
skip normalization, so the query reaching this helper is not always a map.
dissoc on a string threw ClassCastException; the search formatter's old
string guard had tolerated exactly this shape. Strings now pass through
verbatim, and the helper's namespace gets its first tests.
* Use a doc link for escape-xml in the escape-xml-content docstring
* Read orphaned sources' string keys in extract-sql-content
Orphaned transform sources skip normalization and keep their JSON string
keys, so the keyword probes missed them and the model got EDN of a
string-keyed map instead of the SQL. Multi-stage queries now fall through
to the EDN branch too: stage 0 alone is partial SQL and hid the rest.
Also tags the read-check 404 alongside the 403, closing the race where
the database vanishes between the exists? check and the read.
Update E2E test timings with averaged data
Collected timing data from 7 days of test runs.
All specs with timing data were included (simple averages).
🤖 Generated automatically by update-e2e-timings workflow
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: GitHub Actions <actions@github.com>
We no longer upload JUnit XML to S3, so the action and all 47 call sites
go. Test results stay visible through the JUnit reporter and CI
Conductor.
`id-token: write` drops from minimal-tests and transforms-python-stress-test,
whose only OIDC consumer was the upload's AWS credential handshake. It
stays in run-tests, which must keep granting it to drivers.yml's ECR login.
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Brings `actionlint` (v1.7.12, the version CI pins) to zero findings, down
from 186.
Real problems, not just lint noise:
- `pr-env-destroy-manual.yml` used an unquoted YAML scalar containing
` #`, so YAML truncated the command at the comment marker and the step
actually ran `echo "Destroying PR environment for PR` — an unterminated
quote. Moved to a block scalar with the PR number in an env var.
- `yaml.yml`'s `yaml-linter` job read `needs.files-changed.outputs.yaml`
without declaring `needs:`, so its `if` was always false and the linter
never ran.
- `containerize-uberjar.yml` referenced `github.event.inputs.commit` in
three places, but its `workflow_dispatch` declared no inputs.
- The three Slack notification workflows interpolated
`github.event.issue.title` straight into a `github-script` body; it now
goes through an env var.
- Several step-summary blocks wrote `echo '... ${{ ... }}'` in single
quotes, which never expanded as intended.
The rest is mechanical: quoting `$GITHUB_OUTPUT`/`$GITHUB_ENV`/
`$GITHUB_STEP_SUMMARY` redirect targets, grouping consecutive appends
into `{ ...; } >> file`, replacing `ls | xargs basename` with a glob
loop, `A && B || C` with `if/then`, `[[ "$?" != 0 ]]` with `if ! cmd`,
and assorted quoting.
`containerize-jar.yml`'s `release` input is now a real `boolean` rather
than a string of `"true"`/`"false"`, which also removes the
`${{ 'true' }}` workaround at its call sites.
Adds `.github/actionlint.yaml` to declare the `arm-ubuntu-24-2core`
self-hosted runner label.
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
* Split the research catalog tool to keep its payload bounded
get_research_candidates returned every research-eligible metric with every
candidate dimension inlined — on instances with many metrics the JSON blob
overwhelmed the following LLM request's context.
Two-tier replacement:
- list_research_metrics (new): slim index — id, name, description
(truncated to 150 chars), in_library — no dimensions. Optional q filters
by substring on metric and dimension names.
- get_research_candidates now requires metric_ids (max 20 per call) and/or
q, and returns a deduped shape: same-group dimensions resolve to the same
underlying Field, so descriptive fields (name, types, interestingness)
are stated once per dimension group, with each metric's dimension id and
per-metric display name (renames are common) in
dimension_id_and_name_by_metric. A q match beyond the cap is ranked
library-first-then-interestingness, truncated, and stamped
{truncated, shown, matched}. Argument-less calls return an instructive
error instead of the catalog.
add_research_groups is unchanged — the ids the model echoes back are the
same per-metric dimension ids as before. explorations.selmer updated to
describe the new contract (factual sync only).
* Cap the list_research_metrics index
Load-testing with 3,000 metrics (~62k dimensions) showed the unfiltered
index was still a ~394 KB / ~100k-token blob — slim rows don't save you
when there are thousands of them. Above 500 matches the index is now
ranked library-first-then-interestingness, truncated, and stamped
{truncated, shown, matched} — the same contract as the
get_research_candidates q cap — so the model knows to narrow with q.
Under the cap, rows keep their natural library-first-then-name order.
At the 3,000-metric load the unfiltered index drops to ~66 KB; q-filtered
responses are unaffected.
* Code review cleanup
---------
Co-authored-by: Nathan Voxland <nathan@metabase.com>
* Declare two request params the handlers read but never declared
Request decoding drops keys a schema does not declare, so a handler that
reads an undeclared key silently gets nil.
- POST /api/card/pivot/:card-id/query destructured `parameters` and
passed it to `process-query-for-card`, but declared only
`ignore_cache` (and `dashboard_id`) — parameterised pivot queries
silently ignored their parameters. Declared exactly as the non-pivot
sibling in the same namespace does.
- POST /api/notify/db/:id destructured `synchronous?`, the documented
flag that makes the sync run synchronously, without declaring it, so
every call ran asynchronously.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
* Add regression tests for both undeclared params
Each test fails on master and passes with the schema fix:
- pivot: reuses the native card whose `{{category}}` template tag is
`:required`, so the query only runs if the request body's value reaches
the query processor. Without the declaration the request 500s.
- notify: blocks the redefined `sync-database!` on a promise and asserts
the response is still outstanding, which can only hold when the handler
derefs the sync future. Without the declaration `{:success true}` comes
back immediately.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
* Avoid row-constructor IN in multi-catalog Databricks sync (GHY-4263)
On a multi-catalog Databricks connection, describe-fields and describe-fks
filtered schemas with `(table_catalog, table_schema) IN ((?, ?))`. Databricks'
planner is orders of magnitude slower on that form against
`system.information_schema.columns` than on AND-ed equalities: on a Unity
Catalog workspace with ~220k columns the same query drops from 3m27s to 2.0s
when the predicate is rewritten. sync-fields issues one such query per schema,
so a ~50-schema sync took hours and never ran to completion.
Emit `catalog = ? AND schema = ?` per pair, OR-ed together for multi-pair
calls, instead of the row constructor.
* Update comment
* groundwork cleanup & gap fixing in provider adapters
* moonshot / kimi BE support
* FE support
* fix provider name & default model to kimi k3
* clean comments
* Honor suite-level quarantine rules in the gate
/api/quarantine serves a top-level `suites` array of glob rules alongside
`tests`. A rule matching the job's reporting suite quarantines the whole
suite, so the gate passes without comparing failures test by test.
Globs match as an unanchored, case-insensitive substring and support `*`,
`?` and `{a,b}` — a subset of the minimatch semantics ci-conductor
validates against, compiled inline because this package carries no
runtime dependencies and CI runs it straight from the checkout.
Both response keys are read as optional, so the script and the server
need no lockstep deploy.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* Drop the driver "info" status
Suite-level quarantine rules say the same thing from CI Conductor, for
every suite rather than drivers alone: run it, but don't let it gate.
mage now reads only "skip" from ci-test-config.json, and no longer emits
a per-driver status; the workflows drop the plumbing that carried it,
along with the job summary and PR comment that reported an info driver's
real outcome. The quarantine gate's own verdict and CI Conductor's UI
cover that visibility.
"skip" stays: not running a cloud driver at all saves real spend and has
no equivalent in the quarantine endpoint.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* Remove dead checks from drivers
* Tweak comment
* Drop the "info" status from notification
---------
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
The upload named the artifact after the event's own commit, while every
lookup around it — the concurrency group, check-existing-artifacts, the
bundle-stats upload and check-jar-health — keys on the dispatch input
first. A dispatch with an explicit commit therefore built that commit and
uploaded it under the dispatch branch's tip.
Inline the composite action into its only caller so the name comes from
the same expression as the lookups instead of a second copy of the
fallback chain.
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
* ignore missing sandbox/perms features when syncing
for consistency with other code, we should not worry about whether the
premium feature in question is actually enabled. If sandboxing is
enabled and configured and we're runnning EE code we should respect it
when a new table/group/db is synced.
* dedupe implied rows when setting data permissions
build-database-permission and build-table-permissions merge the results
of their recursive implication calls with `merge-with concat`, and
several implication paths can produce the same row:
- setting view-data to :blocked implies `transforms :no` both directly
and via the implied `create-queries :no`, so the DB-level path
INSERTed the transforms row twice
- coalescing table perms to a DB-level row (all tables agreeing on a
create-queries value) implies a DB-level `view-data :unrestricted`
row, and the recursive table-level view-data call coalesces to that
same row, so the table-level path INSERTed the view-data row twice
Nothing deduped the merged :to-insert list, so every such call silently
persisted duplicate data_permissions rows. Duplicate DB-level rows are
the arming condition for the UXW-4927 sandbox data leak: the
going-granular path for newly-synced tables only deleted one row per
(group, perm-type), and a surviving duplicate out-votes the new table's
:blocked row at enforcement time.
* make the going-granular DELETE+INSERT atomic
The bulk path for newly-synced tables issued its DELETE and INSERT
outside any transaction, so a failed insert — e.g. the unique-constraint
violation added in the next commit — could leave the DB-level rows
deleted with the table-level expansion half-written. Wrap the pair in a
transaction. Nested transactions are savepoint-based (see
metabase.app-db.connection), so this is safe under callers that already
run inside a transaction, like the Table after-insert hook.
* dedupe data_permissions and add a unique constraint
data_permissions had no unique constraint on
(group_id, perm_type, db_id, table_id), so nothing prevented duplicate
rows for the same scope, and duplicate DB-level rows armed the UXW-4927
sandbox data leak. Three v58 changesets (v58 is the oldest release this
will be backported to):
1. Delete existing duplicates, keeping the most restrictive perm_value
per scope, lowest id as tie-break.
2. Add a generated column unique_perms_helper = COALESCE(table_id, -1).
All three app DBs treat NULLs as distinct in unique indexes, H2 has
no partial indexes, and MariaDB has no functional indexes, so a
NULL-safe generated column is the only portable mechanism — same
approach as metabase_table.unique_table_helper (v55.2025-05-19).
Unlike that precedent the base column carries an ON DELETE CASCADE
foreign key, which MySQL rejects for STORED generated columns
(errno 1215), so MySQL/MariaDB use a VIRTUAL column; Postgres has no
virtual generated columns and no such restriction, so it keeps
STORED. Dedupe, constraint enforcement, and FK cascade verified on
Postgres 17, MySQL 8.0, and MariaDB 10.6.
3. Add a unique constraint on
(group_id, perm_type, db_id, unique_perms_helper).
Inserts stay fail-loud: after the previous commits, a constraint
violation can only mean a genuine bug, and it should abort the write
(the going-granular DELETE+INSERT now runs in a transaction) rather
than be silently absorbed.
The generated column is excluded from the model (after-select) and from
load-from-h2/dump-to-h2 copies, mirroring unique_table_helper. Tests
that inserted raw duplicate rows on top of the hook-created defaults now
set permissions through the API instead.
* sidebar empty states
* fix e2e test
* fix tests after rebase on master
* keep empty failed/canceled threads in the sidebar so they can be restarted
* review comment
* Prefetch a code-split page when a link to it is hovered
* Show a fallback while the router hydrates a lazy route
* Split the query builder into its own chunk
* Move the query builder's action types off the actions barrel
* Split the question selectors into their own module
* Read the question selectors from the leaf module in the app shell
* Import the query builder's view components directly in table editing
* Keep a pending lazy navigation from being replaced
* Prefetch the query builder chunk from links into it
* Drop the redundant job id from the granular-rerun artifact name
For most drivers `github.job` and `junit-name` are the same string, so the name
came out as `failed-tests-be-tests-snowflake-ee-be-tests-snowflake-ee-Driver
Tests-attempt-1`.
`junit-name` plus `artifact-suffix` already discriminate every call site in the
artifact namespace that run-tests.yml shares between app-db.yml, backend.yml,
drivers.yml and semantic-search.yml — including the `be-tests-postgres` job that
exists in both app-db.yml and drivers.yml, where the two derive the same
`be-tests-postgres-ee` junit-name but differ on suffix.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* Stop the rerun artifact restore from erroring when there is nothing to restore
An attempt that fails with nothing attributable to a var uploads no failed-tests
artifact, so the next attempt's restore legitimately finds none and falls back to
the full suite. That is the designed path, but `download-artifact`'s `name` input
treats a miss as an error, printing a red annotation on a job whose real failure
is elsewhere.
`pattern` filters the artifact list the action already fetches and downloads
nothing on zero matches, so this is the same single API call as before and costs
no extra rate-limit budget. The pattern is an exact literal, and artifact names
are unique within a run, so it matches at most one.
`merge-multiple` is deliberately left unset: a single match already lands at
`target/failed-tests`, and setting it would merge a hypothetical second match into
that one file — letting a matrix leg rerun another leg's failures — where the
default nests them and falls back to the full suite instead.
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-authored-by: Chris Vandoodewaard <chris.vandoodewaard@metabase.com>
* Render pinned collection items in a single grid of compact static cards
* Fix pinned-card e2e waits that relied on removed previews
* Register explorations as pinnable and movable drag types
* Address review comments
* Memoize search model hooks
Cache model hooks by the registered spec methods using a single-entry FIFO. This avoids repeated per-row hook construction while bounding reload-time cache growth and keeping invalidation best-effort.
* Trim the model-hooks cache test
The finally-clause recompute was dead: restoring the method restores the cache key, so
the probe's entry can never be hit again. Split the caching assertion out of the
invalidation test.
* Assert the model-hooks cache is warm before probing invalidation
The first call registers spec methods as it resolves models, so it caches under a key
that is stale by the time it returns. Probing from that state proves nothing: the miss
comes from the stale key, not from the redefinition.
* Note that the model-hooks cache key is only an invalidation token
* Set *warn-on-reflection* in the search spec test namespace
Clj-Kondo flags any interop call in a namespace that leaves the var unset; the
.addMethod probes are the first interop here.
* [Metabot] Pretty print tool call results
Split a tool call's result into Response, Structured output and Other
fields sections instead of dumping the raw JSON-encoded result map into
a single JSON code editor. Text output renders with real newlines, and
structured_output (also the kebab-case spelling kept by rows migrated
from the v1 format) gets its own section.
* Simplify the tool call result parser
* Centralize remote-sync read-only handling in useTransformPermissions
Fixes Create index (and other transform mutations) being offered in
remote-sync read-only mode and failing with 403. The hook now returns
combined readOnly plus granular permissionsReadOnly/remoteSyncReadOnly,
and all direct getIsRemoteSyncReadOnly usages across transforms are
replaced with hook values or props. Run button stays gated by
permissions only (running is allowed in remote-sync read-only).
* Rename RunButtonSection readOnly prop to permissionsReadOnly
* Drop redundant remoteSyncReadOnly from incremental switch disabled state
* Fix plugin setup order in indexes page spec
* Enable Kimi K3 via OpenRouter
* Enable DeepSeek V4 Flash via OpenRouter
* Raise the conversation-title output budget from 128 to 512 tokens
Reasoning models spend the output budget on reasoning before emitting the forced
tool call, so at 128 tokens the structured call often returns no tool call at all
and no title is saved.
* Fixed failing CI
* Add query timestamp, refactor error marker
* no-mbql-introspection pass
* Order temporal-axis exploration variants by date, not aggregation
When the row cap fires on a time series, aggregation-desc ordering keeps
the highest-count buckets scattered across the full range: gaps in the
x-axis, old spikes beating recent data. Temporal axes now order by the
temporal breakout desc (aggregation desc as tiebreak) before the limit,
so the most recent contiguous window survives.
- time-facet: order by the temporal breakout (second breakout) desc
- default: branch on dim type — temporal dims by recency, categorical
dims keep aggregation desc
- per-value-time-series: gains the same date-desc cap (had none, so a
single fine-grained series could blow query-caching-max-kb)
- all three also filter the NULL date bucket: it can't render on a time
axis, and its position under ORDER BY DESC is warehouse-dependent
(Postgres NULLs first, MySQL last), making cap survival
non-deterministic otherwise
- temporal-pattern-day/hour unchanged (bounded, no cap interaction)
* Added debug info to interestingness calculations
* even less mbql introspection
* oops, delete .scratch
* Sidebar UI fixes - error marker, loading shimmer
* Interestingness >= 0.7
* Adjust prompts list animation
* Fix e2e
* Note that the LLM query planner is a POC, not used in production
* Fix searching for explorations
* Use llm-metabot-provider for everything
* Log interestingless calculation as info for now
* Log interestingless calculation back as debug
* Fix FE lint
* fix ai summary showing dirty while generating
* Use llm-metabot-provider
* collection breadcrumbs
* add back null values for temporal variants
* order by for temporal pattern variants
* fix top-n-other
* doh, fix temporal pattern queries
* fix name in viz header, don't show be error in tooltip
* sort numeric bins in heat map header, fix tests
* UXW-4483 Add matching colors for category + breakout charts (#75912)
* Fix CI
* Fix units, fix loki
* UXW-4522 Update last query run timestamp in background
* Fix loki
* Fix e2e
* Improve Explorations viz types (#75958)
* UXW-4483 Add matching colors for category + breakout charts
* improve viz types first pass
* add error boundary
* add row chart support for dimension_value_colors
* Fix typecheck
* fix tests
* better handling of segments with row fallback, fix null color bug
---------
Co-authored-by: Denis Berezin <denis.berezin@metabase.com>
* UXW-4402: Make explorations respect sandboxing/impersonation/db routing (#75572)
* Make explorations respect sandboxing/impersonation/db routing
* some self-review fixes
* more cleanup
* test fixes
* doh
* Fixing CI failures
* capitalize 'By' consistently
* Fixed CI failure
* add loki snapshots
* Review fix
* Fixes after merge
* Rework Explorations UX (#76300)
* comment on exploration viz first pass
* wip
* wip
* wip
* UX changes first pass finished
* code cleanup, fix, add tests
* address review comments
* Code cleanup
---------
Co-authored-by: Nathan Voxland <nathan@metabase.com>
* Refine explorations with metabot (#75326)
* teach metabot about new exploration ux
* refine explorations
* address self-cr
* pass over FE
* CR response
* fix test
* Add :model/Exploration to api + eid-translation model-imports
These modules reference :model/Exploration after merging the latest
explorations base; declare the imports to satisfy model boundaries.
---------
Co-authored-by: Kevin Kelleher <kevin.kelleher@metabase.com>
* Harden explorations runner:
- planner crash recovery
- Bound timeline scoring
- add metrics
* UX tweaks
* Additional logging of errors on ai summary, plus increased thinking budget
* Improved timeline interestingness calculation
* Handle sonnet 5 etc better
* Make explorations groups a real thing (#76643)
* Rename ExplorationThreadGroup model to ExplorationBlock
Explorations organize their queries in a hierarchy: an Exploration has
Threads (one per run), a Thread has Blocks, and a Block is one
metrics-by-dimensions grouping that renders as a single sidebar heading.
That persisted grouping entity was named ExplorationThreadGroup, and the
word "group" was overloaded across the backend and frontend. Rename the
entity to Block — the name the frontend already uses — so the larger
data-model changes that follow start from one clear name. No behavior
change.
Renames :model/ExplorationThreadGroup -> :model/ExplorationBlock and
table exploration_thread_group -> exploration_block.
- Append a renameTable changeset rather than editing the createTable in
place: the original is already applied to running dev DBs, so an
in-place edit would fail checksum validation and force a DB rebuild.
- Rename the model namespace/file; update all call sites, the require
alias (:as block), and insert-blocks!.
- Update model registrations (resolution, serdes models, copy, entity-id
test) and re-sort the entries.
- Refresh the model's docstrings to describe a "block".
The planner's :thread-groups ctx key and the read-side group-tree are
left untouched here; they belong to later read-side work.
* Persist exploration pages with stable ids reconciled across reruns
Within a Block, an Exploration's queries are organized into Pages: a Page
is the bundle of queries for one (metric, dimension, query_type). Until
now a Page existed only as a derived, content-addressed key computed at
read time, so anything that wanted to point at a Page had to reference a
string that changed whenever the grouping definition changed. This makes
a Page a real persisted row whose surrogate id is its identity, so
comments (and stars, coming next) can anchor to a Page and survive both a
rerun — where queries are deleted and regenerated — and a future
re-grouping.
Introduce ExplorationPage: the persisted (block, card, dimension,
query_type) page within a block.
- New exploration_page table + model, FK to exploration_block (cascade)
and report_card. Registered in resolution, serdes (excluded), copy,
and the entity-id test.
- Add exploration_query.page_id (NOT NULL, FK, cascade): the reconciler
stamps it on every query, so it's required from the start. group_id
stays for now as the block pointer the read-side and deep-link URLs
use; dropping it waits for the read-side rework.
- Reconcile in insert-plan-rows!, the single insert point for both
initial Start and rerun: find-or-create a page per distinct key
(stamping page_id), then GC pages left with no queries. reset-thread-
for-rerun! is unchanged — pages survive a query delete (cascade is
page->query), so find-or-create recovers the same page ids on rerun.
- No DB unique index on the key: dimension_id is text (not portably
indexable on MySQL) and planning is single-threaded, so uniqueness is
enforced by find-or-create and matching is done in Clojure.
- Migration changesets appended (the table/group_id changesets are
already applied to the running dev db).
Tests cover page-id stamping, rerun id-stability, orphan GC, and perms.
* Return nested blocks→pages from the read API; drop query.group_id
An Exploration Thread's structure is Blocks, each containing its Pages,
each Page bundling the ids of its queries. Until now the backend instead
returned a hand-rolled flat list of typed nodes (sidebar/singleton/page)
linked by parent_group_id and keyed by auto:<...> strings, which the
frontend then re-nested. Now that Blocks and Pages are real rows, return
the nested shape the tables already encode: GET /:id returns :blocks,
each with its :pages, built from ExplorationBlock + ExplorationPage rows
and queries grouped by page_id.
- New metabase.explorations.blocks (renamed from .groups): blocks-tree
builds the nested shape; block headings + page names + score-sorting
ported over; block-display-name made public for the LLM planner ctx.
- Deep links go to a page by id: page-url → /question/research/<exp>/
page/<page-id>; the AI-summary and document-append builders pass the
primary query's page_id.
- Drop exploration_query.group_id (and its FK/index); a query reaches its
block through its page now — build-row-context (used by the runner to
finalize each row's MBQL) resolves the block via the page, which is what
the group_id read there used to do.
- API schemas: ExplorationQueryGroup → ExplorationBlockNode +
ExplorationPageNode.
- Migration changesets appended (the table is live on the dev db).
- Rewrite the read-model + endpoint tests for the nested shape; every
ExplorationQuery test fixture now hangs off a page (page_id NOT NULL).
The create payload key stays :groups for now (separate input contract,
touches the data-picker flow). FE consumption of :blocks lands next.
* Consume nested blocks→pages on the frontend
The frontend renders an Exploration Thread as sidebar headings — one per
Block — with a navigable item per Page (a Page being the bundle of
queries for one metric/dimension/variant). It used to receive a flat
typed-node `groups` list and re-nest it itself; this switches it to the
nested `blocks` array (block → pages) the read API now returns, and
renames the navigable unit from "group" to "page" throughout. Users see
the same sidebar and charts — this lines the frontend up with the
backend's shape.
- exploration.ts: add ExplorationBlockNode / ExplorationPageNode; thread
`.groups` → `.blocks`; remove ExplorationQueryGroup / its display-type;
the navigable id is now a page node id (kept as an opaque string).
- Sidebar utils build the tree directly from blocks→pages (one heading
per block, one child per page); score-sorting + empty-block filtering
preserved.
- The navigable unit is renamed group→page end to end: the route segment
becomes /page/<id>, matching the backend's AI-summary deep links. The
selection/url/comment-draft id stays a string via String(page.id), so
the existing string-keyed plumbing is unchanged.
- Visualization toolbar/header/comment-draft props renamed group→page;
the mark-as-interesting handler is left a stub for the stars feature.
- Specs, stories, and test-utils rewritten to the nested shape.
The create-payload request key stays `groups` (separate input contract).
* Retain comment-anchored pages across rerun GC; point comment URLs at /page/
A Page bundles the queries for one (metric, dimension, query_type), and a
comment can be anchored to a Page — comment.child_target_id holds the page
id. On a rerun an Exploration's queries are deleted and regenerated, and
any Page left with no queries is garbage-collected. A Page a comment
points at must survive that cleanup even when it ends up empty, or the
comment would resolve to nothing. This makes the rerun GC comment-aware
and moves comment deep-links onto the page route.
- gc-orphan-pages! now retains an orphan page when some "exploration"-
targeted comment's child_target_id equals the page id (compared as
strings). Empty + no comment is still GC'd as before. Page ids are
globally-unique PKs, so matching on child_target_id alone is
unambiguous.
- exploration-comment-url: /group/<child> → /page/<child>, matching the
page route and blocks/page-url; stale "group ID" docstring fixed.
- explorations imports :model/Comment for the retention lookup (comments
doesn't depend on explorations, so no dependency cycle).
- No backfill: the dev Postgres has zero exploration comments and zero
auto:* child_target_id rows, so the optional migration is skipped.
- Test: comment on a page → drop that page's selection → rerun → the page
is retained and the comment still resolves; a no-comment empty page in
the same rerun is still GC'd.
* Generate short and long page names from metric, dimension, and variant
A Page — the bundle of queries for one (metric, dimension, query_type) —
is shown in the sidebar under its Block's heading, but is also referenced
on its own elsewhere (comment deep-links, AI-summary chart titles,
search). Page names are derived, not stored, so a Page needs both a short
label that omits whatever its Block heading already shows (e.g. under a
"Number of Orders" heading: "By Category over time") and a full,
self-describing label for context-free places ("Number of Orders by
Category over time"). This generates both from the Page's parts — the
metric name, the dimension name, and a short per-variant qualifier —
rather than by string-slicing a finished name.
- variants/variant-qualifier: query_type → a short, param-free localized
qualifier ("over time", "(Day of week)", ...) or nil for the default
breakdown. Lives next to query-name so all variant naming is in one
place. Unit-tested.
- blocks: page-long-name and page-short-name generate from parts (metric
+ dimension + qualifier) using tru templates and the block's anchor;
page-node emits :name and :long_name.
- API ::ExplorationPageNode and the FE ExplorationPageNode type gain
:long_name (FE display wiring left for separate work).
- Same-dimension pages that differ only by variant (default vs
top-n-other) are now distinctly named instead of colliding; a
blocks-tree test that asserted the collision now asserts they differ.
* Add ability to star pages, sidebar tabs UX (#76755)
* add ability to star, sidebar tabs
* - renamed "user interestingness" to "starred"
- Let starred just be true|false, no null
- Removed DELETE starred API, just PUT now
---------
Co-authored-by: Nathan Voxland <nathan@metabase.com>
* Fix CI: test compile error, migration boolean type, e2e formatting
- documents/search_test.clj: qualify text->prose-mirror-ast with the
documents.test-util/ alias in the exploration-thread exclusion test.
The unresolved symbol broke backend test compilation, which cascaded
to every app-db/driver test job and clj-kondo ("0 tests" everywhere).
- 20260428_explorations.yaml: use ${boolean.type} instead of bare
boolean for exploration_page.starred (required post-v49 so MySQL
uses BIT(1)).
- oxfmt formatting for two explorations frontend files (fe-lint).
* Update e2e specs for reworked comment URL + explorations sidebar
- comment helper: match the node comment link by path only. The
`?new=true` marker was removed from comment URLs (base commit
"respect query params in all urls"); the CommentsMenu unit test was
updated but this e2e helper was missed. Verified the comment flow
still works (opening a node's thread shows the new-comment editor).
- explorations spec: align with the reworked sidebar —
* `by <dim>` leaf label is now `By <dim>`
* Scratchpad doc test: only Scratchpad is created synchronously; the
AI Summary doc is created asynchronously, so stop asserting it (and
the removed "Findings" grouping) at creation time.
* AI-summary completion test: the AI Summary doc row now hangs under
the thread heading ("Initial investigation"), not a "Findings" group.
* remove the two "Add to document" tests — that control was removed.
* Fix stylelint rule-empty-line-before in explorations CSS modules
Add blank line before nested &:hover rules in ActionToolbar.module.css
and ExplorationGroupVisualization.module.css. These surfaced in fe-lint's
lint-css step once the oxfmt formatting failure was resolved.
* - cleaned up migration file
- Improve group renaming in code
- preserve starts between re-runs like comments
* Fixing CI failures
---------
Co-authored-by: Kevin Kelleher <kevin.kelleher@metabase.com>
Co-authored-by: Nathan Voxland <nathan@metabase.com>
* Don't send empty messages to the LLM
* address review comments from 76755
* UXW-4765 Hide charts (#77488)
* UXW-4765 Hide charts
* Review fixes
* Fix the failing explorations unit tests and Loki stories (#77564)
* Fix the failing explorations unit tests and Loki stories at the branch tip
* Restore the original all-hidden copy and drop thread headings from it
* Delete the outdated ExplorationGroupVisualization stories and their Loki references
* Rephrase the hide toast to put the chart name first
* UXW-4764 Remove interestingness badges on results (#77544)
* UXW-4764 Remove interestingness badges on results
* Drop e2e assertion for the removed sidebar interestingness markers
* Colocate the interesting and error markers with their only consumers
* Remove the interestingness marker from the timeline picker too
* fix types, lints, formatting
* Uxw 4779 update tab menu to match designs (#77543)
* UXW-4779 Update sidebar tab menu to match designs
* Show a pointer cursor across the whole SegmentedControl
* Give the icon tabs img semantics and hide the decorative icon names
* Let getExplorationSidebarTabsInfo own the tab icon config
* Revert the SegmentedControl root pointer cursor, Mantine already puts it on the labels
* Regenerate modules config after master merge
The merge left 'explorations' and master's new 'entity-retrieval' out of
sorted order, which metabase.core.modules-test asserts on.
* Float the action toolbar pinned to the bottom of the chart canvas (#77667)
* fix types
* revert timelines in documents
* Review fixes: stop requiring the QP's internal cache.impl from other modules
Explorations code that saves and loads query-result snapshots was requiring
metabase.query-processor.middleware.cache.impl directly — an internal
namespace of the query-processor module that other modules aren't supposed
to reach into. Since the module's :api set is generated from actual usage,
those requires had put cache.impl on the module's public surface — something
master never exposed. Modules talk to each other through a small public
facade (metabase.query-processor.core) instead.
The two helpers those callers need — do-with-serialization (write a QP
result as a gzipped blob) and with-reducible-deserialized-results (read one
back) — are now re-exported from the qp.core facade, and the explorations
callers go through it. One consumer can't: queries/cached_result.clj is
itself loaded while qp.core is loading, so requiring the facade from there
would be a circular-load error at boot. It gets a proper home instead — a
new small, documented public namespace,
metabase.query-processor.result-serialization, re-exporting the same two
helpers. With nothing outside the module requiring cache.impl anymore, the
regenerated :api drops it, matching master.
* Review fixes: explorations planner and AI-summary polish
- Nested cardEmbeds: the AI-summary postprocessor wrapped only top-level
cardEmbed nodes in resizeNode containers, while the card materializer
walks the whole document — so a cardEmbed the LLM emitted inside e.g. a
blockquote got a real card but rendered collapsed to zero height. The
wrapper now walks the tree too (skipping already-wrapped nodes).
- Awareness-tier sort validation: phase 2 validates that embedded
categorical charts carry an explicit sort instruction, but the validation
set was built only from top-tier breakouts — an awareness-tier categorical
chart escaped validation entirely, and its prompt block (slim-block) never
told the model a sort was required. Both tiers are validated now and
slim-block emits the same per-chart sort directive as full-block.
- Contextual-scoring gate order: safe-score+describe resolved its scoring
context (a Card select and a SQL compile) before checking whether the row
is scorable at all (chart config present, creator known, non-blank thread
prompt) — wasted DB reads and a query compile for every unscorable row.
The cheap gates run first now. Its docstring also claimed the LLM gate
throws and is swallowed; it actually returns nil from a pre-flight cond.
- Discovery-cache TTL: the top-K category discovery cache was a
process-lifetime LRU, so a stale top-K result could be served forever on
a long-lived instance; entries now also expire after 15 minutes.
- Chart-name i18n: filtered-subset chart names built their multi-value
summary as (str (count values) " values") — a hardcoded, unpluralized
English fragment inside an otherwise-translated name. It uses trun now.
- Docstring/comment corrections: effective-cardinality documented (and
branched on) a :num-bins bucket that no caller can produce — branch and
docs removed; run-one-plan-iteration! said it runs "the LLM planner" when
the default is the mechanical one; a variants-test section comment claimed
temporal variants add a not-null filter that no variant adds.
Also repoints this module's result-blob serialization calls at the qp.core
facade (the consumer half of the cache.impl cleanup in this stack's first
commit).
* Review fixes: explorations API permissions, documents, and search
- Creating an exploration accepted arbitrary card ids (in blocks) and
timeline ids without checking the caller could read them, so you could
attach someone else's private metric or timeline and see its name/events
later. Both are read-checked at create time now, and timeline hydration
filters out timelines the viewer can't read. A duplicated timeline id used
to hit a unique constraint and 500 the request; ids are deduped.
- The restart endpoint re-selected the Exploration row after resetting the
thread, but the reset never touches that row — the redundant select is
dropped.
- GET /api/document listed documents that belong to exploration threads,
which search, recents, and collection items all deliberately hide. It
applies the same filter now — which as a side effect removes an N+1
pattern where each listed doc could trigger several permission queries.
- The document validator rejected {:type "doc" :content []} — the exact
empty document the backend itself creates for new threads. An empty root
doc is valid now.
- Search: is_published was deliberately hidden from search responses on
master (#66841) and this branch accidentally re-exposed it — hidden again
in both search engines. Exploration descriptions are searchable in the
legacy engine too, matching the new engine.
- Data picker: metrics and dimension groups now use the same
interestingness filter (a metric could previously reference dimension ids
that appeared in no group), search matches the displayed
"<group> - <dimension>" text, and a collection-ids query that ran twice
per request runs once.
- Consistency: the create event publishes after the transaction commits
(observers could previously fire before the data was visible).
* Review fixes: metabot / LLM infrastructure
- Asking Claude for extended thinking while also forcing a tool call
(tool_choice "required") produced a request Anthropic rejects with a 400.
The guard that already handled this for structured output now covers the
general tools path too.
- The e2e guard that stops tests from hitting a real LLM provider threw a
raw MalformedURLException on unparseable URLs; it now fails closed with
its normal friendly message — and it was moved out of a catch-all that
would have rerouted it, so it fails loudly.
- The timeline tools handed the LLM broken XML (an attribute quote and a tag
never closed) and interpolated user-written names/descriptions unescaped,
letting content break the markup. Output is well-formed and escaped.
- Hardening/docs: the retryable-error check bounds its exception cause-chain
walk at 10 (a cyclic getCause could loop forever); the
contextual-interestingness response schema now really forbids extra
properties (its docstring claimed it already did); docstrings about
thinking model buckets, return shapes, and inputs corrected.
- Also repairs three test breaks that already existed at HEAD: an arity
mismatch from the OpenRouter merge, a test asserting thinking blocks are
dropped (they now stream as reasoning parts), and tests that didn't stub
the LLM gate's newer usage/permission checks.
* Review fixes: lib-metric, metrics transforms, startup sync, interestingness
- group-by-source was re-exported for JVM Clojure but missing from the
ClojureScript branch of lib-metric.core; added.
- Metric/measure display-info inherited a default :long-display-name even
where the method overrode :display-name, so the two could disagree;
they're set together now.
- A failure in the startup metric-dimension sync aborted server boot
entirely; it's wrapped to log an error and continue.
- The interestingness "leftovers" backfill loaded every unscored field for
the database into memory in one select; it now streams one field at a
time, and the namespace docstring mentions this second recompute trigger.
- Added a unit test covering the dimensions :out transform (kebab-key
renames + type-value keywordization).
* Review fixes: enterprise perms/sync/deps, cached card query, test quality
- Remote sync: an event payload that simply omitted a key used to satisfy a
nil condition (e.g. "exploration_thread_id must be nil") — conditions now
require the key to be present with the right value, checked with
(find object k), which distinguishes a present-but-nil entry from an
absent key. Vector values in removal conditions rendered as broken
equality clauses; they render as IN.
- Dependency graph: smartLink nodes with nil/placeholder ids added nil to
the dependency set (the sibling cardEmbed branch already guarded this);
guarded, with regression tests for both branches.
- Test quality: the revisions exclusion test asserted the impl against its
own private set, so it could never fail; it checks a hardcoded expected
list now. Exploration bookmarks get real API coverage
(create/list/delete/ordering/archived), and a mislabeled test string was
fixed.
* Review fixes: v4 instance-analytics views with explorations usage sources
The AI-usage analytics views translate machine source names ("exploration",
"contextual_interestingness", …) into human-readable labels with a SQL CASE.
The explorations stack added its new sources to the v1 SQL files — but the
views actually installed on instances are v3, and nothing reinstalled them,
so users saw the raw machine names fall through the ELSE branch.
Adds v4 of ai_usage_log and metabot_conversations (v3 plus the new CASE
branches, with one consistent "Explorations" label — v1 said "Exploration"
for the source but "Explorations" for the profile), a migration that
reinstalls the views at v4, an updated pointer comment in usage.clj, and a
parity test that fails whenever a known source/profile is missing a CASE
branch in the latest views — exactly the drift that happened here.
* Review fixes: frontend (minimal) and e2e
Only the low-risk frontend fixes from the review pass, plus the e2e test
fixes; the other frontend findings are deliberately left out to keep this
branch's FE diff small.
- getDimensionIcon existed byte-for-byte in two files; the duplicate module
is deleted and its one importer now uses the copy in
common/utils/columns.ts.
- e2e: an assertion that searched for text inside another text node (and so
always passed) is meaningful now; readSelectedText no longer breaks
Cypress's retry behavior by asserting after a .then; and the seemingly
pointless GET /api/metric/:id in seedMetrics got a comment explaining the
side effect it exists for (it persists metric dimensions).
* Remove the LLM query planner
Explorations plans queries with the mechanical planner only now. Removed:
the LLM planner namespace, its prompt templates (plan.selmer /
repair.selmer) and prompt loader, the prompt-markdown renderers in the
planner-context namespace, the explorations-query-planner setting
(mechanical/llm/auto), and the :skip-no-llm exit for "planner requires an
LLM but none is configured".
pick-planner! now always returns the mechanical planner — kept as a seam
so tests can stub it and a future planner has an obvious dispatch point.
The planner-contract protocol and the shared context builder stay (they
are planner-agnostic), and the query_plan_transcript column stays (every
planner writes a transcript); its remark no longer describes an LLM
exchange.
* remove graph._dimension_value_colors (per bar colors)
* Add row count to explorations API (#77740)
Already available, we just need to persist it and thread it through the
API.
* Format fix
* Exploration UX... um explorations. (#76901)
* click thru
* better comment links
* chart avatars
* avatars
* allow subsubexplorations
* disable explore further for Other group in top n charts and dimension anchored blocks
* fix explore further for binned numeric columns
* comment highlighting, remove avatars
* comment label formatting
* map and table highlighting
* use built in viz click actions popover instead of custom
* fix comment badge in dark mode
* self review cleanup
* fix and simplify new thread toast
* fix be lints, add tests
* fix tests
* add fe, e2e tests
* - POST /:id/explore-further now verifies the clicked page
belongs to this exploration before copying its block, closing a hole where
any page in the instance could be copied into any writable exploration.
- key on the stable :explore-filters spec instead of a hash of
the per-row-reconstructed :dataset_query (which minted fresh :lib/uuids and so
never hit). build-row-context exposes :explore-filters on the row context for
the cache to key on.
Also reuse metabase.util/capitalize-first-char in the block title and thread
name helpers instead of hand-rolled str/upper-case.
* Code cleanup
* Gate exploration derived metadata by viewer's permissions
* fix lints
* Code review improvements
* Den/uxw 4741 explorations new sidebar (#77674)
* Update tabs view, add filter button menu
* Fix tests
* Add sidebar items vertical lines + icons
* Other sidebar changes - viewed state, sticky hierarchy
* Show sub-explorations near it's parent item
* Fix issues
* Self review
* Fixes after merge
* Self review
* Review fixes
* Fix empty states
* Review fixes
---------
Co-authored-by: John Swanson <john.swanson@metabase.com>
Co-authored-by: Kevin Kelleher <kevin.kelleher@metabase.com>
Co-authored-by: Nathan Voxland <nathan@metabase.com>
Co-authored-by: Denis Berezin <denis.berezin@metabase.com>
* Explorations self-review cleanup (#77717)
* Consolidate the explorations migration in place
The 063 explorations migration had grown by accretion: tables were created
and then altered by later changesets in the same unreleased file. Since the
branch convention is to edit unmerged changesets in place, the final shapes
are folded into the createTables:
- exploration_query gains params, nullable dataset_query/entity_id, and
page_id inline (the page_id FK and index stay separate — exploration_page
is created later in the file, so they can't run earlier).
- stored_result gains data_access_token inline; its backfill changeset was
deleted outright — created and backfilled in the same release, it could
only ever run against an empty table.
- exploration_query_result gains chart_stats / metric_description /
chart_description inline, which also retires the randomly-named changeset
id v63.gr324asd.
- idx_eqti_query_id deleted: the unique constraint on the same leading
column already provides that index.
The completed_at and analysis_started_at backfills are deleted outright:
the explorations feature merges to master in one piece, so at upgrade time
no instance can have pre-existing exploration threads to backfill. (They
were also dangerous — they stamped columns the cancel endpoint and the
completion handler gate on being NULL, permanently locking out any thread
in flight during the migration.)
Cosmetics: backfilled default docs are named "Scratchpad" to match runtime
(the backfill said "Findings"); stale remarks corrected (chart_stats is EDN
not JSON, the real query_type variant list, what collection_id actually
defaults to).
Existing dev databases will need a migration reset.
* Set exploration thread started_at in the INSERT
The thread was inserted without started_at and then immediately updated to
set it, on the theory that the background worker must not see the thread
before its dependent document/block/timeline rows exist. The whole create
shares one transaction, so intra-transaction ordering is invisible to the
worker (it reads on another connection; the commit is atomic) — the flag
can simply be set in the INSERT, saving a write.
* Guard exploration restart against in-flight work
The restart endpoint could fire while the planner was still working on the
thread, deleting its rows mid-plan and wedging the thread in a state no
worker would ever pick up again. reset-thread-for-rerun! is now a CAS: it
applies only to a terminal thread (completed_at set) with no query still
mid-execution, and the endpoint returns 409 ("cancel it before restarting")
otherwise. A successful reset leaves the thread exactly in the shape the
planning worker's claim predicate picks up.
The endpoint also published :event/exploration-update, which provably did
nothing: restart changes no Exploration column, and the revision system
skips unchanged objects. The publish is removed; the AI-summary doc reset
still records a Document revision, which is what /mine's last-touched
ordering composes.
* Route mid-stream LLM errors through the provider-friendly rethrow
LLM responses stream lazily; a connection timeout between SSE chunks
escaped as a raw SocketTimeoutException because the provider error handling
only wrapped request setup — the reduction happens later, in the agent
loop. Every provider adapter's SSE reducible (claude, openai, openrouter,
azure, bedrock) is now wrapped so consumption-time failures get the same
translation as request-time errors.
* Grant the read-only timelines scope under metabot-nlq
The explorations metabot profile instructs the model to use the timeline
tools, but those tools were granted only under
:permission/metabot-other-tools while the profile is gated on NLQ — so for
NLQ-only users they were silently filtered out of scope while the prompt
kept referring to them. agent:timelines:* (whose only member is the
read-only list/details scope) is now granted under both permissions.
* Don't retry deterministically-failing fields in the interestingness leftovers pass
The leftovers pass selects on dimension_interestingness IS NULL, so a field
whose scoring deterministically throws was re-attempted on every sync
forever. A per-process attempted-marker now skips fields that already
failed in this process; a restart re-enables retry, so transient failures
still converge. Process-local by design — no schema change, and no sentinel
score leaking into product surfaces that read the column as a number.
* Fail closed when a sandboxed field's user attributes are missing
sandbox-token-for-table returned nil when the field was sandboxed but the
user's sandbox-attribute lookup came up empty — and the downstream
compatibility gate reads nil as "unrestricted". That's fail-open. It now
returns a user-scoped "indeterminate" sentinel that matches nobody else's
token (two indeterminate users don't match each other either): fail-closed.
Also adds direct and end-to-end tests for the impersonation and routing
token producers, which previously had none.
* Require the (card, stored_result) pairing on the cached card-query path
POST /api/card/:id/query with a stored_result_id read-checked the card but
then served whatever snapshot id the client sent — nothing verified the
snapshot actually belonged to that card. The pairing must now exist in
stored_result_use (404 when it doesn't, so an unpaired probe can't confirm
a snapshot exists). Both materialization paths already insert the pairing
rows, and the table ships in the same unreleased release, so no legacy data
lacks them.
The viewer-lens check also denied a nil dataset_query only by accident (a
schema exception swallowed by the catch — and in prod, where mu/defn logs
instead of throwing, it would have failed open); it denies explicitly now.
Plus: consistent t2/select-one forms in the endpoint, a stored_result_use
docstring update, and the generated module config picking up the new
StoredResultUse consumption.
* Apply the personal-collection default when it resolves after first render
The "default new explorations to my personal collection" logic captured
personalCollectionId once in a useState initializer — if the selector
resolved after the first render, the default never applied. It now
reconciles when the id arrives, without overriding an explicit user choice.
* Move the explorations migrations to v64
The v63 release branch has been cut, so the explorations feature will ship
in v64 and its migrations belong in the 064 directory. Both files move
(20260428_explorations.yaml and 20260519_exploration_bookmark.yaml) and
every changeset ID is reprefixed v63. -> v64.
No changeset content changes. Since we're already editing unreleased
changesets in place on this branch, existing dev databases needed a
migration reset anyway.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* v_metabot_conversations v4: restore include-deleted semantics lost in merge
Master's #77542 (conversation branches) and this branch both created
metabot_conversations/v4 in parallel: master's dropped the
deleted_at IS NULL filters so soft-deleted (rewound/regenerated) messages
are counted; ours added CASE branches for the new explorations usage
sources and profile. The 'Merged from master' resolution kept only our
copy, silently losing the deleted-message semantics — which is why
deleted-messages-included-test and last-message-at-test fail: they assert
master's behavior against our view.
This drops the three filter lines from each dialect's v4 file so v4 is the
union of both changes. Both installing changesets (our v63 and master's
v64) are runOnChange: true and checksum the referenced file, so every
instance reinstalls the corrected view on upgrade — no new version or
migration needed.
* Drop the redundant v_metabot_conversations v4 installer
Master's #77542 (064/20260713_metabot_analytics_views_include_deleted.yaml)
already installs metabot_conversations v4 via a runOnChange changeset against
the same SQL files, which now carry both sides' changes: the explorations
CASE branches and the include-deleted semantics. A second installer here was
redundant — and would have collided with master's changeset IDs
(v64.2026-07-13T00:00:00/01) had this file been mechanically renumbered to
v64 like the other explorations migrations.
The ai_usage_log v4 changeset stays: nothing on master installs that view.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* Move the ai_usage_log v4 migration to 064
Same reasoning as the explorations tables: v63 is cut, this ships in v64.
The changeset ID takes the next free slot for its date
(v64.2026-07-13T00:00:02 — master's include-deleted file owns 00 and 01)
rather than a mechanical v63->v64 rewrite that would have collided.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
* Show a crossed-eye indicator with a tooltip on hidden pages in the exploration sidebar (#77907)
* Explorations no AI Summary (#77878)
* Extract AI Summary + document-in-exploration out of explorations
Removes the LLM-generated "AI Summary" feature and the ability to display a
document within an exploration. The feature-restoring (inverse) commit is kept
on branch `explorations-ai-summary`; cherry-pick it to bring the functionality
back.
Backend:
- Delete the explorations.ai-summary.* pipeline, composite, document-perms,
prompt templates, and their tests
- Drop exploration->Document creation, the /thread/:id/documents endpoints,
the :documents hydrate, ai_summary_document_id, and the AI-summary reset
- Revert the documents-module coupling: content-visibility gate, the
document.exploration_thread_id FK/column, the view-log skip, and the
GET /document list filter; delete documents/validate.clj
- Drop the collection-items show_exploration_documents option and the
exploration-doc exclusions in recents / search / serdes / remote-sync;
drop the search exploration-thread-id attribute
- Remove the ai_summary_document_id and document.exploration_thread_id
migration changesets (edited in place)
Frontend:
- Delete the ExplorationDocument component and its tests
- Remove the AI-summary sidebar node, completion toast, and the document
API / urls / types; keep the shared comments sidesheet and page routing
* Extract static card-embed in documents out of explorations
Removes the static (cached-snapshot) card-embed capability from documents, so it
can land on its own branch (`static-embed`) independently of the AI-Summary
feature. The feature-restoring (inverse) commit lives on `static-embed`.
Keeps the base `stored_result` model, the exploration query-result cache, and the
cached-result permission helpers (`assert-can-view-cached-result!`) that the
exploration `/query/:id` streaming endpoint uses. Also keeps the general
`report_card.document_id` "card in a document" feature (it predates this work).
Removed:
- The cached-result serve path (`cached-dataset`, `apply-sort`,
`deserialize-cached-result`, `allowed-chart-sorts`, `viewer-can-view-cached-result?`)
and the `stored_result_id`/`sort` branch of `POST /api/card/:id/query`
- `stored_result_use.card_id` (column + model arm; keeps `exploration_id`)
- The documents serdes static-node filter (reverted to master's live-only rewrite)
and the prose-mirror static docstrings
- Frontend: the tiptap `cardEmbed` static rendering (the `stored_result_id` + `sort`
+ `chart_href` trio) across the CardEmbed node/menu, document hooks, redux, and
`card.ts`; deleted the CardEmbedNode static unit spec
* Fix pre-existing cljfmt/oxfmt formatting to unblock CI
These 4 files (a stray blank line each) are unchanged from the base explorations
branch — the format checks were failing on inherited, not extraction, changes.
* more documents cleanup
* Update Loki Snapshots
* change references from entity (which was page or document) to just page
---------
Co-authored-by: Kevin Kelleher <kevin.kelleher@metabase.com>
Co-authored-by: Metabase Automation <github-automation@metabase.com>
* Restore fix
* Fix Metabot exploration tools streaming empty results to the client (#77990)
Since 6912a50e85 ([Metabot] Chat internal format improvements, #77301)
only a tool result's :output string makes it onto the tool-output-available
SSE event; a bare map streams as "". The exploration plan tools returned
bare maps, so the FE — which applies plan edits by parsing the streamed
tool result — silently dropped every edit while the LLM (which still saw
the full result) reported success.
Wrap each tool's payload in {:output (json/encode ...)} and add a
regression test that runs real tool results through parts->aisdk-sse-xf
and asserts the payload survives onto the wire.
* Explorations timeline fixes (#77958)
* fix timelines
* disable timelines for row charts
* Explorations sort order (#77980)
* sort empty pages to bottom
* sort subexplorations by the initial exploration's interestingness
* Add e2e tests
* Explorations use persistent queue (#77901)
* explorations: run planning/queries/scoring on the persistent queue
Switch explorations from the hand-rolled polling runner to the mq persistent
queue: three queues (plan, query, timeline-score) with per-queue on-error
handlers and worker-count-bounded concurrency.
The generic mq subsystem changes this builds on live in a separate branch
(mq-queue-improvements) so they can land on master independently. Until that
lands and flows back into `explorations`, this branch depends on those mq
features (:on-error, :max-concurrent-batches) and won't run standalone.
* Remove explorations settings-test
* Code cleanup
* Code cleanup
* Use in-memory queue backend for e2e tests
The persistent Quartz queue backend stores its durable-job state in the app
DB's QRTZ_* tables. e2e restores app-db snapshots between tests (H.restore),
which drops those tables out from under the running scheduler while the
per-process ensured-jobs cache still believes the jobs exist. Post-restore
publishes then target a non-existent job and never deliver, so exploration
planning produces no queries and the detail-page sidebar stays empty.
e2e is a single node, so the in-memory backend (no app-db state, unaffected by
snapshot restore) is the correct fit. Set via the :e2e alias so it applies to
both local (runFromSource) and CI (runFromJar, via getJvmOptsFromDepsEdn).
* fix type
* Remove Timeline interestingness from explorations (#78013)
* Remove Timeline interestingness from explorations
Removes the LLM-driven per-(query, timeline) interestingness scoring
feature and all of its wiring
* Improve names (#77862)
* improve names
* review changes
* fix dimension disambiguation not working
* Fix explore-further on expression columns (#77900)
1. The 500 — normalize-target-ref (in metrics/transforms.clj) silently dropped the expression name. The :field ref schema has a legacy-form decoder that relocates the id into MBQL-5 position; the :expression schema has none, so a
valid legacy ["expression" "Gear"] normalized to [:expression {…} nil]. That nil-named ref then failed Malli validation inside find-matching-column mid-request → 500.
2. The drill couldn't work anyway — a top-n-other chart breaks out on a synthetic CASE expression that exists only in the variant query, never on the metric card. So even a valid expression ref could never resolve against the card.
* Code cleanup
---------
Co-authored-by: Nathan Voxland <nathan@metabase.com>
* Update Loki Snapshots
* Improved LLM error handling
* Fixed queueng of explore-further
* Fix e2e tests
* Add copy link hotkey support
* Fix e2e tests
* Fix CI
* Fix after merge
* Make highlighted state in line charts more obvious when hovering over comment badge (#77998)
* show tooltip when highlighting comment for cartesian charts
* set blurScope to blur across stacked chart grids, only show tooltip for single line series
* review feedback
* add types file that was supposed to be included with 77998
* self review
* Auto-switch the add metrics modal to the All tab when the library has nothing to show (#78070)
* Fix back button (#77995)
* Fix back button
Co-Authored-By: Andrei Chernykh <a.d.chernykh@gmail.com>
* Remove the stray /question/explore route, fix the entry spec's removed setMode prop, and clean up dead code
* Keep the research draft and conversation alive across the entry/plan step and browser back/forward via a shared provider on the research route
* Fold the plan-step URL assertion into the existing manual-flow e2e test instead of a separate one
---------
Co-authored-by: Ethan McCue <ethan@mccue.dev>
* Explorations: remove dormant extended-thinking support; unify LLM timeouts (#78173)
* Remove unused Anthropic extended-thinking support
Extended thinking was fully dormant: the Claude adapter supported it
(streaming translation of thinking blocks, config normalization, request-body
support) and call-llm-structured-with-trace forwarded a :thinking opt, but no
production caller ever passed :thinking, and the multi-turn agent path
(call-llm) never accepted it at all. Remove the adapter support and the
structured-path plumbing until a real consumer exists.
This also resolves several open review questions about the thinking code
(signature round-trip on multi-turn continuations, structured-output
guarantees under thinking, per-model capability guards) by removing the code
they applied to.
* Unify LLM HTTP timeout knobs across metabot.self and llm.anthropic
metabase.metabot.self.core/request hardcoded its connection (10s) and
socket (120s) timeouts, diverging from the metabase.llm.anthropic HTTP
path, which already reads the operator-tunable llm-connection-timeout-ms /
llm-request-timeout-ms settings. Operators had two uncoordinated knobs.
Delete the two private constants and read the settings at call time in
`request` (keeping the per-request `(merge req)` override). Raise the
setting defaults to the former constants' values (5000 -> 10000,
60000 -> 120000) so no path loses its safety margin, and move the
constants' streaming/inter-byte rationale into the defsetting docstrings.
Both LLM HTTP paths now share a single operator knob pair.
Resolves PR #1 review comment 2.
* UXW-4758 Ability to view all timeline events
* Show proper empty state message in sidebar for initial load
* Bump prefer-with-dynamic-fn-redefs ratchet to 28
The explorations MQ queue tests need three real with-redefs (not
mt/with-dynamic-fn-redefs): the handlers run on an MQ worker thread that
does not carry the calling thread's dynamic bindings, as documented
inline at test/metabase/explorations/queues_test.clj:102,153,265. Seed
the budget to match so metabase.core.kondo-ratchet-test passes.
* fix regression of #77667 caused by #76901
* Restore explorations CASE branches in v_metabot_conversations (v6)
master's #77549 rewrote v_metabot_conversations to v5 (summary->title
rename) without the explorations source/profile CASE branches that v4
carried, so the exploration/contextual_interestingness sources and the
explorations profile surfaced as raw machine names and failed
metabase-enterprise.metabot.usage-test/known-sources-and-profiles-have-view-case-branches-test
across every dialect.
v5's installer is not runOnChange and is already released on master, so
edit it in place would break the checksum. Instead add v6 (v5 + the three
branches) and a runOnChange installer that reinstalls the view after the
v5 rename, matching the v4 installer pattern.
* fix errors caused by merging master
* UXW-4758 Close timeline events sidebar when comments are opened
* Explorations queries api performance (#78175)
* revert error message display changes
* Avoid a full fingerprint version increment
* Fix unit tests error logs
* Removed the composed LRU + TTL cache that is not supported by the core.cache factories. The data should be naturally bounded enough that TTL should be good enough.
* Added scopes for explorations in metabot
* Explorations PUT safety (#78188)
Don't allow random assignment on PUT /api/exploration/:id
* Pick the right column as the metric for interestingness scoring (#78269)
Pick the right column as the metric
* Explorations unstranded threads (#78250)
Fixing ways threads can be stranded
* Hide SQL error messages from API response
* Explorations pass research plan (#78236)
Correctly pass research plan
* Revert in-place edits to historical instance-analytics view SQL
The explorations CASE branches were added directly to the v1/v4 view SQL
files, but those files back runOnChange changesets: editing them changes
the checksum, so on an already-migrated DB the old installers re-fire.
metabot_conversations v1 still selects c.summary, which the v64 rename
(summary -> title) has dropped, so booting the branch on a master-migrated
DB fails with 'column c.summary does not exist' (all six cross-branch
migration CI jobs).
The branches live on in the branch-new final versions (ai_usage_log v4,
metabot_conversations v6), so nothing is lost on fresh installs.
* Raise prefer-with-dynamic-fn-redefs ratchet to 29
The branch added four with-redefs ignores in explorations/queues_test.clj
(all justified: the redefined handlers run on MQ worker threads, which do
not see thread-local dynamic bindings, so mt/with-dynamic-fn-redefs cannot
work there) but the budget was only bumped to 28. Verified all 29 ignores
in the tree still sit on live with-redefs calls, so none can be removed
instead.
* Improving exploration error handling
* Fix with-current-user binding in exploration runner (#78163)
* Better error vs. empty handling in query cache
Conflicts:
src/metabase/explorations/query_plan/variants.clj
* self review
* Added FKs
* fix e2e
* Added visible-metric-cards-where-clause to queries.core
* Cleaned up removed-in-master agent:table scope
* Reuse Event card design from QueryBuilder
* Handle prompt injection
* Fix hotkeys hijack default actions
* Reuse the query instead of continually re-running it
* Fix extra rerenders on queries polling
* Review fixes
* Review fixes
* Fix chart skeleton for all screens
* review comments
* Fix Andrey's review comments
* Fix Andrey's review comments
* Fix Nemanja's review comments for e2e
* Fix Nemanja's review comments for e2e
* more review comments
* Ensure user has read permissions on cards when doing an explore-further
* Handle numeric PKs/FKs better in explorations (UXW-4757) (#77991)
Numeric PK/FK dimensions produced unusable one-bar-per-id charts and were
auto-selected as interesting dimensions. Two root causes:
1. The query planner treated any numeric dim as auto-binnable, but...
they're not, and we emitted unbounded bare breakouts, and ruled out
the top-n-other variant.
2. Dimension interestingness hard-zeros PKs but not FKs, and the
cardinality scorer's auto-binnable branch gave numeric FKs 0.9 for a
binned chart the QP can never produce, so they got auto-selected.
Fixes:
- default-bucket-for-dim returns nil for :Relation/* dims, so numeric keys
band by raw distinct count: <=100 -> default, >20/unknown -> top-n-other.
- top-n-other stringifies its CASE THEN values so numeric dims compile to
a single-typed CASE (mixed numeric/string CASE breaks strict warehouses).
- type-penalty hard-zeros numeric FKs (semantic FK + numeric effective/base
type). Non-numeric FKs (e.g. a country-code FK) stay scoreable.
- The cardinality scorer excludes :Relation/* from the auto-binnable
branch, matching lib's own binning rule.
* UXW-4475: self-heal missing metric dimensions in explorations read path (#78061)
Metrics whose dimensions were never successfully synced (dimensions NULL) -
e.g. metrics created on a model before it had result_metadata, or SQL-model
metrics predating the source-card sync fix - showed zero dimensions in the
explorations picker and Metabot research tools forever, because those paths
read persisted dimensions without ever syncing.
hydrated-metrics now runs sync-dimensions! for loaded metrics that have a
dataset_query but empty dimensions, then reloads the healed rows. Metrics
that compute nothing persist nothing and retry cheaply on a later request
(app-DB reads only, no warehouse queries, no write when nothing changed).
* Fix stored-snapshot permission gate ignoring card-sourced queries (#78455)
The data-access-lens capture derived its table set from
query->source-table-ids on the raw stored dataset_query, which only
collects numeric :source-table ids. For queries built on cards
(:source-table "card__N"), neither the creator's nor the viewer's token
picked up any tables, so both sandbox dimensions were empty and
data-access-compatible? returned true — letting a sandboxed viewer
stream an unsandboxed creator's snapshot.
Add query-perms/query->resolved-source-table-ids, which preprocesses the
query (as admin, via the existing preprocess-query helper) so source
cards — nested chains and card-sourced joins included — resolve to their
underlying table ids, and use it at both lens-capture sites (the
exploration runner's token write and the cached-result read gate). The
preprocess binds qp.i/*skip-middleware-because-app-db-access*, the same
flag the offline semantic checker uses, so the routing/impersonation
preprocess middlewares don't resolve the *viewer's* destination/role
(and throw on lapsed premium features) while we're only after the
query's table footprint. The helper throws when the card chain can't be
resolved (deleted card); non-admin viewers are then denied while
admins — never sandboxed, impersonated, or routed off the router db —
fall back to the same admin-only access as a nil token.
Also make stored_result.dataset_query NOT NULL via a follow-up
addNotNullConstraint changeset: the gate derives the viewer's lens from
it, so a query-less snapshot has no defensible verdict. A gate caller
passing a map without the key now gets a loud exception (500) instead of
a quiet fallback, so trimmed-row bugs surface in development.
Also fix explore-further-nested-thread-name-omits-metric-prefix-test to
reference a real exploration_page instead of a hardcoded id 999 that
violates the source_page_id FK on fresh databases.
* Fix build
* Fixed comment
* Better cached result testing
Conflicts:
src/metabase/queries/cached_result.clj
* Hash data access token info
* Added test from before
* Fix unit tests
* Fix Andrey's review comments
* Sloan review comments
* Explorations - fix metrics case (#78411)
* Add delay for Sidebar shimmer text, split ExplorationTreeNode
* Fixes for routing after merge
* Latest reviews fixes
* UXW-4908 new Comments UI
* add more analytics
* Code review fixes
* Fix scroll to view for focused comment
* Correct permissions "lenses" (#78697)
* Correct permissions "lenses"
Some early code had a huge, obvious bug, asserted in tests, for database routing.
For explorations, because queries are pre-computed, we need to calculate "lenses" representing the state
of database routing, sandboxing, and impersonation. These lenses are calculated when the exploration is
created, and then calculated when someone attempts to view the exploration. The idea is that if the lens
reveals that the viewer would have been able to view the underlying queries, then they should also be able
to view your exploration.
Until now, a viewer who was "unrestricted" on one of those dimensions was allowed to see a restricted
snapshot. I.e., a "non-sandboxed" user could see a sandboxed user's exploration, a "non-impersonated" user
could see an impersonated user's snapshot, and a "routed-to-the-router" user could see a "routed-to-a-destination"
user's snapshot.
The issue here... "unrestricted" does not necessarily mean "unrestricted." It's entirely possible that:
- User A is sandboxed with a sandbox that replaces Table T with card C, which actually queries Table FT.
User B is permitted to see Table T (unrestricted) but is blocked on Table FT.
- User A is impersonated - to a *more* permissive role. User B is unimpersonated, but should not be able to see
the results of a query made by User A.
- User A has database routing, configured to point to `MySecretTenantData`. User B is routed to `__METABASE_ROUTER__`,
which holds a template schema and maybe some data to play around with. User B should not be able to see the results of
A's exploration.
This simplifies the code here:
- superusers can see every exploration, regardless of lens
- everyone else sees an exploration's results *ONLY* when their data-access lens is *identical* to the creator's. Same sandbox
(and relevant user attributes), same impersonation role, same routing destination. Unrestricted only matches unrestricted.
Also hardened the write side: capturing the creator's lens fingerprint used to be best-effort, silently saving a snapshot with
no fingerprint when it failed. Now the exploration query fails visibly instead, and the read side treats a
fingerprint-less snapshot as a bug (denied to non-admins, logged as an error) rather than something to adjudicate.
* Apply suggestions from code review
clean up test descriptions and other docstrings
Co-authored-by: Timothy S. Dean <timothy@timothydean.dev>
* Respond to CR
* fix impersonation test
---------
Co-authored-by: Timothy S. Dean <timothy@timothydean.dev>
* Fixed compile issue
* Address query-plan review comments on #73244
- context.clj: correct the column-fingerprint-for-target docstring (missing
fingerprints make default/time-facet eligibility fail closed and route
dims to top-n-other, not the reverse), fix the applicability return-shape
doc, and extend the ns docstring to cover build-row-context.
- context.clj + variants.clj: remove the dead temporal_dimension_id override
plumbing (nothing ever sets that param; the mechanical planner never emits
per-value-time-series items).
- variants.clj: drop the unreferenced known-variants var.
- query_plan.clj + planner.clj: validate every planner result against
result-schema in the orchestrator, treating a mismatch as a planner
failure, and document that the schemas are enforced.
- planner.clj: fix the plan! docstring's claim that the ctx shape is
documented on generate-query-plan!.
* Remove unused planner result schemas instead of enforcing them
Drops plan-item-schema and result-schema from the planner contract (review
follow-up on #73244): they were never applied to any value, and rather than
wiring validation into the orchestrator we document the plan!/result shapes
in prose on the protocol. Also rewords the context ns docstring's
plan/validate/materialize mention accordingly.
* Trim redundant clause from query-plan.context ns docstring
* Fix FE typecheck and unit tests
* Fix some tests
* Fix dimensions display in NewExplorationData
* Select all dimensions for a metric by default
* Fix metric-dimensions dependency test vs metric dimension auto-sync
The test (from master) created a metric card with hand-written
dimension_mappings and nil dimensions. On this branch a metric with nil
dimensions counts as uncurated, so the after-insert dimension auto-sync
seeds dimensions from the query and overwrites the provided mappings —
the mapped categories table never became a dependency.
Set the mappings via a plain update after insert instead: the
after-update hook only re-syncs when dataset_query changes, so they
stick, and the test still exercises what it means to — the dependency
backfill turning mapped-dimension tables into table dependencies.
* Fix metric dashcard perms test vs metric dimension auto-sync
The tests (from master) expect a metric dashcard query to return the
bare aggregation. On this branch the after-insert auto-sync seeds
metric dimensions, and qp.dashboard applies the default time dimension
as a breakout, turning the scalar count into a monthly series.
Suppress the auto-sync when creating the metric so the test stays
focused on what it exercises: permissions on the metric's source
model.
* Fix FE typecheck and unit tests
* Skip tests that will be addressed in the next PR
* Fix loki tests
* Add logs to identify e2e test failure reason
* Fix CI e2e failure
* Review improvements
* Redact raw driver errors from the exploration payload
`GET /api/exploration/query/:id` deliberately allowlists its 409 payload and
substitutes a generic message, because a driver error names tables, columns,
and row values from a query the reader may not have permission to run.
`GET /api/exploration/:id` hydrates the same `ExplorationQuery` rows onto its
response and returned `:error_message` verbatim, so the raw error reached
anyone who could read the exploration. Declaring the key in the response
schema does not help: a `defendpoint` response schema validates and encodes
but does not strip.
Give the substitution one home in `redact-query-error` and route both paths
through it — `hydrate-exploration` covers all five `::HydratedExploration`
endpoints at a single point.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* Track data_access_token with thread and query instead of relying on the (not always there) cached result
* Key the derived-data verdict on both query projections
* Remove eager dimension-sync for metrics
* Fix unit tests after merge
* Fix typecheck
* Code review fixes
* Revert "Select all dimensions for a metric by default"
This reverts commit d5e8883c95.
* Code review fixes
* Fixing tests
* Fix e2e test
* fix issues after merge
* fix error when saving exploration in Our Analytics
---------
Co-authored-by: Nathan Voxland <nathan@metabase.com>
Co-authored-by: Denis Berezin <denis.berezin@metabase.com>
Co-authored-by: John Swanson <john.swanson@metabase.com>
Co-authored-by: Andrei Chernykh <andrei.chernykh@metabase.com>
Co-authored-by: Kyle Doherty <5248953+kdoh@users.noreply.github.com>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Co-authored-by: Metabase Automation <github-automation@metabase.com>
Co-authored-by: Ethan McCue <ethan@mccue.dev>
Co-authored-by: Timothy S. Dean <timothy@timothydean.dev>
* refactor: extract reusable typed schema module
* remove obsolete database-name query parameter
* refactor: split typed schemas rest endpoint module
* refactor: move public facade to core namespace
* refactor: use bare namespace for typed schema endpoint
* docs: describe typed schema options
* refactor: narrow typed schema query parameter
* docs: explain typed schema database names
* docs: clarify typed schema model scope
* re-export render-typescript from core namespace
* refactor: tighten typed schema options
* docs: add docstrings to typed schema query parameters
test: clarify typed schema parser coverage
* refactor: isolate typed schema model scope
* docs: clarify typed schema model scope
* test: simplify typed schema model scope coverage
* test: simplify rest api tests
* test: separate typed schema API and core coverage
* test: cover typed schema database selection
* test: assert generated typed schema tables
* test: normalize typed schema timestamps
* refactor: introduce TypeScript AST between schema and rendering
schema->ast is a pure policy stage producing a tagged-vector AST
(documented and malli-schema'd in typed-schemas.javascript); render-js
is an option-free printer over it. Replaces print-time policy callbacks,
path-based node-kind guessing, and the reference magic-map. Output
verified byte-identical against the previous renderer.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01G69dZFxucVXhR1s11ZxmuB
* refactor: split typed schema fetching from pure assembly
fetch-items is now the pipeline's only impure stage; create-schema
assembles Items into the semantic schema value and takes an info map so
generated-at/instance-url can be pinned for deterministic output.
build-semantic-schema composes the two, unchanged for callers. Folds
the base-schema namespace into core, drops the duplicate
database-id-filter-clause, and rewrites core tests against literal
items. Output verified byte-identical.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01G69dZFxucVXhR1s11ZxmuB
* refactor: reify typed schema data access as a protocol
SchemaSource names every read fetch-items performs; app-db-source is
the production implementation. Tests reify the protocol with literal
values instead of redefining selection functions across namespaces.
Output verified byte-identical.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01G69dZFxucVXhR1s11ZxmuB
* test: add real-db coverage above the in-memory pyramid
source-test exercises app-db-source against with-temp rows — the one
seam where typed schemas meet the database. core-test gains a single
end-to-end test running every entity kind (question, metric, model with
action, table with fields) from with-temp rows through fetch, assembly,
AST and rendering.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01G69dZFxucVXhR1s11ZxmuB
* test: run the pipeline e2e against the test-data dataset
mt/dataset provides a real synced database, so the e2e cards sit on
real tables and fields — the question and metric need no hand-written
result_metadata, and metric columns are computed from real metadata.
Replaces the hand-rolled temp database/table/field rows.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01G69dZFxucVXhR1s11ZxmuB
* test: assert the e2e schema with =? pattern matching
One structural pattern replaces the key-plucking assertions; a single
exactness check keeps the database-scoping guarantee that partial map
matching would silently lose.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01G69dZFxucVXhR1s11ZxmuB
* test: sweep typed schema tests into =? pattern style
select-keys plucking becomes =? patterns (sets stay exact under =?, so
scope assertions keep their meaning); try/catch exception capture
becomes (is (thrown? ...)) + =? on ex-data; arg-asserting mocks become
stubs, with the argument pass-through now covered by the dataset e2e;
vestigial #' derefs on now-public fns are dropped; the five
prefer-with-dynamic-fn-redefs lint warnings are fixed.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01G69dZFxucVXhR1s11ZxmuB
* docs: state the module's separation rules in namespace docstrings
Future changes should keep the boundaries: reads go through
SchemaSource, stages downstream of Items stay pure with environment
values entering via info, render policy stays schema->AST, TypeScript
syntax stays in the printer, and REST strings decode once at the
endpoint boundary. core-test documents where new tests belong in the
pyramid.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01G69dZFxucVXhR1s11ZxmuB
* refactor: drop convenience arities orphaned by the source protocol
app-db-source always passes full arguments, so the single-argument
arities of select-schema-cards, select-tables, question-schemas,
metric-schemas and model-schemas had no production callers left. Each
fetch function now has one full-arity signature.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01G69dZFxucVXhR1s11ZxmuB
* refactor: define scopes and trim the library scope to its live keys
scope.clj now defines the vocabulary: references are portable ways to
name content (ids, names, entity ids); a scope is what they resolve to
on this instance — permission-filtered id sets with descendant
expansion, nil-vs-empty semantics, and the 404-vs-empty asymmetry
stated. LibraryScope gets a closed malli schema; its :collection-ids
and :library-collections keys were computed and asserted but read by
nothing, and are gone.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01G69dZFxucVXhR1s11ZxmuB
* refactor: move scope compilation to the fetch layer as scope-filter-clause
scope.clj now only resolves references into scopes; the fetch layer's
schema.common owns the one function that compiles a scope into a where
clause. The docstring states the nil-vs-empty enforcement and warns
against (seq ...) guards at call sites, which would turn 'resolved to
nothing' into 'unscoped' and select everything.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01G69dZFxucVXhR1s11ZxmuB
* refactor: batch collection reference resolution in typed schema scopes
Adds collection/descendants-flat-for (descendants of many collections
in one query, selecting :type) and rewrites scope resolution on top of
it: references partition into id and entity-id batches, resolve in at
most two selects, and 404s name every unresolved reference instead of
a bare 'Not found.'. Query count is now constant in the number of refs.
library-scope enforces its LibraryScope return schema via mu/defn.
Addressing, eligibility, and authorization are distinct layers in
resolve-collection-refs!, and their failures are deliberately
indistinguishable.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01G69dZFxucVXhR1s11ZxmuB
* refactor: shrink the typed-schemas public surface to its contract
core now exports exactly three vars — SemanticSchemaOptions,
build-semantic-schema, render-typescript — mirroring the module's
data/print split; the pipeline implementation moves verbatim to the
internal pipeline namespace. New exports are compatibility promises,
and nothing needed the other five. The endpoint returns a ring map
directly instead of calling respond (async comes free), and
database-ids-for-ref uses select-one on the id path, documenting that
name references can genuinely match several databases (union semantics
pending a product decision).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01G69dZFxucVXhR1s11ZxmuB
* refactor: define the public API in core and name the build side
core now defines build-semantic-schema and render-typescript as visible
compositions of the internal stages instead of potemkin re-exports —
the public API is deliberate and decoupled from internal names. The
internal namespace becomes 'build': it interprets options into
SchemaSource reads and assembles the items into the schema value,
pairing with render/javascript as the module's two sides. The
compositions move out of build and render accordingly.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01G69dZFxucVXhR1s11ZxmuB
* test: build e2e card queries with Lib instead of deprecated mbql-query
The e2e's cards now carry lib-built MBQL 5 queries
((lib/query mp (lib.metadata/table ...)) + lib/aggregate) rather than
hand-rolled legacy MBQL via the deprecated mt/mbql-query macro — which
also proves the pipeline handles MBQL 5 dataset_query cards end to end.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01G69dZFxucVXhR1s11ZxmuB
---------
Co-authored-by: dan sutton <dan@dpsutton.com>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Drop dead QueryExecution delete from embed download test
with-query-execution! redefs save-execution-metadata!* and captures the
query execution in a promise, so nothing is written to or read from the
QUERY_EXECUTION table. The delete! only served to reach the app DB before
anything had initialized it, failing with "this database is empty".
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
* Persist the download format on export, never on format switching
The download popover remembers your last-used export format, but it was
persisting that preference on every click of the format segmented control,
spraying identical PUT requests at
/api/user-key-value/.../download_format_preference while the user just
looked around.
Switching formats now sends nothing. The preference is written as part of
handling an actual download, a fire-and-forget side task on top of the
export work, and only when it differs from the value the server already
has (the widget fetches it anyway to preselect the format, and the
mutation keeps the cached copy in sync, reverting it if the write fails).
Downloading with an unchanged format sends nothing at all. User-visible
behavior is unchanged: the popover still reopens on the format you last
exported with. The downloads e2e spec asserted the old
selection-persists behavior and is updated to the new semantics.
Considered and declined in review: guarding the comparison against a
still-loading preference (a download racing the initial GET can send one
redundant idempotent write; ordering a fire-and-forget path is not worth
it), and serializing overlapping preference writes (two different-format
exports within one HTTP round-trip could persist out of order; a mutation
queue for a preference value is overkill). Both races are rare, bounded,
and self-correcting on the next export.
* Trim the comments and the old-behavior test guards
Review feedback: the comment on persistExportFormat ran longer than the
function and said what the code already says, and both the e2e comment
and the no-request-on-switch assertions pinned behavior we removed
rather than behavior we built.
Also makes the optional chaining on formatPreference consistent within
persistExportFormat.
Refresh the bundled browserslist data to clear the "caniuse-lite is
outdated" warning on dev builds; the data was several months stale.
Lockfile only: bumps caniuse-lite to 1.0.30001806 and
baseline-browser-mapping to 2.11.9, and drops stale nested duplicates so
they resolve to the hoisted versions. Hand-edited because
update-browserslist-db misbehaves with bun in this repo.
* Support manifest-based runtime plugins
Summary
- Extract shared dependency pruning for runtime plugin uberjars.
- Initialize non-driver manifest plugins eagerly behind plugin API version 1.
- Preserve lazy loading for legacy driver manifests, while treating an empty driver list as a non-driver plugin.
Scope
This is capability-neutral infrastructure. It does not define an embedding API or ship an embedder.
Testing
- Plugin uberjar pruning tests.
- Plugin initialization, version-gate, empty-driver, and lazy-driver tests.
* Make runtime plugins lazy by default
* Document lazy plugin loader constraints
* Name plugin dependency checks after registration, not init
A satisfied `plugin:` dependency means the other manifest is registered, not that its
code is loaded. Rename the argument threaded through the dependency checks to say so, and
record where load ordering actually comes from for the two driver manifests that declare
one.
* Make a failed lazy-driver load retryable
Load the plugin before removing the placeholder `initialize!` method, so a load
that throws leaves the driver retryable instead of stranded on the no-op default.
Guard the removal so a driver that installs its own `initialize!` during load
keeps it.
Also clarify the `register-plugin!` contract (manifest plugins defer the classpath
add and code load; bare dependency JARs go on the classpath immediately), rename
`prune-provided-libs` to `prune-provided-libs!`, and mark its pure test parallel.
* Drop ^:parallel from prune-provided-libs-test
prune-provided-libs! logs via u/announce, so kondo's :metabase/validate-deftest
rejects the destructive fn inside a parallel test. Keep the ! name, drop the tag.
* Generalize bundled plugin loading to non-drivers; review nits
Rename bundled-driver-manifest-paths / load-bundled-driver-plugin-manifests! to
drop the driver-only framing -- the classpath glob already discovers any bundled
manifest -- and document that bundled plugins carry no add-to-classpath!, so their
code never comes from the writable plugins directory. Add a test asserting a
bundled plugin activates by running init steps only.
Also delay the plugin-uberjar core basis so requiring the namespace no longer pays
for a full dependency-tree resolution, and drop the unused init-plugin-with-info!
compatibility alias.
* Use with-dynamic-fn-redefs in the plugin init tests
Every binding here redefs a plain defn, so prefer the thread-safe dynamic redefs
(clears :metabase/prefer-with-dynamic-fn-redefs) and align each binding block's
values consistently.
* Register bundled manifests before plugins-directory manifests
Registration is first-wins, so a JAR dropped in the writable plugins directory
could shadow a bundled plugin's name and run its own code under that identity.
Load bare dependency JARs first (for class: deps), then bundled manifests, then
user manifests -- so bundled, root-owned code always claims the name.
* Reserve bundled plugin names so plugins-dir JARs cannot shadow them
A bundled plugin with an unmet dependency (Oracle/Vertica without their JDBC JAR)
never registers, leaving its name unclaimed -- a plugins-directory JAR could then
register under it and run as the bundled plugin. Reserve every bundled name up
front and refuse a colliding plugins-directory manifest.
* Reserve bundled names against the dependency batch too
Classification and registration read the writable JAR separately, so a JAR that
looks like a bare dependency but resolves to a bundled-name manifest could
register under it. Compute the reserved set once and pass it to both batches.
* Fail fast on concurrent plugin loading
* Align plugin framework forms
* Guard plugin registration and loading with one lock
Registration synchronized on the registered-plugins atom while loading claimed a second atom by compare-and-set, so
two different locks guarded two views of the same state.
Both paths now take one ReentrantLock: registration waits for it, and loading still fails fast when another thread
holds it.
Because that lock is reentrant, a plugin whose init step loads another plugin no longer trips the concurrency check
on its own thread -- nesting is not concurrency.
The registered and loaded name sets collapse into a single registry atom keyed by plugin name, which leaves
active-plugin-load as a diagnostic that names the loader in the rejection message.
* Reject plugin load cycles
A reentrant lock let a plugin load itself, directly or through a cycle like A -> B -> A.
`:loaded?` is only set once the init steps finish, so each pass through the cycle ran them again until the stack blew.
The lock now carries a stack of the loads in progress: nesting distinct plugins is still fine, re-entering one that is
already loading throws and names the cycle.
A rejected concurrent load no longer reports that plugin `nil` holds the lock when the holder is registering rather
than loading.
* Tidy the plugin cycle tests
Hoist the shared registration out of the first `testing` block, drop the closure that wrapped it, and share one uuid
between the two plugin names.
* Drop the temporary atom in update-unsatisfied-deps!
Everything here runs under plugin-lock, so a read and a reset does the job.
* Allow dropping items into sub-collections shown in the collection items table
* Trim comments in collection drop target
* Accept documents and metrics in collection drag-and-drop targets
* Improve collection drag-and-drop feedback
* code clean up
* Remove redundant React keys from drag source table row
* Retire redux navigation in favour of useNavigate
`push`/`replace`/`goBack` and the `CALL_HISTORY_METHOD` transport are gone.
Components navigate with `useNavigate`. Code that cannot call a hook (thunks,
the session middleware, a reducer, the permission and visualization config
objects) uses a module-level `navigate`, which is the navigator holder `AppShell`
already filled in, exported directly.
`metabase/router` exports its own `useNavigate`, which falls back to that
`navigate` when no router is mounted rather than throwing, because redux
navigation had no opinion on whether one was. The SDK mounts none, which is what
`useMaybeLocation` already exists for.
Four things this could not do mechanically:
* Relative targets resolved from the root under redux navigation, because
`AppShell` sits on a pathless layout route. `useNavigate` resolves against the
component's own match, so `SearchBar`, `SearchApp` and the command palette
needed absolute paths.
* A `LocationDescriptorObject` is assignable to v7's `To`, so passing one to
`navigate` type-checks and drops `state` silently. Six sites carried state,
including the QB card-state contract.
* `mapDispatchToProps: { onChangeLocation: push }` cannot hold `navigate`, since
`connect` dispatches what a bound creator returns. Five containers became
function components, the dashboard context supplies it in the provider, and
three `reference/` pages had it wired but never read it.
* A permission `actionCreator` returned an action to dispatch. It is now
`onSelect`, returning void, and `updateDataPermission` treats the presence of a
post action as handled rather than inferring it from a truthy return.
LOCATION_CHANGE and the location mirror stay: they are the router-to-redux
direction, feeding `isNavbarOpen`, `errorPage` and trace-id rotation.
* Hold navigations only when a router is on its way
A host that builds no router never registers a `navigate` for the holder to
flush against, so anything held is held for the life of the page. The SDK is
such a host and runs for a long time, so every `navigate` call from it retained
a closure and the `state` that closure captured.
`createAppRouter` and `createMemoryAppRouter` now say that a registration is
coming, because both put an `AppShell` above the tree. Without that, a
navigation with nowhere to go is dropped rather than queued, so a router-less
host retains nothing.
The queue stays a queue. Collapsing it to the latest navigation would change the
history stack: a push followed by a replace would apply the replace to the entry
the user arrived on, and lose the intermediate one.
* GHY-4251: Bound dependency backfill memory and honour its off-switch
The BackfillDependencies job OOM-killed instances with large warehouses. Native-query dependency
calculation resolved parsed table names by fetching the Database's entire table catalog and scanning
it in Clojure, so cost scaled with warehouse size rather than query size. Reading an entity also
attaches a MetadataProvider to its query, and the job bound no provider cache, so each entity in a
batch got a private provider holding its own copy of that catalog -- all of it retained for the
lifetime of the batch. At batch size 500 against a ~20k-table warehouse that is ~10M retained table
rows, which no amount of heap fixes.
Three changes:
- Look up only the tables a query names. New :name-ci metadata spec matches names
case-insensitively, since normalize-name lower-cases while sync stores whatever the warehouse
reported (Snowflake reports upper-case). Unlike :id/:name it does not widen the result set to
inactive or hidden objects, so it narrows an otherwise-unfiltered fetch without changing which
tables are eligible. The SQL narrows and find-table-or-transform still decides, so matching
semantics are unchanged. Transforms keep the existing scan -- their target name lives in a JSON
column with nothing to query -- which is fine at their cardinality.
- Scope a MetadataProvider cache to the backfill batch, spanning the select as well as the
processing loop, so entities share one provider per database instead of one each. This is the
invariant deps.findings/upsert-analysis! already enforces by throwing.
- Honour a non-positive batch size in schedule-run! for both dependency jobs. It is the supported
off-switch, but only task/init! checked it; the event-driven trigger did not, so entity changes
kept waking the jobs every second to do nothing. schedule-run! is the one function all three
scheduling paths share. The setting is env-var-only, so it cannot change while the process runs
and the job stays unscheduled until restart.
* GHY-4251: Record the failure when a fatal error kills a backfill entity
backfill-entity-batch! caught Exception, so an Error -- an OutOfMemoryError in the reported case --
escaped without recording anything. The next run then selected the identical batch and died the same
way, indefinitely, with no progress and nothing in dependency_status to show for it.
Catch Throwable instead and record the failure before letting an Error propagate, so a poisonous
entity backs off and eventually goes terminal and the batch can drain. Errors are still rethrown
rather than swallowed: the rest of the batch is abandoned and the job fails, because the JVM is in a
state we cannot reason about. Exceptions keep their existing per-entity behaviour.
* GHY-4251: Only let pending retries keep the backfill job scheduled when licensed
The job reschedules itself when a full batch was selected or when entities are in retry backoff. The
retry check ran outside the :dependencies feature gate that guards the work itself, so an instance
without the feature kept rescheduling hourly forever: a retry marker is only cleared by processing
the entity -- on success via upsert-status!, or by going terminal past max-retries -- and both are
gated on the feature, so the job could never resolve the condition keeping it awake. It took the
"No more entities to backfill for, stopping" branch never.
Gate the check on the feature. Requires the feature to be enabled, and gives up an accidental warm
standby: with the feature off and retries pending the job previously stayed scheduled and would pick
up work the moment the feature flipped. Recovery does not depend on that -- setting the token
publishes :event/set-premium-embedding-token, and every content-change handler triggers the job -- so
the remaining exposure is an upstream-only change to a token's feature set, which starts the backfill
on the next content change rather than immediately.
* GHY-4251: Correct the off-switch rationale in schedule-run! docstrings
Both docstrings claimed the batch size setting is env-var-only. It is not: get-raw-value consults the
application database after the env var, and :setter :none only blocks writes through the settings API,
so a stored value would be honoured if one existed. The conclusion is unchanged -- there is no
supported way to change it at runtime, so a cold stop until restart is right -- but the stated reason
was wrong, and it is the reason a future reader would rely on.
* GHY-4251: Stop the entity check job rescheduling when it cannot do anything
DependencyEntityCheck is a periodic checker with no terminal state, so it always queues another run.
That is correct while licensed, but check-entities! is gated on the :dependencies feature while the
reschedule was not, so an instance without the feature woke every 30-90 minutes forever to do nothing.
Same defect as the backfill job's retry check, reached by a different route.
Extract the reschedule out of the Quartz job body -- it only needed the scheduler, not the whole
execution context, which also makes it testable -- and gate it on the feature. If the feature is
enabled later the job restarts from the content-change event handlers, or from task/init! on the next
restart.
* Update docstrings
* GHY-4251: Record a failure for entities that have no dependency_status row yet
record-failure! only updated an existing row, so it silently did nothing for an entity that had never
been processed. Those are exactly the entities instances-for-dependency-calculation picks up first --
it matches a null status row -- so on a first backfill every entity is in that state. A failure
recorded nothing, the entity was reselected unchanged on the next run, and the crash loop the previous
commit set out to break stayed intact for the population most likely to cause it.
Upsert instead, via app-db/update-or-insert! like the other writers in this namespace. An inserted row
takes the column defaults for stale (false) and dependency_analysis_version (0); 0 is below
current-dependency-analysis-version, which is what keeps the entity eligible for the retry once
next_retry_at elapses.
This also fixes the "failure count: null" log line in the backfill catch path, which read back a row
that did not exist.
Found in review by escherize.
* GHY-4251: Make the dependency jobs' reschedule decisions licence-failure tolerant
Two problems with gating the reschedule on has-feature?, both introduced by the previous two commits.
The entity check job had no event wiring at all, while the backfill job derives
:event/set-premium-embedding-token. Stopping its periodic chain when unlicensed therefore left no way
back except a content change or a restart, because the content-change handlers that would revive it are
themselves gated on the feature. Enabling the feature on an idle instance left the checker dead. Mirror
the backfill job's wiring so setting a token restarts it.
has-feature? also collapses "no licence" and "could not check the licence" into the same false:
*token-features* returns #{} on any Throwable, so a network timeout or tripped circuit breaker reads as
unlicensed. Where that decides whether to schedule it is terminal, since nothing re-fires when the check
later succeeds -- the token never changed. Both inputs to the backfill job's decision consult the
licence, so a single blip stopped that job too.
Resolve the licence for scheduling with canonically-has-feature?, which reports nil when it could not
tell, and stop only on a definitive false. No token at all still answers false, so the common unlicensed
case still goes cold; a token that cannot be validated keeps the job alive. Doing the licensed work
still uses has-feature? and still fails closed -- fail closed on work, fail open on scheduling.
Tests for both jobs now drive all three licence states. The previous two-state tests expressed
"unlicensed" with mt/with-premium-features, which only rebinds *token-features* and so cannot produce a
definitive false; they were passing for the wrong reason.
Found in review by escherize and metamben.
* GHY-4251: Select the analysis batch inside its metadata-provider cache
analyze-batch! selected its instances before entering with-metadata-provider-cache, so each entity in
the batch got a private MetadataProvider attached to its query -- the same shape as the backfill bug,
same default batch size, other job.
Unlike the backfill job this is not a live leak: upsert-analysis! ignores the embedded provider, builds
its own from the entity's database id inside the cache, and passes it explicitly to check-entity, so
those providers stay empty. It is a trap rather than a bug -- the moment an entity is analyzed through
its own query, a batch of 500 retains 500 providers' worth of fetched metadata. upsert-analysis! already
throws when no cache is bound, so the cache was always meant to cover this work; only the select sat
outside it.
Moving the select inside the cache is behaviour-neutral today. The new test pins the intent by comparing
provider identity, since providers compare equal when they wrap the same database id.
Found in review by escherize.
* GHY-4251: Back the dependency test fixture's Tables with the application database
The shared fixture composed its Cards and snippets over metabase.lib.test-metadata, so its Tables existed
only in memory. That is fine for metadata the provider legitimately supplies, but resolving a table name
parsed out of native SQL is a lookup against the appdb, and an in-memory base has no rows to find.
Swap the base to the appdb-backed provider. The H2 test-data database already has the tables these tests
reference, so nothing needs creating; expectations move from mock ids to the real ones. Cards and snippets
stay mocked, since nothing looks those up by name.
These pass both before and after the implementation change in the next commit, which is the point: they
verify behaviour rather than a particular lookup mechanism.
* GHY-4251: Resolve table names with a t2 query instead of a MetadataProvider spec
Narrowing the catalog fetch was implemented as a new :name-ci MetadataProvider spec key. That put the
contract on every implementation for the benefit of one caller, and they did not agree: the overriding
provider ignored the key and fell into its fetch-everything branch, the composed provider took a
fetch-then-refetch path, and the cached provider keyed it by whole-spec so batch sharing was poor.
Resolving a table name parsed out of native SQL is a lookup against the application database, not query
metadata, so do it directly: lib-be/tables-by-name, case-insensitive because normalize-name lower-cases
while some warehouses report upper-case. The SQL still only narrows -- find-table-or-transform continues
to decide -- so matching semantics are unchanged.
Two consequences worth naming. sql-tools now uses lib-be, where the provider previously hid the appdb from
it. And a native query naming a transform's not-yet-created output table now resolves to {:transform id}
via the transform branch rather than {:table <invented-id>} from an override; a dangling invented table id
was the worse answer, but it is a change.
Verified with the OOM repro: 0 catalog fetches, completes in a 1200m heap where master OOMs. Per-card appdb
query volume is unchanged from the spec-key version.
Suggested in review by Alex.
* GHY-4251: Reschedule after a failed run, and keep a disabled job warm
Two changes to when the dependency jobs schedule themselves.
A failure no longer ends the chain. Both jobs are one-shot self-rescheduling with no cron backstop, so an
Error escaping the batch skipped scheduling entirely and the job stayed dead until a content-change event
or a restart. That is survivable for an OutOfMemoryError, which usually takes the process with it and lets
task/init! recover on boot, but not for a StackOverflowError from pathological SQL, which leaves a live
process behind. The backfill job body is extracted to run-and-reschedule! -- it only ever needed the
scheduler, not the whole execution context, which also makes it testable -- and reschedules on the way out
of a failure before rethrowing.
A non-positive batch size now suppresses only the event-driven trigger rather than all scheduling. The
1-second trigger fires on every entity change and never consulted the batch size, which is what made a
disabled job wake roughly once a second; that is the part worth stopping. The job's own periodic schedule
is left alone, so a disabled job keeps ticking slowly and resumes if the setting ever becomes positive
instead of staying cold until a restart. task/init! schedules regardless and logs that nothing will be
processed.
Note the resulting asymmetry is deliberate: scheduling now fails open on an indeterminate licence but still
stops on a non-positive batch size, because the batch size is definitively knowable and a token's status is
not.
Raised in review by metamben; Error case from escherize.
* GHY-4251: Cover the case folding tables-by-name depends on
The narrowed table fetch is a prefilter, not the decision: it selects with SQL lower(name) against names
lowered by u/lower-case-en, then find-table-or-transform matches precisely using the driver's
normalize-name. That is only correct while the prefilter returns a superset of what the precise match would
accept, and one of the two assumptions behind that is testable -- the appdb's lower() has to agree with
Java's.
Add a test that stores tables under ASCII and accented-Latin names and looks each one up by its own,
lower-cased and upper-cased spellings. It needs no per-driver harness: the backend workflow runs it on H2
and app-db.yml runs it on Postgres and MariaDB, so a collation that folds differently fails it there.
Deliberately out of scope, and noted as such: locale-dependent folds like U+0130, where lower() legitimately
differs between engines. The other assumption -- that a driver's normalize-unquoted-name is a pure case
fold -- cannot be tested from here, so it is recorded as a comment at the call site.
* GHY-4251: Cover the other assumption behind the narrowed table fetch
tables-by-name is a prefilter and find-table-or-transform makes the decision, so the narrowing is only sound
while the prefilter returns a superset of what the precise match accepts. The previous commit tested one of
the two assumptions behind that and claimed the other could not be tested from here. It can: "a driver's
normalize-unquoted-name is a pure case fold" is just the assertion that lower-casing the input and the
normalized output gives the same string. Trimming, stripping or substituting characters all break it, and
any of them would make the prefilter drop rows that should have matched.
Each implementation is invoked directly rather than through driver dispatch, so no driver needs registering
or initialising. The test also asserts which implementations it found, because `methods` only sees those
loaded in the run -- without that it would quietly keep passing while covering less and less.
Comment at the call site updated: both assumptions now name their test rather than one being described as
untestable.
* GHY-4251: Avoid the deprecated mbql-query in the new findings test
The full kondo lint that CI runs reports cross-namespace deprecation warnings that per-file runs do not,
and CI fails on warnings. The new provider-sharing test used mt/mbql-query, deprecated since 0.61.0 for
returning legacy MBQL 4.
The test only needs two cards with distinct queries, so use native ones, matching the equivalent test in
backfill_test. That avoids the deprecation rather than adding a namespace-level exclusion for it.
* GHY-4251: Resolve table ids with t2 in sql-tools, then fetch metadata through the provider
Replaces the lib-be/tables-by-name added in 88b011c527. Resolving a name parsed out of native SQL is a
lookup, so sql-tools does that itself against the appdb and gets back ids only; the metadata for those ids
then comes through the MetadataProvider via lib.metadata/bulk-metadata, which is the ordinary :id spec.
Better than fetching the metadata directly on three counts. lib-be grows no public API. Overrides and
caching still apply to the metadata, because it goes through the provider. And the :id spec is cached
per-id, so entities in a batch naming overlapping tables share those entries -- the whole-spec keying of the
earlier version meant {orders} and {orders, products} were separate cache entries fetching orders twice.
The active and visibility filter moves into the id query: an :id lookup does not apply it, and the catalog
fetch this replaced was active-only, so it has to happen where the ids are chosen. The case-folding test
moves with the function, and gains coverage of that filter.
sql-tools no longer depends on lib-be. The module config records :model-imports #{:model/Table} instead,
which scopes the new dependency to the one model rather than a whole module.
Verified with the OOM repro: still 0 catalog fetches, completes in a 1200m heap where master OOMs.
Review feedback from Alex.
* GHY-4251: Drop an incidental reformat from lib_be/metadata/jvm.clj
Left over from the :name-ci spec key that was added and then removed. Restoring the line takes lib_be out
of the diff entirely, so this PR no longer touches any Querying Platform owned path.
* Add backend granular-rerun collector
Scan hawk's JUnit output and emit the failed test vars as a hawk `:only`
vector so a failed backend/driver run can be rerun targeting only those
tests instead of the whole suite. Reuses the existing backend JUnit
adapter. Writes nothing unless every failure is attributable to a var, so
an unattributable failure falls back to a full rerun.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* Wire granular reruns into the driver test action
On a re-run of a failed backend/driver job, restore the tests that failed
on the prior attempt and run only those via hawk `:only`, instead of the
whole suite. After a failed run, collect the failed vars and upload them
as an artifact keyed by junit-name; a re-run downloads it and targets those
tests. Falls back to the full suite whenever no confident failed-test set
exists (first attempt, missing/empty artifact, or unattributable failures).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* Exempt stress workflows from granular reruns
Add a `granular-rerun` input (default true) to the driver test action,
gating the restore/collect/upload steps on it. Stress workflows set it to
false: they intentionally run a fixed test set `:times N` to hunt
flakiness, and narrowing a re-run to only the failed vars would drop that
repetition.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* Wire granular reruns into the non-driver backend tests
Extend the be-tests job with the same restore/collect/upload flow as the
driver action: on a re-run of a failed job, run only the tests that failed
on the prior attempt via hawk `:only`. The rerun drops the tag and
partition args, since the restored vars are already this partition's own
failures — re-partitioning the small set would drop half of them. Each
partition keeps a distinct artifact, so it reruns exactly its own failures.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* Give app-db partition legs a per-job rerun artifact key
The granular-rerun failed-tests artifact was named solely from junit-name.
app-db's partition legs (Enterprise Tests, EE App DB Part 1/2) share one
junit-name, so they collided on a single artifact and a rerun could restore
the wrong leg's failed set. Add an optional rerun-artifact-key input to the
test-driver action and set a per-job-unique value for those legs.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* Collect hawk's var-less-errors JUnit file for granular reruns
hawk (PR #50) now writes fixture-init and namespace load/compile errors to a
fixed-name mb_hawk_var_less_errors.xml, with a testcase that has a name but no
classname. The backend collector only scanned *_test.xml, so it silently
ignored that file and the run's failing set stayed incomplete. Match the file
too; its classname-less testcase parses to a namespace-less failure, which
correctly forces a full rerun instead of a misleadingly narrow one.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* Add a granular-rerun knob
* Scope the rerun artifact to the attempt that recorded it
The failed-tests artifact was named per job, so it outlived the attempt
that wrote it. An attempt that passed, or that failed with nothing
attributable to a test var, uploads no artifact, leaving the previous
one in place for the next attempt to restore -- narrowing a run that
should have gone wide.
Name it `...-attempt-N` on upload and read `attempt - 1` on restore, so
only what the immediately preceding attempt recorded is ever used and
anything else falls back to the full suite.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* Compute the previous attempt number in the shell
Workflow expressions have no arithmetic operators, so
`${{ github.run_attempt - 1 }}` is not a valid expression -- actionlint
rejects it, and the runner would not evaluate it either.
Resolve it with `$((GITHUB_RUN_ATTEMPT - 1))` in a preceding step and
read it back from the env context.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* Clear pre-existing actionlint findings in the stress workflows
These workflows are only linted when a PR touches them, so the findings
below sat unreported until this branch edited both files.
`transforms-python-stress-test.yml` omitted the required
`previous-step-outcome` input on all eight `upload-test-results` steps;
each `test-driver` step already carries `id: test-driver`, so wire its
`test-outcome` output through.
`drivers-stress-test.yml` tripped shellcheck: an unquoted `$JAVA_HOME`
(SC2086) and three consecutive appends to an unquoted `$GITHUB_OUTPUT`
(SC2086, SC2129), now a single quoted group redirect.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* Rename logs-artifact-suffix to artifact-suffix and reuse it for reruns
The failed-tests artifact needs a per-leg discriminator, which test-driver
cannot derive on its own: `matrix` and `strategy` are unavailable inside a
composite action, and `github.job` is the job id, identical across legs.
`logs-artifact-suffix` already carried exactly that value, so reuse it rather
than take a second input for the same concept.
Name both artifacts from `github.job` + `junit-name` + `artifact-suffix`.
Neither of the first two is unique on its own once run-tests.yml pulls app-db,
backend, drivers and semantic-search into one run and one artifact namespace:
`be-tests-postgres` is a job id in two of them and both derive
`be-tests-postgres-ee`.
Fill in the suffix wherever a matrix runs concurrent legs but did not set one,
or set one that did not separate them. Those legs already collided on the test
logs artifact; without this they would also restore each other's failed-test
set on a rerun and re-run the wrong tests.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* Correct the var-less error description in the rerun collector
hawk writes var-less errors to mb_hawk_var_less_errors.xml without a
`classname`; the header claimed they leave no JUnit trace at all. What makes
them unusable for a narrow rerun is the missing namespace, not a missing file.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* Let a colliding failed-tests upload fail instead of overwriting
The artifact name is scoped to the job and the attempt, so nothing legitimately
uploads it twice. With overwrite, a name collision would silently resolve to
last-writer-wins and a rerun would restore another job's failed set; without it
the upload 409s and says so. The step stays continue-on-error, so a collision
degrades to a full rerun rather than failing the job.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* Clear the restored failed-tests file when a rerun can't be narrowed
The collector signals "rerun everything" by leaving no file, but on a rerun that
path already holds the previous attempt's set, restored for the test step to
read. Declining to write left it in place, so the upload shipped last attempt's
list as this one's and the next attempt narrowed to it again — a namespace-load
failure could keep the full suite from ever running again.
Delete it on the no-confident-set path, and again in the CI step before the
collector runs, which also covers the collector dying before it decides.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* Move metabase:// protocol link helpers from metabot to urls
The metabase:// URL protocol helpers (parse/create functions, regex
constants, and entity model types) are a cross-cutting URL scheme used
by both metabot and rich_text_editing, so they belong in metabase/urls
rather than metabot/utils. Moved links.ts (and its spec) to
urls/metabase-protocol.ts, exported it from the urls index barrel, and
updated all importers.
* mysql mbql5
* clickhouse mbql5
* starburst mbql5
* sqlite mbql5
* presto-jdbc mbql5
* druid-jdbc mbql5
* vertica mbql5
* athena mbql5
* hive-like mbql5
* fix databricks
* fix path
* sqlserver mbql5
* oracle mbql5
* fix sqlserver tests
* fix test failures
* fix sqlserver test errors
* clean up
* kondo, test fix
* remove sql-mbql5
* remove from more drivers
* update empty string is null driver
* simplify helpers
* remove unneeded defmulti
* fix field clause
* sqlserver mbql5
* oracle mbql5
* fix sqlserver tests
* fix test failures
* fix sqlserver test errors
* clean up
* kondo, test fix
* small changes
* remove mbql-clause and mbql-clause-with-opts
* make fixes
* more fixes
* remove new mbql5 usages
* Require metabase.driver.sql.pivot so the [:sql :pivot] emitter registers
Removing the sql-mbql5 driver moved sql_mbql5/pivot.clj to sql/pivot.clj,
but the side-effect require moved nowhere: on master
metabase.driver.sql-mbql5 required its pivot namespace, and after the move
no production namespace loaded metabase.driver.sql.pivot at all. Without
that load the apply-top-level-clause [:sql :pivot] method never registers,
so native pivot queries compile to a plain GROUP BY with no pivot-grouping
column (breaking pivot results at runtime). Whole-suite CI masked this
because loading the pivot namespace's own test file registers the method.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* lower ratchets
* fix ratchets whitespace
* use map over mapv
* remove oracle :value
* fix bigquery
* add back comments, update change log
* move comments
* remove unneeded mbql4 cases
* move pivot to sql
* destructure after clause check
---------
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
* GDGT-2938: Review fixes
* Keep the broken-dependents column wide enough for its header
Capping the column truncated the 'Broken dependents' header, which turned on
the Ellipsified tooltip and left two copies of that text in the DOM once the
header was clicked. Its cells hold a short count, so auto width measures the
header and is already stable across pages.
* GDGT-2938: Address PR comments
* Load permission caches whole, with tables in a RoaringBitmap
Each of the three request-scoped permission caches now loads every
database on its first miss, instead of loading the scope it was asked
about and tracking which scopes it covers. That removes the reason the
prime functions existed: a caller no longer has to declare what it is
about to check, so `prime-table-perms-cache`, `prime-db-perms-cache` and
`prime-schema-perms-cache` are gone along with their twenty-odd call
sites, and with them the whole class of bug where a new call path silently
becomes a query per table.
Loading everything is only affordable because the table cache stops
storing a value per table. It now keeps, per permission type, one
RoaringBitmap per permission value holding the IDs of the tables that
resolve to it — a partition, so a lookup walks the type's values and stops
at the one that matches, and each table ID is stored once. Table IDs are
unique across databases, so nothing is keyed by database any more, and
only tables with a grant of their own appear at all.
The bitmaps are built by mutation while private to the load and never
written to once published.
Uncached callers — a check on a user other than the requester, or inside
`disable-perms-cache` — still load just the one table or database they
ask about, since no later check will amortize a wider load.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
* Only use a permission cache that a request scope actually bound
The three caches are dynamic vars, and `use-cache?` checked only that the
user being asked about was the requester -- not that anything had bound a
fresh atom. Code running outside `with-relevant-permissions-for-user` read
and wrote the JVM-wide root atom.
That was survivable while a cache tracked which databases and tables it
covered, since an unseen one still forced a load. Now that a load covers
everything, the first check for a user writes "all of it, for good" into
an atom with no lifetime, and every later check for that user in the same
JVM is answered from it -- so a permission granted afterwards is never
seen. Tests, which set `*current-user-id*` freely outside any request
scope, poison it on the first check.
The root binding is now nil and `use-cache?` takes the cache, so an
unbound one reads through to the database. This is what bounds the
staleness each cache's docstring promises.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
* Record which databases a cache load covered
A cache that loads everything cannot tell "the user has no rows for this
database" from "this database did not exist when I loaded", and answered
both as no-permissions. Callers fall back to the least permissive value,
so a database created inside a request scope came back `:blocked` --
which is what broke the driver and app-db suites: `mt/dataset` builds a
database inside the scope `mt/with-test-user` has already opened, and
every check against it was denied.
Each cache now stores, per user, the set of database IDs its load
covered alongside the index. A question about a database outside that
set reloads. The set is read before the load, so a database appearing in
between counts as uncovered and forces a reload rather than being
wrongly marked covered.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
* Revert the bitmap cache, back to scoped permission caches
Loading every database on a miss traded per-call scoping for a cache that
then has to prove what it covered, and it cost two correctness bugs of
the same family -- the cache over-claiming, first across time (no
lifetime) and then across databases (no coverage record). The memory win
did not pay for that surface.
Back to master's caches, which load the table or database they were asked
about and record it. The next commit adds what this branch was really
after: making an unprimed check loud instead of silently expensive.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
* Add strict cache-miss detection and prime the paths it found
Adds *perms-cache-misses-are-errors?*, bound true by
with-relevant-permissions-for-user in dev and test. A cache's first load
is ordinary; a later miss means an earlier load already covered a
different scope, which is the signature of an unprimed run.
prime-table-perms-cache now also primes the db cache and takes :db-ids,
since table-permission-for-user reads the database's own grant too, and
call sites pass :db-ids wherever they have them.
Genuine N+1s this found and fixed: readable-fields-only, metric
dimension filtering, and the schema listing's table filter.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
* Throw on the fifth cache miss, not the second
Measured across the search, sandbox, dashboard and sync suites: at a
limit of three, 30 tests fail; at five, 18; and raising it to eight or
twelve changes nothing. So the cases between three and five are bounded
-- a dashboard whose parameters span two databases, an entity read plus
a chain-filter elsewhere -- and everything still caught at five is an
unbounded run. MB_PERMS_CACHE_MISS_LIMIT overrides it for re-measuring.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
* Drop the miss-limit env var, keep the measured constant
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
* Correct the miss-limit measurement, revert the unfounded search prime
The 18 failures I measured against were pre-existing: the same batch on
the base commit produces them identically, so the apparent plateau from
five upward was those failures, not the limit catching anything.
Re-measured against that baseline: a limit of two flags 19 sites and
three flags 10, all bounded -- an entity read plus a chain-filter, a
dashboard spanning two databases -- and at four and above none flag. Five
stays, but as a backstop against runs that grow with the data rather than
a linter for every second check.
Also reverts the search prime, which was added on the assumption those
failures were permission misses. They were not, and it bought a query per
search for nothing.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
* Prime the db perms cache for search's per-table permission checks
`check-permissions-for-model` resolves `table` and `indexed-entity`
results against their database, so a search spanning N databases cost N
queries. Results are consumed as a stream, so the databases involved are
never known up front -- and a search ranges over all of them anyway, so
prime every database.
Caught by the strict cache-miss check: metabase.search.api-test's
permissions-test failed with 24 database-level misses, all from
search.impl/can-query?. Postgres-only because the appdb search engine is.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
* Order broken pulses by id so notification order is deterministic
`broken-pulses` selected with no ORDER BY, so the order the notifications
went out in was whatever order the rows came back in.
handle-broken-subscriptions-due-to-bad-parameters-test asserts the second
email is the one about the slack pulse, so it fails whenever the two
broken pulses come back the other way round. Shard-dependent, since the
physical row order shifts as other tests churn the table -- it does not
reproduce standalone, and it fails on master too.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
* Exclude destination databases from full-database selects
Destination databases are reachable only through their router and never
carry data_permissions rows of their own, so listing them just widens the
load. The permission graph in permissions_rest already filtered them;
admin-permission-graph and the new search prime did not.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
* Prime table perms, not just db perms, for search's table results
A `table` search result is checked with `can-query?`, which goes through
table-permission-for-user -- and that reads the table's own grant as well
as its database's. The database half short-circuits only when the db-level
grant is already maximally permissive, which is why the CI failure showed
database misses alone; a user with table-level grants would have gone on
to miss the table cache once per result.
prime-table-perms-cache covers both caches. Passing only :db-ids keeps it
affordable: the table cache holds data_permissions rows carrying a
table_id, so the load is bounded by the user's table-level grants rather
than by how many tables exist.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
* Point the cache-miss error at one prime function and one entity
The message listed all three prime functions and the ex-data carried only
a kind, so it said a check was slow without saying which check.
cached-database-perms takes the table-id its caller was checking, when it
has one. A miss reached that way is a table check -- table-permission-for-user
reads the database cache on its way through -- so it now reports kind
:table and names prime-table-perms-cache, which primes both halves;
prime-database-perms-cache would have left the table half unprimed.
The kind -> function map is spelled out rather than built from the kind, so
renaming one of those functions turns it up. Also renames db to database in
the function names touched here.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
* Rename full-db-permission-for-user to full-database-permission-for-user
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
* Batch the db-ids table-perms load, and stop formatting ids as numbers
Two review findings, both real:
prime-table-perms-cache batched only its :table-ids branch. The :db-ids
branch passed the whole set to one statement, and search now hands it every
non-destination database, so a large enough instance would blow the
bound-parameter cap. Batched like its sibling and like
prime-database-perms-cache.
The miss message went through tru, so MessageFormat rendered table 1234567
as "table 1,234,567". It only ever throws in dev and test, so it does not
need translating -- plain format, no grouping. The test used id 42, which is
below the grouping threshold and so passed against the broken format; it now
uses ids past the thousands.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
The query_execution_partitioned migration failed on stats because the primary key
constraint had an inconsistent name; it originally came from h2 so it didn't match
the constraint name we saw in development. This migration should do nothing unless
the DB is in the same situation as stats where it has a bad constraint name.
`=?` applies an expected function to the actual value, so `complement` --
`clojure.core/complement`, the higher-order function that negates a
predicate -- evaluates to `(complement <actual-value>)`, which returns a
new function. Functions are truthy, so those checks passed for every
input and asserted nothing. `:hawk/key-not-present` is the assertion the
lines were reaching for: the key is absent from the spec.
Hawk also stopped feeding absent keys into the expected predicate as a
truthy sentinel, and now reports a missing key as a diff outright, so
the remaining checks over keys that are not present no longer pass
vacuously either.
In `can-change-from-password-test` that exposed two assertions the test
DB could never satisfy: it authenticates with an uploaded private key
and sets `use-password false`, so `:password` was absent from both the
details and the resulting spec, while the first two cases assert that a
password is present and takes precedence over a key file. The details
now carry a password to switch away from, which is what the test is
about.
In `normalize-use-password-test`, `normalize-details` only ever assocs
`:use-password true`, and only when no private key is configured. The
DB under test has a private key id, so the key is never added -- absent
rather than false.
* Validate OAuth token scopes for MCP
* Test MCP tool calls are refused outside token scopes
* Clean up scoped MCP token test
* Use scoped credentials for MCP UI
* Use MCP public API for UI credentials
* Fix credential test regex, auth-method arity test, reflection warnings
- api_test: the uiCredential extraction regex was over-escaped (matched
literal backslashes) and could never match the rendered HTML, so
mcp-ui-credential-validation-test failed on every backend job
- middleware session_test: auth-method grew a 5th arg (mcp-ui-info) but
auth-method-test still called it with 4, throwing ArityException; also
cover the new "mcp-ui" auth method
- session.clj: type-hint the signature comparison so Eastwood's
reflection check passes
- align let/map bindings per style guide; mention the MCP UI credential
path in wrap-current-user-info's docstring
- frontend: scope the no-literal-metabase-strings disable to the header
line with a reason; drop now-unused setSessionTokenHeader and update
stale comments that referenced it
---------
Co-authored-by: Bryan Maass <bryan.maass@gmail.com>
The "Failed to cancel sync" toast used the app-wide 5s default, which is
too short to read and screenshot, especially since the same handler
dismisses the sync modal underneath it.
Fixes#79029
BOT-1937: Restore the tool call details popup in Metabot chats
Moving the AI analytics views to Monitor replaced the self-contained tool
call modal with an onSelect callback, wired up only by the Monitor
conversation detail page. Normal Metabot chats pass no callback, so
clicking a tool call in a /debug chat did nothing.
Bring the modal back for callers that don't render details themselves,
composed from the existing ToolCallTitle and ToolCallDetailsContent so
the chat and the Monitor sidebar keep sharing one details renderer.
* GDGT-2931: Make Alerts Management tabs visible by default
Removes the full-page blocking loader on Alerts Management so tabs,
filters, and search render immediately, matching the pattern used by
Dependency Diagnostics and CLI/MCP analytics. The grid now shows an
animated skeleton while loading instead of blocking the whole page.
All three tabs (All/Failing/Ownerless) always render with a live count
badge instead of hiding tabs with zero results.
* Add positive assertion for the tab count loading skeleton
Give the skeleton badge a test id so the loading-state test can assert
it's actually rendered, not just that the count text is absent.
* Drop the visually-hidden loading status for the table skeleton
Reverting a restore I mistakenly applied earlier — this simplification
was an intentional manual edit, not something to undo.
* Remove the tab-switch refetch test and its listGate scaffolding
* GDGT-2931: Address PR comments
* Use react-router's own <Navigate>
* Interpolate redirect targets with generatePath and resolve them like v7
* Justify queryToSearch's encoding on its own terms, not history@3's
* Re-export react-router from the barrel instead of a file each
* Flatten the v7 folder into metabase/router
* Drop the V7 suffixes and merge the two router providers
* Use react-router's own hooks and types instead of mirroring them
* Drop the dead function form of a Link target
The subscription save path borrowed getActionErrorMessage from
metabase/actions/utils, whose fallback text ("Something went wrong
while executing the action") is wrong for this call site. Call
getResponseErrorMessage from metabase/utils/errors directly with a
subscription-specific fallback, and drop the actions import.
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
* Put CI MySQL/MariaDB datadirs on tmpfs
A full app-db migration chain on a stock mysql:8.0 container is ~59s of
which ~44s is fsync waiting: MySQL 8 images default to full durability
with the binlog on (log_bin=1, sync_binlog=1, flush_log_at_trx_commit=1),
every DDL statement implicitly commits, and DDL additionally fsyncs the
table files it creates — one measured setup-db! run performed ~50k data
fsyncs + ~37k redo-log fsyncs across 1,464 changesets. No statement is
hot; it is a uniform ~30ms/changeset disk tax, paid for a database CI
deletes minutes later.
Durability knobs cannot recover it: the dynamic ones
(flush_log_at_trx_commit=2, sync_binlog=0) win back only ~20s because DDL
data-file fsyncs are not governed by them, and --skip-log-bin needs a
server command-line flag GHA service containers cannot pass. --tmpfs fits
the one field they do accept (options:) and neutralizes every fsync class:
measured 59.3s -> 15.8s per setup-db!, reproduced.
Applied to every mysql/mariadb service container (app-db, drivers,
semantic-search, cross-branch-migrations). Expected effects: the :db init
timeouts that outran #79014's 120s bump within a day become structurally
impossible (~5-16s init), the ~25min of Liquibase inside a mysql app-db
job mostly evaporates, and job duration stops tracking runner disk
contention. Cost: the app db lives in runner RAM (test DBs are MBs) and
images with pre-baked datadirs re-initialize at container start.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WCo9W7tgK13nNSfWTtQoUc
* Add required description to the cross-branch-migration-test action
Pre-existing actionlint error surfaced by touching
cross-branch-migrations.yml (actionlint scopes to changed files).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WCo9W7tgK13nNSfWTtQoUc
---------
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
* Run route-leave guards on native useBlocker
* Chain the route-leave guards and scope the registry to the provider
* Keep the prompted destination when another navigation supersedes it
* Keep the held navigation in the ref, not resynced from state
* Resume the captured navigation instead of reissuing it
* Revert the supersede handling, keeping the guard fan-out
* Stop the dashboard syncing its URL while a leave prompt is up
* Partition the query_execution table when using Postgres AppDB.
The `query_execution` table gets very big on large installs and poses
performance problems. Partitioning it will allow us to get those under control.
This will only apply when using Postgres as the AppDB. If you have an H2 AppDB,
it's probably not big, and if you have a MySQL AppDD, then, uh, well, too bad.
The basic plan is to rename the existing `query_execution` to
`query_execution_old` and then create a new `query_execution` partitioned table
with the same schema. Once that's done, we can attach `query_execution_old` as
a partition to the main table, and all the data should be available transparently.
Unfortunately partitioned tables have gotchas that complicate things. This
means the migration plan is split into different steps.
The partitioned table must not have a primary key that doesn't include the
partition key. In order to do this, we have to stop using `id` as the primary
key and replace it with a composite key of `id, started_at`.
The first migration swaps the primary keys around, dropping the ones
based on the id alone and replacing them with ones based on `id` and `started_at`.
The second migration renames `query_execution` to
`query_execution_old` and then creates the new `query_execution` table
which is based on `query_execution_old` but partitioned by
`start_at`. Then it takes the `query_execution_old` table and attaches it as a
partition into the full `query_execution` table. This partition covers up to
the current month, plus one more after, in order to handle the case where the
migration runs at the end of the month and query executions need to be written
to disk before the job to create the next partition gets a chance to run.
The table's IDENTITY has to be dropped since this can't be done in a partition;
it should be handled by the higher level table.
The v_query_log view references the old query execution (now `query_execution_old`)
table. We need to re-create the view to point it at the new table instead.
Finally a migration dynamically calculates a new name for `query_execution_old`
to turn it into `query_execution_2026_08` or whatever, depending on
the date of the data that the table will actually contain.
* Add a metabase.app-db.partitions jobs for managing old/new partitions.
The migration puts us in a good state for the present, but as time
goes on, we'll need new partitions for new months. The new
`metabase.app-db.partitions` namespace handles this by defining a new
`ManagePartitions` job that runs daily to see if any new partitions
need to be created or whether any existing partitions need to be
detached.
Each run checks the current month's partition and the next month's
partition and creates them if they haven't been created yet. It also
checks for the retention cutoff period to see which partitions to
detach.
The individual logic functions and the `manage-partitions` function
that tie them together are tested, but the `defjob` is not; I don't
know if there is a way to test jobs.
There is no mechanism for actually deleting the old data once it
passes out of the retention period; it merely becomes detached. We may
need to document this somewhere so that users know they can clean it
up to save disk space. (It's unclear to me where that would be
documented.)
* Disable truncate-audit-tables on query_execution when using postgres.
* Address review suggestions.
* Set sequence on new partitioned table to match old sequence value
* Ensure partitioning job is loaded at startup
* Fix off-by-one error in partition boundaries for dropping
* Use locale-independent yyyy year formatting
* Move to metabase.audit-app.task.partitions
Still needs to address the fact that rollback only converts a single
table instead of all partitions back into one table.
* More review suggestions:
* Fix rollback sequence restoration to target old identity sequence.
* Fix rollback of table renames to use the oldest partition.
* Make partition management cron fire immediately in cases when it's been missed.
* No need to export retention days setting in audit-app module.
* Don't recreate primary key with id, started_at; just drop the primary key.
As far as I can tell, the primary key is not used any more since the
field_usage table got dropped; all reads of this table are just counting.
* Drop partitions instead of just detaching when cycled out of retention.
* Fix rollback to drop view first, then recreate at the end.
* Avoid relying on max(id) in partition migration.
The index will be gone by the time this runs, so it would result in a
full table scan of a HUUUUUUGE table.
* Bump `hawk` to `1.0.18`
This corresponds to
c17e2b9b4e
* fix bad tests
* select data key
---------
Co-authored-by: Riley Thompson <rileythompson99@gmail.com>
* Retry the quarantine list fetch before giving up
A dropped socket or a 5xx from ci-conductor made the gate report COULD NOT
CHECK on the first try. Transport errors, timeouts, 429s and 5xx now get three
attempts with exponential backoff and jitter; a 4xx returns straight away.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* improve retry api
---------
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-authored-by: Ryan Laurie <iethree@gmail.com>
* Add allowedCustomVisualizations setting to EAJS embeds
This commit adds the plumbing necessary to pass down the
value of the allowlist for custom viz.
It warns when wrong names are passed (ie: not starting with `custom:`),
and forces [] for guest embed as the feature is not supported there.
* Move sandbox-host to its own namespace to remove +auth from it
The donor document is loaded as an iframe src, which cannot carry session
headers, so the endpoint must be reachable without authentication. Auth is
applied per-namespace by ns-handler in custom-viz-plugin.api and cannot be
lifted for a single endpoint, so the endpoint moves verbatim to its own
namespace, mounted without +auth next to the session-authed routes (both
still behind the :custom-viz premium gate). No behavior change other than
dropping the auth requirement.
* Serve the custom viz sandbox donor to EAJS embeds
Makes the donor iframe-able as long as either a valid session is passesed
or the correct sec-fetch-site headers are passed.
* Allow blob: images on the EAJS embed page for custom viz icons
Custom viz icons in the chart type picker are served by the SDK asset
manager as blob: URLs (a cross-origin img cannot carry the session
header, and EAJS has no cookies). The modular embedding page runs on the
Metabase origin under a strict img-src that omits blob:, so the browser
blocked the icons and left a broken image in the picker. Add blob: to
img-src only for the /embed/sdk/ entrypoint, gated on a URI predicate so
the main app and static or public embeds keep the stricter policy.
* chore: tidy the SDK release dispatch forms and bump output
The dispatch form renders an input's description as the control's label,
which is a single plain text node: newlines collapse to spaces and no
markdown is parsed. Both SDK release workflows had multi-line
description blocks written as if they rendered, so release_type's
occupied ten rendered lines and pushed the select below the fold.
- release_type: one-line description keyed by option, options reordered
into release-cycle order, and default: alpha pinned so the reorder
does not move the preselection. Option values are unchanged.
- Detail that no longer fits the form moved to the file header.
- Shortened the two read_first option strings that overflowed the
collapsed select at 44 and 81 characters.
- Gave the version-bump PR body's table a real header instead of an
empty one (EMB-2167).
- Capture the new PR's URL and append it to the job summary; the
existing summary step runs before the PR exists (EMB-2168).
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01ENnTtrTcig1vH3QADbsGgo
* chore: capitalise release type names in the dispatch label
The label renders as plain text, so no markdown emphasis is available.
Caps makes each option findable when scanning the wrapped paragraph.
Values stay lowercase - this is display emphasis only.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01ENnTtrTcig1vH3QADbsGgo
* chore: say what each dispatch input is for, not what it isn't
prerelease_id read as an output description, so it never said what to
type. Phrase it as a mapping: enter data-apps, get 0.62.5-data-apps.0.
Same length, same six rendered lines.
The release workflow's label opened with a double negative - use it
'when neither the bump PR nor re-running the failed jobs will do it' -
which left the reader to infer the purpose. It is a manual override, so
say that first. Retry guidance stays in the file header, where there is
room for it. Eight rendered lines down to six.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01ENnTtrTcig1vH3QADbsGgo
* chore: drop a redundant comment above the dispatch label
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01ENnTtrTcig1vH3QADbsGgo
---------
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Dependency upgrade: align both drivers that ship org.apache.thrift/libthrift on the latest release (0.24.0).
hive-like: bump the existing direct pin 0.16.0 → 0.24.0. Since 0.19.0, libthrift declares runtime deps (httpclient5, httpcore5, jakarta.servlet-api) that only serve its THttpClient/TServlet classes; hive-jdbc is built against httpclient 4.x (already pinned in this module) and cannot use them, so they are excluded to keep the driver jar lean. Verified by scanning every class in the built jar: the only classes referencing those packages are libthrift's own five HTTP/servlet classes - nothing in the hive/hadoop stack touches them.
databricks: pin 0.24.0 over the 0.23.0 bundled with databricks-jdbc-thin 3.4.1, following this module's existing pattern of pinning newer transitives (lz4, bouncycastle, jackson, commons-configuration2).
Previously the two driver jars bundled different libthrift versions (0.16.0 and 0.23.0) into the shared plugin classloader; this aligns them on a single version.
Also updates the Spark SQL connection form's "additional JDBC options" placeholder from ;transportMode=http to ;transportMode=binary - binary is the default and supported transport; hive-jdbc's http transport mode is not compatible with libthrift ≥ 0.19 (it passes an httpclient 4.x object to a constructor that now takes httpclient 5.x, see #46793), so the form should not suggest it.
Routine dependency upgrade: bumps the pinned at.yawk.lz4/lz4-java (maintained fork of the discontinued org.lz4:lz4-java) to the latest release, 1.11.1, in the ClickHouse and Databricks driver modules.
Upstream changes between the pinned versions and 1.11.1 are additive only (new fast-reset compressor classes, a linux-riscv64 native binary, bugfixes) — no classes removed, same net.jpountz packages, so it is a drop-in update for both JDBC drivers. Neither driver module calls the library directly.
* GDGT-2884: Move AI analytics to Monitor (#78222)
* GDGT-2874: Relocate AI analytics source under monitor/ai-auditing
* GDGT-2874: Add Monitor URL helpers for AI Auditing
* GDGT-2874: Repoint hardcoded legacy paths to Monitor URL helpers
* GDGT-2874: Swap MetabotAdminLayout for MonitorMain in AI Auditing pages
* GDGT-2874: Fix stale audit_app import in metabot-analytics test-support mock
Plan gap: this file wasn't in Task 1's moved-tree scope and wasn't listed as an
external consumer to fix in a later task. Caught during Task 4 verification
via type-check-pure.
* GDGT-2874: Add AI Auditing sections to monitor_section_clicked event
* GDGT-2874: Add canAccessAiAuditing selector and route guard
* GDGT-2874: Add combined AI Auditing routes getter and plugin index
* GDGT-2874: Move AI Auditing plugin wiring from PLUGIN_AUDIT to PLUGIN_MONITOR
* GDGT-2874: Mount /monitor/ai-auditing route block
* GDGT-2874: Add AI Auditing tab to Monitor nav
* Add regression tests for AI Auditing getActiveSection ordering
Lock in the order of the mcp/conversations/usage-stats route-matching
clauses in getActiveSection: the bare Urls.monitorAiAuditing() path
string-prefixes both sub-paths, so the more specific clauses must be
checked first (ts-pattern's P.string.startsWith is first-match-wins).
Without this, a future reorder would silently break active-tab
highlighting for the two sub-routes.
* GDGT-2874: Remove Auditing folder from Admin AI nav
* GDGT-2874: Add legacy redirects for /admin/metabot/usage-auditing
* GDGT-2874: Fix moved AI Auditing spec mocks and path assertions
* GDGT-2874: Repoint slash-command URL helper off the legacy admin path
The Metabot /inspect slash command still built its target URL via
metabase-enterprise/urls.ts's adminMetabotUsageAuditingConversation,
which hardcoded the old /admin/metabot/usage-auditing/conversations
path. It only worked because it bounced through the legacy redirect.
Delete that helper (its only caller) and use the canonical
Urls.monitorAiAuditingConversationDetail directly.
* GDGT-2874: Update E2E specs to canonical /monitor/ai-auditing URLs
usage-auditing.cy.spec.ts and mcp-analytics.cy.spec.ts predate the
Monitor migration and still visited/asserted on the legacy
/admin/metabot/usage-auditing* URLs, which only worked via the legacy
redirect. In-app navigation (row clicks, breadcrumbs, back button) now
lands on /monitor/ai-auditing* URLs, so any post-navigation URL
assertion in these specs was exercising stale paths. Point every
cy.visit()/pathname assertion at the new URLs, and add a small
regression test asserting the legacy URL still redirects, since the
main specs no longer exercise it directly.
* GDGT-2874: Apply AI Auditing cross-review fixes
* GDGT-2874: Add AI Auditing availability placeholders
* GDGT-2874: Gate AI Auditing pages with settings placeholders
* GDGT-2874: Keep unavailable AI Auditing links navigable
* GDGT-2874: Prioritize global AI settings for MCP availability
* GDGT-2874: Fix empty screen layout
* GDGT-2874: Use semantic AI Auditing navbar icons
* GDGT-2874: Restore Audit plugin ownership
* GDGT-2874: Test legacy redirects in app routes
* GDGT-2874: Add canonical Usage stats route
* GDGT-2874: Cleanup
* GDGT-2874: Address PR review feedback
- Rename AiAuditingAvailabilityLayout -> AvailabilityLayouts: the file
exports two layout components (Metabot and MCP), neither named after
the file/folder.
- Use !!useSetting(...) for the enabled/configured flags, matching the
convention used everywhere else these settings are read.
* GDGT-2883: Restyle AI Auditing to match Monitor (#78356)
* GDGT-2874: Fix moved AI Auditing spec mocks and path assertions
* GDGT-2874: Apply AI Auditing cross-review fixes
* GDGT-2883: Restyle AI Auditing to match Monitor
* GDGT-2883: Fix inconsistencies
* Fix issues surfaced by restacking onto gdgt-2874
- Remove index.unit.spec.ts: orphaned artifact from a superseded
gdgt-2874 draft, testing a ./index module that was never actually
relocated here (registration logic still lives in audit_app/index.js,
already covered by routes.unit.spec.tsx).
- Un-mock ConversationHeader in ConversationDetailPage.unit.spec.tsx:
the blanket mock predated the conversation-title test and was
hiding the real header it needs to assert against; the endpoint
mocks setup() already provides make the real component safe to
render.
- oxfmt fix in ConversationsTable.tsx.
* GDGT-2883: Fix Monitor layout consistency across AI Auditing pages
- Contain the Tool calls grid to the viewport (MonitorMain only clips
when its parent is a flex container, so wrap it in a Flex h="100%"
on the events tab; the charts tab keeps natural page scroll)
- Do the same for the Conversations grid page
- Hide the events table's pagination until it has loaded once, instead
of showing a nonsensical "1 - 0 of N" range
- Add showTotal to the Conversations page's pagination, which was
missing it entirely
- Match tabs/filters/content spacing (md) to Background Tasks on Usage
stats and MCP analytics, which had drifted to lg
- Vertically center the ellipsis menu button against the conversation
title in ConversationHeader
* GDGT-2883: Make all AI Auditing grid columns sortable, fix non-clickable cursor
- TreeTable: only show the pointer cursor on rows that actually respond
to a click (onRowClick or getRowHref set), instead of unconditionally
on every row. Fixes MCP analytics' Tool calls grid looking clickable
when it isn't, and the same latent bug on several other display-only
TreeTable consumers.
- MCP analytics Tool calls: wire up sorting for the remaining columns
(ID, Client version, Tenant, IP address, Error type, Error message).
The query builder already supported ordering by any valid view
column, so this is purely frontend wiring.
- Conversations: add Title as a sortable column, including the backend
allow-list/schema entry it needs. Queries and Searches stay
unsortable — they're computed in-memory per page from message
content, not real SQL columns.
- Fix a stale field name (summary -> title) in ConversationsPage's
test mock that was failing type-check.
* GDGT-2883: Show tool call details in a sidebar on the Conversation Detail page
Replace the tool-call-click modal with a Monitor sidebar, matching the
rest of the AI Auditing views. The live Metabot chat widget keeps its
modal — AgentToolCallMessage now takes an optional onSelect callback
that, when provided, defers to the host instead of opening the modal.
Also style the sidebar's Request/Response copy buttons with the same
Copy/Copied! tooltip used elsewhere in the conversation view, and fix
a pre-existing bug where copying re-stringified the already-formatted
JSON text, copying an escaped string instead of clean JSON.
* GDGT-2883: Toggle the tool call sidebar closed on a second click
Clicking the same tool call record that's already selected now closes
the sidebar instead of re-opening it with the same content.
* GDGT-2883: Constrain conversation content to an 800px column, fix page scroll
Wrap the whole page body (header, stats, feedback, conversation, and
generated queries) in a single horizontally-centered 800px column
instead of leaving it full width.
Fix the page-level scroll along the way: the Flex wrapper added for
the tool call sidebar's width measurement gave MonitorMain's parent
display:flex, which made MonitorMain's flex+overflow:hidden CSS
actually take effect and clip content instead of letting the page
scroll. This page has no internal scroll container of its own (unlike
table-driven Monitor views), so drop the bounded height and let it
scroll naturally.
Also add the missing Copy/Copied! tooltip to AgentToolCallMessage's
own copy button, matching the one already added to the sidebar's
Request/Response copy buttons.
* GDGT-2883: Make AI Auditing pagination abortable, sync page in URL instantly
Every paginated AI Auditing grid (MCP analytics Tool calls, Conversations)
kept a stale in-flight request running when the user paged again before it
resolved, and the URL's page param lagged behind clicks by the shared
useUrlState hook's 300ms debounce — unlike Dependency Diagnostics, where
page changes are both abortable and instant.
- Swap McpEventsTable's and ConversationsPage's queries onto
useAbortableQuery (the same hook DD already uses), so switching pages
cancels the stale request instead of letting it complete in the
background.
- Add an opt-in `immediate` option to useUrlState's patchUrlState that
bypasses the debounce for a single patch, and use it for pagination
clicks specifically. Filter/sort changes keep debouncing as before —
this only affects callers that pass the new option.
* GDGT-2883: Sync pagination to the URL instantly across other Monitor list pages
Same delayed-URL-on-pagination-click bug as AI Auditing, found on a sweep
of every useUrlState consumer with a paginated grid: TaskListPage,
TaskRunsPage, NotificationsAdminPage, and ErrorOverview. Pass the new
patchUrlState immediate option on their pagination handlers too.
TaskListPage's pagination test asserted the old debounced timing (URL
unchanged immediately after a click, then advance the fake timer); updated
it to assert the new instant update instead. The other pages' tests only
asserted the eventual URL state, so they needed no changes.
* GDGT-2883: Use the empty-dashboard illustration for the disabled MCP analytics page
Swap the generic metabot icon for the EmptyDashboardBot illustration and
make the CTA a secondary button, matching other Monitor empty states.
Scoped to the MCP-disabled case only — the AI-features-disabled and
not-configured states elsewhere in this file are unchanged.
EmptyState gains an optional actionVariant prop (defaults to "filled",
preserving every other caller's current look).
* GDGT-2883: Apply cross-review a11y/correctness fixes
- Empty-dashboard illustration + secondary CTA now apply to all four
AI Auditing unavailable states (AI disabled, not configured, MCP
disabled), not just MCP-disabled — that was the intended scope; a
narrower revert from a review pass was itself reverted here.
- Give the tool call copy buttons accessible names (were icon-only).
- Make TreeTable sort column headers keyboard-operable.
- Clear the selected tool call when navigating to a different
conversation, so the sidebar can't reopen with stale data.
- Move focus into the tool call details sidebar on open and back to
the trigger on close; label it as a region.
- Ignore a hidden/PII-gated sort column from a stale URL instead of
querying by it.
- Add a Title-sort UI test and a one-shot immediate-bypass regression
test.
* GDGT-2883: Fix code-review findings in TreeTable and AI Auditing
- Stop sortable header keydowns from bubbling to the treegrid, which
could also activate a stale keyboard-focused row
- Give every populated TreeTable header the columnheader role, not
just sortable ones
- Drop the redundant onRowClick push() on conversation rows, which
double-navigated alongside the row's own link
- Render the MCP analytics loading/empty states inside the active
Tabs.Panel so the selected tab's aria-controls stays valid
* GDGT-2883: Trim redundant comments in McpEventsTable and ConversationDetailPage
* GDGT-2883: Cleanup
* GDGT-2883: Fix e2e tests
* GDGT-2883: Address PR comments
* GDGT-2919: Migrate CLI usage analytics to Monitor (#78900)
* Project MCP events query to curated PII-aware columns
Restricts buildEventsQuery to exactly the columns the events table
displays, keyed off tenant/PII retention settings, instead of shipping
the view's full row (including ip_address/error_message) to the
browser regardless of the setting.
* GDGT-2919: Migrate CLI analytics to Monitor
* GDGT-2919: Drop ai-features-enabled? gate for CLI analytics
The gate was new, not preserved: the old admin route only checked
audit_app/admin access, with no ai-features-enabled? check, and CLI's
v_agent_api_calls data is independent of Metabot/LLM configuration.
* Fix oxfmt formatting
* GDGT-2919: Bring CLI analytics UI in line with the rest of Monitor
- Move the calls filter bar under the tabs and use MCP's spacing values,
keeping tabs/filters mounted (and visible) through loading and empty
states instead of swapping the whole Tabs block out.
- Replace CliEventsTable's AdminDataTable (admin-only, fixed
maxBodyHeight) with TreeTable, matching McpEventsTable's structure so
the Calls tab fills the available height like the rest of Monitor.
* Replace withRouteProps with useRouter across Monitor pages
withRouteProps is documented as a temporary v3-router migration shim
that reads location/params via useRouter() and re-injects them as
props. Have the Monitor pages call useRouter() directly instead and
mount them as plain route elements, dropping the wrapper.
* GDGT-2919: Make all Calls tab columns sortable
Matches MCP's events table, where every column (including the PK and
PII columns) is server-sortable. call_id/tenant_name/ip_address/
error_message are all plain columns on v_agent_api_calls, so ordering
by them needs no query changes beyond adding them to the sort list.
* GDGT-2919: Project CLI events query to curated PII-aware columns
buildEventsQuery shipped the view's full row (including ip_address and
error_message) to the browser regardless of the tenant/PII retention
settings; the frontend only hid those columns from the rendered table.
Add cliEventColumnKeys as the single source of truth for both the
query projection (Lib.withFields) and the display columns, mirroring
the same fix already applied to MCP analytics.
* Use absolute imports for cross-module paths in ai-auditing
Replace ../ relative imports (mcp-analytics/cli-analytics reaching
into metabot-analytics, and cross-directory imports within
metabot-analytics) with the metabase-enterprise/... absolute alias.
Same-directory ./ imports are left as-is.
* GDGT-2919: Show an error instead of spinning forever in CliEventsTable
useCliEventsQuery didn't surface the query's error, so a failed
per-page fetch left data undefined and the table stuck on the
skeleton indefinitely. Surface the error and render
LoadingAndErrorWrapper instead.
Also drop useRetainedValue: RTK Query's data (unlike currentData)
already stays sticky across page/sort changes until the next result
resolves, so the manual retention was redundant.
* GDGT-2919: Show an error instead of spinning forever in McpEventsTable
Mirrors the CliEventsTable fix: useMcpEventsQuery now surfaces the
query error, and McpEventsTable renders LoadingAndErrorWrapper instead
of leaving the skeleton up indefinitely on a failed fetch. Drops
useRetainedValue there too, for the same reason (RTK Query's data is
already sticky across page/sort changes).
Also trims redundant comments in both event tables.
* GDGT-2919: Address PR comments
* Revert gratuitous DependencyDiagnosticsPageOwnProps rename
Keep master's DependencyDiagnosticsPageProps naming so this Graphy-owned
file isn't touched by the master merge for no functional reason.
* Fix jest.mock("metabase/AppComponent") to match its named export
Master's router refactor changed AppComponent from a default-exported
connect() component to a plain named export (App). The mock still
returned a default export, so import { App } resolved to undefined and
broke every route render in routes.unit.spec.tsx.
* GDGT-2874: Fix usage audit e2e test
The treemap label model imported VisualizationGridSize from the
enterprise custom-viz package, a cross-boundary type import. A
structurally identical type already exists in
metabase/visualizations/types, so use that instead.
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
* Resolve relative navigation against v7's own match
* Resolve redirect targets off the v7 match branch
* Drop the router argument from useDashboardUrlQuery
* Free Palette and McpAnalyticsPage from the router context
* Read location and params from v7 instead of useRouter
* Scope route-leave hooks from the match instead of an injected route
* Delete RouterBridge, the router contexts, and the v3 route types
HoneySQL munges dashes to underscore in a keyword's namespace, but not in its name. The code was relying on HoneySQL default behavior to turn namespaced keywords into table references, but the inconsistency bit me in some new development work.
Two paths currently pass schema-qualified keywords in cases where a table might leak:
- `drop-table!` targeted a nonexistent object and silently "succeeded" (not dropping the table)
- `compile-transform` (the SQL variant) could target a nonexistent database.
Both `drop-table-sql` and `create-table-sql` are updated (the latter via `quote-table`) to keep them in sync.
Collapsing the namespace into the keyword's name keeps everything verbatim as you'd expect. Callers already passing in `"schema.name"` weren't impacted.
* Fix QB perf
* Forbid loading every field via GET /api/database/:id/metadata
Make skip_fields a required `true` on the FE request type so no caller can ask
for field-level metadata, which grows with every column in the database.
The two callers that needed fields now scope their fetch to a single table:
- Data Reference field pages fetch the table's own query metadata (which also
pulls FK target tables and their fields)
- useAuditTable resolves the table from the plain database response, then loads
/api/dataset/query_metadata for that table
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
* Withhold the audit table until its fields load
ConversationStatsPage renders charts straight from useAuditTable's table
without checking isLoading, so handing back a table built from the field-less
database metadata made it paint once and re-render when the ad-hoc query
metadata arrived, detaching the chart nodes mid-assertion.
Also make the segment field list spec's database mock honour skip_fields, so
unit tests can catch a caller that stops asking for fields.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
* fix be
---------
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
* BE Add Mistral metabot provider
* BE Add prompt-cache-key to LLMRequestOpts for Mistral prompt caching support
Mistral requires a `prompt_cache_key` to be set to enable prompt caching, so add it as an optional key to
`LLMRequestOpts` and thread it through `self/call-llm` and `self/call-llm-structured`.
* FE Add Mistral metabot provider
* Review feedback: fold all provider selectable tests into a single test
* Review feedback: use findPutRequests to reduce verbosity
* BE Add Z.AI metabot provider
* FE Add Z.AI metabot provider
* Move generic chat-completions code to metabase.metabot.self.openai.chat-completions
And use it in both the openrouter and zai providers.
hashSearchTerm and shouldReportSearchTerm are analytics reporting
policy (settings read + SHA-256 hashing), not search UI, so they move
from common/search/term.ts to the lib-tier analytics module and are
exported from its barrel. Consumers now import them from
metabase/analytics, and the common/search barrel no longer re-exports
them.
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Relocate the isVisualizerDashboardCard type guard from
metabase/visualizer/utils to metabase-types/guards/dashboard, alongside
the other dashboard type guards, and update all importers to the new
path. Remove the re-export from the visualizer utils barrel and delete
the old file.
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
* Add parseSearchQuery and a useNavigationType facade hook
* Move the location.action readers to useNavigationType
* Move the location.query readers to search params
* Drop query and action from the facade Location
* Replace connect() with hooks in the app shells and permissions pages
* Rewrite the reference containers as function components
* Delete withRouteProps
* Pass the props reference children read but do not select
* Cover every reference container page with a render test
* Fetch reference container data before the child's first render
* Add Snowplow event tracking to the metric Dimensions tab
* Update events' definitions to comply with new approach
* Update analytics-events claude skill to account for new approach defined in #71631
TableBrowser only needs the table list, but GET /api/database/:id/metadata
returns every field of every table. Pass skip_fields=true so the response
stays proportional to the number of tables.
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
HoneySQL inlines true/false as TRUE/FLASE tokens even when the query is otherwise parameterized, and not every driver target has a boolean literal. SQL Server parses a bare TRUE as an identifier and `insert-into!` fails with `Invalid column name 'TRUE'`. Wrapping boolean row values in `[:lift v]` binds them as paremeters, which is what we do for other value types.
Impacts every `:sql-jdbc` driver's `insert-into!`. Drivers with boolean literals are not impacted.
Fix quadratic :is_personal hydration that made GET /api/collection take ~23s
On an instance with 28k collections — 6k of them personal — `GET /api/collection`
took 23.5s while making only 13 DB calls. Nearly all of it was the `:is_personal`
batched hydration: for every collection it scanned the full set of personal
collection IDs, building a prefix string with `format` on each step, so a request
did ~20k x 6k = 125M `String/format` calls.
Personal Collections only ever live in the Root Collection, so a collection is
inside one exactly when the *first* ID of its location path is a personal
collection. Test that single ID against the set instead of scanning.
Measured on a seeded app DB matching the reported shape (28.6k collections, 6,011
personal roots, 13.6k live non-personal, 7.8k archived, depth to 13):
GET /api/collection 11.2s -> 1.9s
:is_personal hydrate 9.2s -> 75ms
The remaining hot spot is `:can_write` (~1.1-1.5s), which linearly scans the
user's permission set per collection — left for a separate change since it
touches the shared permission check.
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
* Convert the store factory and whitelabel color bootstrap to TypeScript
* Name the deliberately loose reducer map type
* Condense the reducer map comment
* BigQuery: reject oversized queries client-side with a humanized error
BigQuery rejects jobs whose SQL exceeds 1 MB (1,048,576 characters,
including comments and whitespace) with a raw 400 INVALID_ARGUMENT,
which Metabase surfaces to the user as "Error executing query: <raw
message>" -- accurate but technical and not actionable.
Check the final compiled SQL (including the appended remark) before the
BigQuery client/job is created and throw a localized, actionable
:invalid-query error instead. The raw DB error path remains as a
fallback if this limit ever drifts from BigQuery's own.
Quota reference: https://cloud.google.com/bigquery/quotas#query_limitsCloses#79019
* Update error message for SQL query length validation
* Address review: express limit as (* 1024 1024), slim the tests
- max-sql-query-length-chars: write the value as (* 1024 1024) so the
1 MiB intent reads off the page.
- validate-query-length-test: collapse to the two boundary cases (at the
limit allowed; over the limit throws :invalid-query), error type only,
per review feedback to keep it simple.
* Address review: trim noise from max-sql-query-length-chars docstring
Drop the cross-reference to validate-query-length! and the "not ^:const"
justification — commentary, not contract. Leaves what the constant is,
how BigQuery counts, the failure mode, and the quota link.
* Add AI provider setup to onboarding
- Insert AI config step into setup flow
- Track connect and skip analytics events
- Update onboarding tests and schema version
* Preserve existing AI provider during setup
- Wait for the saved provider before mounting the form
- Show the connected provider in the setup step and cover it in tests
* Defer AI provider sync until after setup
- Keep the AI config step visible during onboarding
- Skip legacy Metabot provider migration on fresh instances
* Add AI provider setup to onboarding
- Hide the step when AI features are disabled
- Support Anthropic onboarding setup in e2e
- Mock Anthropic model catalog for credential validation
* Match license step layout in AI setup step
* Fix modules config and flaky setup toast assertion
* Add free_units to managed AI e2e add-on fixture
* Condense AI step ordering comment
* Make AI setup opt-in from the completed step
The AI provider step is no longer part of the wizard by default. The
"You're all set up!" screen offers a "Set up AI" button that appends
ai_config as the last step; connecting or skipping returns to it.
* Scope the settings-list mock to the test that needs it
settingOverrides was only used by one embedding test, so drop it from the
shared setup helper and let that test override the settings-list route.
* Type the visualization registry as components or bare definitions
* Convert the dashboard viz register module to TypeScript and use typed named exports for Action and IFrameViz
* Strip large literal IN lists before GraalPy SQL parsing
Large literal IN lists (like large VALUES clauses) blow GraalPy's ~114KB/AST-node
memory cost and its 30s call timeout: a 300k-item list that sqlglot parses in
1.7s on CPython times out at 30s on GraalPy, poisoning a pooled context on every
background dependency-analysis pass over the offending query.
Extend the JVM-side pre-processing to also replace literal-only IN lists with
more than 100 items with `IN (NULL)`. Only lists consisting purely of literals
(numbers, quoted strings, NULL/TRUE/FALSE) are stripped; column references,
function calls, subqueries, and bind parameters disqualify a list, since
stripping those would change analysis results or parameter counts. Lists nested
inside unstripped IN subqueries are still found.
All callers get the same handling: the stripping happens in the one shared
strip-large-literal-lists (renamed from strip-large-values) used by every
analysis entry point, and is-single-stmt-of-type? keeps the #74284 restore
behavior — validate the stripped SQL, return the original text — so
impersonated queries still execute with their literal lists intact.
With the fix, field-references on a 300k-item IN list drops from a 30s timeout
to ~80ms.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* Simplify literal-list stripping to a single shared pass
Review cleanup: collapse the two per-keyword scanners into one walk over the
SQL driven by a combined VALUES|IN pattern with small per-keyword handlers,
extract the quote/doubled-quote scanning into a shared skip-string-literal
(also fixing count-top-level-commas, which didn't handle doubled-quote
escapes), and replace the substring-allocating literal-only-list? check with a
single index-based early-exit pass, so IN (SELECT ...) is rejected without
scanning or copying the subquery region. No-strip calls now return the
original string identity.
field-references on a 300k-item IN list drops from ~80ms to ~27ms; all
existing stripping tests pass unchanged.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* Add exhaustive tests for literal-list stripping
Cover threshold boundaries (100 vs 101) for both VALUES and IN, every accepted
literal shape, a disqualifier grid (17 non-literal items x 3 positions),
byte-for-byte preservation of surrounding SQL, nesting (IN inside VALUES
tuples, stripped-VALUES swallowing nested lists, deep IN-subquery chains),
idempotency, original-instance identity on the no-strip path, malformed input
(unbalanced parens, unterminated strings, empty lists), keyword false
positives (margin, JOIN, INT, quoted identifiers), and an end-to-end pass of
every analysis entry point over stripped SQL.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* Also strip large ARRAY literals and literal tuple IN lists
Two more static-data shapes reproduce the GraalPy 30s timeout / severe CPU
burn: id = ANY(ARRAY[...]) (the Postgres idiom ORMs emit instead of IN; 200k
items hit the 30s timeout) and composite-key tuple lists
(a, b) IN ((1, 1), (2, 2), ...) (50k tuples burned 21s).
Generalize the flat literal-list scanner to a parameterized closing delimiter
and reuse it for ARRAY[...] -> ARRAY[NULL], and add a tuple-list scanner that
requires every tuple to be literal-only, stripping to IN ((NULL, ...)) with
the first tuple's arity. Subscript access (arr[1], arr[1:5]), subquery tuples,
and any non-literal item still disqualify a list. The is-single-stmt-of-type?
restore behavior covers both new shapes.
After the change: tuple-IN 50k tuples 21.6s -> 525ms; ANY(ARRAY[200k])
30s timeout -> 1.1s. CASE WHEN chains and UNION ALL literal tables remain
unstripped (no lossless strip exists for them).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* Treat double quotes as a disqualifier in literal-list scanning
In identifier-quoting dialects (Postgres, Snowflake, ...) "x" inside an IN
list or ARRAY literal is a column reference, not a string, so stripping a
large list of double-quoted items would drop real field references from
analysis. The literal-list scan is dialect-agnostic, so be conservative: only
single-quoted strings count as literals; a bare double quote leaves the list
untouched. skip-string-literal still skips both quote styles where it is used
for paren/comma balance scanning, where either interpretation is a
skip-region.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* Use [[strip-threshold]] docstring links instead of backticks
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* Fix Eastwood reflection warning: cast close-ch to primitive char
(= ch close-ch) compared a primitive char against the untyped close-ch
parameter, which Eastwood flags as an unresolvable Util.equiv call.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* Ignore keyword matches in strings/comments; reject empty list elements
Review follow-ups:
The keyword matcher previously rewrote text inside string literals and
comments — SELECT 'IN (0, ..., 149)' became SELECT 'IN (NULL)', and an
unbalanced quote inside such a region could corrupt the statement. The walker
now gap-scans between matches with a string/comment-aware cursor
(code-region-end) and copies any match that sits inside '...', "...",
-- ..., or /* ... */ through untouched.
The flat-list scanner also accepted empty elements, so invalid SQL like
IN (,,,) with >100 commas was rewritten into valid IN (NULL), masking syntax
errors from validation. Both the flat and tuple scanners now require content
in every element and reject leading/trailing/consecutive commas.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* Simplify IN/ARRAY stripping to one simple-list scanner
Stripping feeds fail-soft analysis only, and execution paths restore the
original SQL, so the defensive machinery accumulated from review rounds
(literal word allow-list, tuple/arity validation, empty-element checks,
string/comment-aware outer scanning) bought correctness for cases that don't
matter at the cost of real complexity. Replace it all with one rule: a list is
strippable when, from its opening ( or [, it contains only digits, signs,
single-quoted strings, commas, whitespace, and balanced parens (which covers
tuple lists with no special logic), and holds at least 100 commas. Flat IN,
tuple IN, and ARRAY all collapse into a single ~20-line scanner and one
decision fn; NULL/TRUE/FALSE items now conservatively block stripping. VALUES
keeps its pre-existing count-based path. The best-effort contract is stated in
the public docstring.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Seven sites across four drivers were escaping SQL string literals the same way. This consolidates, and there's a single place to warn not to use escape-sql to sanitize user input.
No behavior changes.
MBQL treats `java.math.BigInteger` and `clojure.lang.BigInt` interchangeably. Only the Clojure class was in the parameter table, so a BigInteger fell through in one of my tests to `:default` and bound as a string.
* Show the output schema and table for a transform run
Adds a TransformOutput component that renders the run's target as `schema › table`
breadcrumbs with a copy button. Both crumbs link out when they exist: the schema to
the data model, the table to the query builder. Wired into the run sidebar
("Output table" row) and the job run sidebar ("Output:" line).
Also fixes the job run sidebar not shrinking below its content width (missing
min-width: 0), which let long table names push it past the viewport.
* fix tests
* drop newTab prop, always open output links in new tab
* addressing PR's comments
* fix race condition
The three preceding PRs removed every shared-tier import from the module: the
route guards, the redux location mirror, and the styled-Link re-export. All that
is left is metabase/utils, which basic may import.
basic rather than lib: router is a React component library, the same shape as
metabase/ui, and nothing in basic/* imports it, so lib would only widen what
lib/utils is allowed to reach for no gain.
* fix(sdk): sdk crashes when resizing column
* do the same on ExecutionOutputTable too
* reword test to make it better
* make sure cols/rows are not new arrays every render when output is undefined
* Derive Tabs value type from the tab values
Mantine types the Tabs value and onChange argument as `string | null`. Make
TabsProps generic over the tab value so callers get their own union back in
onChange instead of a bare string.
* Drop Tabs onChange type workarounds at call sites
Every `<Tabs>` call site now annotates its own tab union instead of casting or
re-narrowing the change value: the six `as SomeTab` casts (CommentsSidesheet,
PythonEditorResults, GroupLimitsSettingsSection, McpAnalyticsPage,
CliAnalyticsPage, ConversationStatsPage), the four in the admin/permissions and
tenants listings, and the handlers that laundered `string | null` through a
lookup or a literal comparison (AddDataModal, NotificationsTabs, TableSection,
the date pickers, TabPanel). AddDataModal's `isValidTab` guard existed only for
this and is gone; DatasetFieldMetadataSidebar's TAB map is now `as const`.
* Cleanup
* Apply suggestions from code review
Co-authored-by: Kamil Mielnik <kamil@kamilmielnik.com>
---------
Co-authored-by: Kamil Mielnik <kamil@kamilmielnik.com>
* Mark metabase/router as side-effect free for rspack
* Route deep metabase/router imports through the barrel
* Let static-viz import the router barrel instead of a deep module
* Guard the side-effect-free contract and scope the include to the directory
metabase/router/Link.ts re-exported metabase/common/components/Link so every
routing import had one home. But that Link is not a routing primitive: it wraps
RouterLink with a Tooltip from metabase/ui, a CSS module, and disabled/variant
props, so the barrel dragged shared/common and basic/ui into the router module
and looped back into router/router-link.
Call sites import it from metabase/common/components/Link again. RouterLink and
RouterLinkProps stay in metabase/router, which is where the primitive belongs.
The boundaries linter never flagged this: element-types checks imports, not
'export ... from'. So the edge had to go rather than be waived, or a lower tier
would have been green while the dependency was still there.
* Fix N+1 query patterns in GET /api/database and GET /api/native-query-snippet
GET /api/database?saved=true&include=tables issued ~6-7 app-DB queries per
saved question (one metabase_database select-one plus a field select and
4-5 hydrations each) — ~3000 queries on instances with a few hundred
eligible cards:
- card->virtual-table: only fetch the database when include-database? is
set, instead of unconditionally per card
- extract cards->card-id->metadata-fields so the result_metadata fields of
all cards are fetched and hydrated in one batch (reused by
batch-fetch-card-query-metadatas), and pass it from cards-virtual-tables
- ids-of-dbs-that-support-source-queries: one select instead of
select-pks-set + per-id select-one
GET /api/native-query-snippet ran user-has-any-perms-of-type? — an
uncached EXISTS against data_permissions — once per snippet via
mi/can-read?, even though the answer is per-user, not per-snippet:
- user-has-any-perms-of-type? now primes the request-scoped permissions
cache for all databases (the same pattern query-creation-capabilities
already uses on GET /api/user/current) and answers by scanning the
cached entries; repeated calls in one request are pure cache hits
- prime-all-dbs-cache records completeness with an ::all-dbs sentinel in
the existing :db-ids set so the database id list is fetched only once
- the uncached EXISTS path is kept for cross-user checks and
disable-perms-cache
Note: the cache prime excludes permission rows for inactive tables, so a
grant existing only on a deactivated table no longer satisfies
user-has-any-perms-of-type?.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* Track full-prime state under a dedicated :all-dbs-primed? cache key
Move the all-databases-primed marker out of the :db-ids set (where it sat
as a ::all-dbs sentinel among real database IDs) into its own
:all-dbs-primed? key on *permissions-for-user*. prime-db-cache now
preserves unrelated keys by assoc-ing instead of reset!-ing the atom.
Make prime-all-dbs-cache public with an optional pre-fetched-ids arity,
and switch query-creation-capabilities — the one caller that manually
primed all databases — to use it, so the full-prime work is shared with
user-has-any-perms-of-type? within a request.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* Simplify prime-all-dbs-cache: single arity, :all-dbs? key
Drop the pre-fetched-ids arity — the function always fetches the database
ID list itself — and rename the completeness marker from :all-dbs-primed?
to :all-dbs?.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* Align :all-dbs? docstring continuation line
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* Add call-count regression tests for the two fixed N+1 endpoints
Both tests measure warm app-DB call counts via t2/with-call-count at two
entity counts and assert the count doesn't scale:
- GET /api/database?saved=true&include=tables with 2 vs 18 eligible cards;
allows the 1-per-card toucan2 identity-query overhead from
do-after-select but fails on the pre-batching one-select-per-card
behavior (verified failing against the pre-fix code).
- GET /api/native-query-snippet with 2 vs 10 snippets as a non-admin;
fails on the per-snippet permission EXISTS check (verified failing
against the pre-fix code).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* Use mt/with-model-cleanup in call-count tests
Replaces the try/finally + manual delete pattern, so fixture rows are
cleaned up even when insertion or the warm-up request throws (addresses
review feedback).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* Rework any-perms-of-type caching to a dedicated request-scoped cache
Instead of priming the full permissions cache for every database and
scanning it, cache per [user-id perm-type] the set of database IDs
granting the most permissive value, loaded across all databases in a
single query. This answers every :exclude-db-ids variant from one small
fetch and keeps semantics identical to the uncached query.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* perms cache
* nil fix
* fix
* current state
* cleanup
* Test that inactive-table-only grants are ignored by aggregate perm checks
user-has-any-perms-of-type? previously counted grants existing only on
inactive tables (its raw EXISTS query had no active-table filter); now
that it is served by the distinct-perms cache it ignores them, matching
every other cached check. This is intentional — cover it with a test.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* Apply code-review fixes to the permissions cache rework
- Restore batch priming in with-can-run-adhoc-query via a new public
prime-full-perms-cache (table-level checks otherwise load the cache
one database at a time, up to 2 queries per database on card lists)
- Scope the non-cached distinct-perms load to one database instead of
scanning all of the user's permission rows per call
- Extract the shared FROM/JOIN/WHERE of the two cache queries into
perm-rows-query-base so their row sets cannot drift apart
- Simplify the full-perms interner binding and drop a dead fnil and a
redundant double-check inside the distinct-perms swap!
- Select only :id and :engine in ids-of-dbs-that-support-source-queries
(full rows decrypt :details per database, twice per request)
- Own the batched virtual-table Field fetch in a new
cards->virtual-tables entry point; the helper is private again
- Use mt/with-dynamic-fn-redefs in permission-caches-load-per-type-test
- Document that the request caches are point-in-time snapshots
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* Treat cached false as a hit in fast-interner
or-based lookups skip false, so interning false re-ran canon-fn and
putIfAbsent on every call (result was still correct). Use if-some so
presence, not truthiness, decides the hit.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* Document why distinct perms load all databases and why the uncached interner exists
Capture the review-thread answers in the docstrings: distinct-perms
preloads every database because most consumers ask cross-database
questions, and full-perms builds a per-call interner even without a
request cache because group values are highly repetitive.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* Batch-prime the full perms cache in the remaining per-table check loops
Endpoint tracing found three more loops that fill the full permissions
cache one database at a time now that priming is on demand: the
include=tables branch of GET /api/database (per-table can-read?/
can-query?/can-write?), batch-fetch-query-metadatas* (per-table
can-read? behind dashboard/card query_metadata), and the EE
filter-tables-by-data-model-perms loop. Each now batch-primes the perm
types it checks, one query per type instead of up to 3 per database.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* Simplify perms caches to all-types loading; rename to table/schema caches
Per-type bookkeeping wasn't worth it — nearly every consumer needs
several permission types, and there are only a handful. Both caches now
load every permission type at once: the table-level cache (was
*full-perms-cache*) per database, the schema-level cache (was
*distinct-perms-cache*, table_id collapsed so schema is the finest
granularity it answers) for all databases in a single query on first
use. prime-table-perms-cache takes just database IDs and no-ops for
superusers, whose checks never read the cache.
Also batch-prime the remaining per-table check loops found by a
repo-wide audit: search result filtering, GET /api/table, recent views
(also reusing the already-fetched table instead of re-selecting it),
typed-schemas table listings, and the segment/measure list endpoints.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* Drop the search prime and scope the recents prime to actual table recents
Benchmarks on a 10M-row data_permissions Postgres (200k rows for the
test user) showed both primes regressing their endpoints: search went
40ms to 840ms because priming all databases loads every permission row
of the user on each request while matching results only touch a few
databases, and recents primed the whole instance because table-recents
returns every table when given no IDs. Search now relies on lazy per-db
loads; table-recents returns nothing for empty IDs so the prime no-ops
naturally.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* Load the schema perms cache per database, with a priming function
Benchmarks showed the all-databases SELECT DISTINCT costing ~230ms per
request at 200k user rows, taxing endpoints that ask about one database
(dashcard query 228ms -> 445ms). The schema cache now mirrors the table
cache: single-database questions load one database at a time (indexed,
all permission types at once), loops over many databases call
prime-schema-perms-cache, and cross-database questions load everything
once, recorded under :all-dbs? so repeated checks stay one query.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* Load perms caches via next.jdbc/plan instead of toucan2 rows
JFR allocation profiling showed toucan2's per-row machinery (row delays,
transient instances, realize maps) dominating the cache loads' share of
request allocation. plan reads columns straight off the ResultSet with
no per-row maps; the folds are unchanged. Runs on the current toucan2
connection so transactional writes stay visible. On the 10M-row bench
this cuts per-request allocation ~10-15% further (e.g. /api/user/current
570MB on master -> 376MB) and 50-90ms of latency on load-heavy
endpoints. Note plan queries are invisible to t2/with-call-count, so
one test now spies on the load fn instead of counting calls.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* Store perms caches as mutable Java collections, skipping persistent copies
The folds now build nested HashMaps/HashSets that the caches store
as-is: no per-row persistent update-in churn and no whole-structure
persistent copy at the end. Clojure's lookup/seq functions read Java
collections directly, so consumers are unchanged. Only the two outer
levels (perm-type, db-id) are ConcurrentHashMaps that later loads extend
via putAll of missing databases; everything below is written once by its
load and then only read. Numeric keys are canonicalized to Long at both
fold and lookup time — JDBC hands back Integer for int columns and Java
maps compare boxed keys with equals, not Clojure's equiv. The entry
interner is gone with the persistent copy; instances with thousands of
identical per-database profiles lose that sharing (#76077), which is the
trade-off for the allocation win.
On the 10M-row Postgres bench (200k rows for the user) vs master:
/api/user/current 570 -> 268 MB allocated per request, GET /api/database
?saved=true&include=tables 1773 -> 1032 MB and 4.1s -> 2.5s,
GET /api/table 1152 -> 865 MB and 1.9s -> 0.9s.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* Answer cross-database permission questions from a light per-function cache
Instances with thousands of databases and groups make loading tuples
for every database (schema cache with :all-dbs?) scale badly: the
distinct tuple set alone reaches hundreds of thousands of rows. The
questions that actually span databases only need one value per database
for one permission type, so most-permissive-database-permission-for-user
(signature unchanged) and user-has-any-perms-of-type? now share
*most-permissive-perms-cache*: {user-id {perm-type {db-id value}}},
loaded by one indexed query returning distinct (database, value) pairs
and coalesced per database. The :all-dbs? machinery is gone and
query-creation-capabilities needs no priming — its per-database loop
hits the cache after one query.
On the 10M-row bench this takes GET /api/user/current from 722ms/268MB
allocated to 556ms/182MB — 5MB above the response-validation floor
measured with all permission checks short-circuited (master: 833ms/570MB).
Also reorganize the caching section (atoms, then loaders, then the
functions using them; coalesce moves above since a loader needs it) and
relax the xrays call-count expectation: permission-cache loads go
through next.jdbc and are invisible to t2/with-call-count.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* Coalesce permissions in SQL and give every check its own per-function cache
Each permission-checking function now owns one atom, one hand-written
aggregate query, and a cache structure fitting exactly its question,
organized per function. Values map to ranks (value-rank-case) and
GROUP BY collapses tables, duplicate values and — via two nested levels
of aggregation for the most-restrictive-within-group-then-most-
permissive-across-groups questions — group identity too, so result sets
scale with tables/schemas/databases rather than raw row counts
(dbs x groups x tables x values). A bucket's [min max] rank pair
reconstructs coalesce exactly, verified exhaustively per permission
type including :perms/view-data's special rule; full-schema keeps its
per-group db-level union by expanding db-level rows into every schema
inside its query. native-download delegates to full-db-permission
(identical semantics), user-has-any derives from the most-permissive
cache.
With the row-count multiplier gone the loads are back to plain t2/query
and persistent Clojure collections: the next.jdbc/plan machinery, the
mutable Java collections, the Long key canonicalization, and the
interner are all gone, and cache loads are visible to
t2/with-call-count again. No public API changes; the batch primer is
now prime-db-perms-cache and warms every per-database cache.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* Drop a docstring reference to the removed full-perm-rows
The ~15x realize-before-key-access observation stands on its own; the
function it pointed at no longer exists now that the permission caches
load SQL-side aggregates.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* Rewrite app-db NOT IN (SELECT ...) queries as NOT EXISTS anti-joins
NOT IN (subquery) degrades to a per-row subplan on Postgres once the
subquery result exceeds work_mem. Rewrites all runtime occurrences to
correlated NOT EXISTS (or a literal comparison where the subquery was
0/1 rows). MySQL keeps the double-wrapped subquery in
delete-database-fields! since it cannot reference the DELETE target
table in a subquery.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* Keep is_audit predicate as NOT EXISTS in set-default-database-permissions!
Tests create audit DBs with random ids, so the fixed audit-db-id
constant is not equivalent to the is_audit = true subquery here.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Snowflake: don't over-escape schema names in the dynamic-table probe
The FK sync path pre-escaped schema/table names (`_` -> `\_`) before handing
them to `dynamic-table?`, which uses the schema as an identifier in
`IN SCHEMA <db>.<schema>`. `"RAW\_DATA"` is not a real schema, so Snowflake
threw for every table in any underscore-named schema and we logged a full
stack trace each time.
`dynamic-table?` now takes raw names and escapes only the `LIKE` argument,
where `_` is actually a wildcard. Escaping for the JDBC metadata call moved
to the call site. Also log the caught exception without the stack trace,
since it fires once per table.
Fixes#78541
Co-authored-by: da <paoliniluis@gmail.com>
* Generate a fresh title for a forked conversation from its first new message
* Show "Forked conversation" instead of "New conversation" in fork header
* Rename first-valid-user-message to first-non-forked-user-message
* Align map value columns in metabot conversation schemas (PR review nits)
* feat: fork a Metabot conversation at an assistant message
* feat: surface Metabot conversation fork origin in analytics
Record forked_from_conversation_id on the conversation and
forked_from_message_id on each copied message, expose them through the
metabot-analytics API and audit views, and link a fork back to its
original thread.
* Update git_branch icon
* Show source conversation title in fork boundary and header
* Revert fork boundary to the attempt where the fork happened
Showing the fork boundary divider on every regenerated attempt was
misleading: the fork only happened on the inherited attempt, so the
divider matches that single message again.
* Refine fork boundary rendering and tests
* Exclude forked (copied) messages from v_metabot_messages
* Address PR review: unify message-action guard, extract ForkBoundary, assert fork toasts via UI
* Reuse createMockMetabotConversationDetail for the fork test fixture
* Replace fork props on Messages with a renderAfterMessage slot; move ForkBoundary to EE
* Drop the redundant UndoListing now that the metabot test setup mounts it
* Index and FK-constrain forked_from_conversation_id so deleting an origin nulls the link
* Split forked messages out of usage metrics with new_message_count
v_metabot_conversations gains new_message_count, which leaves out the
messages a fork copied in from its origin. message_count keeps counting
the whole thread, so the conversations list, the detail page and the
shipped model all report what the reader can actually see.
The usage stats charts aggregate the new column instead, which makes
SUM(new_message_count) reconcile with COUNT(*) FROM v_metabot_messages.
metabot-stats leaves copies out of :metabot-queries and :metabot-users
for the same reason — copies inherit role/ai_proxied and are stamped
with a fresh created_at, so a fork was reporting LLM calls that never
happened.
* Don't offer to fork while a turn is streaming
Only the streaming message's actions were hidden, so earlier assistant
messages kept their fork button mid-turn. Forking then swapped the
conversation out from under an in-flight stream whose reducers are keyed
by agentId alone, so the old turn's remaining deltas rendered inside the
fork.
Withholding onFork while isDoingScience is set is preferable to
cancelling the request: the user asked to branch off an earlier message,
not to stop the response, and loadConversation already avoids cancelling
so an aborted turn isn't recorded.
* Realign the analytics schema maps and fix a migration author
The fork columns are longer than the alignment column the surrounding
maps used. ConversationDetail had already drifted into two alignment
columns before this branch; both maps are now uniform.
Extends the dynamic JSX formatter pattern from #77658 (which did links)
to markdown and email. formatValue and formatEmail now call a registered
renderer through the registry instead of importing ReactMarkdown/Mustache
and ExternalLink directly; the renderers live in ui.tsx and are wired up
at app boot by registerJsxFormatting (called from registerVisualizations).
This keeps the react-markdown/micromark/mustache stack and the mailto
ExternalLink out of the static-viz bundle, which imports the formatting
engine through shared echarts code but never registers the JSX renderers
(so jsx formatting degrades to plain text there, as it already did for
links). No behaviour change in the app.
Static-viz bundle: 3,450,947 -> 3,297,087 bytes (~150KB, 4.5%).
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
metabase.metrics.core re-exports sandbox-restricted-fields from metabase.metrics.permissions, and metabase.metabot.tools.entity-details calls it through that API. The passthrough to metabase.permissions.metric was missing, so importing the re-export threw at load and broke every namespace downstream of metabase.metrics.core.
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Transform failure emails list root-cause failures by name and summarize dependents that never ran as a count, so the test asserts the tally instead of the dependent's name.
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
* Add Metric Dimensions section
* [metrics] Field IDs are not unique! Multiple FKs to same table.
Fixes a few places where the field ID alone is assumed to be a unique
key for dimension mappings.
* [metrics] Add basic CRUD API for metric v2 dimensions
Fixes UXW-4631.
Fixes UXW-4633.
* [metrics] Add `/api/metric/$ID/dimension/set-default` endpoint
Fixes UXW-4632.
This sets the dimension of a Metric v2 with the specified
`dimension_id` (a body parameter) to be the `:default true`.
All other dimensions have `:default` removed, so there is at
most one default dimension for any metric.
* [metrics] Metric dimensions create dependencies on the tables
Fixes UXW-4636.
* [metrics] Serdes support for curated Metrics v2 dimensions
Fixes UXW-4634.
Previously `:dimensions` and `:dimension_mappings` were marked as
`:skip` on serdes since the dimensions were always synced on read.
Now that they're curated and only synced when blank, we need to
properly serialize dimensions and mappings.
* Adjust Metric dimensions UI + connect to api
* Add dimensions reorder with dnd
* Add metric dimensions usage in metric viewer
* Change Metric Viewer to group dimensions by type
* Implement shared dimensions logic, add e2e tests
* Use default dimension in Metric Viewer
* Align headings in panels / Make trash and add buttons the same height
* Move and simplify the Set as default button
* Make about page use the default curated dimension in the preview chart
* [dimensions] Bump `:card_schema` to 24; backfill `:dimensions` in old style
Fixes UXW-4808.
This gives pre-existing v2 Metrics the full `breakoutable-columns` as
their dimensions, rather than just the main group as is now the
default. That preserves the old behaviour for the dimensions of the
metrics.
* Respect default metric dimensions on dashboards / Add projectDimension lib function (#77910)
* Respect default metric dimensions on dashboards
* Consolidate metric dashboard E2E coverage
* Remove default time dimension from metric editor
* Implement a few design tweaks in Dimensions tab UI
* Use curated metric dimensions on dashboard metric card filters
* Add dimension picker to metric About page
* Auto-pick default dimensions for new metrics
* Additional cleanup to the About page dimension select element
* Expand all metrics in See all dimensions
* Remove table names from metric filters
* Don't show sub-headers in 'See all' dimension view in Metric Viewer
* Remove metric overview tab
* [UXW-4859] Handle empty curated dimension list and removed default dimension edge cases
* Fix default viz type when adding metric to dashboard
* Remove the 'See all' option on explorer page for single metric visualization
* [UXW-4863] Don't use custom display name as field reference in dimension detail column
* [backend] Fix persistence of upgraded metric dimensions
* [UXW-4880] Re-add Overview tab, show curated dimensions
* [UXW-4848] Make sure we show all shared dimensions in the Shared Dimensions view, if the underlying fields match
* [UXW-4771] Make pinned metric in collection respect default curated dimension
* Update e2e tests to account for PR updates [part 1]
* Seed explicitly joined metric dimensions
* Fix metric dimension serialization dependencies
* Fix/adjust backend unit tests
* Fix metric dimension lint issues
* Run metrics with saved card permissions
* Update e2e tests to account for PR updates [part 2]
* Update kondo ratchet for metric serialization test
* Update metric permissions e2e expectation [part 3]
* Fix BE sandbox unit test
* Prefix upgraded metric dimensions with connected table names (#78410)
[UXW-4896] Prefix upgraded metric dimension names
* Address FE issues from July 22 reviews
* use curated dimensions on series breakout list / fix dimension display name on breakout legend and dimension pill bar
* Optimize getMetricMappingOptions
* Adjust about page dimension select margin-bottom and dropdown positining
* Fix curated metric dimension selection across duplicate foreign key paths
* Improve logic to show correct dimension label in the CenterControls button
* Exclude orphaned dimensions from runtime metric metadata
* Address Ethan's comments / Clean up
* Use metadata field cache when checking user permission / Improve field validation when adding dimension
* Address latest reviews
* Update serialization baseline card_schema 23 -> 24
* Tighten docstrings and comments per review feedback
Remove caller references, ticket refs, and usage context from docstrings
and comments; describe contracts instead.
---------
Co-authored-by: Braden Shepherdson <braden@metabase.com>
Co-authored-by: Anderson <anderson@metabase.com>
Co-authored-by: Anderson <ands.alves.nunes@gmail.com>
* Stabilize SSO group-mapping delete flake by re-reading settled state
Selecting groups for a mapping optimistically patches the settings cache,
but the setting update also invalidates and refetches session properties.
A trailing stale refetch can revert the last-added group after the test's
per-selection checkbox assertion has already passed, leaving the mapping
with only one non-admin group. The delete then fires a single deleteGroup
request and cy.wait times out waiting for the second.
Re-load the auth settings page between adding groups and deleting the
mapping, and assert the row shows its full group summary before acting.
The fresh page load aborts in-flight refetches and reads a single settled
server state, so every mapped group is present when the delete fires.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* Serialize SSO group-mapping edits against their setting round-trips
Adding a group to a mapping PUTs the group-mappings setting (built from the
widget's current mappings cache) and invalidates session properties, which
triggers a background refetch. The widget optimistically patches the cache,
but a trailing stale refetch can revert the just-added group; the next
selection then builds its PUT from that stale set and overwrites the previous
group server-side (last-write-wins). The mapping ends up with fewer groups
than the test expects, so fewer deleteGroup requests fire and cy.wait times
out waiting for the second one.
Wait for each mapping mutation's PUT and its trailing session-properties
refetch to settle before the next interaction, in createMapping,
addGroupsToMapping (per group), and deleteMappingWithGroups, so every edit is
built from a fully committed round-trip. deleteGroup/clearGroup invalidate
only permissions-group, so updateSetting is the sole source of session
refetches and the waits stay aligned one-to-one.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* update getExtraTagsForVersion to append -lts if isLts is set
* add ability to check if a version is an LTS version
* set islts
* update function to determine if we should tag with lts
* add condition for multiple LTS releases so we only tag the latest
* fix actionlint errors
* update lts tag to include major version
* await result of getExtraTagsForVersion
* fix type
* fix merge conflict, don't throw error
* GDGT-2851 Remove color-mix from theme color definitions
color-mix can only be resolved by a browser, so any theme color using it
breaks static viz, which renders on the server (see UXW-4690, where the
gauge failed on background_page-tertiary-inverse).
Per design's proposal, introduce a new orion-95 stop and use solid scale
values for the page backgrounds:
- dark: page primary/secondary/tertiary -> orion 95/100/110
- light: page primary/secondary/tertiary inverse -> orion 95/100/110
The remaining mixes are collapsed to their literal equivalents:
shadow-default to a plain rgba, and tooltip-background-focused to the
next stop down from tooltip-background.
The mixes that derive from whitelabel colors (core-brand, core-filter,
core-summarize) are left alone, since their values aren't known ahead of
time. A unit test now guards against reintroducing color-mix anywhere
else in the theme definitions.
* Update Loki Snapshots
* Remove var() references and non-hsla literals from theme definitions
Static viz feeds theme colors straight into Color(), which only parses
literals, so a var() throws there exactly like a color-mix does — even
when it points at a perfectly ordinary color. Collapse the remaining
var() references to the underlying base color, and normalize the few
hex and bare-keyword values to hsla so every theme color is a literal.
The guard test drops the transitive resolver it no longer needs and just
asserts that every non-whitelabel color is an hsla literal.
* Fix regression in pivot table
* Empty commit to trigger CI
* Manually update pivot table loki snapshot
* Update e2e colour assertions for the new background tokens
Dark background_page-primary moved from orion-100 to the new orion-95,
so anything asserting rgb(7, 23, 34) for it now expects rgb(12, 28, 39).
The dark mode body assertion was matching color(srgb 0.0204 0.06792
0.0996) — Chrome's computed form for the old color-mix on
background_page-secondary. That token is a plain hsla now, so Chrome
reports it as rgb(7, 23, 34).
* Fix e2e
* Fix e2e
* Revert changes in assertDarkMode
* Fix oss e2e spec
* Revert SDK theming assertions to the deprecated background token
The SDK theme presets resolve colours through the first entry of
SDK_TO_MAIN_APP_COLORS_MAPPING, and for `background` that is the
deprecated `background-primary`, which still points at orion-100.
`background_page-primary` sits at index 1 and never gets read, so
table-root is unaffected by the move to orion-95.
* Address review feedback on the colour theme changes
Move the gold and upsell-gem literals into base-colors, since each was
repeated across both theme files. Drop the intermediate staticColors
binding in the guard test in favour of a second filter, and drop the
`?? {}` on theme.colors, which is a required field.
---------
Co-authored-by: Metabase Automation <github-automation@metabase.com>
* Fix click-behavior multi-stage e2e flake: settle mapping list between selections
The "navigating to questions with filters applied in every stage" test sets 11
click-target mappings sequentially, using index-based selectors (.first(),
.last(), .eq(2)) against the unset-click-mappings list. That list re-renders as
each mapping moves from the unset to the set section, but there was no barrier
between steps, so an index-based lookup could resolve mid-re-render and land on
the wrong target — yielding a stable-but-wrong filter set (a duplicated
"Product → Title" filter replacing the dropped "Reviews - Product → Reviewer"
one, which is exactly the assertion that fails).
Extract source selection into selectClickMappingSource(), which asserts the
source picker popover has closed before returning, so every subsequent
index-based lookup runs against a settled list.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* Fix click-mapping settle barrier to assert popover closed correctly
The previous settle barrier used `H.popover().should("not.exist")`, but the
`H.popover()` helper internally chains `.filter(":visible").should("be.visible")`,
so the assertion is self-contradictory: it requires a visible popover to exist
and then asserts it does not, failing deterministically the moment the source
selection closes the popover.
Assert the popover is gone with `cy.get(H.POPOVER_ELEMENT).should("not.exist")`
— the idiomatic pattern used across the e2e suite — so the mapping list still
settles between index-based selections without the contradictory visibility check.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* Scope click-mapping settle barrier to the visible popover
The unset-click-mappings list re-renders as each mapping moves unset->set.
A bare `cy.get(POPOVER_ELEMENT).should("not.exist")` can never settle in the
click-behavior sidebar: it keeps ~60 mantine popover/combobox dropdowns
mounted-but-hidden, so they are continuously found in the DOM. Filter to
`:visible` so the barrier only waits for the open source picker to close.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* Export queryToSearch and searchToQuery from metabase/router
* Serialize the query with queryToSearch at the navigation call sites
* Read search, not query, off a navigation descriptor
* Drop query from LocationDescriptorObject
* Thread QB popState off the routing slice
* Thread dashboard edit-mode off the routing slice
* Thread Metabot profile selection off the routing slice
* Thread document Metabot context off the routing slice
* Migrate component location readers to useLocation
* Retire the state.routing slice
* Convert getExpandedCollectionsById to TypeScript and fix the stale ExpandedCollection type
* Type fake collection ids as literals instead of casting to CollectionId, dropping the String() workaround
* Audit log statements: no stack traces, no queries/user input, no pretty-printing
- Replace throwable args to log macros with (ex-message e) so ex-data and
stack traces never reach the logs (~450 sites)
- Remove SQL/MBQL queries, prepared-statement params, HTTP request/response
bodies, PII, and user-authored names from log output (IDs kept)
- Replace u/pprint-to-str in log calls with pr-str, or drop the value
- Silence sqlglot parser warnings in the GraalPy bootstrap (they quote raw
chunks of customer SQL to stderr)
- Rewrite leftover functional colors (hsl/hsla/rgb/rgba) to hex in static-viz
SVG output so Batik never errors (and stack-dumps) on them
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* Use logf variants for exception messages; restore serdes --full-stacktrace
- Convert print-style `(log/warn "msg:" (ex-message e))` calls to
`(log/warnf "msg: %s" (ex-message e))`
- Serdes import/export logs the full stack trace again when the
--full-stacktrace CLI option / full_stacktrace API param is passed
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* Update swap-clauses tests for clause-free warning messages
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* Update log-content test assertions for the new logging policy
Log calls no longer attach throwables (entries have :e nil) and append
ex-message text to the message instead; body/secret leak tests now assert
the payload never appears in log output.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* Address review comments
- rethrow-api-error!: update docstrings — response body is never logged,
it travels on the thrown ex-data
- expand-macros: log the segment filter-clause count once instead of an
identical trace line per clause
- settings cache: log the whole SQLException message chain (the old
print-sql-exception-chain walked .getNextException; ex-message alone
dropped the chained causes)
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Add Claude Opus 5 to the anthropic openrouter and bedrock whitelists
- anthropic claude-opus-5
- openrouter anthropic/claude-opus-5
- bedrock anthropic.claude-opus-5
* Scope the invite group picker to the groups with access to the shared item
* Imply All Users membership only for invites with a target, and mock the scoped groups endpoint in share menu specs
* Show all groups when inviting to view and mark which ones can access the item
* Make the invite picker's group section labels visible and tidy up UserForm's membership mapping
* Generalize the group picker into ordered labeled sections with group a11y roles, keeping invite-target semantics in UserForm
* Use the compact Alert size for the invite picker's no-access warning
Suppress api-key-missing exception when failing to generate sample question.
Not having an API key is the default situation and should not result
in an error being logged.
* Let library retrieval use pgvector on the app db
The tables were bare-named, so the drop-and-rebuild in ensure-tables! would have
resolved against the application database's default schema. That is what kept
the feature dedicated-only.
Table names now come from the pgvector mode, the way semantic search picks
between its two index-metadata configs: bare for a dedicated store, so no
existing index moves, and qualified by a library_retrieval schema when sharing
the app db. Its own schema, not semantic search's, whose migration drops every
table in it.
Raw SQL interpolating a qualified name into "%s" would read it as one
identifier, so those sites go through quote-table.
The sync task's scheduling gate stays boot-safe: a dedicated URL, or the license
plus a Postgres app db, decided without resolving the mode. available? checks
the license before the store for the same reason.
* Cover the app-db round trip end to end
Opt-in like the semantic-search one, and picked up by the same appdb-mode CI
job. Asserts the tables land in library_retrieval, the application schema is
untouched, retrieval returns the indexed entity, and a semantic-search wipe
leaves the library index alone -- the reason the schema is separate.
* Check our own schema before reporting the store ready
Semantic search's predicate answers for semantic_search. A role that can use
that schema but cannot create library_retrieval passed it, and then failed every
reconcile inside ensure-tables!. Probe for our schema too: present, or creatable
in a rolled-back transaction, cached because available? runs on every write
nudge and search.
* Gate the round trip on the capability the feature checks
Gating on semantic search's check let the harness pass where a role that can use
semantic_search but not create library_retrieval fails in production.
* Check schema privileges, not schema existence
A role with USAGE but not CREATE passed the existence check and then failed
creating tables. Ask Postgres for both privileges instead, and only fall back to
the rolled-back CREATE SCHEMA when the schema isn't there at all.
The whole probe is inside the try now: the existence query wasn't, so a
transient database failure could throw out of request-entity-sync!, which the
write hooks call fire-and-forget.
* Put the dedicated index in the schema too
A dedicated store kept the index in the default schema, which is exactly what
semantic search's migration wipes -- so enabling or migrating semantic search
dropped the library index and silently re-embedded the whole library. It fires
whenever the stored schema version trails the code, not just on first init.
Both modes now use the library_retrieval schema, so default-tables and
app-db-tables collapse into one. An index built before the schema existed is
moved into it with ALTER TABLE ... SET SCHEMA, keeping its rows; the move runs
before the IF NOT EXISTS creates, which would otherwise leave an empty table
beside it. Creating the schema now needs CREATE on the dedicated database, so a
failure says so.
* Drop only semantic search's own tables in a dedicated store
A dedicated store's default schema is shared -- library retrieval keeps its
index there until it moves -- so wiping every table in it could drop that index
before entity retrieval had a chance to adopt it. Both run as background jobs,
so nothing ordered them.
Scoped to the index_/dlq_/repair_ prefixes, and only in dedicated mode: the
module's own schema in app-db mode holds nothing else, so a reset there is still
free to clear leftovers. The app-db sentinel refusal still reads every table in
scope, since an app db is recognized by tables we would never drop.
* Match the names semantic search generates, not their prefixes
A cohabitant in a shared dedicated store is free to call something
index_history, which the prefix check would have dropped. Match the control
tables exactly, index tables through index-table-name? -- the same contract
orphan cleanup uses -- and DLQ and repair tables by their generated shapes.
Also covers the adoption path end to end: a legacy unqualified index with a row
in it, moved by ensure-tables! and still holding the row afterwards. Asserted
there rather than after the reconcile, which GCs documents the library no longer
implies.
* Stop the availability tests depending on the app db
available?-gates-test left the real schema probe live while stubbing only
the pgvector mode. The probe is Postgres-only SQL, so it threw and read as
unusable on the H2 and MySQL runs, and the case failed there.
The round trip's legacy table declares a vector(4) column before
ensure-tables! runs, and the support check verifies CREATE EXTENSION in a
transaction it rolls back, so nothing has installed the type by then.
* Answer review on the dedicated-store reset
Rename ours? to semantic-search-table?, and point the two docstring
references at namespaces the reader can resolve.
Cover the case where a legacy bare table and a qualified one both exist:
adoption has to leave them alone, since SET SCHEMA onto an occupied name
errors and would abort the whole ensure-tables! transaction.
* Latch the app-db schema check once it passes
The check catches everything and returns false, so on a plain TTL a single
transient app-db failure took retrieval offline for five minutes: available?
went false, search returned nothing and the write nudge no-oped.
A confirmed yes now sticks for the life of the process. Only a no keeps
re-checking, so a privilege granted while we run is still picked up.
* Stub the evals trigger
* use ref arg
* dummy to trigger build
* Clean up
* Use appropriate token
* Simplify pr message
* Use .evals_ref for default ref
* Comment cleanup
* Adjust ref
* Use evals branch
* Use general automation token
* readme for .evals_branch
* Use main of evals for label trigger
* Revert "dummy to trigger build"
This reverts commit e6502c6ff076a47d94eb829b516fc45b37cd7680.
* whitespace
* Address review remarks
* Use ai benchmark naming so we can trigger from pr
* Address review comments
* Remove now redundant branch pinning
* Review remarks
* feat: add lookback
* validate lookback
* feat: apply lookback
* tests
* fix
* fix test
* feat: only support lookback on temporal checkpoints
* update test
* FE
wip
* add :type/Time test
* add test with watermark row removed
* refactor to avoid double get-in
* only support date/datetime
* tests
* fix: support coerced cols
* add more tests
* refactor
* more tests
* test for coerced instant
* use utils in FE too
* chorec
* further validate
* use lib utils
* use strict comparison
* allow clearing lookback
* handle race
* chore: migrate registerVisualization to typescript
* chore: drop 'as unknown as Visualization' cast in custom viz plugin loading
applyDefaultVisualizationProps now takes the plain component and returns
the typed Visualization via Object.assign. The SDK's date_style format
option is widened to string | null to match the ColumnSettings value the
host actually passes to checkRenderable.
* Unit test the native editor run button's unmapped-tag tooltip
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* Unit test the Save button's unmapped-tag tooltip and disabled state
Covers both dimension and temporal-unit template tags.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* Cover temporal-unit tags in the canRun template tag validation test
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* Drop the unmapped field filter e2e test in favour of unit coverage
Every assertion it made is now covered by NativeQueryEditorRunButton and
ViewTitleHeader unit tests.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
* Read useLocation and useParams straight from react-router v7
* Re-export react-router's Outlet instead of shimming it through context
* Disable the command palette by path instead of route props
* Drop the now-unused route props plumbing
* Add mdb/with-unshared-connection: appdb connections exempt from ambient reuse
A connection holding fragile long-lived state -- a streaming result set
(postgres portal/cursor), or FOR UPDATE row locks -- must not be picked
up by ambient connection reuse: a borrower that inherits it via
*current-connectable* and commits destroys that state. This is the
failure mode behind the first-boot hang in #78238, the revert (#77048)
of appdb streaming (#76645), and "cursor/portal does not exist" errors
when field updates run mid-sync-stream.
Built on toucan2's new unshared-connection! (camsaul/toucan2#212): the
connection is used when passed explicitly -- (t2/reducible-query conn
...), (t2/with-transaction [t conn] ...) -- but never bound as
*current-connectable*, so toucan calls that resolve their connection
ambiently run (and commit) on other pooled connections.
mdb/with-unshared-connection is checkout -> mark -> body -> close; in
every other respect the connection is ordinary JDBC, configured by the
body (e.g. .setAutoCommit false to stream). The pg test reproduces the
exact #77048 failure on an unmarked control connection (portal killed by
a nested commit mid-reduction) and shows the unshared connection
streaming to completion.
deps.edn temporarily pins toucan2 to the PR sha via :git/sha; a
follow-up commit will switch back to :mvn/version once a toucan2
release includes it.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01G7FfdpzW5pr7CX9aNwa98f
* Use released toucan2 1.0.572 for unshared-connection support
camsaul/toucan2#212 merged and released; drop the :git/sha pin.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01G7FfdpzW5pr7CX9aNwa98f
* Hold the audit DB cluster lock on a detached connection (fixes boot stall)
Since #76551 the audit install/load/sync pipeline runs inside a cluster
lock whose SELECT FOR UPDATE ties lock lifetime to transaction lifetime:
the whole pipeline became one long transaction on the lock's connection.
The serdes load's ~3,600 per-entity nested transactions piled up as
savepoints and subtransaction XIDs, and every row lock was held until a
single commit at the end -- stalling first boot for 10-25+ minutes and
blocking concurrent work like config-file database provisioning (#78238).
with-cluster-lock now supports :detached? -- the lock rows are held on a
dedicated unshared connection (toucan2 unshared-connection!) while the
body runs on ordinary pooled connections, committing incrementally in
its own short transactions. Locks release at body completion (commit) or
throw (pool check-in rollback). Re-acquiring a lock this thread already
holds detached is a no-op in both detached and transactional form;
upgrading a detached :share hold to :exclusive throws instead of
self-deadlocking; :retry-transient? is rejected (a re-run would
double-apply incrementally-committed work).
The audit pipeline opts in: per-entity serdes isolation (#74412) is
restored, and a crash mid-pipeline is repaired by the next boot (the
checksum only advances on completion; reconcile runs every boot), which
is what makes incremental commit safe here.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01G7FfdpzW5pr7CX9aNwa98f
* Address review: confine detached retry to acquisition; repair dialect-sync window
Review findings (galdre), all majors addressed:
- Major 1: retry/with-retry wrapped the whole detached body, so a
retryable-looking error thrown by the body (e.g. a nested cluster lock
timing out inside the serdes load) re-ran an incrementally-committed
body and then surfaced as "Failed to obtain cluster lock", which
ensure-audit-db-installed! swallows as benign contention. An acquired?
volatile now confines both the retry predicate and the acquisition-
failure wrapper to the pre-acquisition phase in detached mode; body
errors propagate raw with intact cause chains.
- Major 2: a failed or killed engine-changed dialect sync was never
repaired once the content checksum had advanced. New durable setting
audit-db-dialect-sync-pending: set before the sync attempt, cleared on
success, a third sync trigger on every boot.
- Major 3: the mysql/h2 engine-flip visibility window under detached
mode is documented as a KNOWN WINDOW at adjust-audit-db-to-source!
(restores pre-#76551 semantics; overlay-based resolution is follow-up).
- Majors 4-6: reviewer's contract tests adopted (throw-path lock
release, check-in rollback, behavioral pin on :detached?, crash-mid-
load self-heal, plus the two Major-1 reproductions); public docstrings
updated to describe all three lock modes and the upgrade throw.
Minors: unified cross-mode reentrancy registry with eager share->
exclusive upgrade validation (fail-fast in any detached interleaving;
legal same-transaction upgrade preserved); h2 read->write upgrade throws
instead of hanging; lock-row INSERT gets a query timeout (bounds the
N-node cold-start pile-up); install-database! and each reconcile group
are single transactions; unshared checkout goes through *application-db*
so the restore-time read-lock gate applies; registry conveyance and
explicit-tx-on-unshared-connection caveats documented; dual kondo
lint-as entries; test hygiene (finally cleanup, nil ambient contract).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01G7FfdpzW5pr7CX9aNwa98f
* Right-size the engine-flip visibility comment
The committed, briefly-visible engine flip is this pipeline's
longstanding behavior; only #76551's single-transaction era hid it for
one release. A neutral note fits better than a warning block.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01G7FfdpzW5pr7CX9aNwa98f
* Tighten comments to constraints; drop review-context narration
Comments now state what the code can't show (crash windows, why raw
JDBC, why the marker precedes the sync) and drop justification written
for reviewers: issue-number history, caller name-drops, repeated
tellings of the retry-confinement story, and cost reassurances.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01G7FfdpzW5pr7CX9aNwa98f
* Make the detached lock a bespoke entry point instead of a with-cluster-lock mode
Reviewer pressure on the :detached? option had generalized
with-cluster-lock into a two-mode API whose cross-mode interleavings all
needed guarding: a held-locks registry, mode-upgrade validation, an h2
upgrade guard, and an acquired? volatile threading retry confinement
through the shared machinery — all protecting call patterns nothing in
the tree performs.
with-detached-cluster-lock is now its own function: exclusive-only,
single lock, documented non-reentrant. Structure replaces machinery —
the body sits outside the acquisition retry, so body errors are never
retried nor rewrapped as acquisition failure (no volatile), and
same-name re-entry throws via a two-line dynamic-set guard instead of a
registry (nested *different* locks, which the audit body does take,
work unchanged). with-cluster-lock returns to its previous shape; the
lock-row INSERT keeps its new query timeout, which bounds the
multi-node cold-start pile-up in both modes. Error shape is unchanged,
so audit-lock-contention? still classifies correctly.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01G7FfdpzW5pr7CX9aNwa98f
---------
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
* Compact in-memory format for the per-request data-permissions cache
The request-scoped cache in metabase.permissions.models.data-permissions
previously stored raw data_permissions rows as full Toucan instances, keyed
{user-id {perm-type {db-id [row ...]}}}. On instances with thousands of
databases and users in many groups a single prime-db-cache call could
materialize over a million instances (hundreds of MB retained per request,
plus the matching GC churn).
Store a compact entry per (perm-type, db-id) instead:
{:groups {group-id perm-value} ; db-level rows
:tables {table-id [schema-name {group-id perm-value}]}} ; table-level rows
and intern the group->value maps at load time: on big instances they are
typically identical across most databases, so the retained size becomes
proportional to the number of distinct permission profiles instead of
groups * databases.
Also fetch only the columns the checks actually consume (group_id, perm_type,
db_id, table_id, schema_name, perm_value) instead of p.*, and fix an O(n^2)
already-cached filter in prime-db-cache.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* small fix
* Use metabase.util.performance helpers in the cache hot path
perf/concat + perf/mapv in entry-group-value-pairs. Deliberately NOT
perf/update-vals for the interning pass: it skips replacing values that
are = to the old ones, which would undo the interning.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* Drop perf helpers from entry-group-value-pairs
perf/concat over an already-built vector is just (into a b), and reduce-kv
iterates the map faster than mapv over entry seqs. Plain into + reduce-kv,
one less require.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* Address code-review findings on the compact permissions cache
- Duplicate data_permissions rows (no unique constraint on
group/type/db/table) no longer silently last-win: cache slots promote to
a set when duplicate rows disagree, so every value reaches the consumers'
coalescing logic exactly as raw rows did. Regression test added.
- schema_name is stored per table-level row ({:schema .. :value ..})
instead of first-row-wins per table, since it is denormalized and rows
for one table can disagree.
- The interner now lives in the *permissions-for-user* atom, so value
sharing spans every prime in the request (db-at-a-time priming
previously got a fresh pool per call), and it reuses
metabase.util.memoize/fast-interner instead of a bespoke pool.
- rows->cache-entry returns nil for empty rows so cached and non-cached
get-permissions branches agree; the non-cached branch skips interning.
- schema-permission-for-user and most-permissive-database-permission-for-user
now go through entry-group-value-pairs instead of hand-rolled traversals.
- Dropped the impossible-nil guards in prime-db-cache.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* fix
* Address CodeRabbit nitpicks
- gid->slot -> group-id->slot (no abbreviations)
- native-download-permission-for-user reuses most-restrictive-per-group
instead of re-implementing the grouping inline
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* Rename local intern -> interner (Eastwood local-shadows-var)
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* Explain the slot rationale in conj-slot/slot-seq docstrings
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* simplify slots
* Document interning with a worked example in the cache docstrings
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014V8QbHjoEkJ8eZLG13sYUe
---------
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Co-authored-by: dan sutton <dan@dpsutton.com>
* Restyle the sharing popover with Copy link and Embed buttons above the actions list
* Reserve the copy button width with hidden pseudo-elements instead of inline styles
* swap subscriptions action with auto-refresh action
* hide Subscriptions for non-admins if slack and email aren't set up
* small adjustments
* e2e adjustments
* more e2e adjustments
---------
Co-authored-by: Nick Fitzpatrick <nickfitz.582@gmail.com>
* Convert the legacy chart legend components to TypeScript function components
* Share the legend prop types and extract a typed getChartLayout helper with tests
Deleting a Database hard-deletes its Cards with raw SQL, but nothing
rewrites the `parameters` JSON on Cards/Dashboards that used one of
those Cards as a dropdown source. The parameter keeps a
`values_source_config.card_id` pointing at a row that is gone.
On export that made `*export-fk*` throw `:target-not-found`, which
`log-and-extract-one` swallows as a skip — silently dropping the whole
Card or Dashboard from the archive.
Export each parameter on its own now, and when one carries a card
values-source that can no longer be resolved, emit it without
`:values_source_type`/`:values_source_config`. That is the same shape
the app writes when a source Card is archived, so the parameter comes
back as a plain dropdown over its connected fields.
A source Card that merely isn't part of the export is unaffected —
`*export-fk*` only throws when the row is absent, so those still fail
dependency validation as before.
Fixes GHY-3259
The helper inserts straight into the table to bypass the model's transforms,
which bypasses the hook that fills in entity_id too. #78452 made that column NOT
NULL, so every App DB Tests job has failed on it since.
* Eagerly load tenant auth-provider; fail tenant SSO logins loudly (UXW-4898)
Root cause: metabase-enterprise.tenants.auth-provider — the login! method
that validates a login's tenant claim and stamps :tenant_id onto the user
data — was never eagerly loaded. The JWT/SAML providers only `derive` from
its dispatch keyword, which does not load the namespace, so on a freshly
started instance the tenant step was silently absent from the login!
method chain until unrelated traffic (e.g. /api/user/current hydrating
user attributes) lazily loaded it. A JWT provisioning login in that window
created the user with tenant_id = nil even though the token carried a
valid @tenant, the tenant-group assignment failed and was swallowed, and
the login returned 200 — permanently wedging the account (every later
login 403s with "Cannot add tenant claim to internal user").
Fix, plus defense in depth so no future tenant-drop can wedge an account:
- Add metabase-enterprise.tenants.init requiring the auth-provider, wired
into metabase-enterprise.core.init, so the tenant login step is always
registered at startup.
- create-user! throws instead of silently dropping :tenant_id when the
sso-user-fields field list would strip a tenant assignment stamped by
the upstream tenant flow.
- sync-group-memberships*! no longer swallows real tenant/group mismatch
errors (bad pairs whose :group-is-tenant? is non-nil); the login fails
loudly instead. Nonexistent groups keep log-and-continue. This also
covers EMB-2118 (JWT omitting @tenant with a mapped tenant group).
- The login! provisioning chain (tenant creation, user create/update,
group sync) runs in one transaction, so a failed login leaves no
half-provisioned user row behind.
* Fix cljfmt formatting in jwt_test.clj
* Stop the location-syncing call sites from replacing the url they just read
* Cover the documents /new to /new warning with a unit test
* Adopt v7 navigation semantics, drop the v3 parity guards
The v7 provider mirrored every location into state.routing by calling
useDispatch itself, which is what kept metabase/router bound to the store.
createLocationMirror takes a plain dispatch function instead, so the router
publishes locations without knowing redux exists. app.js and the test harness
supply it as RouterProvider's onLocationChange prop. V7ReduxBridge is renamed
V7RouterBridge now that nothing in it is redux-specific.
metabase/router now passes a basic-tier boundaries trial with zero violations.
Guards are authorization policy, not routing primitives. They were the largest
source of shared-tier imports inside metabase/router (redux, plugins, and four
selector modules), blocking the module from dropping to a lower tier.
The generic factory and the app-wide auth/setup guards move to a new shared
metabase/route-guards module. The monitor and data-studio guards move into their
own features, which lets getMonitorRoutes and getDataStudioRoutes stop taking
them as parameters.
Router violations under a basic-tier trial go from 13 to 3; the rest are the
redux location mirror in the v7 provider.
* Add GeoJSON error mocks and cover map load failures in unit tests
`setupGeoJSONEndpoint` could only produce a valid FeatureCollection, so the
error branches of the custom map flow had no unit coverage.
- `setupMalformedGeoJSONEndpoint` returns a 200 with a payload the client-side
validator in `metabase/api/geojson` rejects, since those checks run in
`transformResponse` and need the payload to reach the client.
- `setupGeoJSONEndpointWithError` returns a non-2xx with a bare string body,
matching what `api-exception-response` sends for an `ex-info` carrying only
a status code.
- `createMockProjectedGeoJSONFeatureCollection` supplies projected coordinates
so every bounds corner falls outside lat/lng range.
Covers the invalid-location, missing-features, and out-of-bounds errors in
CustomGeoJSONWidget.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* Drop map GeoJSON error e2e tests in favour of unit coverage
All three asserted error strings are now covered more cheaply. The two
backend-owned messages are pinned by metabase.geojson.api-test against mocked
HTTP, and the modal rendering them is covered by CustomGeoJSONWidget. The two
client-side validation messages come from transformResponse in
metabase/api/geojson, which the e2e tests reached with the backend fully
stubbed or bypassed.
Two of them also drove real outbound requests from CI — one to www.metabase.com
and one to a file on the public repo's master branch.
projected.geojson had no other consumer; the equivalent payload is now built by
createMockProjectedGeoJSONFeatureCollection.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* Drop admin settings e2e tests that duplicate existing unit coverage
Each of these asserts the same condition as a unit test that already exists,
not merely an adjacent one:
- the HTTPS redirect check failure message is asserted in HttpsOnlyWidget
("should display an error message if the site is not accessible via https")
- the billing info fetch failure is asserted in LicenseAndBillingSettings
("renders error for billing info for store managed billing and info request
fails")
- the absent license input for hosted instances is asserted in
LicenseAndBillingSettings ("does not render an input when token has hosting
feature enabled")
mockBillingTokenFeatures and its feature-key constants had no remaining
callers.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* Persist disabling a collection from remote sync (GHY-4189)
Disabling a collection deleted its RemoteSyncObject rows outright, so the
removal left no dirty state to push and the collection's files stayed on the
remote. A later pull then re-imported the collection (its YAML still carried
is_remote_synced: true), re-enabling it.
Mark the collection's and its contents' RemoteSyncObject rows as "removed"
instead — a pending deletion the next export pushes to the remote — while
dropping rows still in "create" (never pushed, so nothing to delete remotely).
* Don't delete contents when a collection is re-synced before the push
Disabling a collection records a pending removal on its contents'
RemoteSyncObject rows. Re-enabling it re-tracked only the collection's own row,
so the contents stayed marked for deletion and the next export removed their
files from the remote.
Clear the recorded removal when a collection is re-synced before that removal
was pushed. Restore to "update" rather than "synced": edits made while the
collection was un-synced are not tracked, so the entity must be re-serialized
for the remote to be guaranteed to match local.
* Stop core-test leaking RemoteSyncObject rows between tests
Disabling a collection now marks its RemoteSyncObject rows rather than deleting
them, so rows seeded by a test outlive it and leak dirty state into every later
test. The old delete-on-disable behavior had been doing this cleanup by
accident.
Reset the table around each test with the clean-object fixture.
* Assert the pull succeeds in the disable round-trip test
The test discarded impl/import!'s result, so an errored pull would leave the
collection un-synced for the wrong reason and the following assertion would
pass vacuously.
* Re-check eligibility at write time so a pending removal isn't resurrected
An event handler computes eligibility from its event payload, then writes the
RemoteSyncObject row much later. A disable committing in between left the
handler writing a tracked status over the "removed" status the disable had
recorded, so the entity stayed on the remote after being un-synced.
Re-check eligibility against current DB state before letting a tracked status
overwrite a pending removal, and let the removal stand when it no longer holds.
* Row-lock the sync entry so the un-sync race has no remaining window
Re-checking eligibility at write time narrowed the race but did not close it: a
disable could still commit between the re-check and the update, and the losing
write would take effect.
Row-lock the entry for the transaction so a tracking write and a concurrent
un-sync settle in a fixed order instead of losing one of the two: whichever
locks first commits, and the other observes that result — the un-sync re-marking
the row, or the eligibility re-check seeing the entity leave the synced set.
The race test drives the disable on its own thread and unparks the handler
without waiting for it, since the parked handler holds the row lock the disable
needs. Its assertion holds for either order, so the choreography only biases
which one is exercised.
* Address review: honest race coverage, insert-path guard, tighter asserts
- Split the concurrency test into two, each parking the handler at the point
that exercises one ordering: disable-first (guarded by the write-time
eligibility re-check) and handler-first (guarded by the row lock, which the
disable now provably blocks on). The prior single test parked on the plain
existing-entry read, so it only ever exercised the re-check ordering while its
comment claimed lock contention.
- Re-check eligibility in the insert branch too: a stale tracked-status event
that finds no row inserts a fresh one, a phantom the row lock cannot cover.
This narrows, but cannot fully close, that window.
- Round-trip pull test asserts = :success rather than not= :error, so a
:conflict result can no longer slip through.
- restore-removed-rsos! docstring no longer claims it only clears removals from
record-removed-rsos!; it clears any 'removed' row in the subtree.
* Re-track never-pushed contents when a collection is re-synced
Disabling a collection drops its contents' never-pushed 'create' RemoteSyncObject
rows (the remote never had them, so nothing to delete there). Re-enabling only
restored the rows that were marked 'removed', so a card added-but-never-pushed
before the disable was left with no row — and the next incremental export, seeing
no dirty row for it, silently omitted it.
On re-enable, insert a 'create' row for every eligible content entity in the
re-enabled subtree that has none, so the next export pushes it. The
RemoteSyncObject table only records pending changes, so already-synced content is
legitimately untracked; re-marking it 'create' re-serializes it harmlessly (a
'create' onto its own path is a no-op).
* scaffold: two-step SDK release process (EMB-1518)
Replaces the hardcoded version/dist-tag computation in
release-embedding-sdk.yml with a new sdkRelease metadata block in
package.template.json (distTag, tagAsLatest), plus a new
sdk-version-bump-pr.yml workflow that bumps the version via `npm version`
and opens a PR labeled sdk-release-trigger for review. Merging that PR
triggers the release workflow directly.
Unverified: scaffolded per EMB-1518 ahead of Nicolo/Roman's review, since
this can only really be exercised by triggering a real SDK release. See
.claude/kelvin/2026-07-09-emb-1518-simplify-sdk-release-process-and-checklist-maintenance/
for the plan and open questions (particularly around distTag semantics for
stable/patch/minor releases).
* fix: correct distTag comment, <major>-stable is real (EMB-1518)
npm dist-tag ls @metabase/embedding-sdk-react confirms 51-stable
through 62-stable are already live, actively-used tags - not a new
concept as the prior comment assumed. Resolves the one open question
this scaffold couldn't settle from the proposal text alone.
* fix: address review cycle 1 findings (EMB-1518)
Fixes release-breaking and correctness issues found in review cycle 1
of the SDK release process scaffold:
- beta release_type now defaults to <major>-beta dist-tag instead of
the literal (non-existent) "beta" tag
- validate branch/release_type combinations that the old
getNextSdkVersion used to hard-error on (master requires alpha,
release branches reject alpha)
- pull_request trigger on release-embedding-sdk.yml now scoped to
package.template.json via a paths filter, and the concurrency group
no longer cancels an in-flight release on unrelated PR closures
- readme-PR follow-up steps now skip cleanly when the readme was
already up to date, instead of failing and firing a false alert
- tagAsLatest now always resets to false rather than carrying forward
a prior true, and the actual value is shown in the summary/PR body
- version-bump branch name includes the run id to avoid collisions on
retry
- renamed a step whose name didn't match what it does
See the "Review cycle 1 (fable)" section and the new "Fixes applied
for review cycle 1" subsection in the plan file for full detail on
each finding and its disposition.
* fix: address review cycle 2 findings + simplify version bump (EMB-1518)
Cycle-2 findings: add a preminor release_type so master's next-major
alpha bump no longer needs a hand-made PR (#9); make the Slack
failure notification independent of an artifact upload that may
never have run (#10); guard against two open version-bump PRs racing
each other (#11); stop interpolating the free-text branch input
directly into shell scripts (#12).
Also replaces the npm-version rename dance (rename to package.json,
bump, rename back, plus a .bak copy) with direct calls to the semver
package the bump logic already wraps, writing the result via jq
instead - no functional change, removes the dead .bak file (#13).
* fix: address review cycle 3 findings + rename resolve-release job (EMB-1518)
Fixes findings #14-#16 from the third fable review cycle:
- #14: the failure Slack alert's messageId interpolation was unquoted,
rendering a JS syntax error whenever send-slack-notification-start was
skipped (the exact scenario cycle-2's fix#10 was meant to cover). Now
quoted, and falls back to sendSlackMessage (new top-level message)
instead of sendSlackReply when there's no thread to reply to.
- #15: finished the cycle-2 injection-hardening pass by routing the two
remaining inline branch-value usages (workflow-summary's heredoc,
publish-npm's gh pr create --base) through env: instead of ${{ }}.
- #16: added a semver validity pre-check before the bump step, so an old
release branch's 0.N.x placeholder version fails with a clear ::error::
instead of a silent exit 1 from the semver CLI.
#17 (minor's version-collision caveat on old release branches) is left as
an open question for Nicolo/Roman rather than removed - documented with a
comment and Judgment call #6. #18 (merge-commit-sha race) is deferred -
it fails loudly rather than silently, lower priority than the others.
Also renames the resolve-release job to prepare-release throughout
release-embedding-sdk.yml, per explicit request (not a review finding).
See the plan file's "Fixes applied for review cycle 3" section for the
full verification detail.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01U2Va4ryWuZbQvmUt2N85ec
* feat: auto-compute tagAsLatest from vars.LATEST_VERSION (EMB-1518)
Resolves Judgment call #2: the SDK version-bump workflow previously
always defaulted tagAsLatest to false because there was no reliable
way to know if a branch is the current "latest" major. The repo
already exposes vars.LATEST_VERSION (major version of the most recent
gold/stable release) for exactly this, so patch/minor bumps now set
tagAsLatest=true when MAJOR_VERSION matches it, false otherwise
(including when the variable is unset). alpha/beta/preminor stay
false. The PR review gate still allows overriding the computed value
before merging.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01U2Va4ryWuZbQvmUt2N85ec
* refactor: use github.ref_name instead of inputs.branch (EMB-1518)
github.ref_name is already set by GitHub's "Use workflow from" ref
picker on every workflow_dispatch run, so the custom inputs.branch
free-text field was duplicating a selection that was already made -
and could silently diverge from it. Switch every functional read of
inputs.branch to github.ref_name in both release workflows; keep the
branch input field itself, but make it explicitly non-functional
(required: false, no default, description points at the ref picker).
type: choice was considered and rejected again here: it would close
the typo risk but can't accept an unregistered branch, breaking
one-off custom-branch releases without editing the workflow file per
release - exactly the maintenance burden EMB-1518 removes.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01U2Va4ryWuZbQvmUt2N85ec
* fix: remove dead readme-update mechanism (EMB-1518)
update_readme's two sed patterns match zero occurrences in the current
README (restructured by 15f7d3d20e, no version-pinned content remains),
so publish-npm's readme-follow-up-PR step has been a silent no-op on
every release, not just re-runs. Deleted the entire dead block (7 steps,
the env vars and pull-requests:write permission it alone used) and the
now-unused update_readme function.
* fix: drop minor release_type option (EMB-1518)
`semver -i minor` on a release branch jumps to the next major's
version line, colliding with that major's own branch. `patch` already
covers graduation and `preminor` already covers the legitimate
next-major jump (scoped to master). Drop `minor` outright rather than
add compensating validation - resolves review-cycle-3 finding #17 /
Judgment call #6.
* fix: add explicit reviewer request for embedding team on bump PR (EMB-1518)
CODEOWNERS is deliberately stripped on release branches (a repo-wide
practice to cut auto-backport review noise, unrelated to this
workflow), so the bump PR's "code owner review is automatic" comment
was false there - it got zero reviewers requested on release
branches. EMB-1000 documented the identical gap in the old system's
follow-up PR. Fix targets the metabase/embedding team (not a named
individual - the old system's hardcoded reviewer has since left).
* fix: restrict release types by branch, add custom releases (EMB-1518)
A one-off branch could publish as the stable release. Validation only
guarded master and release-x.*, so `patch` on e.g. sdk-data-apps stripped
the prerelease id (0.62.5-data-apps.0 -> 0.62.5), computed the
<major>-stable dist-tag, and - when the major matched vars.LATEST_VERSION
- set tagAsLatest, shipping a feature branch to npm as the stable release.
Only check-git-tag stood in the way, and only if that version had already
shipped.
Each branch class now gets only the release types that make sense for it:
master takes alpha/preminor, release branches take beta/patch, any other
branch takes custom. tagAsLatest is additionally gated on a release-x.*
branch, so nothing published from master or a one-off branch can take the
npm `latest` tag.
The custom type keeps one-off branches (esbuild, data-apps) releasable
under that restriction: prerelease_id is set on the first cut and becomes
both the version's prerelease id and the <major>-<id> dist-tag, matching
the shape already live on npm (56-esbuild, 63-data-apps). Later bumps
leave it empty and semver reuses the id from the committed version.
Also drops the dead release_utils.bash (zero functions, zero callers),
replaces both free-text dispatch inputs with single-option choices that
cannot be typed into, points the release workflow's form at the bump PR,
and removes the scaffold banner comments.
* fix: read vars.LATEST_MAJOR_VERSION, not LATEST_VERSION (EMB-1518)
The variable is LATEST_MAJOR_VERSION - it is what release.yml already
reads to decide Docker's `latest` tag, and what the release checklist has
a github admin bump each cycle. `vars.LATEST_VERSION` does not exist, so
tagAsLatest would have computed false on every bump, including at gold,
without any error - the fallback for an unset variable is false.
* refactor: make the release concurrency group readable (EMB-1518)
Same three cases as before, but each arm now names what it produces
(sdk-release-<branch> for a dispatch or a merged bump PR, sdk-release-noop-<run id>
for a closed PR this workflow ignores), and the comment above maps them in
the order the expression tries them.
The noop arm is not decoration: concurrency is evaluated before any job's
`if:` runs, so a closed PR that prepare-release skips would still enter the
release group without it. cancel-in-progress stays false so a release is
never interrupted mid-publish.
Groups are prefixed rather than bare because concurrency groups are
repo-wide, not per-workflow.
* refactor: build the release scripts once, in their own job (EMB-1518)
The Slack jobs each got at release/dist/index.cjs a different way:
send-slack-notification-start checked out and built it (then uploaded it as
an artifact), send-slack-notification-failure built its own second copy, and
send-slack-notification-successful downloaded the artifact. The duplicate
build existed because the failure job has to fire even when the metadata read
fails - in which case the start job never ran, so its artifact never existed.
A build-release-scripts job now builds it once and uploads it, and all three
Slack jobs download it. The job hangs off prepare-release only, so the
artifact is there no matter how early the release fails - which removes the
reason the second build existed.
* fix: quote the Slack messageId in the success notification (EMB-1518)
A Slack ts is a string. Interpolated bare it becomes a JS number literal and
drops trailing zeros - 1783602196.214000 renders as 1783602196.214, and the
reply targets a thread that does not exist. Pre-existing on master; the
failure job already quoted it, so this makes the two consistent.
* refactor: resolve the release branch in one expression (EMB-1518)
The env: BRANCH indirection was injection hardening from when the value was
the free-text `inputs.branch`. It is github.ref_name now - a real branch
name, same as github.event.pull_request.base.ref, which the next line already
inlined - so it guarded nothing and made the step asymmetric.
base.ref is empty on a dispatch, so `base.ref || ref_name` picks the right one
without branching on github.event_name.
* refactor: drop the duplicate-bump-PR pre-flight (EMB-1518)
It guarded a case the design already handles safely. Two open bump PRs
against the same branch both compute from the branch's committed version, so
both propose the same next version: the first to merge publishes and tags it,
and the second dies at check-git-tag with "Tag ... already exists" (npm would
reject the republish too). Nothing is published twice, and the error says
exactly what happened.
Getting there needs someone to dispatch the bump twice and leave the first PR
open, with both PRs sitting labelled in the list. That is not worth an API
call and 16 lines on every bump.
* chore: trim the bump step's comments (EMB-1518)
* refactor: drop the semver validity pre-check (EMB-1518)
Every supported release branch (57 through 63) already carries a valid
semver, and the new process never writes a `0.N.x` placeholder - gold is a
patch bump (0.63.0-beta.3 -> 0.63.0). The placeholder only ever existed
transiently under the old checklist, which rewrote it back to a concrete
version after each release.
Also saves an npx invocation on every bump.
* chore: spell out PRERELEASE_ID in the bump workflow (EMB-1518)
* refactor: spell out the semver call in each release type (EMB-1518)
alpha and beta shared an arm that passed --preid="$RELEASE_TYPE", and patch
passed -i "$RELEASE_TYPE" - you had to know the variable's value to know what
command ran. Each arm now names its own call.
The release_type description now leads with the phase (before gold, at gold,
after gold), which is what the person dispatching actually knows. It also
corrects preminor: master moves to the next major *after gold*, not at branch
cut - the 62 cycle did it in the after-gold checklist (#75487, 0.62.0-alpha ->
0.63.0-alpha), and 61 before it (#74279).
* docs: stop calling the bump PR a review gate (EMB-1518)
Nothing blocks on approval: the bump PR is not auto-merged but it is not
gated on a reviewer either, and workflow_dispatch can publish with no PR at
all. The computed distTag/tagAsLatest are defaults made visible and editable
- whoever dispatched the bump has already decided.
* docs: note EMB-2095 (auto-merge the bump PR) in the workflow (EMB-1518)
* docs: tighten the tagAsLatest comment (EMB-1518)
* refactor: collapse the tagAsLatest case into one condition (EMB-1518)
* docs: say what the sdkRelease write does (EMB-1518)
* refactor: drop the label-creation step (EMB-1518)
The sdk-release-trigger label is repo-level, so it only ever needed creating
once - it now exists. Re-running `gh label create ... || true` on every bump
swallowed every error (including auth) to no purpose.
* fix: fail on a leftover bump branch instead of side-stepping it (EMB-1518)
The run-id suffix made every retry a fresh branch, so an abandoned bump left
a branch on origin forever and the name was unpredictable. The branch is now
named after the version, and an existing one is an error: an earlier bump to
the same version was never merged, which is worth knowing before opening a
second PR alongside it.
* refactor: drop the no-backport label step (EMB-1518)
Backporting is opt-in - auto-backport.yml only fires on a `backport` label -
and backport-reminder.yml only demands a decision label when a PR touches
docs/**. The bump PR touches one JSON file, so no-backport guarded nothing.
Keeps the reviewer rationale, moved next to the --reviewer flag it explains.
* feat: show the target branch and computed values in the bump PR body (EMB-1518)
The PR body did not say which branch it targeted, which is the first thing
you want to know when a release PR shows up. Branch, release type, version,
dist-tag and tagAsLatest are now a table at the top.
* fix: pin semver and validate prerelease_id (EMB-1518)
npx --yes semver resolved whatever the registry served at dispatch time, on
a job that installs no dependencies - so release-critical version math could
change under us while package.json already pins ^7.6.3. Pinned to semver@7.
prerelease_id was passed to semver unchecked. An id like `DataApps` bumped
fine and then died three steps later in the dist-tag read-back, whose regex
only matches [a-z][a-z0-9-]*, with an error blaming the computed version
instead of the input. It is now rejected up front, with a message naming the
rule.
* fix: take semver's version from package.json (EMB-1518)
Pinning the workflow to semver@7 fixed the drift but created a second
version to keep in step. Read the range the repo already pins instead
(devDependencies.semver, currently ^7.6.3) and hand it to npx, so bumping
semver in package.json is the only place it happens.
* refactor: rename bump workflow input to read_first (EMB-1518)
* feat: reject prerelease_id on non-custom release types (EMB-1518)
* refactor: drop the preminor bump-only warning from the PR body (EMB-1518)
* refactor: drop the duplicate rerun wording in tag-exists error (EMB-1518)
* fix: reject reserved alpha/beta as a custom prerelease_id (EMB-1518)
* refactor: tidy and clarify comments in the SDK bump workflow (EMB-1518)
* fix: satisfy actionlint in the SDK release workflow (EMB-1518)
* feat: extract embedding SDK release logic into a tested helper (EMB-1518)
* refactor: remove dead embedding SDK helpers from version-helpers (EMB-1518)
* fix: drop trailing blank line in the SDK bump workflow (EMB-1518)
* refactor: run the SDK version bump from a tested helper, not inline bash (EMB-1518)
* docs: note the env block guards github-script against injection (EMB-1518)
* refactor: drop the unused LATEST_MAJOR_VERSION fallback in the bump script (EMB-1518)
* feat: show the target branch in the bump PR title (EMB-1518)
* docs: drop the manual-edit invitation from the bump PR body (EMB-1518)
* refactor: tidy the SDK release helper and its tests (EMB-1518)
* refactor: read the SDK release metadata via the tested helper (EMB-1518)
* refactor: env-route the branch name in the bump PR title (EMB-1518)
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0156DAaukfiU6xceXd3brbZN
* refactor: simplify SDK release concurrency group (EMB-1518)
Collapse the three-case concurrency expression to one per-branch group, and
switch every multi-item needs array to a block sequence.
The dropped noop group guarded a compound case: GitHub keeps one pending run
per group and cancels any earlier one, so a closed PR touching the template
could evict a release waiting to start. That needs two same-branch releases
in flight at once, so we take the simpler expression. The clean fix is
queue: max, which actionlint cannot parse yet.
---------
Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
* Deflake transform revision history spec
The "view and revert transform revisions" test changed the transform
name twice, each firing a "Transform name updated" success toast that
lives for the default 5s undo timeout. Unlike every other update flow in
this spec, these toasts were never dismissed. On a fast runner the test
reached the failed-revert error toast within that window, so a lingering
success toast coexisted with it and H.undoToast() (a single-element
findByTestId on [data-testid="toast-undo"]) failed with "Found multiple
elements".
Dismiss each success toast right after it appears, matching the existing
convention used elsewhere in this file, so exactly one toast is present
when the error toast is asserted.
---------
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-authored-by: Nemanja <31325167+nemanjaglumac@users.noreply.github.com>
* Add semantic search health checks
* Split HNSW recovery from semantic health
* Use HoneySQL literals for repair queries
* Use dynamic redefs in semantic search test
* Auto-scroll to a dashcard whenever it becomes the scrollTo target, and keep the scroll until the grid settles
* Drop a stray brace from the auto-scroll spec's first visit URL
* Simplify the auto-scroll correction to a fixed frame window
* Delete the react-router v3 engine
* Sever the react-router package dependency
* Give the SDK data-app router its own browser history on v7
* Rename the react-router-v7 alias to react-router
* Rename getDataAppHistory to getRawBrowserHistory, drop dead getCurrentHistory
Scope existence preconditions to their table; lint against name-only checks
A foreignKeyConstraintExists precondition with only foreignKeyName (or
indexExists with only indexName) forces Liquibase to snapshot the entire
schema to answer the check -- ~2s per precondition on a large postgres
appdb. 27 such preconditions (24 FK, 3 index; 21 in one v59 workspace
block) added ~46s to every full migration: 50.2s of total changeset time,
of which 24 changesets cost 45.9s and the other 1,182 cost 3.7s.
Add the scoping key to all 27. Measured on a fresh postgres migration:
total changeset time drops 50.2s -> 5.1s, with zero changesets over 1s.
Editing released changesets' preconditions is safe: Liquibase checksums
cover the changes, not preConditions. Verified two ways -- ChangeSet
.generateCheckSum is byte-identical before/after for all 27 (v9
checksums), and a clone of an already-migrated appdb re-validates
cleanly against the edited files.
lint-migrations-file: generalize require-primary-key-exists-has-table-name
into require-table-scoped-existence-preconditions, covering
primaryKeyExists/indexExists/uniqueConstraintExists (tableName) and
foreignKeyConstraintExists (foreignKeyTableName), walking the full
precondition tree in both its flat-map and list forms.
Claude-Session: https://claude.ai/code/session_01G7FfdpzW5pr7CX9aNwa98f
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
* Fall back to .yaml if .yml not found
Update docs to mention `.yaml` support
* Fall back to .yaml if .yml not found
Update docs to mention `.yaml` support
* Update file.clj
* Fix tests
* Address feedback
* Drop field_usage table as it is no longer used.
* Stop saving preprocessed_query in metadata since it's not used any more.
This used to be used for saving off field usage, but that hasn't been
a thing for a very long time.
* Make model-illustration.svg work well in both light and dark mode
* whitelabel svg
---------
Co-authored-by: Nick Fitzpatrick <nickfitz.582@gmail.com>
* Migrate PinMap to a typed function component and extract a pure getPoints
* Type the pin metric as nullable, derive MapSettings from VisualizationSettings, and add createMockVisualizationProps
* Trim comments, replace the map ref cast with an instanceof guard, and cover instance clearing
* Don't block dashboard saves on pre-existing foreign internal cards
UXW-4870: the guard added in #77236 (reject assigning a dashcard's card
to a question internal to another dashboard) checked updated dashcards
unconditionally, so any dashboard with a pre-existing foreign internal
card association became unsavable — every edit, even a resize, returned
400 "Invalid Request.".
Scope the guard to newly-introduced card ids: card ids the dashboard
already references (as a dashcard's card or in a series) are
grandfathered, so existing associations never block saving — including
moving such a card to a different dashcard in the same save. Newly
assigning a question internal to another dashboard is still rejected.
* Clarify arg names in internal-dashcard guard
Rename the guard's params to existing-dashboard/new-dashcards to make
clear the grandfather set comes from stored state while the checked
dashcards are the incoming save.
* Guest embed filter dropdowns should use the router database (UXW-4881)
PR #68952 made guest-embed query execution bypass DB routing by wrapping it
in with-database-routing-off, but the parameter-values endpoints (dropdown
values, search, remapping) didn't get the same treatment: chain-filter runs
through the QP, where attach-destination-db-middleware throws "Anonymous
users cannot access a database with routing enabled."
- Wrap the query-running sections of the four param helpers in
metabase.embedding-rest.api.common in with-database-routing-off, fixing
all six /embed/{card,dashboard}/:token/params/* endpoints plus their
preview_embed equivalents.
- Wrap the guest-embed tiles endpoints, which have the same anonymous-user
bug.
- Wrap preview_embed's query-execution endpoints (query, dashcard, pivot,
tiles) so the admin's embed preview shows exactly what the published
embed will show (router DB data) instead of routing via the previewing
admin's user attribute.
- Replace the manual *current-user-permissions-set*/*is-superuser?*
bindings in the param helpers with the equivalent request/as-admin.
Public sharing has the same gap but whether routed DBs should work over
public links at all is an open product question, so it is intentionally
left out.
* Consolidate embed routing-off wraps into the shared execution helpers
Review feedback: rather than a macro that each endpoint must remember to
use, move with-database-routing-off inside the shared execution helpers in
metabase.embedding-rest.api.common (process-query-for-card-with-params,
process-query-for-dashcard, process-tiles-query-for-dashcard, and a new
process-tiles-query-for-card wrapper), which every /api/embed and
/api/preview_embed endpoint funnels through. Endpoints built on these
helpers now bypass routing automatically, so a future endpoint can't
reintroduce this class of bug.
Removes all per-endpoint wraps from embed.clj (including the two original
ones from #68952) and preview_embed.clj; the invariant now lives in one
file with one explanatory comment.
* Fix tooltip scrollbar flicker in TreeTable (GDGT-2822)
Hovering a cell tooltip in a virtualized TreeTable while scrolling made a
page-level vertical scrollbar flash on and off. The tooltip is portaled to
<body> and positioned with real top/left insets, and with the document root
sized to the exact viewport height, a tooltip positioned past the fold grew
the document and flashed a scrollbar.
Portal cell tooltips into the table's own scroll container instead of <body>,
so a tooltip positioned past the fold is clipped by the (already-scrolling)
table rather than stretching the page. This is wired through a small
TooltipPortalTargetProvider context so the metabase/ui Tooltip wrapper picks
up the target automatically — no per-tooltip wiring, and any TreeTable with
cell tooltips benefits (e.g. the notifications admin table).
* Address PR comment
* Fix Tooltip wrapper not forwarding ref
* Add monitor dir in metabase-enterprise
* GDGT-2676: Migrate Dependency Diagnostics under Monitor space
* GDGT-2677: Migrate Admin -> Tools under Monitor space
* GDGT-2676: Fix paths
* GDGT-2676: Create separate high-level directories for migrated features
* GDGT-2657, GDGT-2686: Monitor space + migrate Dependency Diagnostics (#76527)
* GDGT-2657: Introduce Monitor space layout and routing
* GDGT-2686: Migrate Dependency Diagnostics to Monitor
* Monitor: Add Monitor plugin. Cleanup + fixes
* Monitor: Update graphy ownership path for Dependency Diagnostics
* Monitor: Cleanup PR
* GDGT-2697: Update hotkeys hints for Toggle navbar
* GDGT-2684: Address Copilot review comments
* GDGT-2684: Address PR comments
* GDGT-2684: Rename SpaceLayout to AreaLayout. Split to separate files
* GDGT-2684: Refactor route guards + add tests
* GDGT-2684: Move Admin Tools pages to the Monitor space (#76606)
* GDGT-2684: Move Admin Tools to Monitor space
* Monitor: Extract diagnostics sidebar into shared Monitor sidebar outlet
Add a useMonitorSidebar context to MonitorContent so Monitor pages can
render sidebars in a dedicated right-hand region outside the padded,
scrollable main content area. Rewire Dependency Diagnostics to render its
detail sidebar through this outlet, and promote the resizable shell
(SidebarResizableBox) into shared monitor/components.
* Monitor: Add MonitorHeaderTitle component for consistent page headers
Introduce a shared MonitorHeaderTitle matching the small secondary text
style used by Data Studio breadcrumbs, and apply it across Monitor pages:
Dependency Diagnostics header, Jobs, Logs, Model cache log, Task list/runs,
Task and task-run details, Erroring questions, and Alerts management.
* GDGT-2684: Cleanup
* GDGT-2684: Update monitoring tools access permissions
* GDGT-2684: Add handling for not-found routes
* GDGT-2684: Fix not-found routes handling in Data Studio
* GDGT-2684: Extend route tests. Minor styling refactoring
* GDGT-2684: Refactor Alerts Management sidebar
* GDGT-2684: Move old SidebarResizableBox to ModelSibar
* GDGT-2684: Cleanup
* GDGT-2684: Refactor Sidebar solution to use Portal
* GDGT-2684: Fix Monitor Tools E2E specs (#76886)
* Format monitor router files after the master merge
* Import Route from metabase/router facade in AreaTab spec
Fixes the no-restricted-imports lint error banning direct react-router imports.
* GDGT-2713: Update Tasks page styling (#77088)
* GDGT-2713: Update Tasks page styling
Restyle the Monitor Tasks and Runs pages to match Dependency diagnostics:
- Replace the SettingsSection card + default tabs shell with the Monitor
frame (fixed-height page, pills tabs; only the table scrolls)
- Rebuild TasksTable and TaskRunsTable on the TreeTable data-grid
(sticky header, virtualized rows, skeleton loading, a11y labels)
- Move filters into a Group bar and pagination below the table with a
total results count
- Keep the 2-state sort contract (always sorted, desc/asc) on top of
TanStack's 3-state cycle; asc-first on all sortable columns
- Add MonitorHeaderTabs (Monitor-owned pills tabs, no data-studio
dependency) and extract the shared LinkTab router-link tab to
metabase/common/components
* GDGT-2713: Fix e2e test
* Migrate LinkTab to the metabase/router facade
* Remove unused Link import from PaneHeader
The Tasks styling switched the tabs to LinkTab, leaving Link unused.
* Address PR review comments and fix entity_name typing
Use strict undefined comparisons instead of loose null checks where
values can only be undefined (optional props, RTK Query error,
destructured SortingState), and strict null comparisons where the
value can only be null (Task.db_id).
Also fix TaskRun/RunEntity entity_name typing: the backend always
assocs this key (nil when the entity lookup misses), so it's
string | null on the wire, not optional.
* GDGT-2712: Update Jobs page styling (#77100)
* GDGT-2712: Update Jobs page styling
Restyle the Monitor Jobs page to match Dependency diagnostics:
- Rebuild the page on the Monitor frame (fixed-height page; only the
jobs table scrolls, with a sticky header)
- Render the scheduler info in a Code block instead of a bare pre
inside a settings card
- Rebuild the jobs table on the TreeTable data-grid (skeleton loading,
empty state, a11y label, keyboard row activation); row click still
opens the job triggers modal
- Move JobInfoApp into a folder and add unit coverage; add
TaskInfo/Job API mocks and a task-info server mock
* Make the Monitor sidebar outlet resizable
Move the resizing behavior from the Dependency diagnostics sidebar into
the shared Monitor Sidebar so every Monitor view gets it without
duplication:
- Relocate SidebarResizableBox from metabase-enterprise/monitor into
MonitorLayout and wrap the Sidebar portal content with it
- Sidebar API: optional resizable (default true) with containerWidth
required while resizable, optional defaultWidth (512 by default);
consumers measure their content area with useElementSize and pass it
- Protect the Monitor content area from text selection while dragging
inside Sidebar instead of per-view
- Simplify DiagnosticsSidebar/DependencyDiagnostics accordingly and
drop the now-empty metabase-enterprise/monitor/components tree
- Make the Alerts management detail sidebar resizable (fills the
resizable box, keeps its 560px default width)
* GDGT-2712: Jobs page styling fixes
* Replace the job triggers modal with a Monitor sidebar
Clicking a job on the Jobs page now opens the job's triggers in the
resizable Monitor sidebar outlet instead of a modal:
- New JobTriggersSidebar renders each trigger as a vertical key/value
block (all fields the modal table had), with a divider between
triggers and a close action that returns to /monitor/jobs
- The /monitor/jobs/:jobKey route stays, so deep links and back/forward
keep working; JobInfoApp reads the jobKey param and renders the
sidebar, measuring its content area for the resize constraint
- Delete JobTriggersModal; add a Trigger API mock
- Give the scheduler info Code block the old card padding and keep it
from shrinking
* Apply cross-review fixes for the jobs sidebar work
- Render trigger details as a dl/dt/dd list for proper label/value
semantics (no visual change)
- Cover the shared Sidebar resize side effect (user-select class on the
Monitor container across drag start/stop/unmount)
- Cover keyboard row activation, all trigger attributes with null
placeholders and the may-fire-again 'No' branch, and the no-triggers
empty state
* GDGT-2712: Cleanup
* GDGT-2712: Add distinct error message when job is not found
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
* Migrate JobInfoApp to the metabase/router facade
* Disable text selection on body while resizing the sidebar
Address review comment: dragging the resize handle otherwise causes
the browser to continuously select/deselect page text, since the
sidebar is portaled and doesn't share a DOM ancestor with the main
content to scope a CSS class to.
---------
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
* GDGT-2711: Update Logs page styling (#77160)
* GDGT-2711: Extract LogsViewer to common Monitor components
* GDGT-2711: Update Logs page styling
* GDGT-2711: Cleanup
* GDGT-2711: Address PR comments
* GDGT-2711: Address PR review comments
Rename LogsViewer's process prop to processUUID, drop the unused
logs-content id, and remove a few comments flagged as redundant.
* GDGT-2661: Update Erroring questions page (styling + refactoring) (#77167)
* GDGT-2661: Restyle Erroring questions page
Rewrite the EE /monitor/errors page off the legacy audit_app viz onto the
Monitor recipe used by the other Monitor tools pages, keeping the same
backend audit query:
- TreeTable with the 11 audit columns, 2-state sortable headers, and native
row selection; "Rerun Selected" moved to the standard bottom BulkActionBar
- Dependency-diagnostics-style debounced filter inputs
- Page no longer scrolls; the table body is the sole scroll region
- Pagination outside the table with a total count (new bad-table-count audit
query), hidden until the data has loaded, and synced to a ?page URL param
- Requests abort the previous in-flight one on paging/sorting/filtering
Replaces the AuditParameters/AuditTable/ErrorMode stack; deletes the old
ErrorOverview and its click-mode.
* Remove dead legacy audit-table v1 code
The Erroring questions page was the last consumer of the Audit v1 table
stack. With it rebuilt on TreeTable, these are unreferenced and removable:
AuditTable, QuestionLoadAndDisplay, AuditTableVisualization (the last
"audit-table" display), AuditParameters, the bad_table card def, and
AuditMode. The rest of audit_app (unsubscribe UI, InsightsLink,
metabot-analytics) is untouched.
* Fix duplicate/incorrect cards in the erroring-questions audit query
The latest_qe CTE joined query_execution to the per-card MAX(started_at)
subquery on started_at alone, so executions across cards that shared a
timestamp fanned out — surfacing duplicate rows (and cards whose latest run
didn't actually error). Join on card_id as well. Regression test covers a
shared-timestamp collision.
* GDGT-2661: Address review feedback
- Surface count-query failures in the page error state so a failing count
can't silently hide pagination
- Filter bar reflows (wrap) instead of clipping at narrow widths / zoom
- Skeleton renders the checkbox column to avoid a layout shift on load
- Centralize the internal-dataset-query cast to a single boundary
- Add tests: count db/collection filter clauses (backend), 202-body
data.error path, real filter-reset assertion, rerun failure recovery
* GDGT-2661: Fold pagination total into the rows query via COUNT(*) OVER()
The page ran two /api/dataset requests per view — the bad-table rows and a
separate bad-table-count — each recomputing the expensive latest_qe scan over
all query executions. Add a COUNT(*) OVER () total_count column to bad-table so
the full filtered total rides on the rows response, and drop the count query.
Halves the round-trips and eliminates the second latest_qe scan (~2.1s -> ~1.4s
of DB work on the Stats dump); the window count adds no measurable overhead
since it runs over the already-materialized result set before LIMIT.
* Migrate ErrorOverview to the metabase/router facade
* GDGT-2661: Return erroring-questions total_count at the response root
The bad-table query computes COUNT(*) OVER () as a column, which repeated the
total on every row. Strip that column via an xform and surface the value once
as a top-level total_count on the response; the FE reads it from there.
* GDGT-2661: Non-blocking per-row rerun with loader in erroring questions
Rerun now runs in the background per card instead of blocking on the whole
batch: clicking Rerun Selected closes the bulk bar immediately and fires each
request independently, so more questions can be queued while slow ones run.
A rerunning row shows a loader in place of its checkbox, is excluded from
Select all, and its loader clears when the request settles; the list refetches
once the last rerun finishes.
TreeTable gains a generic isRowLoading prop and makes the whole checkbox cell
a click target so a near-miss no longer navigates to the row.
Also hoist total_count onto the Dataset type and drop the local helper.
* Extract shared generic sorting helpers for monitor tables
The monitor tables (erroring questions, tasks, dependency diagnostics) each
duplicated getSortingState/getSorting/isSortColumn. Extract them into a generic
metabase/utils/sorting module keyed by the column type. getSorting is overloaded
so the un-sort behaviour is type-driven: pass the current sort for always-sorted
tables (direction flips) or omit it to allow an unsorted result.
Refactor the three monitor tables onto the shared helpers.
* Update erroring questions E2E selectors for the new table
Point the tests at the TreeTable-based ErrorOverview: erroring-questions-table
container, erroring-question rows, and the always-enabled filters / bulk bar
that closes on rerun instead of a disabled button.
* Bind *driver* when reducing audit internal queries
The MySQL timestamp reader calls results-timezone-id, whose driver arg is
malli-validated as a keyword in tests. reduce-results* bound the timezone
override but not *driver*, so it was nil and audit queries returning timestamps
errored on MySQL/MariaDB. Bind *driver* to the app-db driver alongside the
override.
* GDGT-2661: Address PR comments
* GDGT-2661: Merge erroring questions input into one, extend to support name search
* GDGT-2661: Fix checkbox cell width in rows
* GDGT-2661: Fix e2e test
* GDGT-2661: Address PR comments
* GDGT-2710: Update Model cache log page styling (#77277)
* GDGT-2713: Update Model cache log page styling
Restyle to match Dependency diagnostics: the page no longer scrolls
vertically, only the table does. Replace the hand-rolled AdminS.ContentTable
with a metabase/ui TreeTable (skeleton while loading, empty state, refresh
action), and drop the emotion-styled ModelCacheRefreshJobs.styled.tsx in
favor of a CSS module.
Model and Collection columns are the two stretching columns (fill remaining
width) and support client-side sorting, since the underlying /api/persist
endpoint has no sort param.
* Unify monitor tables on 3-state sortable headers
Erroring questions and Tasks forced a 2-state flip-forever cycle by passing
the current sort back into getSorting(); drop that so a 3rd click clears the
header's sort arrow and falls back to the page's default sort, matching
Dependency diagnostics and Notifications.
The default-sorted column on each page needs sortDescFirst matching its
default direction (desc), otherwise the very first click on that column is a
no-op: TanStack's per-column cycle treats desc as the removable/last state
when sortDescFirst is false, so a column that starts at desc goes straight to
removed on click one.
* Make all applicable columns sortable in Model cache log and Jobs
Both are fully-loaded, unpaginated lists, so enabling client-side sorting on
the remaining columns is safe. Model cache log's action column stays
unsortable (no header text or data). Tasks and Task runs are left as-is:
their backends only support a fixed set of sort columns (or none), and
sorting the rest client-side would silently reorder only the current page.
* GDGT-2710: Address PR comments
* GDGT-2709: Update Alerts management page styling (#77364)
* GDGT-2709: Update Alerts management page styling
Restyle to match Dependency diagnostics: the page no longer scrolls
vertically, only the table does. Move pagination outside the table into a
plain footer and show the total result count via PaginationControls'
showTotal.
* Restrict Alerts management to admins only
The /api/notification/admin* endpoints are superuser-gated on the backend,
but the frontend route and nav link were reachable by anyone with the
broader can_access_monitoring permission (shared with the other Monitor
Tools pages), so a non-admin holding it could reach the page while every
API call inside it 403'd. Add a dedicated canAccessAlertsManagement
selector and route guard so the route/nav link match the backend's
admin-only restriction.
* Fix misleading test name in MonitorLayout spec
The test asserts the link is hidden for a non-admin analyst, not that it's
shown for an admin.
* GDGT-2794: Update Task/Run details page styling
Extract the "Back to Tasks"/"Back to Runs" breadcrumb into a shared
MonitorBackLink component, styled and positioned like other Monitor
page headers. Wrap both detail pages in the same non-scrolling shell
used elsewhere in Monitor so only the content area scrolls internally,
instead of the whole page scrolling behind the floating AppSwitcher.
* Extract MonitorPageContent to replace SettingsSection in Monitor
SettingsSection carries admin-page title/description logic that Monitor
pages don't use. MonitorPageContent keeps the same card styling with none
of that logic, dropping the last two /monitor call sites still importing
from metabase/admin.
* GDGT-2709: Fix layout in Tasks Details page
* GDGT-2709: Refactor NotificationsTable row selection logic. Fix selection with spacebar
* GDGT-2791: Abort superseded in-flight list requests in Monitor (#77694)
* GDGT-2791: Abort superseded in-flight list requests in Monitor
Changing page/sort/filter on a Monitor list page left the previous, often
expensive, list request running to completion on the server. Subscribed RTK
useXQuery hooks only abort when their last subscriber unmounts, so introduce
a shared useAbortableQuery hook built on lazy triggers (whose promise exposes
.abort()) that cancels the superseded request on arg change, unmount, and
skip transitions, and suppresses the abort error so cancellations never flash
the error view.
Caching mirrors useQuery: cache-first by default, refetchOnMountOrArgChange
to opt into always-fetch; manual refetch() always hits the network. Every
converted call site keeps its pre-migration caching semantics.
Converted: TaskListPage, TaskRunsPage, TaskRunEntityPicker,
ModelCacheRefreshJobs, NotificationsAdminPage (main list),
DependencyDiagnostics, BrokenDependentsSection, and ErrorOverview (its
one-line useAbortableAdhocQuery wrapper is inlined and hooks.ts deleted).
* GDGT-2791: Cleanup
* GDGT-2791: Fix type narrowing in TaskRunEntityPicker
* GDGT-2791: Clamp pagination range to total
* GDGT-2791: Test that tag invalidation does not refetch superseded args
* GDGT-2791: Scope query aborts per hook instance via injected cache key
* GDGT-2791: Migrate _refetchDeps to the shared __rtkCacheKey mechanism
* GDGT-2791: Refine baseQuery typings. Cleanup
* GDGT-2791: Replace interface in params declarations with type. Refine BaseQuery params type
* GDGT-2832: Group Monitor navbar items into sub-menus (#77706)
* GDGT-2832: Move AreaContent to AreaLayout and normalize Monitor page spacing
* GDGT-2832: Group Monitor navbar items into sub-menus
* GDGT-2832: Cleanup. Fix AreaTab outline overflow
* GDGT-2832: Match Alerts Management tab spacing to Dependency Diagnostics
* GDGT-2832: Replace collapsible navbar groups with static labelled sections
* GDGT-2832: Update Monitor views headings to match navbar. Fix e2e test
* Monitor section design tweaks (#78007)
* icon and spacing updates
* remove info icon from background tasks page heading
* improve empty states
* GDGT-2810: Enable sorting in all Monitor views (#77842)
* GDGT-2810: Enable sorting on all Background tasks and Task runs columns
* GDGT-2810: Remove bold styling from first Monitor table columns
* GDGT-2810: Fix reflection lint warnings in task history API test
* GDGT-2810: Qualify task_run/task_history filter columns
The entity_name and task_count sorts add joins (report_card and
report_dashboard both have their own entity_id column), so unqualified
filter columns become ambiguous: /api/task/runs?entity-id=N&sort_column=entity_name
returned a 500. Qualify the filter columns and add a regression test
combining filters with the derived-column sorts.
* GDGT-2810: Cleanup
* GDGT-2810: Use kebab-case sort params on the task runs endpoint
* GDGT-2810: Extract shared useSortingStateChange hook for TreeTable sorting
* GDGT-2810: Extract shared page/sort URL param parsers
* GDGT-2810: Fix e2e tests
---------
Co-authored-by: Ngoc Khuat <qn.khuat@gmail.com>
* GDGT-2843: Integrate Monitor upsell pages (#77948)
* GDGT-2843: Migrate Monitor upsell pages into monitor scope, link to /upgrade/monitor
* GDGT-2843: Tighten monitor module boundaries to standard shared tier
* GDGT-2843: Change Monitor app icon from gauge to pulse
* GDGT-2843: Update pulse icon and Monitor logo
* GDGT-2843: Address PR comments
* GDGT-2843: Fix e2e tests
* Fix Monitor code review findings (#78106)
* Fix Monitor code review findings
* Remove obsolete dependencies redirects
* Monitor review: Cleanup
* Address review comments
* Cleanup
* Remove Dependency diagnostics link from Data Studio nav
It already lives under Monitor (/monitor/dependency-diagnostics); the
Data Studio route just redirects there now.
---------
Co-authored-by: Kamil Mielnik <kamil@kamilmielnik.com>
Co-authored-by: YB Tan <tybmeta@gmail.com>
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Maz Ameli <maz@metabase.com>
Co-authored-by: Ngoc Khuat <qn.khuat@gmail.com>
* WIP: start flipping unit-test harness to v7
Default routerEngine to v7 and render bare (non-Route) children under a
catch-all route so v7's <Routes> renders them. role=link nav failures still
to diagnose. Suite not yet green.
* Give v7 tests a history handle and drop DOM-wrapped route config
The harness now owns the v7 memory history and hands specs a v3-shaped adapter
(getCurrentLocation/push/replace/goBack/listen), so the ~42 specs that drive and
assert navigation through the returned history keep working on v7.
Route config wrapped in a <div> is a v3 artifact that v7's <Routes> cannot read;
swap those two for fragments.
* Pin the v3-vs-v7 conformance suites to the v3 engine
They exist to prove the facade on v3 matches real v7 and are built from raw v3
route config, so they must not follow the harness default now that it is v7.
* Resolve a relative initial route in the v7 test history
* Buffer navigations dispatched before the v7 router registers
* Carry route props onto the v7 matched-route stubs
* Fix the v48 /browse/:dbId-:slug legacy redirect on v7
* Render facade routes in renderHookWithProviders on v7
* Migrate unit specs off v3-only routing behavior for the v7 flip
* Drop buffered v7 navigations when the router unmounts
* Default use-v7-router to true
* Re-home history@3 types onto the metabase/router facade
* Format the re-homed import blocks
* Merge duplicate router imports and drop unused lint directives
* Re-home the router-listen spec's history Location type
Wait for the DebouncedFrame to settle via ensureChartIsActive() before
clicking a group tile to drill in. While the frame transitions it carries
pointer-events: none, so the drill click was silently swallowed and the
breadcrumb button never rendered.
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* Migrate AuditTableVisualization to a typed function component
* Fix Erroring Questions column sorting by forwarding sorting props through Visualization
* Extract typed viz definition to drop casts per review
* GHY-4016: Replace Field YAMLs with FieldUserSettings in git sync (PR 1/3)
Git sync currently exports one Field YAML per database column. For a
typical customer this is ~6396 files — 84% of the entire repo. Most of
that data (active, database_type, position, dimensions, ...) is sync
metadata that git sync has no business tracking; the user-curated part
lives in FieldUserSettings.
This PR switches git sync to export FieldUserSettings instead of Fields.
Only fields with user-curated metadata get a YAML at all.
Measured against the stats-remote-sync repo (196 published tables,
~20 columns each):
Import: 43s → 14s (3x faster, 7659 → 1263 files)
Export: 6.3s → 2.0s (3x faster, 2155 → 239 entities)
Files: 7659 → 1263 (84% reduction)
For a large customer (563k files, 522k Fields), ingest time alone drops
from 116s to 7s.
Full Field serdes is unchanged — non-git-sync export/import is
unaffected. FieldUserSettings already had complete serdes support
(generate-path, storage-path, load-find-local, make-spec); this PR
wires it into the git sync paths.
Changes:
- `serdes/descendants "Table"`: branches on `:user-edits-only` opts.
When set, emits `["FieldUserSettings" field_id]` only for fields with
a FUS row; otherwise emits `["Field" field_id]` as before.
- `spec.clj`: named `git-sync-extract-opts` constant (with
`:user-edits-only true`); public `pk-col` helper for the non-standard
FieldUserSettings PK (`:field_id` not `:id`).
- `extract.clj` (OSS): local `pk-col` for `:where [:in pk-col ids]`
(module boundary prevents depending on EE spec).
- `impl.clj`: `extract-chunk` uses `spec/pk-col` for FieldUserSettings.
- `models.clj`: adds `"FieldUserSettings"` to `data-model-in-collection`.
- `events.clj`: removes `:model/Field` from `register-events-for-spec!`
so field-update events no longer create Field RSOs. A dedicated
`::field-update-event` handler creates FieldUserSettings RSOs only
when a FUS row exists. This completes the incremental path: full
export uses `descendants` + `:user-edits-only`; incremental export
uses FUS RSO rows.
Tests assert on the full RSO table so stray Field RSOs fail immediately.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* Address PR review: defensive get for dynamic pk-col, fix content-hash test for FieldUserSettings
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* don't need maps, k/v pairs work
* Declare :model/FieldUserSettings module boundary export/import
remote-sync references :model/FieldUserSettings in events.clj, so it must
be exported by warehouse-schema and imported by enterprise/remote-sync.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* Remove unused f1-id binding in extract test (kondo)
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* Test that Field and FieldUserSettings are serdes descendants leaf nodes
Cements the design discussed in review: Table's descendants method is the
only source of field-level entries, and the :user-edits-only branch is
exclusive — you get Field refs or FieldUserSettings refs, never both via
a transitive Field -> FieldUserSettings edge.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
* WIP: dag run BE
* split into new table
* run utils
* remove dag changes on job runs
* new dag run model
* abstract over parent-run
* separate dag model/api
* factor common job/dag code in a coordinated-run primitive
* fixes
* dag test
* fix migration
* kondo
* missing resolution
* exclude model
* kondo config
* fix test
* fix: exclude for deser
* fixes
* fix: exclude for deser
* fix: missing fk
* random ids
* chore: rename
* feat: add unioned runs
* remove dead endpoints
* api tests
* extend filter semantics
* trim docstrings
* kondo
* gap: allow canceling job runs
* share code
* fix: job start should return proper job id
* fix: mig to 64
* simplify
* simplify further
* simplify
* dead code
* add unique constraint on dag runs
* tighten schemas
* tighten permissions
* fix test
* relax schema
* fix: set null on delete
* handle deleted jobs/transforms for dag runs
* appease the sql linter
* fix test
* simplify rollbacks
* simplify callsite
* fix tests from bad rebase
* fix: fallback when transform is deleted
* add transform_count
* fix tests
* log with context
* varchar 32 for status
* Add UI for running transform with upstream or downstream (#77367)
* GDGT-2625 wip: reprocess DAG UI for transforms
- Split "Run now" button (upstream/downstream) + confirmation modal
- runDag / dag-transforms / cancel-dag API + analytics event
- Tabbed Runs page: transform-graph runs (default) + individual transform runs
- Graph runs table (Run/Type cols), filter bar (Type/Transform/Started/Status),
sidebar with member transforms; shapes aligned to bronsa's /api/transform/runs
- List + members still served from a mock (transform-graph-runs.mock) via queryFn
* GDGT-2625 wip: align graph runs to /api/transform/runs, sidebar + cancel
- Reshape TransformGraphRun to backend RunSummaryResponse (run_type/entity_id/name)
- Filters mirror Transform runs: Type/Transform/Started/Ended/Status/Trigger
- Sidebar: title = Run name, Type as first metadata row, cancel section below
metadata (dag/job/transform), confirm modal; job-run cancel endpoint wired
- Mock covers job/dag/transform in all states (succeeded/started/failed/timeout)
- Reorder columns: Status/Trigger after Duration
* GDGT-2625 wip: sidebar header entity links
Add "View this job" / "View this transform" (+ dependency graph for
transforms) action links to the graph-run sidebar header, mirroring the
individual transform-runs sidebar.
* GDGT-2625 wip: migrate graph runs to real /api/transform/runs
- Swap list + member queryFn mocks for real queries (list endpoint, dag/job
member drill-down; standalone runs synthesized). Delete the mock fixture.
- Rewrite page spec against real endpoints via fetch-mock + new mock factories.
- Reword DAG run confirm modal (drop "in this order", bullet list) since members
can run in parallel.
* Adjust wording
* GDGT-2625 wip: scheduled state + cleanup
- Show a "scheduled" run state on the transform page after a DAG reprocess is
triggered, so the queued transform gives feedback instead of looking idle
- Drop dead DAG-run types/tag-providers/mock helpers left from earlier iterations
* Address code-review findings for reprocess DAG UI
Correctness:
- Migrate transforms.cy.spec.ts to /runs/individual (the /runs route now
renders the graph-runs page, not the individual run table)
- Gate the "Run all" DAG confirm button on the mutation, preventing a
double-click from triggering two DAG runs
- Toast when run-dag returns a null dag_run_id instead of silently no-op
- Stop the "Scheduled" banner from polling forever when the member-runs
request errors
- Keep the graph-run sidebar open when the selected run scrolls off the
current page during polling
- Invalidate transform state on cancelDagRun/cancelJobRun so member
transforms' last_run refreshes after a cancel
Cleanup:
- Translate whole phrases (with context) for DAG run names
- Replace a typecast with a type guard in TypeFilterWidget
- Memoize parsed params and filter options on the graph-runs page
- Drop the unused direction field from the scheduled-DAG-run state
- Convert the sidebar's IIFE switch to ts-pattern match
* Handle deleted entities in graph runs; fix run-list cache invalidation
- Graph runs table: grey a deleted job/transform's run name (italic +
"has been deleted" tooltip), keeping the row clickable
- Sidebar: keep showing member transforms for deleted DAG and standalone
runs (fetched by dag-run id / synthesized from the run); hide the section
only for deleted job runs, whose members can't be fetched without the
(now-null) job id
- api/transform.ts: invalidate the run lists on deleteTransform (a
transform's runs survive its deletion) and cancelCurrentTransformRun, so
the Runs pages reflect the change instead of relying on polling
- Trim narrating comments from the transform API types
* Cleanup
* Fix transforms cancel e2e: click renamed "Cancel run" confirm button
The transform-page cancel confirmation button was renamed from "Yes" to
"Cancel run", but two cancelation tests still clicked "Yes". The cancel was
never confirmed, so the slow (pg_sleep) transform kept running and held a lock
on the writable DB, hanging the next test's resetWritableDb and failing the
suite. Point those two clicks at the new label (the individual-runs-page cancel
still uses "Yes" and is left unchanged).
* Avoid type cast in RunSection spec to satisfy no-unjustified-type-casts
Replace `body as string` with a typeof guard when reading the run-dag request
body, which the new lint rule flags.
* Show dependency count in the graph-run label
Use bronsa's new transform_count to label DAG runs as
"{n} dependencies › {name}" (upstream) / "{name} › {n} dependents"
(downstream), n = transform_count - 1, joined by a small chevron icon.
Bold the job/transform name; fall back to the plain name when the count is
absent (job/standalone runs, or DAG runs predating the column).
* Replace runs tabs with Detailed view switch; polish graph-run name rendering
- Swap the two runs tabs for a Detailed view switch that toggles grouped
(/runs) and detailed (/runs/individual) views in place; add RunsPage shell
and DetailedViewSwitch, remove RunTabs
- Render the sidebar header run name like the table (dimmed deps count +
chevron), with wider gap and aligned font size/weight
- Keep the Detailed view switch label on one line
- Bump graph-runs page size to 25 to match the individual runs page
- Fix stale react-router-redux import
* Preserve shared filters across Detailed-view toggle; drop deprecated CSS util
Toggling the Detailed-view switch previously called the run-list URL helpers
with no arguments, discarding the active filters. Forward the params common to
both views (statuses, transform ids, start/end time, run methods) via a new
CommonRunListParams/pickCommonRunListParams helper; page and sort are dropped
since the two views paginate differently and use incompatible sort columns.
Also replace the deprecated CS.overflowHidden global utility class in the new
graph-run table with an inline overflow style prop.
* Track Detailed-view toggle in transform runs analytics
Emit a transform_runs_view_toggled simple event from the DetailedViewSwitch
toggle handler with the resulting view mode (detailed/grouped), so both the
grouped and individual run-list call sites are covered.
* Fix reprocess run status: don't re-show Scheduled after a member completes
- Derive the DAG scheduled state from this transform's own member run:
a member run row only exists once the transform starts, so its presence
means the transform already ran this cycle. Prevents isScheduled from
re-arming (and masking the completed status) after this transform's run
finishes while sibling DAG members are still running
- Keep the graph-run sidebar polling while the top-level run is canceling,
mirroring isPollingNeeded
* Cleanup
* Address PR review nits from Uladzimir
- Extract isActiveRunStatus helper and use it for the duplicated
started/canceling checks (RunButton, both isPollingNeeded, sidebar
shouldPoll, RunSection dagFinished)
- DetailedViewSwitch: replace styles prop with a CSS module class
- Convert renderRunName/renderRunEntityName to RunName/RunEntityName
components
- Drop redundant String() around an already-string cell value
- Cover RunButton onRun click and the run-options menu show/hide
* Adapt runs routes to the react-router migration on the rebased base
The base migrated transforms routes to element=/withRouteProps with
Outlet-based layouts. Make RunsPage an Outlet layout and render the runs
list pages via element in the specs.
* wip
---------
Co-authored-by: Oleh K <oleh@metabase.com>
* Treat a v7 Link with an empty `to` as a button, not a navigation
* Make a v7 replace to the current URL a no-op so URL-sync effects settle
* Provide router.listen on the v7 engine and keep the router shim stable
* Serialize a v3 location query into the search string on v7
* Fix lint and type issues in the v7 router shim
* Keep the v7 matched-route branch stable across renders
* Republish the v7 splat param under v3's `splat` name
* Enable the react-router v7 engine by default
* Commit the v7 location update synchronously instead of in a transition
* Mirror the v7 location into redux from the history subscription
* Format the v7 router provider
* Sort query keys when serializing a v3 location query
* Match v3 query encoding and resolve bare link paths against the root
* Keep v3 query encoding and leave absolute link urls alone
* Always push when a v7 link targets the current url
* conversation list endpoint
* add rtkquery endpoints
* generate conversation title
* better title generation logic, poll for title if not recieved before end of first message
* render conversation title in ui and show conversation list
* allow resuming historical conversations
* show exploration conversation history
* refresh current conversation ui for 409 case
* surface conversation titles in usage analytics
* add /metabot/conversation/:id route
* remove pending title data part, assume from lack of title data part we need to poll, fix loading a conversation that is mid-response
* title backfill job
* rename title event
* FE prevent polling multiple times for conversation title
* improve title generation prompt
* ci fixes
* automated review feedback
* e2e test fixes
* reduce diff size
* self-review
* Address PR review feedback
- api: pass profile-id through valid-usage-profile-id when ensuring a
conversation title, matching the backfill job
- title backfill: gate readiness on check-usage-limits! (instance-level)
so the job pauses and reschedules cleanly when a usage limit is hit
- title backfill: bound the title future .get with a 60s timeout so a
stuck generation can't block the backfill loop indefinitely
* Add Metabot option to the Transforms New menu (GDGT-1773)
Adds a "Metabot" entry as the first option in the Transforms "New" menu.
Clicking it opens the data-studio Metabot sidebar with a pre-seeded
"Create a transform that " prompt (not submitted), gated on Metabot access.
Also fixes the tiptap prompt input so a programmatically seeded prompt
places the caret at the end instead of the start (autofocus "end" plus a
caret-to-end after external setContent), which also improves the retry path.
* Cleanup
* Cleanup unnecessary code
* Test CreateTransformMenu Metabot item without mocking
Render the real CreateTransformMenu + Metabot via the shared metabot test
setup so the menu item, permission gating, and prompt seeding are exercised
end to end instead of through stubbed hooks.
Bump babashka postgresql pod to 0.1.6 so mage runs on Apple Silicon
The 0.1.0 pod manifest has no macos-aarch64 entry -- only macos-amd64.
Babashka downloads that x86_64 artifact and caches it under the aarch64 path
anyway, so on an Apple Silicon machine without Rosetta every mage command dies
before doing any work:
Cannot run program ".../mac_os_x/aarch64/pod-babashka-postgresql":
Exec failed, error: 86 (Bad CPU type in executable)
Clearing the pod cache does not help -- the re-download is the same x86_64
binary. 0.1.5 is the first release with a real macos-aarch64 artifact; this
bumps to the current 0.1.6, which resolves to a native arm64 executable.
0.1.6 keeps every platform 0.1.0 shipped (macos-amd64, linux-amd64,
windows-amd64) and adds macos-aarch64 and linux-aarch64, so no one loses
coverage. `pg/execute!` is the only pod fn used anywhere in the repo, and its
signature is unchanged.
Verified on arm64: the cached binary is now `Mach-O 64-bit executable arm64`,
`mage.bot.setup-worktree` loads, and `./bin/mage fix-kondo-ratchets` runs to
completion.
Claude-Session: https://claude.ai/code/session_01X5dVcVX59aLonqqsYsqU65
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* Restore navigation blocking on the v7 engine via a blocking history
* Render the v7 Link on the v7 engine so links navigate in modals
* Scope v7 leave hooks to their route so within-route navigation is allowed
* Don't let a v7 Link with no destination navigate on click
* Type the app Link's `to` as optional to match the runtime contract
* Apply activeClassName on the v7 engine via NavLink
* Hide settings tab in TagEditorSidebar if there is no tags
* Don't crash query metadata fetch on un-normalizable queries
batch-fetch-query-metadata filtered empty queries before normalization,
but normalize-query returns {} when a query can't be normalized (e.g. a
native query with an incomplete field-filter/table template tag). That {}
then failed batch-fetch-query-metadata*'s strict ::query schema, 400ing the
whole request and tripping the query editor error boundary. Filter empties
after normalization too so an unnormalizable query just yields no metadata.
* Hide template-tag 'Try it' examples in transforms editor
The template-tag help sidebar's 'Try it' buttons load an example query
against the Sample Database. Transforms can't target the Sample Database,
so clicking 'Try it' switched the transform to an invalid database and left
the editor in a broken state. Add a canUseSampleDatabase QueryEditor option
(default on) that transforms set to false, which withholds the sample
database id so TagEditorHelp hides the 'Try it' buttons.
* Add regression tests for transform variables sidebar fixes
- batch-fetch-query-metadata skips queries that normalize to {} instead of
400ing the whole request (the P1 crash)
- TagEditorHelp hides the 'Try it' examples when no sample database is available
- TagEditorSidebar hides the Settings/Help tabs until the query has a variable
* Address PR review feedback
- Simplify batch-fetch-query-metadata with (keep (comp not-empty normalize-query))
- Replace the hidden-tabs settings behavior with an empty state in the Settings
tab (no more layout shift / auto-switch to Help); use the shared EmptyState
- Make sampleDatabaseId a required, DatabaseId-typed setup option
- Add a NativeQuerySidebar test asserting 'Try it' is hidden for transforms
(canUseSampleDatabase=false) and shown otherwise
- Re-add the getEditorOptions canUseSampleDatabase assertion
* split use-metabase-query.unit.spec.tsx into multiple files
* split test fixtures from setup.tsx
* move useMetabaseQuery tests into tests folder
* rename useMetabaseQuery hook test
* remove unused query test fixture fields
* rename mock dataset query helper
* rename bundle exports
* chore: tiny formatting changes
* test: rewrite stale query results test
* test: extract public api tests to type and runtime tests
* Convert the Table visualization from a class to a function component
* Stabilize the Table question so async metadata updates don't rebuild columns mid-hover
* Type useSyncedQuestion via VisualizationProps so the SDK d.ts doesn't leak internal types
* Author route paths in v7 syntax, shim to v3 in the facade
* Add use-v7-router flag and frontend engine reader
* Add flag-gated react-router v7 engine (provider, tree mapper, bridge)
* Re-point redux navigation and location to the v7 router
* Support both router engines in renderWithProviders
* Switch the app mount to the v7 router behind the flag
* Point setRouteLeaveHook no-op comment at DEV-2374
* Revoke schema-level grants when destroying Redshift workspace isolation
destroy-workspace-isolation! discovered schemas to revoke via
svv_relation_privileges, which only surfaces relation-level grants: a granted
input schema with no tables (or whose tables were dropped after provisioning)
kept its schema-level USAGE grant, and DROP USER then failed with 'user ...
cannot be dropped because the user has a privilege on some object'. Union in
svv_schema_privileges so schema-only grants are revoked too.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* Rename workspace+det to workspace in the new destroy test
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* Run Redshift workspace teardown statements individually, not as a JDBC batch
A failure now surfaces as the plain server error instead of the
BatchUpdateException wrapper ('Batch entry N ... Call getNextException to see
other errors in the batch.'); the transaction still aborts as a whole on the
first error.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* Revert "Run Redshift workspace teardown statements individually, not as a JDBC batch"
This reverts commit 4bf2d241b5.
* Unwrap BatchUpdateException in Redshift workspace teardown
Keep the batch (there can be many granted schemas, two revokes each) but
rethrow .getNextException so failures read as the plain server error instead
of 'Batch entry N ... Call getNextException to see other errors in the batch.'
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* remove comment
* Unwrap BatchUpdateException across all workspace-isolation batch sites
New driver.u/batch-exception util returns the per-statement exception hiding
under .getNextException (or the throwable unchanged). Applied to every
workspace init/grant/destroy executeBatch site (h2, mysql, postgres,
clickhouse, redshift) — init sites unwrap before scrub-exceptions — so
failures read as the plain server error instead of 'Batch entry N ... Call
getNextException to see other errors in the batch.'
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* Keep the batch exception on init-workspace-isolation! failures
The init statements carry secrets and pgjdbc echoes the aborted statement in
the BatchUpdateException message — scrub-exceptions redacts it, and the
GHY-3660 test (init-workspace-isolation-never-exposes-password-test) relies on
that redacted CREATE USER echo as proof the run reached the statement.
Unwrapping to the inner server error discarded the echo. grant/destroy keep
the unwrap: their statements carry no secrets and the plain server error is
the better message there.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* Unwrap the batch envelope on init-workspace-isolation! too
All workspace batch sites now rethrow the underlying per-statement exception;
init scrubs the unwrapped exception so passwords stay out of the message. The
GHY-3660 test now proves the flow reached CREATE USER by asserting the
server's permission error instead of the redacted batch echo (which no longer
exists).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* Test batch-envelope removal for grant! and password scrubbing after unwrap
- driver-level: a failed grant-workspace-read-access! must not surface a
BatchUpdateException (h2 runs locally; pg/mysql/clickhouse in driver CI;
non-batching impls pass trivially)
- unit: scrub-exceptions composed with batch-exception redacts the password
in the unwrapped per-statement message
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* Batch init/grant/destroy on every SQL workspace driver
Redshift grant, and all three Snowflake / SQL Server workspace methods, now
run their statements as JDBC batches like the other drivers — one round trip,
per-statement exceptions unwrapped via driver.u/batch-exception (init unwraps
before scrub-exceptions). Redshift grants commit before grant! returns, so a
subsequent describe-database from another connection still sees the
privileges; the destroy pre-loop for foreign-grantor default-priv rows stays
per-statement on purpose (each must survive the others failing).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* Fold Redshift foreign-grantor revokes into the destroy batch
The per-statement autocommit pre-loop is gone: one transactional batch now
covers the FOR USER default-priv revokes, relation/schema revokes, and the
drops. A stale discovery row fails the whole destroy, which is fine — the
workspace layer records the failure and the deprovision retry re-discovers
fresh state.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* Escape workspace passwords in DDL like every other interpolated value
CREATE/ALTER USER statements interpolated the generated password raw into the
SQL string on postgres, redshift, h2, clickhouse, and snowflake (mysql and
sqlserver already escaped). Escape with sql.u/escape-sql :ansi (quote
doubling, valid on all five engines) and scrub both the raw and escaped forms
from failure messages.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* Unwrap batch exceptions from all JDBC connectors; fix sqlserver batch var scope
The grant-envelope test caught two real issues in driver CI:
- MariaDB's connector chains the per-statement exception via .getCause and
mssql-jdbc chains nothing at all, so batch-exception now falls back to the
cause and, failing that, rebuilds a plain SQLException with the same
message/SQLState/error code.
- mssql-jdbc executes a Statement batch in one T-SQL scope, so the two
sqlserver destroy statements both DECLAREing @sql collided; the variable
names are now unique.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* Type-hint the rebuilt SQLException ctor to avoid reflection
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* Return the batch exception as is when nothing is chained
mssql-jdbc's BatchUpdateException message is already the plain server error —
only pgjdbc adds the 'Batch entry N ... getNextException' wording. So the
unwrap falls back to the original exception instead of synthesizing a
SQLException, and the driver test asserts the envelope wording stays out of
the message rather than checking the exception class.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* Interpolate the escaped password in Snowflake workspace user DDL
CREATE/ALTER USER spliced the raw password while every other workspace
driver uses the ANSI-escaped value; the escaped form was already
computed and scrubbed here.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
A STRUCT column's INFORMATION_SCHEMA data_type spells out its entire
nested schema, and the COLUMN_FIELD_PATHS LEFT JOIN repeats it on every
nested-leaf row -- O(n^2) bytes for a column with n leaves. Dynamic-key
STRUCTs (e.g. Cloud Functions log-sink tables whose JSON payloads use
user IDs as keys) reach thousands of leaves with ~270KB type strings;
one such table produced ~4,000 rows x 271KB = 1.09GB buffered in
sync-fields!'s partition-by, OOMing the instance mid-sync.
Only short type-string prefixes are ever consumed
(raw-type->database+base-type dispatches on prefixes up to 10 chars and
stores at most RANGE<TIMESTAMP> verbatim), so SUBSTR(..., 1, 200) in the
query -- applied server-side, before the wire -- is behaviorally
identical while collapsing response size, parse cost, and the buffered
partition (1.09GB -> ~2.8MB for the table above).
The test needs no dataset or network: it captures the compiled SQL via
*process-native* and asserts both data_type columns are truncated.
* remove the last test file from the legacy structure
* convert test table into format of newer tests
* remove test for undefined since this isn't a valid type for RowValue
Update E2E test timings with averaged data
Collected timing data from 7 days of test runs.
All specs with timing data were included (simple averages).
🤖 Generated automatically by update-e2e-timings workflow
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: GitHub Actions <actions@github.com>
* migrate TokenField
* fix RemappedOption type for Autocomplete ComboboxStringItem
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* address RecipientPicker review: memoize maps, share lookup/label helpers
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* fix single-value search: accept top match on Enter
Autocomplete only completes on Enter when an option is highlighted, but
search options load async so none is. Restore old TokenField behavior via
onKeyDown.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* fix letters swallowing
* addressing the PR's comments
* added missing tests
* introduce Autocomplete + refactoring
* apply filter by clicking on enter
* fix test
* refactor autocomplete
* refactor autocomplete
* fix edge case + added unit tests
* fix type
* fix test
* fix test
* get rid of lint suppression + allow autoselect on open for multiautocomplete
* revert autoselect destructuring, restore eslint suppression (tests failing)
* explain exhaustive-deps suppression in MultiAutocomplete effect
---------
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* Remove withRouter from production components
* Delete the public withRouter export
* Type the route-leave hook for PlainRoute and drop the route casts
* Add element/Outlet + useRoute support to the router facade
* Convert root route tree to element/Outlet
* Convert admin route tree to element/Outlet
* Convert data-studio, models and metrics route trees to element/Outlet
* Convert transforms, account, monitor and public route trees to element/Outlet
* Convert enterprise route trees to element/Outlet and drop dead library segment routes
* Convert tenants admin route tree to element/Outlet
* Convert redirect, createEntityIdRedirect and modalRoute to element
* Ban component= on routes with eslint
* Drop component= from the facade Route type and convert remaining routes
* Drop the component= eslint rule, the facade type now enforces it
* Remove the unused withOutlet helper
* Assert the required tenantId route param instead of coercing to NaN
* Bridge element={null} to a nothing-rendering component, matching v7
* Memoize router context value so element routes re-render once per navigation
* Convert data_apps and impersonation routes merged from master to element
* initial proof-of-concept for data apps
POC 2
POC 3
POC 4
Revert "POC 4"
This reverts commit 1f62e574dc.
POC 5
POC 6
* Expose all SDK components to child realm
* Setup app publishing and serving
* Refactor distortion helpers
* Adjust skill
* Add useQuestionQuery SDK hook (#74900)
Co-authored-by: Timofei <timofei@metabase.com>
* Mock METABASE_EMBEDDING_SDK_BUNDLE for data app sdk hooks/components
* Adjust skill for TS project
* Update skill to clearly mention TS
* Add blocked APIs list to skill
* Prototype schema exports and typed hooks for custom data apps (#74978)
* add prototype for schema generation
* add formatting for generated schemas
* add skill file
* useQuestionQuery should take only ID based on @sanex3339 feedback
* add segments, measures, tables and useMetabaseQuery hook
* Add useMetabaseQuery to endowments
* Fix queryDataset
---------
Co-authored-by: Timofei <timofei@metabase.com>
* Add ability to update app bundle
* Simplify hook
* Add spinner instruction
* Mode Data Apps to OSS
* Instruct to not not use blocked event handlers
* Improve API, add styles isolation, add router components (#75418)
* Prepare skills for data app template + adjust router components (#75490)
* Fixes after rebase
* Move migration to v63
* Move migration to v63 #2
* Fix config.edn
* Fix BE tests
* Fix team name
* Fix duplicated config
* Adjust version text in skill
* Fix data app bundle uploading after legacy client removal
* Remove `actions` from config edn
* Fix failing test
* Fix failing test #2
* Add custom actions for data apps (#75627)
* Fix imports
* Fix type annotations
* Adjust migration
* Adjust permissions
* Fixed iframe security
* Data Apps publishing via remote repository POC (#75702)
* Update gitignore with data-app.html
* [PoC] Type safety and modularity improvements for custom data apps (#75381)
* allow separate libraries in schemas
* add metric type checking
* add collections parameter
* check that segments and measures belongs to table in useMetabaseQuery
* require less type information for measures
* require less type information for segments
* use metric dataset endpoint when querying measures in metrics
* add validation for breakout buckets
* fix filter operation type scoping
* expose filter type helper for nicer autocompletions
* update skill file
* add breakout schema helper for better type errors
* update breakout schema helper to accept binning and bucket
* move fields that are not used in compile and runtime to comments to shorten type errors
* enforce data truthfulness in skill
* enforce semantic layer richness in skill
* handle dimension key collisions mapped from many tables
* update skill description
* prevent hallucinated measures and breakouts
* fix field serialization
* add guidance on using filter matcher
* add skill guidance on long results and units
* accept query object in StaticQuestion and InteractiveQuestion
* extract use-metabase-query to multiple files
* update skill file to use sdk question by default
* update skill file to avoid hallucinated derived calculations
* add new functions into sandbox
* kondo and cljfmt lint fixes
* update CSP rules to allow style tags for data apps
* address review feedback on scoping and metric filters
* address code review feedback
* Add aggregations support for breakouts in query hooks (#75723)
* add aggregations support
* add additional aggregation helpers
* Add metrics support in useMetabaseQueryObject hook (#75728)
* add metric queries
* indicate metric support in skill file for useMetabaseQueryObject
* add new version for release
* fix query dataset tests
* update release workflow temporarily
* update skill description
* update skill to use more generic examples
* hotfix: allow esm usage in vite dev mode
* skill file update for table caveat
* update sandbox
* fix type errors in useMetabaseQuery
* add v0.63.2 to release workflow
* update skill file to use the 63-data-apps version
* Move data apps to the EE directory and under `data-apps` token feature (#75871)
* remove skill file alias
* Add error handling for data apps
* Add spinner while iframe is loading
* Add OSS/no token upsell banner placeholder for data apps (#75893)
* ensure a fixed height is set if container has height
* Allow including everything in data and metric library in schema endpoint (#75944)
* add filter for including library and data collections
* address review on skill file
* remove id field from rendered comment
* remove useQuestionQuery and useMetricQuery hook
* Improve env files lookup in skills (#75955)
* Allow `unsafe-eval` for data app iframe (#76004)
* Fix lint
* Fixes after rebase
* Fix duplicated test
* Improve Providers and typesafety (#75934)
* chore: reformat single expression functions to arrow
* Use entity id in questionCollections when generating typed schemas (#76230)
* allow using entity ids for question collection filter
* guide the agent towards using entity ids
* add failing test for invalid entity id
* Make custom viz work in data apps (#76237)
* Allow blob: images in data-app CSP (EMB-1923)
SDK custom visualizations load their icon assets via blob: URLs (to avoid
the cross-origin auth issue), but the /embed/data-app/ route allowed
'unsafe-eval' without the blob: scheme in img-src, so icons were blocked
under a production CSP. Add blob: to img-src scoped to the data-app iframe
route only; the main app img-src stays strict.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* EMB-1880: support for `providerProps` in data apps
---------
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* Reduce schema bloat in dimensions of generated metrics when many tables are involved (#76003)
* Add useQuestionQuery SDK hook (#74900)
Co-authored-by: Timofei <timofei@metabase.com>
* Prototype schema exports and typed hooks for custom data apps (#74978)
* add prototype for schema generation
* add formatting for generated schemas
* add skill file
* useQuestionQuery should take only ID based on @sanex3339 feedback
* add segments, measures, tables and useMetabaseQuery hook
* Add useMetabaseQuery to endowments
* Fix queryDataset
---------
Co-authored-by: Timofei <timofei@metabase.com>
* [PoC] Type safety and modularity improvements for custom data apps (#75381)
* allow separate libraries in schemas
* add metric type checking
* add collections parameter
* check that segments and measures belongs to table in useMetabaseQuery
* require less type information for measures
* require less type information for segments
* use metric dataset endpoint when querying measures in metrics
* add validation for breakout buckets
* fix filter operation type scoping
* expose filter type helper for nicer autocompletions
* update skill file
* add breakout schema helper for better type errors
* update breakout schema helper to accept binning and bucket
* move fields that are not used in compile and runtime to comments to shorten type errors
* enforce data truthfulness in skill
* enforce semantic layer richness in skill
* handle dimension key collisions mapped from many tables
* update skill description
* prevent hallucinated measures and breakouts
* fix field serialization
* add guidance on using filter matcher
* add skill guidance on long results and units
* accept query object in StaticQuestion and InteractiveQuestion
* extract use-metabase-query to multiple files
* update skill file to use sdk question by default
* update skill file to avoid hallucinated derived calculations
* add new functions into sandbox
* kondo and cljfmt lint fixes
* update CSP rules to allow style tags for data apps
* address review feedback on scoping and metric filters
* address code review feedback
* Add aggregations support for breakouts in query hooks (#75723)
* add aggregations support
* add additional aggregation helpers
* Add metrics support in useMetabaseQueryObject hook (#75728)
* add metric queries
* indicate metric support in skill file for useMetabaseQueryObject
* add new version for release
* fix query dataset tests
* remove useQuestionQuery and useMetricQuery hook
* Data Apps publishing via remote repository POC (#75702)
* avoid duplicating dimension fields
namespace dimensions by table
add dimension id to mapping
pick from table fields for only allowed dimensions
fix incorrect type guards
allow querying metric using question as source
remove createMetabaseQuery from public interface
add sourceFieldId to metric
* doc generation type fixes
---------
Co-authored-by: Timofei <timofei@metabase.com>
* Add allowed hosts handling for data apps (#76094)
* Update version
* Properly include data app changes in `pull changes` modal (#76294)
* Move app template into main repo, handle skill versioning flow (#76288)
* feat(data-apps): allow passing down a card with viz settings (#75993)
support card prop in sdk question component
* Rename data app env variables and read them only in dev mode (#76604)
* Add custom data apps layout with sliding toolbar (#76600)
* Remove unused helper
* Dev env Sandboxing, Human Readable Errors and Copy Helper (#76597)
* Revert wrong rebase changes
* Improve creating filtering interfaces in custom data apps (#76401)
* update skill files to guide filter usages
* update skill file to prevent filtering mistakes
* move filtering ui patterns into a separate file
* do not treat mb_replace_me as valid creds
* use combobox for filters
* pass title false to sdk question
* combobox trigger before typing
* Add missing exported types
* Apply visualizations and visualization settings from SDK package (#76659)
* ask the agent to pull possible visualizations
* ask the agent to use card prop
* teach the agent when to use viz settings
* Support query-only SDK cards
* Remove cross references between data apps skills (#76661)
Remove data app skill cross references
* Move Data App Dev Vite App setup into SDK package (#76695)
* Don't navigate on ad-hoc question chart clicks (#76751)
* Fix sandboxing crashes and update skill file mistakes (#76858)
Fix data app sandboxing and skill guidance
* Block form submissions in data apps via CSP (#76882)
* Fix SDK theming hover text colors in generated data apps (#77083)
update guidance on colors
* Remove SDK question query prop for data apps use cases (#77080)
* Remove SDK question query prop
* update tests to use card prop instead
* Adopt Lib.createTestQuery for table sources when querying in data apps (#75957)
* Add base metabase-lib query implementation
* switch to Lib.createTestQuery DSL
* Load query metadata at runtime for metabase-lib instead of hard-coding (#76874)
* load query metadata dynamically
* accept id-only source references
* Wait for SDK login before resolving query metadata
* refactor: use useAsyncFn to handle states
* pass source-name to createTestQuery
* Add metric aggregations for useMetabaseQuery via Lib.createTestQuery (#76947)
* add metric sources for useMetabaseQuery via Lib.createTestQuery
* use MetabaseQueryObject instead of opaque type marker
* Fix metric schema and join source validation
* add metric library filtering to skill files
* make metric an aggregation expression instead of join sources
* mark tableId as never in joined fields
* export public MetabaseQueryObject type
* use camelCase for source name and field id
* Add initial data app e2e setup (#77101)
* Add sorting and fields to useMetabaseQuery via Lib.createTestQuery (#76966)
* add sorting, limiting and fields to useMetabaseQuery
* refactor: cleanup sorting and aggregations
* Fix metric order by validation
* Support saved questions for useMetabaseQuery via Lib.createTestQuery (#77073)
* add saved questions as useMetabaseQuery input
* remove extra operators for saved cards
* Add unit fe/be tests for data apps (#76979)
* Fix lint
* Use modern api in query-dataset.ts
* Fix BG color
* Fix execute-action tests
* Update SDK-data-apps version
* Move migrations to v64 folder
* Adjust router components to use metabase/router
* Allow opening a data app for any signed user
* Rename data app skills (#77266)
* Update DataAppLayout.tsx `go back` button logic
* Harden data apps skills and action validation errors (#77261)
* harden data apps skills and actions error validation
* do not display models as questions in data apps
* clarify action result types
* Agents should report errors when models, actions or questions cannot be schema generated (#77260)
* return an error if models or saved questions are invalid
* return an error if actions are invalid
* update skill file guidance on model generation
* fix model boundary
* Add data apps admin UI + routing/sandboxing/isolation e2e (#77271)
* Rename data-apps paths to /apps (#77298)
* Include all action-included models in the typed schema when a mutation is specified (#77291)
* add includeModels=true flag to include all models
* omit model columns and models without actions
* remove extra metadata
* Adjust admin UI for Data Apps (#77420)
* Add more e2e/unit tests for Data Apps (#77524)
* Update bun.lock
* Fix errors after rebase
* Parallelize dts rollup (#77663)
* revert EMB-1840 measures support in metric dataset endpoint
* chore: add data app module (#77705)
* test ci
* apply code review suggestion
* Setup separate entries for client and node dev data app entries (#77687)
* Fix type casts lint errors (#77696)
* Adjust data apps setup + add proper analytics setup (#77604)
* Extract common and scope helpers + tests for typed schema module (#77403)
* refactor: extract common and query param helpers
* refactor: use kebab-case for query params
* refactor: cleanup schema mapping
ci: fix module config
* refactor: rename schema scope helpers to scope.clj
* refactor: extract scope_test.clj
* Extract typed schema rendering helpers to render.clj (#77703)
* extract schema render function
* move runtime compact test out
* remove javascript and json endpoints from schema generator
* refactor: cleanup javascript.clj and render.clj
* Extract typed schema generators for tables and questions (#77729)
* refactor: extract schema generator modules
* remove question-details from model-schemas
* fix: batch queries in table and question schema generator
* remove N+1 in typed schema field table lookup
* Extract typed schema generators for models and actions (#77739)
* extract model schema generator
* batch model action querying to avoid n+1 queries
* Extract typed schema generator for metrics (#77806)
* Extract typed schema generators for models and actions (#77739)
* extract model schema generator
* batch model action querying to avoid n+1 queries
* extract metric schema generator
* test: simplify fields in metric_test fixtures
* refactor: clean up metric schema generation
* remove n+1 from metric field dimension read
* disable default temporal dimension breakout
* surface errors if metric schema cannot be built
* revert metrics/api.clj to master to remove outdated comments
* Fix routes
* Take a data app's slug from its directory name (#77688)
* Merge data-apps specific migrations into one
* ci: fix failing backend test and cljfmt lint
* Add a beta badge in data apps admin page heading (#77877)
marks the data app as beta
* Revert "Merge data-apps specific migrations into one"
This reverts commit 383a5fa28d.
* Revert "Revert "Merge data-apps specific migrations into one""
This reverts commit 948d904b62.
* ci: actionlint fix
* Fix data apps route
* Use getSubpathSafeUrl in attachIframeUrlMirror
* Fix comment
* Do not send apps with sync error for token check
* Add data app listing sidebar section for non-admins (#77977)
* show data apps in sidebar for non-admins
* add app icon to data apps
* use available=true to filter no sync error and enabled apps
* fix navbar keys in data apps
* test: use non-admin user to assert non-admin response
* fix toucan query in data apps listing endpoint
* re-export entity detail helpers and result column to metabot core
* open data apps in new tab from the sidebar
* Prevent race conditions in responses of useMetabaseQuery hook (#78023)
prevent stale responses in useMetabaseQuery
* ci: remove tools.util from module declaration
* Merge migrations into a single one
* reset release-embedding-sdk.yml to master
---------
Co-authored-by: Phoomparin Mano <poom@metabase.com>
Co-authored-by: Phoomparin Mano <phoomparin@gmail.com>
Co-authored-by: Nicolò Pretto <info@npretto.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Some cleanup of some false alarms I encountered in test environments while working on a different issue. Destination databases (those with `router_database_id` set) are reached only through their router: metadata, sync, permissions, etc. Only execution goes against the destination's connection. A `data_permissions` row pointing at a destination is never consulted -- it's already actively stripped.
There are test utilities that helpfully create in-test permissions for all databases created, so in a test environment it would look exactly like the invariant was being violated, but that was only because the test utilities deviated from production behavior. This brings the test behaviors into line with production (no permissions get written).
There should be no change in production behavior in this PR, but as a bonus, I figured that a safety guard filtering rows out on reads is not as good as filtering them out on reads *and* failing loudly on writes. I also added guards that a normal database can't be turned into a destination database, which was a behavior tests were relying on for easier setup (but again, the tests then deviated once more from production behavior).
Since 4.28 Liquibase ships an opt-out analytics feature that, on the first
migration run, fetches config from config-endpoint.liquibase.com and sends
anonymized usage events.
* Parse :absolute-datetime string literals in repr resolve
Metabot's representations repair pass wraps bare ISO `between` bounds as
`[:absolute-datetime {} "2024-01-01" :day]` to satisfy the lib `:between`
temporal schema, leaving the literal as a string. That form is schema-valid
(so it passes construct_notebook_query validation), but the JVM query
processor expects a real java.time value — optimize-temporal-filters throws
on a bare string, and the QP's own wrap-value-literals only parses temporal
strings that arrive as raw values, not ones already wrapped in
:absolute-datetime.
Parse those string literals into java.time values in resolve-query, where the
portable repr form becomes QP-ready pMBQL, alongside the existing
annotate-field-types pass.
* Address review: tighten docstrings, use update over assoc
- parseable-temporal-string?: contract-first two-line docstring
- parse-absolute-datetime-literals: move the "why" into inline comments at
the walk, keep the docstring to contract + idempotency
- use (update node 2 u.date/parse) instead of assoc
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* Address review: year/year-month, fail-fast, export round-trip
- parse year/year-month :absolute-datetime literals too (u.date/parse maps
them to LocalDate), so they are QP-ready instead of crashing later
- fail fast on a structurally-ISO but invalid literal (e.g. "2024-13-45"):
re-throw as an :agent-error? ex-info so the LLM gets a clean message
- export: format java.time values back to ISO-8601 strings in
portable-repr-form so resolve -> export round-trips to the documented
string form
- tests: per-shape parsing (date/datetime/offset/year/year-month),
idempotency, invalid-literal, and export round-trip
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* Move :absolute-datetime string parsing to the QP layer
Revert the resolve-layer parsing and instead teach wrap-value-literals to
parse string-valued :absolute-datetime clauses, not just raw values. The
repair pass wraps bare `between` bounds as `[:absolute-datetime {} "2024-01-01"
:day]`; wrap-value-literals previously only parsed temporal strings that
arrived as raw values, so a pre-wrapped string slipped through and
optimize-temporal-filters threw on the bare string.
The QP is the right layer: wrap-value-literals runs at execution time, where
both the report timezone (bound) and the comparison field's type are
available — so the parse is type- and timezone-aware for free, and the stored
query keeps its portable ISO-string form (no export round-trip to invert).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* Fail fast on malformed :absolute-datetime literals in resolve
Add a resolve-side validation pass that rejects temporal literals which pass
the schema's structural regex but are not real dates (e.g. "2024-13-45"), so
the agent gets an actionable error at construct time instead of a failure at
query execution.
Validation only — it does not coerce. The string -> java.time coercion stays
in the QP's wrap-value-literals, where the field type and report timezone are
available. Accepts every valid :absolute-datetime shape: date, datetime,
offset-datetime, year, and year-month.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* Preserve unit and wrap mixed :between bounds for string :absolute-datetime
When re-parsing a string-valued :absolute-datetime in wrap-value-literals:
- Carry the clause's own temporal unit through instead of discarding it. The
comparison field's bucket still wins, but for an unbucketed field the literal's
unit is now the fallback (rather than :default), so a year/year-month literal
like [:absolute-datetime {} "2024" :year] keeps its bucket instead of silently
collapsing to a single instant.
- Wrap each :between bound independently so a mixed bound (string :absolute-datetime
on one side, :now / :relative-datetime / an already-parsed :absolute-datetime on
the other) still coerces the wrappable side instead of falling through unwrapped.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* Guard comparison literals and tidy :absolute-datetime validation
- wrap-value-literals: guard the field position in the :=/:!=/<…> arms so a
comparison with a wrappable literal on both sides can't bind a literal as the
field and leave it unparsed; it now falls through unchanged.
- resolve: validate :absolute-datetime literals via lib.walk/walk-clauses
(no tree rebuild, shares clause-of-type? detection with the QP) instead of
clojure.walk/postwalk.
- Move cross-subsystem rationale out of the string-valued-absolute-datetime
docstring into an inline comment.
- Add a test covering the two-literal comparison fall-through.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* Parse fieldless :absolute-datetime comparisons and bound bad-literal errors
Wrap a string-valued :absolute-datetime on both sides of a comparison
(no field to wrap against) so each literal is parsed with its own unit
instead of leaving a bare string for later middleware to choke on.
Truncate the untrusted literal before it lands in the agent-facing
exception message and ex-data, so a pathological value can't flood logs.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* Treat :default field unit as unbucketed for :absolute-datetime literal fallback
A temporal field can carry :temporal-unit :default (add-default-temporal-unit
stamps it on :temporal/requires-default-unit drivers, and an LLM can author it
directly). The unit-fallback guard only skipped a nil unit, so a :default field
dropped the literal's own unit and collapsed e.g. [:absolute-datetime {} "2024"
:year] to a single :default-bucketed instant. Treat :default as unbucketed too.
* Validate :absolute-datetime literals with plain walk to keep resolve gate friendly
validate-absolute-datetime-literals walked the query with lib.walk/walk-clauses,
a mu/defn whose input schema validates the whole query against ::lib.schema/query.
Under test instrumentation that threw a raw "Invalid input" error on an otherwise-
malformed query (e.g. an :offset in :expressions), pre-empting the friendlier
query-not-runnable agent error the downstream canRun gate is meant to surface.
Use clojure.walk/postwalk and a bare [:absolute-datetime _ s _] shape check, matching
the sibling annotate-field-types pass, so only the literals are inspected.
* Use match/match-many for :absolute-datetime literal validation, add mixed-between raw-value test
Addresses PR review feedback: swap the manual walk/postwalk in
validate-absolute-datetime-literals for match/match-many, and cover a
:between with a raw (unwrapped) string bound plus a :relative-datetime
bound in wrap_value_literals_test.
---------
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* Show honest "still working" state when a Git sync push stalls (GHY-4132)
A large push can stop reporting progress while the export is still running; the
backend then reports the task as :timed-out with ended_at nil. The push modal
treated that as neither success nor error and stayed frozen on "Exporting
content…" until the page was refreshed.
Render a distinct stalled state instead: drop the frozen progress bar and show
"Still working. No update for N minutes." (internationalized, singular/plural
aware). The counter derives from last_progress_report_at and stays fresh via the
existing 2s current-task poll — no separate ticker.
* Add throttled monotonic progress reporter for remote-sync tasks (GHY-4132)
* Ramp remote-sync export progress across plan + serialize phases (GHY-4132)
* Report remote-sync export commit checkpoint before push (GHY-4132)
* Report remote-sync push progress via JGit ProgressMonitor (GHY-4132)
* Fix throttle, dead 0.66 write, and push liveness in export progress (GHY-4132)
commit-staged! was passed a pre-forced report closure, so every push
ProgressMonitor tick bypassed the 10s throttle and hammered the DB. Force only
the one-shot commit checkpoint (0.8) explicitly in finish-commit!, and let
push ticks go through the throttled reporter.
The forced 0.66 serialize-boundary write ran after commit-staged! returned,
by which point the 0.8 checkpoint (and push ticks) had already advanced the
monotonic reporter past it, so it was silently dropped. Move it to the end of
each export's stage-fn, before the checkpoint/push run.
The push monitor only reported during the exactly-titled "Writing objects"
phase with a known positive total; a non-English JVM locale renames that
title and JGit may report an unknown (zero) total, so the push phase could go
silent for its whole duration and re-trip the stale-progress timeout. Report
on every update tick regardless of phase/total, falling back to the 0.8
push-start fraction — repeated equal values are monotonic no-ops but still
bump last_progress_report_at, throttled to at most once per 10s.
* Use dayjs diff for the stalled-progress minute counter (GHY-4132)
* Expose search's ranked set and page-hydration as separate steps
`search` already ran in two stages: rank the permission-filtered result set
(capped at max-filtered-results), then slice that set to the requested page
and hydrate only those rows. The comment on `search-results` spells out why
the slice comes first — hydration does per-row work, including the expensive
`can_write` check, so it should only touch rows that are actually returned.
Both stages were sealed inside `search`, so a caller holding several ranked
sets had no way to combine them and paginate the combination. Its only option
is to ask `search` for N hydrated pages and merge those, which hydrates N
pages to return one, and — because each page is sliced from its own ranking
before merging — cannot page the merged order correctly at all.
Split the two stages into `ranked-results` and `search-results` and export
both from the module API. `search` is now their composition and behaves
identically; no call site changes.
The motivating caller is metabot's multi-query search, which runs a search per
query and fuses the rankings with RRF. Fusing at the ranked-results seam lets
it hydrate only the page it returns, page the fused order coherently, and
report a total for the fused set.
* Remove comments from search core.clj
Removed comments explaining the purpose of `ranked-results` and `search-results`.
* kick ci
* Convert sandboxes and advanced_permissions plugin modules to TypeScript
* Address review comments: remove typecasting from sandboxes actions and unit spec
@@ -27,9 +27,28 @@ This guide covers Clojure and ClojureScript coding conventions for Metabase. See
- Format using Markdown conventions
- Reference other vars with `[[other-var]]` not backticks
A docstring states the *contract*: what it returns (including when it returns nil), what its
params mean, what it throws versus swallows, and any precondition or invariant the caller must
respect. A params block for non-obvious or optional args is good documentation — length is not
the problem.
These seven things are, and each is a reason to cut or relocate:
| Anti-pattern | Why | Where it goes instead |
| --- | --- | --- |
| **Facts you don't own** — how a *callee*, another namespace, or another repo works internally | Drifts silently; nothing fails when it goes stale | Cut. Name the callee and let its own docstring say it |
| **Said twice** — ns docstring restating its vars', a caller restating its callee's, sibling fns repeating boilerplate | Two copies means one is already wrong | Cut the copy. One fact, one owner |
| **Design rationale** — why this approach, why it lives in this module, what the alternative was | The caller can't act on it | Commit message, or an inline comment on the specific line it constrains |
| **Development history** — "matching master", "the X branch", "currently", "for now", "used to" | Stale by construction | Cut |
| **Instructions to the reader** — "callers should trust this", "don't bother checking", "you should use X" | State the contract; the caller decides | Cut, keeping any actual guarantee as a plain statement |
| **Invented precision** — score bands, thresholds, or tables no code enforces | Reads as spec, isn't | Cut, or make the code enforce it |
| **A note on the wrong var** — describing how overrides behave, on the base/default impl | Nobody reading the override sees it | Move to the overrides it describes |
**Comments:**
-`TODO` format: `;; TODO (Name YYYY-MM-DD) -- description`
- Prefer an inline comment over docstring prose for anything that explains *the body*. Put it on
the line it explains, not at the top of the function.
- Run targeted tests, and lint continuously during development
- Prioritize understanding existing patterns before implementing
- Don't commit changes, leave it for the user to review and make commits
- Minimize comments. Document only non-obvious intent, trade-offs, or constraints that the code itself cannot convey. Do not narrate what the code does — keep code self-descriptive through naming and structure.
@@ -10,23 +10,23 @@ This skill helps you add product analytics (Snowplow) events to track user inter
## Quick Reference
Analytics events in Metabase use Snowplow with typed event schemas. All events must be defined in TypeScript types before use.
Analytics events in Metabase use Snowplow with typed event schemas. Simple events are declared **where they are used** — `trackSimpleEvent` is generic and validates the payload at the call site.
**Key Files:**
-`frontend/src/metabase-types/analytics/event.ts` - Event type definitions
@@ -37,14 +37,16 @@ First check if snapshots exist:
ls e2e/snapshots/default.sql 2>/dev/null &&echo"snapshots exist"||echo"snapshots missing"
```
Always pass the spec path via `--spec`. The runner feeds its arguments to Cypress's CLI parser (`cypress.cli.parseRunArguments`), which silently drops bare positional paths and then runs the entire suite. A leading `run` argument is harmless, but the path itself must follow `--spec`. Sanity-check that the first `Running:` line says `(1 of 1)`.
If snapshots exist, skip regeneration for speed:
```bash
MB_EDITION=oss CYPRESS_VIDEO=falseCYPRESS_RETRIES=0CYPRESS_GUI=falseGENERATE_SNAPSHOTS=false bun test-cypress $ARGUMENTS
MB_EDITION=oss CYPRESS_VIDEO=falseCYPRESS_RETRIES=0CYPRESS_GUI=falseGENERATE_SNAPSHOTS=false bun test-cypress --spec path/to/spec.cy.spec.js
```
If snapshots are missing (first run), let the runner generate them:
```bash
MB_EDITION=oss CYPRESS_VIDEO=falseCYPRESS_RETRIES=0CYPRESS_GUI=false bun test-cypress $ARGUMENTS
MB_EDITION=oss CYPRESS_VIDEO=falseCYPRESS_RETRIES=0CYPRESS_GUI=false bun test-cypress --spec path/to/spec.cy.spec.js
```
When running enterprise tests, replace `MB_EDITION=oss` with `MB_EDITION=ee` in the commands above.
@@ -67,9 +69,11 @@ GREP="test name here" MB_EDITION=oss bun test-cypress --spec path/to/spec.js
(The stress script forwards its arguments to `bun test-cypress`, so the same `--spec` rule applies.)
Set `E2E_STRESS_RUNS=N` for more iterations (default: 5). Stops on first failure and prints screenshot paths. Read those screenshots to analyze the failure.
description: Deciding where frontend code lives — tier model, move mechanics, extension points, side effects, naming, and the traps. Use when moving code between modules, carving new modules, fixing boundary violations, adding a barrel or an endpoint, or reviewing module-shape decisions.
---
# Frontend modularization
The module is the unit of blast radius: dependency direction, test selection,
bundle cones, and review scope all follow module boundaries. Every decision
here serves one goal: **make the file locations tell the truth about who owns
what and who may depend on whom.**
## Hard rules (check these before anything else)
- NEVER add an `export … from` re-export at an old path, and NEVER `export *`
from a barrel. Move the code, codemod every call site, delete the old path,
all in the same PR.
- NEVER import past a module's `index.ts` when that module has
`enforcePublicApi: true`, and never past `metabase/ui`'s index at all. If
the barrel is heavy, the fix is making the module side-effect free (below),
not a deep import.
- NEVER rewrite a consumer to a lower-level idiom to make the linter pass. If
the fix makes the consumer worse, the plan is wrong.
- NEVER give a new module `enforceSharedTiers: false`; new modules ship
enforced with `enforcePublicApi: true` and an explicit `index.ts`.
- NEVER move code without the consumer-tier check (procedure step 2).
- ALWAYS delete a module's `enforceSharedTiers: false` line in the PR that
takes it to zero violations.
- ALWAYS measure `bun run module-boundaries` before and after, and report the
numbers.
- ALWAYS import `dayjs` from `metabase/dayjs`, routing from `metabase/router`,
Mantine from `metabase/ui`, react-redux hooks from `metabase/redux`; the
| side-effect-free directories and their exceptions | `frontend/build/shared/rspack/side-effect-free-modules.js` (`SIDE_EFFECT_FREE_PATHS`, `SIDE_EFFECT_PATHS`) |
| the lint rules that enforce that promise | `metabase/no-module-side-effects`, `metabase/no-base-api-access` in `frontend/lint/eslint-plugin-metabase/rules/` |
| public-api enforcement | `metabase/enforce-module-public-api` rule, driven by `getPublicApiModules()` |
## The model
Tiers: `lib < basic < shared < feature < app`. Imports point downward, never
sideways at feature tier (features may not import features; app and EE may
import anything). The shared tier is subdivided in `shared-tiers.mjs` into
shared-utils (U0…), shared-platform (P0…), and shared-domain, each ordered
into levels. A module imports only *strictly lower* levels of its sub-tier
plus the sub-tiers below; same-level peers are deliberately forbidden, which
is what makes cycles structurally impossible. A domain may hold two seats
when consumers demand it: a low core and a high surface (metabase-lib below
**Primary standard: the [`typescript-write`](../typescript-write/SKILL.md) skill.** Load it first — it defines the authoring rules this review enforces, alongside `frontend/CLAUDE.md` and `docs/developers-guide/frontend.md`.
Adherence to `typescript-write` is the **highest-priority** review dimension: rank any violation of its provisions above all other findings. Treat its **no-`any` hard rule** (no explicit *or* implicit `any` in new code) as **blocking**, and verify it with the LSP rather than by eye.
Adherence to `typescript-write` is the **highest-priority** review dimension: rank any violation of its provisions above all other findings. Treat its **no-`any` hard rule** (no explicit *or* implicit `any` in new code) as **blocking**. Use TypeScript LSP tools to inspect inferred types when available; otherwise rely on type-checking and linting.
Review in this priority order:
1.**Violations of [`typescript-write`](../typescript-write/SKILL.md) provisions** — no-`any`, type tightening, type modeling, null/undefined handling, naming, structure, comments. Highest priority; block on the no-`any` rule.
1.**Violations of [`typescript-write`](../typescript-write/SKILL.md) provisions** — no-`any`, type tightening, type modeling, function signatures, null/undefined handling, naming, structure, comments. Highest priority; block on the no-`any` rule. Apply conditional guidance in context: explain the unsupported type guarantee or concrete readability problem, rather than treating every preference as a blanket ban.
2. Compliance with `frontend/CLAUDE.md`.
3. Readability and maintainability.
4. Appropriate test coverage.
4. Appropriate test coverage. For internal typed callers, avoid requesting tests solely for inputs the type system excludes. External API data, deserialised values, storage and JavaScript callers can violate annotations: test runtime validation and nontrivial assumptions at those boundaries. Types do not replace behavioural, security or data-integrity tests.
## Blind spots — act as the missing reviewer
@@ -31,3 +31,4 @@ These rarely surface in team reviews, so this skill should raise them. They are
- **Security.** Evaluate potential security issues in new code.
- **Bundle size.** Flag new large dependencies, default imports from icon or util libs, and heavy modules imported at route-load time.
- **Analytics.** User-facing flows should emit tracking events. If a PR adds a new flow (button, modal, navigation) without a tracking event, ask whether one is expected.
- **Public API surface** (embedding SDK). Consumers should be able to use public signatures and name types they need to import. Export those types deliberately and document public behaviour, including `@deprecated` for deprecated APIs; a referenced structural type does not automatically need its own named export.
@@ -13,26 +13,34 @@ description: Write TypeScript and JavaScript code following Metabase coding stan
- **New code must not introduce `any`, explicit or implicit.** No `any` annotations, no `as any` / `as unknown as`, no untyped parameters or returns that infer `any`, no implicitly-`any` destructures or array/object literals.
- **Untyped third-party / boundary values** must be typed at the boundary (a declared type, `unknown` + type guard, or a small typed wrapper) — never let `any` propagate inward.
- **Mandatory LSP verification.** After writing or editing any TS/TSX, inspect the changed symbols with the TypeScript Language Server (hover to read inferred types; `goToDefinition` to confirm sources) and run the project type-check.
- **Mandatory type verification.** Before finishing a TS/TSX change, run `bun run type-check-pure`. If TypeScript LSP tools are available, also inspect changed symbols with hover and go-to-definition; otherwise skip LSP check.
## Type tightening
- **Avoid type casts and loose `unknown`** — fix the signature instead.
- **If a function only needs one field of a wide object, accept that field** — not the wide object. The cast often disappears once the signature is right.
- **Reach for `Partial<T>`, `Pick<T, K>`, `Record<K, V>`, and generics** before reaching for a cast.
- **Match dictionary keys to the data.** Use a finite key union when the keys are known. Open dictionaries can use an index signature, `Record<string, T>`, or `Map`; index signatures still constrain values, and changing their spelling to `Record<string, T>` does not make missing-key access safe.
- **Prefer making props/components generic** (`<T>`) when a value flows through unchanged and the caller knows the type.
- **Prefer `unknown` over loose typing** and narrow before use — an `unknown` value forces a guard at the point of use.
- **`satisfies` for object literals** that must conform without widening (config objects, lookup maps, discriminated literals) — better than `: T` (widens) or `as T` (unsafe).
- **Avoid non-null assertions (`!`)**. Prefer a guard, early return, or `?.`. Use `!` only when non-nullness is provably true and localized, with a comment.
- **Guard indexed lookups that may miss.** Arrays and dictionaries can return `undefined` even when the inferred type omits it. Use an iteration form that preserves the key/value relationship, and only assert `keyof` when the runtime keys are known to belong to the declared type.
- **Type guards belong in `frontend/src/metabase-types/guards/`**. Do not redefine them locally.
- **A cast you can't avoid needs a real justification comment.** The `metabase/no-unjustified-type-casts` rule accepts any preceding comment — state the actual reason the cast is safe. NEVER write the legacy `// Unjustified type cast. FIXME` placeholder; it exists only on casts that predated the rule, and copying it sneaks an unjustified cast past the linter. If you can't articulate why the cast is correct, the cast is wrong — fix the types.
- **Keep unavoidable assertions local.** Isolate a repeated or complex assertion behind a helper when that makes its invariant easier to enforce. Test nontrivial runtime assumptions that justify it; a trivial assertion does not automatically need a new helper or test. Do not weaken a public signature just to silence implementation errors.
## Type modeling
- **Reuse existing types; don't re-declare them.** Use canonical IDs and domain entity types from `metabase-types/api` (`FieldId`, `TableId`, `ConcreteTableId`, `SchemaName`, …) and key data structures by them (`new Map<ConcreteTableId, …>()`). Don't duplicate generated/API types — compose or derive (`Pick`, `Omit`, indexed access `SomeType["field"]`, `ReturnType`).
- **Use generics to allow TypeScript to infer correct types** when creating functions and components that need to be reusable and type-safe. Don't hesitate to introduce complex generics if they allow to derive types automatically instead of manual narrowing.
- **Generics must make promises the implementation can keep.** A caller-selected `get<T>(): T` must not disguise an unchecked assertion about external data. Return `unknown` and validate, or accept a validator that establishes `T`. A factory such as `function empty<T>(): T[] { return []; }` is valid; judge the implementation, not how often `T` appears in the signature.
- **Prefer an honest type over false precision.** Use generics when they express a real relationship. If a complex type cannot model the behaviour accurately, choose a simpler type or `unknown` with narrowing instead of asserting an unsupported guarantee.
- **Model the actual data contract; keep types narrow.** Optional `field?: T` for a key that may be absent, `field: T | undefined` only when the key is always present but the value may be undefined, `| null` for explicit API nulls. Prefer domain unions over broad `string` / `number` / loose `Record`.
- **Refer to API implementation** when defining or refining types to ensure they match the actual data structure. When considering a type cast, first consider if the type should be refined to match the actual data structure.
- **Optionality must reflect absence.** Keep required fields required and optional fields optional. Normalise input when the application has a meaningful default, not merely to remove a type error. Preserve the actual wire shape in API types.
- **Represent related absence together when modelling internal state.** If several fields exist or disappear together, consider a nullable containing object or a discriminated union. Do not reshape a raw API declaration unless the data actually has that shape.
- **Prefer explicit special states when designing a format.** Use nullability or named union variants when sentinel values such as `-1` hide meaning. Preserve established protocol sentinel values unless the behaviour is deliberately changed, or normalise them at an explicit boundary.
- **Discriminated unions for variant state, with exhaustive checks.** Model "one of N shapes" as a union with a literal discriminant rather than a bag of optional fields, and exhaust it with ts-pattern's `.exhaustive()` so adding a variant becomes a compile error:
```ts
import { match } from "ts-pattern";
@@ -44,20 +52,30 @@ description: Write TypeScript and JavaScript code following Metabase coding stan
.exhaustive(); // Compile-time guarantee all cases handled
```
- **Derive union types from constants** (`as const` + `typeof`/`keyof`) so the type and the values can't drift.
- **`readonly` / immutability where mutation isn't intended** — component props, shared constants, exported config. Prefer `readonly T[]` / `ReadonlyArray<T>` for inputs you don't mutate.
- **`readonly` / immutability where mutation isn't intended** — component props, shared constants, exported config, and unmutated parameters. Prefer `readonly T[]` / `ReadonlyArray<T>` for inputs you don't mutate. Functions that mutate caller-owned data should make that behaviour explicit.
- **Construct complete, well-typed objects where practical.** Prefer an object expression when it avoids partially initialised state or assertions. Incremental construction is fine when it is clearer and maintains the type's invariants.
- **Treat `metabase-lib` opaque types as black boxes.** Use `metabase-lib` functions to work with types such as `Lib.Query`, rather than casting into their internal representation.
- **Type async and error states explicitly** (a discriminated union or the data-layer's typed result) — never leave loading/error/empty implicit.
## Function signatures
- **Make public contracts explicit where it improves stability and clarity.** Annotate parameters and return types at shared boundaries when useful; let local values infer. Use `satisfies` or an annotation when a declaration needs an explicit shape check.
- **Accept the inputs the operation supports and return the most precise honest result.** Narrow avoidable uncertainty inside the function, but preserve meaningful nullability and union variants in its return type.
- **Use named options when positional arguments are easy to confuse.** Consecutive arguments with the same type are a useful warning sign, not an automatic requirement to rewrite a clear API.
- **Use `async`/`await` when it clarifies control flow or error handling.** Returning an existing promise directly is also valid; a Promise return type alone does not require adding `async`.
## Null and undefined
- **Narrow at the source**. If a value is optional only in a corner case, don't thread `undefined` through every layer — guard at the producer.
- **Sensible defaults for optional values**. Use `?.` and `??` at the consumer.
- **Lists should be filtered** before being used in a map or other iteration.
- **Narrow nullable list elements when the operation requires present values.** Filter with a type guard when missing entries should be discarded; preserve them when their absence or position carries meaning.
- **Avoid non-strict null comparisons** (`X != null`) when `X` can never be `null` — use a strict check or narrow the type. Use `checkNotNull` where necessary.
- **Check actual nullability against API implementation**. Find the API endpoint implementation and check if the field can actually be null.
## Naming
- **Names describe the entity, not the mechanism**. A name must reflect what the value holds.
- **Use domain vocabulary and include units where useful** (`timeoutMs`, `widthPx`, `temperatureC`). Prefer a more specific name when `Info`, `Data`, or `Entity` obscures the meaning, while keeping established terminology when it is clear.
- **Align sibling concepts**: keep verb conventions consistent across a related API.
- **No names that encode implementation history** rather than current meaning. Suffixes like `Base`, `New`, `Old`, `Initial` need a real semantic distinction, otherwise drop them.
- **Avoid cryptic identifiers** (`v`, `n`, `$n`) for domain values; short names are fine only in tiny conventional contexts (loop index `i`, coordinates `x`/`y`, generic params `T`/`K`/`V`).
@@ -74,10 +92,6 @@ description: Write TypeScript and JavaScript code following Metabase coding stan
- **No comments by default**. Well-named identifiers carry the `what`.
- **Comments should be concise**. Add a short, concise comment only when the `why` is non-obvious: a workaround, a hidden invariant, a subtle ordering constraint, a clever reduction. Never document the actual implementation, focus on the intent and the why.
## TypeScript Migration
**When touching existing JavaScript files, propose to convert them to TypeScript first**. Create a separate PR for the conversion, then implement the changes.
## Verify before done
- **Run the project type-check** when finished (see the shared TypeScript commands above).
{clojure.core/print{:message"Use metabase.util.log instead of clojure.core/print"}
{metabase-enterprise.entity-retrieval.core/search-unfiltered{:message"Permission bypass risk: library-index hits are unfiltered — resolve each entity and drop what the current user can't read"}
metabase.entity-retrieval.core/search-unfiltered{:message"Permission bypass risk: library-index hits are unfiltered — resolve each entity and drop what the current user can't read"}
metabase.entity-retrieval.mirror/search-unfiltered{:message"Permission bypass risk: library-index hits are unfiltered — resolve each entity and drop what the current user can't read"}
clojure.core/print{:message"Use metabase.util.log instead of clojure.core/print"}
clojure.core/println{:message"Use metabase.util.log instead of clojure.core/println"}
clojure.core/printf{:message"Use metabase.util.log instead of clojure.core/printf"}
clojure.core/prn{:message"Use metabase.util.log instead of clojure.core/prn"}
(are[task-namespace](=?[{:message"Module task should not be used in the api module. [:metabase/modules api :uses]"
:type:metabase/modules}]
@@ -43,6 +56,16 @@
'{:metabase/modules{api{:uses#{search}}
search{:api#{metabase.search.core}}}}))))
(deftest^:parallelmodule-checker-empty-api-test
(is(=?[{:message"Namespace metabase.embeddings.core is not an allowed external API namespace for the embeddings module. [:metabase/modules embeddings :api]"
echo "Could not cherry pick because of a conflict"
echo "has-conflicts=true" >> $GITHUB_OUTPUT
echo "has-conflicts=true" >> "$GITHUB_OUTPUT"
# Add a shell script for resolving conflicts
echo "git reset HEAD~1" > ./backport.sh
echo "rm ./backport.sh" >> ./backport.sh
echo "git cherry-pick ${COMMIT}" >> ./backport.sh
echo "echo 'Resolve conflicts and force push this branch.\n\nTo backport translations run: bin/i18n/merge-translations <release-branch>'" >> ./backport.sh
write_backport_script "${COMMIT}"
chmod +x ./backport.sh
git add ./backport.sh
git commit -m "Add backport resolution script"
@@ -57,44 +57,43 @@ runs:
#${ORIGINAL_PULL_REQUEST_NUMBER}
> [!IMPORTANT]
> Manual conflict resolution is required.
> Checkout the branch and run \`./backport.sh\` script. Force push your changes after cherry-picking. Check the box below when done.
> Checkout the branch and run \`./backport.sh\` script. Force push your changes after cherry-picking. Mark the PR as ready for review once conflicts have been addressed.
printf'%s\n'"printf 'Resolve conflicts and force push this branch.\\n\\nTo backport translations run: bin/i18n/merge-translations <release-branch>\\n'"
@${githubUsername} You have modified embedding code. Please make sure the PR title follows [Conventional Commits](https://www.conventionalcommits.org/en/v1.0.0/) style.
) 2>&1|whileIFS=read -r line ||[ -n "$line"];doprintf'[p%s] %s\n'"$idx""$line";done
rc="${PIPESTATUS[0]}"
echo"$rc" > "$STATUS_DIR/p$idx"
}
rm -rf "$JVM_DIRS_ROOT"
mkdir -p "$STATUS_DIR"
echo"Runner $TEST_RUNNER_INDEX of $TEST_RUNNER_COUNT: running $TEST_JVMS_PER_RUNNER concurrent JVMs for partition(s) ${INDEXES[*]} of $PARTITION_TOTAL (-Xmx$TEST_HEAP_PER_JVM each)"
`Import attempt ${attempt+1} failed (${importError.message}); retrying in ${delay}ms`,
);
awaitsleep(delay);
}
}
}
Some files were not shown because too many files have changed in this diff
Show More
Reference in New Issue
Block a user
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.
