Compare commits

...

79 Commits

Author SHA1 Message Date
Chris Truter
48026e557a Tie plugin modules to their directories, and find plugin tests
A file under modules/X now belongs to module/X whatever its namespace says, and a config test fails when a
namespace resolves to a plugin from outside its directory, or a file in the directory resolves elsewhere.
dev.deps-graph infers namespaces for files under a plugin's src and test roots, so test selection finds plugin
tests. A module inside a plugin's own subtree with :uses :any still counts as that plugin's dependent.
2026-09-17 12:21:38 -04:00
Chris Truter
60d2407b5c Make separately built plugins modules of their own
A plugin under modules/X is now module/X, implying the namespace prefix metabase-module.X, the way enterprise/X
implies metabase-enterprise.X. Nothing under src/ or enterprise/ can use that namespace, so a plugin's files
can't shadow core ones, and core code can't land inside a plugin's prefix. The in-process embedder moves from
enterprise/search.embeddings.embedder and metabase-enterprise.embedder to module/embedder and
metabase-module.embedder; its directory was already outside enterprise/.

A plugin may be missing at runtime, so only its own subtree may use it. The resolver refuses it to anyone else,
:uses :any included, and mage's CI selection no longer counts :any modules as its dependents. Every file in a
plugin's directory maps to it.

The directory rule lives in hooks.common.modules/module-directory, shared by mage, dev.deps-graph, the module
explorer and the test runner. The runtime log attribution in metabase.util.log follows the same prefix rule.
CI's kondo run now lints the embedder, and the ratchet scan counts suppressions in each plugin's src and test
directories rather than whole plugin trees, so build copies under target/ never count twice.

A note in the resolver records the plan for drivers: module/driver.X, keeping metabase.driver.X in
modules/drivers/X, so driver-test selection can come from the module graph.
2026-09-17 07:00:35 -04:00
github-automation-metabase
b4f8c8e8aa chore(sdk): bump SDK version on master to 0.64.0-alpha.4 (#82641)
chore(sdk): bump SDK version to 0.64.0-alpha.4 (alpha)
2026-09-17 13:52:53 +04:00
YB
b826490105 [GDGT-2870] fix non-admins unable to create alerts on OSS jars with a paid token installed (#82555)
gate notification permissions on enable-advanced-permissions? not has-feature?
2026-09-17 17:45:03 +08:00
Stas Gavrylov
499bc1160b Consistent typography (part 1): Text/Title on home, collection, Metrics, Data, and Models (#81888)
* Use Title for EmptyState heading and keep heading role

* Use h1 for the Metrics page title

* Use h1 for the Data page title

* Use h1 for collection name and Title/Text in collection empty state

* Fix EmptyState spec to use the illustrationElement prop

* Add hidden h1 and use Text for home greeting and captions

* Use h1 for the Models page title and fix heading levels

- Page title: Title order={2} -> order={1} (parity with Metrics/Data)
- Empty state title: order={3} -> order={2} (avoids h1 -> h3 gap)
- Explanation banner: Title order={6} -> Text size="md" lh="sm" fw={700}
  (banner copy is not a heading; removes h1 -> h6 skip, style-parity swap)

* Remove heading level unit tests

Drops the getByRole heading-level assertions added across the branch
(8 tests in 7 spec files, incl. the new EmptyState spec). Heading
semantics remain covered by the component changes themselves; e2e/a11y
audits own regression coverage.

* Address Copilot comments

* Update HomeCaption to use div

* Fix e2e test broken by HomeCaption's new wrapper element

HomeCaption now wraps its children in a Text component, which added
an extra DOM level. The custom-viz e2e test relied on .parent() from
the caption text landing on the section's shared ancestor with the
recent-item links, which no longer holds.

Scope the assertion via a stable data-testid instead of DOM depth, so
it doesn't re-break on unrelated markup changes.

* GDGT-3181: Address PR comments
2026-09-17 11:57:29 +03:00
Phoomparin Mano
64e7f5cfc8 Whitelist product analytics ingestion origins for data apps (#82483)
whitelist product analytics ingestion origins for data apps
2026-09-17 16:54:49 +08:00
Mahatthana (Kelvin) Nomsawadi
0cf4271e33 feat: warn when the SDK host app runs React 18 (#82432)
* feat: add a host React major version helper to the SDK

Reads the major from the React the host app hands to the bundle, so
both the React 18 deprecation warning and the upcoming unsupported
React version guard read the same source.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UtrkKRneS6NAuJmdYXEHHi

* feat: warn when the SDK host app runs React 18

The SDK will require React 19 in an upcoming release. Surface that
through the existing usage-problem banner and its console message when
the host app renders the SDK with React 18 on localhost or in the host
app's development mode. Production hosts get the same styled console
warning without the banner, and it ignores allowConsoleLog because
losing React 18 support is a breaking change for them. The iframe embed
is excluded since its React version is Metabase's own.

The new warning has the lowest precedence so license errors, the
API-key evaluation warning and the missing "exp" warning still show
when present.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UtrkKRneS6NAuJmdYXEHHi
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01RkRuMTMz2vB8zeicbr9nvq

* fix: log SDK usage problems found after mount

The usage problem is computed into the store in an effect, so on the
first render the hook still reads null. The log-once effect ran with
that null, logged nothing and marked itself done, so the real problem
that arrived a render later was never logged.

Only log once init has finished and a problem exists. Before init
finishes, settings still hold placeholder values, so a problem
computed then can be the wrong one, such as the licensed API key
warning on an unlicensed instance.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01RkRuMTMz2vB8zeicbr9nvq

* fix: show the React 18 warning before other problems

API-key users on localhost only ever saw the API key warning, so they
would not learn about the React 18 deprecation until they moved to SSO
or production. The React 18 case now comes first in the usage problem
match, right after the MCP app opt-out.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix: keep usage errors above the React 18 warning

Putting the React 18 warning first hid the license, disabled-SDK and
non-localhost API key errors on React 18 dev hosts, and let SDK
components render where those errors would stop them. Errors now come
before every warning, and React 18 stays first among the warnings.

The API key error gains `isLocalhost: false` so it can sit above the
localhost API key warnings without catching them.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KsyRP238wpM1hKnPg2VRh2

---------

Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-17 09:48:42 +01:00
Tyler Plude
69b65837bb [GHY-4523] Add remote sync support for glossary (#82390)
* add entity_id to glossary and key its serialization on it

* route glossary writes through glossary.core

* track glossary entries in remote sync when the Library is synced

* export and import glossary entries through remote sync

* tweaks

* lock glossary edits on read-only instances and backfill ledger rows on upgrade

* fe support

* tests

* resolve term-keyed glossary files to local entity ids during remote sync imports

* fix tests

* e2e adjustments

* fix test

* resolve term-keyed glossary paths without a shape check

* resolve same-term glossary files by term in the pre-load conflict check; fix migration

* Backfill glossary tracking per entry, not all-or-nothing

* fix test order
2026-09-16 22:35:14 -07:00
Romeo Van Snick
6636c786e2 Read the reference database pages from their own requests (#82481)
* Read the reference database pages from their own requests

* Remove the entity selectors from the reference store

* Pass the entity the reference update actions read
2026-09-17 13:34:01 +10:00
Chris Truter
63b1899372 Tidy up some Search comments (#82623)
* Tidy the comments and docstrings from the search index metadata fix

The sql-errors registries carried a diary block about the change instead of a note per code.
Put each vendor's state above its entry, move the driver-alignment note next to the set it
guards, and put the vendor-code rationale on the cond branch that needs it.

Elsewhere: restore a dropped blank line, move the IF EXISTS rationale into the fn body,
and rewrap the comments and docstrings that broke mid-sentence.

* Name MariaDB alongside MySQL in the sql-errors comments

* Restore SQL error registry maintenance context
2026-09-17 01:21:29 +00:00
Romeo Van Snick
0a441968d1 Make getMetadata private to the metadata store (#82551) 2026-09-17 10:28:20 +10:00
Romeo Van Snick
b4513dcdca Move the card copy form next to the other copy forms (#82549)
* Move the card copy form next to the other copy forms

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FeHJzstcAE34DTu4G6btjx

* Point the side-effect ledger at the card copy form's new path

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FeHJzstcAE34DTu4G6btjx

---------

Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-17 10:28:00 +10:00
Chris Truter
1f1643d5b8 Fix search index metadata deletion guard (#80309)
* Fix search index metadata deletion guard

Delete only the pending SearchIndexMetadata row rather than whichever row happens to match the coordinate, and
state that contract in the docstring.

The cleanups the fix leaned on, in the same change:

- Write the index DDL as Honey SQL maps instead of threading `sql.helpers`, which earns its place only when
  clauses are added conditionally.
- Identify a missing index table by SQL state rather than exception class, covering every state our app databases
  report, and name those states and the vendor error codes instead of spelling out literals at each call site.
- Take the connection as an argument rather than encoding it in an `-on-conn!` name suffix.
- Let the index poll survive an index that has not been built yet.
- Move rationale out of the docstrings into comments: a docstring is the caller's contract.
- Skip `with-model-cleanup`'s reindex when no search-relevant rows were deleted, and give `multiple-limits-test`
  its own search data so it stops depending on another test's rows.

* State which safe-batch-upsert! failures throw and which are skipped

The docstring claimed failed retries propagate; only a missing table
after refresh, a missing table on the retry, and interruption do. Any
other failure is logged and the batch is skipped.

* Classify app-db SQL errors centrally instead of matching states in search

app-db.sql-errors keeps the named SQLSTATE and vendor-code registries and
classifies an exception's cause chain into an error class keyword
(:table-not-found, :duplicate-key) that call sites dispatch on. The
search module's private copy of the missing-table states goes away, and
with it the comment about keeping them aligned with the drivers; the
duplicate-key entries are ready for their caller in the lease work
stacked above. MySQL's catch-all integrity-constraint state is
disambiguated by the vendor code, so no db-type parameter is needed.

* Relax the reset pre-delete to any non-active index metadata

delete-pending-index! becomes delete-non-active-index!: a reset now also
clears retired metadata rows for the coordinate, never active ones. The
lifecycle test covers deleting retired and pending rows while the active
row is refused.

* Count every cleanup deletion and reindex when any rows were removed

The search-hooks filter had a blind spot: raw deletes cascade (a user's
personal collection), removing search-relevant rows even when the listed
model has none. Count the total with a reduce and rebuild whenever
something was deleted; only an empty cleanup skips.

* Address search index review feedback
2026-09-16 20:13:38 -04:00
Eric Normand
95368200b1 Least-privilege MCP v2 scopes with step-up (GHY-4543) (#82452)
* List every MCP v2 tool regardless of token scopes (GHY-4543)

tools/list filtered out tools the token lacked scope for, so a client
holding a narrow grant never saw the write, SQL or delivery tools and
had nothing to attempt or step up for. Scope stays enforced at
tools/call; the disabled-tools and client-extension filters still apply.

With scopes out of the listing, list-tools and tools-hash drop their
token-scopes parameter, and the transport's tools-hash-fn becomes
zero-arity.

* Explain MCP v2 scope denials to the model (GHY-4543)

Clients replace a scope denial with their own text ("token expired",
"Insufficient scope", "Tool execution failed"), so the model tells the
user their login expired. Server instructions now say that failure is a
missing permission, and how the user reconnects to grant it. Every tool
description ends with a static sentence naming its required permission by
its consent-screen label, identical for every token: Claude Code caches the
first description it loads for the whole session.

* Answer MCP v2 scope denials with 403 insufficient_scope (GHY-4543)

A scope denial was an HTTP 200 carrying a JSON-RPC error or an isError
result, so clients never stepped up: Claude Code only saves a step-up
scope, and mcp-remote only starts one, on a real 403 with an
insufficient_scope WWW-Authenticate challenge. The registry gate and the
four in-handler scope checks now mark their refusal, and the transport
answers a single (non-batched) message with 403 plus a challenge whose
scope is the held v2 scopes plus the required one.

* Keep a DCR client's registration scope from narrowing its step-up (GHY-4543)

Claude Code registers with the scope it read from the protected-resource
metadata, then steps up on the same client_id with the wider scope from a
403 insufficient_scope challenge. oidc-provider validates a requested scope
against the client's registered scopes, so that step-up was refused with a
400 invalid_request. Once scopes_supported is narrowed to the baseline, no
Claude Code user could reach write, SQL, or delivery.

A dynamically registered client's ceiling is now its registered scopes plus
the default grant scopes, computed on read. Registering can widen the
ceiling (a client that registers for mb:full keeps it) but never narrows
it. RFC 8707 resource narrowing is unchanged.

* Declare each MCP v2 tool's OAuth scope in securitySchemes (GHY-4543)

ChatGPT only steps up for scopes a tool declares: without per-tool
securitySchemes it re-authorizes for its narrow login scope in a loop,
and with them it re-authorizes once for held plus declared scopes.
The scheme is derived from the tool's static :scope in the cached
manifest, so it is the same for every token.

* Advertise only the baseline MCP v2 scopes to new clients (GHY-4543)

The protected-resource metadata `scopes_supported` and the 401 challenge
`scope` now list only `agent:content:read agent:resource:read`, so users
who never need write, SQL, or delivery never grant them. Claude Code and
the Claude connectors take their first-login scope from the resource
metadata; ChatGPT takes it from the challenge. All of them reach the rest
through the 403 `insufficient_scope` step-up and per-tool
`securitySchemes`.

What the resource accepts is unchanged: RFC 8707 narrowing still keeps all
six v2 scopes, and the authorization-server metadata stays wide because
Codex requests that list on every login and cannot step up.

* List every MCP v2 resource regardless of token scopes (GHY-4543)

resources/list now matches tools/list: it is token-independent, so a client can
see the resources it could step up for, and a client that pre-fetches the MCP
Apps shells before stepping up to agent:query:run does not cache them as
missing. resources/read keeps its scope gate.

* Answer MCP v2 resources/read scope denials with 403 insufficient_scope (GHY-4543)

resources/list now shows every resource, so answering a denied read with
"Resource not found" hides nothing and leaves a client with no step-up
signal. A client that pre-fetches an MCP Apps shell with a baseline token
now gets the same challenge tools/call sends, naming agent:query:run.
Unknown URIs stay -32602; inside a batch the denial is an in-band -32600.

* Lead MCP v2 tool descriptions with their required permission (GHY-4543)

Clients truncate long tool descriptions (Claude Code at 2048 characters),
so the appended "Requires the ... permission" sentence was cut from
document_write and execute_query. The model needs that sentence to name
the missing permission after a scope denial, so it now opens every
description, and the server instructions say it starts the description.

* Serve every MCP v2 resource on resources/read regardless of token scopes (GHY-4543)

Claude Desktop reads an MCP Apps shell alongside the tool call. When that
read was refused with 403 insufficient_scope, Desktop never stepped up after
the tool call's own 403; with the shell served, it stepped up automatically
and retried. The shell carries no data and, for a token without the shell's
scope, no UI credential: read-resource now withholds the credential instead
of refusing the read. Chart data stays gated by the tool call,
refresh_ui_credential, and the UI-surface route costs.

* Restore the scope check on MCP v2 data resource reads (GHY-4543)

Only the MCP Apps UI shells need to be served to any token: a refused shell
read stopped Claude Desktop stepping up after the tool call's 403. The fields
catalog carries data and has no such constraint, so reading it without
agent:resource:read again gets the 403 insufficient_scope challenge (in-band
-32600 inside a batch). UI shells stay unscoped, minting a credential only for
a token holding their scope.

* Fit MCP v2 tool descriptions under Claude Code's 2048-character limit (GHY-4543)

Claude Code truncates tool descriptions at 2048 characters, so document_write and
execute_query lost their closing guidance, including execute_query's pointer away
from execute_sql for filtered-dashboard cards. Condense both, and get_content for
margin, deferring detail to learn() topics and argument schemas. A test now holds
every listed description to the limit.

Also correct docstrings and comments left stale by the resources/read and
tools/list scope changes and by credential delivery through refresh_ui_credential.

* Spell out the scope union in the MCP v2 challenge docstring (GHY-4543)

The whitespace linter rejects the confusable Unicode character U+222A.

* Add agent:query:run to the MCP v2 baseline scopes (GHY-4543)

Claude Desktop retries a tool after a step-up over a session that does not
declare MCP Apps support, so a stepped-up visualize_query is refused and the
first chart after a step-up never embeds. Querying and charting therefore
must not require a step-up. The protected-resource metadata, the 401
challenge and default-ask-scopes now advertise agent:content:read
agent:query:run agent:resource:read; writes, raw SQL and delivery still step
up.

* Point MCP v2 chart requests at visualize_query in the server instructions (GHY-4543)

The routing line named only execute_query, and clients like Codex load tool descriptions on demand, so chart requests could miss visualize_query.

* Gate native query handles in transform_write on agent:sql:run and the kill switch (GHY-4543)

A query handle is not proof the SQL gates were spent: /api/embed-mcp/drills
stores a native query under agent:query:run alone. transform_write now runs
the same scope and mcp-execute-sql-enabled checks on a handle that resolves
to native SQL as it does on an inline native definition, matching
question_write.

* Tighten resource registration, step-up scopes, and the DCR ceiling (GHY-4543)

- register-resource! takes a closed map and strips :ui?, so a data resource
  can never be served to a token without its scope.
- step-up-scopes matches held scopes instead of looking them up, so a token
  holding a wildcard such as agent:content:* keeps the surface scopes it
  covers when a client replaces its grant with the challenged scope. It now
  takes the single required scope both callers pass.
- The default grant ceiling is applied only while dynamic registration is
  enabled, so turning registration (or MCP) off stops widening existing
  dynamic clients. The merge is now a pure function with unit tests, and an
  HTTP test pins that static clients are never widened. The legacy-client
  migration test runs with registration disabled, where the stored snapshot
  alone decides, so it exercises the migration again.

* Correct MCP v2 tool descriptions, the documents skill, and server instructions (GHY-4543)

- document_write lists the valid entity-link models again.
- The documents skill said edits store new_str literally; the tool parses it
  as Markdown, as edit-replacement-parses-markdown-test pins.
- execute_query no longer claims to save cards (its handles save as cards),
  and names the previous-stage field ref shape instead of an ambiguous
  column-name string.
- The server instructions attribute the rewritten permission failure to the
  client and cover resource reads, which get the same 403.

* Pin the MCP v2 scope-denial and UI-credential contracts in tests (GHY-4543)

Closes gaps the review found, where removing a check would stay green:
- question_write and subscription_write in-handler denials carry
  :insufficient-scope at the registry, like alert_write's.
- A denied tools/call sent as a notification is a bare 202, no challenge.
- Wildcard and unrestricted tokens get the shell credential; the fields
  catalog mints none, over the registry and over HTTP.
- tool-and-resource-scopes-agree-test asserts the credential is realized,
  since any token now reads a shell.
- advertised-scopes-are-distinct-test covers the advertised baseline, which
  is a vector and can repeat, and its docstring says so.

* Clean up MCP v2 scope-denial helpers, header quoting, and docs (GHY-4543)

- Share one scope-denial message and detail builder between tool calls and
  resource reads, one step-up challenge helper, and a common
  throw-insufficient-scope! for the four in-handler scope checks.
- quoted-string replaces " and \ instead of backslash-escaping them, since
  RFC 6750 excludes both from scope and error_description values; the 401
  challenge now quotes through it too.
- Trim docstrings to their contract and move the reasons into comments.
- The unrestricted UI credential comes from any unrestricted session (cookie,
  API key, or an mb:full token), not only cookie sessions.
- Note that the DCR ceiling applies only while registration is enabled, and
  fix stale comments, test names and strings, long lines, order-sensitive
  scope assertions, and hard-coded consent labels in tests.

* Always widen dynamic OAuth clients to the MCP scopes (GHY-4543)

Claude Code registers with the baseline scopes and steps up on the same
client_id. With dynamic registration turned off, that step-up /authorize
was answered with a raw 400. A dynamic client now always gains the six v2
MCP scopes on read, each still behind consent; the remaining agent-API
ceiling is added only while registration is enabled. Registration itself
stays blocked.

* Close the MCP v2 scope test gaps (GHY-4543)

Cover the transform_write update path's native gates, the HTTP 403
insufficient_scope challenge question_write and transform_write send from
inside their handlers, saving a drill handle minted through
/api/embed-mcp/drills, and the closed resource registration schema. Give
the locale test a positive control, merge the two tools/list
token-independence tests into one that compares the whole payload, and
drop the permission-sentence truncation test the fit and starts-with
tests already cover.

* Tidy MCP v2 scope wording and naming (GHY-4543)

Rename registry/insufficient-scope to insufficient-scope-detail so it
does not read like the transport fn of the same name. Drop the server
instructions clause telling the model to read a resource's scope off an
error clients hide. Document agent:resource:read and the handler-internal
scope checks in the module README, correct the stale comments about UI
resources gating on agent:query:run, and wrap the lines this branch left
over 120 columns.

* Let the test fallback template win over a local frontend build (GHY-4543)

The MCP Apps shell renders from the built `embed-mcp.html` when one exists,
so a worktree that has run a frontend build shadowed the inline fallback the
credential tests install. The production template has embedded no
`uiCredential` since #81041, so those tests failed locally and passed on CI,
where backend-only runs produce no build.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014QT1v8on6aPT2UeHCojGj8

* Gate the MCP grant-ceiling widening on the MCP kill switch (GHY-4543)

A dynamically registered client gained the MCP surface scopes on every read of
its client record, unconditionally. Only the wider `default-grant-scopes`
ceiling was gated, on `oauth-server-dynamic-registration-enabled`.

`agent:resource:read` is one of those MCP scopes and is also the declared scope
of `POST /api/agent/v1/read-resource`, which a separate lever gates. So with MCP
disabled and the agent API enabled, a dynamic client could authorize for a scope
its registration never included and spend it at a surface still serving.

Read `mcp-enabled?` in the effectful client-store wrapper and pass no MCP scopes
down when it is off; `widen-to-grant-ceiling` stays pure. While MCP is enabled
the existing behavior is unchanged: the MCP scopes are still added whatever the
registration setting says, so a client registered with the baseline can step up.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014QT1v8on6aPT2UeHCojGj8

* Cover the built MCP embed template's no-credential property (GHY-4543)

Serving MCP Apps UI shells to any token holder rests on the built
embed-mcp.html discarding the credential it is handed (since #81041) — the
iframe fetches its own through `refresh_ui_credential`. Nothing asserted that,
so a frontend change reintroducing a `uiCredential` placeholder would land
silently now that the outer scope gate on shell reads is gone.

Add `metabase.mcp.ui-resource-test` rendering the built template with a
credential and asserting the output carries neither the credential nor a
`uiCredential` placeholder. Backend-only CI runs have no frontend build, so the
test logs a skip instead of failing when the resource is absent.

Also gate the test fallback template's precedence over the built one on
`is-test?`, so the reorder can only ever apply in a test process.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014QT1v8on6aPT2UeHCojGj8

---------

Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-16 18:21:34 -05:00
Romeo Van Snick
2602fec3cd Record how many bytes of CSS a page loads (#82541)
Claude-Session: https://claude.ai/code/session_01171Lkmug15JzrcN2aMBLMk

Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-17 06:38:33 +10:00
Kevin Kelleher
b9d8328b19 Adds 'Upgrade now' banner (#82214)
* initial upgrade now banner

* remove security center integration

* fix compareVersions types

* automation, cleanup, tests, fetch version-info.json every hour

* Change CheckForNewVersions poll interval to every 6 hours

https://github.com/metabase/metabase/pull/82214#discussion_r3993430218

---------

Co-authored-by: Mike Appleby <86076+appleby@users.noreply.github.com>
2026-09-16 20:27:32 +00:00
bryan
7d94211ecd Fix two closed-schema regressions from #82447 (#82581)
* Accept localized names in CollectionInstance

The sidebar fails to load: any call passing the Trash collection through
`CollectionInstance` throws.

    Invalid input: ["an instance of the root Collection, got: (toucan2.instance/instance
    :model/Collection {... :name #metabase.util.i18n.UserLocalizedString{:format-string
    "Trash" ...} :type "trash" ...})"]

`trash-collection` sets `:name` with `deferred-tru`, yielding a
`UserLocalizedString` rather than a `String`. #82447 added `CollectionInstance`
with `[:name :string]`, which that value cannot satisfy, so the Trash
collection matches neither branch of `[:or RootCollection CollectionInstance]`.
Malli reports only the first branch's `:error/message`, which is why the error
misleadingly names the root Collection.

Widen `:name` to accept localized strings via the existing
`ms/LocalizedString`. The deferral is deliberate: `trash-collection` is
memoized across requests, so eager-rendering with `str` would freeze the name
in whichever locale filled the cache (#59662). `:slug` and `:description` are
left alone; they are not localized.

Reproduced and verified against a running instance: `shared-tenant-collection?`,
`children-location` and `descendant-ids` no longer throw on the Trash
collection, and `GET /api/collection/tree`, `/api/collection`,
`/api/collection/:trash-id` and `/api/collection/:trash-id/items` all return
200 with the name rendering as "Trash".

* Reopen the cols schema in the static-viz color picker

Rendering a table with conditional formatting throws:

    Invalid input: {:cols [{:display_name ["disallowed key, got: \"Category\""],
                            :base_type ["disallowed key, got: :type/Text\""]} ...]}
    fn-name: cell-background-colors

`QueryResults` describes the value handed to the shared color-picking JS. Its
docstring says it is "a pretty loose schema, more as a safety net" that "just
needs column names from `:cols`" — callers pass whole column metadata maps and
only `:name` is ever read. #82447 closed the inner `:cols` map, so every caller
passing `:display_name` or `:base_type` now fails validation.

Mark that map `::mr/deliberately-open` to restore the documented intent. The
outer map stays closed; the extra keys only ever appear on columns.
2026-09-16 20:23:32 +00:00
Chris Truter
7c771f3c33 Add GPT-6 Astra support (#82499)
* Add GPT-6 Astra to the OpenAI, Azure and Bedrock model pickers

The OpenAI allowlist stopped at the 5.6 family, and the reasoning/temperature
check keyed off a gpt-5 prefix, so a forced request would have sent
temperature, which Astra rejects. Widen the check to GPT-5 and GPT-6 and add
the model to the three adapters that sell it directly.

OpenRouter is left out: its allowlist needs a live reasoning probe per model.

* Correct Bedrock context window comment
2026-09-16 20:17:17 +00:00
Romeo Van Snick
3c3979838a Move the SDK storybook instance URL out of the storybook config (#82544)
Claude-Session: https://claude.ai/code/session_01FeHJzstcAE34DTu4G6btjx

Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-17 06:06:06 +10:00
Eric Normand
49a7152e97 Give flat-table-content typed argument schemas so body.clj loads (#82591)
* Give flat-table-content typed argument schemas so body.clj loads

#82447 made mu/defn throw at macroexpansion when an argument's schema does
not declare its shape. #82277 landed flat-table-content with an unschema'd
card argument and an unschema'd open map, so requiring
metabase.channel.render.body now fails to load.

Use the same argument schemas as its sibling prep-for-html-rendering, which
receives the same two values from `render :table`.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014QT1v8on6aPT2UeHCojGj8

* Narrow simple-pivot-content's color-data cols to :name

The second collision between #82447 and #82277: #82447 closed the cols
entries of js.color's QueryResults to exactly {:name :string}, and #82277's
simple-pivot-content hands cell-background-colors full cols, so every
simple-pivot-* test errors with :metabase.util.malli.fn/invalid-input.

Narrow the cols the way the flat-table path already does. The colors are
unaffected: the JS reads cols only to build a name->index map
(makeCellBackgroundGetter -> getColumnIndexesByName).

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014QT1v8on6aPT2UeHCojGj8

---------

Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-16 21:32:54 +02:00
Timofey Kachalov
a367aac5d8 Require defineQuery and defineAction exports in data-app hooks (#82489) 2026-09-16 20:19:27 +04:00
Maksym Yakubych
f02360567a Make Metabot metric mentions clickable (#82348) 2026-09-16 16:12:47 +00:00
Eric Normand
6318ae1d86 Build MCP v2 agent-facing text with msg so interpolated values are quoted and escaped (#82437)
* Add a kondo lint for unquoted interpolation into agent-facing prose

Teaching errors, paging hints, and steering lines read to an LLM as the
server speaking. A warehouse table name, driver error, or card name
interpolated into that prose unquoted can carry newlines and text that
pose as server-authored instructions.

`:metabase/unquoted-prose-interpolation` flags `format` %s arguments,
`tru`-family {n} arguments, and non-literal arguments of a `str` call
over a prose literal, unless they are literals, `pr-str` calls, or a
`str/join` over `(map pr-str ...)`. It is enabled only for
`metabase.mcp.v2.*` and `metabase.agent-lib.*` source namespaces.

The rule currently reports 243 findings; a follow-up tunes out the
noise and fixes the real sites.

* Scope the prose-interpolation lint to MCP v2 and the MCP transport

* Add the MCP v2 message builder and lint its lines

`metabase.mcp.v2.message` builds agent-facing prose: `msg` takes a
vector of line format strings plus arguments, and `render` joins the
lines with newlines and interpolates the arguments, cleaning each one
unless it is `raw` or a nested message. `clean` quotes with `pr-str`
and escapes invisible, line-breaking, and quote-like characters as
\uXXXX. `render` never throws: a malformed message, a format failure,
or anything that isn't a message renders with every part cleaned.

`:metabase/agent-message-lines` checks `msg` calls: the lines must be a
literal vector of string literals with no line breaks, `%n`, or
control characters, and the arguments must match the specifiers.

* Render MCP v2 messages at the tool-content and JSON-RPC exits

Teaching errors thrown with a message render it into the exception
message and carry the message in ex-data, so the caller-facing error
paths hand it on to be rendered. error-content, success-content, and
jsonrpc-error render messages; plain strings pass through unchanged for
now. message? is now public.

* Lint that MCP v2 exits receive msg-built text

Add :metabase/agent-message-exit, enabled for agent-prose-namespaces. It flags the message argument of throw-teaching-error, error-content, jsonrpc-error, and success-content when it is syntactically text (a string literal, a string-building call, or a threading, branching, or body form yielding one), and ex-info calls whose data literal carries a 4xx :status-code or an error-code key.

Point the unquoted-prose-interpolation messages at msg instead of pr-str.

* Scope the agent-prose lints in their hooks instead of with :off switches

The prose-interpolation and agent-message-exit linters are :warning globally; their hooks run only in non-test metabase.mcp.v2.* and metabase.mcp.transport namespaces. agent-message-lines runs in any non-test file. This keeps config.edn free of new :off switches for the kondo ratchet.

* Build shared MCP v2 prose with msg

Convert the shared response helpers, the transport's JSON-RPC errors, the
registry's pre-dispatch errors, and the API's method and resource errors to
msg. Registry errors now carry a message, humanized schema detail cleans its
paths and expectations, and the query pipeline's rewrapped schema failure
keeps the original message as a cleaned argument. Adds message-ex-info and
list-message to common.

* Build shared MCP v2 write, resolve, and skills prose with msg

Convert the _write dispatch, id resolution, recipient, projection, skills,
and recovery-hint text. Caller-supplied values (ids, clear names, methods,
recipients) are cleaned; server-declared names stay bare. The skills
catalog, pack, and reference texts are now messages, and the recovery
hints render msg-built English text for the shared pipeline.

* Build MCP v2 bookmark and UI credential prose with msg

* Build MCP v2 alert and subscription prose with msg

* Build MCP v2 transform prose with msg

* Build MCP v2 collection, duplicate, content, and document prose with msg

Card types, include sections, and document snippets are now cleaned in
teaching errors; the tools' own argument keys and type names stay bare.
The default copy name stays a plain content name, and get_content passes
its payload to success-content for encoding.

* Build MCP v2 metric, segment, measure, and question prose with msg

Normalizer and pipeline exception text, card types, template tag names,
column names, and schema summaries are cleaned in teaching errors. The
definition-shape and metric shape-rule sentences are messages nested into
the errors that end with them, so resolve-external-query receives a
message hint. The native-SQL scope refusal goes through
throw-teaching-error.

* Build MCP v2 parameter-values and dashboard-op prose with msg

Stored parameter ids and names, template-tag names and types, and text-card
placeholders are cleaned in teaching errors, listed with list-message. The
get_parameter_values steering lines are messages nested after the JSON
payload, including the shared truncation line. op-error! takes a message
detail; op and argument names the server declares stay bare.

* Build search, learn, and visualize MCP v2 prose with msg

* Build query and dashboard MCP v2 prose with msg

* Build browse MCP v2 prose with msg

* Name the op in dashboard op errors instead of in their detail

op-error! now takes the op name separately and renders `op <idx> (<op-name>): <detail>`, nesting the
detail message. Callers that still passed strings (the unknown-card check, add_link, and the unknown-op
fallback) rendered doubly quoted; the unknown-op fallback cleans the caller's unmatched op name itself.

* Clean plain strings whole at every MCP v2 exit

error-content, success-content, and jsonrpc-error now render everything
through message/render, so a plain string (a string teaching error, or an
upstream exception message surfaced by ->mcp-error-content or
caller-safe-error-message) reaches the client quoted and escaped instead
of passing through. Only msg values render as prose; "Internal error" is
now a msg so it stays unquoted.

* Scope MCP v2 prose linters by config and retire unquoted-prose-interpolation

Remove the format/str/tru interpolation checks now that msg cleans values at
runtime, restoring the i18n hooks to strict-apostrophes. msg lines may be split
with str over literals, and %s can't take width, precision, or flags, nor %S.
The linters are enabled for MCP transport and v2 source namespaces through an
ns-group instead of in-hook namespace checks.

* Harden MCP v2 message rendering, truncation, and rewrapped errors

clean escapes only double-quote look-alikes, keeping single quotes such as
U+2019 so names survive being copied back. render treats %S and %s with
flags, width, or precision as malformed, and renders a fixed sentence when
an argument can't be printed. ellipsize shortens string arguments inside
their quotes, falling back to message/truncate, which cuts a rendering
without leaving a quoted value open. An exception rewrapped with new text
surfaces that text instead of its stale ex-data message.

* Send MCP v2 recovery hints, throttle, and extension errors as prose

Recovery hints are now messages nested after the quoted pipeline text,
added by the v2 query entry point instead of appended by the shared
pipeline. The throttle refusal and missing-extensions error are
server-written messages rather than quoted strings, and the default
duplicate name is localized again with tru.

* Describe recovery-hint contract now that MCP v2 attaches its own hints

* Surface rewrapped exception text in MCP v2 query and definition errors

Use common/exception-message instead of queries' private copy, which
preferred the stale ex-data message over a rewrapped exception's new text.

* Restore plain MCP code restructured only for the retired prose lint

Inline the SSE framing and native summary prefix, return the WWW-Authenticate
builder and schema descriptions to plain str/format, and restore master's text
for developer-only load-time and 500 errors. Agent-facing output is unchanged.

* Tighten MCP v2 message docstrings to their own contracts

* Give MCP v2 message fns malli schemas

* Wrap long MCP v2 msg lines and build the learn catalog as one message

* GHY-4544: Widen agent-message lint to all MCP source and message-taking helpers

Rename hooks.metabase.prose-interpolation to hooks.metabase.agent-message.
Check msg lines in every non-test metabase.mcp namespace; keep the exit
check on transport and v2, whose 4xx errors are MCP exits. Flag malformed
format specifiers, treat cond/condp/case results as text, and check the
messages passed to op-error!, check-execute-sql-enabled!, ellipsize, and
list-content's :empty-hint. Split lint-msg into helpers.

* Never lengthen a quoted value when truncating a message

A value one character over its budget kept its full text and still gained an ellipsis, rendering longer than the uncut message. Cut values and whole values are now measured the same way, and the cut search never considers the whole value.

* Harden message cleaning and share render's formatting with truncation

clean binds the print vars that affect quoting, so a caller's *print-readably* false can't strip the quotes off a value, and is now private. More double-quote look-alikes are escaped. render and truncation share one formatting helper, and truncation helpers are named for what they return.

* Render schema failures as readable detail and add schemas to common message helpers

An invalid-input schema failure now names its problems with humanize-detail instead of printing the humanized data as one quoted value. message-ex-info, exception-message, list-message, and ellipsize are mu/defns, the message helpers move ahead of the teaching errors that use them, and pure helpers are named for what they return.

* Omit a missing exception message from MCP v2 errors and keep the original as cause

A nil exception message rendered as null (or an empty quoted string) in the
query pipeline rewraps and the definition, metric, transform, and inline
question normalization errors. Those messages now leave that text out.
with-schema-detail and with-recovery-hint rethrow with the original exception
as the cause, so logged traces show the pipeline's throw site.

* Unquote get_content item errors and name alert schedule fields readably

A get_content item's error is a JSON string value, so a plain upstream
message goes in as it is instead of quoted and escaped twice. alert_write's
uncompilable-schedule error lists the schedule's fields with their values
cleaned, instead of a double-printed map.

* Tidy MCP v2 message construction and naming

Send the throttle library's refusal as its own text, nest raw values
directly, inline duplicate's copy-name, rename query-failed and browse's
message locals, replace search's empty filler messages with a list join,
trim two Metabot docstrings to their contract, and mark two pure
dashboard_ops tests parallel.

* Judge threading forms by their result and check :empty-hint through browse's paged-list-content

The exit check read the threaded value of -> and ->> rather than what the form
returns. ->, ->>, some->, and some->> now take text-ness from their last step,
cond-> and cond->> from the initial value or any step, as-> from its last form,
and doto from its target; ex-message counts as building a string. The
:empty-hint check is also registered for browse's paged-list-content, which
passes its options map to list-content.

* Keep a truncated MCP v2 message's pieces so truncating it again closes its quotes

A truncation is now a message of the pieces it kept: server text as raw text and
a cleaned value as itself or its cut excerpt, so a later truncation cuts each
piece as the kind it is. Also registers the message and raw schemas, shares one
surrogate-safe string-prefix, merges the argument dispatch, and renders keywords
as their names.

* Wrap long lines in the MCP v2 common tests

* Pass server text to MCP v2 helpers as messages and name missing identifiers

Helpers that interpolated a string parameter through message/raw on the strength of a
comment about their call sites now take a message: alert and subscription scope and
schedule checks, parameter lookup, segment/measure name checks, duplicate's trash check,
alert id checks, and dashboard op errors. Rendered text is unchanged, except that
dashboard_write's unknown-card refusal now quotes the op name as the caller sent it.

A missing tool name, method, op, or dashcard/tab/parameter id now gets its own sentence
instead of rendering as null.

* Keep surrogate pairs whole when cutting document_write error snippets

* Wrap MCP v2 test lines longer than 120 characters

* Cut document_write error snippets with the shared surrogate-safe string-prefix

* Pass single-line server text to MCP v2 messages as raw

A message with one line and no arguments that is only ever interpolated into another message renders the same as raw text, so those call sites now pass (message/raw "..."). Messages that reach an exit, render on their own, or are returned as a top-level message keep msg. Helper docstrings now describe those parameters as server text.

* Inline one-use MCP v2 message helpers

require-hour! in alert_write and subscription_write and catalog-line in skills existed only to keep message-wrapped lines short. With raw text at the call sites they read as direct calls again.

* Allow a plain subject for the MCP v2 execute_sql kill-switch check

The helper interpolates its subject into its own msg, so a string literal is cleaned like any value.

* Minimize raw in MCP v2 prose

Identifiers interpolated into agent-facing messages are now cleaned rather than
passed as raw; raw moves to where server text is defined. Leftover call-site
comments are removed, lists of names render via list-message/lines-message, and
test expectations are updated to match.

* Quote field names in MCP v2 errors and clean remaining identifiers

Field, argument, and tool names in fixed agent-facing msg lines use double quotes
instead of backticks, matching names interpolated through cleaning. Code snippets
keep their backticks. common.clj cleans the not-found model name, the schema-failure
function name, select-fields type names and nearest catalog paths, and the
truncation-line param instead of passing them raw. alert_write's ignored-field error
says "An" before an hourly schedule, as subscription_write does. Test expectations
updated to match.

* Close MCP v2 message arg schemas for mu/defn's shape check and name not-found types as server text

* Allow only %s, %d, and %% in MCP v2 msg lines and govern enterprise MCP namespaces

Match the runtime's allowlist: a `msg` line may use `%s` with no flags but `<`, `%d` with flags and
width but no precision, and a bare `%%`; any other conversion would alter or hide a cleaned value.
Flag private-use characters too, since the truncation markers delimit arguments with them, and cap
the argument index at nine digits so an absurd one reports as malformed instead of overflowing.

The two MCP ns-groups now also match `metabase-enterprise.mcp.*`, which they left ungoverned.

* Bound MCP v2 message printing to 100 elements and 10 levels

An unbounded sequence argument made render hang and a multi-million-element vector rendered
megabytes of text, since printed pinned *print-length* and *print-level* to nil.

* Take only strings as MCP v2 raw message text

A non-string raw was interpolated into the formatted template instead of being kept as a text
piece, which could collide with the private-use truncation markers. Message lines holding
private-use characters are now malformed for the same reason.

* Allowlist MCP v2 message format conversions and widen quote look-alikes

Only %%, a plain %s, and a %d without precision format; every other conversion — %b, %h, %c,
%x, %f, %tY and friends, which reinterpret an already-cleaned argument — makes the message
malformed, so it renders fully cleaned. The double-quote look-alike set gains the ornaments,
ditto mark, and prime family found by sweeping the Unicode blocks it draws from.

* Clean get_content per-item error text instead of trusting JSON quoting

JSON escapes control characters and double quotes but leaves category-Cf
characters (a bidi override, a zero-width space) intact, so a plain-string
caller-facing error reached the model raw. Render it like every other agent
message, which quotes and \uXXXX-escapes the whole string.

* Wrap MCP v2 message lines the branch pushed past 120 columns

Text is unchanged: each line splits into a `str` of the same pieces, or moves
a word to the continuation line.

* Flag msg lines whose argument reference String/format can't resolve

The kondo hook counted a leading %< and an explicit 0$ as consuming no
argument, so a msg using either linted clean while the runtime's format
threw and fell back to rendering every part quoted.
2026-09-16 15:53:05 +00:00
Bartosz Rybacki
f3d1360240 GDGT-2751 Apply the Table viz's pivot toggle in subscriptions (#82277)
* GDGT-2751 Apply the Table viz's pivot toggle in subscriptions

A Table card with the "Pivot table" toggle on arrives flat in a
dashboard subscription or alert. The toggle is a browser-only
transform; the three settings reach the static renderer, which
never read them.

Port the browser's pivot into `render :table`, including its sort
rule, and draw the grid with the existing pivot renderer. The grid
is built from every row and is not cut at the attachment row limit,
like a Pivot Table card. The server-side dashboard PDF gives such a
card the pivot treatment too, so it no longer prints the source row
count under the grid. CSV and XLSX attachments stay flat; they are
a separate code path.

3 functions needed to be moved. A pure move, no changes: 
`setting-value`, `pivot-cell->str` and `pivot->hiccup` go from 
the pivot section to a section of their own under the 
`render` multimethod.
2026-09-16 17:37:44 +02:00
Sloan Sparger
efe0882aca [Metabot] Remove transform_codegen profile (#82208) 2026-09-16 09:17:13 -05:00
Sébastien
f1a48e2b5c SDK entity navigation drops the click behavior filter when the target is a GUI (MBQL) question (#82409)
* Fix SDK entity navigation dropping click behavior filters on GUI questions

* Give ad-hoc question targets their own click behavior type

* Cover the GUI click behavior target with an e2e test

* Fix the internal-navigation dashcard setup clobbering itself
2026-09-16 14:51:54 +02:00
Chris Vandoodewaard
9b59d17d10 notify release managers if auto release is skipped (#82528) 2026-09-16 08:03:13 -04:00
Will Fry
b084ade539 GDGT-3224: migrate TabRow to CSS modules to fix gradient theme issue (#82482)
* GDGT: migrate TabRow to CSS modules to fix gradient theme issue

* Update Loki Snapshots

* increase scroll gradient size

* Update Loki Snapshots

* Add dark theme story for narrow dashboard with many tabs

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* Add placeholder Loki reference for dark theme narrow tabs story

Copy of the light theme snapshot so the loki-update workflow detects a
diff and replaces it with the CI-generated image.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* Update Loki Snapshots

* move left/right styles into existing classes

---------

Co-authored-by: Metabase Automation <github-automation@metabase.com>
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
2026-09-16 11:01:26 +00:00
Nemanja Glumac
2980fd7f29 Stub floating-ui positioning in unit tests (#82493) 2026-09-16 12:53:55 +02:00
Will Fry
6faa93e40c GDGT-3230 fix lack of highlight in CardEmbedNode title input (#82520) 2026-09-16 11:49:58 +01:00
Romeo Van Snick
d3db1a1c53 Read three single-entity lookups from their own requests (#82550)
Read the segment's table, the list's table and the editor's database from their own requests
2026-09-16 19:39:20 +10:00
Timofey Kachalov
82d075c73c Expose a DateRangePopover in the data-app SDK namespace (#82291) 2026-09-16 11:40:43 +04:00
Romeo Van Snick
9f00ac4fc0 Dissolve the hoc module into its four consumers (#82547)
Claude-Session: https://claude.ai/code/session_01FeHJzstcAE34DTu4G6btjx

Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-16 15:18:19 +10:00
Romeo Van Snick
4b5f24e0a9 Read the reference segment pages from their own requests (#82475)
* Read the reference segment pages from their own requests

* Remove the segment selectors from the reference store
2026-09-16 13:59:59 +10:00
Fraser
46541523bc Tier documents and require explicit shared-tier coverage (#82542)
* Tier documents and require explicit shared-tier coverage

* Check shared tier coverage directly in the unit spec
2026-09-16 15:44:38 +12:00
Fraser
1b0ff2917f Preserve the visual-test required check for Loki (#82539) 2026-09-16 15:44:27 +12:00
bryan
4a3d32ae0a Bind inline values in app-DB queries with [:auto/param ...] (#82373)
* Require app-DB value slots to hold bound parameters

HoneySQL resolves a map in a value slot toward SQL structure rather than
data, so a value can reach the database as syntax instead of as a bound
parameter. `[:param ...]` is the only form that always binds.

Adds `metabase.app-db.value-guard`:

  - `auto-param` rewrites inline `[:param* v]` markers into HoneySQL's
    `[:param :kN]` plus the params map, so callers do not keep a separate
    map in sync.
  - `long*`/`longs` coerce ids, which covers the common lookup without
    param noise and rejects anything that is not a number.
  - `assert-values-wrapped!` walks the compiled query at the Toucan
    compile step and throws when a value slot holds something that could
    become SQL.

The walk is clause-aware: it descends only into `:where`/`:having`/`:set`/
`:values`/`:join-by` and knows which argument of each operator is a value,
so column and operator positions are left alone. An operator that is not
classified fails closed.

This is an allowlist and lives apart from `honeysql-guard`, which is a
blocklist of known shapes and is backported to older versions.

Two forms needed handling beyond plain scalars and params: h2x wrappers,
which are checked by their payload rather than trusted by their wrapper,
and HoneySQL's `:%fn.col` call shorthand, which is a closed enumerated set
rather than a prefix test so that a constructed keyword cannot pass.

`strict-modules` is empty for now. A module joins it once its call sites
carry params or coercions, at which point plain scalars are rejected there
too.

* Accept java.time values in value slots

Toucan's before-insert hooks add created_at/updated_at as ZonedDateTime,
which clojure.core/inst? does not cover, so every insert was rejected.
All java.time.temporal.Temporal values bind as parameters.

* Check nested values, report the offending leaf, defer installation

Three changes, all from measuring the check against a real module.

A dev-authored marker on a subquery asserts that its SQL is dev-authored.
It says nothing about the values inside it, which are just as reachable
from a request, so the check now descends into a marked subquery instead
of passing it wholesale. `honeysql-guard` also descends here, but with a
blocklist, so a nested unwrapped value passes it today.

When a nested value was rejected the error named the enclosing subquery
rather than the value, which pointed debugging at the wrong place. The
nested check now throws on the leaf.

The check is no longer installed at the Toucan compile step. Against the
`collections` module it errored 91 of 148 tests, and none of those was an
unparameterized user value -- they were correlated subqueries, a UNION
used as an operand, and SQL function calls in `:set`, all of which are
real SQL that happens to sit in a value-bearing clause. Installing it
globally is therefore not the right first step; per-namespace opt-in is,
and that opt-in has to key off the calling namespace rather than the
model, since every Toucan model keyword is `:model/Something` and carries
no module information.

* Adopt the value check in the content-translation queries

Gives the check one real caller instead of leaving it unused. `locale`
reaches `translations-for-locale` from a request query parameter, so it is
coerced where it enters the query and the query is checked before it runs.

Adoption is an ordinary function call at the site it protects -- no compile
hook, no ambient state -- so the next namespace adopts independently and
nothing changes for namespaces that have not.

Adds `str*` alongside the id coercions. It asserts a value is already a
string, which rejects the maps and keyword vectors HoneySQL would compile as
SQL while leaving a legitimate string untouched.

`checked` deliberately does not demand bound parameters. A coercion returns
a plain scalar, indistinguishable at runtime from an unwrapped one, so that
stronger property cannot be asserted from the compiled form.

Also records a blocker found while writing this: `[:param ...]` needs its
values in a separate `:params` map, and `honeysql-guard` rejects any unmarked
nested map, so that map never reaches HoneySQL. `[:param]` is therefore
unusable through Toucan until `honeysql-guard` learns about `:params`, which
is a change to a backported namespace and is left alone here.

* Scope the value check to namespaces that have adopted it

Runs the check inside Toucan's compile step again, but only for queries
issued from a namespace listed in `enforcing-namespace-prefixes`. A query
from anywhere else compiles exactly as before, so a namespace adopts on its
own schedule and nothing has to be switched on everywhere at once.

The alternative was a `checked` call at each query site, which only works as
long as everyone remembers to write it. Putting it in the pipeline means an
adopting namespace cannot forget.

Scoping by namespace rather than by model because a model is shared: opting
in `:model/Card` would opt in collections, search, dashboards and
serialization together, which is the big-bang this is trying to avoid. A
`db.clj` is the unit a team actually cleans up, and a prefix entry covers a
whole module once it is ready.

`enforcing-caller?` reads the call stack rather than an ambient binding. The
check runs several frames below the caller inside Toucan, so a dynamic var
would have to be threaded through Toucan internals and would not survive a
thread hand-off; the stack already carries the answer. Measured at ~4us per
query against a DB round trip of hundreds, and only paid at the compile step.

`enforcing-namespace-prefixes` is public so a clj-kondo hook can read the
same set and check adopted namespaces at author time.

Tests prove both directions: a query from the adopting namespace is rejected,
and the same query from a non-adopting namespace is not. A bare keyword in a
value slot is the discriminator, since `honeysql-guard` permits it and this
check does not. `collections` stays at the baseline 1958 assertions, 0 errors.

* Bind values inline with [:auto/param ...]

Makes the design doc's ergonomic half real. A caller writes the value where
it belongs and `bound` lifts it into the params map HoneySQL wants, so the
two never have to be kept in sync:

    (bound [query {:where [:= :locale [:auto/param locale]]}]
      (t2/select :model/ContentTranslation query))

Two things had to be fixed for this to reach the database.

`honeysql-guard` rejected the `:params` map as an unmarked nested map. It
now treats `:params` like `:set` and `:values` -- keys that hold values
rather than nested queries -- so its contents are still checked. Nothing in
`:params` can become SQL regardless of shape, since HoneySQL binds it as a
parameter, so checking it catches a value JDBC would later refuse rather
than an injection.

HoneySQL also takes params as a format option rather than a query clause, so
`bound` passes them through `toucan2.honeysql2/*options*`.

The marker is `:auto/param` rather than a bare keyword so it cannot collide
with a HoneySQL operator, and an unresolved one now throws: HoneySQL would
otherwise compile `[:auto/param v]` into a `PARAM(?)` function call, so
forgetting `bound` would produce wrong SQL instead of an error.

* Lift [:auto/param ...] in the pipeline instead of at the call site

A caller now writes the marker inline and nothing else:

    (t2/select :model/ContentTranslation
               {:where    [:= :locale [:auto/param locale]]
                :order-by [:msgid]})

The compile step lifts every marker into the params map and hands it to
HoneySQL, so the query is an ordinary Toucan query and the wrapper macro is
gone. One less thing to remember, and a query that forgets the wrapper is no
longer possible to write.

The method is `:around` rather than `:before` because it has to replace the
query it passes on, not only inspect it.

An unresolved marker still throws. It now means the query reached the
database by some route other than the compile step, since anything through
the pipeline gets lifted.

* Pin what an unlifted marker does, and correct why a hostile value is refused

HoneySQL does not recognise `:auto/param`, so a marker handed straight to
`sql/format` compiles as a call to a function named PARAM over its argument.
The database then refuses the statement -- `Function "PARAM" not found` --
so it is broken rather than dangerous, but it is worth pinning, since the
marker only does its job on the path that lifts it.

Also corrects a test comment that credited JDBC with refusing a hostile
non-scalar. Through Toucan it never reaches the driver: `honeysql-guard`
rejects the unmarked map at the compile step first.

* Check every clause that can hold a value, not a list of five

The walk inspected `:where`, `:having`, `:set`, `:values` and `:join-by`.
HoneySQL has about 92 clauses, so that missed real ones -- notably join ON
conditions, which are the most used clause family in the codebase (~650
sites across `:join`, `:left-join` and friends), plus `:do-update-set` and
`:on-duplicate-key-update`.

Worse than the omissions was the direction. Unknown *operators* failed
closed, but an unlisted *clause* was skipped silently, so a clause nobody
had classified went unchecked. The walk now lists the clauses that hold only
structure -- `:select`, `:from`, `:order-by`, the DDL -- and checks
everything else, so a clause added to HoneySQL later is inspected rather
than ignored.

A join clause is a flat sequence alternating target and ON condition, and
both are keyword-headed vectors. The operator table tells them apart: a
condition starts with a known operator, anything else is a table.

Also drops `long*`, `longs` and `str*`. They were `clojure.core/long` and
`str` with docstrings -- an alias layer with no added behaviour, and nothing
used them once the marker existed. `[:auto/param]` is what the walk can
actually see; a coercion returns a plain scalar it cannot distinguish from
an uncoerced one. GHY-4475 wants amending to match.

* Load the check from app-db core, and move test scaffolding out of src

Removing the `keep-me` no-op exposed a real gap: nothing loaded this
namespace, so neither the marker lift nor the check ran in a running system.
It was masked while the adopting namespace still required it. `app-db.core`
now requires it the way it requires `honeysql-guard`, and `keep-me` is a
reference inside `comment` rather than a defn, matching the convention.

The test-only query that proved the check is scoped to its caller no longer
lives in `src`. The test namespace adds itself to the enforcing set instead,
which works now that the munged class prefixes are derived on each call
rather than captured once at load; two copies of that set could drift.

Adds a test that an unmarked string is compared as a literal. A string was
never the danger -- HoneySQL binds it -- and saying so keeps the marker's
purpose clear: it is for values that are not strings, which HoneySQL would
otherwise compile as structure.

Records that the marker belongs in a query map rather than a kv-arg. Toucan
builds kv-args into the where clause after the lift runs, so a marker passed
that way ends up inside a PARAM call instead of a comparison.

* Reduce to the value binding, and drop the walk

Review found the walk failing open in every position it did not explicitly
classify -- a map reaching `check-clause!` returned nil rather than throwing,
which left `{:where [:and [:= :a 1] {:raw "1=1 OR true"}]}` compiling to
`WHERE (a = ?) AND (1=1 OR true)`. The left-hand side of every comparison,
arguments past the declared indexes, `:join-by` conditions, and
`:limit`/`:offset`/`:fetch` were all unchecked for the same reason. A check
that advertises a guarantee it does not provide is worse than no check, so
the walk is gone rather than patched. It can come back built fail-closed.

What remains is the part that worked: a `[:auto/param v]` marker is lifted
at the compile step into HoneySQL's `[:param :kN]` plus its params map, so a
value written inline binds as `?`.

Four bugs in that half are fixed. `postwalk` descended into a marker's own
payload, so a nested marker bound a `[:param]` vector instead of the value;
it now uses `prewalk` and takes the payload whole. Toucan re-enters `compile`
with the `[sql & args]` vector it produced, so the method ran twice per
query; it now only walks a map. Param keys were sequential and guessable,
letting a `[:param :p1]` from request data bind another slot's value; keys
are now random. The options binding replaced `:params` wholesale, dropping an
enclosing compile's params; it merges.

`honeysql_guard.clj` is reverted. `:params` is never a key in a query map --
it reaches HoneySQL as a format option -- so that change was dead code in a
file backported to v58, v61 and v63, and the comment claiming `[:param]`
needed it was wrong.

* Reject a malformed marker, and do not treat a map entry as one

A marker-headed vector of the wrong arity now throws. HoneySQL does not
recognise the leftover form and compiles it into a call to a function named
PARAM, so `[:auto/param v extra]` previously failed at the database with an
error pointing nowhere near the typo.

A `MapEntry` is a two-element vector whose first element is its key, so the
map `{:auto/param v}` would otherwise look like a marker and the map holding
it would be rewritten away. That map means "a column named :auto/param", so
reaching it takes misuse rather than ordinary use -- but excluding the type
costs one check, so the walk does it rather than leaving the shape to chance.

Testing `vector?` instead of `sequential?` does not help here, since
`MapEntry` implements `IPersistentVector`.

This also drops support for a marker written as a seq. Nothing writes one,
and the looser test was widened for surface area rather than a caller.

Adds tests for the write path and for a query with no markers, neither of
which the unit tests reached: they exercise `auto-param` directly, so the
pipeline method's own branches were untested.

* Accept one marker shape, and never let one reach SQL

`[:auto/param value]` is the one supported form. Anything else headed by the
marker -- a seq, the wrong arity -- now throws instead of being left to
compile, because HoneySQL does not recognise the keyword and turns whatever
is left into a call to a function named PARAM that fails at the database
with nothing pointing back to the query.

Two assertions hold the line. Nothing marker-shaped may survive the rewrite,
and every value lifted out has to come back as a bound argument: a marker in
a slot HoneySQL formats as an identifier is rewritten but never consumed, so
`{:from [[:auto/param :t]]}` silently produced `FROM param AS p3lg...`,
dropping the value and putting the generated key in the SQL text. That now
throws.

A marked nil is left to HoneySQL rather than bound. It turns a literal nil in
a comparison into `IS NULL`, while a bound parameter gets `= ?`, which no row
satisfies -- so wrapping a nullable value silently returned nothing.

A `MapEntry` is still excluded from the marker test, so a column named
`:auto/param` is left alone rather than rewritten away.

* Look for the marked value among the arguments, not for a count

`assert-every-value-bound!` compared how many values were lifted to how many
arguments came back, which any arguments satisfy -- not specifically the ones
that were lifted. A query carrying unmarked literals has arguments to spare,
so a dropped marker hid behind them:

    {:from [[:auto/param :t]] :where [:and [:= :a 1] [:= :b 2]]}
    ;; => FROM param AS p16vl... WHERE (a = ?) AND (b = ?)   -- two args, one
    ;;    param, count satisfied, value silently gone

It now looks for each marked value among the arguments. A sequential value is
spread across one argument per element, so its elements are looked for rather
than the collection; anything else, including the map this exists to
neutralize, binds whole.

Tests cover the shapes where the number of arguments and the number of marked
values disagree: a collection, the same value marked twice, and marked values
sitting alongside unmarked ones.

* Drop the bound-value assertion, and say what is not checked

`assert-every-value-bound!` rejected ordinary queries. It inferred from a
value's Clojure type whether HoneySQL would spread it across several
arguments, but that is a decision HoneySQL makes per operator: `:in` spreads
a collection, `=` binds one whole. So a vector compared with `=`, and a set
in an `:in`, were each checked the wrong way and threw. A collection holding
`nil` or `false` threw as well, because the check used a set as a predicate
and a set returns nil for those members even when it contains them.

It was also unsound in the direction it existed for. Comparing against the
argument list loses which argument came from which value, so a dropped
marker passed whenever an unrelated literal elsewhere happened to equal it:
`{:from [[:auto/param "de"]] :where [:= :msgid "de"]}` was accepted, with the
value gone and the generated key left in the SQL as an identifier.

No post-hoc check can be sound here -- HoneySQL gives no signal for a param
it did not consume -- so the namespace docstring now states the limitation
instead of a check pretending to cover it. A marker still may not survive the
rewrite, and a malformed one still throws; both of those are about the marker
itself rather than about what HoneySQL did with it.

* Lint a converted namespace's values for a missing marker

Marking a value does nothing to keep it marked: a later edit adding an
unmarked value slot to a converted namespace passes silently. This adds the
author-time check that holds the line, extending the hook that already
enforces `db.clj` confinement rather than adding a second one -- the two are
halves of the same rule, and that hook already knows which namespaces are
`db.clj`.

It reports a symbol reaching a value slot without a marker, following the
clause shapes a query map uses. Only a symbol: a literal cannot carry a
request value, and a value built inside the function is out of reach of a
check that does not follow it across a call. Only a `db.clj`, which is what
makes it decidable -- the values there are named in the parameter vector a
few lines up.

Off by default, enabled per namespace. Run globally it reports 269 sites, of
which 196 are ids that Malli already constrains to an integer and that
therefore cannot become SQL. Warning about those would bury the 73 that
matter -- strings and enums like `perm-type`, `setting-key`, `namespace`. A
namespace is added to `:config-in-ns` once its own values are marked, so the
list grows with the migration rather than ahead of it.

`metabase.content-translation.db` is the first entry. Removing its marker now
fails the lint.

* Bind a marked kv-arg, which is how queries are actually written

`(t2/select :model/X :locale [:auto/param v])` did not work, and kv-args are
how nearly every query in the codebase is written: 1125 call sites across
`*/db.clj` against 3 that pass a query map. So the marker covered the style
almost nobody uses.

Toucan builds a kv-arg into the where clause before the compile step sees it,
taking the marker keyword for the operator and folding the column in, so
`:locale [:auto/param v]` arrives as `[:auto/param :locale v]`. That arity
was previously rejected as malformed. It is now recognised and rewritten to
`[:= :locale [:param k]]`, which is the comparison Toucan would have built.

A marked kv-arg holding nil still compares as `IS NULL` rather than binding,
matching an unmarked one.

The lint reads the same style now. With it enabled across `src` it reports
1006 sites rather than the 269 it saw when it only walked query maps, of
which 769 are id-shaped and 237 are not -- close to the 78% the project's
estimate predicted, and the 237 are the ones that need a marker rather than
a coercion.

* Pair kv-args from the model, and do not lint written values

Several query fns take an argument before the model -- (t2/select-one-fn
:value :model/Setting :key k) -- so pairing the kv-args from a fixed offset
misaligned them and the values went unreported. They are now paired from the
model keyword, which is where the pairs actually start.

Insert calls are no longer linted for unmarked values. A value written into a
row is not a where-clause value an attacker can turn into SQL structure, and
a define-before-insert hook may read it before the query compiles, so a
marker there breaks the write rather than protecting anything.

* Reject a marked operator form, and check every argument of one

A kv-arg value may itself be an operator -- `:locale [:in [...]]` -- and
Toucan folds that into `[:auto/param :locale [:in [...]]]`, which the kv
rewrite turned into `[:= :locale [:param k]]`. That binds the operator form
as a single value and changes the comparison, so `IN (?, ?)` became `= ?`
against a list. It now throws and points at the working form, which is the
marker inside the operator rather than around it: `[:in [:auto/param vs]]`.

The linter examined an operator's arguments only when there were exactly
two, so `[:between :col lo hi]` and any other arity went unchecked -- a false
negative, which is the expensive direction for this lint. It now takes every
argument after the column, and `between` joins the operator set.

`read-call?` no longer assumes the call head is Named. Kondo always passes a
resolved var so this was unreachable, but the failure would have been a
linter crash rather than one bad file.

Also drops a duplicated paragraph that described the key generation as
gensymed when it uses ThreadLocalRandom, and replaces the ns docstring's
description of a misplaced marker with the statement it actually produces.

* Lint the query shape as well as its values, under one finding

A query written as `:column value` pairs is now reported alongside a value
that reaches a slot unmarked, under a single `:metabase/unsafe-app-db-query`
finding. A namespace opts into both at once, and the pair can be flipped on
globally when every `db.clj` has been converted.

Toucan builds kv-args into the where clause itself, so the query's shape is
not on the page -- there is nothing for a reader, this linter, or the `.sql`
extraction that follows this project to read. Every call has a map form:
`{:where [...]}` for a select, and for `update!` a conditions map, since its
spec takes `conditions-map-or-query?` rather than only pairs.

Checking the shape rather than the pairs' values also removes the reason the
marker had to understand a kv-arg at all. The value check now walks query
maps only.
2026-09-16 03:27:26 +00:00
Alexander Polyankin
48d493c293 Fix the code review findings of the closed argument schemas PR (#82537) 2026-09-15 22:44:21 -04:00
Andrei Chernykh
01f2c4b624 Redesign the LDAP settings page and its group mapping card (#82389)
* Redesign the LDAP settings page and its group mapping card

* Keep the mapping editor fields aligned when a name is rejected

* Merge the LDAP group mapping e2e flows into one

* Address the LDAP page review feedback

* Drop comments that restate the code

* Lock the LDAP attributes card and group the locked cards
2026-09-15 19:37:24 -07:00
Romeo Van Snick
86a761845a Read a card's UI parameters off the question (#82466) 2026-09-16 12:03:10 +10:00
github-automation-metabase
c19c1bfdeb Auto-update documentation for master (#82532)
Update generated docs (master)

Co-authored-by: github-actions <github-actions@github.com>
2026-09-16 02:02:55 +00:00
bryan
47644f8e40 Raise describe timeout in handle-circular-refs-test (DEV-2333) (#82540)
`handle-circular-refs-test` has been failing intermittently since #82447.

The last assertion wraps `(umd/describe :metabase.lib.schema/query)` in a
500ms `with-timeout`. That timeout exists as a smoke test against runaway ref
expansion -- before `max-depth` was added to `metabase.util.malli.describe`,
describing this schema took literal minutes. It was never meant to be a
performance budget.

#82447 required closed map schemas in `mu/defn`, which enlarged the schemas
`describe` walks to its `max-depth` of 3. Since expansion cost is roughly
exponential in branching factor, this made the call ~4.5x slower. Measured on
one machine, isolated runs:

    before #82447: 89ms cold, 33-50ms warm
    after  #82447: 232ms cold, 158-193ms warm

Headroom against the 500ms budget dropped from ~14x to ~2.8x. The test is
`^:parallel` and `with-timeout` runs its body in a `future`, so it competes for
the thread pool with every other parallel test; under CI contention ~175ms
crosses 500ms. That is why it presents as flaky rather than consistently red,
and why it showed up on the Postgres 14 and Java 25 suites -- those runners are
slower/more contended, not semantically different.

The describe output itself is unchanged (identical 187-char string), so only
speed regressed, not correctness.

Raising the timeout to 5000ms keeps the runaway-recursion guard meaningful (that
failure mode is minutes, not milliseconds) while surviving pool contention.

Not treating the 4.5x as a perf bug to chase: `umd/describe`'s only production
caller is the error-explanation path in `metabase.api.macros`, reached only
after validation has already failed, and it describes the nested sub-schema
rather than the full query schema. Validation itself is unaffected by #82447
(sub-microsecond before and after).
2026-09-16 01:55:21 +00:00
Romeo Van Snick
980c6b0018 Measure each commit in German and in English (#82536)
Claude-Session: https://claude.ai/code/session_01171Lkmug15JzrcN2aMBLMk

Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-16 11:18:22 +10:00
Fraser
e677b6b267 Use affected-test plans to narrow Loki visual tests (#82050) 2026-09-16 12:37:30 +12:00
Romeo Van Snick
548573f22d Move the public question and the expression editor off the Metadata object (#82313)
* Build the public question from the store

* Pass the expression editor its database, not the Metadata object

* Build the FieldValuesWidget test metadata with createMockMetadata

* Name the endpoints the public question's parameters come from
2026-09-16 08:56:28 +10:00
Aleksandr Lesnenko
00232ddc3a drop disable mcp tools setting (#82533) 2026-09-15 22:00:53 +00:00
bryan
f6caace66a Refuse to save a query handle that carries bound parameter values (#82434)
Refuse to save a query handle carrying bound parameters

A handle minted by execute_sql re-attaches the :parameters it ran with, but
:parameters is runtime-only and serialize-query strips it on the way into
dataset_query. Saving such a handle persisted the query without its filter and
without complaining, so the card returned rows the agent's own run excluded.

Fail closed in resolve-query-handle-for-save!, the chokepoint shared by
question_write, metric_write and transform_write, with a teaching error naming
the concrete alternative.

Closes GHY-4547
2026-09-15 14:51:30 -06:00
technomancy
ba327647bc Update oldest MySQL version to 8.4. (#82507)
According to https://endoflife.date/mysql version 8.0 went EOL in
April. Now 8.4 is the oldest supported release.
2026-09-15 23:34:51 +03:00
github-automation-metabase
dfbff65b08 Update translations 2026-09-15 (#82474)
* Update translation sources 2026-09-15

* Update translations

---------

Co-authored-by: github-actions <github-actions@github.com>
2026-09-15 20:26:58 +00:00
github-automation-metabase
f787cc5b89 Tighten ratchets (#82523)
Co-authored-by: github-actions <github-actions@github.com>
2026-09-15 20:02:24 +00:00
bryan
1b57292d09 Send the SAML SessionIndex in single logout requests (#82441)
* Send the SAML SessionIndex in single logout requests

Metabase's LogoutRequest identified the session to end by NameID alone, which
is the user's email. Auth0 issues a distinct user ID per connection, so one
email can own several sessions and Auth0 cannot tell which to end:

    client_error: No active session(s) found matching LogoutRequest

SAML Profiles 4.4.4.1 makes SessionIndex the element that scopes a
LogoutRequest to a session. Record the SessionIndex the IdP sends in its login
assertion on the session it creates, and send it back at logout.

SessionIndex is per session, not per user, so it lives on core_session rather
than auth_identity, which has a unique constraint on (user_id, provider) and
would let concurrent logins overwrite each other.

A hardcoded SessionIndex was tried once and broke Azure AD (#41600). Only an
index the IdP itself issued is ever sent; sessions without one (every session
predating this column, and every non-SAML session) produce exactly the request
they produce today. The existing saml-logout test, which asserts on the full
signed logout URL for a session with no SessionIndex, passes unchanged.

Requires saml20-clj 4.3.0, which parses SessionIndex out of login assertions
and can emit it in a LogoutRequest.

Fixes #77837

* Send the IdP's NameID, not the user's email, in logout requests

An IdP matches a LogoutRequest on NameID plus SessionIndex. We sent the user's
email as the NameID, but the IdP's NameID is not necessarily an email: Auth0
sends an opaque `auth0|<id>`. A LogoutRequest naming an identifier the IdP never
issued matches no session, so Auth0 answered:

    No active session(s) found matching LogoutRequest

even once we started sending the correct SessionIndex.

Record the NameID and its Format from the login assertion alongside the
SessionIndex, and send those at logout. Sessions predating these columns have a
NULL NameID and fall back to email, so logout keeps working across an upgrade.

Verified end to end against a live Auth0 tenant: the LogoutRequest now carries
NameID `auth0|<id>` with Format `unspecified` and the assertion's SessionIndex,
Auth0 returns StatusCode Success, and the session row is deleted.
2026-09-15 13:56:42 -06:00
Chris Vandoodewaard
6fdc4439da record E2E statistics (#82496)
* add script to upload e2e test timings

* update workflow, add tests

* fix team name
2026-09-15 14:27:09 -04:00
Chris Vandoodewaard
ad51ac8b78 only run some semantic search tests on PRs (#82440)
* only run some semantic search tests on PRs

* add label to force run semantic search tests

* fix whitespace

* trigger ci
2026-09-15 14:26:17 -04:00
Michael Eby
94aea9ae24 docs: 2FA v2 (#81962)
* docs: 2FA v2

* verify admin path and require options

* edit from jeff
2026-09-15 12:46:43 -04:00
Alexander Polyankin
e79f40e3d3 Require typed, closed argument schemas in mu/defn (#82447) 2026-09-15 16:45:35 +00:00
Nick Fitzpatrick
9ce408d114 Uxw 4979 manage unsynced content modal (#81226)
* Manage Unsynced Content modal

* add actions

* Show dependies when attempting to remote sync

* adjust spacing of dependency rows

* handling snippet and dashboard dependencies

* reshaping API response for FE consumption

* save from modal again

* cleanup

* PR Feedback, Design Feedback

* handle instance analytics case

* fixing function string

* Mike's feedback

* fix getIcon hook
2026-09-15 13:16:19 -03:00
Stas Gavrylov
6f423c147d GDGT-3199: Stabilize document comment tests (#82480)
* GDGT-3199: Stabilize document comment tests

* GDGT-3199: Assert editor focus before typing
2026-09-15 18:17:56 +03:00
Stas Gavrylov
7b271d25b6 GDGT-3108: Stabilize document revision history test (#82484)
* GDGT-3108: Stabilize document revision history test

* GDGT-3108: Apply review feedback

* GDGT-3108: Target final toast assertion by text
2026-09-15 18:17:46 +03:00
Will Fry
75398ad620 GDGT-3131: refactor ai usage stats page to use nested routes (#82420) 2026-09-15 15:48:28 +01:00
Stas Gavrylov
34c44f0f7f GDGT-3138: Make inline update rollback deterministic (#82418)
* GDGT-3138: Make inline update rollback deterministic

* GDGT-3138: Clarify reinitialization behavior

* GDGT-3138: Wait for incremental switch readiness

* GDGT-3138: Cleanup
2026-09-15 16:44:05 +03:00
Chris Truter
968b105665 Never send the model an empty user message from a Slack thread (#82443)
A file shared with no text, or a message that only mentions the bot, reached the thread history as an empty user
message, and Anthropic rejects the whole request for it. Name the attachments instead, or leave the message out.
2026-09-15 15:30:26 +02:00
Chris Truter
aecbc3c3cb Warm the Kondo cache so cache-reading hooks run in CI (#82388)
* Warm the Kondo cache so cache-reading hooks run in CI

`hooks/ns-analysis` reads only `.clj-kondo/.cache`, and Kondo writes that cache after every file in a run has
been analysed. `mage kondo` deleted the cache and linted in one pass, so a hook that asks about another
namespace's vars found nothing and skipped its check. `:metabase/prefer-with-dynamic-fn-redefs` therefore never
fired in CI while warning locally against a warm cache.

Lint once and discard the findings before the real pass. `kondo-redundant-ignores` and `kondo-insert-ignores`
get the same warm-up: on a cold cache the former reports needed suppressions as redundant.

Turning the linter on exposes 327 flagged `with-redefs` forms. 316 become `mt/with-dynamic-fn-redefs`. Nine keep
`with-redefs` where the redefined function runs on a thread a dynamic binding does not reach, or on a value the
proxy cannot wrap. Two `.cljc` sites are covered by a namespace entry in config.edn instead, since the macro is
JVM only.

Delegating replacements capture the original with `mt/original-fn`. A bare symbol or `@#'the-var` resolves to
the proxy once a var has been patched, which recurses on the second test in a JVM.

Inline ignore budget 28 to 37, config suppressions 3 to 4.

* Stop counting prefer-with-dynamic-fn-redefs ignores

Genuine cases are common: Quartz workers, pool callbacks, queue delivery workers and values the proxy cannot
wrap all need a root swap. A numeric ceiling adds churn to every PR that meets one. Each ignore still needs a
comment explaining why.

* Harden the Kondo warm-up and keep a primitive fn on with-redefs

`next-page-size` in the BigQuery driver takes and returns primitive longs, so compiled callers invoke it
through `IFn$LLLLL`. The dynamic-redefs proxy is a plain variadic fn, so the conversion threw
ClassCastException, and since patching is permanent every later BigQuery query in the JVM would too.

The warm pass now also lints `dev/src`, which defines vars that `dev/test` redefines. That exposes six sites
in the kondo ratchet tests, now converted. A warm pass that exits with anything but 0, 2 or 3 fails the
task rather than leaving a partial cache. `kondo-updated` warms when no cache exists, and the ratchet tools
warm the roots they lint.

* Only skip kondo-updated's warm pass after a finished one

A cache directory can be partial, left by an editor or an interrupted run, and `kondo-updated` then skipped
warming and missed cache-dependent findings. `warm-cache!` now writes a marker when it finishes, deleting it
first so an interrupted pass can't leave an old one, and `kondo-updated` checks for the marker instead.

* Only let a full warm pass mark the Kondo cache warm

A warm over a narrower set of roots leaves the cache incomplete, so it must not write the marker
`kondo-updated` trusts, nor clear one that a full pass wrote.

* Point the Athena TODO at the macro the test now uses

* Say why kondo-updated accepts a stale cache
2026-09-15 15:30:02 +02:00
github-automation-metabase
5c18aea2c7 Auto-update documentation for master (#82444)
Update generated docs (master)

Co-authored-by: github-actions <github-actions@github.com>
2026-09-15 09:14:04 -04:00
github-automation-metabase
18a6ad127c Widen snowplow-micro poll budget to de-flake event assertions (#77383)
Snowplow events are POSTed asynchronously and then ingested by
snowplow-micro before they can be read back via `micro/good`. The
1000ms retry budget in the Cypress helper was too tight for the event
to land on loaded CI runners or under network throttling, producing
spurious "Snowplow retry timeout" failures. Bump the budget to 5000ms;
this only affects how long a failing assertion waits, since matching
events still return early.

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-09-15 14:30:40 +02:00
Fraser
afd98b94a5 Repoint embedding-sdk's ClickObject import to metabase-lib (#82455) 2026-09-15 23:22:26 +12:00
Phoomparin Mano
9a5524063b Infer semantic schema export scope from the data app prompt (#82221)
add scope guidance for agents
2026-09-15 17:01:39 +07:00
Romeo Van Snick
d46d7a47e4 Read the reference lists from their own responses (#82473)
* Drop the unused metadataFields prop from the reference detail pages

* List the reference databases from the fetched response

* Fetch the reference segment list where it is rendered
2026-09-15 19:54:14 +10:00
Romeo Van Snick
8d67bf7e3d Keep the segment retire action closed until its table loads (#82471) 2026-09-15 19:54:00 +10:00
Will Fry
4084c9254b GDGT-3219: fix broken navigation from unsaved sql (#82433) 2026-09-15 08:30:18 +01:00
Romeo Van Snick
5f8f065921 Read the embedding data picker's entities from the store (#82407) 2026-09-15 15:49:11 +10:00
Romeo Van Snick
7ff180c80a Read the table and database from their own requests, not the entities mirror (#82472) 2026-09-15 05:46:55 +00:00
Romeo Van Snick
6964b63d2b Record when the page draws on the second and later visits (#82467)
* Record when the page draws on the second and later visits

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01171Lkmug15JzrcN2aMBLMk

* Wrap the two long readings for the formatter

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01171Lkmug15JzrcN2aMBLMk

---------

Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-15 14:44:59 +10:00
Romeo Van Snick
98486ccd56 Restore the changes #82405 reverted (#82463) 2026-09-15 03:33:50 +00:00
Fraser
7e9e46602f Give types, settings-components and embedding-ee shared-tier levels (#82462) 2026-09-15 14:50:15 +12:00
Fraser
7ea204ed6e Enforce the shared tiers for api (#82460) 2026-09-15 14:50:04 +12:00
Romeo Van Snick
7e36117dbc Read the data selector's entities from the store (#82405)
* Add relation selectors to the metadata store

* Read the data selector's entities from the store

* Take the data selector's entities as plain objects in its consumers

* Format the data selector spec

* Inline the table schema lookup

* Name the data selector's lookups prop entityLookups
2026-09-15 11:59:21 +10:00
Nick Fitzpatrick
72ea707064 Add dataPointsVisible extension to eCharts (#81721)
* useRenderedDataVisibility

* static viz, other adjustments

* remove dead hook

* Split panels, simplification

* handling line series with a range very close to chart boundary, ignore goal line

* return true on traversal if intersection is found
2026-09-14 22:44:43 -03:00
Fraser
781e650301 Move security-center endpoints into enterprise security_center (#79351)
* Move security-center endpoints into enterprise security_center

Moves security-center.ts out of the shared metabase/api module into
enterprise/frontend/src/metabase-enterprise/security_center/api.ts. All
of its endpoints hit /api/ee/ URLs and every consumer lives in the EE
security_center module, so it was an OSS api file with zero OSS
consumers.

The file now uses EnterpriseApi.injectEndpoints, matching the existing
in-module EE precedents (monitor/ai-auditing/metabot-analytics/api.ts,
table-editing/api/index.ts). The security-advisory tag type moves from
the OSS TAG_TYPES to ENTERPRISE_TAG_TYPES alongside it.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* Cast test store state for EnterpriseApi selectors

The listSecurityAdvisories endpoint selector is now typed against the
enterprise tag union, while renderWithProviders builds a store typed
with the OSS State. The runtime state shape is unchanged, so the spec
casts through a helper with a justification comment.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* Use relative imports for EnterpriseApi inside the enterprise module

security_center/api.ts was reaching its own module through the
metabase-enterprise alias; same-module imports are relative per the
import-style convention.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* Import EnterpriseApi through the metabase-enterprise alias

../api/api reaches the sibling api module, not security_center's own
files, so the same-module-imports-are-relative convention does not
apply here. The alias form matches the other in-module EE endpoint
owners (metabot-analytics, table-editing) and is the specifier the
no-base-api-access rule tracks EnterpriseApi by.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* Rewrap the cast comment in the promo card spec

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* Verify security-center mutations invalidate the advisory cache

* Simplify security-center tests and flush query updates

* Match the security-center api spec titles to the folder's should idiom

---------

Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-09-15 11:37:42 +12:00
Fraser
53f48ed3be Use affected-test plans to narrow frontend unit tests (#82049)
* Use affected-test plans to narrow frontend unit tests

* Pass affected test plans by artifact and reject Jest runner failures

* Extract unit test selection preparation from workflow YAML

* Follow the shared Jest config files in the frontend path filters

* Drop the unused import/no-commonjs directive from the filter spec

* Leave heap retention specs out of the planner's unit list

* Check unit test results in a script and let suite hook failures reach quarantine

* Warn when a narrowed unit test run matches none of Jest's test paths

* Reword the workflow cast comment in the test plan spec
2026-09-15 11:04:56 +12:00
1631 changed files with 73909 additions and 39310 deletions

View File

@@ -74,6 +74,10 @@
:metabase/prefer-with-dynamic-fn-redefs {:level :warning}
:metabase/t2-query-namespace {:level :warning}
:metabase/table-or-field-query {:level :warning}
;; App-DB query hygiene: a value reaching a value slot unmarked, and a query written as
;; `:column value` pairs rather than a map. Off by default and enabled per namespace under
;; `:config-in-ns`; flip it on globally once every `db.clj` has been converted.
:metabase/unsafe-app-db-query {:level :off}
:metabase/toucan-model-ns {:level :warning}
:metabase/require-shape-checker {:level :warning}
:metabase/tests-must-live-in-known-modules {:level :warning}
@@ -591,6 +595,7 @@
clojure.core/defprotocol hooks.clojure.core.def/lint-def
clojure.core/defrecord hooks.clojure.core.def/lint-def
clojure.core/deftype hooks.clojure.core.def/lint-def
clojure.core/ex-info hooks.metabase.agent-message/lint-ex-info
clojure.core/ns hooks.clojure.core.ns/lint-ns
clojure.core/require hooks.clojure.core.require/lint-require
clojure.core/requiring-resolve hooks.clojure.core.require/lint-requiring-resolve
@@ -606,7 +611,6 @@
metabase-enterprise.cache.config-test/with-temp-persist-models hooks.common/with-seven-bindings
metabase-enterprise.dependencies.api-test/with-dependents-test! hooks.common/with-two-bindings
metabase-enterprise.metabot-v3.tools.dependencies-test/with-dependent-transforms! hooks.common/with-two-bindings
metabase-enterprise.metabot.tools.dependencies-test/with-dependent-transforms! hooks.common/with-two-bindings
metabase-enterprise.semantic-search.db.connection/with-migrate-tx hooks.common/let-one-with-optional-value
metabase-enterprise.serialization.api-test/with-serialization-test-data! hooks.common/with-three-bindings
metabase-enterprise.serialization.test-util/with-temp-dpc hooks.toucan2.tools.with-temp/with-temp
@@ -686,6 +690,15 @@
metabase.lib.test-util.macros/mbql-query hooks.metabase.test.data/mbql-query
metabase.lib.test-util.macros/mbql-5-query hooks.metabase.test.data/mbql-query
metabase.lib.test-util.macros/with-testing-against-standard-queries hooks.metabase.lib.test-util.macros/with-testing-against-standard-queries
metabase.mcp.transport/jsonrpc-error hooks.metabase.agent-message/lint-jsonrpc-error
metabase.mcp.v2.common/ellipsize hooks.metabase.agent-message/lint-ellipsize
metabase.mcp.v2.common/error-content hooks.metabase.agent-message/lint-teaching-exit
metabase.mcp.v2.common/list-content hooks.metabase.agent-message/lint-list-content
metabase.mcp.v2.common/success-content hooks.metabase.agent-message/lint-success-content
metabase.mcp.v2.common/throw-teaching-error hooks.metabase.agent-message/lint-teaching-exit
metabase.mcp.v2.dashboard-ops/op-error! hooks.metabase.agent-message/lint-op-error
metabase.mcp.v2.message/msg hooks.metabase.agent-message/lint-msg
metabase.mcp.v2.tools.browse/paged-list-content hooks.metabase.agent-message/lint-list-content
metabase.mcp.v2.tools.parameters-test/with-fixtures hooks.common/with-one-binding
metabase.metrics.api-dimension-test/with-seeded-metric hooks.common/with-one-binding
metabase.model-persistence.api-test/with-persistence-setup! hooks.common/with-one-top-level-binding
@@ -898,7 +911,19 @@
;; run Toucan 2 queries anywhere.
;;
{:pattern "^metabase\\.app-db\\..*"
:name app-db-namespaces}]
:name app-db-namespaces}
;;
;; MCP source namespaces, OSS and enterprise, whose prose reaches an LLM agent, excluding test namespaces.
;;
{:pattern "^(?!.*-test(?:\\..*)?$)(?!.*test-util)metabase(?:-enterprise)?\\.mcp\\..*"
:name agent-message-namespaces}
;;
;; MCP source namespaces whose caller-facing errors are MCP exits: `metabase.mcp.transport` and `metabase.mcp.v2.*`,
;; and their enterprise counterparts. Other MCP namespaces' 4xx `ex-info`s are admin API and middleware errors, not
;; MCP exits.
;;
{:pattern "^(?!.*-test(?:\\..*)?$)(?!.*test-util)metabase(?:-enterprise)?\\.mcp\\.(?:transport$|v2\\..*)"
:name agent-exit-namespaces}]
:config-in-ns
{metabase.app-db.custom-migrations
@@ -924,12 +949,25 @@
app-db-namespaces
{:linters {:metabase/t2-query-namespace {:level :off}}}
agent-message-namespaces
{:linters {:metabase/agent-message-lines {:level :warning}}}
agent-exit-namespaces
{:linters {:metabase/agent-message-exit {:level :warning}}}
;; Namespaces whose values are marked. Adding one here is what keeps it that way; see
;; `metabase.app-db.value-guard`.
metabase.content-translation.db
{:linters {:metabase/unsafe-app-db-query {:level :warning}}}
;; .cljc tests run in both Clojure and ClojureScript — `with-dynamic-fn-redefs` is JVM-only,
;; so leave these alone.
metabase.lib.aggregation-test
{:linters {:metabase/prefer-with-dynamic-fn-redefs {:level :off}}}
metabase.util.time-test
{:linters {:metabase/prefer-with-dynamic-fn-redefs {:level :off}}}
metabase.lib.metadata.calculation-test
{:linters {:metabase/prefer-with-dynamic-fn-redefs {:level :off}}}
;; classloader sits below `util` in the module graph, so it can't use `metabase.util.log` without creating
;; a cycle; it gets to use tools.logging directly.

View File

@@ -116,7 +116,10 @@
util
warehouse-schema-overlay
warehouses}
:model-exports #{:model/Action}
:model-exports #{:model/Action
:model/HTTPAction
:model/ImplicitAction
:model/QueryAction}
:model-imports #{:model/Card
:model/Collection
:model/Dashboard
@@ -262,6 +265,7 @@
events
lib.schema
models
request
settings
util
util.config}
@@ -345,11 +349,12 @@
app-db
{:team "UX West"
:api #{metabase.app-db.cluster-lock ; TODO FIXME
:api #{metabase.app-db.cluster-lock ; TODO (Chris 2026-09-16) -- callers should go through app-db.core
metabase.app-db.core
metabase.app-db.init
metabase.app-db.setting
metabase.app-db.setup
metabase.app-db.sql-errors
metabase.app-db.transient-error}
:uses #{auth-provider
connection-pool
@@ -437,8 +442,15 @@
:api #{metabase.batch-processing.core
metabase.batch-processing.init}
:uses #{app-db
parameters
query-processor.cache-backend
settings
util}}
util}
:model-imports #{:model/Card
:model/Collection
:model/Dashboard
:model/Document
:model/Table}}
bookmarks
{:team "UX West"
@@ -505,6 +517,7 @@
metabase.channel.init
metabase.channel.models.channel
metabase.channel.render.core
metabase.channel.schema
metabase.channel.settings
metabase.channel.slack
metabase.channel.template.core
@@ -672,11 +685,11 @@
content-verification
{:team "UX West"
:api #{metabase.content-verification.core
metabase.content-verification.init}
:api #{metabase.content-verification.core metabase.content-verification.init metabase.content-verification.schema}
:uses #{app-db
lib.schema
models
users
util}
:model-exports #{:model/ModerationReview}
:model-imports #{:model/Card :model/Dashboard :model/User}}
@@ -734,14 +747,17 @@
metabase.dashboards.schema
metabase.dashboards.settings
metabase.dashboards.write}
:uses #{analytics
:uses #{actions
analytics
api
app-db
audit-app
collections
content-verification
embedding
events
lib
lib.be
lib.schema
models
parameters
@@ -786,6 +802,7 @@
dashboards
embedding
events
lib
lib.be
lib.schema
models
@@ -1004,7 +1021,7 @@
explorations
{:team "UX West"
:api #{metabase.explorations.core metabase.explorations.init}
:api #{metabase.explorations.core metabase.explorations.init metabase.explorations.schema}
:uses #{analytics
analytics.interface
api
@@ -1089,6 +1106,7 @@
events
lib.schema
models
remote-sync
util}
:model-exports #{:model/Glossary}
:model-imports #{:model/User}}
@@ -1126,8 +1144,8 @@
interestingness
{:team "UX West"
:api #{metabase.interestingness.core}
:uses #{util}}
:api #{metabase.interestingness.chart.types metabase.interestingness.core}
:uses #{lib.be util}}
lib
{:team "Querying Platform"
@@ -1148,6 +1166,7 @@
metabase.lib.normalize
metabase.lib.options
metabase.lib.pivot
metabase.lib.ref
metabase.lib.types.isa
metabase.lib.util
metabase.lib.walk}
@@ -1312,6 +1331,7 @@
:uses #{activity-feed
ai-tracing
api
api-scope
api.eid-translation
app-db
bookmarks
@@ -1394,14 +1414,17 @@
events
lib
lib.be
lib.metric
lib.schema
metrics
models
permissions
remote-sync
search
users
util
warehouse-schema-overlay}
warehouse-schema-overlay
warehouses.schema}
:model-exports #{:model/Measure}
:model-imports #{:model/Collection :model/Table :model/User}}
@@ -1469,17 +1492,16 @@
task
timeline
transforms
transforms.base
util
util.config
warehouse-schema-overlay
warehouses
warehouses.schema}
warehouses}
:model-exports #{:model/AiUsageLog
:model/Metabot
:model/MetabotConversation
:model/MetabotFeedback
:model/MetabotMessage}
:model/MetabotMessage
:model/MetabotPrompt}
:model-imports #{:model/Card
:model/Collection
:model/Dashboard
@@ -1672,20 +1694,53 @@
lib.legacy-mbql
lib.metadata
lib.schema
parameters
util
util.classloader
warehouse-schema-overlay}
:model-imports #{:model/Card
:model-imports #{:model/Action
:model/Card
:model/Channel
:model/ChannelTemplate
:model/Collection
:model/CustomVizPlugin
:model/Dashboard
:model/DashboardCard
:model/DashboardCardSeries
:model/DashboardTab
:model/Database
:model/Dimension
:model/Document
:model/EmbeddingTheme
:model/Exploration
:model/Field
:model/FieldUserSettings
:model/FieldValues
:model/Glossary
:model/HTTPAction
:model/ImplicitAction
:model/Measure
:model/Metabot
:model/MetabotPrompt
:model/NativeQuerySnippet
:model/Notification
:model/NotificationCard
:model/NotificationHandler
:model/NotificationRecipient
:model/NotificationSubscription
:model/OsiAiContext
:model/PythonLibrary
:model/QueryAction
:model/Segment
:model/Table
:model/TableIndex
:model/Timeline
:model/TimelineEvent
:model/Transform
:model/TransformJob
:model/TransformJobTransformTag
:model/TransformTag
:model/TransformTransformTag
:model/User}}
mq
@@ -1701,8 +1756,7 @@
native-query-snippets
{:team "Graphy"
:api #{metabase.native-query-snippets.api
metabase.native-query-snippets.core}
:api #{metabase.native-query-snippets.api metabase.native-query-snippets.core metabase.native-query-snippets.schema}
:uses #{api
collections
events
@@ -1712,6 +1766,7 @@
permissions
premium-features
remote-sync
users
util}
:model-exports #{:model/NativeQuerySnippet}
:model-imports #{:model/Collection :model/User}}
@@ -1754,8 +1809,10 @@
util
util.config}
:model-exports #{:model/Notification
:model/NotificationCard
:model/NotificationHandler
:model/NotificationRecipient}
:model/NotificationRecipient
:model/NotificationSubscription}
:model-imports #{:model/Card
:model/Channel
:model/ChannelTemplate
@@ -1796,6 +1853,7 @@
metabase.parameters.shared}
:uses #{api
app-db
dashboards
lib
lib.be
lib.metadata
@@ -1835,6 +1893,8 @@
util
util.config
warehouse-schema-overlay
warehouses
warehouses.schema
enterprise/permissions}
:model-exports #{:model/ApplicationPermissionsRevision
:model/CollectionPermissionGraphRevision
@@ -2266,6 +2326,7 @@
lib.schema
permissions
premium-features
server
session
settings
users
@@ -2282,13 +2343,17 @@
collections
dashboards
events
explorations
lib
lib.be
lib.schema
measures
models
parameters
queries
query-permissions
segments
transforms
util
util.config}
:model-exports #{:model/Revision}
@@ -2406,8 +2471,10 @@
permissions
remote-sync
search
users
util
warehouse-schema-overlay
warehouses.schema
xrays}
:model-exports #{:model/Segment}
:model-imports #{:model/Collection :model/Table :model/User}}
@@ -2455,6 +2522,7 @@
:api #{metabase.session.api
metabase.session.core
metabase.session.init
metabase.session.schema
metabase.session.settings} ; TODO -- shouldn't be exposed as an API namespace
:uses #{api
app-db
@@ -2606,7 +2674,8 @@
staleness
{:team "UX West"
:api #{metabase.staleness.core}}
:api #{metabase.staleness.core}
:uses #{util}}
startup
{:team "DevEx"
@@ -2767,7 +2836,7 @@
lib.schema
models
util}
:model-exports #{:model/Timeline}
:model-exports #{:model/Timeline :model/TimelineEvent}
:model-imports #{:model/Collection :model/User}}
tracing
@@ -2826,7 +2895,8 @@
:model/TransformJobRun
:model/TransformJobTransformTag
:model/TransformRun
:model/TransformTag}
:model/TransformTag
:model/TransformTransformTag}
:model-imports #{:model/Collection
:model/Database
:model/Field
@@ -2868,11 +2938,12 @@
transforms.indexes
{:team "Gadget"
:ns-prefix "metabase.indexes"
:api #{metabase.indexes.models.table-index metabase.indexes.reconcile}
:api #{metabase.indexes.models.table-index metabase.indexes.reconcile metabase.indexes.schema}
:uses #{driver
lib.schema
models
util}
util
warehouses}
:model-exports #{:model/TableIndex}}
transforms.indexes.rest
@@ -3637,10 +3708,7 @@
enterprise/dependencies
{:team "Graphy"
:module-exports #{enterprise/dependencies.replacement}
:api #{metabase-enterprise.dependencies.api
metabase-enterprise.dependencies.core
metabase-enterprise.dependencies.init
metabase-enterprise.dependencies.models.dependency}
:api #{metabase-enterprise.dependencies.api metabase-enterprise.dependencies.init metabase-enterprise.dependencies.models.dependency}
:uses #{api
app-db
collections
@@ -3651,14 +3719,18 @@
lib
lib.be
lib.metadata
lib.metric
lib.schema
measures
models
native-query-snippets
permissions
premium-features
queries
query-processor
request
revisions
segments
settings
sql-tools
task
@@ -3666,7 +3738,8 @@
transforms.base
util
util.graph
warehouse-schema-overlay}
warehouse-schema-overlay
enterprise/permissions.sandbox}
:model-imports #{:model/Card
:model/Collection
:model/Dashboard
@@ -3696,6 +3769,7 @@
measures
model-persistence
models
parameters
queries
segments
task
@@ -3791,7 +3865,6 @@
metabase-enterprise.metabot.api.routes
metabase-enterprise.metabot.init}
:uses #{api
lib.be
lib.schema
metabot
metabot.llm
@@ -3801,14 +3874,8 @@
settings
task
util
enterprise/cloud.add-ons
enterprise/dependencies
enterprise/transforms.python}
:model-imports #{:model/AiUsageLog
:model/Card
:model/PermissionsGroup
:model/PermissionsGroupMembership
:model/Transform}}
enterprise/cloud.add-ons}
:model-imports #{:model/AiUsageLog :model/PermissionsGroup :model/PermissionsGroupMembership}}
enterprise/metabot.agent-api
{:team "Metabot"
@@ -3963,7 +4030,7 @@
enterprise/permissions.sandbox
{:team "UX West"
:ns-prefix "metabase-enterprise.sandbox"
:api #{metabase-enterprise.sandbox.api.routes metabase-enterprise.sandbox.api.util}
:api #{metabase-enterprise.sandbox.api.routes metabase-enterprise.sandbox.api.util metabase-enterprise.sandbox.schema}
:uses #{api
app-db
audit-app
@@ -4037,6 +4104,7 @@
:model/Document
:model/Field
:model/FieldUserSettings
:model/Glossary
:model/Measure
:model/NativeQuerySnippet
:model/PythonLibrary
@@ -4063,12 +4131,6 @@
:api #{metabase-enterprise.embeddings.client}
:uses #{enterprise/search.semantic}}
enterprise/search.embeddings.embedder
{:team "Metabot"
:ns-prefix "metabase-enterprise.embedder"
:api #{}
:uses #{search.embeddings util util.classloader}}
enterprise/search.entity-retrieval
{:team "Metabot"
:ns-prefix "metabase-enterprise.entity-retrieval"
@@ -4372,7 +4434,12 @@
util
warehouse-schema-overlay
warehouses.schema}
:model-imports #{:model/Database :model/Field :model/Table}}}
:model-imports #{:model/Database :model/Field :model/Table}}
module/embedder
{:team "Metabot"
:api #{}
:uses #{search.embeddings util util.classloader}}}
;; namespaces matching these patterns (with `re-find`) are excluded from the module linter. Since regex literals
;; aren't allowed in EDN just used the `(str <regex>)` version i.e. two slashes instead of one.

View File

@@ -14,9 +14,9 @@
:consistent-alias 2
:def-fn 1
:deprecated-namespace 95
:deprecated-var 96
:deprecated-var 95
:discouraged-java-method 4
:discouraged-namespace 71
:discouraged-namespace 69
:discouraged-var 130
:duplicate-require 1
:equals-true 1
@@ -26,7 +26,7 @@
:metabase/i-like-making-cams-eyes-bleed-with-horrifically-long-tests 13
:metabase/modules 24
:metabase/namespace-name 4
:metabase/prefer-with-dynamic-fn-redefs 28
:metabase/prefer-with-dynamic-fn-redefs :unlimited
:metabase/test-helpers-use-non-thread-safe-functions 18
:metabase/validate-defendpoint-has-response-schema 427
:metabase/validate-defendpoint-query-params-use-kebab-case 48
@@ -59,9 +59,10 @@
:main-without-gen-class 1
:metabase/defsetting-must-specify-export 1
:metabase/defsetting-namespace 1
:metabase/prefer-with-dynamic-fn-redefs 3
:metabase/prefer-with-dynamic-fn-redefs 4
:metabase/t2-query-namespace 2
:metabase/table-or-field-query 1
:metabase/unsafe-app-db-query 1
:missing-docstring 1
:private-call 1
:redundant-ignore 1

View File

@@ -110,11 +110,11 @@
(when-not (contains? (hooks.common/ignored-linters ns-symb-node) :metabase/namespace-name)
(let [parts (str/split (str (hooks/sexpr ns-symb-node)) #"\.")]
(when (and (<= (count parts) 2)
(#{"metabase" "metabase-enterprise"} (first parts))
(#{"metabase" "metabase-enterprise" "metabase-module"} (first parts))
;; exclude test namespaces for now
(not (str/ends-with? (last parts) "test")))
(hooks/reg-finding! (assoc (meta ns-symb-node)
:message "Metabase namespaces should have the form metabase[-enterprise].<module>.* [:metabase/namespace-name]"
:message "Metabase namespaces should have the form metabase[-enterprise|-module].<module>.* [:metabase/namespace-name]"
:type :metabase/namespace-name)))))))
(defn- ns-form-node->import-node [ns-form-node]

View File

@@ -21,12 +21,30 @@
;;;; Dots express nesting: `lib.schema` is a child of `lib`. A declared
;;;; `enterprise/X` module is also a child of OSS module `X`. A namespace
;;;; belongs to the module with the most specific matching `:ns-prefix`.
;;;;
;;;; A `module/X` module is a plugin built from `modules/X`, shipped as its own jar and possibly absent at
;;;; runtime, so only its own subtree may use it.
;; TODO (Chris 2026-09-17) -- Driver plugins are not modules yet. The plan is `module/driver.X`, implying
;; `metabase.driver.X` in `modules/drivers/X`, so drivers and the third-party plugins built like them keep
;; their namespaces. With each driver a module, "should this change run driver tests" can come from the module
;; graph instead of the override list in `mage.modules`. The drivers' `metabase.test.data.X` helpers will need
;; resolving by directory, or a rename.
(defn plugin-module?
"Whether `module` is a plugin, built from `modules/<name>` and shipped as its own jar."
[module]
(= "module" (namespace module)))
(defn default-ns-prefix
"Namespace prefix a module owns unless it sets `:ns-prefix`: `metabase.lib.schema` for `lib.schema`,
`metabase-enterprise.foo` for `enterprise/foo`."
`metabase-enterprise.foo` for `enterprise/foo`, `metabase-module.embedder` for `module/embedder`."
[module]
(str (if (= "enterprise" (namespace module)) "metabase-enterprise." "metabase.") (name module)))
(str (case (namespace module)
"enterprise" "metabase-enterprise."
"module" "metabase-module."
"metabase.")
(name module)))
(defn module-ns-prefix
"Namespace prefix `module` owns: its `:ns-prefix`, else [[default-ns-prefix]]."
@@ -34,6 +52,21 @@
(or (get-in modules [module :ns-prefix])
(default-ns-prefix module)))
(defn module-directory
"Repo-relative directory holding `module`'s namespaces in `tree`, `\"src\"` or `\"test\"`, e.g.
`src/metabase/lib/schema`, `enterprise/backend/test/metabase_enterprise/sso` or
`modules/embedder/src/metabase_module/embedder`."
[modules module tree]
(let [ns-prefix (module-ns-prefix modules module)]
(str (cond
(plugin-module? module)
(str "modules/" (first (str/split (name module) #"\.")) "/")
(str/starts-with? ns-prefix "metabase-enterprise.")
"enterprise/backend/")
tree "/"
(-> ns-prefix (str/replace "." "/") (str/replace "-" "_")))))
(defn build-prefix->module
"Map each declared namespace prefix to its module, for [[declared-module]]."
[modules]
@@ -56,6 +89,7 @@
(resolve-module prefix->module 'metabase.qp.middleware.wow) => 'qp
(resolve-module prefix->module 'metabase-enterprise.whatever.core) => enterprise/whatever
(resolve-module prefix->module 'metabase-module.whatever.core) => module/whatever
An unmatched Metabase namespace resolves to its first segment so the linter can report an undeclared module."
[prefix->module ns-symb]
@@ -69,6 +103,9 @@
(some->> (re-find #"^metabase-enterprise\.([^.]+)" (str ns-symb))
second
(symbol "enterprise"))
(some->> (re-find #"^metabase-module\.([^.]+)" (str ns-symb))
second
(symbol "module"))
(some-> (re-find #"^metabase\.([^.]+)" (str ns-symb))
second
symbol))))
@@ -119,17 +156,37 @@
(recur ancestor)
{:ancestor ancestor, :child child}))))
(defn plugin-root
"The top-level plugin `module` belongs to, e.g. `module/embedder` for `module/embedder.model`.
Returns `nil` when `module` is no plugin."
[modules module]
(when (plugin-module? module)
(last (take-while some? (iterate #(parent-module modules %) module)))))
(defn- plugin-error
"Explain why `caller` may not use plugin `target`, or return `nil` when `target` is no plugin or `caller`
sits inside the plugin."
[modules caller target]
(when-let [plugin (plugin-root modules target)]
(when-not (descendant-of? modules caller plugin)
(format (str "Module %s is a plugin that may not be installed; %s may not use it. Load it through "
"metabase.plugins instead. [:metabase/modules %s :uses]")
target caller caller))))
(defn namability-error
"Explain why `caller` may not refer to `target` in `:uses`, or return `nil` if it may.
A nested module is private to the nearest ancestor with a missing export.
Exporting every link makes it public."
Exporting every link makes it public.
A plugin is private to itself, since it may be missing at runtime."
[modules caller target]
(when-let [{:keys [ancestor child]} (blocking-export modules target)]
(when-not (descendant-of? modules caller ancestor)
(format (str "Module %s is nested and not exported by its ancestors; %s may not use it. Add %s to %s's "
":module-exports, or move the caller into the %s subtree. [:metabase/modules %s :module-exports]")
target caller child ancestor ancestor ancestor))))
(or (plugin-error modules caller target)
(when-let [{:keys [ancestor child]} (blocking-export modules target)]
(when-not (descendant-of? modules caller ancestor)
(format (str "Module %s is nested and not exported by its ancestors; %s may not use it. Add %s to "
"%s's :module-exports, or move the caller into the %s subtree. "
"[:metabase/modules %s :module-exports]")
target caller child ancestor ancestor ancestor)))))
;;;; Lint rules. These functions take the full linter config.

View File

@@ -0,0 +1,375 @@
(ns hooks.metabase.agent-message
"Hooks linting agent-facing prose built with `metabase.mcp.v2.message/msg` (`:metabase/agent-message-lines`) and the
exits and helpers that carry it to the agent (`:metabase/agent-message-exit`). Each hook checks only when its
linter's level is configured and not `:off`, and returns its input unchanged."
(:require
[clj-kondo.hooks-api :as hooks]
[clojure.string :as str]))
(defn- level-on?
[config linter-key]
(not= :off (get-in config [:linters linter-key :level] :off)))
(defn- call-name
"The symbol at the head of list `node`, or nil."
[node]
(when (hooks/list-node? node)
(let [head (first (:children node))]
(when (hooks/token-node? head)
(let [v (hooks/sexpr head)]
(when (symbol? v) v))))))
(defn- calls?
"Whether `node` is a call to a var named `var-name` in one of `ns-names` (nil for an unqualified call)."
[node ns-names var-name]
(when-let [sym (call-name node)]
(and (= var-name (name sym))
(contains? ns-names (namespace sym)))))
(def ^:private core-ns #{nil "clojure.core"})
(def ^:private string-ns #{"str" "string" "clojure.string"})
(defn- literal-string
"The string value of `node` when it is a string literal or a `str` call over string literals, else nil."
[node]
(cond
(hooks/string-node? node)
(hooks/sexpr node)
(and (calls? node core-ns "str")
(every? hooks/string-node? (rest (:children node))))
(apply str (map hooks/sexpr (rest (:children node))))))
(defn- digits->long
"The value of a string of decimal digits."
[s]
;; Kondo's hook interpreter has no `parse-long`.
(reduce (fn [n c] (+ (* 10 n) (- (int c) (int \0)))) 0 s))
(def ^:private format-specifier
"A `java.util.Formatter` specifier; groups are explicit index, flags, width, precision, and conversion. The index is
capped at nine digits, so a longer one reads as malformed rather than a number too large to work with."
#"%(?:(\d{1,9})\$)?([-#+ 0,(<]*)(\d+)?(?:\.(\d+))?([tT]?[a-zA-Z%])")
(defn- consumed-args
"`[index conversion]` for each argument-consuming specifier in `fmt`, with zero-based argument indexes. An index of
-1 marks a reference no argument can satisfy: an explicit `0$`, or a `<` with no specifier before it to repeat."
[fmt]
(loop [[[_ explicit flags _width _precision conversion] & more] (re-seq format-specifier fmt)
next-index 0
last-index nil
acc []]
(if-not conversion
acc
(let [consumes? (not (contains? #{"%" "n"} conversion))
index (cond
(not consumes?) nil
explicit (dec (digits->long explicit))
(str/includes? flags "<") (or last-index -1)
:else next-index)
implicit? (and consumes? (not explicit) (not (str/includes? flags "<")))]
(recur more
(cond-> next-index implicit? inc)
(or index last-index)
(cond-> acc
index (conj [index conversion])))))))
(def ^:private msg-linter :metabase/agent-message-lines)
(defn- reg-msg-finding!
[node message]
(hooks/reg-finding! (assoc (meta node) :message message :type msg-linter)))
(defn- line-break?
"Whether format string `line` contains a line-breaking, control, or private-use character, or a `%n` specifier."
[line]
;; Private-use characters delimit the argument markers `render` formats with, so a line holding one could fake one.
(boolean (or (re-find #"[\p{Cc}\p{Cf}\p{Co}\p{Zl}\p{Zp}]" line)
(some #(= "n" (nth % 5)) (re-seq format-specifier line)))))
(defn- allowed-specifier?
"Whether specifier match `specifier` renders its value as `render` passes it: a bare `%%`, a `%s` with no flags but
`<`, or a `%d` with any flags and width but no precision. An explicit argument index is allowed on each."
[[_ explicit flags width precision conversion]]
(case conversion
"%" (not (or explicit (seq flags) width precision))
"s" (not (or width precision (seq (str/replace flags "<" ""))))
"d" (not precision)
false))
(defn- disallowed-specifier
"The first specifier in format string `line` that isn't [[allowed-specifier?]], as it is written, or nil."
[line]
(some #(when-not (allowed-specifier? %) (first %)) (re-seq format-specifier line)))
(def ^:private valid-conversion
"The conversions `java.util.Formatter` accepts, with each date/time conversion's required suffix."
#"[bBhHsScCdoxXeEfgGaA%n]|[tT][HIklMSLNpzZsQBbhAaCYyjmdeRTrDFc]")
(defn- malformed-specifier?
"Whether format string `line` has a `%` that doesn't begin a valid `java.util.Formatter` specifier."
[line]
(let [valid-removed (str/replace line format-specifier
(fn [[specifier & groups]]
(if (re-matches valid-conversion (last groups)) "" specifier)))]
(str/includes? valid-removed "%")))
(defn- line-problem
"The finding message for the first problem with a `msg` line, given its literal string or nil; nil when it has none."
[line]
(cond
(nil? line)
(str "Each line of a `msg` must be a string literal or a `str` of string literals; "
"pass values as arguments after the vector.")
(line-break? line)
(str "A `msg` line can't contain a line break, `%n`, or a control or private-use character; "
"put each line in its own string.")
(malformed-specifier? line)
"A `%` in a `msg` line must begin a format specifier; write a literal percent sign as `%%`."
:else
(when-let [specifier (disallowed-specifier line)]
(str "A `msg` line may only use `%s`, `%d`, or `%%`; `" specifier "` would alter or hide the value."))))
(defn- lint-lines!
"Flag each of `line-nodes` whose literal string, the matching element of `lines`, has a problem."
[line-nodes lines]
(doseq [[line-node line] (map vector line-nodes lines)]
(when-let [problem (line-problem line)]
(reg-msg-finding! line-node problem))))
(defn- plural
[n word]
(str n " " word (when-not (= 1 n) "s")))
(defn- lint-arg-count!
"Flag `msg` call `node` when a literal line refers to an argument that can't exist, or when its `args` don't number
the arguments its literal `lines` consume."
[node lines args]
(let [indexes (map first (consumed-args (str/join "\n" lines)))
expected (if (seq indexes) (inc (apply max indexes)) 0)
given (count args)]
(cond
;; `String/format` throws on these, so the message would render with every part quoted instead.
(some neg? indexes)
(reg-msg-finding! node (str "A `msg` line refers to an argument that can't exist; "
"argument indexes start at `1$`, and `%<` repeats the specifier before it."))
(not= expected given)
(reg-msg-finding! node (format "The `msg` lines take %s but %d %s given."
(plural expected "argument") given (if (= 1 given) "is" "are"))))))
(defn lint-msg
"Flag `msg` calls whose lines aren't a literal vector of single-line literal strings, that hold a malformed
specifier or a conversion other than `%s`, `%d`, or `%%`, or whose arguments don't match the lines' specifiers."
[{:keys [node config] :as input}]
(when (level-on? config msg-linter)
(let [[fn-node lines-node & args] (:children node)]
(if-not (and lines-node (hooks/vector-node? lines-node))
(reg-msg-finding! (or lines-node fn-node)
"`msg` takes a literal vector of line strings, one string per line of the message.")
(let [line-nodes (:children lines-node)
lines (map literal-string line-nodes)]
(lint-lines! line-nodes lines)
;; A malformed specifier can't be counted, and its line already has a finding.
(when (and (every? some? lines) (not-any? malformed-specifier? lines))
(lint-arg-count! node lines args))))))
input)
(def ^:private exit-linter :metabase/agent-message-exit)
(defn- exit-enabled?
[{:keys [config]}]
(level-on? config exit-linter))
(defn- reg-exit-finding!
[node message]
(hooks/reg-finding! (assoc (meta node) :message message :type exit-linter)))
(defn- calls-core?
"Whether `node` is a call to a `clojure.core` var or special form named in `var-names`."
[node var-names]
(when-let [sym (call-name node)]
(and (contains? var-names (name sym))
(contains? core-ns (namespace sym)))))
(defn- calls-i18n?
[node]
(when-let [sym (call-name node)]
(contains? #{"tru" "trs" "deferred-tru" "deferred-trs"} (name sym))))
(defn- calls-json-encode?
"Whether `node` calls `encode` in a namespace or alias whose name ends in `json`."
[node]
(when-let [sym (call-name node)]
(and (= "encode" (name sym))
(boolean (some-> (namespace sym) (str/ends-with? "json"))))))
(defn- builds-string?
"Whether `node` is a string literal or a call that builds a string."
[node]
(or (hooks/string-node? node)
(calls-core? node #{"str" "format" "pr-str" "ex-message"})
(calls-i18n? node)
(calls? node string-ns "join")
(calls-json-encode? node)))
(defn- clause-results
"The result nodes of `clauses`, alternating test and result nodes, with a trailing default when their count is odd."
[clauses]
(concat (take-nth 2 (rest clauses))
(when (odd? (count clauses)) [(last clauses)])))
(defn- result-fn-marker?
[node]
(and (hooks/keyword-node? node) (= :>> (hooks/sexpr node))))
(defn- condp-results
"The result nodes of `condp` `clauses`: each test's result, skipping `:>>` result functions, and any default."
[clauses]
(loop [[_test result & more :as clauses] clauses
acc []]
(cond
(empty? clauses) acc
(nil? (next clauses)) (conj acc (first clauses))
(result-fn-marker? result) (recur (next more) acc)
:else (recur more (conj acc result)))))
(defn- branch-results
"The nodes branching form `node` can evaluate to, or nil when it isn't one."
[node]
(let [args (rest (:children node))]
(cond
(calls-core? node #{"if" "if-not" "if-let"}) (rest args)
(calls-core? node #{"or"}) args
(calls-core? node #{"cond"}) (clause-results args)
(calls-core? node #{"case"}) (clause-results (rest args))
(calls-core? node #{"condp"}) (condp-results (drop 2 args)))))
(defn- step-call
"Threading step `node` as the call it stands for: a list as is, anything else as a call with no written arguments."
[node]
(if (hooks/list-node? node) node (hooks/list-node [node])))
(defn- threading-results
"The nodes whose text-ness decides what threading form `node` evaluates to, or nil when it isn't one."
[node]
(let [[init & steps] (rest (:children node))]
(cond
(calls-core? node #{"->" "->>" "some->" "some->>"}) [(if steps (step-call (last steps)) init)]
(calls-core? node #{"cond->" "cond->>"}) (cons init (map step-call (take-nth 2 (rest steps))))
(calls-core? node #{"as->"}) [(or (last (rest steps)) init)]
(calls-core? node #{"doto"}) [init])))
(defn- tail-result
"The node body form `node` evaluates to, or nil when it isn't one."
[node]
(let [args (rest (:children node))]
(cond
(calls-core? node #{"when" "when-not" "when-let" "let"}) (when (next args) (last args))
(calls-core? node #{"do"}) (last args))))
(defn- stringy?
"Whether `node` syntactically evaluates to text: a string literal, a call that builds a string, or a threading,
branching, or body form whose result is one."
[node]
(boolean
(or (builds-string? node)
(some stringy? (branch-results node))
(some stringy? (threading-results node))
(some-> (tail-result node) stringy?))))
(defn- lint-exit-text!
"Flag `arg`, the message passed to `target` (a description of where it goes), when it is text rather than a `msg`."
[arg target]
(when (and arg (stringy? arg))
(reg-exit-finding! arg (format (str "`%s` passes text to %s; "
"build agent-facing text with `msg` so interpolated values are cleaned.")
(pr-str (hooks/sexpr arg))
target))))
(defn- lint-exit-arg!
"Flag the argument at child `index` of call `node` when it is text rather than a `msg`."
[node index]
(lint-exit-text! (nth (:children node) index nil) (str "`" (name (call-name node)) "`")))
(defn lint-teaching-exit
"Flag a call throwing or returning a teaching error whose message, the first argument, is text rather than a `msg`."
[{:keys [node] :as input}]
(when (exit-enabled? input)
(lint-exit-arg! node 1))
input)
(defn lint-jsonrpc-error
"Flag a `jsonrpc-error` whose message, the third argument, is text rather than a `msg`."
[{:keys [node] :as input}]
(when (exit-enabled? input)
(lint-exit-arg! node 3))
input)
(defn lint-op-error
"Flag an `op-error!` whose detail message, the third argument, is text rather than a `msg`."
[{:keys [node] :as input}]
(when (exit-enabled? input)
(lint-exit-arg! node 3))
input)
(defn lint-ellipsize
"Flag an `ellipsize` whose message, the first argument, is text rather than a `msg`."
[{:keys [node] :as input}]
(when (exit-enabled? input)
(lint-exit-arg! node 1))
input)
(defn- map-value
"The value node of keyword `k` in map literal `node`, or nil."
[node k]
(when (hooks/map-node? node)
(some (fn [[key-node value-node]]
(when (and (hooks/keyword-node? key-node) (= k (hooks/sexpr key-node)))
value-node))
(partition 2 (:children node)))))
(defn lint-list-content
"Flag a `list-content`, or a wrapper taking its options map as the third argument too, whose literal options map
has an `:empty-hint` that is text rather than a `msg`."
[{:keys [node] :as input}]
(when (exit-enabled? input)
(lint-exit-text! (map-value (nth (:children node) 3 nil) :empty-hint)
(str "`" (name (call-name node)) "`'s `:empty-hint`")))
input)
(defn lint-success-content
"Flag a `success-content` whose first argument is text rather than a payload or a `msg`."
[{:keys [node] :as input}]
(when (exit-enabled? input)
(let [text (second (:children node))]
(when (and text (stringy? text))
(reg-exit-finding! text (str "`success-content` takes data to JSON-encode or a `msg`; "
"pass the payload itself, or build the text with `msg`.")))))
input)
(defn- caller-facing-data?
"Whether `node` is a map literal with a 4xx `:status-code` literal or an `error-code` key."
[node]
(and (hooks/map-node? node)
(some (fn [[k v]]
(when (hooks/keyword-node? k)
(let [kw (hooks/sexpr k)]
(or (= "error-code" (name kw))
(and (= :status-code kw)
(hooks/token-node? v)
(let [status (hooks/sexpr v)]
(and (number? status) (<= 400 status 499))))))))
(partition 2 (:children node)))))
(defn lint-ex-info
"Flag an `ex-info` whose data marks it as a caller-facing error."
[{:keys [node] :as input}]
(when (exit-enabled? input)
(when (caller-facing-data? (nth (:children node) 2 nil))
(reg-exit-finding! node (str "Caller-facing errors must be thrown with `throw-teaching-error` and a `msg`, "
"so their text is rendered and cleaned."))))
input)

View File

@@ -29,6 +29,100 @@
[filename]
(boolean (and filename (re-find #"(?:^|/)test/" filename))))
(def ^:private value-operators
"Operators whose second argument is a value rather than a column."
'#{= not= < > <= >= like not-like ilike not-ilike in not-in between})
(defn- marked?
"Whether `node` is an `[:auto/param v]` marker."
[node]
(and (hooks/vector-node? node)
(= :auto/param (some-> (first (:children node)) hooks/sexpr))))
(defn- value-nodes
"The nodes sitting in a value slot of `node`, following the clause shapes a query map uses."
[node]
(cond
;; A query map -- walk its clause values, not its keys.
(hooks/map-node? node)
(mapcat value-nodes (take-nth 2 (rest (:children node))))
(hooks/vector-node? node)
(let [[head & args] (:children node)
op (some-> head hooks/sexpr)]
(cond
(contains? #{:and :or :not} op)
(mapcat value-nodes args)
;; `[:= col v]`, `[:in col vs]`, `[:between col lo hi]` -- the first argument is the
;; column and the rest are values, whatever the arity.
(and (keyword? op) (contains? value-operators (symbol (name op))))
(rest args)
:else
(mapcat value-nodes (:children node))))
:else nil))
(def ^:private write-fns
"Calls whose values are written rather than filtered on. A value here is not a where-clause value
an attacker can turn into SQL structure, and a `define-before-insert` hook may read it before the
query compiles, so a marker would break it rather than protect anything."
'#{insert! insert-returning-instance! insert-returning-instances! insert-returning-pk!
insert-returning-pks! save!})
(defn- read-call?
"Whether `node` is a query call whose values are filtered on rather than written."
[node]
(let [f (some-> (first (:children node)) hooks/sexpr)]
(not (contains? write-fns (some-> f name symbol)))))
(defn- kv-arg-pair
"The first `:column value` pair a query call passes after its model, if any.
`(t2/select :model/X :locale locale)`. Several of these fns take an argument before the model --
`(t2/select-one-fn :value :model/X :key k)` -- so the pairs do not start at a fixed offset; they
start after the `:model/...` keyword. A call that does not name a literal model is not checked."
[args]
(let [after-model (->> args
(drop-while #(not (and (hooks/keyword-node? %)
(= "model" (namespace (hooks/sexpr %))))))
rest)
[k v] after-model]
(when (and k v (hooks/keyword-node? k))
k)))
(defn- lint-kv-args!
"Register a finding for a query written as `:column value` pairs.
Toucan builds those into the where clause itself, so the query's shape is not on the page: there
is nothing for a reader, this linter, or the `.sql` extraction that follows this project to read.
The same call takes a map -- `{:where [:= :locale v]}` for a select, `{:locale v}` for an
`update!` -- which is the statement, near enough to read off."
[node]
(when-let [k (kv-arg-pair (rest (:children node)))]
(hooks/reg-finding!
(assoc (meta k)
:message (format "Pass this query a map rather than `%s ...` pairs, so its shape is on the page."
(hooks/sexpr k))
:type :metabase/unsafe-app-db-query))))
(defn- lint-unmarked-values!
"Register a finding for each argument of the enclosing function that reaches a value slot unmarked.
Only a symbol is reported. A literal cannot carry a request value, and a value built inside the
function is out of reach of a check that does not follow it across a call."
[node]
(doseq [value (mapcat value-nodes (rest (:children node)))
:when (and (hooks/token-node? value)
(symbol? (hooks/sexpr value))
(not (marked? value)))]
(hooks/reg-finding!
(assoc (meta value)
:message (format "`%s` reaches a SQL value slot unmarked. Write it as [:auto/param %s] so it is bound as a parameter."
(hooks/sexpr value) (hooks/sexpr value))
:type :metabase/unsafe-app-db-query))))
(defn lint-query-call
"Register a `:metabase/t2-query-namespace` finding when a Toucan 2 query call appears outside a `<module>.db`
namespace."
@@ -43,4 +137,10 @@
(hooks/sexpr fn-node))
:type :metabase/t2-query-namespace))))
((requiring-resolve 'hooks.metabase.warehouse-schema-overlay.table-or-field-query/lint-read) input)
(when (and ns
(db-namespace? (modules/config input) ns)
(not (test-file? filename))
(read-call? node))
(lint-unmarked-values! node)
(lint-kv-args! node))
input)

View File

@@ -0,0 +1,334 @@
(ns hooks.metabase.agent-message-test
(:require
[clj-kondo.hooks-api :as hooks]
[clj-kondo.impl.utils]
[clojure.edn :as edn]
[clojure.test :refer :all]
[hooks.metabase.agent-message :as agent-message]))
(defn- findings
"Run `hook-fn` over `form`, a form or its source string, with the linters at `level` (unconfigured when nil) in
namespace `ns` of file `filename`; return the findings' messages."
([hook-fn form]
(findings hook-fn form :warning))
([hook-fn form level]
(findings hook-fn form level 'metabase.mcp.v2.tools.browse "src/metabase/mcp/v2/tools/browse.clj"))
([hook-fn form level ns filename]
(let [config (if level
{:linters {:metabase/agent-message-lines {:level level}
:metabase/agent-message-exit {:level level}}}
{})]
(binding [clj-kondo.impl.utils/*ctx* {:config config
:ignores (atom nil)
:findings (atom [])
:namespaces (atom {})}]
(let [input {:node (hooks/parse-string (if (string? form) form (pr-str form)))
:ns ns
:filename filename
:config config}
output (hook-fn input)]
(is (identical? (:node input) (:node output))
"the hook must return the node unchanged so Kondo's normal analysis still runs")
(mapv :message @(:findings clj-kondo.impl.utils/*ctx*)))))))
(def ^:private hook-cases
"`[hook form]` pairs that each report one finding when their linter is enabled."
[[agent-message/lint-msg '(msg lines x)]
[agent-message/lint-teaching-exit '(throw-teaching-error "x")]
[agent-message/lint-jsonrpc-error '(jsonrpc-error id -32602 "x")]
[agent-message/lint-success-content '(success-content "x")]
[agent-message/lint-ex-info "(ex-info \"x\" {:status-code 404})"]
[agent-message/lint-op-error '(op-error! idx "add_link" "x")]
[agent-message/lint-ellipsize '(ellipsize "x" 300)]
[agent-message/lint-list-content '(list-content data total {:empty-hint "x"})]])
(defn- linter-enabled-in-ns?
"Whether `config` enables `linter` in namespace `ns-name` through an ns-group's `:config-in-ns` entry."
[config linter ns-name]
(boolean
(some (fn [{:keys [pattern name]}]
(and (string? pattern)
(re-find (re-pattern pattern) ns-name)
(not= :off (get-in config [:config-in-ns name :linters linter :level] :off))))
(:ns-groups config))))
(deftest ^:parallel config-scope-test
(let [config (edn/read-string (slurp ".clj-kondo/config.edn"))]
(testing "GHY-4544: `msg` lines are checked in every OSS and enterprise MCP source namespace, and no test one"
(doseq [[ns-name enabled?] {"metabase.mcp.ui-resource" true
"metabase.mcp.transport" true
"metabase.mcp.v2.tools.browse" true
"metabase-enterprise.mcp.usage" true
"metabase-enterprise.mcp.v2.tools.browse" true
"metabase.mcp.ui-resource-test" false
"metabase.mcp.v2.tools.browse-test" false
"metabase.mcp.test-util" false
"metabase-enterprise.mcp.usage-test" false
"metabase-enterprise.mcp.v2.tools.browse-test" false
"metabase.queries.models.card" false}]
(testing ns-name
(is (= enabled? (linter-enabled-in-ns? config :metabase/agent-message-lines ns-name))))))
(testing "GHY-4544: exits are checked in MCP transport and v2 source namespaces, whose 4xx errors reach the agent"
(doseq [[ns-name enabled?] {"metabase.mcp.transport" true
"metabase.mcp.v2.common" true
"metabase.mcp.v2.tools.browse" true
"metabase-enterprise.mcp.v2.common" true
"metabase.mcp.settings" false
"metabase.mcp.validation" false
"metabase-enterprise.mcp.usage" false
"metabase.mcp.transport-test" false
"metabase.mcp.v2.tools.browse-test" false
"metabase-enterprise.mcp.v2.tools.browse-test" false
"metabase.queries.models.card" false}]
(testing ns-name
(is (= enabled? (linter-enabled-in-ns? config :metabase/agent-message-exit ns-name))))))
(testing "GHY-4544: `:empty-hint` is checked on `list-content` and on wrappers passing their options map to it"
(doseq [fn-sym '[metabase.mcp.v2.common/list-content
metabase.mcp.v2.tools.browse/paged-list-content]]
(testing fn-sym
(is (= 'hooks.metabase.agent-message/lint-list-content
(get-in config [:hooks :analyze-call fn-sym]))))))
(testing (str "GHY-4544: `check-execute-sql-enabled!` isn't checked, since it interpolates its subject into "
"its own `msg` and a plain string is the intended way to name the tool")
(is (nil? (get-in config [:hooks
:analyze-call
(symbol "metabase.mcp.v2.queries" "check-execute-sql-enabled!")]))))))
(deftest ^:parallel scope-test
(testing "GHY-4544: the hooks leave scoping to config, reporting in any namespace where their linter is enabled"
(doseq [[hook form] hook-cases
[ns filename] [['metabase.mcp.v2.tools.browse "src/metabase/mcp/v2/tools/browse.clj"]
['metabase.queries.models.card "src/metabase/queries/models/card.clj"]
['metabase.mcp.v2.tools.browse-test "test/metabase/mcp/v2/tools/browse_test.clj"]]]
(testing (str ns " " (pr-str form))
(is (= 1 (count (findings hook form :warning ns filename)))))))
(testing "nothing is reported when the linter is off or unconfigured"
(doseq [[hook form] hook-cases
level [:off nil]]
(testing (str level " " (pr-str form))
(is (empty? (findings hook form level)))))))
(deftest ^:parallel msg-test
(testing "GHY-4544: a vector of single-line string literals with matching arguments is accepted"
(is (empty? (findings agent-message/lint-msg
'(msg ["Table %s: %d of %d fields." "Continue with `offset: %d`."] t n total next)))))
(testing "specifiers that consume no argument aren't counted, and an explicit index can repeat an argument"
(is (empty? (findings agent-message/lint-msg '(msg ["100%% done."]))))
(is (empty? (findings agent-message/lint-msg '(msg ["%1$s, then %1$s again, then %<s"] x)))))
(testing "the lines must be a literal vector"
(is (=? [#".*vector.*"]
(findings agent-message/lint-msg '(msg lines x))))
(is (=? [#".*vector.*"]
(findings agent-message/lint-msg '(msg "one line" x)))))
(testing "a line may be split across a `str` of string literals"
(is (empty? (findings agent-message/lint-msg '(msg [(str "Table %s has " "%d fields.")] t n))))
(is (=? [#".*1 argument.*2.*"]
(findings agent-message/lint-msg '(msg [(str "only " "%s")] a b)))))
(testing "each line must be a string literal or a `str` of string literals"
(is (=? [#".*string literal or a `str` of string literals.*"]
(findings agent-message/lint-msg '(msg ["ok" line]))))
(is (=? [#".*string literal or a `str` of string literals.*"]
(findings agent-message/lint-msg '(msg [(str "split " x)])))))
(testing "a line can't contain a line break, control, or private-use character; each line is its own string"
(is (=? [#".*own string.*"]
(findings agent-message/lint-msg '(msg ["first\nsecond"]))))
(is (=? [#".*own string.*"]
(findings agent-message/lint-msg '(msg [(str "first\n" "second")]))))
(is (=? [#".*own string.*"]
(findings agent-message/lint-msg (list 'msg [(str "first" (char 0x2028) "second")]))))
(is (=? [#".*own string.*"]
(findings agent-message/lint-msg '(msg ["tab\tseparated"]))))
(is (=? [#".*own string.*"]
(findings agent-message/lint-msg '(msg ["done.%n"]))))
(testing "GHY-4544: a private-use character, which the truncation markers use, is flagged too"
(is (=? [#".*own string.*"]
(findings agent-message/lint-msg (list 'msg [(str "first" (char 0xE000) "second")]))))
(is (=? [#".*own string.*"]
(findings agent-message/lint-msg (list 'msg [(str "marked" (char 0xF8FF))]))))))
(testing "the argument count must match the specifiers across all lines"
(is (=? [#".*2 arguments.*1.*"]
(findings agent-message/lint-msg '(msg ["%s and" "%s"] a))))
(is (=? [#".*1 argument.*2.*"]
(findings agent-message/lint-msg '(msg ["only %s"] a b)))))
(testing (str "GHY-4544: a reference `String/format` can't resolve is flagged, since the runtime would fall back "
"to rendering every part quoted")
(testing "`%<` with no specifier before it to repeat, which counts as no argument at all"
(is (=? [#".*can't exist.*"]
(findings agent-message/lint-msg '(msg ["Value: %<s"]))))
(is (=? [#".*can't exist.*"]
(findings agent-message/lint-msg '(msg ["100%% done," "then %<d"])))))
(testing "an explicit index of zero, which is below the first argument"
(is (=? [#".*can't exist.*"]
(findings agent-message/lint-msg '(msg ["Value: %0$s"]))))
(is (=? [#".*can't exist.*"]
(findings agent-message/lint-msg '(msg ["Value: %0$s"] x)))))
(testing "a `%<` that repeats the first argument is still accepted"
(is (empty? (findings agent-message/lint-msg '(msg ["%s then %<s"] x)))))))
(deftest ^:parallel msg-specifier-test
(testing "GHY-4544: only `%s` with no flags but `<`, `%d` with flags and width, and `%%` are allowed"
(doseq [line ["%s" "%d" "%1$s" "%,d" "%05d" "%-8d" "%(d" "%1$,10d"
"%s again %<s" "%d again %<d" "%s (100%%)"]]
(testing line
(is (empty? (findings agent-message/lint-msg (list 'msg [(str "Value: " line)] 'x)))))))
(testing "GHY-4544: any other conversion, or a `%s` or `%d` that reshapes its value, is flagged"
(doseq [line ["%b" "%h" "%c" "%x" "%X" "%f" "%e" "%o" "%g" "%a" "%tY" "%TY" "%B"
"%S" "%.20s" "%10s" "%-5s" "%#s" "%1$.3s" "%-10S" "%.1f" "%.2d" "%1$.2d"]]
(testing line
(is (=? [#"A `msg` line may only use `%s`, `%d`, or `%%`; .* would alter or hide the value\."]
(findings agent-message/lint-msg (list 'msg [(str "Value: " line)] 'x)))))))
(testing "GHY-4544: a `%` that doesn't begin a valid format specifier is flagged, since `String/format` rejects it"
(doseq [line ["Braces %}" "Dot %." "Trailing %" "Space % " "Unknown %q" "Bad date %tq" "%" "%1$"
;; GHY-4544: an argument index too large to be one is malformed, not an arithmetic overflow
"%999999999999999999999999$s"]]
(testing (pr-str line)
(is (=? [#"A `%` in a `msg` line must begin a format specifier.*`%%`.*"]
(findings agent-message/lint-msg (list 'msg [line]))))))
(testing "an index at the digit cap still parses, and is reported as an argument count"
(is (=? [#".*999999999 arguments.*"]
(findings agent-message/lint-msg '(msg ["%999999999$s"] x))))))
(testing "valid specifiers, including an escaped percent sign, aren't flagged as malformed"
(doseq [form ['(msg ["100%% done"])
'(msg ["%,d rows"] n)
'(msg ["%1$s, then %<s"] x)
'(msg [(str "50%" "% of %s")] x)]]
(testing (pr-str form)
(is (empty? (findings agent-message/lint-msg form)))))))
(def ^:private stringy-texts
"Message arguments built as text rather than with `msg`."
['"Query failed."
'(str "Query failed: " err)
'(format "Table %s not found" (pr-str t))
'(cond-> (json/encode body) message (str "\n" message))
'(if x "a" (str "b" y))
'(if x (msg ["a"]) (str "b" y))
'(when-let [e (:error r)] (str/join ", " e))
'(or (:message r) (tru "Unknown error"))
'(let [s (pr-str x)] (str "Bad value " s))
'(cond (:a x) (msg ["a"]) :else "b")
'(condp = x :a (str "a" y) (msg ["b"]))
'(condp = x :a (msg ["a"]) "b")
'(case x :a (msg ["a"]) "b")
'(case x :a "a" :b (msg ["b"]))
'(->> bad (map name) (str/join ", "))
'(-> x (str " suffix"))
'(-> x name str)
'(some-> e ex-message)
'(some->> xs (map name) (clojure.string/join ", "))
'(cond->> xs x (map name) y (str/join ", "))
'(cond-> "x" y (subs 1))
'(as-> x v (assoc v :a 1) (json/encode v))
'(doto (str "a" b) println)
'(let [s x] (-> s (format y)))])
(def ^:private message-texts
"Message arguments that aren't syntactically text."
['(msg ["Table %s not found."] t)
'message
'(if x (msg ["a"]) (msg ["b"]))
'(render-error e)
'(cond (str x) (msg ["a"]) :else (msg ["b"]))
'(condp = x "a" (msg ["a"]) (msg ["b"]))
'(condp get x "a" :>> render-error (msg ["b"]))
'(case x "a" (msg ["a"]) ("b" "c") (msg ["b"]))
'(-> "x" count)
'(->> "x" (map str) set)
'(some-> (str "x") render-error)
'(-> x)
'(-> e :message)
'(cond-> {:a 1} x (assoc :b "c"))
'(as-> "x" v (count v))
'(doto x (println "a"))])
(deftest ^:parallel teaching-exit-test
(doseq [fn-name ["throw-teaching-error" "error-content"]]
(testing (str "GHY-4544: text passed to `" fn-name "` is flagged")
(doseq [text stringy-texts]
(testing (pr-str text)
(is (=? [(re-pattern (str ".*passes text to `" fn-name "`.*`msg`.*"))]
(findings agent-message/lint-teaching-exit (list (symbol "common" fn-name) text)))))))
(testing (str "`" fn-name "` accepts a `msg`, a symbol, and other calls")
(doseq [text message-texts]
(testing (pr-str text)
(is (empty? (findings agent-message/lint-teaching-exit
(list (symbol fn-name) text {:status-code 404}))))))))
(testing "only the message argument is checked"
(is (empty? (findings agent-message/lint-teaching-exit '(error-content (msg ["x"]) (str "code"))))))
(testing "nothing is reported when the linter is off"
(is (empty? (findings agent-message/lint-teaching-exit '(throw-teaching-error "x") :off)))))
(def ^:private helper-cases
"`[hook fn-name form-fn]`: a message-taking helper's hook, the name its finding cites, and a function of a message
returning a call that passes it to the helper."
[[agent-message/lint-op-error "op-error!" #(list 'dashboard-ops/op-error! 'idx "add_link" %)]
[agent-message/lint-ellipsize "ellipsize" #(list 'common/ellipsize % 300)]
[agent-message/lint-list-content "list-content" #(list 'common/list-content 'rows 0 {:offset 0 :empty-hint %})]
[agent-message/lint-list-content "paged-list-content"
#(list 'paged-list-content 'args 'rows {:empty-hint %} 'identity)]])
(deftest ^:parallel message-helper-test
(doseq [[hook fn-name form-fn] helper-cases]
(testing (str "GHY-4544: text passed as `" fn-name "`'s message is flagged")
(doseq [text stringy-texts]
(testing (pr-str text)
(is (=? [(re-pattern (str ".*passes text to `" fn-name "`.*`msg`.*"))]
(findings hook (form-fn text)))))))
(testing (str "`" fn-name "` accepts a `msg`, a symbol, and other calls as its message")
(doseq [text message-texts]
(testing (pr-str text)
(is (empty? (findings hook (form-fn text))))))))
(testing "only the message argument is checked"
(is (empty? (findings agent-message/lint-op-error '(op-error! (str "0") (str "op") (msg ["x"])))))
(is (empty? (findings agent-message/lint-ellipsize '(ellipsize (msg ["x"]) (str "300")))))
(is (empty? (findings agent-message/lint-list-content '(list-content (str "data") 0 {:param (str "x")}))))
(is (empty? (findings agent-message/lint-list-content '(list-content data 0 opts))))
(is (empty? (findings agent-message/lint-list-content '(list-content data 0))))))
(deftest ^:parallel jsonrpc-error-test
(testing "GHY-4544: text passed as the JSON-RPC error message is flagged"
(doseq [text stringy-texts]
(testing (pr-str text)
(is (=? [#".*passes text to `jsonrpc-error`.*`msg`.*"]
(findings agent-message/lint-jsonrpc-error (list 'transport/jsonrpc-error 'id -32602 text)))))))
(testing "a `msg`, symbol, or other call as the message is accepted"
(doseq [text message-texts]
(testing (pr-str text)
(is (empty? (findings agent-message/lint-jsonrpc-error (list 'jsonrpc-error 'id -32602 text)))))))
(testing "only the third argument is the message"
(is (empty? (findings agent-message/lint-jsonrpc-error '(jsonrpc-error "id" (str "code") (msg ["x"])))))))
(deftest ^:parallel success-content-test
(testing "GHY-4544: text passed to `success-content` is flagged"
(doseq [text stringy-texts]
(testing (pr-str text)
(is (=? [#"`success-content` takes data to JSON-encode or a `msg`.*"]
(findings agent-message/lint-success-content (list 'common/success-content text)))))))
(testing "a payload, a `msg`, or a symbol is accepted"
(doseq [text (conj message-texts '{:data rows} '[a b])]
(testing (pr-str text)
(is (empty? (findings agent-message/lint-success-content (list 'success-content text))))))
(is (empty? (findings agent-message/lint-success-content
'(success-content {:data rows} (str "structured")))))))
(deftest ^:parallel ex-info-test
(testing "GHY-4544: an ex-info carrying a 4xx status code or an error code is flagged"
(doseq [form ["(ex-info \"Card not found\" {:status-code 404})"
"(ex-info (str \"Bad \" x) {:status-code 400 :field f})"
"(ex-info \"Nope\" {::common/error-code c})"
"(ex-info \"Nope\" {:error-code c})"
"(ex-info (msg [\"Nope\"]) {:error-code c})"]]
(testing form
(is (=? [#".*`throw-teaching-error` and a `msg`.*"]
(findings agent-message/lint-ex-info form))))))
(testing "other ex-info calls aren't flagged"
(doseq [form ["(ex-info \"boom\" {:status-code 500})"
"(ex-info \"boom\" {:status-code code})"
"(ex-info \"boom\" {:foo 1})"
"(ex-info \"boom\" data)"
"(ex-info \"boom\" (merge {:status-code 400} data))"
"(ex-info \"boom\")"]]
(testing form
(is (empty? (findings agent-message/lint-ex-info form))))))
(testing "nothing is reported when the linter is off"
(is (empty? (findings agent-message/lint-ex-info "(ex-info \"x\" {:status-code 404})" :off)))))

View File

@@ -5,8 +5,15 @@
[clojure.test :refer :all]
[hooks.metabase.toucan.db-ns :as toucan.db-ns]))
(defn- confinement-findings
"Only the `t2-query-namespace` findings, so these tests are not affected by the query-hygiene
linter that shares this hook."
[findings]
(filter #(= :metabase/t2-query-namespace (:type %)) findings))
(defn- lint-query-call [form ns-sym & [filename modules]]
(binding [clj-kondo.impl.utils/*ctx* {:config {:linters {:metabase/t2-query-namespace {:level :warning}}}
(binding [clj-kondo.impl.utils/*ctx* {:config {:linters {:metabase/t2-query-namespace {:level :warning}
:metabase/unsafe-app-db-query {:level :warning}}}
:ignores (atom nil)
:findings (atom [])
:namespaces (atom {})}]
@@ -25,11 +32,11 @@
:message #".*`t2/select-one`.*"}]
(lint-query-call '(t2/select-one :model/Card :id 1) 'metabase.queries.models.card))))
(testing "metabase.<module>.db is allowed"
(is (empty? (lint-query-call '(t2/select-one :model/Card :id 1) 'metabase.queries.db))))
(is (empty? (confinement-findings (lint-query-call '(t2/select-one :model/Card :id 1) 'metabase.queries.db)))))
(testing "metabase-enterprise.<module>.db is allowed"
(is (empty? (lint-query-call '(t2/select-one :model/Card :id 1) 'metabase-enterprise.sandbox.db))))
(is (empty? (confinement-findings (lint-query-call '(t2/select-one :model/Card :id 1) 'metabase-enterprise.sandbox.db)))))
(testing "metabase.driver.<driver>.db is allowed for driver modules"
(is (empty? (lint-query-call '(t2/select-one :model/Database :id 1) 'metabase.driver.bigquery-cloud-sdk.db))))
(is (empty? (confinement-findings (lint-query-call '(t2/select-one :model/Database :id 1) 'metabase.driver.bigquery-cloud-sdk.db)))))
(testing "a nested db namespace is not a module db namespace"
(is (=? [{:type :metabase/t2-query-namespace}]
(lint-query-call '(t2/query {:select [:*]}) 'metabase.queries.models.db))))
@@ -71,3 +78,61 @@
(lint-query-call '(t2/query {:select [:*]}) 'metabase.metabot.llm.models.db nil modules))))
(testing "enterprise modules resolve through the metabase-enterprise root"
(is (empty? (lint-query-call '(t2/select :model/Card) 'metabase-enterprise.sandbox.db nil modules))))))
(deftest ^:parallel unmarked-value-in-a-db-namespace-test
(testing "a symbol reaching a value slot is flagged"
(is (=? [{:type :metabase/unsafe-app-db-query
:message #"`locale` reaches a SQL value slot unmarked.*"}]
(lint-query-call '(t2/select :model/X {:where [:= :locale locale]}) 'metabase.foo.db))))
(testing "a marked value is not flagged"
(is (empty? (lint-query-call '(t2/select :model/X {:where [:= :locale [:auto/param locale]]})
'metabase.foo.db))))
(testing "a literal cannot carry a request value and is not flagged"
(is (empty? (lint-query-call '(t2/select :model/X {:where [:= :locale "de"]}) 'metabase.foo.db))))
(testing "a column reference is not a value"
(is (empty? (lint-query-call '(t2/select :model/X {:where [:= :a.id :b.id]}) 'metabase.foo.db))))
(testing "values nested under a boolean connective are reached"
(is (=? [{:message #"`b`.*"}]
(lint-query-call '(t2/select :model/X {:where [:and [:= :x [:auto/param a]] [:= :y b]]})
'metabase.foo.db))))
(testing "only db namespaces are linted for unmarked values"
(is (empty? (filter #(= :metabase/unsafe-app-db-query (:type %))
(lint-query-call '(t2/select :model/X {:where [:= :locale locale]})
'metabase.foo.models.thing)))))
(testing "a test source tree is exempt"
(is (empty? (lint-query-call '(t2/select :model/X {:where [:= :locale locale]})
'metabase.foo.db "test/metabase/foo/db_test.clj")))))
(deftest ^:parallel kv-arg-style-test
(testing "a query written as :column value pairs is flagged whatever the values are"
(are [form] (=? [{:type :metabase/unsafe-app-db-query, :message #"Pass this query a map.*"}]
(lint-query-call form 'metabase.foo.db))
'(t2/select :model/X :locale locale)
'(t2/select :model/X :archived false)
'(t2/select :model/X :locale [:auto/param locale])
'(t2/select-one :model/X :id (long id))))
(testing "one finding per call, not one per pair"
(is (= 1 (count (lint-query-call '(t2/select :model/X :locale locale :msgid msgid)
'metabase.foo.db)))))
(testing "a fn with an argument before the model is still recognised"
(is (=? [{:message #"Pass this query a map.*"}]
(lint-query-call '(t2/select-one-fn :value :model/Setting :key k) 'metabase.foo.db))))
(testing "a map query is not flagged"
(are [form] (empty? (lint-query-call form 'metabase.foo.db))
'(t2/select :model/X {:where [:= :locale [:auto/param locale]]})
'(t2/update! :model/X {:locale [:auto/param locale]} {:msgstr "x"}))))
(deftest ^:parallel write-calls-are-not-linted-for-values-test
(testing "an insert's values are written, not filtered on, so they are not flagged"
(are [form] (empty? (filter #(= :metabase/unsafe-app-db-query (:type %))
(lint-query-call form 'metabase.foo.db)))
'(t2/insert! :model/X :key k :value v)
'(t2/insert-returning-instances! :model/X :key k :value v)))
(testing "a select's values are still flagged"
(is (=? [{:type :metabase/unsafe-app-db-query}]
(lint-query-call '(t2/select :model/X :key k) 'metabase.foo.db)))))
(deftest ^:parallel operator-arity-test
(testing "a value operator with an unexpected arity still has its args examined"
(is (seq (lint-query-call '(t2/select :model/X {:where [:= :a b c]}) 'metabase.foo.db)))
(is (seq (lint-query-call '(t2/select :model/X {:where [:between :a lo hi]}) 'metabase.foo.db)))))

View File

@@ -2,7 +2,7 @@
// imported, vs the *allowed* edges in module-boundaries.mjs). Consumed by
// .github/scripts/create-test-plan.ts.
//
// bunx depcruise frontend/src enterprise/frontend/src \
// bunx depcruise frontend/src enterprise/frontend/src .storybook/preview.tsx \
// --config .dependency-cruiser.cjs --output-type json > dependency-graph.json
/** @type {import('dependency-cruiser').IConfiguration} */
@@ -10,10 +10,13 @@ module.exports = {
options: {
// Keep only edges between our own source files (drops node_modules etc.),
// which keeps the JSON small.
includeOnly: "^(frontend/src|enterprise/frontend/src)/",
includeOnly: "^(frontend/src|enterprise/frontend/src|\\.storybook)/",
doNotFollow: { path: "node_modules" },
// No rules here, so the cycle and orphan analysis has no consumer and only costs time.
skipAnalysisNotInRules: true,
// Resolve the `"*": ["./frontend/src/*", ...]` aliases so bare imports like
// `metabase/lib/foo` map back to a real file.
tsConfig: { fileName: "tsconfig.json" },

View File

@@ -1,5 +1,5 @@
name: Create test plan
description: Cruise the frontend import graph and compute the affected-tests plan.
description: Compute the affected-tests plan for the changed files.
inputs:
changed-files:
@@ -28,21 +28,15 @@ outputs:
stats:
description: FE Affected Tests stats row (JSON).
value: ${{ steps.compute.outputs.stats }}
fe_unit_specs_to_run:
description: Unit specs to run (JSON array).
value: ${{ steps.compute.outputs.fe_unit_specs_to_run }}
loki_stories_to_run:
description: Loki stories to run (JSON array).
value: ${{ steps.compute.outputs.loki_stories_to_run }}
e2e_specs_to_run:
description: E2E specs to run (JSON array).
value: ${{ steps.compute.outputs.e2e_specs_to_run }}
artifact-id:
description: Artifact containing test-plan.json for this run.
value: ${{ steps.upload.outputs.artifact-id }}
runs:
using: composite
steps:
- name: Prepare Node + Bun
uses: ./.github/actions/prepare-node-bun
- name: Prepare front-end environment
uses: ./.github/actions/prepare-frontend
- name: Resolve closest-ancestor e2e coverage manifest
id: manifest
uses: actions/github-script@v9
@@ -61,16 +55,6 @@ runs:
headSha: context.payload.pull_request?.head?.sha ?? context.sha,
});
core.setOutput("path", result?.path ?? "");
- name: Build dependency graph
shell: bash
run: |
# Quiet on success; surface output only if install fails.
bun install --frozen-lockfile > /tmp/bun-install.log 2>&1 \
|| { echo "bun install issues (continuing):"; tail -20 /tmp/bun-install.log; }
bunx depcruise frontend/src enterprise/frontend/src \
--config .dependency-cruiser.cjs \
--output-type json > dependency-graph.json || true
- name: Create test plan
id: compute
shell: bash
@@ -82,15 +66,20 @@ runs:
LOKI_INFRA_TOUCHED: ${{ inputs.loki-infra-touched }}
E2E_INFRA_TOUCHED: ${{ inputs.e2e-infra-touched }}
SHARED_SOURCES_TOUCHED: ${{ inputs.shared-sources-touched }}
DEP_GRAPH_JSON: dependency-graph.json
E2E_SPEC_MANIFEST: ${{ steps.manifest.outputs.path }}
run: |
# stdout is the plan JSON; the planner's own info logs go to stderr.
# The planner writes this step's GITHUB_OUTPUT entries.
bun .github/scripts/create-test-plan.ts > test-plan.json
echo "stats=$(jq -c '.stats' test-plan.json)" >> "$GITHUB_OUTPUT"
echo "fe_unit_specs_to_run=$(jq -c '.fe_unit_specs_to_run' test-plan.json)" >> "$GITHUB_OUTPUT"
echo "loki_stories_to_run=$(jq -c '.loki_stories_to_run' test-plan.json)" >> "$GITHUB_OUTPUT"
echo "e2e_specs_to_run=$(jq -c '.e2e_specs_to_run' test-plan.json)" >> "$GITHUB_OUTPUT"
- name: Upload test plan
id: upload
uses: actions/upload-artifact@v7
with:
name: test-plan-${{ github.run_id }}-${{ github.run_attempt }}
path: test-plan.json
if-no-files-found: error
retention-days: 7
- name: Report test plan
shell: bash
@@ -110,6 +99,10 @@ runs:
echo
echo "Changed: $(jq '.stats.fe_files_changed' "$plan") FE / $(jq '.stats.be_files_changed' "$plan") BE file(s) · $(jq '.stats.fe_modules_affected_usage' "$plan") FE module(s) affected"
echo
if [ "$(jq '.stats.loki_preview_touched' "$plan")" = "true" ]; then
echo "**Loki: full run**: a file the Storybook preview imports changed."
echo
fi
e2e_run=$(jq '.stats.e2e_specs_to_run_usage' "$plan")
e2e_all=$(jq '.stats.e2e_specs_all' "$plan")
if [ "$e2e_run" -eq "$e2e_all" ]; then

View File

@@ -0,0 +1,39 @@
name: Prepare test selection
description: Download this run's test plan and write the suite's selected test paths.
inputs:
suite:
description: The suite to select tests for (unit, loki or e2e).
required: true
test-plan-artifact-id:
description: ID of this run's test-plan.json artifact.
required: true
require-download:
description: Fail the step when the test plan download fails, instead of running the full suite.
required: false
default: "false"
outputs:
paths-file:
description: JSON file listing the selected test paths, or '' when the suite runs in full.
value: ${{ steps.selection.outputs.paths-file }}
selection:
description: full, narrowed or empty, or '' when the plan is unavailable and the suite runs in full.
value: ${{ steps.selection.outputs.selection }}
runs:
using: composite
steps:
- name: Download test plan
id: test-plan
continue-on-error: ${{ inputs.require-download != 'true' }}
uses: actions/download-artifact@v8
with:
artifact-ids: ${{ inputs.test-plan-artifact-id }}
path: ${{ runner.temp }}/test-plan
- name: Prepare test selection
id: selection
shell: bash
env:
PLAN_DOWNLOADED: ${{ steps.test-plan.outcome }}
run: bun .github/scripts/prepare-test-selection.ts "${{ inputs.suite }}"

View File

@@ -63,12 +63,10 @@ frontend_specs: &frontend_specs
- "frontend/**/tests/**"
- "frontend/**/*.unit.*"
- "enterprise/frontend/**/*.unit.*"
- ".github/**/*.unit.spec.*"
# file-paths.unit.spec.ts asserts what this file's own patterns match, so a change to it has to
# run the specs that guard it.
- ".github/file-paths.yaml"
- "jest.config.js"
- "jest.tz.unit.conf.json"
- "jest.base.conf.js"
- "jest.esm-packages.js"
- "jest.tz.unit.conf.js"
release_source: &release_sources
- "release/**"
@@ -80,6 +78,7 @@ frontend_all: &frontend_all
frontend_loki_ci: &frontend_loki_ci
- ".github/workflows/loki.yml"
- "loki.config.js"
- ".loki/**"
- ".storybook/**"
@@ -87,14 +86,52 @@ frontend_unit_infra: &frontend_unit_infra
- *frontend_configs
- *frontend_ci
- "jest.config.js"
- "jest.tz.unit.conf.json"
- "jest.base.conf.js"
- "jest.esm-packages.js"
- "jest.tz.unit.conf.js"
- "frontend/test/**"
- "frontend/**/__mocks__/**"
- "frontend/src/embedding-sdk-shared/jest/**"
frontend_loki_infra: &frontend_loki_infra
- *frontend_configs
- *shared_ci
- *frontend_loki_ci
- "frontend/build/shared/**"
- "frontend/test/__support__/**"
- "patches/**"
# dependency-cruiser doesn't follow CSS @import, so global styles must run every story.
- "frontend/src/metabase/css/**/*.css"
# Storybook serves resources/frontend_client as static files, and the stories bundle the app's SVGs.
- "resources/frontend_client/**"
- "frontend/**/*.svg"
- "enterprise/frontend/**/*.svg"
frontend_loki_all:
- *frontend_sources
- *frontend_loki_infra
# Everything the ci-scripts jest project runs or depends on.
# test-unit ignores that project, so this filter is the only thing that reruns it.
ci_scripts:
- *frontend_ci
- ".nvmrc"
# file-paths.unit.spec.ts asserts what this file's own patterns match, so a change to it has to
# run the specs that guard it.
- ".github/file-paths.yaml"
- ".github/scripts/**"
- ".github/actions/**"
- ".github/workflows/loki.yml"
- ".storybook/story-files.cjs"
- "frontend/lint/module-boundaries.mjs"
- "frontend/lint/shared-tiers.mjs"
- "frontend/test/jest-test-paths-filter.ts"
- ".dependency-cruiser.cjs"
- "jest.config.js"
- "jest.base.conf.js"
- "jest.esm-packages.js"
- "package.json"
- "bun.lock"
# e2e harness/support code (commands, support file, runner). Not in the
# instrumented bundle, so it maps to no covered module — a change here forces a

View File

@@ -105,6 +105,28 @@ export function getAffectedFiles(
return affected;
}
/** The file plus every file it transitively imports. */
export function getTransitiveDependencies(
fileDependencies: FileDependency[],
file: string,
): Set<string> {
const dependenciesOf = new Map(
fileDependencies.map(({ source, dependencies }) => [source, dependencies]),
);
const reached = new Set([file]);
const queue = [file];
while (queue.length > 0) {
const current = queue.shift()!;
for (const dependency of dependenciesOf.get(current) ?? []) {
if (!reached.has(dependency)) {
reached.add(dependency);
queue.push(dependency);
}
}
}
return reached;
}
export type ModuleGraph = {
nodes: ModuleNode[];
/**

View File

@@ -6,6 +6,7 @@ import {
getAffectedFiles,
getAffectedModules,
getChangedModules,
getTransitiveDependencies,
mapFileToModule,
parseCruiseModules,
} from "./affected-modules";
@@ -123,6 +124,31 @@ describe("affected modules", () => {
});
});
describe("getTransitiveDependencies", () => {
it("should return the file and everything it transitively imports", () => {
expect(
[...getTransitiveDependencies(FILE_DEPS, "src/app.js")].sort(),
).toEqual(["src/app.js", "src/foo/foo.tsx", "src/utils/colors.ts"]);
});
it("should return only the file when it imports nothing", () => {
expect([
...getTransitiveDependencies(FILE_DEPS, "src/utils/colors.ts"),
]).toEqual(["src/utils/colors.ts"]);
});
it("should stop at an import cycle", () => {
const cyclic = [
{ source: "src/a.ts", dependencies: ["src/b.ts"] },
{ source: "src/b.ts", dependencies: ["src/a.ts"] },
];
expect([...getTransitiveDependencies(cyclic, "src/a.ts")].sort()).toEqual(
["src/a.ts", "src/b.ts"],
);
});
});
describe("buildUsageModuleGraph + getAffectedModules", () => {
const usageGraph = buildUsageModuleGraph(ELEMENTS, FILE_DEPS);

View File

@@ -11,6 +11,7 @@ import {
buildUsageModuleGraph,
getAffectedModules,
getChangedModules,
getTransitiveDependencies,
mapFileToModule,
} from "./affected-modules";
@@ -21,6 +22,7 @@ export type TestPlanStats = {
be_files_total: number;
unit_infra_touched: boolean;
loki_infra_touched: boolean;
loki_preview_touched: boolean;
shared_sources_touched: boolean;
fe_modules_total: number;
fe_modules_changed: number;
@@ -48,9 +50,12 @@ export type CreateTestPlanInput = {
elements: ModuleDef[];
rules: Rule[];
changedFiles: string[];
// Parsed dependency-cruiser edges, or null to fall back to the rules graph.
fileDependencies: FileDependency[] | null;
// Loads the dependency-cruiser edges, called only when some suite can be narrowed.
// Null falls back to the rules graph.
loadFileDependencies: () => FileDependency[] | null;
testFilesBySuite: { unit: string[]; loki: string[]; e2e: string[] };
// Storybook's preview file, whose imports wrap every story.
lokiPreviewFile: string;
e2eSpecFiles: Record<string, string[]> | null;
unitInfraTouched: boolean;
lokiInfraTouched: boolean;
@@ -63,6 +68,7 @@ export type CreateTestPlanInput = {
};
// Tests whose owning module is affected.
// A spec outside the module tree always runs, because the graph cannot scope it.
export function filterAffectedTests(
nodes: ModuleNode[],
affected: Set<string>,
@@ -70,7 +76,7 @@ export function filterAffectedTests(
): string[] {
return testFiles.filter((file) => {
const module = mapFileToModule(nodes, file);
return module !== null && affected.has(module);
return module === null || affected.has(module);
});
}
@@ -130,8 +136,9 @@ export function createTestPlan({
elements,
rules,
changedFiles,
fileDependencies,
loadFileDependencies,
testFilesBySuite,
lokiPreviewFile,
e2eSpecFiles,
unitInfraTouched,
lokiInfraTouched,
@@ -143,16 +150,11 @@ export function createTestPlan({
beFilesTotal,
}: CreateTestPlanInput): TestPlan {
const rulesGraph = buildModuleGraph(elements, rules);
const usageGraph = fileDependencies
? buildUsageModuleGraph(elements, fileDependencies)
: rulesGraph;
const nodes = rulesGraph.nodes;
// Distinct module types (an element type can span several patterns).
const totalModules = new Set(elements.map((el) => el.type)).size;
const changedModules = getChangedModules(nodes, changedFiles);
const rulesAffected = getAffectedModules(rulesGraph, changedFiles);
const usageAffected = getAffectedModules(usageGraph, changedFiles);
// The coarse "feature" tier is the only set the e2e manifest is ever
// collapsed to (see filterAffectedE2eSpecs).
@@ -161,7 +163,7 @@ export function createTestPlan({
// cljc/cljs compile into the FE bundle, so they force a full run that module
// selection can't narrow — same effect as a suite's own infra changing.
const unitForceAll = unitInfraTouched || sharedSourcesTouched;
const lokiForceAll = lokiInfraTouched || sharedSourcesTouched;
const lokiInfraForceAll = lokiInfraTouched || sharedSourcesTouched;
// e2e is integration-level, so anything the FE-coverage manifest can't see
// forces a full run: cljc/cljs in the bundle (sharedSourcesTouched), a backend
// change that can break the UI (beFilesChanged), an e2e harness/support change
@@ -172,6 +174,28 @@ export function createTestPlan({
beFilesChanged > 0 ||
e2eSpecFiles === null;
// The cruise takes seconds, so it is skipped when every suite runs in full anyway.
const edges =
unitForceAll && lokiInfraForceAll && e2eForceAll
? null
: loadFileDependencies();
const usageGraph = edges
? buildUsageModuleGraph(elements, edges)
: rulesGraph;
const usageAffected = getAffectedModules(usageGraph, changedFiles);
// The preview wraps every story, so a change to a file it imports can change any screenshot.
const previewDependencies = edges
? getTransitiveDependencies(edges, lokiPreviewFile)
: new Set<string>();
const lokiPreviewTouched = changedFiles.some((file) =>
previewDependencies.has(file),
);
// Without the cruise, the rules graph cannot account for the preview's
// global dependencies, so it cannot safely narrow Loki.
const lokiForceAll =
lokiInfraForceAll || lokiPreviewTouched || edges === null;
const select = (forceAll: boolean, affected: Set<string>, files: string[]) =>
forceAll ? files : filterAffectedTests(nodes, affected, files);
@@ -208,6 +232,7 @@ export function createTestPlan({
be_files_total: beFilesTotal,
unit_infra_touched: unitInfraTouched,
loki_infra_touched: lokiInfraTouched,
loki_preview_touched: lokiPreviewTouched,
shared_sources_touched: sharedSourcesTouched,
fe_modules_total: totalModules,
fe_modules_changed: changedModules.size,

View File

@@ -26,8 +26,9 @@ const E2E_FILES = ["e2e/test/scenarios/a.cy.spec.ts"];
const baseInput = {
elements: ELEMENTS,
rules: RULES,
fileDependencies: null,
loadFileDependencies: () => null,
testFilesBySuite: { unit: UNIT_FILES, loki: LOKI_FILES, e2e: E2E_FILES },
lokiPreviewFile: ".storybook/preview.tsx",
e2eSpecFiles: null,
unitInfraTouched: false,
lokiInfraTouched: false,
@@ -41,7 +42,10 @@ const baseInput = {
describe("createTestPlan", () => {
it("runs only specs in affected modules (rules graph)", () => {
const plan = createTestPlan({ ...baseInput, changedFiles: ["src/foo/x.ts"] });
const plan = createTestPlan({
...baseInput,
changedFiles: ["src/foo/x.ts"],
});
expect(plan.stats.fe_modules_changed).toBe(1);
expect(plan.stats.fe_modules_affected_rules).toBe(1);
@@ -50,8 +54,8 @@ describe("createTestPlan", () => {
"src/foo/foo.unit.spec.ts",
]);
expect(plan.stats.unit_specs_to_run_rules).toBe(2);
expect(plan.stats.loki_stories_to_run_rules).toBe(1);
// With no file deps, usage falls back to rules.
expect(plan.loki_stories_to_run).toEqual(LOKI_FILES);
// With no file deps, unit usage falls back to rules, but Loki runs in full.
expect(plan.stats.unit_specs_to_run_usage).toBe(2);
});
@@ -60,7 +64,7 @@ describe("createTestPlan", () => {
...baseInput,
changedFiles: ["src/utils/colors.ts"],
// Only feature/foo actually imports lib/utils.
fileDependencies: [
loadFileDependencies: () => [
{ source: "src/foo/foo.tsx", dependencies: ["src/utils/colors.ts"] },
],
});
@@ -91,14 +95,69 @@ describe("createTestPlan", () => {
...baseInput,
changedFiles: ["docs/readme.md"],
unitInfraTouched: true,
loadFileDependencies: () => [],
});
expect(plan.stats.unit_specs_to_run_rules).toBe(UNIT_FILES.length);
expect(plan.stats.loki_stories_to_run_rules).toBe(0);
});
// The preview imports the theme, which imports the colors.
const PREVIEW_DEPS = [
{ source: ".storybook/preview.tsx", dependencies: ["src/utils/theme.ts"] },
{ source: "src/utils/theme.ts", dependencies: ["src/utils/colors.ts"] },
{ source: "src/foo/foo.tsx", dependencies: ["src/foo/x.ts"] },
];
it("forces a full Loki run when the Storybook preview imports a changed file", () => {
const plan = createTestPlan({
...baseInput,
changedFiles: ["src/utils/colors.ts"],
loadFileDependencies: () => PREVIEW_DEPS,
e2eSpecFiles: { "e2e/test/scenarios/a.cy.spec.ts": ["src/bar/b.ts"] },
});
expect(plan.stats.loki_preview_touched).toBe(true);
expect(plan.stats.loki_stories_to_run_rules).toBe(LOKI_FILES.length);
expect(plan.loki_stories_to_run).toEqual(LOKI_FILES);
// Unit and e2e still narrow to what the usage graph reaches from the change.
expect(plan.fe_unit_specs_to_run).toEqual(["src/utils/utils.unit.spec.ts"]);
expect(plan.e2e_specs_to_run).toEqual([]);
});
it("narrows Loki when the Storybook preview does not import the changed file", () => {
const plan = createTestPlan({
...baseInput,
changedFiles: ["src/foo/x.ts"],
loadFileDependencies: () => PREVIEW_DEPS,
});
expect(plan.stats.loki_preview_touched).toBe(false);
expect(plan.loki_stories_to_run).toEqual(["src/foo/Foo.stories.tsx"]);
});
it("runs all Loki stories when the cruise fails even if rules select fewer", () => {
const plan = createTestPlan({
...baseInput,
changedFiles: ["src/foo/x.ts"],
loadFileDependencies: () => null,
e2eSpecFiles: { "e2e/test/scenarios/a.cy.spec.ts": ["src/bar/b.ts"] },
});
expect(plan.stats.loki_preview_touched).toBe(false);
expect(plan.loki_stories_to_run).toEqual(LOKI_FILES);
expect(plan.stats.loki_stories_to_run_rules).toBe(LOKI_FILES.length);
expect(plan.stats.loki_stories_to_run_usage).toBe(LOKI_FILES.length);
// Unit and E2E keep their rules-graph fallback.
expect(plan.fe_unit_specs_to_run).toEqual(UNIT_FILES.slice(0, 2));
expect(plan.e2e_specs_to_run).toEqual([]);
});
it("runs the full e2e suite when no coverage manifest is available", () => {
const plan = createTestPlan({ ...baseInput, changedFiles: ["src/foo/x.ts"] });
const plan = createTestPlan({
...baseInput,
changedFiles: ["src/foo/x.ts"],
});
expect(plan.stats.e2e_specs_to_run_rules).toBe(E2E_FILES.length);
expect(plan.stats.e2e_specs_to_run_usage).toBe(E2E_FILES.length);
@@ -229,21 +288,53 @@ describe("createTestPlan", () => {
expect(plan.stats.fe_files_changed).toBe(5);
expect(plan.stats.be_files_changed).toBe(3);
});
it("does not build the usage graph when every suite runs in full", () => {
const loadFileDependencies = jest.fn(() => null);
createTestPlan({
...baseInput,
changedFiles: ["src/foo/x.ts"],
sharedSourcesTouched: true,
loadFileDependencies,
});
expect(loadFileDependencies).not.toHaveBeenCalled();
});
it("builds the usage graph when any suite can be narrowed", () => {
const loadFileDependencies = jest.fn(() => null);
createTestPlan({
...baseInput,
changedFiles: ["src/foo/x.ts"],
unitInfraTouched: true, // loki can still be narrowed
loadFileDependencies,
});
expect(loadFileDependencies).toHaveBeenCalledTimes(1);
});
});
describe("filterAffectedTests", () => {
it("keeps only tests inside affected modules", () => {
it("keeps tests inside affected modules, drops tests in unaffected ones", () => {
const nodes = buildNodes(ELEMENTS);
const affected = new Set(["feature/foo", "lib/utils"]);
const tests = [
"src/foo/foo.unit.spec.ts",
"src/utils/colors.unit.spec.ts",
"src/bar/bar.unit.spec.ts",
"docs/unrelated.unit.spec.ts",
];
expect(filterAffectedTests(nodes, affected, tests)).toEqual([
"src/foo/foo.unit.spec.ts",
"src/utils/colors.unit.spec.ts",
]);
});
it("keeps a test that maps to no module (unknown scope)", () => {
const nodes = buildNodes(ELEMENTS);
const affected = new Set(["feature/foo"]);
const tests = ["docs/unrelated.unit.spec.ts", "src/bar/bar.unit.spec.ts"];
expect(filterAffectedTests(nodes, affected, tests)).toEqual([
"docs/unrelated.unit.spec.ts",
]);
});
});

View File

@@ -0,0 +1,182 @@
import { spawnSync } from "node:child_process";
import { readFileSync, statSync } from "node:fs";
import { join } from "node:path";
const TEST_UNIT_COMMAND = ["bun", "run", "test-unit-keep-cljs"];
type SuiteResult = {
name: string;
status: string;
assertionResults: unknown[];
};
type JestResults = {
wasInterrupted: boolean;
testResults: SuiteResult[];
};
function isSuiteResult(value: unknown): value is SuiteResult {
return (
typeof value === "object" &&
value !== null &&
"name" in value &&
typeof value.name === "string" &&
"status" in value &&
typeof value.status === "string" &&
"assertionResults" in value &&
Array.isArray(value.assertionResults)
);
}
function readResults(file: string): JestResults | null {
const value = readJson(file);
if (
typeof value === "object" &&
value !== null &&
"wasInterrupted" in value &&
typeof value.wasInterrupted === "boolean" &&
"testResults" in value &&
Array.isArray(value.testResults) &&
value.testResults.every(isSuiteResult)
) {
return {
wasInterrupted: value.wasInterrupted,
testResults: value.testResults,
};
}
return null;
}
function parseJson(text: string): unknown {
try {
return JSON.parse(text);
} catch {
return null;
}
}
function readJson(file: string): unknown {
try {
return parseJson(readFileSync(file, "utf8"));
} catch {
return null;
}
}
function isStringArray(value: unknown): value is string[] {
return (
Array.isArray(value) && value.every((item) => typeof item === "string")
);
}
function hasContent(file: string): boolean {
try {
return statSync(file).size > 0;
} catch {
return false;
}
}
export function findRunnerFailure(env: NodeJS.ProcessEnv): string | null {
const {
UNIT_TEST_RESULTS_FILE,
JEST_JUNIT_OUTPUT_DIR,
JEST_JUNIT_OUTPUT_NAME,
} = env;
if (
!UNIT_TEST_RESULTS_FILE ||
!JEST_JUNIT_OUTPUT_DIR ||
!JEST_JUNIT_OUTPUT_NAME
) {
throw new Error(
"UNIT_TEST_RESULTS_FILE, JEST_JUNIT_OUTPUT_DIR and JEST_JUNIT_OUTPUT_NAME are required",
);
}
const results = readResults(UNIT_TEST_RESULTS_FILE);
if (results === null || results.wasInterrupted) {
return "Jest did not finish the run.";
}
if (!hasContent(join(JEST_JUNIT_OUTPUT_DIR, JEST_JUNIT_OUTPUT_NAME))) {
return "Jest did not write a JUnit report.";
}
const failedSuites = results.testResults.filter(
(suite) => suite.status === "failed",
);
// jest-junit leaves out suites with no test results, so the quarantine gate never sees them.
const suitesWithoutResults = failedSuites.filter(
(suite) => suite.assertionResults.length === 0,
);
if (suitesWithoutResults.length > 0) {
const names = suitesWithoutResults.map((suite) => suite.name).join(", ");
return `Test suites failed to run: ${names}`;
}
if (failedSuites.length === 0) {
return "Jest failed without a failing test suite.";
}
return null;
}
function listTests(
command: readonly string[],
env: NodeJS.ProcessEnv,
): string[] | null {
const [program, ...args] = command;
const result = spawnSync(program, [...args, "--listTests", "--json"], {
env,
encoding: "utf8",
stdio: ["ignore", "pipe", "inherit"],
});
if (result.status !== 0) {
return null;
}
const listed = parseJson(result.stdout);
return isStringArray(listed) ? listed : null;
}
// --passWithNoTests lets every shard pass when the selection matches nothing Jest discovers.
// A single empty shard is normal, so only an unsharded listing that is also empty warns.
export function findEmptySelection(
env: NodeJS.ProcessEnv,
command: readonly string[] = TEST_UNIT_COMMAND,
): string | null {
const { UNIT_TEST_RESULTS_FILE, JEST_TEST_PATHS_FILE } = env;
if (!UNIT_TEST_RESULTS_FILE || !JEST_TEST_PATHS_FILE) {
return null;
}
const results = readResults(UNIT_TEST_RESULTS_FILE);
const selection = readJson(JEST_TEST_PATHS_FILE);
if (
results === null ||
results.testResults.length > 0 ||
!isStringArray(selection) ||
selection.length === 0
) {
return null;
}
const listed = listTests(command, env);
if (listed === null) {
return "Could not list Jest's unit tests to check the selection.";
}
if (listed.length > 0) {
return null;
}
return `None of the ${selection.length} selected unit specs match a test Jest discovers, so no unit tests ran.`;
}
if (require.main === module) {
if (process.env.TEST_OUTCOME === "failure") {
const failure = findRunnerFailure(process.env);
if (failure !== null) {
console.error(`::error::${failure}`);
process.exitCode = 1;
}
} else {
const warning = findEmptySelection(process.env);
if (warning !== null) {
console.warn(`::warning::${warning}`);
}
}
}

View File

@@ -1,28 +1,31 @@
// I/O entrypoint: gather inputs (env vars, the cruise graph, the test-file
// lists) and hand them to createTestPlan, which does the computing.
// I/O entrypoint: gather inputs (env vars, the test-file lists, the cruise graph on request),
// hand them to createTestPlan, which does the computing, and publish its small stats output.
import { execFileSync } from "node:child_process";
import { existsSync, readFileSync } from "node:fs";
import { execFileSync, spawnSync } from "node:child_process";
import { appendFileSync, existsSync, readFileSync } from "node:fs";
import micromatch from "micromatch";
import { MAIN_APP_STORY_GLOBS } from "../../.storybook/story-files.cjs";
import { elements, rules } from "../../frontend/lint/module-boundaries.mjs";
import { type FileDependency, parseCruiseModules } from "./affected-modules";
import { createTestPlan } from "./affected-tests";
import { listSpecFiles } from "./e2e-spec-globs.mjs";
const UNIT_ROOTS = ["frontend/src", "enterprise/frontend/src"];
// The specs `bun run test-unit` runs.
const UNIT_ROOTS = ["."];
const UNIT_GLOBS = [
"frontend/src/**/*.unit.spec.{js,jsx,ts,tsx}",
"enterprise/frontend/src/**/*.unit.spec.{js,jsx,ts,tsx}",
"**/*.unit.spec.{js,jsx,ts,tsx}",
"!.github/**", // the ci-scripts project, which test-unit ignores
"!release/**", // has its own jest config
"!enterprise/frontend/src/custom-viz/**", // modulePathIgnorePatterns in jest.config.js
"!**/*.leak.unit.spec.{js,jsx,ts,tsx}", // run only by jest.memory.conf.js
];
const STORY_ROOTS = ["frontend", "enterprise/frontend"];
const STORY_GLOBS = [
"frontend/**/*.stories.{js,jsx,ts,tsx}",
"enterprise/frontend/**/*.stories.{js,jsx,ts,tsx}",
];
const FRONTEND_ROOTS = ["frontend", "enterprise/frontend"];
const LOKI_PREVIEW_FILE = ".storybook/preview.tsx";
// `git ls-files -- frontend enterprise/frontend` already prints just over a
// megabyte of paths, and node's default maxBuffer is exactly 1 MiB: past that
@@ -32,12 +35,11 @@ const LS_FILES_MAX_BUFFER = 64 * 1024 * 1024;
// Returns the tracked files under `roots` that match `globs`. The `dot: true`
// option means files inside dot-directories such as `.storybook` are included.
function listFiles(roots: string[], globs: string[]): string[] {
const tracked = execFileSync("git", ["ls-files", "--", ...roots], {
const tracked = execFileSync("git", ["ls-files", "-z", "--", ...roots], {
encoding: "utf8",
maxBuffer: LS_FILES_MAX_BUFFER,
})
.split("\n")
.map((line) => line.trim())
.split("\0")
.filter(Boolean);
return micromatch(tracked, globs, { dot: true });
}
@@ -49,26 +51,43 @@ const csvToList = (csv: string | undefined) =>
.map((s) => s.trim())
.filter(Boolean);
// Reads the dependency-cruiser graph (DEP_GRAPH_JSON). Null falls back to the
// rules graph, so a missing or unparseable file never breaks the plan.
function readFileDependencies(): FileDependency[] | null {
const path = process.env.DEP_GRAPH_JSON;
if (path && existsSync(path)) {
try {
const { modules } = JSON.parse(readFileSync(path, "utf8"));
process.stderr.write(`Using usage graph from ${path}.\n`);
return parseCruiseModules(modules);
} catch (error) {
process.stderr.write(
`Failed to read ${path}; falling back to rules graph: ${error}\n`,
);
}
} else {
process.stderr.write(
"No DEP_GRAPH_JSON found; falling back to rules graph.\n",
// Runs dependency-cruiser over the frontend sources and the Storybook preview and parses its edges.
// Null falls back to the rules graph, so a failed cruise never breaks the plan.
function loadFileDependencies(): FileDependency[] | null {
const output = "dependency-graph.json";
process.stderr.write("Building usage graph with dependency-cruiser.\n");
try {
// stdout is the plan JSON, so the cruise's own output goes to stderr.
const result = spawnSync(
"bunx",
[
"depcruise",
"frontend/src",
"enterprise/frontend/src",
LOKI_PREVIEW_FILE,
"--config",
".dependency-cruiser.cjs",
"--output-type",
"json",
"--output-to",
output,
],
{ stdio: ["ignore", 2, 2] },
);
if (result.error) {
throw result.error;
}
if (result.status !== 0) {
throw new Error(`depcruise exited with status ${result.status}`);
}
const { modules } = JSON.parse(readFileSync(output, "utf8"));
return parseCruiseModules(modules);
} catch (error) {
process.stderr.write(
`Failed to build usage graph; falling back to rules graph: ${error}\n`,
);
return null;
}
return null;
}
// Reads the nightly coverage manifest (E2E_SPEC_MANIFEST): { builtAt, specs:
@@ -98,12 +117,13 @@ const testPlan = createTestPlan({
elements,
rules,
changedFiles: csvToList(process.env.CHANGED_FILES),
fileDependencies: readFileDependencies(),
loadFileDependencies,
testFilesBySuite: {
unit: listFiles(UNIT_ROOTS, UNIT_GLOBS),
loki: listFiles(STORY_ROOTS, STORY_GLOBS),
loki: listFiles(FRONTEND_ROOTS, MAIN_APP_STORY_GLOBS),
e2e: listSpecFiles(),
},
lokiPreviewFile: LOKI_PREVIEW_FILE,
e2eSpecFiles: readE2eSpecFiles(),
unitInfraTouched: process.env.UNIT_INFRA_TOUCHED === "true",
lokiInfraTouched: process.env.LOKI_INFRA_TOUCHED === "true",
@@ -111,8 +131,15 @@ const testPlan = createTestPlan({
sharedSourcesTouched: process.env.SHARED_SOURCES_TOUCHED === "true",
feFilesChanged: csvToList(process.env.FE_CHANGED_FILES).length,
beFilesChanged: csvToList(process.env.BE_CHANGED_FILES).length,
feFilesTotal: listFiles(["frontend", "enterprise/frontend"], ["**"]).length,
feFilesTotal: listFiles(FRONTEND_ROOTS, ["**"]).length,
beFilesTotal: listFiles(["src", "enterprise/backend"], ["**"]).length,
});
process.stdout.write(JSON.stringify(testPlan) + "\n");
if (process.env.GITHUB_OUTPUT) {
appendFileSync(
process.env.GITHUB_OUTPUT,
`stats=${JSON.stringify(testPlan.stats)}\n`,
);
}

View File

@@ -11,12 +11,17 @@ type Filter = string | Filter[] | { [changeType: string]: Filter };
// A filter is a list of patterns, and an entry may name the change types it applies to as
// { "added|modified": pattern }. Either way the patterns are what matters here.
const patterns = (value: Filter): string[] => {
if (typeof value === "string") return [value];
if (Array.isArray(value)) return value.flatMap(patterns);
if (typeof value === "string") {
return [value];
}
if (Array.isArray(value)) {
return value.flatMap(patterns);
}
return Object.values(value).flatMap(patterns);
};
describe("file-paths.yaml", () => {
// js-yaml returns an untyped value, and this repository file defines named path filters.
const filters = load(readFileSync(FILTERS, "utf8")) as Record<string, Filter>;
// dorny/paths-filter matches with picomatch, which micromatch wraps, so this asks the same
@@ -44,14 +49,47 @@ describe("file-paths.yaml", () => {
"%s skips a ratchets-only change and still follows the rest of .clj-kondo",
(name) => {
expect(matches(name, ".clj-kondo/ratchets.edn")).toBe(false);
expect(
matches(name, ".clj-kondo/config/modules/ratchets.edn"),
).toBe(false);
expect(matches(name, ".clj-kondo/config/modules/ratchets.edn")).toBe(
false,
);
expect(matches(name, ".clj-kondo/config.edn")).toBe(true);
expect(matches(name, ".clj-kondo/config/modules/config.edn")).toBe(true);
},
);
it("runs CI script tests when the Node version changes", () => {
expect(matches("ci_scripts", ".nvmrc")).toBe(true);
});
it.each([
"jest.config.js",
"jest.base.conf.js",
"jest.esm-packages.js",
"jest.tz.unit.conf.js",
])("runs every frontend unit test when %s changes", (path) => {
expect(matches("frontend_specs", path)).toBe(true);
expect(matches("frontend_unit_infra", path)).toBe(true);
});
it("runs every frontend unit test when an SDK jest setup file changes", () => {
expect(
matches(
"frontend_unit_infra",
"frontend/src/embedding-sdk-shared/jest/setup-env.ts",
),
).toBe(true);
});
it.each([
"jest.config.js",
"jest.base.conf.js",
"jest.esm-packages.js",
".storybook/story-files.cjs",
".github/workflows/loki.yml",
])("runs CI script tests when %s changes", (path) => {
expect(matches("ci_scripts", path)).toBe(true);
});
it("runs the ratchet check on the ratchets file", () => {
expect(matches("project_ratchet_checks", ".clj-kondo/ratchets.edn")).toBe(
true,
@@ -63,4 +101,42 @@ describe("file-paths.yaml", () => {
),
).toBe(true);
});
it.each([
"loki.config.js",
".storybook/story-files.cjs",
".storybook/preview.tsx",
".loki/reference/chrome.laptop-Button.png",
".github/workflows/loki.yml",
".github/workflows/run-tests.yml",
".github/actions/prepare-frontend/action.yml",
"frontend/build/shared/rspack/css-config.js",
"frontend/test/__support__/custom-viz-fixtures/calendar-heatmap/index.js",
"patches/@loki+browser+0.35.0.patch",
"frontend/src/metabase/css/index.module.css",
"frontend/src/metabase/css/core/base.module.css",
"resources/frontend_client/app/fonts/Lato/lato-v16-latin-regular.woff2",
"resources/frontend_client/app/assets/img/no_results.svg",
"frontend/src/metabase/ui/components/icons/Icon/icons/warning.svg",
"enterprise/frontend/src/metabase-enterprise/google_drive/database-error.svg",
])("runs all Loki stories when %s changes", (file) => {
expect(matches("frontend_loki_all", file)).toBe(true);
expect(matches("frontend_loki_infra", file)).toBe(true);
});
it.each([
"frontend/src/metabase/ui/Button.stories.tsx",
"frontend/src/metabase/ui/components/feedback/Alert/Alert.module.css",
"frontend/build/embedding-sdk/rspack.config.js",
])("allows a change to %s to narrow the Loki run", (file) => {
expect(matches("frontend_loki_all", file)).toBe(true);
expect(matches("frontend_loki_infra", file)).toBe(false);
});
it.each([
".github/workflows/frontend.yml",
"frontend/test/metabase/scenarios/Button.unit.spec.tsx",
])("does not run Loki stories when %s changes", (file) => {
expect(matches("frontend_loki_all", file)).toBe(false);
});
});

View File

@@ -0,0 +1,237 @@
import { spawnSync } from "node:child_process";
import {
mkdirSync,
mkdtempSync,
readFileSync,
realpathSync,
rmSync,
writeFileSync,
} from "node:fs";
import { tmpdir } from "node:os";
import { join, resolve } from "node:path";
import { load } from "js-yaml";
import {
findEmptySelection,
findRunnerFailure,
} from "./check-unit-test-results";
import { prepareTestSelection } from "./prepare-test-selection";
type Step = { name: string; run?: string; "continue-on-error"?: boolean };
// js-yaml returns an untyped value, and this repository workflow defines the job's steps.
const workflow = load(
readFileSync(resolve(__dirname, "../workflows/frontend.yml"), "utf8"),
) as { jobs: { "fe-tests-unit": { steps: Step[] } } };
const steps = workflow.jobs["fe-tests-unit"].steps;
const checkScript = resolve(__dirname, "check-unit-test-results.ts");
const filter = resolve(
__dirname,
"../../frontend/test/jest-test-paths-filter.ts",
);
describe("frontend test plan handoff", () => {
let dir: string;
let env: NodeJS.ProcessEnv;
beforeEach(() => {
jest.spyOn(console, "warn").mockImplementation(() => {});
// Jest reports real paths, and macOS's temporary directory is behind a symlink.
dir = realpathSync(mkdtempSync(join(tmpdir(), "frontend-test-plan-")));
mkdirSync(join(dir, "test-plan"));
writeFileSync(join(dir, "output"), "");
env = {
...process.env,
NODE_OPTIONS: "",
JEST_TEST_PATHS_FILE: "",
RUNNER_TEMP: dir,
GITHUB_OUTPUT: join(dir, "output"),
PLAN_DOWNLOADED: "success",
JEST_JUNIT_OUTPUT_DIR: dir,
JEST_JUNIT_OUTPUT_NAME: "junit.xml",
UNIT_TEST_RESULTS_FILE: join(dir, "unit-test-results.json"),
};
});
afterEach(() => {
jest.restoreAllMocks();
rmSync(dir, { recursive: true, force: true });
});
function plan(files: string[], total = 2) {
return {
fe_unit_specs_to_run: files,
stats: { unit_specs_all: total, unit_specs_to_run_usage: files.length },
};
}
function writePlan(value: unknown) {
writeFileSync(join(dir, "test-plan/test-plan.json"), JSON.stringify(value));
}
function jestCommand(useFilter: boolean) {
return [
process.execPath,
require.resolve("jest/bin/jest"),
"--config",
JSON.stringify({
rootDir: dir,
testMatch: ["**/*.spec.cjs"],
reporters: ["default", require.resolve("jest-junit")],
...(useFilter && { filter }),
}),
];
}
function runJest(
files: Record<string, string>,
useFilter = false,
args: string[] = [],
) {
for (const [name, source] of Object.entries(files)) {
writeFileSync(join(dir, name), source);
}
const [program, ...commandArgs] = jestCommand(useFilter);
return spawnSync(
program,
[
...commandArgs,
"--runInBand",
"--watch=false",
"--json",
`--outputFile=${env.UNIT_TEST_RESULTS_FILE}`,
...(useFilter ? ["--passWithNoTests"] : []),
...args,
],
{ cwd: dir, env, encoding: "utf8" },
);
}
function readResults() {
return JSON.parse(
readFileSync(join(dir, "unit-test-results.json"), "utf8"),
);
}
function writeSelection(files: string[]) {
env.JEST_TEST_PATHS_FILE = join(dir, "unit-specs.json");
writeFileSync(env.JEST_TEST_PATHS_FILE, JSON.stringify(files));
}
it("lets ordinary assertion failures reach quarantine", () => {
expect(
runJest({ "fail.spec.cjs": "test('fails', () => expect(1).toBe(2));" })
.status,
).toBe(1);
expect(findRunnerFailure(env)).toBeNull();
});
it("lets suite hook failures reach quarantine", () => {
expect(
runJest({
"hook.spec.cjs":
"test('passes', () => {}); afterAll(() => { throw new Error('hook'); });",
}).status,
).toBe(1);
expect(readFileSync(join(dir, "junit.xml"), "utf8")).toContain(
"Test execution failure",
);
expect(findRunnerFailure(env)).toBeNull();
});
it("fails a filter crash even though the test step permits errors", () => {
env.JEST_TEST_PATHS_FILE = join(dir, "broken.json");
writeFileSync(env.JEST_TEST_PATHS_FILE, "[");
expect(
runJest({ "pass.spec.cjs": "test('passes', () => {});" }, true).status,
).toBe(1);
env.TEST_OUTCOME = "failure";
const result = spawnSync("bun", [checkScript], { env, encoding: "utf8" });
expect(result.status).toBe(1);
expect(result.stderr).toContain("::error::");
const step = steps.find((step) =>
step.run?.includes("check-unit-test-results.ts"),
);
expect(step?.["continue-on-error"]).toBeUndefined();
});
it("fails suite startup errors even alongside ordinary assertion failures", () => {
expect(
runJest({
"crash.spec.cjs": "throw new Error('startup crash');",
"fail.spec.cjs": "test('fails', () => expect(1).toBe(2));",
}).status,
).toBe(1);
expect(readFileSync(join(dir, "junit.xml"), "utf8")).not.toContain(
"crash.spec.cjs",
);
expect(findRunnerFailure(env)).toContain("crash.spec.cjs");
});
it("fails when JUnit results are missing", () => {
runJest({ "fail.spec.cjs": "test('fails', () => expect(1).toBe(2));" });
rmSync(join(dir, "junit.xml"));
expect(findRunnerFailure(env)).not.toBeNull();
});
it.each([true, false])(
"loads the TypeScript filter natively with an empty selection: %s",
(empty) => {
writePlan(plan(empty ? [] : [join(dir, "pass.spec.cjs")]));
prepareTestSelection("unit", env);
env.JEST_TEST_PATHS_FILE = join(dir, "unit-specs.json");
const result = runJest(
{
"pass.spec.cjs": "test('passes', () => {});",
"fail.spec.cjs":
"test('excluded', () => { throw new Error('must not run'); });",
},
true,
);
expect(result.status === 0 ? "" : result.stderr).toBe("");
expect(result.status).toBe(0);
expect(readResults()).toMatchObject({
numTotalTestSuites: empty ? 0 : 1,
});
},
);
it("warns when a narrowed run matches none of Jest's test paths", () => {
writeSelection([join(dir, "renamed.spec.cjs")]);
expect(
runJest({ "pass.spec.cjs": "test('passes', () => {});" }, true).status,
).toBe(0);
expect(findEmptySelection(env, jestCommand(true))).toContain(
"None of the 1 selected unit specs",
);
});
it("warns through Bun when the unit test step passed", () => {
writeSelection([join(dir, "renamed.spec.cjs")]);
runJest({ "pass.spec.cjs": "test('passes', () => {});" }, true);
env.TEST_OUTCOME = "success";
const result = spawnSync("bun", [checkScript], { env, encoding: "utf8" });
expect(result.status).toBe(0);
expect(result.stderr).toContain(
"::warning::None of the 1 selected unit specs",
);
});
it("does not warn for a shard the selection leaves empty", () => {
writeSelection([join(dir, "pass.spec.cjs")]);
const result = runJest(
{ "pass.spec.cjs": "test('passes', () => {});" },
true,
["--shard=2/2"],
);
expect(result.status).toBe(0);
expect(readResults()).toMatchObject({ numTotalTestSuites: 0 });
expect(findEmptySelection(env, jestCommand(true))).toBeNull();
});
it("does not list tests for an empty selection", () => {
writeSelection([]);
runJest({ "pass.spec.cjs": "test('passes', () => {});" }, true);
expect(findEmptySelection(env, ["false"])).toBeNull();
});
});

View File

@@ -0,0 +1,44 @@
import fs from "node:fs";
import os from "node:os";
import path from "node:path";
// eslint-disable-next-line @typescript-eslint/no-require-imports -- Match Jest's direct CommonJS loading contract.
const filter = require("../../frontend/test/jest-test-paths-filter");
describe("jest-test-paths-filter", () => {
const originalPathsFile = process.env.JEST_TEST_PATHS_FILE;
let tempDir: string;
let pathsFile: string;
beforeEach(() => {
tempDir = fs.mkdtempSync(path.join(os.tmpdir(), "jest-test-paths-"));
pathsFile = path.join(tempDir, "paths.json");
process.env.JEST_TEST_PATHS_FILE = pathsFile;
});
afterEach(() => {
fs.rmSync(tempDir, { recursive: true, force: true });
if (originalPathsFile === undefined) {
delete process.env.JEST_TEST_PATHS_FILE;
} else {
process.env.JEST_TEST_PATHS_FILE = originalPathsFile;
}
});
it("should keep the listed paths and drop the rest", () => {
const one = path.resolve("frontend/one.unit.spec.ts");
const two = path.resolve("frontend/two.unit.spec.ts");
const three = path.resolve("frontend/three.unit.spec.ts");
fs.writeFileSync(pathsFile, JSON.stringify([one, two]));
expect(filter([one, three, two])).toEqual({ filtered: [one, two] });
});
it("should resolve relative paths against the working directory", () => {
fs.writeFileSync(pathsFile, JSON.stringify(["frontend/one.unit.spec.ts"]));
const one = path.resolve("frontend/one.unit.spec.ts");
const two = path.resolve("frontend/two.unit.spec.ts");
expect(filter([one, two])).toEqual({ filtered: [one] });
});
});

View File

@@ -0,0 +1,122 @@
import { appendFileSync, readFileSync, writeFileSync } from "node:fs";
import { join } from "node:path";
import type { TestPlan, TestPlanStats } from "./affected-tests";
type NumberStat = {
[K in keyof TestPlanStats]: TestPlanStats[K] extends number ? K : never;
}[keyof TestPlanStats];
type Suite = {
name: string;
filesKey: Exclude<keyof TestPlan, "stats">;
totalKey: NumberStat;
selectedKey: NumberStat;
pathsFile: string;
};
export const SUITES = {
unit: {
name: "unit",
filesKey: "fe_unit_specs_to_run",
totalKey: "unit_specs_all",
selectedKey: "unit_specs_to_run_usage",
pathsFile: "unit-specs.json",
},
loki: {
name: "Loki",
filesKey: "loki_stories_to_run",
totalKey: "loki_stories_all",
selectedKey: "loki_stories_to_run_usage",
pathsFile: "loki-stories.json",
},
e2e: {
name: "E2E",
filesKey: "e2e_specs_to_run",
totalKey: "e2e_specs_all",
selectedKey: "e2e_specs_to_run_usage",
pathsFile: "e2e-specs.json",
},
} satisfies Record<string, Suite>;
function isSuiteName(name: string): name is keyof typeof SUITES {
return Object.hasOwn(SUITES, name);
}
function isRecord(value: unknown): value is Record<string, unknown> {
return typeof value === "object" && value !== null;
}
function isStringArray(value: unknown): value is string[] {
return (
Array.isArray(value) && value.every((item) => typeof item === "string")
);
}
function isCount(value: unknown): value is number {
return typeof value === "number" && Number.isInteger(value) && value >= 0;
}
// null means a full run, and [] is an explicit selection of no tests.
function readSelection(planFile: string, suite: Suite): string[] | null {
const plan: unknown = JSON.parse(readFileSync(planFile, "utf8"));
const files = isRecord(plan) ? plan[suite.filesKey] : undefined;
const stats = isRecord(plan) && isRecord(plan.stats) ? plan.stats : {};
const total = stats[suite.totalKey];
if (
!isStringArray(files) ||
files.some((file) => file.length === 0) ||
!isCount(total) ||
files.length !== stats[suite.selectedKey] ||
files.length > total
) {
throw new Error(`Invalid ${suite.name} test selection`);
}
return files.length === total ? null : files;
}
export function prepareTestSelection(
suiteName: keyof typeof SUITES,
env: NodeJS.ProcessEnv = process.env,
) {
const suite: Suite = SUITES[suiteName];
const { RUNNER_TEMP, GITHUB_OUTPUT, PLAN_DOWNLOADED } = env;
if (!RUNNER_TEMP || !GITHUB_OUTPUT) {
throw new Error("RUNNER_TEMP and GITHUB_OUTPUT are required");
}
let files: string[] | null;
try {
if (PLAN_DOWNLOADED !== "success") {
throw new Error("Test plan download failed");
}
files = readSelection(join(RUNNER_TEMP, "test-plan/test-plan.json"), suite);
} catch (error) {
console.warn(
`::warning::Test plan unavailable or invalid, running the full ${suite.name} suite: ${error}`,
);
return;
}
if (files === null) {
appendFileSync(GITHUB_OUTPUT, "selection=full\n");
return;
}
const pathsFile = join(RUNNER_TEMP, suite.pathsFile);
writeFileSync(pathsFile, JSON.stringify(files));
appendFileSync(
GITHUB_OUTPUT,
`paths-file=${pathsFile}\nselection=${files.length === 0 ? "empty" : "narrowed"}\n`,
);
}
if (require.main === module) {
const suiteName = process.argv[2] ?? "";
if (!isSuiteName(suiteName)) {
throw new Error(
`Unknown test suite "${suiteName}", expected one of: ${Object.keys(SUITES).join(", ")}`,
);
}
prepareTestSelection(suiteName);
}

View File

@@ -0,0 +1,268 @@
import { spawnSync } from "node:child_process";
import {
mkdirSync,
mkdtempSync,
readFileSync,
rmSync,
writeFileSync,
} from "node:fs";
import { tmpdir } from "node:os";
import { join, resolve } from "node:path";
import { load } from "js-yaml";
import { SUITES, prepareTestSelection } from "./prepare-test-selection";
type Suite = (typeof SUITES)[keyof typeof SUITES];
type Step = {
id?: string;
if?: string;
uses?: string;
run?: string;
env?: Record<string, string>;
with?: Record<string, string | boolean>;
};
const SCRIPT = resolve(__dirname, "prepare-test-selection.ts");
const SUITE_NAMES = ["unit", "loki", "e2e"] satisfies (keyof typeof SUITES)[];
function loadWorkflow(file: string) {
// js-yaml returns an untyped value, and these repository workflows define jobs with steps.
return load(
readFileSync(resolve(__dirname, "../workflows", file), "utf8"),
) as {
jobs: Record<
string,
{ if?: string; outputs?: Record<string, string>; steps?: Step[] }
>;
};
}
function loadAction() {
// js-yaml returns an untyped value, and this repository action defines outputs and steps.
return load(
readFileSync(
resolve(__dirname, "../actions/prepare-test-selection/action.yml"),
"utf8",
),
) as {
outputs: Record<string, { value: string }>;
runs: { steps: Step[] };
};
}
describe("prepareTestSelection", () => {
const { loki } = SUITES;
let dir: string;
let env: NodeJS.ProcessEnv;
beforeEach(() => {
jest.spyOn(console, "warn").mockImplementation(() => {});
dir = mkdtempSync(join(tmpdir(), "test-selection-"));
mkdirSync(join(dir, "test-plan"));
writeFileSync(join(dir, "output"), "");
env = {
...process.env,
RUNNER_TEMP: dir,
GITHUB_OUTPUT: join(dir, "output"),
PLAN_DOWNLOADED: "success",
};
});
afterEach(() => {
jest.restoreAllMocks();
rmSync(dir, { recursive: true, force: true });
});
function plan(files: unknown, total = 2, suite: Suite = loki) {
return {
[suite.filesKey]: files,
stats: {
[suite.totalKey]: total,
[suite.selectedKey]: Array.isArray(files) ? files.length : 0,
},
};
}
function writePlan(value: unknown) {
writeFileSync(join(dir, "test-plan/test-plan.json"), JSON.stringify(value));
}
const output = () => readFileSync(join(dir, "output"), "utf8");
it.each([{ files: ["one.spec.cjs"] }, { files: [] }])(
"preserves an explicit selection: $files",
({ files }) => {
writePlan(plan(files));
prepareTestSelection("loki", env);
expect(output()).toBe(
`paths-file=${join(dir, loki.pathsFile)}\nselection=${files.length > 0 ? "narrowed" : "empty"}\n`,
);
expect(
JSON.parse(readFileSync(join(dir, loki.pathsFile), "utf8")),
).toEqual(files);
},
);
it("writes the selection when invoked through Bun in CI", () => {
writePlan(plan(["one.spec.cjs"]));
const result = spawnSync("bun", [SCRIPT, "loki"], {
env,
encoding: "utf8",
});
expect(result.status === 0 ? "" : result.stderr).toBe("");
expect(result.status).toBe(0);
expect(output()).toBe(
`paths-file=${join(dir, loki.pathsFile)}\nselection=narrowed\n`,
);
});
it("writes no paths file for a full selection", () => {
writePlan(plan(["one.spec.cjs", "two.spec.cjs"]));
prepareTestSelection("loki", env);
expect(output()).toBe("selection=full\n");
});
it.each([
null,
{},
plan("[]"),
plan([null]),
{ ...plan([]), [loki.filesKey]: ["one.spec.cjs"] },
plan([""]),
plan(["one.spec.cjs"], 0),
])("runs in full for an invalid plan: %j", (value) => {
writePlan(value);
prepareTestSelection("loki", env);
expect(console.warn).toHaveBeenCalledWith(
expect.stringContaining("::warning::"),
);
expect(output()).toBe("");
});
it.each([
{ scenario: "the test plan is missing", setup: () => {} },
{
scenario: "the test plan is truncated",
setup: () => {
writeFileSync(
join(dir, "test-plan/test-plan.json"),
`{"${loki.filesKey}":[`,
);
},
},
{
scenario: "the download failed",
setup: () => {
writePlan(plan([])); // Even a partial download must not be trusted.
env.PLAN_DOWNLOADED = "failure";
},
},
])("runs in full when $scenario", ({ setup }) => {
setup();
prepareTestSelection("loki", env);
expect(console.warn).toHaveBeenCalledWith(
expect.stringContaining("::warning::"),
);
expect(output()).toBe("");
});
it.each(SUITE_NAMES)(
"reads the %s selection from its own plan keys",
(suiteName) => {
const suite = SUITES[suiteName];
writePlan(plan(["one.spec.cjs"], 2, suite));
prepareTestSelection(suiteName, env);
expect(output()).toBe(
`paths-file=${join(dir, suite.pathsFile)}\nselection=narrowed\n`,
);
expect(
JSON.parse(readFileSync(join(dir, suite.pathsFile), "utf8")),
).toEqual(["one.spec.cjs"]);
},
);
it("ignores the other suites' selections", () => {
writePlan({
...plan(["one.spec.cjs", "two.spec.cjs"]),
[SUITES.unit.filesKey]: [],
[SUITES.e2e.filesKey]: [],
});
prepareTestSelection("loki", env);
expect(console.warn).not.toHaveBeenCalled();
expect(output()).toBe("selection=full\n");
});
it("rejects an unknown suite", () => {
const result = spawnSync("bun", [SCRIPT, "toString"], {
env,
encoding: "utf8",
});
expect(result.status).not.toBe(0);
expect(result.stderr).toContain('Unknown test suite "toString"');
});
});
describe("test selection workflow steps", () => {
const ACTION = "./.github/actions/prepare-test-selection";
const CONSUMERS: Record<string, { suite: keyof typeof SUITES }> = {
"frontend.yml": { suite: "unit" },
"loki.yml": { suite: "loki" },
};
it("skips Loki visual tests only for an empty story selection", () => {
const { jobs } = loadWorkflow("loki.yml");
expect(jobs["story-selection"].outputs?.selection).toBe(
"${{ steps.loki-selection.outputs.selection }}",
);
expect(jobs["visual-test"].if).toContain(
"needs.story-selection.outputs.selection != 'empty'",
);
expect(jobs["visual-test"].if).not.toMatch(/outputs\.[\w-]+ *[!=]= *'\d+'/);
});
it.each(Object.entries(CONSUMERS))(
"selects tests through the composite action in %s",
(file, { suite, ...inputs }) => {
const steps = Object.values(loadWorkflow(file).jobs).flatMap(
({ steps = [] }) => steps.filter((step) => step.uses === ACTION),
);
expect(steps.length).toBeGreaterThan(0);
for (const step of steps) {
expect(step.with).toEqual({
suite,
"test-plan-artifact-id": expect.stringMatching(/^\$\{\{ .+ \}\}$/),
...inputs,
});
}
},
);
it("downloads the test plan before preparing the selection", () => {
const { outputs, runs } = loadAction();
const [download, prepare] = runs.steps;
expect(download).toMatchObject({
uses: expect.stringMatching(/^actions\/download-artifact@/),
"continue-on-error": "${{ inputs.require-download != 'true' }}",
with: {
"artifact-ids": "${{ inputs.test-plan-artifact-id }}",
path: "${{ runner.temp }}/test-plan",
},
});
expect(prepare).toMatchObject({
env: { PLAN_DOWNLOADED: `\${{ steps.${download.id}.outcome }}` },
run: expect.stringContaining(
'prepare-test-selection.ts "${{ inputs.suite }}"',
),
});
expect(outputs).toMatchObject({
"paths-file": {
value: `\${{ steps.${prepare.id}.outputs.paths-file }}`,
},
selection: { value: `\${{ steps.${prepare.id}.outputs.selection }}` },
});
});
});

226
.github/scripts/story-files.unit.spec.ts vendored Normal file
View File

@@ -0,0 +1,226 @@
import { execFileSync } from "node:child_process";
import { mkdtempSync, readFileSync, rmSync, writeFileSync } from "node:fs";
import { tmpdir } from "node:os";
import { join, resolve } from "node:path";
import micromatch, { isMatch } from "micromatch";
import { normalizeStories } from "storybook/internal/common";
import {
MAIN_APP_STORY_GLOBS,
getStories,
hasExplicitTitle,
} from "../../.storybook/story-files.cjs";
const ROOT = resolve(__dirname, "../..");
const tracked = execFileSync(
"git",
["ls-files", "-z", "--", "frontend", "enterprise/frontend"],
{ cwd: ROOT, encoding: "utf8", maxBuffer: 64 * 1024 * 1024 },
)
.split("\0")
.filter(Boolean);
const planned = micromatch(tracked, MAIN_APP_STORY_GLOBS, { dot: true }).sort();
describe("getStories", () => {
let tempDir: string;
let pathsFile: string;
beforeEach(() => {
tempDir = mkdtempSync(join(tmpdir(), "story-files-"));
pathsFile = join(tempDir, "stories.json");
});
afterEach(() => {
rmSync(tempDir, { recursive: true, force: true });
});
it("loads the main app stories and docs when there is no plan", () => {
const patterns = getStories({ pathsFile: "", filter: "" });
expect(
isMatch("../frontend/src/metabase/ui/Button.stories.tsx", patterns),
).toBe(true);
expect(isMatch("../frontend/src/metabase/ui/Button.mdx", patterns)).toBe(
true,
);
expect(
isMatch(
"../frontend/src/embedding-sdk-shared/Button.stories.tsx",
patterns,
),
).toBe(false);
});
it("builds only the files in the JSON plan", () => {
const files = planned.slice(0, 2);
writeFileSync(pathsFile, JSON.stringify(files));
expect(getStories({ pathsFile })).toEqual(
files.map((file) => `../${file}`),
);
});
it("builds a planned story with an explicit title", () => {
const story = join(tempDir, "Titled.stories.tsx");
writeFileSync(
story,
'const meta = { title: "Components/Titled" };\nexport default meta;\n',
);
writeFileSync(pathsFile, JSON.stringify([story]));
expect(getStories({ pathsFile })).toEqual([`../${story}`]);
});
it("rejects a planned story without an explicit title", () => {
const story = join(tempDir, "Untitled.stories.tsx");
writeFileSync(story, "export default { component: Untitled };\n");
writeFileSync(pathsFile, JSON.stringify([story]));
expect(() => getStories({ pathsFile })).toThrow(story);
});
it("rejects an untitled Alert story even though its args have a title", () => {
const source = readFileSync(
join(
ROOT,
"frontend/src/metabase/ui/components/feedback/Alert/Alert.stories.tsx",
),
"utf8",
).replace(' title: "Components/Feedback/Alert",\n', "");
const story = join(tempDir, "Alert.stories.tsx");
writeFileSync(story, source);
writeFileSync(pathsFile, JSON.stringify([story]));
expect(() => getStories({ pathsFile })).toThrow(story);
});
it("rejects an empty plan even when the CSV filter is set", () => {
writeFileSync(pathsFile, "[]");
expect(() =>
getStories({ pathsFile, filter: "frontend/src/Unselected.stories.tsx" }),
).toThrow("No stories selected");
});
it("preserves the stress test's CSV filter", () => {
expect(
getStories({
filter: "frontend/One.stories.tsx,frontend/Two.stories.tsx",
}),
).toEqual(["../frontend/One.stories.tsx", "../frontend/Two.stories.tsx"]);
});
it.each(["{", "null", '""', "{}", "[null]"])(
"rejects an invalid plan: %s",
(contents) => {
writeFileSync(pathsFile, contents);
expect(() => getStories({ pathsFile })).toThrow();
},
);
});
describe("hasExplicitTitle", () => {
it.each([
{
form: "double quotes",
source: 'export default {\n title: "Components/Button",\n};\n',
},
{
form: "single quotes",
source: "export default {\n title: 'Components/Button',\n};\n",
},
{
form: "a title constant",
source: 'const title = "Components/Button"; export default { title };',
},
{
form: "a meta constant",
source:
'const meta: Meta<typeof Button> = {\n title: "Components/Button",\n};\n\nexport default meta;\n',
},
{
form: "a satisfies expression",
source:
'const meta = { title: "Components/Button" } satisfies Meta<typeof Button>; export default meta;',
},
])("accepts a title in $form", ({ source }) => {
expect(hasExplicitTitle(source)).toBe(true);
});
it.each([
{
form: "no title",
source: "export default {\n component: Button,\n};\n",
},
{
form: "only a title control",
source:
'const argTypes = {\n title: {\n control: { type: "text" },\n },\n};\n\nexport default { component: Button, argTypes };\n',
},
{
form: "only a title variable",
source:
"export default {\n component: Button,\n args: {\n title: defaultTitle,\n },\n};\n",
},
{
form: "only a string-valued args title",
source: 'export default { args: { title: "Button label" } };',
},
{
form: "only another object's title",
source:
'const other = { title: "Other" }; export default { component: Button };',
},
{
form: "only a commented title",
source:
'export default {\n // title: "Components/Button",\n component: Button };',
},
{
form: "an empty title",
source: 'export default { title: "" };',
},
{
form: "a template literal unsupported by Storybook",
source: "export default { title: `Components/Button` };",
},
])("rejects $form", ({ source }) => {
expect(hasExplicitTitle(source)).toBe(false);
});
});
describe("MAIN_APP_STORY_GLOBS", () => {
it("matches the same tracked files as Storybook's own story matcher", () => {
const specifiers = normalizeStories(getStories(), {
configDir: join(ROOT, ".storybook"),
workingDir: ROOT,
});
const built = tracked
.filter(
(file) =>
!file.endsWith(".mdx") &&
specifiers.some((specifier) =>
specifier.importPathMatcher.test(`./${file}`),
),
)
.sort();
expect(planned.length).toBeGreaterThan(0);
expect(planned).toEqual(built);
});
it("matches no docs entries", () => {
expect(planned.filter((file) => file.endsWith(".mdx"))).toEqual([]);
});
it("matches only stories with an explicit title", () => {
const untitled = planned.filter(
(file) => !hasExplicitTitle(readFileSync(join(ROOT, file), "utf8")),
);
expect(untitled).toEqual([]);
});
});

View File

@@ -23,9 +23,13 @@ interface Condition {
coldLargestPaintMs: number;
coldPageReadyMs: number;
warmPageReadyMs: number;
warmLargestPaintMs: number;
steadyLargestPaintMs: number;
steadyPageReadyMs: number;
locale: string;
scripts: number;
scriptKb: number;
cssKb: number;
totalKb: number;
runs: number;
}
@@ -69,9 +73,13 @@ function buildRows(
"Cold largest paint ms": condition.coldLargestPaintMs,
"Cold page ready ms": condition.coldPageReadyMs,
"Warm page ready ms": condition.warmPageReadyMs,
"Warm largest paint ms": condition.warmLargestPaintMs,
"Steady largest paint ms": condition.steadyLargestPaintMs,
"Steady page ready ms": condition.steadyPageReadyMs,
Locale: condition.locale,
Scripts: condition.scripts,
"Script kb": condition.scriptKb,
"Css kb": condition.cssKb,
"Total kb": condition.totalKb,
Runs: condition.runs,
}));

View File

@@ -0,0 +1,73 @@
/**
* Collect some stats about our e2e tests:
* - total number of spec files
* - total time to run all specs
* - average spec runtime
* - max spec runtime
*
*/
import { readFileSync } from "node:fs";
import { importStats, StatsRow } from "./stats-import";
const TABLE = "e2e_test_timings";
export interface Timings {
durations?: { spec?: string; duration?: unknown }[];
}
export const buildTestTimingsRow = (
timestamp: string,
timings: Timings,
): StatsRow => {
const durations = timings.durations ?? [];
const specCount = durations.length;
if (specCount === 0) {
throw new Error("No durations found.");
}
let totalRunTime = 0;
let maxSpecRuntime = 0;
for (const { duration } of durations) {
if (typeof duration !== "number") {
continue;
}
totalRunTime += duration;
maxSpecRuntime = duration > maxSpecRuntime ? duration : maxSpecRuntime;
}
const averageSpecRuntime = Math.round(totalRunTime / specCount);
return {
date: timestamp,
spec_count: specCount,
total_runtime: totalRunTime,
avg_spec_runtime: averageSpecRuntime,
max_spec_runtime: maxSpecRuntime,
};
};
export async function main() {
const path = process.env.TIMINGS_PATH || "e2e/support/timings.json";
try {
const timings = JSON.parse(readFileSync(path, "utf-8")) as Timings;
const row = buildTestTimingsRow(new Date().toISOString(), timings);
console.log(row);
await importStats({ table: TABLE, rows: [row] });
console.log(`Successfully uploaded e2e test timings to ${TABLE}.`);
} catch (error) {
const message = error instanceof Error ? error.message : String(error);
console.log(
`::warning::Uploading e2e test timings from ${path} failed; leaving run green: ${message}`,
);
}
}
if ((import.meta as ImportMeta & { main?: boolean }).main) {
main();
}

View File

@@ -0,0 +1,61 @@
import { buildTestTimingsRow } from "./upload-e2e-test-timings";
const timestamp = "2026-06-30T00:00:00.000Z";
describe("buildTestTimingsRow", () => {
it("summarizes spec durations", () => {
const row = buildTestTimingsRow(timestamp, {
durations: [
{ spec: "a.cy.spec.js", duration: 1000 },
{ spec: "b.cy.spec.js", duration: 3000 },
{ spec: "c.cy.spec.js", duration: 2000 },
],
});
expect(row).toEqual({
date: timestamp,
spec_count: 3,
total_runtime: 6000,
avg_spec_runtime: 2000,
max_spec_runtime: 3000,
});
});
it("rounds the average runtime", () => {
const row = buildTestTimingsRow(timestamp, {
durations: [
{ spec: "a.cy.spec.js", duration: 1000 },
{ spec: "b.cy.spec.js", duration: 1001 },
],
});
expect(row.avg_spec_runtime).toBe(1001);
});
it("counts specs with a non-numeric duration but ignores their value", () => {
const row = buildTestTimingsRow(timestamp, {
durations: [
{ spec: "a.cy.spec.js", duration: 1000 },
{ spec: "b.cy.spec.js", duration: null },
{ spec: "c.cy.spec.js" },
],
});
expect(row).toEqual({
date: timestamp,
spec_count: 3,
total_runtime: 1000,
avg_spec_runtime: 333,
max_spec_runtime: 1000,
});
});
it("throws when there are no durations", () => {
expect(() => buildTestTimingsRow(timestamp, { durations: [] })).toThrow(
"No durations found.",
);
expect(() => buildTestTimingsRow(timestamp, {})).toThrow(
"No durations found.",
);
});
});

View File

@@ -93,9 +93,9 @@ jobs:
],
mysql: [
{
name: 'MySQL 8.0',
image: 'mysql:8.0',
'junit-name': 'be-tests-mysql-8-0-ee',
name: 'MySQL 8',
image: 'mysql:8.4',
'junit-name': 'be-tests-mysql-8-ee',
env: { 'enable-ssl-tests': 'false', 'enable-aws-iam-tests': 'false' },
latest: false
},

View File

@@ -450,9 +450,9 @@ jobs:
env:
enable-ssl-tests: 'false'
enable-aws-iam-tests: 'false'
- name: MySQL 8.0
junit-name: be-tests-mysql-8-0-ee
image: mysql:8.0
- name: MySQL 8
junit-name: be-tests-mysql-8-ee
image: mysql:8.4
env:
enable-ssl-tests: 'false'
enable-aws-iam-tests: 'false'

View File

@@ -12,6 +12,13 @@ on:
skip-custom-viz:
type: boolean
default: false
# ID of this run's test-plan.json artifact, or '' to run the full suite.
test-plan-artifact-id:
type: string
default: ""
skip-ci-scripts:
type: boolean
default: false
workflow_dispatch:
inputs:
skip:
@@ -100,6 +107,7 @@ jobs:
SHARDS: 2 # should match matrix above
JEST_JUNIT_OUTPUT_DIR: ./target/junit
JEST_JUNIT_OUTPUT_NAME: test-report-frontend-unit.xml
UNIT_TEST_RESULTS_FILE: unit-test-results.json
steps:
- uses: actions/checkout@v6
- name: Prepare front-end environment
@@ -110,11 +118,29 @@ jobs:
# Resolve JOB_ID before the long test step so the ci-conductor reporter
# below can link failures to this exact job. Best-effort.
uses: ./.github/actions/resolve-job-id
- name: Prepare unit test selection
id: unit-selection
if: ${{ inputs.test-plan-artifact-id != '' }}
uses: ./.github/actions/prepare-test-selection
with:
suite: unit
test-plan-artifact-id: ${{ inputs.test-plan-artifact-id }}
- name: Run frontend unit tests
id: fe-tests
run: bun run test-unit --silent --shard=${{ matrix.shard }}/${{ env.SHARDS }}
# Keep this step continue-on-error because the quarantine logic controls whether it ultimately fails.
env:
JEST_TEST_PATHS_FILE: ${{ steps.unit-selection.outputs.paths-file }}
# Jest exits non-zero on a shard with no files unless --passWithNoTests is set.
run: |
rm -f "$UNIT_TEST_RESULTS_FILE" "$JEST_JUNIT_OUTPUT_DIR/$JEST_JUNIT_OUTPUT_NAME"
bun run test-unit --silent --shard=${{ matrix.shard }}/${{ env.SHARDS }} --json --outputFile="$UNIT_TEST_RESULTS_FILE" ${{ steps.unit-selection.outputs.paths-file && '--passWithNoTests' || '' }}
# Ordinary assertion failures can be quarantined, but runner failures cannot.
continue-on-error: true
- name: Check unit test results
if: ${{ !cancelled() && (steps.fe-tests.outcome == 'failure' || steps.unit-selection.outputs.paths-file) }}
env:
TEST_OUTCOME: ${{ steps.fe-tests.outcome }}
JEST_TEST_PATHS_FILE: ${{ steps.unit-selection.outputs.paths-file }}
run: bun .github/scripts/check-unit-test-results.ts
- name: Report failed tests to ci-conductor
if: ${{ !cancelled() && ( steps.fe-tests.outcome == 'failure' || github.run_attempt != '1' ) }}
# This step should NEVER fail the job!
@@ -144,6 +170,18 @@ jobs:
- name: Run frontend timezones tests
run: bun run test-timezones
# No CLJS compile: the CI-script specs don't import app code.
fe-tests-ci-scripts:
if: ${{ !inputs.skip-ci-scripts }}
runs-on: ${{ vars.DEFAULT_RUNNER_KEY }}
timeout-minutes: 10
steps:
- uses: actions/checkout@v6
- name: Prepare front-end environment
uses: ./.github/actions/prepare-frontend
- name: Run CI script tests
run: bun run test-ci-scripts
# The custom-viz package is an isolated Bun workspace under
# enterprise/frontend/src/custom-viz
fe-custom-viz:
@@ -174,6 +212,7 @@ jobs:
- fe-type-check
- fe-tests-unit
- fe-tests-timezones
- fe-tests-ci-scripts
- fe-custom-viz
if: always() && !cancelled()
runs-on: ${{ vars.SLIM_RUNNER_KEY }}

View File

@@ -4,16 +4,52 @@ on:
workflow_call:
inputs:
skip:
description: "Skip the Loki visual tests."
type: boolean
default: false
test-plan-artifact-id:
description: "ID of this run's test-plan.json artifact, or '' to run the full suite."
type: string
default: ""
concurrency:
group: ${{ github.workflow }}-${{ github.head_ref && github.ref || github.run_id }}-loki
cancel-in-progress: true
jobs:
story-selection:
if: ${{ !inputs.skip && inputs.test-plan-artifact-id != '' }}
runs-on: ${{ vars.SLIM_RUNNER_KEY }}
timeout-minutes: 5
outputs:
selection: ${{ steps.loki-selection.outputs.selection }}
test-plan-artifact-id: ${{ steps.loki-selection.outputs.paths-file && inputs.test-plan-artifact-id || '' }}
steps:
- name: Checkout Repository
uses: actions/checkout@v6
with:
sparse-checkout: |
.github
/.nvmrc
/package.json
sparse-checkout-cone-mode: false
- name: Prepare Node + Bun environment
uses: ./.github/actions/prepare-node-bun
- name: Prepare Loki story selection
id: loki-selection
uses: ./.github/actions/prepare-test-selection
with:
suite: loki
test-plan-artifact-id: ${{ inputs.test-plan-artifact-id }}
visual-test:
if: ${{ !inputs.skip }}
name: loki-run
needs: story-selection
# A missing job output compares equal to '0' in GitHub expressions,
# so an empty selection is marked with a non-numeric value.
if: ${{ !cancelled() && !inputs.skip && needs.story-selection.outputs.selection != 'empty' }}
runs-on: ${{ vars.DEFAULT_RUNNER_KEY }}
timeout-minutes: 30
services:
@@ -41,8 +77,22 @@ jobs:
- name: Compile CLJS
run: NODE_ENV=development bun run build-pure:cljs
- name: Prepare Loki story selection
id: loki-selection
if: ${{ needs.story-selection.outputs.test-plan-artifact-id != '' }}
uses: ./.github/actions/prepare-test-selection
with:
suite: loki
test-plan-artifact-id: ${{ needs.story-selection.outputs.test-plan-artifact-id }}
- name: Build Storybook
env:
STORYBOOK_STORY_PATHS_FILE: ${{ steps.loki-selection.outputs.paths-file }}
run: bun run build-storybook
- name: Run Loki Visual Tests
run: bun run test-visual:loki:ci
# Loki drops stories with parameters.loki.skip, and exits non-zero when that leaves the selection empty.
run: bun run test-visual:loki --reactUri file:./storybook-static --verboseRenderer ${{ steps.loki-selection.outputs.paths-file && '--passWithNoStories --requireReference' || '' }}
- name: Generate Visual Report on Failure
if: failure()
@@ -57,3 +107,38 @@ jobs:
include-hidden-files: true
path: .loki/
if-no-files-found: ignore
loki-tests-result:
name: visual-test
needs: [story-selection, visual-test]
if: always() && !cancelled()
runs-on: ${{ vars.SLIM_RUNNER_KEY }}
timeout-minutes: 5
steps:
- name: Check Loki job status
uses: actions/github-script@v9
env:
needs: ${{ toJson(needs) }}
with:
script: | # js
const needs = JSON.parse(process.env.needs);
const jobs = Object.entries(needs).map(
([jobName, jobValues]) => ({
name: jobName,
result: jobValues.result
}));
// are all jobs skipped or successful?
if (jobs.every(job => (job.result === 'skipped' || job.result === 'success'))) {
console.log('All jobs are skipped or successful');
process.exit(0);
}
// otherwise, something failed
console.log('Some Loki jobs failed');
jobs.forEach((job) => {
if (job.result !== 'success') {
console.log(`${job.name} - ${job.result}`);
}
});
process.exit(1);

View File

@@ -56,6 +56,16 @@ jobs:
core.setOutput('artifact-count', result.artifactCount);
- uses: ./.github/actions/prepare-node-bun
- name: Upload e2e test timings to Stats
if: ${{ github.repository == 'metabase/metabase' }}
env:
TIMINGS_PATH: e2e/support/timings.json
ENG_STATS_URL: ${{ secrets.ENG_STATS_URL }}
API_KEY: ${{ secrets.ENG_STATS_API_KEY }}
run: bun .github/scripts/upload-e2e-test-timings.ts
- name: Configure git and commit changes
id: commit
run: |
@@ -110,7 +120,7 @@ jobs:
--body "$BODY" \
--label ".CI & Tests" \
--label "no-backport" \
--reviewer metabase/devexp)
--reviewer metabase/devex)
PR_NUMBER=${URL##*/}

View File

@@ -57,11 +57,10 @@ jobs:
build_scripts: ${{ steps.changes.outputs.build_scripts }}
embedder_plugin: ${{ steps.changes.outputs.embedder_plugin }}
frontend_all: ${{ steps.changes.outputs.frontend_all }}
frontend_ci: ${{ steps.changes.outputs.frontend_ci }}
frontend_sources: ${{ steps.changes.outputs.frontend_sources }}
frontend_unit_infra: ${{ steps.changes.outputs.frontend_unit_infra }}
frontend_loki_ci: ${{ steps.changes.outputs.frontend_loki_ci }}
frontend_loki_infra: ${{ steps.changes.outputs.frontend_loki_infra }}
frontend_loki_all: ${{ steps.changes.outputs.frontend_loki_all }}
shared_sources: ${{ steps.changes.outputs.shared_sources }}
documentation: ${{ steps.changes.outputs.documentation }}
embedding_documentation: ${{ steps.changes.outputs.embedding_documentation }}
@@ -69,6 +68,7 @@ jobs:
embedding_sdk_host_sample_apps: ${{ steps.changes.outputs.embedding_sdk_host_sample_apps }}
embedding_sdk_ci: ${{ steps.changes.outputs.embedding_sdk_ci }}
custom_viz_all: ${{ steps.changes.outputs.custom_viz_all }}
ci_scripts: ${{ steps.changes.outputs.ci_scripts }}
project_backend_checks: ${{ steps.changes.outputs.project_backend_checks }}
project_migration_checks: ${{ steps.changes.outputs.project_migration_checks }}
project_ratchet_checks: ${{ steps.changes.outputs.project_ratchet_checks }}
@@ -97,16 +97,14 @@ jobs:
filters: .github/file-paths.yaml
create-test-plan:
needs: files-changed
needs: [should-run, files-changed]
name: Create test plan
if: ${{ github.event_name == 'pull_request' }}
if: ${{ github.event_name == 'pull_request' && needs.should-run.outputs.verdict == 'defer' }}
runs-on: ${{ vars.DEFAULT_RUNNER_KEY }}
timeout-minutes: 10
outputs:
stats: ${{ steps.compute.outputs.stats }}
fe_unit_specs_to_run: ${{ steps.compute.outputs.fe_unit_specs_to_run }}
loki_stories_to_run: ${{ steps.compute.outputs.loki_stories_to_run }}
e2e_specs_to_run: ${{ steps.compute.outputs.e2e_specs_to_run }}
artifact-id: ${{ steps.compute.outputs.artifact-id }}
steps:
- uses: actions/checkout@v6
- name: Create test plan
@@ -262,6 +260,10 @@ jobs:
skip: >-
${{ needs.should-run.outputs.verdict != 'force-run' &&
needs.files-changed.outputs.backend_all != 'true' }}
smoke-only: >-
${{ needs.should-run.outputs.verdict != 'force-run' &&
!contains(github.event.pull_request.labels.*.name, 'ci:run-semantic-search-tests')
}}
app-db-tests:
if: |
@@ -292,7 +294,7 @@ jobs:
force-run: ${{ needs.should-run.outputs.verdict == 'force-run' }}
frontend-tests:
needs: [should-run, files-changed]
needs: [should-run, files-changed, create-test-plan]
if: |
!cancelled() &&
(needs.should-run.outputs.verdict == 'force-run' ||
@@ -303,6 +305,8 @@ jobs:
skip: >-
${{ needs.should-run.outputs.verdict != 'force-run' &&
needs.files-changed.outputs.frontend_all != 'true' }}
# A skipped/failed planner supplies no artifact, so the suite runs in full.
test-plan-artifact-id: ${{ needs.create-test-plan.result == 'success' && needs.create-test-plan.outputs.artifact-id || '' }}
skip-lint: >-
${{ needs.should-run.outputs.verdict != 'force-run' &&
needs.files-changed.outputs.frontend_all != 'true' &&
@@ -310,9 +314,12 @@ jobs:
skip-custom-viz: >-
${{ needs.should-run.outputs.verdict != 'force-run' &&
needs.files-changed.outputs.custom_viz_all != 'true' }}
skip-ci-scripts: >-
${{ needs.should-run.outputs.verdict != 'force-run' &&
needs.files-changed.outputs.ci_scripts != 'true' }}
loki-tests:
needs: [should-run, files-changed]
needs: [should-run, files-changed, create-test-plan]
if: |
!cancelled() &&
(needs.should-run.outputs.verdict == 'force-run' ||
@@ -321,9 +328,9 @@ jobs:
with:
skip: >-
${{ needs.should-run.outputs.verdict != 'force-run' &&
needs.files-changed.outputs.frontend_ci != 'true' &&
needs.files-changed.outputs.frontend_sources != 'true' &&
needs.files-changed.outputs.frontend_loki_ci != 'true' }}
needs.files-changed.outputs.frontend_loki_all != 'true' }}
# A skipped/failed planner supplies no artifact, so the suite runs in full.
test-plan-artifact-id: ${{ needs.create-test-plan.result == 'success' && needs.create-test-plan.outputs.artifact-id || '' }}
e2e-tests:
needs: [should-run, files-changed, uberjar]

View File

@@ -6,6 +6,10 @@ on:
skip:
type: boolean
default: false
smoke-only:
type: boolean
default: false
description: Only run Postgres Latest to do a quick resource-efficient sanity check, intended to be set for PRs.
# Run one semantic-search job on demand, against the ref you dispatch on, optionally under a
# profiler. Everything else about the job -- services, matrix legs, test args, quarantine gating --
@@ -54,7 +58,7 @@ env:
jobs:
semantic-search-tests-h2:
if: ${{ !inputs.skip && (inputs.suite == '' || inputs.suite == 'h2') }}
if: ${{ !inputs.skip && !inputs.smoke-only && (inputs.suite == '' || inputs.suite == 'h2') }}
runs-on: ${{ vars.DEFAULT_RUNNER_KEY }}
timeout-minutes: 40
strategy:
@@ -98,7 +102,7 @@ jobs:
:exclude-tags '[${{ matrix.job.exclude-tag }}]'
semantic-search-tests-mariadb:
if: ${{ !inputs.skip && (inputs.suite == '' || inputs.suite == 'mariadb') }}
if: ${{ !inputs.skip && !inputs.smoke-only && (inputs.suite == '' || inputs.suite == 'mariadb') }}
runs-on: ${{ vars.DEFAULT_RUNNER_KEY }}
timeout-minutes: 40
strategy:
@@ -163,7 +167,7 @@ jobs:
:exclude-tags '[${{ matrix.job.exclude-tag }}]'
semantic-search-tests-mysql:
if: ${{ !inputs.skip && (inputs.suite == '' || inputs.suite == 'mysql') }}
if: ${{ !inputs.skip && !inputs.smoke-only && (inputs.suite == '' || inputs.suite == 'mysql') }}
runs-on: ${{ vars.DEFAULT_RUNNER_KEY }}
timeout-minutes: 40
strategy:
@@ -250,6 +254,10 @@ jobs:
junit-name: semantic-search-tests-postgres-latest
postgres-image: postgres:latest
pgvector-image: pgvector/pgvector:pg17
exclude: >-
${{ inputs.smoke-only
&& fromJSON('[{"version":{"name":"Postgres 14.x with pgvector pg17"}}]')
|| fromJSON('[]') }}
job:
- name: Semantic Search Tests
build-static-viz: false
@@ -302,7 +310,7 @@ jobs:
# MB_PGVECTOR_DB_URL is deliberately unset. The dedicated-harness tests gate themselves off; the
# appdb-pgvector-mode round trip (and the URL-independent unit tests) run for real.
semantic-search-tests-appdb-mode:
if: ${{ !inputs.skip && (inputs.suite == '' || inputs.suite == 'appdb-mode') }}
if: ${{ !inputs.skip && !inputs.smoke-only && (inputs.suite == '' || inputs.suite == 'appdb-mode') }}
runs-on: ${{ vars.DEFAULT_RUNNER_KEY }}
timeout-minutes: 40
strategy:

View File

@@ -1,8 +1,8 @@
name: Bundle Load Stats
# Reuses the uberjar built on every master merge: boots it, signs in, loads the
# app in a throttled headless Chrome across four network and CPU conditions, and
# appends the times to eng-stats-importer.
# app in a throttled headless Chrome across four network and CPU conditions and
# two site locales, and appends the times to eng-stats-importer.
#
# The instance is blank on purpose. `index.html` is built from settings and the
# signed-in user alone, so seeding content would not change a single byte of the
@@ -59,7 +59,7 @@ jobs:
needs: [uberjar-built]
if: ${{ needs.uberjar-built.outputs.found == 'true' }}
runs-on: ${{ vars.DEFAULT_RUNNER_KEY }}
timeout-minutes: 10
timeout-minutes: 20
env:
HEAD_SHA: ${{ inputs.commit || github.event.workflow_run.head_sha }}
MB_JETTY_PORT: "4000"
@@ -67,6 +67,9 @@ jobs:
# Loads per condition. The first is the cold reading and the rest give the
# median, so this trades run time for a steadier number.
RUNS: "8"
# Measured in this order, so the translated reading comes from the same
# state the instance is set up in.
LOCALES: "de en"
steps:
- name: Check out the code
uses: actions/checkout@v6
@@ -114,19 +117,36 @@ jobs:
env:
CHROME_PATH: ${{ steps.setup-chrome.outputs.chrome-path }}
run: |
bun frontend/test/bench/matrix.ts "$SITE/" "$RUNS" \
| tee artifacts/load-times.json
# The site locale decides which translation catalogue the document
# carries, and a user with no locale of their own falls back to it.
# Measure both, so every commit has a translated reading and an
# English one, and the difference is what translation costs.
for locale in $LOCALES; do
curl -sS -X PUT -o /dev/null \
-H "X-Metabase-Session: ${SESSION_COOKIE}" \
-H "content-type: application/json" \
-d "{\"value\":\"${locale}\"}" \
"$SITE/api/setting/site-locale"
# One request to absorb the settings cache miss the switch causes.
curl -sS -o /dev/null "$SITE/"
bun frontend/test/bench/matrix.ts "$SITE/" "$RUNS" \
| tee "artifacts/load-times-${locale}.json"
done
- name: Show the Metabase log if something went wrong
if: failure()
run: tail -100 artifacts/metabase.log || true
- name: Import load times into Stats
if: ${{ inputs.commit == '' }}
env:
ROWS: artifacts/load-times.json
# Passed via env (not interpolated into the shell) so the commit
# message can't inject commands; the script keeps only the subject.
COMMIT_MESSAGE: ${{ github.event.workflow_run.head_commit.message }}
COMMIT_TIMESTAMP: ${{ github.event.workflow_run.head_commit.timestamp }}
ENG_STATS_URL: ${{ secrets.ENG_STATS_URL }}
API_KEY: ${{ secrets.ENG_STATS_API_KEY }}
run: bun .github/scripts/upload-bundle-load-stats.ts
run: |
# Each file carries its own `Locale`, read from the served document.
for locale in $LOCALES; do
ROWS="artifacts/load-times-${locale}.json" \
bun .github/scripts/upload-bundle-load-stats.ts
done

Binary file not shown.

Before

Width:  |  Height:  |  Size: 22 KiB

After

Width:  |  Height:  |  Size: 22 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 27 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 13 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 9.8 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 14 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 16 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 18 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 36 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 31 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 26 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 24 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 25 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 24 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 18 KiB

View File

@@ -12,21 +12,16 @@ const {
} = require("../frontend/build/shared/rspack/side-effect-free-modules");
const appConfig = require("../rspack.main.config.js");
const mainAppStories = [
"../frontend/**/*.mdx",
"../frontend/*/!(embedding-sdk-bundle|embedding-sdk-shared)/**/*.stories.@(ts|tsx)",
"../enterprise/frontend/*/!(embedding-sdk-ee|embedding-sdk-package)/**/*.stories.@(ts|tsx)",
];
// Allow filtering to specific story files via env var (used by stress tests)
// STORYBOOK_STORIES_FILTER: comma-separated file paths relative to repo root
// e.g. "frontend/src/.../Button.stories.tsx,frontend/src/.../Alert.stories.tsx"
const stories = process.env.STORYBOOK_STORIES_FILTER
? process.env.STORYBOOK_STORIES_FILTER.split(",").map((f) => `../${f}`)
: mainAppStories;
const { getStories } = require("./story-files.cjs");
const config: StorybookConfig = {
stories,
stories: getStories({
pathsFile: process.env.STORYBOOK_STORY_PATHS_FILE,
// Allow filtering to specific story files via env var (used by stress tests)
// STORYBOOK_STORIES_FILTER: comma-separated file paths relative to repo root
// e.g. "frontend/src/.../Button.stories.tsx,frontend/src/.../Alert.stories.tsx"
filter: process.env.STORYBOOK_STORIES_FILTER,
}),
staticDirs: [
"../resources/frontend_client",
"./msw-public",

View File

@@ -0,0 +1,54 @@
const { readFileSync } = require("node:fs");
const { resolve } = require("node:path");
const ROOT = resolve(__dirname, "..");
const MAIN_APP_STORY_GLOBS = [
"frontend/*/!(embedding-sdk-bundle|embedding-sdk-shared)/**/*.stories.@(ts|tsx)",
"enterprise/frontend/*/!(embedding-sdk-ee|embedding-sdk-package)/**/*.stories.@(ts|tsx)",
];
// Storybook autotitles a story from its path relative to the entry's directory,
// so an untitled story gets a different title in a narrowed build and Loki writes a fresh reference for it.
function hasExplicitTitle(source) {
// Only load the parser when validating a build; the planner also imports this
// module for its story globs.
const { loadCsf } = require("storybook/internal/csf-tools");
try {
const { meta } = loadCsf(source, { makeTitle: (title) => title }).parse();
return typeof meta?.title === "string" && meta.title.length > 0;
} catch {
return false;
}
}
function getStories({ pathsFile, filter } = {}) {
if (pathsFile) {
const files = JSON.parse(readFileSync(pathsFile, "utf8"));
if (
!Array.isArray(files) ||
!files.every((file) => typeof file === "string")
) {
throw new Error("Story paths must be a JSON array of strings");
}
if (files.length === 0) {
throw new Error("No stories selected");
}
const untitled = files.filter(
(file) => !hasExplicitTitle(readFileSync(resolve(ROOT, file), "utf8")),
);
if (untitled.length > 0) {
throw new Error(
`Stories without an explicit title: ${untitled.join(", ")}`,
);
}
return files.map((file) => `../${file}`);
}
const files = filter
? filter.split(",")
: ["frontend/**/*.mdx", ...MAIN_APP_STORY_GLOBS];
return files.map((file) => `../${file}`);
}
module.exports = { MAIN_APP_STORY_GLOBS, getStories, hasExplicitTitle };

View File

@@ -98,6 +98,9 @@ interactive explorer with a dependency graph.
target's `:api`.
- Each `:module-exports` entry widens a nested module's visibility by one ancestor. Export every link to
make it available everywhere. OSS module `X` exports its `enterprise/X` companion automatically.
- A `module/X` module is a plugin built from `modules/X` and shipped as its own jar, with namespaces under
`metabase-module.X`. It may be missing at runtime, so only its own subtree may use it, and `:uses :any` does not
reach it; the rest of the backend loads it through `metabase.plugins`.
## Ratchets

View File

@@ -364,7 +364,7 @@
"webpack-notifier": "1.15.0",
"webpack-stats-plugin": "^1.1.3",
"yaml-lint": "~1.6.0",
"zrender": "^6.0.0",
"zrender": "6.1.0",
},
},
},
@@ -5353,7 +5353,7 @@
"zod-to-json-schema": ["zod-to-json-schema@3.25.2", "", { "peerDependencies": { "zod": "^3.25.28 || ^4" } }, "sha512-O/PgfnpT1xKSDeQYSCfRI5Gy3hPf91mKVDuYLUHZJMiDFptvP41MSnWofm8dnCm0256ZNfZIM7DSzuSMAFnjHA=="],
"zrender": ["zrender@6.0.0", "", { "dependencies": { "tslib": "2.3.0" } }, "sha512-41dFXEEXuJpNecuUQq6JlbybmnHaqqpGlbH1yxnA5V9MMP4SbohSVZsJIwz+zdjQXSSlR1Vc34EgH1zxyTDvhg=="],
"zrender": ["zrender@6.1.0", "", { "dependencies": { "tslib": "2.3.0" } }, "sha512-oEGMDB6pOP2S6OwRR4PdVv610zrjnA3Bh+JnSG12fYJlBKjtNAoEb5fSUoCOOINlH96I2fU38/A2UpRKs67xYQ=="],
"zustand": ["zustand@4.5.7", "", { "dependencies": { "use-sync-external-store": "^1.2.2" } }, "sha512-CHOUy7mu3lbD6o6LJLfllpjkzhHXSBlX8B9+qPddUsIfeF5S/UZ5q0kmCsnRqT1UHFQZchNFDDzMbQsuesHWlw=="],
@@ -5993,8 +5993,6 @@
"echarts/tslib": ["tslib@2.3.0", "", {}, "sha512-N82ooyxVNm6h1riLCoyS9e3fuJ3AMG2zIZs2Gd1ATcSFjSA23Q0fzjjZeh0jbJvWVDZ0cJT8yaNNaaXHzueNjg=="],
"echarts/zrender": ["zrender@6.1.0", "", { "dependencies": { "tslib": "2.3.0" } }, "sha512-oEGMDB6pOP2S6OwRR4PdVv610zrjnA3Bh+JnSG12fYJlBKjtNAoEb5fSUoCOOINlH96I2fU38/A2UpRKs67xYQ=="],
"elliptic/bn.js": ["bn.js@4.11.9", "", {}, "sha512-E6QoYqCKZfgatHTdHzs1RRKP7ip4vvm+EyRUeE2RF0NblwVvb0p6jSVeNTOFxPn26QXN2o6SMfNxKp6kU8zQaw=="],
"endent/dedent": ["dedent@0.7.0", "", {}, "sha1-JJXduvbrh0q7Dhvp3yLS5aVEMmw="],

View File

@@ -93,7 +93,7 @@
:exclusions [org.clojure/data.avl]}
medley/medley {:mvn/version "1.4.0"} ; lightweight lib of useful functions
metabase/connection-pool {:mvn/version "1.2.0"} ; simple wrapper around C3P0. JDBC connection pools
metabase/saml20-clj {:mvn/version "4.2.1"
metabase/saml20-clj {:mvn/version "4.3.0"
:exclusions [; EE SAML integration TODO: bump version when we release the library
org.bouncycastle/bcpkix-jdk15on
org.bouncycastle/bcprov-jdk15on
@@ -576,6 +576,7 @@
"bin/load-namespaces/test"
"bin/release-list/src"
"bin/release-list/test"
"modules/embedder/src"
"modules/drivers/athena/src"
"modules/drivers/athena/test"
"modules/drivers/bigquery-cloud-sdk/src"

View File

@@ -1,16 +1,30 @@
(ns dev.add-load
(:require [clojure.walk :as walk]
[dev.with-perm :as perm]
[metabase.util.malli :as mu]))
[metabase.util.malli :as mu]
[metabase.util.malli.registry :as mr]))
(def ^:private logical-kw
"A keyword that starts with a question mark, used to track values created by from-script."
[:and :keyword [:fn #(= "?" (namespace %))]])
(def ^:private Attrs
"The attributes of an entity to insert, as `mt/with-temp` takes them for its model."
[:map {:closed false, ::mr/deliberately-open true, :description "model attributes"}])
(def ^:private Bindings
"A map of logical keywords to the columns of an inserted entity they bind."
[:map {:closed false, ::mr/deliberately-open true, :description "logical keyword bindings"}])
(def ^:private Inserted
"An inserted entity of any model."
[:map {:closed false, ::mr/deliberately-open true, :description "inserted entity"}])
(mu/defn- extract-bindings
"Extracts the bindings from the script and inserts the values into the ids map. This is the data oriented equivalent
of let."
[bindings :- [:or :map logical-kw] inserted]
[bindings :- [:or Bindings logical-kw]
inserted :- Inserted]
(cond (map? bindings)
(update-vals bindings #(get inserted %))
(and (keyword? bindings) (= "?" (namespace bindings)))
@@ -18,14 +32,15 @@
(mu/defn- fill-attrs
"Data oriented value resolution used to add attributes to a model from the [[attrs]] map."
[ids :- :map attrs :- :map]
[ids :- Bindings
attrs :- Attrs]
(walk/postwalk
(fn [x] (if-let [value (get ids x)] value x))
attrs))
(def ^:private BindingForm [:or [:= :?] logical-kw :map])
(def ^:private BindingForm [:or [:= :?] logical-kw Bindings])
(def ^:private NormalizedTuple [:tuple :keyword BindingForm :map])
(def ^:private NormalizedTuple [:tuple :keyword BindingForm Attrs])
(def ^:private Tuple [:or
[:tuple :keyword]

View File

@@ -22,7 +22,7 @@
(mu/defn ->dbdef :- [:map [:database-name :string]]
"Coerce something to a Database Definition. Can resolve string symbols from the current namespace or
from [[metabase.test.data.dataset-definitions]]."
[dbdeffable]
[dbdeffable :- [:or :symbol tx/ValidDatabaseDefinition]]
(if (symbol? dbdeffable)
(->dbdef (data.impl/resolve-dataset-definition (symbol (ns-name *ns*)) dbdeffable))
(tx/get-dataset-definition dbdeffable)))
@@ -30,7 +30,8 @@
(mu/defn ->tabledef :- [:map [:table-name :string]]
"Coerce something to a Table Definition. You can pass in a string table name to find the matching table from a
Database Definition."
[dbdeffable tabledeffable]
[dbdeffable :- [:or :symbol tx/ValidDatabaseDefinition]
tabledeffable :- [:or :string tx/ValidTableDefinition]]
(cond
(and (map? tabledeffable)
(:table-name tabledeffable))

View File

@@ -22,6 +22,12 @@
;; mechanism for bounded caching of those parsed files.
(def ^:dynamic *parsed-file-cache* nil)
(def ^:private File
"A path string or `java.io.File`, as accepted by `rewrite-clj.parser/parse-file-all` and friends."
[:or
string?
[:fn {:error/message "Instance of a java.io.File"} #(instance? java.io.File %)]])
(defn- parse-file-all
"Calls `rewrite-.clj.parser/parse-file-all`, but first checks in `*parsed-file-cache*` if it is bound."
[file]
@@ -70,15 +76,19 @@
^java.io.File []
(io/file (str (.getAbsolutePath (project-root-directory)) "/enterprise/backend/src")))
(declare kondo-config)
(mu/defn- plugin-source-roots :- [:sequential (ms/InstanceOfClass java.io.File)]
"Source roots of the separately built plugin modules under `modules/`: every driver, plus the siblings that
ship as their own jar. They are off the main classpath, so nothing else here would find them."
"Source roots of the separately built plugins under `modules/`: every driver, plus `modules/X/src` for each
declared `module/X`. They are off the main classpath, so nothing else here would find them."
[]
(let [modules (io/file (str (.getAbsolutePath (project-root-directory)) "/modules"))]
(concat
(for [driver (.listFiles (io/file modules "drivers"))]
(io/file driver "src"))
[(io/file modules "embedder" "src")])))
(for [module (keys (kondo-config))
:when (modules/plugin-module? module)]
(io/file modules (name module) "src")))))
(mu/defn- find-source-files :- [:sequential (ms/InstanceOfClass java.io.File)]
[]
@@ -121,7 +131,7 @@
(mu/defn- find-dynamically-loaded-namespaces :- [:set simple-symbol?]
"Find the set of namespace symbols for namespaces loaded by `require` and friends in a `file`."
[file]
[file :- File]
(try
(find-required-namespaces file)
(catch Throwable e
@@ -142,7 +152,7 @@
(mu/defn find-defenterprises
"using rewrite-clj, find and return all the namespaces 'required' by defenterprise forms in a file."
[file]
[file :- :string]
;; We want to know what namespace defendpoint 'requires': so do not need to parse anything in the enterprise dir.
(if (str/includes? file "/metabase_enterprise/")
[]
@@ -160,7 +170,7 @@
(mu/defn find-defenterprise-schemas
"using rewrite-clj, find and return all the namespaces 'required' by defenterprise-schema forms in a file."
[file]
[file :- :string]
;; We want to know what namespace defendpoint 'requires': so do not need to parse anything in the enterprise dir.
(if (str/includes? file "/metabase_enterprise/")
[]
@@ -204,19 +214,37 @@
don't include them in our deps tree."
'{metabase.config.core #{metabase-enterprise.core.dummy-namespace metabase.test.dummy-namespace}})
(mu/defn- file-dependencies :- [:map
[:namespace simple-symbol?]
[:filename string?] ; filename is relative to [[project-root]]
[:module symbol?]
[:deps [:sequential
[:map
[:namespace simple-symbol?]
[:module symbol?]
[:dynamic {:optional true} :keyword]]]]]
[prefix->module :- map?
file :- [:or
string?
[:fn {:error/message "Instance of a java.io.File"} #(instance? java.io.File %)]]]
(def ^:private ModuleConfig
[:map {:closed true}
[:team {:optional true} :string]
[:ns-prefix {:optional true} :string]
[:api {:optional true} [:or [:= :any] [:set :symbol]]]
[:uses {:optional true} [:or [:= :any] [:set :symbol]]]
[:friends {:optional true} [:set :symbol]]
[:model-exports {:optional true} [:or [:= :any] [:set :keyword]]]
[:model-imports {:optional true} [:or [:= :bypass] [:set :keyword]]]
[:module-exports {:optional true} [:set :symbol]]])
(def ^:private ModulesConfig
[:map-of :symbol ModuleConfig])
(def ^:private PrefixToModule
[:map-of :string :symbol])
(def ^:private FileDependencies
[:map {:closed true}
[:namespace {:optional true} [:maybe simple-symbol?]]
[:filename string?] ; filename is relative to [[project-root]]
[:module symbol?]
[:deps [:sequential
[:map {:closed true}
[:namespace simple-symbol?]
[:module symbol?]
[:dynamic {:optional true} :keyword]]]]])
(mu/defn- file-dependencies :- FileDependencies
[prefix->module :- PrefixToModule
file :- File]
(try
(let [decl (ns.file/read-file-ns-decl file)
ns-symb (ns.parse/name-from-ns-decl decl)
@@ -254,8 +282,6 @@
{:file file}
e)))))
(declare kondo-config)
(comment
(file-dependencies (modules/build-prefix->module (kondo-config)) "src/metabase/app_db/setup.clj")
;; should ignore the entries from [[ignored-dependencies]]
@@ -612,7 +638,8 @@
(mu/defn- module->source-files :- [:set :string]
"Return the set of all *source* filenames (relative to the [[project-root]] directory) for a `module`."
[deps module]
[deps :- [:sequential FileDependencies]
module :- symbol?]
(into
(sorted-set)
(comp (filter #(= (:module %) module))
@@ -628,6 +655,7 @@
[file]
(-> file
file->path-relative-to-project-root
(str/replace #"^modules/(?:drivers/)?[^/]+/(?=(?:src|test)/)" "")
(str/replace #"^enterprise/backend/" "")
(str/replace #"^(?:(?:src)|(?:test))/" "")
(str/replace #"\.clj[cs]?$" "")
@@ -706,10 +734,7 @@
[".clj" ".cljc" ".cljs" ".bb"])
(defn- module->test-path-prefix [modules-config module]
(let [ns-prefix (modules/module-ns-prefix modules-config module)]
(str (when (str/starts-with? ns-prefix "metabase-enterprise.") "enterprise/backend/")
"test/"
(-> ns-prefix (str/replace "." "/") (str/replace "-" "_")))))
(modules/module-directory modules-config module "test"))
(defn- existing-test-file-paths [path-prefix]
(into (sorted-set)
@@ -721,11 +746,11 @@
(mu/defn- module->test-files :- [:set :string]
"Return the set of test filenames associated with a `module`."
([modules-config :- map?
([modules-config :- ModulesConfig
module-sym :- :symbol]
(module->test-files modules-config (modules/build-prefix->module modules-config) module-sym))
([modules-config :- map?
prefix->module :- map?
([modules-config :- ModulesConfig
prefix->module :- PrefixToModule
module-sym :- :symbol]
(let [path-prefix (module->test-path-prefix modules-config module-sym)
test-dir (io/file path-prefix)
@@ -767,7 +792,7 @@
(mu/defn find-model-keywords :- [:set :keyword]
"Find all `:model/X` keywords referenced in a source file, ignoring comments."
[file]
[file :- File]
(try
(let [models (atom #{})]
(walk-parsed-ignore-comments!
@@ -787,7 +812,7 @@
(mu/defn find-model-definitions :- [:set :keyword]
"Find all models with their `t2/table-name` defined in this file."
[file]
[file :- File]
(let [models (atom #{})]
(walk-parsed-ignore-comments!
(fn [node]

View File

@@ -281,8 +281,27 @@
(known-linter-hint))
{:unknown (vec unknown)}))))
(def ^:private source-roots
["src" "test" "enterprise" "modules/drivers" "dev" "bin" "mage"])
(defn plugin-source-roots
"Each plugin's `src` and `test` directories under `modules/`, drivers included, relative to `repo-root`.
A plugin's other directories hold no code of its own, and a local build copies its sources into `target/`,
which would otherwise count twice."
([]
(plugin-source-roots "."))
([repo-root]
(let [subdirs (fn [parent]
(->> (.listFiles (io/file repo-root parent))
(filter #(.isDirectory ^java.io.File %))
(map #(str parent "/" (.getName ^java.io.File %)))
sort))]
(for [plugin (concat (remove #{"modules/drivers"} (subdirs "modules"))
(subdirs "modules/drivers"))
tree ["src" "test"]
:let [root (str plugin "/" tree)]
:when (.isDirectory (io/file repo-root root))]
root))))
(defn- source-roots []
(into ["src" "test" "enterprise" "dev" "bin" "mage"] (plugin-source-roots)))
(def ^:private source-extensions
[".clj" ".cljc" ".cljs"])
@@ -461,7 +480,7 @@
Returns `{:file \"src/...\", :line 42, :linters [...], :justified? boolean}` maps.
Forms inside string literals or line comments don't count."
([]
(scan source-roots))
(scan (source-roots)))
([roots]
(for [root roots
^java.io.File f (file-seq (io/file root))

View File

@@ -122,8 +122,8 @@
#_{:clj-kondo/ignore [:discouraged-var]}
(println (format "Rollbacked %d migrations. Latest migration: %s" n (latest-migration)))))
([k :- [:enum :id :count "id" "count"]
target]
([k :- [:enum :id :count "id" "count"]
target :- [:or :int :string]]
(let [n (case (keyword k)
:id (migration-since target)
:count (maybe-parse-long target))]
@@ -195,9 +195,10 @@
;; =>
{:forward \"DROP INDEX public.idx_user_id_device_id;\",
:rollback \"CREATE INDEX idx_user_id_device_id ON public.login_history(session_id, device_id);\"}"
([id]
([id :- :string]
(migration-sql-by-id id (mdb/db-type)))
([id db-type :- [:enum :postgres :mysql :mariadb :h2]]
([id :- :string
db-type :- [:enum :postgres :mysql :mariadb :h2]]
(t2/with-connection [conn]
(liquibase/with-liquibase [^Liquibase liquibase conn]
(let [database (liquibase-database db-type)

View File

@@ -19,14 +19,15 @@
"The path segments used to display `module` in the tree.
An enterprise module appears under its OSS counterpart when one exists.
Otherwise its full `enterprise/` name appears at the root."
Otherwise its full `enterprise/` name appears at the root, as a `module/` plugin's always does."
[modules-config module]
(let [segments (str/split (name module) #"\.")]
(if (= (namespace module) "enterprise")
(if (contains? modules-config (symbol (first segments)))
;; Keep an enterprise subtree together under its OSS module.
(into [(first segments) "enterprise"] (rest segments))
(into [(str "enterprise/" (first segments))] (rest segments)))
(case (namespace module)
"enterprise" (if (contains? modules-config (symbol (first segments)))
;; Keep an enterprise subtree together under its OSS module.
(into [(first segments) "enterprise"] (rest segments))
(into [(str "enterprise/" (first segments))] (rest segments)))
"module" (into [(str "module/" (first segments))] (rest segments))
segments)))
(defn explicit-ns-prefix
@@ -36,13 +37,6 @@
(when (and prefix (not= prefix (modules/default-ns-prefix module)))
prefix)))
(defn- ns-prefix->source-dir
"Repo-relative source directory for a namespace prefix: `metabase.lib-be` -> `src/metabase/lib_be`."
[ns-prefix]
(str (when (str/starts-with? ns-prefix "metabase-enterprise.") "enterprise/backend/")
"src/"
(-> ns-prefix (str/replace "." "/") (str/replace "-" "_"))))
(defn- set->strings
"Sort and stringify a set-valued config field."
[x]
@@ -88,7 +82,7 @@
"One module's config as the plain data the page consumes."
[modules-config module->uses used-by module]
(let [{:keys [api uses friends] :as entry} (get modules-config module)
source-dir (ns-prefix->source-dir (modules/module-ns-prefix modules-config module))]
source-dir (modules/module-directory modules-config module "src")]
{:id (str module)
:enterprise (= (namespace module) "enterprise")
:team (modules/module-team modules-config module)
@@ -108,7 +102,7 @@
;;; Per-module source metrics, from tracked files and git history.
(def ^:private source-dirs ["src" "enterprise/backend/src" "test" "enterprise/backend/test" "modules/drivers"])
(def ^:private source-dirs ["src" "enterprise/backend/src" "test" "enterprise/backend/test" "modules"])
(defn- git-output!
"Run Git and return stdout, or throw with its stderr when it fails."
@@ -123,7 +117,8 @@
str/split-lines
(filter #(re-find #"\.clj[cs]?$" %))))
(def ^:private source-root #"^(?:enterprise/backend/)?(?:src|test)/|^modules/drivers/[^/]+/(?:src|test)/")
(def ^:private source-root
#"^(?:enterprise/backend/)?(?:src|test)/|^modules/drivers/[^/]+/(?:src|test)/|^modules/[^/]+/(?:src|test)/")
(defn- file->ns
"The namespace a source or test file declares, going by its path."

View File

@@ -40,13 +40,19 @@
;;;; Ordering — must match `metabase.core.modules-test`
;;;; ---------------------------------------------------------------------------
(defn- enterprise? [module-name]
(str/starts-with? (str module-name) "enterprise/"))
(defn- module-group
"Where `module-name` sorts: OSS modules first, then `enterprise/` ones, then `module/` plugins."
[module-name]
(let [s (str module-name)]
(cond
(str/starts-with? s "enterprise/") 1
(str/starts-with? s "module/") 2
:else 0)))
(defn- sort-module-names
"Sort module symbols with `enterprise/` modules last. Mirrors the test's `sort-module-names`."
"Sort module symbols by [[module-group]], then by name. Mirrors the test's `sort-module-names`."
[module-names]
(sort-by (juxt #(if (enterprise? %) 1 0) str) module-names))
(sort-by (juxt module-group str) module-names))
(defn- sort-for-key
"Return the sorted element vector for a generated key. `:uses` sorts modules enterprise-last; everything
@@ -246,7 +252,7 @@
(str/join ", " to-add)))
(not sorted?)
(conj "Modules are not sorted by name (enterprise/ last) — reorder them by hand."))))
(conj "Modules are not sorted by name (enterprise/, then module/, last) — reorder them by hand."))))
;;;; ---------------------------------------------------------------------------
;;;; Entry point

View File

@@ -103,10 +103,10 @@
thrown? (atom false)]
(binding [kondo-ratchet/*ratchets-file* (.getPath budgets)
kondo-ratchet/*module-ratchets-file* (.getPath modules)]
(with-redefs [kondo-ratchet/known-linters (constantly (set (keys (:ignore-counts ratchets))))
kondo-ratchet/config-suppressions (constantly {})
kondo-ratchet/module-escape-hatches (constantly {})
kondo-ratchet/scan (constantly occurrences)]
(mt/with-dynamic-fn-redefs [kondo-ratchet/known-linters (constantly (set (keys (:ignore-counts ratchets))))
kondo-ratchet/config-suppressions (constantly {})
kondo-ratchet/module-escape-hatches (constantly {})
kondo-ratchet/scan (constantly occurrences)]
{:lines (str/split-lines
(with-out-str
(try
@@ -193,7 +193,7 @@
:config-counts {}
:comment-exempt #{}})))]
(binding [kondo-ratchet/*ratchets-file* (.getPath budgets)]
(with-redefs [kondo-ratchet/known-linters (constantly #{:a})]
(mt/with-dynamic-fn-redefs [kondo-ratchet/known-linters (constantly #{:a})]
(let [out (with-out-str
(is (thrown-with-msg? clojure.lang.ExceptionInfo #"names 1 unknown linter: :bogus"
(kondo-ratchet/check))))]

View File

@@ -153,6 +153,21 @@
:extra 1}
(kondo-ratchet/actual-counts occurrences))))))
(deftest ^:parallel plugin-source-roots-test
(let [dir (.toFile (java.nio.file.Files/createTempDirectory
"plugin-roots"
(make-array java.nio.file.attribute.FileAttribute 0)))]
(doseq [path ["modules/embedder/src"
"modules/embedder/resources"
"modules/embedder/target/classes"
"modules/drivers/mongo/src"
"modules/drivers/mongo/test"
"modules/drivers/mongo/target"]]
(.mkdirs (io/file dir path)))
(testing "plugins and drivers contribute their src and test directories, never target or resources"
(is (= ["modules/embedder/src" "modules/drivers/mongo/src" "modules/drivers/mongo/test"]
(kondo-ratchet/plugin-source-roots (.getPath dir)))))))
(deftest ^:parallel scan-error-identifies-file-test
(let [dir (.toFile (java.nio.file.Files/createTempDirectory
"kondo-ratchet-error-test"
@@ -361,10 +376,10 @@
{:file "f.clj", :line 2, :linters [:free]}]]
(binding [kondo-ratchet/*ratchets-file* (.getPath budgets)
kondo-ratchet/*module-ratchets-file* (.getPath modules)]
(with-redefs [kondo-ratchet/known-linters (constantly #{:free})
kondo-ratchet/scan (constantly occurrences)
kondo-ratchet/config-suppressions (constantly {})
kondo-ratchet/module-escape-hatches (constantly {})]
(mt/with-dynamic-fn-redefs [kondo-ratchet/known-linters (constantly #{:free})
kondo-ratchet/scan (constantly occurrences)
kondo-ratchet/config-suppressions (constantly {})
kondo-ratchet/module-escape-hatches (constantly {})]
(is (= ["seeded :free at 2"
(str "wrote " (.getPath budgets))]
(str/split-lines (with-out-str (kondo-ratchet/fix! {:seed "free"}))))))
@@ -383,8 +398,8 @@
(spit (kondo-ratchet/render-module-ratchets {})))]
(binding [kondo-ratchet/*ratchets-file* (.getPath budgets)
kondo-ratchet/*module-ratchets-file* (.getPath modules)]
(with-redefs [kondo-ratchet/known-linters (constantly #{:free})
kondo-ratchet/scan (fn [] (throw (AssertionError. "scanned before validating the seed")))]
(mt/with-dynamic-fn-redefs [kondo-ratchet/known-linters (constantly #{:free})
kondo-ratchet/scan (fn [] (throw (AssertionError. "scanned before validating the seed")))]
(is (thrown-with-msg? clojure.lang.ExceptionInfo
#"^cannot seed :bogus: not a known linter -- policies must name"
(kondo-ratchet/fix! {:seed ":bogus"})))
@@ -405,10 +420,10 @@
run! #(str/split-lines (with-out-str (kondo-ratchet/fix!)))]
(binding [kondo-ratchet/*ratchets-file* (.getPath budgets)
kondo-ratchet/*module-ratchets-file* (.getPath modules)]
(with-redefs [kondo-ratchet/known-linters (constantly #{:free :empty :gone :zero})
kondo-ratchet/scan (constantly occurrences)
kondo-ratchet/config-suppressions (constantly {})
kondo-ratchet/module-escape-hatches (constantly {})]
(mt/with-dynamic-fn-redefs [kondo-ratchet/known-linters (constantly #{:free :empty :gone :zero})
kondo-ratchet/scan (constantly occurrences)
kondo-ratchet/config-suppressions (constantly {})
kondo-ratchet/module-escape-hatches (constantly {})]
(is (= ["dropped :gone (no ignores left)"
"dropped :zero (no ignores left)"
"WARNING: :unlimited policies with no ignores left: :empty -- delete an entry by hand once its linter no longer needs one"
@@ -437,10 +452,10 @@
{:api-any 2, :friend-edges 5, :uses-any 1})))]
(binding [kondo-ratchet/*ratchets-file* (.getPath budgets)
kondo-ratchet/*module-ratchets-file* (.getPath modules)]
(with-redefs [kondo-ratchet/known-linters (constantly #{})
kondo-ratchet/scan (constantly [])
kondo-ratchet/config-suppressions (constantly {})
kondo-ratchet/module-escape-hatches (constantly {:api-any 1, :friend-edges 4, :uses-any 0})]
(mt/with-dynamic-fn-redefs [kondo-ratchet/known-linters (constantly #{})
kondo-ratchet/scan (constantly [])
kondo-ratchet/config-suppressions (constantly {})
kondo-ratchet/module-escape-hatches (constantly {:api-any 1, :friend-edges 4, :uses-any 0})]
(is (= ["lowered module :api-any 2 -> 1"
"lowered module :friend-edges 5 -> 4"
"dropped module :uses-any (no escape hatches left)"

View File

@@ -87,17 +87,18 @@
modules'))))
(defn- sort-module-names
"Sort module names in order but sort the `enterprise/` modules last."
"Sort module names in order, with `enterprise/` modules after the OSS ones and `module/` plugins last."
[module-names]
(sort-by (fn [module-name]
[(if (str/starts-with? module-name "enterprise/")
1
0)
[(cond
(str/starts-with? module-name "enterprise/") 1
(str/starts-with? module-name "module/") 2
:else 0)
module-name])
module-names))
(deftest modules-should-be-sorted-by-name-test
(testing "Modules configs should sorted by module name with enterprise/modules appearing last"
(testing "Modules are sorted by name, with enterprise/ modules and then module/ plugins last"
(let [actual (module-names-in-file-order)
expected (sort-module-names actual)]
(is (= expected
@@ -189,6 +190,24 @@
(testing (format "Remove %s from %s" (pr-str extraneous) (pr-str ks))
(is (empty? extraneous)))))))
(deftest plugin-namespaces-live-in-their-plugin-directory-test
(testing "a namespace resolves to plugin module/X exactly when its file sits under modules/X/"
(let [config (dev.deps-graph/kondo-config)]
(doseq [{:keys [filename module]} (dev.deps-graph/dependencies)
:let [file (#'dev.deps-graph/file->path-relative-to-project-root filename)
directory (second (re-find #"^modules/(?!drivers/)([^/]+)/" file))
plugin (some->> module (modules/plugin-root config))]
:when (or directory plugin)]
(testing (format "\n%s resolves to %s" file (pr-str module))
(is (= directory (some-> plugin name))))))))
(deftest ^:parallel plugin-files-name-their-namespaces-test
(is (= '[metabase-module.embedder.plugin metabase-module.embedder.plugin-test metabase.driver.mongo]
(map #'dev.deps-graph/file->namespace
["modules/embedder/src/metabase_module/embedder/plugin.clj"
"modules/embedder/test/metabase_module/embedder/plugin_test.clj"
"modules/drivers/mongo/src/metabase/driver/mongo.clj"]))))
(deftest ^:parallel uses-references-must-be-namable-test
(testing "every module named in :uses is visible to the caller"
(let [config (dev.deps-graph/kondo-config)]
@@ -256,8 +275,10 @@
(testing "`:module` resolves through each module's :ns-prefix, so dotted and renamed modules find their tests"
(is (= ["test/metabase/lib/schema"
"test/metabase/actions_rest"
"enterprise/backend/test/metabase_enterprise/transforms_python"]
(metabase.test-runner/module-folders '[lib.schema actions.rest enterprise/transforms.python])))))
"enterprise/backend/test/metabase_enterprise/transforms_python"
"modules/embedder/test/metabase_module/embedder"]
(metabase.test-runner/module-folders
'[lib.schema actions.rest enterprise/transforms.python module/embedder])))))
(deftest do-not-use-rest-modules-in-other-modules-test
(doseq [[module {:keys [uses], :as _config}] (dev.deps-graph/kondo-config)

View File

@@ -23,7 +23,6 @@ Here's a non-exhaustive list of things Metabot can help with:
- [Edit SQL directly in the native editor](#inline-sql-editing).
- [Analyze a chart](#analyze-charts-with-metabot).
- [Fix errors in SQL code](#have-metabot-fix-sql-queries).
- [Generate transforms](../data-modeling/transforms/transforms-overview.md#use-metabot-to-generate-code-for-transforms)
- [Generate charts in documents](../documents/introduction.md)
- [Answer questions from Slack](./metabot-slack.md).

View File

@@ -36,8 +36,6 @@ See [full docs for Metabot](metabot.md) and [embedded AI chat](../embedding/ai-c
Connect your favorite third-party AI tool - like Claude or Codex - to the Metabase MCP server. MCP servers are designed for answering in-the-moment questions like "hey btw what's our q3 revenue?" Metabase's MCP server is also useful when combined with other MCP servers. For example, you can ask Claude a question about your customers that combines data from Metabase, your CRM, and your support ticket platform.
Compared to the built-in Metabot, MCP server has somewhat restricted functionality (for example, it can't generate code or build transforms for now). See [the ever-expanding list of MCP server tools](mcp.md#available-tools).
**Controls**: Metabase MCP server requires people to authenticate into Metabase, and all the responses it provides will be scoped to their permissions. However, unlike built-in Metabot, MCP server doesn't come with granular control over which _tools_ people can use, or disable MCP server altogether.
**Provider**: Requests to Metabase's MCP server are handled by the provider you choose to connect to the MCP server (e.g. Claude, Cursor, etc).

View File

@@ -62,6 +62,7 @@ Supported models:
| GPT-5.6 Luna | `gpt-5.6-luna` | 922,000 |
| GPT-5.6 Sol | `gpt-5.6-sol` | 922,000 |
| GPT-5.6 Terra | `gpt-5.6-terra` | 922,000 |
| GPT-6 Astra | `gpt-6-astra` | 922,000 |
Credentials:
@@ -247,6 +248,7 @@ Supported models:
| GPT-5.4 (2026-03-05) | `openai.gpt-5.4-2026-03-05` | 272,000 |
| GPT-5.5 | `openai.gpt-5.5` | 272,000 |
| GPT-5.5 (2026-04-23) | `openai.gpt-5.5-2026-04-23` | 272,000 |
| GPT-6 Astra | `openai.gpt-6-astra` | 922,000 |
Credentials:

View File

@@ -73,17 +73,16 @@ Each conversation is tagged with a source (where in Metabase the conversation to
### Profiles
A profile is the configuration Metabot uses for a conversation: which prompt, which tools, and what it's allowed to do. The Conversations admin page, the **By profile** chart in [Usage stats](#usage-stats), and the **Metabot Conversations** model (see [Building custom reports](#building-custom-reports)) all show the human-readable profile name. The [AI Usage Log](#building-custom-reports) model exposes the raw `profile_id` instead (e.g. `internal`, `transforms_codegen`, `embedding_next`).
A profile is the configuration Metabot uses for a conversation: which prompt, which tools, and what it's allowed to do. The Conversations admin page, the **By profile** chart in [Usage stats](#usage-stats), and the **Metabot Conversations** model (see [Building custom reports](#building-custom-reports)) all show the human-readable profile name. The [AI Usage Log](#building-custom-reports) model exposes the raw `profile_id` instead (e.g. `internal`, `embedding_next`).
| Profile | What it does |
| -------------------- | ------------------------------------------------------------------------------------------------------------------------- |
| `Internal` | The default Metabot in the chat sidebar. Can build query-builder questions _and_ write SQL. |
| `NLQ` | Natural-language querying only. Always returns a [query builder](../questions/query-builder/editor.md) result, never SQL. |
| `SQL` | SQL writing only. Used by [inline SQL editing](./metabot.md#inline-sql-editing) and similar surfaces. |
| `Slackbot` | The persona behind [Metabot in Slack](./metabot-slack.md). |
| `Embedding` | The Metabot persona used inside [embedded Metabase](../embedding/start.md). |
| `Transforms codegen` | Generates [transform](../data-modeling/transforms/transforms-overview.md), SQL, or Python. |
| `Documents` | Generates content inside [Documents](../documents/start.md). |
| Profile | What it does |
| ----------- | ------------------------------------------------------------------------------------------------------------------------- |
| `Internal` | The default Metabot in the chat sidebar. Can build query-builder questions _and_ write SQL. |
| `NLQ` | Natural-language querying only. Always returns a [query builder](../questions/query-builder/editor.md) result, never SQL. |
| `SQL` | SQL writing only. Used by [inline SQL editing](./metabot.md#inline-sql-editing) and similar surfaces. |
| `Slackbot` | The persona behind [Metabot in Slack](./metabot-slack.md). |
| `Embedding` | The Metabot persona used inside [embedded Metabase](../embedding/start.md). |
| `Documents` | Generates content inside [Documents](../documents/start.md). |
## Conversations
@@ -118,7 +117,7 @@ The detail view is a full audit of a single conversation. It includes:
- **Stat tiles**: Messages, Total tokens, Queries run, Searches.
- **Feedback** (if any): thumbs-up or thumbs-down and comments. The agent response that triggered the feedback is shown alongside.
- **Conversation transcript**: the full message-by-message exchange. Tool calls (search calls, query construction, etc.) are inlined. You can click "View" to open a modal with the info.
- **Queries generated**: every SQL or [query builder](../questions/query-builder/editor.md) (MBQL) query Metabot wrote during the conversation, with the referenced tables listed underneath. Hit **Visit** on a query to open the item in a new tab and run it yourself. Transform code-gen queries are shown read-only and can't be re-run from here.
- **Queries generated**: every SQL or [query builder](../questions/query-builder/editor.md) (MBQL) query Metabot wrote during the conversation, with the referenced tables listed underneath. Hit **Visit** on a query to open the item in a new tab and run it yourself.
### The `/inspect` shortcut

14994
docs/api.json generated

File diff suppressed because it is too large Load Diff

View File

@@ -147,9 +147,7 @@ config:
embedding-homepage: hidden
embedding-secret-key: null
enable-embedding-interactive: false
enable-embedding-sdk: false
enable-embedding-simple: false
enable-embedding-static: false
enable-embedding-modular: false
enable-password-login: true
enable-pivoted-exports: true
enable-public-sharing: true
@@ -272,6 +270,7 @@ config:
metabot-sql-system-prompt: ''
mfa-challenge-signing-key: null
mfa-enforcement: 'off'
mfa-requirement-deadline: null
native-query-autocomplete-match-style: substring
nested-field-columns-value-length-limit: 50000
no-data-illustration: default

View File

@@ -731,7 +731,17 @@ Allow admins to securely embed questions and dashboards within other application
Allow admins to embed Metabase via interactive embedding?
### `MB_ENABLE_EMBEDDING_SDK`
### `MB_ENABLE_EMBEDDING_MODULAR`
- Type: boolean
- Default: `false`
- [Configuration file name](./config-file.md): `enable-embedding-modular`
Allow admins to embed Metabase via modular embedding, the SDK for React, and guest embeds?
### `MB_ENABLE_EMBEDDING_SDK [DEPRECATED]`
> DEPRECATED: 0.65.0
- Type: boolean
- Default: `false`
@@ -739,7 +749,9 @@ Allow admins to embed Metabase via interactive embedding?
Allow admins to embed Metabase via the SDK?
### `MB_ENABLE_EMBEDDING_SIMPLE`
### `MB_ENABLE_EMBEDDING_SIMPLE [DEPRECATED]`
> DEPRECATED: 0.65.0
- Type: boolean
- Default: `false`
@@ -747,7 +759,9 @@ Allow admins to embed Metabase via the SDK?
Allow admins to embed Metabase via modular embedding?
### `MB_ENABLE_EMBEDDING_STATIC`
### `MB_ENABLE_EMBEDDING_STATIC [DEPRECATED]`
> DEPRECATED: 0.65.0
- Type: boolean
- Default: `false`
@@ -1883,7 +1897,15 @@ Key used to sign MFA challenge tokens. Generated automatically on first use.
- Default: `off`
- [Configuration file name](./config-file.md): `mfa-enforcement`
Controls whether two-factor authentication is available to users. :off disables it entirely; :optional allows users to enroll voluntarily.
Controls whether two-factor authentication is available to users. :off disables it entirely; :optional allows users to enroll voluntarily, :required mandates users enroll.
### `MB_MFA_REQUIREMENT_DEADLINE`
- Type: timestamp
- Default: `null`
- [Configuration file name](./config-file.md): `mfa-requirement-deadline`
Time after which mfa-enforcement will take effect for all users.
### `MB_NATIVE_QUERY_AUTOCOMPLETE_MATCH_STYLE`

Binary file not shown.

Before

Width:  |  Height:  |  Size: 42 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 62 KiB

View File

@@ -126,8 +126,6 @@ To create a transform:
If you're writing a SQL transform, variables _must_ be wrapped in optional blocks (`[[ ]]`), or given a default value. See [variables in SQL transforms](query.md#variables-in-sql-transforms) for more details.
If [Metabot is enabled](../../ai/settings.md#enable-ai-features), you can [use Metabot](#use-metabot-to-generate-code-for-transforms) to generate code for your transform.
6. Click **Save** in the top right corner and fill out the transform information:
- **Name** (required): The name of the transform.
@@ -140,28 +138,6 @@ To create a transform:
7. Optionally, once the transform is saved, assign tags to your transform. Tags are used by [jobs](./jobs-and-runs.md) to run transforms on schedule.
## Use Metabot to generate code for transforms
> Code generation for transforms requires [AI features](../../ai/settings.md#enable-ai-features).
You can ask Metabot to generate a new SQL or Python-based transform, or edit an existing transform.
1. Go to **Data studio > Transforms**. If you want Metabot to edit an existing transform, navigate to the transform.
2. While in Transforms view, click on the Metabot icon in top right.
3. Describe the transform that you'd like Metabot to write.
You can specify which kind of transform you want (Python or SQL) and @-mention specific data sources to help Metabot understand your request.
![Create a new transform with codegen](../images/transform-codegen.png)
4. Metabot will create a to-do list for itself that will show its thinking process, and then work through the list.
5. If you asked Metabot to create a new transform, review the code that Metabot offers in the chat window and click on "Create" under the code snippet.
![Create snippet](../images/create-transform.png)
You can continue working with Metabot to refine your code. Metabot will suggest code changes, and give you the option of accepting or rejecting the changes.
## Edit a transform
_Data Studio > Transforms_
@@ -184,7 +160,7 @@ To edit the transform's query or script:
2. Find the transform you'd like to edit and click on **Edit definition** above the transform definition.
3. Edit the query or script.
See [query-based transforms](query.md) and [Python transforms](python.md) for more information. You can [use Metabot](#use-metabot-to-generate-code-for-transforms) to help edit your transform.
See [query-based transforms](query.md) and [Python transforms](python.md) for more information.
Once you change the transform's query or script, the next transform run (manual or scheduled) will use the updated query and write the results into the target table. If you've changed the table's columns, and you have questions that query the table, they might break. For example, if your new transform query no longer includes a column that a downstream question was relying on, that question will break.

View File

@@ -17,7 +17,8 @@ This is either:
<br/></li>
<li>the string ID found in the <code>entity_id</code> key of the dashboard object when using the API directly or using the SDK Collection Browser to return data</li>
</ul>
</div></li><li class="tsd-parameter"><h5 id="datapickerprops"><code class="tsd-tag">Optional</code><span class="tsd-kind-property">dataPickerProps</span><span class="tsd-signature-symbol">?: </span><a href="https://www.typescriptlang.org/docs/handbook/utility-types.html#picktype-keys" class="tsd-signature-type external" target="_blank">Pick</a><span class="tsd-signature-symbol">&lt;</span><a href="SdkQuestionProps.html" class="tsd-signature-type tsd-kind-interface">SdkQuestionProps</a><span class="tsd-signature-symbol">,</span> <span class="tsd-signature-type">&quot;entityTypes&quot;</span><span class="tsd-signature-symbol">&gt;</span></h5><div class="tsd-comment tsd-typography"><p>Additional props to pass to the query builder rendered by <code>InteractiveQuestion</code> when creating a new dashboard question.</p>
</div></li><li class="tsd-parameter"><h5 id="datapickerprops"><code class="tsd-tag">Optional</code><span class="tsd-kind-property">dataPickerProps</span><span class="tsd-signature-symbol">?: </span><a href="https://www.typescriptlang.org/docs/handbook/utility-types.html#picktype-keys" class="tsd-signature-type external" target="_blank">Pick</a><span class="tsd-signature-symbol">&lt;</span><a href="SdkQuestionProps.html" class="tsd-signature-type tsd-kind-interface">SdkQuestionProps</a><span class="tsd-signature-symbol">,</span> <span class="tsd-signature-type">&quot;entityTypes&quot;</span> <span class="tsd-signature-symbol">|</span> <span class="tsd-signature-type">&quot;dataPicker&quot;</span><span class="tsd-signature-symbol">&gt;</span></h5><div class="tsd-comment tsd-typography"><p>Additional props to pass to the query builder rendered by <code>InteractiveQuestion</code> when creating a new dashboard question.</p>
<p>Set <code>dataPicker: &quot;staged&quot;</code> to always get the staged data picker. By default, Metabase shows a simple dropdown menu with tables and models, and only switches to the staged picker when there are 100 or more data sources. <code>entityTypes: [&quot;question&quot;]</code> only takes effect in the staged picker.</p>
</div></li><li class="tsd-parameter"><h5 id="drillthroughquestionheight"><code class="tsd-tag">Optional</code><span class="tsd-kind-property">drillThroughQuestionHeight</span><span class="tsd-signature-symbol">?: </span><span class="tsd-signature-type">Height</span><span class="tsd-signature-symbol">&lt;</span><span class="tsd-signature-type">string</span> <span class="tsd-signature-symbol">|</span> <span class="tsd-signature-type">number</span><span class="tsd-signature-symbol">&gt;</span></h5><div class="tsd-comment tsd-typography"><p>Height of a question component when drilled from the dashboard to a question level.</p>
</div></li><li class="tsd-parameter"><h5 id="drillthroughquestionprops"><code class="tsd-tag">Optional</code><span class="tsd-kind-property">drillThroughQuestionProps</span><span class="tsd-signature-symbol">?: </span><a href="DrillThroughQuestionProps.html" class="tsd-signature-type tsd-kind-interface">DrillThroughQuestionProps</a></h5><div class="tsd-comment tsd-typography"><p>Props of a question component when drilled from the dashboard to a question level.</p>
</div></li><li class="tsd-parameter"><h5 id="enableentitynavigation"><code class="tsd-tag">Optional</code><span class="tsd-kind-property">enableEntityNavigation</span><span class="tsd-signature-symbol">?: </span><span class="tsd-signature-type">boolean</span></h5><div class="tsd-comment tsd-typography"><p>When true, internal click behaviors (links to dashboards/questions) are preserved.

File diff suppressed because one or more lines are too long

File diff suppressed because one or more lines are too long

File diff suppressed because one or more lines are too long

View File

@@ -17,7 +17,6 @@ This is either:
<br/></li>
<li>the string ID found in the <code>entity_id</code> key of the dashboard object when using the API directly or using the SDK Collection Browser to return data</li>
</ul>
</div></li><li class="tsd-parameter"><h5 id="datapickerprops"><code class="tsd-tag">Optional</code><span class="tsd-kind-property">dataPickerProps</span><span class="tsd-signature-symbol">?: </span><a href="https://www.typescriptlang.org/docs/handbook/utility-types.html#picktype-keys" class="tsd-signature-type external" target="_blank">Pick</a><span class="tsd-signature-symbol">&lt;</span><a href="SdkQuestionProps.html" class="tsd-signature-type tsd-kind-interface">SdkQuestionProps</a><span class="tsd-signature-symbol">,</span> <span class="tsd-signature-type">&quot;entityTypes&quot;</span><span class="tsd-signature-symbol">&gt;</span></h5><div class="tsd-comment tsd-typography"><p>Additional props to pass to the query builder rendered by <code>InteractiveQuestion</code> when creating a new dashboard question.</p>
</div></li><li class="tsd-parameter"><h5 id="hiddenparameters"><code class="tsd-tag">Optional</code><span class="tsd-kind-property">hiddenParameters</span><span class="tsd-signature-symbol">?: </span><span class="tsd-signature-type">string</span><span class="tsd-signature-symbol">[]</span></h5><div class="tsd-comment tsd-typography"><p>A list of <a href="https://www.metabase.com/docs/latest/embedding/public-links.html#appearance-parameters">parameters to hide</a>.
<br/></p>
<ul>

File diff suppressed because one or more lines are too long

File diff suppressed because one or more lines are too long

View File

@@ -9,7 +9,7 @@
| <a id="autorefreshinterval"></a> `autoRefreshInterval?` | `number` | The interval between auto refreshes on the dashboard, in seconds. |
| <a id="classname"></a> `className?` | `string` | A custom class name to be added to the root element. |
| <a id="dashboardid"></a> `dashboardId` | `string` \| `number` | The ID of the dashboard. <br/> This is either: <br/> - the numerical ID when accessing a dashboard link, i.e. `http://localhost:3000/dashboard/1-my-dashboard` where the ID is `1` <br/> - the string ID found in the `entity_id` key of the dashboard object when using the API directly or using the SDK Collection Browser to return data |
| <a id="datapickerprops"></a> `dataPickerProps?` | [`Pick`](https://www.typescriptlang.org/docs/handbook/utility-types.html#picktype-keys)\<[`SdkQuestionProps`](./api/SdkQuestionProps.md), `"entityTypes"`\> | Additional props to pass to the query builder rendered by `InteractiveQuestion` when creating a new dashboard question. |
| <a id="datapickerprops"></a> `dataPickerProps?` | [`Pick`](https://www.typescriptlang.org/docs/handbook/utility-types.html#picktype-keys)\<[`SdkQuestionProps`](./api/SdkQuestionProps.md), `"entityTypes"` \| `"dataPicker"`\> | Additional props to pass to the query builder rendered by `InteractiveQuestion` when creating a new dashboard question. Set `dataPicker: "staged"` to always get the staged data picker. By default, Metabase shows a simple dropdown menu with tables and models, and only switches to the staged picker when there are 100 or more data sources. `entityTypes: ["question"]` only takes effect in the staged picker. |
| <a id="drillthroughquestionheight"></a> `drillThroughQuestionHeight?` | `Height`\<`string` \| `number`\> | Height of a question component when drilled from the dashboard to a question level. |
| <a id="drillthroughquestionprops"></a> `drillThroughQuestionProps?` | [`DrillThroughQuestionProps`](./api/DrillThroughQuestionProps.md) | Props of a question component when drilled from the dashboard to a question level. |
| <a id="enableentitynavigation"></a> `enableEntityNavigation?` | `boolean` | When true, internal click behaviors (links to dashboards/questions) are preserved. When false (default for SDK), these click behaviors are filtered out. |

View File

@@ -51,8 +51,6 @@ function InteractiveDashboard(
| "treemap"
| "list",
) => void;
} & {
dataPickerProps?: Pick<SdkQuestionProps, "entityTypes">;
} & {},
): Element;
```
@@ -63,9 +61,9 @@ A dashboard component with drill downs, click behaviors, and the ability to view
<!-- [<snippet parameters>] -->
| Parameter | Type | Description |
| :-------- | :---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | :---------- |
| `props` | \{ `autoRefreshInterval?`: `number`; `dashboardId`: [`SdkDashboardId`](./api/SdkDashboardId.md); `drillThroughQuestionHeight?`: `Height`\<`string` \| `number`\>; `drillThroughQuestionProps?`: [`DrillThroughQuestionProps`](./api/DrillThroughQuestionProps.md); `initialParameters?`: [`ParameterValues`](./api/ParameterValues.md); `onParametersChange?`: (`payload`: [`ParameterChangePayload`](./api/ParameterChangePayload.md)) => `void`; `parameters?`: [`ParameterValues`](./api/ParameterValues.md); `plugins?`: [`MetabasePluginsConfig`](./api/MetabasePluginsConfig.md); `renderDrillThroughQuestion?`: () => [`ReactNode`](https://github.com/DefinitelyTyped/DefinitelyTyped/blob/0b728411cd1dfb4bd26992bb35a73cf8edaa22e7/types/react/index.d.ts#L478); \} & \{ `dashboardId?`: [`SdkDashboardId`](./api/SdkDashboardId.md) \| `null`; `token?`: `string` \| `null`; \} & \{ `enableEntityNavigation?`: `boolean`; `hiddenParameters?`: `string`[]; `initialParameters?`: [`ParameterValues`](./api/ParameterValues.md); `withCardTitle?`: `boolean`; `withDownloads?`: `boolean`; `withSubscriptions?`: `boolean`; `withTitle?`: `boolean`; \} & \{ `className?`: `string`; `style?`: [`CSSProperties`](https://github.com/DefinitelyTyped/DefinitelyTyped/blob/0b728411cd1dfb4bd26992bb35a73cf8edaa22e7/types/react/index.d.ts#L2579); \} & \{ `onLoad?`: (`dashboard`: [`MetabaseDashboard`](./api/MetabaseDashboard.md) \| `null`) => `void`; `onLoadWithoutCards?`: (`dashboard`: [`MetabaseDashboard`](./api/MetabaseDashboard.md) \| `null`) => `void`; `onVisualizationChange?`: (`visualization`: \| `"object"` \| `"table"` \| `"bar"` \| `"line"` \| `"pie"` \| `"scalar"` \| `"row"` \| `"area"` \| `"combo"` \| `"pivot"` \| `"smartscalar"` \| `"gauge"` \| `"progress"` \| `"funnel"` \| `"map"` \| `"scatter"` \| `"boxplot"` \| `"waterfall"` \| `"sankey"` \| `"treemap"` \| `"list"`) => `void`; \} & \{ `dataPickerProps?`: [`Pick`](https://www.typescriptlang.org/docs/handbook/utility-types.html#picktype-keys)\<[`SdkQuestionProps`](./api/SdkQuestionProps.md), `"entityTypes"`\>; \} & \{ \} | |
| Parameter | Type | Description |
| :-------- | :----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | :---------- |
| `props` | \{ `autoRefreshInterval?`: `number`; `dashboardId`: [`SdkDashboardId`](./api/SdkDashboardId.md); `drillThroughQuestionHeight?`: `Height`\<`string` \| `number`\>; `drillThroughQuestionProps?`: [`DrillThroughQuestionProps`](./api/DrillThroughQuestionProps.md); `initialParameters?`: [`ParameterValues`](./api/ParameterValues.md); `onParametersChange?`: (`payload`: [`ParameterChangePayload`](./api/ParameterChangePayload.md)) => `void`; `parameters?`: [`ParameterValues`](./api/ParameterValues.md); `plugins?`: [`MetabasePluginsConfig`](./api/MetabasePluginsConfig.md); `renderDrillThroughQuestion?`: () => [`ReactNode`](https://github.com/DefinitelyTyped/DefinitelyTyped/blob/0b728411cd1dfb4bd26992bb35a73cf8edaa22e7/types/react/index.d.ts#L478); \} & \{ `dashboardId?`: [`SdkDashboardId`](./api/SdkDashboardId.md) \| `null`; `token?`: `string` \| `null`; \} & \{ `enableEntityNavigation?`: `boolean`; `hiddenParameters?`: `string`[]; `initialParameters?`: [`ParameterValues`](./api/ParameterValues.md); `withCardTitle?`: `boolean`; `withDownloads?`: `boolean`; `withSubscriptions?`: `boolean`; `withTitle?`: `boolean`; \} & \{ `className?`: `string`; `style?`: [`CSSProperties`](https://github.com/DefinitelyTyped/DefinitelyTyped/blob/0b728411cd1dfb4bd26992bb35a73cf8edaa22e7/types/react/index.d.ts#L2579); \} & \{ `onLoad?`: (`dashboard`: [`MetabaseDashboard`](./api/MetabaseDashboard.md) \| `null`) => `void`; `onLoadWithoutCards?`: (`dashboard`: [`MetabaseDashboard`](./api/MetabaseDashboard.md) \| `null`) => `void`; `onVisualizationChange?`: (`visualization`: \| `"object"` \| `"table"` \| `"bar"` \| `"line"` \| `"pie"` \| `"scalar"` \| `"row"` \| `"area"` \| `"combo"` \| `"pivot"` \| `"smartscalar"` \| `"gauge"` \| `"progress"` \| `"funnel"` \| `"map"` \| `"scatter"` \| `"boxplot"` \| `"waterfall"` \| `"sankey"` \| `"treemap"` \| `"list"`) => `void`; \} & \{ \} | |
<!-- [<endsnippet parameters>] -->

View File

@@ -9,7 +9,6 @@
| <a id="autorefreshinterval"></a> `autoRefreshInterval?` | `number` | The interval between auto refreshes on the dashboard, in seconds. |
| <a id="classname"></a> `className?` | `string` | A custom class name to be added to the root element. |
| <a id="dashboardid"></a> `dashboardId` | `string` \| `number` | The ID of the dashboard. <br/> This is either: <br/> - the numerical ID when accessing a dashboard link, i.e. `http://localhost:3000/dashboard/1-my-dashboard` where the ID is `1` <br/> - the string ID found in the `entity_id` key of the dashboard object when using the API directly or using the SDK Collection Browser to return data |
| <a id="datapickerprops"></a> `dataPickerProps?` | [`Pick`](https://www.typescriptlang.org/docs/handbook/utility-types.html#picktype-keys)\<[`SdkQuestionProps`](./api/SdkQuestionProps.md), `"entityTypes"`\> | Additional props to pass to the query builder rendered by `InteractiveQuestion` when creating a new dashboard question. |
| <a id="drillthroughquestionheight"></a> `drillThroughQuestionHeight?` | `Height`\<`string` \| `number`\> | Height of a question component when drilled from the dashboard to a question level. |
| <a id="drillthroughquestionprops"></a> `drillThroughQuestionProps?` | [`DrillThroughQuestionProps`](./api/DrillThroughQuestionProps.md) | Props of a question component when drilled from the dashboard to a question level. |
| <a id="enableentitynavigation"></a> `enableEntityNavigation?` | `boolean` | When true, internal click behaviors (links to dashboards/questions) are preserved. When false (default for SDK), these click behaviors are filtered out. |

View File

@@ -9,7 +9,6 @@
| <a id="autorefreshinterval"></a> `autoRefreshInterval?` | `number` | The interval between auto refreshes on the dashboard, in seconds. |
| <a id="classname"></a> `className?` | `string` | A custom class name to be added to the root element. |
| <a id="dashboardid"></a> `dashboardId?` | [`SdkDashboardId`](./api/SdkDashboardId.md) \| `null` | The ID of the dashboard. <br/> This is either: <br/> - the numerical ID when accessing a dashboard link, i.e. `http://localhost:3000/dashboard/1-my-dashboard` where the ID is `1` <br/> - the string ID found in the `entity_id` key of the dashboard object when using the API directly or using the SDK Collection Browser to return data |
| <a id="datapickerprops"></a> `dataPickerProps?` | [`Pick`](https://www.typescriptlang.org/docs/handbook/utility-types.html#picktype-keys)\<[`SdkQuestionProps`](./api/SdkQuestionProps.md), `"entityTypes"`\> | Additional props to pass to the query builder rendered by `InteractiveQuestion` when creating a new dashboard question. |
| <a id="hiddenparameters"></a> `hiddenParameters?` | `string`[] | A list of [parameters to hide](https://www.metabase.com/docs/latest/embedding/public-links.html#appearance-parameters). <br/> - Combining [initialParameters](./api/InteractiveDashboardProps.md#initialparameters) and [hiddenParameters](./api/InteractiveDashboardProps.md#hiddenparameters) to filter data on the frontend is a [security risk](https://www.metabase.com/docs/latest/embedding/sdk/authentication.html#security-warning-each-end-user-must-have-their-own-metabase-account). <br/> - Combining [initialParameters](./api/InteractiveDashboardProps.md#initialparameters) and [hiddenParameters](./api/InteractiveDashboardProps.md#hiddenparameters) to declutter the user interface is fine. |
| <a id="initialparameters"></a> `initialParameters?` | [`ParameterValues`](./api/ParameterValues.md) | Initial values for query parameters, slug-keyed. Applied once on mount; user widget edits afterwards are not reflected back to the host. <br/> For each parameter: <br/> - set to a value (string for a single option, array of strings for multiple): that value is applied. <br/> - set to `null`: strictly cleared, ignoring the parameter's default. <br/> - omitted (or set to `undefined`): falls back to the parameter's default (or `null` if it has no default). <br/> <br/> - Combining [initialParameters](./api/InteractiveDashboardProps.md#initialparameters) and [hiddenParameters](./api/InteractiveDashboardProps.md#hiddenparameters) to filter data on the frontend is a [security risk](https://www.metabase.com/docs/latest/embedding/sdk/authentication.html#security-warning-each-end-user-must-have-their-own-metabase-account). <br/> - Combining [initialParameters](./api/InteractiveDashboardProps.md#initialparameters) and [hiddenParameters](./api/InteractiveDashboardProps.md#hiddenparameters) to declutter the user interface is fine. |
| <a id="onload"></a> `onLoad?` | (`dashboard`: [`MetabaseDashboard`](./api/MetabaseDashboard.md) \| `null`) => `void` | Callback that is called when the dashboard is loaded. |

View File

@@ -45,6 +45,8 @@ If your admin has turned on [two-factor authentication](./two-factor-authenticat
6. Enter the six-digit code from the app.
7. Save your recovery codes. Metabase shows these ten codes only once, so copy them and store them somewhere safe. Each recovery code only works once. If you [regenerate recovery codes](#regenerate-recovery-codes), the old codes will no longer work.
If your admin requires 2FA, Metabase prompts you to set up 2FA the next time you log in.
### Logging in with two-factor authentication
Once you set up two-factor authentication, Metabase will ask for a code every time you log in. After you enter your email and password:
@@ -61,6 +63,8 @@ Recovery codes are single-use, so it's worth generating a fresh set if you're ru
In your account settings, on the **Authentication** tab, click **Disable**. Your account will be protected by your password only, and your recovery codes will stop working.
If your admin requires 2FA, you can't disable it.
## Account login history
The login history lists each login, along with some location information (if available), and some client information (like Browser (Firefox/Windows)). If you see any suspicious login attempts, change your password and notify your administrator.

View File

@@ -13,14 +13,21 @@ Metabase's native 2FA applies to password logins and LDAP logins. 2FA for Single
## Turn on two-factor authentication
An admin can turn 2FA on for your Metabase:
To enable two-factor authentication:
1. Go to **Admin settings** > **Settings** > **Authentication**.
2. Find the **Two-factor authentication** card.
3. Toggle it to **Enabled**.
Once enabled, a **Two-factor authentication** section shows up in each person's account settings under the **Authentication** tab, where they can enroll in 2FA.
1. Go to **Admin** > **Settings** > **Authentication** > **Overview**.
2. Scroll to the **Two-factor authentication** card.
3. Enable the **Allow two-factor authentication** toggle. Once enabled, people who log in with a password or LDAP can enroll in 2FA from the **Authentication** tab in their account settings.
4. In **Require two-factor authentication**, select an option. You must set up 2FA for your own account before you can require it.
- **Don't require**: People can enroll in 2FA, but they don't have to. This is the default.
- **Require now**: Metabase logs out everyone who hasn't logged in with 2FA. Anyone who hasn't enrolled sets up 2FA at their next login.
- **Require by a certain date**: Under **Enrollment deadline**, select a date. When the deadline passes, 2FA is required for everyone and Metabase logs out anyone who hasn't logged in with 2FA.
If you configure Metabase through environment variables or a [config file](../configuring-metabase/config-file.md), the matching setting is [`MB_MFA_ENFORCEMENT`](../configuring-metabase/environment-variables.md#mb_mfa_enforcement). Set `MB_MFA_ENFORCEMENT` to `optional` to let people enroll, or `off` to turn 2FA off.
Requiring 2FA doesn't affect API keys.
If you configure Metabase through environment variables or a [config file](../configuring-metabase/config-file.md), the matching setting is [`MB_MFA_ENFORCEMENT`](../configuring-metabase/environment-variables.md#mb_mfa_enforcement). Set `MB_MFA_ENFORCEMENT` to `optional` to let people enroll, `required` to make enrollment mandatory, or `off` to turn 2FA off.
To set the enrollment deadline, use [`MB_MFA_REQUIREMENT_DEADLINE`](../configuring-metabase/environment-variables.md#mb_mfa_requirement_deadline).
## Supported methods
@@ -44,7 +51,7 @@ Metabase requires people to use an email address as a login, but that doesn't me
## See who's enrolled
When 2FA is on, go to **Admin settings** > **Settings** > **Authentication** > **Two-factor authentication**. The 2FA card shows how many people have enrolled in 2FA and how many haven't.
When 2FA is on, go to **Admin** > **Settings** > **Authentication** > **Overview** and scroll to the **Two-factor authentication** card. The card shows how many people have enrolled in 2FA and how many haven't. Click a count to see the list of people.
## Further reading

View File

@@ -608,6 +608,7 @@ The Topic column on the [Activity log](#activity-log) model takes one of:
- measure-update
- mfa-disabled
- mfa-enrolled
- mfa-required-enrollment-failed
- mfa-verification-failed
- notification-create
- notification-unsubscribe
@@ -664,6 +665,7 @@ The Topic column on the [Activity log](#activity-log) model takes one of:
The Query Source column on the [Query log](#query-log) model takes one of:
- action
- action-execute
- ad-hoc
- agent
- cache-refresh
@@ -680,6 +682,7 @@ The Query Source column on the [Query log](#query-log) model takes one of:
- json-download
- map-tiles
- metric
- public-action-execute
- public-csv-download
- public-dashboard
- public-json-download

View File

@@ -1,9 +1,16 @@
import { StaticQuestion } from "@metabase/embedding-sdk-react";
import { useMetabaseQueryObject } from "@metabase/embedding-sdk-react/data-app";
import {
defineQuery,
useMetabaseQueryObject,
} from "@metabase/embedding-sdk-react/data-app";
import { useEffect } from "react";
const ORDERS_TABLE_ID = 5;
const OrdersQuery = defineQuery({
source: { type: "table", id: ORDERS_TABLE_ID },
});
const ALLOWED_URL = "https://allowed.data-app.test/ping";
const BLOCKED_URL = "https://blocked.data-app.test/ping";
@@ -17,9 +24,7 @@ function SandboxProbe() {
}
export default function App() {
const { query } = useMetabaseQueryObject({
source: { type: "table", id: ORDERS_TABLE_ID },
});
const { query } = useMetabaseQueryObject(OrdersQuery);
return (
<div data-testid="dev-app-content" style={{ padding: 24 }}>

View File

@@ -1,12 +1,14 @@
import {
InteractiveQuestion,
StaticQuestion,
useAction,
} from "@metabase/embedding-sdk-react";
import {
copy,
DataAppLink,
DataAppRouter,
defineAction,
defineQuery,
useAction,
useDataAppLocation,
useMetabaseQuery,
useMetabaseQueryObject,
@@ -24,9 +26,9 @@ const PROBE_COLOR = "rgb(0, 128, 0)";
function Overview() {
const { scalarQuery, questionQuery } = getTestEnv();
const ordersCount = useMetabaseQuery(scalarQuery);
const ordersCount = useMetabaseQuery(defineQuery(scalarQuery));
const totalOrders = ordersCount.data?.rawRows?.[0]?.[0];
const ordersQuery = useMetabaseQueryObject(questionQuery);
const ordersQuery = useMetabaseQueryObject(defineQuery(questionQuery));
return (
<div data-testid="data-app-content" style={{ padding: 24 }}>
@@ -82,8 +84,8 @@ function Details() {
function QueryStates() {
const { scalarQuery, errorQuery } = getTestEnv();
const broken = useMetabaseQuery(errorQuery!);
const working = useMetabaseQuery(scalarQuery);
const broken = useMetabaseQuery(defineQuery(errorQuery!));
const working = useMetabaseQuery(defineQuery(scalarQuery));
const count = working.data?.rawRows?.[0]?.[0];
return (
@@ -130,7 +132,7 @@ function DownloadQuestionPage() {
// to a custom viz — the example `demo-viz` requires exactly one result column.
function CustomVizPage() {
const { scalarQuery } = getTestEnv();
const q = useMetabaseQueryObject(scalarQuery);
const q = useMetabaseQueryObject(defineQuery(scalarQuery));
return (
<div data-testid="data-app-custom-viz" style={{ padding: 24 }}>
@@ -148,7 +150,7 @@ function CustomVizPage() {
function StaticQuestionPage() {
const { questionQuery } = getTestEnv();
const q = useMetabaseQueryObject(questionQuery);
const q = useMetabaseQueryObject(defineQuery(questionQuery));
return (
<div data-testid="data-app-static-question" style={{ padding: 24 }}>
@@ -198,7 +200,10 @@ function Actions() {
// the copy the definition points at is the authored action itself.
const action = useAction(
actionId
? { action: { id: actionId, parameters: [] }, copiedActionId: actionId }
? defineAction({
action: { id: actionId, parameters: [] },
copiedActionId: actionId,
})
: null,
);
const [output, setOutput] = useState("idle");

View File

@@ -1,6 +1,7 @@
import { useAction } from "@metabase/embedding-sdk-react";
import { useAction } from "@metabase/embedding-sdk-react/data-app";
import { useState } from "react";
// @ts-expect-error generated at test time
import { CreateScore } from "../actions/orders.action";
/**

View File

@@ -0,0 +1,10 @@
name: Library
entity_id: librarylibrarylibrary
slug: library
created_at: '2026-09-11T00:00:00Z'
type: library
serdes/meta:
- id: librarylibrarylibrary
label: library
model: Collection
is_remote_synced: true

View File

@@ -0,0 +1,11 @@
name: Data
entity_id: librarylibrarydatadat
slug: data
created_at: '2026-09-11T00:00:00Z'
type: library-data
parent_id: librarylibrarylibrary
serdes/meta:
- id: librarylibrarydatadat
label: data
model: Collection
is_remote_synced: true

View File

@@ -0,0 +1,11 @@
name: Metrics
entity_id: librarylibrarymetrics
slug: metrics
created_at: '2026-09-11T00:00:00Z'
type: library-metrics
parent_id: librarylibrarylibrary
serdes/meta:
- id: librarylibrarymetrics
label: metrics
model: Collection
is_remote_synced: true

View File

@@ -0,0 +1,8 @@
created_at: '2026-09-11T00:00:00Z'
creator_id: admin@metabase.test
definition: Annual recurring revenue
entity_id: glossaryfixturearr001
term: ARR
serdes/meta:
- id: glossaryfixturearr001
model: Glossary

View File

@@ -19,6 +19,8 @@ export const SYNCED_COLLECTION_FIXTURE_PATH =
export const SYNCED_TRANSFORMS_COLLECTION_FIXTURE_PATH =
Cypress.config("projectRoot") +
"/e2e/support/assets/example_synced_transforms_collection";
export const SYNCED_LIBRARY_FIXTURE_PATH =
Cypress.config("projectRoot") + "/e2e/support/assets/example_synced_library";
// Copy the sample synced collection from the fixture folder to the working directory
export const copySyncedCollectionFixture = () => {
@@ -34,6 +36,13 @@ export const copySyncedTransformsCollectionFixture = () => {
destination: LOCAL_GIT_PATH,
});
};
// Copy the sample remote-synced Library (with one glossary entry) from the fixture folder to the working directory
export const copySyncedLibraryFixture = () => {
cy.task("copyDirectory", {
source: SYNCED_LIBRARY_FIXTURE_PATH,
destination: LOCAL_GIT_PATH,
});
};
export const checkoutSyncedCollectionBranch = (branch: string) => {
cy.exec("git -C " + LOCAL_GIT_PATH + ` checkout -b '${branch}'`);

View File

@@ -2,7 +2,11 @@ import { updateSetting } from "e2e/support/helpers";
const SNOWPLOW_URL = "http://localhost:9090";
const SNOWPLOW_INTERVAL = 100;
const SNOWPLOW_TIMEOUT = 1000;
// Snowplow events are POSTed asynchronously and then ingested by snowplow-micro, so a
// matching event can take a moment to become queryable via `micro/good` — especially on
// loaded CI runners or with network throttling. A short budget flakes the poll before the
// event lands; this only affects how long a *failing* assertion waits (matches return early).
const SNOWPLOW_TIMEOUT = 5000;
export const enableTracking = () => {
updateSetting("anon-tracking-enabled", true);

View File

@@ -1012,10 +1012,6 @@
"spec": "../test/scenarios/metabot/native-sql-generation.cy.spec.ts",
"duration": 24167
},
{
"spec": "../test/scenarios/metabot/transforms-codegen.cy.spec.ts",
"duration": 35428
},
{
"spec": "../test/scenarios/metrics/browse.cy.spec.ts",
"duration": 35277

View File

@@ -28,8 +28,22 @@ const DASHBOARD_B_FILTER: Parameter = createMockActionParameter({
sectionId: "number",
});
// Click-behavior target pointing at the GUI question's own QUANTITY column.
// Mirrors the shape the click-behavior UI writes for a structured question.
const GUI_QUESTION_QUANTITY_DIMENSION = [
"dimension",
["field", ORDERS.QUANTITY, { "base-type": "type/Integer" }],
{ "stage-number": 0 },
];
const GUI_QUESTION_QUANTITY_TARGET_ID = JSON.stringify(
GUI_QUESTION_QUANTITY_DIMENSION,
);
describe("scenarios > embedding-sdk > internal-navigation", () => {
describe("dashboard", () => {
// Assigned while the setup commands run, before the dashcard is created.
let guiQuestionId: number;
beforeEach(() => {
signInAsAdminAndEnableEmbeddingSdk();
@@ -63,7 +77,19 @@ describe("scenarios > embedding-sdk > internal-navigation", () => {
cy.wrap(drillableQuestion.id).as("drillableQuestionId");
});
// 3. Create Dashboard B with a filter and a card
// 3. Create a GUI (MBQL) question used as a click behavior target. It
// returns a single column so the applied filter is easy to assert on.
H.createQuestion({
name: "GUI Question",
query: {
"source-table": ORDERS_ID,
fields: [["field", ORDERS.QUANTITY, { "base-type": "type/Integer" }]],
},
}).then(({ body: guiQuestion }) => {
guiQuestionId = guiQuestion.id;
});
// 4. Create Dashboard B with a filter and a card
// Dashboard B will have a click behavior linking to a question
cy.get<number>("@drillableQuestionId").then((drillableQuestionId) => {
H.createDashboard({
@@ -114,7 +140,7 @@ describe("scenarios > embedding-sdk > internal-navigation", () => {
});
});
// 4. Create Dashboard A with click behaviors
// 5. Create Dashboard A with click behaviors
cy.get<number>("@dashboardBId").then((dashboardBId) => {
cy.get<number>("@nativeQuestionId").then((nativeQuestionId) => {
H.createDashboard({
@@ -130,65 +156,120 @@ describe("scenarios > embedding-sdk > internal-navigation", () => {
limit: 5,
},
}).then(({ body: questionA }) => {
H.addOrUpdateDashboardCard({
card_id: questionA.id,
dashboard_id: dashboardA.id,
card: {
row: 0,
col: 0,
size_x: 24,
size_y: 8,
visualization_settings: {
column_settings: {
// ID column links to Dashboard B with parameter
[`["ref",["field",${ORDERS.ID},null]]`]: {
click_behavior: {
type: "link",
linkType: "dashboard",
linkTextTemplate: "Go to Dashboard B",
targetId: dashboardBId,
parameterMapping: {
[DASHBOARD_B_FILTER.id]: {
source: {
type: "column",
id: "ID",
name: "ID",
// A second, single-column question whose QUANTITY cells link to
// the GUI question. Kept separate so the column is always on
// screen, and so the other tests keep using card 0 unchanged.
H.createQuestion({
name: "Quantities for Dashboard A",
query: {
"source-table": ORDERS_ID,
fields: [
["field", ORDERS.QUANTITY, { "base-type": "type/Integer" }],
],
limit: 5,
},
}).then(({ body: quantitiesQuestion }) => {
// Both cards must be set in a single call: addOrUpdateDashboardCard
// replaces the dashboard's whole dashcard list, so a second call
// would silently drop the first card.
H.updateDashboardCards({
dashboard_id: dashboardA.id,
cards: [
{
card_id: questionA.id,
row: 0,
col: 0,
size_x: 24,
size_y: 8,
visualization_settings: {
column_settings: {
// ID column links to Dashboard B with parameter
[`["ref",["field",${ORDERS.ID},null]]`]: {
click_behavior: {
type: "link",
linkType: "dashboard",
linkTextTemplate: "Go to Dashboard B",
targetId: dashboardBId,
parameterMapping: {
[DASHBOARD_B_FILTER.id]: {
source: {
type: "column",
id: "ID",
name: "ID",
},
target: {
type: "parameter",
id: DASHBOARD_B_FILTER.id,
},
id: DASHBOARD_B_FILTER.id,
},
},
target: {
type: "parameter",
id: DASHBOARD_B_FILTER.id,
},
id: DASHBOARD_B_FILTER.id,
},
},
},
},
// PRODUCT_ID column links to native question with parameter
[`["ref",["field",${ORDERS.PRODUCT_ID},null]]`]: {
click_behavior: {
type: "link",
linkType: "question",
linkTextTemplate: "Go to Native Question",
targetId: nativeQuestionId,
parameterMapping: {
id: {
source: {
type: "column",
id: "PRODUCT_ID",
name: "Product ID",
// PRODUCT_ID column links to native question with parameter
[`["ref",["field",${ORDERS.PRODUCT_ID},null]]`]: {
click_behavior: {
type: "link",
linkType: "question",
linkTextTemplate: "Go to Native Question",
targetId: nativeQuestionId,
parameterMapping: {
id: {
source: {
type: "column",
id: "PRODUCT_ID",
name: "Product ID",
},
target: {
type: "variable",
id: "id",
},
id: "id",
},
},
target: {
type: "variable",
id: "id",
},
id: "id",
},
},
},
},
},
},
},
{
card_id: quantitiesQuestion.id,
row: 8,
col: 0,
size_x: 12,
size_y: 6,
visualization_settings: {
column_settings: {
// QUANTITY column links to a GUI (MBQL) question, mapping
// the clicked quantity onto the target's own column
[`["ref",["field",${ORDERS.QUANTITY},null]]`]: {
click_behavior: {
type: "link",
linkType: "question",
linkTextTemplate: "Go to GUI Question",
targetId: guiQuestionId,
parameterMapping: {
[GUI_QUESTION_QUANTITY_TARGET_ID]: {
source: {
type: "column",
id: "QUANTITY",
name: "Quantity",
},
target: {
type: "dimension",
id: GUI_QUESTION_QUANTITY_TARGET_ID,
dimension: GUI_QUESTION_QUANTITY_DIMENSION,
},
id: GUI_QUESTION_QUANTITY_TARGET_ID,
},
},
},
},
},
},
},
],
});
});
});
});
@@ -202,6 +283,7 @@ describe("scenarios > embedding-sdk > internal-navigation", () => {
cy.intercept("POST", "/api/dashboard/*/dashcard/*/card/*/query").as(
"dashcardQuery",
);
cy.intercept("POST", "/api/dataset").as("datasetQuery");
});
it("should pass parameters to the linked dashboard", () => {
@@ -273,6 +355,71 @@ describe("scenarios > embedding-sdk > internal-navigation", () => {
});
});
it("should pass the click behavior filter to the linked GUI (MBQL) question", () => {
cy.get<number>("@dashboardAId").then((dashboardAId) => {
mountSdkContent(
<InteractiveDashboard
dashboardId={dashboardAId}
enableEntityNavigation
/>,
);
});
cy.wait("@getDashboard");
cy.wait("@dashcardQuery");
getSdkRoot().within(() => {
// Verify we're on Dashboard A
cy.findByText("Dashboard A").should("be.visible");
// Click on the custom link text that navigates to the GUI question
H.getDashboardCard(1)
.findAllByText("Go to GUI Question")
.first()
.click();
});
// A GUI target cannot consume the mapped value as a parameter, so it opens
// as an ad-hoc question carrying the value as a filter instead.
cy.wait("@datasetQuery").then(({ request }) => {
// Request body is a pMBQL query; a filter clause is
// [operator, opts, fieldRef, value].
const [{ filters }] = request.body.stages;
const [filter] = filters;
expect(filter[0]).to.equal("=");
expect(filter[2]).to.include(ORDERS.QUANTITY);
cy.wrap(filter[3]).as("filteredQuantity");
});
getSdkRoot().within(() => {
cy.findByTestId("visualization-root").should("be.visible");
// The toolbar counts the mapped value as a filter on the question.
// Before the fix the saved question opened unfiltered, so this read
// "Filter" instead.
cy.findByTestId("filter-dropdown-button")
.should("be.visible")
.and("contain", "1 filter");
// Every returned row matches the quantity we clicked on
cy.get<number>("@filteredQuantity").then((filteredQuantity) => {
H.tableInteractiveBody()
.findAllByTestId("cell-data")
.should("have.length.at.least", 1)
.each((cell) => {
expect(cell.text()).to.equal(String(filteredQuantity));
});
});
// Verify breadcrumb shows Dashboard A, and that going back works
cy.findByText("Back to Dashboard A").should("be.visible").click();
cy.findByText("Dashboard A").should("be.visible");
});
});
it("should navigate to the question when clicking on the dashcard title", () => {
cy.get<number>("@dashboardAId").then((dashboardAId) => {
mountSdkContent(

Some files were not shown because too many files have changed in this diff Show More