Compare commits

...

363 Commits

Author SHA1 Message Date
Stas Gavrylov
6ecffd0c9b Regenerate SDK host app lockfiles with npm 11 2026-08-25 08:44:05 -07:00
Stas Gavrylov
0a96d3d9dd Upgrade Node.js to 24 LTS 2026-08-25 07:29:23 -07:00
Riley Thompson
45516ce5cf Fix string to boolean literal coercion (#80093)
fix literal coercion
2026-08-25 10:03:43 -04:00
dpsutton
4917982a2d better development against audit db (#80349)
* better development against audit db

* Bump :discouraged-namespace kondo ratchet to 81

connection.clj now requires metabase.audit-app.core with a
:discouraged-namespace ignore (audit-db dev support), so the budget
needs +1.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BjQcjQx7NPGr3Rpf6QaG4b

---------

Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-08-25 08:17:31 -05:00
Jakub Chodorowicz
a4fa8cf2ca harden the custom viz dev server url handling (#80222) 2026-08-25 14:04:21 +02:00
Fraser
b3f2191d7b Prepare visualizations for the viz-core split (#80784) 2026-08-25 23:29:47 +12:00
Chris Truter
858eee14df Add hierarchy visualization renderers (#80618)
* Add hierarchy visualization utilities

* Normalize hierarchy visualization label newlines

* Preserve ordered hierarchy traversal in visualizations

* Format hierarchy utility docstrings

* Move hierarchy visualization utilities to dev

* Move hierarchy visualization tests alongside the dev namespace

Also sort nodes unreachable from a root, so malformed hierarchies still
render in a stable order, and mark the hierarchy tests parallel.

* Keep hierarchy visualization tests discoverable by the test runner

The runner skips `dev` directories and non-`metabase.*` namespaces, so a
`dev.*` test never runs. Follow `metabase.core.modules-test` and
`metabase.core.kondo-ratchet-test`, which test dev namespaces from a
`metabase.*` test namespace, and say so in the namespace docstring.

* List hierarchy nodes without drawing a tree

`nodes` returns every tag as a flat vector, leaves-to-roots by default so a node
precedes its parents, or alphabetically on request. Ordered hierarchies break ties
by derivation order and the rest fall back to `pr-str`, both of which `hierarchy->graph`
already worked out, so there is no separate code path for the two kinds.

A test pins the topological order to `ordered-hierarchy/sorted-tags` so the dev output
cannot drift from the order the runtime uses.
2026-08-25 13:20:48 +02:00
Sébastien
5ae667c0b0 Fix flaky JWT user-session e2e in sdk iframe embed setup get-code (#80642)
* Fix flaky JWT user-session e2e in sdk iframe embed setup get-code

* Assert terms acceptance with a plain not.exist check
2026-08-25 13:08:50 +02:00
Chris Truter
a5bd647dc4 Fix ordered hierarchy ordering and validation (#80720)
* Fix ordered hierarchy ordering and validation

* Format hierarchy utility docstrings

* Mark ordered hierarchy tests parallel

* Return nil from first-common-ancestor when tags share no ancestor

Passing a tag with no ancestors as the second argument made the lookup call nil as a
predicate, so disconnected roots and tags outside the hierarchy threw an NPE instead of
returning nil.

* Reject children declared after a tag's first occurrence in any basis

The check only ran for nested vectors, so a tag introduced in one basis could pick up
children in a later one. Hoisting it into derive-basis applies the same rule to every
top-level basis.

* Ratchet down kondo ignore budgets

---------

Co-authored-by: Metabase Automation <github-automation@metabase.com>
2026-08-25 12:49:56 +02:00
Nicola Mometto
79c18e5f89 Check ChannelTemplate write permission on notification handler templates (#80469)
The notification create/update endpoints let a handler attach an existing
ChannelTemplate by id, or carry an inline template that creates, overwrites,
or deletes one, but didn't apply the permission check used when writing a
ChannelTemplate directly. Gate any such template write on
mi/can-write? :model/ChannelTemplate, mirroring the existing inline-channel
check. Seeding and the pulse->notification migration go through the model
layer directly, so they're unaffected.
2026-08-25 10:07:57 +00:00
Nemanja Glumac
4bcfa01f00 Fix kondo ratchets total ignored forms count (#80811) 2026-08-25 11:33:56 +02:00
Fraser
dd1cc22c22 Early exit the loki stress test when every changed story is loki-skipped (#80790) 2026-08-25 20:51:51 +12:00
Nicola Mometto
421f547421 Require native query permission to author native query actions (#80612)
POST /api/action and PUT /api/action/:id now require the current user to
have ad-hoc native query permission on the target database when the
submitted dataset_query is native.
2026-08-25 09:24:19 +01:00
Fraser
4071663479 Tidy module side-effects lint rule (#80734) 2026-08-25 14:12:47 +12:00
Romeo Van Snick
b9846a5b64 Load enterprise route pages through plugin route slots (#80268)
* Load the enterprise admin pages through plugin route slots

* Load the tenant collection pages through plugin route slots

* Resolve every plugin route slot in a test

* Name the chunks the plugin route slots load
2026-08-25 12:12:02 +10:00
Nemanja Glumac
adf654542a Remove ability to skip test suites upfront (#80760) 2026-08-24 17:49:51 -06:00
Fraser
e8b4febba7 Move session and subscription request types into metabase-types/api (#79346) 2026-08-25 11:27:19 +12:00
Fraser
ecdaad1ddc Prepare the visualization types for a viz-core module (#80404) 2026-08-25 11:25:00 +12:00
Alexander Polyankin
1fbd338057 Check revision revert perms against the table of the restored definition (#80553)
Check revert perms against the table of the restored definition

For Segment and Measure reverts, table_id is re-derived from the
definition on update, so check write perms against the source table of
the restored revision's definition when it has one, falling back to the
revision's stored table_id otherwise.

Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-08-24 19:05:04 -04:00
Alexander Polyankin
a408a8822c Default undeclared endpoint params to an empty map schema (#80575)
* Default undeclared endpoint params to an empty map schema

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* Ignore unparsed request bodies instead of declaring them

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* Pass multipart params as the request body of multipart endpoints

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* Declare query and route params read by dashboard and database endpoints

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* Ignore the kebab-case linter for newly declared snake_case query params

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-08-24 19:04:46 -04:00
Ben Grabow
2258a06f60 Don't create nested transaction which would block view of task progress (#80402) 2026-08-24 17:29:18 -05:00
github-automation-metabase
ad801f22e9 Auto-update documentation for master (#80733)
Update generated docs (master)

Co-authored-by: github-actions <github-actions@github.com>
2026-08-24 21:22:09 +00:00
Alex Yarosh
037092060e docs: remove billing page (#80722)
* remove billing

* tending to links
2026-08-24 17:00:29 -04:00
Jeff Bruemmer
93d7163cd2 docs - embedding dashboards (#79915) 2026-08-24 19:28:20 +00:00
Anderson
0361fce560 Fix flaky remote-sync conflict-modal e2e test (#80396)
* Anchor remote-sync conflict-modal test on the async conflict lifecycle

* Reword test comments per writing conventions

* Trim comments and move timeouts to request/test-level options

* Use navigation helper when navigating to the transforms page

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>

* Fail fast on terminal sync task statuses

---------

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
2026-08-24 15:52:21 -03:00
Nemanja Glumac
5e53c6b689 Reduce flakiness of table-editing e2e tests (#80598) 2026-08-24 20:52:18 +02:00
Ben Grabow
6fa6eee688 Debounce tracking table writes (#80662) 2026-08-24 18:35:28 +00:00
github-automation-metabase
8339e57d2c Fix detached-DOM flake in upload table deletion e2e test (#78293)
* Fix detached-DOM flake in upload table deletion e2e test

Re-query the upload-tables-table container on each assertion instead of
caching it via within(). The table re-renders when getUploadTables
resolves, detaching the node within() captured, so a retrying query
against the stale subtree got stuck on an intermediate render.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* Appease the linter

---------

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-authored-by: Nemanja <31325167+nemanjaglumac@users.noreply.github.com>
2026-08-24 20:32:30 +02:00
Nemanja Glumac
4b8fd9848e Cover DB management permissions with targetted tests (#80594) 2026-08-24 20:25:17 +02:00
Phoomparin Mano
4852413258 Check embedding saml popup source (#80494)
* check saml popup origins

* throw when site-url does not match instance-url

* Fix SAML embedding popup validation
2026-08-24 17:39:25 +00:00
Nemanja Glumac
eb401d8b6e Skip drivers-decision analysis on force-run (#80617) 2026-08-24 19:31:26 +02:00
Chris Truter
59b8696296 Fix test-user caching across appdb swaps (#80436)
* Key cached session tokens on the app db

with-empty-h2-app-db! swaps in a different application database, and this cache was keyed on the username
alone. A token minted against the previous database names a session that does not exist in the new one, so
the request 401s, and the token minted inside leaks back out when the original is restored. The retry in
client-fn cleared the whole cache and tried again, which papered over it often enough to look like flake:
"Failed to authenticate X after three tries".

Key it on the app db as well, like every neighbouring cache does via memoize-for-application-db.

databases-list-can-upload-test and dismiss-spinner-test go from failing to passing; both were in the CI
failure list, the former across 12 jobs.

* Track globally-created test users per application database

with-empty-h2-app-db! swaps in an empty app db, where the users another database created do not exist.
`created-globally?` was a boolean, so it stayed true across the swap and user->id kept handing back ids from
the previous database. Authenticating then inserted a Session for a user that was not there:

  Failed to authenticate :crowberto after three tries: Referential integrity constraint violation:
  FK_SESSION_REF_USER_ID: PUBLIC.CORE_SESSION FOREIGN KEY(USER_ID) REFERENCES PUBLIC.CORE_USER(ID) (1)

Track the set of app dbs that have them instead. The memo beside it was already keyed this way via
memoize-for-application-db; only the flag was not.

databases-list-can-upload-test failed 8/8 before this and 0/8 after; the whole namespace now passes.

* Say what the token cache holds
2026-08-24 19:20:47 +02:00
metabase-bot[bot]
ff25772d0c [chore] Update E2E test timings with recent data (#79933)
Update E2E test timings with averaged data

Collected timing data from 7 days of test runs.
All specs with timing data were included (simple averages).

🤖 Generated automatically by update-e2e-timings workflow

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: GitHub Actions <actions@github.com>
Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: iethree <30528226+iethree@users.noreply.github.com>
2026-08-24 13:13:09 -04:00
Nicola Mometto
4a47fda8d8 Reject impersonation role equal to the driver default-role sentinel (#80616)
Reject impersonation role attribute equal to the driver default-role sentinel

connection-impersonation-role now rejects a role attribute that (case-insensitively)
equals the driver's default-database-role sentinel (e.g. NONE / DEFAULT), which would
otherwise emit SET ROLE NONE and drop the connection to the base warehouse credential.
2026-08-24 17:24:16 +01:00
Alexander Polyankin
70504458f6 Strip template tags from public and embed native queries (#80574)
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-08-24 12:19:28 -04:00
Alexander Polyankin
54d059982c Resolve card UI parameter fields from param_fields for saved cards (#80569)
* Resolve card UI parameter fields from param_fields for saved cards

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* Resolve parameters against the query for questions opened from a dashboard

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-08-24 12:19:27 -04:00
Alexander Polyankin
2350e6a952 Remove unused data from embed and public endpoints (#80510)
* Remove unused data from embed and public endpoints

Public and static-embed card/dashboard/document endpoints now return a
minimal placeholder dataset_query (a blank native query against the same
database) instead of selecting individual query keys. The frontend only
needs the database id here. Parameter combining and :param_fields
hydration in public-card now run before the card is trimmed since both
read template tags from the full query.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* Add tests for embed card and dashboard dataset_query

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* Assert strictly on dataset_query in public and embed tests

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* Handle native queries in Table viz

* Fix column shadowing in HeaderCellWithColumnInfo

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* Use param_fields for public question parameters

Public and static-embed card payloads carry param_fields instead of the
real query, so PublicOrEmbeddedQuestion now builds UI parameters from
them via getCardUiParametersFromParamFields, mirroring the saved-mode
dashboard path. The query-based helpers are renamed to the FromQuery /
FromParamFields scheme on both the card and dashboard sides, and the
ParameterWithTarget casts are dropped now that getParameterTargetField
accepts an optional target.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* Use param_fields for embed setup card parameters

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* Use param_fields for guest embed SDK question parameters

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* Route card UI parameters by param_fields presence

Restores getCardUiParameters as the entry point: it delegates to
getCardUiParametersFromParamFields when the card carries param_fields
(public and static-embed payloads) and to the query-based variant
otherwise. The SDK SQL parameter surfaces use it since they serve both
guest and authenticated contexts. Also reverts the dashboard helper
renames back to getSaved/getUnsavedDashboardUiParameters.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* Fix preview embed card param values

The preview-embed card values and remapping endpoints get their card via
card-for-unsigned-token -> public-card, which now replaces the query with
a blank native placeholder, so field-filter parameters could no longer be
resolved and the response schema rejected the nil result. Bind
qp.perms/*param-values-query* around the card fetch, the escape hatch
remove-card-non-public-columns already honors, matching the public
sharing values endpoints.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* Add preview embed test for remapped parameter fields

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* fix

* vscode fix

* Backfill template tag parameters in param_fields UI parameters

GET /api/card/:id returns parameters as stored, unlike the public and
embed endpoints which combine them with template tags, so
getCardUiParametersFromParamFields now derives parameters via
getParametersFromCard like the query-based variant always did. Guest
embed payloads are unaffected: their parameters arrive already combined
and the blank query derives nothing.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* Use getCardUiParameters dispatcher in SDK load-question

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* Guard applyTemplateTagParameters by param_fields presence

Cards whose payloads carry param_fields have parameters already combined
with template tags by the backend, so re-applying from the query would
wipe them. Keying the guard on the card payload keeps it in sync with
how the card was loaded, replacing the isGuestEmbed flag checks.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* Use getCardUiParameters everywhere and build test queries via Lib

Question.parameters and the query builder call sites go back to the
getCardUiParameters entry point, which now routes by param_fields
presence, leaving those files unchanged relative to master. The
template tag fallback test builds its query with
Lib.createTestJsNativeQuery instead of an inline legacy map.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* Make getCardUiParameters the only entry point

The FromQuery and FromParamFields variants are private to cards.ts now;
all consumers call getCardUiParameters, which routes by param_fields
presence.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* Build preview embed remapped-fields test query via lib

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* Restore null fallback in useAvailableParameters

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* Fix

* Preserve query type in stripped public and embed card queries

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* Add placeholder aggregations to stripped MBQL queries

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* Keep parameter template tags in stripped native queries and revert FE workarounds

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* Export parameter-template-tag? via import-vars

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* Add test for :param_fields hydration on GET /api/card/:id

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-08-24 12:19:27 -04:00
Andrei Chernykh
c0ed9ab9af Remove unused @types/ packages (#80568)
Remove orphaned @types stubs, move @types/js-yaml to devDeps
2026-08-24 08:50:48 -07:00
Chris Truter
6482f14a58 Respect Metabot permission mode when resolving access (#80468)
* Clear All tenant users metabot permissions when switching to group-level mode

Switching metabot permissions to group-level only wiped the All Users magic group's rows.
Under tenants the simple mode also writes to All tenant users, and since resolution takes the
most permissive value across a user's groups, a leftover :yes there overrode every group-level :no.

POST /advanced now clears both magic groups; DELETE /advanced keeps both instead of dropping the
tenant toggle. The confirmation dialog names both groups when tenants are on.

Fixes #80394

* Backfill the tenant permission cleanup and make both mode-switch dialogs tenant-aware

Instances that switched to group-level mode before the fix still carry the seeded all-yes rows
for All tenant users, so a migration clears them wherever metabot-advanced-permissions is true.

The disable dialog now says both magic groups keep their access under tenants, and both dialogs
use the "All internal users" name the permissions UI shows when tenants are on.

* Explain why the tenant-permission migration only reads the setting row

* Move the tenant-permission migration to 063 for the single backport

* Move the tenant-permission migration to 061 so it backports to every affected release

* Scope metabot permission resolution to the groups the current mode shows

Resolution took the most permissive value across every group a user belongs to regardless of
mode, so any row on a group the admin UI hides -- the All Users and All tenant users rows in
group-level mode, the seeded data-analyst rows and any API-written group in simple mode --
silently overrode every visible :no. The write-site fixes (deleting rows on POST /advanced, the
backfill migration gated on the setting row) only covered the paths that went through the
endpoint; PUT /api/setting, serdes import and PUT / on a hidden group all bypassed them.

Filter in the resolver instead: simple mode only counts Administrators, All Users and All tenant
users; group-level mode counts everything but those two default groups. Hidden rows stay in the
table and come back into force on switching modes, so POST /advanced no longer deletes them and
DELETE /advanced genuinely keeps them, which also restores the simple-mode values after a round
trip instead of leaving the default groups at all-no. With leftover rows inert the migration is
no longer needed.

* Leave the mode-switch dialog copy as it was

* Refuse the metabot permission mode switch while an env var forces the setting

The setter writes the row regardless, but the env var keeps winning on read, so DELETE /advanced would
delete every group's rows and leave the instance in group-level mode with nobody but admins allowed in.

* Flip the metabot permission mode before dropping the group-level rows

Deleting first meant a failed setting write left group-level mode in place with its permissions already gone. In the new order a failed delete only leaves rows that simple mode ignores.

* Say in the docstrings that resolution only sees the active mode's groups

* Clear the default-group rows again when switching to group-level mode

Preserving them so they returned on a switch back was a silent widening: the 061 migration seeds every default group all-yes, so a round trip through group-level mode handed everyone AI access again, including the tenant rows that caused this bug.

Both mode-switch dialogs promise removal, and nothing in the docs describes a mode remembering its old configuration, so the modes now delete the rows the other one owns, as before. Resolution stays scoped to the active mode's groups: that is what makes rows created by other write paths, and the rows already sitting on instances in the field, harmless without a migration.

* Switch the metabot permission mode in one transaction

Ordering the two writes only chose which half-applied switch an admin got: the mode changed while the rows did not, or the rows went while the mode stayed. The setting write goes last inside the transaction, since it updates the settings cache in place and a rollback would leave the new value there.

* Reload the settings cache when a mode switch rolls back

The setting write updates the cache in memory, which the rollback does not undo, so a commit failure would leave this node serving a mode the database does not have.

* Cover the settings cache in the mode-switch rollback test

The redefined setter now writes the setting for real before throwing, so the test fails if the rollback leaves the new mode in the cache.

* Extract the mode's group list to one definition

Both the resolution query and the mode-switch endpoints had their own copy of the
Administrators / All Users / All tenant users list, and read as opposites: one
selected with :in, the other deleted with :not-in. They now share
visible-groups-clause, with hidden-groups-clause defined as its complement.

* Improve Metabot permissions documentation

* Address Metabot permissions review feedback
2026-08-24 15:01:14 +00:00
Tyler Plude
3cd5fc8e1e [BOT-1899] Metabot DeepSeek direct provider support (#80054)
* initial deepseek support

* [BOT-1899] Port DeepSeek onto the LLM provider connection registry (#80100)

Wire DeepSeek into the LLM provider connection registry

DeepSeek arrived as an adapter reading its own credential settings, which is how
providers worked before the connection list. Make it a connection type like the
rest:

- `deepseek` joins `metabase.llm.provider`'s registry with the api-key and
  base-url fields the admin connection form renders itself. No key prefix:
  DeepSeek keys carry the same `sk-` an OpenAI key does, so matching on it would
  claim OpenAI keys and reject nothing.
- `llm-deepseek-api-key` / `-api-base-url` become the type's single-provider
  environment variables, reading and writing through the connection they
  configure like every other provider's.
- The adapter takes `:credentials` off the request instead of reading settings,
  so two DeepSeek connections can hold different keys.
- The frontend mirrors the new type name and gets a DeepSeek mark.

The app-db migration that folds the old per-provider settings into the
connection list is a frozen snapshot of the registry as it stood then, and stays
untouched: no released version wrote a DeepSeek setting, so there is nothing to
adopt, and a rollback drops the connection because the versions it rolls back to
cannot serve DeepSeek either. A test pins that.

Regenerated the settings docs, and listed DeepSeek among the providers you can
bring a key for.

* revert docs changes

* review feedback

---------

Co-authored-by: Maksym Yakubych <maksym@metabase.com>
Co-authored-by: Maksym Yakubych <undrfined@gmail.com>
2026-08-24 10:35:50 -04:00
Kamil Mielnik
5f8ff14492 GDGT-3141 [Flaky Test]: should render saved top level blocks (#80614) 2026-08-24 21:24:04 +07:00
Sloan Sparger
185e6d7878 [Metabot] Add prompt toggle button to the SQL editor header (#80379) 2026-08-24 11:08:59 -03:00
Stas Gavrylov
a90f1170ba GDGT-2460 Update Toast styles (#80327)
* GDGT-2460 Update Toast styles

* GDGT-2460 Port Toast to Mantine Notification

* Update Loki Snapshots

---------

Co-authored-by: Metabase Automation <github-automation@metabase.com>
2026-08-24 16:46:10 +03:00
github-automation-metabase
3af49a24f7 Auto-update documentation for master (#80555)
Update generated docs (master)

Co-authored-by: github-actions <github-actions@github.com>
2026-08-24 09:39:32 -04:00
Chris Vandoodewaard
c15af0329d update driver-test-results with missing driver tests (#80551)
* add dependency on all driver tests

* sort the list
2026-08-24 13:28:01 +00:00
Chris Vandoodewaard
19747d61b0 use CI Conductor base url from vars instead of secrets (#80552) 2026-08-24 09:23:59 -04:00
Kevin Kelleher
3235182102 Remove group prefixes from Explorations, use curated dimension display name (#79603)
* Remove group prefixes from dimension labels, use curated display-name

* fix error after merge
2026-08-24 13:14:04 +00:00
github-automation-metabase
e1ed14d77f Fix flaky native database source selectDatabase detach (#78266)
* Fix flaky native database source selectDatabase detach

The database picker popover opens automatically and re-mounts its rows as the database list finishes loading, which could detach the target row while Cypress waited for it to become actionable ("elements ... disappeared from the page"). Anchor the click in selectDatabase behind a visibility assertion so Cypress re-queries the popover chain on detach and clicks a freshly attached, settled row.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* Force-click the native database picker row to avoid detach

The database picker popover opens automatically and, as GET /api/database
resolves, its onQueryStarted hook re-writes shared metadata. That hands the
selector a fresh databases array reference and re-mounts the AccordionList rows.
The previous visibility-anchored click still entered Cypress's actionability
wait, during which the target row detached ("elements ... disappeared from the
page"). Click the row with { force: true } so Cypress fires the moment the row
is found and visible, skipping the actionability wait window in which the
re-mount can detach it.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-authored-by: Uladzimir Havenchyk <125459446+uladzimirdev@users.noreply.github.com>
2026-08-24 14:18:45 +02:00
Paolo Rascunà
638d0e4cb2 BOT-1606: Truncate over-long Metabot answers instead of going silent (#80105)
* BOT-1606: Truncate over-long Metabot answers instead of going silent

`channel/final-text-blocks` put the whole answer into a single Slack `section`
block. Slack caps a section's `text.text` at 3000 characters, so any longer
answer came back as `invalid_blocks` ("must be less than 3001 characters") and
the thread simply went quiet -- no answer, no error, nothing to retry.

Elide the answer at the limit, and when that actually cuts it, follow it with a
`context` block saying so and linking to the full answer in Metabase. The notice
is deliberately kept out of the message's `:text`, which `thread->history`
replays back to the model as the assistant's own words.

The answer stays a single mrkdwn `section` block, as before. Answers that already
fit are byte-for-byte unchanged: `u.str/elide` returns its input untouched.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* BOT-1606: Address review comments on the truncation notice

Works through appleby's review nits on #80105:

- Docstring summary for `truncation-notice` is one sentence (continuation
  aligned to 3 spaces, matching the rest of the file).
- The notice now names the limit, so a reader can judge how much to trim:
  "This answer was longer than 3000 characters."
- Adopted the suggested "...and try again." ending.
- Added a debug log on the truncation path, recording the answer length
  against the limit.
- Moved the shared `oversized-answer` fixture into `slackbot.test-util`;
  `channel_test` and `api_test` had identical copies.
- `slack-section-text-limit` now derives from the production constant
  rather than restating it, which makes the drift test redundant.

`log/debugf` evaluates its args only once the level is enabled, so the
debug log was verified through the REPL with log capture on rather than
through the suite, which cannot reach it.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Ew4fCZUQC2fu4MZ7JamMLm

* BOT-1606: Cap the visualization title so Slack accepts the message

The truncation shipped so far only guarded the answer block. The error in
BOT-1606 names `/blocks/1`, which is the visualization title block, not the
answer: `format-viz-title` embeds a `/question#<base64-query>` link, and for
a large query that base64 alone runs past Slack's 3000-character section
limit, so `chat.postMessage` rejects the whole message.

Reproduced with a native query of the size Metabot routinely writes: a
7,119-character SQL string yields a 9,770-character link and a 9,820-character
title block, and Slack returns the reported error verbatim.

A cut URL is a broken URL, so drop the link rather than truncate it and keep
the title; with no title left, `viz-output->blocks` already falls back to
"Query results". `section-text-limit` becomes public for this -- streaming
already required the channel namespace, so no new dependency.

Both this and the answer truncation log at `info` rather than `debug`:
`metabase.slackbot` inherits INFO from log4j2.xml, so a debug line there is
invisible on a default instance. Link drops get their own counter rather than
reusing `responses-truncated`, which counts responses; a long answer and a
long query link are different problems and worth telling apart.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Ew4fCZUQC2fu4MZ7JamMLm

* BOT-1606: Fall back to plain text when Slack rejects the blocks

`send-channel-response` logged a failed `chat.postMessage` and stopped
there, so a user whose message Slack refused saw the "is doing science..."
status and then nothing. `send-dm-response` already retries with a
plain-text reply; do the same here, with the same wording.

A plain-text reply carries no blocks for Slack to reject, so it survives
whatever made the block message fail -- including causes we have not
anticipated and so cannot cap the way the answer and viz title are capped.

Add `responses-undeliverable`, incremented from both paths when the
fallback fails too. It counts only total delivery failure: a response the
fallback rescues is deliberately not counted, so the metric answers "how
often did a user get nothing at all", which is the alertable question.

Left the error-cleanup fallback in `send-dm-response` uncounted -- that
path is already reporting a thrown exception, and its fallback carries an
error notice rather than an answer.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Ew4fCZUQC2fu4MZ7JamMLm

* BOT-1606: Cap alt_text, and stop miscounting title-only elisions

Follow-ups from a Clojure review of the branch.

Slack caps an image block's `alt_text` at 2000 characters, not the 3000 it
allows a section block, and `viz-output->blocks` passed the raw title
straight through. A title of 2001-3000 characters therefore cleared
`format-viz-title` untouched -- comfortably inside the section limit -- and
then failed `chat.postMessage` with `invalid_blocks`: the same silence this
branch exists to fix, reachable through the function it hardened. Cut the
title with `limit-chars` rather than `elide`, since alt text is plain and an
ellipsis would only eat into the budget.

`format-viz-title` also reported a dropped link whenever the rendered title
was over the limit, including when there was no link at all -- logging
`link_length=0` while claiming a drop, and incrementing `viz-links-dropped`
for something that never happened. Both cases still elide the title; only the
reporting now branches.

Docstrings: several first lines ran on into the next line, which
https://guide.clojure.style/#docstring-summary asks not to happen. Also
`format-viz-title` claimed to return nil only when given neither a title nor
a link, though it also returns nil for a link too long to keep -- as the
branch's own test asserts -- and `slack-section-text-limit` restated the
sentence from the var it aliases.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Ew4fCZUQC2fu4MZ7JamMLm

---------

Co-authored-by: Paolo Rascuna <1936871+prascuna@users.noreply.github.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-24 14:01:11 +02:00
github-automation-metabase
1081fb8c02 Fix flaky alerts toggle e2e in sdk iframe embed setup select-embed-options (#78285)
* Fix flaky alerts toggle e2e: re-query preview iframe for Alerts click

The 'toggles alerts for question when email is set up' spec cached the
embed preview iframe body via within() and chained the Alerts button
visibility assertion straight into the click. Toggling the sidebar
options re-renders the embedded question, so the cached body and the
found ActionIcon detach mid-click ('the page updated while this command
was executing'). Re-query the iframe body fresh and split the visibility
assertion from the click so Cypress retries against a stable element.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* Give re-render-dependent alerts lookups a longer timeout

Iteration 1 removed the mid-click detach by re-querying the preview
iframe body, and 49/50 burn-in runs passed. The residual failure is a
different flake: toggling a sidebar option re-renders (and re-runs the
query of) the embedded question inside the already-loaded iframe, and
that round-trip occasionally exceeds Cypress's default 4s command
timeout, so findByRole for the "Alerts" button times out with only a
"group" role present in the iframe body. Give the Alerts button and
alert-modal lookups that depend on a re-render a 15s budget so Cypress
waits the slow re-fetch+render out instead of asserting mid-render.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-authored-by: Sébastien <sebastien@metabase.com>
2026-08-24 13:13:39 +02:00
github-automation-metabase
4ba63550a4 Harden embedding-hub sandbox column-update e2e flake (#78620)
The "should update existing sandboxes when changing column selection"
test raced in two ways:

- It waited on @updatePermissionsGraph with the default 5s request
  timeout, but the "Next" click first runs several sequential metadata
  fetches (list tables, list databases, permissions-graph revision)
  before issuing that PUT, so under load the request can appear later
  than 5s and the wait times out. Give it a 15s timeout.

- It asserted H.undoToast().should("not.exist") with no positive anchor,
  which can pass against an unrendered page. Anchor it on the positive
  success signal ("Select data" step marked complete) first.

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-authored-by: Sébastien <sebastien@metabase.com>
2026-08-24 13:00:31 +02:00
Phoomparin Mano
582d66e9c1 Allow embedding sdk target for saml popups (#80479)
* check for origins in saml popups

* log origin of login
2026-08-24 10:46:48 +00:00
Ngoc Khuat
04b84f8b77 Add connection and read timeouts to python-runner requests (#80080)
* Add connection and read timeouts to python-runner requests

* Tighten timeout docstrings

* Cancel the runner on a client timeout, retry transient log-poll timeouts

Review follow-ups on the python-runner timeouts:
- a client-side timeout on /execute now fires /cancel, so the runner
  stops working on a run nobody is waiting for
- a read timeout while polling /logs is retried instead of ending the
  poll loop for the rest of the run
- use u/seconds->ms instead of inline math
2026-08-24 16:34:27 +07:00
YB
9adb94e945 [GDGT-3085] adhere to HTTP put spec for notification API (#79821)
PUT /api/notification/:id accepted a different id from what was in the URL which diverges from normal PUT behaviour. This fix aligns it with how we've handled PUT updates in this codebase by only handling allowlisted params for the update.
2026-08-24 09:10:26 +00:00
github-automation-metabase
9139563930 Deflake SDK question data-picker target-collection test (#78105)
* Deflake SDK question data-picker target-collection test

The "change target collection to a different entity id" test blindly
pressed Escape then clicked "Pick your starting data" to open the data
picker. In the embedding SDK the DataStep always renders the data picker
and auto-opens it when no source table is selected, and that open state
settles asynchronously after the collection re-render. When the popover
auto-opened after the blind Escape, the follow-up click toggled it back
closed, so H.popover() never appeared and the Orders lookup timed out.

Wait until the popover is actually gone after Escape before reopening it
from a known-closed state, and use visibility anchors instead of exist,
removing the toggle race.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* Deflake SDK question data-picker target-collection test

The embedding data picker uses isInitiallyOpen={!table}; with questionId="new"
no source table is ever selected, so the picker auto-opens on every re-render and
stays open. The prior approach toggled it (esc to close, click to reopen), which
raced the async auto-open: sometimes the click toggled the just-opened popover
shut, sometimes esc failed to close it at all. Interact with the already-open
popover directly to remove the toggle race.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-08-24 10:55:14 +02:00
Fraser
dd308471eb Remove the dead push-event arms from the affected-tests stats upload (#80583)
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-08-24 09:37:02 +02:00
metabase-bot[bot]
5f354e3df2 [chore] Update E2E test timings with recent data (#80580)
Update E2E test timings with averaged data

Collected timing data from 7 days of test runs.
All specs with timing data were included (simple averages).

🤖 Generated automatically by update-e2e-timings workflow

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: GitHub Actions <actions@github.com>
2026-08-23 20:28:05 +02:00
Ben Grabow
fbc2d9d323 Use explicit hierarchy for metabot reaction types (#79465)
Reaction types registered by defreaction derived from :metabot/registered-action in Clojure's
global hierarchy, sharing one mutable namespace with model keywords, hook traits and every
other subsystem. Move them into a hierarchy var private to this namespace, mutated through
derive! -- the same shape as metabase.events.hierarchy, kept in-file because defreaction is
the only writer and known-reaction-types the only reader.

This also narrows the schema-rebuild watch. It was attached to #'clojure.core/global-hierarchy,
so it woke on every derive anywhere in the application and compared descendant sets to decide
whether anything relevant had changed; now it only fires for reaction registrations.

Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-22 21:57:54 -05:00
Ben Grabow
c24af85666 Use explicit hierarchy for search engines (#79462)
The legacy-cookie alias :search.engine/fulltext -> :search.engine/appdb lived in Clojure's
global hierarchy, sharing one mutable namespace with model keywords, hook traits and every
other subsystem. Move it into metabase.search.hierarchy, mirroring metabase.events.hierarchy:
a defonce var holding a hierarchy map, mutated through derive!.

All thirteen engine multimethods now dispatch on that hierarchy, and known-engine? walks it
rather than the global one.

The hierarchy has a single writer inside the search module, so it exposes only derive! and
ancestors -- the operations that are actually used.

Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-22 12:22:19 -05:00
Sloan Sparger
1d961d5cc5 [Metabot] Remove dead chart analysis code (#80371) 2026-08-21 22:50:30 +00:00
github-automation-metabase
df58523076 Fix flake in writable-connection model persistence e2e test (#78277)
The model persistence test clicked the global and per-database persistence
toggles without waiting for their requests to complete, then immediately
POSTed to persist a card. The card-persist endpoint returned 400
("Persisting models not enabled for database") when it raced ahead of the
per-database setting, and the per-database toggle could fail to render when
the database page booted from a stale global setting.

Intercept and wait on both persist-enable requests before proceeding.

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-08-22 00:48:17 +02:00
github-automation-metabase
b9292b6b6c Deflake notifications unsubscribe test on modal entrance transition (#78580)
The "unsubscribe from an alert when the user has not created it" test
clicked the Unsubscribe button immediately after a bare findByText for
the confirm modal's title. That query resolves the instant the title
mounts, while the Mantine modal is still animating in and its title
transiently covers the button's click point, so cy.click() intermittently
failed with "covered by another element".

Anchor on the title being visible first so the click is deferred until
the modal's entrance transition has settled.

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-08-21 16:08:43 -06:00
Sloan Sparger
bd86545b79 [Metabot] Remove image_base_64 context (#80364)
remove image_base_64 context
2026-08-21 17:03:48 -05:00
Sloan Sparger
92ab58c480 [Metabot] Remove explain chart button (#80355)
* remove explain chart

* update docs

* doc change
2026-08-21 17:03:48 -05:00
John Swanson
319a3664f9 Guard queue declarations by ownership, not config equality (#80388)
Re-running mq init in a REPL (dev server startup plus test init, or a
retried :mq test-init step) threw for the exploration queues:

```
  Queue :queue/exploration-plan is already registered with different config.
```

This is because of an attempt to catch cases where we were declaring
the same queue multiple times with different config. However, this was
subtly broken in two ways:

- first, there are two phases to queue registration: declaration, and
  registration. Declaration just means "write down what queue X's config
  is." Registration means actually executing that. Our check ran on
  *registration*, not on declaration. So if there were two different
  calls to `def-queue!` for the same queue, the first one would declare
  its `def-queue*`, then the second would overwrite it, and there would
  be no meaningful check left.

- second, doing registration twice would always throw, because some
  queue configs contained functions, and declaring the function twice
  resulted in non-equal results.

Instead, we want to just check that we're not *declaring* the same queue
multiple times in different namespaces. We allow declaring the same
queue twice in the same namespace for REPL convenience - technically a
loophole, but this is similar to a check we have for `defsetting`s and
it doesn't seem to have been a problem.
2026-08-21 14:41:38 -07:00
John Swanson
e964676a23 Pin exploration auto-binning to an explicit bin width (UXW-4391) (#78060)
* Pin exploration auto-binning to an explicit bin width (UXW-4391)

Exploration charts fan one metric/dim query out into several
segment-filtered variants that share an x-axis. With {:strategy :default}
binning, the QP re-derives the bin width per variant from that variant's
own filters, so a segment that range-filters the binned column (e.g.
"Highly rated products", rating >= 4) got 0.125-wide bins while its
siblings got 0.75-wide bins.

Fix: resolve the width the QP would use for the unsegmented query
(card-level filters included, per-segment filter excluded) and pin it on
the breakout as {:strategy :bin-width}. The binning middleware never
renegotiates an explicit bin width, and bin edges snap to multiples of
the width, so every variant lands on the same bin grid. Card-level range
filters on the dim still narrow the bins, since they're shared by all
variants.

- move filters->field-map + extract-bounds from the binning middleware
  into metabase.lib.binning.util (middleware delegates; extract-bounds
  now returns nil and the middleware keeps the qp.error-type throw)
- add metabase.lib.binning/default-bin-width (exported via lib.core):
  the width :default binning resolves for a column given the query's
  own filters, nicer-breakout fixed point included
- pin it in metabase.explorations.query-plan.mbql/apply-default-bucket

* Address CR

- default-bin-width now keys its filter lookup by the column ref's own
  id-or-name (like the QP binning middleware) instead of (or (:id column)
  (:name column)) with a regression test for the bug this fixed.
- Build the segment :definition test fixtures through lib instead of the
  mt/mbql-query macro
2026-08-21 14:34:57 -07:00
Cam Saul
a816155366 Clean up usages of 'pMBQL' in our documentation (#80390) 2026-08-21 21:00:00 +00:00
Kevin Kelleher
faa3018ebf Explorations toggle for contextual interestingness (#78239)
* toggle for contextual interestingness

* update toggle, language in AI usage views

* fix test
2026-08-21 20:56:19 +00:00
Anderson
3b6a98b66b Hide the empty dashcard menu on visualizer cards whose source card cannot be downloaded (#80113)
* Hide the empty dashcard menu on visualizer cards when downloads are not permitted

* Address review comments

* Type the dashcard menu's underlying-question items as an array and simplify code
2026-08-21 20:24:30 +00:00
Nemanja Glumac
d4268ac46e Remove unused merge_group CI logic (#80535) 2026-08-21 21:58:10 +02:00
Anderson
ed16a0bc37 Migrate home cards from Emotion to Mantine (#80335)
* Migrate home cards from Emotion

* Fix x-ray icon sizing
2026-08-21 16:06:16 -03:00
Nick Fitzpatrick
866b1b7bb0 add copy value click action (#79837)
* add copy value click action

* move details above filter

* move copy into it's own section

* fix import

* e2e adjustments

* Adding unit test for column settings

* Don't show copy action on PK columns
2026-08-21 15:54:06 -03:00
Sloan Sparger
f53640862e [Metabot] Handle conversation reaching max window size (#79859) 2026-08-21 18:36:21 +00:00
Chris Vandoodewaard
29afd6c0ad turn off slack messages in engineering-ci and create Linear issues instead (#80533)
* create tickets instead of sla
ck notifications

* add CI Triage label

* add pagination

* fix

* address pr review
2026-08-21 14:31:17 -04:00
Anderson
e86c35cd4d Migrate collection header caption styles from Emotion to Mantine (#80331)
Migrate CollectionCaption from Emotion to Mantine and CSS module
2026-08-21 14:21:23 -03:00
Alexander Polyankin
7e8be17997 Improve mongo value compilation (#80509)
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-08-21 16:56:34 +00:00
Alexander Polyankin
13cddfb68d Don't splice unvalidated database-type into JSON-field casts (#80482)
h2x/cast already guards cast targets: a plain type name is emitted raw,
anything else is quoted as an identifier. The MySQL and Postgres
json-query implementations bypassed that route and spliced the stored
field's database-type into SQL unvalidated (CONVERT(.., <raw>) and
::<raw> respectively).

Expose the plain-type-name predicate as h2x/raw-type-name? and use it:
Postgres quotes a non-plain type as an identifier (same rule as
h2x/cast); MySQL rejects it with an invalid-query error, since
CONVERT's target type cannot be a quoted identifier.

Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-08-21 12:49:23 -04:00
Alexander Polyankin
0ea74a39ff Reject implicit-join field refs in tile endpoints (#80480)
Tile endpoints resolve latField/lonField legacy refs via resolve-field;
a ref with :source-field (implicit join) now fails fast with a 400
instead of producing a broken query. The check lives in resolve-field,
which every tile endpoint (ad-hoc, card, dashcard, public, embed,
preview-embed) reaches through tiles-query.

Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-08-21 12:49:22 -04:00
Alexander Polyankin
abf7c47f3c Strip internal query-processor keys at the query boundary (#80392)
Internal query-processor and permission keys (`:qp/...`, `:query-permissions/...`,
`:impersonation/...`, `:persisted-info/...`, `:source-query/...`) are added to a
query during preprocessing and drive permission enforcement, so a client must
not be able to supply them. Previously five separate stripper middlewares
removed them one key at a time.

Replace those with a single declarative rule: a qualified map key outside the
`:lib` namespace is an internal key and gets stripped when a query is
deserialized from JSON. The rule is a Malli `:decode/api` (and
`:encode/serialize`) transformer on the query/stage/join/clause-options schemas,
run via `lib.serialize/prepare-after-deserialization` at the JSON boundaries
(REST API request decode and app-DB read) and once as a preprocessing
middleware, `qp.perms/remove-internal-keys`, gated by `qp.pipeline/*pivot?*` so
pivot sub-query keys survive their re-run.

Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
2026-08-21 12:49:22 -04:00
Sloan Sparger
4dbf593d23 [Metabot] Key conversation state by conversation id (#78759) 2026-08-21 11:35:01 -05:00
Alexander Polyankin
ede6a0ec56 Improve API schemas (#80332)
* Improve API schemas

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* Keywordize nested parameter option map keys

`::parameter-with-value` only keywordizes its top-level keys (via
`normalize-map-no-kebab-case`), so on the query-string parameter path -- where
the whole value arrives as a JSON string with string keys at every level -- the
nested `parameter.options` and `dimension.options` maps kept string keys. The
api decode's `strip-extra-keys` pass then dropped those undeclared string keys,
silently emptying `:options` (a case-insensitive filter would run case
sensitively, a relative date filter would lose `:include-current`, a
stage-targeted dimension would lose `:stage-number`).

Give both option schemas a `:decode/normalize` of `lib.schema.common/normalize-map`
so their keys are keywordized before the strip pass, matching how the rest of
the lib schemas decode map keys.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* Address agent-api / xrays review feedback

- /v2/query continuation-token branch now normalizes-and-validates the embedded
  query like the query_handle branch does, instead of trusting the unsigned
  token's shape; the total-row budget is read off that validated query.
- reject-native-query! normalizes the payload to MBQL 5 (best-effort) and checks
  for a native stage with lib/any-native-stage?, so native detection no longer
  depends on how the JSON was decoded (keyword vs string keys); /v1/execute and
  the query_handle path decode with plain json/decode and let normalization do
  the keywordizing. Malformed payloads still fall through to the existing shape
  checks for their clean 400s.
- Renamed validate-query -> normalize-and-validate-query (it normalizes and
  strips, not just validates) and led its docstring with what it returns.
- xrays cell-query route params decode declaratively via a ::cell-query schema
  (:decode/api base64-decodes + validates ::root.cell-query) instead of
  base64-decoding once in the schema and again in the handler.
- Tiles reject test asserts a clean 400 (status-code) rather than any Exception.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* Update sandbox dimension-target test for keywordized option keys

The dimension-target `:options` map now keywordizes its keys (e.g. `"stage-number"`
-> `:stage-number`), so the attribute-remappings round-trip test expects the
normalized keyword form on the way back out.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

---------

Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-08-21 11:15:18 -04:00
Brad Anderson
a4eb75df3a GRO-829 Commit embedding docs (#80186) 2026-08-21 11:14:08 -04:00
Nicolò Pretto
9ad311d127 attempt to cut admin <-> sdk cycle in modules (#80146) 2026-08-21 16:54:34 +02:00
Chris Vandoodewaard
7abb55964b display CI Conductor link for failing tests (#79601)
* add permalink to quarantine entry, update compare function to return QuarantineEntry type

* print search url to make tests easy to find

* update tests

* remove a few comments

* simplify search url
2026-08-21 10:31:14 -04:00
Egor Grushin
a28a9af735 "Send now" in the question alert modal is never disabled (#80439)
Disable "Send now" when the alert has no delivery target (#79881)

Extract alertHasValidTarget from alertIsValid: the handler and channel
checks without the subscriptions one. Send now is a one-off send that
ignores the schedule, so it gates on the target only, while Done keeps
the full check.
2026-08-21 14:44:56 +01:00
Egor Grushin
24fa76cfa8 Fix the disabled Done button in the question alert modal (#80438)
Fix disabled Done button in question alert modal (#79914)

bg is a style prop, lands inline on the root and beats the :disabled
rule in Button.module.css, so the button kept its brand background while
the label switched to the dark text-disabled token. color feeds
--button-bg instead, which the disabled rule overrides.
2026-08-21 14:27:28 +01:00
Nicola Mometto
dad470d197 Add validation and permission checks for content endpoints (#80277)
* Run full query-permission checks when compiling a native form (SEC-822)

POST /api/dataset/native compiled and returned the query after checking only
database-level native perms, so a referenced source card or card template tag
was resolved without a read check. Preprocess the query once (with inline
parameter compilation enabled), run the same check-query-permissions* checks
the run path uses (surfacing a permission failure as a 403), then compile the
preprocessed query. Also trim the response to :query and :params.

* Validate link-card entity ids on write for dashboard cards (SEC-836, SEC-832)

DashboardCard validated the link-card entity id only when building the read
query; nothing checked it on write, so a non-integer id could be stored and
would then break every hydration of the dashboard. Add a write-side check in
define-before-insert and a new define-before-update, and wrap the notification
link-card renderer in try/catch so one bad card degrades to a missing part
instead of failing the whole subscription. Adds regression tests.

* Require positive-integer entity ids in dependency backfill (SEC-830)

replace-dependencies! placed to_entity_id directly in the insert, and the
dashboard click_behavior extraction forwarded targetId with no type check. Add
an explicit pos-int? check at the insert where to-add is built (a runtime
check, not an mu/defn schema, which is stripped from prod builds), and filter
targetId to positive integers during extraction so bad ids drop rather than
making the backfill job throw. Adds regression tests.

* Validate click-behavior target ids on write for dashboard cards (SEC-830)

Review follow-up: the entity ids the dependency backfill consumes come from
dashcard click_behavior targetIds, so validate them at the write boundary as
well, keeping the deep guard in replace-dependencies! as defense in depth.
Covers both the top-level click_behavior and per-column column_settings ones,
on insert and update. Adds regression tests.
2026-08-21 08:32:38 -04:00
Romeo Van Snick
72b3d9c271 Drop the two selector-level display readers of the metadata mirror (#80400) 2026-08-21 21:21:10 +10:00
Nicolò Pretto
be612d2dc6 support passing multiple fonts in embedding (#80442) 2026-08-21 13:15:28 +02:00
github-automation-metabase
3d99e1332f Wait for query run before asserting native-conversion results (#78274)
The 'converting question to a native query' e2e test asserted on
[data-testid=cell-data] immediately after clicking 'Convert this
question to a native query' (and after 'Explore results'), without
waiting for the POST /api/dataset run to settle. Under load the
assertion raced a stale/mid-flight results table and timed out.

Mirror the working convertToSql() helper: intercept POST /api/dataset
and cy.wait for it before each cell-data assertion.

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-08-21 13:05:29 +02:00
Kevin Kelleher
5e5b740287 Fix commenting on a chart without a timeline selected in explorations (#80378)
don't send null timeline_id to the BE in a comment's context
2026-08-21 07:00:22 -04:00
Kevin Kelleher
57ca3a007d Add row count check to column cardinality cache (#80176)
add row count check to column cardinality cache
2026-08-21 07:00:00 -04:00
Romeo Van Snick
a5e4adb11a Move plan, frame-paint, and export-format helpers out of common (#80049)
* Move plan util from common into lib utils

* Move waitUntilNextFramePainted from common into lib utils

* Move export format types from common into metabase-types

* Move plan helpers into the settings module
2026-08-21 20:32:45 +10:00
YB
866c840355 gate pulse test-send behind the subscription application permission (#80136) 2026-08-21 18:32:13 +08:00
YB
d914e47f76 escape parameter & heading text when rendering notification emails (#80055)
escape parameter and heading text when rendering notification emails
2026-08-21 18:31:57 +08:00
Andrei Simionescu
537b242202 PRO-180: Add copy to clipboard for query results (#79341)
Add a copy button to the query builder's view footer, next to download.
It copies whatever is on screen: tabular results (tables, pivot grids,
or a chart toggled to the raw data view) as tab-separated text plus a
text/html table flavor so spreadsheets paste a grid, and any other
chart as a png rendered through the same pipeline as the png download.

Copies are always formatted, matching the rendered table: remapped
columns, column visibility and order, renamed titles, click-behavior
link text, and pivot grids with their headers and totals. Raw data
stays a download concern. Copying is capped at the rows loaded in the
browser, with a toast saying when only the first slice was copied.

The clipboard write starts inside the click's user activation so Safari
accepts it, cell click metadata is built only for columns whose text
needs it so wide tables serialize in linear time, and the selected-cell
copy now shares the quoting TSV serializer.
2026-08-21 10:31:18 +01:00
Ngoc Khuat
118ee5715c Cover database detail handling on serialization import (#80334)
Cover Database connection details on serialization import

Pins the two checks in load-one! for Database: an H2 engine is refused, and
low-level JDBC keys are dropped from details, write_data_details and
admin_details.
2026-08-21 16:13:09 +07:00
Chris Truter
5f4bb206a1 Treat a 403 from construct_notebook_query as agent-facing (#79983) 2026-08-21 10:36:06 +02:00
Ngoc Khuat
a6345c149b Quote table names in the MySQL rename, and cover transform target names (#80068) 2026-08-21 08:26:26 +00:00
Nicola Mometto
992d8c36e2 Add checks to document card-creation and AST reference ids (#80145)
* Harden document card-creation permission checks

Route document create/update/copy card creation through the same checks
POST /api/card applies: create-check the target collection, read-check
each card copied by document_id FK, and query-permission-check the fields
that parameter targets name.

* Guard document AST entity ids before app-DB lookups

smartLink/cardEmbed reference ids come from free-form document JSON and
may be any value. Add a shared node-entity-id accessor that yields the id
only when it is a positive integer, and use it in the raw-entity readers
(serdes export transform, export-time dependencies, and descendants) so a
malformed reference is skipped rather than passed to a Toucan query.
2026-08-21 09:23:57 +01:00
Nicola Mometto
024aa144a0 Add access and visibility checks to comments and mentions (#80142)
* Add access and type checks to comment notifications

Mentioned users are now resolved to active users whose permissions cover
reading the commented entity before being added to the notification
recipients, matching how other delivery surfaces scope their audiences.

The TipTap text-node renderer now coerces non-string :text values to
strings so they always compile as escaped text rather than as hiccup
elements.

* Scope the comment mentions listing to visible users

GET /api/comment/mentions now applies the same audience rules as
GET /api/user/recipients: non-superusers only see users in their own
tenant, further narrowed by the user-visibility setting (:none -> self,
:group -> shared-group members, :all -> everyone in the tenant). This
also resolves the endpoint's long-standing TODO about tenant handling.

same-groups-user-ids moves from the user REST namespace into the user
model so both endpoints share one implementation.
2026-08-21 09:23:37 +01:00
Nemanja Glumac
3fea6bf3fc Fix the action creator modal flake (#80282) 2026-08-21 10:11:07 +02:00
Fraser
79f3813c3e Tolerate stale self entries in the side-effect registry spec (#80401) 2026-08-21 20:10:50 +12:00
Fraser
5516e40607 Add extension points in viz for sdk behaviour (#79436) 2026-08-21 16:23:06 +12:00
Romeo Van Snick
ebe7663ba3 Read the query's database from the API, not the entities mirror (#80267)
Read the query's database from the API in the three feature-check consumers
2026-08-21 14:09:18 +10:00
Romeo Van Snick
234c695a25 Dissolve the lib-tier type cycle between metabase-types, metabase-lib, and utils (#80053)
* Move JoinConditionOperator from metabase-lib into metabase-types

* Move font family and DeepPartial types into metabase-types

* Generate mock document ids without the lib utils uuid helper

* Import JoinConditionOperator from the api barrel in metabase-lib

* Drop lib-utils re-export shims and inline docs at the moved declarations
2026-08-21 12:26:21 +10:00
Fraser
e247190827 Make metabase/ui side-effect free (#79962) 2026-08-21 14:17:50 +12:00
Tyler Plude
8661fd4524 [BOT-1930] Self hosted vLLM metabot support (#79437)
* init vllm support

* tmp improvements

* cleanups; bump max tokens floor for vllm; env shadowed cred field guards

* fix openrouter temperature

* vllm reasoning

* fixes

* cleanups

* make vllm transport errors non-retryable

* fix vllm connect diagnostics; add recorded stream fixtures

* [BOT-1930] Rebuild vLLM support on the provider connection registry (#80103)

* Rebuild vLLM support on the provider connection registry

vLLM becomes a provider type in `metabase.llm.provider` rather than a set of
`llm-vllm-*` settings read by the adapter: a required base URL and an optional
API key, no default model, and an `MB_LLM_VLLM_API_BASE_URL` group so the
environment can configure one connection. The base URL is the credential — a
server started without `--api-key` takes no key — so it alone brings a usable
connection into existence, and the admin form renders from the descriptor
instead of a bespoke component.

Two things the adapter used to reach out for now travel with the connection:

- The connect-time preflight, which exercises tool calling and structured
  output against the model that will be used, runs from `verify-credentials!`
  under a new `probe?` flag. Only writes probe; a model listing must not spend
  a generation on the operator's server.

- Whether the served model streams reasoning, which only the probe can answer,
  is reported back as `:learned-config` and stored on the connection instead of
  in a global setting, so two vLLM connections no longer overwrite each other's
  verdict. The request path reads it from the credentials it is handed.

A vLLM connect has no default model to select, so it adopts the model its probe
exercised, and an edit re-probes against the model the connection is actually
serving Metabot.

* Document connecting a vLLM server

The provider list in the AI settings docs is hand-maintained. vLLM is the one
provider an admin can misconfigure outside Metabase, so it gets the server flags
the connect-time check looks for alongside its entry.

* fix kondo

* revert docs

---------

Co-authored-by: Tyler Plude <tsplude@gmail.com>

* review comments

---------

Co-authored-by: Maksym Yakubych <maksym@metabase.com>
Co-authored-by: Maksym Yakubych <undrfined@gmail.com>
2026-08-20 22:00:58 -04:00
Fraser
e7a84b1625 Fix Mongo native preview sidebar assertions after default-limit removal (#80395) 2026-08-21 13:39:57 +12:00
github-automation-metabase
449e3f5c76 Auto-update documentation for master (#80376)
Update generated docs (master)

Co-authored-by: github-actions <github-actions@github.com>
2026-08-20 21:18:29 +00:00
Kevin Kelleher
ce55f0cdc6 Fix axis labels for timezone naive strings (#80121)
* fix axis labels for timezone naive strings

* Update Loki Snapshots

* fix lint after rebase

* don't throw in tests

---------

Co-authored-by: Metabase Automation <github-automation@metabase.com>
2026-08-20 17:11:25 -04:00
Denis Berezin
706c3862ab UXW-4718 Dashboard tab selection not persisted on page refresh (#79669)
* UXW-4718 Dashboard tab selection not persisted on page refresh

* review fixes

* Fix unit tests

* Fix unit tests
2026-08-20 23:32:07 +03:00
Anderson
723aebae08 Aggregate region and grid map tooltips when the mapped dimensions are not unique (#80027)
* Aggregate map rows sharing a region or grid cell and warn about unaggregated data

* Fix comment
2026-08-20 17:19:59 -03:00
Ben Grabow
06f7a3ca8b Bgrabow/profile tests workflow (#80258)
* Add workflow dispatch/profiling options to app-db, backend, semantic-search workflows
2026-08-20 20:12:09 +00:00
Chris Vandoodewaard
e30fb0cec1 report all test results on rerun to CI Conductor (#80326)
* upload all pass/failures for backend tests on rerun

* also for frontend tests

* add some tests

* cleanup whitespace

* copilot review

* a few more fixes from local review

* determine rerun from github context instead of input
2026-08-20 14:50:15 -04:00
Ben Grabow
9add75e67f Use content-based cache keys for m2, bun, cypress, eslint. (#80219)
* Use content-based cache keys for m2, bun, cypress, eslint.
Eagerly populate cache on push to master/release for m2, bun, cypress.
Read-write eslint cache from every branch.

* Handle multiline outputs

* Remove unneeded M2 restore on prepare-frontend

* Handle potentially multiple cyprss version matches. Update commentary.

* Validate list of alias combinations in use across the repo

* Check preresolve aliases in parallel
2026-08-20 12:38:42 -05:00
github-automation-metabase
af4fd281b4 Auto-update documentation for master (#80250)
Update generated docs (master)

Co-authored-by: github-actions <github-actions@github.com>
2026-08-20 13:27:35 -04:00
Nicolò Pretto
6f36c446e6 remove --no-package-lock from embedding host app tests (#80094)
remove --no-package-lock
2026-08-20 18:30:08 +02:00
lbrdnk
c010ae765a Reuse workflow to dockerize an image in perf test workflow (#80205)
* Reuse dockerize workflow

* Remove uberjar job from labeled workflow
2026-08-20 18:17:55 +02:00
appleby
b6edd4379c Enable Qwen 3.8 Max via OpenRouter (#79735)
* Enable Qwen 3.8 Max via OpenRouter

* Downgrade tool_choice "required" to "auto" for qwen3.8-max

Alibaba returns a 400 request error if you pass tool_choice "required" and unlike moonshot / Kimi K3, does not allow
you to disable thinking mode for Qwen 3.8.

* Add coerce-stringified-scalars to transform tool call args

Qwen 3.8 sometimes sends integer args as strings, like `{:limit "15"}`. Use the tool's declared schema to
automatically transform any simple numeric or boolean scalars.
2026-08-20 08:29:53 -07:00
Timofey Kachalov
7e3d167c05 Adjust skills to instruct using default Question components when possible (#80248) 2026-08-20 19:24:21 +04:00
Nicola Mometto
ef87a8221e Guard transform revision history per snapshot (#80088)
* Test: transform revision diffs should not include past source bodies

A transform's revision history diffs two full snapshots, so every past
:source (native SQL / Python body) and :source_database_id shows up in
the diff - even for versions that read from a database the current
viewer has no query access to. Add a test asserting the diff only
describes that the source changed, without reproducing the old source.

* Guard transform revision history per snapshot, not per current row

* extra guards in metabot tool
2026-08-20 15:16:56 +01:00
Timofey Kachalov
4c5abeb77e Add description field for data apps (#79544) 2026-08-20 17:57:34 +04:00
Sloan Sparger
fdc8caffe7 [Metabot] Handle tool-calls finish reason (#79860) 2026-08-20 14:53:51 +01:00
Stas Gavrylov
62c29ab1eb GDGT-2932: Refactor AI Analytics views to use internal routes and shared layout (#79010)
* GDGT-2932: Refactor MCP/CLI analytics to shared-layout + nested routes

Replaces the tab-query-param pages with a shared layout (header, tab
navigation, filter bar, and a single "has any data" gate) rendering
usage/events sub-routes through an Outlet, mirroring the
dependency-diagnostics pattern. New URLs:
- /monitor/ai-auditing/mcp/{usage,events}
- /monitor/ai-auditing/cli/{usage,calls}
with an index redirect from the bare section path to /usage.

Shared filter/pagination state is passed to the active leaf route via a
typed React context (following MonitorContent's existing
useMonitorSidebarContext precedent) rather than react-router's untyped
useOutletContext.

Also fixes two bugs found in cross-review: tab links were dropping the
current query string (clearing filters on every tab switch, since
LinkTab doesn't inherit location.search), and the index redirect was
gated behind the "has data" query so a bare /mcp or /cli with zero rows
never canonicalized to /usage. Bundles a parity fix for useCliHasData
(now surfaces query errors, matching useMcpHasData) so the CLI usage
view doesn't spin forever on a failed count query.

* Fix MonitorHeaderTabs to use aria-current instead of the ARIA tabs pattern

MonitorHeaderTabs rendered its options with role="tab" (via Mantine
Tabs.Tab + a Link component), but each "tab" is a real, bookmarkable
route rather than in-place content switching — the ARIA tabs pattern
implies activating a tab doesn't navigate, which isn't true here. It
also never rendered a matching tabpanel/aria-controls pairing, since
the actual content lives in a react-router <Outlet/> outside the Tabs
tree entirely.

Rewritten as plain nav links with aria-current="page" on the active
one, matching the existing AreaTab component's approach for the same
kind of route-driven navigation. Visual pill styling is preserved via
a small CSS module. Updates the two dependency-diagnostics/tasks/mcp/
cli consumers' tests and e2e specs that asserted the old role="tab".

* GDGT-2932: Review

* GDGT-2932: Keep error count queries mounted across analytics routes

* GDGT-2932: Cleanup

* Fix task tabs E2E selector

* GDGT-2932: Address PR comments

* GDGT-2932: Cleanup

* GDGT-2932: Remove LinkTab. Introduce shared AiAnalyticsSectionLayout. Fix comments
2026-08-20 16:21:00 +03:00
Stas Gavrylov
f78ab5b83f Upgrade Scalar API Reference to 1.65.1 (#80289) 2026-08-20 15:54:15 +03:00
Fraser
7cedc004a8 Fix the module side-effect lint errors in querying/expressions (#80285) 2026-08-20 23:35:31 +12:00
Fraser
d2c5d44b63 Add a lint rule that keeps side-effect-free modules side-effect free (#80047) 2026-08-20 22:22:11 +12:00
Nemanja Glumac
ee92e5d470 Reproduce 23103 using Jest instead of e2e (#80237) 2026-08-20 11:01:14 +02:00
Nicola Mometto
f0bf525611 Require write permission to send notifications and to use inline channels (#80069) 2026-08-20 09:57:40 +01:00
Romeo Van Snick
d7e2aa3d5b Check database features through the API helper, not the v1 wrapper (#80158)
* Check database features through the API helper, not the v1 wrapper

* Move the database feature checks out of common, behind a databases barrel
2026-08-20 15:46:50 +10:00
Romeo Van Snick
da126bb88c Give the remaining app routes their own chunks (#80160)
* Give the remaining app routes their own chunks

* Fetch the home page chunk while the user signs in
2026-08-20 13:54:35 +10:00
Romeo Van Snick
4711e0c845 Always invalidate remote sync on card, dashboard and document mutations (#80148) 2026-08-20 13:54:19 +10:00
appleby
6d22e0c553 Add deepseek-v4-pro-0813 to openrouter supported-models (#80018) 2026-08-19 16:55:48 -07:00
Sloan Sparger
8ed4b9d8dc [Metabot] Reasoning style updates (#80127) 2026-08-19 22:30:28 +00:00
Romeo Van Snick
f5bdbf045d Load the dependency graph and schema viewer pages lazily (#80050)
* Move the xyflow stylesheet into the graph components

* Give the schema viewer one loader for its route and its prefetch

* Load the dependency graph from the plugin slot lazily
2026-08-20 08:22:14 +10:00
bryan
9450affe98 Sign session key hashes with a server-side secret (#80046) 2026-08-19 22:04:09 +00:00
Maksym Yakubych
25ed6e5a58 Update the OpenRouter provider logo to the refreshed brand mark (#80253) 2026-08-19 21:50:55 +00:00
Maksym Yakubych
84be938a6e fix(select): don't render option groups emptied by search (#80197) 2026-08-19 21:46:36 +00:00
Maksym Yakubych
909156c65d Deflake metabot error boundary test by loading the chat module at suite setup (#80213)
* Deflake metabot error boundary test by loading the chat module at suite setup

* Preload the chat module in the shared metabot test setup

* Use the shared metabot test setup in the error boundary spec
2026-08-19 20:56:53 +00:00
Nathan Voxland
9a0e229370 Better tracking of attached data warehouse connections (#79910) 2026-08-19 15:07:30 -05:00
Andrei Chernykh
68746a6d43 Clean up leftover JS-era config, deps, and docs after the TypeScript migration (#80032)
* Clean up leftover JS-era config, deps, and docs after the TypeScript migration

* Address review: keep .jsx in resolve.extensions, drop .js from loader tests, includes-based e2e validator filter

* Keep @babel/node for a separate deps cleanup round

* Keep js/jsx in rspack loader tests
2026-08-19 12:48:52 -07:00
github-automation-metabase
dc81dff870 Fix flaky dashboard parameter-removal e2e test (#78302)
* Fix flaky dashboard parameter-removal e2e test

The "should be able to remove parameter (metabase#17933)" test asserted on
the full dashboard URL query string with cy.location("search").should("eq",
...). The order in which parameter slugs are serialized into the URL is not
deterministic, so a correct-values-but-different-order string
(?text_starts_with=G&text_ends_with= vs ?text_ends_with=&text_starts_with=G)
could never match and the retry just spun until timeout. Empty/stale-URL
timing races produced the other observed failures.

Replace the four exact-string assertions with an order-independent helper that
parses the query string via URLSearchParams and deep-equals the expected
param map. It still retries through cy .should(callback), so async URL updates
settle before the assertion resolves.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* Wait for parameter-remove button to enable before clicking

The parameter-removal step in the dashboard parameters spec clicked the
sidebar's Remove control via cy.findByText("Remove"). That query resolves to
the button's inner label <span>, not the <button> element. A span is never
"disabled", so Cypress's actionability check did not wait: it clicked the span
while the underlying button was still disabled.

The Remove button stays disabled until the dashboard's subscriptions have
loaded (the sidebar warns before archiving any subscription bound to the
filter). When that GET /api/pulse request was slow, the click landed on the
disabled button as a silent no-op, the parameter was never removed, and the
subsequent URL assertion timed out waiting for the slug to disappear.

Target the button element by role and assert it is enabled before clicking so
the click waits for the subscriptions request to settle.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* Fix linting error

---------

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-authored-by: Nemanja <31325167+nemanjaglumac@users.noreply.github.com>
2026-08-19 21:30:30 +02:00
github-automation-metabase
8107c34e1d Auto-update documentation for master (#80120)
Update generated docs (master)

Co-authored-by: github-actions <github-actions@github.com>
2026-08-19 15:22:08 -04:00
Nick Fitzpatrick
97ecbeab61 Update auto 'Show Dots' logic (#80071)
* Update auto 'Show Dots' logic

* Update Loki Snapshots

* PR feedback

---------

Co-authored-by: Metabase Automation <github-automation@metabase.com>
2026-08-19 16:09:30 -03:00
github-automation-metabase
3ea4987f1f Fix flaky revision-history dashboard test (#78280)
The test opened the info sidebar right after an ad-hoc edit-mode save that
only awaited the PUT /api/dashboard request. The trailing query_metadata
reload re-rendered the header, dropping the 'More info' click so the
sidesheet never opened. Use the H.saveDashboard() helper, which waits for the
full save-and-settle sequence before proceeding.

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-08-19 21:07:10 +02:00
bryan
58c85c338c Only serialize Channel :details for callers who can write the channel (#80057) 2026-08-19 13:00:45 -06:00
github-automation-metabase
d056853a8c Fix flaky segment_created tracking e2e test (#78610)
* Fix getTable intercept in segment_created tracking e2e test

The segment editor's DataStep loads a table via fetchTableMetadataAndForeignKeys, which calls GET /api/table/:id/query_metadata. The test intercepted the bare GET /api/table/:id endpoint instead, so cy.wait("@getTable") always timed out because that request never fires in this flow. Point the intercept at the query_metadata endpoint the editor actually calls.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* Gate segment filter click on enabled Add-filters button

The segment editor renders a disabled "Add filters to narrow your answer"
button until onChange builds the query from fully-loaded table + FK metadata.
Waiting on the getTable response alone isn't enough — the trailing foreign-key
metadata loads race the click, so the click lands on the still-disabled button
and the filter popover never opens. Assert the button is enabled first.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-08-19 20:55:18 +02:00
Nemanja Glumac
368c50f7e4 Explicitly exclude old-migrations-test tag (#80214) 2026-08-19 20:50:00 +02:00
Chris Vandoodewaard
3c70c47538 Revert "only publish Github release for major and gold releases" (#80220)
Revert "only publish Github release for major and gold releases (#79478)"

This reverts commit 814b57b74a.
2026-08-19 18:45:08 +00:00
Nick Fitzpatrick
b872cccdde Uxw 5027 model metadata editor UI issues (#79909)
* everything except model index

* use-should-index

* remove 0.88em form label
2026-08-19 14:38:17 -03:00
Stas Gavrylov
0c5d3b3a59 GDGT-2459: Update Tooltip component (#79994)
* GDGT-2459 Update Tooltip styles

* Update Loki Snapshots

* Add Tooltip Overview Loki reference

* GDGT-2459 Add Tooltip spacing and shadow

* GDGT-2459 Use standard small Tooltip shadow

* Update Loki Snapshots

---------

Co-authored-by: Metabase Automation <github-automation@metabase.com>
2026-08-19 20:19:40 +03:00
Chris Truter
3989cb5eb3 Standardize nonparallel tests on ^:synchronized (#80064) 2026-08-19 17:15:36 +00:00
Chris Vandoodewaard
83adb8f686 make cross version tests run using Postgres (#79091)
* add sample db

* delete cross version 57 since we don't support it anymore

* update cross version tests to use postgres

* fix modal selector to work on v1.60

* fix modal selector for v60 and v63 compatibility

* add better logging

* use qa image

* use herestring to handle large output properly

* use data from WA image

* fix flake in nested model assertion

* cleanup the README

* code review fixes
2026-08-19 13:11:33 -04:00
Nick Fitzpatrick
3e9460c53e Uxw 4977 show collection that couldnt be synced (#79652)
* updating remote sync enpoint error response

* adding FE types

* Show required dependencies on sync enable

* modal version

* tests

* maz feedback

* Handle Our Analytics case

* Code review fixes

* Fixed bad fix

* Rebase and cleanup

---------

Co-authored-by: Nathan Voxland <nathan@metabase.com>
2026-08-19 13:38:03 -03:00
Chris Truter
6327ba0828 Stop LLM impersonation tests leaking cached field values (#80178) 2026-08-19 18:03:07 +02:00
Anderson
6b4af53319 Keep metric dimension rows full-width by placing the vertical scrollbar in the column gutter (#80008)
Keep metric dimension rows full-width by moving scrollbar into column gutter
2026-08-19 13:00:28 -03:00
Eric Normand
6bd66a6d5e Validate imported queries against this instance's MBQL schema (GHY-4241) (#79496)
* Validate imported queries against this instance's MBQL schema (GHY-4241)

An export decides the shape of the queries it carries, so an import can hand us
a query shape this version has no representation for. `lib/normalize` does not
always reject one: it can quietly produce something the schema rejects, which
is then stored and breaks the card on read.

After `import-mbql` normalizes an ingested query, validate the result against
`::lib.schema/query` and refuse the import if it fails. Only full queries are
checked - bare refs and the MBQL fragments inside visualization settings have no
standalone schema to validate against.

Normalization failures stay non-fatal. They fire today on content this instance
produced itself (pivot column refs in visualization settings raise
`:malli.core/invalid-schema`), so promoting them would reject legitimate
imports. Validating the successfully-normalized result catches the real problem
without that fallout.

`MB_SERIALIZATION_SKIP_SCHEMA_VALIDATION` opts out. `metabase.models.serialization`
cannot read an EE setting, so the flag reaches it through a dynamic var bound by
`load-metabase!` - the one choke point every import path funnels through.

Also validate the bundled instance analytics content against the same schema in
CI. That content re-imports on every EE boot and a failure there propagates out
of `init!` to `System/exit`, so a bundle regeneration that broke it would
crash-loop rather than degrade.

* Fix import validation documentation

* Refactor documentation for import validation settings

* update docs

* Report the schema refusal as a 400 and stop duplicating its errors (GHY-4241)

The serdes import API reads `:status` off ex-data itself rather than going
through the API layer's exception handling, so `:status-code` left it nil and
the refusal surfaced as a 500 for what is a client input error.

`u/strip-error` already renders ex-data into the message, so the truncated
`Schema errors:` tail was a second, shorter copy of `:schema-errors`.

* Drop the unreachable status code from the schema refusal (GHY-4241)

`load-one!` rewraps every exception from the load block in a fresh ex-info
carrying `path-error-data`, and the import API reads `:status` off the
outermost exception only. Neither `:status` nor `:status-code` attached here
was ever visible, so claiming one in a comment was worse than saying nothing.

* Name both causes in the refusal, and pin bundle ingest errors (GHY-4241)

`mu/defn` schemas are not instrumented in prod, so an app DB can hold MBQL the
QP tolerates but `::lib.schema/query` rejects - a stale `:join-alias`, a field
filter with no `:widget-type`. Such content exports intact and is then refused
on import, so "most likely exported by a newer Metabase" was the wrong
diagnosis for a whole class of refusals. Name both causes, and point at
continue_on_error as the per-entity escape rather than only the global one.

The bundle test proved every query imports but not that every file parses.
`ingest-list` collects unreadable files in an errors atom rather than failing,
so a malformed bundled YAML would drop out of the queries under test while
still crashing EE boot. Assert the atom is empty too.

* Move refusal rationale out of docstrings, harden a test helper (GHY-4241)

Docstrings state the contract; the reasoning behind the check, the message
wording and the sentinel moves to comments beside the code it explains.
`*skip-schema-validation?*` no longer names who binds it, which would go stale
silently. `validate-imported-query!` now says legacy MBQL 4 queries are left
alone as well, which was true but unstated.

`cause-chain-messages` used `map ex-message`, so an exception with no message
would put a nil in the vector and NPE the callers' `re-find` - failing the
test for a reason unrelated to what it asserts.

* kick ci

* Simplify the import schema check's control flow (GHY-4241)

`validate-imported-query!` returns its argument and no longer consults the
opt-out, so it is a plain step in `import-mbql`'s pipeline and the caller
decides whether to run it. That removes the sentinel `normalize-imported`
needed to keep the refusal outside its `catch`, along with the comment
explaining the sentinel.

`some->` replaces `->`, which makes `normalize-imported`'s nil guard
redundant.

* update comment
2026-08-19 15:38:28 +00:00
Michael Eby
50568b4841 docs: remove db secrets export from serialization docs (#80114) 2026-08-19 11:20:28 -04:00
Chris Vandoodewaard
272b17ffdd [Flaky Test]: should send snowplow events through admin settings (#80005)
* don't assert on toast

* dismiss toast after assertion
2026-08-19 10:57:24 -04:00
Ben Grabow
64b5ba8500 Fix flaky tests: Error rolling back after previous error: Deadlock detected. (#80126)
Retry on search reindex deadlock
2026-08-19 09:42:02 -05:00
Chris Truter
285d779b78 Use the in-process embedder for retrieval and complexity score (#78314)
* Route semantic embeddings through the provider SPI

Summary

- Register the existing ai-service, OpenAI, and Ollama implementations through the shared provider registry.
- Route semantic embedding, readiness, resolution, and preparation through the SPI while preserving the existing public adapter.
- Preserve the compatibility client model-dimensions contract by normalizing it to vector-dimensions at the provider boundary.
- Keep semantic-specific batching, metrics, and settings policy in semantic search.

Scope

This migrates consumers to the SPI. Persisted embedding-space compatibility and in-process provider selection are intentionally handled by later stack layers.

Testing

- Built-in provider behavior and existing semantic embedding adapter tests.
- Compatibility-client dimension normalization and conflict handling.

* Describe the embeddings client contract in SPI terms

The batch entry point delegates to the provider registry rather than a multimethod.

* Port complexity-test fake embedder to the provider SPI

get-embeddings-batch is a plain fn now, so the old defmethod no longer compiles
and broke every job that loads this namespace. Register a capturing provider
instead and assert on the descriptor it receives.

* Induce the dimension mismatch from the model, not the mock vector

The SPI rejects a vector whose length doesn't match the resolved model, so a dim-1 mock embedding
no longer reaches pgvector. Shrink the configured model instead, which is what an upgrade does.

* Persist immutable embedding-space identity

Summary

- Persist embedding-space IDs in semantic and library-entity index metadata.
- Backfill semantic metadata through a preserving schema migration and rebuild legacy entity vectors once when identity is unknown.
- Include immutable space identity in compatibility checks and physical table names.
- Enforce the PostgreSQL 63-byte identifier limit using UTF-8 byte length.

Scope

This establishes durable compatibility. Synchronization between entity compatibility checks and rebuilds follows in the next stack layer.

Testing

- Semantic migration preservation and backfill.
- Entity-index rebuild and compatibility behavior.
- Embedding-space table naming and non-ASCII identifier limits.

* Resolve the model in the index-cleanup test helper

record-new-index-table! now persists embedding_space_id into a NOT NULL column,
so create-test-index must build a resolved model that carries one.

* Track the index-status rename after rebase onto master

* Resolve the model-v2 variant in the reconcile tests

A bare assoc keeps the v1 embedding-space-id on a renamed model, so the rebuild wrote a meta row
whose space no longer matched what the provider resolves and nothing repopulated.

* Serialize entity index queries with rebuilds

Summary

- Coordinate library searches and reconciles with shared and exclusive PostgreSQL advisory locks.
- Keep compatibility rechecks and vector queries on the same locked connection so they cannot straddle a rebuild.
- Use non-blocking search lock acquisition so waiting searches cannot exhaust the pgvector pool.
- Preflight compatibility before embedding and return no results while a reconcile owns the index.

Scope

This PR only addresses library entity-index concurrency and degradation behavior.

Testing

- Search call ordering and locked connection reuse.
- Incompatible-index preflight and reconcile-contention degradation.

* cljfmt entity-retrieval core-test

* Follow the search-unfiltered rename in the entity-retrieval tests

Rebase fallout: the parent branch renamed `search` to `search-unfiltered`, and these call sites came across from
before the rename.

* Enable the in-process embedding provider

Summary

- Allow plugin-defined providers in semantic settings and safely encode provider names in physical index identifiers.
- Enable the bundled in-process Arctic provider with dimension readiness, query prefixing, and token metrics.
- Document the optional plugin and its glibc 2.34+ Linux or Apple Silicon macOS runtime requirement.
- Build the embedder jar in CI and run one artifact-only smoke path through the provider SPI on the OSS classpath.

Scope

This intentionally supports one pinned Arctic model. Custom model sources, MiniLM, and library overrides remain out of scope.

Testing

- Settings, prefix, token-accounting, readiness, and derived table-name regressions.
- Jar-manifest discovery, platform failure modes, batching, no-network behavior, and real ONNX inference.

* Re-hash qualified index table names over the 63-byte limit

qualify-index wraps the model table name with the metadata qualifier but did not
re-check Postgres's 63-byte identifier limit. A longer provider-encoded name plus
the qualifier could exceed it, so Postgres truncated the name on CREATE while
table-exists? still looked it up by the full name -- reporting the index table as
missing. Re-apply hash-identifier-if-exceeds-pg-limit to the qualified name.

* Hash only the table component of a qualified index name

Postgres's 63-byte limit is per identifier. Hashing the whole schema-qualified
name dropped the schema, so in app-db mode a long semantic_search.<table> became
a bare index_<sha1> created outside the isolated schema. Split off the schema and
hash only the table component.

* Cover in-process MiniLM complexity scoring

* Activate the in-process embedding provider on demand

* Stub lazy provider activation in token usage test

* Update Kondo ratchets for embedder changes

* Rely on bootstrap for in-process provider registration

* Cover the bundled complexity-score model in the artifact smoke test

The artifact job only exercised the retrieval model, so a broken or missing MiniLM
bundle -- which uses a different pooling and the three-input INT8 graph -- would have
shipped unnoticed.

* Document both bundled embedder models

* Register the in-process provider in standalone CLI mode

Provider registration happens in metabase.core.core/init!, which `--mode complexity-score`
skips entirely. A synonym axis configured for the in-process provider therefore failed with
:provider-not-registered on the CLI while working on the server.

The activation moves into metabase.embeddings.startup so both entry points run the same
routine instead of drifting apart.

* Assert the descriptor the in-process synonym embedder sends

The test claimed the embedder passes the MiniLM descriptor to the provider SPI but only
checked that it was a function, so an Arctic descriptor would have passed too.

* Pin the CLI's provider registration with a test

Nothing caught the standalone CLI skipping provider registration the first time, so assert
appdb mode registers before it scores.

* Honour :snowplow? false in the ai-service adapter

Routing single embeddings through the batch SPI dropped the option, so the health and
circuit-recovery probes started reporting their synthetic embeddings as organic token usage.

* Make the dimension-mismatch test reach pgvector

Shrinking the model with assoc kept the old embedding-space id, so resolution failed and the
query never ran -- the same path search-degrades-when-configured-model-cannot-be-resolved-test
already covers. Use a self-consistent model, stub the compatibility preflight aside, and assert
the warning that only the 22000 branch emits.

* Use dynamic redefs for the mock index's table suffix

Pre-existing on master: a plain defn redefined on one thread, so nothing here needs the root
swap. Same mock-index value either way.

* Document the tiny bundled models as a test tier

* Fix index-health-check false degraded report

row->index's :embedding-model now carries :embedding-space-id and
:embedding-spi-version, which get-configured-embedding-model doesn't
return, so the equality check in index-health-check always failed and
reported every healthy index as degraded. Compare only the keys
settings actually configure.

* Give v1 Arctic-embed models their own query prefix

model-family-query-prefixes matched every snowflake-arctic-embed
model name and prefixed queries with "query: ", which is only correct
from v1.5 onward. v1 models (including the bundled xs test-tier
model) expect the longer CQE-style instruction instead.

* Carry the plugin-name pin into embeddings.startup

The provider registration this branch moved into
embeddings.startup/ensure-in-process-provider! re-duplicated the
manifest name as its own private literal. Make it the one public
constant both the startup logic and the tests reference, instead of
reintroducing a third copy.

* Correct the Arctic Embed query-prefix cutover to v2.0

The generation split is at v2.0, not v1.5. m-v1.5 was getting "query: "
when it wants the longer instruction, and m-long matched neither pattern
so it got no prefix at all.

Replace the two mutually-exclusive patterns with an ordered first-match-wins
table. That drops the exclusivity constraint the previous shape needed and
covers every model in the family.

* Stop the query-prefix setting claiming `query: ` family-wide

The description predates the generation split and tells admins the whole
snowflake-arctic-embed family expects `query: `, which is wrong for the
bundled model.

* Anchor the Arctic Embed patterns to the model-name segment

Unanchored, they matched any name merely containing the family name, and
a false positive there is unsuppressable: `ee-embedding-query-prefix` can
replace a matched prefix but never remove one.

* Compare embedding spaces in the semantic health check

Settings can stay identical while a provider upgrade resolves the same
model into a new space, stranding the index. The space id is what
actually decides whether the stored vectors are still usable, so compare
that rather than provider/model/dimensions.

Resolution reaches the provider, so it can fail outright when the plugin
is absent or the platform is unsupported. That is its own incident, not
a model mismatch.

* Document the in-process semantic search model

* Add in-process semantic search integration test

* Drop incompatible legacy entity indexes

* Address embedder review findings

* Quote legacy index identifiers separately
2026-08-19 16:39:01 +02:00
Chris Truter
c9ee97eba1 Run LLM context queries with user-aware database access (#80149) 2026-08-19 13:31:13 +00:00
Nicolò Pretto
1da1d694ff fix(eajs): allow copying to clipboard in chrome (#80072) 2026-08-19 15:01:37 +02:00
Ben Grabow
ff8e7993d3 Internal load-namespaces tool to check for compilation errors (#80009)
* Internal load-namespaces tool to check for compilation errors

* Ignore load-namespaces in module checks

* lint load-namespaces

* Reject positional args

* Test the new utility

* Guard against nonexistent root.
Make the code more testable.
2026-08-19 07:56:18 -05:00
Nemanja Glumac
ac3cd1d211 Make mage driver-decisions honor the "force-run" (#80128) 2026-08-19 14:44:04 +02:00
Nemanja Glumac
22581bf881 Add e2e instructions for copilot (#80097) 2026-08-19 14:06:55 +02:00
Romeo Van Snick
5d77c90c36 Delete the reference loading slice (#80137)
* Give the reference edit forms their own save state

* Pass reference fetch state as props and drop it from the slice
2026-08-19 21:08:41 +10:00
Romeo Van Snick
2ca2e0f6d3 Carry the upload destination name in upload state (#80135) 2026-08-19 21:08:31 +10:00
Romeo Van Snick
a444fcd2af Preload the app bundles from the top of <head> (#80134)
* Preload the app bundles from the top of <head>

* Narrow the preload hint type with flatMap
2026-08-19 20:51:31 +10:00
Nicola Mometto
1a84c0616c remove unused transformer (#80117) 2026-08-19 09:41:16 +01:00
Romeo Van Snick
16906d4383 Load CodeMirror editors lazily so they leave the initial bundle (#79809)
* Load CodeMirror editors lazily so they leave the initial bundle

* Inline the single-use dynamic imports into the lazy calls

* Share the codemirror split chunk with the embedding SDK build

* Split the snippet, Python and public document pages by route instead of wrappers

* Dedupe modules shared by async chunks instead of naming CodeMirror

* Drop the duplicate metrics viewer route that master already added

* Keep barrel imports and mark the expression language side-effect free

* Move the action editor boundary to the new action-creator module

* Drop the codemirror cache group the SDK config no longer needs

* Drop props types exported for wrappers that route splitting replaced

* Delay the code fallback so a fast chunk load never flashes uncoloured code

* Render nothing while the code editor chunk loads

* Delay the whole expression widget instead of showing a spinner inside it

* Open the editors from routes and modals instead of loading states

* Fold the Metabot preview chunk into the message's own loading state

* Fold the query editor chunk into the transform pages' own loading state

* Prefetch the expression widget from the notebook so its popovers open ready

* Await the run error message now that the code editor loads lazily

* Hold the custom column popover shut until the expression widget arrives

* Wait for the expression editor to hold focus before typing into it

* Type into the page directly when asserting the read-only native editor

* Read the editor value only once its document stops changing
2026-08-19 16:02:49 +10:00
Nathan Voxland
dbe9a0acc1 Improve geojson and http requests (#80111) 2026-08-19 00:29:36 -05:00
Romeo Van Snick
3a0ac1ecba Move the segment fetch thunks into the reference module (#80059)
* Move the segment fetch thunks into the reference module

* Inject the settings endpoints in the api cache seed helper
2026-08-19 15:08:10 +10:00
Romeo Van Snick
2d2e90cf66 Fetch every plugin route factory when its route is matched (#79637)
* Fetch the routes-only plugin pages when their route is matched

* Fetch the remaining plugin route factories when their route is matched

* Load the AI settings pages as one chunk

* Say why these route files register no prefetch

* Derive the AI settings prefetch paths from the route paths

* Prefetch the model migration page like its siblings

* Count the python transform pages master already made lazy

* Give each AI settings page its own chunk
2026-08-19 15:05:14 +10:00
Nemanja Glumac
b7f4881ae8 Decide whether to build uberjar based on files changed (#80129) 2026-08-19 16:02:50 +12:00
Ryan Laurie
9b71465b85 Use eng stats api for data uploads (#79929) 2026-08-19 15:16:38 +12:00
Fraser
522b0458e0 Create FE Modularization skill (#80132) 2026-08-19 14:48:53 +12:00
Romeo Van Snick
70181282f3 Keep tiptap and prosemirror out of the initial bundle (#79964)
* Move useEntityData out of the tiptap SmartLink node module

* Load the Metabot chat panel lazily

* Load the public document page lazily

* Recover from a failed fetch of the Metabot chat panel

* Fetch the public document chunk at startup
2026-08-19 12:40:29 +10:00
Romeo Van Snick
1f9d8f05ba Give the remaining route subtrees their own chunks (#79738)
* Fetch the data studio, model and snippet pages when their route is matched

* Fetch the monitor and timeline pages when their route is matched

* Name the chunks the route loaders build

* Fetch the AI auditing and library pages when their route is matched
2026-08-19 12:37:47 +10:00
Romeo Van Snick
b8584138cb Resolve the segment thunks' table and database from responses, not the mirror (#80058) 2026-08-19 12:25:52 +10:00
Romeo Van Snick
4525787c27 Fetch reference pages through one loading bridge instead of two (#80056) 2026-08-19 09:35:51 +10:00
Nemanja Glumac
63f77fa7c3 Decide whether jobs run at all before hitting the test-plan (#80091) 2026-08-18 23:06:47 +00:00
Denis Berezin
4c73becf0e Fix e2e test flakes (#79791)
* Fix e2e test flakes

* Revert timeout
2026-08-19 02:04:21 +03:00
Egor Grushin
741ed577d9 Security Center: Markdown (#79737)
* GDGT-2977 Style full markdown output in shared Markdown component

- Markdown.module.css covers every CommonMark+GFM element as :where() base layer
- compact prop for line-clamped previews
- disallowHeading renders headings as paragraphs instead of unwrapping to text
- drop duplicated rules from AIMarkdown, delete AdvisoryCard.module.css
- kitchen sink fixture + stories + spec failing on unstyled elements

* GDGT-2977 Trim markdown specs to our own behavior

- drop tests that only exercise react-markdown/remark-gfm
- cover urlTransform (metabase://, data:image, unsafe protocols)
- assert headings render as p when disallowed
- merge AdvisoryCard markdown cases

* GDGT-2977 Fix markdown spec asserting against mantine style tag

- container.firstElementChild was the injected <style>, so every
  querySelector check passed vacuously
- render into a host element and select the markdown root from it
- javascript: urls collapse href to empty, which drops the link role

* addressing the PR's comments

* GDGT-2977 Match skeleton caption tooltip to legend caption

compact + lineClamp, per PR review

* fix small regression
2026-08-18 23:36:09 +01:00
Fraser
e81d02d15d Give dayjs a facade module (#79963) 2026-08-19 10:25:12 +12:00
Maksym Yakubych
9b159ccf78 Let conversation titles run on their own model (#78954)
* Let conversation titles run on their own model

Every LLM call read the one `llm-metabot-provider` setting, so naming a
conversation — a short, high-volume call — burned the same model the chat runs
on. Add an `llm-title-model` setting whose getter resolves rather than defaults:
the stored value, else the fastest model of the connection Metabot uses, else
that model itself. Provider types now carry a `:mini-model` alongside
`:default-model`, taken from each adapter's own supported-models list so the
fallback names a model the picker actually offers; Azure and the managed
provider have none and fall through.

Move model selection out of the AI providers card into its own Models section,
so per-use-case choices have somewhere to live. Deleting a connection clears a
title model pinned to it, reading the raw stored value so a merely-derived one
is left alone.

* Name a mini model for every provider type that has one

Mistral and Z.AI were the only types with no `:mini-model`, so conversation
titles on those connections fell through to the model Metabot chats on. Both
adapters whitelist exactly one model today, so their mini model is that same
model — the fall-through by another name, but spelled out where a real small
model goes when one is added.

Two tests hold the line. The registry map fails when a type gains or loses a
`:mini-model`, since a missing one reads as nil and quietly sends titles to
the full-size model. The listability test fails when a type names a model its
adapter will not offer: `list-models` returns the whitelist intersected with
the provider's catalog, so an id the whitelist omits never reaches the admin
picker, which renders it as a blank selection.

That second test caught Anthropic naming `claude-haiku-4-5` while the Claude
whitelist carried only `claude-haiku-4-5-20251001`, leaving the new title
picker blank on the commonest setup there is. The whitelist was the odd one
out — the adapter already defaults to the undated alias — so it loses the date.

* Sell the title model as the mini model it already was

Review asked whether each feature should get its own picker or whether one
cheap model should serve every quick task; the answer was one model. The
setting already resolved through the registry's `:mini-model` and nothing but
titles read it, so this is a rename — `llm-title-model` becomes
`llm-mini-model` — plus copy that presents naming conversations as the first
of the quick tasks rather than the point.

The derivation tests now pin the stored value to nil instead of assuming a
clean database; a crashed run had left a stored value behind, which the old
tests read as the derived one.

* Return Haiku's dated id, which is what the Anthropic catalog serves

Dropping the date assumed /v1/models would list the bare alias. Review
testing with a live key showed it does not: the catalog carries only
claude-haiku-4-5-20251001, so the undated whitelist key filtered Haiku out
of the picker, and the derived mini-model reference matched no option —
titles kept running on Haiku while the picker sat blank.

The alias stays where it always worked: inference-call defaults. The
whitelist and the registry go back to the catalog's id, which is also the
whitelist's convention — the 2025 models are dated, the 2026 models are not,
tracking what the catalog returns for each.

* Move a pinned mini model with the deployment its connection now serves

Editing an Azure connection's deployment already moves the Metabot
selection, because the reference bakes the deployment name in. A mini model
pinned to that connection went stale the same way, so it moves the same
way. A derived mini model is left unstored — it follows the Metabot
selection on its own.
2026-08-18 17:01:37 -03:00
Jeff Bruemmer
fbd41598bf docs - embedding intro edits (#80109)
* auth props

* edits

* Update docs/embedding/introduction.md
2026-08-18 19:47:56 +00:00
github-automation-metabase
b4eb9ba39f Auto-update documentation for master (#80030)
Update generated docs (master)

Co-authored-by: github-actions <github-actions@github.com>
2026-08-18 15:41:17 -04:00
Maksym Yakubych
3e6fe863dd BOT-1642 Show a clearer error for a second AI-managed subscription (#80017) 2026-08-18 18:53:15 +00:00
Kevin Kelleher
da492ba066 Fix cardinality cache collisions (#79856)
* use json_query as the cache key to prevent cardinality cache collisions

* fix e2e, persist default settings that depend on cardinality

* review comments
2026-08-18 13:26:55 -04:00
Anderson
b1e7ed37dd Add setting to display percentages instead of values on stacked bar charts (#79922)
* Display percentages instead of values on stacked bar charts

Adds a Stack value format setting that lets stacked bar chart segments display their percentage of the stack total.

Addresses #67702

* Handle percetanges for negative values

* Show decimals for fractional stack percentages

* Show percentage totals for stacked bars

* Improve condition readability

---------

Co-authored-by: Abhijnya K G <53806883+Abhijnya002@users.noreply.github.com>
2026-08-18 14:21:20 -03:00
Maksym Yakubych
2c98f5d2bf BOT-1989: Restore the conversation title size on the Conversation detail page (#79996) 2026-08-18 17:13:32 +00:00
Nick Fitzpatrick
15e8fd286d Update user-visibility subtext (#78147) 2026-08-18 14:08:08 -03:00
Maksym Yakubych
35a1199a35 Support multiple LLM provider connections at once (#78860)
* Invalidate the cached models list when credentials change

A credential write can change which models the provider serves, e.g. a different Bedrock region, Google location, or
Azure resource.

* Support multiple LLM provider connections at once

Metabot could only ever be connected to one AI provider, because each
provider's credentials lived in its own hardcoded settings
(llm-anthropic-api-key, llm-azure-api-key/-base-url, llm-bedrock-*, ...)
and llm-metabot-provider named the single provider in use. Adding a
provider meant editing a `case` in six namespaces plus three constant
maps on the frontend, and two keys for the same provider were impossible.

Replace that with a list of provider *connections* stored in one
`llm-providers` JSON setting, fronted by a CRUD API rather than edited as
a raw blob -- the same shape the `oidc-providers` setting already uses.
Provider types are now a pure-data registry describing each type's
credential fields, so the admin UI renders a connection form from the
server descriptor instead of a bespoke component per provider, and the
model picker lists models from every connection, grouped by connection.

Connection keys are slugs defaulting to the provider type, so existing
`llm-metabot-provider` values keep resolving unchanged. `metabase`
becomes an ordinary connection, which collapses the three-segment
`metabase/provider/model` special case into the generic rule. The
per-provider env vars keep working and each synthesizes one read-only
connection; MB_LLM_PROVIDERS puts the whole list under env control.

* Fix managed provider handling in the connection list

Five issues from reviewing the new admin UI, all around the Metabase-managed
provider:

- The managed provider's models were fetched from the proxy, so instances
  that cannot reach it showed "Couldn't load models". The proxy serves one
  benchmarked model rather than a listable catalog, so the registry now
  carries that catalog directly and nothing is fetched. It doubles as the
  allow-list `llm-metabot-provider` is validated against, replacing the
  separate copy in metabot.settings.

- The managed provider had an Edit action that opened an empty modal (or one
  offering to re-Connect something already connected). There is nothing to
  edit, so managed connections now only offer Remove.

- It could be added twice. Provider types can declare themselves singletons;
  the API rejects a second connection and the FE drops already-connected
  singletons from the Add dropdown.

- The empty state repeated the section description verbatim.

- Cards showed the raw type slug under the name ("Metabase" / "metabase").
  The provider type's label is now shown only when it differs from the
  connection's name, i.e. when the admin renamed it.

Also drops the startup step that materialized a `metabase` connection on any
proxy-configured instance, which made the managed provider look connected when
nobody had connected it. It is now materialized only where a `metabase/...`
model reference is actually written: the legacy metabot-v3 sync, and the
migration of an instance already pointed at it.

* Address review comments on the provider e2e helpers and managed setup spec

- Drop the doc comments added to e2e-metabot-helpers.ts.

- Move the mock LLM server's base URL into the connection's own config rather
  than the legacy `llm-anthropic-api-base-url` setting. Both worked, because
  the adapters fall back to that setting when the connection carries no base
  URL, but relying on the fallback left the connection describing one endpoint
  while requests went to another.

- Assert that the add-on purchase really does precede the connection POST,
  which the test claimed in its name but never checked, and add the missing
  case: a failed purchase must not create the connection.

* Stop repeating the model family in the model picker

Providers already carry the family in the model's display name, so prefixing
it with the group produced labels like "Opus · Claude Opus 4.8" and
"Anthropic · Anthropic: Claude Haiku 4.5". Show the display name on its own.

The group still comes back from the API: the backend orders each provider's
models by it, so dropping it here keeps the grouping-driven order without
showing the label twice.

* Select a default model on connect, and simplify the ad-hoc connect view

- Connecting a provider when Metabot had nothing usable to run on now points
  `llm-metabot-provider` at that connection's default model, so the first
  provider you add leaves the instance working instead of connected with no
  model selected. An existing selection that still resolves is left alone, so
  adding a second provider does not silently switch Metabot over to it. The
  "did it have a usable model" check has to run before the new connection is
  saved: the setting's default names the `anthropic` connection, which a
  freshly saved Anthropic connection would otherwise satisfy retroactively.

- The ad-hoc "connect to a model" modal no longer renders the admin list with
  an "Add a provider" button that opens a second modal. It is one connect form,
  and once a provider exists it is the model picker. Extracted
  ProviderConnectionForm so the admin modal and this view share it.

- Show a skeleton while the connection and provider-type lists load.

- Replace the add-on ordering assertion, which compared indexes into the call
  history, with a purchase that only resolves once the test releases it: the
  connection must not be created while it is in flight.

* Fix the managed-provider-limit spec for the single-form connect view

The ad-hoc connect modal no longer offers an "Add a provider" button that
opens a nested modal, so assert on the provider picker the form now renders
directly.

* Show the model picker after connecting from the ad-hoc modal

Connecting a provider in the ad-hoc view called onClose straight away, so the
modal shut before the picker it was supposed to hand over to could render.
Stay open and show the picker instead, except for a provider that serves a
single fixed model (the managed one), where there is nothing to pick.

* Split the AI provider list and setup views apart

The one component branched on an `isModal` prop to be either the admin list
(cards, add/edit/remove, model picker) or the ad-hoc connect flow, which meant
neither was composable and the modal branch read as an afterthought. Split it
into AIProviderList and AIProviderSetup, each owning its own data and state,
sharing ProviderConnectionForm, LlmModelPicker and ProviderListSkeleton.
Call sites pick the one they mean instead of passing a flag.

Also use `useDisclosure` for the add-provider modal. `editing` and `deleting`
stay on `useState` because they carry the row being acted on rather than a flag.

* Cancel the managed AI add-on from the backend, not a plugin hook

`PLUGIN_METABOT.useOnProviderRemoved` only ran when the managed connection was
removed through that one component, so removing it any other way — the API
directly, or the fallback path in the delete handler — left the customer paying
for an add-on they no longer used.

Move the cancellation into `DELETE /api/llm/providers/:key` behind a
`defenterprise`, which is where the invariant actually holds: removing the
managed connection cancels its add-on for every caller. The EE side decides
which product type to cancel from the same premium features the frontend was
reading, and keeps the rule that an instance on legacy tiered pricing with no
metered offer to move to is left alone. Extracted the Store call the
cloud-add-ons endpoints already make into `cloud-add-ons.core` so both share it.

The plugin hook and its EE implementation are gone.

Also clear fetch-mock state between AI provider spec setups. The file grew a
second `describe`, and its routes and call history leaked into whichever suite
jest packed next into the same worker.

* Address review comments on the model picker and provider API shape

- Snake_case the two kebab-case keys the provider-type endpoint put on the
  wire (docs-url, default-model), matching display_name alongside them. The
  internal registry keeps its idiomatic kebab-case keys.
- Drop the model picker's own success toast: useAdminSetting's updateSetting
  already toasts both outcomes, so picking a model fired two toasts and the
  error path only looked unhandled.
- Show SetByEnvVar under the picker when llm-metabot-provider comes from an
  env var, so the disabled Select says why.

* Remove the unused GET /api/llm/list-models endpoint

Nothing called it: no frontend, SDK, or e2e reference, only its own tests. It
listed models for whichever connection Metabot was pointed at, which
GET /api/llm/models now covers for every connection.

It is in the generated OpenAPI spec, so this is a breaking change for anyone
calling it directly. The spec is regenerated by the openapi-self-healing CI job
rather than by hand — a local run rewrites far more of the file than this change
touches.

* Pick a provider type from an icon grid instead of a dropdown

Replace ProviderTypeSelect with a ProviderTypePicker card grid, giving each
provider type an icon the backend names in the provider-type registry.

Connections now also report the env vars that configured them, so the UI can
name them rather than just saying the connection is env-managed.

* Use real provider logos, drop the stepper, lead with Metabase

- Drop the two-step "Provider / Configure" indicator. The modal swaps content
  in place rather than paging, so it rendered as a lone empty circle next to a
  floating label and conveyed no progress.

- Ship brand logos for the five bring-your-own-key providers, mirroring how
  driver logos already work: a file per provider under
  app/assets/img/llm-providers, looked up by provider type, falling back to the
  registry's icon when there is no logo (the managed provider keeps `metabot`).
  Sourced from svgl. The three monochrome marks are rewritten to currentColor so
  they survive dark mode; Azure and AWS keep their brand colour. Bedrock uses the
  AWS mark for want of a Bedrock-specific one.

- Offer the Metabase-managed provider first when the instance can actually use
  it, since it is the option to reach for before bringing a key.

- Fix the blank managed connect screen. MetabaseAIProviderSetup decided between
  "connect" and "already connected" from `llm-metabot-configured?`, which under
  multi-provider is true as soon as *any* connection works — so adding Metabase
  alongside an existing Anthropic connection rendered the connected-state card
  for a subscription that was never bought, which draws nothing. The caller now
  passes whether the managed provider itself is connected.

The provider-types order assertions became environment-dependent once ordering
keys off proxy availability (this worktree's .env sets MB_LLM_PROXY_BASE_URL), so
they now assert the set of types plus the lead position explicitly.

* Drop the provider picker heading that repeated the modal title

* Right-align the managed provider footer to match the other setup screens

* Focus the confirm button so Enter confirms instead of dismissing the modal

* Tuck the optional connection fields behind Advanced settings

The connect form led with Display name and API base URL, so the one field
that actually matters — the API key — was buried among inputs that already
have working defaults.

Show only the fields a provider type marks required, and collapse the rest
plus Display name into an Advanced settings disclosure at the bottom. The
split comes from the field descriptors rather than a hardcoded list, so
Azure (whose base URL is required) keeps both inputs up front.

Editing a connection that already customized an optional field opens the
section, so a non-default value is never hidden.

* Let the registry say which connection fields are advanced

Inferring it from `required` was wrong for Bedrock. `required` means
validation fails without a value, which is a different question from whether
an admin has to think about the field. Region has a default, so it is not
required — but it decides which models are reachable and where inference
runs, so burying it hands EU customers a silent us-east-1 default.

Add an explicit `advanced?` to the field descriptors and split on that. Only
the base URLs and Bedrock's session token carry it; region now stays up
front where it belongs.

* Cover submitting the provider connection form with the enter key

* Treat the single-provider env vars as a supported way to configure a provider

They were named and documented as a legacy compatibility shim — `legacy-env-connections`,
`migrate-legacy-provider-connections!`, and a doc line promising they "keep working".
That undersells them: configuring one provider by environment variable is the simpler
option when you do not want to hand-write the `llm-providers` JSON, and it is resolved
from the environment on every read, so editing a variable is picked up rather than being
frozen into a copy at first startup.

Rename to say what they are, and reword the docs to position the two mechanisms as
alternatives rather than old and new. The one-shot at startup only ever adopted
credentials stored in the app DB, so name it for that instead of for migrating away from
something we are keeping.

* Type the LLM provider logo lookup and move it out of the icon component

The logo map lived inline in ProviderTypeIcon as a Record<string, string>, so a new
provider type silently fell back to the generic icon instead of failing the build.

Give provider types a LlmProviderTypeName union and key the map on it, so adding a type
without deciding on a logo is a compile error. The map moves to metabot/constants.ts and
the lookup to metabot/utils/llm-provider.ts, matching how databases/ splits ENGINE_LOGO
from getEngineLogo.

* Focus the first credential field once a provider is picked

* Show the Metabase logo for the managed provider

* Only badge a provider connection when something is actually wrong

The green check was driven by `usable`, which only asks whether the required fields are
non-blank. The connect form already refuses to submit until they are, so every connection
made through the UI got a check — including one whose API key the provider rejects, right
above a "Couldn't load models" error saying so.

Drop the check and keep the warning, which still carries a real signal for connections
that arrive incomplete from MB_LLM_* or MB_LLM_PROVIDERS, and label it so it says what is
wrong rather than just showing a red triangle.

* Lighten the provider list to divider-separated rows

Each connection was a bordered Card, but the list already sits inside SettingsSection,
which is itself a bordered, padded card — so every row read as a card inside a card.

Follow DatabaseInfoSection instead: one container, rows separated by a divider. Also drop
the icon from 40 to 32, since the tile no longer has to anchor a card of its own.

* Tighten the provider list spacing and align the add button with the rows

The list and its add button sat in the same lg-gap stack as the model picker, so a control
that belongs to the list was pushed 24px away from it, as far as the next setting. Group
them in their own sm stack and keep lg for the gap to the picker.

The subtle button also carries horizontal padding, so its + sat indented from the provider
logos below it. Drop the padding, matching the Advanced settings toggle on the connect
form, so both start on the same edge.

Tighten the section header gap from lg to md via the stackProps SettingsSection already
exposes for this.

* Fetch provider models once and report failures on the provider that failed

A model-loading failure was only visible under the model picker, which reads as a problem
with the picker rather than with Anthropic. It also meant the failure was invisible
anywhere a picker was not rendered.

/api/llm/models already returns every connection in one response with a per-connection
error, and RTK Query already caches it under the llm-models tag, so the fetch is shared by
construction. Wrap it in useLlmConnectionModels, which hands back the grouped select
options and the errors keyed by connection. Any future picker goes through that hook and
gets the same cache entry and the same keying, rather than re-deriving either.

The provider row now renders its own error and the picker no longer lists them; the picker
keeps the request-level error, which is a different failure.

* Pick the provider from a pasted API key

Paste a key anywhere on the provider grid and it jumps straight to that
provider's form with the key filled in, ready to connect.

The registry already declared a `:prefix` per credential field — the backend
validates against it — so this just exposes it and reuses it for recognition.
Anthropic, OpenAI and OpenRouter carry one; nothing else is guessable.

Longest prefix wins, since OpenAI's `sk-` also matches an Anthropic
`sk-ant-` key. Text that matches nothing is left alone.

* Leave the paste-to-detect shortcut unannounced

Drop the hint under the provider grid. The shortcut stays; it is just for
people who find it. `canRecognizeApiKeys` existed only to decide whether to
render that line, so it goes too rather than sitting unused.

* Gate the managed provider usage panel on the features that fill it

The panel was shown whenever the metabot plugin was enabled, which is a
weaker condition than having usage and pricing to report. Move the check
behind a hasMetabaseManagedProviderDetails plugin hook so the enterprise
side decides from the premium features, and OSS answers false.

* Register the Mistral and Z.AI provider types in the connection registry

master added both providers to the single-provider settings architecture this branch
replaces, so rebasing dropped them from the admin UI: the adapters, the model listers and
the MB_LLM_MISTRAL_*/MB_LLM_ZAI_* settings all survived, but nothing offered either type
as something an admin could connect.

Registry entries restore that, and adding them to `single-provider-settings` is what keeps
an instance already configured through those env vars working — the same entry drives both
the env-resolved connection and the app-DB adoption at startup.

* Resolve provider connections in the LLM call tests

`parse-provider-model` stopped deriving a provider from the model string's first segment
and started resolving it against a configured connection, but this file still called it
with nothing configured, so every test that reached an adapter failed on "No LLM provider
connection named". Configuring the default connections is the fixture those tests were
always implying.

The invalid-format cases now assert the error the new resolution actually raises: an
unparseable string and an unknown connection are the same failure, because a name that is
not a connection is all "unknown provider" can mean once the registry is the source.

* Move the onboarding AI step onto the provider connection list

The setup wizard's AI step was built against the single-provider form this branch removed,
so it still imported `AIProviderConfigurationForm` and the provider enum and no longer
compiled. It now renders `AIProviderSetup`, the same connect-then-pick-a-model view the
ad-hoc modal uses, so onboarding and the admin page stay one flow rather than two.

Preselecting `metabase` goes away with the dropdown it was written for. The backend already
sorts the managed provider first in the type picker, which is what the preselection was for,
and picking it in a grid of logos is not a step the wizard needs to skip on the admin's
behalf.

* Add the Mistral logo and make a missing provider logo fail the build

LlmProviderTypeName is written by hand and had drifted: the registry gained mistral and
zai, the union did not, so the exhaustive LLM_PROVIDER_LOGOS record was exhaustive over a
stale set and both types fell through to the generic icon without a type error.

Add them to the union, which does make the record fail to compile until each is decided.
Mistral gets its mark from svgl; svgl has no Z.AI logo, so zai is an explicit null and
keeps the fallback icon until we have a real asset.

Nothing links the registry to the union, so add a backend test over the registry's type
names that names both frontend files to update.

* Thread per-connection credentials through the Mistral and Z.AI call paths

Both adapters took :credentials in list-models but ignored them in mistral-raw/zai-raw,
which read llm-mistral-api-key / llm-zai-api-key directly. So a connection's models listed
under its own key while every actual chat call went out under the global setting: a second
Mistral connection would silently answer with the first one's key, and a connection with no
global setting configured would fail outright.

Match the openrouter/claude shape — credentials first, configured settings for whatever
they do not carry — for both the key and the base URL, and cover the call path, which no
adapter had a test for.

* Give Z.AI its logo

Taken from z.ai's own logo.svg, since svgl carries no Z.AI mark. Stripped the Illustrator
export's ~10.5KB of unused .st0-.st194 declarations — only two of them were referenced —
and kept the geometry and colours exactly as shipped.

Every registered type now has a mark, so the fallback is only reachable when the backend
serves a type this union has not caught up with. Test it that way instead.

* Let instances that are only offered managed AI reach the purchase flow

The managed provider tile is enabled by `type-available?`, which asks whether
the LLM proxy URL is readable. That setting was gated on `metabase-ai-managed` or
`metabot-v3` — features granted only by the AI add-on products — while
`offer-metabase-ai-managed` comes from the starter and pro-cloud plans. Every
cloud instance eligible to buy the add-on therefore saw the tile as unavailable
and had no way to reach the purchase flow that grants the feature.

* Cover per-connection credentials on the remaining providers' call paths

Every provider tested credentials through list-models and none through the call path,
which is why Mistral and Z.AI could list models under a connection's own key while sending
every chat request under the global setting.

Claude, OpenAI and OpenRouter resolve credentials inline like Mistral did, so they get the
same explicit/blank pair. Azure and Bedrock instead forward :credentials into a shared
request helper the list-models tests already exercise, so what is worth pinning there is
the forwarding: with no settings saved, the request only goes out if the passed-in
credentials arrive.

* Pin every provider type's default model

The removed settings-put-connect-*-defaults-model tests asserted this per provider; the
connection API replaced them with one test that a first connection gets a model selected,
and provider-types-test only spot-checked anthropic and azure. A typo in mistral's or
zai's :default-model would have shipped silently.

* Alias metabase.llm.test-util as llm.tu, not lct

The letters do not spell anything in the namespace, so at a use site like
lct/default-connections there is no way to tell which namespace it came from. The repo
convention for a module's test utilities is <module>.tu — lib.tu, notification.tu.

Add it to kondo's :consistent-alias map too; it was absent, which is why the alias could
be picked freely in the first place.

* Make the whole provider header toggle the usage details

Only the chevron was clickable, so the icon and name next to it did nothing — unlike
Accordion.Control, which is what makes the entire header hit the toggle everywhere else.

Promote the header itself to the control and leave the chevron as the affordance, with
the options menu as a sibling rather than nested inside it. SetByEnvVar moves below the
header for the same reason: it carries a docs link, and a link inside a button is invalid.

Not Mantine's Accordion, because expandability is per-row here — only the managed provider
has a panel, and wrapping the rest would give them a control that opens nothing.

* Let the provider modal play its closing animation

The modal hardcoded opened and the parent mounted it conditionally, so closing unmounted
it outright and Mantine never got to run the exit transition. Opening animated because
mounting still runs the enter one.

Drive it with opened instead and keep it mounted, the way ConfirmModal a few lines below
already does. Clearing the edited connection moves to onExitTransitionEnd so the title
does not flip to "Add a provider" on the way out.

* Fix the edit modal never closing

Openness was derived from isAdding || editing != null, but closing only cleared isAdding
and left editing to be cleared after the exit transition — which could not start, because
editing still held the modal open. Editing a provider trapped it.

Track openness in its own disclosure so closing is one flag, and let editing carry only
which connection to show, cleared after the transition. Adding worked before, which is why
the existing test missed this; cover closing from edit too.

* Revert the provider modal close-animation work

Reverts 7e78358857d and 1a7c37d6412. The missing close animation is not this component's:
the Modal theme attaches enter-only CSS keyframes (popInFromBottom, fadeIn) to every modal
in the app and animation.module.css defines no exit counterpart, so driving this one with
opened bought nothing but state to keep in sync — and the first attempt at it trapped the
edit modal open.

Back to mounting it conditionally. Keeps the fetch-mock teardown from the reverted commit,
which is unrelated hygiene: without it this spec's routes leak into
MetabotAgentDataSourcePills and fail it in a combined run.

* Serve a request from its own connection's credentials, nothing else

Every adapter fell back to the single-provider `llm-*-api-key` setting when
the credentials it was handed came up blank. That made sense while the two
systems coexisted, but the env vars are themselves connections now, so the
fallback could only ever fire by lending one connection another's key — a
connection whose key an admin cleared would silently keep working on someone
else's, while `connection-usable?` reported it as broken.

Resolution now fills each field's registry default into the config, so
`:credentials` carries everything the adapter needs, including the base URL
it used to read from the settings. The adapters take credentials and nothing
else, and throw when they are missing.

Tests drive the adapters the way production does — by passing credentials —
rather than by setting a global and calling with none.

* Finish removing the settings fallback: Mistral, Z.AI, and the docs

Mistral and Z.AI landed alongside the previous commit and were copied from
the old shape, so both still reached for their `llm-*-api-key` setting when
the credentials came up blank. A half-applied invariant is worse than either
state — the next adapter gets copied from whichever one you happen to open.

The docstrings across all seven still advertised the fallback I had already
deleted, including the canonical `LLMRequestOpts` contract. Code that lies
about itself in the one place people go to read the rule is how the rule
comes back.

* Drop the editorializing line from the credentials doc

The entry should say what the field is, not argue for it.

* Reject runtime writes to the per-provider credential settings

These configure a connection only when set by an environment variable, which
llm.provider/connections resolves on every read. Writing one to the app DB landed somewhere
nothing reads: the settings API returned 200 and the request kept using whatever the
connection held. The e2e specs are the proof — one pointed llm-anthropic-api-base-url at a
mock server and would have talked to the real Anthropic instead.

Mark all sixteen :setter :none, so a write raises "you cannot set X; it is a read-only
setting" instead. Tests keep working because with-temporary-setting-values passes
:bypass-read-only?; connections are managed through /api/llm/providers.

Give each a :doc, since env-var-dox drops a read-only setting from the generated docs unless
it has one, and format-config-name then labels them environment-variable-only.

The custom setters and normalize-llm-base-url are unreachable now, so they go; base URLs on
connections were never normalized, only the setting path was.

* Disable the managed-AI Connect button instead of hiding it

An admin who had not ticked the Terms box saw no Connect button at all, so
the modal gave no sign that connecting was possible or what unlocked it.
Show it disabled: the checkbox is now visibly the thing standing in the way.

The gate moves out of `connectAction`, which keeps meaning "there is nothing
to do here" — already connected, or not an admin — and stays null in those
cases so the button remains hidden.

* Let a connection name the model it serves, so Azure is selectable

Azure's listing endpoint returns the regional catalog rather than the customer's
deployments, so list-models always came back empty and the picker had nothing to offer: you
could connect Azure and never pick a model. Master collected the deployment alongside the
credentials; the connection API dropped that and nothing replaced it.

Give the registry a :model-field naming the config key holding the model, and Azure a
required Deployment field. /api/llm/models still makes the call — it is what verifies the
credentials — but reports the configured deployment instead of the empty list. The connect
form needs no frontend change: it is descriptor-driven, so the field renders itself.

Two things this pulled in. config-complete? now demands the deployment, which is right for
"can this connection serve a request" but wrong for the adapter's auth check, so that
gets credentials-complete?, which ignores the model field. And the environment path needs
to express the deployment too, or an env-configured Azure connection reports unusable, so
llm-azure-model joins the single-provider settings.

Also give the unsupported-model error a 400: the deployment is typed by hand, so a bad
prefix is a bad request, not an outage.

* Import the provider logos instead of pointing at asset paths

The filename map plus a path-building helper came from ENGINE_LOGO, but that is the older
pattern: 57 places import an SVG as a module, and DatabaseHostnameWithProviderField's
ProviderIcon does exactly this — vendor logos colocated in the component directory, imported
and mapped by provider name.

So colocate them and import them. The bundler resolves the reference, which means a renamed
or missing file fails the build rather than 404ing at runtime, and the path string stops
being something to keep in sync by hand. The map moves next to the imports, which drops
metabot/utils/llm-provider.ts and the constants entry entirely.

The per-file src assertions go: jest maps every SVG to one stub, so what is left to check
is which of the three marks renders — the Metabase logo, a vendor image, or the fallback.

* Judge the onboarding AI step on a usable connection, not the first listed

connections[0] answered three questions it could not answer well. A listed connection is
not necessarily one that can serve a request — one configured by env var with a field
missing reports usable false — so onboarding would call itself connected, hide the skip
button, and report that provider to analytics.

Ask for the first usable connection instead. The skip button also waited on the list being
empty, which is equally true while the request is in flight, so it flashed on every load;
gate it on isLoading as well.

The completed-step title inferred skipping from a missing label, but the label comes from
the provider-types request, so a connected provider with that request still in flight read
as skipped. Pass whether a provider connected separately from what it is called.

* Match the loading skeleton to the list it stands in for, and colocate it

It still described the card-per-connection layout: two 4rem blocks with rounded corners and
a gap between them. The list has been divider-separated rows for a while now, so the
skeleton was a size and a shape that no longer appear anywhere.

Rebuild it from the row it mimics — PROVIDER_ICON_SIZE plus the row's py="sm" — with a
divider between and the button that follows, and move it into AIProviderList.tsx beside
those constants, so the next layout change has it in view rather than a file away.

Cover it too: nothing asserted the skeleton rendered at all, which is how it drifted.

* Keep the Metabase mark on the managed provider when whitelabelled

LogoIcon resolves through PLUGIN_LOGO_ICON_COMPONENTS, which the whitelabel plugin fills
with the customer's logo. That is right nearly everywhere, and wrong here: this row is
Metabase's own hosted AI service, so a whitelabelled instance was labelling it "Metabase"
— the registry string is not rebranded — beside someone else's logo.

Use DefaultLogoIcon, as WelcomePage and SdkUsageProblemBanner already do where the mark has
to be Metabase's regardless of branding.

* Stop standing in for an unknown provider with a real one

Gemini is a provider we could plausibly add, and the day we do, every test asserting it is
unknown starts asserting the opposite of what it means. Use a name nobody will ship.

Same for the Azure family placeholder, which picked gemini for a family Azure does not
serve. The genuine google/gemini-2.5-flash model references stay.

* Match on the secondary button instead of nesting a && inside a ternary

isEditing ? (onCancel && <Cancel/>) : <Back/> reads as two branches but is three, and the
one that renders nothing is the one you cannot see. The rest of this component already
picks its shape with match, so this now does too — and onCancel is narrowed by the pattern
rather than guarded separately.

* Pick the Azure model provider from a dropdown again

The rewrite collapsed master's Model provider select and Deployment name
input into one text field where the admin types `openai/` as a prefix. Split
them back apart and compose the model from the two, so the wire family is
picked rather than spelled.

`:model-field` becomes `:model-fields`, a vector composed with `/`, which is
the shape the Azure adapter already parses back out. A required field the
registry gives a `:default` now counts as filled — the form shows it
pre-selected, and resolution supplies it — so the family select does not
block Connect on a connection that simply accepted it.

* Drop two explanatory comments

Both restated what the line below already says.

* Shrink the loading skeleton to the row's content height

The skeleton painted the full 3rem row, padding included, so it read as two
heavy slabs butted against the divider. Wrap it in the row's own py="sm" and
size the bar to the provider icon instead: the row still occupies the same
height, so nothing shifts when the list arrives.

* Name the provider on the closed model picker

A model name alone does not say who serves it, so the picker gave no way to tell which
provider was selected without opening it. With several connections — two Anthropic keys,
say — that is the part you most need to see.

The option label carries the provider, since that is what a closed Select renders. The
dropdown keeps rendering the model alone through renderOption, because the group heading
above it already names the provider.

* Only treat a usable connection as one worth moving on from

Setup moved past the provider picker as soon as any connection was listed,
while the step around it counted only usable ones. A connection that exists
but cannot serve a request landed between them: the picker was replaced by a
model dropdown with nothing in it, the step still offered to skip, and there
was no way to connect anything.

That is reachable, not theoretical — the single-provider env vars synthesize
a connection as soon as one credential field is set, so MB_LLM_AZURE_API_KEY
on its own produces one Azure needs three fields to use.

* Serve the disconnect test's stream from the connection under test

The fake OpenRouter server was reached through the per-provider base-URL
setting, which no longer feeds a request now that credentials come from the
connection: the stream went to openrouter.ai and the turn finalized as an
error rather than an abort.

Disabling clj-http decompression on the LLM request also had to move from
http/post to the fn the adapters actually call, so the aborted stream no
longer races a ZLIB EOF against the cancel.

* Stop shipping an icon name the frontend never reads

The frontend keys its logos on the provider type and fails to compile for a
type it has no logo for, so the registry icon was only ever reachable for a
type the server knows and the client does not — which cannot happen when the
two ship as one artifact. Seven of the eight entries said "ai" anyway, and
the eighth said "metabot" for the managed provider, which is drawn from the
Metabase logo component before the icon is consulted.

Drop it from the registry, the response, and the type, and let
ProviderTypeIcon own the generic fallback it was already defaulting to.

* Point the setup wizard's AI e2e tests at the connection API

All three drove the single-provider flow this branch replaced: a Provider Select that is
now a grid of buttons, and PUT /api/metabot/settings, which no longer exists. The Anthropic
one also aimed the mock server via llm-anthropic-api-base-url, which stopped being writable
when those settings became read-only — it now types the base URL into the connection's own
advanced settings, which is where a connection reads it from.

The managed test additionally stubs GET /api/llm/provider-types and /api/llm/providers: the
managed type is only connectable once the LLM proxy is configured, and that has no backing
in e2e, so it is patched the same way the token features already are.

* Match the required API key label the way the unit specs do

Mantine appends a required marker to a required field's label, so the setup
spec's exact-string lookup found nothing once the API key field came from the
provider registry with :required? true. The unit specs already match on a
regex; the e2e spec was the last one on the bare string.

Also name do-with-entitlements! for the thread-unsafe settings it binds, which
Kondo flags.

* Edit the stored connection list rather than the environment-resolved one

The environment winning on read is right, but writes rebuilt the list from
connections, where an env-configured connection replaces the stored one with
the same key. Saving anything else then dropped the shadowed credentials from
llm-providers for good, so taking the env var back off did not restore them.

Editing the stored list also lets PUT report a shadowed key as env-configured
instead of 404ing, and stops the startup managed-connection write from landing
in the app DB when the whole list comes from MB_LLM_PROVIDERS -- which would
have pointed Metabot at a connection that never resolves.

* Configure an Azure deployment from the environment the way the form does

MB_LLM_AZURE_MODEL landed under a :model config key that nothing reads --
azure declares :model-fields [:model-family :deployment-name] -- so an
environment-configured Azure connection composed no model and counted as
incomplete, which hid the Metabot UI instance-wide behind
llm-metabot-configured?. Replace it with one variable per config key.

The same completeness rule dropped Azure from the startup adoption entirely,
because the credentials that existed before this branch never named a
deployment: they carried it in llm-metabot-provider. An upgraded instance was
left pointing at a connection that did not exist, with Metabot failing on every
request. A deployment names what to call rather than what authenticates the
call, so stop requiring it, and recover the pair from the model reference
during adoption so the connection can still offer it in the picker.

* Add a provider alongside the managed one instead of cancelling it

Running out of included tokens is a reason to bring your own key, not to end the
subscription. "Use a different AI provider" was wired straight to the add-on
cancellation: it charged ahead with no confirmation, offered no provider to
switch to, and left the metabase connection and llm-metabot-provider pointing at
each other afterwards, so nothing on screen changed.

Let the button open its own modal again. AIProviderSetup short-circuits to the
model picker once any connection exists, which is exactly wrong here, so let a
caller start on the connection form; the managed connection is left alone and
the model picker follows the save. Removing the connection outright is already
the list's Remove action, which cancels the add-on server-side.

* Let the single-colour provider marks follow the theme

The marks were loaded through <img>, and an SVG in an <img> is its own document:
currentColor resolves against that document's initial color, black, with no way
to reach the color the tile sets. Anthropic, OpenAI and OpenRouter therefore
painted black on a near-black tile in dark mode, and Bedrock's navy "aws"
wordmark left only the orange smile visible.

Inline those four instead, so currentColor picks up the tile's text colour --
dark on light, light on dark, which is how each brand publishes its monochrome
mark. Mistral, Z.AI and Azure keep their own colours and stay images. Bedrock's
wordmark becomes currentColor, reproducing AWS's own light and dark variants.

OpenRouter pointed at a clipPath the file never defined; harmless while it was
sandboxed in an <img>, but an id to collide with once inlined.

* Optimize the provider logos and normalize them to one size

The files came from seven different sources and showed it: viewBoxes ranged from
24x24 to 512x512, some carried fixed width/height, and each baked in its own
arbitrary padding. Rendered into the same 20px tile the marks came out anywhere
from 12px to 20px tall, so the row read as a jumble.

Run everything through SVGO (viewBox kept, dimensions and editor cruft dropped),
then re-cut each viewBox to a square centred on the mark's real ink extent. Every
mark now fills the tile's width, with padding controlled by the tile alone rather
than by whoever exported the file. Z.AI is inset 12%: its mark is itself a filled
rounded square, so at an equal bounding box it reads far heavier than the open
marks.

* Say when the provider lists fail to load

Both components defaulted the two queries to empty arrays and never read
their error, so a failed request was indistinguishable from having nothing
configured. The provider-types failure was the worse of the two: it rendered
an empty picker grid — no providers to choose, no explanation, no way on.
The connections failure is quieter but wronger, since an admin with several
providers would be told to connect their first one.

Report the error where the list or the picker would have gone, using the
same getErrorMessage the rest of this feature already uses.

* Tighten the provider section's spacing and hang the icon off the name

Sloan flagged three gaps as feeling high even where they match other sections: drop the
section's header gap from md to sm, and the gap out to the model picker from lg to md. The
row-to-button gap was already the tightest of the three and stays.

The icon was centred against the whole text block, so it sat slightly off the name and
drifted further as an error or env-var line appeared beneath it. Give the name the icon's
height and centre it there, so the two line up on one row however many follow.

* Collapse the add button's box, which was most of the spacing

Trimming gaps barely moved this because the space around "Add another provider" was not a
gap. Metabase's Button defaults to size md, so it is 42px tall; p={0} removes the padding
but not the height, leaving its ~20px label centred with about 11px of dead space above and
below it. That is the two gaps flagged around the button, and no gap value could reach them.

Let it take its label's height once it is an inline link. Rows go from py="sm" to "xs" as
well, since a 32px icon does not need 8px above and below it.

* Give the add button its height back

Collapsing it to its label bought vertical space but cost the button its shape. Keep the
tighter rows from the same change.

* Draw every provider logo on one 24x24 grid

Making each viewBox square left seven different squares -- 24, 88, 300.23, 256,
259.53, 511, 30.72 -- so the files only matched once a consumer sized them. Bake
each mark's scale and centring into its path data instead, so every file is
literally width=24 height=24 viewBox="0 0 24 24" with the mark centred on that
grid and its longest side spanning it.

Z.AI keeps its 12% inset, for the same reason as before: its mark is a filled
rounded square and reads heavier than the open marks at an equal box.

* Sit the row actions on the icon band and cut the gap to the add button

* Fit a provider's name and type inside the icon's height

* Give each provider row a little more room

* Inline the remaining provider logos instead of loading them by URL

Mistral, Z.AI and Azure rendered as broken images. Their files are valid; the
copies the bundler emitted were not. BannerPlugin is configured with no test, so
it prepends the licence header to every emitted asset -- and re-applies it on
each watch rebuild, so the served .svg started with six stacked comment blocks
before its <svg>, which no browser will parse as an image. Editing these files
repeatedly is what re-emitted them and made it visible.

Inline all seven the way the other four already were: SVGR turns them into
modules, so nothing reaches the asset pipeline and one import style covers the
set. Azure's gradients were on ids a/b/c, which are too generic to share a
document with, so they are namespaced now.

The banner covers every other emitted asset too, so any .svg re-emitted during a
dev session hits this; that is worth fixing in the build config separately.

* Call the managed provider Metabase AI service and tighten the add-provider modal

Drops the redundant 'Selected provider' subtitle and 'About Metabase AI service'
heading, puts the free-token and pricing lines in their own callout, and swaps the
row's ellipsis button for a neutral ActionIcon.

* Pin the base URL the credential-adoption tests read

* Mock the provider endpoints the onboarding AI modal calls

* Give the explorations e2e helper a provider connection instead of read-only settings

* Make Moonshot AI a provider type you connect to

* Regenerate the settings docs

* Answer the review on the model listing endpoint

- Drop the per-provider :group heuristics. The picker groups by connection
  name and never reads the field, so it was computed on every admin page load
  for nothing.
- Key the model cache on a hash of the connection's config rather than on the
  config itself, so a credential is not held in a process-wide cache, and a
  rotated one gets its own entry instead of the old list.
- Take the model a connection composes from its own config into account when
  Metabot's selection has to move: Azure has no default model, so it was never
  a fallback, and editing its deployment left the selection naming one that no
  longer exists.

* Migrate the per-provider credential settings instead of adopting them at startup

The startup adoption left the settings it copied from behind, frozen: nothing
writes to them any more and they are read-only, so an instance rolled back to
code that still reads them would run on whatever was last stored there rather
than on what is configured. A reversible migration fixes both ends — it clears
the settings on the way up, and writes the connection list back into them (and
rewrites a model reference older code cannot resolve) on the way down.

Also pulls the copy of `trimmed-string` the two LLM namespaces shared into
`metabase.util`.

* Wait for the model list before opening the picker in its spec

* Make Google Gemini a provider type you connect to

Carries the Gemini Enterprise Agent Platform provider over to the connection
list: a registry entry with the same form every other provider gets, credentials
resolved from the connection rather than from the `llm-google-*` settings (which
become environment-only like the rest), and the connection naming the model it
serves — the platform has no listing endpoint worth trusting, so connecting
validates the model with a free countTokens probe, the way Azure validates its
deployment.

* Cover the Google credentials in the settings migration

* Name the Gemini models Metabot works best with

* Use Moonshot AI's own mark, not Kimi's

The file was Kimi's rounded-square K — Kimi is Moonshot's chatbot, not the
company we connect to. Icon sets ship the two separately and it is an easy
one to pick wrong.

Replaced with the MoonshotAI mark, kept in the same shape as the other
single-colour logos here so it takes the theme's text colour.

* Give Google the form the upstream PR built for it

Three field kinds the registry could not express before, so the connection form
can offer what the settings page did:

- a fixed model list, replacing the free-text field and the paragraph of advice
  about which IDs work;
- an authentication method to pick, rendered as a segmented control;
- a file to upload for the service account key, read into the config the way
  every other credential is typed into it.

A field can now hang off another one's value (`show_when`), which is what makes
the credential follow the method — and a field the form hides is cleared on
save, so switching methods drops the credential it replaced.

* Let Gemini's models come from the shared picker

A connection only names its own model when nothing else can name it — Azure,
whose deployments its listing endpoint does not return, and which serves one of
them. Gemini's models are a known list, so pinning one per connection put a
second model picker inside the form and hid the other model from the one below
it. The list becomes the type's fixed catalog instead, the way the managed
provider's is, and both models show up per connection in the model picker.

A fixed catalog no longer skips the provider call: that is what verifies the
credentials, so it is made against the catalog's first model and the catalog is
what comes back. The managed provider still answers from the registry alone —
there is nothing to call, and instances that cannot reach the proxy would fail.

* Answer the migration review notes

Rename the frozen table's :credentials to :required-keys — they are the keys a
provider needs to count as configured, and "credentials" made the optional ones
sound like something else.

Stop the downgrade encrypting every value it writes. write-setting! ran
maybe-encrypt unconditionally, so base URLs, regions and the model reference —
settings declared :encryption :no — came out of a rollback holding ciphertext.
Reads decrypt opportunistically so nothing broke, but a setting should hold what
its declaration says it holds. Only the :sensitive? settings are encrypted now,
and a test pins the raw storage both ways.

The Google row is also renamed to Google Gemini Enterprise alongside the
registry, to keep the adopted connection's display name in step.

* Close the review's holes in the provider registry and its API

Redact the Google service account key. secret-field-keys covered only
:password fields, and the key is a :file upload, so GET /api/llm/providers
returned the whole credential — private key included — in plaintext. Fields can
now be marked :secret? regardless of input type, which also makes the masked
echo-back flow cover the key on edit.

Require what Google actually requires. Every field was individually optional
because either credential will do, so an empty config counted as complete —
usable: true on a connection that can serve nothing. Provider types now declare
alternative credential groups (:required-any): a service account key on its own,
or an OAuth token together with the project ID it bills to. Completeness and
validation both honor the groups.

Treat blank credentials as absent. The form clears the field it hides when the
authentication method changes, but the empty string it sent was stored and
passed to the adapter, which fed "" to the service-account parser — the
"no JSON input found" from the review. Blank values are dropped on save, and
the adapter trims what reaches it so an env-supplied blank cannot shadow the
other credential or the project ID a key carries.

Reserve the metabase connection key. Any type could claim it, splitting
managed-model-ref? from managed-type? — a bring-your-own-key connection whose
requests would be tagged as proxied in metabot_message and confuse the managed
free-tier limit. Only the managed type may hold the key now, a squatter blocks
managed connection with a clear message instead of colliding, and the unused
proxied-model-ref? is gone.

Also: per-field :validate hooks in the registry (Google's project ID and
location get the setting-specific validation the old defsettings had), select
values are checked against their options, with-entitlements takes a kwargs map,
and the cond-> in self.clj is realigned.

* Pick the Google model in the connection form

Connecting a Google provider validates the credentials with a countTokens probe
that needs a model, but the form collected none, so the probe ran against a
hardcoded default the chosen location may not serve — a confusing rejection the
admin had no control over, and no guarantee the model they wanted works at all.

Provider types now ship their fixed catalog and alternative credential groups to
the frontend. A type with a catalog gets a model picker in its connection form,
the probe runs against the picked model, and Metabot starts on it. The Connect
button also honors the credential groups, so it no longer offers to connect a
Google provider with nothing filled in.

* Quiet the kondo warnings CI caught

u/lower-case-en over str/lower-case in the reserved-key check, and the
validate-config! additions split into their own deftests instead of growing the
original past the one-concern-per-test lint.

* Report the provider Metabot landed on, not the first usable connection

The setup step resolved its connected provider with find(usable) over the
connection list, which is insertion order. Connect two providers during setup
and pick the second one's model, and the step still reported the first — in its
title and in what submitAiConfig records. Resolve the connection from the
llm-metabot-provider reference instead, the way the pre-connection-list code
compared against the saved value.

* Spell the field-level secret flag :sensitive?

Same word as the defsetting option it mirrors, so a grep for :sensitive?
surfaces the registry's secret fields alongside the settings.

* Give the LLM settings migration its own test file, and a managed round trip

Twice now a reviewer could not find the migration's coverage, because the tests
sat in the shared custom_migrations_test and referenced the migration only by
its changeset id. They live in a file mirroring the migration's path now, with a
docstring naming the change class that connects the two.

The round trip crisptrutski asked about also surfaced what he predicted: the
managed connection is not in the migration's provider table, so every rollback
of a managed-AI instance logged a false 'no equivalent setting: metabase'
warning. It holds no credentials and its model reference survives the rollback,
so it is dropped silently now, and a new test walks the managed round trip.

* Let the environment shadow connections field by field, and settings write through

Answers crisptrutski's review of the environment and settings story.

The MB_LLM_* variables now shadow a stored connection's config per field, the
way an env var shadows any other setting, instead of swapping the whole
connection out. A lone MB_LLM_ANTHROPIC_API_BASE_URL reaches the stored
connection's base URL rather than doing nothing; everything the environment
does not supply stays editable, and env_fields tells the form exactly which
inputs to disable. A standalone env connection still only comes into existence
when a credential variable is set — which is why Azure's base URL is no longer
marked as one: alone it used to synthesize an unusable connection that shadowed
the stored credentials wholesale. A metabase/ reference pinned by
MB_LLM_METABOT_PROVIDER synthesizes the managed connection it names, so that
reference resolves without an app-DB row.

The per-provider settings are a view over the connection list rather than
read-only. :setter :none broke config.yml provisioning outright — the
provisioner calls setting/set! on every entry and nothing catches the throw, so
the instance failed to start — and orphaned generate-sql and semantic search,
which still read llm-anthropic-api-key and llm-openai-api-key directly after
the migration deleted the rows underneath them. Getters resolve the connection
field with the environment and registry defaults layered in; setters write
through to the connection, creating it on first write and keeping the old
setters' field validation. Both docs files are regenerated: the config
template carries the settings again.

Also from the review: deleting the managed connection requires superuser — it
cancels the Store subscription, and everything else that can cancel add-ons is
superuser-only — and the confirmation dialog says so; the automatic
llm-metabot-provider writes (first connection, edit follow-up, delete
fallback) are skipped when the env var pins the selection, restoring what
check-not-env-shadowed! used to guarantee; the edit form starts the model
picker on the model the connection is actually serving and a changed pick
follows through to the selection for fixed-catalog types; and redact masks
every field of a provider type the registry does not recognize, instead of
none of them.

* Reconcile the restack onto bot-1875-gemini-models-fe

Mechanical conflict resolutions during the rebase left a few files at the wrong
side of the divergence; this restores the reviewed merge result: the shared
tool-function conversion in the Claude and OpenAI adapters, the settings
endpoints stripped from the relocated metabot API slice, and
gemini-3.7-flash — new on the Gemini branch — joining the fixed catalog.

* Restore the stop-reason propagation the restack dropped from the adapters

* Strip trailing slashes off base URLs when a connection resolves

* Slug-proof the reserved key, follow the effective config, and seed the models cache

* Count a blank-config connection as dropped on rollback

* Warn that deleting the anthropic or openai connection turns off the feature reading it

* Regenerate the environment variable docs against current master

* Drop the env-var doc section master removed in the backend hardening

* Update src/metabase/llm/settings.clj

Co-authored-by: Chris Truter <crisptrutski@users.noreply.github.com>

* Mark the dev rollback subqueries so the app-DB honeysql guard allows them

* Gate the settings-access delete test on EE, where advanced permissions exist

* Wait for the AI setup buttons to render before clicking them

---------

Co-authored-by: Mike Appleby <86076+appleby@users.noreply.github.com>
Co-authored-by: Chris Truter <crisptrutski@users.noreply.github.com>
2026-08-18 13:42:53 -03:00
Anderson
6ab7bfb2c6 Add tracking events for collection filtering, selection, pinning, and move interactions (#79926)
* Add collection interaction analytics

* Code cleanup

* Address review comments
2026-08-18 15:37:24 +00:00
Nicola Mometto
50a3a1177d Align hardening with x.63 backport (#80066) 2026-08-18 16:27:27 +01:00
Denis Berezin
461830362b UXW-3641 Remove emotion from EntityItem (#79848)
* UXW-3641 Remove emotion from EntityItem

* Review fixes
2026-08-18 18:19:44 +03:00
Chris Truter
2550ef763e Add embedding provider SPI and Arctic embedder plugin (#78310)
* Add a versioned embedding provider SPI

Summary

- Add a runtime registry for embedding providers with readiness, model resolution, preparation, and batch embedding.
- Define immutable embedding-space descriptors and legacy deterministic identities.
- Validate provider results, dimensions, counts, and finite numeric vector components.
- Make SPI version 1 a compile-time constant; provider artifacts must embed the literal version they were built against.

Scope

This defines the neutral provider contract only. It does not migrate semantic-search consumers or ship a provider plugin.

Testing

- Registration and version compatibility.
- Identity stability and revision round trips.
- Registry snapshot consistency and malformed provider output.

* Rename shadowing local in provider/prepare!

The local bound from :prepare! shadowed the prepare! var (eastwood local-shadows-var).

* Add the Arctic embedder plugin

Summary

- Build a separate runtime plugin containing the pinned Snowflake Arctic embedding model and DJL/ONNX dependencies.
- Bundle arm64 and AVX2 exports, lazy-load native inference, and cap local batches.
- Register literal embedding SPI version 1 and derive immutable space identity from the exact model/runtime bundle.
- Fail readiness closed for unsupported platforms, musl, and glibc older than 2.34.
- Disable DJL network telemetry and revision-address bundle URLs so model upgrades cannot reuse stale cached content.

Scope

This PR ships and registers the provider but does not yet select it from semantic-search settings. Consumer integration follows upstack.

Testing

- Model bundle and plugin uberjar build tests.
- Catalog, classloader, platform-readiness, identity, cache-key, and batching coverage.
- Artifact-only real inference with zero outbound network attempts.

* Follow prune-provided-libs! rename in the embedder plugin build

* Merge nested let in embedder-plugin artifact test

Kondo flagged the inner let as redundant; fold both binding vectors into one.

* Deref the now-delayed core provided-libs in the embedder build

* Bundle MiniLM for in-process complexity scoring

* Activate the embedder plugin in its artifact smoke test

* Activate lazy embedding providers on demand

* Use plugin initializer in artifact smoke test

* Make lazy provider activation best effort

* Log lazy embedding provider activation failures

* Declare the embeddings plugin dependency

* Keep embeddings provider internal

* Activate the in-process embedder during bootstrap

* Fix embedder validation and optional plugin startup

* Declare the embedder plugin as a module

The deps graph only scanned src, enterprise/backend/src, and modules/drivers, so the plugin's
namespaces belonged to no module and anyone linting them got six unresolved-dependency
warnings. Scanning modules/embedder/src too lets the generator record what the plugin actually
takes from the host.

* Move embedder model caching into the weekly cache generator

Review feedback on #78310:
- Drop the redundant persist-credentials: false checkout options.
- Stop writing the model cache from the CI job. The weekly cache-generator
  workflow is the single cache writer; the build job now restores only.
  The downloader is content-addressed (SHA-256 verified, refetches deltas),
  so a stale weekly cache still protects against HuggingFace rate limits
  without affecting correctness.

* Bundle snowflake-arctic-embed-xs instead of Arctic-L

The bundled models are a test tier: they exercise the packaging, DJL
runtime, tokenizer, and pooling machinery without shipping the 1.1 GB
Arctic-L artifacts. xs keeps the same family, query-prefix scheme, and
cls pooling as Arctic-L at 23 MB per architecture. A release-tier build
with the full model comes later, when the plugin is published for
customers or offline use.

At ~90 MB of pinned downloads there is nothing worth caching, so the
cache-generator wiring is reverted and the CI job downloads directly.

* Fail loudly when a catalog entry lacks this architecture's export

resolved-model computed the embedding-space-id from
get-in spec [:architectures arch :sha256] without checking it was
present. A model missing this architecture's export would silently
bake a nil into the identity hash instead of failing, producing a
syntactically valid but bogus embedding-space-id.

* Pin the embedder plugin manifest name to one constant

"Metabase In-Process Embedder" was duplicated in core.clj (the
registered?/load-plugin! lookup key) and the artifact test. A rename
in modules/embedder/resources/metabase/embedder/metabase-plugin.yaml
without updating both call sites would silently disable the plugin,
since registered? just returns false for a name it doesn't recognize.

* Downgrade the hashed-table-name log line to debug

Hashing fires deterministically for every identifier derived from a
long provider/model pairing (e.g. "in-process" + the bundled xs
model), so it was logging a warning on every lookup rather than once
at creation.

* Cover the missing-architecture catalog guard

The error path added alongside the guard had no test, so the earlier
silent nil-hashing behaviour could come back unnoticed.

* Assert the reason on the missing-architecture guard

The message alone leaves `:reason :architecture-not-bundled` untested,
which is the part callers actually branch on.

* Drop the hashed-table-name log line

It fires from a pure name-derivation helper on every lookup. The table
creation it hints at is already logged at info by the caller, and the
parameters behind the hash are in index_metadata.

* Extract the missing-architecture check out of the smoke test

The guard's assertions pushed the smoke test to 155 lines, past the
150-line deftest limit, so Clj-Kondo failed. Pull it into a helper the
way the platform-support checks already are.
2026-08-18 17:11:24 +02:00
Ramiro Nuñez Dosio
c78c9fed48 Update README with new image URL and bug reporting info (#80073)
* Update README with new image URL and bug reporting info

Updated the image URL for the Metabase product screenshot and added a bug reporting guide.

* put metabase-product-screenshot-updated.png to the docs

* fix link in the readme

---------

Co-authored-by: Timur Mashara <timur@metabase.com>
2026-08-18 15:43:18 +02:00
Oleh K
1821975d0c Use Breadcrumb component in Detail View (#79892)
* Use Breadcrumb component in Detail View (GDGT-2596)

Detail View had its own Breadcrumb, duplicating the shared one introduced
in #75114. Point TableNav, ModelNav and CollectionBreadcrumb at
metabase/common/components/Breadcrumb and delete the local copy.

The shared component hardcoded showTooltip={false}, which would have lost
the full-text tooltip Detail View shows on truncated crumbs, so make it a
prop. It defaults to false, leaving every other consumer unchanged.

Crumbs now take the shared component's sizing (0.875em, bold) instead of
inheriting 14px regular; the separator is sized to match so the row stays
consistent. Colour and icons are unchanged.

* Use URL helpers for the Detail View breadcrumb links

TableNav built the database and schema URLs by hand. Extract databaseSlug
from browseDatabase so browseSchemaBySlug can reuse it, then build both
links from the helpers. Both now carry the database slug rather than only
the id.

* Style official collections in the Detail View breadcrumbs

Collection crumbs hardcoded icon="folder", so an official collection
rendered like a regular one. Use CollectionBadge, which resolves the icon
from the collection's authority level, the same way the model page does.

Drop the text-secondary override too: it made Detail View darker than
every other breadcrumb in the app bar.

* Align Detail View breadcrumbs with the rest of the app bar

The detail view nav carried a -13px left margin, putting its first crumb
13px closer to the logo than the collection breadcrumbs shown on every
other page. Drop it so both start at the same offset.

* Match the Detail View breadcrumb separator to the app bar

Its color, font size and line height all differed from the separator the
collection breadcrumbs render. Use the same values, and Box rather than
Text so it inherits the line height instead of setting its own.

* Update the database breadcrumb href expectation

The database crumb now goes through Urls.browseDatabase, which appends
the slug, so the link is /browse/databases/1-sample-database rather than
/browse/databases/1.
2026-08-18 15:39:42 +02:00
Nemanja Glumac
812e970d41 Remove the reduntant skip input from bundle-size step (#80070) 2026-08-18 14:35:31 +02:00
Oleh K
836c398dc7 Make query builder breadcrumb text the same size (#79889)
* Make query builder breadcrumb text the same size (GDGT-2609)

The database and schema crumbs rendered smaller than the rest of the
breadcrumb. HeadBreadcrumbs spread {...rest} after className, so a caller
forwarding className -- even as undefined -- wiped the computed Container
and headVariant classes, and those crumbs fell back to Breadcrumb's
0.875em. Merge the incoming className instead of letting it clobber.

Size the head-variant dividers and the leading icon to match the crumbs
too, so the whole row reads as one line of text.

* Address review comment

Keep the props spread last and exclude className from it, rather than
reordering the spread ahead of className.
2026-08-18 14:35:03 +02:00
Chris Truter
548bae6986 Adopt bare library index tables in dedicated store only (#79986) 2026-08-18 13:00:20 +02:00
Chris Truter
2e45128a8e Drop departed members from the Metabot team (#80031) 2026-08-18 12:22:46 +02:00
lbrdnk
f768659552 Unify build logic in PR env and AI benchmarks (#78534)
* Stub the evals trigger

* use ref arg

* dummy to trigger build

* Clean up

* Use appropriate token

* Simplify pr message

* Use .evals_ref for default ref

* Comment cleanup

* Adjust ref

* Use evals branch

* Use general automation token

* readme for .evals_branch

* Use main of evals for label trigger

* Revert "dummy to trigger build"

This reverts commit e6502c6ff076a47d94eb829b516fc45b37cd7680.

* whitespace

* Address review remarks

* Use ai benchmark naming so we can trigger from pr

* Address review comments

* Remove now redundant branch pinning

* Review remarks

* Enable passing commit to uberjar from other workflows

* Unify ai-benchmark and pr-env build logic

* linter

* checkout action

* separate build step

* Use wip branch

* adjust job output

* Make image building manually triggerable

* tmp comment

* Address review remarks on the build extraction

- ai-benchmark's `image` regained the label guard `uberjar` used to carry; without
  it any label on any same-repo PR built ee+oss and pushed to ECR.
- New `resolve` job turns the dispatch `pr`/`ref` inputs into one full sha, and
  feeds it to both the image build and the eval-run dispatch. `pr` was ignored
  before, so `pr: N` benchmarked the launch branch while reporting against the PR;
  `ref: master` looked for an artifact named after a branch.
- prepare-uberjar-artifact takes a `commit` so the upload key matches what
  check-existing-artifacts/fetch-artifact look up. Without it a build for an
  explicitly passed commit uploaded under the event sha and then failed to
  retrieve itself.
- uberjar's build checkout honours an explicit commit, but stays on the event ref
  otherwise so PR builds keep compiling the merge ref.
- Concurrency group includes the calling workflow, so a re-label on one caller no
  longer evicts the other's queued run.
- ECR existence check targets the pushed registry explicitly and no longer hides
  real errors behind "not found".
- Scoped `secrets: inherit` down to the two secrets dockerize needs, added
  read-only default permissions, dropped the dead `jar xf`, restored the comment
  explaining `finalize`, and reset EVALS_BRANCH to main.

* Point smoke evals back at the evals-docker-registry branch

Temporary while metabase/evals#32 is unmerged — the ECR image logic is only on
that branch. TODO comment retained so this cannot merge as-is.

* Align ai-benchmark pr/ref precedence with evals eval-run

eval-run.yml resolves both inputs with `ref` overriding the PR head; this
resolved `pr` first, so the same two inputs meant opposite things either side of
the dispatch. Flip precedence and the run-name to match, and point the `ref`
description at eval-run's containment rule instead of claiming pr wins.

* Cleanup the uberjar worfklow

* Smoke should be triggered on !cancelled()

* Reset pointer to evals branch
2026-08-18 12:14:46 +02:00
bryan
7b9b376765 Harden JSON-encoded parameter-values request schema (#80035) 2026-08-18 09:52:00 +01:00
Romeo Van Snick
3933c11183 Register visualization definitions eagerly, load chart components on demand (#79745)
* Give List, Map, ObjectDetail and PivotTable their own definition file

* Let a visualization register a loader for its component

* Load every chart component on demand

* Load the chart chunk while the query is still running

* Delete the echarts lazy boundary the chart split made redundant

* Dedupe modules shared by the SDK's async chunks (#79748)

The chunked SDK entry's splitChunks filter only matched chunks whose
name starts with "embedding-sdk-chunk", so the unnamed chunks created by
import() were excluded from splitChunks entirely. With every chart now
loaded through import(), each echarts-based chart chunk bundled its own
copy of echarts, zrender and the shared viz code, growing the reachable
SDK total from 3.3MB to 6.2MB gzipped.

Include async chunks in the filter and add an asyncCommons cache group
(minChunks: 2) so shared modules move into a common async chunk. The
vendors group keeps its own name-based filter so async modules are not
merged into the eagerly loaded vendor chunks. Initial payload is
unchanged; the legacy monolith and the bootstrap stay excluded.

Co-authored-by: Claude Fable 5 <noreply@anthropic.com>

* Lower the size floor for the SDK's shared async chunks

* Load chart components before a story renders

* Retry a chart chunk that fails to download

* Keep one loading state while a chart chunk loads

* Delete the leaflet and pivot-table lazy boundaries the chart split made redundant

* Say why onInit is late now that the renderer is not lazy

* Register the charts a test renders before it runs

---------

Co-authored-by: Nicolò Pretto <info@npretto.com>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-08-18 11:21:10 +10:00
Fraser
8dbbb58c57 Create current-user module (#79357) 2026-08-18 12:58:58 +12:00
Romeo Van Snick
e7de47b608 Module boundaries follow-ups: storybook element order, dead window-bridge rule (#79803)
* Order the storybook config element before embedding-sdk-shared

* Drop the dead window-bridge whitelist rule
2026-08-18 10:25:04 +10:00
Anderson
f6e9cd0a58 Adapt the collection bulk action bar to pinned, unpinned, and mixed selections (#79857)
* Adapt the collection bulk action bar to pinned, unpinned, and mixed selections

* Align the bulk action bar per-state actions with the Figma spec

* Show the bulk action overflow for unpinned selections and fix its trigger styling

* Move bulk action logic into UnarchivedBulkActions and address review feedback

* Center the bulk action overflow trigger icon via a label class

* Make BulkActionButton code nicer

* Address review comments
2026-08-17 23:38:59 +00:00
Romeo Van Snick
c883033bb6 Make the provider props store payload opaque (#79801) 2026-08-18 09:37:11 +10:00
Cam Saul
34863099c0 Migrate MongoDB to MBQL 5 (#77673)
* [MongoDB] remove `:metadata/key-constraints` driver feature flag (incorrectly set)

* Migrate MongoDB to MBQL 5 [WIP]

* cljfmt

* Test fix 🔧

* Test fixes 🔧

* Test fixes 🔧

* Test fixes 🔧

* value.clj => value.cljs

* value.cljs => value.cljc 🤦

* 🔧

* Let's not deprecate too many things in this PR

* Fix more stuff? 🔧

* Test fixes 🔧

* Revert SQL driver changes

* Test fixes 🔧

* cljfmt

* Test fix 🔧

* Kondo (partial)

* Test fixes 🔧

* cljfmt

* kondo

* Address PR feedback 🔧

* Update ratchets 🔧

* 🔧

* 🔧

* Test fix 🔧

* Update ratchets.edn
2026-08-17 15:34:42 -07:00
Ryan Laurie
fbfa5c9c40 Use ubuntu slim image for 59 little jobs (#80028)
use ubuntu slim image for 59 little jobs
2026-08-17 22:18:37 +00:00
Denis Berezin
7cd842e414 UXW-5013 Support adding a metric to recent items (#79661)
* UXW-5013 Support adding a metric to recent items

Co-authored-by: Ruchir D <111691121+Ruchir-D@users.noreply.github.com>

* Code review fixes

---------

Co-authored-by: Ruchir D <111691121+Ruchir-D@users.noreply.github.com>
Co-authored-by: Nathan Voxland <nathan@metabase.com>
2026-08-18 01:09:23 +03:00
Nemanja Glumac
3fdbcf9ac6 Delete minimal-tests workflow (#80029) 2026-08-17 23:40:16 +02:00
github-automation-metabase
e791ef41da Auto-update documentation for master (#79919)
Update generated docs (master)

Co-authored-by: github-actions <github-actions@github.com>
2026-08-17 19:48:45 +00:00
github-automation-metabase
68496eb7a0 Auto-update documentation for master (#79847)
Update generated docs (master)

Co-authored-by: github-actions <github-actions@github.com>
2026-08-17 15:48:30 -04:00
Nicola Mometto
af35f7f0af Backend hardening (#80003)
Co-authored-by: Alexander Polyankin <alexander.polyankin@metabase.com>
Co-authored-by: Alexander Lesnenko <alxnddr@users.noreply.github.com>
Co-authored-by: Ben Grabow <ben.grabow@gmail.com>
Co-authored-by: Bryan Maass <bryan.maass@gmail.com>
Co-authored-by: Dan Sutton <dan@dpsutton.com>
Co-authored-by: Eric Normand <ericwnormand@gmail.com>
Co-authored-by: Nemanja Glumac <31325167+nemanjaglumac@users.noreply.github.com>
Co-authored-by: Timothy Dean <timothy@timothydean.dev>
Co-authored-by: Tyler Plude <tsplude@gmail.com>
2026-08-17 13:44:13 -06:00
technomancy
c7166aaa15 Only run latest app db variants on PRs (#78755)
* Don't test MySQL/MariaDB AppDB on anything but latest outside release/master.

* generate matrix up front

# Conflicts:
#	.github/workflows/app-db.yml

* bugfix

* parameterize it all

* make things more uniform

---------

Co-authored-by: Ryan Laurie <iethree@gmail.com>
2026-08-17 13:40:56 -06:00
Jeff Bruemmer
8f7a033678 docs - map tile notes (#80006)
* map tile notes

* edit

* image
2026-08-17 19:35:23 +00:00
Ben Grabow
6d4769584e Use an isolated executor pool for test isolation (#79913)
* Use an isolated executor pool for test isolation

* Fix another test

* Fix some other likely flaky tests

* Consistently use the mock executor in the tests that need isolated behavior.
2026-08-17 14:10:21 -05:00
Chris Vandoodewaard
a68d9424ca disable CodeQL overlay DB cache (#79987)
* disable cache

* cleanup codeql cleanup workflow
2026-08-17 14:35:46 -04:00
github-automation-metabase
7582bb77ec Update translations 2026-08-17 (#79988)
* Update translation sources 2026-08-17

* Update translations

---------

Co-authored-by: github-actions <github-actions@github.com>
2026-08-17 17:42:59 +00:00
Ben Grabow
a0d8a1758b Check via .getDataset instead of .listDatasets when all datasets are exactly named (#79921)
* Check via .getDataset instead of .listDatasets when all datasets are exactly named

* Use perf/every?

* Handle some corner case differences between .listDatasets and .getDataset method behaviors
2026-08-17 11:40:08 -05:00
Kamil Mielnik
7470cf2485 GDGT-2646 Refresh leads to custom viz reverting to standard viz (#79888) 2026-08-17 20:28:55 +07:00
Michael Eby
c1fc823ce5 docs: callout for AI Analytics Week (#79918) 2026-08-17 09:18:22 -04:00
Chris Vandoodewaard
b0ba105d0c add file, not line number to .po files (#79723)
add file, not line number
2026-08-17 09:03:55 -04:00
Nick Fitzpatrick
e49a6d6754 Uxw 4937 combine account password and security tabs (#79677)
* Moving Account Security Panel

* tests

* adjustments

* update docs

* Product Feedback, rebasing adjustments

* fix unit test

* adjust e2e
2026-08-17 09:18:28 -03:00
Stas Gavrylov
6b311219dc Fix duplicated rows on the Erroring questions page (#79520)
The `latest_qe` CTE joined `query_execution` to its per-card max
`started_at`, which returns every execution tied at that timestamp
rather than one. A card whose two most recent runs landed in the same
clock tick therefore fanned out into two identical rows on
`/monitor/errors` — same card_id, duplicate row keys in the table, and
an inflated `total_count`. Rank the tied executions and keep one.

Also give the ORDER BY a unique tiebreaker: no sort column is unique,
so tied cards were ordered arbitrarily and could repeat on one page of
LIMIT/OFFSET results while going missing from another.
2026-08-17 15:07:21 +03:00
Oleh K
8129fff836 Match SelectButton styling to Select (#79879)
* Update SelectButton styling to match Select (#GDGT-2705)

SelectButton diverged from the new Mantine Select: bolder text, a smaller
lighter chevron, a taller radius and, on long values, wrapping to two
centered lines instead of truncating.

Match the md Select input on every measurable property — 40px height, 4px
radius, regular weight, left aligned, 16px chevron in text-primary, and
Select's disabled tokens — and truncate long values with an ellipsis. The
clear affordance now sits alongside the chevron rather than replacing it,
matching Select's layout.

Migrate the component from Emotion to CSS modules while here.

Also fixes three Select issues surfaced by comparing the two side by side:

- long values cut off mid-glyph instead of ellipsizing
- with a clear button, the value ran underneath it, because the right-hand
  padding was pinned to a flat 40px while Mantine widens that section to
  60px (md) / 41px (xs)
- the disabled right section painted an opaque background over the field's
  translucent disabled fill, showing a square-cornered box on any
  non-white surface

* Remove comment

* Address review comments

- use !! instead of a null check, onClear is never null
- drop size={16} on both icons, it is Icon's default
- drop overflow: hidden from .SelectInput; the input already computes
  overflow: clip, so the declaration never applied and the ellipsis works
  without it
- cover sm, lg and xl in the clear-button padding fix, not just md and xs
2026-08-17 13:01:49 +02:00
Romeo Van Snick
0b0e6c0f4b Delete the unread collections entity slice (#79870)
* Stop hydrating the collections entity slice

* Delete the unread collections entity slice

* Drop the orphaned createMockEntitiesState import

* Wait for the collection query in the TableCollection link test

* Await the collection tree in the SavedEntityPicker tests
2026-08-17 20:55:14 +10:00
Ngoc Khuat
2366d9fc6b Don't let a failed advisory matching query override the version check (#79466)
* security center: don't let a failed matching query override the version check

an advisory whose matching query names a table the instance doesn't have yet
fails at parse time, and we mapped that to :error. the notification task treats
:error the same as :active, so instances that the version range had already
ruled out got notified anyway. that's how SC-2026-015 reached 1.58.

affected means in-range AND matched, so an undeterminable query result shouldn't
outrank a version check that already said no. same as SQL giving FALSE AND NULL
= FALSE. in-range instances with a query we genuinely couldn't run still report
:error.

* security center: split evaluate-advisory!-test to stay under the kondo length limit
2026-08-17 09:53:42 +00:00
Romeo Van Snick
3a065165fb Read database, table, and field display data from RTK Query (#79810)
* Read database and field display data from RTK Query, not state.entities

* Read table and database breadcrumb data from RTK Query, not state.entities

* Delete the legacy useTableQuery and useDatabaseQuery entity hooks

* Delete the legacy useDatabaseListQuery entity hook

* Keep FileUploadStatus on the mirror-backed table hook
2026-08-17 18:39:07 +10:00
Romeo Van Snick
c9c8b101cd Import only the metadata actions the reference module uses (#79869)
* Drop the dead metadata action spreads in the reference module

* Import only the metadata actions the reference module uses
2026-08-17 13:52:33 +10:00
Fraser
fe94806a72 Re-tier metabot to platform P4 and detail-view to P2 (#79866) 2026-08-17 12:40:31 +12:00
Fraser
a9ad0ee017 Replace the custom-homepage cache patch with tag invalidation (#79867) 2026-08-17 10:38:23 +12:00
metamben
47e50180d6 Return nil when cached query is canceled (like non-cached queries) (#79625)
closes QUE2-106
2026-08-16 15:33:32 +03:00
Romeo Van Snick
45dfb5a837 Make the theme editor an app-tier module (#79802) 2026-08-15 09:21:36 +10:00
Egor Grushin
eb009eb8f9 Require selecting an alert schedule time (#79623)
* Refactoring Schedule, force a user to pick an hour in alerts and subscriptions
2026-08-14 23:54:58 +01:00
Michael Eby
f258c5de52 docs: update outdated starter pricing (#79890) 2026-08-14 17:04:45 -04:00
appleby
fb3f08713d Gemini Enterprise Agent Platform: FE changes (#79558)
* Add Gemini Enterprise Agent Platform provider settings

* Invalidate the cached models list when credentials change

A credential write can change which models the provider serves, e.g. a different Bedrock region, Google location, or
Azure resource.

* Give the FileInput placeholder and clear button a visible color

The theme resolves these to transparent making them invisible withouth the override.

* Review feedback: whitelist recommended models

* Fix useUpdateMetabotSettingsMutation import

* Add gemini-3.7-flash to list of recommended models
2026-08-14 12:31:07 -07:00
appleby
c6e544cb1c Gemini Agent Platform: BE support Gemini models (#78193)
* Add llm-google-* settings for the Gemini Enterprise Agent Platform

The location, project ID and model become path segments of every request, so each is validated as URL-safe.

* Wire the google provider into Metabot provider settings

* Add Google Gemini provider via streamGenerateContent API

Add new provider supporting Gemini models via the Gemini Enterprise Agent Platform

- New ns `metabase.metabot.self.google`. Handles generic provider concerns like
  - contructing appropriate endpoint urls for the given model and location
  - validating caller-provided credentials via a countToken call
  - credentials handling
  - validating config
  - remapping upstream errors with improved error messages
  - toplevel request entrypoint
- New ns `metabase.metabot.self.google.stream-generate-content`
  - handles wire format and aisdk chunk tranlation for Google's streamGenerateContent API

* Wire Google credentials into the admin API

Optional credential settings that are set by an env var no longer block connect or disconnect.

* Skip the provider setting write when shadowed by an env var

* Drop the unused llm-bedrock-configured? setting

Nothing read it: the frontend never referenced it, and its last backend caller went away when
configured-provider-credentials switched to provider-credentials-complete? for the bedrock branch.

* Extract the shared tool->function conversion into schema/tool-function

The mu/defn doc-stripping + filter-schema-by-features + JSON Schema transform boilerplate was copy-pasted across the
claude, openai, chat-completions, and google adapters.

* Review feedback: allow settings to be cleared even if env shadowed

There was a mismatch where check-not-env-shadowed! allowed nil values but set-setting-unless-env-shadowed! did
not. Update set-setting-unless-env-shadowed! to preform nil writes so that a disconnect fully clears the appdb
settings even if some are env shadowed.
2026-08-14 12:31:07 -07:00
technomancy
1ca1c69b3f Track datasets created in with-dynamic-table. (#79862)
Update snowflake's sql.tx/qualified-name-components to not ignore db-name.
2026-08-14 11:43:40 -07:00
Chris Vandoodewaard
6c7ddff080 add workflow to run security scans daily (#79843)
* add workflow to run security scanners

* add newline

* add slack notification on failure
2026-08-14 14:29:21 -04:00
Chris Vandoodewaard
0d1f7d911c used cache deps for Mage (#79903) 2026-08-14 17:35:42 +00:00
Andrei Simionescu
0e70b3eb71 BOT-1956: Read the Python library source from :source (#79573)
Change `get_transform_python_library_details` to read from `:source` instead
of `:content` to match what the API sends. The code is wrapped in a Markdown
fence longer than any backtick run inside it, prefixed with a line telling
the model to treat it as data, and cut off at 100k characters with a
truncation marker when the library is bigger than that.
2026-08-14 18:22:52 +01:00
Andrei Chernykh
0c9e3642c9 Convert enterprise embedding dataselector to ts (#79789)
* Convert enterprise embedding DataSelector to TypeScript

* Replace default exports with named exports in the embedding data picker
2026-08-14 10:00:49 -07:00
Ben Grabow
e3708068c3 Count only the query executions under test (#79726) 2026-08-14 11:37:09 -05:00
Nathan Voxland
5dbc8db1d2 Only default warehouse-allowed-networks to external-only on prod (#79672) 2026-08-14 11:17:55 -05:00
Andrei Chernykh
cab11e6bcc chore: Convert dashboard click drill to ts (#79626)
* Rename dashboard-click-drill and its unit spec to TypeScript

* Convert dashboard-click-drill to TypeScript

* Address review feedback: discriminate click-behavior types and replace non-null assertions with checkNotNull

* Deduplicate base intersections in click-behavior union types
2026-08-14 09:02:00 -07:00
Kevin Kelleher
f0f11536f6 Avoid rerendering visualization on every SQL editor keystroke (#79825)
* Avoid rerendering visualization on every SQL editor keystroke

* fix unnecessary `card` assignment

Co-authored-by: Denis Berezin <denis.berezin@metabase.com>
2026-08-14 15:41:27 +00:00
Riley Thompson
33ac781b25 Snowflake passphrase protected private keys (#79542)
* update tests

* passphrase protected keys

* clean up

* improve tests
2026-08-14 15:36:29 +00:00
Michael Eby
4ba5c162e5 docs: add metric dimensions page (#79772)
* docs: add metric dimensions page

* Optimized 3 images, saved 262.3 KB (53.0% smaller)

* Edits from Jeff

* more consolidation

---------

Co-authored-by: Metabase Automation <github-automation@metabase.com>
2026-08-14 11:05:06 -04:00
Ben Grabow
e29527e5b3 Assert only on password reset email count (#79721)
* Assert only on password reset email count

* Initialize :db :web-server :test-users once for support access grant session tests
2026-08-14 08:52:55 -05:00
Chris Vandoodewaard
c980d85154 print out basic summary for release workflow (#79844) 2026-08-14 08:55:16 -04:00
Chris Vandoodewaard
19899d290d cache Babashka, also guard cache generation (#79841)
* cache Babashka, also guard cache generation

* fix linting

* also cache whitespace-linter deps
2026-08-14 08:54:13 -04:00
Kamil Mielnik
22e69fce8b GDGT-3076 Toast notifications are truncated for unavailable visualizations (#79766) 2026-08-14 09:45:18 +00:00
Oleh K
0a42bfbac2 Keep the remove-table button in the row when table names are long (#79760)
* Keep remove-table button in the row when table names are long

The table selector laid its label button and remove button out in a Group
that wraps by default. The label button is a flex item, so it kept the
default min-width: auto and refused to shrink below the width of its
longest line. A long database or schema name made it claim the whole row
and pushed the remove button onto a line of its own, at the left edge.

Stop the group wrapping, let the button shrink, and truncate the label
lines instead. The lines use max-width rather than width so a short line
keeps the left alignment the Stack gives it -- Button centers its label,
so a full-width line would be centered.

Also space the two label lines apart slightly.

* Space the remove button with margin instead of padding

Padding right sat inside the ActionIcon, so its hover background ran past
the icon and the glyph sat off centre. Margin keeps the same gap from the
row edge with a symmetric 28x28 hit area.

* Use Ellipsified for the table selector label lines

Manual truncation clipped the text with no way to read the full name.
Ellipsified brings the overflow styles and a tooltip that appears only
when the text is actually cut.

Dropping the Stack's align=start matters here: with flex-start each line
sized to its max-content width, so it never overflowed itself and the
truncation check never fired -- the button label clipped the text
instead, without an ellipsis. Stretched lines re-expose the button's
centered text-align, hence ta=left.

* Drop no-op layout props from the table selector

The button root class duplicated miw={0}, which already lands as an
inline min-width. The Stack's w/miw/justify did nothing: it stretches to
the button label by default, the label already carries min-width 0, and
its height is content-driven. ActionIcon has min-width equal to its
natural size, so it cannot shrink and flex="0 0 auto" changed nothing.

Verified by removing each and re-measuring at 800/1440/2560px.

* Drop redundant truncation styles and the empty-state Ellipsified

Mantine's button label already sets overflow hidden, and Ellipsified
carries its own min-width 0, so both declarations restated what was
already there. The css module is back to its state before this branch.

"Select a table..." is a fixed short string that never truncates, so it
goes back to a plain Box.
2026-08-14 10:04:29 +02:00
Anderson
32a179075c Add Escape and Delete shortcuts for collection item selection (#79670)
* Add collection selection keyboard shortcuts

* Address review comments
2026-08-14 00:32:36 -03:00
Fraser
acaeecd506 Delete dead ai-sql-fixer types, move revision diff guards into querying (#79865) 2026-08-14 13:24:19 +12:00
Fraser
fbb89ab47a Delete dead SandboxedPortal and unused formatting barrel exports (#79864) 2026-08-14 13:24:07 +12:00
Romeo Van Snick
8f06a28894 Compare a PR's bundle size against the base ref its uberjar was merged with (#79858) 2026-08-14 11:23:49 +10:00
Fraser
5f6f2ba892 Move card-embed editor extensions into documents (#79695) 2026-08-14 12:39:37 +12:00
Ben Grabow
c5e7d8024a Fix flaky trigger metadata sync test (#79771)
* Isolate async task execution from other tests and ensure task is cleaned up

* Use dynamic redefs
2026-08-13 17:20:02 -05:00
Andrei Chernykh
e0342c82dd Bump react-router from 7.18.1 to 7.18.2: CVE-2026-22030 (#79838) 2026-08-13 14:53:25 -07:00
Maz Ameli
662e74ba16 Add Data Studio guide page and reorganize sidebar navigation (#76707)
* Add Data Studio guide page and reorganize sidebar navigation (#76707)

* adjustments

* transform tabs, e2e adjustments

* oss test

* redirect first timers to the guide page and clean things up

* PR Feedback

* OSS copy, re-adding tests

---------

Co-authored-by: Nick Fitzpatrick <nickfitz.582@gmail.com>
2026-08-13 17:32:21 -03:00
Ryan Laurie
87dc99836a Auto-update documentation for master (#79792)
Update generated docs (master)

Co-authored-by: github-actions <github-actions@github.com>
2026-08-13 19:22:26 +00:00
Alex Yarosh
f5e7923fb5 docs: yet more AI providers (#79395)
* Revise AI provider details in settings documentation

Updated AI provider list to include Mistral and Z.AI models.

* Update docs/ai/settings.md

Co-authored-by: Michael Eby <38187276+michaeleby1@users.noreply.github.com>

---------

Co-authored-by: Michael Eby <38187276+michaeleby1@users.noreply.github.com>
2026-08-13 18:23:39 +00:00
Riley Thompson
0218df7cd7 Offset window function support for clickhouse (#79541)
* backwards compatible version

* clean up

* remove check
2026-08-13 18:06:33 +00:00
Andrei Simionescu
28d47ffafe BOT-1956: Render MBQL transform sources as portable-repr JSON for Metabot (#79575)
Notebook-built transform sources now reach Metabot as the same portable JSON that `construct_notebook_query` consumes, with tables and fields by name, while native SQL still passes through verbatim. Card entity IDs inside an exported query now come from the permission-checked content store, so a source card the reader can't open renders no query at all instead of falling back to an EDN dump.
2026-08-13 18:25:29 +01:00
Michael Eby
f9ac58f586 docs: add Monitor docs section (#79577)
* docs: add Monitor docs section

* update inbound links for profiling-metabase move

* tiny edits

* docs - CLI analytics (#79551)

* edits from jeff

---------

Co-authored-by: Jeff Bruemmer <jeff.bruemmer@gmail.com>
2026-08-13 12:58:41 -04:00
Nick Fitzpatrick
93fb2a5515 Chart Settings Popover horizontal margin (#79786) 2026-08-13 13:58:07 -03:00
Phoomparin Mano
633781cc4e Replace flaky embed code e2e test with jest tests (#79818)
* replace e2e with unit tests

* Update frontend/src/metabase/embedding/embedding-iframe-sdk-setup/components/tests/SdkIframeEmbedSetup.unit.spec.tsx

Co-authored-by: Nicolò Pretto <info@npretto.com>

* remove embed-snippet unit test

---------

Co-authored-by: Nicolò Pretto <info@npretto.com>
2026-08-13 16:30:57 +00:00
YB
98ea8d0a0a [GDGT-3087] mount /api/email routes with auth middleware (#79807)
mount /api/email routes with +auth middleware
2026-08-14 00:23:24 +08:00
Denis Berezin
5efdec98b5 UXW-4961 Convert data-grid to TS (#79788)
* UXW-4961 Convert data-grid to TS

* review fixes
2026-08-13 19:19:37 +03:00
Chris Vandoodewaard
f40f61f6ee add Mage deps to the cache (#79787)
resolve mage deps
2026-08-13 08:08:28 -04:00
Fraser
4724b7bde4 Split viz registry from the index file (#79682) 2026-08-13 23:16:30 +12:00
Fraser
9bed0e2f99 Move the query-action plug to querying and re-tier actions to platform (#79428) 2026-08-13 22:50:14 +12:00
Fraser
1be7ee6d1b Move metabot's endpoints into the metabot module (#79563) 2026-08-13 22:39:12 +12:00
Romeo Van Snick
21600b34b2 Make the metabot state channel payload opaque (#79800)
* Make the metabot state channel payload opaque

* Make MetabotStateChannel generic over the state type
2026-08-13 20:09:59 +10:00
Romeo Van Snick
1f6a8476dc Move the SDK bundle window declaration next to its exports type (#79799) 2026-08-13 20:04:13 +10:00
Romeo Van Snick
aa1d39ebc4 Move SDK storybook themes into embedding-sdk-shared (#79796) 2026-08-13 20:03:51 +10:00
Fraser
644d271c6d Sever visualizer, custom-viz and data-grid edges (#79355)
* Invert visualizer click-object formatting via a Visualization prop

Visualization no longer imports formatVisualizerClickObject from the
visualizer module; DashCardVisualization supplies a transformClickObject
callback instead, so the dashboard side owns the visualizer coupling.
2026-08-13 21:51:41 +12:00
Oleksandr Yakushev
8a83354b64 [perf] Optimize Malli validator cache (#79274) 2026-08-13 08:56:22 +00:00
Fraser
59646dd4b0 Move query-endpoints into querying (#79564) 2026-08-13 20:49:27 +12:00
Fraser
b249fe1b3d Move admin and single-consumer endpoints out of api (#79354) 2026-08-13 20:48:53 +12:00
Romeo Van Snick
a021efac20 Give the transforms, Data Studio, monitor and account pages their own chunks (#79701)
* Give the transforms and Data Studio pages their own chunks

* Give the monitor and account pages their own chunks

* Load the transform pages as one chunk
2026-08-13 15:49:38 +10:00
Romeo Van Snick
7a8ce63393 Give the chart-rendering feature pages their own chunks (#79700)
* Give the chart-rendering feature pages their own chunks

* Load the metric pages as one chunk

* Fetch the Data Studio Library metric pages when their route is matched

* Load the metric pages and their Data Studio wrappers as one chunk

* Say why the metric wrappers share the core chunk
2026-08-13 15:04:30 +10:00
Romeo Van Snick
917c65d41f Remove the unused ?source SVG imports from the icon map (#79741) 2026-08-13 15:01:13 +10:00
Romeo Van Snick
7d228d6fbe Move builtin auth plugin registration into the auth module (#79798) 2026-08-13 13:09:08 +10:00
Romeo Van Snick
fb8cf2d724 Move NewItemMenu integration spec next to NewModals (#79797)
* Move NewItemMenu+NewModals integration spec next to NewModals

* Drop the stale NewItemMenu spec pattern from module boundaries
2026-08-13 13:09:04 +10:00
Romeo Van Snick
a5268784d3 Move customVizColumnTypes into enterprise custom_viz (#79795) 2026-08-13 13:07:35 +10:00
Romeo Van Snick
033ca3a319 Give the dashboard its own chunk (#79645)
* Split the app shell's dashboard selectors into their own module

* Declare the dashboard card actions beside their type constants

* Keep the dashboard reducer off the parameter and data-fetching actions

* Split the tab thunks off the tab reducer

* Fetch the embed setup modals when one is opened

* Give the dashboard its own chunk

* Keep the query builder URL sync from replacing a pending navigation

* Keep the table picker's automatic selection off a pending navigation

* Read the pending navigation from the router, not from a render

* Wait for the code-split embed modal before probing for its terms card

* Drop the note restating what import type already says

* Outrank the grid placeholder style the dashboard chunk now loads later
2026-08-13 13:05:25 +10:00
Anderson
e406a40c47 Reveal column descriptions on hover in the Detail visualization (#78875)
* Show column descriptions in the Detail visualization via an info icon tooltip

* Reveal the Detail column description icon on hover instead of always

* Reserve the description icon's width so hovering never shifts the Detail table

* Address review comments

* Remove delay showing info icon
2026-08-12 23:56:09 -03:00
Anderson
c9f16511fa Allow selecting collection items with shift+click and a Select menu (#79663)
* Add a shift-pressed hook and shift+click selection for rows and pinned cards

* Add a Select entry to the collection item overflow menu

* Cover shift+click and the Select menu entry with an e2e test

* Keep shift+click on row actions from selecting the row

* Simplify collection item selection callbacks

* Address collection selection review feedback

* Scope pinned card queries now that pinned items are listed as rows

A pinned item renders both as a card and as a table row, so unscoped link
queries by item name now match twice. Scope them to the pinned section and
wait for the pinned query before asserting.

* Address review comments
2026-08-12 22:57:14 -03:00
Nemanja Glumac
5147989def Run a configured-skip driver when its own files change (#79793)
Drop the quarantine-conflict flag: a driver marked `skip` in ci-test-config
whose own `modules/drivers/*` files changed now simply runs, instead of
spinning up a runner that fails immediately and demands a `ci:run-<driver>`
label. Changes to another driver's files, or to shared code, leave the skip
in place.

Rename the decision logic's "quarantine" vocabulary to "skip", matching the
config's own `status: skip`. "Quarantine" now refers only to ci-conductor's
suite-level gating rules.

Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-12 23:34:15 +00:00
Fraser
c917cb51a5 Reduce UI imports from viz-core (#79684) 2026-08-13 10:30:07 +12:00
Fraser
d7a8cb8f84 Resolve settings widgets by name (#79683) 2026-08-13 10:29:31 +12:00
Anderson
32d8cbe5d8 Allow pinned collection items to be selected alongside table rows (#79614)
* Make useListSelect key-safe and pinned cards selectable

* Cover shared pinned and table selection with tests

Adds a CollectionContent selection harness and end-to-end coverage for
selecting the same item from its pinned card and its table row, and points
the pinned item menu helper at the card element so it still resolves once
the card becomes a selection target rather than a link.

* Swap the pinned card icon for a checkbox on hover in select mode

* Reset pinned card hover state outside select mode

* Target projected metric dataset requests in e2e

* Simplify pinned selection and select-all handling

* Simplify pinned selection inputs

* Share one selection entry between pinned cards and table rows

Pinned items now appear in the collection contents list (#79679), so the
pinned grid and the table render the same logical item twice. Both read and
write the single useListSelect entry keyed by id plus model, and select-all
covers the listed page once instead of composing a separate pinned set.

* Simplify selected pinned card styling
2026-08-12 19:17:34 -03:00
Nemanja Glumac
b4366d39b7 Consolidate driver ci labels under ci:run-<driver> (#79668)
* consolidate driver CI labels under ci:run-<driver>

break-quarantine-<driver> and ci:run-<driver> were two spellings of the
same override: ci:run-* is checked at Priority 3, ahead of the quarantine
check, so it already forces a quarantined driver to run.

Drop the break-quarantine-* family and point the quarantine-conflict hint,
the mage help, and the workflow error messages at ci:run-<driver>.

* rename ci:all-cloud-drivers to ci:run-all-cloud-drivers

Every other driver force-run label is spelled ci:run-*.


Co-authored-by: Chris Vandoodewaard <chris.vandoodewaard@metabase.com>
2026-08-12 22:38:11 +02:00
Sloan Sparger
d678585e39 [Metabot] Handle expired sessions (#79742)
fix
2026-08-12 12:51:04 -05:00
Sloan Sparger
cb5adad43b [Metabot] Better scrolling on historical convo load (#79734) 2026-08-12 17:49:21 +00:00
Sloan Sparger
ab6756d28f [Metabot] Hide Metabot copy action for textless responses (#79727) 2026-08-12 12:48:26 -05:00
Chris Vandoodewaard
7b8de67c1c only run milestone workflows in metabase/metabase (#79767)
* guard all milestone workflows so they only run in metabase/metabase

* add guards/notes for other workflows doing milestone things

* format

* typo

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>

* another typo

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>

---------

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
2026-08-12 16:26:10 +00:00
Ben Grabow
b59cd74dc8 Fix flaky cleanup of poll thread (#79729) 2026-08-12 10:59:27 -05:00
Andrei Chernykh
b0826c98f3 Convert app.js and app-main.js entry points to TypeScript (#79724)
* Convert app.js and app-main.js entry points to TypeScript

* Throw instead of logging when the root element is missing
2026-08-12 08:55:03 -07:00
Andrei Chernykh
893c614bed Convert metabase-lib/v1 parameter-values utils to TypeScript (#79555)
* Convert metabase-lib/v1 parameter-values utils to TypeScript

* Address review feedback: keep falsy values in conjunct, flatten test describes
2026-08-12 08:54:03 -07:00
Andrei Chernykh
6580d35948 chore: Convert schema.js to TypeScript (#79685)
* Convert schema.js to TypeScript

* Shorten comments

* Remove dead CacheConfigSchema

* Replace db_id cast with checkNotNull
2026-08-12 08:50:33 -07:00
Chris Vandoodewaard
3e231408dd add repo to release notification (#79770) 2026-08-12 11:37:16 -04:00
Sloan Sparger
a33d96bbd3 [Metabot] Handle content-filter finish reason (#79336)
handle content-filter finish reason
2026-08-12 10:15:31 -05:00
Sloan Sparger
7514efdd2a [Metabot] Handle length finish reason (#79335)
* support length finish reason at the api level

* add length stop reason ui

* TEMP COMMIT

* sure up new providers

* remove trailing whitespace
2026-08-12 10:15:31 -05:00
Jeff Bruemmer
b8e6d94a47 docs - impersonation notes (#79768) 2026-08-12 14:17:57 +00:00
Oleh K
1944ddd2c7 Stop the alert run-status icon from shrinking (#79747)
* Stop the alert run-status icon from shrinking

The warning icon sits in a flex row next to the run's error message, and
with flex-shrink at its default of 1 a long message squeezed it down to a
few pixels wide while its height stayed at 16px.

* Use FixedSizeIcon for the alert run-status icon
2026-08-12 14:22:24 +02:00
Andrei Simionescu
c30815f910 BOT-1921: Document additional Metabot environment variables (#79662)
Ten Metabot settings were marked :doc false, excluding them from
environment-variables.md. Remove the flag and write real descriptions
so MB_METABOT_NAME, MB_METABOT_ICON, MB_METABOT_SHOW_ILLUSTRATIONS,
the three system prompt settings, and the four EE usage-limit/
permissions settings show up in the generated docs.
2026-08-12 10:57:08 +01:00
Romeo Van Snick
08beefc835 Give the admin section its own chunks (#79643)
* Give the admin section its own chunks

* Load the admin pages as two chunks

* Fetch the tenant people and group pages when their route is matched

* Fetch the permissions pages when their route is matched

* Export AdminApp and DataPermissionsPage by name

* Say why the tenant wrappers get their own chunk
2026-08-12 13:57:40 +10:00
Fraser
74b08a51c8 Update module-boundaries config (#79733) 2026-08-12 14:33:34 +12:00
Fraser
244700d893 Create shared value-formatting module (#77771) 2026-08-12 13:19:22 +12:00
Romeo Van Snick
2a3f535c39 Prefetch a code-split page when a link to it scrolls into view (#79688) 2026-08-12 10:39:11 +10:00
Denis Berezin
eebccaad10 UXW-4973 Correctly show cache durations (#79499)
* UXW-4973 Correctly show cache durations

* Add validation

* Review fixes

* Fix CI

* Review fixes

* Fix e2e test
2026-08-12 02:09:14 +03:00
Andrei Chernykh
3b936768f0 Show pinned items in the collection contents list (#79679)
* Show pinned items in the collection contents list

* Update e2e specs for pinned items appearing in the contents list

* Simplify the collection empty state offset to a plain vh value

* Simplify bookmark state, skip the pinned query in trash, and drop the dead toolbar prop

* Use a real pointer click for the pinned item menu so hover cannot be raced away
2026-08-11 16:00:16 -07:00
Kevin Kelleher
1d56ea8657 Use consistent icon for explorations (#79728)
use consistent icon for explorations
2026-08-11 21:58:28 +00:00
Kevin Kelleher
4981a05a19 Include library in Entity and Mini Pickers after downgrade to OSS (#79678)
include library in Entity and Mini pickers after downgrade to OSS
2026-08-11 17:38:30 -04:00
Mihael Konjević
bd8bab1d36 fix: run source-card field-type inference before cross-stage inference (BOT-1604) (#78799)
* fix: run source-card field-type inference before cross-stage inference (BOT-1604)

The repair pipeline mini-resolves earlier stages to type a later stage's
cross-stage field refs, but ran that pass before stamping base-type onto a
source-card stage's own bare-name field refs. The mini-resolve of a
source-card-sourced prefix then always failed schema validation and was
silently skipped, leaving cross-stage refs untyped and the query permanently
unrunnable for any multi-stage construct_notebook_query built on a saved
question/model.

* docs: fix stale pass numbering in repair pipeline docstring (BOT-1604)

Number the new source-card field-type pass 4.5 and keep the pre-existing
cross-stage pass as 5, instead of bumping cross-stage to 5.5 - this leaves
the other in-file references to "Pass 5" correct and matches the file's
existing fractional-pass-number convention.
2026-08-11 23:28:22 +02:00
Mihael Konjević
56117f5bb8 Surface metric explicit-join dimensions via agent feedback (BOT-1612) (#74989)
- metric-details surfaces the metric's FK-less join dimensions (which the
  consumer-framed dimension list silently drops), each carrying the exact
  portable join clause to paste into `joins:` and an alias-qualified field
  reference. Fail closed: an entry is emitted only when its join clause exports
  and targets a real table; join-key columns are excluded.
- the :no-fk-path repair error is now actionable, pointing the agent at the
  metric's /dimensions resource for the join to add.

Also fix a pre-existing option-key typo in resources.clj
(:with-queryable-dimensions / :with-field-values -> the ?-suffixed forms the
readers expect) so the plain metric resource correctly skips dimensions and
field-value fetches.
2026-08-11 23:14:06 +02:00
Chris Vandoodewaard
9b9b40a5f9 centralize linting workflows (#79673)
* init file

* fix linting

* delete files moved into lint.yml

* try parallel steps

* more parallel

* rework so that dependencies are ordered properly

* match naming

* remove whitespace

* update file-paths, add concurrency block
2026-08-11 11:15:25 -04:00
Denis Berezin
24e36a9f0e UXW-4712 Fix non-applicable empty series tooltip values for funnel bar charts (#79602) 2026-08-11 17:33:55 +03:00
github-automation-metabase
c2d55422d1 Auto-update documentation for master (#79680)
Update generated docs (master)

Co-authored-by: github-actions <github-actions@github.com>
2026-08-11 10:21:23 -04:00
Timur Mashara
c0fe1712f6 docs - host the model button image in the repo (#79707)
* docs - host the model button image in the repo

The troubleshooting guide linked the model button screenshot from
https://www.metabase.com/learn/images/models/model-icon.png, which now
404s with no redirect. Learn images were moved out of the marketing repo
into a CDN bucket, and that key space has been renamed several times with
old keys hard-deleted rather than redirected.

This broke the nightly Update Docs job in metabase/docs.metabase.github.io
on Aug 8-10. It only goes green today because the link gate tolerates one
broken link and the other one got fixed upstream.

Commit the image here instead of chasing the CDN. It matches how every
other image in docs/ is referenced, it gets versioned along with the
release branch so older doc snapshots keep the image they shipped with,
and it is the only form our link checkers actually validate: lychee runs
--offline (external URLs skipped) and the docs site's htmlproofer runs
--disable-external.

The source file was a WebP served with a .png extension, so it has been
transcoded to a real PNG - pngcrush in optimize-images.yml would have
failed on the original bytes.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* Optimized 1 images, saved 1.8 KB (1.7% smaller)

---------

Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-authored-by: Metabase Automation <github-automation@metabase.com>
2026-08-11 16:16:18 +02:00
Egor Grushin
24b402cf95 Fix icon and text vertical alignment in incremental transform alert (#79681)
fix(ui): align Alert icon with text when there is no title

Icon box was locked to the 24px title row, so title-less alerts (e.g.
incremental transform warning) hung the icon above a single line of text.
Without a title, icon and close button now center on the first message line.
2026-08-11 15:04:31 +01:00
Timofey Kachalov
ca89dc06f1 Dev Data app fixes (#79604) 2026-08-11 17:50:44 +04:00
Chris Vandoodewaard
698956f32f milestones should only be updated/expected in metabase/metabase (#79539)
* onyl run set-milestone in metabase/metabase

* don't trigger cloud issues if this isn't metabase/metabase

* run ci?
2026-08-11 09:05:08 -04:00
Nemanja Glumac
4b10b24deb Run the block-master-source-branch-prs workflow conditionally (#79706) 2026-08-11 11:37:00 +00:00
Paolo Rascunà
0635ba585d Add prascuna to Metabot team and Slack map (#79704)
Co-authored-by: Paolo Rascuna <1936871+prascuna@users.noreply.github.com>
2026-08-11 12:30:14 +02:00
Andrei Simionescu
e9d98d62c3 BOT-1953: Suppress the Metabot sidebar on the conversation route (#79574)
* BOT-1953: Suppress the Metabot sidebar on the conversation route

The full-page Metabot surface disables the app-bar button and the Cmd+E
shortcut so the chat cannot open on top of itself. That check compared the
pathname against /question/ask exactly, but #77549 added a second route
rendering the same MetabotAsk, /metabot/conversation/:convoId, which
MetabotAsk itself redirects to after the first message. The sidebar came
back from there on.

useIsFullPageMetabot matches either route and drives both suppression
sites. useIsAskPage stays strict for MetabotAsk's redirect, which must not
re-fire once it is already on the conversation route. The route path now
has one spelling, Urls.CONVERSATION_BASE_PATH, so a rename cannot silently
turn the suppression off again.

The conversation matcher pins the route shape rather than a prefix, with an
optional trailing slash because react-router matches the route either way.

Cmd+E had no coverage because a note in the test harness claimed tinykeys
shortcuts do not fire under jsdom. They do, bound to Control rather than
Meta since jsdom reports an empty navigator.platform.

* BOT-1953: Match the Metabot route predicates with react-router

useIsAskPage compared the pathname to /question/ask by string equality,
so /question/ask/, which react-router serves as the same page, left the
app-bar button and Cmd/Ctrl+E active on a full-page Metabot surface.

Both predicates now go through matchPath, the matcher the router itself
uses: trailing slash and case tolerated, extra segments and an empty
conversation id rejected.

* BOT-1953: Keep the urls module to plain URL builders

The two route predicates move into the metabot hooks that use them, which
now call matchPath directly; urls/ goes back to exporting only functions
that build strings. Their url-level specs go with them, since the metabot
ui and app-bar button specs already pin the behavior.
2026-08-11 09:18:45 +01:00
Fraser
a6b5e09200 Count dynamic imports as module boundary edges (#79693) 2026-08-11 17:42:21 +12:00
Romeo Van Snick
fa3af4772e Give the reference section its own chunk (#79641) 2026-08-11 15:20:08 +10:00
Romeo Van Snick
8350e6cc4a Fetch the AI auditing pages when their route is matched (#79565) 2026-08-11 11:54:21 +10:00
Denis Berezin
2f3d32a2ba UXW-4983 disable download to pdf for empty dashboard tab (#79490)
* UXW-4983 disable download to pdf for empty dashboard tab

* Add tooltip

* Update Loki Snapshots

* Review fixes - hide button when dashboard is empty

* Update Loki Snapshots

* Fix e2e

* Review fix

---------

Co-authored-by: Metabase Automation <github-automation@metabase.com>
2026-08-11 00:45:44 +03:00
Kevin Kelleher
c9da277bd3 Fix dashboard click behavior for date columns without a unit (#79664)
add fallback unit for formatting unbinned date columns
2026-08-10 17:43:06 -04:00
github-automation-metabase
d225a5f8d0 Auto-update documentation for master (#79622)
Update generated docs (master)

Co-authored-by: github-actions <github-actions@github.com>
2026-08-10 16:09:01 -04:00
Jeff Bruemmer
16d281da2a docs - custom viz embedding web components (#79403) 2026-08-10 19:57:07 +00:00
Chris Vandoodewaard
7592d39e42 combine all notification workflows into issue-notifactions.yml (#79659)
* combine all notification workflows into issue-notifactions.yml

* try proposed naming schema

* add newline

* remove unused on-new-feature-label job
2026-08-10 15:42:21 +00:00
Ryan Laurie
9e6e08bffb remove dead workflows and support code (#79624)
* remove dead workflows and support code

# Conflicts:
#	.github/workflows/build-for-release.yml
#	.github/workflows/main.yml
#	.github/workflows/presto-kerberos-integration-test.yml

* remove dead clojure files
2026-08-10 09:32:38 -06:00
Andrei Chernykh
9bdf46f3d7 Stop autopinning models and metrics on creation (#79548)
* Stop autopinning models and metrics on creation

* Fix e2e assertions that expected new models to be pinned

* Remove negative-case pin tests per review
2026-08-10 08:26:16 -07:00
Chris Vandoodewaard
814b57b74a only publish Github release for major and gold releases (#79478)
* only publish Github release for major and gold releases

* pr review fixes
2026-08-10 10:16:58 -04:00
Denis Berezin
570a0e18c8 UXW-4929 Resolve unknown string dashboard ids (#79586) 2026-08-10 16:43:48 +03:00
Romeo Van Snick
bfa63e2a92 Replace the Bun globals in the yarn.lock alias script to fix the type check (#79650)
* Use node:fs in the yarn.lock alias script to fix the type check

* Wrap the alias script in a main function instead of top-level await
2026-08-10 14:14:54 +02:00
Chris Vandoodewaard
ec9e8c40f0 post warning about deprecated labels (#79618)
* post warning about deprecated labels

* cleanup double/triple backport logic, fail if labels set

* fix message, whitespace

* silent api response
2026-08-10 08:00:34 -04:00
Romeo Van Snick
518ca39d56 CI: rebuild node_modules from the bun store on install (#79644)
Rebuild node_modules from bun store to avoid stale CI cache
2026-08-10 13:47:08 +02:00
Kevin Kelleher
0fc5582ae2 Prefetch queries from Exploration toolbar navigation (#79615)
* move prefetch up into ExplorationPage

* fix e2e
2026-08-10 07:37:22 -04:00
Romeo Van Snick
b750f8d8fa Split documents into its own chunk (#79349)
* Add lazyModalRoute for modals under a code-split page

* Split documents into its own chunk

* Prefetch the documents chunk from links into it

* Prefetch the comments sidesheet with the document page
2026-08-10 21:02:14 +10:00
Romeo Van Snick
fc7d75b01f Fix Snyk scan by renaming bun's aliased yarn.lock keys (#79646)
* Fix Snyk scan by renaming bun's aliased yarn.lock keys

* Trigger the Snyk workflow on changes to the alias fix script
2026-08-10 21:02:02 +10:00
Uladzimir Havenchyk
af266d1426 fix: pass isSensible to custom viz (#79325)
* fix: pass isSensible to custom viz

* fixup

* address review

* fixup

* address review

* address review

* fixup

* fixup

* address custom viz re-render

* fixup
2026-08-10 12:12:01 +02:00
Andrei Simionescu
660b7ac2db BOT-1944: Fix two Metabot tool failures (#79498)
* Fix two Metabot tool failures surfaced by the reliability data

get_transform_details passed the transform's query map straight into the XML
formatter. clojure.data.xml reads a map in second position as an attribute map
and then throws on MBQL 5's namespaced :lib/type key, so every query-source
transform failed and only Python ones (no :query key) got through. That is 72
errors in 75 calls over 90 days, across 15+ instances. The query now renders as
SQL text, matching how the line below already handles :target.

create_sql_query has two implementations sharing one tool name, and neither used
the shared agent-error contract. The :sql profile's code-edit variant had no
error handling at all, so the :agent-error? exceptions validate-database-access
raises escaped uncaught and were tracked as tool failures. That variant errored
on 26% of calls against 0.04% for the :internal one. Both now delegate to
handle-agent-error, as the other tools do: the intended validation message goes
back to the model, and anything unexpected is rethrown so real defects stay
visible in the telemetry rather than being reported as successes.

Neither tool had a test; both now do.

* Address review: transform rendering and 403 classification

Transform source queries now render verbatim (data.xml escaped <, >, &,
which breaks old_string matching against the real query), python bodies
render, and the source type attribute is no longer a keyword literal.
Notebook-built queries fall back to EDN with the attached metadata
provider stripped, shared with transform->xml, whose string guard always
dropped the normalized (map) query.

validate-database-access tags the plain 403 from read-check so both
create_sql_query variants classify permission failures as agent errors,
and the pre-classification ERROR logs are gone; run-tool already logs
both paths at the right levels.

* Tolerate string queries in transform-query->text

Legacy transforms stored raw SQL strings under :query, and orphaned sources
skip normalization, so the query reaching this helper is not always a map.
dissoc on a string threw ClassCastException; the search formatter's old
string guard had tolerated exactly this shape. Strings now pass through
verbatim, and the helper's namespace gets its first tests.

* Use a doc link for escape-xml in the escape-xml-content docstring

* Read orphaned sources' string keys in extract-sql-content

Orphaned transform sources skip normalization and keep their JSON string
keys, so the keyword probes missed them and the model got EDN of a
string-keyed map instead of the SQL. Multi-stage queries now fall through
to the EDN branch too: stage 0 alone is partial SQL and hid the rest.

Also tags the read-check 404 alongside the 403, closing the race where
the database vanishes between the exists? check and the read.
2026-08-10 10:26:23 +01:00
Fraser
cc136d9725 Move the query pickers into querying (#79561) 2026-08-10 20:50:57 +12:00
Romeo Van Snick
aee69e552a Drop prettier/standalone from the expression formatter and load the printer on demand (#79639)
* Print expressions with prettier's doc printer instead of prettier/standalone

* Load the expression printer on demand
2026-08-10 13:13:28 +10:00
metabase-bot[bot]
4dc84c1615 [chore] Update E2E test timings with recent data (#79628)
Update E2E test timings with averaged data

Collected timing data from 7 days of test runs.
All specs with timing data were included (simple averages).

🤖 Generated automatically by update-e2e-timings workflow

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: GitHub Actions <actions@github.com>
2026-08-09 09:13:08 +02:00
Nemanja Glumac
fb00ed06ef Remove the test-results S3 upload (#79552)
We no longer upload JUnit XML to S3, so the action and all 47 call sites
go. Test results stay visible through the JUnit reporter and CI
Conductor.

`id-token: write` drops from minimal-tests and transforms-python-stress-test,
whose only OIDC consumer was the upload's AWS credential handshake. It
stays in run-tests, which must keep granting it to drivers.yml's ECR login.

Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-08 00:41:53 +02:00
Anderson
7675914513 Add in-collection type filters with dynamic options (#79340)
* Add dynamic collection type filters

* Complete collection filter coverage

* Harden dynamic collection filters

* Simplify collection type filters

* Address review feedback on collection type filters

* Additional clean up
2026-08-07 18:57:13 -03:00
Anderson
28160265ec Implement in-collection text search (#79333)
* Add in-collection item search

* Add collection search end-to-end coverage

* Fix collection search navigation and coverage

* Add loading state and fix spacing for collection search

* Keep collection tables mounted during refetches

* A couple UI refinements

* Show collection search loading inline
2026-08-07 18:57:12 -03:00
Anderson
3ec36759e7 Backend: search and type-filter metadata for collection items (#79332)
* Add collection item search and filter metadata

* Add collection authority filters

* Cover collection search edge cases

* Keep snippet pinned-state behavior unchanged

* Never report snippets in available_models

* Remove collection authority filters

* Code cleanup

* Cover explorations in available models

---------

Co-authored-by: Nathan Voxland <nathan@metabase.com>
2026-08-07 18:57:12 -03:00
Denis Berezin
d8a19a330a UXW-4939 Fix crash on dashboard tab duplication (#79554)
Co-authored-by: azharizz <azharizzannada@gmail.com>
2026-08-08 00:34:11 +03:00
3336 changed files with 866867 additions and 763101 deletions

View File

@@ -0,0 +1,248 @@
---
name: fe-modularization
description: Deciding where frontend code lives — tier model, move mechanics, extension points, side effects, naming, and the traps. Use when moving code between modules, carving new modules, fixing boundary violations, adding a barrel or an endpoint, or reviewing module-shape decisions.
---
# Frontend modularization
The module is the unit of blast radius: dependency direction, test selection,
bundle cones, and review scope all follow module boundaries. Every decision
here serves one goal: **make the file locations tell the truth about who owns
what and who may depend on whom.**
## Hard rules (check these before anything else)
- NEVER add an `export … from` re-export at an old path, and NEVER `export *`
from a barrel. Move the code, codemod every call site, delete the old path,
all in the same PR.
- NEVER import past a module's `index.ts` when that module has
`enforcePublicApi: true`, and never past `metabase/ui`'s index at all. If
the barrel is heavy, the fix is making the module side-effect free (below),
not a deep import.
- NEVER rewrite a consumer to a lower-level idiom to make the linter pass. If
the fix makes the consumer worse, the plan is wrong.
- NEVER give a new module `enforceSharedTiers: false`; new modules ship
enforced with `enforcePublicApi: true` and an explicit `index.ts`.
- NEVER move code without the consumer-tier check (procedure step 2).
- ALWAYS delete a module's `enforceSharedTiers: false` line in the PR that
takes it to zero violations.
- ALWAYS measure `bun run module-boundaries` before and after, and report the
numbers.
- ALWAYS import `dayjs` from `metabase/dayjs`, routing from `metabase/router`,
Mantine from `metabase/ui`, react-redux hooks from `metabase/redux`; the
raw specifiers are lint-forbidden.
## Files that are ground truth
| what | where |
|---|---|
| module elements, tiers, `enforceSharedTiers`, `enforcePublicApi` | `frontend/lint/module-boundaries.mjs` |
| shared sub-tiers and levels, cluster rules | `frontend/lint/shared-tiers.mjs` |
| standalone boundaries lint (all violations, incl. grandfathered) | `bun run module-boundaries` (config `eslint.config.module-boundaries.mjs`) |
| PR lint (only enforced modules fail CI) | `bun run lint-eslint-pure` |
| side-effect-free directories and their exceptions | `frontend/build/shared/rspack/side-effect-free-modules.js` (`SIDE_EFFECT_FREE_PATHS`, `SIDE_EFFECT_PATHS`) |
| the lint rules that enforce that promise | `metabase/no-module-side-effects`, `metabase/no-base-api-access` in `frontend/lint/eslint-plugin-metabase/rules/` |
| public-api enforcement | `metabase/enforce-module-public-api` rule, driven by `getPublicApiModules()` |
## The model
Tiers: `lib < basic < shared < feature < app`. Imports point downward, never
sideways at feature tier (features may not import features; app and EE may
import anything). The shared tier is subdivided in `shared-tiers.mjs` into
shared-utils (U0…), shared-platform (P0…), and shared-domain, each ordered
into levels. A module imports only *strictly lower* levels of its sub-tier
plus the sub-tiers below; same-level peers are deliberately forbidden, which
is what makes cycles structurally impossible. A domain may hold two seats
when consumers demand it: a low core and a high surface (metabase-lib below
questions below query_builder; viz-core below visualizations).
Enforcement is per module: `enforceSharedTiers: false` on an element exempts
it from the level rules (counted by `bun run module-boundaries`, invisible to
PR lint). Modules with `enforcePublicApi: true` may only be imported from
outside via `metabase/<module>`, and import relatively inside.
## Deciding where code lives (apply in order; first decisive test wins)
1. **Who owns the concept?** Code lives with the module that owns its
concept, not the module that renders or calls it. A mode encoding dashboard
click behaviour is dashboard code even if viz executes it. Metabot
conversation state is metabot state even though the store registers it.
2. **Consumer-tier check.** The destination must sit at or below the file's
*lowest* consumer. A single consumer below the proposed home vetoes the
move; report it, don't force it.
3. **Only four legitimate fixes** for a bad edge: move the code to its
terminal home; invert through a designed extension point (a prop, an
injected component, a contract type owned by the socket); fix the tier
placement when the declared level is wrong; delete a thin wrapper (a rename
with no derivation isn't an abstraction — deleting it *is* the migration).
4. **A bridge lives above the lower endpoint.** Code that needs both sides of
a boundary belongs in or above the higher module.
5. **"Genuinely shared" is proven by consumers.** A shared address needs two
or more consumers the tier rules can't serve another way. One consumer
pretending to be shared is a feature module at the wrong address.
## Extension points
The socket's owner defines the contract; every plug lives with its extender.
Visualization owns the `mode` prop and click interfaces, each surface supplies
its mode; the editor owns its extension contract, metabot supplies tiptap
extensions; querying owns `TemplateTagsSidebarProps`, parameters implements
it.
- Prefer plain props/injection at composition sites. `PLUGIN_*` registries are
for edition gating; single-reader slots move out to their reading module;
slots with many readers live in the module that owns the concept (every
slot the whitelabel plugin fills lives in `whitelabel`), tier permitting.
Keep contracts type-light so implementations can load lazily.
- Identical injection at every callsite is acceptable until a fourth consumer
or second slot appears; then the composition deserves its own module above
both parts. Don't pre-build the wrapper.
## Store shape
A module that has redux state owns it:
- `store/` is private: reducer, plain creators/action types, and selectors,
typed against the module's own state (`QueryBuilderStoreState`), never the
global `State`. Thunks live in `actions/` and import `store/`, never the
reverse.
- The barrel is the state API: selectors for reads, creators/thunks for
writes.
- Store roots (`reducers-main`, `reducers-public`, the SDK store) import each
slice from its module barrel and derive their own state type; no slice
registry, no `declare module` merging.
- Cross-module writes: a callback prop when a composition point exists;
otherwise dispatch the owner's exported creator; an event only when the
emitter must not know its listeners. Raw `dispatch({ type: "…" })` strings
are banned. A reducer or listener matching another module's *internal*
action is the coupling to remove.
- Slices that mirror one endpoint get deleted (the RTK cache is the state); a
fold over many sources with one writer (the metadata store) is kept as a
module-private slice behind the barrel.
## Endpoints
Endpoints migrate to their owning module via `Api.injectEndpoints` from the
module's `api/` file (settings, metabot, transforms precedents); `metabase/api`
trends toward the client only. There is one `Api` object per backend because
tag invalidation only works within one instance. Consumers reach endpoints
only through the owner's exports (`useGetCardQuery`, `cardApi.endpoints.…`),
never by name through the base object; `metabase/no-base-api-access` allows
injection only in `metabase/api` and `**/api/**` / `**/api.ts`, and reaching by
name only in `metabase/api` and test support (`invalidateTags` /
`resetApiState` are fine anywhere). Each endpoint move carries its
metadata-store hookup: the owner calls `entitiesFetched` from its own
`onQueryStarted`.
## Import-time side effects
Directories in `SIDE_EFFECT_FREE_PATHS` are marked `sideEffects: false` for
rspack, so importing one export from their barrel no longer drags the whole
barrel into a bundle. The price: nothing in those directories may do work at
import time, or production silently drops it. `metabase/no-module-side-effects`
enforces this over every listed directory (module-scope calls, writes to
imported objects or globals, bare imports, control flow at import; escape
hatches are `/* #__PURE__ */` and the rule's pure-callee list). Files that
must run at import are listed in `SIDE_EFFECT_PATHS` (files or directories).
- **A library the app configures gets a facade module** owning the vendor
import, the registrations, and the types, with the raw specifier
lint-forbidden outside it: `metabase/router`, `metabase/ui`,
`metabase/redux`, `metabase/dayjs`. Facades are the effect, so they never
enter `SIDE_EFFECT_FREE_PATHS`. Never re-register a vendor plugin locally
(`dayjs.extend`, `echarts` `use`).
- **A patch a component needs is an explicit registration module,
bare-imported by the provider that needs it** and listed in
`SIDE_EFFECT_PATHS` (Mantine's `Popover.Dropdown` replacement, imported by
`ThemeProvider`).
- **Effects with no value to import** (global CSS, polyfills, the CSP nonce,
EE plugin bootstrap) live in the entry's side-effects file and nowhere else.
- **Registries are filled from the composition root at boot**
(`initializePlugins`, `registerVisualizations`), never by a module
registering itself at import.
- **RTK injection stays at import**; when a module with an `api/` folder is
declared side-effect free, list `<module>/api/` in `SIDE_EFFECT_PATHS`.
## Naming
The feature keeps the plain product-surface name; a shared carve is named for
what the shared part actually is: the concept when the concept is the shared
thing (`settings`, `current-user`, `whitelabel`, `dayjs`), or a qualified
capability when the plain noun is taken (`search-ui`, `metrics-ui`,
`viz-core`, `data-studio-ui`). Never two modules distinguished only by tier
prefix; never generic `-shared` / `-feature` suffixes.
## Procedure for a move
1. **Enumerate consumers honestly** (hook-name greps miss the last three):
```
grep -rn "metabase/<module>" frontend/src enterprise/frontend/src frontend/test e2e
grep -rn "jest.mock(\"metabase/<module>" frontend enterprise/frontend
grep -rn "addMatcher(.*<endpoint>" frontend/src enterprise/frontend/src
```
plus sibling files importing the moved file relatively. Classify each by
module and tier from `module-boundaries.mjs` / `shared-tiers.mjs`.
2. **Consumer-tier check**: destination at or below the lowest consumer. If
not, stop and report the blocking consumer.
3. **Measure before**: `bun run module-boundaries 2>&1 | tail -1` on the base
(write the number down).
4. **Move**: `git mv`; codemod every importer to `metabase/<module>` (outside)
or a relative path (inside); add named exports to the destination barrel
for every symbol an outside consumer needs (each must have a real
consumer). Delete the old path. If the module didn't exist: add its element
in `module-boundaries.mjs` (ordered before any element it's carved from,
first match wins), its level in `shared-tiers.mjs`, `enforcePublicApi:
true`, an `index.ts`. If the move empties a module's violations, delete
its `enforceSharedTiers: false` line. If the module has an `api/` folder
and is side-effect free, list `<module>/api/` in `SIDE_EFFECT_PATHS`.
5. **Verify** (all of these, report the numbers):
```
grep -rn "<old path>" frontend enterprise/frontend e2e # must be empty
bunx eslint --no-warn-ignored <touched files>
bun run lint-eslint-pure # enforce-module-public-api included
bun run type-check-pure
bun run test-unit-keep-cljs <touched folders' specs>
bun run module-boundaries 2>&1 | tail -1 # after; no violation may name the new files
bun run build-release:static-viz # only if viz/static-viz/ui/dayjs touched; hard 3.5 MiB budget
```
6. **Report**: count before/after (state count-neutral explicitly), the edges
removed by file, flags/rules deleted, barrel exports added, anything you
skipped and why, and any consumer you found that vetoed part of the move.
## Traps (each has bitten at least once)
- **Late binding.** A bare function reference in a `createSelector` input
array, a component identity captured for comparison, a `PLUGIN_*` object
read at module scope: all capture at import time and dodge jest spies, EE
overrides, and lazy loading. Read at call time (`(_state) => fn()`), or
inject.
- **Plans go stale.** Verify every recipe against current code before
executing; things get merged, renamed, or turn out never to have existed.
Skip-and-report beats force.
- **The linter's blind spots**: `export … from` re-exports, bare-specifier
packages (custom-viz), pre-seeded caches that never register tags. A grep
for the old path is the stronger claim than a green lint run.
- **Path-keyed baselines reset on rename**: a generated per-file ledger
silently loses a file's history when it moves.
- **A heavy barrel is a bundle hazard until its module is side-effect free.**
One constant imported from the `metabase/ui` barrel pulled Mantine into the
static-viz bundle and broke its budget; the fix was `SIDE_EFFECT_FREE_PATHS`,
never a deep import.
- **Type augmentation is program-wide.** `declare module "dayjs"` (and
similar) makes plugin types available to any file in the same tsconfig, so
a passing type-check never proves the setup ran; only importing the facade
does.
- **Environment before diagnosis**: after a dependency-touching merge,
`bun install` before declaring a failure real; worktrees share staleness
through the node_modules symlink, and a stale `target/cljs_dev` makes
`type-check-pure` fail on `metabase-lib/metric/core.ts` (rebuild cljs in the
worktree).
## PR shape
One logical change per PR; the body names the specific edges removed (files,
imports, before/after count) and gives each move's ownership reason in plain
language a reviewer outside the effort can follow, without campaign labels.
Stack only for semantic dependency or same-hot-file serialisation, and say
so. Consider codeowner surface when chunking. Open as a draft until measured
and green.

View File

@@ -75,10 +75,6 @@ description: Write TypeScript and JavaScript code following Metabase coding stan
- **No comments by default**. Well-named identifiers carry the `what`.
- **Comments should be concise**. Add a short, concise comment only when the `why` is non-obvious: a workaround, a hidden invariant, a subtle ordering constraint, a clever reduction. Never document the actual implementation, focus on the intent and the why.
## TypeScript Migration
**When touching existing JavaScript files, propose to convert them to TypeScript first**. Create a separate PR for the conversion, then implement the changes.
## Verify before done
- **Run the project type-check** when finished (see the shared TypeScript commands above).

View File

@@ -403,6 +403,7 @@
metabase.channel.render.core channel.render
metabase.cloud-migration.core cloud-migration
metabase.eid-translation.core eid-translation
metabase.llm.test-util llm.tu
metabase.login-history.core login-history
metabase.model-persistence.core model-persistence
metabase.notification.core notification
@@ -523,7 +524,6 @@
metabase.test.data.interface/defdataset clojure.core/def
metabase.test.data.interface/defdataset-edn clojure.core/def
metabase.test.embedded-postgres.core/with-system clojure.core/let
metabase.test.gentest/defgentest clojure.test/deftest
metabase.test/defdataset clojure.core/def
metabase.test/let-url clojure.core/let
metabase.test/with-actions clojure.core/let
@@ -682,7 +682,6 @@
metabase.test.data/dataset hooks.metabase.test.data/dataset
metabase.test.data/mbql-query hooks.metabase.test.data/mbql-query
metabase.test.data/run-mbql-query hooks.metabase.test.data/mbql-query
metabase.test.gentest/iterate hooks.metabase.test.gentest/iterate*
metabase.test.util.async/with-open-channels hooks.common/let-with-optional-value-for-last-binding
metabase.test.util.generators.jvm/with-random-cards hooks.metabase.test.util.generators.jvm/with-random-cards
metabase.test.util.log/with-log-level hooks.common/with-ignored-first-arg
@@ -801,7 +800,7 @@
:name test-namespaces}
{:pattern "^(?!.*-test(?:\\..*)?$)(?!.*test-util)(?!^metabase\\.test.*).*$"
:name source-namespaces}
{:pattern "(?:(?:^metabase\\.cmd.*)|(?:^build.*)|(?:^metabuild-common.*)|(?:^release.*)|(?:^i18n.*)|(?:^lint-migrations-file$))|(?:^mage.*)"
{:pattern "(?:(?:^metabase\\.cmd.*)|(?:^build.*)|(?:^metabuild-common.*)|(?:^release.*)|(?:^i18n.*)|(?:^lint-migrations-file$))|(?:^mage.*)|(?:^load-namespaces.*)"
:name printable-namespaces}
{:pattern "^metabase\\.(?:(?:lib)|(?:legacy-mbql))\\..*"
:name metabase-lib}

View File

@@ -473,6 +473,7 @@
app-db
config
events
lib
models
permissions
premium-features
@@ -623,8 +624,7 @@
collections-rest
{:team "UX West"
:api #{metabase.collections-rest.api
metabase.collections-rest.init}
:api #{metabase.collections-rest.api}
:uses #{api
app-db
collections
@@ -638,7 +638,6 @@
queries
request
revisions
settings
tracing
transforms
upload
@@ -735,6 +734,7 @@
permissions
public-sharing
queries
query-permissions
query-processor
search
settings
@@ -783,17 +783,19 @@
revisions
util
xrays}
:model-imports #{:model/Card
:model-imports #{:model/Action
:model/Card
:model/Collection
:model/Dashboard
:model/DashboardCard
:model/DashboardCardSeries
:model/DashboardTab
:model/Field
:model/ParameterCard
:model/Pulse
:model/PulseChannel
:model/PulseChannelRecipient
:model/Query
:model/Table
:model/User}}
data-studio
@@ -831,7 +833,9 @@
batch-processing
collections
events
lib-be
models
parameters
public-sharing
queries
query-permissions
@@ -874,6 +878,7 @@
:uses #{analytics-interface
app-db
appearance
audit-app
auth-provider
classloader
config
@@ -974,6 +979,7 @@
eid-translation
embedding
events
lib
models
notification
parameters
@@ -985,6 +991,11 @@
util}
:model-imports #{:model/Card :model/Dashboard :model/DashboardCard :model/EmbeddingTheme}}
embeddings
{:team "Metabot"
:api #{metabase.embeddings.provider metabase.embeddings.startup}
:uses #{plugins util}}
entity-retrieval
{:team "Metabot"
:api #{metabase.entity-retrieval.core
@@ -1209,7 +1220,8 @@
metabase.lib.schema.util
metabase.lib.schema.validate
metabase.lib.types.isa
metabase.lib.util}
metabase.lib.util
metabase.lib.walk}
:uses #{config
graph
legacy-mbql
@@ -1224,9 +1236,9 @@
lib-be
{:team "Querying Platform"
:api #{metabase.lib-be.core
metabase.lib-be.init}
:uses #{lib
:api #{metabase.lib-be.core metabase.lib-be.init metabase.lib-be.schema}
:uses #{legacy-mbql
lib
models
settings
util}
@@ -1257,6 +1269,7 @@
{:team "Metabot"
:api #{metabase.llm.api
metabase.llm.init
metabase.llm.provider
metabase.llm.settings
metabase.llm.startup}
:uses #{analytics
@@ -1267,6 +1280,8 @@
lib-be
metabot
models
parameters
permissions
premium-features
request
settings
@@ -1350,7 +1365,6 @@
metabot
{:team "Metabot"
:api #{metabase.metabot.api
metabase.metabot.api.entity-analysis
metabase.metabot.config
metabase.metabot.core
metabase.metabot.init
@@ -1550,6 +1564,7 @@
driver
embedding
events
lib
models
parameters
permissions
@@ -1675,6 +1690,7 @@
:uses #{api
audit-app
events
lib
permissions
premium-features
request
@@ -1759,6 +1775,7 @@
api
dashboards
dashboards-rest
documents
events
lib
models
@@ -1856,6 +1873,7 @@
view-log
warehouse-schema}
:model-exports #{:model/Card
:model/ParameterCard
:model/Query
:model/QueryExecution
:model/QueryTable
@@ -1879,8 +1897,7 @@
queries-rest
{:team "Querying Platform"
:api #{metabase.queries-rest.api}
:uses #{analyze
api
:uses #{api
collections
eid-translation
embedding
@@ -1917,7 +1934,7 @@
query-processor
request
util}
:model-imports #{:model/Card :model/Field}}
:model-imports #{:model/Card :model/Field :model/Table}}
query-processor
{:team "Querying Platform"
@@ -2030,22 +2047,28 @@
metabase.revisions.core
metabase.revisions.init}
:uses #{api
collections
config
dashboards
events
lib
lib-be
models
parameters
queries
query-permissions
util}
:model-exports #{:model/Revision}
:model-imports #{:model/Card
:model/Collection
:model/Dashboard
:model/DashboardCard
:model/DashboardCardSeries
:model/DashboardTab
:model/Document
:model/Exploration
:model/Measure
:model/ParameterCard
:model/Segment
:model/Transform}}
@@ -2163,6 +2186,7 @@
request
session
settings
setup
sso
system
tracing
@@ -2254,6 +2278,7 @@
channel
formatter
lib
llm
metabot
permissions
query-processor
@@ -2496,7 +2521,6 @@
models
permissions
premium-features
queries
query-processor
remote-sync
revisions
@@ -2801,6 +2825,7 @@
:uses #{analytics
api
app-db
collections
database-routing
driver
events
@@ -2817,7 +2842,8 @@
util
warehouse-schema
xrays}
:model-imports #{:model/Database
:model-imports #{:model/Collection
:model/Database
:model/Dimension
:model/Field
:model/FieldValues
@@ -2939,8 +2965,7 @@
enterprise/advanced-config
{:team "UX West"
:api #{metabase-enterprise.advanced-config.api
metabase-enterprise.advanced-config.api.logs
:api #{metabase-enterprise.advanced-config.api.logs
metabase-enterprise.advanced-config.file
metabase-enterprise.advanced-config.init
metabase-enterprise.advanced-config.settings}
@@ -3026,6 +3051,7 @@
app-db
audit-app
classloader
config
driver
lib
models
@@ -3095,7 +3121,7 @@
enterprise/cloud-add-ons
{:team "Cloud"
:api #{metabase-enterprise.cloud-add-ons.api}
:api #{metabase-enterprise.cloud-add-ons.api metabase-enterprise.cloud-add-ons.core}
:uses #{api
events
premium-features
@@ -3107,6 +3133,7 @@
{:team "Cloud"
:api #{metabase-enterprise.cloud-proxy.api}
:uses #{api
lib
premium-features
util
enterprise/harbormaster}}
@@ -3184,6 +3211,7 @@
collections
config
driver
embeddings
metabot
models
premium-features
@@ -3297,6 +3325,11 @@
settings
util}}
enterprise/embedder
{:team "Metabot"
:api #{}
:uses #{classloader embeddings util}}
enterprise/embedding-hub
{:team "Embedding"
:api #{metabase-enterprise.embedding-hub.api}
@@ -3390,7 +3423,7 @@
warehouse-schema
enterprise/sandbox}
:model-exports #{:model/ConnectionImpersonation}
:model-imports #{:model/DataPermissions :model/PermissionsGroupMembership}}
:model-imports #{:model/DataPermissions :model/Database :model/PermissionsGroupMembership}}
enterprise/internal-stats
{:team "UX West"
@@ -3442,6 +3475,7 @@
metabot}
:uses #{api
lib-be
llm
metabot
models
permissions
@@ -3449,6 +3483,7 @@
settings
task
util
enterprise/cloud-add-ons
enterprise/dependencies
enterprise/transforms-python}
:model-imports #{:model/AiUsageLog
@@ -3583,6 +3618,7 @@
events
legacy-mbql
lib
lib-be
models
permissions
premium-features
@@ -3669,6 +3705,7 @@
collections
config
connection-pool
embeddings
events
health-inspector
lib-be
@@ -3688,7 +3725,10 @@
enterprise/serialization
{:team "Graphy"
:api #{metabase-enterprise.serialization.api metabase-enterprise.serialization.cmd metabase-enterprise.serialization.core}
:api #{metabase-enterprise.serialization.api
metabase-enterprise.serialization.cmd
metabase-enterprise.serialization.core
metabase-enterprise.serialization.init}
:friends #{enterprise/memoize-monitor}
:uses #{analytics
api
@@ -3703,6 +3743,7 @@
premium-features
search
server
settings
setup
util}
:model-imports :bypass}
@@ -3729,6 +3770,7 @@
embedding
premium-features
request
server
session
settings
sso

View File

@@ -5,15 +5,15 @@
;; to defend in your PR.
;; :all is the vector-less ignore form, which suppresses every linter on the next form.
{:ignore-counts {:aliased-namespace-symbol 3
:all 50
:all 48
:case-symbol-test 1
:clojure-lsp/unused-public-var 9
:def-fn 1
:deprecated-namespace 95
:deprecated-var 79
:deprecated-var 84
:discouraged-java-method 3
:discouraged-namespace 79
:discouraged-var 107
:discouraged-namespace 81
:discouraged-var 111
:equals-true 1
:main-without-gen-class 1
:metabase/defsetting-namespace 2
@@ -24,8 +24,8 @@
:metabase/namespace-name 4
:metabase/prefer-with-dynamic-fn-redefs 28
:metabase/test-helpers-use-non-thread-safe-functions 17
:metabase/validate-defendpoint-has-response-schema 441
:metabase/validate-defendpoint-query-params-use-kebab-case 47
:metabase/validate-defendpoint-has-response-schema 440
:metabase/validate-defendpoint-query-params-use-kebab-case 50
:metabase/validate-defendpoint-route-uses-kebab-case 44
:metabase/validate-deftest 9
:missing-docstring 10
@@ -44,5 +44,5 @@
:unused-binding 9
:unused-excluded-var 2
:unused-namespace 1
:unused-private-var 8
:unused-private-var 9
:unused-value 1}}

View File

@@ -71,6 +71,24 @@
(some-> (ns-form-node->ns-symb-node ns-form-node)
hooks/sexpr))
(defn- lint-test-parallel-metadata [ns-form-node]
(when-let [ns-symb-node (ns-form-node->ns-symb-node ns-form-node)]
(let [ns-metadata (meta (hooks/sexpr ns-symb-node))]
(cond
(:synchronous ns-metadata)
(hooks/reg-finding!
(assoc (meta ns-symb-node)
:message (str "Use `^:synchronized` to mark this namespace explicitly non-parallel; "
"`^:synchronous` is ignored by Hawk. [:metabase/validate-deftest]")
:type :metabase/validate-deftest))
(and (contains? ns-metadata :parallel)
(false? (:parallel ns-metadata)))
(hooks/reg-finding!
(assoc (meta ns-symb-node)
:message "Use `^:synchronized` instead of `^{:parallel false}`. [:metabase/validate-deftest]"
:type :metabase/validate-deftest))))))
(defn- lint-modules [ns-form-node config]
(let [ns-symb (ns-form-node->ns-symb ns-form-node)]
(when-not (modules/ignored-namespace? config ns-symb)
@@ -140,6 +158,7 @@
lint-requires-on-new-lines
(lint-modules (modules/config x))
lint-namespace-name
lint-test-parallel-metadata
lint-jsqlparser-imports)
x)

View File

@@ -28,11 +28,11 @@
(walk form)))
(defn- deftest-check-parallel
"1. Check if test is marked ^:parallel / ^:synchronized correctly.
"1. Check if test is marked `^:parallel` / `^:synchronized` correctly.
2. Make sure disallowed forms are not used in ^:parallel tests.
3. Disallow ^:parallel tests in namespaces marked ^:synchronous -- the
ns marker is the author's blanket promise that nothing inside runs
in parallel, so an individual ^:parallel deftest contradicts it."
3. Disallow `^:parallel` tests in namespaces marked `^:synchronized` --
the ns marker is the author's blanket promise that nothing inside runs
in parallel, so an individual `^:parallel` deftest contradicts it."
[{[_ test-name & body] :children, :as _node} ns-sym config]
(let [test-metadata (:meta test-name)
metadata-sexprs (map hooks/sexpr test-metadata)
@@ -44,28 +44,48 @@
(completing merge)
{}
metadata-sexprs)
parallel? (:parallel combined-metadata)
synchronized? (:synchronized combined-metadata)
ns-synchronous? (boolean (:synchronous (meta ns-sym)))]
parallel? (true? (:parallel combined-metadata))
synchronized? (true? (:synchronized combined-metadata))
explicitly-nonparallel? (or synchronized?
(and (contains? combined-metadata :parallel)
(false? (:parallel combined-metadata))))
ns-metadata (meta ns-sym)
ns-explicitly-nonparallel? (or (true? (:synchronized ns-metadata))
(and (contains? ns-metadata :parallel)
(false? (:parallel ns-metadata))))]
(when (and (contains? combined-metadata :parallel)
(false? (:parallel combined-metadata)))
(hooks/reg-finding! (assoc (meta test-name)
:message (str "Use `^:synchronized` instead of `^{:parallel false}`. "
"[:metabase/validate-deftest]")
:type :metabase/validate-deftest)))
(when (:sequential combined-metadata)
(hooks/reg-finding! (assoc (meta test-name)
:message (str "Use `^:synchronized` to mark this test explicitly non-parallel; "
"`^:sequential` is ignored by Hawk. "
"[:metabase/validate-deftest]")
:type :metabase/validate-deftest)))
(when (and parallel? synchronized?)
(hooks/reg-finding! (assoc (meta test-name)
:message "Test should not be marked both ^:parallel and ^:synchronized"
:message "Test should not be marked both `^:parallel` and `^:synchronized`"
:type :metabase/validate-deftest)))
(when (and parallel? ns-synchronous?)
(when (and parallel? ns-explicitly-nonparallel?)
(hooks/reg-finding!
(assoc (meta test-name)
:message (str "Test should not be marked ^:parallel in a namespace marked ^:synchronous -- "
:message (str "Test should not be marked `^:parallel` in a namespace marked "
"`^:synchronized` -- "
"the ns marker promises no parallel execution. "
"Drop ^:parallel from the deftest, or drop ^:synchronous from the ns. "
"Drop `^:parallel` from the deftest, or drop `^:synchronized` from the ns. "
"[:metabase/validate-deftest]")
:type :metabase/validate-deftest)))
;; only when the custom `:metabase/deftest-not-marked-parallel-or-synchronized` is enabled: complain if tests are
;; not explicitly marked `^:parallel` or `^:synchronized`. This is mostly to encourage people to mark everything
;; `^:parallel` in places like `metabase.lib` tests unless there is a really good reason not to.
(when-not (or parallel? synchronized?)
;; Only when the custom `:metabase/deftest-not-marked-parallel-or-synchronized` is enabled:
;; complain if tests are not explicitly marked `^:parallel` or `^:synchronized`. This is
;; mostly to encourage people to mark everything `^:parallel` in places like `metabase.lib`
;; tests unless there is a really good reason not to.
(when-not (or parallel? explicitly-nonparallel?)
(hooks/reg-finding!
(assoc (meta test-name)
:message "Test should be marked either ^:parallel or ^:synchronized"
:message "Test should be marked either `^:parallel` or `^:synchronized`"
:type :metabase/deftest-not-marked-parallel-or-synchronized)))
(when parallel?
(doseq [form body]
@@ -153,6 +173,7 @@
"build."
"i18n." ; bin/i18n
"lint-migrations-file-test"
"load-namespaces." ; bin/load-namespaces
"mage."
"main-test" ; bin/release-list
"metabase.deps-edn-test"
@@ -232,18 +253,20 @@
(defn use-fixtures
"Flag `:parallel/unsafe` forms inside fixtures. Skipped when the surrounding
namespace is marked `^:synchronous` -- the ns marker is the author's
namespace is marked `^:synchronized` -- the ns marker is the author's
explicit opt-in to a single-threaded test ns where destructive setup is
safe. Without that opt-in, kondo can't know whether sibling deftests will
be `^:parallel`, so the conservative default is to flag.
Sister to [[deftest-check-parallel]]: the deftest hook rejects `^:parallel`
tests inside a `^:synchronous` ns, so the two checks together form a
coherent opt-in: either the whole ns is synchronous (fixtures can be
tests inside a `^:synchronized` ns, so the two checks together form a
coherent opt-in: either the whole ns is explicitly non-parallel (fixtures can be
anything; no parallel tests allowed) or it isn't (fixtures must be
parallel-safe)."
[{:keys [node config], ns-sym :ns, :as input}]
(when-not (:synchronous (meta ns-sym))
(when-not (or (true? (:synchronized (meta ns-sym)))
(and (contains? (meta ns-sym) :parallel)
(false? (:parallel (meta ns-sym)))))
(let [linter-config (get-in config [:linters :metabase/validate-deftest])]
(doseq [form (rest (:children node))]
(warn-about-disallowed-parallel-forms form linter-config))))

View File

@@ -71,7 +71,7 @@
(let [module (module ns-symb)
module-api-namespaces (module-api-namespaces config module)
module-friends (module-friends config module)]
(or (empty? module-api-namespaces)
(or (nil? module-api-namespaces)
(contains? module-api-namespaces ns-symb)
(contains? module-friends current-module))))

View File

@@ -1,14 +0,0 @@
(ns hooks.metabase.test.gentest
(:require
[clj-kondo.hooks-api :as hooks]))
(defn iterate*
"(iterate {:as limit-spec} [s1 val1 & more] body-form body-form...)"
[{{:keys [children] :as node} :node :as orig}]
(let [[token limit-spec bindings & body-nodes] children
node* (-> (hooks/list-node
(list token limit-spec
(hooks/list-node (list* (hooks/token-node 'let) bindings
body-nodes))))
(with-meta (meta node)))]
(assoc orig :node node*)))

View File

@@ -19,6 +19,19 @@
(defn- lint-modules [form config]
(lint-ns form (assoc-in config [:linters :metabase/modules :level] :warning)))
(deftest ^:parallel explicit-nonparallel-namespace-metadata-test
(let [config {:linters {:metabase/validate-deftest {:level :warning}}}]
(testing "Hawk's supported namespace marker passes"
(is (empty? (lint-ns '(ns ^:synchronized example.test) config))))
(testing "the verbose Hawk spelling is not accepted as Metabase style"
(is (=? [{:type :metabase/validate-deftest
:message #(re-find #"Use `\^:synchronized` instead" %)}]
(lint-ns '(ns ^{:parallel false} example.test) config))))
(testing "the legacy marker explains that Hawk ignores it"
(is (=? [{:type :metabase/validate-deftest
:message #(re-find #"`\^:synchronous` is ignored by Hawk" %)}]
(lint-ns '(ns ^:synchronous example.test) config))))))
(deftest ^:parallel module-checker-allowed-modules-test
(are [task-namespace] (=? [{:message "Module task should not be used in the api module. [:metabase/modules api :uses]"
:type :metabase/modules}]
@@ -43,6 +56,16 @@
'{:metabase/modules {api {:uses #{search}}
search {:api #{metabase.search.core}}}}))))
(deftest ^:parallel module-checker-empty-api-test
(is (=? [{:message "Namespace metabase.embeddings.core is not an allowed external API namespace for the embeddings module. [:metabase/modules embeddings :api]"
:type :metabase/modules}]
(lint-modules
'(ns metabase.metabot.core
(:require
[metabase.embeddings.core :as embeddings]))
'{:metabase/modules {metabot {:uses #{embeddings}}
embeddings {:api #{}}}}))))
(deftest ^:parallel module-checker-friends-test
(is (= []
(lint-modules

View File

@@ -21,6 +21,54 @@
#{:athena}}}}})
(mapv :message @(:findings clj-kondo.impl.utils/*ctx*))))
(defn- parallel-metadata-findings
([form]
(parallel-metadata-findings form 'example.test))
([form ns-sym]
(let [linters {:metabase/deftest-not-marked-parallel-or-synchronized {:level :warning}
:metabase/validate-deftest {:level :warning}}]
(binding [clj-kondo.impl.utils/*ctx* {:config {:linters linters}
:ignores (atom nil)
:findings (atom [])
:namespaces (atom {})}]
(hooks.clojure.test/deftest {:node (api/parse-string form)
:lang :clj
:ns ns-sym
:config {:linters {:metabase/validate-deftest {}}}})
@(:findings clj-kondo.impl.utils/*ctx*)))))
(deftest ^:parallel explicit-nonparallel-metadata-test
(testing "Hawk's supported test markers satisfy the explicit metadata linter"
(are [metadata] (empty? (parallel-metadata-findings
(format "(deftest %s example-test (is true))" metadata)))
"^:parallel"
"^:synchronized"))
(testing "an unmarked test still gets the opt-in linter's guidance"
(is (=? [{:type :metabase/deftest-not-marked-parallel-or-synchronized
:message "Test should be marked either `^:parallel` or `^:synchronized`"}]
(parallel-metadata-findings "(deftest example-test (is true))"))))
(testing "the unsupported marker explains that Hawk ignores it"
(is (some #(and (= (:type %) :metabase/validate-deftest)
(re-find #"ignored by Hawk" (:message %)))
(parallel-metadata-findings
"(deftest ^:sequential example-test (is true))"))))
(testing "the verbose Hawk spelling is not accepted as Metabase style"
(is (some #(and (= (:type %) :metabase/validate-deftest)
(re-find #"Use `\^:synchronized` instead" (:message %)))
(parallel-metadata-findings
"(deftest ^{:parallel false} example-test (is true))"))))
(testing "a test cannot request both execution modes"
(is (some #(and (= (:type %) :metabase/validate-deftest)
(re-find #"both `\^:parallel` and `\^:synchronized`" (:message %)))
(parallel-metadata-findings
"(deftest ^:parallel ^:synchronized example-test (is true))"))))
(testing "a namespace-level non-parallel promise rejects parallel tests"
(is (=? [{:type :metabase/validate-deftest
:message #(re-find #"namespace marked.*`\^:synchronized`" %)}]
(parallel-metadata-findings
"(deftest ^:parallel example-test (is true))"
(with-meta 'example.test {:synchronized true}))))))
(deftest ^:parallel disallow-hardcoded-driver-names-in-tests-test
(is (= []
(deftest-warnings

View File

@@ -12,6 +12,7 @@
"enterprise/backend"
"bin/build"
"bin/lint-migrations-file"
"bin/load-namespaces"
"bin/release-list"
#_"mage/src"
#_"mage/test"

2
.github/CODEOWNERS vendored
View File

@@ -42,6 +42,8 @@ docs/api.json # auto-generated
docs/configuring-metabase/environment-variables.md # auto-generated
docs/configuring-metabase/config-template.md # auto-generated
docs/installation-and-operation/commands.md # auto-generated
docs/embedding/sdk/api # auto-generated
docs/embedding/eajs/snippets # auto-generated
# @metabase/devexp
.github @metabase/devexp

View File

@@ -3,3 +3,11 @@ self-hosted-runner:
# selected through `vars.DEFAULT_RUNNER_KEY` and is not checked.
labels:
- arm-ubuntu-24-2core
paths:
# GitHub added parallel steps (`parallel`, `background`, `wait`, `wait-all`,
# `cancel`) in June 2026, and actionlint 1.7.12 does not know the schema yet:
# https://github.blog/changelog/2026-06-25-actions-steps-can-now-be-run-in-parallel/
.github/workflows/lint.yml:
ignore:
- 'step must run script with "run" section or run action with "uses" section'

View File

@@ -3,17 +3,15 @@ description: |
Measures app + embedding SDK bundle sizes from the current and base-ref
uberjars (reusing the builds that already ran) plus their bundle-stats
artifacts, and flags gzipped size changes beyond the threshold for the main
app's initial bundle and the chunked SDK's reachable total. Skips when the
base-ref artifacts aren't available rather than rebuilding from source (which
would compare across pipelines).
app's initial bundle and the chunked SDK's reachable total. The base ref is
read back off the current uberjar so that both sides differ only by the PR.
Skips when the base-ref artifacts aren't available rather than rebuilding from
source (which would compare across pipelines).
inputs:
edition:
description: 'Metabase edition (ee or oss)'
required: true
base-ref-commit:
description: 'Commit SHA of the base ref.'
required: true
threshold-percent:
description: 'Percent threshold for bundle size change'
required: true
@@ -41,12 +39,35 @@ runs:
commit: ${{ github.event.pull_request.head.sha || github.sha }}
edition: ${{ inputs.edition }}
extract-path: temp/current-uberjar
- name: Resolve the base ref the current uberjar was built against
id: base-ref
shell: bash
env:
GH_TOKEN: ${{ github.token }}
run: |
set -uo pipefail
# The uberjar job checks out refs/pull/<n>/merge (the PR head already
# merged into the base branch tip) but names its artifact after head.sha,
# so the comparable base ref is that merge's first parent. Reading it back
# off the jar describes the artifact actually being measured. Using
# pull_request.base.sha instead compares against a stale snapshot of the
# base branch, reporting everything that landed in between as this PR's
# own bundle delta.
unzip -o -j temp/current-uberjar/metabase.jar version.properties -d temp/current-version >/dev/null
built=$(sed -n 's/^hash=//p' temp/current-version/version.properties)
base=$(gh api "/repos/${GITHUB_REPOSITORY}/commits/${built}" \
--jq 'if (.parents | length) == 2 then .parents[0].sha else empty end' 2>/dev/null) || base=""
echo "sha=${base}" >> "$GITHUB_OUTPUT"
echo "The current uberjar was built from ${built}; its base ref is ${base:-unknown}"
- name: Download base-ref uberjar
id: base-jar
if: steps.base-ref.outputs.sha != ''
continue-on-error: true
uses: ./.github/actions/fetch-artifact
with:
commit: ${{ inputs.base-ref-commit }}
commit: ${{ steps.base-ref.outputs.sha }}
edition: ${{ inputs.edition }}
extract-path: temp/base-uberjar
- name: Measure and compare bundle sizes
@@ -56,7 +77,7 @@ runs:
GH_TOKEN: ${{ github.token }}
EDITION: ${{ inputs.edition }}
HEAD_SHA: ${{ github.event.pull_request.head.sha || github.sha }}
BASE_SHA: ${{ inputs.base-ref-commit }}
BASE_SHA: ${{ steps.base-ref.outputs.sha }}
THRESHOLD: ${{ inputs.threshold-percent }}
MEASURE_BUNDLES: app,embedding-sdk-legacy,embedding-sdk-chunked
# The gated bundles, as "<bundle>/<kind>". Each one's step outputs are
@@ -79,6 +100,7 @@ runs:
# The base ref needs an already-built uberjar. Rebuilding it from source
# would compare across two build pipelines (the bug this check used to
# have), so skip instead when it — or its stats — isn't available.
[ -n "${BASE_SHA}" ] || skip "no comparable base ref for the current uberjar"
[ -f temp/base-uberjar/metabase.jar ] || skip "base-ref uberjar not found for ${BASE_SHA}"
extract_js() { # $1=jar $2=dest-root

View File

@@ -0,0 +1,53 @@
name: Collect profile
description: >-
Attach a flamegraph and per-test wall times to a job whose tests ran under clj-async-profiler.
Does nothing when the run was not profiled.
inputs:
profiled:
required: true
description: >-
"true" when the test step ran under the profiler. Passed in rather than read from the
environment so the decision stays recorded by the step that made it.
artifact-name:
required: true
description: Name for the uploaded artifact. Must be unique across the jobs of one run.
runs:
using: "composite"
steps:
# Unlike a JUnit failure report, these run on a green build too — a suite that passes slowly is
# exactly what profiling is for.
- name: Summarize test wall times
if: ${{ inputs.profiled == 'true' }}
continue-on-error: true
shell: bash
run: |
{
echo "### Slowest test namespaces (seconds)"
echo '```'
grep -h '<testsuite' target/junit/*.xml 2>/dev/null \
| sed -E 's|.*<testsuite name="([^"]*)" time="([^"]*)".*|\2 \1|' \
| sort -rn | head -25
echo '```'
echo "### Slowest tests (seconds)"
echo '```'
grep -h '<testcase' target/junit/*.xml 2>/dev/null \
| sed -E 's|.*classname="([^"]*)" name="([^"]*)" time="([^"]*)".*|\3 \1/\2|' \
| sort -rn | head -25
echo '```'
} >> "$GITHUB_STEP_SUMMARY"
# Only the rendered flamegraph is kept: the raw collapsed stacks it was built from run to
# hundreds of MB on a long suite, and are reproducible locally from the same test selection.
- name: Upload profile
if: ${{ inputs.profiled == 'true' }}
continue-on-error: true
uses: actions/upload-artifact@v7
with:
name: ${{ inputs.artifact-name }}
path: |
target/clj-async-profiler/results/*flamegraph.html
target/junit/*.xml
retention-days: 14
if-no-files-found: warn

View File

@@ -1,45 +0,0 @@
name: Get cache keys
description: |
Get deterministic cache keys for Maven and Node.js caches based on the centrally controlled cache prefix,
and the suffix based on the runner OS and the current week of the year.
outputs:
m2-cache-key:
description: "The cache key for the Maven cache."
value: ${{ steps.keys-factory.outputs.m2-cache-key }}
m2-restore-key:
description: "The fallback restore key prefix for the Maven cache."
value: ${{ steps.keys-factory.outputs.m2-restore-key }}
node-cache-key:
description: "The cache key for the Node.js cache."
value: ${{ steps.keys-factory.outputs.node-cache-key }}
node-restore-key:
description: "The fallback restore key prefix for the Node.js cache."
value: ${{ steps.keys-factory.outputs.node-restore-key }}
eslint-cache-key:
description: "The cache key for Eslint cache"
value: ${{ steps.keys-factory.outputs.eslint-cache-key }}
cypress-cache-key:
description: "The cache key for Cypress cache"
value: ${{ steps.keys-factory.outputs.cypress-cache-key }}
runs:
using: "composite"
steps:
- name: Generate and output cache keys
id: keys-factory
run: |
M2_CACHE_PREFIX='M2'
NODE_CACHE_PREFIX='node-modules'
ESLINT_CACHE_PREFIX='eslint'
CYPRESS_CACHE_PREFIX='cypress'
WEEK_OF_YEAR=$(/bin/date "+%G-%V")
CACHE_SUFFIX="${{ runner.os }}-$WEEK_OF_YEAR"
echo "m2-cache-key=$M2_CACHE_PREFIX-$CACHE_SUFFIX" >> "$GITHUB_OUTPUT"
echo "m2-restore-key=$M2_CACHE_PREFIX-" >> "$GITHUB_OUTPUT"
echo "node-cache-key=$NODE_CACHE_PREFIX-$CACHE_SUFFIX" >> "$GITHUB_OUTPUT"
echo "node-restore-key=$NODE_CACHE_PREFIX-" >> "$GITHUB_OUTPUT"
echo "eslint-cache-key=$ESLINT_CACHE_PREFIX-$CACHE_SUFFIX" >> "$GITHUB_OUTPUT"
echo "cypress-cache-key=$CYPRESS_CACHE_PREFIX-$CACHE_SUFFIX" >> "$GITHUB_OUTPUT"
shell: bash

View File

@@ -32,10 +32,6 @@ runs:
exit 1
fi
- name: Get M2 cache key
uses: ./.github/actions/get-cache-keys
id: get-cache-keys
- name: Install uv (for Python package installation)
uses: astral-sh/setup-uv@08807647e7069bb48b6ef5acd8ec9567f424441b # v8.1.0
with:
@@ -43,12 +39,14 @@ runs:
version: "0.10.2" # keep in sync with mise.toml
github-token: ${{ github.token }}
- name: Cache spec
id: cache-spec
shell: bash
run: .github/scripts/cache-keys.sh | tee -a "$GITHUB_OUTPUT"
- name: Restore M2 cache
uses: actions/cache/restore@v5
with:
path: |
~/.m2
~/.gitlibs
key: ${{ steps.get-cache-keys.outputs.m2-cache-key }}
restore-keys: |
${{ steps.get-cache-keys.outputs.m2-restore-key }}
path: ${{ steps.cache-spec.outputs.m2-path }}
key: ${{ steps.cache-spec.outputs.m2-key }}
restore-keys: ${{ steps.cache-spec.outputs.m2-restore-key }}

View File

@@ -25,15 +25,17 @@ runs:
chrome-version: 144
id: setup-chrome
- name: Get cache keys
uses: ./.github/actions/get-cache-keys
id: get-cache-keys
- name: Cache spec
id: cache-spec
shell: bash
run: .github/scripts/cache-keys.sh | tee -a "$GITHUB_OUTPUT"
- name: Get Cypress cache
- name: Restore Cypress binary
uses: actions/cache/restore@v5
with:
path: ~/.cache/Cypress
key: ${{ steps.get-cache-keys.outputs.cypress-cache-key }}
path: ${{ steps.cache-spec.outputs.cypress-path }}
key: ${{ steps.cache-spec.outputs.cypress-key }}
restore-keys: ${{ steps.cache-spec.outputs.cypress-restore-key }}
- name: Ensure that Cypress executable is ready
run: |

View File

@@ -16,43 +16,37 @@ runs:
with:
node-version: ${{ inputs.node-version }}
- name: Get cache keys
uses: ./.github/actions/get-cache-keys
id: get-cache-keys
- name: Restore M2 cache
uses: actions/cache/restore@v5
with:
path: |
~/.m2
~/.gitlibs
key: ${{ steps.get-cache-keys.outputs.m2-cache-key }}
restore-keys: |
${{ steps.get-cache-keys.outputs.m2-restore-key }}
- name: Get bun cache directory
id: bun-cache
run: |
BUN_CACHE_DIR=$(bun pm cache)
echo "Bun cache directory: $BUN_CACHE_DIR"
echo "dir=$BUN_CACHE_DIR" >> "$GITHUB_OUTPUT"
- name: Cache spec
id: cache-spec
shell: bash
run: .github/scripts/cache-keys.sh | tee -a "$GITHUB_OUTPUT"
- name: Restore node_modules cache
# The spec names the store path rather than discovering it, so that a job can restore the cache
# before bun exists. Exported job-wide rather than per-step because every later bun invocation has
# to resolve against the same store. Confirm bun agreed: if the variable stopped taking effect, bun
# would fill its default directory and the cache would silently save an empty tree.
- name: Point bun at the cached store
shell: bash
env:
STORE: ${{ steps.cache-spec.outputs.bun-store-path }}
run: |
echo "BUN_INSTALL_CACHE_DIR=$STORE" >> "$GITHUB_ENV"
ACTUAL=$(BUN_INSTALL_CACHE_DIR="$STORE" bun pm cache)
if [ "$ACTUAL" != "$STORE" ]; then
echo "::error::bun reports its store at $ACTUAL, not $STORE; BUN_INSTALL_CACHE_DIR is not taking effect"
exit 1
fi
- name: Restore bun store
uses: actions/cache/restore@v5
id: node-modules-cache
with:
path: |
${{ steps.bun-cache.outputs.dir }}
node_modules
key: ${{ steps.get-cache-keys.outputs.node-cache-key }}
restore-keys: |
${{ steps.get-cache-keys.outputs.node-restore-key }}
path: ${{ steps.cache-spec.outputs.bun-store-path }}
key: ${{ steps.cache-spec.outputs.bun-store-key }}
restore-keys: ${{ steps.cache-spec.outputs.bun-store-restore-key }}
# postinstall lifecycle hook can be dangerous in CI so we only run it locally (see package.json)
# If a job wants to patch packages, it does so explicitly in the CI. This setting is opt-out.
# For example, we never want to cache node_modules with patches already applied so cache-generator.yml skips that step.
- name: Clean install
- name: Install front-end dependencies
run: bun install --frozen-lockfile
shell: bash
- name: Patch frontend dependencies

View File

@@ -25,6 +25,7 @@ runs:
TARGET_BRANCH: ${{ github.event.pull_request.base.ref || github.ref_name }}
CI_CONDUCTOR_TEST_SUITE: ${{ inputs.test-suite }}
ADAPTER: ${{ inputs.adapter }}
IS_RERUN: ${{ github.run_attempt != '1' }}
# Keep the path constrained to known entrypoints.
run: |
case "$ADAPTER" in

View File

@@ -101,7 +101,21 @@ runs:
# vector reaches clojure -X as a single argument.
run: |
BASE="clojure -X:dev:ci:${{ inputs.edition }}:${{ inputs.edition }}-dev:drivers:drivers-dev:test"
if [ "${{ inputs.granular-rerun }}" = "true" ] && [ "${{ github.run_attempt }}" != "1" ] && [ -s target/failed-tests ]; then
# `:profile-tests` overrides `:test`'s exec-fn, so it has to come last. See dev.profile-tests.
# The `profiled` output is what the collection steps below key off, so that the decision is
# recorded by the step that acts on it rather than re-derived from the environment.
if [ "$PROFILE_TESTS" = "true" ]; then
echo "Profiling this run; a flamegraph will be attached to the job."
echo "profiled=true" >> "$GITHUB_OUTPUT"
BASE="$BASE:profile-tests"
fi
# An explicit selection wins over the rerun narrowing below: it was asked for by hand, on this
# attempt. Names may be separated by commas or spaces; hawk wants them as one vector argument.
if [ -n "$ONLY_TESTS" ]; then
ONLY="[$(echo "$ONLY_TESTS" | tr ',' ' ')]"
echo "Running only: $ONLY"
$BASE :only "$ONLY"
elif [ "${{ inputs.granular-rerun }}" = "true" ] && [ "${{ github.run_attempt }}" != "1" ] && [ -s target/failed-tests ]; then
echo "Granular rerun of previously-failed tests: $(cat target/failed-tests)"
$BASE :only "$(cat target/failed-tests)"
else
@@ -113,8 +127,15 @@ runs:
# ultimately fails.
continue-on-error: true
- name: Collect profile
if: ${{ !cancelled() }}
uses: ./.github/actions/collect-profile
with:
profiled: ${{ steps.run-test.outputs.profiled }}
artifact-name: ${{ format('profile-{0}{1}', inputs.junit-name, inputs.artifact-suffix && format('-{0}', inputs.artifact-suffix) || '') }}
- name: Report failed tests to ci-conductor
if: ${{ !cancelled() && steps.run-test.outcome == 'failure' }}
if: ${{ !cancelled() && ( steps.run-test.outcome == 'failure' || github.run_attempt != '1' ) }}
# This step should NEVER fail the job!
continue-on-error: true
uses: ./.github/actions/report-failures-to-ci-conductor

View File

@@ -1,41 +0,0 @@
name: Update cache
description: |
Update a GitHub Actions cache based on a provided path and key.
If a cache entry already exists for the given key, it is first removed.
This ensures immutability of cache entries, and prevents runtime errors.
inputs:
path:
description: "The path(s) to cache, separated by newlines."
required: true
key:
description: "The cache key to use."
required: true
runs:
using: "composite"
steps:
- name: Check if the cache entry exists
id: cache-lookup
uses: actions/cache/restore@v5
with:
path: ${{ inputs.path }}
key: ${{ inputs.key }}
lookup-only: true # Important bit: the action will not fetch anything, just check for the cache existence.
- name: Delete the cache entry if it exists
if: ${{ steps.cache-lookup.outputs.cache-hit == 'true' }}
run: |
echo "Cache entry found for key '${{ inputs.key }}'. Deleting it to ensure immutability."
gh api \
--method DELETE \
-H "Accept: application/vnd.github+json" \
-H "X-GitHub-Api-Version: 2022-11-28" \
/repos/${{ github.repository }}/actions/caches?key=${{ inputs.key }}
env:
GH_TOKEN: ${{ github.token }}
shell: bash
- name: Save the new cache entry
uses: actions/cache/save@v5
with:
path: ${{ inputs.path }}
key: ${{ inputs.key }}

View File

@@ -1,40 +0,0 @@
name: Upload Test Results to s3
description: Uploads test results files to s3
inputs:
input-path:
required: true
output-name:
required: true
default: test
role-to-assume:
required: true
aws-region:
required: true
default: us-east-1
bucket:
required: true
runs:
using: "composite"
steps:
- name: Configure AWS credentials
uses: aws-actions/configure-aws-credentials@d979d5b3a71173a29b74b5b88418bfda9437d885
with:
role-to-assume: ${{ inputs.role-to-assume }}
aws-region: ${{ inputs.aws-region }}
- name: zip test results
env:
INPUT_DIR: ${{ inputs.input-path }}
OUTPUT_FILE: ${{ inputs.output-name }}
shell: bash
run: | # sh
zip ${OUTPUT_FILE}.zip $INPUT_DIR/*.xml
- name: Upload test results to S3
env:
BUCKET: ${{ inputs.bucket }}
OUTPUT_FILE: ${{ inputs.output-name }}
shell: bash
run: | # sh
DATE=$(date '+%Y-%m-%d')
aws s3 cp ${OUTPUT_FILE}.zip s3://$BUCKET/$DATE/$GITHUB_RUN_ID/$GITHUB_RUN_ATTEMPT/

View File

@@ -1,22 +0,0 @@
name: 'Verify Organization Membership'
description: 'Check if a user is a member of a GitHub organization'
inputs:
github_token:
description: 'Token with access to the organization'
required: true
organization_name:
description: 'Organization name'
required: true
username:
description: 'Username'
required: true
outputs:
is_member:
value: ${{ steps.verify-membership.outputs.is_member }}
description: "Is the user a member of the organization"
runs:
using: "composite"
steps:
- run: bash ${{ github.action_path }}/verify_membership.sh ${{ inputs.github_token }} ${{ inputs.organization_name }} ${{ inputs.username }}
id: verify-membership
shell: bash

View File

@@ -1,25 +0,0 @@
#!/bin/bash
TOKEN=$1
ORG_NAME=$2
USERNAME=$3
response=$(curl -L -o /dev/null -w "%{http_code}" \
-H "Accept: application/vnd.github+json" \
-H "Authorization: Bearer $TOKEN" \
-H "X-GitHub-Api-Version: 2022-11-28" \
"https://api.github.com/orgs/$ORG_NAME/members/$USERNAME")
# Github returns an empty response with 204 status if user is a member and 404 status otherwise
# See: https://docs.github.com/en/rest/orgs/members?apiVersion=2022-11-28#check-organization-membership-for-a-user
if [ "$response" == "204" ]; then
echo "The user $USERNAME is a member of $ORG_NAME"
echo "is_member=true" >> $GITHUB_OUTPUT
elif [ "$response" == "404" ]; then
echo "The user $USERNAME is not a member of $ORG_NAME"
echo "is_member=false" >> $GITHUB_OUTPUT
else
echo "Failed to determine the membership status for $USERNAME in $ORG_NAME"
echo "is_member=false" >> $GITHUB_OUTPUT
exit 1
fi

View File

@@ -106,13 +106,6 @@ backend_ci: &backend_ci
- ".github/workflows/backend.yml"
- ".github/workflows/drivers.yml"
backend_presto_kerberos:
- "**/presto_jdbc/**"
- "**/presto-jdbc/**" # the driver module has a hyphen. underscores are classpaths
- "**/presto_jdbc.clj"
- "**/presto_jdbc_test.clj"
- ".github/workflows/presto-kerberos-integration-test.yml"
backend_sources: &backend_sources
- *shared_sources
- "**deps.edn"
@@ -183,7 +176,7 @@ embedding_sdk_host_sample_apps:
- "e2e/test-host-app/**/*.cy.*.{js,ts}"
snowplow: &snowplow
- ".github/workflows/snowplow.yml"
- ".github/workflows/lint.yml"
- "snowplow/**"
- "bin/lint-snowplow-schemas.bb"
- "bin/lint-snowplow-schemas-test.bb"
@@ -212,7 +205,7 @@ yaml:
- "**/*.yaml"
codeql:
- ".github/workflows/codeql.yml"
- ".github/workflows/lint.yml"
- "frontend/src/**"
- "enterprise/frontend/src/**"
@@ -220,26 +213,43 @@ i18n:
- *default
- ".github/actions/prepare-frontend/**"
- ".github/actions/prepare-backend/**"
- ".github/workflows/i18n.yml"
- ".github/workflows/lint.yml"
- *sources
visualizations:
- "frontend/src/metabase/visualizations/**"
- "frontend/src/metabase/static-viz/**"
- "src/metabase/pulse/**"
cljs:
- *shared_sources
- *shared_specs
- ".github/workflows/cljs.yml"
- "shadow-cljs.edn"
embedder_plugin:
- ".github/actions/prepare-backend/**"
- ".github/file-paths.yaml"
- ".github/workflows/build-scripts.yml"
- "bin/build-embedder-plugin.sh"
- "bin/build/src/build/embedder_model.clj"
- "bin/build/src/build/plugin_uberjar.clj"
- "bin/build/src/build_embedder_plugin.clj"
- "deps.edn"
- "modules/embedder/**"
- "enterprise/backend/src/metabase_enterprise/data_complexity_score/complexity_embedders.clj"
- "enterprise/backend/src/metabase_enterprise/data_complexity_score/settings.clj"
- "enterprise/backend/src/metabase_enterprise/data_complexity_score/synonym_source.clj"
- "enterprise/backend/src/metabase_enterprise/semantic_search/embedding.clj"
- "enterprise/backend/src/metabase_enterprise/semantic_search/settings.clj"
- "enterprise/backend/test/metabase_enterprise/semantic_search/appdb_pgvector_mode_test.clj"
- "enterprise/backend/test/metabase_enterprise/semantic_search/embedding_test.clj"
- "src/metabase/embeddings/provider.clj"
- "src/metabase/embeddings/startup.clj"
- "src/metabase/plugins/**"
- "test/metabase/embeddings/embedder_plugin_artifact_test.clj"
serialization_baseline:
- ".github/workflows/serialization.yml"
- "test_resources/serialization_baseline/**"
openapi:
- ".github/workflows/openapi.yml"
- ".github/workflows/lint.yml"
- "src/metabase/api/**"
- "src/metabase/api_routes/**"
- "src/metabase/*_rest/api.clj"

View File

@@ -0,0 +1,12 @@
---
applyTo: "e2e/**/*.cy.spec.{js,ts}"
---
# Cypress E2E Spec Review
When reviewing Cypress E2E specs, apply the guidelines in these files:
- `.claude/skills/_shared/cypress-conventions.md`
- `.claude/skills/e2e-test-review/SKILL.md`
These are in addition to the TypeScript/JavaScript instructions, which also match these files.

View File

@@ -1,152 +0,0 @@
/* eslint-disable import/no-commonjs */
// Prepares the rows the bundle-size stats logger uploads to stats.metabase.com.
//
// For each measured (bundle, kind) it computes the delta — raw, gzipped and
// brotli (as-served) bytes plus percent — against the previously *plotted* data
// point (restored from a rolling cache), so the deltas can be charted directly.
// It also decides whether this commit is worth recording at all: a point is kept
// when the as-served size moved by at least MIN_DELTA_PERCENT, or when a new
// bundle/kind appears. Only master builds run through here — releases form
// non-linear lines whose deltas are meaningless, so they're not recorded (their
// historical sizes are backfilled separately).
//
// buildStatsRows is pure (data in, data out) so it can be unit-tested; main()
// is the thin I/O wrapper that reads env/files and writes the rows, cache and
// step outputs.
const fs = require("fs");
const path = require("path");
const deltaPercent = (current, base) =>
base ? Math.round(((current - base) * 10000) / base) / 100 : "";
/**
* Turn the current measurements + the last plotted point into upload rows, the
* slim cache rows for the next diff, and the keep/skip decision.
*
* The significance threshold tracks the as-served size: brotli where the build
* ships a precompressed .br, otherwise gzip. Current measurements always carry
* brotliBytes (estimated when no .br shipped), but a cached base point recorded
* before brotli logging may not, so the delta still falls back to gzip per row.
*/
function buildStatsRows({ measurements, previous, threshold, date, commit, commitMessage, version }) {
const previousOf = (bundle, kind) => previous.find(row => row.bundle === bundle && row.kind === kind);
let maxServedDeltaPercent = 0;
let hasNewSeries = false;
const rows = measurements.map(measurement => {
const base = previousOf(measurement.bundle, measurement.kind);
const hasBrotli = measurement.brotliBytes != null && base?.brotliBytes != null;
const gzipDeltaPercent = deltaPercent(measurement.gzipBytes, base?.gzipBytes);
const brotliDeltaPercent = hasBrotli ? deltaPercent(measurement.brotliBytes, base.brotliBytes) : "";
if (!base) {
hasNewSeries = true;
} else {
const servedDeltaPercent = hasBrotli ? brotliDeltaPercent : gzipDeltaPercent;
maxServedDeltaPercent = Math.max(maxServedDeltaPercent, Math.abs(servedDeltaPercent));
}
return {
"Date": date,
"Version": version,
"Commit": commit,
"Bundle": measurement.bundle,
"Kind": measurement.kind, // "initial" or "total"
// The stats table carries a free-text Description column; append-csv
// requires every table column to be present. Populate it with the commit
// subject so the chart's points are self-describing.
"Description": commitMessage,
"Raw bytes": measurement.rawBytes,
"Gzip bytes": measurement.gzipBytes,
"Brotli bytes": measurement.brotliBytes, // as-served (shipped .br, else estimated)
"File count": measurement.fileCount,
"Raw bytes delta": base ? measurement.rawBytes - base.rawBytes : "",
"Gzip bytes delta": base ? measurement.gzipBytes - base.gzipBytes : "",
"Brotli bytes delta": hasBrotli ? measurement.brotliBytes - base.brotliBytes : "",
"Raw delta %": deltaPercent(measurement.rawBytes, base?.rawBytes),
"Gzip delta %": gzipDeltaPercent,
"Brotli delta %": brotliDeltaPercent,
};
});
const firstPoint = previous.length === 0;
const significant = firstPoint || hasNewSeries || maxServedDeltaPercent >= threshold;
// Slim rows the next run diffs against. The workflow only persists this to the
// cache when we actually push, so the cached reference always stays the last
// *plotted* point (cumulative drift is caught).
const cacheRows = measurements.map(({ bundle, kind, rawBytes, gzipBytes, brotliBytes }) => ({
bundle,
kind,
rawBytes,
gzipBytes,
brotliBytes,
}));
const reason = firstPoint
? "first point"
: hasNewSeries
? "new bundle/kind series"
: `max served Δ ${maxServedDeltaPercent.toFixed(2)}% (threshold ${threshold}%)`;
return { rows, cacheRows, significant, reason, maxServedDeltaPercent, firstPoint, hasNewSeries };
}
module.exports = { buildStatsRows };
const readJson = filePath => (filePath && fs.existsSync(filePath) ? JSON.parse(fs.readFileSync(filePath, "utf8")) : null);
const writeJson = (filePath, value) => {
fs.mkdirSync(path.dirname(filePath), { recursive: true });
fs.writeFileSync(filePath, JSON.stringify(value));
};
const setOutput = (env, name, value) => {
if (env.GITHUB_OUTPUT) {
fs.appendFileSync(env.GITHUB_OUTPUT, `${name}=${value}\n`);
}
};
// version.properties (extracted from the uberjar) carries the build's tag.
// Non-release (master) builds report tag=vUNKNOWN, so leave Version empty there.
function readVersion(versionPropsPath) {
const raw =
versionPropsPath && fs.existsSync(versionPropsPath)
? (fs.readFileSync(versionPropsPath, "utf8").match(/^tag=(.*)$/m)?.[1]?.trim() ?? "")
: "";
return raw === "vUNKNOWN" ? "" : raw;
}
function main() {
const env = process.env;
const measurements = readJson(env.CURRENT);
if (!Array.isArray(measurements) || measurements.length === 0) {
console.error(`::error::No bundle measurements found at ${env.CURRENT}`);
process.exit(1);
}
const result = buildStatsRows({
measurements,
// The last plotted point, as slim rows: [{ bundle, kind, rawBytes, gzipBytes, brotliBytes }].
previous: readJson(env.LAST) || [],
threshold: Number(env.MIN_DELTA_PERCENT ?? 1),
date: new Date().toISOString().slice(0, 10), // YYYY-MM-DD
commit: (env.HEAD_SHA || "").slice(0, 12),
commitMessage: (env.COMMIT_MESSAGE || "").split("\n")[0], // subject line only
version: readVersion(env.VERSION_PROPS),
});
writeJson(env.ROWS_OUT, result.rows);
writeJson(env.CACHE_OUT, result.cacheRows);
console.log(`${result.significant ? "RECORD" : "SKIP"} — ${result.reason}`);
setOutput(env, "significant", result.significant ? "true" : "false");
setOutput(
env,
"max_delta_percent",
result.firstPoint || result.hasNewSeries ? "" : result.maxServedDeltaPercent.toFixed(2),
);
}
if (require.main === module) {
main();
}

View File

@@ -0,0 +1,225 @@
// Prepares the rows the bundle-size stats logger imports into eng-stats-importer.
//
// For each measured (bundle, kind) it computes the delta — raw, gzipped and
// brotli (as-served) bytes plus percent — against the previously *plotted* data
// point (restored from a rolling cache), so the deltas can be charted directly.
// It also decides whether this commit is worth recording at all: a point is kept
// when the as-served size moved by at least MIN_DELTA_PERCENT, or when a new
// bundle/kind appears. Only master builds run through here — releases form
// non-linear lines whose deltas are meaningless, so they're not recorded (their
// historical sizes are backfilled separately).
//
// buildStatsRows is pure (data in, data out) so it can be unit-tested; main()
// is the thin I/O wrapper that reads env/files and writes the rows, cache and
// step outputs.
import {
appendFileSync,
existsSync,
mkdirSync,
readFileSync,
writeFileSync,
} from "node:fs";
import { dirname } from "node:path";
/** One measurement emitted by measure-bundle-sizes. */
export interface Measurement {
bundle: string;
kind: "initial" | "total";
rawBytes: number;
gzipBytes: number;
brotliBytes: number;
fileCount: number;
}
/**
* The slim row cached between runs, holding the last *plotted* point.
* `brotliBytes` is optional because a point cached before brotli logging
* landed does not carry it — see buildStatsRows' per-row gzip fallback.
*/
export type CacheRow = Pick<Measurement, "bundle" | "kind" | "rawBytes" | "gzipBytes"> & {
brotliBytes?: number;
};
export interface BuildStatsRowsInput {
measurements: Measurement[];
previous: CacheRow[];
threshold: number;
date: string;
commit: string;
commitMessage: string;
version: string;
}
/** An upload row, keyed by the importer's column display names. */
export type StatsRow = Record<string, string | number | null>;
export interface BuildStatsRowsResult {
rows: StatsRow[];
cacheRows: CacheRow[];
significant: boolean;
reason: string;
maxServedDeltaPercent: number;
firstPoint: boolean;
hasNewSeries: boolean;
}
// null, not "", for an absent delta: the stats importer coerces values
// strictly and rejects "" for a numeric column.
const deltaPercent = (current: number, base: number | undefined | null): number | null =>
base ? Math.round(((current - base) * 10000) / base) / 100 : null;
/**
* Turn the current measurements + the last plotted point into upload rows, the
* slim cache rows for the next diff, and the keep/skip decision.
*
* The significance threshold tracks the as-served size: brotli where the build
* ships a precompressed .br, otherwise gzip. Current measurements always carry
* brotliBytes (estimated when no .br shipped), but a cached base point recorded
* before brotli logging may not, so the delta still falls back to gzip per row.
*/
export function buildStatsRows({
measurements,
previous,
threshold,
date,
commit,
commitMessage,
version,
}: BuildStatsRowsInput): BuildStatsRowsResult {
const previousOf = (bundle: string, kind: string) =>
previous.find((row) => row.bundle === bundle && row.kind === kind);
let maxServedDeltaPercent = 0;
let hasNewSeries = false;
const rows = measurements.map((measurement): StatsRow => {
const base = previousOf(measurement.bundle, measurement.kind);
// Non-null only when BOTH sides carry brotli: a point cached before brotli
// logging landed has none, and that row falls back to the gzip delta.
const baseBrotli =
measurement.brotliBytes != null && base?.brotliBytes != null ? base.brotliBytes : null;
const gzipDeltaPercent = deltaPercent(measurement.gzipBytes, base?.gzipBytes);
const brotliDeltaPercent =
baseBrotli != null ? deltaPercent(measurement.brotliBytes, baseBrotli) : null;
if (!base) {
hasNewSeries = true;
} else {
const servedDeltaPercent = baseBrotli != null ? brotliDeltaPercent : gzipDeltaPercent;
maxServedDeltaPercent = Math.max(maxServedDeltaPercent, Math.abs(servedDeltaPercent ?? 0));
}
return {
Date: date,
Version: version,
Commit: commit,
Bundle: measurement.bundle,
Kind: measurement.kind, // "initial" or "total"
// The stats table carries a free-text Description column. Populate it
// with the commit subject so the chart's points are self-describing.
Description: commitMessage,
"Raw bytes": measurement.rawBytes,
"Gzip bytes": measurement.gzipBytes,
"Brotli bytes": measurement.brotliBytes, // as-served (shipped .br, else estimated)
"File count": measurement.fileCount,
"Raw bytes delta": base ? measurement.rawBytes - base.rawBytes : null,
"Gzip bytes delta": base ? measurement.gzipBytes - base.gzipBytes : null,
"Brotli bytes delta": baseBrotli != null ? measurement.brotliBytes - baseBrotli : null,
"Raw delta %": deltaPercent(measurement.rawBytes, base?.rawBytes),
"Gzip delta %": gzipDeltaPercent,
"Brotli delta %": brotliDeltaPercent,
};
});
const firstPoint = previous.length === 0;
const significant = firstPoint || hasNewSeries || maxServedDeltaPercent >= threshold;
// Slim rows the next run diffs against. The workflow only persists this to the
// cache when we actually push, so the cached reference always stays the last
// *plotted* point (cumulative drift is caught).
const cacheRows = measurements.map(
({ bundle, kind, rawBytes, gzipBytes, brotliBytes }): CacheRow => ({
bundle,
kind,
rawBytes,
gzipBytes,
brotliBytes,
}),
);
const reason = firstPoint
? "first point"
: hasNewSeries
? "new bundle/kind series"
: `max served Δ ${maxServedDeltaPercent.toFixed(2)}% (threshold ${threshold}%)`;
return {
rows,
cacheRows,
significant,
reason,
maxServedDeltaPercent,
firstPoint,
hasNewSeries,
};
}
const readJson = <T,>(filePath: string | undefined): T | null =>
filePath && existsSync(filePath) ? (JSON.parse(readFileSync(filePath, "utf8")) as T) : null;
const writeJson = (filePath: string, value: unknown) => {
mkdirSync(dirname(filePath), { recursive: true });
writeFileSync(filePath, JSON.stringify(value));
};
const setOutput = (name: string, value: string) => {
if (process.env.GITHUB_OUTPUT) {
appendFileSync(process.env.GITHUB_OUTPUT, `${name}=${value}\n`);
}
};
// version.properties (extracted from the uberjar) carries the build's tag.
// Non-release (master) builds report tag=vUNKNOWN, so leave Version empty there.
function readVersion(versionPropsPath: string | undefined): string {
const raw =
versionPropsPath && existsSync(versionPropsPath)
? (readFileSync(versionPropsPath, "utf8").match(/^tag=(.*)$/m)?.[1]?.trim() ?? "")
: "";
return raw === "vUNKNOWN" ? "" : raw;
}
export function main() {
const env = process.env;
const measurements = readJson<Measurement[]>(env.CURRENT);
if (!Array.isArray(measurements) || measurements.length === 0) {
console.error(`::error::No bundle measurements found at ${env.CURRENT}`);
process.exit(1);
}
const result = buildStatsRows({
measurements,
// The last plotted point, as slim rows: [{ bundle, kind, rawBytes, gzipBytes, brotliBytes }].
previous: readJson<CacheRow[]>(env.LAST) || [],
threshold: Number(env.MIN_DELTA_PERCENT ?? 1),
date: new Date().toISOString().slice(0, 10), // YYYY-MM-DD
commit: (env.HEAD_SHA || "").slice(0, 12),
commitMessage: (env.COMMIT_MESSAGE || "").split("\n")[0], // subject line only
version: readVersion(env.VERSION_PROPS),
});
writeJson(env.ROWS_OUT!, result.rows);
writeJson(env.CACHE_OUT!, result.cacheRows);
console.log(`${result.significant ? "RECORD" : "SKIP"} — ${result.reason}`);
setOutput("significant", result.significant ? "true" : "false");
setOutput(
"max_delta_percent",
result.firstPoint || result.hasNewSeries ? "" : result.maxServedDeltaPercent.toFixed(2),
);
}
// Only run the I/O wrapper when invoked directly, so importing buildStatsRows
// (from the spec) has no side effects.
if ((import.meta as ImportMeta & { main?: boolean }).main) {
main();
}

View File

@@ -1,7 +1,16 @@
const { buildStatsRows } = require("./bundle-size-stats-prepare");
import {
buildStatsRows,
type CacheRow,
type Measurement,
} from "./bundle-size-stats-prepare";
const meta = { date: "2026-06-30", commit: "abc123", commitMessage: "Fix the thing (#123)", version: "" };
const measurement = (over = {}) => ({
const meta = {
date: "2026-06-30",
commit: "abc123",
commitMessage: "Fix the thing (#123)",
version: "",
};
const measurement = (over: Partial<Measurement> = {}): Measurement => ({
bundle: "embedding-sdk-chunked",
kind: "total",
rawBytes: 1000,
@@ -23,8 +32,8 @@ describe("buildStatsRows", () => {
expect(result.firstPoint).toBe(true);
expect(result.significant).toBe(true);
expect(result.reason).toBe("first point");
expect(result.rows[0]["Gzip bytes delta"]).toBe("");
expect(result.rows[0]["Brotli delta %"]).toBe("");
expect(result.rows[0]["Gzip bytes delta"]).toBeNull();
expect(result.rows[0]["Brotli delta %"]).toBeNull();
expect(result.rows[0]["Description"]).toBe("Fix the thing (#123)");
expect(result.cacheRows[0]).toEqual({
bundle: "embedding-sdk-chunked",
@@ -49,7 +58,15 @@ describe("buildStatsRows", () => {
});
it("gates significance on the brotli (as-served) delta when both sides have it", () => {
const previous = [{ bundle: "embedding-sdk-chunked", kind: "total", rawBytes: 1000, gzipBytes: 400, brotliBytes: 300 }];
const previous: CacheRow[] = [
{
bundle: "embedding-sdk-chunked",
kind: "total",
rawBytes: 1000,
gzipBytes: 400,
brotliBytes: 300,
},
];
const big = buildStatsRows({
...meta,
@@ -75,13 +92,14 @@ describe("buildStatsRows", () => {
const result = buildStatsRows({
...meta,
measurements: [measurement({ gzipBytes: 440, brotliBytes: 300 })], // gzip +10%
// No brotliBytes: this point was cached before brotli logging landed.
previous: [{ bundle: "embedding-sdk-chunked", kind: "total", rawBytes: 1000, gzipBytes: 400 }],
threshold: 1,
});
expect(result.maxServedDeltaPercent).toBeCloseTo(10);
expect(result.significant).toBe(true);
expect(result.rows[0]["Brotli bytes delta"]).toBe(""); // no base brotli to diff against
expect(result.rows[0]["Brotli bytes delta"]).toBeNull(); // no base brotli to diff against
expect(result.rows[0]["Gzip delta %"]).toBeCloseTo(10);
});
});

145
.github/scripts/cache-keys.sh vendored Executable file
View File

@@ -0,0 +1,145 @@
#!/usr/bin/env bash
# Single definition of every CI dependency cache: what it holds, what identifies it, what it falls back to.
#
# A GitHub cache entry is identified by its key, a hash of the declared path list, and the ref that saved
# it. A step declaring a different path list than the one a cache was saved with computes a different
# identity and can never match it, whatever its key says. So paths and keys are emitted together here and
# every cache step takes both from this script.
#
# Emits `name=value` lines in $GITHUB_OUTPUT format:
#
# .github/scripts/cache-keys.sh >> "$GITHUB_OUTPUT"
#
# Run it with no redirect to see the whole scheme, or pass one name to print a single value:
#
# .github/scripts/cache-keys.sh
# .github/scripts/cache-keys.sh m2-key
#
# Hashes are computed here rather than with GitHub's hashFiles() so that a local run and a CI run of the
# same commit produce the same keys, and so that an empty match can fail loudly - a hashFiles() call that
# matches nothing silently yields "", collapsing the key onto its own restore prefix and pointing every
# dependency set at one entry.
set -euo pipefail
cd "$(git rev-parse --show-toplevel)"
OS="${RUNNER_OS:-$(uname -s)}"
[ "$OS" = "Darwin" ] && OS="macOS"
sha256() {
if command -v sha256sum >/dev/null 2>&1; then
sha256sum | cut -d' ' -f1
else
shasum -a 256 | cut -d' ' -f1
fi
}
# Digest of a set of files: hash each one, then hash the concatenation, so the result depends on every
# file's contents and on which files exist.
digest() {
local f
if [ "$#" -eq 0 ]; then
echo "::error::cache-keys.sh: no files matched; refusing to emit a key that would collide across dependency sets" >&2
exit 1
fi
for f in "$@"; do
[ -f "$f" ] || { echo "::error::cache-keys.sh: $f does not exist" >&2; exit 1; }
sha256 < "$f"
done | sha256
}
shopt -s nullglob
DEPS_FILES=(deps.edn modules/drivers/deps.edn modules/drivers/*/deps.edn)
shopt -u nullglob
DEPS_HASH="$(digest "${DEPS_FILES[@]}")"
LOCK_HASH="$(digest bun.lock)"
# The Cypress binary depends on the Cypress version alone, so keying it on the whole lockfile would
# throw away a 200MB download every time any frontend dependency moved.
#
# Reading a version out of a lockfile with a regex is only safe while exactly one line matches. Insist
# on that rather than taking the first of several: a second match means the format moved under us, and
# picking one at random would key the binary under a version it is not.
CYPRESS_VERSIONS="$(sed -n 's/.*"cypress": \["cypress@\([^"]*\)".*/\1/p' bun.lock)"
CYPRESS_MATCHES="$(printf '%s\n' "$CYPRESS_VERSIONS" | grep -c . || true)"
if [ "$CYPRESS_MATCHES" != "1" ]; then
echo "::error::cache-keys.sh: expected exactly one resolved cypress version in bun.lock, found $CYPRESS_MATCHES" >&2
exit 1
fi
CYPRESS_VERSION="$CYPRESS_VERSIONS"
# An incremental lint cache is only ever an accelerator, so it keys on the commit and always falls back
# to the newest entry on the prefix. The exact key never pre-exists, which keeps the entry current.
ESLINT_SHA="${GITHUB_SHA:-$(git rev-parse HEAD)}"
emit_multiline() {
local name="$1"; shift
printf '%s<<CACHE_KEYS_EOF\n' "$name"
printf '%s\n' "$@"
printf 'CACHE_KEYS_EOF\n'
}
# Paths under the home directory are emitted absolute rather than as `~/...`. actions/cache would expand
# a tilde, but a shell assigning one of these to a variable would not, and would go on to create a
# directory named `~`. Paths inside the checkout stay relative; actions/cache resolves those against the
# workspace.
spec() {
emit_multiline m2-path "$HOME/.m2" "$HOME/.gitlibs" "$HOME/.deps.clj" 'bin/bb'
echo "m2-key=M2-$OS-$DEPS_HASH"
echo "m2-restore-key=M2-$OS-"
# Pinned rather than discovered via `bun pm cache` so that the path is known before bun is installed;
# prepare-frontend exports BUN_INSTALL_CACHE_DIR to match and fails if bun disagrees.
echo "bun-store-path=$HOME/.bun/install/cache"
echo "bun-store-key=bun-store-$OS-$LOCK_HASH"
echo "bun-store-restore-key=bun-store-$OS-"
echo "cypress-path=$HOME/.cache/Cypress"
echo "cypress-key=cypress-$OS-$CYPRESS_VERSION"
echo "cypress-restore-key=cypress-$OS-"
echo "eslint-path=.eslintcache"
echo "eslint-key=eslint-$OS-$ESLINT_SHA"
echo "eslint-restore-key=eslint-$OS-"
}
# Print one output's value, including the multiline form, and fail on a name that does not exist. A
# caller that silently received "" would go on to build a key with a hole in it.
query() {
local want="$1" line delim found=0 in_block=0
while IFS= read -r line; do
if [ "$in_block" = 1 ]; then
if [ "$line" = "$delim" ]; then
in_block=0
else
printf '%s\n' "$line"
fi
continue
fi
case "$line" in
"$want="*)
printf '%s\n' "${line#*=}"
found=1
;;
"$want<<"*)
delim="${line#*<<}"
in_block=1
found=1
;;
esac
done <<EOF
$(spec)
EOF
if [ "$found" = 0 ]; then
echo "::error::cache-keys.sh: no output named '$want'" >&2
exit 1
fi
}
if [ "$#" -eq 0 ]; then
spec
else
query "$1"
fi

80
.github/scripts/cache-keys.unit.spec.ts vendored Normal file
View File

@@ -0,0 +1,80 @@
import { execFileSync } from "child_process";
import { join } from "path";
const SCRIPT = join(__dirname, "cache-keys.sh");
const CACHES = ["m2", "bun-store", "cypress", "eslint"];
// stderr is inherited by default, so the deliberate-failure case below would print a literal
// `::error::` line - which GitHub Actions reads as an annotation directive, making a passing run
// annotate itself with an error.
const run = (args: string[] = [], env: NodeJS.ProcessEnv = {}, quiet = false) =>
execFileSync(SCRIPT, args, {
encoding: "utf8",
stdio: quiet ? ["ignore", "pipe", "ignore"] : "pipe",
env: { ...process.env, RUNNER_OS: "Linux", GITHUB_SHA: "abc123", ...env },
});
const get = (name: string, env: NodeJS.ProcessEnv = {}) =>
run([name], env).trimEnd();
describe("cache-keys.sh", () => {
it("is deterministic for a given tree", () => {
expect(run()).toEqual(run());
});
it("emits every cache's path, key and restore key", () => {
for (const cache of CACHES) {
expect(get(`${cache}-path`)).not.toEqual("");
expect(get(`${cache}-key`)).not.toEqual("");
expect(get(`${cache}-restore-key`)).not.toEqual("");
}
});
// Silently returning "" for a name that does not exist is how a caller ends up building a key with a
// hole in it.
it("fails on an unknown output name", () => {
expect(() => run(["no-such-output"], {}, true)).toThrow();
});
// A key equal to its own restore prefix points every dependency set at one entry, and nothing reports
// it. This is the failure the script exists to prevent.
it("never emits a key equal to its own restore prefix", () => {
for (const cache of CACHES) {
const key = get(`${cache}-key`);
const prefix = get(`${cache}-restore-key`);
expect(key.startsWith(prefix)).toBe(true);
expect(key.length).toBeGreaterThan(prefix.length);
}
});
it("scopes every key to the runner OS", () => {
for (const cache of CACHES) {
expect(get(`${cache}-key`, { RUNNER_OS: "Linux" })).not.toEqual(
get(`${cache}-key`, { RUNNER_OS: "macOS" }),
);
}
});
// Home paths must be absolute: actions/cache expands a tilde but a shell assigning one to a variable
// does not, and would create a directory literally named `~`.
it("emits absolute paths for home directory locations", () => {
expect(get("bun-store-path").startsWith("/")).toBe(true);
expect(get("cypress-path").startsWith("/")).toBe(true);
for (const line of get("m2-path").split("\n")) {
expect(line.startsWith("~")).toBe(false);
}
});
// The Cypress binary depends on the Cypress version alone, so an unrelated dependency bump must not
// invalidate a 200MB download.
it("keys the Cypress binary on the resolved version, not the lockfile", () => {
expect(get("cypress-key")).toMatch(/^cypress-Linux-\d+\.\d+\.\d+/);
});
it("keys the ESLint cache on the commit", () => {
expect(get("eslint-key", { GITHUB_SHA: "deadbeef" })).toEqual(
"eslint-Linux-deadbeef",
);
});
});

69
.github/scripts/check-preresolve-aliases.sh vendored Executable file
View File

@@ -0,0 +1,69 @@
#!/usr/bin/env bash
# Fails when a `clojure` invocation under .github/ uses an alias combination that cache-warm does not
# pre-resolve.
#
# tools.deps selects dependency versions globally per classpath, so two alias combinations can resolve
# the same transitive library to different versions. A combination the warm job never resolved is
# therefore not merely absent from the cache - the cache holds *a* version of those libraries, just not
# the one this classpath wants, and the job fetches its delta straight from Maven Central on every run.
# That is the intermittent-403 failure of DEV-2094, and it is invisible: the build still passes.
#
# Nothing about a missing combination shows up at runtime, so this check is the only thing standing
# between a newly added alias combination and a permanently half-cold cache.
#
# .github/scripts/check-preresolve-aliases.sh
set -euo pipefail
cd "$(git rev-parse --show-toplevel)"
WARM=.github/workflows/cache-warm.yml
# Invocations deliberately left out of the warm job. Each needs a reason, because the default answer is
# to add the combination to the list instead.
WAIVED="$(
cat <<'EOF'
:deps
:outdated
EOF
)"
# :deps - tools.deps built-in used by write-poms.sh; contributes no dependencies of its own.
# :outdated - scheduled dependency-bump workflow only; a cold resolve there costs nothing anyone waits on.
# Alias combinations the warm job pre-resolves.
declared="$(sed -n '/combos=(/,/^ *)/p' "$WARM" | grep -oE '":[^"]*"' | tr -d '"' | sort -u)"
if [ -z "$declared" ]; then
echo "::error::check-preresolve-aliases.sh: found no combos=( ... ) list in $WARM" >&2
exit 1
fi
# Expressions we know how to expand. An unrecognised one has to stop the check rather than be skipped:
# silently ignoring it is how a whole matrix of combinations would go unchecked.
unknown="$(grep -rhoE '\$\{\{[^}]*\}\}' --include='*' \
--exclude='*.png' --exclude='*.jpg' .github/ 2>/dev/null \
| sort -u | grep -v 'edition' || true)"
scan() { # scan <edition>: alias strings used by clojure invocations, with expressions substituted
grep -rhE 'clojure[[:space:]]+(-P[[:space:]]+)?-[XMATP]?:' \
--include='*' --exclude='*.png' --exclude='*.jpg' .github/ 2>/dev/null \
| sed "s/\\\${{[^}]*edition[^}]*}}/$1/g" \
| grep -oE 'clojure[[:space:]]+(-P[[:space:]]+)?-[XMATP]?:[A-Za-z0-9:_.-]+' \
| grep -oE ':[A-Za-z0-9:_.-]+' \
| sed 's/:$//'
}
found="$( { scan ee; scan oss; } | sort -u )"
covered="$(printf '%s\n%s\n' "$declared" "$WAIVED" | sort -u)"
missing="$(comm -23 <(printf '%s\n' "$found") <(printf '%s\n' "$covered") || true)"
if [ -n "$missing" ]; then
echo "::error::Alias combinations used under .github/ that cache-warm does not pre-resolve:" >&2
printf '%s\n' "$missing" | sed 's/^/ /' >&2
echo >&2
echo "Add each to the combos list in $WARM, or waive it in this script with a reason." >&2
echo "Leaving it uncovered means those jobs re-fetch their version deltas from Maven on every run." >&2
exit 1
fi
echo "All $(printf '%s\n' "$found" | grep -c .) clojure alias combinations under .github/ are pre-resolved."

View File

@@ -1,91 +0,0 @@
/**
* jsonData must be in the format:
* [
* { column1: value1, column2: value2, ... },
* { column1: value3, column2: value4, ... },
* ...
* ]
*/
function jsonToCsvFormData(jsonData) {
const formData = new FormData();
const csvHeader = Object.keys(jsonData[0]).join(',') + '\n';
const csvContent = jsonData.map(row =>
Object.values(row).map(value => `"${String(value).replace(/"/g, '""')}"`).join(',')
).join('\n');
const blob = new Blob([csvHeader + csvContent], { type: 'text/csv' });
formData.append('file', blob, 'data.csv');
return formData;
}
// Transient failures worth retrying: throttling and server-side errors. A 4xx
// (bad request, auth) won't succeed on retry, so those fail fast.
const RETRYABLE_STATUS = new Set([429, 500, 502, 503, 504]);
class UploadError extends Error {
constructor(message, { retryable }) {
super(message);
this.retryable = retryable;
}
}
async function attemptUpload(url, jsonData, timeoutMs) {
let response;
try {
response = await fetch(url, {
method: 'POST',
headers: {
"x-api-key": process.env.API_KEY
},
// Rebuilt per attempt: a consumed request body can't be reused.
body: jsonToCsvFormData(jsonData),
// Without this the request can hang indefinitely on a stalled server,
// eating the whole job timeout. An abort surfaces below as a retryable
// network error, so it retries with backoff instead.
signal: AbortSignal.timeout(timeoutMs)
});
} catch (networkError) {
throw new UploadError(networkError.message, { retryable: true });
}
if (response.ok) {
return;
}
const body = await response.text().catch(() => '');
throw new UploadError(
`Upload failed: ${response.status} ${response.statusText} ${body}`.trim(),
{ retryable: RETRYABLE_STATUS.has(response.status) }
);
}
const sleep = ms => new Promise(resolve => setTimeout(resolve, ms));
/**
* Uploads jsonData as CSV, retrying transient API errors with exponential
* backoff. `retries` is the number of extra attempts after the first (so the
* default 2 means up to 3 tries). `timeoutMs` is a per-attempt total deadline
* (default 60s) that aborts a stalled request. Throws the last error once
* attempts are exhausted or the error is non-retryable.
*/
async function uploadCsvToMb({ baseUrl, tableId, jsonData, mode = 'append', retries = 2, retryDelayMs = 1000, timeoutMs = 60_000 }) {
const operation = mode === 'replace' ? 'replace-csv' : 'append-csv';
const url = `${baseUrl}/api/table/${tableId}/${operation}`;
for (let attempt = 0; ; attempt++) {
try {
await attemptUpload(url, jsonData, timeoutMs);
return { success: true };
} catch (error) {
if (!error.retryable || attempt >= retries) {
throw error;
}
const delay = retryDelayMs * 2 ** attempt;
console.warn(`Upload attempt ${attempt + 1} failed (${error.message}); retrying in ${delay}ms`);
await sleep(delay);
}
}
}
module.exports = { uploadCsvToMb };

View File

@@ -1,50 +0,0 @@
const { uploadCsvToMb } = require("./csv-to-mb");
const ok = () => ({ ok: true });
const fail = status => ({ ok: false, status, statusText: "err", text: async () => "" });
const upload = (over = {}) =>
uploadCsvToMb({
baseUrl: "https://stats.example.com",
tableId: 1,
jsonData: [{ a: 1 }],
retryDelayMs: 1,
...over,
});
describe("uploadCsvToMb retries", () => {
afterEach(() => {
delete global.fetch;
});
it("retries transient server errors then succeeds", async () => {
global.fetch = jest.fn().mockResolvedValueOnce(fail(503)).mockResolvedValueOnce(fail(500)).mockResolvedValueOnce(ok());
await expect(upload({ retries: 2 })).resolves.toEqual({ success: true });
expect(global.fetch).toHaveBeenCalledTimes(3);
});
it("retries network errors", async () => {
global.fetch = jest
.fn()
.mockRejectedValueOnce(new Error("ECONNRESET"))
.mockResolvedValueOnce(ok());
await expect(upload({ retries: 2 })).resolves.toEqual({ success: true });
expect(global.fetch).toHaveBeenCalledTimes(2);
});
it("gives up after exhausting retries and throws the last error", async () => {
global.fetch = jest.fn().mockResolvedValue(fail(500));
await expect(upload({ retries: 2 })).rejects.toThrow("Upload failed: 500");
expect(global.fetch).toHaveBeenCalledTimes(3); // 1 initial + 2 retries
});
it("does not retry a non-transient 4xx", async () => {
global.fetch = jest.fn().mockResolvedValue(fail(400));
await expect(upload({ retries: 2 })).rejects.toThrow("Upload failed: 400");
expect(global.fetch).toHaveBeenCalledTimes(1);
});
});

View File

@@ -1,40 +0,0 @@
async function findLinearIssue({ githubIssueUrl, linearApiKey }) {
const response = await fetch("https://api.linear.app/graphql", {
method: "POST",
headers: {
"Content-Type": "application/json",
Authorization: linearApiKey,
},
body: JSON.stringify({
query: `
query {
attachmentsForURL(url: "${githubIssueUrl}") {
nodes {
issue { identifier }
}
}
}
`,
}),
});
const data = await response.json();
if (!response.ok) {
return { success: false, message: `Linear API error: ${response.status}` };
}
if (!data.data?.attachmentsForURL?.nodes?.length) {
return {
success: false,
message: `No Linear issue linked to ${githubIssueUrl}`,
};
}
return {
success: true,
issue: data.data.attachmentsForURL.nodes[0].issue.identifier,
};
}
module.exports = { findLinearIssue };

View File

@@ -0,0 +1,66 @@
/**
* Renames aliased dependency keys in a bun-generated yarn.lock.
*
* Yarn names an alias entry after the alias, for example
* `"typescript7@npm:typescript@7.0.2"`. Bun names it after the aliased package
* instead, for example `"typescript@npm:typescript@7.0.2"`. Snyk follows the
* yarn convention and fails to resolve the dependency.
*/
import { readFile, writeFile } from "node:fs/promises";
async function main() {
const packageJsonPath = `${process.cwd()}/package.json`;
const lockPath = `${process.cwd()}/yarn.lock`;
const packageJson = JSON.parse(await readFile(packageJsonPath, "utf8"));
const dependencies = Object.entries<string>(
Object.assign(
{},
packageJson.dependencies,
packageJson.devDependencies,
packageJson.optionalDependencies,
),
);
let lock = await readFile(lockPath, "utf8");
const renamedKeys: string[] = [];
for (const [alias, specifier] of dependencies) {
if (!specifier.startsWith("npm:")) {
continue;
}
const aliasedPackage = specifier
.slice("npm:".length)
.replace(/@[^@]*$/, "");
if (aliasedPackage === alias) {
continue;
}
const bunKey = `"${aliasedPackage}@${specifier}"`;
const yarnKey = `"${alias}@${specifier}"`;
if (lock.includes(bunKey)) {
lock = lock.replaceAll(bunKey, yarnKey);
renamedKeys.push(`${bunKey} -> ${yarnKey}`);
}
}
if (renamedKeys.length > 0) {
await writeFile(lockPath, lock);
}
process.stdout.write(
renamedKeys.length > 0
? `Renamed alias keys in yarn.lock:\n${renamedKeys.join("\n")}\n`
: "No alias keys to rename in yarn.lock\n",
);
}
main().catch((error) => {
process.stderr.write(`Failed to rename alias keys in yarn.lock: ${error}\n`);
process.exit(1);
});

View File

@@ -1,84 +0,0 @@
const githubSlackMap = require("../../release/github-slack-map.json");
async function getIssueData({ context, github, manualIssueNumber }) {
let issueNumber;
let issueData;
let prReviews = [];
const triggerType = context.eventName;
if (context.eventName === "issues") {
issueNumber = context.payload.issue.number;
issueData = context.payload.issue;
} else if (context.eventName === "workflow_dispatch") {
issueNumber = parseInt(manualIssueNumber);
const { data } = await github.rest.issues.get({
owner: context.repo.owner,
repo: context.repo.repo,
issue_number: issueNumber,
});
issueData = data;
} else if (context.eventName === "pull_request_review") {
const prNumber = context.payload.pull_request.number;
const { data: prData } = await github.rest.pulls.get({
owner: context.repo.owner,
repo: context.repo.repo,
pull_number: prNumber,
});
// Extract issue number from branch name (e.g., "claude-fix/issue-12345-some-title")
const branchMatch = prData.head.ref.match(/claude-fix\/issue-(\d+)/);
if (!branchMatch) {
throw new Error(
`Could not extract issue number from branch name: ${prData.head.ref}`,
);
}
issueNumber = parseInt(branchMatch[1]);
const { data } = await github.rest.issues.get({
owner: context.repo.owner,
repo: context.repo.repo,
issue_number: issueNumber,
});
issueData = data;
const { data: reviews } = await github.rest.pulls.listReviews({
owner: context.repo.owner,
repo: context.repo.repo,
pull_number: prNumber,
});
prReviews = reviews.filter(
(review) =>
review.state === "CHANGES_REQUESTED" &&
isMetabaseMember(review.user.login),
);
}
const { data: comments } = await github.rest.issues.listComments({
owner: context.repo.owner,
repo: context.repo.repo,
issue_number: issueNumber,
});
const employeeComments = comments.filter((comment) =>
isMetabaseMember(comment.user.login),
);
return {
issueNumber,
issueData,
employeeComments,
prReviews,
triggerType,
isIssueAuthorEmployee: isMetabaseMember(issueData.user.login),
};
}
function isMetabaseMember(username) {
return username in githubSlackMap;
}
module.exports = {
getIssueData,
};

109
.github/scripts/linear-issues.js vendored Normal file
View File

@@ -0,0 +1,109 @@
async function findLinearIssue({ githubIssueUrl, linearApiKey }) {
const response = await fetch("https://api.linear.app/graphql", {
method: "POST",
headers: {
"Content-Type": "application/json",
Authorization: linearApiKey,
},
body: JSON.stringify({
query: `
query {
attachmentsForURL(url: "${githubIssueUrl}") {
nodes {
issue { identifier }
}
}
}
`,
}),
});
const data = await response.json();
if (!response.ok) {
return { success: false, message: `Linear API error: ${response.status}` };
}
if (!data.data?.attachmentsForURL?.nodes?.length) {
return {
success: false,
message: `No Linear issue linked to ${githubIssueUrl}`,
};
}
return {
success: true,
issue: data.data.attachmentsForURL.nodes[0].issue.identifier,
};
}
async function createLinearIssue({
teamId,
title,
description,
projectId,
labelIds,
linearApiKey,
}) {
const response = await fetch("https://api.linear.app/graphql", {
method: "POST",
headers: {
"Content-Type": "application/json",
Authorization: linearApiKey,
},
body: JSON.stringify({
query: `
mutation CreateIssue($input: IssueCreateInput!) {
issueCreate(input: $input) {
success
issue {
id
identifier
title
url
}
}
}
`,
variables: {
input: {
teamId,
title,
...(description ? { description } : {}),
...(projectId ? { projectId } : {}),
...(labelIds?.length ? { labelIds } : {}),
},
},
}),
});
if (!response.ok) {
return { success: false, message: `Linear API error: ${response.status}` };
}
const data = await response.json();
if (data.errors?.length) {
return {
success: false,
message: `Linear API error: ${data.errors.map((e) => e.message).join(", ")}`,
};
}
if (!data.data?.issueCreate?.success) {
return {
success: false,
message: `Failed to create Linear issue "${title}"`,
};
}
return {
success: true,
issue: data.data.issueCreate.issue,
};
}
module.exports = {
findLinearIssue,
createLinearIssue,
};

View File

@@ -1,65 +0,0 @@
#! /usr/bin/env bash
# runs one or more Metabase test(s) against a Kerberized Presto instance
set -eo pipefail
# Need Java commands on $PATH, which apparently is not yet the case
export JAVA_HOME="/usr/lib/jvm/msopenjdk-current"
export PATH="$PATH:$JAVA_HOME/bin"
# ensure java commmand is available
which java
# install clojure version needed for Metabase
curl -O https://download.clojure.org/install/linux-install-1.12.0.1488.sh
chmod +x linux-install-1.12.0.1488.sh
./linux-install-1.12.0.1488.sh
RESOURCES_DIR=/app/source/resources
# ensure the expected files are in place, in the resources dir
if [ ! -f "$RESOURCES_DIR/ssl_keystore.jks" ]; then
echo "$RESOURCES_DIR/ssl_keystore.jks does not exist; cannot run test" >&2
exit 11
fi
if [ ! -f "$RESOURCES_DIR/krb5.conf" ]; then
echo "$RESOURCES_DIR/krb5.conf does not exist; cannot run test" >&2
exit 12
fi
if [ ! -f "$RESOURCES_DIR/client.keytab" ]; then
echo "$RESOURCES_DIR/client.keytab does not exist; cannot run test" >&2
exit 13
fi
# Copy the JDK cacerts file to our resources
cp $JAVA_HOME/lib/security/cacerts $RESOURCES_DIR/cacerts-with-presto-ca.jks
# Capture the Presto server self signed CA in PEM format
openssl s_client -showcerts -connect presto-kerberos:7778 </dev/null \
| openssl x509 -outform PEM >$RESOURCES_DIR/presto-ssl-root-ca.pem
# Convert the Presto server self signed CA to DER format
openssl x509 -outform der -in $RESOURCES_DIR/presto-ssl-root-ca.pem -out $RESOURCES_DIR/presto-ssl-root-ca.der
# Add Presto's self signed CA to the truststore
keytool -noprompt -import -alias presto-kerberos -keystore $RESOURCES_DIR/cacerts-with-presto-ca.jks \
-storepass changeit -file $RESOURCES_DIR/presto-ssl-root-ca.der -trustcacerts
# Set up the environment variables pointing to all of this, and run some tests
DRIVERS=presto-jdbc \
MB_ENABLE_PRESTO_JDBC_DRIVER=true \
MB_PRESTO_JDBC_TEST_HOST=presto-kerberos \
MB_PRESTO_JDBC_TEST_PORT=7778 \
MB_PRESTO_JDBC_TEST_SSL=true \
MB_PRESTO_JDBC_TEST_SSL_KEYSTORE_PATH=$RESOURCES_DIR/ssl_keystore.jks \
MB_PRESTO_JDBC_TEST_SSL_KEYSTORE_PASSWORD=presto \
MB_PRESTO_JDBC_TEST_SSL_TRUSTSTORE_PATH=$RESOURCES_DIR/cacerts-with-presto-ca.jks \
MB_PRESTO_JDBC_TEST_SSL_TRUSTSTORE_PASSWORD=changeit \
MB_PRESTO_JDBC_TEST_KERBEROS=true \
MB_PRESTO_JDBC_TEST_USER=bob@EXAMPLE.COM \
MB_PRESTO_JDBC_TEST_KERBEROS_PRINCIPAL=bob@EXAMPLE.COM \
MB_PRESTO_JDBC_TEST_KERBEROS_REMOTE_SERVICE_NAME=HTTP \
MB_PRESTO_JDBC_TEST_KERBEROS_KEYTAB_PATH=$RESOURCES_DIR/client.keytab \
MB_PRESTO_JDBC_TEST_KERBEROS_CONFIG_PATH=$RESOURCES_DIR/krb5.conf \
clojure -X:dev:test:drivers:drivers-dev :only metabase.driver.presto-jdbc-test

140
.github/scripts/stats-import.ts vendored Normal file
View File

@@ -0,0 +1,140 @@
// Pushes stats rows to eng-stats-importer, which writes them to Postgres.
//
// The importer's origin comes from ENG_STATS_URL and its key from API_KEY, both
// repository secrets — this repo is public, so neither is hardcoded here. Visit
// the importer to see the tables and mint keys.
// Transient failures worth retrying: throttling and server-side errors. A 4xx
// (bad request, auth, unknown table) won't succeed on retry, so those fail fast.
const RETRYABLE_STATUS = new Set([429, 500, 502, 503, 504]);
/**
* One row, keyed by column. Values are `unknown` rather than a scalar union
* because a `json` column takes arbitrary nested data.
*/
export type StatsRow = Record<string, unknown>;
export interface ImportStatsOptions {
/** Table name as it appears in `/api/import/{table}`, e.g. `emotion_files`. */
table: string;
rows: StatsRow[];
/** Importer origin. Defaults to ENG_STATS_URL. */
baseUrl?: string;
/** Extra attempts after the first, so the default 2 means up to 3 tries. */
retries?: number;
retryDelayMs?: number;
/** Per-attempt total deadline; aborts a stalled request. */
timeoutMs?: number;
}
export interface ImportStatsResult {
success: true;
inserted?: number;
/** Payload keys with no matching column, when `ignoreUnknown` is in play. */
ignoredKeys?: string[];
}
class ImportError extends Error {
retryable: boolean;
constructor(message: string, { retryable }: { retryable: boolean }) {
super(message);
this.retryable = retryable;
}
}
async function attemptImport(
url: string,
rows: StatsRow[],
timeoutMs: number,
): Promise<ImportStatsResult> {
let response: Response;
try {
response = await fetch(url, {
method: "PUT",
headers: {
"x-api-key": process.env.API_KEY ?? "",
"content-type": "application/json",
},
body: JSON.stringify(rows),
// Without this the request can hang indefinitely on a stalled server,
// eating the whole job timeout. An abort surfaces below as a retryable
// network error, so it retries with backoff instead.
signal: AbortSignal.timeout(timeoutMs),
});
} catch (networkError) {
const message =
networkError instanceof Error ? networkError.message : String(networkError);
throw new ImportError(message, { retryable: true });
}
if (!response.ok) {
// Failures carry an `{ error }` body naming the offending row and column,
// which is the only way to tell a type mismatch from a typo'd key.
const body = await response.text().catch(() => "");
throw new ImportError(
`Import failed: ${response.status} ${response.statusText} ${body}`.trim(),
{ retryable: RETRYABLE_STATUS.has(response.status) },
);
}
// Best-effort: the row count is for logging, so a body that doesn't parse
// shouldn't turn a successful import into a failure.
const result = (await response.json().catch(() => ({}))) as {
inserted?: number;
ignoredKeys?: string[];
};
return { success: true, inserted: result.inserted, ignoredKeys: result.ignoredKeys };
}
const sleep = (ms: number) => new Promise((resolve) => setTimeout(resolve, ms));
/**
* Imports `rows` into the `table` table, retrying transient API errors with
* exponential backoff.
*
* Keys are matched leniently against the table's columns -- `Emotion Files`,
* `emotionFiles` and `emotion_files` all land in `emotion_files` -- but values
* are matched strictly, and a key matching no column is an error rather than a
* silent drop. Rows are appended; the importer has no replace mode.
*
* Throws the last error once attempts are exhausted or the error is
* non-retryable.
*/
export async function importStats({
table,
rows,
baseUrl = process.env.ENG_STATS_URL,
retries = 2,
retryDelayMs = 1000,
timeoutMs = 60_000,
}: ImportStatsOptions): Promise<ImportStatsResult> {
// The importer accepts an empty payload, but there's no point spending a
// request (and a rate-limit slot) to insert nothing.
if (rows.length === 0) {
return { success: true, inserted: 0 };
}
if (!baseUrl) {
throw new Error("ENG_STATS_URL is not set, so there is no importer to push to");
}
// Tolerate a trailing slash: the origin comes from a hand-entered secret.
const url = `${baseUrl.replace(/\/+$/, "")}/api/import/${table}`;
for (let attempt = 0; ; attempt++) {
try {
return await attemptImport(url, rows, timeoutMs);
} catch (error) {
const importError = error as ImportError;
if (!importError.retryable || attempt >= retries) {
throw error;
}
const delay = retryDelayMs * 2 ** attempt;
console.warn(
`Import attempt ${attempt + 1} failed (${importError.message}); retrying in ${delay}ms`,
);
await sleep(delay);
}
}
}

View File

@@ -0,0 +1,160 @@
import { importStats } from "./stats-import";
const ok = (body: unknown = { inserted: 1 }) => ({ ok: true, json: async () => body });
const fail = (status: number) => ({
ok: false,
status,
statusText: "err",
text: async () => "",
});
const load = (over: Record<string, unknown> = {}) =>
importStats({
baseUrl: "https://stats.example.com",
table: "emotion_files",
rows: [{ a: 1 }],
retryDelayMs: 1,
...over,
});
describe("importStats requests", () => {
afterEach(() => {
// @ts-expect-error -- removing the stub the tests installed
delete global.fetch;
delete process.env.API_KEY;
delete process.env.ENG_STATS_URL;
});
it("defaults the origin to ENG_STATS_URL", async () => {
const fetchMock = jest.fn().mockResolvedValue(ok());
global.fetch = fetchMock;
process.env.ENG_STATS_URL = "https://importer.example.com";
await load({ baseUrl: undefined });
expect(fetchMock.mock.calls[0][0]).toBe(
"https://importer.example.com/api/import/emotion_files",
);
});
it("tolerates a trailing slash on the origin", async () => {
const fetchMock = jest.fn().mockResolvedValue(ok());
global.fetch = fetchMock;
await load({ baseUrl: "https://importer.example.com/" });
expect(fetchMock.mock.calls[0][0]).toBe(
"https://importer.example.com/api/import/emotion_files",
);
});
it("fails loudly when no origin is configured", async () => {
const fetchMock = jest.fn();
global.fetch = fetchMock;
await expect(load({ baseUrl: undefined })).rejects.toThrow("ENG_STATS_URL is not set");
expect(fetchMock).not.toHaveBeenCalled();
});
it("PUTs the rows as JSON to the table's import endpoint", async () => {
const fetchMock = jest.fn().mockResolvedValue(ok());
global.fetch = fetchMock;
process.env.API_KEY = "secret";
await expect(load({ rows: [{ a: 1 }, { a: 2 }] })).resolves.toEqual({
success: true,
inserted: 1,
ignoredKeys: undefined,
});
const [url, options] = fetchMock.mock.calls[0];
expect(url).toBe("https://stats.example.com/api/import/emotion_files");
expect(options.method).toBe("PUT");
expect(options.headers).toMatchObject({
"x-api-key": "secret",
"content-type": "application/json",
});
expect(JSON.parse(options.body)).toEqual([{ a: 1 }, { a: 2 }]);
});
it("skips the request entirely when there are no rows", async () => {
const fetchMock = jest.fn();
global.fetch = fetchMock;
await expect(load({ rows: [] })).resolves.toEqual({ success: true, inserted: 0 });
expect(fetchMock).not.toHaveBeenCalled();
});
it("succeeds even if the response body does not parse", async () => {
global.fetch = jest.fn().mockResolvedValue({
ok: true,
json: async () => {
throw new Error("not json");
},
});
await expect(load()).resolves.toMatchObject({ success: true });
});
});
describe("importStats retries", () => {
afterEach(() => {
// @ts-expect-error -- removing the stub the tests installed
delete global.fetch;
});
it("retries transient server errors then succeeds", async () => {
const fetchMock = jest
.fn()
.mockResolvedValueOnce(fail(503))
.mockResolvedValueOnce(fail(500))
.mockResolvedValueOnce(ok());
global.fetch = fetchMock;
await expect(load({ retries: 2 })).resolves.toMatchObject({ success: true });
expect(fetchMock).toHaveBeenCalledTimes(3);
});
it("retries throttling", async () => {
const fetchMock = jest.fn().mockResolvedValueOnce(fail(429)).mockResolvedValueOnce(ok());
global.fetch = fetchMock;
await expect(load({ retries: 2 })).resolves.toMatchObject({ success: true });
expect(fetchMock).toHaveBeenCalledTimes(2);
});
it("retries network errors", async () => {
const fetchMock = jest
.fn()
.mockRejectedValueOnce(new Error("ECONNRESET"))
.mockResolvedValueOnce(ok());
global.fetch = fetchMock;
await expect(load({ retries: 2 })).resolves.toMatchObject({ success: true });
expect(fetchMock).toHaveBeenCalledTimes(2);
});
it("gives up after exhausting retries and throws the last error", async () => {
const fetchMock = jest.fn().mockResolvedValue(fail(500));
global.fetch = fetchMock;
await expect(load({ retries: 2 })).rejects.toThrow("Import failed: 500");
expect(fetchMock).toHaveBeenCalledTimes(3); // 1 initial + 2 retries
});
it("does not retry a non-transient 4xx", async () => {
const fetchMock = jest.fn().mockResolvedValue(fail(400));
global.fetch = fetchMock;
await expect(load({ retries: 2 })).rejects.toThrow("Import failed: 400");
expect(fetchMock).toHaveBeenCalledTimes(1);
});
it("does not retry an unknown table", async () => {
const fetchMock = jest.fn().mockResolvedValue(fail(404));
global.fetch = fetchMock;
await expect(load({ retries: 2 })).rejects.toThrow("Import failed: 404");
expect(fetchMock).toHaveBeenCalledTimes(1);
});
});

View File

@@ -1,11 +1,11 @@
// Appends the test-plan stats row produced by create-test-plan.ts to the
// "FE Affected Tests" table on stats.metabase.com.
// Reads STATS_JSON, TRIGGERED_BY, PR_NUMBER, HEAD_SHA, BASE_SHA, API_KEY from env.
// "FE Affected Tests" table in eng-stats-importer.
// Reads STATS_JSON, PR_NUMBER, HEAD_SHA, BASE_SHA, API_KEY from env.
const { uploadCsvToMb } = require("./csv-to-mb.js");
import type { TestPlanStats } from "./affected-tests";
import { importStats } from "./stats-import";
// "FE Affected Tests" uploaded table on stats.metabase.com.
const TABLE_ID = 81818;
const TABLE = "fe_affected_tests";
async function main() {
// create-test-plan doesn't always produce stats (e.g. on release branches
@@ -16,13 +16,13 @@ async function main() {
return;
}
const s = JSON.parse(raw);
const s = JSON.parse(raw) as TestPlanStats;
console.log("stats", s);
const row = {
Date: new Date().toISOString(),
// "pr_update" or "merge_to_master". ("Trigger" is a reserved SQL word.)
"Triggered By": process.env.TRIGGERED_BY,
// ("Trigger" is a reserved SQL word.)
"Triggered By": "pr_update",
PR: Number(process.env.PR_NUMBER),
"Head SHA": process.env.HEAD_SHA,
"Base SHA": process.env.BASE_SHA,
@@ -43,17 +43,15 @@ async function main() {
"Loki Stories All": s.loki_stories_all,
"Loki Stories To Run (Rules)": s.loki_stories_to_run_rules,
"Loki Stories To Run (Usage)": s.loki_stories_to_run_usage,
"E2E Specs All": s.e2e_specs_all,
"E2E Specs To Run (Rules)": s.e2e_specs_to_run_rules,
"E2E Specs To Run (Usage)": s.e2e_specs_to_run_usage,
// Spelled as the column name rather than a display name: the importer
// splits a digit-then-capital run, so "E2E Specs All" would normalize to
// `e2_e_specs_all` instead of `e2e_specs_all`.
e2e_specs_all: s.e2e_specs_all,
e2e_specs_to_run_rules: s.e2e_specs_to_run_rules,
e2e_specs_to_run_usage: s.e2e_specs_to_run_usage,
};
await uploadCsvToMb({
baseUrl: "https://stats.metabase.com",
tableId: TABLE_ID,
jsonData: [row],
mode: "append",
});
await importStats({ table: TABLE, rows: [row] });
console.log("Data uploaded successfully");
}

View File

@@ -0,0 +1,38 @@
// Appends the measured bundle sizes (with deltas) to the "Bundle Sizes" table
// in eng-stats-importer. Reads ROWS (a JSON file of rows) and API_KEY from env,
// and reports whether the import landed via the `uploaded` step output.
import { appendFileSync, readFileSync } from "node:fs";
import { importStats, type StatsRow } from "./stats-import";
/** `core.setOutput` without the github-script runtime. */
function setOutput(name: string, value: string) {
const file = process.env.GITHUB_OUTPUT;
if (file) {
appendFileSync(file, `${name}=${value}\n`);
}
}
async function main() {
const rowsPath = process.env.ROWS || "artifacts/upload-rows.json";
const rows = JSON.parse(readFileSync(rowsPath, "utf8")) as StatsRow[];
console.table(rows);
try {
await importStats({ table: "bundle_sizes", rows });
console.log("Bundle sizes uploaded successfully");
setOutput("uploaded", "true");
} catch (error) {
// Stats logging is best-effort: if the importer is still unreachable after
// the client's retries, keep the run green. The last-plotted cache is only
// saved on a successful upload, so this point is re-attempted on the next
// master build and nothing is silently dropped.
const message = error instanceof Error ? error.message : String(error);
console.log(`::warning::Bundle-size upload failed after retries; leaving run green: ${message}`);
setOutput("uploaded", "false");
}
}
main();

25
.github/scripts/upload-emotion-stats.ts vendored Normal file
View File

@@ -0,0 +1,25 @@
// Appends the weekly @emotion/* importer count to the "Emotion Files" table in
// eng-stats-importer. Reads EMOTION_FILES and API_KEY from env.
import { importStats } from "./stats-import";
async function main() {
console.log("EMOTION_FILES", Number(process.env.EMOTION_FILES));
// Keys must match the target table's columns (the importer normalizes both
// sides, so casing/spacing is forgiving).
const rows = [
{
Date: new Date().toISOString().slice(0, 10), // YYYY-MM-DD
"Emotion Files": Number(process.env.EMOTION_FILES),
},
];
await importStats({ table: "emotion_stats", rows });
console.log("Data uploaded successfully");
}
main().catch((error) => {
console.error("Error uploading data:", error);
process.exit(1);
});

View File

@@ -0,0 +1,30 @@
// Appends the weekly module-boundary counts to the "Module Boundaries" table in
// eng-stats-importer. Reads NAMED_MODULES, ENFORCED_MODULES, UNMODULED,
// VIOLATIONS and API_KEY from env.
import { importStats } from "./stats-import";
async function main() {
console.log("NAMED_MODULES", Number(process.env.NAMED_MODULES));
console.log("ENFORCED_MODULES", Number(process.env.ENFORCED_MODULES));
console.log("UNMODULED", Number(process.env.UNMODULED));
console.log("VIOLATIONS", Number(process.env.VIOLATIONS));
const rows = [
{
Date: new Date().toISOString().slice(0, 10), // YYYY-MM-DD
Modules: Number(process.env.NAMED_MODULES),
"Enforced modules": Number(process.env.ENFORCED_MODULES),
"Unmoduled folders": Number(process.env.UNMODULED),
Violations: Number(process.env.VIOLATIONS),
},
];
await importStats({ table: "module_boundaries", rows });
console.log("Data uploaded successfully");
}
main().catch((error) => {
console.error("Error uploading data:", error);
process.exit(1);
});

6
.github/team.json vendored
View File

@@ -56,16 +56,14 @@
"label": ".Team/Metabot",
"members": [
"appleby",
"noahmoss",
"retro",
"sloansparger",
"somtom",
"tsplude",
"lbrdnk",
"piranha",
"crisptrutski",
"undrfined",
"andreis"
"andreis",
"prascuna"
]
},
{

View File

@@ -1,44 +0,0 @@
name: Actionlint
on:
push:
branches:
- 'master'
- 'release-**'
paths:
- '.github/workflows/**'
pull_request:
paths:
- '.github/workflows/**'
concurrency:
group: ${{ github.workflow }}-${{ github.head_ref && github.ref || github.run_id }}
cancel-in-progress: true
jobs:
actionlint:
name: Lint Github Actions Workflows
runs-on: ${{ vars.DEFAULT_RUNNER_KEY }}
steps:
- uses: actions/checkout@v6
- name: Get changed Github actions
uses: dorny/paths-filter@fbd0ab8f3e69293af611ebaee6363fc25e6d187d # v4.0.1
id: changes
with:
token: ${{ github.token }}
filters: .github/file-paths.yaml
list-files: shell
- name: Setup actionlint
if: steps.changes.outputs.actionlint == 'true'
id: download
shell: bash
run: |
if command -v actionlint; then
echo "executable=actionlint" >> "$GITHUB_OUTPUT"
else
# script sets executable output, for details see: https://github.com/rhysd/actionlint/blob/main/docs/usage.md#use-actionlint-on-github-actions
bash <(curl -fsSL https://raw.githubusercontent.com/rhysd/actionlint/v1.7.12/scripts/download-actionlint.bash)
fi
- name: Run actionlint
if: steps.changes.outputs.actionlint == 'true'
run: ${{ steps.download.outputs.executable }} --color --verbose ${{ steps.changes.outputs.actionlint_files }}

View File

@@ -1,5 +1,5 @@
name: AI Benchmark
run-name: "dispatch evals · ${{ github.event_name == 'pull_request' && format('smoke PR#{0}', github.event.pull_request.number) || (inputs.pr != '' && format('PR#{0}', inputs.pr) || (inputs.ref != '' && inputs.ref || 'master')) }}"
run-name: "dispatch evals · ${{ github.event_name == 'pull_request' && format('smoke PR#{0}', github.event.pull_request.number) || (inputs.ref != '' && inputs.ref || (inputs.pr != '' && format('PR#{0}', inputs.pr) || 'master')) }}"
on:
pull_request:
@@ -11,7 +11,7 @@ on:
required: false
default: ""
ref:
description: "Metabase ref"
description: "Metabase branch, tag or sha (resolved to a full sha; overrides the PR head, and must sit on the PR's branch when both are given)"
required: false
default: ""
scope:
@@ -38,7 +38,7 @@ jobs:
github.event_name == 'pull_request'
&& github.event.label.name == 'evals/smoke'
&& github.event.pull_request.head.repo.full_name != github.repository
runs-on: ${{ vars.DEFAULT_RUNNER_KEY }}
runs-on: ${{ vars.SLIM_RUNNER_KEY }}
timeout-minutes: 5
steps:
- run: |
@@ -50,7 +50,7 @@ jobs:
github.event_name == 'pull_request'
&& github.event.label.name == 'evals/smoke'
&& github.event.pull_request.head.repo.full_name == github.repository
runs-on: ${{ vars.DEFAULT_RUNNER_KEY }}
runs-on: ${{ vars.SLIM_RUNNER_KEY }}
timeout-minutes: 5
outputs:
comment_id: ${{ steps.post.outputs.comment_id }}
@@ -74,22 +74,88 @@ jobs:
const { data } = await github.rest.issues.createComment({ owner, repo, issue_number, body });
core.setOutput('comment_id', String(data.id));
uberjar:
needs: [announce]
resolve:
# Collapse the dispatch inputs into the one full sha everything downstream needs: it keys the
# uberjar artifact, tags the ECR image, and tells eval-run which commit was measured. `pr` was
# not consulted at all before, so `pr: 12345` built the tip of whatever branch the run was
# launched from and then reported the result against the PR.
if: github.event_name == 'workflow_dispatch'
runs-on: ${{ vars.DEFAULT_RUNNER_KEY }}
timeout-minutes: 5
permissions:
contents: read
pull-requests: read
outputs:
sha: ${{ steps.sha.outputs.sha }}
steps:
- name: Resolve pr/ref to a full commit sha
id: sha
env:
GH_TOKEN: ${{ github.token }}
PR: ${{ inputs.pr }}
REF: ${{ inputs.ref }}
DEFAULT_SHA: ${{ github.sha }}
run: |
set -euo pipefail
# `ref` wins over `pr`, matching evals' eval-run.yml, which this dispatches into — same two
# inputs with opposite precedence in chained workflows would be a trap. eval-run also
# rejects a ref that doesn't sit on the PR's branch; that check stays there rather than
# being duplicated here.
if [[ -n "$REF" ]]; then
# `ref` accepts a branch or tag; fetch-artifact and the image tag need a full sha.
SHA=$(gh api "repos/${GITHUB_REPOSITORY}/commits/${REF}" --jq .sha)
echo "ref ${REF} -> ${SHA}"
elif [[ -n "$PR" ]]; then
SHA=$(gh api "repos/${GITHUB_REPOSITORY}/pulls/${PR}" --jq .head.sha)
echo "PR #${PR} head -> ${SHA}"
else
SHA="$DEFAULT_SHA"
echo "neither pr nor ref given; using this run's commit -> ${SHA}"
fi
if [[ ! "$SHA" =~ ^[0-9a-f]{40}$ ]]; then
echo "::error::could not resolve a full commit sha (got '${SHA}')"
exit 1
fi
echo "sha=${SHA}" >> "$GITHUB_OUTPUT"
image:
needs: [resolve]
# `resolve` is workflow_dispatch-only, so tolerate it being skipped. The label guard is not
# optional: the trigger is `pull_request: [labeled]` with no filter of its own, so without it
# any label on any same-repo PR builds ee+oss and pushes to ECR — and since announce/smoke/
# finalize all skip in that case, a failure shows up as a bare red check with nothing
# explaining it.
if: >-
github.event_name == 'pull_request'
&& github.event.label.name == 'evals/smoke'
&& github.event.pull_request.head.repo.full_name == github.repository
uses: ./.github/workflows/uberjar.yml
secrets: inherit
!cancelled()
&& (needs.resolve.result == 'success' || needs.resolve.result == 'skipped')
&& (github.event_name == 'workflow_dispatch'
|| (github.event_name == 'pull_request'
&& github.event.label.name == 'evals/smoke'
&& github.event.pull_request.head.repo.full_name == github.repository))
uses: ./.github/workflows/build-private-ee-docker-image.yml
secrets:
PR_ENV_IAM_ROLE: ${{ secrets.PR_ENV_IAM_ROLE }}
PR_ENV_AWS_ACCOUNT_ID: ${{ secrets.PR_ENV_AWS_ACCOUNT_ID }}
with:
# Empty on the label path on purpose — the image workflow then falls back to the PR head,
# which is the artifact run-tests has already built and cached under that sha.
commit: ${{ needs.resolve.outputs.sha }}
smoke:
needs: [announce, uberjar]
needs: [announce, image]
# `!cancelled()` is load-bearing, not decoration: on the label path `resolve` is skipped by
# design, and `image` only runs at all because it opts back in with its own `!cancelled()`.
# GitHub propagates that skip *transitively*, so a plain boolean `if` here — carrying the
# implicit `success()` — skips smoke even though image succeeded and the image is in ECR.
# Opting out of the implicit check means the needs' results have to be asserted by hand.
if: >-
github.event_name == 'pull_request'
!cancelled()
&& needs.announce.result == 'success'
&& needs.image.result == 'success'
&& github.event_name == 'pull_request'
&& github.event.label.name == 'evals/smoke'
&& github.event.pull_request.head.repo.full_name == github.repository
runs-on: ${{ vars.DEFAULT_RUNNER_KEY }}
runs-on: ${{ vars.SLIM_RUNNER_KEY }}
timeout-minutes: 5
env:
COMMENT_ID: ${{ needs.announce.outputs.comment_id }}
@@ -114,14 +180,14 @@ jobs:
finalize:
# If the build or dispatch fell over, eval-run never took over the comment — leave a
# failure verdict instead of a comment stuck at "building the image…".
needs: [announce, uberjar, smoke]
needs: [announce, image, smoke]
if: always() && needs.announce.result == 'success' && needs.smoke.result != 'success'
runs-on: ${{ vars.DEFAULT_RUNNER_KEY }}
runs-on: ${{ vars.SLIM_RUNNER_KEY }}
timeout-minutes: 5
env:
COMMENT_ID: ${{ needs.announce.outputs.comment_id }}
SMOKE_RESULT: ${{ needs.smoke.result }}
UBERJAR_RESULT: ${{ needs.uberjar.result }}
IMAGE_RESULT: ${{ needs.image.result }}
steps:
- name: Comment (build/dispatch failed)
uses: actions/github-script@v9
@@ -130,7 +196,7 @@ jobs:
script: |
const runUrl = `${context.serverUrl}/${context.repo.owner}/${context.repo.repo}/actions/runs/${context.runId}`;
// A concurrency-cancelled retrigger lands here too; don't stamp it as an infra failure.
const cancelled = [process.env.SMOKE_RESULT, process.env.UBERJAR_RESULT].includes('cancelled');
const cancelled = [process.env.SMOKE_RESULT, process.env.IMAGE_RESULT].includes('cancelled');
const body = cancelled
? `🤖 **Evals smoke** — ⚪ cancelled before evals started ([logs](${runUrl}))`
: `🤖 **Evals smoke** — ❌ build or dispatch failed before evals started ([logs](${runUrl}))`;
@@ -150,8 +216,12 @@ jobs:
}).catch(() => {});
dispatch:
needs: [resolve, image]
# Deliberately no `always()`: benchmarking without the image this run was supposed to produce
# would measure a stale build or nothing at all. A failed `image` fails the run, which is the
# signal here — unlike the PR path there is no comment for `finalize` to stamp.
if: github.event_name == 'workflow_dispatch'
runs-on: ${{ vars.DEFAULT_RUNNER_KEY }}
runs-on: ${{ vars.SLIM_RUNNER_KEY }}
timeout-minutes: 5
steps:
- name: Dispatch eval-run
@@ -159,7 +229,8 @@ jobs:
GH_TOKEN: ${{ secrets.METABASE_AUTOMATION_USER_TOKEN }}
EVALS_BRANCH: ${{ github.event.inputs.evals_branch }}
PR: ${{ inputs.pr }}
REF: ${{ inputs.ref }}
# The sha the image was actually built and tagged with, not the raw `ref` input.
REF: ${{ needs.resolve.outputs.sha }}
SCOPE: ${{ inputs.scope }}
SAMPLE: ${{ inputs.sample }}
run: |

View File

@@ -6,6 +6,34 @@ on:
skip:
type: boolean
default: false
force-run:
description: Run the full suite regardless of the diff (master / release branches).
type: boolean
default: false
# Run one app-DB job on demand, against the ref you dispatch on, optionally under a profiler.
# Everything else about the job -- services, matrix legs, test args, quarantine gating -- is
# whatever the normal CI run uses, because it is the same job.
workflow_dispatch:
inputs:
app-db:
description: 'Which app-DB job to run. Only this one runs; the others are skipped.'
type: choice
required: true
options:
- mariadb
- mysql
- postgres
tests:
description: 'Optional: run only these tests, comma- or space-separated, e.g. metabase.db.custom-migrations-test/some-test. Bare namespaces work too. Blank runs the job''s usual set.'
type: string
required: false
default: ''
profile:
description: 'Profile the test run and attach a flamegraph to the job. Adds sampling overhead.'
type: boolean
required: false
default: false
concurrency:
group: ${{ github.workflow }}-${{ github.head_ref && github.ref || github.run_id }}-app-db
@@ -15,28 +43,105 @@ concurrency:
# ci-conductor (secrets: inherit is already set on the app-db call in
# run-tests.yml). See DEV-2224.
env:
CI_CONDUCTOR_BASE_URL: ${{ secrets.CI_CONDUCTOR_BASE_URL }}
CI_CONDUCTOR_BASE_URL: ${{ vars.CI_CONDUCTOR_BASE_URL }}
CI_CONDUCTOR_WEBHOOK_SECRET: ${{ secrets.CI_CONDUCTOR_WEBHOOK_SECRET }}
# Both read by the test-driver action, which is what actually runs the tests. Set here rather than
# passed as inputs to each job so that turning either on is one line, not one line per job.
# Empty (and so off) on workflow_call, where neither input exists.
#
# ONLY_TESTS replaces the job's own test selection, so every matrix leg of the chosen job runs the
# whole list -- each app DB has three.
PROFILE_TESTS: ${{ inputs.profile }}
ONLY_TESTS: ${{ inputs.tests }}
jobs:
be-tests-mariadb:
app-db-matrix:
if: ${{ !inputs.skip }}
runs-on: ${{ vars.DEFAULT_RUNNER_KEY }}
timeout-minutes: 5
name: Determine app DB versions
outputs:
mariadb: ${{ steps.versions.outputs.mariadb }}
mysql: ${{ steps.versions.outputs.mysql }}
postgres: ${{ steps.versions.outputs.postgres }}
steps:
- name: Select versions to test
id: versions
uses: actions/github-script@v9
env:
RUN_ALL: ${{ inputs.force-run }}
with:
script: | # js
const runAll = process.env.RUN_ALL === 'true';
const versions = {
mariadb: [
{
name: 'MariaDB 10.6',
image: 'metabase/mariadb:10.6',
'junit-name': 'be-tests-mariadb-10-6-ee',
env: { 'enable-ssl-tests': 'false' },
latest: false
},
{
name: 'MariaDB Latest',
image: 'mariadb:latest',
'junit-name': 'be-tests-mariadb-latest-ee',
env: { 'enable-ssl-tests': 'false' },
latest: true
},
],
mysql: [
{
name: 'MySQL 8.0',
image: 'mysql:8.0',
'junit-name': 'be-tests-mysql-8-0-ee',
env: { 'enable-ssl-tests': 'false', 'enable-aws-iam-tests': 'false' },
latest: false
},
{
name: 'MySQL 9',
image: 'mysql:9',
'junit-name': 'be-tests-mysql-9-ee',
env: { 'enable-ssl-tests': 'true', 'enable-aws-iam-tests': 'true' },
// treating 9.0 as latest until we introduce mysql 26+ support
latest: true
},
],
postgres: [
{
name: 'Postgres 14.x',
'junit-name': 'be-tests-postgres-ee',
image: 'postgres:14-alpine',
env: { 'enable-ssl-tests': 'false', 'enable-aws-iam-tests': 'false' },
latest: false,
},
{
name: 'Postgres Latest',
'junit-name': 'be-tests-postgres-latest-ee',
image: 'postgres:latest',
env: { 'enable-ssl-tests': 'true', 'enable-aws-iam-tests': 'true' },
latest: true,
},
],
};
for (const [db, dbVersions] of Object.entries(versions)) {
const selected = runAll ? dbVersions : dbVersions.filter((version) => version.latest);
core.info(`${db}: ${selected.map((version) => version.name).join(', ')}`);
core.setOutput(db, JSON.stringify(selected));
}
be-tests-mariadb:
needs: app-db-matrix
if: ${{ !inputs.skip && (inputs.app-db == '' || inputs.app-db == 'mariadb') }}
runs-on: ${{ vars.DEFAULT_RUNNER_KEY }}
timeout-minutes: 40
strategy:
fail-fast: false
matrix:
version:
- name: MariaDB 10.6
junit-name: be-tests-mariadb-10-6-ee
image: metabase/mariadb:10.6 # https://github.com/metabase/metabase-mariadb-docker-image
env:
enable-ssl-tests: 'false'
- name: MariaDB Latest
junit-name: be-tests-mariadb-latest-ee
image: mariadb:latest
env:
enable-ssl-tests: 'false'
version: ${{ fromJSON(needs.app-db-matrix.outputs.mariadb) }}
job:
- name: Enterprise Tests
build-static-viz: false
@@ -70,21 +175,9 @@ jobs:
MARIADB_ALLOW_EMPTY_ROOT_PASSWORD: true
MARIADB_DATABASE: mb_test
env:
# Only run old migrations tests on pushes to master or release branches. All other branches should skip tests
# with the tag `mb/old-migrations-test`. `__ADDITIONAL_EXCLUDED_TAG__` is not used anywhere outside of splicing
# it in to the command below.
__ADDITIONAL_EXCLUDED_TAG__: >-
${{
(
github.event_name == 'push' &&
(
github.ref_name == 'master' ||
startsWith(github.ref_name, 'release-x')
) &&
''
) ||
':mb/old-migrations-test'
}}
# __ADDITIONAL_EXCLUDED_TAG__: ${{ !inputs.force-run && ':mb/old-migrations-test' || '' }}
# GDGT-3126: re-enable and properly fix old migrations tests
__ADDITIONAL_EXCLUDED_TAG__: ':mb/old-migrations-test'
# actual serious env vars below
CI: 'true'
DRIVERS: mysql
@@ -125,45 +218,16 @@ jobs:
${{ matrix.job.test-args }}
:exclude-tags '[ ${{ matrix.job.exclude-tag }} ${{ env.__ADDITIONAL_EXCLUDED_TAG__ }}]'
quarantine-engine: ${{ vars.QUARANTINE_ENGINE }}
- name: Upload Test Results
uses: ./.github/actions/upload-test-results
if: always()
with:
input-path: ./target/junit/
output-name: ${{ github.job }}
bucket: ${{ vars.AWS_S3_TEST_RESULTS_BUCKET }}
role-to-assume: ${{ secrets.CI_TEST_RESULTS_IAM_ROLE }}
aws-region: ${{ vars.AWS_REGION }}
be-tests-mysql:
if: ${{ !inputs.skip }}
needs: app-db-matrix
if: ${{ !inputs.skip && (inputs.app-db == '' || inputs.app-db == 'mysql') }}
runs-on: ${{ vars.DEFAULT_RUNNER_KEY }}
timeout-minutes: 40
strategy:
fail-fast: false
matrix:
version:
- name: MySQL 8.0
junit-name: be-tests-mysql-8-0-ee
image: mysql:8.0
env:
enable-ssl-tests: 'false'
enable-aws-iam-tests: 'false'
- name: MySQL 9
junit-name: be-tests-mysql-9-ee
image: mysql:9
env:
enable-ssl-tests: 'true'
enable-aws-iam-tests: 'true'
# `mysql:latest` now resolves to MySQL 26.x, which fails in a way that floods the log
# until the runner cancels the job. A cancelled job can't be soft-failed with
# continue-on-error, so the leg is disabled until 26.x is supported.
# - name: MySQL Latest
# junit-name: be-tests-mysql-latest-ee
# image: mysql:latest
# env:
# enable-ssl-tests: 'false'
# enable-aws-iam-tests: 'false'
version: ${{ fromJSON(needs.app-db-matrix.outputs.mysql) }}
job:
- name: Enterprise Tests
build-static-viz: false
@@ -197,21 +261,9 @@ jobs:
MYSQL_ALLOW_EMPTY_PASSWORD: true
MYSQL_DATABASE: mb_test
env:
# Only run old migrations tests on pushes to master or release branches. All other branches should skip tests
# with the tag `mb/old-migrations-test`. `__ADDITIONAL_EXCLUDED_TAG__` is not used anywhere outside of splicing
# it in to the command below.
__ADDITIONAL_EXCLUDED_TAG__: >-
${{
(
github.event_name == 'push' &&
(
github.ref_name == 'master' ||
startsWith(github.ref_name, 'release-x')
) &&
''
) ||
':mb/old-migrations-test'
}}
# __ADDITIONAL_EXCLUDED_TAG__: ${{ !inputs.force-run && ':mb/old-migrations-test' || '' }}
# GDGT-3126: re-enable and properly fix old migrations tests
__ADDITIONAL_EXCLUDED_TAG__: ':mb/old-migrations-test'
# actual serious env vars below
CI: 'true'
DRIVERS: mysql
@@ -260,36 +312,16 @@ jobs:
${{ matrix.job.test-args }}
:exclude-tags '[ ${{ matrix.job.exclude-tag }} ${{ env.__ADDITIONAL_EXCLUDED_TAG__ }}]'
quarantine-engine: ${{ vars.QUARANTINE_ENGINE }}
- name: Upload Test Results
uses: ./.github/actions/upload-test-results
if: always()
with:
input-path: ./target/junit/
output-name: ${{ github.job }}
bucket: ${{ vars.AWS_S3_TEST_RESULTS_BUCKET }}
role-to-assume: ${{ secrets.CI_TEST_RESULTS_IAM_ROLE }}
aws-region: ${{ vars.AWS_REGION }}
be-tests-postgres:
if: ${{ !inputs.skip }}
needs: app-db-matrix
if: ${{ !inputs.skip && (inputs.app-db == '' || inputs.app-db == 'postgres') }}
runs-on: ${{ vars.DEFAULT_RUNNER_KEY }}
timeout-minutes: 40
strategy:
fail-fast: false
matrix:
version:
- name: Postgres 14.x
junit-name: postgres-ee
docker-image: postgres:14-alpine
env:
enable-ssl-tests: 'false'
enable-aws-iam-tests: 'false'
- name: Postgres Latest
junit-name: postgres-latest-ee
docker-image: postgres:latest
env:
enable-ssl-tests: 'true'
enable-aws-iam-tests: 'true'
version: ${{ fromJSON(needs.app-db-matrix.outputs.postgres) }}
job:
- name: Enterprise Tests
build-static-viz: false
@@ -313,21 +345,9 @@ jobs:
exclude-tag: ':mb/driver-tests :mb/transforms-python-test'
name: "${{ matrix.version.name }} ${{ matrix.job.name }}"
env:
# Only run old migrations tests on pushes to master or release branches. All other branches should skip tests
# with the tag `mb/old-migrations-test`. `__ADDITIONAL_EXCLUDED_TAG__` is not used anywhere outside of splicing
# it in to the command below.
__ADDITIONAL_EXCLUDED_TAG__: >-
${{
(
github.event_name == 'push' &&
(
github.ref_name == 'master' ||
startsWith(github.ref_name, 'release-x')
) &&
''
) ||
':mb/old-migrations-test'
}}
# __ADDITIONAL_EXCLUDED_TAG__: ${{ !inputs.force-run && ':mb/old-migrations-test' || '' }}
# GDGT-3126: re-enable and properly fix old migrations tests
__ADDITIONAL_EXCLUDED_TAG__: ':mb/old-migrations-test'
# actual serious env vars below
CI: 'true'
DRIVERS: postgres
@@ -350,7 +370,7 @@ jobs:
AWS_SECRET_ACCESS_KEY: ${{ secrets.AWS_AURORA_IAM_SECRET_ACCESS_KEY }}
services:
postgres:
image: ${{ matrix.version.docker-image }}
image: ${{ matrix.version.image }}
ports:
- "5432:5432"
env:
@@ -364,28 +384,20 @@ jobs:
uses: ./.github/actions/test-driver
with:
build-static-viz: ${{ matrix.job.build-static-viz }}
junit-name: 'be-tests-${{ matrix.version.junit-name }}'
junit-name: ${{ matrix.version.junit-name }}
artifact-suffix: ${{ matrix.job.name }}
test-args: >-
${{ matrix.job.test-args }}
:exclude-tags '[ ${{ matrix.job.exclude-tag }} ${{ env.__ADDITIONAL_EXCLUDED_TAG__ }}]'
quarantine-engine: ${{ vars.QUARANTINE_ENGINE }}
- name: Upload Test Results
uses: ./.github/actions/upload-test-results
if: always()
with:
input-path: ./target/junit/
output-name: ${{ github.job }}
bucket: ${{ vars.AWS_S3_TEST_RESULTS_BUCKET }}
role-to-assume: ${{ secrets.CI_TEST_RESULTS_IAM_ROLE }}
aws-region: ${{ vars.AWS_REGION }}
app-db-tests-result:
needs:
- app-db-matrix
- be-tests-mariadb
- be-tests-mysql
- be-tests-postgres
runs-on: ${{ vars.DEFAULT_RUNNER_KEY }}
runs-on: ${{ vars.SLIM_RUNNER_KEY }}
timeout-minutes: 5
name: app-db-tests-result
if: always() && !cancelled()

View File

@@ -7,7 +7,8 @@ on:
jobs:
update_pr:
runs-on: ${{ vars.DEFAULT_RUNNER_KEY }}
runs-on: ${{ vars.SLIM_RUNNER_KEY }}
timeout-minutes: 5
steps:
- uses: tibdex/github-app-token@3beb63f4bd073e61482598c45c71c1019b59b73a # v2.1.0

View File

@@ -99,6 +99,36 @@ jobs:
)
runs-on: ${{ vars.DEFAULT_RUNNER_KEY }}
steps:
- name: "!! double-backport and triple-backport label deprecation warning !!"
if:
|
github.event.pull_request.merged == true && (
github.event.action == 'closed' && (
contains(github.event.pull_request.labels.*.name, 'double-backport') ||
contains(github.event.pull_request.labels.*.name, 'triple-backport')
) ||
github.event.action == 'labeled' && (
github.event.label.name == 'double-backport' ||
github.event.label.name == 'triple-backport'
)
)
env:
REPO: ${{ github.repository }}
PR: ${{ github.event.pull_request.number }}
GH_TOKEN: ${{ secrets.METABASE_AUTOMATION_USER_TOKEN }}
run: |
msg="🚨🚨 Backport failed because \`double-backport\` and \`triple-backport\` labels are **deprecated**! Use the \`backport\` label to port changes to all supported releases instead. 🚨🚨"
gh api \
--silent \
--method POST \
-H "Accept: application/vnd.github+json" \
-H "X-GitHub-Api-Version: 2026-03-10" \
/repos/"$REPO"/issues/"$PR"/comments \
-f "body=$msg"
echo "::error::$msg"
exit 1
- uses: actions/checkout@v6
with:
fetch-depth: 0
@@ -110,29 +140,6 @@ jobs:
pull-request-number: ${{ github.event.pull_request.number }}
base-ref: ${{ github.event.pull_request.base.ref }}
- name: how many backports?
id: how-many-backports
uses: actions/github-script@v9
with:
script: | # js
const labels = [
...${{ toJson(github.event.pull_request.labels) }},
{ name:"${{ github.event.label.name }}" }
];
const backportLabelNames = labels.filter(label => label.name.includes('backport')).map(label => label.name);
let result = 1;
if (backportLabelNames.includes("triple-backport")) {
result = 3;
} else if (backportLabelNames.includes("double-backport")) {
result = 2;
}
console.log("backports", result);
return result;
- name: get release versions
id: get-latest-release-version
uses: actions/github-script@v9
@@ -157,21 +164,3 @@ jobs:
backport-commit: ${{ steps.find_commit.outputs.commit }}
github-token: ${{ secrets.METABASE_AUTOMATION_USER_TOKEN }}
github-token-2: ${{ secrets.GITHUB_TOKEN }}
- name: Double Backport
if: ${{ always() && fromJson(steps.how-many-backports.outputs.result) >= 2 }}
uses: ./.github/actions/create-backport
with:
target-version: ${{ fromJson(steps.get-latest-release-version.outputs.result).secondLatest }}
backport-commit: ${{ steps.find_commit.outputs.commit }}
github-token: ${{ secrets.METABASE_AUTOMATION_USER_TOKEN }}
github-token-2: ${{ secrets.GITHUB_TOKEN }}
- name: Triple Backport
if: ${{ always() && fromJson(steps.how-many-backports.outputs.result) >= 3 }}
uses: ./.github/actions/create-backport
with:
target-version: ${{ fromJson(steps.get-latest-release-version.outputs.result).thirdLatest }}
backport-commit: ${{ steps.find_commit.outputs.commit }}
github-token: ${{ secrets.METABASE_AUTOMATION_USER_TOKEN }}
github-token-2: ${{ secrets.GITHUB_TOKEN }}

View File

@@ -17,7 +17,8 @@ on:
jobs:
setup-oss:
runs-on: ${{ vars.DEFAULT_RUNNER_KEY }}
runs-on: ${{ vars.SLIM_RUNNER_KEY }}
timeout-minutes: 5
outputs:
modules: ${{ steps.get-modules.outputs.modules }}
steps:
@@ -64,7 +65,8 @@ jobs:
flags: back-end
setup-ee:
runs-on: ${{ vars.DEFAULT_RUNNER_KEY }}
runs-on: ${{ vars.SLIM_RUNNER_KEY }}
timeout-minutes: 5
outputs:
modules: ${{ steps.get-modules.outputs.modules }}
steps:

View File

@@ -6,6 +6,26 @@ on:
skip:
type: boolean
default: false
force-run:
description: Run the full suite regardless of the diff (master / release branches).
type: boolean
default: false
# Run the backend test suite on demand, against the ref you dispatch on, optionally under a
# profiler. Everything else about the jobs -- matrix legs, tag filters, quarantine gating -- is
# whatever the normal CI run uses, because they are the same jobs.
workflow_dispatch:
inputs:
tests:
description: 'Optional: run only these tests, comma- or space-separated, e.g. metabase.query-processor-test/some-test. Bare namespaces work too. Blank runs each leg''s usual set.'
type: string
required: false
default: ''
profile:
description: 'Profile the test run and attach a flamegraph to the job. Adds sampling overhead.'
type: boolean
required: false
default: false
concurrency:
group: ${{ github.workflow }}-${{ github.head_ref && github.ref || github.run_id }}-backend
@@ -15,8 +35,15 @@ concurrency:
# (secrets: inherit is already set on the backend call in run-tests.yml).
# See DEV-2224.
env:
CI_CONDUCTOR_BASE_URL: ${{ secrets.CI_CONDUCTOR_BASE_URL }}
CI_CONDUCTOR_BASE_URL: ${{ vars.CI_CONDUCTOR_BASE_URL }}
CI_CONDUCTOR_WEBHOOK_SECRET: ${{ secrets.CI_CONDUCTOR_WEBHOOK_SECRET }}
# Both read by the be-tests run step below. Set here rather than passed per job so that turning
# either on is one line. Empty (and so off) on workflow_call, where neither input exists.
#
# ONLY_TESTS replaces the leg's own test selection, so every matrix leg runs the whole list: the
# partition args no longer apply, and legs that hold none of the named tests run nothing.
PROFILE_TESTS: ${{ inputs.profile }}
ONLY_TESTS: ${{ inputs.tests }}
jobs:
be-linter-clj-kondo:
@@ -45,7 +72,7 @@ jobs:
be-linter-eastwood-test:
# Master / release safety net: lint every backend test namespace.
if: ${{ !inputs.skip && github.event_name == 'push' && (github.ref_name == 'master' || startsWith(github.ref_name, 'release-x')) }}
if: ${{ !inputs.skip && inputs.force-run }}
runs-on: ${{ vars.DEFAULT_RUNNER_KEY }}
timeout-minutes: 45
name: Eastwood (tests only)
@@ -57,8 +84,8 @@ jobs:
name: Run Eastwood linter on our backend tests
be-linter-eastwood-test-changed:
# PR / merge queue: lint only the test namespaces touched by the diff.
if: ${{ !inputs.skip && (github.event_name == 'pull_request' || github.event_name == 'merge_group') }}
# Lint only the test namespaces touched by the diff.
if: ${{ !inputs.skip && !inputs.force-run && github.event_name == 'pull_request' }}
runs-on: ${{ vars.DEFAULT_RUNNER_KEY }}
timeout-minutes: 15
name: Eastwood (changed tests only)
@@ -72,8 +99,8 @@ jobs:
- name: Compute changed test namespaces
id: changed
env:
BASE_SHA: ${{ github.event.pull_request.base.sha || github.event.merge_group.base_sha }}
HEAD_SHA: ${{ github.event.pull_request.head.sha || github.event.merge_group.head_sha || github.sha }}
BASE_SHA: ${{ github.event.pull_request.base.sha }}
HEAD_SHA: ${{ github.event.pull_request.head.sha || github.sha }}
run: |
set -euo pipefail
@@ -158,21 +185,9 @@ jobs:
:partition/index 1
name: "Java ${{ matrix.java-version }} ${{ matrix.job.name }}"
env:
# Only run old migrations tests on pushes to master or release branches. All other branches should skip tests
# with the tag `mb/old-migrations-test`. `__ADDITIONAL_EXCLUDED_TAG__` is not used anywhere outside of splicing
# it in to the command below.
__ADDITIONAL_EXCLUDED_TAG__: >-
${{
(
github.event_name == 'push' &&
(
github.ref_name == 'master' ||
startsWith(github.ref_name, 'release-x')
) &&
''
) ||
':mb/old-migrations-test'
}}
# __ADDITIONAL_EXCLUDED_TAG__: ${{ !inputs.force-run && ':mb/old-migrations-test' || '' }}
# GDGT-3126: re-enable and properly fix old migrations tests
__ADDITIONAL_EXCLUDED_TAG__: ':mb/old-migrations-test'
steps:
- uses: actions/checkout@v6
- name: Prepare front-end environment
@@ -230,7 +245,21 @@ jobs:
# whole `[ns/var ...]` vector reaches clojure -X as a single argument.
run: |
BASE="clojure -X:dev:ci:test:${{ matrix.job.edition }}:${{ matrix.job.edition }}-dev"
if [ "${{ github.run_attempt }}" != "1" ] && [ -s target/failed-tests ]; then
# `:profile-tests` overrides `:test`'s exec-fn, so it has to come last. See dev.profile-tests.
# The `profiled` output is what the collection steps below key off, so that the decision is
# recorded by the step that acts on it rather than re-derived from the environment.
if [ "$PROFILE_TESTS" = "true" ]; then
echo "Profiling this run; a flamegraph will be attached to the job."
echo "profiled=true" >> "$GITHUB_OUTPUT"
BASE="$BASE:profile-tests"
fi
# An explicit selection wins over the rerun narrowing below: it was asked for by hand, on this
# attempt. Names may be separated by commas or spaces; hawk wants them as one vector argument.
if [ -n "$ONLY_TESTS" ]; then
ONLY="[$(echo "$ONLY_TESTS" | tr ',' ' ')]"
echo "Running only: $ONLY"
$BASE :only "$ONLY"
elif [ "${{ github.run_attempt }}" != "1" ] && [ -s target/failed-tests ]; then
echo "Granular rerun of previously-failed tests: $(cat target/failed-tests)"
$BASE :only "$(cat target/failed-tests)"
else
@@ -239,8 +268,15 @@ jobs:
# Keep this step continue-on-error because the quarantine logic controls whether it ultimately fails.
continue-on-error: true
- name: Collect profile
if: ${{ !cancelled() }}
uses: ./.github/actions/collect-profile
with:
profiled: ${{ steps.run-java-tests.outputs.profiled }}
artifact-name: profile-be-tests-java-${{ matrix.java-version }}-${{ matrix.job.test-group-name }}
- name: Report failed tests to ci-conductor
if: ${{ !cancelled() && steps.run-java-tests.outcome == 'failure' }}
if: ${{ !cancelled() && ( steps.run-java-tests.outcome == 'failure' || github.run_attempt != '1' ) }}
# This step should NEVER fail the job!
continue-on-error: true
uses: ./.github/actions/report-failures-to-ci-conductor
@@ -281,16 +317,6 @@ jobs:
test-suite: be-tests-java-${{ matrix.java-version }}-${{ matrix.job.test-group-name }}
enforce-quarantine: ${{ vars.QUARANTINE_ENGINE == 'ci-conductor' }}
- name: Upload Test Results
uses: ./.github/actions/upload-test-results
if: always()
with:
input-path: ./target/junit/
output-name: be-tests-java-${{ matrix.java-version }}-${{ matrix.job.edition }}
bucket: ${{ vars.AWS_S3_TEST_RESULTS_BUCKET }}
role-to-assume: ${{ secrets.CI_TEST_RESULTS_IAM_ROLE }}
aws-region: ${{ vars.AWS_REGION }}
- name: Publish Test Report (JUnit)
uses: dorny/test-reporter@a43b3a5f7366b97d083190328d2c652e1a8b6aa2 # v3.0.0
if: failure()
@@ -326,7 +352,7 @@ jobs:
uses: ./.github/actions/prepare-backend
# TODO -- we should probably also check WITHOUT ee -- do a check with only OSS namespaces on the classpath
- name: Check namespaces
run: clojure -M:ee:drivers:check
run: clojure -M:ee:drivers:load-namespaces
be-cljfmt:
if: ${{ !inputs.skip }}
@@ -350,7 +376,7 @@ jobs:
- be-check
- be-tests
if: always() && !cancelled()
runs-on: ${{ vars.DEFAULT_RUNNER_KEY }}
runs-on: ${{ vars.SLIM_RUNNER_KEY }}
timeout-minutes: 5
env:
needs: ${{ toJson(needs) }}

View File

@@ -12,7 +12,8 @@ concurrency:
jobs:
flags:
runs-on: ${{ vars.DEFAULT_RUNNER_KEY }}
runs-on: ${{ vars.SLIM_RUNNER_KEY }}
timeout-minutes: 5
outputs:
has_backport_label: ${{ steps.c.outputs.value }}
steps:
@@ -27,7 +28,7 @@ jobs:
files-changed:
name: Check which files changed
runs-on: ${{ vars.DEFAULT_RUNNER_KEY }}
runs-on: ${{ vars.SLIM_RUNNER_KEY }}
timeout-minutes: 3
outputs:
docs: ${{ steps.changes.outputs.docs }}
@@ -44,7 +45,7 @@ jobs:
needs: [ files-changed, flags ]
if: ${{ !cancelled() }}
name: Decide whether to backport or not
runs-on: ${{ vars.DEFAULT_RUNNER_KEY }}
runs-on: ${{ vars.SLIM_RUNNER_KEY }}
timeout-minutes: 5
permissions: {}
steps:
@@ -59,7 +60,7 @@ jobs:
priority-backport-reminder:
name: Remind about backporting P0 + P1 fixes
runs-on: ${{ vars.DEFAULT_RUNNER_KEY }}
runs-on: ${{ vars.SLIM_RUNNER_KEY }}
timeout-minutes: 5
needs: [flags]
permissions:

View File

@@ -5,10 +5,11 @@ on:
jobs:
check-source-branch:
runs-on: ${{ vars.DEFAULT_RUNNER_KEY }}
if: github.head_ref == 'master'
runs-on: ${{ vars.SLIM_RUNNER_KEY }}
timeout-minutes: 5
steps:
- name: Block master branch PRs
if: github.head_ref == 'master'
run: |
echo "PRs from \`master\` branches are not allowed, please use a new branch name for your PR."
exit 1

View File

@@ -1,273 +0,0 @@
# Builds a fresh release JAR from scratch at a given commit.
#
# You should almost never have to use this. The normal release flow relies on a
# commit that has already been built and tested on its branch — see
# `tag-for-release.yml`, which tags that existing JAR. Building from scratch
# here is only for exceptional cases.
name: Release 0 - Build Release Artifact
run-name: Build ${{ inputs.version }}
on:
workflow_dispatch:
inputs:
version:
description: 'Metabase version (e.g. v0.46.3)'
type: string
required: true
commit:
description: 'A full-length commit SHA-1 hash'
required: true
jobs:
start-message:
runs-on: ${{ vars.DEFAULT_RUNNER_KEY }}
timeout-minutes: 5
steps:
- uses: actions/checkout@v6
with:
sparse-checkout: |
release
.github
- name: Prepare release scripts
uses: ./.github/actions/prepare-release-scripts
- name: Send build start message
uses: actions/github-script@v9
env:
SLACK_RELEASE_CHANNEL: ${{ vars.SLACK_RELEASE_CHANNEL }}
SLACK_BOT_TOKEN: ${{ secrets.SLACK_BOT_TOKEN }}
with:
script: | # js
const { sendPreReleaseMessage } = require('${{ github.workspace }}/release/dist/index.cjs');
await sendPreReleaseMessage({
github,
owner: context.repo.owner,
repo: context.repo.repo,
version: '${{ inputs.version }}',
runId: '${{ github.run_id }}',
releaseSha: '${{ inputs.commit }}',
userName: '${{ github.actor }}',
channelName: '${{ vars.SLACK_RELEASE_CHANNEL }}',
}).catch(console.error);
trigger-milestone-check:
# Posts a Slack FYI about issues that still need review for this release.
# Informational only — nothing depends on it, so it never blocks the release.
needs: [check-version, check-commit]
permissions:
contents: read
issues: read
uses: ./.github/workflows/release-milestone-check.yml
secrets: inherit
with:
version: ${{ inputs.version }}
commit: ${{ inputs.commit }}
check-version:
runs-on: ${{ vars.DEFAULT_RUNNER_KEY }}
timeout-minutes: 10
outputs:
ee: ${{ fromJson(steps.canonical_version.outputs.result).ee }}
oss: ${{ fromJson(steps.canonical_version.outputs.result).oss }}
steps:
- name: Fail early on the incorrect version format
if: ${{ !(startsWith(inputs.version,'v0.') || startsWith(inputs.version,'v1.')) }}
run: |
echo "The version format is invalid!"
echo "It must start with either 'v0.' or 'v1.'."
echo "Please, try again."
exit 1
- uses: actions/checkout@v6
with:
sparse-checkout: |
release
.github
- name: Prepare release scripts
uses: ./.github/actions/prepare-release-scripts
- name: Get Release Version
uses: actions/github-script@v9
id: canonical_version
with:
script: | # js
const { isValidVersionString, getCanonicalVersion, hasBeenReleased } = require('${{ github.workspace }}/release/dist/index.cjs');
const version = '${{ inputs.version }}';
if (!isValidVersionString(version)) {
throw new Error("The version format is invalid! It must start with either 'v0.' or 'v1.'.");
}
const versions = {
ee: getCanonicalVersion(version, 'ee'),
oss: getCanonicalVersion(version, 'oss'),
};
const released = await hasBeenReleased({
github,
owner: context.repo.owner,
repo: context.repo.repo,
version: versions.oss,
});
if (released) {
throw new Error("This version has already been released!", version);
}
return versions;
check-commit:
runs-on: ${{ vars.DEFAULT_RUNNER_KEY }}
timeout-minutes: 10
steps:
- name: Check out the code to verify the release branch
uses: actions/checkout@v6
with:
fetch-depth: 0 # IMPORTANT! to get all the branches
sparse-checkout: |
release
.github
- name: Prepare release scripts
uses: ./.github/actions/prepare-release-scripts
- name: Get Release Branch
uses: actions/github-script@v9
id: release_branch
with:
result-encoding: string
script: | # js
const { getReleaseBranch } = require('${{ github.workspace }}/release/dist/index.cjs');
const version = '${{ inputs.version }}';
const releaseBranch = getReleaseBranch(version);
return releaseBranch;
- name: Ensure that the specified commit exists in the ${{ steps.release_branch.outputs.result }} release branch
run: | # bash
RELEASE_BRANCH=${{ steps.release_branch.outputs.result }}
git checkout "$RELEASE_BRANCH"
if git branch --contains ${{ inputs.commit }} | grep -q "$RELEASE_BRANCH"; then
echo "Commit found in correct release branch"
else
echo "Commit not found in correct release branch"
exit 1
fi
git checkout -
get-build-requirements:
runs-on: ${{ vars.DEFAULT_RUNNER_KEY }}
timeout-minutes: 10
outputs:
java_version: ${{ fromJson(steps.dependencies.outputs.result).java_version }}
node_version: ${{ fromJson(steps.dependencies.outputs.result).node_version }}
build_process: ${{ fromJson(steps.dependencies.outputs.result).build_process }}
steps:
- uses: actions/checkout@v6
with:
sparse-checkout: |
release
.github
- name: Prepare release scripts
uses: ./.github/actions/prepare-release-scripts
- name: Get build dependencies
uses: actions/github-script@v9
id: dependencies
with:
script: | # js
const { getBuildRequirements, getMajorVersion } = require('${{ github.workspace }}/release/dist/index.cjs');
const version = '${{ inputs.version }}';
const requirements = getBuildRequirements(version);
const build_process = getMajorVersion(version) < 46 ? 'legacy' : 'new';
console.log('Build process:', build_process);
return {
java_version: requirements.java,
node_version: requirements.node,
build_process,
};
manage-milestones:
# Close the milestone we're about to ship and open the next one BEFORE building,
# so any PRs that merge while the build is in flight are tagged with the next
# milestone (not the one about to be released).
#
# Sequenced after trigger-milestone-check so that the check sees the milestone
# in its open state. milestone-check is informational and must never block, so
# we run regardless of its outcome; only preflight failures suppress this.
needs: [check-version, check-commit, trigger-milestone-check]
if: ${{ !cancelled() && needs.check-version.result == 'success' && needs.check-commit.result == 'success' }}
permissions:
contents: read
issues: write
uses: ./.github/workflows/release-milestone-manage.yml
with:
version: ${{ inputs.version }}
build-uberjar-for-release:
needs: [check-version, check-commit, get-build-requirements]
runs-on: ${{ vars.DEFAULT_RUNNER_KEY }}
timeout-minutes: 50
strategy:
matrix:
edition: [oss, ee]
env:
INTERACTIVE: false
steps:
- name: Get correct version number
run: | # bash
if [ '${{matrix.edition}}' == 'ee' ]; then
version='${{ needs.check-version.outputs.ee }}'
else
version='${{ needs.check-version.outputs.oss }}'
fi
echo "VERSION=$version" >> "$GITHUB_ENV"
- name: Setup Java ${{ needs.get-build-requirements.outputs.java_version }}
uses: actions/setup-java@v5.2.0
with:
distribution: 'temurin'
java-version: ${{ needs.get-build-requirements.outputs.java_version }}
- name: Setup Node.js ${{ needs.get-build-requirements.outputs.node_version }}
uses: actions/setup-node@v6.4.0
with:
node-version: ${{ needs.get-build-requirements.outputs.node_version }}.x
- name: Check out the code
uses: actions/checkout@v6
with:
ref: ${{ github.event.inputs.commit }}
- name: Prepare front-end environment
uses: ./.github/actions/prepare-frontend
- name: Prepare back-end environment
uses: ./.github/actions/prepare-backend
- name: Build Metabase
if: ${{ needs.get-build-requirements.outputs.build_process != 'legacy' }}
run: ./bin/build.sh :edition :${{ matrix.edition }} :version "$VERSION"
- name: Build Metabase (legacy)
if: ${{ needs.get-build-requirements.outputs.build_process == 'legacy' }}
run: cd ./bin/build-mb && clojure -X build/build! :edition :${{ matrix.edition }} :version "$VERSION"
- name: Store commit's SHA-1 hash
run: echo ${{ inputs.commit }} > COMMIT-ID
shell: bash
- run: mv ./target/uberjar/metabase.jar ./metabase.jar
- name: Calculate SHA256 checksum
run: sha256sum ./metabase.jar > SHA256.sum
shell: bash
- name: Upload JARs as artifact
uses: actions/upload-artifact@v7
with:
name: metabase-release-${{ matrix.edition }}-${{ inputs.commit }}-uberjar
path: |
./metabase.jar
./COMMIT-ID
./SHA256.sum
run-pre-release-tests:
name: Run pre-release tests for ${{ inputs.version }}
needs: build-uberjar-for-release
uses: ./.github/workflows/pre-release.yml
secrets: inherit
with:
commit: ${{ inputs.commit }}
version: ${{ inputs.version }}

View File

@@ -0,0 +1,140 @@
name: Build & push private Metabase Docker image
# Builds the ee uberjar for a commit and pushes it to the private ECR repo, tagged with that
# commit's full SHA. Callers get the tag back as an output.
#
# The tag is per-commit rather than per-PR, so a PR leaves one image behind per pushed commit.
# Nothing in this repo prunes them — retention is whatever lifecycle policy the ECR repo carries.
on:
workflow_dispatch:
inputs:
commit:
description: Full sha of commit to build, optional
workflow_call:
inputs:
commit:
description: Full sha of commit to build, optional
type: string
secrets:
PR_ENV_IAM_ROLE:
description: Role assumed via OIDC to reach ECR
required: true
PR_ENV_AWS_ACCOUNT_ID:
description: Account hosting the private ECR registry
required: true
outputs:
tag:
description: "Image tag (the commit SHA)"
value: ${{ jobs.dockerize.outputs.tag }}
# Keyed on the calling workflow as well as the commit: pr-env and ai-benchmark both land here, and
# a shared group lets a re-label on one evict the other's queued run (`cancel-in-progress: false`
# still drops a *pending* run when a third queues). Before the extraction the two paths were
# independent; this keeps them that way.
concurrency:
group: build-private-ee-docker-image-${{ github.workflow }}-${{ inputs.commit || github.event.pull_request.head.sha || github.sha }}
cancel-in-progress: false
permissions:
contents: read
actions: read # fetch-artifact reads the artifact API
jobs:
build:
if: github.event_name == 'workflow_dispatch' || github.event.pull_request.head.repo.full_name == github.repository
uses: ./.github/workflows/uberjar.yml
with:
# Pass through only an *explicit* commit and let uberjar apply its own event fallback.
# Resolving head.sha here instead would pin the PR path's checkout to the head commit, while
# run-tests' uberjar keeps building the merge ref — both upload under the same
# `metabase-ee-<head.sha>-uberjar` name, so whichever raced first would decide what the tag
# actually contains.
commit: ${{ inputs.commit }}
dockerize:
runs-on: ${{ vars.DEFAULT_RUNNER_KEY }}
needs: [build]
if: needs.build.result == 'success'
timeout-minutes: 60
permissions:
id-token: write
contents: read
actions: read
outputs:
tag: ${{ steps.meta.outputs.tag }}
steps:
# Deliberately the workflow's own revision, not `inputs.commit`: this checkout is where
# `./.github/actions/fetch-artifact` below comes from, and pinning it to the target commit
# would run that commit's copy of the action instead of this one. Nothing else in this job
# reads the tree.
- name: Checkout source code
uses: actions/checkout@v6
- name: Configure AWS credentials
uses: aws-actions/configure-aws-credentials@d979d5b3a71173a29b74b5b88418bfda9437d885 # v6.1.1
with:
role-to-assume: ${{ secrets.PR_ENV_IAM_ROLE }}
role-session-name: GitHub_to_AWS_via_FederatedOIDC
aws-region: us-east-1
- name: Login to Amazon ECR
uses: aws-actions/amazon-ecr-login@183a1442edf41672e66566b7fc560e297a290896 # v2
with:
registries: "${{ secrets.PR_ENV_AWS_ACCOUNT_ID }}"
- name: Compute image tag
id: meta
env:
SHA: ${{ inputs.commit || github.event.pull_request.head.sha || github.sha }}
run: echo "tag=${SHA}" >> "$GITHUB_OUTPUT"
- name: Check whether image already exists
id: exists
env:
TAG: ${{ steps.meta.outputs.tag }}
# Same registry the push below targets — the assumed role's default registry is not
# necessarily this account, and querying the wrong one always reports "not found".
REGISTRY_ID: ${{ secrets.PR_ENV_AWS_ACCOUNT_ID }}
run: |
set +e
OUT=$(aws ecr describe-images \
--registry-id "${REGISTRY_ID}" \
--repository-name metabase-enterprise \
--image-ids imageTag="${TAG}" 2>&1)
STATUS=$?
set -e
if [[ "${STATUS}" -eq 0 ]]; then
echo "Image already present for ${TAG}; reusing it."
echo "found=true" >> "$GITHUB_OUTPUT"
elif grep -q 'ImageNotFoundException' <<< "${OUT}"; then
echo "No image for ${TAG}; building."
echo "found=false" >> "$GITHUB_OUTPUT"
else
# Anything else (missing ecr:DescribeImages, wrong registry, throttling) is
# indistinguishable from "not found" if we swallow it, and we'd rebuild forever
# without ever saying why. Rebuilding is still the safe answer, so warn and go on.
echo "${OUT}" >&2
echo "::warning::Could not determine whether ${TAG} exists in ECR (exit ${STATUS}); building unconditionally."
echo "found=false" >> "$GITHUB_OUTPUT"
fi
- name: Retrieve uberjar artifact for ee
if: steps.exists.outputs.found == 'false'
uses: ./.github/actions/fetch-artifact
with:
commit: ${{ steps.meta.outputs.tag }}
edition: ee
extract-path: "target/uberjar"
output-name: "metabase.jar"
- name: Move uberjar to bin/docker
if: steps.exists.outputs.found == 'false'
run: mv target/uberjar/metabase.jar bin/docker/metabase.jar
- name: Build container
if: steps.exists.outputs.found == 'false'
uses: docker/build-push-action@f9f3042f7e2789586610d6e8b85c8f03e5195baf # v7.2.0
with:
context: bin/docker/
platforms: linux/amd64
network: host
tags: ${{ secrets.PR_ENV_AWS_ACCOUNT_ID }}.dkr.ecr.us-east-1.amazonaws.com/metabase-enterprise:${{ steps.meta.outputs.tag }}
push: true

View File

@@ -9,6 +9,74 @@ on:
jobs:
files-changed:
name: Check which files changed
runs-on: ${{ vars.DEFAULT_RUNNER_KEY }}
timeout-minutes: 3
permissions:
contents: read
outputs:
embedder_plugin: ${{ steps.changes.outputs.embedder_plugin }}
steps:
- uses: actions/checkout@v6
- name: Test which files changed
uses: dorny/paths-filter@fbd0ab8f3e69293af611ebaee6363fc25e6d187d # v4.0.1
id: changes
with:
token: ${{ github.token }}
filters: .github/file-paths.yaml
test-embedder-plugin:
needs: files-changed
if: needs.files-changed.outputs.embedder_plugin == 'true'
runs-on: ${{ vars.DEFAULT_RUNNER_KEY }}
timeout-minutes: 20
permissions:
contents: read
services:
postgres:
image: pgvector/pgvector:pg17
env:
POSTGRES_USER: mb_test
POSTGRES_DB: mb_test
POSTGRES_HOST_AUTH_METHOD: trust
ports:
- 5432:5432
steps:
- uses: actions/checkout@v6
- name: Prepare back-end environment
uses: ./.github/actions/prepare-backend
- name: Build embedder plugin
run: ./bin/build-embedder-plugin.sh
- name: Verify packaged plugin resources
run: |
jar tf modules/embedder/target/metabase-embedder-plugin.jar | grep -Fx metabase/embedder/metabase-plugin.yaml
jar tf modules/embedder/target/metabase-embedder-plugin.jar | grep -Fx metabase/embedder-model-catalog.edn
jar tf modules/embedder/target/metabase-embedder-plugin.jar | grep -Fx metabase-embedder/snowflake-arctic-embed-xs-d8c86521100d3556476a063fc2342036d45c106f-arm64.zip
jar tf modules/embedder/target/metabase-embedder-plugin.jar | grep -Fx metabase-embedder/snowflake-arctic-embed-xs-d8c86521100d3556476a063fc2342036d45c106f-avx2.zip
jar tf modules/embedder/target/metabase-embedder-plugin.jar | grep -Fx metabase-embedder/all-minilm-l6-v2-1110a243fdf4706b3f48f1d95db1a4f5529b4d41-arm64.zip
jar tf modules/embedder/target/metabase-embedder-plugin.jar | grep -Fx metabase-embedder/all-minilm-l6-v2-1110a243fdf4706b3f48f1d95db1a4f5529b4d41-avx2.zip
- name: Run artifact-only plugin smoke test
env:
MB_EMBEDDER_ARTIFACT_TEST: "true"
MB_PLUGINS_DIR: modules/embedder/target
run: >-
./bin/test-agent --oss :only
'[metabase.embeddings.embedder-plugin-artifact-test]'
- name: Run in-process semantic-search round trip
env:
MB_DB_TYPE: postgres
MB_DB_PORT: 5432
MB_DB_HOST: localhost
MB_DB_DBNAME: mb_test
MB_DB_USER: mb_test
MB_APPDB_PGVECTOR_MODE_TEST: "true"
MB_IN_PROCESS_EMBEDDER_SEMANTIC_SEARCH_TEST: "true"
MB_PLUGINS_DIR: modules/embedder/target
run: >-
./bin/test-agent --drivers=postgres :only
'[metabase-enterprise.semantic-search.appdb-pgvector-mode-test/appdb-pgvector-in-process-embedder-round-trip-test]'
test-build-scripts:
runs-on: ${{ vars.DEFAULT_RUNNER_KEY }}
steps:
@@ -24,3 +92,7 @@ jobs:
run: clojure -M:test
working-directory: bin/lint-migrations-file
timeout-minutes: 15
- name: Run load-namespaces tests
run: clojure -M:test
working-directory: bin/load-namespaces
timeout-minutes: 15

View File

@@ -2,7 +2,7 @@ name: Bundle Size Stats
# Reuses the uberjar built on every master merge: downloads the jar (for the
# emitted JS) plus the bundle-stats and embedding-sdk artifacts, measures
# initial/total bundle sizes, and appends them to stats.metabase.com.
# initial/total bundle sizes, and appends them to eng-stats-importer.
on:
workflow_run:
@@ -32,6 +32,7 @@ jobs:
uses: actions/checkout@v6
with:
ref: ${{ github.event.workflow_run.head_sha }}
- uses: ./.github/actions/prepare-node-bun
- name: Download EE uberjar
uses: ./.github/actions/fetch-artifact
with:
@@ -104,42 +105,15 @@ jobs:
# Passed via env (not interpolated into the shell) so the commit
# message can't inject commands; the script keeps only the subject.
COMMIT_MESSAGE: ${{ github.event.workflow_run.head_commit.message }}
run: node ./.github/scripts/bundle-size-stats-prepare.js
- name: Upload sizes (with deltas) to Stats
run: bun ./.github/scripts/bundle-size-stats-prepare.ts
- name: Import sizes (with deltas) into Stats
id: upload
if: steps.prepare.outputs.significant == 'true'
uses: actions/github-script@v9
env:
API_KEY: ${{ secrets.STATS_API_KEY }}
with:
script: | # js
const fs = require("fs");
const { uploadCsvToMb } = require("${{ github.workspace }}/.github/scripts/csv-to-mb.js");
const data = JSON.parse(
fs.readFileSync("${{ github.workspace }}/artifacts/upload-rows.json", "utf8"),
);
console.table(data);
try {
await uploadCsvToMb({
baseUrl: "https://stats.metabase.com",
tableId: 79702,
jsonData: data,
mode: "append",
});
console.log("Bundle sizes uploaded successfully");
core.setOutput("uploaded", "true");
} catch (error) {
// Stats logging is best-effort: if stats.metabase.com is still
// unreachable after the client's retries, keep the run green. The
// last-plotted cache is only saved on a successful upload (below),
// so this point is re-attempted on the next master build and
// nothing is silently dropped.
core.warning(`Bundle-size upload failed after retries; leaving run green: ${error.message}`);
core.setOutput("uploaded", "false");
}
ROWS: artifacts/upload-rows.json
ENG_STATS_URL: ${{ secrets.ENG_STATS_URL }}
API_KEY: ${{ secrets.ENG_STATS_API_KEY }}
run: bun .github/scripts/upload-bundle-size-stats.ts
- name: Remember this as the last plotted point
if: steps.upload.outputs.uploaded == 'true'
uses: actions/cache/save@v4

View File

@@ -7,11 +7,6 @@ name: Bundle Size
on:
workflow_call:
inputs:
skip:
description: Whether to skip the check (e.g. no front-end sources changed, or the uberjar failed to build).
required: true
type: boolean
concurrency:
group: bundle-size-${{ github.workflow }}-${{ github.head_ref && github.ref || github.run_id }}
@@ -20,7 +15,6 @@ concurrency:
jobs:
bundle-size-check:
name: "Bundle size check"
if: ${{ !inputs.skip }}
continue-on-error: true
runs-on: ${{ vars.DEFAULT_RUNNER_KEY }}
timeout-minutes: 20
@@ -32,7 +26,6 @@ jobs:
id: compare
with:
edition: ee
base-ref-commit: ${{ github.event.pull_request.base.sha }}
threshold-percent: ${{ env.BUNDLE_SIZE_THRESHOLD_PERCENT }}
- name: Comment on PR if app bundle size has increased

View File

@@ -1,83 +0,0 @@
# CodeQL's overlay analysis saves a ~400MB "base database" cache on every master run, keyed by commit SHA.
#
# Only the newest entry is ever restored: CodeQL looks its cache up by key *prefix* and takes the most recent match,
# so everything behind it is dead weight. This job keeps the newest few and deletes the rest.
name: CodeQL Cache Cleanup
on:
schedule:
- cron: "0 4 * * *" # daily at 04:00 UTC
workflow_dispatch:
inputs:
keep:
description: "How many of the newest CodeQL caches to keep"
required: false
default: "2"
dry-run:
description: "List what would be deleted without deleting it"
type: boolean
required: false
default: false
concurrency:
group: cache-cleanup-codeql
cancel-in-progress: false
permissions:
actions: write # required to delete cache entries
contents: read
jobs:
cleanup:
# don't run the schedule on forks
if: ${{ github.event_name != 'schedule' || github.repository == 'metabase/metabase' }}
runs-on: ${{ vars.DEFAULT_RUNNER_KEY }}
timeout-minutes: 10
env:
GH_TOKEN: ${{ github.token }}
KEEP: ${{ github.event.inputs.keep || '2' }}
DRY_RUN: ${{ github.event.inputs.dry-run || 'false' }}
PREFIX: codeql-overlay-base-database-
steps:
- name: Delete superseded CodeQL caches
shell: bash
run: |
set -euo pipefail
# created_at is ISO-8601, so a reverse lexicographic sort is newest-first.
gh api --paginate "/repos/${GITHUB_REPOSITORY}/actions/caches?per_page=100" \
--jq ".actions_caches[] | select(.key | startswith(\"${PREFIX}\")) | \"\(.created_at)\t\(.id)\t\(.size_in_bytes)\t\(.key)\"" \
| sort -r > all.tsv || true
found=$(wc -l < all.tsv | tr -d ' ')
echo "Found $found cache(s) matching '${PREFIX}'; keeping the newest $KEEP."
if [ "$found" -eq 0 ]; then
echo "Nothing to do." >> "$GITHUB_STEP_SUMMARY"
exit 0
fi
tail -n +$((KEEP + 1)) all.tsv > doomed.tsv || true
doomed=$(wc -l < doomed.tsv | tr -d ' ')
freed=0
while IFS=$'\t' read -r created id size key; do
[ -z "${id:-}" ] && continue
echo " deleting $key ($((size / 1000000))MB, created $created)"
if [ "$DRY_RUN" = "true" ]; then
echo " (dry run — not deleted)"
else
# A cache can vanish between listing and deleting (concurrent eviction); don't fail the job over it.
gh api --method DELETE "/repos/${GITHUB_REPOSITORY}/actions/caches/${id}" >/dev/null \
|| echo " (already gone, skipping)"
fi
freed=$((freed + size))
done < doomed.tsv
{
echo "### CodeQL cache cleanup"
echo ""
echo "- matched: **$found**"
echo "- kept (newest): **$KEEP**"
echo "- deleted: **$doomed**"
echo "- reclaimed: **$((freed / 1000000)) MB**${DRY_RUN:+ (dry run: $DRY_RUN)}"
} >> "$GITHUB_STEP_SUMMARY"

View File

@@ -1,173 +0,0 @@
# This workflow is the single source of truth for generating and updating various caches used by our CI pipeline.
# It sets the fresh cache each Monday but provides a manual trigger to update it on-demand.
#
# All other jobs only restore from this cache; *they never update it*!
# They should always use `actions/cache/restore@vX` because `actions/cache@vX` would attempt to save the cache.
#
# Our agreed-upon strategy is to use a deterministic cache key based on the week of the year.
# Jobs can utilize the prior week's cache if the current week's cache is not found.
#
#
# PRINCIPLES:
# - The cache is never wrong!
# - Caches are immutable once created.
# - Caches are for speeding up builds, not for correctness.
# - It is faster to download a large cache than to re-resolve missing dependencies.
#
#
# DESIGN:
#
# 1. Prepare
# - A job first prepares the local environment (e.g., installs JDK, Clojure, Node.js, etc.)
# - This fetches the dependencies from the cache (if it exists) and warms up the local cache for this workflow.
# 2. Resolve
# - The next step pre-resolves all dependencies, ensuring that the local cache is fully populated and up-to-date.
# 3. Update
# - The final step saves the local cache back to GitHub Actions cache with a deterministic key.
# - If a cache entry already exists for that key, it is deleted first to ensure immutability.
#
name: Cache Generator
on:
schedule:
- cron: "0 3 * * 1" # Mondays at 03:00 UTC
workflow_dispatch: # manually update the cache
concurrency:
group: cache-generator
cancel-in-progress: true
jobs:
get-cache-keys:
if: ${{ github.event_name != 'schedule' || github.repository == 'metabase/metabase' }}
runs-on: ${{ vars.DEFAULT_RUNNER_KEY }}
steps:
- uses: actions/checkout@v6
- name: Get cache keys
uses: ./.github/actions/get-cache-keys
id: get-cache-keys
outputs:
m2-cache-key: ${{ steps.get-cache-keys.outputs.m2-cache-key }}
node-cache-key: ${{ steps.get-cache-keys.outputs.node-cache-key }}
eslint-cache-key: ${{ steps.get-cache-keys.outputs.eslint-cache-key }}
cypress-cache-key: ${{ steps.get-cache-keys.outputs.cypress-cache-key }}
generate-caches:
runs-on: ${{ vars.DEFAULT_RUNNER_KEY }}
timeout-minutes: 30
needs: get-cache-keys
env:
M2_CACHE_KEY: ${{ needs.get-cache-keys.outputs.m2-cache-key }}
NODE_CACHE_KEY: ${{ needs.get-cache-keys.outputs.node-cache-key }}
CYPRESS_CACHE_KEY: ${{ needs.get-cache-keys.outputs.cypress-cache-key }}
ESLINT_CACHE_KEY: ${{ needs.get-cache-keys.outputs.eslint-cache-key }}
steps:
- uses: actions/checkout@v6
# Backend preparation and cache
- name: Prepare backend
uses: ./.github/actions/prepare-backend
# Pre-resolve every Clojure dependency classpath CI actually uses, so the
# saved cache is a SUPERSET of what each job resolves at runtime.
#
# Why a single `-P` is not enough: tools.deps selects dependency *versions*
# globally per classpath, so different alias combinations can resolve the
# same transitive lib to different versions (e.g. `org.clojure/data.json`
# 2.4.0 vs 2.5.x, `org.ow2.asm/asm` 9.2 vs newer). Resolving only one combo
# leaves the others to fetch their version-deltas straight from Maven
# Central / Clojars on every run -> intermittent 403s. See DEV-2094.
#
# Keep this list in sync with the `clojure` invocations across .github/.
# Each entry notes its consumer. We resolve the FULL alias strings (not a
# hand-reduced set) on purpose: if a currently dep-less alias later gains a
# dependency, this still covers it instead of silently drifting back into a
# cache gap.
- name: Pre-resolve Clojure dependencies
shell: bash
run: |
set -euo pipefail
combos=(
":build:dev:ee:ee-dev:drivers:drivers-dev:cljs" # build + uberjar (combined classpath)
":cljs" # shadow-cljs (build:cljs); resolves :cljs alone -> covers asm:9.2
":dev:ee:ee-dev:drivers:drivers-dev:test:eastwood" # backend.yml (eastwood + eastwood/test)
":dev:ci:ee:ee-dev:drivers:drivers-dev:test" # test-driver.yml (ee) -> covers data.json:2.4.0
":dev:ci:oss:oss-dev:drivers:drivers-dev:test" # test-driver.yml (oss)
":dev:ci:test:ee:ee-dev" # backend.yml matrix (ee) + generative-query-test.yml
":dev:ci:test:oss:oss-dev" # backend.yml matrix (oss)
":dev:ci:ee:ee-dev:test:cloverage" # backend-cloverage.yml
":dev:drivers:drivers-dev:test" # reset-cloud-test-dataset.yml + presto script
":dev:drivers:build:build-dev:build-test:ci" # build-scripts.yml
":ee:drivers:check" # backend.yml (check; git dep athos/clj-check)
":ee:doc" # docs-generate-base.yml
":kondo" # resolve-backport-conflicts.yml (replace-deps)
":run:ee" # cross-branch-migration-test
":dev:ee:migrate" # cross-branch-migration-test
)
for combo in "${combos[@]}"; do
echo "::group::clojure -P -A${combo}"
clojure -P -A"${combo}"
echo "::endgroup::"
done
- name: Update M2 cache
uses: ./.github/actions/update-cache
with:
path: |
~/.m2
~/.gitlibs
key: ${{ env.M2_CACHE_KEY }}
# Frontend preparation and caches
# `prepare-frontend` action (1) prepares Node.js and (2) fetches the node_modules cache if it exists.
# It then (3) runs `bun install --frozen-lockfile` which ensures the local cache is fully populated.
# IMPORTANT: Caching node_modules after patches are already applied will break any PR that attempts to either:
# - upgrade a patched library
# - change a patch
# Hence why we explicitly exclude patches from the cache (controlled by the `apply-patches` action input).
- name: Install front-end dependencies
uses: ./.github/actions/prepare-frontend
with:
apply-patches: "false"
- name: Get bun cache directory
id: bun-cache
run: |
BUN_CACHE_DIR=$(bun pm cache)
echo "Bun cache directory: $BUN_CACHE_DIR"
echo "dir=$BUN_CACHE_DIR" >> "$GITHUB_OUTPUT"
- name: Ensure that Cypress executable is ready
run: |
bun run cypress install
bun run cypress cache path
bun run cypress cache list
bun run cypress verify
shell: bash
- name: Update node_modules cache
uses: ./.github/actions/update-cache
with:
path: |
${{ steps.bun-cache.outputs.dir }}
node_modules
key: ${{ env.NODE_CACHE_KEY }}
- name: Update Cypress cache
uses: ./.github/actions/update-cache
with:
path: ~/.cache/Cypress
key: ${{ env.CYPRESS_CACHE_KEY }}
# ESLint cache (requires both backend and frontend)
- name: Compile CLJS
run: bun run build-pure:cljs
- name: Run ESLint to populate cache
run: bun run lint-eslint-pure
- name: Update eslint cache
uses: ./.github/actions/update-cache
with:
path: ${{ github.workspace}}/.eslintcache
key: ${{ env.ESLINT_CACHE_KEY }}

172
.github/workflows/cache-warm.yml vendored Normal file
View File

@@ -0,0 +1,172 @@
# Writes the three large dependency caches. Every other job restores them and none of them writes.
#
# A single writer rather than the usual let-every-job-save arrangement, because a pull request here fans
# out to roughly 200 jobs, ~75 of which install Cypress. Letting each of those save would have them all
# build a 200MB archive and race for one reservation on every lockfile change, and all pull the Cypress
# binary at once on a version bump.
#
# The ESLint cache is the exception and saves from the lint job itself: it is under a megabyte and only a
# couple of jobs write it, so there is no herd to avoid.
#
# A cache is readable from the ref that saved it, from that ref's pull requests, and - for every ref -
# from the default branch. Warming on push to each merge target therefore serves the pull requests aimed
# at it, and master's copy backs up every ref that has none of its own.
#
# Keys are content-derived (see .github/scripts/cache-keys.sh), so a push whose dependencies already have
# an entry does nothing but three lookups. There is no cron: nothing here is keyed by date.
#
#
# PRINCIPLES:
# - The cache is never wrong!
# - Caches are immutable once created.
# - Caches are for speeding up builds, not for correctness.
# - It is faster to download a large cache than to re-resolve missing dependencies.
name: Cache Warm
on:
push:
branches:
- master
- "release-x.*.x"
workflow_dispatch:
concurrency:
group: cache-warm-${{ github.ref }}
# Pushes to master land a median of 27 minutes apart and a full resolve takes longer, so cancelling in
# progress would abort most warms before they save and a ref could stay cold indefinitely. Queueing
# costs one wait instead: the queued run finds every key present and exits in about a minute.
cancel-in-progress: false
jobs:
warm:
# Forks read the upstream default branch's caches and cannot usefully write their own, so warming
# there would burn a runner for nothing.
if: ${{ github.repository == 'metabase/metabase' }}
runs-on: ${{ vars.DEFAULT_RUNNER_KEY }}
timeout-minutes: 30
steps:
- uses: actions/checkout@v6
- name: Cache spec
id: cache-spec
run: .github/scripts/cache-keys.sh | tee -a "$GITHUB_OUTPUT"
# `lookup-only` with no restore-keys asks whether this exact dependency set is already cached.
# Paths come from the spec, the same ones the save below declares - a lookup that named different
# paths would compute a different cache identity and report a miss every time.
- name: Look up M2 cache
id: m2-lookup
uses: actions/cache/restore@v5
with:
path: ${{ steps.cache-spec.outputs.m2-path }}
key: ${{ steps.cache-spec.outputs.m2-key }}
lookup-only: true
- name: Look up bun store
id: bun-lookup
uses: actions/cache/restore@v5
with:
path: ${{ steps.cache-spec.outputs.bun-store-path }}
key: ${{ steps.cache-spec.outputs.bun-store-key }}
lookup-only: true
- name: Look up Cypress binary
id: cypress-lookup
uses: actions/cache/restore@v5
with:
path: ${{ steps.cache-spec.outputs.cypress-path }}
key: ${{ steps.cache-spec.outputs.cypress-key }}
lookup-only: true
- name: Prepare backend
if: steps.m2-lookup.outputs.cache-hit != 'true'
uses: ./.github/actions/prepare-backend
# Pre-resolve every Clojure dependency classpath CI actually uses, so the saved cache is a SUPERSET
# of what each job resolves at runtime.
#
# Why a single `-P` is not enough: tools.deps selects dependency *versions* globally per classpath,
# so different alias combinations can resolve the same transitive lib to different versions (e.g.
# `org.clojure/data.json` 2.4.0 vs 2.5.x, `org.ow2.asm/asm` 9.2 vs newer). Resolving only one combo
# leaves the others to fetch their version-deltas straight from Maven Central / Clojars on every
# run -> intermittent 403s. See DEV-2094.
#
# Keep this list in sync with the `clojure` invocations across .github/. Each entry notes its
# consumer. We resolve the FULL alias strings (not a hand-reduced set) on purpose: if a currently
# dep-less alias later gains a dependency, this still covers it instead of silently drifting back
# into a cache gap.
- name: Pre-resolve Clojure dependencies
if: steps.m2-lookup.outputs.cache-hit != 'true'
shell: bash
run: |
set -euo pipefail
combos=(
":build:dev:ee:ee-dev:drivers:drivers-dev:cljs" # build + uberjar (combined classpath)
":cljs" # shadow-cljs (build:cljs); resolves :cljs alone -> covers asm:9.2
":dev:ee:ee-dev:drivers:drivers-dev:test:eastwood" # backend.yml (eastwood + eastwood/test)
":dev:ci:ee:ee-dev:drivers:drivers-dev:test" # test-driver.yml (ee) -> covers data.json:2.4.0
":dev:ci:oss:oss-dev:drivers:drivers-dev:test" # test-driver.yml (oss)
":dev:ci:test:ee:ee-dev" # backend.yml matrix (ee)
":dev:ci:test:oss:oss-dev" # backend.yml matrix (oss)
":dev:ci:ee:ee-dev:test:cloverage" # backend-cloverage.yml
":dev:drivers:drivers-dev:test" # reset-cloud-test-dataset.yml
":dev:drivers:build:build-dev:build-test:ci" # build-scripts.yml
":test" # build-scripts.yml (bin/build unit tests)
":ee:drivers:load-namespaces" # backend.yml (load-namespaces)
":ee:doc" # docs-generate-base.yml
":kondo" # resolve-backport-conflicts.yml (replace-deps)
":run:ee" # cross-branch-migration-test
":dev:ee:migrate" # cross-branch-migration-test
":whitespace-linter" # whitespace linter
)
for combo in "${combos[@]}"; do
echo "::group::clojure -P -A${combo}"
clojure -P -A"${combo}"
echo "::endgroup::"
done
echo "::group::./bin/mage"
./bin/mage
echo "::endgroup::"
- name: Save M2 cache
if: steps.m2-lookup.outputs.cache-hit != 'true'
uses: actions/cache/save@v5
with:
path: ${{ steps.cache-spec.outputs.m2-path }}
key: ${{ steps.cache-spec.outputs.m2-key }}
# `prepare-frontend` installs bun, points it at the store path, and runs a frozen-lockfile install,
# which is what fills the store. Patches are deliberately not applied: a store cached with patches
# already in place breaks any pull request that changes or upgrades a patched dependency.
#
# This also runs when only the Cypress cache is missing, because `bun run cypress install` below
# needs the node_modules this step creates. Narrowing the condition to the bun store alone would
# leave that step with no Cypress binary to run.
- name: Install front-end dependencies
if: steps.bun-lookup.outputs.cache-hit != 'true' || steps.cypress-lookup.outputs.cache-hit != 'true'
uses: ./.github/actions/prepare-frontend
with:
apply-patches: "false"
- name: Save bun store
if: steps.bun-lookup.outputs.cache-hit != 'true'
uses: actions/cache/save@v5
with:
path: ${{ steps.cache-spec.outputs.bun-store-path }}
key: ${{ steps.cache-spec.outputs.bun-store-key }}
- name: Ensure that Cypress executable is ready
if: steps.cypress-lookup.outputs.cache-hit != 'true'
shell: bash
run: |
bun run cypress install
bun run cypress cache path
bun run cypress cache list
bun run cypress verify
- name: Save Cypress binary
if: steps.cypress-lookup.outputs.cache-hit != 'true'
uses: actions/cache/save@v5
with:
path: ${{ steps.cache-spec.outputs.cypress-path }}
key: ${{ steps.cache-spec.outputs.cypress-key }}

View File

@@ -3,11 +3,11 @@ name: Prevent backport.sh from being merged
on:
pull_request:
types: [opened, synchronize, reopened]
merge_group:
jobs:
check-backport-script:
runs-on: ${{ vars.DEFAULT_RUNNER_KEY }}
runs-on: ${{ vars.SLIM_RUNNER_KEY }}
timeout-minutes: 5
steps:
- name: Checkout code
uses: actions/checkout@v6

View File

@@ -14,7 +14,7 @@ concurrency:
jobs:
files-changed:
name: Check which files changed
runs-on: ${{ vars.DEFAULT_RUNNER_KEY }}
runs-on: ${{ vars.SLIM_RUNNER_KEY }}
timeout-minutes: 3
outputs:
frontend_all: ${{ steps.changes.outputs.frontend_all }}

View File

@@ -14,7 +14,7 @@ concurrency:
jobs:
files-changed:
name: Check which files changed
runs-on: ${{ vars.DEFAULT_RUNNER_KEY }}
runs-on: ${{ vars.SLIM_RUNNER_KEY }}
timeout-minutes: 3
outputs:
cljs: ${{ steps.changes.outputs.cljs }}

View File

@@ -1,39 +0,0 @@
name: "CodeQL"
on:
push:
branches:
- 'master'
- 'release-**'
pull_request:
jobs:
files-changed:
name: Check which files changed
runs-on: ${{ vars.DEFAULT_RUNNER_KEY }}
timeout-minutes: 3
outputs:
codeql: ${{ steps.changes.outputs.codeql }}
steps:
- uses: actions/checkout@v6
- name: Test which files changed
uses: dorny/paths-filter@fbd0ab8f3e69293af611ebaee6363fc25e6d187d # v4.0.1
id: changes
with:
token: ${{ github.token }}
filters: .github/file-paths.yaml
analyze:
needs: files-changed
if: needs.files-changed.outputs.codeql == 'true'
runs-on: ${{ vars.DEFAULT_RUNNER_KEY }}
steps:
- name: Checkout repository
uses: actions/checkout@v6
- name: Initialize CodeQL
uses: github/codeql-action/init@v4
with:
languages: javascript
config-file: ./.github/codeql/codeql-config.yml
trap-caching: false
- name: Perform CodeQL Analysis
uses: github/codeql-action/analyze@v4

View File

@@ -67,33 +67,6 @@ jobs:
build-args: |
GIT_COMMIT_SHA=${{ github.event.inputs.commit || github.event.pull_request.head.sha || github.sha }}
run-trivy:
needs: [get-container-info, containerize]
strategy:
matrix:
edition: ${{ fromJson(needs.get-container-info.outputs.editions) }}
runs-on: ${{ vars.DEFAULT_RUNNER_KEY }}
name: Run Trivy vulnerability scanner
steps:
- name: Check out the code
uses: actions/checkout@v6
with:
ref: ${{ github.event.inputs.commit || github.event.pull_request.head.sha || github.sha }}
- name: Run Trivy vulnerability scanner
uses: aquasecurity/trivy-action@ed142fd0673e97e23eac54620cfb913e5ce36c25 # v0.36.0
env:
TRIVY_OFFLINE_SCAN: true
repo: ${{ fromJson(needs.get-container-info.outputs[matrix.edition]).repo }}
tag: ${{ fromJson(needs.get-container-info.outputs[matrix.edition]).tag }}
with:
image-ref: docker.io/${{ env.repo }}:${{ env.tag }}
format: sarif
output: trivy-results.sarif
- name: Upload Trivy scan results to GitHub Security tab
uses: github/codeql-action/upload-sarif@v3
with:
sarif_file: "trivy-results.sarif"
post-comment:
needs: [get-container-info, containerize]
if: github.event_name == 'pull_request'

View File

@@ -27,7 +27,8 @@ jobs:
# Generate matrix for scheduled/PR runs
generate-matrix:
if: github.event_name != 'workflow_dispatch' && github.repository == 'metabase/metabase'
runs-on: ${{ vars.DEFAULT_RUNNER_KEY }}
runs-on: ${{ vars.SLIM_RUNNER_KEY }}
timeout-minutes: 5
outputs:
matrix: ${{ steps.set-matrix.outputs.matrix }}
steps:
@@ -97,9 +98,12 @@ jobs:
- name: Show logs on failure
if: failure()
run: |
cd cross-version
docker compose logs metabase || true
docker compose logs postgres || true
for f in cross-version/logs/*.log; do
[ -e "$f" ] || continue
echo "::group::$f"
cat "$f"
echo "::endgroup::"
done
# Notify Slack on nightly failures
notify-on-failure:
@@ -221,6 +225,9 @@ jobs:
- name: Show logs on failure
if: failure()
run: |
cd cross-version
docker compose logs metabase || true
docker compose logs postgres || true
for f in cross-version/logs/*.log; do
[ -e "$f" ] || continue
echo "::group::$f"
cat "$f"
echo "::endgroup::"
done

View File

@@ -1,88 +0,0 @@
name: Disabled Drivers Slack Notification
on:
workflow_dispatch:
schedule:
- cron: "0 9 * * 1" # every Monday at 9:00 UTC
jobs:
notify-disabled-drivers:
if: ${{ github.event_name != 'schedule' || github.repository == 'metabase/metabase' }}
runs-on: ${{ vars.DEFAULT_RUNNER_KEY }}
timeout-minutes: 5
steps:
- uses: actions/setup-node@v6.4.0
with:
node-version: lts/Jod
- name: Install Slack SDK
run: npm install @slack/web-api
- name: Check disabled drivers and notify Slack
uses: actions/github-script@v9
env:
SLACK_BOT_TOKEN: ${{ secrets.SLACK_BOT_TOKEN }}
with:
script: |
const { WebClient } = require('@slack/web-api');
// Fetch the config
const response = await fetch('https://raw.githubusercontent.com/metabase/ci-test-config/refs/heads/master/ci-test-config.json');
const config = await response.json();
const drivers = config.drivers || [];
const lastUpdated = config.metadata?.last_updated || 'unknown';
const reason = config.metadata?.reason || 'No reason provided';
// "skip" is the only status CI reads: the driver is not run at all. A driver that
// runs but must not gate is expressed as a suite-level rule in CI Conductor, which
// reports it in its own UI.
const skipped = drivers.filter(d => d.status === 'skip').map(d => d.name);
if (skipped.length === 0) {
console.log('No drivers are disabled. Skipping Slack notification.');
return;
}
const list = (names) => names.map(d => `\`${d}\``).join(' ');
const lines = [`:no_entry: *Disabled (not run):* ${list(skipped)}`];
if (!process.env.SLACK_BOT_TOKEN) {
throw new Error('SLACK_BOT_TOKEN is required');
}
const slack = new WebClient(process.env.SLACK_BOT_TOKEN);
await slack.chat.postMessage({
channel: 'engineering-ci',
text: 'CI Driver Status Report',
blocks: [
{
type: 'section',
text: {
type: 'mrkdwn',
text: `*CI Driver Status*\n${lines.join('\n')}`
},
accessory: {
type: 'button',
text: {
type: 'plain_text',
text: 'View Config',
emoji: true
},
url: 'https://github.com/metabase/ci-test-config/blob/master/ci-test-config.json'
}
},
{
type: 'context',
elements: [
{
type: 'mrkdwn',
text: `${reason} · :clock1: *${lastUpdated}*`
}
]
}
]
});
console.log('Slack notification sent successfully');

View File

@@ -33,6 +33,9 @@ jobs:
- name: Prepare backend environment
uses: ./.github/actions/prepare-backend
- name: Prepare frontend environment
uses: ./.github/actions/prepare-frontend
- name: Set up Babashka
uses: turtlequeue/setup-babashka@d78ec6570aea3b614bc695cafcceb82eb45c2af9 # v1.8.0
with:
@@ -75,6 +78,22 @@ jobs:
- name: Generate usage analytics documentation
run: ./bin/generate-usage-analytics-docs.bb
- name: Generate embedding SDK documentation
id: generate_embedding_sdk_docs
run: |
set -e # Exit on any error
echo "Generating embedding SDK documentation..."
bun run embedding-sdk:docs:generate
continue-on-error: false
- name: Generate embedding EAJS documentation
id: generate_embedding_eajs_docs
run: |
set -e # Exit on any error
echo "Generating embedding EAJS documentation..."
bun run embedding-eajs:docs:generate
continue-on-error: false
- name: Check for changes
id: check_changes
run: |
@@ -146,6 +165,8 @@ jobs:
- \`clojure -M:ee:doc api-documentation\`
- \`clojure -M:ee:doc command-documentation\`
- \`./bin/generate-usage-analytics-docs.bb\`
- \`bun run embedding-sdk:docs:generate\`
- \`bun run embedding-eajs:docs:generate\`
Please review the changes and merge if they look correct." \
--label "Type:Documentation" \

View File

@@ -770,7 +770,7 @@ jobs:
login-to-ecr:
if: inputs.driver == 'Vertica'
runs-on: ${{ vars.DEFAULT_RUNNER_KEY }}
runs-on: ${{ vars.SLIM_RUNNER_KEY }}
timeout-minutes: 15
permissions:
id-token: write

View File

@@ -6,6 +6,49 @@ on:
skip:
type: boolean
default: false
force-run:
description: Run the full suite regardless of the diff (master / release branches, ci:run-all label).
type: boolean
default: false
# Run one driver job on demand, against the ref you dispatch on, optionally under a profiler.
# Everything else about the job -- credentials, services, test args, quarantine gating -- is
# whatever the normal CI run uses, because it is the same job.
workflow_dispatch:
inputs:
driver:
description: 'Which driver job to run. Only this one runs; the usual decision rules are bypassed.'
type: choice
required: true
options:
- athena
- bigquery
- clickhouse
- databricks
- druid-jdbc
- h2
- mongo
- mongo-sharded-cluster
- mongo-ssl
- mysql-mariadb
- oracle
- postgres
- presto-jdbc
- redshift
- snowflake
- sparksql
- sqlite
- sqlserver
- vertica
tests:
description: 'Optional: run only these tests, comma- or space-separated, e.g. metabase.driver.bigquery-cloud-sdk-test/sync-test. Bare namespaces work too. Blank runs the job''s usual set.'
type: string
required: false
default: ''
profile:
description: 'Profile the test run and attach a flamegraph to the job. Adds sampling overhead.'
type: boolean
required: false
default: false
concurrency:
group: ${{ github.workflow }}-${{ github.head_ref && github.ref || github.run_id }}-drivers
@@ -15,8 +58,16 @@ concurrency:
# ci-conductor (secrets: inherit is already set on the drivers call in
# run-tests.yml). See DEV-2224.
env:
CI_CONDUCTOR_BASE_URL: ${{ secrets.CI_CONDUCTOR_BASE_URL }}
CI_CONDUCTOR_BASE_URL: ${{ vars.CI_CONDUCTOR_BASE_URL }}
CI_CONDUCTOR_WEBHOOK_SECRET: ${{ secrets.CI_CONDUCTOR_WEBHOOK_SECRET }}
# Both read by the test-driver action, which is what actually runs the tests. Set here rather than
# passed as inputs to each job so that turning either on is one line, not one line per driver.
# Empty (and so off) on workflow_call, where neither input exists.
#
# ONLY_TESTS replaces the job's own test selection, so every matrix leg of the chosen driver runs
# the whole list -- BigQuery, for instance, has three legs.
PROFILE_TESTS: ${{ inputs.profile }}
ONLY_TESTS: ${{ inputs.tests }}
jobs:
# This job determines which driver tests should run based on PR context.
@@ -45,29 +96,11 @@ jobs:
sqlite-should-run: ${{ steps.driver-decisions.outputs.sqlite-should-run }}
sqlserver-should-run: ${{ steps.driver-decisions.outputs.sqlserver-should-run }}
vertica-should-run: ${{ steps.driver-decisions.outputs.vertica-should-run }}
# Per-driver quarantine conflict flags (set when driver is quarantined but has file changes)
athena-quarantine-conflict: ${{ steps.driver-decisions.outputs.athena-quarantine-conflict }}
bigquery-quarantine-conflict: ${{ steps.driver-decisions.outputs.bigquery-quarantine-conflict }}
clickhouse-quarantine-conflict: ${{ steps.driver-decisions.outputs.clickhouse-quarantine-conflict }}
databricks-quarantine-conflict: ${{ steps.driver-decisions.outputs.databricks-quarantine-conflict }}
druid-quarantine-conflict: ${{ steps.driver-decisions.outputs.druid-quarantine-conflict }}
druid-jdbc-quarantine-conflict: ${{ steps.driver-decisions.outputs.druid-jdbc-quarantine-conflict }}
mongo-quarantine-conflict: ${{ steps.driver-decisions.outputs.mongo-quarantine-conflict }}
mongo-ssl-quarantine-conflict: ${{ steps.driver-decisions.outputs.mongo-ssl-quarantine-conflict }}
mongo-sharded-cluster-quarantine-conflict: ${{ steps.driver-decisions.outputs.mongo-sharded-cluster-quarantine-conflict }}
mysql-mariadb-quarantine-conflict: ${{ steps.driver-decisions.outputs.mysql-mariadb-quarantine-conflict }}
oracle-quarantine-conflict: ${{ steps.driver-decisions.outputs.oracle-quarantine-conflict }}
postgres-quarantine-conflict: ${{ steps.driver-decisions.outputs.postgres-quarantine-conflict }}
presto-jdbc-quarantine-conflict: ${{ steps.driver-decisions.outputs.presto-jdbc-quarantine-conflict }}
redshift-quarantine-conflict: ${{ steps.driver-decisions.outputs.redshift-quarantine-conflict }}
snowflake-quarantine-conflict: ${{ steps.driver-decisions.outputs.snowflake-quarantine-conflict }}
sparksql-quarantine-conflict: ${{ steps.driver-decisions.outputs.sparksql-quarantine-conflict }}
sqlite-quarantine-conflict: ${{ steps.driver-decisions.outputs.sqlite-quarantine-conflict }}
sqlserver-quarantine-conflict: ${{ steps.driver-decisions.outputs.sqlserver-quarantine-conflict }}
vertica-quarantine-conflict: ${{ steps.driver-decisions.outputs.vertica-quarantine-conflict }}
steps:
- name: Checkout repository
uses: actions/checkout@v6
- name: Prepare back-end environment
uses: ./.github/actions/prepare-backend
- name: Fetch base branch
if: github.event_name == 'pull_request' && github.event.pull_request.base.ref != ''
run: |
@@ -80,9 +113,10 @@ jobs:
# branch context, labels, and file changes to decide what runs.
MAGE_ARGS=(
--git-ref="${BASE_REF}"
--is-master-or-release="${IS_MASTER_OR_RELEASE}"
--force-run="${FORCE_RUN}"
--pr-labels="${{ join(github.event.pull_request.labels.*.name, ',') }}"
--skip="${{ inputs.skip }}"
--only-driver="${ONLY_DRIVER}"
)
echo "Running:"
echo "./bin/mage -driver-decisions ${MAGE_ARGS[*]}"
@@ -96,12 +130,13 @@ jobs:
./bin/mage -driver-decisions "${MAGE_ARGS[@]}" --github-output-only >> "$GITHUB_OUTPUT"
env:
BASE_REF: ${{ github.event.pull_request.base.ref || github.ref_name }}
IS_MASTER_OR_RELEASE: ${{ github.ref_name == 'master' || startsWith(github.ref_name, 'release-x') }}
FORCE_RUN: ${{ inputs.force-run }}
ONLY_DRIVER: ${{ inputs.driver }}
shell: bash
be-tests-athena-ee:
needs: determine-driver-skips
if: ${{ needs.determine-driver-skips.outputs.athena-should-run == 'true' || needs.determine-driver-skips.outputs.athena-quarantine-conflict == 'true' }}
if: ${{ needs.determine-driver-skips.outputs.athena-should-run == 'true' }}
runs-on: ${{ vars.DEFAULT_RUNNER_KEY }}
timeout-minutes: 40
env:
@@ -118,16 +153,6 @@ jobs:
MB_ATHENA_TEST_WITHOUT_GET_TABLE_METADATA_SECRET_KEY: ${{ secrets.MB_ATHENA_TEST_WITHOUT_GET_TABLE_METADATA_SECRET_KEY }}
name: Athena
steps:
- name: Fail on quarantine conflict
if: ${{ needs.determine-driver-skips.outputs.athena-quarantine-conflict == 'true' }}
run: |
echo "::error::Athena driver is quarantined but you changed files in modules/drivers/athena/"
echo ""
echo "Your changes will NOT be tested because the driver is currently quarantined."
echo "To run Athena tests, add the label: break-quarantine-athena"
echo ""
echo "If you merge without testing, you risk breaking the Athena driver!"
exit 1
- uses: actions/checkout@v6
- name: Test Athena driver
id: test-driver
@@ -138,19 +163,10 @@ jobs:
:only-tags [:mb/driver-tests]
:exclude-tags [:mb/transforms-python-test]
quarantine-engine: ${{ vars.QUARANTINE_ENGINE }}
- name: Upload Test Results
uses: ./.github/actions/upload-test-results
if: always()
with:
input-path: ./target/junit/
output-name: ${{ github.job }}
bucket: ${{ vars.AWS_S3_TEST_RESULTS_BUCKET }}
role-to-assume: ${{ secrets.CI_TEST_RESULTS_IAM_ROLE }}
aws-region: ${{ vars.AWS_REGION }}
be-tests-bigquery-cloud-sdk-ee:
needs: determine-driver-skips
if: ${{ needs.determine-driver-skips.outputs.bigquery-should-run == 'true' || needs.determine-driver-skips.outputs.bigquery-quarantine-conflict == 'true' }}
if: ${{ needs.determine-driver-skips.outputs.bigquery-should-run == 'true' }}
runs-on: ${{ vars.DEFAULT_RUNNER_KEY }}
timeout-minutes: 70
strategy:
@@ -187,16 +203,6 @@ jobs:
MB_BIGQUERY_CLOUD_SDK_TEST_SERVICE_ACCOUNT_JSON_ROUTING: ${{ secrets.MB_BIGQUERY_CLOUD_SDK_TEST_SERVICE_ACCOUNT_JSON_ROUTING }}
name: BigQuery ${{ matrix.job.name }}
steps:
- name: Fail on quarantine conflict
if: ${{ needs.determine-driver-skips.outputs.bigquery-quarantine-conflict == 'true' }}
run: |
echo "::error::BigQuery driver is quarantined but you changed files in modules/drivers/bigquery-cloud-sdk/"
echo ""
echo "Your changes will NOT be tested because the driver is currently quarantined."
echo "To run BigQuery tests, add the label: break-quarantine-bigquery"
echo ""
echo "If you merge without testing, you risk breaking the BigQuery driver!"
exit 1
- uses: actions/checkout@v6
- name: Setup python-runner
if: ${{ matrix.job.needs-python-runner == true }}
@@ -211,15 +217,6 @@ jobs:
test-args: ${{ matrix.job.test-args }}
artifact-suffix: ${{ matrix.job.name }}
quarantine-engine: ${{ vars.QUARANTINE_ENGINE }}
- name: Upload Test Results
uses: ./.github/actions/upload-test-results
if: always()
with:
input-path: ./target/junit/
output-name: ${{ github.job }}
bucket: ${{ vars.AWS_S3_TEST_RESULTS_BUCKET }}
role-to-assume: ${{ secrets.CI_TEST_RESULTS_IAM_ROLE }}
aws-region: ${{ vars.AWS_REGION }}
- name: Cleanup python-runner
if: ${{ always() && matrix.job.needs-python-runner == true }}
uses: ./.github/actions/cleanup-python-runner
@@ -229,7 +226,7 @@ jobs:
be-tests-clickhouse-ee:
needs: determine-driver-skips
if: ${{ needs.determine-driver-skips.outputs.clickhouse-should-run == 'true' || needs.determine-driver-skips.outputs.clickhouse-quarantine-conflict == 'true' }}
if: ${{ needs.determine-driver-skips.outputs.clickhouse-should-run == 'true' }}
runs-on: ${{ vars.DEFAULT_RUNNER_KEY }}
timeout-minutes: 40
strategy:
@@ -253,16 +250,6 @@ jobs:
MB_CLICKHOUSE_TEST_NGINX_PORT: 8127
name: Clickhouse ${{ matrix.job.name }}
steps:
- name: Fail on quarantine conflict
if: ${{ needs.determine-driver-skips.outputs.clickhouse-quarantine-conflict == 'true' }}
run: |
echo "::error::ClickHouse driver is quarantined but you changed files in modules/drivers/clickhouse/"
echo ""
echo "Your changes will NOT be tested because the driver is currently quarantined."
echo "To run ClickHouse tests, add the label: break-quarantine-clickhouse"
echo ""
echo "If you merge without testing, you risk breaking the ClickHouse driver!"
exit 1
- uses: actions/checkout@v6
- name: Setup python-runner
if: ${{ matrix.job.needs-python-runner == true }}
@@ -293,15 +280,6 @@ jobs:
test-args: ${{ matrix.job.test-args }}
artifact-suffix: ${{ matrix.job.name }}
quarantine-engine: ${{ vars.QUARANTINE_ENGINE }}
- name: Upload Test Results
uses: ./.github/actions/upload-test-results
if: always()
with:
input-path: ./target/junit/
output-name: ${{ github.job }}
bucket: ${{ vars.AWS_S3_TEST_RESULTS_BUCKET }}
role-to-assume: ${{ secrets.CI_TEST_RESULTS_IAM_ROLE }}
aws-region: ${{ vars.AWS_REGION }}
- name: Cleanup python-runner
if: ${{ always() && matrix.job.needs-python-runner == true }}
uses: ./.github/actions/cleanup-python-runner
@@ -311,7 +289,7 @@ jobs:
be-tests-druid-jdbc-ee:
needs: determine-driver-skips
if: ${{ needs.determine-driver-skips.outputs.druid-jdbc-should-run == 'true' || needs.determine-driver-skips.outputs.druid-jdbc-quarantine-conflict == 'true' }}
if: ${{ needs.determine-driver-skips.outputs.druid-jdbc-should-run == 'true' }}
runs-on: ${{ vars.DEFAULT_RUNNER_KEY }}
timeout-minutes: 40
env:
@@ -326,16 +304,6 @@ jobs:
CLUSTER_SIZE: nano-quickstart
name: Druid (JDBC)
steps:
- name: Fail on quarantine conflict
if: ${{ needs.determine-driver-skips.outputs.druid-jdbc-quarantine-conflict == 'true' }}
run: |
echo "::error::Druid JDBC driver is quarantined but you changed files in modules/drivers/druid-jdbc/"
echo ""
echo "Your changes will NOT be tested because the driver is currently quarantined."
echo "To run Druid JDBC tests, add the label: break-quarantine-druid-jdbc"
echo ""
echo "If you merge without testing, you risk breaking the Druid JDBC driver!"
exit 1
- uses: actions/checkout@v6
- name: Test Druid JDBC driver
id: test-driver
@@ -346,19 +314,10 @@ jobs:
:only-tags [:mb/driver-tests]
:exclude-tags [:mb/transforms-python-test]
quarantine-engine: ${{ vars.QUARANTINE_ENGINE }}
- name: Upload Test Results
uses: ./.github/actions/upload-test-results
if: always()
with:
input-path: ./target/junit/
output-name: ${{ github.job }}
bucket: ${{ vars.AWS_S3_TEST_RESULTS_BUCKET }}
role-to-assume: ${{ secrets.CI_TEST_RESULTS_IAM_ROLE }}
aws-region: ${{ vars.AWS_REGION }}
be-tests-databricks-ee:
needs: determine-driver-skips
if: ${{ needs.determine-driver-skips.outputs.databricks-should-run == 'true' || needs.determine-driver-skips.outputs.databricks-quarantine-conflict == 'true' }}
if: ${{ needs.determine-driver-skips.outputs.databricks-should-run == 'true' }}
runs-on: ${{ vars.DEFAULT_RUNNER_KEY }}
timeout-minutes: 50
strategy:
@@ -398,16 +357,6 @@ jobs:
MB_DATABRICKS_TEST_OAUTH_SECRET: ${{ secrets.MB_DATABRICKS_JDBC_TEST_OAUTH_SECRET }}
name: Databricks
steps:
- name: Fail on quarantine conflict
if: ${{ needs.determine-driver-skips.outputs.databricks-quarantine-conflict == 'true' }}
run: |
echo "::error::Databricks driver is quarantined but you changed files in modules/drivers/databricks/"
echo ""
echo "Your changes will NOT be tested because the driver is currently quarantined."
echo "To run Databricks tests, add the label: break-quarantine-databricks"
echo ""
echo "If you merge without testing, you risk breaking the Databricks driver!"
exit 1
- uses: actions/checkout@v6
- name: Test Databricks driver
id: test-driver
@@ -417,19 +366,10 @@ jobs:
artifact-suffix: ${{ matrix.job.name }}
test-args: ${{ matrix.job.test-args }}
quarantine-engine: ${{ vars.QUARANTINE_ENGINE }}
- name: Upload Test Results
uses: ./.github/actions/upload-test-results
if: always()
with:
input-path: ./target/junit/
output-name: ${{ github.job }}
bucket: ${{ vars.AWS_S3_TEST_RESULTS_BUCKET }}
role-to-assume: ${{ secrets.CI_TEST_RESULTS_IAM_ROLE }}
aws-region: ${{ vars.AWS_REGION }}
be-tests-databricks-multi-catalog-ee:
needs: determine-driver-skips
if: ${{ needs.determine-driver-skips.outputs.databricks-should-run == 'true' || needs.determine-driver-skips.outputs.databricks-quarantine-conflict == 'true' }}
if: ${{ needs.determine-driver-skips.outputs.databricks-should-run == 'true' }}
runs-on: ${{ vars.DEFAULT_RUNNER_KEY }}
timeout-minutes: 50
strategy:
@@ -470,16 +410,6 @@ jobs:
MB_DATABRICKS_TEST_OAUTH_SECRET: ${{ secrets.MB_DATABRICKS_JDBC_TEST_OAUTH_SECRET }}
name: Databricks (Multi-Catalog)
steps:
- name: Fail on quarantine conflict
if: ${{ needs.determine-driver-skips.outputs.databricks-quarantine-conflict == 'true' }}
run: |
echo "::error::Databricks driver is quarantined but you changed files in modules/drivers/databricks/"
echo ""
echo "Your changes will NOT be tested because the driver is currently quarantined."
echo "To run Databricks tests, add the label: break-quarantine-databricks"
echo ""
echo "If you merge without testing, you risk breaking the Databricks driver!"
exit 1
- uses: actions/checkout@v6
- name: Test Databricks driver with multi-catalog
id: test-driver
@@ -489,15 +419,6 @@ jobs:
artifact-suffix: ${{ matrix.job.name }}
test-args: ${{ matrix.job.test-args }}
quarantine-engine: ${{ vars.QUARANTINE_ENGINE }}
- name: Upload Test Results
uses: ./.github/actions/upload-test-results
if: always()
with:
input-path: ./target/junit/
output-name: ${{ github.job }}
bucket: ${{ vars.AWS_S3_TEST_RESULTS_BUCKET }}
role-to-assume: ${{ secrets.CI_TEST_RESULTS_IAM_ROLE }}
aws-region: ${{ vars.AWS_REGION }}
be-tests-h2:
needs: determine-driver-skips
@@ -518,15 +439,6 @@ jobs:
:only-tags [:mb/driver-tests]
:exclude-tags [:mb/transforms-python-test]
quarantine-engine: ${{ vars.QUARANTINE_ENGINE }}
- name: Upload Test Results
uses: ./.github/actions/upload-test-results
if: always()
with:
input-path: ./target/junit/
output-name: ${{ github.job }}
bucket: ${{ vars.AWS_S3_TEST_RESULTS_BUCKET }}
role-to-assume: ${{ secrets.CI_TEST_RESULTS_IAM_ROLE }}
aws-region: ${{ vars.AWS_REGION }}
be-tests-h2-oss:
needs: determine-driver-skips
@@ -548,15 +460,6 @@ jobs:
:only-tags [:mb/driver-tests]
:exclude-tags [:mb/transforms-python-test]
quarantine-engine: ${{ vars.QUARANTINE_ENGINE }}
- name: Upload Test Results
uses: ./.github/actions/upload-test-results
if: always()
with:
input-path: ./target/junit/
output-name: ${{ github.job }}
bucket: ${{ vars.AWS_S3_TEST_RESULTS_BUCKET }}
role-to-assume: ${{ secrets.CI_TEST_RESULTS_IAM_ROLE }}
aws-region: ${{ vars.AWS_REGION }}
be-tests-mysql-mariadb:
needs: determine-driver-skips
@@ -621,21 +524,9 @@ jobs:
MYSQL_ALLOW_EMPTY_PASSWORD: true
MYSQL_DATABASE: mb_test
env:
# Only run old migrations tests on pushes to master or release branches. All other branches should skip tests
# with the tag `mb/old-migrations-test`. `__ADDITIONAL_EXCLUDED_TAG__` is not used anywhere outside of splicing
# it in to the command below.
__ADDITIONAL_EXCLUDED_TAG__: >-
${{
(
github.event_name == 'push' &&
(
github.ref_name == 'master' ||
startsWith(github.ref_name, 'release-x')
) &&
''
) ||
':mb/old-migrations-test'
}}
# __ADDITIONAL_EXCLUDED_TAG__: ${{ !inputs.force-run && ':mb/old-migrations-test' || '' }}
# GDGT-3126: re-enable and properly fix old migrations tests
__ADDITIONAL_EXCLUDED_TAG__: ':mb/old-migrations-test'
# actual serious env vars below
CI: 'true'
DRIVERS: mysql
@@ -688,15 +579,6 @@ jobs:
${{ matrix.job.test-args }}
:exclude-tags '[ ${{ matrix.job.exclude-tag }} ${{ env.__ADDITIONAL_EXCLUDED_TAG__ }}]'
quarantine-engine: ${{ vars.QUARANTINE_ENGINE }}
- name: Upload Test Results
uses: ./.github/actions/upload-test-results
if: always()
with:
input-path: ./target/junit/
output-name: ${{ github.job }}
bucket: ${{ vars.AWS_S3_TEST_RESULTS_BUCKET }}
role-to-assume: ${{ secrets.CI_TEST_RESULTS_IAM_ROLE }}
aws-region: ${{ vars.AWS_REGION }}
- name: Cleanup python-runner
if: always()
uses: ./.github/actions/cleanup-python-runner
@@ -706,7 +588,7 @@ jobs:
be-tests-mongo:
needs: determine-driver-skips
if: ${{ needs.determine-driver-skips.outputs.mongo-should-run == 'true' || needs.determine-driver-skips.outputs.mongo-quarantine-conflict == 'true' }}
if: ${{ needs.determine-driver-skips.outputs.mongo-should-run == 'true' }}
runs-on: ${{ vars.DEFAULT_RUNNER_KEY }}
timeout-minutes: 40
strategy:
@@ -743,16 +625,6 @@ jobs:
MONGO_INITDB_ROOT_PASSWORD: metasample123
name: ${{ matrix.version.name }} ${{ matrix.job.name }}
steps:
- name: Fail on quarantine conflict
if: ${{ needs.determine-driver-skips.outputs.mongo-quarantine-conflict == 'true' }}
run: |
echo "::error::MongoDB driver is quarantined but you changed files in modules/drivers/mongo/"
echo ""
echo "Your changes will NOT be tested because the driver is currently quarantined."
echo "To run MongoDB tests, add the label: break-quarantine-mongo"
echo ""
echo "If you merge without testing, you risk breaking the MongoDB driver!"
exit 1
- uses: actions/checkout@v6
- name: Setup python-runner
if: ${{ matrix.job.needs-python-runner == true }}
@@ -767,15 +639,6 @@ jobs:
test-args: ${{ matrix.job.test-args }}
artifact-suffix: ${{ matrix.version.name }}-${{ matrix.job.name }}
quarantine-engine: ${{ vars.QUARANTINE_ENGINE }}
- name: Upload Test Results
uses: ./.github/actions/upload-test-results
if: always()
with:
input-path: ./target/junit/
output-name: ${{ github.job }}
bucket: ${{ vars.AWS_S3_TEST_RESULTS_BUCKET }}
role-to-assume: ${{ secrets.CI_TEST_RESULTS_IAM_ROLE }}
aws-region: ${{ vars.AWS_REGION }}
- name: Cleanup python-runner
if: ${{ always() && matrix.job.needs-python-runner == true }}
uses: ./.github/actions/cleanup-python-runner
@@ -785,7 +648,7 @@ jobs:
be-tests-mongo-ssl:
needs: determine-driver-skips
if: ${{ needs.determine-driver-skips.outputs.mongo-ssl-should-run == 'true' || needs.determine-driver-skips.outputs.mongo-ssl-quarantine-conflict == 'true' }}
if: ${{ needs.determine-driver-skips.outputs.mongo-ssl-should-run == 'true' }}
runs-on: ${{ vars.DEFAULT_RUNNER_KEY }}
timeout-minutes: 40
strategy:
@@ -803,16 +666,6 @@ jobs:
MB_TEST_MONGO_REQUIRES_SSL: true
name: "${{ matrix.version.name }} (SSL)"
steps:
- name: Fail on quarantine conflict
if: ${{ needs.determine-driver-skips.outputs.mongo-ssl-quarantine-conflict == 'true' }}
run: |
echo "::error::MongoDB SSL driver is quarantined but you changed files in modules/drivers/mongo/"
echo ""
echo "Your changes will NOT be tested because the driver is currently quarantined."
echo "To run MongoDB SSL tests, add the label: break-quarantine-mongo-ssl"
echo ""
echo "If you merge without testing, you risk breaking the MongoDB SSL driver!"
exit 1
- uses: actions/checkout@v6
- name: Spin up Mongo docker container
run: >-
@@ -848,19 +701,10 @@ jobs:
:exclude-tags '[:mongo-sharded-cluster-tests :mb/transforms-python-test]'
:only-tags [:mb/driver-tests]
quarantine-engine: ${{ vars.QUARANTINE_ENGINE }}
- name: Upload Test Results
uses: ./.github/actions/upload-test-results
if: always()
with:
input-path: ./target/junit/
output-name: ${{ github.job }}
bucket: ${{ vars.AWS_S3_TEST_RESULTS_BUCKET }}
role-to-assume: ${{ secrets.CI_TEST_RESULTS_IAM_ROLE }}
aws-region: ${{ vars.AWS_REGION }}
be-tests-mongo-sharded-cluster-ee:
needs: determine-driver-skips
if: ${{ needs.determine-driver-skips.outputs.mongo-sharded-cluster-should-run == 'true' || needs.determine-driver-skips.outputs.mongo-sharded-cluster-quarantine-conflict == 'true' }}
if: ${{ needs.determine-driver-skips.outputs.mongo-sharded-cluster-should-run == 'true' }}
runs-on: ${{ vars.DEFAULT_RUNNER_KEY }}
timeout-minutes: 40
env:
@@ -873,16 +717,6 @@ jobs:
- "27017:17017"
name: MongoDB Sharded Cluster
steps:
- name: Fail on quarantine conflict
if: ${{ needs.determine-driver-skips.outputs.mongo-sharded-cluster-quarantine-conflict == 'true' }}
run: |
echo "::error::MongoDB Sharded Cluster driver is quarantined but you changed files in modules/drivers/mongo/"
echo ""
echo "Your changes will NOT be tested because the driver is currently quarantined."
echo "To run MongoDB Sharded Cluster tests, add the label: break-quarantine-mongo-sharded-cluster"
echo ""
echo "If you merge without testing, you risk breaking the MongoDB Sharded Cluster driver!"
exit 1
- uses: actions/checkout@v6
- name: Test MongoDB driver (Sharded cluster)
id: test-driver
@@ -892,19 +726,10 @@ jobs:
test-args: >-
:only metabase.driver.mongo.sharded-cluster-test
quarantine-engine: ${{ vars.QUARANTINE_ENGINE }}
- name: Upload Test Results
uses: ./.github/actions/upload-test-results
if: always()
with:
input-path: ./target/junit/
output-name: ${{ github.job }}
bucket: ${{ vars.AWS_S3_TEST_RESULTS_BUCKET }}
role-to-assume: ${{ secrets.CI_TEST_RESULTS_IAM_ROLE }}
aws-region: ${{ vars.AWS_REGION }}
be-tests-oracle:
needs: determine-driver-skips
if: ${{ needs.determine-driver-skips.outputs.oracle-should-run == 'true' || needs.determine-driver-skips.outputs.oracle-quarantine-conflict == 'true' }}
if: ${{ needs.determine-driver-skips.outputs.oracle-should-run == 'true' }}
runs-on: ${{ vars.DEFAULT_RUNNER_KEY }}
timeout-minutes: 40
strategy:
@@ -955,16 +780,6 @@ jobs:
- "1521:${{ matrix.version.port }}"
name: ${{ matrix.version.name }}
steps:
- name: Fail on quarantine conflict
if: ${{ needs.determine-driver-skips.outputs.oracle-quarantine-conflict == 'true' }}
run: |
echo "::error::Oracle driver is quarantined but you changed files in modules/drivers/oracle/"
echo ""
echo "Your changes will NOT be tested because the driver is currently quarantined."
echo "To run Oracle tests, add the label: break-quarantine-oracle"
echo ""
echo "If you merge without testing, you risk breaking the Oracle driver!"
exit 1
- uses: actions/checkout@v6
- name: Test ${{ matrix.version.name }}
id: test-driver
@@ -976,15 +791,6 @@ jobs:
:only-tags [:mb/driver-tests]
:exclude-tags [:mb/transforms-python-test]
quarantine-engine: ${{ vars.QUARANTINE_ENGINE }}
- name: Upload Test Results
uses: ./.github/actions/upload-test-results
if: always()
with:
input-path: ./target/junit/
output-name: ${{ github.job }}
bucket: ${{ vars.AWS_S3_TEST_RESULTS_BUCKET }}
role-to-assume: ${{ secrets.CI_TEST_RESULTS_IAM_ROLE }}
aws-region: ${{ vars.AWS_REGION }}
be-tests-postgres:
needs: determine-driver-skips
@@ -1018,21 +824,9 @@ jobs:
needs-python-runner: true
name: "${{ matrix.version.name }} ${{ matrix.job.name }}"
env:
# Only run old migrations tests on pushes to master or release branches. All other branches should skip tests
# with the tag `mb/old-migrations-test`. `__ADDITIONAL_EXCLUDED_TAG__` is not used anywhere outside of splicing
# it in to the command below.
__ADDITIONAL_EXCLUDED_TAG__: >-
${{
(
github.event_name == 'push' &&
(
github.ref_name == 'master' ||
startsWith(github.ref_name, 'release-x')
) &&
''
) ||
':mb/old-migrations-test'
}}
# __ADDITIONAL_EXCLUDED_TAG__: ${{ !inputs.force-run && ':mb/old-migrations-test' || '' }}
# GDGT-3126: re-enable and properly fix old migrations tests
__ADDITIONAL_EXCLUDED_TAG__: ':mb/old-migrations-test'
# actual serious env vars below
CI: 'true'
DRIVERS: postgres
@@ -1079,15 +873,6 @@ jobs:
${{ matrix.job.test-args }}
:exclude-tags '[ ${{ matrix.job.exclude-tag }} ${{ env.__ADDITIONAL_EXCLUDED_TAG__ }}]'
quarantine-engine: ${{ vars.QUARANTINE_ENGINE }}
- name: Upload Test Results
uses: ./.github/actions/upload-test-results
if: always()
with:
input-path: ./target/junit/
output-name: ${{ github.job }}
bucket: ${{ vars.AWS_S3_TEST_RESULTS_BUCKET }}
role-to-assume: ${{ secrets.CI_TEST_RESULTS_IAM_ROLE }}
aws-region: ${{ vars.AWS_REGION }}
- name: Cleanup python-runner
if: always()
uses: ./.github/actions/cleanup-python-runner
@@ -1097,7 +882,7 @@ jobs:
be-tests-presto-jdbc-ee:
needs: determine-driver-skips
if: ${{ needs.determine-driver-skips.outputs.presto-jdbc-should-run == 'true' || needs.determine-driver-skips.outputs.presto-jdbc-quarantine-conflict == 'true' }}
if: ${{ needs.determine-driver-skips.outputs.presto-jdbc-should-run == 'true' }}
runs-on: ${{ vars.DEFAULT_RUNNER_KEY }}
timeout-minutes: 40
env:
@@ -1120,16 +905,6 @@ jobs:
env:
JAVA_TOOL_OPTIONS: "-Xmx2g"
steps:
- name: Fail on quarantine conflict
if: ${{ needs.determine-driver-skips.outputs.presto-jdbc-quarantine-conflict == 'true' }}
run: |
echo "::error::Presto JDBC driver is quarantined but you changed files in modules/drivers/presto-jdbc/"
echo ""
echo "Your changes will NOT be tested because the driver is currently quarantined."
echo "To run Presto JDBC tests, add the label: break-quarantine-presto-jdbc"
echo ""
echo "If you merge without testing, you risk breaking the Presto JDBC driver!"
exit 1
- uses: actions/checkout@v6
- uses: ./.github/actions/await-port
with:
@@ -1165,19 +940,10 @@ jobs:
:only-tags [:mb/driver-tests]
:exclude-tags [:mb/transforms-python-test]
quarantine-engine: ${{ vars.QUARANTINE_ENGINE }}
- name: Upload Test Results
uses: ./.github/actions/upload-test-results
if: always()
with:
input-path: ./target/junit/
output-name: ${{ github.job }}
bucket: ${{ vars.AWS_S3_TEST_RESULTS_BUCKET }}
role-to-assume: ${{ secrets.CI_TEST_RESULTS_IAM_ROLE }}
aws-region: ${{ vars.AWS_REGION }}
be-tests-redshift-ee:
needs: determine-driver-skips
if: ${{ needs.determine-driver-skips.outputs.redshift-should-run == 'true' || needs.determine-driver-skips.outputs.redshift-quarantine-conflict == 'true' }}
if: ${{ needs.determine-driver-skips.outputs.redshift-should-run == 'true' }}
runs-on: ${{ vars.DEFAULT_RUNNER_KEY }}
timeout-minutes: 50
strategy:
@@ -1222,16 +988,6 @@ jobs:
MB_REDSHIFT_TEST_PASSWORD: ${{ secrets.MB_REDSHIFT_TEST_PASSWORD }}
name: ${{ matrix.job.name }}
steps:
- name: Fail on quarantine conflict
if: ${{ needs.determine-driver-skips.outputs.redshift-quarantine-conflict == 'true' }}
run: |
echo "::error::Redshift driver is quarantined but you changed files in modules/drivers/redshift/"
echo ""
echo "Your changes will NOT be tested because the driver is currently quarantined."
echo "To run Redshift tests, add the label: break-quarantine-redshift"
echo ""
echo "If you merge without testing, you risk breaking the Redshift driver!"
exit 1
- uses: actions/checkout@v6
- name: Setup python-runner
if: ${{ matrix.job.needs-python-runner == true }}
@@ -1248,15 +1004,6 @@ jobs:
${{ matrix.job.test-args }}
artifact-suffix: ${{ matrix.job.name }}
quarantine-engine: ${{ vars.QUARANTINE_ENGINE }}
- name: Upload Test Results
uses: ./.github/actions/upload-test-results
if: ${{ !matrix.job.skip && !cancelled() }}
with:
input-path: ./target/junit/
output-name: ${{ github.job }}
bucket: ${{ vars.AWS_S3_TEST_RESULTS_BUCKET }}
role-to-assume: ${{ secrets.CI_TEST_RESULTS_IAM_ROLE }}
aws-region: ${{ vars.AWS_REGION }}
- name: Cleanup python-runner
if: ${{ always() && matrix.job.needs-python-runner == true }}
uses: ./.github/actions/cleanup-python-runner
@@ -1266,7 +1013,7 @@ jobs:
be-tests-snowflake-ee:
needs: determine-driver-skips
if: ${{ needs.determine-driver-skips.outputs.snowflake-should-run == 'true' || needs.determine-driver-skips.outputs.snowflake-quarantine-conflict == 'true' }}
if: ${{ needs.determine-driver-skips.outputs.snowflake-should-run == 'true' }}
runs-on: ${{ vars.DEFAULT_RUNNER_KEY }}
timeout-minutes: 70
strategy:
@@ -1306,16 +1053,6 @@ jobs:
MB_SNOWFLAKE_TEST_RSA_ROLE_TEST_DB: RSA_ROLE_TEST_DB
name: Snowflake ${{ matrix.job.name }}
steps:
- name: Fail on quarantine conflict
if: ${{ needs.determine-driver-skips.outputs.snowflake-quarantine-conflict == 'true' }}
run: |
echo "::error::Snowflake driver is quarantined but you changed files in modules/drivers/snowflake/"
echo ""
echo "Your changes will NOT be tested because the driver is currently quarantined."
echo "To run Snowflake tests, add the label: break-quarantine-snowflake"
echo ""
echo "If you merge without testing, you risk breaking the Snowflake driver!"
exit 1
- uses: actions/checkout@v6
- name: Setup python-runner
if: ${{ matrix.job.needs-python-runner == true }}
@@ -1330,15 +1067,6 @@ jobs:
test-args: ${{ matrix.job.test-args }}
artifact-suffix: ${{ matrix.job.name }}
quarantine-engine: ${{ vars.QUARANTINE_ENGINE }}
- name: Upload Test Results
uses: ./.github/actions/upload-test-results
if: always()
with:
input-path: ./target/junit/
output-name: ${{ github.job }}
bucket: ${{ vars.AWS_S3_TEST_RESULTS_BUCKET }}
role-to-assume: ${{ secrets.CI_TEST_RESULTS_IAM_ROLE }}
aws-region: ${{ vars.AWS_REGION }}
- name: Cleanup python-runner
if: ${{ always() && matrix.job.needs-python-runner == true }}
uses: ./.github/actions/cleanup-python-runner
@@ -1348,7 +1076,7 @@ jobs:
be-tests-sparksql-ee:
needs: determine-driver-skips
if: ${{ needs.determine-driver-skips.outputs.sparksql-should-run == 'true' || needs.determine-driver-skips.outputs.sparksql-quarantine-conflict == 'true' }}
if: ${{ needs.determine-driver-skips.outputs.sparksql-should-run == 'true' }}
runs-on: ${{ vars.DEFAULT_RUNNER_KEY }}
timeout-minutes: 40
env:
@@ -1361,16 +1089,6 @@ jobs:
- "10000:10000"
name: Spark SQL
steps:
- name: Fail on quarantine conflict
if: ${{ needs.determine-driver-skips.outputs.sparksql-quarantine-conflict == 'true' }}
run: |
echo "::error::SparkSQL driver is quarantined but you changed files in modules/drivers/sparksql/"
echo ""
echo "Your changes will NOT be tested because the driver is currently quarantined."
echo "To run SparkSQL tests, add the label: break-quarantine-sparksql"
echo ""
echo "If you merge without testing, you risk breaking the SparkSQL driver!"
exit 1
- uses: actions/checkout@v6
- name: Test Spark driver
id: test-driver
@@ -1381,19 +1099,10 @@ jobs:
:only-tags [:mb/driver-tests]
:exclude-tags [:mb/transforms-python-test]
quarantine-engine: ${{ vars.QUARANTINE_ENGINE }}
- name: Upload Test Results
uses: ./.github/actions/upload-test-results
if: always()
with:
input-path: ./target/junit/
output-name: ${{ github.job }}
bucket: ${{ vars.AWS_S3_TEST_RESULTS_BUCKET }}
role-to-assume: ${{ secrets.CI_TEST_RESULTS_IAM_ROLE }}
aws-region: ${{ vars.AWS_REGION }}
be-tests-sqlite-ee:
needs: determine-driver-skips
if: ${{ needs.determine-driver-skips.outputs.sqlite-should-run == 'true' || needs.determine-driver-skips.outputs.sqlite-quarantine-conflict == 'true' }}
if: ${{ needs.determine-driver-skips.outputs.sqlite-should-run == 'true' }}
runs-on: ${{ vars.DEFAULT_RUNNER_KEY }}
timeout-minutes: 40
env:
@@ -1401,16 +1110,6 @@ jobs:
DRIVERS: sqlite
name: SQLite
steps:
- name: Fail on quarantine conflict
if: ${{ needs.determine-driver-skips.outputs.sqlite-quarantine-conflict == 'true' }}
run: |
echo "::error::SQLite driver is quarantined but you changed files that affect it"
echo ""
echo "Your changes will NOT be tested because the driver is currently quarantined."
echo "To run SQLite tests, add the label: break-quarantine-sqlite"
echo ""
echo "If you merge without testing, you risk breaking the SQLite driver!"
exit 1
- uses: actions/checkout@v6
- name: Test SQLite driver
id: test-driver
@@ -1421,19 +1120,10 @@ jobs:
:only-tags [:mb/driver-tests]
:exclude-tags [:mb/transforms-python-test]
quarantine-engine: ${{ vars.QUARANTINE_ENGINE }}
- name: Upload Test Results
uses: ./.github/actions/upload-test-results
if: always()
with:
input-path: ./target/junit/
output-name: ${{ github.job }}
bucket: ${{ vars.AWS_S3_TEST_RESULTS_BUCKET }}
role-to-assume: ${{ secrets.CI_TEST_RESULTS_IAM_ROLE }}
aws-region: ${{ vars.AWS_REGION }}
be-tests-sqlserver:
needs: determine-driver-skips
if: ${{ needs.determine-driver-skips.outputs.sqlserver-should-run == 'true' || needs.determine-driver-skips.outputs.sqlserver-quarantine-conflict == 'true' }}
if: ${{ needs.determine-driver-skips.outputs.sqlserver-should-run == 'true' }}
runs-on: ${{ vars.DEFAULT_RUNNER_KEY }}
timeout-minutes: 30
strategy:
@@ -1472,16 +1162,6 @@ jobs:
MSSQL_MEMORY_LIMIT_MB: 1024
name: ${{ matrix.version.name }} ${{ matrix.job.name }}
steps:
- name: Fail on quarantine conflict
if: ${{ needs.determine-driver-skips.outputs.sqlserver-quarantine-conflict == 'true' }}
run: |
echo "::error::SQL Server driver is quarantined but you changed files in modules/drivers/sqlserver/"
echo ""
echo "Your changes will NOT be tested because the driver is currently quarantined."
echo "To run SQL Server tests, add the label: break-quarantine-sqlserver"
echo ""
echo "If you merge without testing, you risk breaking the SQL Server driver!"
exit 1
- uses: actions/checkout@v6
- name: Setup python-runner
if: ${{ matrix.job.needs-python-runner == true }}
@@ -1496,15 +1176,6 @@ jobs:
test-args: ${{ matrix.job.test-args }}
artifact-suffix: ${{ matrix.version.name }}-${{ matrix.job.name }}
quarantine-engine: ${{ vars.QUARANTINE_ENGINE }}
- name: Upload Test Results
uses: ./.github/actions/upload-test-results
if: always()
with:
input-path: ./target/junit/
output-name: ${{ github.job }}
bucket: ${{ vars.AWS_S3_TEST_RESULTS_BUCKET }}
role-to-assume: ${{ secrets.CI_TEST_RESULTS_IAM_ROLE }}
aws-region: ${{ vars.AWS_REGION }}
- name: Cleanup python-runner
if: ${{ always() && matrix.job.needs-python-runner == true }}
uses: ./.github/actions/cleanup-python-runner
@@ -1539,20 +1210,11 @@ jobs:
# junit-name: 'be-tests-starburst-ee'
# test-args: >-
# :only-tags [:mb/driver-tests]
# - name: Upload Test Results
# uses: ./.github/actions/upload-test-results
# if: always()
# with:
# input-path: ./target/junit/
# output-name: ${{ github.job }}
# bucket: ${{ vars.AWS_S3_TEST_RESULTS_BUCKET }}
# role-to-assume: ${{ secrets.CI_TEST_RESULTS_IAM_ROLE }}
# aws-region: ${{ vars.AWS_REGION }}
login-to-ecr:
needs: determine-driver-skips
if: ${{ needs.determine-driver-skips.outputs.vertica-should-run == 'true' || needs.determine-driver-skips.outputs.vertica-quarantine-conflict == 'true' }}
runs-on: ${{ vars.DEFAULT_RUNNER_KEY }}
if: ${{ needs.determine-driver-skips.outputs.vertica-should-run == 'true' }}
runs-on: ${{ vars.SLIM_RUNNER_KEY }}
timeout-minutes: 15
permissions:
id-token: write
@@ -1583,7 +1245,7 @@ jobs:
be-tests-vertica-ee:
needs: [determine-driver-skips, login-to-ecr]
if: ${{ needs.determine-driver-skips.outputs.vertica-should-run == 'true' || needs.determine-driver-skips.outputs.vertica-quarantine-conflict == 'true' }}
if: ${{ needs.determine-driver-skips.outputs.vertica-should-run == 'true' }}
runs-on: ${{ vars.DEFAULT_RUNNER_KEY }}
timeout-minutes: 40
env:
@@ -1599,16 +1261,6 @@ jobs:
password: ${{ needs.login-to-ecr.outputs.docker_password }}
name: Vertica
steps:
- name: Fail on quarantine conflict
if: ${{ needs.determine-driver-skips.outputs.vertica-quarantine-conflict == 'true' }}
run: |
echo "::error::Vertica driver is quarantined but you changed files in modules/drivers/vertica/"
echo ""
echo "Your changes will NOT be tested because the driver is currently quarantined."
echo "To run Vertica tests, add the label: break-quarantine-vertica"
echo ""
echo "If you merge without testing, you risk breaking the Vertica driver!"
exit 1
- uses: actions/checkout@v6
- name: Make plugins directory
run: mkdir plugins
@@ -1621,42 +1273,34 @@ jobs:
:only-tags [:mb/driver-tests]
:exclude-tags [:mb/transforms-python-test]
quarantine-engine: ${{ vars.QUARANTINE_ENGINE }}
- name: Upload Test Results
uses: ./.github/actions/upload-test-results
if: always()
with:
input-path: ./target/junit/
output-name: ${{ github.job }}
bucket: ${{ vars.AWS_S3_TEST_RESULTS_BUCKET }}
role-to-assume: ${{ secrets.CI_TEST_RESULTS_IAM_ROLE }}
aws-region: ${{ vars.AWS_REGION }}
drivers-tests-result:
needs:
- determine-driver-skips
# if you add a driver job, you must add it to this list for it to be a required check
- be-tests-h2
- be-tests-athena-ee
- be-tests-snowflake-ee
- be-tests-bigquery-cloud-sdk-ee
- be-tests-clickhouse-ee
- be-tests-databricks-ee
- be-tests-databricks-multi-catalog-ee
- be-tests-druid-jdbc-ee
- be-tests-mysql-mariadb
- be-tests-h2
- be-tests-h2-oss
- be-tests-mongo
- be-tests-mongo-ssl
- be-tests-mongo-sharded-cluster-ee
- be-tests-mongo-ssl
- be-tests-mysql-mariadb
- be-tests-oracle
- be-tests-postgres
- be-tests-presto-jdbc-ee
- be-tests-redshift-ee
- be-tests-snowflake-ee
- be-tests-sparksql-ee
- be-tests-sqlite-ee
- be-tests-sqlserver
- be-tests-vertica-ee
# - be-tests-starburst-ee
# TODO(rileythomp, 2025-08-20): Re-enable vertica tests once the docker image is updated
# - be-tests-vertica-ee
- be-tests-clickhouse-ee
runs-on: ${{ vars.DEFAULT_RUNNER_KEY }}
runs-on: ${{ vars.SLIM_RUNNER_KEY }}
timeout-minutes: 5
name: drivers-tests-result
if: always() && !cancelled()

View File

@@ -66,7 +66,8 @@ jobs:
# one place (SHARDS below); the matrix is generated from it.
setup:
name: Build shard matrix
runs-on: ubuntu-22.04
runs-on: ${{ vars.SLIM_RUNNER_KEY }}
timeout-minutes: 5
outputs:
matrix: ${{ steps.gen.outputs.matrix }}
count: ${{ steps.gen.outputs.count }}

View File

@@ -21,7 +21,7 @@ on:
jobs:
build-matrix:
runs-on: ${{ vars.DEFAULT_RUNNER_KEY }}
runs-on: ${{ vars.SLIM_RUNNER_KEY }}
timeout-minutes: 5
outputs:
matrix: ${{ steps.matrix.outputs.config }}
@@ -35,7 +35,7 @@ jobs:
sparse-checkout-cone-mode: false
- uses: actions/setup-node@v6.4.0
with:
node-version: lts/Jod # 22.x.x
node-version: lts/Krypton # 24.x.x
- name: Build e2e matrix
uses: actions/github-script@v9
id: matrix

View File

@@ -88,7 +88,7 @@ on:
jobs:
determine-test-type:
name: Determine test type
runs-on: ${{ vars.DEFAULT_RUNNER_KEY }}
runs-on: ${{ vars.SLIM_RUNNER_KEY }}
timeout-minutes: 5
outputs:
type: ${{ steps.detect.outputs.type }}
@@ -106,7 +106,7 @@ jobs:
workflow-summary:
name: Stress test inputs
needs: [determine-test-type]
runs-on: ${{ vars.DEFAULT_RUNNER_KEY }}
runs-on: ${{ vars.SLIM_RUNNER_KEY }}
timeout-minutes: 5
steps:
- name: Generate workflow summary

View File

@@ -41,7 +41,7 @@ env:
# (e.g. "https://conductor.<host>"); each consumer appends the path it needs
# (the reporter POSTs to "/webhooks/failed-tests"). Callers pass
# `secrets: inherit`; reporting no-ops when it's unset.
CI_CONDUCTOR_BASE_URL: ${{ secrets.CI_CONDUCTOR_BASE_URL }}
CI_CONDUCTOR_BASE_URL: ${{ vars.CI_CONDUCTOR_BASE_URL }}
# Shared secret sent as the `x-internal-secret` header to authenticate.
CI_CONDUCTOR_WEBHOOK_SECRET: ${{ secrets.CI_CONDUCTOR_WEBHOOK_SECRET }}
REPO_ID: ${{ github.event.repository.id }}
@@ -216,17 +216,6 @@ jobs:
echo "No ./target/junit directory; nothing to process."
fi
- name: Upload Test Results
uses: ./.github/actions/upload-test-results
if: always()
with:
input-path: ./target/junit
# Related to our custom upload to S3
output-name: e2e-${{ inputs.name }}
bucket: ${{ vars.AWS_S3_TEST_RESULTS_BUCKET }}
role-to-assume: ${{ secrets.CI_TEST_RESULTS_IAM_ROLE }}
aws-region: ${{ vars.AWS_REGION }}
- name: Quarantine gate
# Only one of the two run steps executes; the other reports `skipped`. Only gate on actual failure.
if: ${{ !cancelled() && (steps.split-e2e.outcome == 'failure' || steps.tagged-e2e.outcome == 'failure') }}

View File

@@ -24,7 +24,7 @@ jobs:
reset-e2e-test-comment:
if: ${{ github.event_name == 'pull_request' }}
runs-on: ${{ vars.DEFAULT_RUNNER_KEY }}
runs-on: ${{ vars.SLIM_RUNNER_KEY }}
timeout-minutes: 5
steps:
- name: Remove test results comment
@@ -138,7 +138,7 @@ jobs:
e2e-tests-result:
needs:
- e2e-tests
runs-on: ${{ vars.DEFAULT_RUNNER_KEY }}
runs-on: ${{ vars.SLIM_RUNNER_KEY }}
timeout-minutes: 5
name: e2e-tests-result
if: always() && !cancelled()

View File

@@ -8,7 +8,7 @@ env:
TERM: xterm
TZ: US/Pacific # to make Node match the instance tz
MB_EDITION: "ee"
CI_CONDUCTOR_BASE_URL: ${{ secrets.CI_CONDUCTOR_BASE_URL }}
CI_CONDUCTOR_BASE_URL: ${{ vars.CI_CONDUCTOR_BASE_URL }}
CI_CONDUCTOR_WEBHOOK_SECRET: ${{ secrets.CI_CONDUCTOR_WEBHOOK_SECRET }}
REPO_ID: ${{ github.event.repository.id }}
COMMIT_SHA: ${{ github.event.pull_request.head.sha || github.sha }}
@@ -111,16 +111,6 @@ jobs:
echo "No ./target/junit directory; nothing to process."
fi
- name: Upload Test Results
uses: ./.github/actions/upload-test-results
if: always()
with:
input-path: ./target/junit
output-name: e2e-embedding-sdk-react-${{ matrix.react-version }}
bucket: ${{ vars.AWS_S3_TEST_RESULTS_BUCKET }}
role-to-assume: ${{ secrets.CI_TEST_RESULTS_IAM_ROLE }}
aws-region: ${{ vars.AWS_REGION }}
- name: Quarantine gate
if: ${{ !cancelled() && steps.run-tests.outcome == 'failure' }}
continue-on-error: ${{ vars.QUARANTINE_ENGINE != 'ci-conductor' }}
@@ -159,7 +149,8 @@ jobs:
await core.summary.addRaw(summary).write();
decide-target-ref:
runs-on: ${{ vars.DEFAULT_RUNNER_KEY }}
runs-on: ${{ vars.SLIM_RUNNER_KEY }}
timeout-minutes: 5
outputs:
target-ref: ${{ steps.get-target-ref.outputs.value }}
steps:
@@ -186,6 +177,7 @@ jobs:
timeout-minutes: 45
env:
MB_RUN_MODE: "e2e"
MB_WAREHOUSE_ALLOWED_NETWORKS: "allow-all"
METASTORE_DEV_SERVER_URL: ${{ secrets.METASTORE_DEV_SERVER_URL }}
CI_CONDUCTOR_TEST_SUITE: embedding-sdk-component-backward-compatibility
# Sensitive env vars (tokens, secrets) starting with `CYPRESS_` are accessed via async `cy.env()`
@@ -290,16 +282,6 @@ jobs:
echo "No ./target/junit directory; nothing to process."
fi
- name: Upload Test Results
uses: ./.github/actions/upload-test-results
if: always()
with:
input-path: ./target/junit
output-name: e2e-embedding-sdk
bucket: ${{ vars.AWS_S3_TEST_RESULTS_BUCKET }}
role-to-assume: ${{ secrets.CI_TEST_RESULTS_IAM_ROLE }}
aws-region: ${{ vars.AWS_REGION }}
- name: Quarantine gate
if: ${{ !cancelled() && steps.run-tests.outcome == 'failure' }}
continue-on-error: ${{ vars.QUARANTINE_ENGINE != 'ci-conductor' }}

View File

@@ -100,6 +100,7 @@ jobs:
env:
MB_EDITION: ee
MB_RUN_MODE: e2e
MB_WAREHOUSE_ALLOWED_NETWORKS: "allow-all"
METASTORE_DEV_SERVER_URL: ${{ secrets.METASTORE_DEV_SERVER_URL }}
CYPRESS_MB_ALL_FEATURES_TOKEN: ${{ secrets.STAGING_MB_ALL_FEATURES_TOKEN }}
SAMPLE_APP_BRANCH_NAME: ${{ needs.get-sample-apps-data.outputs.branch_name }}
@@ -211,6 +212,7 @@ jobs:
env:
MB_EDITION: ee
MB_RUN_MODE: e2e
MB_WAREHOUSE_ALLOWED_NETWORKS: "allow-all"
METASTORE_DEV_SERVER_URL: ${{ secrets.METASTORE_DEV_SERVER_URL }}
CYPRESS_MB_ALL_FEATURES_TOKEN: ${{ secrets.STAGING_MB_ALL_FEATURES_TOKEN }}
SAMPLE_APP_BRANCH_NAME: ${{ needs.get-sample-apps-data.outputs.branch_name }}
@@ -278,6 +280,7 @@ jobs:
env:
MB_EDITION: ee
MB_RUN_MODE: e2e
MB_WAREHOUSE_ALLOWED_NETWORKS: "allow-all"
METASTORE_DEV_SERVER_URL: ${{ secrets.METASTORE_DEV_SERVER_URL }}
CYPRESS_MB_ALL_FEATURES_TOKEN: ${{ secrets.STAGING_MB_ALL_FEATURES_TOKEN }}
SAMPLE_APP_BRANCH_NAME: ${{ needs.get-sample-apps-data.outputs.branch_name }}
@@ -373,7 +376,7 @@ jobs:
# Host apps are the only suite here running through Metabase's Cypress config,
# whose `after:spec` hook reports failures and writes QUARANTINE_FAILURES_FILE.
# Sample-app suites load their own config (no-op), so ci-conductor is scoped to this job.
CI_CONDUCTOR_BASE_URL: ${{ secrets.CI_CONDUCTOR_BASE_URL }}
CI_CONDUCTOR_BASE_URL: ${{ vars.CI_CONDUCTOR_BASE_URL }}
CI_CONDUCTOR_WEBHOOK_SECRET: ${{ secrets.CI_CONDUCTOR_WEBHOOK_SECRET }}
CI_CONDUCTOR_TEST_SUITE: embedding-sdk-host-app
REPO_ID: ${{ github.event.repository.id }}

View File

@@ -5,12 +5,11 @@ on:
types: [auto_merge_enabled, review_requested, labeled, unlabeled, synchronize]
branches:
- "master"
merge_group:
jobs:
files-changed:
name: Check which files changed
runs-on: ${{ vars.DEFAULT_RUNNER_KEY }}
runs-on: ${{ vars.SLIM_RUNNER_KEY }}
timeout-minutes: 3
outputs:
frontend_embedding_sdk_package_sources: ${{ steps.changes.outputs.frontend_embedding_sdk_package_sources }}

View File

@@ -133,7 +133,7 @@ jobs:
- embedding-sdk-api-documentation-generation-validation
- embedding-sdk-component-tests
if: always() && !cancelled()
runs-on: ${{ vars.DEFAULT_RUNNER_KEY }}
runs-on: ${{ vars.SLIM_RUNNER_KEY }}
timeout-minutes: 5
env:
needs: ${{ toJson(needs) }}

View File

@@ -1,4 +1,4 @@
# Weekly count of files importing @emotion/*, uploaded to stats.metabase.com.
# Weekly count of files importing @emotion/*, imported into eng-stats-importer.
# Feeds the "Emotion CSS Elimination Progress" chart tracking the migration
# off Emotion onto Mantine (+ CSS modules as a fallback).
name: Emotion Stats Update
@@ -11,10 +11,11 @@ on:
jobs:
emotion-stats:
if: github.repository == 'metabase/metabase'
runs-on: ${{ vars.DEFAULT_RUNNER_KEY }}
runs-on: ${{ vars.SLIM_RUNNER_KEY }}
timeout-minutes: 5
steps:
- uses: actions/checkout@v6
- uses: ./.github/actions/prepare-node-bun
- name: Get Emotion file count
id: get-emotion-file-count
run: |
@@ -26,8 +27,6 @@ jobs:
git grep -l -E "(from|import) ['\"]@emotion/" -- \
'frontend/**/*.ts' \
'frontend/**/*.tsx' \
'frontend/**/*.js' \
'frontend/**/*.jsx' \
| wc -l || true
)
@@ -35,8 +34,6 @@ jobs:
git grep -l -E "(from|import) ['\"]@emotion/" -- \
'enterprise/frontend/**/*.ts' \
'enterprise/frontend/**/*.tsx' \
'enterprise/frontend/**/*.js' \
'enterprise/frontend/**/*.jsx' \
| wc -l || true
)
@@ -44,35 +41,8 @@ jobs:
echo "EMOTION_FILES=$((oss + ee))" >> "$GITHUB_OUTPUT"
- name: Upload count to Stats
uses: actions/github-script@v9
env:
EMOTION_FILES: ${{ steps.get-emotion-file-count.outputs.EMOTION_FILES }}
API_KEY: ${{ secrets.STATS_API_KEY }}
with:
script: | # js
const { uploadCsvToMb } = require('${{ github.workspace }}/.github/scripts/csv-to-mb.js');
console.log("EMOTION_FILES", Number(process.env.EMOTION_FILES));
// Keys become CSV headers and must match the target table's columns
// (Metabase slugifies both sides, so casing/spacing is forgiving).
const data = [
{
'Date': new Date().toISOString().slice(0, 10), // YYYY-MM-DD
'Emotion Files': Number(process.env.EMOTION_FILES),
}
];
// tableId 76283 = "Emotion Files" table on stats.metabase.com.
// Auth via x-api-key, sourced from STATS_API_KEY (see env block above).
uploadCsvToMb({
baseUrl: 'https://stats.metabase.com',
tableId: 76283,
jsonData: data,
mode: 'append',
}).then(() => {
console.log('Data uploaded successfully');
}).catch((error) => {
console.error('Error uploading data:', error);
process.exit(1);
});
ENG_STATS_URL: ${{ secrets.ENG_STATS_URL }}
API_KEY: ${{ secrets.ENG_STATS_API_KEY }}
run: bun .github/scripts/upload-emotion-stats.ts

View File

@@ -1,38 +0,0 @@
name: Escalation Slack Notification
on:
issues:
types:
- labeled
jobs:
on-escalation-label:
if: github.event.label.name == '.Escalation'
runs-on: ${{ vars.DEFAULT_RUNNER_KEY }}
timeout-minutes: 10
steps:
- name: Format issue title (escape quotes)
uses: actions/github-script@v9
id: vars
env:
ISSUE_TITLE: ${{ github.event.issue.title }}
with:
script: |
core.setOutput('issue_title', process.env.ISSUE_TITLE.replaceAll(/"/g, '\\"'));
- name: Send escalated issue ${{ github.event.issue.number }} to Slack
id: slack
uses: slackapi/slack-github-action@45a88b9581bfab2566dc881e2cd66d334e621e2c # v3.0.3
with:
webhook: ${{ secrets.SLACK_ESCALATIONS_WEBHOOK_URL }}
webhook-type: incoming-webhook
payload: |
{
"blocks": [
{
"type": "section",
"text": {
"type": "mrkdwn",
"text": "🔥🔥🔥 ${{ steps.vars.outputs.issue_title }}\n${{ github.event.issue.html_url }}"
}
}
]
}

View File

@@ -19,7 +19,7 @@ on:
jobs:
workflow-summary:
name: Stress test inputs
runs-on: ${{ vars.DEFAULT_RUNNER_KEY }}
runs-on: ${{ vars.SLIM_RUNNER_KEY }}
timeout-minutes: 5
steps:
- name: Generate workflow summary

View File

@@ -35,7 +35,7 @@ env:
# Consumed by the report-failures-to-ci-conductor step in fe-tests-unit. Absent
# (e.g. fork PRs without secret access) the reporter no-ops. Matches the other
# suite workflows (backend, drivers, e2e, app-db, semantic-search).
CI_CONDUCTOR_BASE_URL: ${{ secrets.CI_CONDUCTOR_BASE_URL }}
CI_CONDUCTOR_BASE_URL: ${{ vars.CI_CONDUCTOR_BASE_URL }}
CI_CONDUCTOR_WEBHOOK_SECRET: ${{ secrets.CI_CONDUCTOR_WEBHOOK_SECRET }}
jobs:
@@ -50,14 +50,17 @@ jobs:
- name: Prepare back-end environment
uses: ./.github/actions/prepare-backend
- name: Get Eslint cache key
uses: ./.github/actions/get-cache-keys
id: get-cache-keys
- name: Restore ESLint cache
uses: actions/cache/restore@v5
- name: Cache spec
id: cache-spec
run: .github/scripts/cache-keys.sh | tee -a "$GITHUB_OUTPUT"
# Small enough, and written by few enough jobs, that this one reads and writes its own cache
# rather than waiting on cache-warm.
- name: ESLint cache
uses: actions/cache@v5
with:
path: ${{ github.workspace}}/.eslintcache
key: ${{ steps.get-cache-keys.outputs.eslint-cache-key }}
path: ${{ steps.cache-spec.outputs.eslint-path }}
key: ${{ steps.cache-spec.outputs.eslint-key }}
restore-keys: ${{ steps.cache-spec.outputs.eslint-restore-key }}
- name: Compile CLJS
run: bun run build-pure:cljs
@@ -113,7 +116,7 @@ jobs:
# Keep this step continue-on-error because the quarantine logic controls whether it ultimately fails.
continue-on-error: true
- name: Report failed tests to ci-conductor
if: ${{ !cancelled() && steps.fe-tests.outcome == 'failure' }}
if: ${{ !cancelled() && ( steps.fe-tests.outcome == 'failure' || github.run_attempt != '1' ) }}
# This step should NEVER fail the job!
continue-on-error: true
uses: ./.github/actions/report-failures-to-ci-conductor
@@ -129,15 +132,6 @@ jobs:
adapter: frontend
test-suite: fe-tests-unit
enforce-quarantine: ${{ vars.QUARANTINE_ENGINE == 'ci-conductor' }}
- name: Upload Test Results
uses: ./.github/actions/upload-test-results
if: always() && github.event_name != 'workflow_dispatch'
with:
input-path: ./target/junit
output-name: frontend-unit
bucket: ${{ vars.AWS_S3_TEST_RESULTS_BUCKET }}
role-to-assume: ${{ secrets.CI_TEST_RESULTS_IAM_ROLE }}
aws-region: ${{ vars.AWS_REGION }}
fe-tests-timezones:
if: ${{ !inputs.skip }}
@@ -182,7 +176,7 @@ jobs:
- fe-tests-timezones
- fe-custom-viz
if: always() && !cancelled()
runs-on: ${{ vars.DEFAULT_RUNNER_KEY }}
runs-on: ${{ vars.SLIM_RUNNER_KEY }}
timeout-minutes: 5
env:
needs: ${{ toJson(needs) }}

View File

@@ -1,58 +0,0 @@
name: Generative Query Test
on:
workflow_dispatch:
inputs:
context-seed:
description: 'Context seed (empty = random)'
type: number
iteration-seed:
description: 'Iteration seed (for iteration replay, empty = dependent on Context seed)'
type: number
iterations:
description: 'Iterations (number of test iterations, overrides seconds)'
type: number
seconds:
description: 'Seconds (Number of seconds after which next iteration will not start)'
test:
description: 'Test or namespace to run'
default: metabase.query-processor.generative-test/execution-with-cards-test
ref:
description: 'Ref to run generative tests on'
default: qgen-ms2
# TODO: Consider renaming the job in next milestones
jobs:
generative-query-test:
runs-on: ${{ vars.DEFAULT_RUNNER_KEY }}
timeout-minutes: 90
name: Generative Query Test
env:
CI: 'true'
MB_GENTEST_CONTEXT_SEED: ${{ inputs.context-seed }}
MB_GENTEST_ITERATION_SEED: ${{ inputs.iteration-seed }}
MB_GENTEST_LIMIT_ITERATIONS: ${{ inputs.iterations }}
MB_GENTEST_LIMIT_SECONDS: ${{ inputs.seconds }}
MB_GENTEST_RUN: true
steps:
- uses: actions/checkout@v6
with:
ref: ${{ inputs.ref }}
- name: Prepare back-end environment
uses: ./.github/actions/prepare-backend
- name: Prepare front-end environment
uses: ./.github/actions/prepare-frontend
- name: Run test
run: clojure -X:dev:ci:test:ee:ee-dev :only '${{ inputs.test }}'
- name: Publish Test Report (JUnit)
uses: dorny/test-reporter@a43b3a5f7366b97d083190328d2c652e1a8b6aa2 # v3.0.0
if: failure()
with:
path: "target/junit/**/*_test.xml"
name: JUnit Test Report (Generative Query test)
reporter: java-junit
list-suites: failed
list-tests: failed
fail-on-error: false

View File

@@ -1,51 +0,0 @@
name: i18n
on:
push:
branches:
- 'master'
- 'release-**'
pull_request:
concurrency:
group: ${{ github.workflow }}-${{ github.head_ref && github.ref || github.run_id }}
cancel-in-progress: true
jobs:
files-changed:
name: Check which files changed
runs-on: ${{ vars.DEFAULT_RUNNER_KEY }}
timeout-minutes: 3
outputs:
i18n: ${{ steps.changes.outputs.i18n }}
steps:
- uses: actions/checkout@v6
- name: Test which files changed
uses: dorny/paths-filter@fbd0ab8f3e69293af611ebaee6363fc25e6d187d # v4.0.1
id: changes
with:
token: ${{ github.token }}
filters: .github/file-paths.yaml
verify-i18n-files:
needs: files-changed
if: needs.files-changed.outputs.i18n == 'true'
runs-on: ${{ vars.DEFAULT_RUNNER_KEY }}
timeout-minutes: 15
steps:
- uses: actions/checkout@v6
- name: Prepare front-end environment
uses: ./.github/actions/prepare-frontend
- name: Prepare back-end environment
uses: ./.github/actions/prepare-backend
- name: Install xgettext if necessary
run: |
if ! command -v xgettext > /dev/null; then
sudo apt install gettext
fi
- name: Check xgettext version
run: xgettext --version
- name: Check i18n tags/make sure template can be built
run: ./bin/i18n/update-translation-template
- name: Verify i18n translations (.po files)
run: ./bin/i18n/build-translation-resources

View File

@@ -1,52 +0,0 @@
name: JS Stats Update
on:
workflow_dispatch:
schedule:
- cron: 40 1 * * 4 # every Thursday at 1:40
jobs:
js-stats:
if: ${{ github.event_name != 'schedule' || github.repository == 'metabase/metabase' }}
runs-on: ${{ vars.DEFAULT_RUNNER_KEY }}
timeout-minutes: 5
steps:
- uses: actions/checkout@v6
- name: get JS file count
id: get-js-file-count
run: |
oss=$(find ./frontend/src/metabase -type f -name "*.js" -o -name "*.jsx" | wc -l)
ee=$(find ./enterprise/frontend/src/metabase-enterprise -type f -name "*.js" -o -name "*.jsx" | wc -l)
echo $((oss + ee)) javascript files
echo "JS_FILES=$((oss + ee))" >> "$GITHUB_OUTPUT"
- name: Upload count to Stats
uses: actions/github-script@v9
env:
JS_FILES: ${{ steps.get-js-file-count.outputs.JS_FILES }}
API_KEY: ${{ secrets.STATS_API_KEY }}
with:
script: | # js
const { uploadCsvToMb } = require('${{ github.workspace }}/.github/scripts/csv-to-mb.js');
console.log("JS_FILES", Number(process.env.JS_FILES));
const data = [
{
'Date': new Date().toISOString().slice(0, 10), // YYYY-MM-DD
'Js files': Number(process.env.JS_FILES),
}
];
uploadCsvToMb({
baseUrl: 'https://stats.metabase.com',
tableId: 73270,
jsonData: data,
mode: 'append',
}).then(() => {
console.log('Data uploaded successfully');
}).catch((error) => {
console.error('Error uploading data:', error);
process.exit(1);
});

View File

@@ -37,6 +37,8 @@ jobs:
with:
ref: ${{ github.event.pull_request.head.ref }}
persist-credentials: false
- name: Prepare back-end environment
uses: ./.github/actions/prepare-backend
- name: Lower ignore budgets to match the source tree
run: ./bin/mage fix-kondo-ratchets
- name: Check for changes

View File

@@ -6,7 +6,8 @@ on:
jobs:
add-issue-references:
runs-on: ${{ vars.DEFAULT_RUNNER_KEY }}
runs-on: ${{ vars.SLIM_RUNNER_KEY }}
timeout-minutes: 5
steps:
- name: Checkout code
uses: actions/checkout@v6

111
.github/workflows/lint.security.yml vendored Normal file
View File

@@ -0,0 +1,111 @@
name: Lint Security
on:
schedule:
- cron: "0 11 * * *" # Daily at 11:00 UTC
workflow_dispatch: # manually run security scans
concurrency:
group: ${{ github.workflow }}-${{ github.ref }}
cancel-in-progress: true
permissions:
contents: read
security-events: write
jobs:
codeql:
runs-on: ${{ vars.DEFAULT_RUNNER_KEY }}
timeout-minutes: 10
env:
# disables overlay DB cache
CODEQL_OVERLAY_DATABASE_MODE: none
steps:
- uses: actions/checkout@v6
- name: Initialize CodeQL
uses: github/codeql-action/init@v4
with:
languages: javascript
config-file: ./.github/codeql/codeql-config.yml
# disable caching since this only runs daily and the CodeQL cache is large
trap-caching: false
dependency-caching: false
- name: Perform CodeQL Analysis
uses: github/codeql-action/analyze@v4
trivy:
strategy:
fail-fast: false
matrix:
include:
- edition: ee
repo: metabase/metabase-enterprise-head
tag: latest
- edition: oss
repo: metabase/metabase-head
tag: latest
runs-on: ${{ vars.DEFAULT_RUNNER_KEY }}
name: Run Trivy for (${{ matrix.edition }})
steps:
- uses: actions/checkout@v6
- name: Run Vulnerability Scan
uses: aquasecurity/trivy-action@ed142fd0673e97e23eac54620cfb913e5ce36c25 # v0.36.0
env:
TRIVY_OFFLINE_SCAN: true
with:
image-ref: docker.io/${{ matrix.repo }}:${{ matrix.tag }}
format: sarif
output: trivy-results-${{ matrix.edition }}.sarif
# disable caching since this only runs daily so rate limiting isn't an issue
cache: false
- name: Upload Results to Github Security
uses: github/codeql-action/upload-sarif@v4
with:
sarif_file: trivy-results-${{ matrix.edition }}.sarif
category: trivy-${{ matrix.edition }}
notify:
needs: [codeql, trivy]
if: ${{ failure() && github.repository == 'metabase/metabase' }}
runs-on: ${{ vars.DEFAULT_RUNNER_KEY }}
timeout-minutes: 5
steps:
- uses: actions/checkout@v6
with:
sparse-checkout: |
release
.github
- name: Prepare release scripts
uses: ./.github/actions/prepare-release-scripts
- name: Notify Slack of Failed Scans
uses: actions/github-script@v9
env:
SLACK_BOT_TOKEN: ${{ secrets.SLACK_BOT_TOKEN }}
with:
script: | # js
const { sendSlackMessage } = require('${{ github.workspace }}/release/dist/index.cjs');
const scans = [
['CodeQL', '${{ needs.codeql.result }}'],
['Trivy', '${{ needs.trivy.result }}'],
];
const failed = scans.filter(([, result]) => result === 'failure').map(([name]) => name);
if (failed.length === 0) {
console.log('No scan reported a failure. Skipping Slack notification.');
return;
}
const runUrl = `https://github.com/${context.repo.owner}/${context.repo.repo}/actions/runs/${context.runId}`;
const list = failed.map(name => `\`${name}\``).join(', ');
await sendSlackMessage({
channelName: 'engineering-ci',
message: `:rotating_light: Daily security scan failed (${list}) — see <${runUrl}|run logs>`,
}).catch(console.error);

173
.github/workflows/lint.yml vendored Normal file
View File

@@ -0,0 +1,173 @@
name: Lint
on:
push:
branches:
- 'master'
- 'release-**'
pull_request:
concurrency:
group: ${{ github.workflow }}-${{ github.head_ref && github.ref || github.run_id }}
cancel-in-progress: true
jobs:
files-changed:
runs-on: ${{ vars.SLIM_RUNNER_KEY }}
timeout-minutes: 10
outputs:
actionlint: ${{ steps.changes.outputs.actionlint }}
actionlint_files: ${{ steps.changes.outputs.actionlint_files }}
codeql: ${{ steps.changes.outputs.codeql }}
i18n: ${{ steps.changes.outputs.i18n }}
openapi: ${{ steps.changes.outputs.openapi }}
snowplow: ${{ steps.changes.outputs.snowplow }}
yaml: ${{ steps.changes.outputs.yaml }}
steps:
- uses: actions/checkout@v6
- name: Test which files changed
uses: dorny/paths-filter@fbd0ab8f3e69293af611ebaee6363fc25e6d187d # v4.0.1
id: changes
with:
token: ${{ github.token }}
filters: .github/file-paths.yaml
list-files: shell
lint:
needs: files-changed
runs-on: ${{ vars.DEFAULT_RUNNER_KEY }}
timeout-minutes: 10
steps:
- uses: actions/checkout@v6
- name: Setup frontend env
if: |
needs.files-changed.outputs.yaml == 'true' ||
needs.files-changed.outputs.openapi == 'true'
uses: ./.github/actions/prepare-frontend
# Always required by whitespace linter, also used by OpenAPI spec validation
- name: Setup backend env
uses: ./.github/actions/prepare-backend
- name: Setup actionlint
if: needs.files-changed.outputs.actionlint == 'true'
id: download
shell: bash
run: |
if command -v actionlint; then
echo "executable=actionlint" >> "$GITHUB_OUTPUT"
else
# script sets executable output, for details see: https://github.com/rhysd/actionlint/blob/main/docs/usage.md#use-actionlint-on-github-actions
bash <(curl -fsSL https://raw.githubusercontent.com/rhysd/actionlint/v1.7.12/scripts/download-actionlint.bash)
fi
- parallel:
# Deliberately ungated. This guards a coupling that drifts silently, so running it only when a
# path filter says .github changed would reintroduce the failure it exists to catch - and the
# filters here cover .github/workflows only, not the actions and scripts it also reads.
- name: Check cache-warm pre-resolves every alias combination CI uses
run: .github/scripts/check-preresolve-aliases.sh
# Lint whitespace, always runs
- name: Run Whitespace Linter
run: clojure -T:whitespace-linter lint :whitespace-linter/progress-bar false
# Lint YAML
- name: Lint YAML files
if: needs.files-changed.outputs.yaml == 'true'
run: bun run lint-yaml
# Lint Github Actions
- name: Run actionlint
if: needs.files-changed.outputs.actionlint == 'true'
run: ${{ steps.download.outputs.executable }} --color --verbose ${{ needs.files-changed.outputs.actionlint_files }}
# Generate and Lint OpenAPI spec
- name: Generate OpenAPI spec
if: needs.files-changed.outputs.openapi == 'true'
run: bun run generate-openapi
# Parallel steps run asynchronously, and explicitly waiting for a parallel step means "if" statements are ignored.
# Move OpenAPI validation outside the parallel block so that spec generation happens asynchronously without requiring
# an explicit "wait".
- name: Validate OpenAPI spec
if: needs.files-changed.outputs.openapi == 'true'
run: bun run lint-openapi
i18n:
needs: files-changed
if: needs.files-changed.outputs.i18n == 'true'
runs-on: ${{ vars.DEFAULT_RUNNER_KEY }}
timeout-minutes: 15
steps:
- uses: actions/checkout@v6
- name: Prepare front-end environment
uses: ./.github/actions/prepare-frontend
- name: Prepare back-end environment
uses: ./.github/actions/prepare-backend
- name: Install xgettext if necessary
run: |
if ! command -v xgettext > /dev/null; then
sudo apt install gettext
fi
- name: Check xgettext version
run: xgettext --version
- name: Check i18n tags/make sure template can be built
run: ./bin/i18n/update-translation-template
- name: Verify i18n translations (.po files)
run: ./bin/i18n/build-translation-resources
snowplow:
needs: files-changed
if: needs.files-changed.outputs.snowplow == 'true'
runs-on: ${{ vars.DEFAULT_RUNNER_KEY }}
steps:
- name: Checkout
uses: actions/checkout@v6
- name: Set up Java
uses: actions/setup-java@v5.2.0
with:
distribution: temurin
java-version: 25
- name: Set up Babashka
uses: turtlequeue/setup-babashka@d78ec6570aea3b614bc695cafcceb82eb45c2af9 # v1.8.0
with:
babashka-version: 1.12.197
- name: Install igluctl
run: |
set -euo pipefail
curl -fsSL -o "$RUNNER_TEMP/igluctl.zip" https://github.com/snowplow/igluctl/releases/download/0.13.0/igluctl_0.13.0.zip
echo "0eac19121fca1431dec44738ada8eacc4242811a95d015ee71e00deec2b4d748 $RUNNER_TEMP/igluctl.zip" | sha256sum -c -
unzip -q "$RUNNER_TEMP/igluctl.zip" -d "$RUNNER_TEMP/igluctl"
chmod +x "$RUNNER_TEMP/igluctl/igluctl"
echo "$RUNNER_TEMP/igluctl" >> "$GITHUB_PATH"
- name: Lint (igluctl)
# igluctl recursively parses every file as JSON, so it chokes on colocated
# *.md docs and on two schemas whose internal `self.version` doesn't match
# the filename version (pending Snowcat audit). Stage a filtered copy and
# patch only the `self.version` mismatch so igluctl still lints the rest
# of those schemas.
run: |
set -euo pipefail
LINT_ROOT="$RUNNER_TEMP/iglu-lint"
rm -rf "$LINT_ROOT"
mkdir -p "$LINT_ROOT"
cp -r snowplow/iglu-client-embedded/schemas "$LINT_ROOT/"
find "$LINT_ROOT" -name '*.md' -delete
# TODO(BOT-1403): drop these patches once the Snowcat audit fixes the
# internal `self.version` mismatch on these two schemas. We rewrite
# only `self.version` in the staged copy so the file's path version
# matches what igluctl expects; every other lint rule still runs.
# Each `sed` invocation uses the schema's wrong/right pair, so it
# silently no-ops once the underlying schema is fixed.
sed -i 's/"version": "1-0-1"/"version": "1-0-2"/' \
"$LINT_ROOT/schemas/com.metabase/embed_share/jsonschema/1-0-2"
sed -i 's/"version": "1-0-0"/"version": "1-0-1"/' \
"$LINT_ROOT/schemas/com.metabase/serialization/jsonschema/1-0-1"
igluctl lint "$LINT_ROOT/schemas"
- name: Lint (Metabase schema rules)
run: ./bin/lint-snowplow-schemas.bb
- name: Test linter rules
run: ./bin/lint-snowplow-schemas-test.bb

View File

@@ -21,7 +21,7 @@ on:
jobs:
detect-changed-stories:
name: Detect changed stories
runs-on: ${{ vars.DEFAULT_RUNNER_KEY }}
runs-on: ${{ vars.SLIM_RUNNER_KEY }}
timeout-minutes: 5
outputs:
story_files: ${{ steps.detect.outputs.story_files }}
@@ -156,6 +156,9 @@ jobs:
- name: Stress-test Loki
run: |
# Loki's output is piped through tee below, and a pipeline normally reports the exit status of its last command.
# Without this a failed loki run would look like a pass, because tee itself succeeded.
set -o pipefail
BURN_IN=${{ inputs.burn_in || '10' }}
STORY_FILES='${{ needs.detect-changed-stories.outputs.story_files }}'
@@ -171,8 +174,11 @@ jobs:
if bun run test-visual:loki \
--reactUri file:./storybook-static \
--verboseRenderer; then
--verboseRenderer 2>&1 | tee .loki-run.log; then
echo "✅ Run $i passed"
elif grep -q 'No stories were found' .loki-run.log; then
echo "::notice::Every story in the changed files is loki-skipped, nothing to stress test"
exit 0
else
echo "❌ Run $i FAILED"
FAILED=$((FAILED + 1))

View File

@@ -2,12 +2,11 @@ name: Loki Visual Regression Testing
on:
pull_request:
merge_group:
jobs:
files-changed:
name: Check which files changed
runs-on: ${{ vars.DEFAULT_RUNNER_KEY }}
runs-on: ${{ vars.SLIM_RUNNER_KEY }}
timeout-minutes: 3
outputs:
frontend_ci: ${{ steps.changes.outputs.frontend_ci }}

View File

@@ -1,39 +0,0 @@
name: Slack Notification for New Feature Requests
on:
issues:
types:
- labeled
jobs:
on-issue-labeled:
runs-on: ${{ vars.DEFAULT_RUNNER_KEY }}
timeout-minutes: 5
if: |
contains(github.event.issue.labels.*.name, 'Type:New Feature')
steps:
- name: Setting title
uses: actions/github-script@v9
id: vars
env:
ISSUE_TITLE: ${{ github.event.issue.title }}
with:
script: |
core.setOutput('issue_title', process.env.ISSUE_TITLE.replaceAll(/"/g, '\\"'));
- name: Send the issue ${{ github.event.issue.number }} to Slack
id: slack
uses: slackapi/slack-github-action@45a88b9581bfab2566dc881e2cd66d334e621e2c # v3.0.3
with:
webhook: ${{ secrets.SLACK_FEATURE_REQUEST_TRIAGE_WEBHOOK_URL }}
webhook-type: incoming-webhook
payload: |
{
"blocks": [
{
"type": "section",
"text": {
"type": "mrkdwn",
"text": "New incoming feature request: ${{ steps.vars.outputs.issue_title }}\n${{ github.event.issue.html_url }}"
}
}
]
}

View File

@@ -11,7 +11,8 @@ on:
- src/metabase/utils/js.cljs
jobs:
mbql-library-changelog-check:
runs-on: ${{ vars.DEFAULT_RUNNER_KEY }}
runs-on: ${{ vars.SLIM_RUNNER_KEY }}
timeout-minutes: 5
if: ${{ !contains( github.event.pull_request.labels.*.name, '.metabase-lib/no-change' ) }}
steps:
# - uses: jesusvasquez333/verify-pr-label-action@v1.4.0

View File

@@ -1,254 +0,0 @@
name: Minimal Tests
on:
pull_request:
types: [opened, synchronize, reopened, labeled]
concurrency:
group: ${{ github.workflow }}-${{ github.head_ref && github.ref || github.run_id }}
cancel-in-progress: true
permissions:
id-token: write
contents: read
checks: write
actions: read
jobs:
# Only run if the PR has the minimal-tests label
check-label:
runs-on: ${{ vars.DEFAULT_RUNNER_KEY }}
timeout-minutes: 1
outputs:
should-run: ${{ steps.check.outputs.result }}
steps:
- name: Check for minimal-tests label
id: check
run: |
if [[ "${{ contains(github.event.pull_request.labels.*.name, 'minimal-tests') }}" == "true" ]]; then
echo "result=true" >> "$GITHUB_OUTPUT"
else
echo "result=false" >> "$GITHUB_OUTPUT"
fi
# Postgres app-db tests - single version, single job
postgres-app-db-test:
needs: check-label
if: needs.check-label.outputs.should-run == 'true'
runs-on: ${{ vars.DEFAULT_RUNNER_KEY }}
timeout-minutes: 40
env:
CI: "true"
DRIVERS: postgres
MB_DB_TYPE: postgres
MB_DB_PORT: 5432
MB_DB_HOST: localhost
MB_DB_DBNAME: mb_test
MB_DB_USER: mb_test
MB_POSTGRESQL_TEST_USER: mb_test
services:
postgres:
image: postgres:latest
ports:
- "5432:5432"
env:
POSTGRES_USER: mb_test
POSTGRES_DB: mb_test
POSTGRES_HOST_AUTH_METHOD: trust
name: Postgres Latest EE App DB Tests (Part 1)
steps:
- uses: actions/checkout@v6
- name: Test Postgres App DB
id: test-driver
uses: ./.github/actions/test-driver
with:
build-static-viz: true
junit-name: "minimal-be-tests-postgres-ee"
quarantine-engine: ${{ vars.QUARANTINE_ENGINE }}
test-args: >-
:only '["test" "enterprise/backend/test"]'
:exclude-tags '[:mb/driver-tests :mb/transforms-python-test]'
:partition/total 2
:partition/index 0
- name: Upload Test Results
uses: ./.github/actions/upload-test-results
if: always()
with:
input-path: ./target/junit/
output-name: ${{ github.job }}
bucket: ${{ vars.AWS_S3_TEST_RESULTS_BUCKET }}
role-to-assume: ${{ secrets.CI_TEST_RESULTS_IAM_ROLE }}
aws-region: ${{ vars.AWS_REGION }}
# Postgres driver tests
postgres-driver-test:
needs: check-label
if: needs.check-label.outputs.should-run == 'true'
runs-on: ${{ vars.DEFAULT_RUNNER_KEY }}
timeout-minutes: 40
env:
CI: "true"
DRIVERS: postgres
MB_DB_TYPE: postgres
MB_DB_PORT: 5432
MB_DB_HOST: localhost
MB_DB_DBNAME: mb_test
MB_DB_USER: mb_test
MB_POSTGRESQL_TEST_USER: mb_test
services:
postgres:
image: postgres:latest
ports:
- "5432:5432"
env:
POSTGRES_USER: mb_test
POSTGRES_DB: mb_test
POSTGRES_HOST_AUTH_METHOD: trust
name: Postgres Latest Driver Tests
steps:
- uses: actions/checkout@v6
- name: Test Postgres Driver
id: test-driver
uses: ./.github/actions/test-driver
with:
junit-name: "minimal-be-tests-postgres-driver-ee"
quarantine-engine: ${{ vars.QUARANTINE_ENGINE }}
test-args: >-
:only-tags [:mb/driver-tests]
:exclude-tags [:mb/transforms-python-test]
- name: Upload Test Results
uses: ./.github/actions/upload-test-results
if: always()
with:
input-path: ./target/junit/
output-name: ${{ github.job }}
bucket: ${{ vars.AWS_S3_TEST_RESULTS_BUCKET }}
role-to-assume: ${{ secrets.CI_TEST_RESULTS_IAM_ROLE }}
aws-region: ${{ vars.AWS_REGION }}
# Frontend unit tests
frontend-unit-tests:
needs: check-label
if: needs.check-label.outputs.should-run == 'true'
runs-on: ${{ vars.DEFAULT_RUNNER_KEY }}
timeout-minutes: 30
env:
JEST_JUNIT_OUTPUT_DIR: ./target/junit
JEST_JUNIT_OUTPUT_NAME: test-report-frontend-unit.xml
name: Frontend Unit Tests
steps:
- uses: actions/checkout@v6
- name: Prepare front-end environment
uses: ./.github/actions/prepare-frontend
- name: Prepare back-end environment
uses: ./.github/actions/prepare-backend
- name: Resolve current job id
# Resolve JOB_ID before the long test step so the ci-conductor reporter
# below can link failures to this exact job. Best-effort.
uses: ./.github/actions/resolve-job-id
- name: Run frontend unit tests
id: run-tests
run: bun run test-unit --silent
# Keep this step continue-on-error because the quarantine logic controls whether it ultimately fails.
continue-on-error: true
- name: Report failed tests to ci-conductor
if: ${{ !cancelled() && steps.run-tests.outcome == 'failure' }}
# This step should NEVER fail the job!
continue-on-error: true
uses: ./.github/actions/report-failures-to-ci-conductor
with:
test-suite: fe-tests-unit
adapter: frontend
- name: Quarantine gate
if: ${{ !cancelled() && steps.run-tests.outcome == 'failure' }}
continue-on-error: ${{ vars.QUARANTINE_ENGINE != 'ci-conductor' }}
timeout-minutes: 3
uses: ./.github/actions/quarantine-gate
with:
adapter: frontend
test-suite: fe-tests-unit
enforce-quarantine: ${{ vars.QUARANTINE_ENGINE == 'ci-conductor' }}
- name: Upload Test Results
uses: ./.github/actions/upload-test-results
if: always()
with:
input-path: ./target/junit
output-name: frontend-unit
bucket: ${{ vars.AWS_S3_TEST_RESULTS_BUCKET }}
role-to-assume: ${{ secrets.CI_TEST_RESULTS_IAM_ROLE }}
aws-region: ${{ vars.AWS_REGION }}
# Summary job
minimal-tests-result:
needs:
- postgres-app-db-test
- postgres-driver-test
- frontend-unit-tests
runs-on: ${{ vars.DEFAULT_RUNNER_KEY }}
timeout-minutes: 5
name: minimal-tests-result
if: always() && !cancelled()
steps:
- name: Check job status
uses: actions/github-script@v9
env:
needs: ${{ toJson(needs) }}
with:
script: | # js
const needs = JSON.parse(process.env.needs);
const jobs = Object.entries(needs).map(
([jobName, jobValues]) => ({
name: jobName,
result: jobValues.result
}));
// Check if all jobs either succeeded or were skipped
// Skipped jobs are acceptable (they were intentionally skipped when the label is absent)
const allPassedOrSkipped = jobs.every(job =>
job.result === 'success' || job.result === 'skipped'
);
// Check if any jobs actually failed (not skipped, not cancelled)
const anyFailed = jobs.some(job =>
job.result === 'failure'
);
if (allPassedOrSkipped && !anyFailed) {
console.log("");
console.log(" _------. ");
console.log(" / , \_ ");
console.log(" / / /{}\ |o\_ ");
console.log(" / \ `--' /-' \ ");
console.log(" | \ \ | ");
console.log(" | |`-, | ");
console.log(" / /__/)/ ");
console.log(" | | ");
console.log("");
console.log("All minimal tests have passed. Fast feedback achieved!");
process.exit(0);
}
// otherwise, something failed
console.log("");
console.log(" .::::::::::. .::::::::::. ");
console.log(" .::::''''''::::. .::::''''''::::. ");
console.log(" .:::' `::::.... ....::::' `:::. ");
console.log(" .::' `:::::::| |:::::::' `::. ");
console.log(" .::| |::::::|_ ___ __|::::::| |::. ");
console.log(" `--' |::::::|_()__()_|::::::| `--' ");
console.log(" ::: |::-o::| |::o-::| ::: ");
console.log(" `::. .|::::::| |::::::|. .::' ");
console.log(" `:::. .::\-----' `-----/::. .:::' ");
console.log(" `::::......::::' `::::......::::' ");
console.log(" `::::::::::' `::::::::::' ");
console.log("");
console.log("Minimal tests have failed.");
jobs.forEach((job) => {
if (job.result !== 'success' && job.result !== 'skipped') {
console.log(`${job.name} - ${job.result}`);
}
});
process.exit(1);

Some files were not shown because too many files have changed in this diff Show More