Compare commits

...

820 Commits

Author SHA1 Message Date
Ben Grabow
cb525c79a5 Fix jar splice 2026-06-26 17:00:57 -05:00
Ben Grabow
dc63a440f3 Splice H2 lib into metabase jar before running 2026-06-26 15:43:44 -05:00
Ben Grabow
01c9771b31 Allow hardcoded :h2 driver name refs in tests (at least for now) 2026-06-26 15:08:26 -05:00
Ben Grabow
05548699e4 Include H2 driver source on path for tests 2026-06-25 16:21:26 -05:00
Ben Grabow
14cf562376 Work around buggy linter result 2026-06-25 16:20:44 -05:00
Ben Grabow
2c30d66bf0 Move H2 driver out of core 2026-06-25 09:13:10 -05:00
Ben Grabow
c103dde120 Sort deps 2026-06-24 16:02:46 -05:00
dan sutton
8605b43a63 Make H2 an optional driver (OSS-bundled, EE-supplied)
H2 is no longer required on the classpath. The OSS uberjar bundles the H2
JDBC JAR; the EE uberjar does not. EE users who want H2 (quick trials, the
dump-to-h2/load-from-h2 tasks, or an H2 app DB) supply the JAR themselves via
the plugins dir or `-cp`. SQLite, not H2, is the sample DB, so H2 isn't needed
by default.

Mechanics, mirroring the oracle/vertica optional-driver pattern:

- deps.edn: H2 moved from a top-level dep to the `:oss` alias only (plus `:dev`
  for the REPL); the `:ee` basis no longer includes it.
- build/uberjar.clj: exclude `metabase.driver.h2` from AOT (its ns `:import`s
  org.h2.* classes that may be absent at EE build time) and inject it into the
  jar as source, compiled lazily at runtime. Driven by a dedicated
  `core-driver-namespaces-excluded-from-aot` set.
- config.core/h2-available?: runtime check for `org.h2.Driver` on the classpath
  (re-checked at boot, so a JAR added via -cp is seen).
- driver.init: load metabase.driver.h2 only `(when (config/h2-available?))`,
  via a runtime require, instead of an unconditional ns require.
- No core (AOT-compiled) namespace references org.h2.* at compile time anymore.
  The H2-class-typed registrations (IResultSetReadColumn for JdbcClob/JdbcBlob,
  cache results-as-bytes for JdbcBlob, query-canceled?* :h2) moved into
  metabase.driver.h2; the owning multimethods were made public and exposed in
  the app-db / query-processor module :api sets. search.appdb.index detects the
  H2 syntax-error exception by class-name string instead of importing it.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-23 16:03:10 -05:00
Ben Grabow
1d050aec9c Bgrabow/monitor memoization cache size (#75967)
* Monitor number of cache entries in unbounded memoization caches

* Refresh metrics every 10 minutes instead of 1 minute
2026-06-23 18:32:31 +00:00
Eric Normand
fdd669322d Log the reason for full remote-sync export/import fallback (GHY-3935) (#76348)
When export! or import! falls back from the incremental fast-path to a
full re-serialize/reload, log which guard forced it so operators can see
why a full sync ran:

- export: forced overwrite, or a pending change can't be applied incrementally
- import: forced, local changes pending, first import, or changes not
  incrementally loadable

Bind import dirty? as a delay so the fast-path check and the reason
derivation share one query without forcing it on the no-op/merge paths.
2026-06-23 18:13:31 +00:00
Andrei Chernykh
c96ad1f49a Fix @tenant.slug tooltip to read 'system defined' on the create tenant user from (#76321)
Fix @tenant.slug tooltip to read 'system defined' on the create tenant user form
2026-06-23 11:01:28 -07:00
appleby
00f5a7c258 Add agent-lib repair pass to fixup invalid lib/type for join clauses (#76309)
* Add metabase.agent-lib.representations.repair/rewrite-lib-type-aliases*

Rewrite the hallucinated lib/type mbql.join/join to the corrected mbql/join. The LLM was observed making this error in BOT-1657.

Similar to the existing rewrite passes:

- rewrite-operator-name-aliases*
- rewrite-temporal-bucket-aliases*
- rewrite-direction-aliases*

* Fill in a missing lib/type for join clauses

We already did this for stages and the top-level query.

* Validate lib/type for ::join clause in metabase.agent-lib.representations/validate-query

* Mark all deftests ^:parallel in metabase.agent-lib.representations-test
2026-06-23 16:03:14 +00:00
Denis Berezin
98e9faea89 UXW-3457 Fix select to overflow viz settings popover (#76297)
Co-authored-by: @syedfardeenjeelani
2026-06-23 18:04:11 +03:00
Kevin Kelleher
025cda0388 Fix non-unique key props in SectionLayoutPreview (#76250)
fix non-unique key props
2026-06-23 10:51:30 -04:00
Riley Thompson
74aa95064b Validate sqlserver additional options when connecting (#76293)
validate sqlserver optios when connecting
2026-06-23 13:48:41 +00:00
github-automation-metabase
daa7c3d33c Auto-update documentation for master (#76303)
Update generated docs (master)

Co-authored-by: github-actions <github-actions@github.com>
2026-06-23 09:01:23 -04:00
Kevin Kelleher
49c604047d Fix bars overlapping in time series with only two bars (#76246)
* account for extra padding in bar width

* add loki test

* Update Loki Snapshots

---------

Co-authored-by: Metabase Automation <github-automation@metabase.com>
2026-06-23 07:10:03 -04:00
Nicola Mometto
0a0589299c Restrict glossary permissions (#76275)
* restrict permissions for glossary

* disable on UI

* adjust FE code + removed hover effect if the row is readOnly

* rename FieldId to GlossaryField

* address comments

---------

Co-authored-by: Egor Grushin <egor@metabase.com>
2026-06-23 10:05:54 +00:00
YB
403107f13f [GDGT-376] [GDGT-2624] fix serdes import oom (#75786)
Serializing the **import** of a very large data model (~1M+ fields) can OOM. The entire retained-memory cost of an import is the **ingest index**, built up-front before any entity loads (a flat memory plateau through the whole load phase).

Fixes:
1. **Drop the dead structures.** The cache value becomes the `File` directly (`{stripped-path File}`); delete `:from-ids` and its `medley` require, removing the second `ingest-list` pass.

2. **Intern path segments while building the index.** Each `{:model … :id …}` segment and its strings are canonicalized through a shared interner so index keys share their `[db schema table]` prefix structure. Value equality is unchanged, so cache lookups and `:seen`-set comparisons behave identically. Interning goes through a new `metabase.util.memoize/fast-interner` - a thread-safe `ConcurrentHashMap`-backed interner that stores the *canonical* value as the key (so the retained object is the string-pooled copy, not the parsed input).
2026-06-23 16:57:30 +08:00
Nemanja Glumac
dc80345e66 Drop the fix-flaky-test command (#76299) 2026-06-23 09:54:20 +02:00
Uladzimir Havenchyk
6787014a55 chore: drop deprecated Button (#76296) 2026-06-23 07:46:09 +00:00
Romeo Van Snick
3d5661eb54 Lazy load echarts module (#75220)
* Lazy load Echarts renderer

* Load Echarts as the vizualisation loads

* Use correct fallback for echarts renderer

* Fix placeholders for some skeletons

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>

* Eagerly load echarts in sdk bundle

* Reenable brush after chart loads

---------

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
2026-06-23 16:24:56 +10:00
Alexander Polyankin
5ec0d46be4 Push sync limit to drivers (#76307)
* Push sync limit to drivers

* fix

* fix

* fix
2026-06-22 19:49:21 -04:00
Romeo Van Snick
e5b06223a1 Lazy load leaflet (#75222)
* Lazy load Leaflet

* Eagerly import map renderer in SDK bundle to fix lazy-loaded leaflet

The lazy `import()` of MapRenderer never resolves in the embedding SDK
bundle, which can't fetch on-demand chunks at runtime. Pull it into the
preloaded SDK bundle via sdk-specific-imports so the dynamic import
resolves from an already-loaded chunk, while the main app keeps its lazy
leaflet chunk.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-23 09:32:04 +10:00
Eric Normand
a60bd4f293 Surface essential-step sync failures instead of reporting initial sync complete (GHY-3943) (#76286)
* Surface essential-step sync failures instead of reporting initial sync complete (GHY-3943)

When an essential metadata sync step (sync-tables, sync-fields) failed
non-transiently mid-initial-sync, the error was logged and swallowed,
and the database's initial_sync_status was still set to "complete".
This left tables with zero fields while reporting success -- e.g. on
MariaDB <10.2, whose information_schema lacks generation_expression, so
describe-fields throws and no fields are synced, yet the query builder
reports 'Table has no Fields associated with it'.

Mark sync-tables and sync-fields as essential steps. Initial sync is now
reported as aborted (not complete) when an essential step fails for any
reason, in addition to the existing transient-error abort path. Non-
essential step failures (indexes, FKs) and per-table failures still
report complete, preserving graceful degradation. The status setters
no-op once a database is already complete, so re-syncs of healthy
databases are unaffected.

* Better factoring

* Remove unused metabase.util require in sync-metadata

The cond-based sync-db-metadata!* no longer uses u/prog1, so the alias
is unused; drop it to satisfy clj-kondo.
2026-06-22 17:04:54 -05:00
Ben Grabow
ebb8e4838b Replace H2 with SQLite for sample data 2026-06-22 21:50:25 +00:00
Braden Shepherdson
57991c5b67 [subs] Generate PDFs for dashboard subscriptions on the BE (#75205)
The PDFs are laid out on either A4 or US Letter paper, portrait orientation, following the size and positioning of cards on the dashboard itself.

This is a nearly complete rendering of the dashboard, akin to what you see in the brower:
- Directly building a PDF in memory with Apache PDFBox, a low-level Java library.
- All chart types supported by static viz
    - Recently expanded to include maps and pivot tables!
- Markdown rendering, including code blocks, clickable links etc.
- Unicode support (Europe, Cyrillic, Greek, Hebrew, Arabic, Hindi, CJK and others) including RTL
- iframe cards have their target (e.g. a YouTube video) rendered as a link
- Dashboard is run as the requesting user, and can have parameter values supplied
- Parameters on dashboard and individual cards are shown with their values.
- Solid API endpoint that allows for parameter values and directly responds with the PDF.
- Reusing query results when sending charts via email or Slack along with a PDF.

---------

Co-authored-by: Nick Fitzpatrick <nick@metabase.com>
2026-06-22 16:40:28 -04:00
Uladzimir Havenchyk
09726c60c7 chore: migrate to FormSubmitButton from metabase/forms (#75709)
* chore: migrate to FormSubmitButton from metabase/forms

* address code review
2026-06-22 23:28:58 +03:00
bryan
fdf5e72f54 dev.module-viz: interactive modules graph visualizer (#75256) 2026-06-22 20:21:49 +00:00
Andrei Chernykh
0c9ca2b55d Fix middle-click on search results (#76123)
* Make the whole search result row a real link so middle/ctrl-click opens a new tab, and migrate its styles to Mantine

* Block navigation on inactive results, fix click-tracking attribution, and harden the title color override
2026-06-22 11:55:42 -07:00
Alexander Polyankin
ea63bc4889 Fix OOM with BigQuery and Redshift sync (#76276) 2026-06-22 14:23:48 -04:00
Egor Grushin
ab33899be5 GIT-5689 get rid of tippy.js (#76247)
* GIT-5689 get rid of tippy.js

* GIT-5689 additional clean ups
2026-06-22 19:19:43 +01:00
Uladzimir Havenchyk
657d0ec515 fix: handle click on cancel button and make title bigger (#76148) 2026-06-22 21:15:17 +03:00
Denis Berezin
1b5aefd862 UXW-4466 fix Y-axis auto position (#76183)
* UXW-4466 fix Y-axis auto position

* Fix failing e2e

* Fix failing e2e
2026-06-22 21:05:34 +03:00
Nick Fitzpatrick
e809dd2715 Fix library tree auto-expand race leaving Data/Metrics collapsed (#76167) 2026-06-22 23:33:07 +05:30
Nick Fitzpatrick
4318830e5e Stabilize flaky datetime cell-edit table-editing e2e test (#76158)
remove randomness from table edit date test
2026-06-22 23:32:19 +05:30
Eric Normand
6bfb3cc0ae Enforce remote-sync dependency guard on card bulk-move (GHY-3791) (#76278)
* Enforce remote-sync dependency guard on card bulk-move (GHY-3791)

The bulk-move endpoint (POST /api/card/collections) was the only mutation
path that hand-rolled the remote-sync dependency guard instead of using
check-for-remote-sync-update. It checked the stale pre-update card objects
(so moving a root-level card into a synced collection slipped through) and
never checked dependents (so bulk-moving a depended-on synced card out of
the sync set was allowed, unlike the single-card PUT path).

Use check-for-remote-sync-update so the post-update state is re-selected
and both directions are checked inside the existing transaction.

Also add an edit-time test for adding a non-synced dashcard via dashboard
PUT, closing a pre-existing coverage gap.

* Unnecessary comment
2026-06-22 17:00:08 +00:00
Ben Grabow
63a2dbfdf4 Empty nodes yields empty traversal instead of error (#76262) 2026-06-22 16:50:11 +00:00
Riley Thompson
52833fb7bb Use table of LHS join column in query builder (#76038)
* use table of the lhs column

* apply suggestion
2026-06-22 16:41:04 +00:00
metamben
630fc8df93 Preserve nulls in day-of-week calculation (#76180)
Closes QUE2-687
2026-06-22 16:21:01 +00:00
Alexander Polyankin
59637becb1 Fix postgres test (#76277) 2026-06-22 11:58:18 -04:00
Riley Thompson
db9803f9b8 Remove mbql5 experiments (#75127)
* remove postgres-mbql5

* remove mbql5 experiment

* fix postgres, add testing code

* fix and clean up

* include h2 test

* ignore native queries

* add function to compare compilation results

* fix tests

* remove old mbql5 references

* move redshift to mbql5

* redshift compilation test

* examples

* save changes

* fix bad merge
2026-06-22 11:21:28 -04:00
Riley Thompson
cea403df68 Dont override lib temporal types if driver type is text (#75663)
* dont override lib temporal types if driver type is text

* make test driver agnostic

* fix tests

* add sqlite test

* rename unreliable-driver-type? to prefer-lib-type?

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-22 11:03:17 -04:00
Nathan Voxland
16080fabce Fix appdb reindex misdirecting writes to the live index (#76199) 2026-06-22 09:56:06 -05:00
Alexander Polyankin
b342ae4aa7 Fix OOM with json/decode (#76221) 2026-06-22 10:41:55 -04:00
Alexander Polyankin
4392d6aff4 Fix OOM with sync metadata for postgres/redshift (#76218) 2026-06-22 10:41:42 -04:00
metamben
67f5557d3b Skip sandboxing during metric expansion (#76125)
Closes QUE2-681
2026-06-22 17:20:41 +03:00
Anderson
e1e1fef04b Fix conditional formatting range matching (#76137) 2026-06-22 10:53:42 -03:00
Anderson
36329b87e1 Fix download permission warning for blocked groups (#76075) 2026-06-22 10:53:31 -03:00
Ngoc Khuat
a65b91edee GDGT-2332: Retry cluster-lock writes on transient deadlocks (#75172) 2026-06-22 13:46:18 +00:00
Egor Grushin
e3ce080ee8 Gdgt 2590 replace deprecated tooltip component (#76117)
* GDGT-2590 remove tippy completely

* GDGT-2590 replace tippy Tooltip with Tooltip from metabase/ui.

* GDGT-2590 fix tests

* GDGT-2590 fix js test

* GDGT-2590 fix fade animation

* GDGT-2590 fix the gap in stacked tooltip

* GDGT-2590 remove the useState+useEffect hack

* GDGT-2590 remove redundant comment
2026-06-22 14:11:55 +01:00
Romeo Van Snick
4b32e29e29 Move collections frontend module to the feature tier (#75986)
* Move shared collection utilities to metabase/common/collections

Relocate the tier-agnostic collection helpers (utils, constants, types,
hooks, schemas, getExpandedCollectionsById, analytics) from
metabase/collections into metabase/common/collections so they remain in
the shared tier once metabase/collections becomes a feature module.
Update all importers to the new path.

* Move shared collection components to metabase/common/collections

Relocate the collection components and containers that are consumed by
other feature and shared modules (FormCollectionPicker,
FormCollectionAndDashboardPicker, CreateCollectionModal,
FormAuthorityLevelFieldContainer, CreateCollectionForm, PinnedItemCard,
ActionMenu, PinDropZone, CollectionRowMenu, QuestionMoveConfirmModal)
into metabase/common/collections. Extract the shared collection-content
column types into metabase/common/collections/columns. Update importers.

* Move metabase/collections to the feature tier

With all cross-module collection utilities and shared components now
living in metabase/common/collections, metabase/collections only holds
the collection browsing/landing experience. Promote it from the shared
tier to the feature tier so it can no longer be imported by shared or
other feature modules.

* Import SortableColumnHeader directly to avoid barrel cycle resolving undefined
2026-06-22 22:48:40 +10:00
Chris Truter
813d41e755 Respect explicit null collection_id in MCP creation (#75727) 2026-06-22 12:02:40 +00:00
Romeo Van Snick
1b0e9e91d4 Fix silent data corruption when a query errors mid-stream (#75586)
* Abort instead of sending json blob to client in streaming responses

* Stop trying to parse 202 as JSON

* Handle error in response.blob

* Stream an error event when metabot fails mid-stream in text/event-stream

* Clean up handleResponse

* Add e2e test for failing download

* Fix typo in handleResponse comment

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* Revert completion flag when aborting the connection fails

If the Jetty .abort throws, the optimistically-set *completed?* flag would
otherwise stay true, causing do-f-async's finally to skip .complete and leave
the async request hanging until Jetty's timeout fires.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* Add frontend tests for mid-stream download error handling

Covers readDownloadBlob mapping a blob() rejection to the localized
download-interrupted error, and handleResponse rejecting (rather than silently
resolving) when a streamed success body fails to read mid-stream — the behavior
that replaced the old 202 error-blob sniffing.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* Fix flaky metabot disconnect test under forced chunked encoding

closing-connection-native-agent-test closed the response body stream to
simulate a client disconnect. Now that streaming responses force
Transfer-Encoding: chunked (for mid-stream abort detection on downloads),
closing the body makes clj-http drain the chunked response to completion —
which the server sees as a normal finish, not a disconnect, so the turn
finalized as :finished? true.

Close the underlying http-client instead, which aborts the connection and is
detected by the server's cancel loop — matching the existing pattern in
streaming_response_test. Adds user-real-request-full-response to expose the
:http-client from a real streaming request.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* Add special error case for interrupted responses

* Render the new stream error differently than just a network error

* Remove stale _status in BE response

* Remove all references to _status

* Be more explicit about HTTP/2 in comment

Co-authored-by: Chris Truter <crisptrutski@users.noreply.github.com>

* Disambiguate singular "Year" granularity label to fix i18n template build

---------

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-authored-by: Chris Truter <crisptrutski@users.noreply.github.com>
2026-06-22 20:52:07 +10:00
Romeo Van Snick
26dc779def Disambiguate singular "Year" granularity label to fix i18n template build (#76231) 2026-06-22 10:51:07 +00:00
Chris Truter
632949fe3d Add search debug API to explain why an entity is missing from results (#75132)
## Summary

Adds a superuser-only **`GET /api/search/debug`** that answers "I searched for X but my card/dashboard/table didn't show up — why?". It takes a normal `/api/search` request plus `expected_result_type` and `expected_result_id`, walks the search pipeline against whatever engine the request resolves to, and returns `{type, details}` for the **first** stage that drops the entity.

| `type` | meaning |
|---|---|
| `not-searchable` | the spec says don't index it (no spec, or its `:where` excludes it) |
| `missing-from-index` | the spec would index it, but it's absent from the active index (stale/failed ingestion) |
| `filtered` | in the index but cut by a structural filter or permissions — `details.excluded-by` says which |
| `not-matching` | passes filters but doesn't match the fulltext/semantic query (semantic includes the 0.7 cosine cutoff) |
| `matched` / `ranked-out` | passes every stage; returned, or ranked below the limit |

## How it works

A new `search.engine/diagnose` multimethod, dispatched per engine — appdb impl in OSS, semantic impl in EE. Shared orchestration in `debug.clj` handles the engine-independent stages (`not-searchable`, terminal `matched`/`ranked-out`); each engine impl handles `missing-from-index` / `filtered` / `not-matching`.

The appdb impl reuses the normal `results` filter chain but **drops the fulltext predicate** — the blank-search-term seam — to separate "filtered" from "not-matching", and probes filters cumulatively to attribute which one excluded the row.

## Notes

- `indexed-entity` is rejected (400) — compound id doesn't fit `PositiveInt`.
- `not-searchable` reflects current spec/DB truth, so it wins over a stale lingering index row.

## Testing

`metabase.search.debug-test` (appdb) and `metabase-enterprise.semantic-search.debug-test` (pgvector, auto-skips without `MB_PGVECTOR_DB_URL`) — one case per result type. Green in a clean JVM.
2026-06-22 11:56:27 +02:00
Ngoc Khuat
0869369208 Admin notifications group by notification id not card id (#75928) 2026-06-22 09:23:04 +00:00
Oleh K
18eeb58dc2 refactor: Sweep border-* tokens to border-neutral-* (#75611)
* Sweep `border-*` tokens to `border-neutral-*` (GDGT-2531)

Renames all internal consumers of the legacy border semantic tokens to
their `border-neutral-*` counterparts (added by the foundation PR):

- `--mb-color-border`           → `--mb-color-border-neutral`
- `--mb-color-border-strong`    → `--mb-color-border-neutral-strong`
- `--mb-color-border-stronger`  → `--mb-color-border-neutral-strongest`
- `--mb-color-border-subtle`    → `--mb-color-border-neutral-subtle`

Sweep covers `var(--mb-color-…)` references in CSS modules and
styled-components, `color()`/`alpha()`/`darken()`/`getColor()`/
`renderingContext.getColor()`/`theme.fn.themeColor()` helper calls,
`theme.colors.border`/`theme.colors.border[0]` accesses, Mantine
`c`/`bg`/`color` props, and a couple of single-site palette lookups.
All four are pure renames — `border-neutral*` resolve to the same
primitives as the legacy tokens in both light and dark modes.

`border` is an SDK v1 theme input key (`MetabaseColors.border`), so
`SDK_TO_MAIN_APP_COLORS_MAPPING.border` now maps to both `["border",
"border-neutral"]`. Without it, an SDK consumer setting
`theme.colors.border = "#hex"` would update only the legacy
`--mb-color-border`; the new `--mb-color-border-neutral` that the swept
code reads would stay at its default. The dual-map covers both SDK v1
paths — the global `colors` object via `getEmbeddingColorPalette` and the
Mantine override via `getEmbeddingThemeOverride`. Unlike
`brand`/`filter`/`summarize`, `border` isn't a whitelabel
(`application-colors`) key, so no whitelabel-passthrough mirror is needed.

Also flips the four `border` references in the emotion-migrate skill to
their `border-neutral` equivalents, matching the per-sweep convention.

The SDK contract surfaces (`embedding-color-palette.ts` left-side input
key, `dynamic-css-vars-config.ts`, `dynamic-sdk-color-defaults.ts`,
`MetabaseTheme.ts`, `apply-theme-preset.ts`) stay on the legacy `border`
name.

* Realign table in SKILL.md
2026-06-22 11:14:18 +02:00
Oleh K
e78aaa2c44 Fix run query button icon being too small on large screens (#76223)
Remove the large-breakpoint padding override on the run button so the icon keeps its intended size across viewport widths.
2026-06-22 11:06:00 +02:00
Oleh K
dc878ea1e5 Add transform job run history page (#76017)
* GDGT-2632 Build frontend for job run history page

Add a "Run history" tab to the individual transform job page
(/data-studio/transforms/jobs/:jobId/runs) showing a paginated,
sortable, status-filterable list of past job runs, with a resizable
right sidebar to drill into the transform runs that made up a run.

- Types/API: TransformJobRun + TransformRunForJobRun, listTransformJobRuns
  and listJobRunTransformRuns endpoints, transformJobRuns URL helper
- Page: JobRunListPage (table, status filter, pagination, polling) and
  JobRunSidebar (job-run info + transform-run items with View transform)
- Tabs: JobTabs (Overview / Run history) wired into JobEditor + routes
- Promote shared sidebar pieces to metabase/transforms/components
  (SidebarResizableBox, SidebarInfoRow, ErrorSection, TimezoneIndicator)
  so both the runs page and job-run page use them instead of
  cross-importing between pages
- ErrorSection gains showTitle option; colors use the new token names

* GDGT-2632 Add run-method and start-time filters to job run history

The base endpoint now supports filtering job runs by run-method and
start-time, so wire them through and surface them in the UI.

- RTKQ/URL: add run-method + start-time to ListTransformJobRunsRequest,
  the transformJobRuns URL helper, and the page param parser
- UI: Trigger Select (manual/cron) and Started Select with relative
  presets reusing TaskRunDatePicker's labels/values
- Tests: cover each filter interaction and URL-driven params; fix an
  async sidebar assertion

* GDGT-2632 Extract TransformRunList to avoid nested ternary

Address PR review: replace the loading/empty/list nested ternary in the
job run sidebar with a small TransformRunList component using early
returns.

* GDGT-2632 Poll sidebar transform runs while a run is in progress

Address PR review: the job run list polls and updates a run's status, but
the sidebar's transform runs are a separate cache entry that never
refetched, so they went stale after the run finished. Poll the sidebar's
transform runs while the run or any of its transforms is still in
progress.

Lets the shared setup*Endpoint helpers accept a response getter so the
test can drive a run from in-progress to finished while the sidebar
stays open.
2026-06-22 09:11:34 +02:00
Nick Fitzpatrick
abeb35e2e6 Fix flaky impersonation caching e2e test (#76162)
After switching to the impersonated user and reloading, the test asserted
query-builder-main before the reloaded query returned; under load the page
outlasted the default 4s timeout (seen in CI at impersonated.cy.spec.js:143).
Wait on the @query response before asserting the error UI.
2026-06-22 07:40:12 +05:30
Nick Fitzpatrick
9eb4bd1407 Fix flake in should preserve correct order of columns after column removal via sidebar (metabase#13455) (#76083) 2026-06-22 07:39:51 +05:30
Nick Fitzpatrick
a56c304e9e Fix flaky document revision-history toast assertion (#76164)
The revert-error check used a singular toast-undo lookup, but the two earlier
"Document saved" undo toasts could still be visible, so it matched multiple
elements and timed out with an ambiguity error. Dismiss each save toast right
after asserting it so they don't stack into the later lookup, keeping the
simple singular assertions.
2026-06-22 07:35:59 +05:30
Nick Fitzpatrick
7c823b037d Fix flaky remote-sync read-only tenant collection test (#76161)
The test created a shared tenant collection right after enabling read-only
sync, racing the auto-triggered import. Both creations bump the implicit
collection-permission revision via a non-atomic max(id)+1 insert, so they
collided on the primary key and returned a 500. Now await the import first.
2026-06-22 07:35:45 +05:30
Nick Fitzpatrick
e2ad12e6bd Fix flaky maps zoom/pan-after-resize e2e test (#76159)
The test compared a pin marker's position before and after a 1px viewport
resize, but read both positions on fixed cy.wait() timers that race leaflet's
~250ms zoom/resize animation. On a slow machine the pre-resize read landed
mid-animation, drifting past the 5px tolerance. Read each position only once it
settles (poll until two consecutive reads agree) instead of waiting a fixed
time; the closeTo tolerance assertions are unchanged.
2026-06-22 07:35:25 +05:30
Nick Fitzpatrick
42446b8240 Stabilize data-studio bulk "edit attributes for db" e2e test (#76085)
The test selected the database, published it, then clicked the database
row to expand it. While a node is selected, the row click only toggles
expansion, which races the post-publish re-render and could collapse the
database so its tables never rendered. Deselect before expanding, then
wait on the schema request before asserting on the table rows.
2026-06-22 07:27:54 +05:30
Fraser
fd771cbb26 Move home to feature tier (#75213) 2026-06-22 13:34:45 +12:00
Fraser
0ef9a1f2a3 Move account to feature tier (#75211) 2026-06-22 13:29:21 +12:00
Romeo Van Snick
45d21618a2 Lazy load react-virtualized (#75680)
* Lazy load react-virtualized

* Force pivot table eager in SDK bundle to fix on-demand chunk failure

The SDK bundle can't fetch on-demand webpack chunks at runtime, so the
lazily-imported react-virtualized-backed pivot table never rendered,
failing the SDK component tests. Importing PivotTableInner from
sdk-specific-imports.ts inlines it eagerly only in the SDK bundle while
keeping the main app lazy.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-22 11:27:18 +10:00
bryan
7766803a0c Bump bun to 1.3.14 (#76037)
* Bump bun to 1.3.14

* Update bun version in actions

---------

Co-authored-by: Fraser <fraser@metabase.com>
2026-06-22 12:01:29 +12:00
Romeo Van Snick
4bb017ab46 Bundle sizes stats fix (#76220)
* Show error when response fails

* Upload bundle sizes to the correct stats table

* Leave Version empty for unknown build tags
2026-06-22 09:38:38 +10:00
Alexander Polyankin
4df92d8239 Do not fingerprint MongoDB binary blobs (#76214) 2026-06-21 11:17:41 -04:00
Romeo Van Snick
dfecbd6799 Convert metabase/redux/metadata.js to TypeScript and remove some deprecated handlers (#76048)
Type redux/metadata, merge metadata-typed, and split out remapping/revision thunks
2026-06-21 15:09:37 +10:00
Romeo Van Snick
9fd9561b4c Bundle size stats (#76043)
* Emit frontend bundle stats from production build

* Track bundle sizes over time in stats.metabase.com

* Fix formatting

* Remove unused lint rule

* Reuse measure-bundle-sizes.js in the SDK bundle size check
2026-06-21 15:08:36 +10:00
Romeo Van Snick
2c8318d72a Remove enterprise legacy services (#76132)
* Convert enterprise services.ts to RTK query hooks

* Convert application permissions API to RTK query hooks
2026-06-21 15:08:07 +10:00
Egor Grushin
3153af0735 Gdgt 2575 replace deprecated modal component (#75563)
* GDGT-2615 fix PopoverWithRef

* GDGT-2615 migration step 2. (checks 1–6 are passed)

* GDGT-2615 migration step 3 put comments

* GDGT-2615 fix comments

* GDGT-2615 fix test

* GDGT-2615 move tippy-leftovers to tooltip folder

* GDGT-2575 hook mantine's modal to RENDERED_POPOVERS and get rid of deprecated Modal

* GDGT-2615 fix import import "metabase/common/components/Tooltip/tippy.module.css";

* GDGT-2575 extra comment

* GDGT-2575 restore pie_chart.cy.spec.js to master

* GDGT-2575 remove RENDERED_POPOVERS stack

* GDGT-2575 fix closing order

* GDGT-2575 simplify a code

* GDGT-2575 restore working state + added comments + keep code simplification

* GDGT-2575 fix animations

* GDGT-2575 change test description

* GDGT-2575 remove legacy modal selector from H.modal()

* GDGT-2575 fix autofocus in CreateActionForm

* GDGT-2575 remove legacy test

* GDGT-2575 fix Cancel button in NewEventModal

* GDGT-2575 fix autofocus

* GDGT-2575 fix cancel buttons

* GDGT-2575 make wide to be 95% instead of 85%

* GDGT-2575 force rerender to be sync

* GDGT-2575 fix SDK test

* GDGT-2575 addressed #75637 PR's comment leftovers

* GDGT-2575 removed missed unused import

* GDGT-2575 rename store to context

* GDGT-2575 move OverlayStackProvider to the app and sdk roots

* GDGT-2575 fix codemirror editor tooltip inside transformed parent

* GDGT-2590 fix unit tests

* GDGT-2590 fix loki tests
2026-06-20 16:15:11 +01:00
Alexander Polyankin
6992729d5e Limit max sync / scan / fingerprint fields (#76189) 2026-06-19 20:05:50 -04:00
Chris Truter
0bc758804b Add search_curated tool for NLQ (#75237)
* Add semantic_layer_index table and the semantic-layer-search OSS module

The semantic layer is a hand-curated library of saved search prompts, each
mapped to the entities that answer it (canonical = one directly-answering
entity, sources = a set to combine), plus agent-facing usage_instructions.

- migration: single createTable for semantic_layer_index (search_prompt,
  usage_instructions, entities JSON, type, verified, entity_id, timestamps)
- :model/SemanticLayerIndex with validation (non-empty entities; canonical =>
  exactly one) and serdes spec (polymorphic entity refs export as portable
  Card entity_ids / Table paths, mirroring export-viz-link-card)
- write hooks only nudge the enterprise background sync (request-sync!
  defenterprise shim, no-op in OSS) - no embedding or pgvector work on the
  write path
- superuser CRUD API at /api/semantic-layer-search
- module config: new semantic-layer-search module entry

* Add the semantic_layer_search Metabot tool and wire it into the nlq profile

The tool matches the user's request by vector similarity against the curated
semantic layer instead of ranking the whole instance:

- over-fetches raw per-prompt hits, dedupes to distinct entity sets (several
  prompts map to one entity), and returns a small best-first list
- hydrates matched entity refs into the enriched search-result shape via a new
  public tools.search/entity-refs->search-results (portable_entity_id,
  database names, fully-qualified names, metric base tables), so the agent can
  build a query without an extra read_resource round-trip
- surfaces raw cosine similarity and flags low-confidence matches (< 0.45)
  with a leading <note> + confidence="weak", so a miss reads as a miss
- the nlq profile uses it in place of nlq-search-tool

* Add the enterprise pgvector mirror with asynchronous reconciliation

Replaces the hackathon design (embedding-service + pgvector writes inside the
appdb write path via Toucan hooks, best-effort and silently drifting) with a
self-healing background sync:

- reconcile! diffs the whole authoritative appdb table against the mirror by
  content hash every run: stale rows are re-embedded (batched), orphans
  deleted, unchanged rows cost nothing. Any missed write - pgvector downtime,
  a crashed node, an import that bypassed the hooks - is repaired on the next
  run with no operator action.
- index-table/ensure-tables! owns the pgvector DDL: vectors table + HNSW
  index + a single-row meta table recording the embedding model identity and
  schema version. A model/dims change (or version bump) drops and recreates
  the vectors table; the next reconcile re-embeds everything. That
  drop-and-rebuild is the whole model-change story - the source table is
  small and always available.
- task.sync runs reconcile! on a 30s Quartz schedule (cluster-singleton,
  DisallowConcurrentExecution); appdb write hooks trigger-now! the job so
  curator edits are searchable within seconds, with trigger coalescing as
  free debounce.
- core/search serves the tool's similarity query (cosine distance + flat
  canonical/verified boosts, regular-search-shaped score breakdown); [] when
  pgvector is unconfigured or the mirror table does not exist yet.

End-to-end test drives CRUD API -> reconcile -> pgvector -> tool against an
isolated mirror with the 4-dim mock embedding model.

* Wire SemanticLayerIndex into serialization export/import

- exported-models + extract/model-set (with a :no-semantic-layer-index
  opt-out, like sibling models) + ingest legal-top-level-paths
  (semantic_layer_index/, matching the model's storage-path label)
- round-trip coverage: in-memory ref transformation and a real on-disk
  export/import (the two layers that have silently dropped models before),
  excused from the shared baseline fixture it covers itself

* Simplify the semantic layer to a single entity per entry

Drop the sources/canonical distinction: every entry now maps one saved prompt
to exactly ONE entity. Simplifications that follow:

- appdb table: `entities` JSON array + `type` column become a single
  `entity` JSON object; the canonical-arity validation hooks disappear
- pgvector mirror: `canonical` column gone; scoring is similarity (1.0) +
  verified boost (0.1) only
- tool: dedupe key is the single entity, matches lose the kind attribute,
  flatten-data loses match_group, and total_count now trivially equals the
  per-entity row count
- API: no type field/filter

Also folds in the roborev branch-review findings (job 1915):

- serdes entity-ref vocabulary now covers ALL the entity models the tool
  hydrates (question was missing -> export threw; card was hydrated nowhere),
  with an export-viz-link-card-style fallback for unmapped model strings
  instead of aborting the export, and round-trip coverage for every card
  flavor (card/model/metric/question) plus table
- the sync job uses task/defjob (log context + tracing span) instead of a
  raw deftype re-implementing half of it

The remaining two findings (PUT canonical revalidation, total_count vs
flattened rows) are moot now that canonical/sources is gone.

* Address Copilot PR feedback: card->question normalization, XML escaping, finite embeddings

- entity refs with :model "card" now hydrate (normalized to the agent-facing
  "question" type via a new public tools.search/ref-model->entity-type, used
  for both hydration grouping and the tool's lookup key)
- curator-entered saved_search_prompt / usage_instructions are XML-escaped in
  the tool output
- format-embedding rejects NaN/Infinity (they satisfy number? but produce SQL
  literals pgvector chokes on)

* Fix nlq-profile tests for the renamed tool + roborev follow-up nits

CI: the pre-existing profile/tool tests asserted the nlq profile exposes a
tool named "search"; it is now "semantic_layer_search". embedding_next
deliberately keeps the general nlq-search tool (switching embedding customers
to a curated-only layer that starts empty was not in scope), so the
matches-nlq invariant test now asserts the tool sets are identical modulo
that one substitution.

roborev (jobs 1916/1917):
- reuse llm-shape/escape-xml (now public) instead of a second, narrower copy
- format-embedding error message covers non-numbers too ("invalid value")
- API EntityRef :model is now an enum of the known entity types, so garbage
  refs are rejected at the boundary; the serdes :else passthrough (for rows
  predating a model string's removal) gets a test
- migration header notes the in-place changeset edit for anyone who ran the
  earlier branch revision

* Relax the entity-ref response schema to tolerate retired model strings

The write-side enum stays (garbage refs still 400 on POST/PUT), but GET
responses accept any :model string: rows can predate a model string's
retirement — serdes already tolerates those — and under dev/test response
validation a single legacy row would otherwise 400 the whole list endpoint.
(roborev job 1918)

* Drop the in-place-edit note from the migration header

The earlier revision never ran anywhere that survives (and the table name is
new on this branch anyway), so there is no checksum-mismatch audience to warn.

* Fix CI: migration 062->063, ASCII minus, mage driver-test override

* Lint with-isolated-mirror as fn, not let (single-symbol binding)

* Address Copilot round 2: rank by blended score in SQL, PUT patch semantics, smaller nits

- search: the SQL LIMIT now orders by distance minus the verified boost
  (equivalent to the blended score), so a verified row whose boost lifts it
  into the top N is no longer cut by the raw-distance limit; covered by a
  ranking test that fails against the old ordering
- PUT: presence of a key (contains?), not nil-ness, decides what changes, so
  an explicit null clears usage_instructions; entity/verified reject null in
  the schema since the columns are non-nullable
- EntityRef write schema: id must be a positive int
- entity-refs->search-results: drop the per-card (set verified) rebuild;
  select-fn-set already returns a set
- tool instructions prose names the match/note elements without literal
  angle-bracket tags, keeping the instructions element well-formed

* Drop the unused HNSW index from the mirror table

The blended ORDER BY (distance minus the verified boost) can't be served by
an HNSW index, so since the ranking fix the index was built and maintained
but never used on the read path. An exact scan is intentional and
microseconds at curated-table scale; documented at the ranking expression
and in the index-table namespace docstring. (roborev job 1923)

* Tighten docstrings: contracts first, one sentence per line

- mirror/search shim now documents its return shape (the caller contract the
  metabot tool builds on)
- upsert-batch! claimed to return the rows; it returns the count
- format-embedding states the contract (throws on non-finite elements); the
  pgvector-chokes rationale moves to an inline comment at the check
- stale plural in the module docstring (entries map to ONE entity now)
- multi-sentence paragraphs reflowed to one sentence per line

* Bump mirror schema-version so existing tables shed the orphaned HNSW index

Dropping the index creation left tables built under version 1 carrying a
maintained-but-unused index until some other rebuild trigger. The version
bump forces the one-time drop-and-rebuild on first sync. (roborev job 1924)

* Test that a schema-version mismatch alone triggers the mirror rebuild

The version bump is the rebuild trigger the previous commit leans on; cover
it directly (stale-write the meta row, assert :rebuilt + emptied vectors
table) so dropping schema_version from the meta comparison can't regress
silently. (roborev job 1926)

* Assert the mirror rebuild self-terminates

After the rebuild cases, a follow-up ensure-tables! must return :ok — i.e.
write-meta! healed the meta row. Without this, breaking schema_version
handling in write-meta! would silently rebuild on every sync and the tests
would still pass. (roborev job 1927)

* Rename semantic-layer-search -> curated-search; tool 'search_curated', augment not replace

Drops the overloaded 'semantic layer' term (it collides with the BI metrics
layer and with 'semantic search' = embedding IR). New vocabulary throughout:

- appdb table semantic_layer_index -> curated_search_entries
  (model :model/CuratedSearchEntry); pgvector mirror -> curated_search_index
  (now the only thing called an 'index')
- modules curated-search / enterprise/curated-search; API /api/curated-search;
  serdes model CuratedSearchEntry, dir curated_search_entries
- tool var curated-search-tool, :tool-name "search_curated" (verb-first like
  the other tools; distinct from the general 'search' so the two can coexist)

Behavior: the nlq profile now offers BOTH the general search and the curated
search tool, rather than replacing one with the other. The curated tool is
gated on the :semantic-search feature (via a :feature-semantic-search
capability derived in profiles/feature-capabilities) and its description tells
the agent to prefer it, using general search as fallback/complement.

Also tightened two over-long inline comment blocks in EE core.

* Fix alphabetical ordering after the curated-search rename

The in-place sed rename left three sorted lists with curated-search/
CuratedSearchEntry in the old 's' slot:
- api_routes route-map (fails keep-routes-sorted-test) + its keep-me
- models/resolution model->namespace map
Move them to their 'c' slots. (roborev job 2041)

* Filter curated-search hydration by current-user read permissions

entity-refs->search-results fetched Tables/Cards via direct t2/select with no
permission check, so an admin-curated entry pointing at a restricted entity
could leak its name/description/db/portable_entity_id to a user who can't
access it. Filter both fetches with mi/can-read? (mirrors the transform
filter already in this pipeline). (Copilot PR review)

* Address claude-code branch review: permission-filter before take-n, locale-safe XML

- build-matches: hydrate (and thus permission-filter) the whole deduped
  candidate set, then take n — taking n first let unreadable top hits shrink
  the result below n while readable matches sat just past the cut. The
  preceding permission fix made this reachable.
- match->xml: use String/format with Locale/ROOT so %.3f always renders a '.'
  decimal separator (clojure.core/format uses the default locale; a
  comma-decimal site locale produced score="0,950").
Both covered by new regression tests. (roborev job 2042)

* Address claude-fable-5 branch review

- core_test e2e: bind a superuser around the tool calls — the permission
  filter (mi/can-read?) needs *current-user-id*, so the test was returning []
  and would fail wherever it actually runs.
- semantic-search.yml: add the curated_search EE test dir to each job's :only,
  so the pgvector-gated tests (reconcile, rebuild, e2e) actually run in CI
  instead of silently no-opping — which is why the broken e2e went unnoticed.
- dedupe-by-entity: normalize the model so a "card" and a "question" entry
  for the same Card collapse to one (both hydrate to one record).
- ordering: move CuratedSearchEntry / curated-search to their alphabetical
  slots in exported-models, mage driver-affecting-overrides, and the kondo
  lint-as map (+ column alignment).
(roborev job 2048)

* Address claude-fable-5 review: schedule sync independent of feature flag, harden hydration + prompt

- sync task: schedule the Quartz job whenever pgvector is configured (boot-fixed),
  not gated on the semantic-search feature. The job body already self-gates, so
  this keeps the periodic safety net + the write-path trigger's target job alive
  when a license is enabled after startup (otherwise the mirror silently never
  syncs while the tool is offered). New pgvector-configured? predicate.
- card-refs->results: emit one record per ref so a card registered under two
  type strings no longer collapses (last-type-wins) and drops a match.
- API: search_prompt is NonBlankString on POST/PUT — reject blank prompts
  (meaningless embedding + degraded serdes slug).
All covered by new tests. (roborev job 2110)
2026-06-20 00:07:54 +02:00
github-automation-metabase
8b90dc2110 Auto-update documentation for master (#76193)
Update generated docs (master)

Co-authored-by: github-actions <github-actions@github.com>
2026-06-19 17:31:48 -04:00
Mihael Konjević
b907ae761d Make SQL guardrail skills always-on (#76176)
* Make SQL guardrail skills always-on

The on-demand skills system demoted read_resource and the SQL
create/edit/replace guidance from always-present system-prompt content to
catalog-only skills the agent had to load itself. With the guardrails gone
from the prompt, the SQL profile would loop read_resource calls (no
"drill, don't re-search"/"honor truncation" rules) and fire redundant
edit_sql_query calls instead of one replace_sql_query (no edit-vs-replace
guidance).

Mark read-resource, create-sql-query, edit-sql-query, and replace-sql-query
as always-on so their bodies are inlined into the prompt prefix again, while
keeping the bulky rarely-used skills on-demand. Harden load_skill
registration to fire when the manifest has any catalog or always-on skills,
so the dialect-preload invariant no longer depends on read-resource sitting
in the catalog.

* Make ask-for-sql-clarification always-on to restore SQL termination

The :sql profile is :required-tool-call?, which maps to Anthropic
tool_choice {:type "any"} — the model is forced to emit a tool call on
every turn and can never stop on free text. The agent loop only terminates
on max-iterations or a tool result carrying :final-response?. Among the SQL
profile's tools, ask_for_sql_clarification is the only one that sets it, so
it is the model's de facto termination signal.

The skills commit (ec842e1) moved the clarification guidance from the
always-inlined tool instructions to a catalog-only on-demand skill. Its
content was unchanged, but it was no longer present in the prompt, so the
model lost awareness of its terminal/handoff role and instead spun on
read_resource until hitting max-iterations (observed: a single edit_sql_query
followed by 19 redundant read_resource calls).

Mark the skill always-on so its guidance is inlined again, restoring the
pre-commit behavior. Together with the other SQL guardrail skills, the
:sql profile's catalog is now empty (all guidance inline, as before);
load_skill stays registered via the manifest's always-on entries so dialect
preload keeps working.

* Exclude always-on skills from the loadable/advertised skill set

Always-on skill bodies are already inlined in the system prompt, so the
model must neither see them in the catalog nor be able to load_skill them.
The catalog already removed them, but record-loadable-skill-ids! still
recorded every visible skill, leaving always-on ids loadable — so the agent
could burn an iteration re-fetching a skill it already has inline.

Record only the on-demand (non-always-on) skills as loadable, and build the
catalog from that same set.

* Make always-on a per-profile decision instead of a skill-global flag

A skill's always-on/on-demand status was a boolean on the skill itself, so it
applied to every profile exposing the matching tool. That is wrong: the SQL
profile wants its SQL skills inlined on every turn, but the internal profile
(which shares read_resource/create/edit/replace_sql_query) wants them
on-demand to keep its prompt prefix lean.

Move the decision to the profile: a profile lists :always-on-skills (skill
ids) whose bodies are inlined; every other relevant skill stays on-demand in
the catalog. build-skill-manifest now partitions on the profile's set rather
than the skill's flag. The skill-level :always-on frontmatter is removed, and
register-profile! validates that :always-on-skills resolve to real skills.

The :sql profile inlines read-resource + the create/edit/replace/clarification
SQL skills; no other profile opts in, restoring their pre-skills on-demand
behavior. Adds tests for the per-profile split and for always-on bodies
actually appearing in the rendered system prompt.

* Don't render the load_skill catalog header when the catalog is empty

Selmer treats an empty vector as truthy, so a profile whose skills are all
always-on (empty catalog) still rendered the "# Available skills … load the
skill(s) you need with load_skill" header with nothing listed — nudging the
model into pointless load_skill calls. Pass the catalog through not-empty so
an empty catalog is nil (falsy) and the header is omitted.

* Let a profile mark tools terminal; end :sql turn after a successful write

Under :required-tool-call? the model is forced to emit a tool call every
turn and can only stop on max-iterations or a :final-response? result. After
a clean edit_sql_query the SQL profile had no terminal tool to reach for, so
it spun on read_resource to max-iterations (e.g. 1 edit + 19 redundant reads).

Add a per-profile :terminal-tools set: a *successful* call to one of these
tools ends the turn. Success is detected by the presence of :structured-output
(the query/chart tools attach it only on success, returning just an error
:output otherwise), so a failed edit still lets the model self-correct.
Terminality is per-profile — the same write tools stay non-terminal in
profiles like :internal that need multi-step flows.

The :sql profile marks create/edit/replace_sql_query terminal. register-profile!
validates that :terminal-tools refer to tools the profile exposes.

* Replace tool-level :final-response? with per-profile :terminal-tools

Terminality was expressed two ways: the new per-profile :terminal-tools and
the older tool-level :final-response? flag (set by ask_for_sql_clarification
and the document_construct_*_chart tools). Two mechanisms for one concept —
and, like always-on on skills, the tool-level flag couldn't vary by profile.

Consolidate on :terminal-tools. Both tools already pair their terminal result
with :structured-output (failures return only :output), so success-gated
:terminal-tools is behaviour-preserving:
- :sql adds ask_for_sql_clarification to its terminal set.
- :document-generate-content marks document_construct_model_chart and
  document_construct_sql_chart terminal (schema collection stays preparatory).

Remove :final-response? from the tools and drop has-final-response? from the
loop. No frontend/persistence consumer read the flag.
2026-06-19 23:28:14 +02:00
Uladzimir Havenchyk
3f54ca2ce3 chore: replace more deprecated buttons with mantine buttons (#76051)
* chore: replace more deprecated buttons with mantine buttons

* cleanup

* fix button size

* fixup

* fixup

* fixup

* fixup

* fixup

* fixup

* chore: replace more deprecated buttons with mantine buttons

* fixup

* apply suggestion

* fix test
2026-06-19 21:05:54 +00:00
appleby
951e37d922 Normalize row charts before sending to metabot in chart configs (#76102)
* Normalize row charts before sending to metabot in chart configs

* Validate chart configs in analyze-chart-tool
2026-06-19 13:08:35 -07:00
Riley Thompson
93954e82de Exclude remapped columns from saved question return columns (#75931)
exclude remapped columns from saved quesiton return columns
2026-06-19 15:59:43 -04:00
Riley Thompson
ff472d84da Re-enable test and clean up fixmes (#75963)
* re-enable data-editing-requires-describe-features-test

* remove trivial fixmes

* set nested-field-columns to false for mariadb

* check for not maria db
2026-06-19 15:59:26 -04:00
Eric Normand
f5e0129bf6 Show success confirmation when remote sync import/export completes (GHY-3747) (#76099)
* Show success confirmation when remote sync import/export completes (GHY-3747)

Remote sync import/export can take a while, so the progress modal now stays
open on success showing a confirmation message and a Close button, instead of
silently auto-dismissing.

The task carries a general-purpose `message` column (reusable beyond success)
populated from the sync result on completion and exposed via GET /current-task.
The modal renders the server message, falling back to generic copy when none is
provided.

* Store sync result as structured outcome, render copy on the client (GHY-3747)

Instead of storing customer-facing copy in remote_sync_task, store a structured
outcome map (e.g. {:kind "pulled" :count 12 :branch "main"}) and let the UI map
it to a localized message. This keeps copy out of the database and renders it in
the viewer's locale rather than freezing the worker thread's locale at write time.

- Migration: remote_sync_task.message -> outcome (JSON)
- complete-sync-task! takes an outcome map (multi-arity); :outcome JSON transform
- impl success paths return :outcome data; drop the tru strings
- Frontend: RemoteSyncOutcome union, modal maps kind -> copy, falling back to
  generic per-task-type copy when the shape isn't recognized

* Drop the no-auto-dismiss middleware test (GHY-3747)

It asserted the absence of the removed auto-dismiss behavior and needed a
subscribe:false workaround to avoid an invalidation refetch loop. The success
state (renders with a Close button, no auto-dismiss) is covered by the
SyncProgressModal component tests.

* Refactor.

* get rid of silly comment

* Test per-kind fallbacks for malformed sync outcomes (GHY-3747)

Cover the kind-specific fallback branches in getSuccessMessage when a known
outcome kind is missing its fields, including the combined pull-and-push copy
for a malformed merged outcome.
2026-06-19 19:33:58 +00:00
Riley Thompson
c95bba80cb Use scanner over regex in make-nestable-sql (#75970)
* use scanner over regex in make-nestable-sql

* clean up
2026-06-19 14:38:57 -04:00
lbrdnk
748eeb1ce7 Fix agent api /query pagination (#76013)
* Fix results pagination

* Adjust description
2026-06-19 20:35:59 +02:00
Anderson
22b1e3d54c Fix native time grouping ordinal labels (#76047)
* Fix native time grouping ordinal labels

* Address review
2026-06-19 15:20:12 -03:00
Sloan Sparger
94dc98834e [Metabot] Add metabot message validation schemas (#75546)
* add malli schemas for ai-sdk v4 + v5/v6 message formats

* add disposable v4 validation endpoint

* add fn to handle historically invalid v4 messages, remove endpoint, add comment form for validation

* split spec version up into files

* automated clean up pass

* v4 naming improvements, v5+ snake case conversion, small correctness changes

* remove ai sdk v4 format given our internal implementation was so incorrect, move to v1 + v2 file names

* tighten types a bit

* remove code comments

* flip v2 schema keys to spec-faithful camelcase

* clean up

* add helpers

* copilot feedback

* impl PR feedback

* second round pr feedback
2026-06-19 18:04:45 +00:00
Denis Berezin
da6c396006 UXW-3079 Add date format settings for Trend viz (#76130)
* UXW-3079 Add date format settings for Trend viz

* Fix CI

* Review suggestions
2026-06-19 20:52:31 +03:00
lbrdnk
07853910a8 Enable searching for models through agent api (#76076) 2026-06-19 19:41:36 +02:00
Sloan Sparger
3fc052ed12 [Metabot] Don't send query in user is viewing context for questions without a data source (#75569)
* ensure there is a data source before registering query context

* disambiguate validation error message

* add extra test
2026-06-19 17:35:59 +00:00
Jeff Bruemmer
bc5719510d docs - rephrase SSO note (#76181)
rephrase
2026-06-19 13:16:20 -04:00
Mahatthana (Kelvin) Nomsawadi
3e09ab817b docs: add embedding telemetry transparency section (#76069) 2026-06-19 11:54:13 -04:00
Phoomparin Mano
03ed418f80 Fix sensible visualization options in MCP Apps visualization selector (#75876)
* fix visualization selector showing illegal visualizations

* reset the stored default display when loading a new query

* refactor: cleanup logic to capture default display

* update unit test name to be clearer

* add unit test to captureDefaultDisplay

* update prompt in MCP resource to not mention display types or viz settings

* do not accept stale query results after query changes
2026-06-19 15:53:53 +00:00
Anderson
4cc540400e Fix pie chart Other slice drill-through (#75938)
* Fix pie chart Other slice drill-through

* Limit grouped pie drill-through actions

* Add e2e test to account for the Other slice drill through options

* Address metamben's
2026-06-19 15:29:38 +00:00
Copilot
69a279d4ad docs: add driver removal steps and required driver exception (#75870) 2026-06-19 11:05:05 -04:00
Nicolò Pretto
598fc88b68 feat(sdk): Custom visualizations in the Embedding SDK (#75920)
* Add custom-viz plugin support to the embedding SDK

Wire the SDK to load EE custom-visualization bundles. The core loader gains a
`sandboxMode` so the SDK can run plugins in an `about:blank` sandbox, fetches
bundles through the authenticated `api.fetch` client, and exposes
`unregisterCustomVizDisplay` / a `loadCustomVizPluginForDisplay` slot so a saved
`custom:*` question keeps its display instead of resetting to a built-in viz.

No allowlist yet: any installed custom-viz plugin loads.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* Gate SDK custom-viz behind allowedCustomVisualizations

Add the `allowedCustomVisualizations` MetabaseProvider prop and gate every
custom-viz plugin slot (load, auto-load, list, icon) on it: only listed plugin
identifiers load, and a display dropped from the allowlist is unregistered so
the question falls back to a built-in visualization.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* Add initialVisualization prop to SDK questions

Let host apps override the saved display when a question loads via an
`initialVisualization` prop on InteractiveQuestion/StaticQuestion. Accepts
built-in chart types and `custom:*` displays; the custom plugin is loaded
concurrently with the query and the prop falls back to the saved display when
the requested one isn't available or allowed.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* Show custom visualizations in the SDK chart-type picker

Eagerly load the allowed custom-viz plugins so their displays register in the
global visualizations map and appear in the chart-type dropdown, mirroring the
main-app ChartTypeSidebar. Lets users switch a question to a custom viz from
the picker.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* Support loading icons via blobs in the sdk

* Update enterprise/frontend/src/embedding-sdk-ee/custom-viz/initialize.ts

Co-authored-by: Kamil Mielnik <kamil@kamilmielnik.com>

* Update enterprise/frontend/src/embedding-sdk-ee/custom-viz/initialize.ts

Co-authored-by: Kamil Mielnik <kamil@kamilmielnik.com>

* Update enterprise/frontend/src/metabase-enterprise/custom_viz/sandbox/distortions-dom-read.ts

Co-authored-by: Kamil Mielnik <kamil@kamilmielnik.com>

* fix icon in collection browser

* use more specific types instead of "number"

* drop call site from code comment

* remove describeNode (unused) and unused imports

* change `allowedCustomVisualizations` to require `custom:` prefix

* warn the host app dev when they pass down plugins that we don't find in the instance

* temporarily disable initialVisualization

* update e2e after the removal of console.error

---------

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-authored-by: Kamil Mielnik <kamil@kamilmielnik.com>
2026-06-19 16:49:57 +02:00
Nick Fitzpatrick
9228b95e0b Fix flaky template-tags transform test by waiting on the save request (#76157)
The undo-toast assertions ran immediately after clicking Save, racing the
PUT /api/transform/* round-trip. Wait on the existing @updateTransform intercept after each Save
2026-06-19 19:36:33 +05:30
Copilot
23c363b9af docs: document auth method switching behavior for existing accounts (#75900) 2026-06-19 09:51:40 -04:00
Alexander Polyankin
e615ca6f97 Fix memory leak with notifications and GraalVM (#76145)
* Fix memory leak with table notifications

* fix
2026-06-19 09:32:19 -04:00
Uladzimir Havenchyk
a57898c7c3 chore: replace more deprecated buttons with mantine buttons (#75903)
* chore: replace more deprecated buttons with mantine buttons

* cleanup

* fix button size

* fixup

* fixup

* fixup

* fixup

* fixup

* fixup

* apply suggestion

* fixup
2026-06-19 12:42:52 +00:00
Sébastien
7dc9d844b3 Fix flaky editable-dashboard create-question e2e test (#76011) 2026-06-19 13:48:22 +02:00
Romeo Van Snick
8948318340 Convert mode.js to TypeScript (#76135) 2026-06-19 20:52:00 +10:00
Oleh K
f399a41952 Fix find-never-defined-css-variables for multi-line and whitespace var() usages (#76054)
* GDGT-2642 Fix find-never-defined-css-variables for multi-line and fallback var()

The CSS-variable lint only matched contiguous `var(--x)`, so any whitespace,
newline, or fallback between `var(`, the name, and `)` silently defeated
detection. Tighten the usage regex to `var(\s*(--name)\s*[,)])` so it catches:

  - multi-line:        var(\n  --foo\n)
  - inner whitespace:  var( --foo )
  - fallback syntax:   var(--foo, red)

Requiring a trailing `)`/`,` also skips dynamic `var(--mb-color-${x})` template
usages we can't statically validate. A `var(--x, fallback)` is treated as safe
unless `--x` is one of our `--mb-*` design-system primitives, which must always
be defined (a fallback would silently mask a typo or a dropped backport).

Add unit tests covering the bare, whitespace, multi-line, fallback, nested,
mb-with-fallback, and dynamic-interpolation cases.

The tightened lint surfaced several latent bugs, now fixed:
  - EmbedFrame: `--mb-color-background-tertiary-inverseer` typo -> `-inverse`
  - ThemeEditor PreviewPanel: nonexistent `--mb-radius-md` -> `--mantine-radius-md`
  - Switch: `--mb-color-accent-3` typo -> `--mb-color-accent3` (invisible error border)
  - Select: removed dead `.Select` rule and `--select-item-*` declarations that
    referenced variables never defined by Mantine or our theme

Also pruned the knownIssues whitelist from 10 entries down to 2 (only the
genuinely Mantine-provided `--button-bg` and runtime-generated
`--mantine-color-accent-gray-light-filled` remain).

* Clarify why --mantine-color-accent-gray-light-filled is allow-listed

getMantineThemeColors registers the color; Mantine derives the -filled variant
var from it at runtime, so it's absent from the static styles.css we scan.

* Add test for mb-* fallback wrapper with nested bare var

Mirror of the non-mb-wrapper case: an mb-* var carrying the fallback is still
validated, and the nested bare var is too, so both are extracted.

* Remove comment
2026-06-19 12:22:58 +02:00
Nick Fitzpatrick
194821f222 Apply column formatting to static viz map legend, add render to side (#76024)
* Apply column formatting to static viz map legend, add render to side

* PR Feedback
2026-06-19 14:27:28 +05:30
Nicola Mometto
fe44abd20a More generic error message on transform timeout (#76071)
chore: don't leak impl detail on error message
2026-06-19 08:49:53 +00:00
Fraser
24ca0fe913 Error for files that aren't in modules and enforce remaining modules (#76126) 2026-06-19 13:59:48 +12:00
Alexander Polyankin
1c12b1931d BigQuery - sample-table with adaptive page size (#76068) 2026-06-18 23:41:36 +00:00
Timothy S. Dean
28d0a1c5a5 Drain stale dependency entities oldest-first to prevent starvation (#76118)
The dependency entity-check job re-analyzes stale entities in batches. Among entities that are equally stale, the order they're selected in is arbitrary. When the stale backlog is larger than one batch, the same subset can keep getting picked while others are repeatedly passed over — so some stale entities can go unprocessed run after run.

### Solution

Select stale entities oldest-analyzed-first. Each entity that's just been analyzed moves to the back of the line, so the job round-robins through the whole backlog and every stale entity is reached.
2026-06-18 17:23:27 -06:00
Timothy S. Dean
c6be4cae58 Clear the stale flag for entities whose database can't be resolved (#76119)
The dependency entity-check job analyzes each stale entity and records a finding, which clears its stale flag.

If an entity's database can't be resolved — and there's no prior error that already explains why — analysis silently does nothing. The stale flag is never cleared, so the entity is re-selected and re-analyzed on every run, forever.

### Solution

Record a terminal error finding for that case, which clears the stale flag and explains why the entity couldn't be analyzed. If its database later becomes resolvable, the entity is analyzed normally again.
2026-06-18 17:23:00 -06:00
Ryan Laurie
0ccaefe55c Auto Stress Test (#76034)
* auto stress test

* better options

* double test

* variable burn

* less ai comment

* less ai comment
2026-06-18 23:18:40 +00:00
Nemanja Glumac
ac0c1c3d3e Add ci-conductor (dry run) quarantine gate for e2e tests (#76014) 2026-06-18 23:56:34 +02:00
github-automation-metabase
366f27d2f0 Auto-update documentation for master (#76120)
Update generated docs (master)

Co-authored-by: github-actions <github-actions@github.com>
2026-06-18 21:50:47 +00:00
dpsutton
a57027861f Fix OOM in table-sync permission setup by SELECTing DISTINCT (#76077) (#76100)
When the sync-and-analyze job creates new tables, `set-default-table-permissions!`
calls `load-perm-context`, which loaded *every* table-level `data_permissions`
row for the database to figure out, per (group, perm-type, schema), the set of
distinct permission values already in use. The result set grew with the table
count, so on databases with very many tables it materialized millions of rows
(2.5M+ `org.postgresql.core.Tuple` / `toucan2.instance.Instance` in the heap
dump that prompted this) and OOM'd the JVM on a scheduler thread.

`schema-vals-idx` only ever needs the *distinct* (group_id, perm_type,
schema_name, perm_value) tuples — it reduces the rows into a set keyed by
(group, perm-type, schema), so duplicate rows were already being collapsed in
memory. Pushing that DISTINCT into SQL makes the loaded set bounded by
groups × perm-types × schemas × values instead of growing with table count.

Rows loaded by the introspection query (measured; one perm-type, one schema):

    groups  tables   before (groups×tables)   after (DISTINCT)
      5       10              100                    20
      5      100            1,000                    20
      5    1,000           10,000                    20   <- flat across tables
     10      100            2,000                    40
     50      100           10,000                   200   <- linear in groups

Before, the result scaled with the table count (the thing that blew up); after,
it is flat across table count and grows only linearly with the number of groups.
The produced `schema-vals-idx` is byte-identical because it accumulates into a
set — verified in a regression test.

Not scoping the query to the synced tables' schemas (a further reduction):
a `schema_name IN (...)` filter would add NULL/casing correctness risk, and the
DISTINCT collapse alone removes the geometric blowup that caused the OOM.

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-18 16:18:41 -05:00
Oreon Lothamer
7d485d539d run PR-Env-Stats-Dump on already labeled PRs (#76095) 2026-06-18 11:13:38 -10:00
dpsutton
718c730aa9 downgrade apache sshd-core (#76104)
Why? Because 3.0.0-M2 leaks connections.

 Revert mina-sshd 3.0.0-M2 → 2.16 to stop SSH channel leak OOMs

  3.0.0-M2 leaks one KexListener per SSH port-forward channel, never
  removed on channel close. Each Metabase DB query over an SSH tunnel
  opens and closes a `direct-tcpip` channel, so the listener list grows
  unbounded for the life of the (long-lived) SSH connection.

  Heap dump from a crashed prod instance (highway):
  - 1 SSH connection, but 67,221 closed TcpipClientChannel /
    Nio2Session / socket-channel objects retained.
  - KexFilter.listeners (a session-scoped CopyOnWriteArrayList) retained
    572 MB ≈ 38% of heap — full of AbstractChannel$$Lambda instances,
    each capturing one closed channel.

  Root cause in mina-sshd 3.x (dev_3.0 branch):
  - AbstractChannel.init() registers an anonymous KexListener on the
    session for every channel and never removes it on close.
  - AbstractSession.removeKexListener() is also broken: it calls
    addKexListener() instead of remove (copy-paste bug), so even a
    correct removal attempt adds a duplicate.

  The 2.x lineage has no KexFilter / addKexListener / per-channel KEX
  listener — the filter-chain mechanism is entirely new in 3.x — so the
  leak cannot occur on 2.16.
2026-06-18 14:01:00 -07:00
Alexander Polyankin
ddbb6d05e5 BigQuery - ignore tables and fields with long names that we cannot insert into the app db (#76074) 2026-06-18 20:41:51 +00:00
Andrei Chernykh
b04009322f Fix user provisioning getting disabled when adding a group mapping (#75829)
* Fix user provisioning getting disabled when adding a group mapping

* Improve group mappings widget layout and add autosave toasts

* Migrate GroupMappingsWidget off Emotion to Mantine and drop deprecated exports

* Migrate GroupSelect off legacy Select to Mantine Combobox and trim widget CSS to style props

* Show Changes saved toast on JWT settings save and add group field placeholders for SAML and LDAP

* Fix group-mappings e2e helper for the Combobox dropdown

* Consolidate redundant OIDC group-sync and tooltip unit tests

* Give group-select options an explicit aria-label for a stable accessible name

* Scope group-mapping consistency e2e assertions to the table to avoid the portaled dropdown

* Use Text instead of Box for the delete-mapping modal heading and subtitle

* Refactor delete-mapping modal to use Mantine Modal from metabase/ui

* Migrate deprecated Button and Radio to metabase/ui in the group mappings widget
2026-06-18 12:14:43 -07:00
Nick Fitzpatrick
6d294e5f4d Further pdf export improvements (#76016)
* use cssText

* include font props on <text>, <tspan>, and <textPath> nodes

* re-apply SVG transform after cssText copy in html2canvas patch

Chrome's getComputedStyle().cssText is empty, so the cssText copy wiped the Leaflet grid SVG's transform and offset its serialized bounds, re-breaking the #68751 PDF-export grid fix. Re-apply transform/transform-origin/transform-box after the cssText copy; Firefox keeps the bulk-copy perf win. Re-enable the GridMapPdfExport loki test with its baseline reference.

* resolve vars in svg props

* fix hidden row chart axis labels

* gauge value positioning

* fix gauge chart labels

* re-skip grid map loki test
2026-06-18 23:54:41 +05:30
Nathan Voxland
4181df4429 Include dashId in downloaded diagnostics (#75803)
* Include dashId in downloaded diagnostics

* e2e test

---------

Co-authored-by: Nick Fitzpatrick <nickfitz.582@gmail.com>
2026-06-18 23:50:40 +05:30
Eric Normand
c70f62de5c Incremental remote-sync import: load only changed entities on pull (GHY-3779) (#75897)
* Add git changed-files primitive for incremental remote-sync import

changed-files diffs two commit trees with jgit's ANY_DIFF filter,
returning {:added :modified :deleted} path sets. jgit prunes unchanged
subtrees during the walk, so cost is proportional to the number of
changed entries rather than the size of the tree. Returns nil when the
base commit is unresolvable or orphaned (force-push/rebase), signalling
callers to fall back to a full import.

This is the change-detection primitive for making import/pull cost
proportional to what changed (GHY-3779).

* Add differential equivalence harness for incremental import

Establishes the full reload (load-snapshot!) as the oracle for incremental
import: a correct incremental import must leave the app DB, RemoteSyncObject
table, transforms setting, and version pointer in exactly the state a full
import would. Rather than assert hand-computed expectations, the harness runs
both paths against the same change and asserts identical resulting state.

import! provides the seam: :force? true always takes the full path (oracle),
:force? false is the path under test (full today, incremental once GHY-3779
lands). The state vector is pk-independent so delete+recreate doesn't produce
false diffs. Covers no-op / edit / add / delete card scenarios; the
rename/move/delete-collection/transform cases come with the incremental path.

GHY-3779.

* Incremental import fast-path: load only changed entities on pull

On a subsequent (non-first, non-forced, non-dirty) import, load only the
entities whose files changed since the last imported version instead of
deserializing and upserting the entire snapshot.

- Diffable protocol + source.p/changed-files wrapper: git diffs two commit
  trees; the in-memory versioned test source diffs its trees; everything else
  returns nil (no diff available) and falls back to full.
- incremental-load-snapshot!: loads only added/modified files (deps resolve
  from the app DB), deletes only genuinely-removed entities, and reconciles
  just the affected RemoteSyncObject rows. Renames are matched by entity_id, so
  a leaf rename updates in place rather than delete+recreate. Skips the full
  backfill and reindex that the whole-snapshot load runs.
- import! tries the fast-path and falls back to the full load-snapshot! for
  anything structural: first import, force/dirty (only full truly discards
  local drift), an unresolvable base (force-push/rebase), a deleted file with
  no recorded path, or a changed Collection/feature model (v1 scope). Also
  hoists the no-op version-skip ahead of conflict detection so an unchanged
  pull reads nothing.

The differential harness now drives the fast-path behind force? false and
proves it equals the full oracle for no-op / edit / add / delete / rename of
leaf content. (The harness caught a real rename bug during development.)

GHY-3779.

* Update search index per-entity on incremental import

The incremental load skips the whole-appdb reindex, so keep the search index
consistent at a cost proportional to the change: after the reconcile commits,
search/delete! the genuinely-removed entities and search/update! the loaded
(added/modified) ones. Adds search to the enterprise/remote-sync module :uses.

Covered by a test asserting only the changed entity is re-indexed, not the
whole synced set.

GHY-3779.

* Extend differential harness with fast-path assertions and fallback scenarios

run-differential! now reports which path the under-test import took, so every
scenario asserts the v1 scope boundary explicitly:
- leaf changes (no-op / edit / multi-edit / add / delete / rename) run
  :incremental and match the full oracle;
- structural changes (collection rename, collection delete with contents) fall
  back to :fallback (the full import) and still match the oracle.

GHY-3779.

* Improvements

* Drop redundant search/update! on incremental import; rely on serdes load hooks

Serdes load already re-indexes added/modified entities: its t2 insert!/update!
fire the :hook/search-index after-insert/after-update hooks, which call
search/update!. So the explicit per-entity search/update! in the incremental
path was redundant double-work — remove it. Deletes have no such hook, so the
explicit search/delete! for removed entities stays.

Tests: replace the live appdb search-index integration test (its index-tracking
state is order-sensitive across tests in a shared JVM, causing flakiness) with a
stable call-level assertion that search/delete! is invoked with the removed
entity's id. The differential harness + update call-spy continue to cover the
rest. Disable index maintenance in the non-search scenarios so their async
ingestion can't interfere.

GHY-3779.

* Flatten import! decision tree and fix incremental/merge fallback bugs

Refactor the remote-sync import path into flat cond-based decision trees
that read top-to-bottom, and fix correctness/perf issues surfaced along
the way:

- import! / incremental-import-plan / import-merged!: replace nested ifs
  with flat conds (one predicate -> one action per clause).
- incremental-import-plan: guard the diff call so a first import (nil
  last-version) returns the fallback sentinel instead of NPEing on a real
  git source (resolve(nil)).
- import! and import-merged!: compute the expensive conflict scan and
  3-way merge lazily (delay), so no-op pulls and the nil-base / force /
  merge paths don't pay for a full-tree read, and the nil-base merge
  returns its :conflict result instead of throwing.
- Consolidate the repeated full-reload success result via a local
  full-import! closure; route finalize! and the result through the bound
  snapshot-version.
- Make incremental-load-snapshot! private; inline handle-import-exception
  into import!'s catch; refactor prog1 calls to do/let.
- Add a first-import-no-force test (incremental fast-path must not run on
  a first import) and fix the path-detection spy to compare against the
  current sentinel var.

Pre-existing conflict-fixture failures in impl_test are unaffected.

* Batch per-entity deletes and search-index removals in incremental import

incremental-load-snapshot! removed entities (and their search-index
entries) one row at a time. Batch both by model type:

- entity deletes: one (t2/delete! model-key :id [:in ids]) per model
  instead of per row. Toucan2 fires before-delete per matched row even
  for an :in delete, and the Card/Dashboard before-delete cascades are
  row-independent, so behavior is preserved. Still inside the existing
  reconcile transaction.
- search-index removal: one search/delete! per model type (it already
  takes a collection of ids).

The remaining per-row lookups (deleted file_path -> RemoteSyncObject,
RemoteSyncObject -> entity_id) are left as-is; they're bounded by the
number of deletions in a single pull.

* Tag remaining incremental-import N+1 queries with comments

Note the two per-row lookups left un-batched in the incremental import
path (deleted file_path -> RemoteSyncObject, and RemoteSyncObject ->
entity_id) and why: both are bounded by the number of deletions in a
single pull, and batching either has real downsides (long file_path IN
clauses; per-model-key IN grouping).

* Reorder import! so the incremental path skips the conflict scan

After merging GHY-3900, blocking-conflicts was computed before the
incremental fast-path, re-introducing a full-tree get-conflicts scan on
every changed pull and undercutting the constant-time incremental win.

Reorder import!'s cond so the incremental clause sits ahead of the
conflict gate: an incremental load touches only changed files and can't
wholesale-delete unsynced transforms, so it needs no conflict scan and
forces only the cheap O(changes) diff. The conflict gate and full reload
fall to the bottom, so every full-reload path (forced, dirty, first
import, structural) is still gated. force? keeps no standalone branch —
it stays off the incremental path and flows through the gate so
force-deletion? semantics hold.

Also defer the get-conflicts scan behind an inner delay so it runs only
when a gate is open (a forced import with force-deletion? true skips it),
and inline the now single-use full-import! body into :else.

* refactor
2026-06-18 17:04:02 +00:00
lbrdnk
928e9098e2 Add pagination to read_resource (#74966)
* Add pagination to read_resource tool

* kondo

* Review remarks
2026-06-18 18:31:43 +02:00
Nathan Voxland
73cd4c7c1e Pass scopes through the oidc check function (#75416) 2026-06-18 11:20:38 -05:00
Sébastien
5ac6536012 test(sdk-bundle): Fix flaky e2e test (data picker) (#76057) 2026-06-18 17:42:28 +02:00
Nemanja Glumac
42e75a07ba (chore): Point ci-conductor reporter at a base-URL secret (#76067)
Replace the CI_CONDUCTOR_WEBHOOK_URL secret (which held the webhooks base)
with CI_CONDUCTOR_BASE_URL holding the bare origin, and have the reporter
build ${base}/webhooks/failed-tests. This lets each consumer append its own
path.

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-18 17:39:17 +02:00
Anderson
11de545b4b Fix public pivot table downloads (#76023) 2026-06-18 11:52:24 -03:00
Anderson
8894b2a7eb Fix conditional formatting for mini bar cells (#76025) 2026-06-18 11:51:37 -03:00
Anderson
6ea899e4b0 Fix progress Goal input styles / Convert ChartSettingFieldPicker to typescript (#75916)
* Fix progress goal input chevron alignment

* Convert components to typescript and css modules / Remove ghost dropdown caret

* Convert test file to ts

* Address Nick's comment
2026-06-18 11:49:54 -03:00
Nemanja Glumac
f35c54150c Harden the flaky text fix command (#76005) 2026-06-18 15:53:00 +02:00
github-automation-metabase
8cfe03b28c Auto-update documentation for master (#76036)
Update generated docs (master)

Co-authored-by: github-actions <github-actions@github.com>
2026-06-18 09:14:18 -04:00
lbrdnk
9331d39f46 Stamp types on integer id field clauses in resolve-query (#75998)
Stamp types on field clauses after resolve

lib.normalize does not inject type info from the metadata provider.
Add annotate-field-types pass in resolve-query to fill this gap for
all [:field opts integer-id] clauses missing :base-type.
2026-06-18 14:35:31 +02:00
Maz Ameli
3f26fa8e53 Fix capitalization of "View schema" table action label (#76035)
Change "View Schema" to "View schema" in the data model table actions
menu and update the affected unit and e2e tests.

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-06-18 11:44:58 +00:00
Nicola Mometto
f0fc22bb27 Add BE endpoints for job run history (#75926)
* feat: link transform runs with their parent job

* feat: allow storing job-run-id

* add endpoints

* tests

* fix

* missing remark

* fix: add to schema

* fix schema

* fmt

* kondo

* fix test

* allow filtering by start-time and run-method

* tests

* chore: use read-check

* backfill job run ids

* chore: don't alias

* perf of the migration

* realias, add mysql/h2 versions, remove rollback

* chore

* prefilter on run_method
2026-06-18 12:02:25 +01:00
Fraser
8e2c0cc178 Remove legacy SettingsApi service (#76046) 2026-06-18 21:12:32 +12:00
Oleh K
94144e3411 Add KeyboardShortcut component and restyle Kbd (#75895)
* WIP: restyle Kbd from Figma + disabled state and showcase

Restyle the Kbd component to the GDGT-2473 Figma spec: flat surface
fill, no border/shadow, 4px radius, Lato Medium 12px. Add a thin
metabase/ui wrapper exposing a `disabled` boolean prop, a CSS `:hover`
state, and a Storybook showcase matrix.

* Add sequential shortcut examples to Kbd showcase

* Add KeyboardShortcut component, simplify Kbd, migrate palette

Per design discussion: drop Kbd's hover/disabled states (Kbd is now just a
restyled building block), add a KeyboardShortcut component that composes Kbd —
simultaneous keys adjacent (no "+"), sequential steps joined by a localized
"then", with a canonical modifier order and symbol/casing normalization. Migrate
the command-palette shortcuts modal to it (replacing its local renderer) and
list multiple shortcuts comma-separated.

* Add KeyboardShortcut Overview story to Loki visual regression tests

* Render Shift as a word in KeyboardShortcut, not a symbol

Per design: drop the SHIFTKEY (⇧) mapping so Shift always renders as the word.

* Extract KeyboardShortcut key parsing to utils with unit tests

Move formatKey/parseShortcut into a colocated utils.ts and cover them with unit
tests (each formatKey branch + chord/sequence/modifier-ordering for parseShortcut).
2026-06-18 08:59:18 +02:00
Romeo Van Snick
66a53ed09b Move SessionApi to RTK query hooks (#76045)
* Add session RTK Query mutations

* Migrate auth thunks to session RTK mutations and remove SessionApi
2026-06-18 15:40:55 +10:00
Romeo Van Snick
21a319ed95 Remove simple services API's (#75866)
* Move `FrontendErrorsApi` to RTK hooks and update callers

Replace the legacy `FrontendErrorsApi.report` service with a
`reportFrontendError` RTK Query mutation. The two callers are class-based
error boundaries (`componentDidCatch` requires a class), so each gets a thin
functional wrapper that calls `useReportFrontendErrorMutation` and passes a
bound `onReportError` callback down to the inner class component.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* Move `UserApi` to RTK hooks and update callers

Add `getCurrentUser` and `updateUserModalQbnewb` endpoints to the user RTK
API (`UserApi.list` was already covered by `listUserRecipients`) and remove
`UserApi` from services.js.

Component callers use the hooks directly (`UserNameDisplay` via
`useListUserRecipientsQuery`); thunk and imperative callers dispatch through
`runRtkEndpoint`, matching the existing redux/api migration pattern.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* Move `SetupApi` to RTK hooks and update callers

Replace the legacy `SetupApi.create` service with a `createSetup` RTK Query
mutation. The sole caller is the `submitUser` setup thunk, which dispatches it
through `runRtkEndpoint` so its existing try/catch error handling is preserved.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* Move `PersistedModelsApi` to RTK hooks and update callers

The `persistApi` already had matching `enablePersist`/`disablePersist`/
`setRefreshSchedule` mutations, but they were unused and `setRefreshSchedule`
pointed at the wrong URL (`set-refresh/schedule` instead of the backend's
`set-refresh-schedule`). Fix that URL and wire `ModelPersistenceConfiguration`
onto the hooks, using `.unwrap()` so the existing error-toast handling still
fires on failure. Remove `PersistedModelsApi` from services.js.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* Move `PermissionsApi` to RTK hooks and update callers

Add the data-permissions graph endpoints (`getPermissionsGraph`,
`getGroupPermissionsGraph`, `getDatabasePermissionsGraph`,
`updatePermissionsGraph`) to `permissionApi`. All callers are thunks or
imperative hook callbacks, so they dispatch through `runRtkEndpoint` (which
preserves the legacy fetch-every-time behavior and cleans up the query
subscription). Remove `PermissionsApi` from services.js.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* Move `ParameterApi` to RTK hooks and update callers

The `parametersApi` already had matching `getParameterValues`/
`searchParameterValues` query endpoints. Route the `fetchParameterValues`
thunk's `loadParameterValues` through `runRtkEndpoint` and remove
`ParameterApi` from services.js.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* Move `StoreApi` to RTK hooks and update callers

Add a `getTokenStatus` query to `premiumFeaturesApi` (the token-status GET
that lived in `StoreApi`) and route `useLicense`'s status fetch through
`runRtkEndpoint`, preserving its 404-specific error handling. Remove
`StoreApi` from services.js.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* Move `RevisionsApi` to RTK hooks and update callers

The legacy `RevisionsApi.get` (`GET /api/revision/:entity/:id`) and the
existing `listRevisions` query (`GET /api/revision?entity=&id=`) both resolve
to the same backend `revisions+details` after a read-check, so route the two
revision-fetching thunks through `runRtkEndpoint` + `listRevisions`. Remove
`RevisionsApi` from services.js.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* Move `ModerationReviewApi` to RTK hooks and update callers

Route the moderation `verifyItem`/`removeReview` helpers through
`runRtkEndpoint` + the existing `editItemVerification` mutation (taking a
`dispatch` argument), and rework their unit tests to assert the real request
against a store-backed `fetchMock` instead of a mocked service. Remove
`ModerationReviewApi` from services.js.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* Move `CollectionsApi` to RTK hooks and update callers

Add `getCollectionPermissionsGraph`/`updateCollectionPermissionsGraph` to
`collectionApi` (the `graph`/`updateGraph` endpoints; `get` already existed as
`getCollection`). Route the collection-permission load/save thunks and the
error-page collection fetch through `runRtkEndpoint`. Remove `CollectionsApi`
from services.js.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* Move `GTAPApi` to RTK hooks and update callers

Add a `validateGroupTableAccessPolicy` mutation to `groupTableAccessPolicyApi`
for the GTAP `validate` endpoint. The `attributes` endpoint already existed as
`userApi.listUserAttributes`, so reuse it: `useUserAttributes` becomes a thin
`useListUserAttributesQuery` wrapper, and the enterprise `fetchUserAttributes`
thunk dispatches it via `runRtkEndpoint`. (`GTAPApi.list` had no callers.)
Remove `GTAPApi` from services.js.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* Move `PulseUnsubscribeApi` to RTK hooks and update callers

Add a `pulseApi` with `unsubscribeFromPulse`/`undoUnsubscribeFromPulse`
mutations (the unauthenticated, hash-based email unsubscribe endpoints) plus
shared `UnsubscribeRequest`/`UnsubscribeResponse` types, and route the pulse
branch of the unsubscribe page through them. Remove `PulseUnsubscribeApi` from
services.js. (Notification branch migrated separately.)

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* Move `NotificationUnsubscribeApi` to RTK hooks and update callers

Add `unsubscribeFromNotificationByEmail`/`undoUnsubscribeFromNotificationByEmail`
mutations to `notificationApi` (distinct from the existing authenticated
by-id `unsubscribeFromNotification`) and migrate the notification branch of the
unsubscribe page onto them. Remove `NotificationUnsubscribeApi` from
services.js.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* Move `EmbedApi` to RTK hooks and update callers

Add `getEmbedCard`/`getEmbedDashboard` query endpoints to `embedApi` (the base
embed card/dashboard fetches) and route the embed dashboard load and the
public/embedded question loader through `runRtkEndpoint`. Remove `EmbedApi`
from services.js.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* Move `PublicApi` to RTK hooks and update callers

Add the base public endpoints to `publicApi` — `getPublicAction`,
`getPublicCard`, `getPublicDashboard`, `getPublicDocument`,
`executePublicAction`, `executePublicDashcardAction`, and
`prefetchPublicDashcardValues` — and route all callers (public dashboard/card/
document/action loaders, action execution, and dashcard prefetch) through
`runRtkEndpoint`. Remove `PublicApi` from services.js.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* Ignore promise return value

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>

* Fix formatting

* Remove unused code

* Remove mention of EmbedApi

---------

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
2026-06-18 14:19:15 +10:00
Fraser
8460bb5470 Fix unit test violations (#75919) 2026-06-18 16:10:47 +12:00
Alexander Polyankin
b0c6b77bfa Avoid duplicate app db requests in public and embedded cards and dashboards (#76026)
* In-memory entity cache for public/embed paths

* now cards

* fix

* fix

* fixes

* fix

* fix

* fix

* fix

* fix

* fix

* fix

* ffix tests

* fix
2026-06-17 16:08:21 -07:00
technomancy
28b1a1507e Improve dataset creation for Bigquery (#76031)
* Stop deleting datasets for now in bigquery.

* Don't assume the database existing means all tables exist.
2026-06-17 22:50:56 +00:00
Nathan Voxland
4b3ee24236 Store SAML relay state in db (#75887) 2026-06-17 16:54:43 -05:00
Cam Saul
391f7141bf Add Kondo linter to check for empty testing forms (#76029)
* Add Kondo linter to check for empty `testing` forms

* Kondo

* Apply suggestion from @bgrabow

Co-authored-by: Ben Grabow <ben.grabow@metabase.com>

* Apply suggestion from @camsaul

---------

Co-authored-by: Ben Grabow <ben.grabow@metabase.com>
2026-06-17 14:32:10 -07:00
github-automation-metabase
94efd72ec4 Auto-update documentation for master (#75972)
Update generated docs (master)

Co-authored-by: github-actions <github-actions@github.com>
2026-06-17 19:54:19 +00:00
Eric Normand
82544e9255 Warn that force push deletes remote-only files (GHY-3917) (#76012)
Force-pushing to a diverged remote silently deleted files that existed
only on the remote branch (content created elsewhere); the confirmation
only said it would "overwrite the remote branch."

- Compute the force-push blast radius (deleted + overwritten entities) in
  the export preflight, reusing the trial merge's already-loaded trees,
  and return it as force_push_casualties.
- Surface it in the conflict modal: a ForcePushWarning with counts and an
  expandable file list shown when force-push is selected.
- Sharpen the force-push option label and split the options into safe
  (history-preserving) and destructive groups.
- Fix history-rewritten detection: commit-sha now verifies the resolved
  object exists, so an orphaned merge base resolves to nil instead of
  throwing MissingObjectException; compute casualties without a base there.
2026-06-17 14:42:31 -05:00
Nick Fitzpatrick
e61954d29b Show empty state in Admin Table Metadata Table Picker (#75927) 2026-06-18 00:32:39 +05:30
Braden Shepherdson
358be53b46 Propagate parent collection :type only for Library types (#76020)
Fixes UXW-4520.

Previously this propagated all `:type`s other than `"trash"`, which
accidentally copied the magic `"tenant-specific-root-collection"` and
broke creating subcollections for tenants.

This flips the logic from "block only trash" to "allow only library",
which fixes this issue and should be more future-proof.
2026-06-17 18:57:38 +00:00
Anderson
d07294ea21 Fix metrics explorer e2e flakes (#75564)
* Improve metrics explorer e2e tests and try to resolve flakes (attempt 2)

* Make sure expressions data points are clickable on metrics viewer charts

* Continue work on making metrics explorer tests more reliable

* Disable devtools in chrome by default when in headless mode

* Additional anti-flakiness improvements

* Adjust useFormulaEditor.ts to recover metric identities when they are dropped while editing

* Use UI state assertion instead of wait @dataset when formula input is run / Other improvements in metrics-explorer.cy

* Remove test that is vastly covered by the other tests

* Fix metric expression name identity recovery

* Keep 'Responsive viewer controls' test order

* Add metrics picker measure coverage
2026-06-17 15:52:03 -03:00
Mihael Konjević
665fed1d4f Fix metabot construct-query bugs: string-filter options + skill gaps (#76015)
Repair layer:
- New pass merges a trailing options-map into position-1 on the N-ary
  string-search filters (contains / does-not-contain / starts-with /
  ends-with). LLMs append the case-sensitivity options as a trailing element
  (e.g. ["contains" {} <field> "@gmail.com" {"case-sensitive" false}]); because
  these clauses are variadic the fixed-arity merge-trailing-options pass skips
  them, and the stray trailing map made lib.normalize fail with a missing
  lib/uuid "Invalid output" error. A string-search value is never a map, so the
  trailing map is unambiguously misplaced options. Reuses merge-trailing-options
  and is idempotent. Tests cover each head, multi-value, keys-win, nested-in-and
  postwalk, no-ops, and idempotency.

Skills (construct-notebook-query-advanced):
- Document that a `distinct` aggregation outputs a column named `count`, so
  cross-stage HAVING filters reference `["field" {} "count"]`, not "distinct".
- Add guidance for breaking out a metric by a dimension on another table via an
  explicit join (an implicit FK reference against a metric is rejected by the QP
  with a 400).
2026-06-17 20:11:28 +02:00
Nathan Voxland
7bd450f1de Force an analyze on pending index table (#75816)
Force an analyze on pending index before promoting it so metrics are correct and queries fast
2026-06-17 11:27:14 -05:00
Timofey Kachalov
1a9b0c7520 feat(sdk): Add use-action hook for SDK (#75327) 2026-06-17 19:01:33 +04:00
Mihael Konjević
4ff0c08a4b Add @retro slack id mapping (#76009) 2026-06-17 16:25:35 +02:00
Nicola Mometto
c6a4c1bb0e Zombie transform reaper (#75194)
* feat: add transform_run heartbeat migration

* custom message on timeout

* reap stale runs

* test

* heartbeat job runs too

* extract in a run-tracking module

* kondo

* mage

* fix: dissoc internal fields

* fix tests

* chore

* style

* factor out system

* ensure jobs don't hang forever

* use future timeouts

* don't hammer with heartbeats

* refactor

* further handle cases

* simplify impl

* get rid of macro for run tracking

* simplify

* external heartbeat

* cancel in flight

* fix

* move migration to 63

* remove dead kondo config

* add indices for locking run performance

* dissoc tracking run fields on dependencies api

* fix: transform prep needs to be within try/catch

* fix: just use future-cancel

* feat: share heartbeat+reconcile

* feat: add last_heartbeat to job_run

* present run

* cancel in flight on abonrmal exit

* ioc

* simplify

* note

* tighten docstrings

* fix kondo

* fix tests

* nil safe

* whitespace from merge

* handle review comments

* just dissoc last_heartbeat on hydration

* fixes

* use minutes->ms
2026-06-17 15:19:56 +01:00
Mahatthana (Kelvin) Nomsawadi
3012ea0cb4 fix(test): fix flaky embedded dashboard tab view_count assertions (#75889)
fix(test): fix flaky embedded dashboard tab test

The test was asserting view_count values that go through a 20-second
async batch queue, causing near-100% failure in stress runs:

- visitDashboard (beforeEach) fires card queries, submitting increments
  to the batch queue before the embed query runs
- When both increments flush in the same batch cycle, view_count jumps
  from 1 to 3, permanently skipping the expected value of 2
- === 2 is never true in burns 2-30, so retryRequest always times out

Drop the view_count assertions entirely. Backend tests already cover the
increment mechanism (card-read-view-count-test,
embedded-dashboard-card-query-view-log-test,
increment-view-counts!*-test). The intercept spy assertions
(calledOnce / not.have.been.called) fully cover the test's stated
behavior: only cards on the active tab are queried.

Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-17 15:14:26 +01:00
Rodrigo López Dato
80520be19e Expose Mcp-Session-Id header for CORS requests (#75838) 2026-06-17 10:49:58 -03:00
Nemanja Glumac
462255ca5e ci(drivers): Don't let cancelled or timed-out info drivers block PRs (#75992)
ci(drivers): don't let cancelled or timed-out info drivers block PRs

The drivers-tests-result gate passes only when every needed job is
success or skipped. An info driver's job is forced green by the upload
step's continue-on-error, but that only neutralizes a step that runs and
fails normally. When an info job is cancelled (fail-fast, manual cancel)
or times out, the controlling step never completes, so the job's result
is cancelled/failure and the gate fails the required check, blocking PRs.

Feed each driver's configured status into the result job and drop info
drivers from the pass/fail check entirely, so their result never gates
regardless of cancelled/timed-out/failure. Empty or unknown status falls
back to required, so the gate can only stop gating an explicitly-info
driver, never weaken a required one.

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-17 15:48:07 +02:00
dpsutton
e86ae3f02b fix(mysql): don't pass getCatalog() as the sync catalog filter (#75929) (#75971)
Table enumeration broke for PlanetScale databases configured with replica
(or any `<db>@replica`-targeted) credentials starting in 0.62.

Root cause
----------
`active-tables :mysql` was changed to pass `(.getCatalog conn)` as the
catalog filter to `DatabaseMetaData.getTables`. `getCatalog()` is backed by
`SELECT DATABASE()`, which on a Vitess/PlanetScale replica connection returns
the routing-qualified name `<db>@replica`. The driver compiles a non-nil
catalog into `WHERE TABLE_SCHEMA = '<db>@replica'`, but `information_schema`
stores the bare `TABLE_SCHEMA = '<db>'`, so the filter matches nothing and
sync sees an empty database. Primary credentials return the bare name, which
is why the same sync worked on primary but not replica creds.

The `@replica` suffix is a Vitess routing directive, not part of the schema
name, so it can never equal a real `TABLE_SCHEMA`. There is no clean catalog
to recover at runtime: getCatalog/DATABASE()/SCHEMA() are all contaminated,
and setCatalog is a no-op on Vitess.

Fix
---
1. Revert `active-tables :mysql` to pass `nil` for the catalog. The MariaDB
   driver's `nullCatalogMeansCurrent=true` default already scopes a nil
   catalog to the connection's current (bound) database — it does NOT
   enumerate every DB the user can read. This delegates "current database"
   resolution to the driver instead of reconstructing a filter string, and
   correctly returns the bound DB's tables on both primary and replica
   PlanetScale connections.

2. Pin `nullCatalogMeansCurrent=true` explicitly in `default-connection-args`
   so the contract is documented and a future driver bump (Connector/J 8.x
   defaults it to `false`) can't silently widen sync to sibling databases.

The original change aimed to stop workspace MySQL users (GRANT SELECT on the
canonical DB + ownership of an iso DB) from leaking the iso DB into sync. That
leak does not occur with the shipped driver: `nullCatalogMeansCurrent=true`
already scopes `nil` to the bound DB. This is corroborated by the test suite
running many databases on one MySQL server with exact-set table assertions —
a true over-enumeration would have failed those deterministically. Verified
directly against live PlanetScale (replica: nil -> 3 tables, getCatalog -> 0;
primary: 3) and a local multi-DB MySQL server (scoped to the bound DB, no leak).

Why no test
-----------
This regression is not reproducible without Vitess, so no local test has teeth.
The bug only manifests when `getCatalog()` returns a value that differs from the
`information_schema` TABLE_SCHEMA — i.e. PlanetScale's `<db>@replica`. On stock
MySQL `getCatalog()` returns the bare bound-DB name, so any local test would
scope correctly and stay green even if `(.getCatalog conn)` were reintroduced.
A behavioral cross-DB test passes whether or not the bug is present; a unit test
asserting the `nullCatalogMeansCurrent` pin is orthogonal to the catalog
argument and likewise passes with the bug reintroduced. Both would give false
confidence. A faithful test requires a live Vitess/PlanetScale instance, which
isn't available in CI. The inline code comments document the trap, the
mechanism, and the verified reproduction as the durable regression guard.

Here's a repro with the following containers:

```
docker run -d --name mysql-local -e MYSQL_ROOT_PASSWORD=mysecretpassword -e MYSQL_ROOT_HOST='%' -p 3306:3306 mysql:8

docker run -d --name vitess-local \
    -p 33574:33574 \
    -p 33575:33575 \
    -p 33577:33577 \
    -e PORT=33574 \
    -e KEYSPACES=stuff,unsharded \
    -e NUM_SHARDS=2,1 \
    -e MYSQL_BIND_HOST=0.0.0.0 \
    -e VTCOMBO_BIND_HOST=0.0.0.0 \
    vitess/vttestserver:mysql80
```

```clojure
(letfn [(probe [label uri]
          (with-open [conn (jdbc/get-connection {:connection-uri uri})]
            (let [cnt (fn [cat] (with-open [rs (.getTables (.getMetaData conn) cat nil "%"
                                                           (into-array String ["TABLE"]))]
                                  (count (jdbc/result-set-seq rs))))]
              {:label label
               :getCatalog (.getCatalog conn)
               :nil-filter (cnt nil)
               :getCatalog-filter (cnt (.getCatalog conn))})))]
  (mapv (fn [[db conn-s]] (probe db conn-s))
        [["MySQL"           "jdbc:mysql://127.0.0.1:3306/stuff?user=root&password=mysecretpassword"]
         ["Vitess"          "jdbc:mysql://127.0.0.1:33577/stuff?user=root&password="]
         ["Vitess @replica" "jdbc:mysql://127.0.0.1:33577/stuff@replica?user=root&password="]]))
[{:label "MySQL",
  :getCatalog "stuff",
  :nil-filter 3,
  :getCatalog-filter 3}
 {:label "Vitess",
  :getCatalog "stuff",
  :nil-filter 3,
  :getCatalog-filter 3}
 {:label "Vitess @replica",
  :getCatalog "stuff@replica",
  :nil-filter 3,
  :getCatalog-filter 0}]
```
2026-06-17 09:12:06 -04:00
Alexander Polyankin
bc6bc90b7e Fix memory leak with GraalVM (#75982)
* Fix memory leak with GraalVM

* fix
2026-06-17 08:56:07 -04:00
Phoomparin Mano
be2fb45f22 Rename switch for MCP inline charts in admin settings (#75859)
* update copy for mcp apps admin settings

* add aria label

* fix failing type checks
2026-06-17 11:34:22 +00:00
Nemanja Glumac
f3d2ef76b2 Add a pre-flight check to the Claude e2e flake fix (#75991) 2026-06-17 11:42:58 +02:00
Mahatthana (Kelvin) Nomsawadi
1ca82c0c89 feat(sdk-bundle): Modular embedding SDK insights collection (#75785)
* Add public Snowplow proxy endpoint for SDK telemetry (#74967)

* Add public Snowplow proxy endpoint for SDK telemetry

Server half of the CSP-bypass transport (EMB-1758). The Embedding SDK
runs in the customer's page where a strict connect-src CSP blocks a
direct POST to the Snowplow collector; the SDK POSTs the tp2 payload to
the instance instead (already allowlisted) and this endpoint forwards it
server-side, where no browser CSP applies.

Public/anonymous by construction: mounted with +public-exceptions, no
+auth. The browser-tracker can't carry a Metabase session cross-origin,
and the collector it fronts is itself public. Forwards status + body
verbatim so the tracker's retry outbox keeps working; 5s timeouts; 502
on an unreachable collector.

Payload-blind passthrough: no knowledge of the event schema. The
existing embedded_analytics_js schema is used only as a test vehicle;
the SDK's real collection schema is separate (Half 2).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix: remove unused analytics.settings require in proxy_api_test

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* refactor: move proxy endpoint to analytics.api.proxy sub-namespace

- Rename proxy_api → analytics/api/proxy (follows <module>.api.* convention)
- Reference directly from routes.clj (matches metabot pattern)
- Drop inline route comment (docstring covers it)
- Drop log/infof on success path; keep log/errorf on failure
- Drop 502 body (tracker ignores it; server log already captures cause)
- Remove proxy-api from kondo :api set; add analytics.api.proxy instead

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* Clean up AI comments

* fix: clean up proxy test — drop ticket ref, remove unused atom

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* fix: proxy test and impl cleanup

- Add :body nil to 502 response so Ring treats it as a response map
- Overload with-collector macro to support no-atom form
- Relay body in relays-collector-status-verbatim-test
- Drop redundant header case checks (clj-http normalizes to lowercase)

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* fix: use sp.metabase.com instead of "configured collector" in proxy comments

---------

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* Add SDK telemetry transport through the instance proxy (#74968)

* Add public Snowplow proxy endpoint for SDK telemetry

Server half of the CSP-bypass transport (EMB-1758). The Embedding SDK
runs in the customer's page where a strict connect-src CSP blocks a
direct POST to the Snowplow collector; the SDK POSTs the tp2 payload to
the instance instead (already allowlisted) and this endpoint forwards it
server-side, where no browser CSP applies.

Public/anonymous by construction: mounted with +public-exceptions, no
+auth. The browser-tracker can't carry a Metabase session cross-origin,
and the collector it fronts is itself public. Forwards status + body
verbatim so the tracker's retry outbox keeps working; 5s timeouts; 502
on an unreachable collector.

Payload-blind passthrough: no knowledge of the event schema. The
existing embedded_analytics_js schema is used only as a test vehicle;
the SDK's real collection schema is separate (Half 2).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix: remove unused analytics.settings require in proxy_api_test

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* refactor: move proxy endpoint to analytics.api.proxy sub-namespace

- Rename proxy_api → analytics/api/proxy (follows <module>.api.* convention)
- Reference directly from routes.clj (matches metabot pattern)
- Drop inline route comment (docstring covers it)
- Drop log/infof on success path; keep log/errorf on failure
- Drop 502 body (tracker ignores it; server log already captures cause)
- Remove proxy-api from kondo :api set; add analytics.api.proxy instead

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* Clean up AI comments

* fix: clean up proxy test — drop ticket ref, remove unused atom

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* fix: proxy test and impl cleanup

- Add :body nil to 502 response so Ring treats it as a response map
- Overload with-collector macro to support no-atom form
- Relay body in relays-collector-status-verbatim-test
- Drop redundant header case checks (clj-http normalizes to lowercase)

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* fix: use sp.metabase.com instead of "configured collector" in proxy comments

* Add SDK telemetry transport through the instance proxy

Client half of the CSP-bypass transport (EMB-1760). Stands up the
Embedding SDK's Snowplow tracker pointed at the instance proxy
(/api/analytics-proxy, EMB-1758) instead of the collector directly, so
telemetry works inside a strict-CSP customer app — connect-src matches
host not path, so the already-allowlisted instance origin passes.

Transport only, schema-agnostic: initSdkTracker stands up a named "sdk"
tracker (post, credentials omit, keepalive, bufferSize 1, no storage,
server anonymisation); trackSdkEvent sends any self-describing event;
isSdkTrackingEnabled gates on the instance's anonymous-tracking settings.
It does NOT decide what the SDK collects, on which schema, or wire
emission into the component lifecycle — that is the collection work
(EMB-1786) on a new SDK-specific schema. The existing
embedded_analytics_js schema is used only as a temporary test vehicle.

Bumps @snowplow/browser-tracker to v4 (for keepalive + credentials:omit).
Shared dep: pin encodeBase64:true on the main-app tracker to preserve its
exact v3 wire format, and adjust the main-app event.ts data cast for v4's
narrower self-describing-event typing — both no-ops for existing events.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* Note SDK analytics exports unused until EMB-1786

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* Trim SDK tracker config and tests

Remove the trivial isSdkTrackingEnabled helper, drop keepalive and
bufferSize (speculative / web default), make contexts and encodeBase64
explicit, and clarify the credentials comment. Trim the spec to the
config flags that fail silently plus tracker-isolation routing.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix: downgrade @snowplow/browser-tracker v4 → v3.24.6 to fix CI OOM crash

v4.8.1 increased the package weight enough to OOM the CI runner during
e2e-group-28-ee (question-reproductions/reproductions.cy.spec.ts, issue 56416),
crashing Chrome on every run. v3.24.6 is the latest v3 patch and behaves
identically for all current usage (same wire format, same API surface).

Reverts the two v4 accommodations: encodeBase64: true (v4 flipped the POST
default; v3 default is already true) and the Record<string, unknown> type cast
(v4 narrowed SelfDescribingJson; v3 accepts the broader union without a cast).

Also removes credentials: "omit" from the SDK tracker (v4-only option; v3
does not send credentialed cross-origin requests by default).

* fix: remove credentials assertion from snowplow spec — v3 uses XHR not fetch

v3.24.6 uses XMLHttpRequest (not fetch), which has no credentials config.
XHR defaults to withCredentials=false, equivalent to credentials:"omit".
The assertion was added for the v4 fetch-based tracker and doesn't apply to v3.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* fix: upgrade @snowplow/browser-tracker 3.1.6→3.2.0, add withCredentials:false

3.1.6 hardcodes `xhr.withCredentials = true` — no config option. This
causes CORS failures against the proxy endpoint which uses
`Access-Control-Allow-Origin: *` (wildcard CORS rejects any credentialed
request).

3.2.0 added `withCredentials?: boolean` (defaults true). Pinning to
exactly 3.2.0 keeps the bundle at 35 KB — smaller than 3.1.6 (38 KB)
and well below the 3.24.6 bundle (46 KB) that caused the Chrome renderer
OOM in e2e-group-28-ee.

---------

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* Add per-mount SDK component usage events (#75402)

* Add per-mount SDK component usage events

Fire one Snowplow event per public SDK component mount (Option B from
the EMB-1785 tech plan) plus a global-only beacon at provider init,
through the instance proxy transport from EMB-1760.

- New forked schema com.metabase/embedding_sdk/jsonschema/1-0-0
- useTrackSdkComponentMount hook: first-mount-wins dedup, opt-out gate
- All 7 public SDK components wired with their dimension props
- Instance-context plugin + authMethod/sdkVersion globals
- sdkTrackerReady flag fixes child-before-parent init race

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* Add e2e for SDK per-mount component events

Cypress component spec asserting the per-mount event + global init
beacon reach /api/analytics-proxy with the embedding_sdk schema, and
that the opt-out gate (anon-tracking-enabled=false) suppresses both.

Not yet executed locally (needs the SDK component Cypress harness);
verified by CI.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* refactor: rename sdk-component-events → events, object params for initSdkTracker, fix format

- Rename analytics/sdk-component-events.ts → analytics/events.ts (mirrors
  core app's analytics/snowplow.ts + analytics/event.ts split)
- Export EMBEDDING_SDK_SCHEMA from events.ts; remove duplicate definition
  in ComponentProvider.tsx
- Convert initSdkTracker positional params to a named-object signature
- Fix oxfmt formatting in e2e spec and snowplow unit spec

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* feat: CreateDashboardModal tracking, type-safe properties, analytics refactor

- Add CreateDashboardModal to SdkComponentName + iglu schema
- Wire useTrackSdkComponentMount in CreateDashboardModalInner
- Add SdkComponentProperties mapped type for per-component compile-time validation
- Make useTrackSdkComponentMount generic over SdkComponentName
- Move useSdkTrackerInit + deriveAuthMethod to analytics/tracker-init.ts
- Make store a required param in initSdkTracker, remove getSdkStore fallback
- Remove dead null guard in createSdkInstanceContextPlugin, add invariant comment
- Named return value wasJustInitialized in initSdkTracker
- Convert module-scope functions to function keyword
- Fix serialization: value == null catches undefined
- Fix Boolean() casts, improve eslint-disable comment, fix screaming-case comments
- Fix withDownloads default in StaticDashboard to satisfy strict boolean type
- Tests: instance context test, CreateDashboardModal presence test, CreateQuestion delegation e2e test

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* fix: format analytics-component-events spec

* refactor: rename analytics files, type-safe snowplow exports

- events.ts → component-events.ts (more descriptive name)
- tracker-init.ts → tracker.ts, useSdkTrackerInit → useInitSdkTracker
- Export SdkAuthMethod union type from snowplow.ts
- Add WasJustInitialized named type alias for initSdkTracker return
- Simplify initSdkTracker early-return logic (no wasJustInitialized temp var)
- Add comment on createSdkInstanceContextPlugin explaining userId omission
- All consumers updated to new module paths and hook name

* refactor: centralize defaults, drop inapplicable tracked properties

- Add SDK_COMPONENT_DEFAULT_PROPERTIES (mirrors modular embedding's
  DEFAULT_VALUES pattern); hook accepts Partial<> and merges defaults
- Split StaticDashboardProperties (4 fields) from DashboardProperties
  (5 fields); StaticDashboard omits enable_entity_navigation entirely
- Replace PresenceProperties with EmptyProperties = Record<never, never>
  for CollectionBrowser and CreateDashboardModal; drop { used: true }
  (trivially always true, no analytical signal)
- Remove ?? false / ?? "auto" fallbacks from call sites — hook fills
  gaps via SDK_COMPONENT_DEFAULT_PROPERTIES spread merge

* fix: filter undefined before default merge, fix StaticQuestion with_title default

Spread-merge does not distinguish between omitted and undefined: passing
{ with_downloads: undefined } silently overrides the default false.
Filter out undefined values before merging so SDK_COMPONENT_DEFAULT_PROPERTIES
fills genuine gaps, not fields that were explicitly passed as undefined.

Also corrects StaticQuestion.with_title default from true → false to match
the actual component default (title = false, hidden for backwards-compat).

* feat: gate component-mount events to npm SDK only, not iframe embed

Uses isEmbeddingSdk() && !isEmbeddingEajs() — same pattern already used
in ComponentProvider. Iframe embed has its own analytics path; per-mount
component events are intended for npm SDK usage tracking only.

* refactor: drop redundant tracking-only prop defaults in dashboard components

EditableDashboard and InteractiveDashboard destructured withTitle/withDownloads
etc. with defaults solely to pass to useTrackSdkComponentMount. The hook now
handles defaults via SDK_COMPONENT_DEFAULT_PROPERTIES, so the defaults at
the destructuring site are redundant. StaticDashboard keeps its defaults
because withDownloads is also used for the dashcardMenu render.

* fix: filter undefined before default merge, fix StaticQuestion with_title default

Spread-merge does not distinguish between omitted and undefined: passing
{ with_downloads: undefined } silently overrides the default false.
Filter out undefined values before merging so SDK_COMPONENT_DEFAULT_PROPERTIES
fills genuine gaps, not fields that were explicitly passed as undefined.

Also corrects StaticQuestion.with_title default from true → false to match
the actual component default (title = false, hidden for backwards-compat).

* refactor: drop redundant tracking-only prop defaults in StaticDashboard

withTitle and withSubscriptions are only used in the tracking hook call;
withDownloads is also used in dashcardMenu but undefined is falsy so the
= false default has no behavioral effect. All flow through {...normalizedProps}
to SdkDashboard. Hook fills defaults via SDK_COMPONENT_DEFAULT_PROPERTIES.

* refactor: destructure InteractiveQuestion props directly instead of type casts

* refactor: drop component enum from embedding_sdk schema

TypeScript enforces valid component names at compile time via SdkComponentName.
Enum in the schema would require a version bump every time a new component is added.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* fix: pass destructured props through to SdkQuestion in InteractiveQuestion, fix stale test assertion

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* refactor: replace store mock in events spec with real store via renderWithProviders

Use renderWithProviders + sdkReducers + setupSdkState instead of jest.mock on
embedding-sdk-bundle/store. State controls (trackingEnabled, sdkTrackerReady) are now
explicit initial state rather than per-call mock return values.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* fix: resolve TS errors in snowplow.unit.spec

Pass sdk: createMockSdkState() to createMockState so the SdkStoreState
overload is selected instead of State. Replace bare {} with
createMockTokenFeatures() for the token-features setting.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* fix: resolve TS errors in snowplow.unit.spec

Pass sdk: createMockSdkState() to createMockState so the SdkStoreState
overload is selected instead of State. Replace bare {} with
createMockTokenFeatures() for the token-features setting.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* test: add length assertions and multi-component test to analytics spec

- Assert capturedEvents.length in every test so event count is explicit
- Add test: two StaticQuestion mounts fire 3 events (2 component + 1 beacon)
- Fix stale comments: encodeBase64 wording and 3.1.6→3.2.0 upgrade reason

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* fix: correct test 2 length assertion and replace dedup-colliding test 3

Test 2: beacon fires once per JS context (module-level singleton); it
already fired in test 1, so assert length 1 not 2.

Test 3: two StaticQuestion mounts with the same questionId share a dedup
key and only fire one event. Replace with InteractiveQuestion +
CollectionBrowser (distinct keys) and assert length 2.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* test: add iframe embed gate test to events spec

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* test: move iframe embed gate assertion to e2e component spec

Replaces the mocked unit test with a real assertion in the existing
analytics-component-events Cypress spec, where the module singleton is
live and no mocking is needed.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* fix: restrict trackSchemaEvent to main-app tracker to prevent double-firing

In the SDK iframe embed context both the main-app tracker ("sp") and the
SDK tracker ("sdk") are in the Snowplow registry. trackSelfDescribingEvent
with no tracker names fires on all registered trackers, so events like
dashboard_pdf_exported were emitted twice — once via "sp" (browser UA)
and once via the SDK proxy ("sdk", Apache-HttpClient UA).

Passing ["sp"] mirrors the existing pattern in trackSdkEvent which already
restricts to ["sdk"]. In pure SDK contexts where "sp" is never registered
the call becomes a no-op, which is also correct.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* fix: skip SDK tracker init in iframe embed context

The SDK tracker ("sdk") should only initialize when running as the npm
SDK package. In the iframe embed context (isEmbeddingEajs) the tracker
was being registered unnecessarily — its per-mount events are already
blocked by useTrackSdkComponentMount, and the init beacon it was firing
had no test coverage or analytical purpose in that context.

Guarding at the useInitSdkTracker level makes the "sdk" tracker
structurally absent from the Snowplow registry in EAJS, complementing
the ["sp"] restriction in trackSchemaEvent as defense in depth.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* test: move EAJS gate test to iframe embedding e2e suite, fix lint

Removes the EAJS gate test from the SDK component spec (wrong layer —
component tests use a simulated environment, not the real iframe embed
entry point) and adds it to the sdk-iframe-embedding e2e suite where
the full EAJS stack runs.

Also removes unused StaticQuestion import that caused the fe-lint failure.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* fix: default anon-tracking to false until settings load

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* fix: format analytics-component-events spec

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* fix: tighten SdkAuthMethod types and refactor event spec to renderHookWithProviders

- Remove SdkAuthMethod | "" — authMethod is now required in initSdkTracker
  and getSdkAuthMethod returns SdkAuthMethod | undefined (undefined only before
  init, which the sdkTrackerReady gate prevents in practice)
- Update snowplow spec: "returns undefined" replaces "returns empty string"
- Replace TestComponent wrapper in events spec with renderHookWithProviders
  per review comment r3402348453
- Default useIsTrackingEnabled to false so opted-out users are not tracked
  during the settings-load window (review comment r3402504036)

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* refactor: extract getSdkPackageVersion shared helper from getBuildInfo

Replaces 4 inline getBuildInfo("...PACKAGE_BUILD_INFO").version ?? null
calls with a single getSdkPackageVersion() util in get-build-info.ts.
SdkDebugInfo.tsx is left unchanged — it uses the full BuildInfo object.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* fix: update test mocks from getBuildInfo to getSdkPackageVersion

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* fix: wait for both analytics-proxy POSTs in component event e2e tests

Snowplow serializes sends via executingQueue: event 2 is only dispatched
after event 1's XHR response arrives, producing two separate requests.
Waiting for only one request left the count assertion flaky in CI.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* fix: stub analytics proxy to respond immediately in component event tests

The Snowplow queue serializes events: event N+1 fires only after event N's XHR
response arrives. In CI, the real analytics proxy can take 10+ seconds to respond,
which blocks the queue and causes subsequent events to miss the 10s cy.wait timeout.

Replace request.continue() with request.reply({ statusCode: 200 }) so the proxy
stubs return immediately. This also prevents ghost events from failed test retries
from blocking the queue during subsequent tests.

Also adds visual waits (cy.findByText("Product ID").should("be.visible")) before
the proxy waits in tests 1 and 3 as a settings-load race guard, mirroring the
existing pattern in test 4.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* fix: run Snowplow Micro in component test CI and revert proxy stub

Snowplow Micro was not started in component test CI (snowplow: false), so
the backend's analytics proxy hit a connection-refused on localhost:9090 and
returned 502. This blocked the Snowplow queue and caused event wait timeouts.

Enable Snowplow Micro in both component test jobs and revert request.reply()
back to request.continue() so the full proxy chain runs as in production.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-17 09:48:12 +01:00
Mahatthana (Kelvin) Nomsawadi
5db67937ad feat(sdk-bundle): Modular embedding SDK insights collection (#75785)
* Add public Snowplow proxy endpoint for SDK telemetry (#74967)

* Add public Snowplow proxy endpoint for SDK telemetry

Server half of the CSP-bypass transport (EMB-1758). The Embedding SDK
runs in the customer's page where a strict connect-src CSP blocks a
direct POST to the Snowplow collector; the SDK POSTs the tp2 payload to
the instance instead (already allowlisted) and this endpoint forwards it
server-side, where no browser CSP applies.

Public/anonymous by construction: mounted with +public-exceptions, no
+auth. The browser-tracker can't carry a Metabase session cross-origin,
and the collector it fronts is itself public. Forwards status + body
verbatim so the tracker's retry outbox keeps working; 5s timeouts; 502
on an unreachable collector.

Payload-blind passthrough: no knowledge of the event schema. The
existing embedded_analytics_js schema is used only as a test vehicle;
the SDK's real collection schema is separate (Half 2).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix: remove unused analytics.settings require in proxy_api_test

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* refactor: move proxy endpoint to analytics.api.proxy sub-namespace

- Rename proxy_api → analytics/api/proxy (follows <module>.api.* convention)
- Reference directly from routes.clj (matches metabot pattern)
- Drop inline route comment (docstring covers it)
- Drop log/infof on success path; keep log/errorf on failure
- Drop 502 body (tracker ignores it; server log already captures cause)
- Remove proxy-api from kondo :api set; add analytics.api.proxy instead

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* Clean up AI comments

* fix: clean up proxy test — drop ticket ref, remove unused atom

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* fix: proxy test and impl cleanup

- Add :body nil to 502 response so Ring treats it as a response map
- Overload with-collector macro to support no-atom form
- Relay body in relays-collector-status-verbatim-test
- Drop redundant header case checks (clj-http normalizes to lowercase)

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* fix: use sp.metabase.com instead of "configured collector" in proxy comments

---------

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* Add SDK telemetry transport through the instance proxy (#74968)

* Add public Snowplow proxy endpoint for SDK telemetry

Server half of the CSP-bypass transport (EMB-1758). The Embedding SDK
runs in the customer's page where a strict connect-src CSP blocks a
direct POST to the Snowplow collector; the SDK POSTs the tp2 payload to
the instance instead (already allowlisted) and this endpoint forwards it
server-side, where no browser CSP applies.

Public/anonymous by construction: mounted with +public-exceptions, no
+auth. The browser-tracker can't carry a Metabase session cross-origin,
and the collector it fronts is itself public. Forwards status + body
verbatim so the tracker's retry outbox keeps working; 5s timeouts; 502
on an unreachable collector.

Payload-blind passthrough: no knowledge of the event schema. The
existing embedded_analytics_js schema is used only as a test vehicle;
the SDK's real collection schema is separate (Half 2).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix: remove unused analytics.settings require in proxy_api_test

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* refactor: move proxy endpoint to analytics.api.proxy sub-namespace

- Rename proxy_api → analytics/api/proxy (follows <module>.api.* convention)
- Reference directly from routes.clj (matches metabot pattern)
- Drop inline route comment (docstring covers it)
- Drop log/infof on success path; keep log/errorf on failure
- Drop 502 body (tracker ignores it; server log already captures cause)
- Remove proxy-api from kondo :api set; add analytics.api.proxy instead

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* Clean up AI comments

* fix: clean up proxy test — drop ticket ref, remove unused atom

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* fix: proxy test and impl cleanup

- Add :body nil to 502 response so Ring treats it as a response map
- Overload with-collector macro to support no-atom form
- Relay body in relays-collector-status-verbatim-test
- Drop redundant header case checks (clj-http normalizes to lowercase)

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* fix: use sp.metabase.com instead of "configured collector" in proxy comments

* Add SDK telemetry transport through the instance proxy

Client half of the CSP-bypass transport (EMB-1760). Stands up the
Embedding SDK's Snowplow tracker pointed at the instance proxy
(/api/analytics-proxy, EMB-1758) instead of the collector directly, so
telemetry works inside a strict-CSP customer app — connect-src matches
host not path, so the already-allowlisted instance origin passes.

Transport only, schema-agnostic: initSdkTracker stands up a named "sdk"
tracker (post, credentials omit, keepalive, bufferSize 1, no storage,
server anonymisation); trackSdkEvent sends any self-describing event;
isSdkTrackingEnabled gates on the instance's anonymous-tracking settings.
It does NOT decide what the SDK collects, on which schema, or wire
emission into the component lifecycle — that is the collection work
(EMB-1786) on a new SDK-specific schema. The existing
embedded_analytics_js schema is used only as a temporary test vehicle.

Bumps @snowplow/browser-tracker to v4 (for keepalive + credentials:omit).
Shared dep: pin encodeBase64:true on the main-app tracker to preserve its
exact v3 wire format, and adjust the main-app event.ts data cast for v4's
narrower self-describing-event typing — both no-ops for existing events.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* Note SDK analytics exports unused until EMB-1786

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* Trim SDK tracker config and tests

Remove the trivial isSdkTrackingEnabled helper, drop keepalive and
bufferSize (speculative / web default), make contexts and encodeBase64
explicit, and clarify the credentials comment. Trim the spec to the
config flags that fail silently plus tracker-isolation routing.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix: downgrade @snowplow/browser-tracker v4 → v3.24.6 to fix CI OOM crash

v4.8.1 increased the package weight enough to OOM the CI runner during
e2e-group-28-ee (question-reproductions/reproductions.cy.spec.ts, issue 56416),
crashing Chrome on every run. v3.24.6 is the latest v3 patch and behaves
identically for all current usage (same wire format, same API surface).

Reverts the two v4 accommodations: encodeBase64: true (v4 flipped the POST
default; v3 default is already true) and the Record<string, unknown> type cast
(v4 narrowed SelfDescribingJson; v3 accepts the broader union without a cast).

Also removes credentials: "omit" from the SDK tracker (v4-only option; v3
does not send credentialed cross-origin requests by default).

* fix: remove credentials assertion from snowplow spec — v3 uses XHR not fetch

v3.24.6 uses XMLHttpRequest (not fetch), which has no credentials config.
XHR defaults to withCredentials=false, equivalent to credentials:"omit".
The assertion was added for the v4 fetch-based tracker and doesn't apply to v3.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* fix: upgrade @snowplow/browser-tracker 3.1.6→3.2.0, add withCredentials:false

3.1.6 hardcodes `xhr.withCredentials = true` — no config option. This
causes CORS failures against the proxy endpoint which uses
`Access-Control-Allow-Origin: *` (wildcard CORS rejects any credentialed
request).

3.2.0 added `withCredentials?: boolean` (defaults true). Pinning to
exactly 3.2.0 keeps the bundle at 35 KB — smaller than 3.1.6 (38 KB)
and well below the 3.24.6 bundle (46 KB) that caused the Chrome renderer
OOM in e2e-group-28-ee.

---------

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* Add per-mount SDK component usage events (#75402)

* Add per-mount SDK component usage events

Fire one Snowplow event per public SDK component mount (Option B from
the EMB-1785 tech plan) plus a global-only beacon at provider init,
through the instance proxy transport from EMB-1760.

- New forked schema com.metabase/embedding_sdk/jsonschema/1-0-0
- useTrackSdkComponentMount hook: first-mount-wins dedup, opt-out gate
- All 7 public SDK components wired with their dimension props
- Instance-context plugin + authMethod/sdkVersion globals
- sdkTrackerReady flag fixes child-before-parent init race

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* Add e2e for SDK per-mount component events

Cypress component spec asserting the per-mount event + global init
beacon reach /api/analytics-proxy with the embedding_sdk schema, and
that the opt-out gate (anon-tracking-enabled=false) suppresses both.

Not yet executed locally (needs the SDK component Cypress harness);
verified by CI.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* refactor: rename sdk-component-events → events, object params for initSdkTracker, fix format

- Rename analytics/sdk-component-events.ts → analytics/events.ts (mirrors
  core app's analytics/snowplow.ts + analytics/event.ts split)
- Export EMBEDDING_SDK_SCHEMA from events.ts; remove duplicate definition
  in ComponentProvider.tsx
- Convert initSdkTracker positional params to a named-object signature
- Fix oxfmt formatting in e2e spec and snowplow unit spec

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* feat: CreateDashboardModal tracking, type-safe properties, analytics refactor

- Add CreateDashboardModal to SdkComponentName + iglu schema
- Wire useTrackSdkComponentMount in CreateDashboardModalInner
- Add SdkComponentProperties mapped type for per-component compile-time validation
- Make useTrackSdkComponentMount generic over SdkComponentName
- Move useSdkTrackerInit + deriveAuthMethod to analytics/tracker-init.ts
- Make store a required param in initSdkTracker, remove getSdkStore fallback
- Remove dead null guard in createSdkInstanceContextPlugin, add invariant comment
- Named return value wasJustInitialized in initSdkTracker
- Convert module-scope functions to function keyword
- Fix serialization: value == null catches undefined
- Fix Boolean() casts, improve eslint-disable comment, fix screaming-case comments
- Fix withDownloads default in StaticDashboard to satisfy strict boolean type
- Tests: instance context test, CreateDashboardModal presence test, CreateQuestion delegation e2e test

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* fix: format analytics-component-events spec

* refactor: rename analytics files, type-safe snowplow exports

- events.ts → component-events.ts (more descriptive name)
- tracker-init.ts → tracker.ts, useSdkTrackerInit → useInitSdkTracker
- Export SdkAuthMethod union type from snowplow.ts
- Add WasJustInitialized named type alias for initSdkTracker return
- Simplify initSdkTracker early-return logic (no wasJustInitialized temp var)
- Add comment on createSdkInstanceContextPlugin explaining userId omission
- All consumers updated to new module paths and hook name

* refactor: centralize defaults, drop inapplicable tracked properties

- Add SDK_COMPONENT_DEFAULT_PROPERTIES (mirrors modular embedding's
  DEFAULT_VALUES pattern); hook accepts Partial<> and merges defaults
- Split StaticDashboardProperties (4 fields) from DashboardProperties
  (5 fields); StaticDashboard omits enable_entity_navigation entirely
- Replace PresenceProperties with EmptyProperties = Record<never, never>
  for CollectionBrowser and CreateDashboardModal; drop { used: true }
  (trivially always true, no analytical signal)
- Remove ?? false / ?? "auto" fallbacks from call sites — hook fills
  gaps via SDK_COMPONENT_DEFAULT_PROPERTIES spread merge

* fix: filter undefined before default merge, fix StaticQuestion with_title default

Spread-merge does not distinguish between omitted and undefined: passing
{ with_downloads: undefined } silently overrides the default false.
Filter out undefined values before merging so SDK_COMPONENT_DEFAULT_PROPERTIES
fills genuine gaps, not fields that were explicitly passed as undefined.

Also corrects StaticQuestion.with_title default from true → false to match
the actual component default (title = false, hidden for backwards-compat).

* feat: gate component-mount events to npm SDK only, not iframe embed

Uses isEmbeddingSdk() && !isEmbeddingEajs() — same pattern already used
in ComponentProvider. Iframe embed has its own analytics path; per-mount
component events are intended for npm SDK usage tracking only.

* refactor: drop redundant tracking-only prop defaults in dashboard components

EditableDashboard and InteractiveDashboard destructured withTitle/withDownloads
etc. with defaults solely to pass to useTrackSdkComponentMount. The hook now
handles defaults via SDK_COMPONENT_DEFAULT_PROPERTIES, so the defaults at
the destructuring site are redundant. StaticDashboard keeps its defaults
because withDownloads is also used for the dashcardMenu render.

* fix: filter undefined before default merge, fix StaticQuestion with_title default

Spread-merge does not distinguish between omitted and undefined: passing
{ with_downloads: undefined } silently overrides the default false.
Filter out undefined values before merging so SDK_COMPONENT_DEFAULT_PROPERTIES
fills genuine gaps, not fields that were explicitly passed as undefined.

Also corrects StaticQuestion.with_title default from true → false to match
the actual component default (title = false, hidden for backwards-compat).

* refactor: drop redundant tracking-only prop defaults in StaticDashboard

withTitle and withSubscriptions are only used in the tracking hook call;
withDownloads is also used in dashcardMenu but undefined is falsy so the
= false default has no behavioral effect. All flow through {...normalizedProps}
to SdkDashboard. Hook fills defaults via SDK_COMPONENT_DEFAULT_PROPERTIES.

* refactor: destructure InteractiveQuestion props directly instead of type casts

* refactor: drop component enum from embedding_sdk schema

TypeScript enforces valid component names at compile time via SdkComponentName.
Enum in the schema would require a version bump every time a new component is added.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* fix: pass destructured props through to SdkQuestion in InteractiveQuestion, fix stale test assertion

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* refactor: replace store mock in events spec with real store via renderWithProviders

Use renderWithProviders + sdkReducers + setupSdkState instead of jest.mock on
embedding-sdk-bundle/store. State controls (trackingEnabled, sdkTrackerReady) are now
explicit initial state rather than per-call mock return values.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* fix: resolve TS errors in snowplow.unit.spec

Pass sdk: createMockSdkState() to createMockState so the SdkStoreState
overload is selected instead of State. Replace bare {} with
createMockTokenFeatures() for the token-features setting.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* fix: resolve TS errors in snowplow.unit.spec

Pass sdk: createMockSdkState() to createMockState so the SdkStoreState
overload is selected instead of State. Replace bare {} with
createMockTokenFeatures() for the token-features setting.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* test: add length assertions and multi-component test to analytics spec

- Assert capturedEvents.length in every test so event count is explicit
- Add test: two StaticQuestion mounts fire 3 events (2 component + 1 beacon)
- Fix stale comments: encodeBase64 wording and 3.1.6→3.2.0 upgrade reason

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* fix: correct test 2 length assertion and replace dedup-colliding test 3

Test 2: beacon fires once per JS context (module-level singleton); it
already fired in test 1, so assert length 1 not 2.

Test 3: two StaticQuestion mounts with the same questionId share a dedup
key and only fire one event. Replace with InteractiveQuestion +
CollectionBrowser (distinct keys) and assert length 2.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* test: add iframe embed gate test to events spec

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* test: move iframe embed gate assertion to e2e component spec

Replaces the mocked unit test with a real assertion in the existing
analytics-component-events Cypress spec, where the module singleton is
live and no mocking is needed.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* fix: restrict trackSchemaEvent to main-app tracker to prevent double-firing

In the SDK iframe embed context both the main-app tracker ("sp") and the
SDK tracker ("sdk") are in the Snowplow registry. trackSelfDescribingEvent
with no tracker names fires on all registered trackers, so events like
dashboard_pdf_exported were emitted twice — once via "sp" (browser UA)
and once via the SDK proxy ("sdk", Apache-HttpClient UA).

Passing ["sp"] mirrors the existing pattern in trackSdkEvent which already
restricts to ["sdk"]. In pure SDK contexts where "sp" is never registered
the call becomes a no-op, which is also correct.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* fix: skip SDK tracker init in iframe embed context

The SDK tracker ("sdk") should only initialize when running as the npm
SDK package. In the iframe embed context (isEmbeddingEajs) the tracker
was being registered unnecessarily — its per-mount events are already
blocked by useTrackSdkComponentMount, and the init beacon it was firing
had no test coverage or analytical purpose in that context.

Guarding at the useInitSdkTracker level makes the "sdk" tracker
structurally absent from the Snowplow registry in EAJS, complementing
the ["sp"] restriction in trackSchemaEvent as defense in depth.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* test: move EAJS gate test to iframe embedding e2e suite, fix lint

Removes the EAJS gate test from the SDK component spec (wrong layer —
component tests use a simulated environment, not the real iframe embed
entry point) and adds it to the sdk-iframe-embedding e2e suite where
the full EAJS stack runs.

Also removes unused StaticQuestion import that caused the fe-lint failure.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* fix: default anon-tracking to false until settings load

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* fix: format analytics-component-events spec

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* fix: tighten SdkAuthMethod types and refactor event spec to renderHookWithProviders

- Remove SdkAuthMethod | "" — authMethod is now required in initSdkTracker
  and getSdkAuthMethod returns SdkAuthMethod | undefined (undefined only before
  init, which the sdkTrackerReady gate prevents in practice)
- Update snowplow spec: "returns undefined" replaces "returns empty string"
- Replace TestComponent wrapper in events spec with renderHookWithProviders
  per review comment r3402348453
- Default useIsTrackingEnabled to false so opted-out users are not tracked
  during the settings-load window (review comment r3402504036)

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* refactor: extract getSdkPackageVersion shared helper from getBuildInfo

Replaces 4 inline getBuildInfo("...PACKAGE_BUILD_INFO").version ?? null
calls with a single getSdkPackageVersion() util in get-build-info.ts.
SdkDebugInfo.tsx is left unchanged — it uses the full BuildInfo object.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* fix: update test mocks from getBuildInfo to getSdkPackageVersion

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* fix: wait for both analytics-proxy POSTs in component event e2e tests

Snowplow serializes sends via executingQueue: event 2 is only dispatched
after event 1's XHR response arrives, producing two separate requests.
Waiting for only one request left the count assertion flaky in CI.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* fix: stub analytics proxy to respond immediately in component event tests

The Snowplow queue serializes events: event N+1 fires only after event N's XHR
response arrives. In CI, the real analytics proxy can take 10+ seconds to respond,
which blocks the queue and causes subsequent events to miss the 10s cy.wait timeout.

Replace request.continue() with request.reply({ statusCode: 200 }) so the proxy
stubs return immediately. This also prevents ghost events from failed test retries
from blocking the queue during subsequent tests.

Also adds visual waits (cy.findByText("Product ID").should("be.visible")) before
the proxy waits in tests 1 and 3 as a settings-load race guard, mirroring the
existing pattern in test 4.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* fix: run Snowplow Micro in component test CI and revert proxy stub

Snowplow Micro was not started in component test CI (snowplow: false), so
the backend's analytics proxy hit a connection-refused on localhost:9090 and
returned 502. This blocked the Snowplow queue and caused event wait timeouts.

Enable Snowplow Micro in both component test jobs and revert request.reply()
back to request.continue() so the full proxy chain runs as in production.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-17 09:47:57 +01:00
Alexander Polyankin
9c204243f0 Fix dependencies cycles handling (#75978) 2026-06-17 01:16:57 +00:00
Denis Berezin
cb574d0ac1 UXW-4342 Fix document crash after removing a chart (#75641)
* UXW-4342 Fix document crash after removing a chart

* Fix typecheck

* Review fixes

* Fix unit tests
2026-06-17 03:39:36 +03:00
Eric Normand
7b274b46ac Don't delete local transforms when remote has none (GHY-3900) (#75898)
Enabling remote-sync-transforms triggered an import; when the remote
contained no transforms, the imported entity-id set was empty, so
remove-unsynced! built no WHERE clause and fell through to a bare
full-table (t2/delete! :model/Transform), wiping locally-created
transforms.

Gate the unconditional full-table delete so it never fires for
all-or-nothing models (those marked :all-on-setting-disable). An empty
import set means the feature was just toggled on, or the remote tracks
none yet -- not that every local entity should be destroyed.
2026-06-16 22:52:21 +00:00
Kevin Kelleher
449527b9fa Handle null x-axis values in time series charts (#75962)
* properly sort nulls for timeseries axis

* Update Loki Snapshots

---------

Co-authored-by: Metabase Automation <github-automation@metabase.com>
2026-06-16 22:02:54 +00:00
Alexander Polyankin
32c871a46e Introduce multi-process stale-while-revalidate behavior with query caching (#75949)
* Cache empty query results

* Introduce stale-while-revalidate behavior with query caching

* fix author

* fix

* fix

* fix

* fix

* fix

* fix migrations

* fix

* fix

* review comments

* fix

* fix mysql

* fix

* fix
2026-06-16 16:38:39 -04:00
Tyler Plude
eca496d66e [BOT-1662] Support for Microsoft Azure BYOK (#75768)
* Azure BYOK support (squashed BOT-1662)

* cleanups

* fix test

* pr feedback
2026-06-16 13:29:12 -07:00
metamben
b0e21b07d6 Set row limit based on last aggregation or join (#75899)
* Set row limit based on last aggregation or join

Closes QUE2-190

* Fix tests
2026-06-16 22:58:22 +03:00
metamben
5a71626691 Take the metric's name into account when replacing references (#75910)
Closes QUE2-159
2026-06-16 22:58:11 +03:00
metamben
a5287f0238 Add test showing that bucketed breakouts truncate properly (#75811)
* Add test showing that bucketed breakouts truncate properly
* Make mongo's ->rvalue for expressions honor temporal bucketing

Closes QUE2-234
2026-06-16 19:57:31 +00:00
Alexander Polyankin
e025170e85 Fix lib-related N+1 issues with DB calls in parameters (#75888)
* remove duplications

* dedup

* fix

* add tests

* fix

* bulk load

* fix

* fixes

* tests

* fixes

* fix

* fixes

* fix

* fix

* fixes

* fix

* fix /values

* fix

* fix

* fix index usage
2026-06-16 15:03:33 -04:00
Nick Fitzpatrick
656540b069 Oauth client event mark ran v63 (#75950)
* Guard v63 oauth_client_event migration against v62 backport

The oauth_client_event migration was backported to v62. Without these
preconditions, a v62 -> v63 upgrade re-runs CREATE TABLE and fails because
the table already exists. Add MARK_RAN preconditions keyed on the paired
v62 changesets, plus empty rollbacks so a v63 -> v62 downgrade keeps the
objects the still-recorded v62 changesets own. Mirrors #72651.

* Add guarded v62 oauth_client_event backport to master

Master must carry the v62 backport file (identical to release-x.62.x) so
that on a fresh v63 install the 062 changeset sorts first and owns the
table, and v63.* mark-ran. Without it, a fresh-v63 -> downgrade-to-v62
path fails: migrate-down removes the v63 changelog row but the empty
rollback keeps the table, so the v62 jar then re-runs CREATE TABLE on an
existing table. Both files coexisting matches #72651's 060/061 layout.

* Adding comment to rollback props
2026-06-17 00:32:10 +05:30
Riley Thompson
1846972a9e Allow single non-select statements for admins on impersonated DBs (#75902) 2026-06-16 21:00:58 +02:00
Riley Thompson
ba3ba7cdc0 Prefer adding field to join with same join alias (#75917) 2026-06-16 20:55:40 +02:00
Copilot
ba5883cd44 Add CFP Franc (XPF) to currency list (#75906)
Add CFP Franc (XPF) to currency list (fixes #42129)

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
2026-06-16 20:42:34 +02:00
technomancy
95f1ed9b8d Disable multithreading for tests under snowflake and bigquery. (#75948) 2026-06-16 11:14:07 -07:00
Alexander Polyankin
6bc02506d7 Cache empty query results (#75946) 2026-06-16 11:08:28 -07:00
Oreon Lothamer
890a48bc56 PR-Env-Stats-Dump creates PR env with stats db (#75367)
* PR-Env-Stats-Dump creates PR env with stats db
2026-06-16 07:51:20 -10:00
Timofey Kachalov
ffe60b4f05 Do not show the sdk-disabled usage problem on localhost (#75930) 2026-06-16 20:34:13 +04:00
appleby
3c173a9fcc Fix translation of aisdk error parts (#75913)
* Handle response.failed in openai->aisdk-chunks-xf

Bedrock was sending this due to server error for GPT models in certain regions.

* Handle response.incomplete in openai->aisdk-chunks-xf

* Ensure :error parts are correctly translated in aisdk-chunks->part

Providers return aisdk v5 format :errorText, anything consuming internal-format parts expects :error instead. Make
sure to do the translation in aisdk-chunks->part. This was causing us to drop streaming errors and fail to report them
in the UI or persist them to MetabotMessage, so they never got surfaced in the AI usage analytics Conversations view.

* Log LLM provider in-stream errors in report-aisdk-errors-xf
2026-06-16 09:25:51 -07:00
Andrei Chernykh
3d290b8530 Convert QueryVisualization components to TypeScript (#75684) 2026-06-16 09:21:22 -07:00
github-automation-metabase
195fff76b2 Auto-update documentation for master (#75907)
Update generated docs (master)

Co-authored-by: github-actions <github-actions@github.com>
2026-06-16 12:19:07 -04:00
metamben
2cdb98f44a Add missing dimensions in underlying-records drill-thru (#75048) 2026-06-16 18:55:25 +03:00
Riley Thompson
ca7f3468ab Use desired column alias when merging model metadata (#75884)
* use desired column alias

* remove legacy conversion in test

* fix kondo
2026-06-16 11:00:55 -04:00
Eric Normand
6cc41b5ad9 Surface YAML parse reason in remote-sync ingestion errors (#75896)
* Surface YAML parse reason in remote-sync ingestion errors

When a pull/ingestion failed on an unreadable or unparseable file, the
error only named the file ("Failed to read N file(s) during ingestion:
<path>"), giving no clue why. The underlying cause was preserved in the
exception chain but never rendered.

- ingestable.clj: read- and parse-failure ex-infos now carry a concise,
  machine-readable :reason in ex-data. For YAML errors the reason is the
  SnakeYAML problem plus its 1-based line/column (its message's first line
  is a generic context wrapper, so first-line alone is unhelpful).
- impl.clj source-error-message: render one line per failed file with its
  :reason, so the sync-error modal explains the failure.

* Render ingestion error as a single line

The modal shows the error in a plain <Text> that collapses newlines, so
the per-file bullet list rendered as a run-on line. Join files with '; '
on one line instead. The machine-readable :file/:reason stay in ex-data
for a structured display later.
2026-06-16 09:49:46 -05:00
Nick Fitzpatrick
89ec6adf08 Maps support static viz (#75387)
* Maps static viz

* adjustments

* linting error

* CI failures

* Sasha Feedback. Removing some duplication

* BE cleanup

* first pass

* second pass

* eastwood linter

* review bot - first pass

* bot review pass 2

* PR feedback
2026-06-16 20:18:56 +05:30
Nick Fitzpatrick
f8822cdd9c Object Detail static viz support (#75386)
* Object Detail static viz support

* adjustments

* braden-bot review
2026-06-16 18:20:01 +05:30
Nick Fitzpatrick
90d86d090c add documents to search on link dashcards (#75892) 2026-06-16 16:56:50 +05:30
Romeo Van Snick
c3590b13b0 Move DashboardApi to RTK hooks and update all callers (#75677)
* Add dashboard parameter-values RTK endpoints

Add getDashboardParameterValues and searchDashboardParameterValues query
endpoints to dashboardApi, mirroring the existing cardApi equivalents, and
forward dashboard_load_id through the getDashboard query. Introduces the
DashboardParameterValuesRequest and SearchDashboardParameterValuesRequest
types in preparation for replacing the legacy DashboardApi client.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* Migrate DashboardApi callers to RTK and remove it

Replace the legacy DashboardApi client with the RTK dashboardApi endpoints:
get -> getDashboard, parameterValues/parameterSearch -> the new dashboard
parameter-values queries. Imperative thunk callers dispatch the endpoints
via runRtkEndpoint / initiate, matching the existing cardApi patterns. Drops
the obsolete DashboardApi jest mocks and removes DashboardApi from services.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* Avoid mutating frozen RTK Query dashboard result in fetchDashboard

The `else` branch of `fetchDashboard` now reads the dashboard through the
`getDashboard` RTK Query endpoint, whose responses are deeply frozen. The
virtual-card merge mutated the dashcards in place, throwing "Cannot assign to
read only property" and breaking dashboard rendering. Rebuild the dashcards
immutably instead.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* Wire abort signal through runRtkEndpoint in fetchDashboard

`runRtkEndpoint` now accepts an optional `signal` that it wires to RTK Query's
`.abort()`, so imperative callers can cancel in-flight requests. Pass
`fetchDashboardCancellation.signal` to the `getDashboard` call so starting a
new dashboard fetch cancels the previous one again, as it did before the
migration off `DashboardApi.get`.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* Clean up RTK Query subscriptions in error-page entity fetch

`getEntityDetails` dispatched `initiate(...).unwrap()` for cards and dashboards
without ever unsubscribing, leaving long-lived subscriptions in the store.
Route both through `runRtkEndpoint`, which unsubscribes in a `finally`.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-16 21:12:21 +10:00
Kamil Mielnik
ad68687351 GDGT-2631 Fix name input border cut off on hover (#75923) 2026-06-16 11:06:28 +00:00
Uladzimir Havenchyk
1e99cb10aa chore: replace easy deprecated Button call sites with metabase/ui Button (#75790)
* chore: replace easy deprecated Button call sites with metabase/ui Button

* chore: simplify migrated button call sites

- Scope EditBar button styling to ButtonsContainer so it can't leak into
  the title/center slots, and raise its specificity above the metabase/ui
  variant hover rules; collapse the subtle/filled selector pair to
  button[data-variant]
- DatabaseSectionField: use the size="compact-sm" prop pattern of its
  sibling section fields and drop its CSS module
- DisconnectButton: size via the ActionIcon size prop instead of CSS
  dimension resets; drop declarations the subtle variant already provides
- Replace single-rule filterButton class with m="sm"; remove no-op CSS
  (createTabButton padding, SnippetButton brand color)
- Dedupe the repeated inline action types in the card reducer

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* chore: migrate ActionButton to metabase/ui Button

ActionButton now renders the Mantine Button; the success/failed states
use variant="filled" with the success/error theme colors instead of the
legacy ButtonsS classes. Consumer flags map 1:1 (primary -> filled,
danger -> filled error, success borderless -> filled success), and the
edit-bar save buttons use size="sm" to match their migrated cancel
buttons. With no legacy buttons left inside EditBar, its ButtonsS
selectors are gone too.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* fix: keep "Show error details" as an inline text link after Button migration

The deprecated onlyText Button rendered as an inline text link (~90x13);
variant="subtle" alone kept Mantine's default md box (146x40), enlarging
the control and shifting the error-modal layout. Restore the inline look
with the size="xs" h="1rem" p={0} idiom.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-06-16 13:39:14 +03:00
Nicola Mometto
00ec2a1dea Generate athena pom on publish (#75925) 2026-06-16 12:35:04 +02:00
Nemanja Glumac
c8350535e7 Design the autonomous e2e flake fixer (#75909)
Replace the human push-approval gate with a branch-existence dedup check:
the branch name is derived solely from the Linear ref
(automated-e2e-flake-fix-<TEAM_SLUG>-<ISSUE_NUMBER>), and an existing remote
branch means a run is already in flight — otherwise it's a go, no approval.
The command stays identity-agnostic, so a human locally and an automated
trigger run the same path.

Also: prefer Linear MCP with a REST/GraphQL fallback for headless runs,
drop the poll cadence to 15 minutes, and lead the Linear audit-trail
comment with total time spent and tokens used.

* Add claude-e2e-flake-fix workflow for autonomous e2e flake fixing

Runs /fix-flaky-test headlessly as the Metabase Automation user, triggered
by workflow_dispatch or repository_dispatch (e.g. from ci-conductor) with a
Linear issue ref. ci-conductor only dispatches; the agent runs here, in CI.

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-16 09:56:33 +02:00
appleby
9f71763026 AWS Bedrock support (#75685)
* openai s/:max_tokens/:max_output_tokens/

Prevents errors like the following when using openai provider

  Provider API request failed: provider=openai status=400 body={:error {:message "Unknown parameter: 'max_tokens'.", :type "invalid_request_error", :param "max_tokens", :code "unknown_parameter"}}

* openai: do not attempt to set temperature on models that do not support it

for example, gpt-5 and the o-series models

* openai: ignore reasoning chunks in openai->aisdk-chunks-xf

* bedrock: raise max-tokens for suggested prompts generator

* BOT-1663 Metabot support for AWS Bedrock provider

* Drop custom SigV4 implementation in bedrock provider and use AwsV4HttpSigner instead

* vibed admin UI changes for configuring bedrock provider and creds

* Validate bedrock creds before save and unify creds handling with api-key providers

- fix metabot.settings/configured-provider-api-key to return a creds map not api key
- modify provider-models-response and list-models to take a creds map rather than api key
- update various other places that expected api-key only
- ensure bedrock provider supports "Disconnect" button in admin UI
- drop Fable models for now

* admin UI change to support bedrock disconnect via :credentials nil

* BE: rename api-key errors -> credentials

* FE: rename api-key errors -> credentials

* Reject ai-proxy requests early in bedrock provider

Supporting this would require some work on the ai-service proxy side, so reject it with a clear error message for now,
but also update the bedrock provider to use core/resolve-auth and core/request so that it's easy to add support in the
future if we decide to.

* feat: admin ui for amazon bedrock (#75805)

* Fix docstrings

* Fix race in extraction-metrics-test

* review feedback: fix clearing of session token and region

* Ensure that a Disconnect in the admin UI clears the AWS region as well

---------

Co-authored-by: Maksym Yakubych <undrfined@gmail.com>
2026-06-15 16:03:20 -07:00
metamben
88fd6b6d90 Add test showing that day-of-week is not a temporal on the next stage (#75821)
Closes QUE2-249
2026-06-15 22:55:37 +00:00
Kevin Kelleher
f9aa7966d3 Fix axis labels being cut off when rotated 45 degrees (#75815)
* fix axis labels being cutoff when rotated 45 degrees

* add png, split stories into files

* try to fix default export error, add comment
2026-06-15 17:42:30 -04:00
Nemanja Glumac
4c586296a9 Pass build_jar to stress-test workflow in /fix-flaky-test (#75891)
The e2e-stress-test-flake-fix workflow gained a build_jar input that
builds the uberjar from the branch instead of downloading a prebuilt
ancestor JAR. Teach the command to set it: build whenever any commit on
the branch touches source (backend or frontend), keeping it on for later
test-only attempts; skip the build only when every commit is test-only.

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-15 23:39:01 +02:00
mendral-app[bot]
08e62e754d Use env var indirection in team-issues-slack-notification (#70291)
Co-authored-by: mendral-app[bot] <233154221+mendral-app[bot]@users.noreply.github.com>
2026-06-15 23:34:45 +02:00
da
f3d28fd439 Update row and column security information for databases (#75885)
* Update row and column security information for databases

Clarified that both MongoDB and Druid support only row-level security and do not support column-level security.

* Update docs/permissions/row-and-column-security.md

Co-authored-by: Jeff Bruemmer <jeff.bruemmer@gmail.com>

---------

Co-authored-by: Jeff Bruemmer <jeff.bruemmer@gmail.com>
2026-06-15 18:16:38 -03:00
thackervishal
d7a5482a8a DOC-fix bug in keycloak page + minor improvements (#75813) 2026-06-15 16:51:03 -04:00
Stas Gavrylov
0f5bed6ad7 GDGT-2579: Replace deprecated EntityMenu internals with Mantine Menu (#75637)
* Refactor EntityMenu to use Menu inside

* GDGT-2579: Remove unused EntityMenu paths. Cleanup

* GDGT-2579: Use Menu directly instead of EntityMenu

* GDGT-2579: Cleanup following comments

* wip

* Fix detail view actions menu hotkeys

* GDGT-2579: Remove legacy wrappers

* GDGT-2579: Cleanup

* GDGT-2579: Get rid of EntityMenuTrigger component

* GDGT-2579: Cleanup

* GDGT-2579: Fix tests

* GDGT-2579: Address review comments

* GDGT-2579: Apply PR comments

* GDGT-2579: Update 'disabled' handling logic
2026-06-15 21:45:42 +01:00
Andrei Chernykh
e49ba02621 Hide public link options from non-admins and when public sharing is disabled (#75681)
* Hide public link options from non-admins and when public sharing is disabled

* Update public link copy tooltip

* Redesign sharing menus with Copy link and Embed header actions for admins and a public link copy for non-admins

* Hoist repeated card action types in the QB reducer to fix a TS2589 instantiation-depth error

* Make sharing menus regular menus with an always-available app link copy and a separate public link copy for non-admins

* Clean up sharing menu leftovers: dead props, unused testid, repeated test setup, and explicit resource branches in e2e

* Update stale Loki reference for the dark-theme embed download story

* Revert "Update stale Loki reference for the dark-theme embed download story"

This reverts commit a3b9c8f70c.

* Update permissions e2e for the always-present non-admin copy button

* Drop unused onCopied prop from CopyLinkMenuItem; use createMockDashboardTab in its spec
2026-06-15 12:32:19 -07:00
metabase-bot[bot]
e5d0ac1636 [chore] Update E2E test timings with recent data (#75840)
Update E2E test timings with averaged data

Collected timing data from 7 days of test runs.
All specs with timing data were included (simple averages).

🤖 Generated automatically by update-e2e-timings workflow

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: GitHub Actions <actions@github.com>
2026-06-15 21:09:48 +02:00
technomancy
0a858152c1 Fix a very foolish mistake around percentages for health inspector. (#75890)
Add a test that catches this.
2026-06-15 18:36:59 +00:00
Nemanja Glumac
f1fd89c939 Allow info level driver runs (#75648)
* mage: read driver skip/info status from ci-test-config

Replace the legacy `ignored.drivers` quarantine list with the new top-level
`drivers` array in ci-test-config.json, where each entry carries a status:

  skip      -- do not run the driver (quarantined; break-quarantine-<driver> overrides)
  info      -- run the driver, but its result must not gate (data collection only)
  required  -- not listed: runs and gates as usual (the default)



Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-15 20:28:37 +02:00
Nick Fitzpatrick
dd1ec20321 Add support for pivot table static viz (#75337)
* Add support for pivot table static viz

* review feedback

* Move pivot postprocess into the pivot module

* deduplicate formatters

* formatting

* fix CI
2026-06-15 23:40:56 +05:30
technomancy
b0da250e69 Temporarily remove belize test and create-replace-table test in snowflake (#75830)
Temporarily comment out problematic tests.

These tests are quarantined, so they don't affect pass/fail
status. However, by examining CI logs, it appears that it's very
common for seemingly-unrelated failures to happen immediately
following one of these failures. Let's see if this still happens when
they don't run at all!
2026-06-15 11:03:51 -07:00
Andrei Chernykh
c1c9bb4f1c Convert SnippetSidebar to TypeScript (#75659)
* Convert SnippetSidebar to TypeScript

* Replace ComponentType<any> in snippet sidebar row renderers with a typed plugin contract
2026-06-15 09:02:50 -07:00
Alex Yarosh
a3176eb0ca docs: erd (#75647)
* erd

* Optimized 1 images, saved 134.0 KB (43.4% smaller)

* link sprinkling

* Apply suggestions from code review

Co-authored-by: Jeff Bruemmer <jeff.bruemmer@gmail.com>

* Apply suggestions from code review

Co-authored-by: Michael Eby <38187276+michaeleby1@users.noreply.github.com>

---------

Co-authored-by: Metabase Automation <github-automation@metabase.com>
Co-authored-by: Jeff Bruemmer <jeff.bruemmer@gmail.com>
Co-authored-by: Michael Eby <38187276+michaeleby1@users.noreply.github.com>
2026-06-15 11:12:03 -04:00
github-automation-metabase
3fbfb979e8 fix(test): resolve flaky TablePicker visibility layer test (#75721)
* fix(test): resolve flaky TablePicker visibility layer test

Consolidate all virtualizer-dependent assertions into a single waitFor
block with a 5000ms timeout. The virtualizer's async layout cycle
(containerRef → measureElement → render rows) can exceed the default
1000ms timeout under CI load, causing intermittent failures.

Fixes UXW-4469

* fix: use single assertion in waitFor to satisfy eslint rule

* Properly mocking ResizeObserver

* Don't click the table row

---------

Co-authored-by: Nick Fitzpatrick <nickfitz.582@gmail.com>
2026-06-15 20:41:33 +05:30
Alex Yarosh
33d8adc109 docs: collections in the library (#75448)
* collections in the library

* Optimized 3 images, saved 123.3 KB (51.4% smaller)

* Apply suggestions from code review

Co-authored-by: Jeff Bruemmer <jeff.bruemmer@gmail.com>

* edits from jeff

---------

Co-authored-by: Metabase Automation <github-automation@metabase.com>
Co-authored-by: Jeff Bruemmer <jeff.bruemmer@gmail.com>
2026-06-15 11:10:05 -04:00
Riley Thompson
5af51e630d Check custom aggregations don't contain unaggregated refs (#75797)
* add test to prove fixed

* check for unaggregated refs

* fix typo and test
2026-06-15 14:36:44 +00:00
Egor Grushin
f44a41ead4 GDGT-2615 Replace deprecated TippyPopover component (#75668)
* GDGT-2615 fix PopoverWithRef

* GDGT-2615 migration step 2. (checks 1–6 are passed)

* GDGT-2615 migration step 3 put comments

* GDGT-2615 fix comments

* GDGT-2615 fix test

* GDGT-2615 fix test

* GDGT-2615 fix test

* GDGT-2615 fix test

* GDGT-2615 fix test

* GDGT-2615 fix test

* GDGT-2615 fix test

* GDGT-2615 move tippy-leftovers to tooltip folder

* GDGT-2615 fix import import "metabase/common/components/Tooltip/tippy.module.css";

* GDGT-2615 fix SDK tests

* GDGT-2615 address the PR's comments

* GDGT-2615 fix test

* GDGT-2615 fix a regression

* GDGT-2615 fix a regression

* GDGT-2615 addressed the review's comments

* GDGT-2615 addressed the review's comments
2026-06-15 13:46:50 +01:00
Oleh K
348c513d47 Restyle Chip component according to Figma spec (#74769)
* Rework Chip story and add reusable Storybook showcase helpers

Turn the Chip story into an overview showcase (sizes section plus a column
per variant with all interactive states, hover/pressed via
storybook-addon-pseudo-states) and extract the layout into a reusable
metabase/ui/stories/showcase module: StoryShowcase, StorySection, StoryRow,
StoryCode, and StoryJsx (a small JSX syntax highlighter).

* WIP: restyle Chip to Figma spec with semantic tokens

Restyle the Chip to match the updated Figma spec (GDGT-2483) using
semantic --mb-color-* tokens throughout — light/filled variants, sm/md
sizes, and the full state matrix (default / hover / pressed / selected /
hover-selected / disabled / disabled-selected). Sizes are wired via a
Mantine vars resolver; per-state styling is keyed off data-variant on
the root and data-checked/data-disabled on the label. Adds data-force-*
hooks so the showcase can render hover and pressed states statically.

Migrates the two existing call sites (BaseSMTPConnectionForm,
WebhookForm) from the invalid variant="brand" to variant="filled".

* wip: chip

* Align Chip colors with updated Figma semantic tokens

Switch to the post-refactor (GDGT-2517) role-based tokens design now binds in
Figma: neutral background_surface-secondary(-hover/-pressed) for unchecked
states, background_surface-brand-subtle + text-brand-strong for selected light,
background_surface-brand-strong for selected filled, and the corresponding
-hover variants. Also renames background-disabled/text-tertiary to
background_surface-disabled/text-disabled.

* Add Chip Overview story to Loki visual regression tests

* Format

* Add Loki reference for Chip Overview story

* Remove redundant comments from Chip restyle

* Cleanup

* Address review comment

* Cleanup
2026-06-15 11:47:28 +00:00
Sébastien
e860f98a43 Fix Guest embed options staying disabled until SSO terms are accepted (#75794) 2026-06-15 12:54:38 +02:00
Fraser
5fabe396ab Fix selectors/app module-boundary violations (#75777) 2026-06-15 21:14:36 +12:00
Sébastien
b55bed1cc1 Fix history.back 404 when Metabase is under a sub-path in interactive embedding (#75700) 2026-06-15 10:42:42 +02:00
Nicola Mometto
04bfeb20f6 deleted database handling for transforms (#74829)
* wip

* wip: fe

* tests

* tests

* serialize as tombstone

* mark dead transforms as stale

* fix tests

* fix

* fix migration

* move check

* fix FE when no dbs are available

* fix e2e test

* handle nil database in validation

* fix for broken dependency surfacing

* fix comment

* lenient for serdes

* fix: don't display while loading

* refactor

* handle migration comments

* move migration to 60

* handle N+1 in batched-hydrate for source-readable? by providing a per-model cache

* refactor

* random ids
2026-06-15 09:34:54 +01:00
Romeo Van Snick
94487e88fe Move AutoApi to RTK hooks and update callers (#75774)
* Add getXrayDashboard RTK endpoint to automagic-dashboards API

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* Migrate AutoApi callers to RTK hooks and remove AutoApi

Use the getXrayDashboard RTK endpoint instead of the legacy AutoApi.dashboard
service: the embedding-hub x-ray picker uses the useLazyGetXrayDashboardQuery
hook, and the transient dashboard fetch dispatches the endpoint via
runRtkEndpoint alongside the existing query-metadata fetch.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* Remove :tag* raw-substitution special case from substituteUrlTags

AutoApi.dashboard was the only route using a multi-segment :tag* placeholder,
and it now builds its URL directly through the RTK endpoint. With no remaining
callers, drop the raw-substitution branch so every URL tag is URL-encoded.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* Propagate abort signal through runRtkEndpoint for transient x-ray fetch

The transient-dashboard branch lost abort support when AutoApi.dashboard moved
to runRtkEndpoint: rapid dashboard navigation could leave stale in-flight
requests and apply late state updates. Thread an optional AbortSignal through
runRtkEndpoint that calls the RTK Query action's abort(), and pass
fetchDashboardCancellation.signal to both transient x-ray fetches.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* Guard getXrayDashboard against path-traversal in user-controlled subPath

The transient x-ray subPath comes from the /auto/dashboard/* route (splat +
location hash), so it is user-controlled and embedded raw into the request URL.
Reject '.'/'..' path segments in the getXrayDashboard query so a crafted URL
cannot walk out of the automagic-dashboards route into another same-origin
endpoint. Validating in the API query keeps the check at the source for every
caller.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-15 15:55:23 +10:00
Romeo Van Snick
f635373830 Move ActivityApi into metabase/api and update callers (#75773)
* Add getMostRecentlyViewedDashboard RTK endpoint

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* Migrate AddToDashSelectDashModal off ActivityApi to RTK hook

Replace the useAsync-based useMostRecentlyViewedDashboard wrapper with the
useGetMostRecentlyViewedDashboardQuery RTK hook and drop ActivityApi from
services.js.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* Test getMostRecentlyViewedDashboard tag invalidation

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* Use findRequests in activity API spec (review feedback)

Replace the fetchMock.callHistory.calls(...) call counters with the shared
findRequests helper, per review. The helpers become async (findRequests flushes
call history and returns the matching requests), so the waitFor callbacks await
them.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-15 15:23:22 +10:00
Fraser
c8535a0013 Fix VisualizationResult violations (#75671) 2026-06-15 14:39:46 +12:00
YB
58d6210131 [GDGT-376] [GDGT-2623] fix serdes export oom and n+1 table lookups (#75595)
* fix serdes export oom and n+1 table lookups: tally :seen as a frequency map and cache table-fk resolution

* formatting

* narrow table and database selects in export-table-fk resolver

* use export-resolver for database lookup

* use u/safe-inc and rename export report :seen to :entity-counts

* add comment on intentionally not routing field lookups through cached resolver
2026-06-15 10:10:56 +08:00
Romeo Van Snick
9cdd481113 Move metrics to feature tier (#75776)
* Extract shared metric primitives to common/metrics

Move the FilterPicker and TemporalBucketPicker components, the metric
dimension/date utilities, and the core metric types out of
metabase/metrics into the shared-tier metabase/common/metrics. These are
the only pieces of metrics that shared-tier consumers (metrics-viewer,
urls) depend on. Relocating them lets metrics become a feature-tier
module in a follow-up without creating shared -> feature import
violations.

Pure relocation: no behavior changes, only file moves and import path
updates.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* Move metrics module to the feature tier

With the shared metric primitives now living in common/metrics, nothing
in the shared tier reaches into metabase/metrics anymore. Reclassify it
as a feature module so the boundary linter enforces that it can only be
consumed by other feature modules (enterprise, public) and the app
shell.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-15 11:57:17 +10:00
Romeo Van Snick
97c046ec87 Move metabase/metrics-viewer to the feature tier (#75775)
* Extract metrics-viewer URL-state codec into shared common/metrics-viewer

The metrics-viewer URL-state codec is the contract shared between code
that builds deep-links into the viewer (shared `urls`/`metrics`) and the
viewer itself. Extract it into a new shared module so the viewer can move
to the feature tier without shared modules importing across the boundary.

- common/metrics-viewer/compact-schema.ts: moved verbatim from the feature
- common/metrics-viewer/types.ts: relocated MetricsViewerDisplayType,
  MetricsViewerDimensionBreakoutType, DimensionFilterValue
- common/metrics-viewer/url-state.ts: serialized types, compact schemas,
  encodeState/decodeState (pure, no metabase-lib runtime dependency)

The metabase-lib-bound live-state conversion stays in the feature's
url-serialization.ts, which now imports the codec from common and
re-exports it; viewer-state.ts and dimension-filters.ts re-export the
relocated types so internal call sites are unchanged. urls/metrics.ts now
imports the codec from common.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* Move metrics-viewer FE module to feature tier

Reclassify metrics-viewer from the shared tier to the feature tier now
that the shared URL-state codec it exposed lives in common/metrics-viewer.
No shared module imports the viewer across the boundary anymore, so the
move introduces no new module-boundary violations.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-15 08:49:38 +10:00
Maksym Yakubych
91628d34bf feat: inline charts (#75550) 2026-06-13 03:52:20 +00:00
Alexander Polyankin
e6670cff35 Fix flaky with-query-execution! test helper delivering stale batched executions (#75831) 2026-06-13 01:28:29 +00:00
Maksym Yakubych
55682e2e8c feat: full screen ai exploration page (#75502)
Closes [BOT-1667: Full-screen AI Exploration page](https://linear.app/metabase/issue/BOT-1667/full-screen-ai-exploration-page)

### Description

It's a full-screen AI Exploration page. Now, instead of opening the sidebar to display the chat with Metabot, it will be on its own page. 

### How to verify

1. Press New, then AI exploration.
2. Send a message to Metabot.

### Demo

_Upload a demo video or before/after screenshots if sensible or remove the section_

### Checklist

- [x] Tests have been added/updated to cover changes in this PR
- [x] If adding new Loki tests: they pass [stress testing](https://github.com/metabase/metabase/actions/workflows/loki-stress-test-flake-fix.yml)
2026-06-12 20:31:59 -03:00
Stas Gavrylov
eacd80a882 GDGT-2617 Hide Security Center setup prompt from non-admins (#75690)
* Hide Security Center setup prompt from non-admins (GDGT-2617)

The SecurityCenterPromoCard rendered for all users in the navbar promo slot,
but links to the admin-only /admin/security-center route, so non-admins hit a
permission error on click. Gate the card on getUserIsAdmin and skip its
admin-only RTK Query requests for non-admins.

* GDGT-2617: Refactor tests

* GDGT-2617 small adjustment

---------

Co-authored-by: Egor Grushin <egor@metabase.com>
2026-06-12 23:59:57 +01:00
Alexander Polyankin
5d1dd5c032 Improve app db connection pool usage with public dashboards (#75770) 2026-06-12 21:17:45 +00:00
Anderson
938ce52bb0 Make metric viewer controls responsive (#75250)
* Add responsive metric viewer controls

* Add dedicated css module files / break down components / Clean up props / Simplify test approach

* Address review comments
2026-06-12 17:14:34 -03:00
thackervishal
9e1b5779f9 docs: update session expiration page. (#74936)
* docs: update session expiration page. Restructures around how MAX_SESSION_AGE, MB_SESSION_TIMEOUT, and MB_SESSION_COOKIES interact
* Updates based on PR feedback

Co-authored-by: Alex Yarosh
2026-06-12 14:21:10 -04:00
technomancy
38fba4025f Avoid recreating datasets in snowflake (#75682)
* Fix describe-database-test in two namespaces to use sets.

This shouldn't be asserting about the full set of tables, just that at
least these tables exist.

* Stop deleting old datasets snowflake driver.

Turn off dynamic dataset loading support.

This is just too unreliable given the race conditions we see in cloud databases.

* Fix some other describe-database tests to assert subset of table names.
2026-06-12 11:19:02 -07:00
Chris Truter
23269606df Guarantee correctness for the search permissions fast path (#73037)
* Guarantee correctness for the search permissions fast path

Semantic search's read filter has a fast path for models whose read permission is a pure function of their denormalized `:collection_id`. Membership in that fast path was a hardcoded map, trusted by hand to stay in sync with each model's actual `mi/can-read?` — a silent-drift hazard: change a model's read perms and the map keeps routing it through the fast path with stale semantics.

Make the declaration authoritative. A model opts in with `define-collection-based-visibility!`, the single source of truth:

- t2-model form: installs `mi/can-read?` as a delegate to the shared `perms/can-read-via-parent-collection?` AND registers the model, in one call — so a model can't be in the fast-path set unless its `can-read?` is genuinely collection-id-only.
- search-model (string) form: registers a search-model whose index-row `:collection_id` is denormalized from a parent at index time (e.g. "indexed-entity" from its parent Card), with a required `:denormalized-from`.

Semantic search derives its fast-path set from these registries (running `search/specifications` first so the model namespaces load before the registries are read; wrapped in a `delay`). The fast path then calls the same helper the slow path installs, so the two can't diverge.

Guards: an identity-check test fails if anyone re-`defmethod`s `mi/can-read?` over a registered model; a contract test structurally traces each `:denormalized-from` claim through the spec's join-equality graph; a cold-start regression pins the specs-before-registry load order.

* Structurally verify keyword-form-derived fast-path search models

The :denormalized-from contract test only covered string-form
registrations. Keyword-form-derived siblings (card/dataset/metric/
dashboard) were swept into the collection-id-only fast path purely
because their spec's :model is a registered t2-model, with no check
that the index row's :collection_id is the entity's own collection.

Add a contract test asserting that, for each keyword-form-derived
search-model, tracing :collection-id through join equalities reaches
the spec's own registered t2-model -- so a future sibling sourcing
:collection-id from a join fails the test instead of silently gaining
incorrect read semantics.

* Guard against vacuous pass in keyword-form fast-path trace test

Add an (is (seq registered-t2-models)) assertion so the test fails
directly if the t2-model registry is ever empty, rather than silently
no-op'ing when the :when filter matches nothing.
2026-06-12 10:23:43 -07:00
thackervishal
8649b4126b DOC - one word fix in models intro (#74609)
* one word fix in models intro
* Update docs/data-modeling/models.md

Co-authored-by: Jeff Bruemmer <jeff.bruemmer@gmail.com>
2026-06-12 13:21:23 -04:00
Uladzimir Havenchyk
3e1f4bd247 fix: add optimistic updates for comments api in documents with many cards (#75763)
* fix: invalidate in-flight queries

* cleanup test

* use different approach

* update comment

* update comments
2026-06-12 16:59:47 +00:00
Riley Thompson
25a108bfb0 Fix quick filter on binned columns (#75767)
apply binning to column in quick filter drill thru
2026-06-12 12:55:34 -04:00
Riley Thompson
a989608ef6 Preserve column selection on joined tables in drill through (#75617)
* only use fields from join in drill thru

* improve and add test
2026-06-12 12:54:56 -04:00
Riley Thompson
498c7e3552 Fix self join column remapping (#75339)
* skip if same join alias

* add test
2026-06-12 12:48:16 -04:00
Eric Normand
a37645a74a Fix restoring items from a remote-synced collection (#75766)
The remote-synced dependents check in check-for-remote-sync-update fired
on any change to :archived, so restoring (true->false) an item in a
remote-synced collection threw "Used by remote synced content." — the
item's own containing collection is reported as a dependent. Guard the
check so it only runs when archiving (false->true), where blocking on
real dependents is the intended behavior.

Fixes GHY-3821
2026-06-12 11:12:22 -05:00
Tyler Plude
70da910602 [BOT-1685] Fix UX of Metabot inline SQL search (#75772)
* Scope inline SQL @-mention search to the selected database via table_db_id

* Recreate the Metabot prompt tiptap editor on database change so @-mention scoping doesn't use a stale onlyDatabaseId
2026-06-12 08:26:50 -07:00
Eric Normand
0376648dc8 GHY-3726 merge non-conflicting remote changes on export instead of blocking (#75050)
* GHY-3726 merge non-conflicting remote changes on export instead of blocking

When the remote branch had advanced past the last synced version, export
hard-failed ("Cannot export changes that will overwrite new changes in the
branch") even when local and remote touched different entities, deadlocking
push and pull.

Export now performs a 3-way merge keyed on serdes identity (model + id)
rather than on-disk path -- paths are derived from entity/collection names,
so a path-keyed merge would corrupt on renames/moves (duplicate entity_ids,
false conflicts). Non-conflicting remote changes are merged in, the merge
commit fast-forwards onto the remote tip, and the merged result is loaded
back into the local app DB (the pull half). Genuine same-entity conflicts
return a :conflict result; force still overwrites; an orphaned merge base
(force-push/rebase) surfaces as a conflict.

- merge.clj: entity-identity three-way-merge + conflict-label
- source.clj: serialize-specs, merge-and-store!
- git.clj/protocol.clj: snapshot-at
- impl.clj: export! merge/reconcile path, factored load-snapshot!

* GHY-3726 make merge an explicit user choice (preflight) instead of auto-merging

Reworks the push UX so merging is a deliberate option rather than something that
happens silently on push.

Backend:
- Gate the 3-way merge behind an explicit :merge? flag. A diverged export with
  neither force nor merge now returns :conflict (the UI decides how to proceed)
  rather than auto-merging.
- Add impl/preview-export-merge + source/preview-merge: a dry-run that reports
  {has_changes, clean, conflicts, summary, reason} without writing.
- New GET /export-preflight endpoint; :merge param on POST /export.

Frontend:
- Push now runs the preflight first. If the remote has advanced it opens the
  conflict modal offering Merge (when clean), Force push, Create new branch, and
  Cancel; a genuine conflict drops Merge and lists the conflicting entities.
- Merge action calls export with merge=true; commit message threads through.
- Invalidate all content tags after any successful sync (a merged export changes
  local data), and stop mapping export-task conflicts to the "setup" modal.

* GHY-3726 add local-only merge to the pull path

Pull now offers the same Merge affordance as push, but merges into local only
(the push is left for later) so a coworker's branch changes can be brought in
without discarding un-pushed local edits.

Backend:
- import! gains a :merge? flag → import-merged!: a local-only 3-way merge that
  loads the merged result into the app DB without writing to the remote.
  Remote-originated changes end up synced; un-pushed local changes are preserved
  as dirty (captured before the load and restored after, including pending
  deletions). Sets last-version to the remote tip so the branch is no longer
  behind. Genuine conflicts and an orphaned base return :conflict.
- source/compute-merge made public; source/specs->snapshot builds an in-memory
  snapshot so merged content can be loaded without going through git.
- :merge param on POST /import.

Frontend:
- Pull with un-pushed local changes runs the export preflight, then opens the
  conflict modal: clean → Merge (local) / Force push / New branch / Discard;
  conflict → drops Merge and lists the conflicting entities. Read-only keeps
  Discard-only. Merge on the pull variant calls import(merge=true).

* GHY-3726 run export preflight on push click for a single-modal flow

Previously, pushing when the remote had advanced showed two modals: the commit
form, then (after submit) the merge/conflict modal. Now the export preflight
runs when the user clicks Push, so we open the correct modal directly:
- remote unchanged -> commit-message modal (plain push)
- remote ahead     -> merge/conflict modal straight away

The commit-message field moves into the conflict modal's push variant (merge /
force / new branch all push), so there's a single modal in every case.

* GHY-3726 show entity name (not entity_id) in merge conflict labels

conflict-label now parses the entity's name from the serialized content and
shows e.g. "Card A (collections/.../card_a.yaml)" instead of the raw serdes
"Card <entity_id>". Falls back to model + id when the content has no name.

* GHY-3726 fix CI: implement snapshot-at on remaining Source mocks, mock export-preflight, stabilize merge API tests

After merging master:
- Add snapshot-at to the Source mocks that lacked it (api_test missing-branch /
  failing sources, wrapping_source_test) so clj-kondo's protocol-method check passes.
- Mock the new /export-preflight (and /export) endpoints in the shared remote-sync
  test server-mocks, fixing GitSyncControls unit test's unmocked-request failure.
- Stub load-snapshot! in the export/import merge API tests: the heavy app-DB
  reconcile load inside the async task is slow/racy on the MySQL/MariaDB app-db
  matrix; merge correctness is covered synchronously by impl-test.

* GHY-3726 reject branch-mismatch on export/preflight (multi-tab CAS guard)

Make branch a required param on POST /export and GET /export-preflight and
reject the request (409 with branch_mismatch + current_branch) when it
disagrees with the authoritative remote-sync-branch setting, so a stale tab
can't push to or preview the wrong branch after another session switched
branches. Thread the validated branch into preview-export-merge. Also fixes
a pre-existing dropped body schema on /export (misplaced ]).

Frontend surfaces the mismatch as a 'this view is out of date' modal with a
Refresh action on both push and pull, instead of swallowing the preflight
error.

* GHY-3726 reject branch-mismatch on import via expected_branch token

/import handles both pull (branch == current) and branch switch (branch is a
different target), so a single branch param can't double as the CAS
assertion like it does on export. Add an optional expected_branch param =
the branch the client believes is currently active; reject (409
branch_mismatch) when it disagrees with the remote-sync-branch setting.
Optional so scheduled auto-import and the admin-settings pull degrade safely.

Frontend sends expected_branch=currentBranch from the app-bar pull/switch and
the conflict-modal merge-import/discard flows.

* GHY-3726 make import expected_branch required

The CAS token is only a real guarantee if every endpoint caller must send it;
an optional token leaves the same opt-in hole we're closing, and is
inconsistent with export's required branch. The scheduled auto-import calls
impl/async-import! directly (not this endpoint), so only FE callers hit
/import and all of them have a current branch to assert. Wire the remaining
caller (admin-settings PullChangesButton) and guard currentBranch on switch.

* GHY-3726 surface branch-mismatch as refresh modal across all sync flows

Route every GitSyncControls catch (push preflight, dirty-pull preflight,
non-dirty pull, branch switch) through a shared showBranchMismatchIfPresent
helper that opens the 'this view is out of date' modal on a 409
branch_mismatch, instead of swallowing it or showing a generic toast.

* GHY-3726 make merge version/status bookkeeping atomic with the reconcile

load-snapshot! gains an optional :finalize! thunk that runs inside its existing
transaction (after remove-unsynced!/sync-objects!). Callers move their
version/status bookkeeping into it so it commits atomically with the
object-table reconcile, or rolls back together.

Closes two crash windows:
- import-merged! could permanently lose captured dirty markers if it died
  between the load (which marks everything synced) and restore-dirty-objects!
  (a retry's capture would then find nothing). Restore now commits with
  sync-objects! or not at all.
- export-merged! now advances the version only after the reconcile load, so a
  crash leaves it at the old base and a retry re-merges (the push is
  idempotent) instead of pointing past un-reconciled local state.

load-metabase! stays outside the transaction (unchanged).

* GHY-3726 fail loudly when a pushed merge commit can't be reconciled locally

export-merged! previously skipped the local reconcile (via when-let) if
snapshot-at returned nil for the just-written commit, yet still advanced the
version and marked everything synced — silently leaving local without the
folded-in remote changes. Throw instead: export!'s catch fails the task, the
version stays at the old base (set-version! is in the reconcile's finalize),
and a retry re-merges and reconciles. The push is idempotent.

* GHY-3726 test: pull merge folds remote change while keeping local deletion dirty

Combined round-trip covering the disjoint-change invariant: one merge applies an
unrelated remote change (folded in, synced) while a pending local deletion is
preserved as dirty via restore-dirty-objects!' re-insert branch, and the version
advances to the remote tip.

* GHY-3726 throw on duplicate entity_id within a merge side

index-by-key warned and silently overwrote when two specs on the same side
shared a serdes identity (same model + entity_id at two paths), dropping one
file — silent corruption. Throw instead so a duplicate entity_id surfaces.
Path-fallback keys (non-serdes files, unique paths) still only warn.

* GHY-3726 show busy state during preflight and toast when mergeability check fails

- Push and dirty-pull both await the export preflight (which re-serializes the
  library and reads remote trees); track it with isCheckingPreflight and fold
  it into the branch control's isLoading so the control shows a spinner and
  disables meanwhile, instead of looking idle.
- A dirty pull whose preflight fails for a non-branch-mismatch reason now toasts
  'Couldn't check whether your changes can be merged...' before opening the pull
  modal (without the merge option), rather than silently degrading.

Test asserts the toast via the undo store (the toaster isn't mounted in this
test harness).

* GHY-3726 fix two branch/merge bugs found in review

- Branch switch dropped .unwrap() on importChanges, so the CAS 409 resolved
  silently instead of throwing: no refresh modal and trackBranchSwitched fired
  as success. Add .unwrap() so it routes to the refresh modal. (review Issue 1)
- A dirty-but-up-to-date merge pull reported a spurious 'remote history was
  rewritten' conflict: async-import! left base-snapshot nil when the remote
  hadn't advanced, which import-merged! treated as an orphaned base. Resolve
  base to the remote tip when not diverged and no-op the merge (keep local
  dirty). Genuine orphaned-base/no-prior-sync still conflict. (review Issue 2)

Tests: clean-switch-rejected refresh modal; import merge no-op when the remote
hasn't advanced (drives async-import! base resolution, review Issue 3). Also
de-flaked two pre-existing race-prone pull tests by waiting for the dirty state
to settle before clicking.

* GHY-3726 review cleanup: atomicity, conflict surfacing, labels, logging

- Plain/force export now advances the version and marks objects synced inside a
  transaction (store! push stays outside), matching the merge path. (Issue 4)
- Export tasks that end in :conflict now toast 'the remote branch changed
  before your push finished...' instead of being silently swallowed by the
  listener middleware (covers the preflight->execute race). (Issue 5)
- conflict-label no longer destructures a [::by-path path] key into garbage
  model/id; uses the filename. (Issue 7)
- merge_test deftests marked ^:parallel. (Issue 9)
- entity-identity parse-failure log raised debug->warn so a malformed
  serialized file is visible. (Note B)
- Documented same?'s conservative cross-side-rename conflict edge. (Note A)

* GHY-3726 test async-{import,export}! base-snapshot resolution via a versioned fake Source

Per review: deeper coverage of the async-*! wrappers without a hand-built
:base-snapshot or stubbed compute-merge. Adds test-helpers/versioned-source (a
Source with a controllable current version + version->files map; snapshot-at
returns nil for an unreachable/orphaned base). Six tests drive the real base
resolution and real 3-way merge end-to-end (load-snapshot! stubbed-with-finalize
to skip the slow app-DB reconcile): not-diverged no-op keeps local dirty,
diverged resolves base and folds in a remote-only entity, base unreachable ->
conflict, no prior sync -> conflict; plus the export mirrors. This is the test
shape that would have caught the dirty-but-up-to-date false conflict directly.

* GHY-3726 rename SyncConflictModal mutation-wrappers.ts to hooks.ts

Per review (alxnddr): repo convention is hooks.ts. Pure rename + import update.

* GHY-3726 surface export-conflict toast via useToast in GitSyncControls

Per review (alxnddr): prefer useToast over addUndo. useToast is a hook and can't
run in the listener middleware, and the shared useSyncStatus hook mounts twice
(StatusListing + GitSyncControls) so it would double-toast. Move the toast to
GitSyncControls (single mount, already has sendToast, observes the task): on an
export task in conflict it toasts once via useToast then clears the task, which
also fixes the latent poll/refetch re-fire of the old middleware addUndo. The
middleware now just dismisses the modal for export conflicts. Drops the undo
reducer from the middleware spec; toast assertion moves to the GitSyncControls
spec.

* GHY-3726 test: force export overwrites upstream changes (full re-serialize)

A force export must drop files the remote diverged with — something the
incremental fast-path (which preserves every untouched file) cannot do. This
test shows the same upstream file is preserved by an incremental export but
removed by a force export, pinning force? = full re-serialize and guarding
against regressing to an incremental force.

* GHY-3726 implement apply-changes! on remaining SourceSnapshot mocks

The master merge added apply-changes! to the SourceSnapshot protocol; three of
my implementations didn't implement it, which clj-kondo flags (and CI fails on):
- source/specs->snapshot (in-memory read-only merge snapshot) -> throws
- source_test entities->snapshot -> records upserts like write-files!
- test_helpers versioned mock snapshot -> applies upserts/deletes, preserves the rest

* Send expected_branch in e2e read-write import helper

The /import endpoint requires expected_branch (CAS guard against stale
multi-tab clients). configureGitAndPullChanges' read-write path posted an
empty body, failing validation. Send expected_branch: "main" to match the
branch configureGit sets.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* Flatten nested conds in export!

The undiverged export path was an inner cond nested inside the outer
dispatch cond. Hoist its inputs (disabled-files, dirty-rows, plan) into
the let — gated so incremental-plan stays lazy on the force/merge paths —
and lift its three branches into the single top-level cond. No behavior
change.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* Update out-of-date push e2e test for preflight-first flow

The push redesign runs the export preflight on push-click and, when the
remote has advanced, opens the conflict modal directly instead of the
"Push to Git" commit-message modal. Update the out-of-date push test to
assert the conflict modal ("The remote branch has new changes…") and pick
"Create a new branch and push changes there" there, matching the new UX.
The stale two-step assertion was masked until the expected_branch helper
fix let setup succeed.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* Match either push conflict-modal title in out-of-date push e2e test

The 3-way merge in this scenario comes back as a conflict, not a clean
merge (the local re-serialization of the synced set isn't byte-identical
to base, so the remote-renamed question reads as changed on both sides),
so the modal title is "Some of your changes conflict with the remote
branch…" rather than "The remote branch has new changes…". Match on
/remote branch/, common to both push-variant titles, so the test passes
regardless of merge cleanliness.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-12 10:03:34 -05:00
Timothy S. Dean
e11255220d Fix: core.async/timeout combined with core.async/go prevented fat dynamic var GC (#75769)
* This has to be the strangest Clojure footgun I've seen in many years.

* Kondo-ignore in the test file what the src file ignores
2026-06-12 11:00:53 -04:00
Oleh K
376db5fb43 refactor: Sweep brand/filter/summarize tokens to core-* (#75332)
* GDGT-2529: Add new color tokens alongside existing ones

Foundation for the GDGT-2517 color token refactor. Adds new and
renamed semantic tokens without changing any existing emission or
call sites. After this PR, both old and new CSS variable names
resolve to the same values; migration sweeps will move consumers
to the new names, and the cleanup PR will remove the old.

* GDGT-2529: Move kept tokens from Legacy to New section

These 14 tokens are not being renamed (text-primary, text-brand,
border-brand, tooltip-*, switch-off, etc.) and have no equivalent
in the New section. Moving them into the New group prevents
readers from assuming they're discontinued when they encounter
the Legacy section's 'Prefer using new ones' comment.

* Update Loki Snapshots

* Rearrange and aort new colors keys alphabetically

* Add explanation to core-brand/filter/summarize copy

* Update Loki Snapshots

* Split light/dark themes into `deprecatedColors` and `actualColors`

Per review feedback on #75227 — replaces the `// Legacy colors` and `// New`
section comments with named top-level constants so the intent shows up in
the import graph rather than as inline prose.

* Sweep `brand`/`filter`/`summarize` to `core-*` (GDGT-2530)

Renames all consumer-side references to the legacy semantic tokens to
their `core-` counterparts added by the foundation PR:

- `--mb-color-brand` → `--mb-color-core-brand` (and `-hover`)
- `--mb-color-filter` → `--mb-color-core-filter`
- `--mb-color-summarize` → `--mb-color-core-summarize`

Sweep covers: `var(--mb-color-…)` references in CSS/styled-components,
`color()`/`alpha()`/`lighten()`/`darken()`/`getColor()` helper calls,
Mantine `color`/`c`/`bg`/`iconColor`/`colorScheme`/`primaryColor`
props, type unions, default destructure values, ternary/fallback
expressions in JSX, static color-name lists, plus three doc files
(emotion-migrate skill, custom-viz template README, mantine-styles
contract test).

Both old and new CSS variables continue to emit during the migration,
and the SDK/whitelabel input keys (`theme.colors.brand`,
`application-colors.brand`) are unchanged; the rename is internal.

* Re-align emotion-migrate SKILL.md tables after the brand → core-brand edits

* Fix two visual regressions from the brand → core-brand sweep

- SDK v1 `theme.colors.brand` (and `filter`/`summarize`) now writes
  to both the legacy and the new core-* internal token via
  `SDK_TO_MAIN_APP_COLORS_MAPPING`, so consumers that have already
  migrated to `var(--mb-color-core-brand)` pick up SDK overrides
  the same way legacy consumers do. The v2 path was already covered
  by the mirror in `derive-theme.ts`; this PR patches the v1 path.

- `<Alert color="core-brand">` lost its tinted background because
  `Alert.config.tsx` was synthesizing the background-color token name
  via `background-${props.color}`, which produced the non-existent
  `background-core-brand`. Added an explicit `core-brand → background-brand`
  case so the tint resolves correctly while the `background-*` family
  is still on the legacy namespace.

* Revert sweep over CssVarsDeclarationPlugin test assertions

The sweep accidentally flipped two `toContain("--mb-color-brand: ;")`
assertions to `core-brand`, but the test's synthetic TS fixtures still
define `type ColorKey = "brand" | …` — the plugin parses those literally
and emits `--mb-color-brand`, so the assertions need to track the fixture,
not the production rename.

* Mirror `brand`/`filter`/`summarize` to `core-*` in embedding palette

The SDK v1 path rebuilds the global `colors` map via
`setGlobalEmbeddingColors` → `getEmbeddingColorPalette` on every theme
or whitelabel change, which bypasses the mirror in `derive-theme.ts`.
Whitelabel `application-colors` comes in via `appPalette` directly
keyed by the legacy `brand`/`filter`/`summarize` names, so before this
fix only the legacy CSS vars picked up the whitelabel override and the
new `core-*` keys silently fell back to their built-in defaults.

Mirror them at the end of the merge so consumers reading
`var(--mb-color-core-brand)` get the right color regardless of where
in the merge chain the brand value came from.

* Try to fix missing core-* variables

* Remove manual patch of core-* colors in getEmbeddingColorPalette

---------

Co-authored-by: Metabase Automation <github-automation@metabase.com>
2026-06-12 15:57:59 +02:00
Riley Thompson
b25f53509f Fix param values from joined source card (#75751)
* fix param values from joined source card

* Update test/metabase/parameters/custom_values_test.clj

Co-authored-by: metamben <103100869+metamben@users.noreply.github.com>

---------

Co-authored-by: metamben <103100869+metamben@users.noreply.github.com>
2026-06-12 09:39:45 -04:00
Oleh K
5b36ecb251 Add Storybook showcase helpers and a Figma component skill (#75693)
* Add reusable Storybook showcase components

Storybook-only presentational primitives for building component showcase
stories: StoryShowcase (bordered panel + title), StorySection, StoryRow,
and StoryJsx (monospace JSX with light, theme-aware syntax highlighting via
tokenizeJsx). Live in metabase/ui/stories/showcase so they stay out of the
metabase/ui barrel.

* Add metabase-ui-component-from-figma skill

Project skill codifying the workflow for restyling/implementing a metabase/ui
component from a Figma spec: check out the issue, map usage/blast radius, build
the Storybook showcase first, style from exact semantic tokens, iterate, then
commit and migrate call sites. Registered in frontend/CLAUDE.md.

* Handle closing tags and text children in StoryJsx tokenizer

Track whether we're inside a tag vs. element content so text children render as
plain text, and stop clearing the tag-name expectation on `/` so the name in a
closing tag (e.g. `</Kbd>`) is recognized as a tag. Handles `<Kbd>⌘</Kbd>`.

* Refine component-from-figma skill per review feedback

Raise the abstraction (drop Chip-specific anecdotes and one-time setup
rationale), split into update-existing vs create-new modes with the
showcase/styling order flipped for new components, derive the showcase matrix
from the component's full state space (theme excluded — it's global, not a
prop), and add a scoped test-first step (custom-interactive components only, not
thin Mantine wrappers).

* Address skill review feedback: trim duplication, add story example

Trim the Phase 1 mode-decision step and Phase 2 path reiteration, add an upfront
Figma desktop MCP availability check, mention playwright-mcp as an alternative
browser MCP, and embed an illustrative finalized matrix story.

* Flag non-color token scale mismatches in component-from-figma skill

Extend the discrepancy guardrail beyond colors: radius/spacing/elevation/type
values that have no matching step in the codebase's scales (e.g. Figma radius
md=12px vs --mantine-radius-md=8px) should be flagged to the user, not silently
baked in as literals.

* Fix typo
2026-06-12 14:09:20 +02:00
Braden Shepherdson
5f7c893f98 Add admin endpoints for OAuth clients and tokens (#74933)
* Add admin endpoints for OAuth clients and tokens

This allows auditing of the outstanding OAuth clients and tokens created
by users on an instance to enable MCP server access for external agents
like Claude Code.

* Add migrations for `decision` and `decided_at` columns; update API endpoints

* Adding unit tests for the new endpoint and adjusting checks on other tests

* Overhaul the approach: now a separate audit log table

Note that the migration was dropped but since it only added columns,
there's no real problem.

* FE - Authorizations page

* e2e tests

* Design Feedback

* Adding redirect-uri column, making the page wider

* column tweaking

* BE lint fixes

* fix be tests

* add fk index to oauth_client_event

---------

Co-authored-by: Nick Fitzpatrick <nickfitz.582@gmail.com>
2026-06-12 17:17:52 +05:30
Stas Gavrylov
19cd3a9b25 GDGT-2597 Order Schema Viewer fields to match the table metadata view (#75716)
GDGT-2597: Order Schema Viewer fields to match the table metadata view

The `/api/ee/erd` endpoint fetched a table's fields with no `:order-by`, so
columns came back in raw DB order. Sort them by `field-order-rule`
([position asc, lower(name) asc]) — the same ordering the `:fields` hydration
applies — so the Schema Viewer's field list matches
`/api/table/:id/query_metadata` instead of appearing unsorted.

Reuse the shared `field-order-rule` constant (declaring the warehouse-schema
module dependency) so the two surfaces can't drift.
2026-06-12 14:25:33 +03:00
Stas Gavrylov
d7c289b747 GDGT-2598: Schema Viewer shows the database name for schema-less DBs (MongoDB, MySQL, …) (#75616)
* GDGT-2598: Make SchemaPickerInput display DB name for schemaless DBs

* Fix metadataUrl generation for Schema Viewer's  info panel header

* GDGT-2598: Cleanup

* GDGT-2598: Address PR comments
2026-06-12 12:23:17 +03:00
Romeo Van Snick
bf0ac23a9f Move embedding-specific rewrites to metabase/embedding (#75582)
* Inline publicBase

* Inline internalBase

* Add middleware for rewriting /api/embed to /api/embed_preview where needed

* Remove getEmbedBase helper and inline /api/embed everywhere

* Tighten the prefix regexp

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>

* Fix e2e tests

* Register embed-preview rewrite from render, not module load

The module-load registration polluted the shared api client in unrelated
unit tests (the SDK JWT subpath test saw a stray unmocked
/api/session/properties). Register it synchronously from usePublicEndpoints
instead: render runs parent-first while effects run child-first, so this
still lands before the child dashboard fetcher's effect and fixes the embed
preview race, but only when an embed page actually renders.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* Guard based on isEmbed

Co-authored-by: Mahatthana (Kelvin) Nomsawadi <me@bboykelvin.dev>

* Simplify unit test

---------

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-authored-by: Mahatthana (Kelvin) Nomsawadi <me@bboykelvin.dev>
2026-06-12 14:52:05 +10:00
Alexander Polyankin
c2c7a41a53 Workspace UI revamped (#75687) 2026-06-11 20:33:37 -04:00
github-automation-metabase
73e2bfc4db Auto-update documentation for master (#75764)
Update generated docs (master)

Co-authored-by: github-actions <github-actions@github.com>
2026-06-11 17:35:26 -06:00
Romeo Van Snick
eff532c50f Move search to feature tier (#75613)
* Relocate non-UI search core to metabase/utils/search (lib tier)

Moves the search filter types, SearchFilterKeys constants, analytics
helpers, and URL/location utils out of metabase/search into the lib-tier
metabase/utils/search so non-feature modules (api, palette, plugins,
entity pickers) can keep importing them once search becomes a feature
module. Pure relocation + import-path rewrites; no behavior change.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* Move shared date filter components to metabase/querying

SearchFilterDateDisplay and SearchFilterDatePicker are generic
querying-date UI (built on metabase/querying + metabase/ui), reused by
the transforms run-filter bar. Move them into
metabase/querying/filters/components so non-search modules can keep
using them once search becomes a feature module.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* Move shared search-result components to metabase/common/components

SearchResult, SearchResultLink, and InfoText are presentational
components reused by the global nav search bar (and via it, dashboard
LinkViz), the command palette, and entity pickers. Move the cluster
into metabase/common/components so these non-search consumers no longer
import from the search module.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* Move search FE module to the feature tier

Reclassify metabase/search from a shared module to a feature module in
the module-boundaries config. Shared/lib modules can no longer import
from search; the reusable primitives previously living under search were
relocated to their natural homes (metabase/utils/search,
metabase/querying, metabase/common/components) in the preceding commits.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* Move search core from utils to metabase/common/search (shared tier)

metabase/utils should stay product-agnostic, but the search filter
types, SearchFilterKeys constants, analytics helpers, and URL/location
utils are Metabase search-domain code. Move them from
metabase/utils/search into metabase/common/search, the shared
Metabase-domain layer (mirroring the querying <-> query_builder split
and sitting alongside the SearchResult components already in
metabase/common/components). Pure relocation + import-path rewrites; no
behavior change.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-12 09:18:54 +10:00
Romeo Van Snick
c0136115c0 Move ActionApi into RTK hooks and clean up callers (#75679)
* Add RTK Query endpoints for action execution and prefetch

Add executeAction, prefetchActionValues, executeDashcardAction, and
prefetchDashcardValues endpoints to actionApi, plus the request/result
types they need. These replace the legacy ActionsApi helpers in
metabase/services; callers migrate in follow-up commits.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* Migrate ActionsApi callers to RTK action endpoints

Replace the legacy ActionsApi.execute/prefetchValues/executeDashcardAction/
prefetchDashcardValues calls with the RTK endpoints:

- Thunks dispatch the endpoints' initiate() (executeAction, executeRowAction).
- Imperative prefetches in components go through the runRtkEndpoint helper.
- DeleteObjectModal uses the useExecuteActionMutation hook.
- Public-dashboard branches keep using PublicApi for now.

The ActionExecuteModal test builds fetchInitialValues via useDispatch +
runRtkEndpoint, mirroring the real callers.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* Remove ActionsApi from metabase/services

All callers now use the RTK action endpoints, so drop the legacy
ActionsApi helper.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* Fix CI failures and address review feedback

- Use runRtkEndpoint in the executeAction/executeRowAction thunks instead of
  dispatching initiate().unwrap() directly. The codebase Dispatch type returns
  `unknown`, so the direct form failed type-check (TS2571); runRtkEndpoint is the
  blessed thunk helper and returns the result.
- DeleteObjectModal: coalesce a missing objectId to null so the execute params
  satisfy ParametersForActionExecution (TS2322).
- ActionVizForm unit test: render with renderWithProviders so the new
  useDispatch/RTK prefetch path has a store.
- Add keepUnusedDataFor: 0 to the prefetch queries (Copilot review): they are
  imperative fetch-and-discard calls with row/dashcard-specific params that
  rarely repeat, so retaining cache entries adds memory for no benefit.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-12 09:18:06 +10:00
metamben
e31c5c451a Add test verifying model persistence with duplicate column names (#75675)
Closes QUE2-182
2026-06-11 22:56:16 +00:00
Eric Normand
aff4a39ab7 Fix circular dependency error for cards with self-referencing parameters (#75731)
* Fix circular dependency error for cards with self-referencing parameters

A card whose parameter sources dropdown values from the card itself (or
from another card whose parameter points back) declares itself as a
serdes dependency. The circular-dependency strip keys for Card did not
include :parameters, so the load retry hit the same self-dependency and
aborted with "Circular dependency on Card <entity_id>".

Add :parameters to the strip keys for Card and Dashboard. The existing
two-pass mechanism then handles it: the circular retry inserts the
entity without parameters, and the outer pass restores them once the
row exists for FK resolution.

Also fix xform-one: coercion re-added stripped keys as explicit nils,
which violated the NOT NULL constraint on report_dashboard.parameters
during stripped updates.

* Address review: strip-mechanism hardening and test cleanup

- Fix the model->circular-dependency-keys docstring: it promised
  vector-path stripping with :* that no consumer ever implemented;
  document the actual top-level-keys contract and the two-pass restore.
- When a load fails for an entity that was stripped to break a
  dependency loop, say so in the error: with continue-on-error the
  stripped row survives the import, and the message now names the keys
  the entity may be missing.
- Exclude stripped keys from the :copy spec in xform-one, so the strip
  contract holds for all spec-processing steps, not just :transform
  and :coerce keys.
- Share one card-sourced-param helper across the three new tests and
  assert both cards' parameters in the mutual-reference test.
2026-06-11 17:22:33 -05:00
Anderson
bf2544ff7d Display column labels below split map charts (#74774)
* Show column labels below split map charts

* Clarify metric dimension item index fields
2026-06-11 18:49:16 -03:00
Anderson
af5aa9e59f Add metric labels to dimension picker selects / Add expr metric occurrence counts (#75292)
* Add metric labels to dimension picker selects + add metric expression occurance counts

* Expand accordions on search / Fix accordion bg color

* Replace ExpressionSubToken.count with ExpressionSubToken.occurrenceCount / Fix unit test
2026-06-11 18:48:08 -03:00
John Swanson
984e06d3a1 Analyze query executions for metadata (#72726)
* Track metric-api executions as QueryExecution rows

Tag metric-plan MBQL queries from /api/metric/dataset and
/api/metric/breakout-values with :executed-by and :context :metric so
process-userland-query writes a QueryExecution row; extend the ::context
enum with :metric to match.

* Usage-metadata: infer implicit dimensions, segments, and metrics from query executions

A new Quartz task extracts facts from each day's QueryExecution rows and
writes daily rollups to four source-keyed tables (SourceDimensionDaily,
SourceMetricDaily, SourceSegmentDaily, SourceDimensionProfileDaily). An
insights layer queries the rollups to surface the most common ad-hoc
aggregations, filters, and breakouts.

Deviation from the original plan: dimension "interestingness"
(single-value, all-null, low-cardinality) was meant to come from cached
"count by <dim>" queries. We read it from the field's existing
Field.fingerprint instead (see
metabase.usage-metadata.models.source-dimension-profile-daily).

* Usage-metadata: treat queries as opaque via metabase.lib

Replaces direct (:stages ...) / (:source-table ...) reads and raw :field
clause pattern-matching in usage_metadata.extract with the kondo-allow-listed
metabase.lib public API (metabase.lib.core + metabase.lib.util.match).

- select-root-owner: uses lib/primary-source-table-id + lib/primary-source-card-id
- extract-usage-facts: uses lib/stage-count
- clause-field-refs: uses lib.util.match/match-many instead of clojure.walk/postwalk
- atomic-filter-clauses: uses lib/clause-of-type?
- field-ref detection: lib/clause-of-type? + lib/field-ref-id, dropping the
  hand-rolled field-ref? / field-ref-info helpers

Also drops the :join-alias fallback and its Field.table_id DB lookup
(*field->table-id* cache + binding wrapper in batch.clj), since
lib/find-matching-column already resolves joined columns via the :table-id
on the returned column metadata.

* random ids, no rollbacks on migration

* Stream the Query-table read in usage-metadata batch

The reducible-select over :model/Query filtered by query_hash could
buffer a large slice of the query table in driver memory on Postgres.
Wrap it in mdb/streaming-reducible so the JDBC fetch actually streams
via a server-side cursor.

* CR response

* update module config

* CR response

* oops, fix stale references in tests

* fix migration version

* address self-review

I was taking one final look at this before merging and noticed a couple
small issues that I wanted to clean up before I merged.

The big one was aggregation types I hadn't thought about, e.g.
`[:- [:metric A] [:metric B]]` is a supported custom expression.

Take a query like this:
```
{
  "lib/type":"mbql/query",
  "stages":[{
    "lib/type":"mbql.stage/mbql",
    "source-table":210,
    "aggregation":[
      [
        "-",
        {"lib/uuid":"bd30ecfd-f921-4e52-a253-2b3d348d63d4","name":"foo","display-name":"foo"},
        ["metric",{"lib/uuid":"27c99d2b-d951-4024-ab21-729cac76b54a","effective-type":"type/Integer"},94],
        ["metric",{"lib/uuid":"084c85de-68ab-456c-a451-a3dbce2be844","effective-type":"type/Integer"},95]
      ]
    ]
  }],
  "database":2
}
```

This would have resulted in a `NOT NULL` crash because we couldn't
extract the source (card or table) from an aggregation like this.

The fix is just to ignore these cases. This does have tradeoffs: it
means we're explicitly going to undercount some usages. However,
handling them "correctly" would require actually resolving the metric
reference during extraction, which we want to avoid for reasons of
efficiency and correctness (the metric may have changed since the query
was run).

Since these are fairly niche queries, I think this is probably okay?

I also added batching to the INSERTs to avoid errors on very large
numbers of parameters, and added a safety harness around calling
`lib-be/normalize-query` in case that throws.

* don't use deprecated `mt/mbql-query`

* Usage-metadata: composite segment candidates (#72982)

* Usage-metadata: mine composite segment candidates from stage filter baskets

Adds a composite rollup sibling to `source_segment_daily`: one row per stage's
top-level filter basket (the implicit `:and` of all top-level filters), keyed
on the canonicalized whole-clause and carrying the sorted array of atom
fingerprints. Storage stays `O(queries)` -- no powerset expansion at write
time.

Read path runs Apriori FIM (size-capped 2..5, absolute + relative support
floors, closed itemsets only) over a rolling window of composite rollups for a
given owner and returns reconstructed `[:and ...]` candidates that recur
across baskets with different tails. Itemsets whose atom-set matches an
existing saved `:model/Segment` definition are filtered out.

* kondo

* seq->vec before normalization

* rm `:count` from `implicit-composite-segments`, document vals

`count` and `support` were the same thing by two different names; I
removed `count` because in this domain `support` and `support-ratio`
are more meaningful.

I also added better descriptions of all the return values, because I had
to refresh my memory on them when I was looking back at this code :)

* fixup migration

* bad john, no raw mbql manipulation

* more CR response

* vectorize lazyseqs as part of normalization

* don't break everything, though

* fix tests for mysql/mariadb

* Move `atomic-filters` and `referenced-columns` to lib

Two new Lib helpers, both re-exported from lib.core:

- `lib.filter/atomic-filters` — stage helper that flattens top-level `:and`'s
  into a list of atomic boolean filter clauses. Mirrors `lib/filters`'s
  return shape.

- `lib.fe-util/referenced-columns` — walks the expression-parts AST and
  returns the `:metadata/column` leaves of a clause. Runtime-gates the
  input via `lib.util/clause?` / `column-metadata?`. Lives in `fe-util` rather
  than next to returned-columns because the impl is built on
  expression-parts and the natural placement would create a cycle —
  noted in the docstring.

`usage-metadata/extract.clj`'s segment, composite and metric fact builders
now compose these helpers. Drops `atomic-filter-clauses`, `participants-from-parts`,
and the `column-metadata?` predicate.

* Move `referenced-columns` out of `lib.fe-util` into its own namespace

* move migrations to 063

* cljfmt
2026-06-11 14:44:15 -07:00
Ben Grabow
8c0e64ac1f Log connection pool destruction at INFO for flaky-test diagnosis (#75754)
Destroying a c3p0 pool closes its checked-out connections, which kills
any in-flight query on another thread with errors like SQL Server's
"The result set is closed" (DEV-2161). These destroy/invalidate events
were only logged at DEBUG, so CI test-log artifacts (which capture
INFO+) could not show whether a pool destroy coincided with a flaky
query failure.

Log destroy-pool! and invalidate-pool-for-db! at INFO so the next
occurrence is diagnosable from the CI artifact alone. In tests these
log lines automatically carry the running test's name via the logging
context, attributing each pool destruction to its trigger.

Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-06-11 16:25:51 -05:00
metamben
55f798f532 Add test demonstrating that #33698 has been fixed (#75555)
Closes QUE2-189
2026-06-12 00:13:22 +03:00
metamben
9ab9b085d4 Fall back on effective-type for time fields (#75442)
Closes QUE2-650
2026-06-12 00:13:09 +03:00
technomancy
f6423a79b5 Use *driver* to test h2/postgres with and without mbql5 (#75729)
Consider this atonement for my crimes of having used with-redefs.
2026-06-11 20:56:53 +00:00
Anderson
daef67027c Fix filter not respected when downloading visualizer-affected card from dashboard (#75720)
Fix filtered dashboard card xlsx downloads
2026-06-11 16:02:10 -03:00
Eric Normand
87e4ecf563 Audit-log background and startup remote-sync imports (#75730)
* Audit-log background and startup remote-sync imports

Auto-imports (Quartz job) and startup imports now publish
:event/remote-sync-import with :user-id nil and :details {:auto true},
so the activity log records every pull on read-only instances.

Also fixes the manual paths: the :on-success callbacks in api.clj were
single-arity but run-async! invokes them with [task-id result], so the
ArityException was silently swallowed and manual sync events were never
recorded either.

Fixes #73335

* Guard audit publish in auto-import; test manual import audit row

publish-event! rethrows handler exceptions, so an audit-log failure
would have marked an already-successful auto-import as failed in task
history. Wrap it in try/catch like run-async! does for :on-success.

Also adds a regression test asserting manual POST /import writes a
remote-sync-import audit row attributed to the user with no :auto flag,
locking in the callback arity fix.
2026-06-11 18:49:51 +00:00
Andrei Chernykh
525aca564b Fix extra divider in metric menu for non-admin users (#75667)
Fix extra divider in metric menu for non-admin users without usage analytics access
2026-06-11 11:25:08 -07:00
metamben
f02742f0a9 Return nested fields from sandboxed queries (#75358)
* Return nested fields from sandboxed queries

Closes QUE2-651
2026-06-11 18:12:06 +00:00
Timothy S. Dean
80646be599 Fix Athena sync failing on partitioned Iceberg tables (#75666)
Syncing a partitioned Iceberg table left it with no fields at all.

When the driver can't get field metadata over JDBC it runs `DESCRIBE` instead (#43980, #58441). `DESCRIBE` output ends with a partition section that re-lists the partition columns. For Hive tables
they reappear with their real types and were already deduplicated; for Iceberg tables they reappear with the partition transform (e.g. `identity`) where the type should be. The parser treated
those rows as real columns, couldn't map `identity` to a base type, and the insert for *all* of the table's fields failed on the null base type.

Fix:

- Stop parsing `DESCRIBE` output at the partition section — it never contains new columns. (Section markers verified against Hive's `TextDescTableFormatter`, where this format comes from; link in
the code.)
- Also drop any row that repeats an earlier column name, in case the marker text ever changes.
- Unknown database types now fall back to `:type/*` instead of failing the sync, same as the JDBC path.
2026-06-11 11:54:57 -06:00
Ben Grabow
9ea5f3c265 Omit case-sensitive option when DB doesn't support it (#75665)
* Omit case-sensitive option when DB doesn't support it

* Accept database as setup opt and construct state in setup
2026-06-11 17:36:30 +00:00
github-automation-metabase
306ce29098 Auto-update documentation for master (#75673)
Update generated docs (master)

Co-authored-by: github-actions <github-actions@github.com>
2026-06-11 13:28:31 -04:00
Chris Truter
a9f80db7fd Prefix semantic search queries with query: for Arctic embedding models (#75718) 2026-06-11 17:19:21 +00:00
Rodrigo López Dato
4015a7408e Add JVM hardware info to diagnostic details (#75561)
Include the number of CPUs, max JVM memory, and total physical host
memory in the bug-reporting diagnostic info, under the system-info
section. The max JVM figure matches the "Maximum memory available to
JVM" line printed at startup; the total host memory shows how much
the machine actually has, making it easier to diagnose performance
problems from a bug report.
2026-06-11 16:49:53 +00:00
Anderson
ce69cec459 Fix schema breadcrumb after table route refresh (#75428) 2026-06-11 13:27:29 -03:00
Anderson
8c750b74fb Fix stale dashboard breadcrumb after rename (#75432)
* Fix stale dashboard breadcrumb after rename

* Fix dashboard breadcrumb context leaks
2026-06-11 13:27:26 -03:00
Gabriel Moreno
f234679608 Fix library e2e empty state flaky test (#75429)
* fix: wait for library to be created

* fix: formatting
2026-06-11 11:43:36 -04:00
Chris Truter
92c67e8c62 Fix inconsistent db defaults for data_layer / data_authority (#75689) 2026-06-11 15:14:50 +00:00
Nathan Voxland
4d1d85814c Tidy query preprocessing, dataset streaming, and permission helpers (#75551)
* Tidy persisted-info handling in query preprocessing and SQL QP

Resolve the persisted-cache source SQL from the metadata provider in the SQL
QP, and clear any stale :persisted-info/native keys during preprocessing.

* Add docstrings to query-permission helpers and simplify related tests

* Normalize query maps before permission checks on card save and revert

* Build dataset-streaming :info server-side and tidy pivot query setup

* Fixed mbql and formatting
2026-06-11 09:38:32 -05:00
Chris Truter
1bb1a3848a Weaken verified and official-collection search weights (#75676) 2026-06-11 16:14:02 +02:00
Riley Thompson
6cb272064f Fix replacement with field filter alias (#75644)
* fix field filter alias

* trigger ci
2026-06-11 13:26:36 +00:00
Brad Anderson
a866b4827c GRO-388 mimic Snowplow.setUserId with metaplow (#75565) 2026-06-11 09:18:44 -04:00
Nicola Mometto
de9f9979c0 Transform scheduler freshness (#75309)
* wip: transform freshness

* fix: don't consider inactive jobs

* fix: don't include unscheduled transforms, use appdb now

* fix npe

* just compute next scheduled run

* test

* chore: return scheduled/dependency from API

* notes on freshness

* handle review comments
2026-06-11 12:28:30 +00:00
Uladzimir Havenchyk
f77ee419cc fix(documents): add proper rendering optimizations of the cards (#70832)
- do not render cards which are not visible at the viewport and close to it, use placeholders instead
- fetch data only when browser is ready to provide the best user experience
- prepare document for printing by rendering all cards in background, so no placeholders are rendered instead of cards
2026-06-11 11:20:42 +00:00
Fraser
f702f1aa45 Fix setup module violation (#75692) 2026-06-11 13:34:53 +03:00
Fraser
e8adb6a17e Fix EmbeddingTheme violations (#75683) 2026-06-11 20:13:00 +12:00
Fraser
5a2357c36f Fix pdf download violations (#75459) 2026-06-11 20:03:42 +12:00
Uladzimir Havenchyk
08fd155739 fix: wait for all databases to be loaded and fix sources of flakinesses (#75173)
* test: attempt to fix flaky test

* fix: only auto-select database in DataSelector when exactly one is available

The segments "Filter by table" picker (and every other multi-database
DataSelector) auto-selected the first database and drilled into its tables,
skipping the database list. This was a regression from #64406, which changed
the skipSteps guard from `=== 1` to `>= 1` while adding disabled-database
filtering. Restore the `=== 1` invariant on the filtered enabled databases.

This was the root cause of the metabase#52411 e2e flake: opening the picker
intermittently drilled into Sample Database (alphabetically first, single
schema) instead of showing the list, depending on metadata-load timing. With
the fix the list is always shown, so the test drops the drill-recovery and
keeps only force-click for the unrelated re-render detach.

- Unit regression test: with multiple databases, none is auto-selected.
- e2e: simplified back to force-click.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix: don't auto-select a database while the picker list is still loading

The earlier === 1 fix addressed the fully-loaded multi-database case, but the
segments picker still drilled into Sample Database intermittently: the picker's
`databases` list is built by hydrating queried database ids through the
metadata store and dropping those not yet hydrated, so mid-load it can
transiently contain only the first database. skipSteps then treated that as
"the only available database" and auto-selected it.

Gate the auto-select on `databasesLoaded` (every queried database hydrated, or
`databases` passed directly), so a transient partial list can't trigger it.
Together with the === 1 guard this fixes both the metabase#52411 flake and the
fully-loaded multi-DB regression at the source.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix(AccordionList): key rows by section identity, not list position

AccordionList rendered each row with `key={index}`, a flat position into a
list that changes length/order as async sections resolve. In the legacy
DataSelector picker, the "Saved Questions" section appears/disappears as the
`/api/database?saved=true` and `/api/search` requests settle, shifting every
later section's index and remounting it. That detached "Writable Postgres12"
mid-click and made metabase#52411 flaky (worse under CI network throttling).

Key rows by `section.key`/`section.name` instead, falling back to the index
when a section has no stable identity. Each row now keeps its React instance
across these updates, fixing the detach at its source for every AccordionList
consumer (data selector, notebook data source, filter pickers, etc.).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* test: drop force-click now that AccordionList keys are stable (52411)

The re-render detach that force-click worked around is fixed at its source by
keying AccordionList rows on section identity (prev commit). "Writable
Postgres12" no longer remounts when the saved-questions/search requests settle,
so a plain click is safe.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* move db loading logic to initializeQB

* fix(AccordionList): key rows by section identity, not list position

The metabase#52411 stress test (50x, qa_db, network throttling) still
failed 21/50 with the initializeQB preload alone: every failure detached
on the first database click ("Writable Postgres12") in the DataSelector
popover. That test runs on /admin/datamodel/segments, which never calls
initializeQB — so the preload can't help it. The real cause is the
database AccordionList re-rendering as databases hydrate into metadata
incrementally: with key={index}, shifting list entries remount the row
mid-click and Cypress loses it.

Key rows by section.key/section.name instead, falling back to the index
when a section has no stable identity, so each row keeps its React
instance across these updates. Restores the fix from the earlier branch
state with a corrected comment (the churn is the database list itself,
not a Saved Questions section). The initializeQB preload is kept as an
orthogonal improvement for the QB data selector.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* Revert "fix(AccordionList): key rows by section identity, not list position"

This reverts commit 0f79b29ca2.

* restore automatic db selection when only 1 db is available

* fix: don't auto-select a database while the picker list is still loading

The === 1 guard alone still let the segments "Filter by table" picker drill
into Sample Database intermittently (metabase#52411): the DataSelector connect
builds `databases` by hydrating queried database ids through the metadata store
and dropping those not yet hydrated, so mid-load the list can transiently hold
only the first database. skipSteps then treated that single entry as "the only
available database" and auto-selected it, advancing into its schema/tables and
unmounting the list mid-click.

Gate the DATABASE_STEP auto-select on `databasesLoaded` (every queried database
hydrated, or `databases` passed directly), so a transient partial list can't
trigger it. Together with the === 1 guard this fixes the flake at its source for
the segments page and every other DataSelector consumer.

Confirmed via CI stress runs that the initializeQB preload could not fix this:
the segments page never calls initializeQB, and === 1 alone leaves the
transient-single race. Adds a unit regression test.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* improve comments

* cleanup

* fixup

* rename function

* cleanup test

* cleanup test

---------

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-11 09:30:35 +03:00
Eric Normand
db96fadd29 GHY-3725 incremental export fast-path for remote sync (#75057)
* GHY-3725 incremental export fast-path for remote sync

Export previously re-serialized the entire synced content set on every push,
regardless of what changed, so a single in-place edit cost as much as a full
export (O(total synced entities)).

Add an incremental fast-path: when every pending dirty change is an in-place
content update of an entity-id model whose serialized path already exists in
the repo holding that same entity, serialize and write only the changed
entities and preserve everything else. Any other case (create, rename, delete,
collection rename, path/hybrid models, mixed batches) falls back to the
existing full export, which reconciles the whole tree correctly.

- source protocol: new apply-changes! (write/overwrite + delete, preserve rest)
- git: apply-changes! copies existing tree entries by object id, inserts only
  changed blobs
- spec: extract-dirty-entities-for-export serializes only the dirty rows
- impl: safe-incremental-upserts gate (verified via a real YAML parse of the
  existing file's entity_id); export! chooses incremental vs full and marks
  only processed rows synced
- RemoteSyncObject: dirty-rows helper
- MockSource: apply-changes! for tests

Measured ~constant (~4ms) regardless of synced-set size vs linear full export
(39x-353x faster for one edited card at 50-500 entities).

* GHY-3725 extend incremental export to renames and deletes

Persist each entity's last-exported file path on the RemoteSyncObject row so
the incremental fast-path can resolve old paths for renames and deletes
(previously these fell back to a full export).

- migration: add remote_sync_object.file_path (nullable)
- source: store!/->file-spec capture {model_type entity_id path} during the
  write pass (path computed once to avoid advancing the slug generator)
- impl: incremental-plan now handles update (in-place + rename: write new +
  delete old path), delete/removed (delete stored path + drop row), and falls
  back to full when file_path is absent (post-import; self-heals on next full
  export) or a name-collision is detected; record-exported-paths! persists
  file_path after every full export (export-only population)

Benchmark: in-place edit, rename, and delete are all ~constant (2-6ms)
regardless of synced-set size vs linear full export (215/650/1650ms at
50/200/500 entities).

* GHY-3725 fix incremental export dropping cross-scope dependencies

The incremental fast-path wrote only the changed entities, so an in-place edit
that newly referenced remote-sync content outside the synced set (e.g. a card
based on a card in another collection) omitted that dependency's file — a full
export pulls it via serdes/descendants, so this was a regression introduced by
incremental updates.

Guard incremental updates with a dependency-closure check: compute the entity's
transitive serdes/descendants + serdes/required closure and fall back to a full
export when any referenced remote-sync content model isn't tracked by a
RemoteSyncObject row. References to already-synced content (which has a row and
is in the repo) stay incremental; non-content deps (databases, resolved by
reference at import) are ignored.

Tests: an update/create that adds a cross-scope reference exports the dependency;
an update referencing a card in another synced collection stays incremental.

* GHY-3725 make creates incremental in remote-sync export

Creates previously forced a full export. Handle them on the incremental
fast-path: write the new entity's file (no old path to delete), guarded by the
same checks as updates — the target path must be free or already hold this
entity (dedup-collision guard), the dependency closure must already be in the
repo (else fall back to full so serdes/descendants pulls it), and all upsert
paths in a batch must be distinct.

Create and in-place update are kept distinct: an update with no stored file_path
to a new path is an unresolvable rename and still falls back to full, whereas a
create to a free path is written directly.

Tests: create is incremental and records file_path while preserving other files;
a create whose path collides with a different entity falls back to full; a create
referencing un-synced content falls back (dependency guard).

* GHY-3725 pull dependency closure incrementally instead of falling back

When an incremental create/update references remote-sync content not yet in the
repo (e.g. a synced card based on a card in a non-synced collection), we now
serialize and write that dependency alongside the change instead of falling back
to a full export. The set written is the change's transitive
serdes/descendants+required closure restricted to untracked content models — the
same subset a full export would emit for the change — so the repo ends up
consistent while everything else is preserved.

- untracked-content-deps returns the untracked content entities in the closure
- incremental-plan accumulates them per row and dep-upserts serializes them
  (reusing the storage context so paths dedupe consistently)
- a path collision while pulling a dep still falls back to full; pulled deps get
  no RemoteSyncObject row, matching full-export behavior

Tests: an update/create that adds a cross-collection reference now stays
incremental and exports the dependency.

* GHY-3725 fix CI: migration dir, kondo, task-leak, transform cleanup

- migration: move file_path changeset to migrations/063 (next dev version)
  and bump id to v63; 062 is the released version (check-migration-versions).
- incremental_export_test: add (fixtures/initialize :db) + clean-remote-sync-state
  fixtures so RemoteSyncTask rows don't leak into later tests (was causing
  "A running task exists" failures in settings-test/snippets-test); replace the
  deprecated mt/mbql-query with explicit query maps (kondo).
- source_test / wrapping_source_test: implement apply-changes! on the
  SourceSnapshot mocks now that it's part of the protocol (kondo
  missing-protocol-method warnings).
- impl: incremental export now falls back to a full export when the repo still
  holds files for content types disabled by settings (transforms /
  python-libraries / snippets), since only a full export reconciles those away.

* Whitespace

* GHY-3725 address review feedback

- dirty-objects is now implemented in terms of dirty-rows (no duplicated query);
  trim caller details from the dirty-rows docstring.
- store! returns {:version :entries} instead of taking a path-atom argument, so
  the exported-path tracking is a return value rather than a threaded side
  effect; ->file-spec's serialization is shared via entity->file-spec.
- rename extract-dirty-entities-for-export -> extract-entities-for-rows (it just
  groups rows by model type and serdes-extracts them; also used for dependency
  rows, not only dirty ones) and trim caller details from its docstring.
2026-06-11 05:31:59 +00:00
Nemanja Glumac
f0ab65ee4f Allow e2e stress-test workflow to build the jar when needed (#75669) 2026-06-11 07:30:21 +02:00
Alexander Polyankin
9c76f5c629 Show upstream archived on inactive dependencies (#73186) 2026-06-11 02:58:27 +00:00
Anderson
79b7c2a604 Clean up metrics viewer token action menu (#75206)
Clean up metric token action menu
2026-06-10 19:32:27 -03:00
Eric Normand
ecbbbd0543 Add regression test: native SQL card flagging itself as broken (GHY-3682) (#75674)
Add regression test for native SQL card flagging itself as broken (GHY-3682)

Editing a native SQL card to use an explicit column list (instead of
SELECT *) previously caused /check-card to report the card being edited
as broken, surfacing the "these changes will break some other things"
popup for the card itself. The underlying native validation was fixed in
639c8deb; this adds an endpoint-level regression test covering the
reported symptom: a native source card plus a downstream consumer, edited
to explicit valid columns, must return success with no broken cards.
2026-06-10 22:24:54 +00:00
Romeo Van Snick
e95194e698 Remove legacy GET in leaflet map (#75577)
* Switch LeafletTilePinMap to use api.request instead of legacy GET

* Await data url read before calling done

* Wait for the tile image to actually load

* Move image loading to asyn helper

* Enable retry

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>

---------

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
2026-06-11 08:24:22 +10:00
Romeo Van Snick
2a1b4e0163 Remove legacy POST in internal-analytics (#75578)
Remove legacy GET in internal-analytics
2026-06-11 08:23:32 +10:00
Romeo Van Snick
02d7cbdded Convert PulseApi to RTK hooks (#75583)
* Convert pulse api to RTK hooks

* Use actual response type

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>

* Fix formatting

---------

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
2026-06-11 08:23:16 +10:00
Kevin Kelleher
84ed39b816 Use dashboard filters in Visualizer data (#75498)
use datasets in visualizer initial state as primary rather than fallback
2026-06-10 18:06:40 -04:00
Eric Normand
8083e6e3aa Abort initial sync status when metadata fetch fails (#75660)
When `describe-database` throws during the metadata-fetch stage (e.g.
Athena credentials lacking `glue:GetDatabases`), the exception was raised
before `initial_sync_status` was ever updated, leaving the database stuck
at "incomplete" and the UI perpetually showing "Syncing tables…".

Wrap the fetch so a throw marks the initial sync aborted before
re-throwing, so the status flips to "aborted" and the UI surfaces the
failure. This is general to any driver, not just Athena.

Fixes GHY-3534.
2026-06-10 20:08:05 +00:00
Eric Normand
b47b12cf6b Stop retrying permanently-failed fingerprinting every sync (#75640)
* Stop retrying permanently-failed fingerprinting every sync

When fingerprinting a field failed, `fingerprint_version` was never
advanced, so `fields-to-fingerprint` re-selected the field via
`fingerprint_version < latest` and retried it on every subsequent sync —
indefinitely. For an effectively-permanent failure (slow view timeout,
permission denied, unsupported type, coercion/dispatch error) this is pure
waste, and amplifies GHY-3266 by leaking a long-running query each sync.

Advance `fingerprint_version` to the latest on failure so the field is not
re-selected, EXCEPT for transient connection/environment errors (DB
unreachable, pool exhausted, SSL handshake), which are left untouched so the
next sync retries them — matching the existing abandon-sync behavior.

Covers both failure paths: the whole-table sampling-query throwable and the
per-field fingerprinter throwable.

* Use public do-not-retry-exception? instead of a single-use wrapper

* Clarify do-not-retry-exception? branch in fingerprint failure handling

* Rename do-not-retry-exception? to transient-exception? with a private hierarchy

Move the exception classification into a private hierarchy populated through a
new public register-transient-exception fn (replacing the global derive), and
rename the keyword to ::transient-exception and the predicate to
transient-exception?. The predicate's truthy meaning now matches its name.
2026-06-10 18:38:09 +00:00
technomancy
e7262aa5b3 Update driver stress test workflow to pass in necessary snowflake env. (#75574)
* Update driver stress test workflow to pass in necessary snowflake env.

Seems bad that this has to be duplicated a bunch of places.

* Unset CDPATH to avoid failures in dev-install.
2026-06-10 11:11:11 -07:00
thackervishal
d24c2551d7 DOC - fixing 2 incorrect links and adding a line about arrays in json unfolding (#74610) 2026-06-10 14:00:35 -04:00
Alex Yarosh
9ac9602d26 docs: alert management (#75478)
* alert management

* Optimized 2 images, saved 245.6 KB (51.8% smaller)

* Apply suggestions from code review

Co-authored-by: Jeff Bruemmer <jeff.bruemmer@gmail.com>

---------

Co-authored-by: Metabase Automation <github-automation@metabase.com>
Co-authored-by: Jeff Bruemmer <jeff.bruemmer@gmail.com>
2026-06-10 17:56:09 +00:00
John Swanson
05a70000b5 Fix metric viewer for metrics based on native sql cards (#72404)
metrics based on native sql support
2026-06-10 10:26:49 -07:00
John Swanson
7d8738db7e UXW-4443: Re-encrypt additional columns during key rotation (#75536)
* UXW-4443: Re-encrypt additional columns during key rotation

There were 6 columns that we weren't correctly rotating during key
rotation. This is a mechanical change to add them to the list of things
we rotate. The "Right Way" to fix this would probably be closer to
UXW-4444 (a refactor that mechanically prevents this from happening
again) but this triage makes sense for now.

[UXW-4443: Re-encrypt additional encrypted columns during key rotation](https://linear.app/metabase/issue/UXW-4443/re-encrypt-additional-encrypted-columns-during-key-rotation)

* fix test
2026-06-10 10:25:44 -07:00
Chris Truter
6d7cbe822c Make app DB authoritative for table existence in table-candidates (#75588) 2026-06-10 18:51:37 +02:00
Timothy S. Dean
f2afce5817 Fix dropped admin email on persist-refresh failures in dev/test (#75342)
The `:event/persisted-model-refresh-error` event schema required `:trigger` to be an org.quartz.Trigger instance, but in dev mode the producer was hardcoding it to the strings `"Scheduled"` or `"Manual".

The schema mismatch had no effect in production, but in dev and test runs `mu/validate-throw` rejected the payload; the resulting exception was swallowed by the try/catch in `publish-refresh-error-event!`. I only noticed because I happened to write a test that exercised the path.

Fix: the real Quartz Trigger was available on the JobExecutionContext that `refresh-job-fn!` receives, so we use it.
2026-06-10 10:16:24 -06:00
Anderson
357cc76125 Fix UI error handling when archiving a collection fails (#75431)
Fix silent archive failures
2026-06-10 13:07:44 -03:00
Aleksandr Lesnenko
31f7b5d27e cli oauth (#75451)
Co-authored-by: Bryan Maass <bryan.maass@gmail.com>
2026-06-10 09:37:04 -06:00
Stas Gavrylov
a1d5e4b071 Fix flaky schema-viewer writable-Postgres test (#75094)
* Fix flaky schema-viewer writable-Postgres test by re-triggering DB sync

The writable-Postgres beforeEach used a single fire-and-forget
resyncDatabase whose wait only required *any* table to exist. sync_schema
submits to a single-threaded task pool (a slow predecessor task blocks the
queue) and is subject to duplicate-op suppression (a sync already in flight
makes ours a no-op), so the newly-created sv_test/sv_extra schemas were
intermittently never discovered and the hook timed out with zero tables.

Replace it with a re-triggering poll that re-POSTs sync_schema periodically
and waits until every required table reaches initial_sync_status=complete,
which also guarantees fields and FKs are synced.

* Refactor resyncDatabase to support waiting for multiple tables to complete sync

* Add /sync_schema issue link
2026-06-10 18:24:54 +03:00
Chris Truter
7ff2080a85 Further MCP improvements for HTTPS detection (#75531) 2026-06-10 09:16:00 -06:00
Alex Yarosh
155bac70a7 docs: metric updates (#75581)
* metric updates

* Optimized 2 images, saved 246.3 KB (49.0% smaller)

* Update docs/data-modeling/metrics.md

Co-authored-by: Jeff Bruemmer <jeff.bruemmer@gmail.com>

---------

Co-authored-by: Metabase Automation <github-automation@metabase.com>
Co-authored-by: Jeff Bruemmer <jeff.bruemmer@gmail.com>
2026-06-10 15:15:41 +00:00
Anderson
ddd34b2742 Fix MySQL/MariaDB collation mismatch failure in v62 legacy_library_root_entity_ids migrations (#75496)
* Fix MySQL collation in Library root migrations

* Change approach: fix existing sql directly and replace changeset ids

* Update unit tests

* Include filepath in the DELETE WHERE clause / Replace only changed changesets' ids
2026-06-10 11:57:31 -03:00
Nathan Voxland
edaf5a64f7 Switch appdb-index-size to be a real db check (#75284) 2026-06-10 09:51:41 -05:00
Maksym Yakubych
862fc7a2eb feat: make slackbot use the default internal metabot id (#74727)
Closes [BOT-1581: Slackbot `metabot` record can not be configured in the admin UI](https://linear.app/metabase/issue/BOT-1581/slackbot-metabot-record-can-not-be-configured-in-the-admin-ui)

### Description

Smartbot was using its own separate Metabot i, which made it unconfigurable via admin settings. Now it uses the same internal id as the Metabot inside the app.

### Checklist

- [x] Tests have been added/updated to cover changes in this PR
- [x] If adding new Loki tests: they pass [stress testing](https://github.com/metabase/metabase/actions/workflows/loki-stress-test-flake-fix.yml)
2026-06-10 11:51:24 -03:00
Ngoc Khuat
c13ba1bcc8 Count webhook channels in alert management (#75379) 2026-06-10 14:18:38 +00:00
Alex Yarosh
442c7ee351 docs: mcp updates (#75580)
* mcp updates

* Optimized 1 images, saved 80.5 KB (48.3% smaller)

* Apply suggestions from code review

Co-authored-by: Jeff Bruemmer <jeff.bruemmer@gmail.com>

* Apply suggestions from code review

Co-authored-by: Jeff Bruemmer <jeff.bruemmer@gmail.com>

---------

Co-authored-by: Metabase Automation <github-automation@metabase.com>
Co-authored-by: Jeff Bruemmer <jeff.bruemmer@gmail.com>
2026-06-10 13:47:14 +00:00
Chris Truter
75ea90c029 Tolerate legacy search weight overrides (#75614)
* fix(search): tolerate legacy flat weight overrides (#BOT-1677)

`/api/search` 500'd on stats with `Don't know how to create ISeq from:
java.lang.Double`. `normalize-override-keys` (added in #75388) reduces over
the persisted `experimental-search-weight-overrides` setting assuming every
value is a per-context weight map. Pre-#50338 the setting was a flat
`{scorer weight}` map, and stats still has one persisted -- so a bare Double
landed where a map was expected and `merge` blew up.

Flat overrides used to apply to every context, so fold their bare weights into
the `:default` base (where an explicit `:default` context override still wins),
rather than crashing or silently dropping them.

* Tighten normalize-override-keys docstring; move rationale inline
2026-06-10 08:58:36 -04:00
Oleh K
09c574b988 Render transform job details before the transform plan loads (#75499)
* GDGT-2595 Render job details before transform plan loads

The transform plan endpoint (`/api/transform-job/:id/transforms`) can take
10+ seconds to compute, which previously blocked the entire job details page
behind a full-screen spinner.

Make only the transform plan non-blocking: the page now renders as soon as the
job and databases queries resolve, and the transform list loads independently
in TransformsSection (which owns its own loading/error states). While the plan
loads, editing stays locked and the run button shows a "Checking permissions…"
state instead of a misleading "no permission" message.

* Address PR review feedback

- Reorder getRunButtonInfo so an in-progress run takes precedence over the
  loading state, dropping the negative status guard
- Pass the transform plan loading flag directly into the run button
- Rewrite JobPage spec to use fake timers + a mock delay instead of a custom
  deferred helper
2026-06-10 14:03:01 +02:00
Romeo Van Snick
403ceceffc Move metabase/browse to feature tier (#75377)
* Make metabase/browser a feature

* Move metrics analytics to metrics module

* Move shared test setups to metabase-types
2026-06-10 22:00:39 +10:00
Romeo Van Snick
b6b6c0f6b9 Move metabase/setup to feature tier (#75584)
* Make setup a feature

* Move getIsHosted and getTokenFeature to metabase/selectors/settings
2026-06-10 21:56:12 +10:00
Chris Truter
f0c53fd0de Look up load_skill ids by string instead of keywordizing (#75587) 2026-06-10 18:52:34 +07:00
Roman Abdulmanov
2b40d21357 Removed deprecated header (#75594) 2026-06-10 08:34:07 -03:00
Uladzimir Havenchyk
e54f13d3ea docs: update the link to custom viz docs (#75603) 2026-06-10 11:31:57 +00:00
Nemanja Glumac
b79f6cdd50 Fix timeline events intermittently not rendered (#75566)
* Show timeline events when timelines load after the question

The query builder dispatched showTimelinesForCollection from an effect
gated only on bookmarks loading, with no dependency on the fetched
timelines. When GET /api/timeline?include=events resolved after the
question and bookmarks had loaded, the effect had already run once with
an empty timeline list and never re-ran, so the events were never pushed
into the chart's visible set and no marker rendered. The race surfaced
most often on ad-hoc native questions, whose load path tends to lose it.

Gate the effect on the timelines being loaded and re-run it when they
arrive, restoring the guarantee that was lost when the Timelines.loadList
HOC was removed.

Add a deterministic Jest regression test that forces the late-timelines
load order (fails without this fix, passes with it), and make the e2e
native-query timeline test deterministic via a monthly aggregation and a
wait on the timelines request.

* Point timeline-events regression test at the product bug

The test guards against the timeline-events load-order regression, so its
description should reference the product bug rather than the flaky-test
ticket it surfaced through.

---------

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-10 13:08:26 +02:00
Nicola Mometto
dacc0adad0 Send daily digest for failing transform jobs (#75472)
* stop sending JIT job failure emails

* send daily digest instead

* add test

* cleanup

* keep disabled

* don't use cutoff, use daily range

* fix tests
2026-06-10 11:14:51 +01:00
Mahatthana (Kelvin) Nomsawadi
27fcd32d5d chore: after gold SDK release checklist on master (#75487)
* chore: bump SDK version to 0.63.0-alpha on master

* chore: add lifecycle comments to tag-latest job
2026-06-10 08:36:12 +01:00
Chris Truter
4246d2979a Fix test flake related to search analytics (#75576) 2026-06-10 08:24:29 +02:00
Fraser
114d345f86 Fix plugins violations (#75076) 2026-06-10 15:37:36 +12:00
YB
1142b7e95f [GDGT-2539] enforce run transform result schema (#75376)
Adds a malli schema to driver/run-transform! and the python equivalent that would throw in dev/test and no-ops in prod (to prevent any performance cost from running malli there).

The schema enforces that the map minimally includes the rows-affected key, which is consumed by instrumentation.
2026-06-10 09:57:29 +08:00
github-automation-metabase
f1211ebc0a Auto-update documentation for master (#75568)
Update generated docs (master)

Co-authored-by: github-actions <github-actions@github.com>
2026-06-09 20:45:36 -04:00
Andrei Chernykh
f8873d9ae9 Migrate copy components off EmotionCSS and replace CopyWidget with CopyTextInput (#75425)
Migrate copy components off Emotion and replace CopyWidget with CopyTextInput
2026-06-09 16:43:04 -07:00
Romeo Van Snick
5dc0b835b0 Clean up API client options and remove XHR calls (#74368)
* Remove XHR code path from api client

* Remove fetch: true option in api calles

* Remove fetch option from api as it is now the default

* Remove unused json option for api

* Make retry options internal

* Remove bodyParamName hack

* Remove superfluous hasBody parameter

* Split out request and legacy _makeMethod so we can be more explicit about boty

* Clean up the retry loop

* Move headers to getClientHeaders

* Remove raw option by extending the url template syntax

* Move locale to global variable

* Remove DEFAULT_OPTIONS

* Pull retry into separate module

* Remove _makeRequestWithRetries

* Use async-await for making requests

* Pull out responseStatus helper

* Simplify _buildHeaders

* Remove cast

* Fold _dispatch into _makeRequest

* Use correct type for body

* Use type of transformed response

* Avoid cast on method

* Add appendSearchParams helper

* Remove method arg to substitute-url

* Add buildUrl method

* Move appendQueryparams into branch

* Simplify options parsing

* Add test for undefined params

* Remove unneeded Request instantiation

* Pull out getResponseBody helper

* Parse the relative path correctly

* Use explicit Request so findRequests works

* Remove query wrapper in favor af a thin baseQuery wrapper

* Remove the reliance on fetch-then-abort semantics from XHR

* Make internal methods private

* Split up ApiClient

* Move substituteUrlTags into client utils

* Remove weird self = this

* Move apiRequestManipulationMiddleware into separate file

* Move errors into separate file

* Move CSRF machinery into separate file

* Add telativePath helper

* Only read header when we need it

* Small code cleanups

* Remove unused query.ts

* Remove unused fetch option

* Add types for event map

* Switch to numeric event names

* Remove unused options from transformResponse

* Remove old function-style transformer option

* Remove lengthy comment that was obvious

* Move up status emit

* Flatten branch

* Remove unused fields from ResponseErrorInfo

* Use EventEmitter for response errors

* Make retry cancellable

* Cancel retries too

* Pass url directly

* Simplify OnBeforeRequestHandlerConfig

* Centralize request options resolution

* Centralize error handling and retrying in _dispatch

* Clarify the response body parser

* Return empty body as null instead of the empty string

* Trust the server's content type

* Move types to shared types file

* Avoid parsing/cloning/buffering responses when it's not needed

* Replace transformResponse with rawResponse

* Statically determine the response type when possible

* Remove incorrect comment

* Make ai-streaming go through api client too

* Update override-requests spec for flat headers API

After rebasing onto master, the overrideRequests tests (added on master)
still used the old nested `options: { hasBody }` shape. This branch
flattened OnBeforeRequestHandlerConfig to an optional `headers`, dropping
`options`/`hasBody`, so the tests no longer type-check. Drop the obsolete
fields; the tests only assert on url/method.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* Return content type in server mocks

* Make sure headers exist

* Add note about 202 respone handling containing a bug

* Avoid manually parsing JSON responses

* Make aiStreamingRequest return the same shape as APIClient.request

* Remove unserializable error tpye

* Standardize on AbortError

* Import RequestMethod from metabase/api/client

* Do not export non-legacy helpers from legacy-client

* Simplify middleware handler init

* Make internal methods private

* Remove rename

* Move all legacy client machinery into legacy-client

* Fix embed URL :token substitution from legacy POST/PUT bodies

* Expose request body to middleware so embed :token resolves on api.request

Guest-embed saved-card queries (run-query.ts) dispatch through RTK with the
embed `token` in the request body. `api.request` only handed `params` to the
onBeforeRequest middleware, so the embed override could rewrite the URL to
/api/embed/card/:token/query but never fill :token — the request went to
/api/embed/card//query and the question never loaded. The previous commit fixed
only the legacy GET/POST helpers; this does the same for api.request, which is
the path the RTK query endpoints actually use (card/pivot/dashcard queries).

Merge the body into the middleware data (params win on clash), then re-split
leftovers: param-origin keys to the querystring, the rest to the JSON body.
Keys consumed by URL :tag substitution (the embed token) land in neither.

Fixes the guest-embed SDK component tests and e2e guest-embed/token-refresh
flows (pivot + map-tile failures were downstream of the question not loading).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* Pass body as a separate argument in middleware

* Fix tests

* Disambiguate network errors from parse errors

* Await visualizer modal save before closing to fix dashcard series race

The visualizer "Save" handlers dispatched the async replace/add thunks
without awaiting them and closed the modal immediately. Those thunks
commit the new card series to the dashcard only after fetching the
referenced cards, so closing early let a subsequent dashboard save run
against the not-yet-updated dashcard and drop the new series.

The blocking XHR transport hid this race; with fetch (dispatched on the
next microtask) it surfaced consistently, e.g. metrics-dashboard.cy
"should be able to combine scalar metrics on a dashcard" where the
dashboard save found no changes, issued no PUT, and the orphaned series
query 404'd ("Some columns are missing").

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* Fix import

* Enable retry and RTK query endpoints

* Retry RTK queries on 503

* Avoid losing status on unparsable responses

* Fix test

---------

Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-06-10 07:43:32 +10:00
Fraser
70604b58c1 Move Metabot QB and Sidebar into query builder (#74740) 2026-06-10 09:39:58 +12:00
Andrei Chernykh
6fba249a18 Migrate moderation components from Emotion to Mantine (#75447)
* Migrate moderation components from Emotion to Mantine

* Remove orphaned ModerationReviewBanner and QuestionModerationSection

* Rename ModerationReviewBanner to ModerationReviewText and use mt prop for icon margin
2026-06-09 14:05:08 -07:00
Alexander Polyankin
4855069483 Fix regexp QP perf issue (#75548)
* fix perf with qp regex

* test
2026-06-09 13:31:43 -07:00
Mihael Konjević
ec842e146c On-demand skills system for Metabot (#75249)
Co-authored-by: Chris Truter <chris@metabase.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-09 20:25:36 +00:00
Chris Truter
92d89811b3 feat(search): retune ranking — boost exact, make library opt-in (#75396) 2026-06-09 22:23:29 +02:00
Timothy S. Dean
e67f0a194a Connection credentials != intent: remove behavioral switches based on connection-type (#75346)
Principles:
1. All database operations should use the default (read) credentials, unless the programmer explicitly upgrades to write credentials.
2. Upgrading to write credentials should not implicitly upgrade downstream code (write-connection should also be able to read, and read operations shouldn't change to write -- or be expected to).

Scenario:
- Refreshing a cached model performs three data-warehouse operations, two of which *require* writes. Its body was wrapped in `with-write-connection`.
- Separately, the impersonation native-query validator was changed to require a different statement type based on which credentials were active. This broke impersonated native-model caching, as the validator rejected the model's SELECT for not being a write.
- `with-default-connection` was introduced to locally downgrade back to read credentials, to patch this up.

Because we don't ever want to accidentally escalate credentials, the system should be rock-solid simple to reason about, and downstream code changing its behavior based on which credentials happen to be active "because it can" is *not that*.
2026-06-09 14:10:54 -06:00
Cam Saul
d5f19b8634 Remove the long-deprecated describe-table-fks method (#75267)
* Remove the long-deprecated `describe-table-fks` method

* Fix typo

* Kondo lint fixes 🔧

* Fix self-require

* Fix accidental use of `driver` alias in `metabase.driver`

* Fix compilation

* Remove unused namespace

* Test fix 🔧

* Test fixes 🔧

* PR feedback & cleanup

* Test fix 🔧

* Test fix 🔧

* Make sure implementations of `describe-fks` support missing options

* Fix docstring

* Snowflake test fix 🔧

* Snowflake test fix 🔧
2026-06-09 13:09:21 -07:00
Alexander Polyankin
c45ee5937e [main] Change missing template tag warning to trace (#75530)
* Change missing template tag warning to trace

* fix
2026-06-09 13:07:16 -07:00
github-automation-metabase
d305bd7b1b fix(e2e): wait for collection API response in cross-version verify tests (#74773)
fix(e2e): wait for collection API response before asserting on entries in cross-version tests

The cross-version verify specs were flaky because cy.findAllByTestId('collection-entry-name')
would fire before the collection items API response was received, hitting the default 4000ms
Cypress retry limit under CI load.

Instead of increasing timeouts, this adds a helper that intercepts the GET /api/collection/root/items
request and waits for it to complete before asserting. This ensures the collection table is fully
rendered before any entry lookups.
2026-06-09 19:22:03 +00:00
Nick Fitzpatrick
95fa251fa6 Uxw 4224 simplify actions and tabs in table details sidebar in data (#75202)
* update TableSection button layout, remove icons from tabs

* move Schema Viewer menu item

* Design Feedback

* e2e and unit test adjustments

* Hide Shame
2026-06-10 00:51:05 +05:30
Sloan Sparger
d80908dbd8 [Metabot] Remove unused metabot REPL (#75518) 2026-06-09 19:18:08 +00:00
Chris Truter
ad23a60790 feat(search): split weight profiles and normalize search contexts (#75388) 2026-06-09 20:51:45 +02:00
Nemanja Glumac
eae3a57f82 Support single-backport escape hatch (#75503) 2026-06-09 20:50:06 +02:00
Chris Truter
701ae37562 Don't resolve inactive tables in portable FK resolution (#75397) 2026-06-09 18:31:32 +00:00
Andrei Chernykh
e81e4c8c9d Show clearer permission state on inaccessible metric pages (#75068)
* Show clearer permission state on inaccessible metric pages

* Update read-only metric e2e test to assert the new permission screen

* Detect metric view permission from the query result, not can_run_adhoc_query

* Lighten MetricPageCard doc comment

* Restore read-only metric test: read-only users can view, editing restricted

* Extract setupForbiddenCardEndpoints test helper for forbidden card mocks
2026-06-09 11:16:22 -07:00
Chris Truter
685421c95a Support query_handle consistently across MCP tools (#75403) 2026-06-09 19:16:27 +02:00
Chris Truter
1cd718fd8e Missed spots with deprecating /mcp route (#75500) 2026-06-09 17:16:07 +00:00
Anderson
a1144c0aa4 Metrics Viewer Dimension Picking Experience Improvements (#74618)
* Metrics viewer: move column picker to viewer controls and remove tabs (#74545)

* Add column picker to metrics viewer controls

* Remove column tabs

* Remove unnecessary mock / Avoid complex relative imports

* Metrics viewer: add column labels visibility toggle (#74562)

* Add metrics viewer column labels menu

* simplify imports

* Implement sidebar dimension picker for metrics viewer (#74627)

* Implement metrics viewer sidebar dimension picker

* Improve sidebar dimension picker accessibility

* Tighten sidebar dimension picker types

* Update metrics viewer dimension icons

* Use available css variables instead of defining raw colors

* Break down/refactor DimensionPickerSidebar

* Implement a few other style improvements / refactoring

* Use accordions to group tables and columns on the sidebar

* Allow accordions to be expanded/collapsed more freely / Fix test

* Avoid some prop drilling / additional clean up

* Fix same-category column select not working

* Address remaining review comments

* Update metrics viewer control styles (#74653)

* Implement metrics viewer sidebar dimension picker

* Improve sidebar dimension picker accessibility

* Tighten sidebar dimension picker types

* Update metrics viewer dimension icons

* Use available css variables instead of defining raw colors

* Break down/refactor DimensionPickerSidebar

* Implement a few other style improvements / refactoring

* Use accordions to group tables and columns on the sidebar

* Allow accordions to be expanded/collapsed more freely / Fix test

* Update metrics viewer control styles

* Fix metrics viewer control hover styles

* Fix a couple bugs on the metrics viewer dimension sidebar state (#74732)

* Fix metrics viewer dimension sidebar state

* Improve getDimensionItemKey

* Refactor metrics viewer code to remove "tab" naming (tabs do not exist anymore) (#74729)

* Refactor metrics viewer dimension breakout naming

* Address remaining 'tab' references

* Make metrics viewer column labels static (#74735)

* Refactor metrics viewer dimension breakout naming

* Make metrics viewer column labels static

* Align column label spacing with designs

* Fix issues with the Metrics Viewer dimension picking sidebar column selection (#74882)

* Fix metrics viewer sidebar field lists

* Fix a couple more issues, clean up

* Show disabled metric pills (#74870)

* Fix metrics viewer sidebar field lists

* Fix a couple more issues, clean up

* Show disabled metric pills

* Code clean up

* Add "No breakout" option to the dimension picking sidebar (#74932)

* Fix metrics viewer sidebar field lists

* Fix a couple more issues, clean up

* Add metrics viewer no breakout option

* Code clean up

* Narrow down useMemo dependencies in MetricsViewerDimensionBreakoutContent

* Update e2e tests for the Metrics Viewer (#74939)

* Fix metrics viewer sidebar field lists

* Fix a couple more issues, clean up

* Add metrics viewer dimension picker e2e tests

* Fix a few tests and restore removed tests

* Do another round of test fixes

* Stabilize metrics viewer E2E coverage

* Address review comment

* Address issues with the new dimension picking UI (#74996)

* Fix sidebar dimension category grouping

* Fix dimension picker See all mapping

* Prevent unmatched See all dimension autofill

* Disable expression pills without mapped dimensions

* Fix blank chart viewer container when all values in a series are 0

* Fix expression pill disabled state / Fix a couple bugs on dimension matching

* Fix bug on metric expression column matching on the sidebar

* Shows expression circle icon on sidebar / Fix accordions not showing up for metrics selected more than once in the formula input

* Show list of columns under category rows instead of column Select element for single-metric view

* Center controls without taking ellipsis menu into consideration

* code clean up

* Fix metrics viewer type checks

* Address review comment

* Refactor metrics-viewer components to resolve over-propping / code clean-up (#75161)

* Refactor metrics-viewer components to use context, reducing over-propping

* Additional refactoring: reorganize hooks, move utility functions, clean up

* Address review comments

* Address most of Kevin's comments

* Fix tests for new hooks and fix ts check

* Fix issues when switching dimension categories / Keep column labels state as we select different dimensions

* Avoid url to grow indefinitely when selecting dimensions

* Fix failing e2e test

* Address review comments

* Fix remaining failing e2e tests
2026-06-09 14:15:02 -03:00
Ryan Laurie
74ca24eb58 Update Backport Reminder (#75504)
* auto pass backport label check for non docs prs

* update backport reminder

* can't change the name

* test docs change

* remove docs change

* ok fine nemanja

* add concurrency limit
2026-06-09 17:04:37 +00:00
Chris Truter
8c71b27b77 Report where MCP saves questions; default to personal collection (#75491) 2026-06-09 17:00:16 +00:00
Chris Truter
fb3416c0c1 feat(search): add a search context enum, default it to api (#75385) 2026-06-09 16:48:11 +00:00
Aleksandr Lesnenko
b7bdfdce6e add admin transforms redirect routes (#75497) 2026-06-09 12:32:54 -04:00
Maksym Yakubych
f88e12ec68 feat: add copy button to metabot code snippets (#74720)
Closes [BOT-1571: Add copy button to Metabot code snippets](https://linear.app/metabase/issue/BOT-1571/add-copy-button-to-metabot-code-snippets)

### Description

Adds a button to copy and embed code snippets.

### How to verify

1. Open Metabot sidebar.
2. Type `Create plain markdown code block without using tools. Write SELECT from accounts repeated 100 times`
3. Send message.

### Demo

![image.png](https://app.graphite.com/user-attachments/assets/eebb4810-fd75-4d55-8607-3339009904e7.png)



### Checklist

- [x] Tests have been added/updated to cover changes in this PR
- [x] If adding new Loki tests: they pass [stress testing](https://github.com/metabase/metabase/actions/workflows/loki-stress-test-flake-fix.yml)
2026-06-09 12:12:24 -03:00
bryan
2aeb340822 GHY-3709 fix Redshift workspace destroy on foreign-grantor default privs (#74919)
* GHY-3709 fix Redshift workspace destroy on foreign-grantor default privs

Workspace destroy was failing with "user cannot be dropped because some
objects depend on it / privileges for default privileges on new relations
belonging to user <grantor> in schema <input-schema>". Two underlying
problems:

1. Destroy-side discovery used `svv_relation_privileges` which only surfaces
   actual relation grants -- not `pg_default_acl` entries. Schemas whose only
   reference to the iso-user was a default-priv row got skipped, leaving the
   row in place. `DROP USER` then failed.

2. Even when the schema was discovered, the REVOKE was issued without
   `FOR USER <grantor>`. `pg_default_acl` rows are keyed by grantor, not
   grantee -- so the no-`FOR USER` form only targets rows owned by
   `current_user`. Rows owned by another role (`stats_admin` in prod) were
   never revoked.

Fix:
- `default-acl-grants-for-user` queries `pg_default_acl` directly and emits
  `(grantor, schema)` pairs referencing the iso-user. Uses
  `array_to_string(defaclacl, ',')` since Redshift refuses `aclitem` casts
  and lacks `aclexplode`.
- Destroy iterates pairs and emits
  `ALTER DEFAULT PRIVILEGES FOR USER "<grantor>" IN SCHEMA "<schema>"
  REVOKE ALL ON TABLES FROM "<iso-user>"`.

Prevention (Redshift-side mirror of #74909):
- Grant-time pre-flight `assert-can-alter-default-privileges!` rejects
  schemas whose relation-owners or pre-existing default-priv grantors cannot
  be impersonated by `current_user`. Redshift has no working
  `pg_has_role(...)` overload on RA3, so impersonation collapses to
  "owner == current_user OR current_user is superuser". Verified live in
  REPL.

Added test skeleton for the foreign-grantor case; body to follow.

* Add driver-level tests for GHY-3709 preconditions and destroy

Redshift (modules/drivers/redshift/test/metabase/driver/redshift_test.clj):
- workspace-precondition-alter-default-privileges-test: mirrors PG's
  precondition test. Creates a non-superuser tenant role and a foreign-
  owned table; asserts the pre-flight raises with status 412 and clears
  after reassignment.
- workspace-precondition-foreign-default-acl-grantor-test: seeds a
  pre-existing pg_default_acl row owned by a foreign grantor and asserts
  the pre-flight catches it -- this is the prevention half of GHY-3709.

PostgreSQL (test/metabase/driver/postgres_test.clj):
- workspace-destroy-survives-foreign-grantor-default-priv-test: regression
  guard. PG sidesteps GHY-3709 because destroy uses DROP OWNED BY, which
  removes default-priv ACL entries where the iso-user is the grantee
  regardless of grantor. The test seeds a foreign-grantor default-priv
  row targeting the iso-user, runs destroy, and asserts the iso-user is
  gone.

Driver:
- Promote `assert-can-alter-default-privileges!` in the Redshift driver
  from private to public so the test ns can call it directly, matching
  the PG driver's visibility on the equivalent fn.

Removed the placeholder skeleton in driver_isolation_test.clj that the
prior commit added -- the Redshift coverage now lives in the
driver-specific test ns next to the existing precondition tests.

All three tests verified live against a real Redshift and Postgres cluster
via REPL before commit.

* Address roborev: stronger test cleanup + randomized PG workspace id

- Redshift tests: add `DROP OWNED BY` for each foreign-grantor role before
  `DROP USER` in the finally blocks. The whole point of these tests is the
  case where DROP USER would fail on lingering pg_default_acl entries, so
  cleanup needs belt-and-suspenders to prevent role accumulation if the
  destroy assertion or the test itself ever regresses.
- PG destroy regression test: switch workspace id from hardcoded
  0xdeadbeef to `(rand-int Integer/MAX_VALUE)`, matching the random-
  suffix convention used by surrounding tests in the same file.

Both still pass live.

* Rename current-user-superuser? -> current-user-usesuper?

Mirror the Redshift catalog column (pg_user.usesuper) rather than the
ambiguous "superuser" label, which already has a load-bearing meaning
elsewhere (cluster admin role).

* Address roborev branch-level findings

- LIKE escape in `default-acl-grants-for-user`: iso-user names always contain
  `_`, which LIKE treats as wildcard, so the unescaped pattern could match
  sibling users' default-priv rows. Added `escape-like-pattern` and `ESCAPE`
  clause. Pipe (`|`) chosen as escape char because Redshift's parser rejects
  `ESCAPE '\\'` as an unterminated string literal.

- `current_user` column-keyword fragility: replaced
  `(:current_user (jdbc/query "SELECT current_user"))` with an aliased
  `SELECT current_user AS me` and `(:me row)`. Removes dependence on the
  JDBC adapter's identifier mangling for a reserved word.

- Widen `relkind` set in `relation-owners-in-schema` from `('r','v')` to
  `('r','v','m','p','f')` to cover materialized views, partitioned tables,
  and foreign tables -- the same scope ALTER DEFAULT PRIVILEGES ... ON TABLES
  affects. Spectrum externals live in svv_external_tables and remain
  excluded; comment added.

- Split destroy batch: the per-grantor default-priv REVOKEs now run as
  individual statements with per-statement try/catch and warn-level logging.
  Previously, a single bad ALTER (e.g. row pointing at a manually-dropped
  schema) would abort the whole batch and leave the iso-user undroppable.

- Test cleanup: `try-execute!` now logs at warn level instead of silently
  swallowing. CI re-runs against transient cleanup failures will now leave
  a trace instead of accumulating orphan roles invisibly.

All four new fns + both new tests verified live against Redshift after the
changes.

* Address roborev: LIKE first-element bug + PG cleanup logging

- Critical bug fix in `default-acl-grants-for-user`: verified live that
  Redshift's `array_to_string(aclitem[])` strips enclosing braces, so the
  output of a row with a single grantee is `name=privs/grantor` -- with no
  leading delimiter. The prior LIKE alternation (`,name=%` OR `{name=%`) only
  matched mid-array entries; single-grantee rows (the common case) were
  silently skipped. Without this, GHY-3709 would have been "fixed" only for
  multi-grantee rows and would have recurred on every typical workspace
  destroy.

  Replaced with a single LIKE that prepends `,` to the haystack:
  `(',' || array_to_string(d.defaclacl, ',')) LIKE ',name=%' ESCAPE '|'`.
  Both single- and multi-grantee rows now match. Verified live with a real
  single-grantee fixture; sibling-with-underscore-substring still does NOT
  match (escape still effective).

- PG cleanup logging: replaced `(catch Throwable _ nil)` in the destroy
  test's finally block with `(catch Throwable t (log/warnf ...))`. CI re-runs
  now surface orphan iso-users instead of masking them behind a downstream
  schema-drop failure.

All tests still green live.

* Update Redshift precondition tests for new error message

Master's PR refactor of raise-unmemberable-default-priv-owners! changed
the error message from one containing 'ALTER DEFAULT PRIVILEGES' to
'not a member of N role(s)'. Update both Redshift precondition tests'
thrown-with-msg? regexes to match, mirroring the PG counterpart at
test/metabase/driver/postgres_test.clj:2557.

* Add Redshift destroy-survives-foreign-grantor integration test

Mirrors the PG counterpart workspace-destroy-survives-foreign-grantor-default-priv-test.
Reproduces GHY-3709 end-to-end: provisions an iso-user, seeds a pg_default_acl
row owned by a foreign role (CREATEUSER + AUTHORIZATION pattern), runs the
real destroy-workspace-isolation!, and asserts DROP USER succeeds.

This is the destroy-path coverage that was missing — the existing
precondition tests only exercise assert-can-alter-default-privileges! at
grant time. This test catches a transaction-aborted regression on the
destroy side if the per-statement REVOKE loop ever poisons its enclosing
JDBC transaction (PG SQLSTATE 25P02).

* Address @ericnormand: autocommit per-grantor REVOKE before destroy tx

Redshift has no SAVEPOINT and aborts the entire transaction on the first
error. The previous shape ran per-grantor REVOKEs inside the destroy
`with-db-transaction`; on any error the tx was poisoned and the
following batch (relation REVOKEs, DROP SCHEMA, DROP USER) silently
rolled back. The catch-and-log was a no-op for cleanup intent.

Fix: split the per-grantor REVOKE loop out into its own auto-commit
phase (jdbc/execute! on the pooled spec, no enclosing tx) BEFORE opening
the destroy transaction. Each REVOKE either commits or fails
independently; one stale row no longer poisons DROP USER.

The main batch stays transactional -- relation REVOKEs + iso-schema
REVOKE + DROP SCHEMA + DROP USER is the cleanup we want atomic.
2026-06-09 09:02:29 -06:00
Nemanja Glumac
6f932522d6 Auto minor releases for every supported major version (#75251)
Auto minor releases for every supported major version.

Schedule-minor now releases a minor for every supported major line, not
just the current one, so the branches we backport into don't go stale.

It reads the supported majors from `major_version_support` in
_version-info.json_ via `getSupportedMajors` — the same source the
auto-backport workflow uses to pick its targets — and dispatches
release-auto.yml once per supported major.

Patch scheduling is left untouched for this first pass.

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-09 16:18:21 +02:00
Nemanja Glumac
530bae6edc Fix flaky test: warn about unsaved changes when navigating away while save modal is open (#75482)
Wait for Mantine's focus trap to move focus inside the leave-confirm modal
before pressing Escape, so the realPress keydown lands on the focus-trapped
content where the closeOnEscape handler lives instead of being lost during the
open transition.

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-09 14:01:15 +00:00
Nemanja Glumac
5183ebbc86 Auto-backport to supported releases (#74969)
* Derive auto-backport targets from version-info.json release_lines

The `backport` label now fans out to every supported major line, read from
version-info.json: a line is a target iff its eol is todayor later
(lts is display-only and ignored here). Targets fan out via a
fail-fast:false matrix so one cherry-pick conflict can't block the others.

double-/triple-backport keep their original CURRENT_VERSION-based depth
behavior, verbatim, in a separate legacy job at the end of the file as a
short-term manual escape hatch.

* Extract supported-majors logic into a unit-tested release helper

auto-backport.yml builds the release scripts via prepare-release-scripts
and calls the helper, so the "which majors are supported" rule has one tested
implementation instead of being duplicated inline.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-09 15:16:31 +02:00
Chris Truter
c81156251c Fix formatDurationLong rounding at unit boundaries (#75474)
## Summary

Fixes a rounding edge in `formatDurationLong`.

The seconds branch formatted via `(durationMs / 1_000).toFixed(1)`, which could round-up to the boundary value we just checked, e.g.

```
formatDurationLong(59_999) // → "60.0s" 
formatDurationLong(60_001) // → "1m 0s" 
```

## Fix

Round the duration to the tenth-of-a-second *before* choosing a branch, so values at th boundary promote to the next unit. The same class of bug existed at the hour boundary, which this also fixes.

## Origin

Introduced in #75102 (`[GDGT-2503] add sorting by duration to transform runs list`)

## Tests

Added boundary cases to `time.unit.spec.ts`; full suite passes and `type-check-pure` is clean.
2026-06-09 14:40:44 +02:00
Timofey Kachalov
df165c37d3 Fix broken parameters api snippets (#75475) 2026-06-09 16:39:04 +04:00
Kamil Mielnik
b324b2bbc8 GDGT-2504 Replace deprecated Badge component (#75114) 2026-06-09 12:19:13 +00:00
Chris Truter
e7ae478c43 Setting to remove recent views from Metabot context (#75470)
Co-authored-by: Mike Appleby <86076+appleby@users.noreply.github.com>
2026-06-09 11:55:05 +00:00
Stas Gavrylov
94ce9b37df Rename App to AppComponent to resolve import conflict with app.js (#75473) 2026-06-09 11:52:41 +00:00
Stas Gavrylov
f716e395a4 Migrate app routes to TypeScript (#75323)
* Migrate app routes to TypeScript

* Cleanup
2026-06-09 13:56:53 +03:00
Nicolò Maria Mezzopera
301179ef9b Add frontend skill patterns from FE PR review feedback (#72689)
* Add frontend skill patterns from Gadget PR review feedback

Codify recurring review feedback from the last 2 months of Gadget
frontend PRs into shared skill patterns so Claude can apply and enforce
them while writing and reviewing TypeScript code.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* Add error/loading, Mantine, i18n, security, bundle-size patterns

Extend frontend skill patterns with themes surfaced by aggregating PR
review feedback across all 29 frontend engineers over the last 3 months.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* Address PR review feedback: restructure skills, remove duplication

- Inline typescript-patterns into typescript-write, delete shared file
- Merge testing-patterns into e2e-test-create, delete shared file
- Move Mantine/i18n guidance to frontend/CLAUDE.md
- Remove duplicate review checklist from typescript-review
- Remove too-specific examples (custom viz, metabase-lib)
- Remove Cypress line from react-redux-patterns
- Add non-strict null comparison rule (egorgrushin)

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* Address second round of PR feedback and fix incongruences

- Revert unstructured blob in e2e-test-create; surgically insert helpers
  into Phase 0, intercept guidance into Phase 3, Cypress.env into
  "What NOT to do"
- Remove redundant "Before you submit" checklist from typescript-write
- Fix frontend/CLAUDE.md tech stack line (CSS Modules > Emotion → Mantine > CSS Modules)
- Add etc. to style props list, remove translation-branch-in-tests line
- Deduplicate "do not derive loading from data" in react-redux-patterns
- Add "check helpers first" preamble to Intercepts example

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* Update Claude skills and guidelines

* Address PR comments

* Updates following PR comments

* Address feedback

* Fix path typo
2026-06-09 13:29:54 +03:00
Ben Grabow
5b2a4365f1 When tables are created in a batch, assign IDs deterministically (#75365)
* When tables are created in a batch, assign IDs deterministically

* Update IDs to match new sort order

* Use a specific table ID instead of assuming that n+1 is valid.

* This test creates a transform targeting the sorted-first schema/table. Update the assertion to expect Schema A, which now sorts first.

* Reorder to match new entity sort order

* Parameterize a hard-coded table id
2026-06-09 05:25:35 +00:00
Nick Fitzpatrick
be3475c670 Send expandedIds when clicking on a table collection in table view of Data Studio (#75404) 2026-06-09 09:22:07 +05:30
Nick Fitzpatrick
6a3b4bd207 Don't move already published tables (#74977)
Don't move alraedy published tables
2026-06-09 09:20:48 +05:30
Nick Fitzpatrick
49b7aa0f57 Unpublish fk linked tables with archiving a collection with published tables (#75046) 2026-06-09 09:20:20 +05:30
Maz Ameli
792592ff00 Restyle the new-device login email with the modern card template (#75246)
Restyle new-device login email with the modern card template

Rewrite login_from_new_device.hbs to mirror transform_failed.hbs (shared
_dashsub_alert_header.hbs partial, centered logo, rounded card, footer),
reshape the context passed by send-login-from-new-device-email! to supply
the namespaced context/payload vars the shared markup expects, and switch
to send-email-with-logo! so custom logos embed via cid. Also simplify the
body copy (no Metabase hyperlink, single greeting paragraph) and update
the session test assertions to match.

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-06-08 17:33:22 -07:00
Alex Yarosh
6f5582f62c docs: Update docs autogen workflow (#75449)
Update default target branch for documentation release
2026-06-09 00:14:58 +00:00
Andrei Chernykh
640d666809 Migrate EmptyState component off Emotion to a CSS module (#75420) 2026-06-08 17:09:51 -07:00
github-automation-metabase
992b2195ea Auto-update documentation for master (#75438)
Update generated docs (master)

Co-authored-by: github-actions <github-actions@github.com>
2026-06-08 20:02:28 -04:00
Egor Grushin
787d9a65be GDGT-2555 Replace deprecated CheckBox with metabase/ui Checkbox (#75372)
* GDGT-2555 Replace deprecated CheckBox with metabase/ui Checkbox

Migrate all remaining usages of the custom common/components/CheckBox to
the Mantine-based Checkbox from metabase/ui, and delete the deprecated
component.

Migrated call sites:
- EntityItem / ItemsTable collection table (per-row select, select-all header)
- ActionCreator FormFieldEditor "Show field" checkbox
- Timeline questions sidebar (TimelineCard, EventCard)
- Enterprise audit table (AuditTableVisualization)

* adjust the checkbox's label color and size

* fix tests

* fix test with a new checkbox
2026-06-09 00:10:57 +01:00
Alexander Polyankin
e839894faa Make workspace db updates in a transaction (#75413) 2026-06-08 23:03:50 +00:00
Chris Truter
e319703c94 Make defaulted defendpoint params Optional in OpenAPI (#75406) 2026-06-08 22:18:23 +00:00
Nemanja Glumac
a866620984 Pilot: self-healing e2e tests (/fix-flaky-test command) (#75434)
Add /fix-flaky-test pilot command for self-healing e2e tests

A bounded, human-supervised loop that diagnoses a quarantined flaky e2e
test, proposes a fix on a branch, and verifies it with CI stress runs
(burn_in=50, fail_fast, network throttling on/off). Pilot, not autonomous:
a human approves before any push and the resulting PRs never auto-merge.
2026-06-09 00:13:13 +02:00
Nathan Voxland
afa196ae00 Fix password-disable when using OIDC (#75361) 2026-06-08 17:02:17 -05:00
metamben
d7b6f616a9 Honor report-timezone in native field filters (#75029)
Closes QUE2-609
2026-06-09 00:59:06 +03:00
Nemanja Glumac
7c0b9e43e2 Fix flaky homepage x-ray save test (#75424)
The "Save this" button on automatic (x-ray) dashboards rendered enabled as
soon as the header was visible, regardless of whether the dashboard had
finished loading. Clicking it before `dashboard` was populated fired a
false `x-ray_saved` analytics event and no-opped `save()`, so no
`POST /api/dashboard/save` request was ever sent.

* Disable the button until `dashboard` is loaded.

* Assert x-ray save via the saved-state UI, not the request alias

cy.wait("@saveDashboard") flaked with 'No request ever occurred' because
it depends on catching the POST in a timing window. Assert the real
user-observable outcome instead: after clicking 'Save this', the header
switches to a 'Saved' button + 'See it' link. Cypress retries that until
the save lands, which (with the disabled-until-loaded guard) is robust.

* Wait for x-ray dashboard readiness (param URL sync) before saving

the dashboard isn't in the store yet. useDashboardUrlQuery syncs the dashboard
params into the URL query string once it's loaded, so a populated location.search
is a reliable readiness signal. Clicking Save before then loses the save.

---------

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-08 23:30:25 +02:00
Andrei Chernykh
295d050934 Convert sandboxes ParameterTargetWidget and container to TS (#75330)
* Convert sandboxes ParameterTargetWidget and container to TS

* Fix typecheck: bridge loose ValueType to ParameterTarget at the widget boundary

* De-flake TablePicker tree-view test by waiting for the first table

* Drop HOC cast and default export, tidy ColumnPicker widget bridge

* Replace sandbox DimensionRef with ParameterTarget and drop the casts
2026-06-08 14:18:27 -07:00
Fraser
e3f84bfc43 Fix AIProvider config violations (#74888) 2026-06-09 09:15:40 +12:00
Nemanja Glumac
6134f7a7ca Pre-resolve all CI alias combos in cache generator (#75258)
The cache generator resolved only one classpath
(`:build:dev:ee:ee-dev:drivers:drivers-dev:cljs`), but CI jobs run other
alias combinations. Because tools.deps selects dependency versions
globally per classpath, those combos resolve some transitive libs to
versions the generator never cached (e.g. data.json 2.4.0 from the test
classpath, asm 9.2 from shadow-cljs's `:cljs`-only classpath). Those
deltas were fetched Maven Central / Clojars direct on every run, which
intermittently 403s from GitHub's shared runner IPs.

Resolve every distinct alias combination CI uses (one `clojure -P` per
combo) so the saved cache is a superset of what every job needs, and
jobs stop hitting upstream registries for these deltas.

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-08 22:46:59 +02:00
technomancy
943ce3d2b2 Health inspector (#74681)
* WIP health inspector.

* Table and routes for health inspector.

* Add API routes for health inspector.

* Address review suggestions.

* Add delay in between validating each query.
* Check for empty results before trying to divide by zero.
* Formatting

* Address review suggestions.

* Rename *delay* -> *delay-ms*
* Fix migration comment
* Parameterize limit
* Select newest runs first
* Switch to counting valid queries using reduce instead of map

* Move health inspector migration to version 63.

* Add a setting and jitter to health inspector reporting job.

Lower thread priority.

* Move settings into separate ns; sleep as long as validation took.

* Add an init namespace; needed to pass metabase.core.init-test.

* Don't rerun health inspector job on misfire or allow concurrent execution.

* Delete unused delay-ms var.

For some reason the linter didn't catch that this was unused! That's
an annoying bug.

Also use `int` instead of `integer` for health in the DB schema.
2026-06-08 13:40:53 -07:00
technomancy
649b0e2b00 Mark snowflake as not having :transforms/accurate-rows-affected. (#75421)
Similarly to bigquery and redshift, the :rows-affected of the result
is incorrect when doing CTAS operations. We need to mark it as such so
that it will use the fallback COUNT-based approach to get the real number.
2026-06-08 13:32:26 -07:00
Romeo Van Snick
84a31f735c Move metabase/auth to the feature tier (#75375)
* Move auth redux actions into shared redux/auth slice

The session/auth actions (login, logout, refreshSession, etc.) are
app-wide concerns used by nav, metabot, and enterprise auth — not part
of the auth feature UI. Co-locate them with the existing auth reducer in
metabase/redux/auth, matching the convention used by other redux slices
(e.g. redux/user). The LoginData payload type moves alongside the login
thunk.

This removes all cross-module imports of metabase/auth/actions, which is
a prerequisite for moving auth to the feature tier (shared/feature
modules may not import from a feature module).

Pure refactor: no behavior change.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* Move auth to feature tier

auth is a user-facing feature (login/logout pages, password reset, auth
provider buttons), not shared infrastructure. Reclassify it from the
shared tier to the feature tier in the module-boundary lint config.

All session/auth redux actions previously imported across modules now
live in the shared metabase/redux/auth slice, so no shared or feature
module imports from metabase/auth anymore. The strict module-boundaries
lint adds zero new violations.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-09 06:30:22 +10:00
Andrei Chernykh
3d0713ce21 Update API keys admin page UI (#75170)
* Modernize API keys admin page with standard empty state, table, and modals

* Polish API keys table layout and modal styling per design

* Add row-menu hover affordance and fix secret-key padding and dark-mode colors

* Fix admin-route and settings tests for new API keys copy

* Add row-click-to-edit, ActionIcon copy button, and align Create/Edit API key modals with the designs

* Align Regenerate API key modal with the shared modal style and mark Key name required on Edit

* Make the API keys e2e group selector case-insensitive after the group label rename

* Remove the shadow from the API key info-note Paper and add a TODO for the future Alert
2026-06-08 13:21:09 -07:00
Alexander Polyankin
70bb44ec4d Limit schemas in workspaces (#75405)
limit schemas in workspaces
2026-06-08 16:17:24 -04:00
Alexander Polyankin
f3dcaa059f Filter out stub databases (#75401)
filter out stub databases
2026-06-08 16:17:16 -04:00
Egor Grushin
6edffa6fb4 GDGT-1784 add a loader to the lens tab (#75151)
* GDGT-1784 add a loader to the lens tab to indicate loading while the tab is active

* fix the indicator for fast lenses

* fix test

* Extract renderIndicator into Indicator component

Co-Authored-By: Egor Grushin <egor@metabase.com>
2026-06-08 21:07:35 +01:00
technomancy
0f23c635f7 Split bigquery driver tests into 2 parts like redshift. (#75366)
They time out otherwise. Might be necessary to do 3 parts honestly.
2026-06-08 11:18:13 -07:00
Riley Thompson
650a933a6f Update transitively dependent cards when card database changes (#75268)
* update dependent cards with new db id

* address feedback
2026-06-08 17:47:35 +00:00
Ryan Laurie
b756e6a1ad Update frontend dev guide (#75419)
update dev guide
2026-06-08 11:06:08 -06:00
Eric Normand
ae5ac5a2ba Re-clone remote sync repo if its temp dir was deleted (GHY-3815) (#75414)
* Re-clone remote sync repo if its temp dir was deleted (GHY-3815)

get-jgit cached Git instances keyed by repo path without checking that
the underlying directory still existed. If $TMPDIR/metabase-git was
deleted, the stale cached instance lost its origin remote and every
remote sync operation failed with 'origin: not found' until restart.
Only return the cached instance when its directory still exists;
otherwise re-clone.

* Type-hint local-path to avoid reflection in git test
2026-06-08 17:02:52 +00:00
John Swanson
aa27de3c3d Fix /api/database/:id/schemas when tables have a NULL schema (#74988)
For databases whose tables have Table.schema = NULL  table-level
`:perms/create-queries` grants were being silently
ignored. The endpoint returned an empty 200 even when the user had
query-builder permission on individual tables.

`schema-permission-for-user` was normalizing its `schema-name` argument
from nil to "" before filtering data_permissions rows. But we weren't
normalizing the other side of the `=` - the
`data_permissions.schema_name`. When this was stored as `NULL`, the
comparison failed.

Note that for mysql, both `:model/Table` and `:model/DataPermissions`
store `NULL` for the schema, triggering this bug - we want the schema
permissions for `NULL` but end up checking the schema permissions for
`""`. There aren't any data permissions for that schema name, so we fall
back to the least permissive permission value and `can-read-schema?`
returned false.

`NULL` and `""` schemas can be treated as functionally equivalent - in
general a single database will *either* have `NULL` or `""` values, so I
think it would be safe to just remove the coercion from both sides of
the equality here (like `(= schema-arg schema-from-data-perm)`), but it
should also be safe to just add coercion to *both* sides of the equation
(like `(= (or schema-arg "") (or schema-from-data-perm ""))`), which is
what I've done here.
2026-06-08 16:28:11 +00:00
Anderson
5bf39260ba Fix picker search fired twice on metrics viewer page load (#75290)
Fix duplicate metric picker search
2026-06-08 13:23:11 -03:00
Andrei Chernykh
04b98c1501 Show value preview for metrics defined on a model (#75169) 2026-06-08 09:03:18 -07:00
Jeff Bruemmer
e6132a0ac6 docs - parameter dropdown labels (#75338) 2026-06-08 11:27:11 -04:00
Oleh K
53baf39e82 Add new color tokens alongside existing ones (#75227)
* GDGT-2529: Add new color tokens alongside existing ones

Foundation for the GDGT-2517 color token refactor. Adds new and
renamed semantic tokens without changing any existing emission or
call sites. After this PR, both old and new CSS variable names
resolve to the same values; migration sweeps will move consumers
to the new names, and the cleanup PR will remove the old.

* GDGT-2529: Move kept tokens from Legacy to New section

These 14 tokens are not being renamed (text-primary, text-brand,
border-brand, tooltip-*, switch-off, etc.) and have no equivalent
in the New section. Moving them into the New group prevents
readers from assuming they're discontinued when they encounter
the Legacy section's 'Prefer using new ones' comment.

* Update Loki Snapshots

* Rearrange and aort new colors keys alphabetically

* Add explanation to core-brand/filter/summarize copy

* Update Loki Snapshots

* Split light/dark themes into `deprecatedColors` and `actualColors`

Per review feedback on #75227 — replaces the `// Legacy colors` and `// New`
section comments with named top-level constants so the intent shows up in
the import graph rather than as inline prose.

---------

Co-authored-by: Metabase Automation <github-automation@metabase.com>
2026-06-08 17:23:21 +02:00
Riley Thompson
984d24be26 Handle null timestamps for druid (#75350)
* fix druid nil timestamp

* remove let block

* use when-let
2026-06-08 14:56:28 +00:00
Oleh K
6bfd5cefd6 Enable JUnit hook-failure rewriting for all e2e tests (#75380) 2026-06-08 16:44:46 +02:00
Chris Truter
dd08672815 Run the Data Complexity Score job weekly instead of daily (#75059) 2026-06-08 12:11:46 +00:00
Egor Grushin
7ee95ea5d8 Fix tooltips rendering under popovers and menus (#75373)
Mantine assigns Tooltip, Popover and Menu the same "popover" z-index
tier (300), so a tooltip tied with them and the winner was decided by
portal DOM order. Once tooltips moved into the fixed PortalContainer
near the top of <body>, later-mounted overlays began painting over
them. Bumping the tooltip z-index to popover+1 keeps it in its tier
while always rendering above same-tier overlays, independent of DOM
order.
2026-06-08 12:07:07 +01:00
Nicola Mometto
514ec567ba Relax schema in security center (#75382)
security center: relax semver schema
2026-06-08 10:56:18 +00:00
Mahatthana (Kelvin) Nomsawadi
0eeccf9ae5 feat: add embedding-devs Slack group to github-slack-map (#75384)
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-08 11:54:26 +01:00
Nemanja Glumac
c33fe8afee Make representations repair idempotent for unwrapped boolean wrappers (#75304)
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-08 11:32:42 +02:00
Fraser
a5f3bdb2e2 Fix notifications violations (#75075) 2026-06-08 21:19:05 +12:00
github-automation-metabase
b8bfd206e7 fix: update misleading copy on transforms upsell page (#74938)
- Replace 'No writable database connection' with 'No compatible database
  connection' since the validation checks database compatibility (not
  sample, not audit, supports transforms feature), not write permissions
- Update the permissioned description to show 'Analysts and Admins' for
  all Pro plans, not just pro-self-hosted
- Update body text in NoWritableDatabasesEmptyState to accurately
  describe the requirement

Fixes GRO-421

Co-authored-by: Benoit Vinay <ben@benoitvinay.com>
2026-06-08 09:15:47 +02:00
Fraser
25a4905945 Move models to feature tier (#75212) 2026-06-08 16:13:08 +12:00
Timothy S. Dean
b1b5d4906b Optimize chain filter inner field projections (#75209)
Chain filter built each join's inner stage with no explicit `:fields` projection, leaving `add-implicit-clauses` to fill it in, which on wide tables meant *every single column*. This PR narrows each join's inner projection to exactly the columns the rest of the query references through it.

## Approach

- `with-join-source-fields` sets `:fields` on a join's source subquery; `nil` or empty reverts to implicit-all prior behavior. Modeled after `with-join-fields`, which governs what the join exposes outward to its containing stage.
- `joined-thing` is a re-export of the existing `lib.join` function, surfaced for callers that need to ask whether a join was built against a Table or a Card.

`tighten-join-projections` runs as the terminal pass in `chain-filter-mbql-query`.

Ordering matters. The pass runs after `add-required-filters-if-needed` so that middleware-injected refs (BigQuery partition filters on joined tables, most relevantly) are present in the query before projections are computed. A regression test pins exactly that.
2026-06-07 19:07:00 -06:00
metabase-bot[bot]
c32e5bdaf8 [chore] Update E2E test timings with recent data (#75369)
Update E2E test timings with averaged data

Collected timing data from 7 days of test runs.
All specs with timing data were included (simple averages).

🤖 Generated automatically by update-e2e-timings workflow

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: GitHub Actions <actions@github.com>
2026-06-07 22:43:32 +02:00
Nick Fitzpatrick
c610661092 Hide collection CTA if user doesn't have publish permisisons (#75275) 2026-06-07 09:19:31 +05:30
github-automation-metabase
b47463a1c9 Auto-update documentation for master (#75359)
Update generated docs (master)

Co-authored-by: github-actions <github-actions@github.com>
2026-06-05 17:37:45 -04:00
Riley Thompson
d202231950 Remove failed bigquery fix attempt (#75344)
remove failed fix
2026-06-05 16:57:42 -04:00
Jeff Bruemmer
8674434557 docs - workspace framing (#75351)
Revise workspaces documentation for clarity
2026-06-05 16:30:25 -04:00
Jeff Bruemmer
a623d06a21 docs - custom viz (#75153) 2026-06-05 20:08:18 +00:00
Andrei Chernykh
6cd06168df refactor(embedding): Convert SavedEntityPicker to TypeScript (#75061)
* Convert SavedEntityPicker to TypeScript

* Pass collectionId to SavedEntityList and drop the tree-node cast

* Gate SavedEntityPicker on currentUser and document children override
2026-06-05 11:36:04 -07:00
Cam Saul
b9fd9c4f3d Add test for #68701 (already fixed) (#75291)
* Add test for #68701 (already fixed)

* cljfmt
2026-06-05 11:24:20 -07:00
Cam Saul
a4f27fd2fc Fix zoom in drill for auto binned lat/lon (#75063)
* Fix zoom in drill for auto binned lat/lon

* Test fix 🔧
2026-06-05 11:24:15 -07:00
technomancy
aa55ae49d3 Add locks around Snowflake dataset creation (#74876)
* Make with-temp-test-data use a randomized dataset name.

This isn't a problem on most DBs, but on cloud DBs where everything is
shared, we have to use a random name unless the dataset is immutable
and never deleted.

* Add dataset-lock implementation for Snowflake.

* Use existing random-name helper for temp-test-dataset names.

* Memoize snowflake locks table creation and use connection pooling.

* Fix stale lock check; apparently the difference can be negative.

* Actually use a separate readlock.

* WIP add debugging info for snowflake locks

* Turn readLock into a no-op for snowflake.

A proper readLock here would almost certainly blow out the timeout
budget. The current implementation is just to check that there isn't
an existing writeLock happening, but it doesn't prevent a new
writeLock from happening, and it's very unreliable, with Snowflake
randomly reporting that the table just disappeared briefly.

Disabling it is bad, but it would still leave us with the writeLock
protecting against the worst of the unreliability.

* Retry when Snowflake tells us the table that definitely exists doesn't exist.

* Grant access to locks table; get rid of retry stuff.

This was caused by an errant use of with-redefs that swapped out the
user accessing the locks table randomly based on parallelism.
2026-06-05 10:06:14 -07:00
Nemanja Glumac
728296f25b Send Cypress failure screenshots to ci-conductor (#75283)
* Send Cypress failure screenshots to ci-conductor (DEV-2000)

Attach each failed test's first failure screenshot to the failed-tests
payload so ci-conductor can upload it to S3 and link it to the run.

- Capture in the after:screenshot hook, the only place Cypress exposes
  testFailure + testAttemptIndex; keep exactly the first failure shot per
  test (testFailure && testAttemptIndex === 0), excluding retries and
  manual cy.screenshot() at the source.
- Drain the per-spec buffer in after:spec and match each failing test to
  its shot by normalized title↔filename containment (after:screenshot
  gives a path but no title; name is null for auto-failure shots).
- Read + base64-inline the file at send time as a data URI; ci-conductor
  stores null when absent.

Best-effort throughout: every step is wrapped, logs on failure, and can
never throw into or interrupt the test run.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* Resolve failure screenshots in after:spec, drop the after:screenshot hook (DEV-2000)

The after:screenshot hook was only there for `testFailure` /
`testAttemptIndex`, but both can be recovered from the filename, and
`results.screenshots[].path` in after:spec is the same path the hook
reported. So everything we need lives in after:spec.

Match each failed test to its screenshot by anchoring the normalized
title on the failure marker:

  normalize(basename).startsWith(normalize(title.join("")) + "failed")

Normalizing both sides to [a-z0-9] cancels every Cypress sanitization
rule (">", " -- ", "/", ...) — no reconstruction. Anchoring on "failed"
excludes manual cy.screenshot() shots (which carry no "(failed)") and is
collision-proof against prefix titles; the shortest matching basename is
the first attempt (no "(attempt N)" suffix).

Removes the recordedFailureScreenshots accumulator, recordFailureScreenshot,
takeRecordedScreenshotPaths, and the after:screenshot wiring in config.js —
one hook, no cross-hook state, and strictly more precise matching.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-05 11:00:20 -06:00
Nemanja Glumac
3d13cfe575 Update the org-level CURRENT_VERSION variable (#75288) 2026-06-05 18:30:18 +02:00
Andrei Chernykh
89969fff33 Convert metabase/hoc files to TypeScript (#75070)
* Convert metabase/hoc files to TypeScript

* Use createMockLocation instead of casting in ModalRoute test
2026-06-05 09:02:17 -07:00
Andrei Chernykh
9b6d1ea98e Migrate NewModelOption to TS and move hover styling to CSS (#75130)
* Migrate NewModelOption to TS and move hover styling to CSS

* Use Mantine components and rem in NewModelOption
2026-06-05 09:01:47 -07:00
Andrei Chernykh
d34785dc6e Migrate snippets CollectionRow from JSX to TS (#75138) 2026-06-05 09:01:26 -07:00
Andrei Chernykh
008feef331 Migrate QuestionModerationSection to TS and drop dead unverify path (#75134) 2026-06-05 09:01:05 -07:00
Andrei Chernykh
3378023d91 Migrate ErrorOverview and mode in tools/ from JS to TypeScript (#75072)
* Migrate ErrorOverview and mode in tools/ from JS to TypeScript

* Coerce cardId to number in handleReloadSelected

* Update enterprise/frontend/src/metabase-enterprise/tools/mode.ts

Co-authored-by: Anderson <ands.alves.nunes@gmail.com>

---------

Co-authored-by: Anderson <ands.alves.nunes@gmail.com>
2026-06-05 09:00:44 -07:00
Andrei Chernykh
49251fcf79 Migrate subscriptions sidebar pulse card from Emotion to Mantine (#74999)
Migrate subscriptions sidebar pulse card from Emotion to Mantine and CSS module
2026-06-05 08:58:03 -07:00
Andrei Chernykh
0de49b9ef7 Migrate database engine warning and section field from Emotion to Mantine (#74997)
Migrate database engine warning and section field from Emotion to Mantine and CSS module
2026-06-05 08:57:00 -07:00
Mahatthana (Kelvin) Nomsawadi
43b804d390 feat: add embedding-sdk-tag-latest workflow for npm dist-tag (#75245)
* feat: add embedding-sdk-tag-latest workflow for npm dist-tag

Adds a standalone reusable workflow that sets the `latest` dist-tag on
a published @metabase/embedding-sdk-react version using a classic npm
token (NPM_RELEASE_TOKEN). Trusted publishing (OIDC) cannot set
dist-tags, so this lives in a separate workflow.

Wires the new workflow into release-embedding-sdk.yml as a `tag-latest`
job (disabled via `if: false`). Remove that line when the release branch
becomes gold. Also removes the old commented-out inline dist-tag step.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* fix: quote GITHUB_STEP_SUMMARY in embedding-sdk-tag-latest workflow

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* Cleanup step by not explicitly create .npmrc and rely on setup-node instead

* ci: use secrets: inherit for npm token in sdk tag-latest workflow

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* ci: use DEFAULT_RUNNER_KEY in sdk tag-latest workflow

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* fix: add concurrency guard, permissions, and shell injection prevention to tag-latest workflow

- Add concurrency group `npm-dist-tag-latest` (cancel-in-progress: false) to prevent parallel runs from racing
- Add `permissions: {}` for least-privilege (workflow doesn't use GITHUB_TOKEN)
- Hoist version to job-level env var `VERSION` to prevent shell injection via inputs.version
- Add verify step to confirm version exists on npm before tagging
- Quote package spec in npm commands

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

---------

Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-05 16:53:58 +01:00
Nemanja Glumac
6d37c69bbd GHY-3805: Make check-card dependency response order deterministic (#75300)
`broken-cards-response` built `:bad_cards`/`:bad_transforms` from
`(keys card)` and an `IN` query with no `:order-by`, so the response
order was nondeterministic. This flaked
check-card-hydrates-dashboard-and-document-test, which matches
`:bad_cards` positionally via `=?`.

Achieve the deterministic ordering by sorting `created_at` anchored by `id`.
---------

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-authored-by: Ben Grabow <ben.grabow@metabase.com>
2026-06-05 17:28:22 +02:00
Nick Fitzpatrick
6a679c2eed Add Data Layer column to table, fix update on undo (#75135)
* Add Data Layer column to table, fix update on undo

* covering undo case in e2e
2026-06-05 20:46:33 +05:30
Jeff Bruemmer
f8a9f7fc47 docs - workspaces (#75274) 2026-06-05 14:54:05 +00:00
Jeff Bruemmer
7273730111 workspace modal copy (#74982) 2026-06-05 10:50:56 -04:00
Nathan Voxland
786417140b Clear setup-token after use (#75198) 2026-06-05 09:29:26 -05:00
Uladzimir Havenchyk
215adbd598 chore: replace more deprecated buttons with mantine button (#75179)
* chore: replace more deprecated buttons with mantine button

* update colors

* fix hover state and button size

* address review

* fix segment form button usage

* use same color
2026-06-05 17:08:44 +03:00
Braden Shepherdson
9393fa8c9b [perms] Break up large SELECTs to avoid the 65K SQL parameter limit (#75032)
Fixes UXW-4107.

SQL queries (at least in Postgres) only allow 65,535 `?` parameters. If
you have a huge set of collections being selected while updating
permissions, Metabase was generating a query with
`WHERE id IN (?, ?, ?, ...)` with more than that many parameters.

With this change, the queries are broken up into chunks of 20k ids
each and the results combined in memory.
2026-06-05 10:04:54 -04:00
Braden Shepherdson
f7acc044e4 [perms] Cap perms for audit collection at :read, don't throw on :write (#74240)
Fixes UXW-3396.

Trying to set `:write` perms for a collection including the magic Audit
collections while setting `Include Subfolders` was throwing a 400.

The cause was that it tried to set `:write` perms for the Audit collection,
which threw an error. This change updates it to treat `:write` as `:read`,
so that the permissions update can succeed.

The FE handles `Include Subfolders` by fetching the tree of
subcollections, and it doesn't handle the Audit collection specially.
2026-06-05 10:04:43 -04:00
Chris Truter
bb6fe57d3f Add Prometheus metric for search result counts (#75186) 2026-06-05 09:28:36 -04:00
Mahatthana (Kelvin) Nomsawadi
5f4211360d fix: pin commons-configuration2 2.15.1 in Databricks driver (#75228)
databricks-jdbc-thin 3.3.1 transitively pulls commons-configuration2
2.11.0 which has CVE-2026-45205 (Uncontrolled Recursion, Medium).
Fixed in 2.15.0; 2.15.1 is the latest patch on that line.

Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-05 14:26:18 +01:00
Mahatthana (Kelvin) Nomsawadi
876f2c4e1c fix: pin netty 4.1.135.Final in Snowflake driver (#75229)
AWS SDK (transitive via snowflake-jdbc-thin) pulls io.netty 4.1.126.Final
which has SNYK-JAVA-IONETTY-16438930 (Data Amplification, High) and
SNYK-JAVA-IONETTY-16438322 (Resource Exhaustion, High). Both fixed in
4.1.133.Final.

Pin netty-codec, netty-handler, and netty-transport-classes-epoll to
4.1.135.Final to ensure the full netty family is on a consistent version.

Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-05 14:26:15 +01:00
github-automation-metabase
6db984ab33 Auto-update documentation for master (#75289)
Update generated docs (master)

Co-authored-by: github-actions <github-actions@github.com>
2026-06-05 09:18:24 -04:00
Ngoc Khuat
af6ea737a3 Honor continue_on_error and full_stacktrace on serialization export (#75122) 2026-06-05 19:15:36 +07:00
Nick Fitzpatrick
7a3ea4c788 allow collection creation when publishing tables (#75261) 2026-06-05 17:32:32 +05:30
Nick Fitzpatrick
d9f40d41ad Remove Our Analytics from library snippet header (#75266) 2026-06-05 17:31:57 +05:30
Ngoc Khuat
6884aea553 Alerts management: failing tab includes abandoned + query failures (#75189)
* Alerts management: failing tab includes abandoned + query failures

The failing tab filtered on last_send (channel delivery) only, so alerts
that never reached delivery — query errors, or runs abandoned on instance
restart — showed as "not failing" even though nobody got the alert.

last_check is the whole-run rollup (success/failed/abandoned), already a
superset of last_send. Add a last_check_status filter and point the failing
tab at it. The explicit "last send attempt" filter still filters delivery
only, so check vs send stay distinct.

* Failing tab: sort by last_check and explain abandoned runs

the failing tab filters on last_check but still sorted by last_send, so abandoned and query-fail alerts (which have no last_send) sank to the bottom, the exact ones it's meant to surface. default the tab to sort by last_check.

abandoned runs have no task_history message, so they showed as failing with a blank reason. synthesize an explanation instead.

* small nitpick

---------

Co-authored-by: Egor Grushin <egor@metabase.com>
2026-06-05 17:43:20 +07:00
Kamil Mielnik
81cc40f669 GDGT-2339 Frontend crashes when notification payload is null (#75107) 2026-06-05 10:14:18 +00:00
Nicola Mometto
3d6b88a2a1 prevent removing table tag from existing incremental transforms (#75101)
* fix: stronger validation, reject updates that would break table incrementality

* fix test

* add tests

* add FE confirmation modal

* update kondo config

* handle FE review comments
2026-06-05 10:10:12 +01:00
Romeo Van Snick
a5630332e5 Fix native editor module boundaries (#75168)
* Remove legacy NativeQueryEditor shim

* Make native-editor toggle buttons fully controlled

* Route native-editor parameter reorder through setDatasetQuery

* Move NativeQueryEditor into the shared querying module

* Move createRawSeries to the shared visualizations lib

* Move TagEditorSidebar into the shared querying module

* Compose template-tag query mutations in transform sidebar

Switching a variable's type fires setTemplateTag and setTemplateTagConfig
back-to-back in one handler. Both derived from the same render-captured
props, so the second onChangeQuery clobbered the first and the type change
was lost — breaking the transform variable editor (TableMappingSelect never
rendered, saves produced no toast).

Track the latest query in a ref and compose each mutation on top of it so
sequential updates no longer overwrite each other. Fixes the failing
transforms template-tags and transforms-incremental e2e specs.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-05 15:04:20 +10:00
Ngoc Khuat
5528bf4740 GDGT-304: Fix unthreadsafe number parsing in csv upload (#75119) 2026-06-05 04:51:37 +00:00
YB
72d014454f [GDGT-2370] add efficiency tracking histogram for incremental transforms (#74651)
Add :metabase-transforms/incremental-rows - a histogram surfacing the efficiency of incremental SQL transforms by emitting two observations per run: rows available in the source (lo, hi] checkpoint window vs. rows actually written to the target.

The :full-incremental-run label separates the first-run / config-change population (where the pair measures whole-table scans) from steady-state windowed runs.

Both numbers derive from a single scan: get-source-range-params now adds count to the existing max-watermark aggregation.

There's some existing gaps which we have mitigated as follows:

Python transforms didn't populate :rows-affected in driver/run-transform!'s return shape, so we are deriving that count from the generated output file from the python runner (number of rows equal number of lines in the file - except for the edge case where no rows were generated)

For full incremental transforms (which translate to CREATE TABLE statements), the drivers for Redshift / BigQuery do not return the number of rows that were in the created tables (they follow this spec https://docs.oracle.com/en/java/javase/21/docs/api/java.sql/java/sql/Statement.html#executeUpdate(java.lang.String)) ; we skip emitting metrics for these 2 drivers in full incremental transforms.
2026-06-05 10:25:05 +08:00
YB
a451945286 [GDGT-2548] chore: use shared constant to set empty cell placeholder values in tables/detail fields (#75223)
replace uses of em-dash empty value placeholder in fields and table cells with shared constant
2026-06-05 10:24:06 +08:00
Cam Saul
4eb1408f48 Remove legacy :druid driver / migrate to :druid-jdbc (#75143)
* Remove legacy `:druid` driver / migrate to `:druid-jdbc`

* Add no-op migration rollback

* Update e2e tests

* Remove additional places mentioning removed Druid driver

* Kondo

* Update modules config
2026-06-04 17:28:37 -07:00
Riley Thompson
6945a211ba Use unicode regex for bigquery alias (#75123)
* use unicode regex for bigquery alias

* trigger ci

* fix workspace e2e test
2026-06-04 23:57:43 +00:00
technomancy
a21f6ab613 Disable bigquery from workspace-full-e2e-test so we can unquarantine (#75254)
Temporarily disable bigquery from workspace-full-e2e-test so we can un-quarantine.
2026-06-04 15:21:05 -07:00
Eric Normand
40af443aaa GHY-3804 require superuser for remote-sync current-task endpoints (#75276)
The /current-task and /current-task/cancel endpoints were protected only
by +auth, letting any authenticated user read sync task details and cancel
active sync operations. Add api/check-superuser to both, matching the rest
of the remote-sync API.
2026-06-04 21:08:20 +00:00
thackervishal
bf4bb7ab10 one word typo fix (#75279) 2026-06-04 16:43:48 -04:00
Eric Normand
18899f65e9 GHY-3801 preserve result_metadata for native cards on serdes export (#75259)
* GHY-3801 preserve result_metadata for native cards on serdes export

Native (non-model) cards can't have their result_metadata re-derived at
import time the way MBQL cards can: query->expected-cols computes MBQL
columns statically from synced metadata, but a native query is opaque SQL
that can only be introspected by executing it. GHY-3339 dropped
result_metadata from all non-model exports on the assumption that import
re-derives it; that holds for MBQL but leaves native source cards with
nil metadata, which breaks downstream GUI questions that join them
("Error adding alias info to join...").

Export the full result_metadata for native non-model cards (mirroring the
pre-GHY-3339 transforms that import-result-metadata already inverts),
while keeping the lean model export and the MBQL non-model skip.

* Whitelist exported keys + assert column names (PR review)

- export-result-metadata: switch the native branch from a blacklist
  (dissoc :fingerprint) to an explicit keep-keys whitelist, so internal
  :lib/* and other computed keys aren't serialized — the bloat GHY-3339
  set out to remove. Keeps the structural type info native needs plus the
  display soft-keys and FK transforms.
- e2e test: assert the imported native card's result_metadata column names
  match the source, not just that metadata is non-empty.
2026-06-04 20:28:19 +00:00
Jeff Bruemmer
b9113d6741 docs - CLI page and revisions on agent-driven workflow (#74976) 2026-06-04 20:10:16 +00:00
Egor Grushin
5efc5f7963 GDGT-2365 fix z-index in PythonEditor (#75052)
* GDGT-2365 fix z-index in PythonEditor

* GDGT-2365 add a repro

* addressing the PR's comments

* addressing the PR's comments

* Update e2e/test/scenarios/data-studio/transforms/reproductions.cy.spec.ts

Co-authored-by: Kamil Mielnik <kamil@kamilmielnik.com>

* Addressing PR's comments

---------

Co-authored-by: Kamil Mielnik <kamil@kamilmielnik.com>
2026-06-04 20:59:41 +01:00
Michael Eby
d5aabfebe4 docs: update homepage settings (#75154)
* docs: update homepage settings

* fix broken link

* edits from jeff and alex
2026-06-04 15:29:13 -04:00
Michael Eby
2626b192f5 docs: update metabase store UI paths (#75136)
* docs: update metabase store UI paths

* fix links

* edits from jeff
2026-06-04 15:28:53 -04:00
Michael Eby
947662623a docs: add drill-through page (#74922)
* docs: add drill-through page

* fix broken links

* edits from alex

* add note about drag to select
2026-06-04 15:28:33 -04:00
Nemanja Glumac
bc3401f4b5 Fix job_id resolution: gh rejects --slurp with --jq (#75260)
#75185 switched the lookup to `gh api --paginate --slurp ... --jq`, but gh
refuses `--slurp` together with `--jq` ("the --slurp option is not supported
with --jq") and exits immediately. The usage error was swallowed by
2>/dev/null, so every attempt produced empty output and JOB_ID was never
written to $GITHUB_ENV — making job_id always null in ci-conductor reports
(it was only ever populated before #75185 when the job landed on page 1).

Slurp with gh and run the filter through a standalone jq instead. The slurped
shape is an array of page objects, which the existing `[.[].jobs[] | ...]`
filter already handles. Verified against a live in-progress run.

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-04 21:27:31 +02:00
Eric Normand
706c4c444b GHY-3802 capture extract-phase logs in serialization export.log (#75255)
* GHY-3802 capture extract-phase logs in serialization export.log

The serialization export endpoint hoisted `extract/extract` out of the
`for-ns` log-capture block so invalid-input errors could still set a
non-200 HTTP status. But escape-analysis warnings (content referencing
entities outside the requested collections) are emitted eagerly during
`extract`, so they were logged outside the capture and never reached
`export.log` — leaving customers with a valid archive and an empty log.

Capture the eager extract logs into the same buffer that becomes
`export.log`, and write that buffer after the storage appender has closed
(and flushed). Add a regression test asserting the escape warning lands in
`export.log`.

* kick ci

* Rename serialization-log-nses to serialization-logger-prefixes

These are log4j2 logger-name prefixes, not loaded namespaces. Clarify the
name and docstring so it's obvious each prefix captures every logger nested
under it.
2026-06-04 19:12:26 +00:00
Egor Grushin
cf2ba4502c GDGT-2558 move alerts management from EE to OSS (#75247)
* GDGT-2558 move alerts management from EE to OSS

* fix build error

* fix: missing feature

---------

Co-authored-by: Nicola Mometto <nicola.mometto@metabase.com>
2026-06-04 20:11:00 +01:00
Egor Grushin
2a053bc762 fix flaky test https://app.trunk.io/metabase/flaky-tests/repo/56d8e0a… (#75253)
fix flaky test https://app.trunk.io/metabase/flaky-tests/repo/56d8e0ad-ae87-4941-a147-3082557f1178/test/44ef37ca-1d2c-5e0f-9e51-a9c64654159d
2026-06-04 19:21:41 +01:00
github-automation-metabase
a001486d53 Auto-update documentation for master (#75207)
Update generated docs (master)

Co-authored-by: github-actions <github-actions@github.com>
2026-06-04 13:36:51 -04:00
Filipe Silva
416633f7bc fix: retry message on gsheets transient failure (#75092)
Fix CLO-5546
2026-06-04 15:44:31 +01:00
Benoit Vinay
3eb882a625 Added new strong-enough password type (#75104)
* Added new `strong-enough` password type

* Revert password policy to default normal

* Numbers only test added

* Misc
2026-06-04 16:41:44 +02:00
Mia Miu
5f8ef77462 docs: standardize upload setup link labels (#75178) 2026-06-04 10:27:49 -04:00
Romeo Van Snick
38783c0797 Wait for loading databases (#75216)
* Wait for databases to load in entity picker

* Add unit test for loading databases

* Make allLoading composable in withSavedDatabasesPrefetch

OR the wrapper's own database-loading state with any incoming allLoading
prop instead of overwriting it, matching withAvailableModels. Addresses
PR review feedback; behavior is unchanged today since this is the
outermost wrapper, but it keeps the loading signal composable.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* Fix types

---------

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-04 20:49:33 +10:00
Andrei Chernykh
6864d60bea Fix flaky caching test in data studio metrics e2e (#75199)
* Fix flaky data studio metrics e2e by updating the caching test for the modal and hardening library clicks

* Scope the cache strategy option lookup to the open dropdown

* Harden library readiness and absence checks in the metrics e2e spec

* Drop redundant navigation logs in the metrics e2e spec

* Fix archive metric test left on the wrong page after dropping the re-visit

* Retry the whole metricItem chain so a slow library render does not flake the click

* Wait for the library list fetch before interacting with its rows

* Navigate metric pages by id and verify list changes via API to avoid the lazy library tree

* Remove the now-unused metricItem library helper
2026-06-04 12:33:42 +02:00
Romeo Van Snick
1d04c71315 Clean table urls (#75079)
* Add Urls.table() builder for clean /table/:slug links

* Route /table/:slug to the QueryBuilder as an ad-hoc question

* Keep /table/:slug URL while the table question is unedited

* Point table links at Urls.table for clean /table/:slug hrefs

* Scope clean /table URL to the table route; update e2e

* Address PR review: restrict clean /table URLs to concrete tables

- Urls.table now accepts only ConcreteTableId; callers that may hold a
  virtual id (Saved Questions virtual DB, card__<id>) fall back to the
  ad-hoc question / tableRowsQuery URL: TableBrowser, detail-view nav,
  search results, notebook data picker, and metric source-table links.
- updateUrl's /table-route check is now basename-aware so the canonical
  URL behavior works under subpath deployments.
- Drop the virtual-id (card__1) URL test that enshrined an unroutable URL.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* Fix CI: keep ad-hoc URL for modelToUrl table links

modelToUrl serves collection/search/library contexts where a user may
only have indirect (published-table) access; the /table/:slug route needs
direct table metadata and 403s for those users (broke the data-studio
table-collection-permissions e2e). Revert that one call site to
tableRowsQuery. Update the InfoText unit test for the search-result table
link, which uses Urls.table directly.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-04 20:16:28 +10:00
Roman Abdulmanov
e0bbcb044f Check webhook secret exists (#75226) 2026-06-04 22:00:15 +12:00
Chris Truter
814398c835 Add semantic-distance scorer for semantic search (#75128) 2026-06-04 10:27:03 +02:00
Phoomparin Mano
eaead01536 Add url to create dashboard and create question tool (#74748)
* create dashboard tool should return the dashboard url

* add unit test on returning dashboard id

* add channel to module config

* fix cljfmt and kondo

* add url to questions
2026-06-04 07:27:55 +00:00
Stas Gavrylov
58a170bf01 GDGT-2525 Introduce PortalContainer for Tooltip component (#75103)
* Set document.body as default target for Tooltip component

* Update selector to 'body' to allow dynamic capturing

* Introduce PortalContainer component as Tooltip target instead of document.body

* Update rendering of PortalContainer, fix tests

* Fix portal rendering in tests

* Make main app portal render in document.body

* Fix storybook provider
2026-06-04 10:12:28 +03:00
Oleh K
b757d6b6bf Add timeout and diagnostic logging to the JUnit hook-rewrite CI step (#75201)
* Bound JUnit hook-rewrite CI step with a timeout and add diagnostic logging

The 'Rewrite JUnit hook failures' step hung for ~30 minutes with no output,
consuming the job's wall-clock budget and cancelling the whole run.
continue-on-error only ignores a non-zero exit, not a hang, so add a
1-minute step timeout so a stall fails just the step.

Also add staged logging (shell marker before bun, then before arg parsing,
after readdir, and per file) so a future stall localizes to a phase instead
of being a black box.

* Remove comment
2026-06-04 08:24:37 +02:00
YB
31fb67c270 [GDGT-2503] add sorting by duration to transform runs list (#75102)
Adds the duration column and make it sortable for the list of transform runs we have in admin view.
2026-06-04 13:19:29 +08:00
Fraser
6f6e012c63 Revert reducers module tier change (#75218) 2026-06-04 16:50:21 +12:00
Fraser
790f4d71ad Reclassify entry-point composition files as app tier (#75073) 2026-06-04 13:47:39 +12:00
Chris Truter
c1ede1fea3 Audit driver/enterprise tests for with-dynamic-fn-redefs (#73940)
Stacked on top of #73140.

Drops the `prefer-with-dynamic-fn-redefs-unaudited` regex from `.clj-kondo/config.edn` so the linter applies to enterprise tests and driver tests too. Converts the ~262 sites it had been silencing across:

- 59 files in `enterprise/backend/test`
- 8 files in `modules/drivers/*/test`

All flagged sites had unambiguously fn-shaped RHSs and defn-style LHSs, so the conversion is mechanical. Four enterprise files needed import adjustments — two had no `metabase.test` alias (added `:as mt`), two already aliased `metabase.test.util :as mt` (added `:as dynamic-redefs` on `metabase.test.util.dynamic-redefs` to avoid the clash).

A second commit reverts 14 SSO-test sites back to `with-redefs` (with inline `#_{:clj-kondo/ignore [...]}` markers) — JWT/SAML/LDAP/OIDC tests hit `/auth/sso` which is served by Jetty workers that don't inherit `*local-redefs*`, so the dynamic redef is invisible to the request handler.

Tested locally: all 919 enterprise tests pass (5649 assertions, 0 failures, 0 errors). Driver tests not run locally — they need DB connections and are expected to be exercised by CI.
2026-06-04 01:24:38 +02:00
Andrei Chernykh
bace1da06a Migrate ArchiveCollectionModal from JSX to TS functional component (#75126)
* Migrate ArchiveCollectionModal from JSX to TS functional component

* Widen ArchiveModal modelId to number | null and pass null for non-numeric collections
2026-06-03 15:08:41 -07:00
Gabriel Moreno
b328d6a4b9 Enable all admin users to activate trial up to pro (#75056)
* refactor: enable non-store admins to go through upsells

* refactor: enable non-store admins to buy transforms

* fix: restore assertions
2026-06-03 14:25:25 -04:00
Chris Truter
729d5a3bbc Support MCP server URL aliases (#75025) 2026-06-03 19:55:48 +02:00
Nathan Voxland
d571a78d1f Default appdb checkoutTimeout to 30s (#75051)
Change c3p0 checkoutTimeout to 30s to avoid forever-waits for connections
2026-06-03 12:20:52 -05:00
Andrei Chernykh
2b4386c784 Migrate License billing info from Emotion to Mantine and CSS modules (#74990) 2026-06-03 08:33:16 -07:00
Ben Grabow
a943448e90 Improve error messages for unsupported H2 functions (#75149)
* Improve error messages for unsupported H2 functions

* regenerate sample db
2026-06-03 10:28:10 -05:00
Nemanja Glumac
d7a4cf2291 Fix job_id resolution on best effort basis (#75185)
Hit the REST jobs API, now with `--paginate --slurp` so the filter sees
every page: a full run has ~185 jobs sharing one run_id, exceeding the
100-per-page cap, so the original single-page lookup silently missed
shards past page 1 and returned null (the original DEV-2117 bug).
Also require status == in_progress and no completed_at so a reused
runner can't match a job it already finished.


Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-03 17:25:23 +02:00
Ngoc Khuat
fdd0b23683 Throttle emails once per message instead of per batch (#75183) 2026-06-03 14:44:34 +00:00
Jakub Chodorowicz
d4be5e5c3f sandbox custom react components (#73941)
* WIP: sandbox custom react components

* improve typing of widget mount

* simplify widget mount module

* refactor widget mount

* refactor widget mount

# Conflicts:
#	enterprise/frontend/src/metabase-enterprise/custom_viz/widget-mount.unit.spec.ts

* refactor viz def settings sanitizer

* improve types

* cleanup naming

* adjust readme and docs

* adjust docs

* use single generic mount to render viz and settings

* improve types

* simplify render in defineConfig

* extract use-plugin-mount

* store test custom viz plugin sources

* fix missing locale in custom viz fixture

* check string setting widgets with allowlist

* add custom viz security custom settings widget security test case

* simplify widget mount stamping to plain property

* use isObject helper, skip non-object values

* add comment explaining rendering flow in usePluginMount

* add performMount and props to explicit deps

* remove obsolete archive with custom plugin

* update test fixture of calendar heatmap

* add react settings widget sandboxing e2e test

* bump sdk version

* revert .only test annotation

* improve test name
2026-06-03 16:14:50 +02:00
github-automation-metabase
2ecf813278 fix: only show trial warning banner when 7 or fewer days remain (#74971)
* fix: only show trial warning banner when 7 or fewer days remain

Previously, the trial banner was shown immediately when starting a new
trial instance, which is overly aggressive. Now the banner only appears
when there are 7 or fewer days left in the trial period.

Closes GRO-452

* fix: update AppBanner integration test for 7-day threshold

Adjust the test to use a date within 7 days of expiry, and add a new
test verifying the banner is hidden when more than 7 days remain.
2026-06-03 13:43:14 +00:00
github-automation-metabase
4c598ca711 Auto-update documentation for master (#75158)
Update generated docs (master)

Co-authored-by: github-actions <github-actions@github.com>
2026-06-03 09:20:15 -04:00
Chris Truter
53f0305835 Add brute-force vector search strategy for semantic search (#75160) 2026-06-03 15:02:53 +02:00
Nicola Mometto
7fc7f35959 fix: use any-transform-enable to gate transform hydration (#75097)
* fix: use any-transform-enable to gate transform hydration

* add test
2026-06-03 11:50:47 +01:00
Kamil Mielnik
96dbeb455a GDGT-1844 Opening a non existant transform from Data Structure folder breaks the ui (#75035) 2026-06-03 16:54:32 +07:00
Kamil Mielnik
40228fc378 Fix "Promise was collected" error in Storybook play functions (#75109) 2026-06-03 16:53:42 +07:00
Uladzimir Havenchyk
d36a73b068 replace deprecated buttons with buttons from ui kit (#75023)
* chore: replace deprecated buttons with mantine button

* more deprecated buttons

* typo
2026-06-03 12:19:39 +03:00
Ngoc Khuat
6b0ea3600e GDGT-2233: Match colliding upload columns by display name on append/replace (#75090) 2026-06-03 08:52:21 +00:00
Uladzimir Havenchyk
346170cf3f chore: rework ChoroplethMap to function component and convert test to ts (#74483)
* chore: convert Map visualization to typescript

* addd missing props to viz definition type

* more tweaks

* more tweaks

* fix typecheck

* fixup

* convert ChoroplethMap to ts

* convert to functional component

* use display name

* oops

* fix comments

* code review

* chore: rework ChoroplethMap and convert test to ts
2026-06-03 11:44:58 +03:00
Romeo Van Snick
13c0db3b5e Typescript 6 (#74140)
* Update to typescript 7.0 (beta)

* Update moduleResolution to bundler

* Switch to tsgo in scripts

* Remove unneeded ts-expect-error

* Add aliases for sdk framework imports

* Add aliases for node_modules types

* Resolve csstype to 3.2.3

* Resolve codemirror types since the package does not export .d.ts files

* Be more precise in SegmentedControl type

* Fix types in CssVarsDeclarationPlugin

* Use correct type for keys in WeakMap

* Use declare context instead of context! to narrow context type

* Import from top-level mantine package

* Remove unused card prop

* Add missing props

* Fix import

* Be more precise in SegmentedControl type

* Convert CardRenderer to TypeScript

* Remove unused @ts-expect-error

* Remove infinite type instantiations

* Do not use declare in type

* Set module to preserve for bundlers

* Add types for tinykeys

* Switch to TypeScript 6.0.3

* Switch back to tsc

* Resolve ts config path directly

* Update @microsoft/api-extractor

* Fix Modal types

* Remove redundant ext-apps ambient module shim

Under the old moduleResolution 'node', the package exports field was
ignored, so @modelcontextprotocol/ext-apps/react could not be resolved
to its real types and this ambient shim re-exported them from the deep
dist path.

With moduleResolution 'bundler', the subpath now resolves natively via
the exports map, while that same exports map blocks the deep dist import
the shim relied on. The shim therefore resolved to nothing and shadowed
the real module, breaking fe-type-check. Deleting it lets the native
types take over.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* Remove deprecated baseUrl from next-15 host app tsconfigs

next build runs a full type-check using the repo's TypeScript (now 6.0),
which hard-errors (TS5101) on the deprecated baseUrl option. Both
next-15 production-bundle e2e jobs failed on it.

baseUrl is unnecessary here: the apps only use @/* path mappings and
real package imports, and under moduleResolution 'bundler' paths
resolve relative to the tsconfig location without baseUrl. This also
matches the modern create-next-app scaffold, which no longer emits it.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* Potential fix for pull request finding

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>

* Remove ts 7 from this branch

* Upgrade typescript-eslint to a version compatible with TS6

---------

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
2026-06-03 17:08:05 +10:00
Andrei Chernykh
fdfbd63fc2 Migrate Group managers UI from Emotion to Mantine and CSS modules (#74993)
* Migrate Group Managers user type components from Emotion to Mantine and CSS modules

* Tidy group detail empty state, member count placement, and admin pane padding

* Update people e2e test for new empty-group copy
2026-06-02 22:14:58 -07:00
Andrei Chernykh
d70174060f Migrate audit table from Emotion to Mantine and CSS modules (#75028) 2026-06-02 22:14:25 -07:00
Fraser
85bf5cfbd8 Fix parameters violations (#75074) 2026-06-03 16:53:32 +12:00
Romeo Van Snick
8d534b519e Move question query runner out of services.js and into metabase/querying (#75016)
* Add RTK query hooks for dashboard card and pivot queries

* Run saved-card queries via RTK in runQuestionQuery

* Add public/embed card and dashcard endpoints

* Add query-endpoints helpers to metabase/api

* Run dashcard queries via RTK in dashboard data-fetching

* Switch to RTK hook in ErrorOverview

* Remove unused pivot-related endpoints from services.js

* Move shouldUsePivotEndpoint into metabase/api/query-endpoints

* Move dispatchQueryEndpoint into metabase/api/query-endpoints

* Move run-query helpers from services.js to metabase/querying/run-query

* Convert run-query to ts

* Use AbortError in query endpoints too

* Remove lazy imports

* Make cardId and token mutually exclusive

* Remove one more dynamic import

* Disambiguate saved-card RTK cache keys with _refetchDeps to fix SDK query abort race

* Avoid Lib database() throw for internal queries in shouldUsePivotEndpoint
2026-06-03 11:34:19 +10:00
Chris Truter
594f46f9ff Fix MCP OAuth discovery behind a TLS-terminating proxy (#75110)
* Fix MCP OAuth discovery behind a TLS-terminating proxy

When Metabase runs behind a proxy that terminates TLS, every OAuth
discovery URL we advertise (issuer, authorize/token/registration
endpoints, protected-resource metadata) is built from the Site URL
setting. If Site URL is stored as http:// the client reaches us over
https but is told to talk to an http:// authorization server, and
rejects the mismatch. Three independent gaps made this hard to recover
from (BOT-1617):

- The OAuth provider config was cached at startup, so correcting Site
  URL had no effect until a restart. Key the cache on Site URL so it
  rebuilds when the setting changes.

- First-boot Site URL auto-seeding read the *-host headers but never
  X-Forwarded-Proto, defaulting to http://. Prepend the forwarded
  scheme (taking the first hop of a comma-separated chain).

- A client probing the bare /.well-known/oauth-protected-resource path
  fell through to the SPA's HTML catch-all, tripping a JSON parse error.
  Serve the metadata at the bare path too.

* Tighten the new docstrings

- Drop the dev-facing "alias of…" note from the bare protected-resource
  endpoint's docstring; defendpoint docstrings render verbatim into the
  OpenAPI docs, and the rationale already lives in the comment above it.
- One sentence per line in forwarded-scheme.

* Match X-Forwarded-Proto case-insensitively

X-Forwarded-Proto may legally be upper/mixed case (e.g. HTTPS). URL
schemes are case-insensitive (RFC 3986), but normalize-site-url's "http"
prefix check is not, so an upper-case scheme would be treated as
scheme-less and mangled into http://HTTPS://… and dropped. Lowercase the
extracted scheme. An unexpected scheme still passes through to fail
normalization, rather than being whitelisted away.

* Guard scheme-prepending against scheme-bearing host headers

Address roborev finding (job 1872): the X-Forwarded-Proto seeding
assumed the *-host headers are scheme-less. A scheme-bearing host header
plus X-Forwarded-Proto would have produced https://https://… → rejected
by normalize-site-url → site-url never seeded, looping on every request.
Only prepend when the host has no scheme; a scheme-bearing host passes
through untouched.

Also tighten the forwarded-scheme docstring to the caller contract
(comma-chain handling moved to an inline comment, covered by a test) and
make the *-host test drive the header from its own label so the case
name can't drift from the header actually set.
2026-06-03 01:24:39 +00:00
Eric Normand
4f60b9d657 Speed up field-values sync via UNION DISTINCT (GHY-3074) (#74533)
* Speed up field-values sync via UNION DISTINCT (GHY-3074)

Replace per-field SELECT DISTINCT queries with a single batched UNION ALL
covering every list-eligible field on a table. Each arm of the UNION runs
its own DISTINCT subquery with the same LIMIT and CAST AS VARCHAR for
type-safe transport. Decoding back to native Clojure values uses the field's
known base_type.

Per-column accuracy is preserved (each column gets up to 1000 distinct
values; has_more_values reflects per-column truncation, not a blanket
table-row sample cap).

Wires the union path through three call sites: scheduled sync, on-demand DB
field-values refresh, and the enterprise data-editor invalidation queue.
Non-SQL drivers (e.g. Mongo) fall back to the per-field path.

* Chunk batched-get-latest-full-field-values IN clause

Wide tables can have thousands of list-eligible fields. A single
`SELECT … WHERE field_id IN (?,?,?…)` with that many parameters can
exceed driver parameter limits (Oracle: 1000, SQL Server: 2100).

Partition the field-id list into batches of 500 and union the results.
Same pattern already used by batch-invalidate-field-values! and
update-field-values-for-on-demand-dbs!.

* Track probed-fields and queries-made in update-field-values task_details

Adds two counters to the sync stats map:
- :probed   number of fields we attempted to fetch distinct values for
- :queries  number of warehouse queries actually issued

On the SQL UNION path these surface the wasted-query ratio that motivated
this PR — a sync run with `{updated:312, probed:5000, queries:100}` shows
that 4688 fields were probed only to confirm "nothing changed", consuming
1 query per ~50 fields instead of 5000 individual queries.

On the non-SQL fallback `:probed` equals `:queries` equals field count
(one DISTINCT per field).

* Fix UNION arm SQL to be driver-portable

The previous shape built `CAST(value AS VARCHAR)` and `LIMIT N` directly in
HoneySQL, which broke on Oracle (needs VARCHAR2(n) + FETCH FIRST), SQL
Server (TOP n), MySQL/MariaDB (CHAR not VARCHAR), and Spark SQL (STRING).

Replace with two driver-aware primitives Metabase already has:

- `sql.qp/cast-to-text` MBQL clause — drivers override to emit their
  native text type (Oracle's VARCHAR2(4000), SQL Server's VARCHAR(MAX),
  Spark SQL's STRING, etc.).
- `sql.qp/apply-top-level-clause :limit` — drivers translate to their
  dialect (Oracle WHERE rownum, SQL Server SELECT TOP N, default LIMIT).

Restructure the arm from a nested SELECT-from-SELECT-DISTINCT to a flat
single-level SELECT with GROUP BY for DISTINCT semantics. GROUP BY
composes cleanly with every driver's :limit transform (notably SQL
Server's TOP N rewrite, which reads from :select and wouldn't have seen
:select-distinct).

Restore the `(isa? driver :sql)` gate so non-SQL drivers (Mongo et al)
fall back to per-field — same blast-radius as master, no regression.

Drops the varchar-type and supports-union-distinct? multimethods I added
in the previous attempt; the per-driver dialect concerns are now handled
by sql.qp's existing extension points.

* Fix UNION arm structure: nest table identifier and parenthesize LIMIT

Two HoneySQL-level bugs from the previous commit, caught by running the
suite in the branch worktree (the earlier "passing" runs were against a
stale REPL in a different worktree):

1. The table identifier went into :from unwrapped, so HoneySQL parsed the
   identifier's own [::identifier :table [...]] vector as a
   [table alias extra] spec. Wrap it as [[expr]] — the same nesting the
   QP's own :source-table handler uses.

2. UNION arms each carried their own LIMIT: `SELECT … LIMIT N UNION ALL
   SELECT … LIMIT N` is illegal SQL (the LIMIT binds to the whole union).
   Nest each limited query as a subquery — `SELECT * FROM (<limited>)` —
   so the LIMIT is parenthesized and the outer arms union cleanly.

Also stop asserting exact :probed/:queries values in the whole-database
sync tests — those counters depend on how many fields across the entire
database have active FieldValues, which is global state these tests
don't pin down. Compare only the create/update/delete/error counts.

* Drop the constant tag from GROUP BY

Postgres rejects a non-integer constant in GROUP BY ("non-integer
constant in GROUP BY") — in Postgres a bare integer in GROUP BY means a
positional column reference, so it disallows other constant literals
there entirely.

A constant in the SELECT list never needs a GROUP BY entry on any
engine, so GROUP BY only the cast expression. The field-name tag stays a
plain SELECT-list literal.

Verified locally against both H2 and Postgres.

* Gate union-distinct tests to SQL drivers

union-distinct-values builds and runs SQL, so the tests that call it
directly only apply to SQL drivers. Previously the integration and
batching tests used a bare mt/dataset, so in a Mongo (or other non-SQL)
CI job they ran against that driver and threw.

Add a sql-test-drivers helper (normal drivers that derive from :sql) and
wrap every direct-call test in mt/test-drivers over it. Non-SQL drivers
exercise the per-field fallback via sync-fields-grouped-by-table!-test,
which works on any driver and is left ungated.

* Add MySQL ::cast-to-text override (CHAR, not TEXT)

MySQL/MariaDB `CAST` does not accept `TEXT` as a target type — its string
cast target is `CHAR`. The `:sql` default for `sql.qp/cast-to-text`
casts to `text`, which is valid on Postgres/H2/Snowflake/Redshift but a
syntax error on MySQL.

This was never caught before because `cast-to-text`'s only other use is
UUID text-comparison, and MySQL has no native UUID type — so the broken
MySQL default was never exercised until the union-distinct field-values
path started casting every column to text.

Override mirrors MySQL's existing `:text` handler, which already casts
to CHAR, and the per-driver override idiom used by Athena, Vertica,
SQL Server, Oracle, etc. MariaDB inherits it via the driver hierarchy.

Verified locally on H2 and Postgres; MySQL relies on CI (local MySQL
test env has an unrelated dataset-teardown failure).

* Add BigQuery ::cast-to-text override (STRING, not TEXT)

BigQuery's string type is STRING; it has no TEXT type, so the :sql
default for sql.qp/cast-to-text (CAST(expr AS text)) is a syntax error.
Same class of bug as the MySQL fix in the previous commit — the default
happens to be valid on Postgres/H2/Snowflake/Redshift but not here.

Override mirrors BigQuery's existing :text handler, which already casts
to STRING.

Verified locally against BigQuery (union-distinct cross-driver and
parity tests).

* kick ci

* Fix docstring continuation indentation

Per the Clojure style guide, docstring continuation lines align with the
opening double-quote (2 spaces), not the first text character (3). Apply
to the docstrings added/modified in this PR.

* Fix docstring indentation in data_editing.clj

* kick ci

* Remove blank line from defn

* Better comment

* Use a private var instead of defn

* Inline union-distinct into models.field-values

The union-distinct namespace was a strategy module being orchestrated
from inside itself, forcing models.field-values to use requiring-resolve
to break the cycle. Merging the 13 forms into models.field-values puts
the orchestration alongside persist-field-values!/limit-values/etc.,
deletes the cycle break, and shrinks the warehouse-schema :api set.

* Rock a comment

* Use transducer for union-distinct batching

* Indentation

* Push empty-input handling into the per-table sync fns

* Use ->map delta helper in per-field sync accounting

* Improvements

* Better comments

* Move both FieldValues caps into persist-field-values!

persist-field-values! now takes raw values from the warehouse, counts them
against *distinct-limit* to detect the row-cap, runs limit-values for the
char-cap, and combines both signals. This removes the QP-level
limit-max-char-len-rff (duplicate of limit-values), drops the
:has_more_values key from distinct-values's return, and updates the
parameter widget caller in parameters.field-values to do its own capping.

Sync code no longer reaches into *distinct-limit* or runs limit-values
manually — both cap concerns are now owned by the warehouse-schema layer.

* Move FieldValues sync orchestration to sync module

The warehouse-schema layer now exposes only run-distinct-batch (the
single-batch UNION ALL primitive) and the persistence primitives. The
batching loop, SQL-vs-non-SQL dispatch, retry semantics, on-demand
refresh, and grouped-by-table orchestration all live in sync.field-values
where they belong.

Removed from warehouse-schema/models/field-values:
- union-distinct-values (batching loop)
- persist-bulk-results! (per-field persist over a batch)
- sql-driver-for-table? (sync-level dispatch concern)
- sync-fields-grouped-by-table! (orchestrator)
- update-field-values-for-on-demand-dbs! (sync workflow)
- table-ids->table-id->is-on-demand? (helper for the above)

Added to sync.field-values:
- fetch-distinct-for-table (batching loop with per-batch try/catch)
- sync-fields-for-table! (unified SQL/non-SQL persister, replaces the
  separate update-field-values-for-table-{union,per-field}! pair)
- sync-fields-grouped-by-table! (moved)
- update-field-values-for-on-demand-dbs! (moved)

Per-batch failures now keep trying through other batches: a recoverable
error (e.g. SQL syntax / query too long) marks the batch's fields as
errors and the loop continues; a non-recoverable error (ConnectException
etc.) propagates out via with-error-handling and aborts the sync.

Updated call sites:
- enterprise/action-v2/data-editing now requires sync.field-values
- queries.models.card and dashboards.models.dashboard call the sync
  version of update-field-values-for-on-demand-dbs!
- dashboards module no longer needs warehouse-schema in its :uses

Updated kondo config to add sync.field-values to sync module's :api set
and to add sync as a :uses for queries and enterprise/action-v2.

* Unify distinct-limit constants and persist code path

- Replaced *absolute-max-distinct-values-limit* with *distinct-limit*
  (both were 1000) and kept the richer docstring.
- Added fetch-distinct-per-field for the non-SQL branch — wraps
  distinct-values per field and returns the same shape as
  fetch-distinct-for-table. Non-SQL drivers now get the same
  :failed-fields semantics: one field's fetch throwing no longer
  cascades into nuking the rest.
- sync-fields-for-table! is now driver-agnostic at the persist layer:
  one reduce, one result->delta, one with-error-handling per persist.
  Branching is contained to the fetch step.

* Drop warehouse-schema from action-v2; push FV-eligibility filter into sync

sync-fields-grouped-by-table! now filters to FV-eligible fields via
field-should-have-field-values? before dispatching. Both callers
(data-editor's batch-invalidate-field-values! and
update-field-values-for-on-demand-dbs!) used to do this filter
themselves; the same logic now lives in one place.

This drops action-v2's only remaining use of warehouse-schema, so it
also disappears from action-v2's kondo :uses.

Also dropped my ^:sync test-metadata typo — it was a no-op tag I
invented; the established convention is ^:synchronized (and these
tests don't mutate global state anyway, so neither applies).

* Add ^:synchronized to FieldValues-writing sync tests

* Gate UNION-DISTINCT path on :nested-queries feature

sync-fields-for-table! used to dispatch to the UNION batch path purely
on (isa? driver-hierarchy :sql). The query structure wraps each arm in
a subquery in FROM, which requires :nested-queries support. All current
SQL drivers declare it (the SQL base class adds it to the default set),
so this is purely defensive — but if a future SQL driver doesn't, it'll
fall back to the per-field path instead of throwing at execution time.

Renamed sql-driver? to can-batch-distinct? to reflect what we actually
require.

* Decode :type/Float and :type/Decimal in UNION results

Auditing decode-value's coverage vs the per-field MBQL path revealed
two real divergences: :type/Float columns were landing as strings via
the UNION path but as Doubles via the per-field path, and same for
:type/Decimal -> BigDecimal vs string. Both round-trip through
:model/FieldValues.values as JSON; a numeric column would JSON-encode
as a number on one path and a string on the other.

Other types we don't decode (Date, DateTime, Time, UUID, IPAddress)
are already JSON-encoded as strings by mi/transform-json, so the
existing string passthrough matches the per-field path's stored
representation — no decoding needed. Added a comment to that effect.

Also hardened Integer's bigint fallback against truly unparseable
input (it threw a second NumberFormatException uncaught for
non-numeric strings) so all numeric branches now fall through to a
string passthrough on malformed input.

decode-value-test extended to cover the full type matrix:
- nil across multiple base-types
- Boolean case-insensitivity (TRUE / True / true / etc)
- Long, BigInteger, BigInteger-via-:type/Integer overflow
- Float, Decimal, Decimal-derived (Currency), Float-derived (Coordinate)
- Malformed numeric input on each numeric branch
- Non-decoded types (Date / DateTime / Time / UUID / IPAddress)

run-distinct-batch-matches-per-field-test extended to include
products.rating (Float) so the parity check would catch a numeric
divergence on every SQL driver in the matrix.

* Fall back to per-field path for tables that require a filter

Partitioned BigQuery tables reject queries without a partition-column
filter. The per-field MBQL path goes through
metadata-from-qp/table-query which has
add-required-filters-if-needed baked into its middleware; the new
run-distinct-batch runs as a native query and bypasses that
middleware, so it would fail on these tables.

can-batch-distinct? now also checks (:database_require_filter table)
and returns false for require-filter tables, dispatching them to the
per-field path which still works.

Tests:
- Unit test on can-batch-distinct? using mt/with-temp tables with the
  flag flipped both ways.
- BigQuery driver test that:
  * Verifies the dispatcher returns false for fv_partitioned_table.
  * Calls sync-fields-grouped-by-table! on the partitioned table and
    asserts that the distinct values ("coffee" / "tea" / "matcha")
    still land in :model/FieldValues correctly via the fallback.

* Route nested-JSON child fields (:parent_id set) to the per-field path

A reviewer caught that two fields can share a name in the same table
when they have different :parent_id values — the unique constraint is
(table_id, parent_id, name), and JSON-column unfolding produces child
fields whose parent JSON column is the disambiguator.

run-distinct-batch had two problems for such fields:

1. by-name collision: (into {} (map (juxt :name identity)) fields)
   keeps only the last same-named field, so the other gets pre-seeded
   :values [] and persist-field-values! would DELETE its existing
   FieldValues row.

2. cast-to-text-honeysql passes (:name field) raw with no parent
   qualification, so the SQL references the wrong column for any
   nested-JSON child.

Both bugs go away by routing :parent_id fields through the per-field
path, which goes through the QP — the QP builds the column reference
correctly (including any parent traversal) and per-field has no
by-name lookup.

sync-fields-for-table! now splits incoming fields by :parent_id:
plain fields go to fetch-distinct-for-table (batch), nested fields go
to fetch-distinct-per-field. merge-fetch-results combines the two
shapes. A table that fails can-batch-distinct? still routes everything
through per-field as before.

Unit test creates two JSON-parent fields with same-named children,
mocks both fetchers, and verifies the dispatcher routes correctly.

* Split run-distinct-batch out into its own namespace to break a cycle

CI surfaced a cyclic load dependency:

    chain-filter → ... → actions → driver-api → sql.qp → field-values →
    field → params.field-values → chain-filter

The new edges from this PR were field-values requiring sql.qp and
query-processor. sql.qp transitively loads actions → ... → eventually
back to field-values via the field/params chain. Local REPL didn't hit
it because everything's already cached; cold-load on CI does.

Fix: move the UNION ALL helpers (build-arm, build-union, run-distinct-batch,
decode-value, table-honeysql, cast-to-text-honeysql, *batch-size*) to a new
namespace, metabase.warehouse-schema.field-values.distinct-batch, which is
the one that imports sql.qp + query-processor. It requires field-values
(for *distinct-limit*). field-values itself no longer requires sql.qp
or query-processor, so the cycle is broken.

Added the new namespace to warehouse-schema's :api set in the kondo
config. sync.field-values and the relevant test files now require the
new namespace alongside field-values.

* Lazy-resolve sync.field-values from card and dashboard to break cycle

Eastwood caught a queries.core <-> queries.models.card cycle. The path:

  queries.models.card → sync.field-values → distinct-batch → sql.qp →
  driver-api → actions.core → actions.execution → query-processor.card →
  queries.core → queries.models.card

queries.models.card's direct (:require [metabase.sync.field-values …]) was
the new edge in this PR. card.clj and dashboard.clj only call
update-field-values-for-on-demand-dbs! at runtime (not at load time), so
requiring-resolve is the standard way to break the static cycle without
changing runtime behaviour. The kondo :uses sync entries on queries and
dashboards modules stay — the dep is still real, just lazy-bound.

* limit-values: keep NULL distinct values

The original PR's limit-values had a (nil? (first values)) -> skip
branch, which made sense when only the UNION batch path called it.
When this branch (in commit de4a6fb7b6) made every persist go through
limit-values, the silent null-stripping started affecting the per-field
MBQL path too — and that path is what backs cached FieldValues for any
field with has_field_values=:auto-list or :list.

The CI regression: parameters.chain-filter-test/nil-values-test creates
a dataset where :myfield contains nil, "", and "value". sync analyses
the field as :auto-list, get-or-create-full-field-values! runs the
QP query (which correctly returns [nil] [""] ["value"]), then
persist-field-values! runs limit-values on those raw values and drops
the [nil] before storing. chain-filter then sees a cached row missing
nil and returns the truncated set.

Fixed by removing the nil-skip clause. nil sorts cleanly into a
sorted-set (it goes first), and parameter widgets care about it as a
"no value set" entry. Updated the unit test to reflect the new
expected behavior.
2026-06-02 22:49:37 +00:00
Fraser
b3e5ae05d1 Fix nav violations (#74738) 2026-06-03 10:10:49 +12:00
Romeo Van Snick
13724dfa7e Migrate ViewNativeQueryEditor to composable NativeQueryEditor (#75080) 2026-06-03 08:10:23 +10:00
Romeo Van Snick
2c3adc56bd Migrate DatasetQueryEditor to composable NativeQueryEditor (#75078) 2026-06-03 08:02:56 +10:00
Romeo Van Snick
3a28d03ece Migrate QueryEditorBody to composable NativeQueryEditor (#75077) 2026-06-03 08:02:53 +10:00
Riley Thompson
71e66269c2 Optimize temporal expressions the same as filters (#74875)
* optimize expressions

* rename for accuracy

* group into one middleware

* include a between filter in test

* rename filter to clause

* improve check for day bucketing, add tests

* fix kondo and a test

* fix test and kondo

* fix test for drivers
2026-06-02 22:00:52 +00:00
technomancy
cdd4c01727 Increase warn time for long tests from 3 seconds to 60 seconds. (#75155)
While in theory it's helpful to know about which tests are "slow", in
practice an awful lot of them are, and no one is working to address
the slow tests, so there is no need to clutter up the CI logs with
information no one is using.

Instead, let's warn when tests go longer than a minute.
2026-06-02 14:58:22 -07:00
Nemanja Glumac
cbe349cac9 Fix set -e abort in e2e-prepare-containers when last service is disabled (#75150)
PR #75096 extracted the docker compose service list into a SERVICES="..."
variable assignment built from command substitutions. Under GitHub Actions'
default bash -e, a variable assignment inherits the exit status of its last
command substitution, so when the last-listed service (snowplow) is disabled,
[ false = true ] && echo ... returns 1, the assignment returns 1, and errexit
aborts the Start Containers step before the retry loop runs. This broke jobs
with snowplow=false (e.g. SDK component tests); e2e-group jobs passed only
because they run snowplow=true.

Build SERVICES as a bash array via standalone test && append statements
(exempt from errexit) and expand with "${SERVICES[@]}", which also fixes the
unquoted-word-splitting on $SERVICES.

DEV-2099

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-02 22:14:37 +02:00
Nathan Voxland
6f3b45b576 Expand Clojure style guide with helpers and idioms (#75124) 2026-06-02 14:55:58 -05:00
bryan
3831a896ca Token scanner: skip image/archive/binary files (#75141) 2026-06-02 21:39:49 +02:00
Timur Mashara
5fb636f09b remove transforms gem from left navigation (#75098) 2026-06-02 21:38:20 +02:00
Anderson
5f0665db9b Fix legacy Library entity ID migration collisions (#74795)
* Fix legacy Library entity ID migration collisions

* Update unit tests
2026-06-02 16:12:34 -03:00
Andrei Chernykh
f74bbbe901 Migrate LogoIcon from JSX class component to TS functional component (#75121) 2026-06-02 12:09:24 -07:00
Nathan Voxland
0d4997a4f3 Improve fingerprint performance (#74683) 2026-06-02 13:16:50 -05:00
Nemanja Glumac
3599d9f885 Send cypress test failures info to ci conductor (#74872)
* Report e2e test failures to ci-conductor (DEV-1999)

Send Cypress failures to the ci-conductor service mid-run, from the
`after:spec` Node hook, so failures can be tied back to the CI run (and
through it the PR / release branch) before the job completes.

- e2e/support/ci_conductor.ts: build the payload and POST it; self-guards
  on a missing webhook URL and never throws into the test run. `job_id`
  lives behind a single seam (sends null for now).
- e2e/support/config.js: call the reporter from `after:spec` (CI only).
- .github/workflows/e2e-test.yml: pass the webhook URL (secret) and REPO_ID.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* Dry-run mode for ci-conductor reporting (DEV-1999)

Add a CI_CONDUCTOR_DRY_RUN toggle that logs the payload instead of POSTing,
so we can validate env resolution and payload shape against real CI failures
before sending any data. Enabled in e2e-test.yml for now; remove that env to
start sending for real.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* Guarantee ci-conductor reporting can never break the e2e run (DEV-1999)

Wrap the whole report at the after:spec call site in try/catch as a hard
backstop, so a throw from extractFailedTests or anything downstream can't
escape into the run (and the video cleanup still runs). Also widen the
reporter's own try/catch to cover payload build + the dry-run log, not just
the fetch. Errors are logged and swallowed.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* Treat the ci-conductor secret as the webhooks base URL (DEV-1999)

The secret now holds the webhooks base (".../webhooks") and the reporter
appends "/failed-tests", so the same secret can serve other webhook endpoints
later and the CI_CONDUCTOR_WEBHOOK_URL name stays accurate. A trailing slash on
the base is tolerated.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* Authenticate ci-conductor requests with x-internal-secret (DEV-1999)

The conductor's failed-tests endpoint requires an x-internal-secret header.
Send it from CI_CONDUCTOR_WEBHOOK_SECRET (a GitHub secret passed via
`secrets: inherit`); the header is omitted when the secret is unset, so local
runs are unaffected. The secret value is never logged.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* Resolve job_id via composite action and send it to ci-conductor (DEV-1999)

GitHub doesn't expose the current job's numeric `workflow_jobs.id` to a
running job. Add a small composite action `./.github/actions/resolve-job-id`
that resolves it once via the Actions API (matched by run_id+run_attempt+
runner_name; picks the latest-started for sequentially-reused runners) and
exports it as $JOB_ID for any later step in the job to read.

Wire it into the e2e job and have the reporter read JOB_ID, so failed-tests
posts now carry the numeric job_id. Best-effort: a failed lookup just warns
and proceeds, and the reporter falls back to null (the column is nullable).

DEV-2036 tracks an optional server-side alternative (resolve job_id from
runner_id on workflow_jobs) for the longer term.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* Retry resolve-job-id lookup to absorb Actions API consistency lag (DEV-1999)

The jobs API is eventually consistent — after a runner picks up a job it can
take a few seconds for runner_name to appear. Doing the lookup early in the
job races with that, so many jobs got "Could not resolve current job id"
warnings while others got lucky. Add a 5x3s retry loop (~12s max wait); the
lookup is cheap and this stays well within job-start budget.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* Enrich ci-conductor payload + report flaky tests too (DEV-1999)

Forward more signal so conductor has what it needs to classify and analyze:

  - Top-level: add run_attempt (GITHUB_RUN_ATTEMPT) and retries (the resolved
    Cypress runMode ceiling, surfaced from setupNodeEvents into the env so the
    reporter can include it without a workflow change).
  - Per test: add duration (ms) and attempts (raw {state} array from Cypress,
    forwarded as-is); collapse the old message/stack pair into a single
    `message` carrying the full displayError blob (Cypress only exposes one).
  - Filter widening: include any test with ≥1 failed attempt, so flaky tests
    (failed then passed) now reach conductor alongside broken ones. Healthy
    tests stay omitted, so a clean spec still triggers zero POSTs.

For flaky tests Cypress doesn't keep displayError after the final attempt
passes, so message arrives as null — the failure pattern is still readable
from the attempts array, which is enough for conductor to classify.

The conductor schema isn't final; extra fields are silently ignored on its
side, so this is forward-safe.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* Send ci-conductor failed-tests for real (DEV-1999)

Drop the dry-run flag so the reporter POSTs to the conductor instead of just
logging the payload. The dry-run logic stays in the code (gated on the env)
in case we want to flip it back on for debugging a future change.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* update schema

* Resolve the job_id just before we run tests

* Fix the docstring

---------

Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Co-authored-by: Ryan Laurie <iethree@gmail.com>
2026-06-02 10:42:34 -06:00
github-automation-metabase
8335330b40 fix: add retry logic to docker login and compose up without third-party actions (#75096)
* fix: add retry logic to docker login and compose up without third-party actions

Replace docker/login-action with a bash retry loop (3 attempts, 15s backoff)
and wrap docker compose up with a retry loop (3 attempts, 30s backoff with
compose down cleanup between retries).

Follows the existing retry pattern from .github/actions/prepare-backend/action.yml.
No third-party retry actions introduced.

Fixes transient Docker Hub connectivity errors that have been causing
~2-4 failures/day for 83+ days.

* fix: pass docker credentials via env vars instead of direct interpolation

- Use env: block to pass username/password as runtime variables
- Prevents special characters from breaking shell interpretation
- Drop unnecessary 2>&1 redirect on docker login
2026-06-02 18:19:03 +02:00
Oleh K
59f0e1214a Replace deprecated Card with Mantine Card (#75020)
* Replace deprecated Card with Mantine Card in BillingInfo

Drops the empty `BillingInfoCard` styled wrapper and renders the Mantine
`Card` directly in BillingInfoTable. The legacy `flat` prop becomes
`withBorder radius="md"` plus Mantine's default no-shadow.

* Replace deprecated Card with Mantine Card in PinnedItemCard

Drops the empty `ItemCard` styled wrapper and renders the Mantine `Card`
directly inside the PinnedItemCard component. `flat` becomes
`withBorder radius="md"` plus Mantine's default no-shadow.

* Match deprecated Card defaults on migrated sites

The Metabase theme's `Paper.config` sets `defaultProps: { shadow: "md" }`,
which Mantine `Card` inherits. Both BillingInfo and PinnedItemCard used
the deprecated `flat` prop (no shadow), so explicitly disable the
inherited shadow via `shadow="none"`.

Also restore the implicit vertical spacing the deprecated card was
contributing via `line-height: 24px`: a small margin-top on PinnedItem's
description and a bit more padding-bottom on its body.

* Replace deprecated Card with Mantine Card in PulsesListSidebar

Swaps the underlying Card import in `PulseCard` to Mantine and moves the
no-shadow / border / radius / no-padding props onto the consumer's
PulseCard render. Drops the no-longer-needed `flat` prop.

* Replace deprecated Card with Mantine Card in LoginHistory

Inline usage in `LoginHistoryItem`. Swaps the Card import to Mantine
and converts the deprecated card's `className + style + p={0}` combo
to native Mantine spacing props (`px="20px" py="sm" my="md"`) plus
`radius="md" withBorder`. Mantine's default `shadow="md"` replaces
the heavier deprecated default.

Mixing `p={0}` with a CS.p* className silently drops the className's
padding — Mantine's own padding wins on specificity — so converting to
Mantine props is the only way to keep the original spacing.

* Replace deprecated Card with Mantine Card in ListItem

Data-reference list item renders Mantine `Card` with native spacing
props (`p="lg" mb="md" w="680px"`) and `radius="md" withBorder`
instead of stacking `hoverable + className(CS.p3, CS.mb2, CS.bgWhite,
CS.rounded, CS.bordered) + style={{width}}` on the deprecated card.
Drops the `hoverable` shadow-on-hover effect — the parent
`<ListItemLink>` already provides hover feedback.

* Replace deprecated Card with Mantine Card in SuggestionsSidebar

X-ray sidebar item card swaps to Mantine `Card` with native spacing
(`p="md"`) and `radius="md" withBorder` instead of the deprecated
`hoverable + className(CS.p2)` combo.

Side effect: the parent Link's hover text-color (`AutomaticDashboardApp.module.css`
.ItemLink :hover) no longer propagates into the card body — Mantine
Paper sets an explicit `color` that blocks the cascade. Not worth
preserving for this view.

* Replace deprecated Card with Mantine Card in BulkActionBar

Swaps `Card` import in `ToastCard` styled wrapper to Mantine and inlines
the `dark` look (dark background, inverse text) so the consumer no
longer needs the deprecated `dark` prop. Also pins
`flex-direction: row` explicitly — Mantine Card defaults to column,
which silently flipped the toast layout vertical.

* Replace deprecated Card with Mantine Card in UndoListing

Converts the `ToastCard` styled wrapper to a function component built on
Mantine `Card`. Maps the legacy `dark` and `noBorder` props to native
Mantine props (`bg`/`c`/`withBorder`) so the consumer's API stays
identical — emotion's `styled()` would have leaked these props to the
DOM as HTML attributes after the swap.

* Remove deprecated Card component

All 8 call sites now use Mantine `Card` (`metabase/ui`). The
`metabase/common/components/Card` directory has no remaining consumers
in either OSS or enterprise code.

* Fix Mantine Card prop types after deprecated Card removal

`styled(MantineCard)` strips Mantine's polymorphic call signature,
which makes consumer props like `onClick`, `color`, `children`, and
`ref` invisible to TypeScript. Two patterns to restore them:

- `BulkActionBar.styled.tsx` ToastCard has no custom props, so cast
  the styled wrapper to `typeof Card` (the same pattern `CardContentSide`
  uses in `UndoListing.styled.tsx`). Drop the now-pointless
  `<Box component={ToastCard}>` wrapper in the consumer — Box wasn't
  adding anything since ToastCard already extends Mantine Card.
- `PulsesListSidebar.styled.tsx` PulseCard needs the custom
  `canEdit` prop, so type the cast as
  `ComponentType<CardProps & ComponentPropsWithoutRef<"div"> & { canEdit }>`.
- `UndoListing.styled.tsx` ToastCard is a function component; extend
  `CardProps & ComponentPropsWithoutRef<"div">` so the consumer's HTML
  attributes type-check.

* Address PR review comments

- LoginHistory: `px="md"` for spacing-token consistency.
- ListItem: `c="inherit"` so the parent `ListItemLink`'s hover color
  propagates into the card body again; explicit `lh="1.5rem"` on the
  title and an `mt` on the subtitle to restore the spacing the
  deprecated card's inherited `line-height: 24px` was providing.
- UndoListing: inline the former `ToastCard` function component at its
  one usage site so `.styled.tsx` only contains styled components.
  Convert the inline-style padding/margin/maxWidth to Mantine
  `px`/`py`/`mt`/`maw` props and simplify the `overflowX` conditional.
- SuggestionsSidebar: `c="inherit"` so the parent `Link`'s hover color
  reaches the card title.

* Don't use rems in lh

* Don't use px in py

* Minor formatting
2026-06-02 18:03:40 +02:00
Nemanja Glumac
072cd1691b DEV-2088: Skip milestone check when there's no prior release to compare (#75085)
The trigger-milestone-check job crashed when cutting the first build of a
new major (e.g. v0.62.0-beta): getLastReleaseTag only looks within the same
major, finds no prior stable tag, and the resulting `undefined` base ref was
fed into the compare API (`compare/undefined...<sha>`), 404ing the job.

Bail early for patches (which share their parent minor's milestone) and
pre-releases (by convention a major's `.0` is always cut as a beta/RC, so a
pre-release is always the first build of a new major with no same-major
baseline). A `!lastTag` guard remains after getLastReleaseTag as a safety net
in case that convention is ever broken.

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-02 17:58:04 +02:00
Egor Grushin
96fc55c8a6 GDGT-2213 Update incremental transform label to "Only process new data" (#75049)
* GDGT-2213 Update incremental transform label to "Only process new data"

* fix failing test
2026-06-02 15:45:42 +01:00
Stas Gavrylov
586dfd96f8 GDGT-2519: Surface hidden tables in Schema Viewer ERD (#75031)
* GDGT-2519: Surface hidden tables in ERD

* GDGT-2519: Add visual mark to hidden table nodes on canvas

* GDGT-2519: Remove 'details-only' and 'normal' from TableVisibilityType
2026-06-02 17:31:31 +03:00
Andrei Chernykh
d971c1216d Migrate public link copy panel from Emotion to Mantine and CSS module (#74998) 2026-06-02 07:15:31 -07:00
Andrei Chernykh
4c2b33b3dc Migrate error components from Emotion to Mantine and CSS modules (#74986) 2026-06-02 07:14:53 -07:00
Andrei Chernykh
cf6eb54cf3 Migrate Object Detail from Emotion to Mantine and CSS modules (#74985) 2026-06-02 07:13:00 -07:00
Ngoc Khuat
0c1e95acdd Fix XLSX exports dropping the currency symbol for non-currency columns (#75026) 2026-06-02 14:12:27 +00:00
Andrei Chernykh
067993f50e Migrate New Model Options from Emotion to Mantine and CSS modules (#74984) 2026-06-02 07:12:24 -07:00
Kevin Kelleher
4c9c985255 Fix failing Update Loki Snapshots workflow (#75106)
* no-verify for loki automation commit

* temp change to test loki update

* Update Loki Snapshots

* revert test changes

---------

Co-authored-by: Metabase Automation <github-automation@metabase.com>
2026-06-02 10:12:04 -04:00
Ngoc Khuat
026b8fcff7 Send transform job-failure emails to all admins in one message (#74751)
* Send transform job-failure emails to all admins in one message

notify-job-failure looped over every admin and published one
:event/transform-failed per admin, producing N separate single-recipient
emails per failure. On instances with many admins this spammed the email
throttler (email-max-recipients-per-second) with N rapid sends, causing
retry storms, and made N SES calls where one would do.

Add a dedicated :event/transform-job-failed event + seeded notification
that targets the admin group and sends a single bcc email, mirroring the
slack-token-error pattern. notify-job-failure now publishes one event.

The per-transform notify-transform-failures path (last editor/creator) is
unchanged.

* Strengthen job-failure test and clean up review nits

- Add an assertion that the email's :bcc field covers the whole admin
  group, proving the send was consolidated rather than fanned out one
  per admin. The previous assertions could not distinguish the two.
- Switch test data to bird-themed emails.
- Point the docstring reference at the seed-notification! var rather
  than the bare namespace.

* Fix cljfmt formatting in notification seed
2026-06-02 20:43:01 +07:00
Ryan Laurie
2cc52e7e9f Convert SearchApp To Typescript (#73740)
* change filename

* convert SearchApp to ts

* simplify

* even better

* fix undefined case
2026-06-02 07:38:34 -06:00
Egor Grushin
af1305dca7 Gdgt 2429 unsaved changes warning does not work while save transform (#75054)
* GDGT-2429 run unsaved changes warning for a new transform if the saving modal is opened and the navigation attempt happen

* GDGT-2429 add tests

* GDGT-2429 extend the test

* GDGT-2429 added "pressing Esc should close the warning, not the saving modal" step to the test
2026-06-02 14:34:47 +01:00
Egor Grushin
a3992af871 GDGT-2516 add snowplow events to the Alerts Management feature. (#75045) 2026-06-02 14:14:45 +01:00
Ngoc Khuat
1ed1a67138 Split oversized email recipient lists to avoid SMTP provider rejection (#74825) 2026-06-02 20:11:39 +07:00
Fraser
116c9343a4 Fix list-view violations (#74877) 2026-06-02 20:30:34 +12:00
Ngoc Khuat
6d926642dd Allow editing MySQL tables when partial_revokes is enabled (#74573) 2026-06-02 08:18:32 +00:00
YB
44847f1ac2 [GDGT-2421] field metadata sandboxing (#74902)
Row & column security already trims columns in query results; this extends the same trimming to the read-only field listings (Data Reference / table metadata) so a sandboxed group sees a consistent column set everywhere. This is a QoL change for sandboxed viewers, OSS/non-sandboxed users and admins are unaffected.
2026-06-02 16:12:05 +08:00
Stas Gavrylov
df4e8ecfa4 GDGT-2515: Fix FK field name crop in Schema Viewer info popup (#75017)
* GDGT-2515: Fix FK field name crop in Schema Viewer info popup

* GDGT-2515: Fix InfoPanel container height

* GDGT-2515: Cleanup legacy styling
2026-06-02 09:46:41 +03:00
Romeo Van Snick
ed34beed0a Replace transformResponse with a rawResponse boolean option (#75003)
* Replace transformResponse with a rawResponse boolean option

The only thing any caller actually used `transformResponse` for in the
legacy client was the `({ response }) => response` escape hatch — "give
me the raw `Response` so I can `.blob()` / `.arrayBuffer()` it" (binary
downloads, map tiles). Replace the whole callback shape with a single
`rawResponse: boolean` option that says exactly that.

Migrations:
- `dataset.ts` `downloadDataset` (both POST and GET branches):
  `transformResponse: ({ response }) => response` → `rawResponse: true`
- `LeafletTilePinMap.tsx`: same swap.
- `query.ts` (RTK baseQuery passthrough): forward `rawResponse` instead.

Side cleanups enabled by removing `transformResponse`:
- Drop the function-form overload on `MethodCreator` (no callers).
- Drop `DEFAULT_OPTIONS.transformResponse`.
- `rawResponse: true` implies the fetch path — XHR has no `Response`
  object, so callers no longer need to pair it with `fetch: true`.

* Always drop Content-Type for formData (not just fetch path)

The previous `options.formData && options.fetch` gate was needed when
`options.fetch` was the only way the formData callers reached the
fetch transport. With `rawResponse: true` now auto-routing through
fetch, those callers no longer pass `fetch: true` — so the gate fails
to fire and the request goes out as `Content-Type: application/json`
with a multipart body. BE rejects with 400.

Fix: gate on `options.formData` alone. XHR's `send(formData)` works
the same way — the browser sets the right Content-Type.

* Strip JSON Content-Type for form-encoded rawResponse downloads

Dropping `fetch: true` from `downloadDataset` (relying on `rawResponse`
to imply the fetch path) regressed the Content-Type handling: the gate
that strips our default `application/json` header keyed off
`options.fetch` alone, so form-encoded download POSTs kept the JSON
header and the backend rejected them with a 400.

Gate the header deletion on the same condition the dispatcher uses to
choose fetch (`isTest || fetch || rawResponse`).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* Fix URLSearchParam body

---------

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-02 14:42:12 +10:00
Nick Fitzpatrick
399290e672 Fix hide when no values for pivot table dashcards (#74851)
* Fix hide when no values for pivot table dashcards

* adding e2e

* fixing test
2026-06-02 09:30:21 +05:30
Nick Fitzpatrick
972bfeda54 Don't invalidate entities on logging activity (#74981)
* Don't invalidate entities on logging activity

* e2e adjustment
2026-06-02 09:29:58 +05:30
Romeo Van Snick
84e629c461 Migrate query action editor (#75015)
* Add composable NativeQueryEditor with backward-compatible shim

* Migrate the QueryActionEditor to new QueryEditor style

* Address PR review comments on native query editor slots

- Pass snippetCollections through the Sidebar slot so the snippets button
  hides correctly when there are no snippets and the root collection isn't
  writable
- Guard the RunButton slot early when run handlers aren't wired up, avoiding
  unnecessary template-tag validation on every render
- Use Children.toArray to collect slot children so React assigns stable keys

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-02 13:41:14 +10:00
Romeo Van Snick
d1fc4802ed Migrate NativeQueryModal to composable NativeQueryEditor (#75021) 2026-06-02 13:40:30 +10:00
Andrei Chernykh
f6105b3cf2 Convert DefaultParametersSection and GroupMappingsWidget specs to TS (#75060)
Convert DefaultParametersSection and GroupMappingsWidget specs to TypeScript
2026-06-01 19:01:04 -07:00
Cam Saul
64abdf6d9f Stop using temporary fork of cljfmt (#75062) 2026-06-01 17:25:54 -07:00
Cam Saul
23f1fb4380 Finish migrating parameters QP/driver code to MBQL 5 (#74467)
* Finish migrating parameters to MBQL 5

* PR feedback (part 1)

* Rework the qp.parameters.values namespace

* Kondo

* Test fixes 🔧

* Test fixes 🔧

* Update docstrings

* Test fix 🔧

* MongoDB test fixes 🔧

* cljfmt

* Revert schema changes to Card query

* Test fix 🔧

* 🔧

* PR feedback 🔧

* Kondo 🔧

* Fix crazy keyboard
2026-06-01 16:52:29 -07:00
Romeo Van Snick
4f4134ebe9 Accept FormData / URLSearchParams as the request body directly (#75005)
* Accept FormData / URLSearchParams as the request body directly

The legacy client previously required uploads to be passed through a
`body: { formData }, formData: true, fetch: true` triple — a wrapper
object, a flag duplicating what `instanceof` could check, and a flag
forcing the fetch transport. Teach the client to detect FormData and
URLSearchParams bodies natively:

- `legacy-client.ts`: in `_makeMethod`, when `rawData` is a `FormData`
  or `URLSearchParams`, use it as the body unchanged. Delete the
  default `Content-Type: application/json` so the browser sets the
  right value (multipart boundary or urlencoded charset). XHR's
  `xhr.send(formData)` honors this too — no transport coupling needed.

- `query.ts` (RTK baseQuery): forward FormData / URLSearchParams bodies
  unspread. The previous `{ ...args?.body, ...args?.params }` spread
  yields an empty object for FormData (keys aren't enumerable) and
  would erase the upload.

- Migrated callers: `table.ts` (append-csv, replace-csv) and
  `advanced-config.ts` now send `body: formData`. The legacy wrapper
  shape still works for any caller that hasn't migrated yet.

Pre-existing `formData: true` opt-in is kept as a fallback for now;
removing it is left for a follow-up after the remaining unmigrated
caller (if any turns up) is converted.

* Pull out helper to check for FormData

* Just use FormData instead of FormData and URLSearchParams
2026-06-02 09:22:13 +10:00
github-automation-metabase
2545ec3374 Auto-update documentation for master (#75058)
Update generated docs (master)

Co-authored-by: github-actions <github-actions@github.com>
2026-06-01 18:09:36 -04:00
Edward Paget
06f43840ce Unblock remote sync when configured branch is deleted upstream (#72885)
fix: unblock remote sync when configured branch is deleted upstream (#72778)

When the configured `remote-sync-branch` has been deleted on the remote,
any operation that snapshots the source throws `Invalid branch: X`.
The most visible symptom is that `GET /has-remote-changes` 500s,
blocking the branch-switching UI the user needs to recover.

- `git/fetch!` now passes `.setRemoveDeletedRefs true` so deleted
  remote branches are pruned locally (deterministic signal).
- `git/snapshot*` throws a typed ex-info
  (`{:error-type :missing-branch, :branch ...}`) so callers can
  discriminate.
- `impl/has-remote-changes?` catches `:missing-branch` and returns
  `{:has-changes? false, :branch-missing? true, ...}` instead of
  propagating. The branch-missing result is intentionally not cached.
- `HasRemoteChangesResponse` gains an optional `:branch_missing`
  field; the API handler includes it when set.

Refactored `has-remote-changes?` into small helpers
(`cache-valid?`, `snapshot-or-missing-branch`, `branch-missing-result`,
`fresh-result!`) to flatten the previously nested conditional flow.

Other callers (`async-import!`, `async-export!`) still surface the
typed error — those are explicit user actions where failing loudly is
correct behavior.
2026-06-01 21:21:37 +00:00
Andrei Chernykh
987b137a2d Migrate action form and implicit action icon from Emotion to Mantine (#75036) 2026-06-01 14:11:18 -07:00
Andrei Chernykh
dfdd6542be Migrate impersonation modal and warning from Emotion to Mantine (#75034) 2026-06-01 14:10:54 -07:00
Andrei Chernykh
5d0322cada Migrate move card toast from Emotion to Mantine and CSS module (#74995) 2026-06-01 14:09:52 -07:00
Uladzimir Havenchyk
2c6bb89a55 deps(custom-viz): use safe version of vite (#75053)
* deps(custom-viz): use safe version of vite

* bump custom viz
2026-06-02 00:08:37 +03:00
Eric Normand
d30920ac1a GHY-3727 add network timeout to remote sync git operations (#75047)
Remote sync pull/push could hang indefinitely (stuck at progress 0 during
import fetch/clone, and 0.3 during export push) because JGit transport
commands had no socket timeout. A stalled connection blocked the sync's
virtual thread forever, and neither the cancel flag (only observed in
update-progress!) nor the diehard interrupt (blocking socket I/O ignores
Thread.interrupt) could unblock it.

Apply .setTimeout to every remote git command via call-remote-command,
driven by a new remote-sync-git-timeout-seconds setting (default 60s), so a
stalled operation throws and flows into the existing error-handling path.
2026-06-01 16:06:23 -05:00
Nathan Voxland
c434e744b7 Add 62 to version-helpers.ts (#75040) 2026-06-01 14:37:19 -05:00
Ben Grabow
50061eab17 Bgrabow/better ci report for e2e tests (#75038)
* Capture output from E2E tests

* Capture output from E2E tests

* Revert back to cheshire
2026-06-01 13:43:05 -05:00
Eric Normand
16593d046c GHY-3754 Order serdes export by created_at for stable filename de-dup (#74992)
Same-named entities in the same folder were assigned their filename
de-dup suffix (`foo.yaml` vs `foo_2.yaml`) by the order rows happened to
arrive from the DB, which has no stable ordering. Across re-exports the
order shifted, so two distinct entities swapped files and their
entity_id/created_at/contents appeared to churn — producing large
phantom git-sync diffs.

Order top-level content extraction by created_at, then entity_id as a
deterministic tiebreaker. created_at is monotonic with creation, so the
oldest sibling keeps the unsuffixed file and adding a new same-named
entity appends a `_2` rather than displacing the existing file — an
append-only property a random entity_id would not have. Applied to the
default extract-query-collections path and the Collection and
NativeQuerySnippet overrides via the shared serdes/stable-storage-order.

Nested fetches (e.g. a Dashboard's DashboardCards) are embedded as lists
inside the parent's file rather than written to their own files, so they
keep their natural order and are explicitly excluded from the ordering.

Adds a regression test and updates the serialization baseline fixture to
the new deterministic ordering.
2026-06-01 13:13:35 -05:00
Nemanja Glumac
11377bdea4 Increase timeout for Eastwood test job to 45 minutes (#75041) 2026-06-01 19:50:35 +02:00
Chris Truter
fa9e51914a Replace abandoned verify-pr-label-action with inline gh check (#74953) 2026-06-01 17:41:03 +00:00
Nick Fitzpatrick
85b80747bc Set oauth dynamic registration to false when MCP is disabled (#74923) 2026-06-01 23:03:17 +05:30
Timur Mashara
981e3ed6f6 add tmashara (me) to slack and gh team map (#75027)
add tmashara to slack and gh team map
2026-06-01 19:00:05 +02:00
Andrei Chernykh
16d2f6509e Replace caching strategy picker radios with a dropdown (#74948)
* Replace caching strategy picker radios with a dropdown

* Migrate caching strategy specs from radios to the dropdown

* Address dropdown review feedback and migrate MetricToolbar caching spec

* Migrate caching e2e specs from radios to the dropdown

* Fix impersonated caching e2e: drop stray Default click that toggled the dropdown

* Size the caching modal to its content instead of a fixed height
2026-06-01 09:43:10 -07:00
Riley Thompson
33beb183c2 Add missing honeysql util methods for mbql5 drivers (#75024)
* add missing methods

* match h2 mbql5
2026-06-01 09:58:01 -06:00
Nathan Voxland
75e8f1854d Short-circut permission group membership queries when there is no user (#73245) 2026-06-01 10:51:06 -05:00
metabase-bot[bot]
a05cff3858 [chore] Update E2E test timings with recent data (#75006)
Update E2E test timings with averaged data

Collected timing data from 7 days of test runs.
All specs with timing data were included (simple averages).

🤖 Generated automatically by update-e2e-timings workflow

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: GitHub Actions <actions@github.com>
2026-06-01 15:19:00 +00:00
metamben
ac5737c926 Prevent writing queries failing the schema checks in the DB (#74884)
Closes QUE2-622
2026-06-01 17:18:46 +03:00
github-automation-metabase
903a9a17c1 Auto-update documentation for master (#74991)
Update generated docs (master)

Co-authored-by: github-actions <github-actions@github.com>
2026-06-01 09:07:32 -04:00
Uladzimir Havenchyk
4cccd299ad chore: replace getAssetUrl with inline images (#74960)
* chore: replace getAssetUrl with inline images

* backend work

* small tightening

* address review

* inline component

---------

Co-authored-by: Nicola Mometto <nicola.mometto@metabase.com>
2026-06-01 15:38:58 +03:00
Nick Fitzpatrick
bb34d6b233 match Viz and dashcard radius (#74843)
* match Viz and dashcard radius

* Updating loki snapshots
2026-06-01 17:58:24 +05:30
Romeo Van Snick
323dca5e46 Entity copy form (#75002)
* Move EntityCopyModal to common and rename it

* Make CopyModal a normal export

* Fix module boundaries for CopyModal

* Throw instead of returning undefined when cloning without underlying question

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-01 21:52:20 +10:00
Oleh K
f2f40a73cf Partial rollout of JUnit hook-failure rewriting for select e2e specs (#74958) 2026-06-01 13:13:18 +02:00
Romeo Van Snick
eb6901f096 Refactor NativeQueryEditor to be more composable (#75014)
* Add composable NativeQueryEditor with backward-compatible shim

* Mock query editor sub components

* Address PR review comments on native query editor slots

- Pass snippetCollections through the Sidebar slot so the snippets button
  hides correctly when there are no snippets and the root collection isn't
  writable
- Guard the RunButton slot early when run handlers aren't wired up, avoiding
  unnecessary template-tag validation on every render

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-01 20:03:12 +10:00
Romeo Van Snick
0750570270 Use RTK hook for settings in AppThemeProvider (#75013) 2026-06-01 19:44:08 +10:00
Romeo Van Snick
8562bc7d9f Use AbortError to signal request cancellation (#75001)
* Standardize on AbortError for request cancellation

Replace the bespoke `{ isCancelled: true }` shape that the API client threw
on abort with the standard `DOMException` AbortError that `fetch()` already
emits on cancellation. Both transports (`_makeRequestWithXhr` and
`_makeRequestWithFetch`) now reject with AbortError, and an `isAbortError`
helper next to `NetworkError` makes the check ergonomic.

Why:
- Matches the web platform (AbortController contract): `fetch` natively
  throws AbortError, no need to wrap.
- One shape, one check: callers don't have to know which transport ran.
- Better stack traces than a plain object literal.

Consumers migrated from `error.isCancelled`/`isCancelledRequestError` to
`isAbortError(error)` in `services.js`, `query_builder/actions/querying`,
dashboard `context`/`data-fetching`/`types`, and the SDK
`use-load-question` hook.

* Make isAbortError a typed guard and expose AbortError

Define `AbortError` as a named type (`Error & { name: "AbortError" }`)
next to `isAbortError`, and let the guard narrow with it. Reach for
the named type in `CancelledFetchDashboardResult` instead of the
inline `{ name: "AbortError" }` shape.

Pure type refinement — no runtime change.
2026-06-01 19:23:07 +10:00
Romeo Van Snick
a7254d757d Harden e2e tests against timing flakiness (#75004)
Defensive Cypress hardening across ~16 specs and 5 helper files,
preempting a class of timing-sensitive flakes that the upcoming
XHR→fetch transport switch is expected to surface.

The pattern across every file is the same:

- Split chained `.within(() => …)` calls when the parent subject can
  become detached mid-async (pinned items run their query on hover,
  Mantine comboboxes mount async, etc.). Re-query the parent so
  Cypress can retry on detach.

- Add small `clickPullOption` / `clickPushOption` / `clickSwitchBranchOption`
  helpers around the git-sync menu — Mantine combobox state can drop a
  synthetic `.click()` if invoked before the dropdown's handlers are
  wired. `realClick` + a "did the menu close?" verify + retry once.

- Expose a `data-testid="mini-picker-list-loader"` on
  `MiniPickerListLoader` so e2e specs can `cy.findByTestId` the
  loading skeleton instead of relying on text that hasn't rendered yet.

- Add an `expectMultipartBody` assertion in `uploads.unit.spec.js`
  that pins down "table.ts hands FormData through unwrapped, the
  multipart Content-Type is set by the browser" — locks in current
  behavior so the upcoming client refactor can't silently break it.

No production-code behavior changes (the MiniPicker `data-testid` is a
test-only attribute). All extracted from the
`remove-xhr-option-in-api` branch's preflight work.
2026-06-01 19:22:56 +10:00
Mihael Konjević
d5d356d3ec Fix agent portable-FK resolution when two tables share a name across schemas (#74931) 2026-06-01 09:22:39 +00:00
Romeo Van Snick
5ef9224d27 Remove requests reducer (#74994)
* Remove only reader of state.requests

* Remove requests reducer and redux slice
2026-06-01 19:22:36 +10:00
Romeo Van Snick
6bdfe920d4 Table API field order shape (#74890)
* Accept {"field_order": [1,2,3]} in table api as wel as [1,2,3]

* Pass {field_order: [1,2,3]} in table api RTK

* Remove bodyParamName option from the API client

The `bodyParamName` option let a caller say "wrap the data under this
key before stringifying" — a workaround for the `PUT
/api/table/:id/fields/order` endpoint that demanded a top-level array.
Now that the endpoint accepts a normal wrapped `{field_order: [...]}`
body, the previous commit migrated the only caller and the option has
no remaining users.

Drop it from `RequestOptions`, from the body-construction path in
`legacy-client`, and from the RTK Query `apiQuery` baseQuery
passthrough.
2026-06-01 19:22:20 +10:00
Oleksandr Yakushev
837cce9552 [perf] Optimize metabase.search.spec for wide tables (#74706) 2026-06-01 12:04:21 +03:00
Jakub Chodorowicz
102682e36f restrict CSP for img (#74347)
* restrict CSP for img and font

* adjust comment indentation

* wip: introduce csp-img-allowed-hosts and csp-img-enabled settings

* change text info style and placement

* adjust cypress tests

* improve coercion on custom viz setter

* copy improvement

* copy change

* cleanup tests

* refactor hosts parsing logic

* remove docs link

* fix OSS csp-img-enabled setting

* revert the default for img csp to include data and self

* update test comment

* link to custom visualization when it's not possible to turn off csp img

* adjust help links to take into account if custom visualizations are available

* remove reduntand gap

* use alert for security warning of custom viz

* always show security risk notice on setup page for custom viz

* fix formatting

* fix cypress test

* pr feecback: refactor security handles

* improve tech docs

Co-authored-by: Jeff Bruemmer <jeffbruemmer@gmail.com>

---------

Co-authored-by: Jeff Bruemmer <jeffbruemmer@gmail.com>
2026-06-01 09:41:46 +02:00
Chris Truter
8be8d22aa4 Seed suggested prompts for each enabled built-in Metabot (#74951) 2026-05-31 17:25:35 +02:00
Oleksandr Yakushev
bcac56435a [perf] Implement efficient insertion in add-pivot-grouping (#74325) 2026-05-31 11:43:46 +03:00
Oleksandr Yakushev
f5d44c019c [match] Use metabase.util.match macros directly instead of driver-api aliases (#73987) 2026-05-31 10:54:25 +03:00
Alexander Polyankin
8fa3fa576f Improve workspace error messages (#74944)
* fix

* fix messages

* fix

* fixes

* fix

* fix: dedupe snowflake quote-schema/quote-field, update postgres test regex

- snowflake.clj had quote-schema/quote-field defined twice (lines 79-80
  and 1068-1069), which clj-kondo flagged as redefined-vars. Drop the
  duplicate copies near init-workspace-isolation!.
- postgres-test workspace-precondition-alter-default-privileges-test
  was asserting the error message contained "ALTER DEFAULT PRIVILEGES",
  but that phrase was removed from raise-unmemberable-default-priv-owners!
  in the recent error-message rewrite. Match the new message instead.

* fix
2026-05-29 17:31:46 -04:00
Eric Normand
706edecd2b GHY-3372 Update parameter values_source_config.card_id during source swap (#74935)
When a card-to-card source replacement runs, dashboards and cards whose
parameters use the old card as a dropdown source were left pointing at
the old card. The dependency graph correctly switched to the new card,
but parameter dropdowns silently kept the old one, leaving the system
inconsistent.

card-swap-source! and dashboard-swap-source! now also rewrite
:parameters values_source_config.card_id for card-to-card swaps. The
existing before-update hooks on Card and Dashboard propagate the change
to the parameter_card join table. Card-to-table swaps are intentionally
left untouched.

Restores walk-parameter-source-card-ids (removed in b12429ac04) and
extends the previously dashboard-only behaviour to cards too.
2026-05-29 16:07:36 -05:00
Eric Normand
7cf846f5d2 GHY-3728 Strip transient lib markers from serdes MBQL export (#74980)
* GHY-3728 Strip transient lib markers from serdes MBQL export

A Card's dataset_query can be stored in the app DB as either legacy MBQL
or MBQL 5. When a legacy query is read back, normalize-query converts it
to MBQL 5 and adds transient runtime markers (:lib.convert/converted? and
:lib/transformation-added-base-type). export-mbql did not strip these, so
whether a query happened to be stored as legacy or MBQL 5 leaked into the
serialized output, producing spurious git-sync diffs.

Run lib/prepare-for-serialization on full queries during export — the same
normalization applied before persisting a query to the app DB. This is
idempotent for already-clean queries, so it introduces no new churn, and
makes a query export identically regardless of its stored format.

* Remove comment
2026-05-29 20:46:46 +00:00
Ngoc Khuat
761a8527ad Admin page for managing notifications and alerts (#72909)
* Add admin page for managing notifications and alerts

Resolves GDGT-2315/2316/2317/2318/2322.

EE admin page at /admin/tools/notifications (gated on :audit-app) for
listing, filtering, and bulk-managing card-type notifications. List shows
ID, card, creator, recipients, schedule, channel, last sent, and a
computed health state (healthy / orphaned_card / orphaned_creator /
failing). Bulk archive, unarchive, and change-owner are available via
POST /api/ee/admin/notifications/bulk. Per-notification detail page at
/admin/tools/notifications/:id shows the hydrated notification plus its
recent send history from task_history.

Model change: creator_id is no longer blocked from updates on
:model/Notification. It is editable only through the new
superuser-gated admin bulk endpoint.

* Fix detail endpoint passing payload_type as string

The payload_type column has a toucan transform that validates values against
the keyword enum notification-types. Passing the raw string "notification/card"
through t2/select-one's kv-args bypassed decoding and failed the enum check,
returning 500 on GET /api/ee/admin/notifications/:id. Pass the keyword instead.

The list endpoint is unaffected because it uses raw honey-sql to build the
WHERE clause, which bypasses the transform entirely.

* Align admin pages with SettingsPageWrapper convention

Wrap list and detail pages in SettingsPageWrapper + SettingsSection to
match the TasksTabs / PeopleListingApp convention. Promote the list page
title to order={1}. Move pagination onto the same row as filters with
justify=space-between, mirroring TaskListPage.

* Fix module boundary config for notification-admin

- Export :model/TaskHistory from task-history module so other modules can
  reference it (our health/detail endpoints query task_history rows).
- Drop models / premium-features / task-history from notification-admin
  :uses; we don't require any namespace from those modules, only a model
  (covered by :model-imports).
- Drop unused :model/NotificationCard from :model-imports.

* Simplify and bound the notifications admin hot paths

Backend:
- health.clj / api.clj bound the task_history scans used for health and
  send-history by a 90-day window and row-count cap. Previously each list
  refresh pulled every notification-send row ever written; on long-running
  instances this was a latent DB bomb.
- Use metabase.util/index-by in place of hand-rolled (into {} (map juxt)).
- bulk-update! dispatches on a keyword instead of string literals.
- Drop a few noisy comments and shorten over-long docstrings.

Frontend:
- Narrow NotificationSendHistoryEntry.status to a typed union so StatusBadge
  is no longer stringly-typed.
- Memoize NotificationRow and pass stable callbacks keyed by id so toggling
  a checkbox at 50 rows doesn't re-render every row.
- bulkNotificationAction invalidates only the admin list tag plus the
  specific row ids it mutated, not the global notification list.
- Replace the local formatDurationMs duplicate with the shared
  formatDuration helper.

* Paginate send history; address review feedback

Backend:
- Split send history out of GET /:id into a dedicated paginated
  GET /:id/send-history?limit&offset endpoint. Detail load stays lean
  and subsequent pages don't refetch the whole notification.
- Bulk endpoint now restricts updates to payload_type = :notification/card
  so an admin can't archive/unarchive/change-owner a system-event
  notification via a stray id.
- health.clj's task_history walker now also checks
  task_details[:original-info :notification_id], so rows rewritten by
  do-with-task-history on failure are correctly picked up.
- Update notification models-test: creator_id updates are now expected to
  succeed, matching the relaxed before-update invariant documented on the
  earlier commit.

Frontend:
- Add useAdminNotificationSendHistoryQuery hook and a paginated
  SendHistorySection on the detail page.
- ChangeOwnerModal: compute the localized title up front instead of
  nesting a t template inside another t — breaks i18n extraction
  otherwise.

* Drive health off TaskRun; link send history out to Tasks > Runs

Notification sends use task_history under the hood, but the parent
notification-send row is marked :success even when an individual
channel-send child fails (exceptions are caught per handler in
metabase.notification.send). The correct aggregation already lives on
:model/TaskRun, whose status is rolled up from ALL children by
complete-task-run!. Switch health.clj to read from TaskRun via the
existing task_history.run_id link so channel failures actually surface.

A side effect: TaskRun's :abandoned status is now a first-class health
state. Add it to the enum, the filter dropdown, and the badge colors.

Since the Tasks > Runs admin page already paginates + filters these,
drop our bespoke GET /:id/send-history endpoint and RTK hook. The
detail page now links out to
/admin/tools/tasks/runs?run-type=alert&entity-type=card&entity-id=<card>
for the full run history.

Tests: reseed TaskHistory rows via TaskRun so health + last_sent_at
exercise the new path. Drop the two send-history endpoint tests. Add
a regression test asserting the run-level status wins over the
parent task_history status.

* Flatten EE notifications route and fold in simplify fixes

Admin notifications endpoints move from /api/ee/admin/notifications
to /api/ee/notifications. Drops the nested-map workaround in
ee-routes-map (the reason it existed was only to thread a single
feature-gated child under /admin — no other /admin/* routes live
there, and the admin intent is already encoded in the ns name
notification-admin, premium-handler gating, and superuser check).

Also folds in a few review items while touching these files:
- Switch the task_history scan in health.clj from ended_at to
  started_at — only started_at is indexed on task_history; a 90-day
  filter + DESC sort on the unindexed ended_at column would seq-scan.
- Refresh the compute-for-rows docstring to include :abandoned.
- Add Urls.adminToolsTasksRunsFor({runType, entityType, entityId})
  using TaskRunType / TaskRunEntityType. SendHistorySection uses
  that helper instead of hand-concatenating a query string — a typo
  now fails at compile time.

* Declare :model/TaskRun export on task-history and import on notification-admin

The module-boundaries check expects every cross-module :model/ reference
to be declared as an :model-export on the owning module and a
:model-import on the consumer. Our health.clj now reads :model/TaskRun,
so register both entries to match.

* Paginate notifications admin list in SQL when no health filter

Previous implementation fetched every matching notification into
memory, computed health for all, filtered/sorted in Clojure, then
sliced to the requested page. Fine at dev-seed scale but O(N) on
every list refresh at instance scale.

Split the list endpoint into two paths:

- No health filter (the common case): push limit/offset + sort into
  SQL. Sort by notification.updated_at desc (indexed). Compute health
  for only the page. O(page size) per request.
- Health filter set (firefighter mode): materialize the full matching
  set, compute health, filter/sort in Clojure, paginate. O(matching
  rows) — kept because :failing / :abandoned / :orphaned_* are
  post-query computed and we can't easily push them into SQL.

Also normalize the response shape to match Metabase's convention for
paginated list endpoints: {:data :total :limit :offset} (was just
{:data :total}). Consistent with /api/user, /api/persisted-info,
/api/comment, etc.

* Drop last_sent_at sort; use updated_at desc everywhere

sort-by-last-sent-desc was only still used in the health-filtered
path. Neither path needs it — a stable updated_at desc ordering is
fine. Hoists the ORDER BY into base-list-query via ordered-list-query
so both paths share it, removes the in-memory sort fn.

* Fold notification-admin list paths into one function

The two paths for with/without a :health filter shared their entire
tail (hydrate + shape response). Collapse into page+total that
returns {:rows :total} and a single list-notifications that handles
hydrate + {:data :total :limit :offset}. Internal branch is still
there (health is post-query computed; can't push into SQL without
schema work) but caller-visible surface is one function.

* Derive health from TaskRun directly; drop task_history scan

Notifications for alerts have a 1:1 mapping from notification →
payload_id → notification_card.card_id, and each alert-type TaskRun
is keyed by (run_type=:alert, entity_type=:card, entity_id=<card-id>).
That means we can derive health by querying :model/TaskRun directly
by card_id — no need to walk task_history to bridge notification_id
to run_id.

Trade-off: health is now per-card, not per-notification. If two
admins set up alerts on the same card, they share the TaskRun
status. In practice rare, and it matches what admins see on
Admin > Tools > Runs filtered by the same card.

Drops the task_history-related constants and the whole in-memory
JSON walk. Tests updated to seed distinct cards per notification
where the per-card approximation would otherwise cross-contaminate.
health-uses-task-run-not-parent-task-history-test removed — the
distinction it tested no longer exists (we never read task_history).

* Push health filtering into SQL; delete notification-admin.health ns

All five health states are now pure SQL filters. A windowed subquery
(ROW_NUMBER() OVER PARTITION BY entity_id ORDER BY started_at DESC)
gives us the latest alert-type TaskRun per card within the lookback
window, joined against LEFT JOINs on notification_card, report_card,
and core_user. health-where turns the requested state into a WHERE
clause with precedence baked in (orphaned_card > orphaned_creator >
failing > abandoned > healthy).

Side effect: health.clj is entirely redundant. The four helper queries
(nid->card, card->archived, inactive-creator-ids, latest-run-by-card)
duplicated what the SQL JOINs already do. Delete the file; replace
compute-for-rows with a 15-line enrich-row helper that reads the
extra columns we now project (__card_id / __card_archived /
__creator_active / __run_status / __run_ended_at) and assocs :health
+ :last_sent_at.

Both the list and detail endpoints share list-query now. No more
two-path split, no more O(N) materialization for the health-filtered
case. Cross-DB: window functions are supported on H2, Postgres 8.4+,
MySQL 8.0+, MariaDB 10.2+.

clj-kondo modules: drop :model/TaskHistory and :model/TaskRun from
both the task-history :model-exports and our :model-imports — we no
longer reference either directly (the SQL uses raw table names).

* Collapse notification-admin health classifier to one SQL CASE

* Drop __ column prefix; shrink notification-admin row coercion

* Debounce recipient_email filter to stop firing a query per keystroke

* Move notifications-admin pagination into a footer under the table

* Align notifications-admin pages with admin/tools design language

List page:
- drop Select labels, use "Filter by X" placeholders like Tasks/Runs
- one-line rows: Ellipsified on Card and Creator, nowrap on Last sent
- merge Channel column into Recipients as inline icon + count
- absolute Last sent timestamp with relative-time tooltip
- fix click dead-zone: only the card Link stops row-click propagation
- use CS.cursorPointer utility instead of inline style
- shorter health labels ("Orphaned", "No owner") + override Badge
  label overflow so labels aren't clipped by sub-pixel rendering
- add info tooltip next to page title

Detail page:
- card name is the page title, with Alert #N as a subtitle
- drop the redundant "Alert" badge
- muted back link matching TaskRunDetailsPage
- h3 section titles, bold inline labels, no Divider rules
- two-column Grid: Details left, Recipients right
- inline list of the last 5 alert runs with status badges
- drop the duplicate Change owner button next to the creator row

Schedule formatter:
- guard schedule_hour with Number.isFinite so bad crons don't render
  "Check daily at NaN"

adminToolsTasksRunsFor:
- optional startedAt so the Runs page entity picker populates when
  linked from the notification detail page

* Test notifications-admin refactor and tighten back-link

- schedule formatter: regression tests for NaN schedule_hour on daily,
  weekly, and monthly
- adminToolsTasksRunsFor URL builder: covers optional startedAt
- list page: row renders, short health labels, filter placeholders, row
  click navigation, empty state
- detail page: card-name title, no redundant Alert badge, single Change
  owner action, sections, recipients, inline send-history runs, back
  link returns to the list

Supporting changes:
- BackLink is now a react-router Link (matches TaskRunDetailsPage), so
  the "back" action doesn't need a dispatch(push(...)) callback
- Ellipsified: use Mantine's maw shorthand instead of style={{maxWidth}}
- TextInput onChange: explicitly type the event to keep tsc happy
- Add createMockAdminNotificationListItem and admin endpoint mocks

* Drop trailing blank line in notification-admin api test

* Fix missing 'abandoned' in HEALTH_VALUES and clean review nits

HEALTH_VALUES dropped 'abandoned', so ?health=abandoned didn't survive
the use-url-state mount-time reserialize — UI let you pick it but the
filter silently evaporated on reload or share. Use 'as const satisfies'
so future mismatches between the literal list and NotificationHealth
are a compile error.

Also: Mantine style props in place of inline flex styles in InfoRow,
drop the redundant 'skip'-guarded arg in ChangeOwnerModal's user query,
and drop the redundant ChangeEvent annotation on the email input.

* Filter recipient_email in SQL; drop raw-value scan and IN splice

The previous implementation read every raw-value notification_recipient
into memory, matched details.value in Clojure, then spliced the matching
notification ids into the outer query as IN (...). That scales poorly
and risks hitting bind-param caps on large instances.

Replace it with an EXISTS correlated on notification.id that joins
notification_handler + notification_recipient + core_user in SQL. The
filter now matches only user recipients by core_user.email; raw-value
(external) recipients are intentionally out of scope — their email
lives inside a JSON column and narrowing it portably would require
either a LIKE on the serialized text or a per-DB JSON operator, neither
of which is worth the abstraction for this one filter. Admins can find
external recipients via channel filters.

* Resolve handler/recipient tables via t2/table-name

Satisfies the :model-imports declaration in modules/config.edn — the
linter sees the :model/NotificationHandler and :model/NotificationRecipient
keywords referenced in source, and the table names stay in sync with the
model defs instead of being duplicated as raw keywords.

* Publish notification-update event on admin bulk actions

Extract publish-notification-update! in the notification API so the
admin bulk endpoint and the self-service PUT endpoint share the same
post-update side effects: recipient emails on :active transitions and
an :event/notification-update audit entry per notification. Without
this the superuser bulk archive / unarchive / change-owner path left
no audit trail.

* Address review feedback on notifications admin PR

* Rename status→active and health→status; add sort and raw-value recipient match

The new admin design surfaces the computed health values as the user-facing
"Status" column, so rename the API/response field accordingly and free the
old `?status=` enum to become a plain `?active=` boolean. Extend
`recipient-email-exists` to match raw-value (external) recipients via a
portable LIKE on the JSON-encoded `details` column — the previous
helper deliberately skipped them. Add `?sort_column` + `?sort_direction`
(malli-enum whitelist), restructure the channel filter as an EXISTS
subquery so we can drop SELECT DISTINCT and sort by any expression, and
fix typo'd `metabase/utils/urls`/`metabase/utils/redux` imports that were
blocking unit tests.

* Match raw-value recipient emails in Clojure, not via JSON-text LIKE

Replace the `recipient-email-exists` helper (which cast `notification_recipient.details`
to text and substring-LIKE-d on the JSON serialization) with a two-path lookup:
- user recipients matched in SQL via indexed `core_user.email`
- raw-value recipients loaded into memory and matched with a Clojure equality
  check on the deserialized `:details.value`

Bounded by the raw-value recipient count, which is admin-search-cadence small
in practice. Removes the dialect-specific cast and the substring false-positive
surface; main query becomes a clean `WHERE notification.id IN (…)`.

* Simplify recipient-email lookup and url-state parsers

- Use metabase.util/lower-case-en (locale-safe) instead of clojure.string/lower-case
- Fold the three-query recipient-email resolution to two: drop handler-ids-with-user-email
  (folded into the final notification-id select) and the separate handler→notification-id
  resolution step
- Drop dead (or … #{}) wrappers around t2/select-fn-set (already returns #{})
- Replace the #{-1} sentinel for empty-handler-set with [:= 1 0]
- Project only :notification_handler_id + :details in the raw-value scan
- Collapse four near-identical parse* enum helpers into one parseEnum<T>
- Extract serializeActive helper instead of nested ternaries inline

* Add ?query= fuzzy substring search

Substring ILIKE OR'd across card name, creator first/last/email, user-recipient
email (correlated EXISTS), and raw-value-recipient email (pre-resolved handler IDs
from the existing in-memory scan, parameterised over a predicate so equality and
substring callers share the helper).

Param name and shape mirror the convention from /api/user (?query=) and
/api/card/:id/series — substring %lower.col% LIKE with u/lower-case-en on input.
?q= remains reserved for the search-index-backed engine.

* Test ?query= matches slack channel name on raw-value recipients

The in-memory raw-value scan never gated on channel_type — it matches details.value
across email, slack, and webhook handlers alike. Add a slack-channel test pinning
that behaviour so a future channel_type filter on the scan can't silently shrink
search coverage. Also tighten handler-ids-with-raw-value-matching's docstring to
spell out the per-channel value shape.

* GDGT-2305 edit

* Reshape notification-admin response for the new design iteration

- Replace `:status` enum + `:last_sent_at` timestamp with `:last_check`
  and `:last_sent` run-summaries `{at, error, status}`. `:last_check` is
  the latest terminal task_run; `:last_sent` is the latest successful
  delivery. Both pull `error` text from task_history's `:message` via a
  one-shot post-page query, no JSON SQL.
- Drop the `?status=` filter and the `?query=` recipient branches; recipients
  stay as the structured `?recipient_email=` filter.
- Add `?last_sent_status=failing|successful` (latest run outcome) and
  `?owner_active=true|false` (owner is_active).
- Rename the API surface `creator_id` -> `owner_id`, `creator_name` sort key
  -> `owner_name`, hydrated `:creator` -> `:owner`. The DB column
  `notification.creator_id` and internal hydrate stay; the rename is at the
  response/request boundary only.
- Replace `:status`-driven and recipient-fuzzy tests with new run-summary
  shape tests and a non-match assertion for ?query=. Use `=?` to keep them
  short.

* Splice owner.is_active onto the admin response

The hydrated :owner (formerly :creator) map is missing :is_active
because metabase.users.models.user/default-user-columns omits it. Project
core_user.is_active from the existing JOIN and merge it onto the owner
map so the FE can render the deactivated-owner icon.

* GDGT-2305 edit

* GDGT-2305 edit

* GDGT-2305 edit

* GDGT-2305 edit

* GDGT-2305 edit

* GDGT-2305 edit

* GDGT-2305 edit

* GDGT-2305 edit

* GDGT-2305 edit

* GDGT-2305 edit

* GDGT-2305 edit

* GDGT-2305 edit

* GDGT-2305 fix sorting

* GDGT-2305 add edit question alert modal

* GDGT-2305 add edit question alert modal

* GDGT-2305 add a copy link button

* GDGT-2305 coloring

* GDGT-2305 rename archive to the delete, remove undelete

* GDGT-2305 remove undelete tests

* remove icon temporary

* remove icon temporary

* restore the icon

* edit

* fix a blinking checkbox

* fix column's name

* small spacing adjustments

* render tabs conditionally based on the whole set, not only the first page

* clean up

* clean up

* clean up

* rollback

* rollback comment regions removing

* added the UserPicker

* remove tests, better write them from scratch

* fix z indices

* inline helpers

* fix stupid lines

* Align notifications-admin API with the alerts management design

- Add ownerless and multi-value channel filters
- Add id/last_check sort columns; default sort to last_send
- last_send now reflects actual channel-send delivery attempts
- Detail endpoint returns per-notification check_history and send_history
  (send entries roll up all channels of a tick)
- Allow superusers to reassign the owner via PUT /api/notification/:id

* Drop the unused bulk unarchive action

The alerts management UI has no restore affordance — "Delete" is a soft
archive with no un-delete — so the bulk unarchive action had no caller.

* apply BE changes

* apply BE changes

* fixes

* fixes

* fixes

* fixes

* Fix CI failures on notifications-admin

- Remove duplicate getTasksRoutes import in admin/routes.tsx (fixes
  fe-type-check, verify-i18n, and the build break)
- Add is_active to createMockUserInfo (UserInfo now requires it)
- Declare :model/TaskHistory in task-history :model-exports and
  enterprise/notification-admin :model-imports (fixes model-boundaries-test)
- Mock GET /api/user in CreateOrEditQuestionAlertModal spec — the new
  NotificationOwner UserPicker fetches users
- cljfmt-format notification_admin api.clj and api_test.clj

* Use creator (not owner) on the notifications-admin API surface

The /ee/notifications endpoints exposed creator_id/creator as owner_id/owner,
diverging from the public /api/notification endpoints which use creator. FE
feedback flagged the inconsistency. Keep creator end-to-end on the wire
(query params, response keys, sort column, and the change-creator bulk action)
to match the public API; owner stays a frontend-only term, translated at the
RTK Query boundary in api/notification.ts.

* Simplify FE: use creator in the data layer, Owner only as a label

Drops the owner<->creator translation layer added at the API-client boundary
(wire types, converters, transformResponse, param/bulk mappers, mock wire
conversion). The admin FE now uses creator_id/creator/creator_active/
creatorless/creator_name and the change-creator action directly, matching the
wire. "Owner" survives only as UI labels, the Ownerless tab, and component
names (ChangeOwnerModal, NotificationOwner). The ?creatorless= URL param
replaces ?ownerless=. NotificationOwner no longer needs an AdminNotification
type guard since both notification shapes now expose creator_id/creator.

* small changes

* addressing the PR's comments

* addressing the PR's comments

* Address review comments on notifications-admin

- creator_id (owner) reassignment is now gated to a single blessed path:
  add `*allow-creator-id-change?*` + `reassigning-creator` to the model and
  have the admin bulk endpoint bind it. The model before-update rejects any
  other creator_id change (incl. unauthenticated/system and superuser-via-PUT).
- public PUT /api/notification/:id rejects creator_id changes with a 400 (was
  a silent strip); echoing back the unchanged value still works.
- escape %/_/\ in the ?query= LIKE wildcard
- error-by-run-id includes abandoned runs and picks the latest row per run via
  SQL ROW_NUMBER instead of an in-memory reduce
- has_failure: boolean CASE read back through bit->boolean (drops the dialect
  cast); list-where-clauses extracted + reduced onto the query
- handler-ids lookup uses reducible-select; guard empty ids in bulk-update!
- exclude orphaned card notifications (payload_id IS NULL) from the list

* Fix cljfmt indentation in notification models_test

* Improve the code and UserPicker UX

* addressed channels column's width if there are either three channels or the count is large

* fix magic numbers for avatar

* fix NotificationRunSummary rendering

* fix the slack icon background when it's dark theme here

* close the details sidebar if the notification is removed/archived

* clear adminNotification tags after editing a notification using a single `updateNotification` action

* invalidate admin notification tags when a new notification is created

* add unit testing

* Simplify creator_id reassignment to superuser-only PUT

Walk back the dynamic-var-gated "only the admin bulk endpoint can change
creator_id" path. The design needs the edit modal to change ownership,
which goes through PUT /api/notification/:id — bronsa's review feedback
was about privilege escalation, not single-path enforcement, and is fully
addressed by superuser gating.

- model `before-update` rejects creator_id changes unless `(mi/superuser?)`;
  unauthenticated/system writes fall through and get rejected too
- public PUT accepts creator_id (via the spec's `:compare-cols`); non-superuser
  attempts get 403 from `mi/can-update?`, model is the backstop
- drop `*allow-creator-id-change?*`/`reassigning-creator` from the model
- drop the `reassigning-creator` wrap from the admin bulk endpoint
- tests: bulk-update via `:id [:in …]` triggers before-update; cover
  superuser-can-reassign and non-superuser-403 from the public PUT

* Notifications admin detail endpoint: surface failing ticks and stream rows

The detail endpoint silently dropped failing ticks from check_history and
send_history, so last_check / last_send came back nil for notifications the
list endpoint correctly marked as failing.

Root cause: `metabase.task_history/do-with-task-history` wraps the caller's
task_details on exception as `{:status :failed :message … :original-info <orig>}`,
so on failure the caller's `notification_id` / `channel_type` move under
`:original-info`. The per-notification filter only looked at the top level.

- `task-details-key` helper checks both shapes; applied to the per-notification
  filter (`matches-notification?`) and `->channel-entry`'s channel_type lookup
- `error-by-run-id` no longer filters by task; an ORDER-BY tiebreaker prefers
  the outer `notification-send` error and falls back to the latest by `ended_at`
  so a channel-only failure still surfaces its error string in `last_check`
- `task-history-rows-for-card` returns a reducible; helpers use
  `(take result-limit)` in the transducer so the JDBC cursor short-circuits
  once we've collected enough matches — common case (one notification per card)
  fetches ~10 rows instead of 500
- realise rows inside the transducer before the reducible closes
- regression tests: failure-wrapped task_details, channel-only failure error
  surfacing on the list endpoint, and outer-error preference on the tiebreaker

* Drop blank lines inside cond in sqlserver to satisfy cljfmt :remove-blank-lines-in-forms?

* make UserPicker UX even better + tests for it

* distinct ownerless and creator_active filters

* Notifications admin list endpoint: populate last_send.error

last_send.error was hardcoded nil at the list level — the Failing tab tooltip
had nowhere to read the channel-send error message from, even though the
notification's status correctly showed :failing.

- expose `lr.run_id` from latest-send-tick-per-card as `:ls_id`
- decorate-runs collects failed ls_ids and looks them up via
  `error-by-run-id task-channel-send …`
- error-by-run-id takes a `task-name` arg now: when non-nil, restricts the
  windowed lookup to that task (last_send wants channel-send specifically);
  when nil, keeps the tiebreaker (last_check, check_history)
- regression test: list endpoint last_send.error returns the channel-send
  task_details.message when the latest send-tick failed

* Simplify error-by-run-id ORDER BY + detail endpoint hydration

- error-by-run-id: always use CASE+ended_at as the partition ORDER BY; the
  CASE is a no-op when task-name restricts the WHERE to one task, so the
  conditional branch on task-name dropped.
- get-notification-detail: drop the decorate-runs call. base-list-query with
  `:skip-run-joins?` doesn't project lc_*/ls_* columns, so decorate-runs was
  a no-op pass (assoc'd nil into last_check/last_send which we overwrite
  immediately from the per-notification histories).

* removed (deactivated) label

* request users of any status in the notification's owner select

* small ux fixes

* Notifications admin include today (#74963)

* add includeToday param

* add Today toggle to the TaskRunDatePicker

* chores

* try to keep selected entity id and type if we are changing only includeToday

* add the tests

* addressing the PR's comments

* fix console error

* fix e2e tests due to TaskRunPicker changes

---------

Co-authored-by: Egor Grushin <egor@metabase.com>
2026-05-29 21:37:12 +01:00
Romeo Van Snick
b6fe71475d Fix deletion of tenant collections (#74536)
* Fix typo

* Make notifying with toast optional in useDeleteItem

* Don't hide unsuccesful bulk archives

* Add repro for 74461

* Fix wording
2026-05-30 06:33:10 +10:00
Romeo Van Snick
41d9fa501d Harden tests against XHR vs fetch timing differences (#74897)
* Fix flakes in cache test

* Fix flake in admin remote-sync test

* Fix snippet flake

* Fix geojson e2e test

* Return content type in server mocks

* Return content type in server mocks

* Fix unit test

* Address review: backwards-compatible lastReqBody, drop redundant within

lastReqBody (and the inline copy in AIQuestionAnalysisButton) assumed the
client calls `fetch(new Request(url, init))` and read the body via
`request.clone().text()`. On master/this branch the client still calls
`fetch(url, init)`, so the first arg is a URL string and `.clone()` throws —
breaking every consumer (history/context/errors/use-metabot + the button
spec). Read both shapes so it's actually backwards-compatible, and route the
button spec through the shared `lastReqBody` to drop the duplicate.

Also collapse the single-element `H.modal().within()` in remote-sync to a
simple chain.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-05-30 06:33:01 +10:00
Eric Normand
94f3c633ce GHY-3266 cancel slow data-warehouse queries via *canceled-chan* timer (#74776)
* GHY-3266 cancel slow data-warehouse queries when *query-timeout-ms* elapses

Statement.setQueryTimeout (added in #74xxx) handles the per-driver path,
but some drivers don't reliably issue a server-side cancel — most
notably the MariaDB connector for MySQL, where the c3p0
unreturnedConnectionTimeout closes the JDBC connection without sending
KILL QUERY, leaving the slow query running on the server. The
fingerprint sync then retries the same view every cycle, accumulating
runaway queries.

This adds a cross-driver backstop: do-with-canceled-chan now schedules a
go-block that puts ::timeout on *canceled-chan* after *query-timeout-ms*.
The existing wire-up-canceled-chan-to-cancel-Statement! then calls
Statement.cancel(), which for MariaDB issues KILL QUERY over a side
connection. Works for any driver with a working .cancel() impl, not just
JDBC.

* Don't consume from *canceled-chan* in the timeout-cancel timer

The first version of schedule-query-timeout-cancel! used
`(alts! [canceled-chan timeout-ch])` so the go-block would early-exit
when the query completed. That broke mongo's
kill-an-in-flight-query-test: the test binds `*canceled-chan*` to a
regular `(a/chan)` (not a promise-chan) and puts a cancel sentinel on
it; the driver listener takes from the chan to abort the in-flight
query. On a regular chan, `alts!` consumes the value, so the timer
go-block raced the driver listener and ate the cancel signal — the
mongo cursor was never aborted.

Replace the read on `*canceled-chan*` with a dedicated `done-ch` we
own. `do-with-canceled-chan` closes it in a `finally`, so the go-block
still exits promptly on completion. The timer never reads from the
caller-supplied channel — it only puts `::timeout` to it after the
timeout elapses, which is safe for both promise-chan and regular chan
callers.

Regression test added.

* Address review feedback on canceled-chan timer

- setup.clj: signal the timer via put! on done-ch rather than close!, so
  the intent ("the query is done") reads more naturally than
  close-as-signal on a promise-chan. Use identical? for the port check.

- setup_test.clj: simplify a deliberately-clever 300ms wait pattern to a
  plain Thread/sleep + deliver. Bound the @taken deref with a 2s
  timeout sentinel so a regression in the timer can't hang CI.

- mysql_test.clj: add a driver test that demonstrates end-to-end
  cancellation against MySQL — bind *query-timeout-ms* to 2s, run
  SELECT SLEEP(60), assert the query throws and completes far short of
  the natural 60s. Timing is the load-bearing assertion; the QP's
  output-schema middleware wraps the underlying SQLException so we
  don't pattern-match on the exception message.

* Refactor
2026-05-29 19:30:10 +00:00
Nemanja Glumac
fb4d33bb38 Fix auto-release rewinding to an older commit (#74768)
The auto-release flow could cut a higher version number from an older commit than the previous release. The fix is a single-function body change in hasCommitBeenReleased. Compare the last release tag's commit to the release branch tip and check membership of the candidate in the returned commits list.

The membership-against-branch-tip framing is deliberate: it enforces "release tags only come from release-x.NN.x" by construction. A candidate that's ahead of lastTagSha but not on the release branch is correctly rejected by the gate.
2026-05-29 11:51:17 -06:00
Timothy S. Dean
680c6ec1a0 Lock config-managed workspaces against runtime edits (#74929)
When a Metabase instance loads its workspace from `config.yml` at boot, that workspace is meant to be managed by the deployment config, not edited at runtime. Nothing stops an admin today from overwriting the config through the workspace-instance API. This leaves the instance state out of sync with the config file that would load on the next restart.

My approach was to treat a workspace as read-only when it's set at boot. This could be a `:workspace` section in `config.yml` or an `MB_INSTANCE_WORKSPACE` env var.

- `workspace-locked-by-config?` is true when the boot process loaded a workspace or the env var is set. The current lock state is not persisted.
- `/api/ee/workspace-instance/current` rejects `POST` and `DELETE` when the workspace is locked. `GET` returns `can_write`.

If you legitimately want to move from locked to unlocked, then remove your boot-time config, restart the instance, and then you can proceed to use the admin API calls.
2026-05-29 11:35:22 -06:00
Ben Grabow
51df60fa51 Add --attempt option to mage ci-report (#74975)
* Add --attempt option

* Support negative relative attempt index
2026-05-29 16:17:32 +00:00
Stas Gavrylov
0b10c75bdb GDGT-1582: Schema Viewer (Entity Relationship Diagram) (#71351)
* GDGT-1582: Add ERD endpoint

* GDGT-1582: First iteration of ERD

* GDGT-1582: Add e2e tests for ERD feature

* GDGT-1582: Fix lint errors

* GDGT-1582: Update backend to support models' ERD

* GDGT-1582: Add search picker. Add models support

* GDGT-1582: Update e2e tests

* wip

* GDGT-1582: Make db type always lowercase

* Add marching ants animation for edges

* Add table icon colors to nodes

* Add Search Input

* Improve entity picking behavior

* GDGT-1582: Add fields sorting

* GDGT-1582: Remove outdated ERD link

* Update schema viewer button icon

* Fix e2e tests

* Replace ERD abbreviature with SchemaViewer

* Refactoring

* Reorganize code

* Update Schema Viewer to support schema picker UI

* Rebuild table picker. Add hops input

* GDGT-1582: Fix tables list fetching

* GDGT-1582: Adjust selectors styling

* GDGT-1582: Fix hops handling on the BE, improve error message output

* GDGD-1582: Add support for expanding FK fields

* GDGT-1582: Add placeholder for table input when no tables are selected

* GDGT-1582: Adjust styling for Hops input

* GDGT-1582: Add schema viewing persistence

* GDGT-1582: Add share link to schema viewer

* GDGT-1582: update e2e tests

* GDGT-1582: Extend /erd to support foreign schema tables

* Update schema viewer icon to network

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* Fix table ids caching

* GDGT-1582: Adjust border styling

* GDGT-1582: Replace number input for hops with a slider

* GDGT-1582: Implement cardinality labels for edges

* GDGT-1582: Refactor

* GDGT-1582: Add table details link and warning on selecting all tables

* GDGT-1582: Improve table selector UX

- Sort visible (but not selected) tables below selected tables
- Add fixed height to list items to prevent layout shift
- Keep dropdown open when canceling select all confirmation
- Close dropdown when confirming select all

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* GDGT-1582: Add collapse/expand and double-click focus to table nodes

- Add collapse/expand button for nodes with more than 10 fields
- Shows first 5 fields when collapsed
- Double-click node to focus/zoom to it
- Double-click again to zoom out to fit all nodes

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* Remove unused active-fields-for-table function

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* REVERTME: Ignore DB deletion to alow staging environment to work

* REVERTME: Fix kondo warning

* GDGT-1582: Update styling of edges and cardinality marks.

* GDGT-1582: Add zoomed-out mode for schema viewer

* GDGT-1582: Add tooltip to compact nodes showing connected tables

Shows table name and list of connected tables with direction/cardinality icons.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* GDGT-1582: Show 20 fields by default and add field count to tooltip

- Change default collapsed field count from 5 to 20
- Start table nodes collapsed by default
- Add field count text under table name in compact mode tooltip

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* GDGT-1582: Add zoom-to-table on FK field click

When clicking on a foreign key field whose target table is already
visible on the canvas, zoom to that table instead of doing nothing.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* GDGT-1582: Fix table selector dropdown close on outside click

Use capturing mousedown listener to close dropdown before ReactFlow
intercepts the event.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* GDGT-1582: Update nodes positioning logic to rearrange in compact view

* GDGT-1582: Improve field zoom transition

* GDGT-1582: Fix positioning inconcistency

* GDGT-1582: Fix tooltip not showing on bottom nodes

* GDGT-1582: Add compact mode toggle button

* GDGT-1582: Update zooming logic

* GDGT-1582: Move button to expand all node fields

* GDGT-1582: Address feedback

* GDGT-1582: Update tests

* GDGT-1582: Update compact mode initialization logic. Remove node collapsing logic

* GDGT-1582: Fix the bug with incorrect number of selected tables displayed

* GDGT-1582: Fix nodes positioning flash

* GDGT-1582: Add AppSwitcher to Schema Viewer route

* GDGT-1582: Implement saving of last selected database to user preferences

* GDGT-1582: Fix table selector bugs. Expose table names

* fix test

* GDGT-1582: Add proper handling for empty schema names

* GDGT-1582: Fix table picker list items width

* Rewrite ERD endpoint: unified pipeline + lazy BFS (#70580)

* Fix nodes rendering on hops change

* GDGT-1582: Addressing design feedback

* wip

* Minor styling fixes

* wip

* Fix eslint

* Fix hops to allow unwrapping FK fields

* fixes

* Fix fk field focus

* Update Node search input

* Implement focus node menu

* Adjust styling

* Add node selection popup

* wip

* Add ?include=schemas param to /database. Remove unit tests

* Add db and schema display in popup. Fix fields list scroll

* Remove old proxy middleware

* Fix CI

* Update table nodes design

* Make search input clear on focus

* Adjust styling. Remove context menu. Add focus node button

* Cleanup

* Add spinner while external FK table is loading

* Cleanup. Fix stuck schem picker when DB is missing

* Update table fields design

* Add Upsell page for Schema Viewer

* SchemaViewer refactoring

* Reorganize code in folders

* More refactoring

* Simplify ERD backend expansion

* Consolidate/delete schema viewer hooks

* More refactoring. Delay /erd fetch until last preferences are restored

* More code reorganization

* Optimize node layout algorithms with caching and better positioning strategies

* Hooks refactoring. Fix stale prefs value

* Refactor zoom logic. Replace FitToCanvas and FitToNewNodes with reducer

* Refactor edges selection. Cleanup

* Optimize nodes rendering

* Optimize viewport zoom logic

* Extract applyLayout functionality

* Update focus node positioning algorithm

* Add minimap nodes shading via SchemaViewerMinimap component

* Move SchemaViewer out of dependencies feature

* Optimize layout functions

* Unit tests for layout functions

* Fix Urls import path

* Add Schema Viewer Upsell page graphics

* E2E and unit tests for Schema Viewer

* Fix table node border color

* Add dark mode support for Minimap

* Replace :dependencies feature token usage with proper :schema-viewer token. Minor tests refactoring

* Further cleanup

* Rename erd.core to erd.impl

core.clj is reserved for Potemkin re-exports of a module's external surface;
this module has no external consumers, so the implementation lives in impl.clj.

* wip

* Replace edges selection z-index setting with CSS solution. More cleanup

* Comments cleanup

* Extract GraphInfoPanel. Fix focus field zoom. Add schema to external table names

* Fix formatting

* Fix cljfmt

* Adjust height of Node panel popup. Remove focus node disabling logic.

* Fix schema selection input closing on outside click

* Register :schema-viewer feature in stats and token-features

Fix backend CI failures:
- Move enterprise/erd block to correct alphabetical position in module
  config and drop non-existent erd.core from :api set
- Add :schema-viewer to snowplow features data
- Add :schema-viewer to expected token-features map in session test

* Address ERD impl.clj review nits

- Make build-erd-field/node/edges/response private (only used in impl.clj)
- Rename readable-tables binding to readable-target-table-ids (it holds IDs)
- Use positional docstring for table-perm-columns instead of ^{:doc}

* GDGT-1582: Addressing review comments

* Extend useUserKeyValue typings to properly infer result value

* GDGT-1582: Addressing review comments

* GDGT-1582: Make table details redirect to Schema Viewer zoom on table

* GDGT-1582: Fix ref fields misalignment in InfoPanelField

* GDGT-1582: Resolve e2e tests comments

* GDGT-1582: Add table description and owner information to table nodes

* GDGT-1582: Rewrite SchemaPickerInput to use Minipicker. Extend MiniPicker to support schema browsing

* GDGT-1582: Cleanup hooks

* GDGT-1582: Update input focus hotkey to 'f'

* GDGT-1582: Replace display_name with name in InfoPanel title

* GDGT-1582: Refactor zoomToNodes

* GDGT-1582: Cleanup flow-graph util

* GDGT-1582: Refactor layout util by splitting to multiple files and introducing PlacementCanvas abstraction

* GDGT-1582: Fix e2e tests

* Fix TS errors

* GDGT-1582: Fix hotkey unit test

* Fix cljfmt

* GDGT-1582: Resolve PR comments

* GDGT-1582: Enable hidden and sensitive fields on /erd

* GDGT-1582: Refactor SchemaViewerEdge + update styling.

* GDGT-1582: Revert extracting GraphInfoPanel to shared. Create own components for Schema Viewer

* GDGT-1582: Make /erd return all fields

* GDGT-1582: Refactor selectedFieldIds computation logic. Cleanup

* GDGT-1582: Fix schema viewer staleness when table metadata is updated

* GDGT-1582: Cleanup

* GDGT-1582: Use lib in flow-graph utils

* GDGT-1582: Replace display_name with name if popup fields

* GDGT-1582: Refactor unit testing mocks

* GDGT-1582: Comments cleanup

* GDGT-1582: Fix fields wrapping in table node info popup

* GDGT-1582: Minor UI fixes. Fix e2e

* GDGT-1582: Fix BE test. Fix field name overflow

---------

Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
Co-authored-by: Nicolò Maria Mezzopera <nicolo.mezzopera@metabase.com>
Co-authored-by: Ngoc Khuat <qn.khuat@gmail.com>
2026-05-29 15:46:26 +00:00
Alexander Polyankin
e50a365f3f Fix pivot setting comparison when migrating from refs (#74972)
* Fix pivot setting comparison when migrating from refs

* fix

* fix
2026-05-29 15:11:47 +00:00
Timothy S. Dean
dc017f71a7 Don't email admin users who haven't accepted invite (#74883)
If someone has been invited as an admin to a Metabase instance, but has not yet accepted that invitation, that person should not get spammed with admin notification emails.

`last_login` = `nil` is our canonical mechanism for determining that a user has not yet accepted an invitation. Acquiring emails for admins should ensure this field is not `nil`.
2026-05-29 08:27:29 -06:00
Romeo Van Snick
c3f9cac18f Return 204 No Content from CSV append/replace endpoints (#74945)
* Return 204 No Content from CSV append/replace endpoints

The `update-csv!` helper hand-rolled a `{:status 200 :body nil}`
response map, which slips through the `wrap-response-if-needed` middleware
(see metabase.api.common.internal): the protocol only rewrites a returned
`nil` to a 204 — a map that already carries `:status` and `:body` is
passed through unchanged. The result was the only FE-facing endpoint in
the codebase that returned a 200 with a literally empty body, which an
HTTP client has to special-case (empty body but a JSON content-type means
`response.json()` throws).

Switch to the existing `api/generic-204-no-content` helper so the
response semantically matches what it always was — a side-effecting
operation with nothing meaningful to return — and so a future move to
`response.json()` in the FE client doesn't have to carry an
empty-body-200 special case.

The FE callers (`appendTableCsv` and `replaceTableCsv` in
frontend/src/metabase/api/table.ts) are typed `void` and the consumer in
frontend/src/metabase/redux/uploads.ts uses `response || modelId`, so
neither the old `null` body nor the new empty 204 changes observable
behavior.

* Fix pre-existing cljfmt violation in transforms timeout_test

CI runs `cljfmt-all --force-check` over every Clojure file, and
test/metabase/transforms/timeout_test.clj currently has blank lines
inside nested `testing` forms that cljfmt rejects. The violation lives
on master, so it fails the Cljfmt check for every open PR (this one
included). Apply the mechanical reformat so this branch's Cljfmt job
can pass.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-05-29 22:01:02 +10:00
Romeo Van Snick
54a770c486 Return empty body as null instead of the empty string (#74895)
* Return empty body as null instead of the empty string

When the BE responded with no body (e.g. `204 No Content`), the legacy
API client surfaced the missing body as the empty string `""`. Callers
then had to disambiguate "I got an empty string back" from "I got the
parsed JSON value that happens to be the empty string", which forced
per-endpoint `transformResponse` workarounds.

Surface empty bodies as `null` instead — the natural "no value" sentinel
in JS, and what RTK Query callers tend to type their response shapes
with already.

Client change:
- Both transport paths (XHR + fetch) coerce an empty body text to `null`
  before any JSON-parse attempt (skipping the parse for empty bodies
  rather than letting it throw and swallow).

Consumer changes (the only two places affected by the new shape):
- `groupTableAccessPolicyApi.getGroupTableAccessPolicy` drops its
  `transformResponse(response: GroupTableAccessPolicy | "" | null)`
  workaround that was coalescing `""` → `null`. The endpoint already
  typed its return as `GroupTableAccessPolicy | null`; the new client
  behaviour just hands it that shape directly.
- `useUserKeyValue` no longer relies on the `""` falsy-ness to fall
  through to `defaultValue` — `null` would `??`-coalesce correctly here
  too, but the comment is updated. The downstream consumer that did
  break is `UserHasSeen`: it had a `useUserAcknowledgement(id, true)`
  default of `true` because the old `""` empty body looked acknowledged
  *and* loading looked acknowledged *and* unacknowledged-key looked
  acknowledged. With the new `null` shape, switch the default to
  `false` and gate on `isLoading || acked` so a previously-dismissed
  banner does not flash before the real state is known. Adds a unit
  test covering the freshly-installed-user case (`null` response).

* Add unit coverage for empty-body → null

Lock in the new behaviour so a future change cannot silently restore
the empty-string return value. Three new specs against the fetch
transport (which is what Jest uses via `isTest`):

- An empty 204 body resolves to `null`.
- An empty 200 body resolves to `null` (not just 204 — any empty body).
- A non-empty JSON body still parses to its decoded value, so the
  empty-text short-circuit does not regress the happy path.

The XHR path mirrors the same `responseText === ""` check; co-locating
both transports under one mock would require a dedicated XHR fake which
this spec does not currently set up. The fetch-path coverage is what
ships in production for any caller running through the test suite.

* Fix unit test
2026-05-29 21:52:42 +10:00
github-automation-metabase
3ca8008471 Auto-update documentation for master (#74942)
Update generated docs (master)

Co-authored-by: github-actions <github-actions@github.com>
2026-05-29 07:47:37 -04:00
Romeo Van Snick
2ed4929758 Remove entity framework 🎈 (#74896)
* Move entityCompatibleQuery to metabase/api/utils/entity-compatible-query.ts

* Move entityTypeForObject / entityTypeForModel into metabase/redux/store/entities

* Move UseQuery rtk type to metabase/api/types

* Wire entity slice reducers directly in metabase/redux/entities

* Drop enhanceRequestsReducer wrapper

* Delete the entity framework

* Rename entityCompatibleQuery to runRtkEndpoint

* Remove dead _list index signature from EntitiesState

* Rename hydrateLegacyEntities to hydrateMetadataStore

* Add tests for new reducer

* Fix comment

* Remove unused Metadata methods

* Fix lint
2026-05-29 20:52:50 +10:00
Chris Truter
d587fb7ffe Boost library-collection items in search ranking (#74806)
* Boost library-collection items in search ranking

- Add :library scorer to the semantic engine (was appdb-only); extract the
  shared expression into metabase.search.scoring/library-score-expr so both
  engines use one implementation.
- Add root_collection_type column to the semantic index schema + ingestion
  path, with a corresponding dynamic-schema migration (v4) to alter existing
  index tables. Existing rows are NULL until re-ingested.
- Rebalance default weights: :library 200 in :default (was unset; only set
  under :metabot at 100); drop the :metabot :library override so it inherits
  the default. 200 comfortably outranks :official-collection 80 + :verified
  80 without dominating all other scorers.

* Bring :data-layer scorer over to semantic and DRY the helpers

- Extract data-layer-score-expr into metabase.search.scoring alongside
  library-score-expr so both engines share one implementation. Refactor
  appdb base-scorers to call the helper.
- Wire :data-layer into semantic base-scorers using the helper (column
  was already in the schema/ingestion from the previous commit).
- Add semantic data-layer-test mirroring the appdb version's per-tier
  weight checks.
- Migration 3 docstring: correct attribution — :library reads
  root_collection_type (added in migration 4), not collection_type.

* Lift curation-tier weights to :default and backfill root_collection_type

- Move :official-collection 80, :verified 80, and the :data-layer per-tier
  weights from :metabot up to :default. Library at 200 stays the dominant
  tier. The :metabot block was the only consumer of these magnitudes, so
  with them lifted the block is empty and removed.
- Extend migration 4 with an in-place backfill: pull root_collection_type
  from the gate's stored document JSON when present, else use the row's
  own collection_type if it's already a library root type. Nested items
  without a refreshed gate doc remain NULL until natural re-ingestion.
- Update metabot-weights-test to assert the new contract (:metabot
  inherits :default verbatim, no overrides).

* Document migration-4 backfill assumptions

Spell out why the `collection_type IN (library...)` fallback is sufficient
today (library-typed collections can't have sub-collections, so any row
with that direct-parent type is at the root) and flag what would need to
change if that ever becomes false. Also note that only the library root
types matter for the fallback — other root types stay NULL because the
`:library` scorer doesn't care about them.

* Drop metabot-weights-test now that :metabot inherits :default with no overrides

* Trim scoring helpers and tighten library-test predicate

- Drop the column-reference parameter from library-score-expr and
  data-layer-score-expr — both columns are unambiguous in every scorer
  subquery, so callers don't need to thread engine-specific qualifiers.
- DRY data-layer-score-expr's per-tier rows via a data-layer-tiers vec
  and a small tier-clauses helper.
- Use a distinctive 'lib-tree' token in the appdb library-test row names
  and check via str/includes?, so the predicate reads off the result
  instead of relying on hand-counted ids or a too-short 'lib' prefix.

* Factor data-layer magnitude up, mirror semantic library-test predicate

- Lift the data-layer magnitude into the top-level :data-layer weight (33)
  and reduce per-tier weights to pure ratios (1 / 0.3 / 0.03). Final tier
  contribution stays at 33, so the magnitude now sits naturally alongside
  :library 200, :official-collection 80, :verified 80 in the config.
- Rewrite data-layer-score-expr as a literal :case vector with a small
  tier-weight closure — clearer than the previous into/concat/mapcat chain.
- Tighten the helper docstrings: caller contracts only, implementation
  rationale moved to inline ;; comments.
- Mirror the appdb library-test's `lib-tree` substring predicate in the
  semantic library-test, and add a tier-ordering case to the semantic
  data-layer-test to match the appdb pair-test for parity.
- Update the appdb data-layer-test tier-ordering case to use the new
  :default magnitudes (was the old :metabot magnitudes).

* Fix semantic data-layer-test tier-ordering case

The tier-ordering case was reading off the per-tier sub-tests' 4-row index,
which includes a "table no layer" row. That row matches the "table" query
(tsvector filter, score-independent), so the assertion would yield [2 3 4 1]
instead of [2 3 4]. Wrap the ordering case in its own 3-row index, mirroring
the appdb counterpart.

* Bump semantic index version to 4 and update affected migration tests

Three existing tests broke because they pinned the previous schema state:
- schema-version-match-default-version-test asserts default-index :version
  equals dynamic-schema-version (now 4) — bumped default-index to 4.
- expected-db-schema-after-migration-test enumerates the index table's
  columns — added :root_collection_type to the expected set.
- init-semantic-search!-test failed due to the version mismatch propagating
  through metadata; resolves once default-index :version matches.

Also drop a couple of unnecessary rationale comments from the appdb and
semantic library-test fixtures — the row data speaks for itself.

* Tidy library-test row alignment in appdb and semantic

* Add migration-4 backfill integration test

Covers the four backfill branches end-to-end against a real pgvector instance:
(1) gate document JSON carries root_collection_type — pulled from there;
(2) gate doc absent but collection_type IS a library root — pulled from
    collection_type; (3) neither — stays NULL; (4) row already populated —
    untouched. The test rolls metadata.index_version back to 3 to force
    migration 4 to re-run against pre-seeded rows.

Also rename the tier-ordering test fixtures from "production table" to
"foo table" — the word wasn't pulling its weight.

* Fix migration-4 backfill test: use mock embedding model and read keys

The test needs the 4-dim mock model in scope so the placeholder embedding
matches the index column dimensions — switch to :mode :mock-initialized.
That also redefines the index-metadata table names, so resolve gate/meta
table names dynamically instead of hardcoding. Read result rows via
as-unqualified-maps so model_id and root_collection_type are accessible
without knowing the dynamic table-name qualifier.

Verified by running against a local pgvector container — 24 tests / 60
assertions pass across scoring-test and migration-test.

* Drop dead with-redefs from migration-4-backfill-test

semantic.core/init! doesn't call semantic.pgvector-api/index-documents! —
the redef was a defensive no-op carried over from sibling tests. Drop it
on this code path since the index_table only ever contains the four rows
inserted by the test itself.

* Annotate data-layer per-tier ratios with effective products

* Sentence-per-line comments in config curation block

* Migration 4 backfill: switch correlated subquery to two set-based UPDATEs

Copilot flagged the per-row correlated subquery into the gate table as slow
on large indexes. Split into two set-based UPDATEs:
  1. UPDATE..FROM join against the gate table (gate-sourced backfill)
  2. UPDATE against collection_type (library-root fallback)

Behaviour-preserving — confirmed by re-running migration-4-backfill-test
against a local pgvector instance.

* Update official-collection-tests ordering for new :official-collection weight

The legacy in-place scoring test asserted [d c b a] for the EE case, relying
on :official-collection at weight 1 being small enough that text matches
still dominated. With the weight lifted to 80 in :default, an official item
now overpowers text matches — d (official) rises to the top.

Test name and intent ("should bump up the value of items in official
collections") is preserved; the assertion just reflects the stronger bump.
Verified: 16 assertions pass.

* Split official-collection-tests into two deftests

The CI kondo run flagged this as containing several logically separate
testing forms. The two testing blocks (official-collection bump vs
verified-items bump) are independent concerns and the split is a clear
readability win.

Renamed to be more specific and use the standard `-test` suffix.

* Drop dead index-documents! redef from migration-lock-coordination-test

The init! code path (init-semantic-search! + gate-updates!) never reaches
index-documents!, so stubbing it in migration-lock-coordination-test's
with-redefs-fn was dead — same as the sibling init!-only tests. Remove it,
keeping the load-bearing maybe-migrate!/write-successful-migration! redefs,
and drop the now-unused semantic.pgvector-api require.

* Clarify ascending-sort mechanics in official-collection comment

The comment said d 'rises to the top' but d appears last in the asserted
vector because sort-by is ascending (highest score last). Reword to make
the ascending-sort placement explicit and avoid a reader misreading it as
a contradiction.

* Revert :data-layer to :metabot-only; keep library/official/verified in :default

The :data-layer scorer is a *structural* signal — every visible warehouse table
has `data_layer "final"` set from `visibility_type`. With :data-layer in
:default, the +33 boost applied to every visible table flipped orderings across
every search surface (search bar, entity picker, document mentions, smart-link
suggestions, etc.) and broke a number of e2e specs that pin specific top results.

Move :data-layer back to :metabot-only with a TODO documenting how to bring it
across in the future (update the e2e snapshot fixture or relax the brittle
assertions). The user-curated tiers (:library, :official-collection, :verified)
stay in :default — they don't break anything because nothing in the snapshot is
flagged with them.

Updated the tier-ordering test labels in appdb and semantic scoring tests to
say ":metabot magnitudes" instead of ":default magnitudes". The tests themselves
use explicit weight overrides so they pass either way.

* Migration 4 backfill: walk Library tree from appdb instead of trusting collection_type

Library sub-collections shipped in #73179 and changed the assumption that every indexed row
with a library `collection_type` is at the root of its own tree. The old heuristic
(`SET root_collection_type = collection_type WHERE collection_type IN library-types`)
fills the column for sub-collections but with the wrong root — a card under
Library/Data/Sub-Data gets `library-data` instead of `library`.

Replace step 3 with an appdb lookup that builds `collection-id → root-collection-type`
for every Library-typed collection (root + sub) and applies one UPDATE per distinct
root type. Library content rules guarantee the whole forest is covered by selecting
collections where `:type` is one of the library types — sub-collections inherit the
parent's type, so a single appdb query catches every depth.

Gate-doc backfill (step 2) still runs first and takes precedence; the appdb walk only
fills rows the gate doc missed (e.g. gate rows written before the spec field existed).

Catch + log on the appdb query so test setups that exercise pgvector before the appdb
schema is up don't fail — production semantic-search init always runs after appdb migration.

Test now stands up library/library-data/nested library-data/library-metrics via
mt/with-temp and asserts every nested case resolves to "library" while a non-library
collection's row stays NULL.

* Migration 4 backfill: walk the Library forest from roots instead of parsing locations

Filter the appdb query by `:location "/"` so we get only the library-typed *root*
collections, then fetch each root's descendants via `:location [:like "/<id>/%"]`.
We know the type for every descendant directly from its root — no path parsing.

Slightly more queries (1 + N for N roots, typically 1–3), but the code is straightforward
and stops conflating "the row is a library type" with "the row is the root of its tree".

* Tighten migration-4 docstrings; relocate implementer notes to inline comments

* Use u/for-map for the Library-forest mapping

* Trim diary comments in migration 4

* Whitespace from outer space
2026-05-29 06:11:39 -04:00
Sébastien
f3d99fc440 Fix flaky guest-embed happy path checkbox visibility (#74952) 2026-05-29 11:48:22 +02:00
Romeo Van Snick
7e79fbb727 Select schema if it is missing (#74586)
* Select schema if it is missing

* Also consider state.schema

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>

* test: run mongo native setup in beforeEach so burn/retries re-set up

The stress workflow (and CI retries) re-run each it via @cypress/grep burn,
which generates N separate tests. A before() hook only sets up the first one;
with testIsolation the page resets for the rest, so iterations 2..N fail with
'never found [data-testid=native-query-editor] .cm-content'. Using beforeEach
re-establishes the editor state for every iteration.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-29 18:53:18 +10:00
Uladzimir Havenchyk
01c49647f7 test: cleanup jest warnings from querying folder (#74753)
* test: cleanup jest warnings from query_builder folder

* test: cleanup jest warnings from querying folder

* fix last warnings

* cleanup

* cleanup and speed up joinstep
2026-05-29 11:13:03 +03:00
Uladzimir Havenchyk
e74ebaa14a chore: drop deprecated ButtonGroup (#74904)
* chore: drop deprecated ButtonGroup

* adjust ci

* Revert "adjust ci"

This reverts commit 7333dc3af8.

* Reapply "adjust ci"

This reverts commit 2ac27c8b69.
2026-05-29 11:12:12 +03:00
YB
44c2c839aa [GDGT-2078] add observability to transform cancelations (#73157)
The transform cancellation subsystem has no Prometheus metrics, no tracing spans, and only trace-level logging. We want better observability over both the user-initiated cancel path and the Quartz sweep that force-cancels stuck runs.

We are adding the following:

* Prometheus counter metabase-transforms/cancelation-requests with label {status} fired once per call to mark-cancel-started-run!
* Prometheus counter metabase-transforms/cancelation-completed with label {outcome}
* Prometheus histogram metabase-transforms/cancelation-latency-ms with label {outcome}, measuring request-time to completion
* Span task.transform.cancel in the :tasks group, with attributes run/id and transform/id
* New audit event :event/transform-run-canceled derived under ::transform-event, published from record-completion! with the full TransformRun row as the object
* Log-level change from trace to info on request, successful cancel, and force-cancel paths
* Also fixed a potential race in cancel-old-canceling-runs! by using a transaction lock on selected rows to be deleted.

* cancel old canceling runs in a transaction

* refactor review comments; properly return cancelled row data to be consumed by audit event log
2026-05-29 06:53:40 +00:00
Andrei Chernykh
1d39f46f11 Metics Homepage: Right side panel (UXW-4238) (#74809)
* Restructure metric About-tab right rail into Source and Relationships subsections (UXW-4238)

* Pluralize verb in '<N> charts use this metric' and tighten relationship-row construction (UXW-4238)

* Combine Last-updated label and relative time into one ttag template

* Inline relationship-row construction

* Use if/else over nested ternaries for relationship rows

* Render Source and Relationships rows declaratively, drop MetadataRow data type

* Polish right-rail metadata rows: white circle icon containers, larger About heading, plain verb phrase

* Detach right rail from chart card and lift sub-section cards above the page

* Connect metadata rows with a vertical line, use dependencies icon, loosen gaps

* Match Figma spec for the right rail and polish per code review

* Gate Dependencies tab and Relationships subsection by admin/analyst role (UXW-4174) and fix ttag extraction

* Test setup via real plugin init and dependency graph endpoint; switch rem() to Mantine size tokens

* Drop caching url from DescriptionSection spec to match the post-rebase MetricUrls type

* Refine right-rail metadata rows: dependent icon, hover color, spacing, and component split

* Seed dependencies token feature into the test store via renderWithProviders

* Update MetricAbout test anchor to the new Source heading
2026-05-28 21:35:09 -07:00
Fraser
6061e2f604 Fix DatePicker css import (#74947) 2026-05-29 15:54:24 +12:00
Fraser
982040aa43 Reclassify schema as shared tier (#74946) 2026-05-29 15:54:14 +12:00
Anderson
482d1dbea5 Fix Sankey click data for extra columns (#74283)
* Fix Sankey click data for extra columns

* Fix error: getting property of undefined on hover

* Address Nick's comment
2026-05-29 03:52:34 +00:00
Romeo Van Snick
2b88cdbb34 Convert the onResponseError to use EventEmitter event on API client and add event types (#74894)
* Use EventEmitter for response errors

`LegacyApi` already extends `EventEmitter` (and uses it for the
status-code events 401/403/etc), but the response-error notification
was a separate parallel mechanism: a public `onResponseError` callback
that the embedding SDK assigned at boot. Two ways for callers to
subscribe to client events on the same object.

Switch to the existing event channel:
- Remove the `onResponseError` field from `LegacyApi`.
- Replace the two `if (this.onResponseError) this.onResponseError(...)`
  guards (XHR path + fetch path) with `this.emit("responseError", info)`.
- Migrate the one external assignment (`api.onResponseError = handler`
  in the SDK init hook) to `api.on("responseError", handler)`. Dedupe by
  handler identity (matching the existing `beforeRequestHandlers`
  pattern with `reactSdkEmbedReferrerHandler`), so other listeners on
  the same event do not prevent the SDK from registering its own.
- Drop the now-unused `ResponseErrorInfo` type alias.

No behaviour change. Just one notification mechanism instead of two.

* Add stricter typing for api client events
2026-05-29 13:31:21 +10:00
Romeo Van Snick
ba93b7ebfb Replace raw option with :tag* URL template syntax (#74891)
The legacy API client supports `:tag` placeholders in URL templates that
get URL-encoded substitutions from request data. For one endpoint
(`/api/automagic-dashboards/:subPath`) the value is a multi-segment path
that must NOT be URL-encoded — slashes have to survive. The opt-out was
a per-call `raw: { subPath: true }` option, plumbed through
RequestOptions just for that one case.

Replace it with a single-character syntax extension: a trailing `*` on
the placeholder. `:tag` encodes (current behavior); `:tag*` substitutes
raw. The decision lives on the URL template, where it belongs, not in a
sibling options object.

Done in this commit:
- New `substituteUrlTags(url, data, method)` helper with its own unit
  spec covering both `:tag` (encoded) and `:tag*` (raw) cases.
- Legacy client uses the helper, drops the inline regex loop.
- `raw: Record<string, boolean>` removed from `RequestOptions` and
  `DEFAULT_OPTIONS`.
- The one caller (`AutoApi.dashboard`) migrates from
  `GET(url, { raw: { subPath: true } })` to `GET("/api/.../:subPath*")`.
2026-05-29 12:30:56 +10:00
Anderson
c63b8ad8b8 Fix stacked chart total labels for combo charts (#74247)
* Fix stacked chart total labels for combo charts

Closes https://linear.app/metabase/issue/UXW-3433/stacked-bar-chart-with-lines-bars-shows-zero-on-horizontal-axis

### Description

Limit stacked-total label placeholders to the series that are actually part of a stack, so line-only points in combo charts do not create stray zero labels.

### How to verify

1. Create a combo chart with stacked bars and a line where one x-axis bucket only has a line value.

2. Enable stacked total labels.

3. Confirm no zero label is rendered for the line-only bucket.

### Checklist

- [x] Tests have been added/updated to cover changes in this PR

* Update loki snapshot
2026-05-28 23:06:33 -03:00
YB
74494aff22 [GDGT-2079] add observability to transform timeout system (#73539)
Add Prometheus instrumentation and an audit event for transform-run and job-run timeouts, and replace the existing non-atomic sweep with a transaction so the emitted metrics can't drift from the rows actually transitioned to :timeout.

Changes:

* Counter :metabase-transforms/timeouts-total {type} - incremented per timed-out run; type is "transform" or "job".
* Histogram :metabase-transforms/timeout-detection-latency-ms {type} - how late the sweeper detected each timeout. Bounded by the 10-min sweep interval
* Audit event :event/transform-run-timeout - fires for both the single-run timeout-run! path and the sweeper path. Payload reflects the post-update :timeout row state.
* Tracing span task.transform.timeout-check {transform.timeout/type} on both Quartz jobs.

---
The transform-side sweeper previously caught and logged all exceptions inside timeout-transform-runs! (log/error from this namespace, then swallow). This wrapper is removed, as a wrapper inside tracing/with-span would have hidden - clj-otel's internal add-exception! only fires if the throw reaches it.

Recovery semantics are unchanged - Quartz catches the throw at the job-execution boundary, logs it at WARN with a JobExecutionException stack trace, and the next 10-minute trigger fires normally. The only functional change is observability: the failure now also populates the task.transform.timeout-check span with status=error.
The job-side sweeper previously used non-atomic UPDATE … RETURNING pks + post-SELECT. The new path is one transaction with SELECT … FOR UPDATE + UPDATE -> this is portable across H2/MySQL/Postgres.

Also fixed a test-isolation bug in metabase.transforms.jobs-test/timeout-old-runs-{notifies,does-not-notify}-*-test - the tests assumed :crowberto was already inserted by suite ordering, so they failed when run alone with :only. Tests now explicitly (mt/fetch-user :crowberto).
2026-05-29 09:45:15 +08:00
Romeo Van Snick
48446fa1aa Replace cancelDeferred with AbortSignal and honor the signal everywhere (#74889)
* Use AbortController in data-fetching instead of deferred

* Use signal instead of cancelDeferred

* Remove unused controller option from legacy-client

* Remove deferred arg

* Pass missing signal

* Handle already-cancelled signal

* Use abortcontroller to abort in test

* Allow delay cancellation

* Remove race in aborting a delay promise

* Cancel retry-backoff waits with the request signal

The retry loop in `_makeRequestWithRetries` was the last place where an
aborted request still finished its wait. `delay()` already accepts an
`AbortSignal`; pass `options.signal` through so an abort interrupts the
backoff sleep immediately instead of running it to completion before the
next attempt (which would only then see the aborted signal).

* Avoid another retry when cancelling

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>

---------

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
2026-05-29 11:08:13 +10:00
Fraser
0a5e7eec56 Decouple ObjectDetail from QB (#72845) 2026-05-29 12:24:07 +12:00
Mihael Konjević
0b269fb4b4 Merge misplaced trailing options-map on fixed-arity MBQL clauses (#74926)
LLMs sometimes place a clause's options at the END of a fixed-arity
tuple clause instead of position 1, e.g.

  ["time-interval" {} <expr> -1 "month" {"include-current" true}]

`time-interval` is a strict 5-tuple, so the extra element fails Malli
with a generic tuple-shape error and the construct_notebook_query tool
rejects the query. The trailing map is unambiguously misplaced options
(no fixed-arity tuple clause legitimately has a map-typed last arg).

Add a generic repair pass `merge-trailing-options*` that detects this
across every fixed-arity tuple clause registered with
`metabase.lib.schema.mbql-clause` — the expected-element-count table is
derived once at namespace load from `tag-registry`, so new fixed-arity
clauses get coverage automatically. Variadic `:catn`/`:repeat` clauses
(`and`, `or`, `case`, `coalesce`, `=`, `in`, …) are excluded — their
arity isn't fixed, so a trailing map there could be a legitimate arg.

When the pattern matches, merge the trailing map into position-1 options
(trailing keys win on conflict, since the trailing map is where the LLM
deliberately wrote content) and drop the trailing element. Idempotent by
construction.

Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-28 17:27:29 -04:00
technomancy
0726c32209 Check for VolumeOperationAllowedLocalPaths connecting to databricks. (#74375)
* Check for VolumeOperationAllowedLocalPaths connecting to databricks.

* Move additional options check to validate-db-details! method.

* Review suggestions: check a few other cases.
2026-05-28 19:02:33 +00:00
Andrei Chernykh
ed8e23b76f Move Explore button into the chart card (UXW-4237) (#74810)
* Move Explore button into the chart card (UXW-4237)

* Address review: drop dead mock, move zIndex to CSS module

* Drop now-redundant Group wrapper around single Menu

* CSS cleanup: drop dead border-radius, move position:relative into class

* Fix metric-page e2e: explore link is now on the chart card, not the header

* Bump Explore button overlay spacing to xl (32px)

* Drop module mocks in MetricAbout spec; use real card metadata + endpoint stubs

* Use size=xs on Explore button for closer-to-design height
2026-05-28 12:02:08 -07:00
appleby
d4238abb10 BOT-1524 Save tables used in a metabot conversation (#74254)
* BOT-1524 Save data sources used in a metabot conversation

* Handle implicit joins

* Simplify query-tables-and-cards with lib/all-referenced-entity-ids

* Cleanup native-tables parsing, remove uneeded macaw-tables

* Use lib more. banish mbql destructuring and raw mbql in tests

* Remove MetabotUsedTable from module config model-exports since nothing external uses it

* banish deprecated mt/mbql-query in tests

* Clean up docstrings in used_tables.clj

* Calm down with "MBQL 5" references

* Improve description for metabot_used_table

* Push seq down into when

* whitespace cleanups

* Use with-temp rather than t2/insert! and t2/delete! in used table tests

* Small formatting cleanup in persistence tests

* Use t2/with-transaction :rollback-only true rather than mt/with-model-cleanup in persistence tests

* Cleanup finalize-records-used-tables-test

* Formatting cleanup in ->transform-sql-parts and ->transform-python-parts

* More persistent test cleanups

* Remove private mp defn for metadata provider

* Rename provider to mp

* Formatting cleanups in used tables tests

* Pass in metadata-provider to extract-used-tables defaulting to lib-be/application-database-metadata-provider

And update the tests to take advantage of not need real cards in the app db and mark tests ^:parallel

* Use mt/with-dynamic-fn-redefs and mark tests ^:parallel

* Remove explicit ^:sequential

* Improve testing description strings

* Improve combined tool calls used tables test

* Only visit cards shared between tool calls once in extract-used-tables

* Fix trailing whitespace

* Fix lint warning about with-redefs

* Fix race in app-mention-triggers-response-test

* Add prometheus metrics for used table extraction duration and failures

* Move used tables extraction to background thread

* review feedback: remove macaw mentions

* review feedback: composed transducer rather than lazy seq

* review feedback: add some debug logs with more context

* review feedback: add bounded queue and increase threads to 4 for the used tables extraction executor

* Add 15s timeout via u/with-timeout for used tables extraction

* Add separate worker and waiter executors to ensure work remains bounded

Don't use u/with-timeout which runs the extraction on the shared future thread pool. Instead, create a separate worker
pool to ensure worker threads remain bounded even if they are stuck in cpu-bound state where they ignore thread
cancellations.

* Rename extract-used-tables-with-timing to extract-used-tables-with-timing!

* Allow idle threads to be reclaimed after 5 minutes

* Add test for extraction timeout and improve test for queue saturation
2026-05-28 14:35:06 -04:00
Cam Saul
7fa844b5bb Revert formatting-breaking commits (#74921)
* Revert "Use existing commit for cljfmt (#74885)"

This reverts commit d4d8c53d83.

* Revert "Fix cljfmt formatting  (#74892)"

This reverts commit ef186e4a60.
2026-05-28 11:25:03 -07:00
Timothy S. Dean
b355eb27f9 Gate workspace config section on :workspaces feature (#74864)
Loading the `:workspace` config-file section (at boot or at runtime via `POST /api/ee/advanced-config/`) only required the `:config-text-file` feature. They should both require `:workspaces`.
2026-05-28 11:51:51 -06:00
Mihael Konjević
9382445c5c Reduce and converge Metabot system prompts on objective-based style (#74758)
* Reduce and converge Metabot system prompts on objective-based style

Converge the embedding and internal system prompts onto the lean,
objective-based style of the NLQ prompt, and collapse the duplicated
verification/grounding guidance into a single source of truth.

- New shared includes (grounding, data-sources, mbql-shape, communication,
  discovery) lifted from the NLQ prompt; nlq/embedding/internal now compose
  them. Retire critical-constraints, verification-triggers, samples-vs-results,
  disclosure-protocol, and anti-patterns (the verification rule was restated
  ~6x; now once).
- internal.selmer 447->107 lines, embedding-next.selmer 419->79; permission
  gating kept but expressed through stable <!-- gate:X -->/<!-- denial:X -->
  anchors so wording can change without breaking tests.
- Verification reframed from "verify EVERY literal, no exceptions" to
  "verify literals whose format you can't predict" to stop wasteful
  sample reads; narration/clarification contradictions resolved.
- Discovery reframed navigate-first: read_resource is a first-class peer of
  search. Fold the four near-identical search prompt files into the discovery
  include; move search parameter guidance into Malli field descriptions and
  enrich the tool docstrings.
- De-duplicate the SQL read-only contract: canonical in create_sql_query.md,
  pointer in edit/replace, one-liner in internal.
- prompt_gating_test asserts on the new anchors instead of prose.

* Address PR review on Metabot prompt converge

- Include grounding/data-sources/discovery in the SQL profile so it gets
  the same discovery and verification guidance as the other profiles
  after the per-tool search prompts were retired.
- Drop the literal tool-name reference in grounding.selmer so the snippet
  is not coupled to specific tool names; discovery.selmer keeps the
  names since it is specifically about distinguishing them.
- Move shared snippets under shared/prompt_snippets/ to mark the
  directory as a contract: files placed there must be safe to embed in
  the cacheable prefix.
- Add prompt-cache tests asserting no Selmer directives appear in
  shared/prompt_snippets/ and that the cacheable prefix of every system
  template is stable across per-request values (current_time, user
  info, viewing context, recent views).

* Tune SQL profile to address regressions found in benchmark runs

- Add an SQL-specific "Searching effectively" section to the SQL
  profile so the agent gets concrete multi-concept and
  semantic/keyword search examples that used to live in the deleted
  sql_search.md.
- Replace the previous "Handle Uncertainty" framing (which described
  ask_for_sql_clarification as "your only way to send text") with an
  explicit tool-routing table. SQL bodies go through
  create_sql_query/edit_sql_query/replace_sql_query;
  ask_for_sql_clarification is reserved for short clarifying
  questions about structural ambiguity only.
- Rewrite ask_for_sql_clarification.md from the ground up: scope to
  short clarifying questions, explicit denylist for SQL bodies and for
  questions discovery can answer, and a one-sentence constraint on
  the question argument.
- Reinforce the routing in the profile's Remember section so the
  cache-prefix tail nails the rule.
- Bump SQL profile max-iterations from 10 to 20 so the agent has
  budget for the longer discovery patterns the new prompt encourages.

* Silence reflection warning in prompt_cache_test

Add (set! *warn-on-reflection* true) so the type hint on ^java.io.File
in snippet-files is checked by the compiler instead of producing a
clj-kondo warning.
2026-05-28 19:41:00 +02:00
Uladzimir Havenchyk
6954909b94 deps: use safe version of js-cookie (#74778)
* deps: use safe version of js-cookie

* pin
2026-05-28 17:37:16 +00:00
Nick Fitzpatrick
b4ebd35fcd Test connection button for Remote Sync (#74665)
* Test connection button for Remote Sync

* remove branch and mode from connection test
2026-05-28 23:05:31 +05:30
Nick Fitzpatrick
dca533be70 Fix Field Filter when there are no search results (#74832) 2026-05-28 23:03:23 +05:30
Phoomparin Mano
ef1d510edc Do not return MCP Apps tools for clients without UI capability (#74743)
* do not return mcp apps tools for clients without ui capability

* use a stronger shape for mcp ui capability detection

* tidy up session.clj session helpers

* add extended session id validation

* mcp ui tools should declare required client extensions

* remove rollback for varchar length extension

* update argument order

* parse uuid with parse-uuid utility

* log when capability payload is blank

* log when capability payload cannot be decoded

* warn if the mcp session id is too long

* document preserving slash with json/decode+kw

* use with-dynamic-fn-redefs for unit test

* remove unused supported-extensions option

* check scope matches first

* re-add non-null constraint after varchar(254) change

* disable mcp apps ui support if capability payload version mismatches

* allow known/older version for capability payload

* fix cljfmt lint
2026-05-28 13:11:32 -04:00
Alexander Polyankin
f6de634363 check pg grant statements rights (#74909)
* check pg grant statements rights

* tests

* fix

* redshift

* fix

* fix

* fix

* fix
2026-05-28 11:56:31 -04:00
Mahatthana (Kelvin) Nomsawadi
4e0dfc2136 bump qs (#74898)
deps: bump qs ^6.14.2 -> 6.15.2

Fixes Snyk medium-severity advisory SNYK-JS-QS-16721866
(CVE-2026-8723, NULL pointer dereference in stringify with
arrayFormat=comma + encodeValuesOnly when arrays contain
null/undefined).
2026-05-28 15:30:52 +00:00
Eric Normand
dd9d119b4c GHY-3701 force let-user-control-scheduling false in workspace config (#74913)
GHY-3701 force let-user-control-scheduling false for non-stub config entries

build-workspace-config emits :details only — not the schedule cron
columns. infer-db-schedules asserts cache_field_values_schedule and
metadata_sync_schedule are populated whenever :let-user-control-scheduling
is true, so a source DB that opted in would blow up the round-trip on
import. Override the key to false in database-entry so the receiving
side takes the auto-schedule branch. Stubs are unaffected — they emit
:details {} and already fall through to the auto-schedule branch.
2026-05-28 15:26:45 +00:00
Alexander Polyankin
db65e87b13 Workspace setup entry point from zero state (#74887)
* setup changes

* fix

* e2e fixes

* fix

* fix

* fix
2026-05-28 15:01:09 +00:00
Romeo Van Snick
ef186e4a60 Fix cljfmt formatting (#74892)
Fix formatting
2026-05-28 17:14:57 +03:00
Braden Shepherdson
72eb2f945c [perms] Fix N+1 query in sandboxing when updating perms (#74734)
Even when sandboxing is not used, 1 read was being generated for each
group X table whose permissions were impacted by the update. In a sample
query from a user, that was 28 groups X 382 tables =  10,696 reads.

This change includes a unit test that synthesizes this same scale and
makes the same request. It did reproduce the same 10,700+ requests the
user was seeing initially, but with these changes to how sandboxes are
checked (one query per database) it now only makes 87 queries. That's
still a lot but not unreasonable for a complex request.
2026-05-28 09:57:54 -04:00
Sébastien
900e1abd5a Fix embedModalEnableEmbedding race with pending React remount (#74815) 2026-05-28 15:50:42 +02:00
Chris Truter
0fe9d30bea Distinguish empty Metabot prompt regeneration from errors (#74406)
* Distinguish empty Metabot prompt regeneration from errors

When an admin clicks "Regenerate suggested prompts" today, three different
outcomes look the same: the LLM crashed, the metabot's library is empty
(no models/metrics to summarise), or the LLM ran but produced no
questions. All three previously returned `204 No Content` and the UI had
no way to tell them apart.

This commit makes `generate-sample-prompts` return a structured outcome
the API endpoint surfaces as JSON:

  {:status :generated  :prompt-count N}             - happy path
  {:status :empty :reason :no-library-content}      - nothing to summarise
  {:status :empty :reason :ai-produced-no-prompts}  - LLM returned 0 questions
  {:status :empty :reason :managed-free-limit-reached}
                                                    - managed AI cap reached

The empty-library and managed-free-limit cases skip the LLM call entirely
so we don't burn tokens generating nothing. The endpoint now returns
`200 OK` with the structured body in place of `204 No Content`; the
response schema is declared so docs and TypeScript signature generation
pick it up.

Errors (LLM 5xx, network, etc.) still surface as exceptions and turn
into 4xx/5xx responses — those remain distinct from the `:empty` cases.

Frontend follow-up needed: the admin panel's regenerate-button toast
should branch on `:status` and `:reason` to render a friendly empty-state
message instead of a silent success. Not in this PR (backend-only).

* Make regenerate-prompts response a discriminated union

Collapse the previous `{:status :empty :reason :x}` shape into a single-tag
union `{:status :x}` so callers can switch on one field, and rename
`:prompt-count` to `:prompt_count` to match wire conventions.

* Handle each regenerate-prompts status in admin UI

Surface the new no-library-content, ai-produced-no-prompts, and
managed-free-limit-reached statuses with their own toast copy instead of
collapsing them into a generic error.

* Tighten regenerate-prompts response and tests

- Drop :managed-free-limit-reached from the discriminated union; the 402
  fires before generate-sample-prompts is ever called, so the case is
  unreachable from this endpoint.
- Match exhaustively in the admin UI via ts-pattern so future status
  additions are a compile error.
- Add admin-UI unit tests for the no-library-content,
  ai-produced-no-prompts, and happy-path branches.
- In the backend test, assert that :prompt_count in the response equals
  the actual MetabotPrompt row count (not just the mock's promise).
- Realize the per-source prompt seqs into vectors so we don't re-run
  ->prompt between counting and inserting.
- "summarise" → "summarize".

* Close TOCTOU race on managed-AI lock in regenerate-prompts

`generate-sample-prompts` now calls `check-metabase-managed-free-limit!`
itself instead of soft-returning `{:status :managed-free-limit-reached}`.
That makes the response union genuinely closed: there's no longer a window
where the endpoint's pre-check passes, the function-level check then sees
the lock flipped, and a status not in the Malli `:multi` schema slips
through (which would have surfaced as a 500 from defendpoint validation).

`update-metabot` still wants best-effort behavior — a settings PUT
shouldn't 402 just because prompt regeneration was raced by the lock —
so wrap its `generate-sample-prompts` call in a try/catch that swallows
the locked-account exception and re-throws everything else.

Add a test for that race-tolerance path, asserting both that the locked
402 is swallowed and that a non-locked ex-info still propagates.

* Tighten happy-path regenerate test anchor

The previous `waitFor(negative-assertion)` succeeded immediately because
no toast renders synchronously on click — by the time the mutation
settled, the assertion had already passed, so a regression where the
happy-path branch wrongly fell through to an empty-state toast could
slip past.

Anchor on the button's idle text instead: "Regenerate suggested prompts"
only re-renders after `isRegenerating` flips back to false, which is the
same render where any mistaken toast would have appeared. Now the
negative assertions run only after the mutation has actually resolved.

* docstrings: trim design-rationale from generate-sample-prompts + endpoint

Both docstrings carried multi-paragraph design rationale ("single discriminator
so the frontend can switch on one field", "closed union so the regenerate
endpoint can declare an exact response schema", "no TOCTOU race between
checks"). All of that is implementer-context for someone who'd refactor the
design, not caller-contract material the IDE-hovering caller needs.

Slim each docstring to the caller contract: what's returned, when 402 fires,
and what callers who want best-effort behaviour should do about the
managed-free-limit race. The schema declaration above the endpoint already
shows the exact response shape.

* Tighten regenerate-prompts response schema

Two narrower invariants in the discriminated union:

- `{:closed true}` on each variant so the OpenAPI / TS output is an
  exact union instead of permitting `additionalProperties`.
- `:prompt_count` is `pos-int?`, not `nat-int?`. The `:generated`
  variant is only returned when total > 0 — the zero case branches
  to `:ai-produced-no-prompts`. Encoding that in the schema removes
  an impossible state from the contract.

Inline comment points the next reader at the function that decides
which branch is returned.

* Pre-guard managed-AI lock in startup prompt task

After 4831cc9a74c made `generate-sample-prompts` always call
`check-metabase-managed-free-limit!`, locked-instance startups would
fail the call and surface as `log/errorf` from the generic catch in
`maybe-generate-suggested-prompts!`. That's noise — the locked state
isn't a failure, just a "not now."

Add a `metabot.usage/managed-free-limit-reached?` branch to the cond so
the task logs a clean info-level skip when the cap is hit, mirroring
the existing "Metabot is disabled" skip. The catch still backstops real
exceptions.

The existing locked-provider test already pins the outcome (no prompts
persisted, generator never called); the new path just changes how the
skip is logged.

* Wrap PUT regenerate in a transaction so a failed regen keeps prompts

* Test that PUT prompt regen rolls back the delete on failure

* Sync OpenAPI specs with regenerate endpoint docstring

* Plainer regenerate-prompts docstring in API docs

* Regenerate OpenAPI specs to include regenerate-prompts response union

The earlier sync commit (9ab0fc923d) updated only the endpoint
description, leaving `responses.2XX` as the generic placeholder. Copilot
review on #74406 flagged that generated clients still wouldn't see the
discriminated union this PR introduces.

Run `bun run generate-openapi` and `clojure -M:ee:doc api-documentation`
so both `resources/openapi/openapi.json` and `docs/api.json` carry the
full `oneOf` schema for the 2XX response (`generated` /
`no-library-content` / `ai-produced-no-prompts`). The regen also flushes
unrelated component drift (workspace schemas, etc.) that the nightly
auto-update workflow had not yet caught up to.

* Drop unrelated EE endpoint drift from docs/api.json

* Drop stray blank lines flagged by cljfmt

The earlier merge commit (`8684854bc85`) was made with `--no-verify`
because husky's pre-commit cljfmt was getting SIGKILL'd mid-merge.
CI caught two stray trailing blank lines inside `let`/`with-redefs`
bodies the merge introduced — apply `bin/mage cljfmt-files` to them.

* Use mt/with-dynamic-fn-redefs for the new function-mocking sites

Copilot review on #74406 flagged four new mocks that use
`clojure.core/with-redefs` to redefine defn-style vars (the project
clj-kondo hook prefers `metabase.test/with-dynamic-fn-redefs` because
it's thread-safe).

Switch the four sites I added in this PR:
- metabot-put-skips/recovers tests for managed-AI lock
- metabot-prompt-regenerate-empty-states-test (no-library-content and
  ai-produced-no-prompts branches)

Leaving the two pre-existing `with-redefs` sites in this file (the
ones not flagged by the reviewer) untouched — those weren't introduced
by this PR.

* nits

---------

Co-authored-by: Sloan Sparger <sloansparger@gmail.com>
2026-05-28 07:52:01 -05:00
Sébastien
84cfb80f6a Update test to reflect label change (#74906) 2026-05-28 14:44:28 +02:00
Ngoc Khuat
fabf312a17 Log resolved email recipients when sending an email (#74816) 2026-05-28 09:21:09 +00:00
Fraser
ccef44cf5c Move NoDatabasesEmptyState into common (#74813) 2026-05-28 20:55:41 +12:00
Uladzimir Havenchyk
07ba1d3459 test: cleanup jest warnings from query_builder folder (#74640)
* test: cleanup jest warnings from query_builder folder

* fix question download widget test

* proper fixes
2026-05-28 11:15:53 +03:00
Ngoc Khuat
52bd3beefd Only re-validate incremental checkpoint field when source changes (#74752) 2026-05-28 15:02:37 +07:00
Romeo Van Snick
5325f240e9 Remove Question entity (#74652)
* Migrate question entity-framework hooks off the Questions entity

* Migrate read-only Question consumers off the Questions entity

* Migrate dashboard card actions off the Questions entity

* Migrate query-builder card actions off the Questions entity

* Remove the questions entity

* Do not drop dashcard sizes in api call

* Handle errors in card query

* Add error/loading state to action creator modal

* Avoid refetching when forbidden

* Avoid stale cards in card mutations
2026-05-28 13:17:56 +10:00
Romeo Van Snick
d4d8c53d83 Use existing commit for cljfmt (#74885) 2026-05-28 12:45:02 +10:00
Phoomparin Mano
a0f8e76a88 Hide feedback buttons for MCP Apps when not on cloud (#74742)
hide mcp feedback buttons if not enabled
2026-05-28 00:28:12 +00:00
github-automation-metabase
ce339dfb19 Update translations 2026-05-26 (#74744)
* Update translation sources 2026-05-26

* Update translations

---------

Co-authored-by: github-actions <github-actions@github.com>
2026-05-27 22:57:39 +00:00
bryan
0537620660 Fix Redshift iso-user leak in test cleanup (#74867) 2026-05-27 16:51:31 -06:00
Ryan Laurie
d29198d4fa Auto-add issues only to open milestones (#74871)
only auto-add issues to open milestones
2026-05-27 22:43:23 +00:00
github-automation-metabase
aabdd5ceb7 Auto-update documentation for master (#74874)
Update generated docs (master)

Co-authored-by: github-actions <github-actions@github.com>
2026-05-27 17:50:43 -04:00
Fraser
bae37c4861 Move setIsNativeEditorOpen into shared (#74807) 2026-05-28 09:38:54 +12:00
Eric Normand
f6bf515793 GHY-3687 Handle is_sample database (#74839)
* GHY-3687 expose sample-database-name from sample-data.core

* GHY-3687 workspace-manager /config emits sample-database entry

* GHY-3687 advanced-config loader handles is_sample

* GHY-3687 add sample-data to driver-affecting-overrides

sample-data is fixture/loader infrastructure, not driver logic. With
enterprise/workspaces (a trigger) now depending on sample-data, the
module would otherwise nudge modules-triggering-drivers from 41 to 42.
2026-05-27 21:12:32 +00:00
bryan
29d703e9d2 MCP: read_resource, write tools, execute_sql (#74458) 2026-05-27 14:34:57 -06:00
Jeff Bruemmer
67745bd3c2 docs - adds dedicated AI privacy page (#74669) 2026-05-27 20:13:57 +00:00
Andrei Chernykh
520fece1d0 feat: Move Caching tab into the overflow menu (UXW-4236) (#74563)
* Move Caching tab into the overflow menu (UXW-3839)

* Refactor caching strategy form and migrate styled-components (UXW-3839) (#74666)

* Open metric caching settings in a modal (UXW-4236) (#74682)

* Remove legacy metric caching page (UXW-4236) (#74691)

* Remove legacy metric caching page (UXW-4236)

* Address review feedback (UXW-4236)

* remove a comment

* Migrate StrategyFormLauncher to CSS module (UXW-4236)

* Merge duplicate @media query in StrategyFormLauncher.module.css (UXW-4236)

* Address review feedback (UXW-3839)

* Use UnstyledButton instead of Anchor for cache strategy trigger (UXW-3839)

* Trim verbose code comments (UXW-3839)

* Address review feedback: rem units, flex alignment, Cancel discard prompt (UXW-3839)

* Fixed-height caching modal and consistent Cancel behavior (UXW-3839)

* Restore Radio description prop and forward description clicks to input (UXW-3839)

* Make strategy radio descriptions clickable via second label (UXW-3839)

* Address review: Radio description prop, modal CSS module, S import name (UXW-3839)

* Tighten Caching modal typography, spacing, and inline-style cleanup (UXW-3839)

* Restore pointer cursor on strategy option label (UXW-3839)
2026-05-27 13:06:42 -07:00
Eric Normand
601be9c2de GHY-3686 Add stub databases (#74789)
* GHY-3686 add is_stub column to metabase_database

Boolean column, NOT NULL, default false. Marks placeholder databases
created during deserialization so they can be excluded from sync and
connection-test paths. Cleared when real details are supplied and the
main connection test passes.

* GHY-3686 serdes :copy + :defaults for is_stub

Round-trip serialization preserves is_stub when true, and elides it
from the exported map when false (matching the column default).

* GHY-3686 reject is_stub in REST API, clear on successful main-conn test

POST /api/database and PUT /api/database/:id throw 400 when :is_stub
appears in the body — setting it is reserved for the advanced-config
code path. PUT clears :is_stub to false when :details or :engine change
and the main connection test passes; write_data_details / admin_details
changes do not clear it.

* GHY-3686 skip stub databases in sync scheduler and job

sync-and-analyze-database! short-circuits when the database is a stub,
mirroring the audit-db guard. check-and-schedule-tasks-for-db! refuses
to register Quartz triggers for stubs.

* GHY-3686 advanced-config loader accepts is_stub

Config-file entries with `is_stub: true` skip the connection-test
gate and the post-insert sync trigger. Spec now optionally accepts
:is_stub as a boolean.

* GHY-3686 inject stub entries into workspace-manager /config

build-workspace-config now appends stub entries to :config.databases
for every database in the instance that isn't already provisioned for
the workspace. Sample (handled by GHY-3687), audit, and routing-target
databases are excluded.

* GHY-3686 chunk workspace-db-ids into 500-row NOT-IN clauses

Avoid feeding an unbounded sequence into a single SQL IN clause —
Oracle's IN cap is 1000 and other drivers have their own parameter
limits. partition-all yields zero chunks (and so no NOT-IN clauses)
when the workspace has no databases, which is the prior behavior.

* GHY-3686 chunk workspace DB lookup IN clause to 500-row batches

Same reasoning as the NOT-IN in stub-databases: a workspace with many
provisioned databases would otherwise hit driver IN-clause limits.
Empty workspace_db_ids yields zero queries (no chunks), matching the
prior empty-case behavior.

* GHY-3686 add :is_stub to table-test db-details fixture

The new is_stub column on metabase_database now appears in API
responses inside the embedded :db map; table-test fixtures asserting
exact :db shapes need to list it. Same pattern as the other is_*
flags in this fixture.

* GHY-3686 stub config entries no-op when a real DB already exists

The round-trip workflow (build-workspace-config -> YAML -> initialize!)
emits stub entries for every non-workspace DB in the instance. When that
config is applied back to the same instance, the stub entries collide
by name+engine with the real databases — without this guard the loader
would update them with :details {} and :is_stub true, breaking working
connections (surfaced as "FATAL: role <user> does not exist" in workspace
e2e tests).

Stubs only ever create rows; they never overwrite existing ones.

* GHY-3686 revert workspace DB IN/NOT-IN chunking

Restoring the single-clause IN and NOT-IN queries in
build-workspace-config and stub-databases. A workspace will not
realistically accumulate enough database refs to hit driver parameter
limits.
2026-05-27 19:59:53 +00:00
Nemanja Glumac
aa84834af4 Upgrade various GitHub actions (#74850)
* Pin nowactions/envsubst comment to v1.0.1

SHA unchanged; the # v1 comment now reflects the exact release.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* Pin trunk-io/analytics-uploader to v2.0.9

Was floating on @main (unpinned); pin to an immutable SHA.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* Upgrade anthropics/claude-code-action to v1.0.133

Pin the exact v1.x release instead of the floating v1 major.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* Upgrade crowdin/github-action to v2.16.2

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* Upgrade hoverkraft-tech/compose-action to v2.6.0

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* Upgrade lycheeverse/lychee-action to v2.8.0

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* Upgrade aquasecurity/trivy-action to v0.36.0

Also adds the missing version comment to the pin.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* Upgrade tailscale/github-action to v4.1.2

Drop-in: all inputs unchanged (version is explicitly pinned to 1.50.1). v4 moves the action to the Node 24 runtime.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* Upgrade azure actions (k8s-set-context, setup-helm) to v5.0.0

Drop-in: inputs unchanged (cluster-type still defaults to generic; setup-helm version still defaults to latest). Both move to the Node 24 runtime.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* Upgrade codecov/codecov-action to v6.0.1

Drop-in: token/files/flags inputs unchanged. v6.0.1 includes the VULN-1652 template-injection fix and the Node 24 runtime.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* Upgrade tj-actions/changed-files to v47.0.6

Drop-in: files input and consumed outputs unchanged. SHA is the GPG-signed v47.0.6 tag, a clean release post-dating the March 2025 incident; SHA pinning stays as the durable protection.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* Upgrade marocchino/sticky-pull-request-comment to v3.0.4

Drop-in: header/message/recreate inputs unchanged. v3 moves to the Node 24 runtime.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* Upgrade chromaui/action to v17.0.1

Drop-in despite the major jump: token/projectToken inputs survive and the removed inputs (allowConsoleErrors, only, etc.) were unused. v17 moves to the Node 24 runtime.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* Upgrade snyk/actions/setup to v1.0.0

Drop-in: we pass no inputs to setup; v1.0.0 is purely additive (new optional os input).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* Upgrade slackapi/slack-github-action to v3.0.3

Migrate the v2+ input model: the incoming-webhook URL moves from the SLACK_WEBHOOK_URL/SLACK_WEBHOOK_TYPE env vars into the webhook/webhook-type inputs; payloads are unchanged. Consolidates the previous v1.23.0 and v1.26.0 pins onto one v3.0.3 SHA. errors stays at the default (false), preserving v1 behavior. v3 runs on Node 24.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* Pin anthropics/claude-code-base-action to v0.0.63

Was floating on @beta (unpinned); pin to the immutable SHA @beta currently resolves to. No behavior change today.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-27 18:32:13 +00:00
bryan
7b466c9b86 Add clj-surgeon config (#74855) 2026-05-27 18:23:43 +00:00
Nick Fitzpatrick
1203790a23 Pass column settings when formatting pivot column headers (#74628) 2026-05-27 23:43:28 +05:30
metamben
00e329ce7e Report synchronous bigquery errors as client errors (#74684)
* Report synchronous bigquery errors as client errors

Closes QUE2-487

* Add missing type hint
2026-05-27 19:12:11 +03:00
metamben
94757c098c Handle nil subtotal nodes (#74605)
Closes #71220
Closes QUE2-460
2026-05-27 19:02:42 +03:00
lbrdnk
8ca87a3ff3 Make default embedding service base url match AI service (#74756)
* Set the default of embedding service base to ai service

* components

* test

* Apply suggestion from @crisptrutski

Co-authored-by: Chris Truter <crisptrutski@users.noreply.github.com>

* review remarks

* review remarks

* test and trim

---------

Co-authored-by: Chris Truter <crisptrutski@users.noreply.github.com>
2026-05-27 17:54:26 +02:00
Alex Yarosh
ecdb062426 docs - you no longer have to be store admin (#74837)
Refine Metabase AI settings documentation

Removed redundant information about the Metabase AI add-on visibility and the requirement for admin email matching. Clarified the process for enabling and disconnecting the Metabase AI Service.
2026-05-27 11:53:09 -04:00
Chris Truter
ae19823c53 Remove worktrunk config from git (#74838) 2026-05-27 15:37:23 +00:00
Riley Thompson
aa971184b0 Don't strip values from impersonation validation (#74440)
* dont strip values from impersonation validation

* add test

* strip values for graalpy, but return original values

* add tests that strip multi statement queries
2026-05-27 14:56:38 +00:00
Chris Truter
2520cb43c2 Bound pr-str when logging LLM errors (#74796) 2026-05-27 16:45:06 +02:00
Nemanja Glumac
0fd4680fdc Fix typo in aws-credentials action version (#74826) 2026-05-27 16:21:16 +02:00
Nick Fitzpatrick
16ba5f7830 Disallow brushing charts with an aggregated x axis (#74646)
Don't allow brushing on charts where the x axis is an aggregate
2026-05-27 19:35:47 +05:30
Nemanja Glumac
ef354ac92f Upgrade package manager actions (#74824)
* Upgrade setup-bun action to v2.2.0

* Upgrade setup-uv action to v8.1.0

* Upgrade setup-chrome action to v2.1.2

* Upgrade setup-babashka action to v1.8.0
2026-05-27 14:05:38 +00:00
Nick Fitzpatrick
fd504cf5ca Add overflow-wrap to tooltip content globally (#74625)
* Add overflow-wrap to tooltip content globally

* e2e test adjustment
2026-05-27 19:31:22 +05:30
Jakub Chodorowicz
b94b50a122 Gdgt 2495 workaround unsafe eval near membrane requirement (#74717) 2026-05-27 15:00:50 +02:00
Chris Truter
3d893c1d75 Rename MCP UI viz scope for metabot consistency (#74820) 2026-05-27 12:56:32 +00:00
Uladzimir Havenchyk
2aabe33583 test: fix assertion in new database modal (#74821) 2026-05-27 14:43:39 +03:00
Oleh K
656c4e99c9 Fix misalignment in horizontal Schedule when "Custom" is selected (#74749)
Anchor the horizontal Schedule row to the top with align-items: flex-start
instead of center, and move the CronExpressionInput's "quartz cron syntax"
hint from above to below the input so the cron input aligns with sibling
controls in the flex container.
2026-05-27 12:49:32 +02:00
Timofey Kachalov
7020531063 Fix ChatGPT drill-through render by giving render_drill_through its own UI resource URI (#74719) 2026-05-27 11:55:58 +04:00
Phoomparin Mano
deddaa08f3 Add CSP configuration to make images load in MCP Apps (#74741)
* add csp for local images

* add base url and baseUriDomains

* address code reviews

* adjust template tests
2026-05-27 13:19:07 +07:00
Phoomparin Mano
52c81f31f0 Fix ChatGPT and Cursor dark theme override for MCP Apps (#74739)
* Add ChatGPT dark theme override

* Allow to override agent overrides if host sends variables

* add additional color overrides

* add unit test for buildMcpAppsTheme

* add override for cursor

* remove padding from cursor body

---------

Co-authored-by: Timofei <timofei@metabase.com>
2026-05-27 12:34:44 +07:00
Aleksandr Lesnenko
e2786c6830 add recent graphy features analytics tracking (#74800) 2026-05-27 01:48:53 +00:00
Alexander Polyankin
da74a8487d Fix schema selection in workspace forms (#74811) 2026-05-27 01:23:14 +00:00
dpsutton
24a5dc82f1 Sanitize workspace creation sql exception messages (#74657)
* Sanitize workspace creation sql exception messages

just str/replace the secrets in the sql statements.
Execute batch returns sql in statements.

I'd really like to do something like this:

```clojure
(defmulti init-workspace-isolation!
  "Initialize database isolation for a workspace. Creates an isolated schema/database,
   user credentials, and grants appropriate permissions for the workspace to operate
   within its own namespace.

   Returns a map with:
   - :schema           - The name of the isolated schema/database created
   - :database_details - Connection details (user, password, etc.) for the isolated user

   Implementations should:
   - Create an isolated schema or database for the workspace
   - Create a user with credentials that can only access that schema
   - Grant appropriate permissions (CREATE, INSERT, SELECT, etc.) on the isolated schema

   This is an enterprise feature. Drivers must also return true for
   (database-supports? driver :workspace database) to indicate support."
  {:added "0.59.0" :arglists '([driver database workspace])}
  dispatch-on-initialized-driver
  :hierarchy #'hierarchy)

;; introduce a new actual public api and have the multimethod be the impl
;; and we can in one place solve this.
(defn init-workspace-isolation
  [driver database workspace]
  (try (init-workspace-isolation! driver database workspace)
       (catch Exception e (scrub-exception e (secrets-of workspace)))))
```

But the current impl is fine as well and allows each driver to catch
particular driver exceptions.

* whoops. committed pseudocode

* bad parens in mysql

* kondo fix
2026-05-26 21:14:32 -04:00
Cam Saul
6cc99e6df5 Warn on large tests with several distinct testing blocks (#74802)
* Warn on large tests with several distinct testing blocks

* Drop threshold to 65

* Drop threshold to 60

* More fixes for threshold of 60

* Fix accidental `:^parallel` in ns `:require`

* cljfmt

* Pull in latest changes
2026-05-26 16:44:15 -07:00
github-automation-metabase
bfbf1746a6 Auto-update documentation for master (#74801)
Update generated docs (master)

Co-authored-by: github-actions <github-actions@github.com>
2026-05-26 19:24:42 -04:00
Cam Saul
6f738a55b8 Enable cljfmt :remove-blank-lines-in-forms? (#74784)
* Update configs

* Update driver features

* cljfmt

* Format new changes

* Merge latest master
2026-05-26 14:47:57 -07:00
Andrei Chernykh
565cccc7b4 Unify Custom homepage and Landing page into a single Homepage radio (#74724)
* Unify Custom homepage and Landing page into a single Homepage radio

* Code review cleanup: inline plugin access, add getHomepageMode tests, drop redundant beforeEach, inline single-use e2e helper

* Update GeneralSettingsPage test for renamed Homepage heading
2026-05-26 14:36:30 -07:00
Maksym Yakubych
006ad73c95 feat: add last calculated date to data complexity (#74535)
### Description

adds last calculated date to data complexity

### How to verify

Describe the steps to verify that the changes are working as expected.

1. New question -> Sample Dataset -> ...
2. ...

### Demo

![image.png](https://app.graphite.com/user-attachments/assets/afb9855b-d389-4e7e-b0d1-d4424cdd5e87.png)



### Checklist

- [x] Tests have been added/updated to cover changes in this PR
- [x] If adding new Loki tests: they pass [stress testing](https://github.com/metabase/metabase/actions/workflows/loki-stress-test-flake-fix.yml)
2026-05-26 17:49:09 -03:00
Alexander Polyankin
bc5e227fc1 Require only transform read permissions to run a transform (#74788)
* check for read permissions when running transforms

* test

* fix
2026-05-26 16:49:03 -04:00
Maksym Yakubych
dc2218a836 fix: simplify e2e ai usage tests (#74673)
Closes [BOT-1585: Simplify e2e AI usage tests](https://linear.app/metabase/issue/BOT-1585/simplify-e2e-ai-usage-tests)

### Description

Simplifies end-to-end AI usage tests by removing testing the labels in the charts. Testing the query creation with unit tests is sufficient. 

### Checklist

- [x] Tests have been added/updated to cover changes in this PR
- [x] If adding new Loki tests: they pass [stress testing](https://github.com/metabase/metabase/actions/workflows/loki-stress-test-flake-fix.yml)
2026-05-26 17:48:46 -03:00
Alexander Polyankin
a734e25ebc Use database select in workspace database form (#74791) 2026-05-26 15:37:47 -05:00
Chris Truter
2368677acc Surface upstream body in LLM provider error logs and messages (#74359)
* Surface upstream body in LLM provider error logs and messages

When an LLM provider (Claude, OpenAI, OpenRouter) returns a non-2xx
response, `core/rethrow-api-error!` used to ship just the canonical
status-specific message ("Anthropic API key expired or invalid",
"OpenAI API is not working but not saying why", etc.). The upstream
body was decoded into `ex-data` but never logged and never shown to
the admin, even when it contained the actionable detail.

This commit:

- Centralizes the body-preview logic in `core/body-preview`. It pulls
  `[:error :message]` out of structured JSON envelopes, falls back to
  `:error`/`:detail`/`:message`, then `pr-str`s the map, and slurps
  `InputStream` bodies. Truncates to 500 chars for the exception
  message; the full body is preserved in logs and `ex-data`.
- `rethrow-api-error!` now appends the preview to the provider's
  canonical message (when present), emits a `log/warn` at the failure
  boundary with the full body, and logs the exception class + message
  on the no-body branch (network errors, timeouts, etc.).
- Drops the per-provider `error-msg` interpolation: each provider's
  `*-errors` helper now only returns the canonical status-specific
  message; the body preview is added centrally so all three providers
  behave consistently and previews can't go missing on edge cases
  (plain-text bodies, HTML error pages, non-JSON envelopes).

This is the master-side fix for the same gap addressed in PR #74356
against `release-x.59.x`; the 59 fix touched the now-removed
`metabot-v3/client.clj` indirection, this fix touches the direct
provider call path it was replaced with.

* Polish: drop pr-str leak, allow-list ex-data, structured warnf

Three follow-ups in response to review of the 59 counterpart (PR #74356):

- `body-preview` no longer falls back to `pr-str` for maps that don't carry
  a recognised human-readable field (`[:error :message]`, `:error`, `:detail`,
  `:message`). Returns nil instead, so users don't see raw
  `{:request-id "..." :trace [...]}` blobs in toasts. The full body is still
  in the log and `ex-data`.
- `ex-data` is now an explicit allow-list of `:status`, `:reason-phrase`,
  `:body`, and the provider tags. The raw clj-http response carries
  `:http-client` (a `Closeable`), `:trace-redirects`, `:orig-content-encoding`,
  and other internals we don't want flowing into Sentry payloads or
  downstream consumers.
- `log/warn` with a trailing map `pr-str`s the map into the message rather
  than emitting structured MDC. Switched to `log/warnf` so the line is one
  greppable formatted blob with explicit field names.

Test coverage extended for each case: unrecognised maps round-trip as nil,
ex-data keys are pinned to the allow-list, and the no-body branch records
the exception class.

* Close remaining pr-str leak and tighten ex-data key-set assertions

- body-preview: drop the :else pr-str fallback; handle JSON arrays by
  probing the first element symmetrically with maps. Arrays of internal
  diagnostic objects (and non-string scalars) now return nil instead of
  leaking pr-str blobs into user-facing toasts.
- self_test: add assert-rethrow-no-internals! to the structured-maps
  branch and a sibling assert-rethrow-no-internals-no-body! for the
  no-:body branch, so any accidental extra ex-data key is caught
  symmetrically on both paths.

* Trim docstrings and tighten tests for upstream-body error surfacing

- Rename `anthropic-errors`/`openai-errors`/`openrouter-errors` →
  `*-error-msg` to reflect that each returns a single, status-specific message
  string (the body preview is appended by [[core/rethrow-api-error!]] now).
- Trim docstrings on `max-body-preview-chars`, `extract-error-message`,
  `body-preview`, and `rethrow-api-error!` — drop asides about what survives in
  `ex-data` / logs; keep only the load-bearing facts. Compress the inline
  `log/warnf` comment to one line about the MDC distinction.
- Slim `body-preview-test`: merge nil/blank/scalar cases with `every? nil?`,
  dedupe the "no recognised error field → nil" branch across maps and arrays,
  drop redundant sub-tests.
- Slim `rethrow-api-error!-test`: introduce a small `caught` helper, drop the
  two `assert-rethrow-no-internals*!` helpers, and inline the exact-key-set
  pin with a one-line comment.
- Fix `call-llm-tool-choice-test`'s synthetic exception to ride
  `:api-error true` so `::skip` survives the new ex-data allow-list — the
  fixture used to ride through the old passthrough `ex-data`.

* Trim *-error-msg docstrings — drop rethrow-api-error! cross-reference

* *-error-msg docstrings: 'separately' reads more naturally than 'out of scope'

* Hint ^Throwable on rethrow-api-error! to silence eastwood reflection warnings

`(some-> e .getClass .getName)` reflected on the un-hinted exception arg.

* Assert upstream body lands on the warn line at the failure boundary

Captures the warn log emitted by rethrow-api-error! and verifies provider,
status, and the full quoted body are present.

* Agent loop API-error log includes the throwable on the body branch

Aligns with the two sibling log/errorf calls in the same catch — every
branch now logs with the stacktrace, so when a provider failure surfaces
the cause chain (rethrow-api-error! → clj-http) is preserved end-to-end.

* Line break between throwable and message template in agent-loop errorf calls

* Revert the line break — sibling errorf already had e on the same line

* let-bind ex-data once in the agent-loop catch

* Destructure ex-data fields in agent-loop catch

* Log exception message in non-api-error agent-loop catch branch

* Skip trailing colon in no-body rethrow when ex-message is blank

Mirror of release-x.59.x #74356's blank-msg fix — an exception like
(RuntimeException.) with no message produced "openai API request failed: null"
(MessageFormat substitutes nil as literal null). Fall back to the colon-less
form when there's nothing to say after it.

* Pin ex-data key set on every rethrow-api-error!-test sub-case

Mirrors the sibling's defensive-pin pattern (release-x.59.x c15732eddbb) —
each test case now independently catches ex-data leaks instead of relying on
the sibling case's pin to cover the same path.

* Only accept string fields in extract-error-message

A structured `:error` map without `:message`, or a vector/map under
`:detail`/`:message`, was being coerced into the user-facing exception
message via `str`. Guard with `string?` so non-string values fall
through to `nil`; structured bodies still survive in ex-data and logs
for debugging.

* body-preview: fall through to later keys when an earlier one is non-string

Mirror of sibling 8239fc661ac. The previous `or` over raw values plus a final
`(string? s)` check returned nil even when a later key had a real string —
e.g. `{:error {:message {:code 500}} :detail "real msg"}`. Filter
per-lookup so non-strings fall through correctly.

* extract-error-message: combine string+blank filter into one fn

Per-lookup `(when (string? v) (not-empty v))` means blank strings also fall through
to a later key (e.g. `{:error "" :detail "real"}` → "real"). Trim the docstring.

* body-preview: warn + pr-str unrecognised body shapes instead of dropping

For non-empty maps/arrays where extract-error-message can't find a clean string,
fall back to pr-str of the body and emit a warn. Better some context in the
user-facing message than none, and the warn gives operators a signal to add
the new envelope shape to extract-error-message.

Nil/empty bodies still return nil — nothing to surface there.

* extract-error-message: trim whitespace so blank scalars fall through

Mirror of sibling e585ae08ccf. `(not-empty "   ")` returns the whitespace
string (only empty strings get rejected) — so `or` short-circuited with it
and the outer trim collapsed it to nil too late to recover via fall-through.
Trim inside the per-key filter instead, matching the empty-string fix from
3ab19e0677.

* Move implementation rationale from docstrings into inline comments

Per the docstring-as-contract guidance: docstrings should describe what the function
does for the caller; implementation rationale (why we filter per-lookup, why warnf, why
the allow-list) belongs as a `;;` comment at the line in the body where it's relevant.

- extract-error-message: drop the "so the raw envelope can't str-leak" rationale into
  a `;;` comment above the per-lookup filter.
- body-preview: drop the "better some context than none" rationale into a `;;`
  comment above the pr-str fallback.
- rethrow-api-error!: drop the "raw clj-http carries :http-client Closeable, …" into
  a `;;` comment above the select-keys allow-list.
- *-error-msg adapter helpers: drop the "body appended separately" clause — that's
  caller behaviour, not what the helper does.

* body-preview: bounded read for InputStream bodies

Slurping the full stream just to truncate to 500 chars meant a multi-MB
upstream body got pulled into memory unnecessarily. Switch to a bounded
read of max-body-preview-chars + 1 — the +1 lets the existing truncation
branch detect overflow without consuming more of the stream.

* rethrow-api-error!: preserve :headers so retry path honors Retry-After

The ex-data allow-list was [:status :reason-phrase :body] — dropping
:headers silently broke metabase.metabot.self/parse-retry-after-header,
which reads (:headers (ex-data e)) to honor upstream Retry-After on
429/529 responses. With :headers gone, those retries fell back to plain
exponential backoff. Add :headers to the allow-list and pin the path
with a test that goes through rethrow-api-error! into retry-delay-ms.

* self-test: drop proxy in favour of .available() to clear Eastwood

Eastwood flagged four reflection warnings in the "InputStream bodies are
read up to the truncation window" test — `.getBytes` on an inferred-Object
String and three `proxy-super read` calls inside the byte-counting
ByteArrayInputStream proxy.

Replace the proxy with a plain ByteArrayInputStream and infer bytes
consumed from `.available()` (BAIS returns unread bytes remaining), and
add a `^String` hint on the literal that feeds `.getBytes`. Same
assertion, no proxy, no reflection.

* decode-bounded-body: cap InputStream slurp before JSON decode

body-preview's bounded InputStream branch was dead code in the real
rethrow-api-error! flow: json/decode-body slurped the stream first and
the cap never fired. Move the cap upstream — slurp the InputStream into
a string of at most max-body-slurp-chars before handing it to decode-body,
and fall back to the raw bounded string when JSON parsing fails (e.g. the
cap truncated us mid-envelope). Tests now exercise the path through
rethrow-api-error! per the review's request.

* docstrings: relocate decode-bounded-body's implementation diary into the body

`max-body-slurp-chars`: move the magnitude justification out of the docstring
to an inline comment above the value. Docstring keeps the caller contract.

`decode-bounded-body`: the docstring was describing *how* the function works
(bound-slurp first, then decode, then fall back) rather than what it returns.
Slim to the return contract and move the "how" to inline comments above the
relevant let-binding.

`body-preview` and `rethrow-api-error!`: pack sentences that were starting
mid-line onto their own lines — same caller-contract content, format matches
the one-sentence-per-line rule for wrapped docstrings.

* body-preview: cap warn pr-str + letfn + narrower catch

- extract-error-message: use letfn instead of let+fn (style nit, per @lbrdnk).
- body-preview: extract truncate-to-preview-limit and apply it to the pr-str
  fallback's warn line, so a multi-MB body doesn't end up as a multi-MB log
  message (per @lbrdnk on the warnf line).
- slurp-bounded / decode-bounded-body: narrow catch from Throwable to
  Exception so JVM-level errors (OOM, StackOverflow) aren't silently
  swallowed in the error-handling path (mirrors Copilot's note on the
  sibling v59 PR).

* Fix CI: split rethrow-api-error!-test + hint char-array in slurp-bounded

- slurp-bounded: add ^chars hint on the buffer so `(String. buf 0 ...)`
  resolves at compile time. Was triggering an Eastwood reflection warning
  in src/metabase/metabot/self/core.clj:529.
- self_test: split the 161-line rethrow-api-error!-test into six focused
  deftests (passthrough / string-body / no-body / input-stream /
  retry-after / warn-log) so clj-kondo's long-test check stays happy and
  the failure messages localise to the right concern.

* slurp-bounded: hint the String. ctor at the call site

The earlier `^chars` hint on the let-binding propagated to the .read call
but not through the loop/recur rebinding, so `(String. buf 0 limit)` still
went through reflection — Eastwood complained, CI failed.

Move the hint to the call site, and explicitly cast the int args so the
matching (char[], int, int) constructor is picked unambiguously.

* rethrow-api-error!: don't splice upstream body on 401/403

Provider auth-error bodies can carry sensitive detail (API keys,
account/org/tenant identifiers). Suppress the body preview from the
user-facing exception message for 401/403; the full body still hits
the warn log for server-side debugging and is still preserved on
ex-data for in-process inspection.

* run-agent-loop: flatten nested catch ifs into a single cond

* Port 59 refinements: chunked slurp-bounded and capped log pr-str

Mirror two improvements landed on the sibling 59 PR (#74356) back to master:

- `slurp-bounded`: switch from pre-allocating a `(char-array max-body-slurp-chars)`
  (1MB even for a 100-byte body) to growing a `StringBuilder` in
  `body-slurp-chunk-chars` (8KB) reads. Worst-case allocation stays at
  ~`max-body-slurp-chars`, but a small error envelope now costs a small allocation.
- New `max-body-log-chars` (2000) + `body-for-log` helper: cap the body
  `pr-str` in `log/warnf` lines so a near-cap slurped stream can't flood the
  logs. The full body still survives in `ex-data`.

Also extracted a generic `truncate-to` helper that `truncate-to-preview-limit`
and `body-for-log` both delegate to.

* self-test: cover the body-for-log warn-line cap (roborev #1797)

Add a case to rethrow-api-error!-warn-log-test with an oversized (5000-char)
body: assert the warn line's body segment is capped at max-body-log-chars with
a trailing ellipsis, while the full untruncated body still survives on ex-data.

* self-test: derive log-cap test bound from max-body-log-chars var (roborev #1798)

Read the cap via @#'self.core/max-body-log-chars and size the body / slice from
it, so a retune of the var adapts the test instead of breaking the suffix match.
2026-05-26 20:33:41 +00:00
Jeff Bruemmer
882a3cd6f3 docs - bitbucket remote sync (#74772)
bitbucket tokens
2026-05-26 16:50:13 -03:00
bryan
07cf459345 MCP: signal tool-list changes to clients (GHY-3661) (#74679) 2026-05-26 19:18:39 +00:00
Ryan Laurie
16156549a7 Allow Admins to purchase Metabot AI (#74602)
* Allow Admins to purchase Metabot AI

* update unit tests
2026-05-26 13:12:25 -06:00
Alexander Polyankin
8a688591df Workspace runtime setup (#74705)
* settings

* setup

* setup

* config

* fix

* fixes

* fixes

* fix

* fix

* fix

* fix

* fix

* fix

* fix

* fix

* fix

* fix

* fix

* fix

* fixes

* fixes

* fix

* hide empty state menu

* disable download button when no databases

* existing workspace check

* add a warning in case someone uses direct urls

* optimistic update

* Revert "optimistic update"

This reverts commit 750dbbccc3.

* delete old remappings

* fix perms

* tests

* fixes

* UI updates

* fix tests

* e2e

* feedback
2026-05-26 14:29:54 -04:00
bryan
c1e6641e28 Workspace provisioning honors admin-connection-details (#74775)
Wrap init/grant/destroy in `driver.conn/with-admin-connection` at the
orchestration layer (`dispatching-provisioner`) so the database's
`:admin-details` overlay applies. Previously the form at
`/admin/databases/<id>/admin` saved a higher-privilege admin user but
provisioning still ran as the regular `:details` user, causing
`CREATE USER ... WITH PASSWORD '...' was aborted: permission denied to
create role` when the regular user lacked CREATEROLE.

Affected all drivers — only `check-isolation-permissions` paths in
mysql.clj and sql_jdbc.clj wrapped before; the production provisioning
path did not.

Wrapping at the orchestrator means a single change covers all 8 drivers
and future drivers inherit it. Added an admin-connection-scope contract
to the multimethod docstrings so direct callers (REPL, tests) know to
bind themselves.

[GHY-3691]
2026-05-26 14:27:39 -04:00
Mihael Konjević
d59dc0499a Strip surrounding double-quotes from source-card/cross-stage column names (#74731)
When the LLM references a source-card or previous-stage column by a name
quoted like a SQL identifier (e.g. ["field" {} "\"app_id\""] instead of
["field" {} "app_id"]), the column-name no longer matches anything the source
returns. The repair pass then can't infer base-type, resolve-query (which only
transforms) lets the typeless string-named ref through, and -- since production
runs with Malli instrumentation off -- the invalid query is stored on the card.
The FE later crashes computing display info: "unable to determine the type of
[field {} \"app_id\"]".

Canonicalise the name in maybe-fill-cross-stage-types: strip surrounding
double-quotes, but only when the stripped name actually matches a column the
source produces, then rewrite the ref to the canonical name and stamp
base-type/effective-type. Rewriting the name (not just stamping the type)
matters -- otherwise the ref would still point at a non-existent column.
Idempotent and conservative: unmatched names and already-correct names are left
untouched.

Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-26 18:28:32 +02:00
Eric Normand
bd0a3d13f3 Fix temporal-unit param field qualification for BigQuery (#74680)
The temporal-unit `->replacement-snippet-info` method lost two field-ref
options (`qp.add.source-table` and `::compiling-field-filter?`) during the
migration from the deprecated `driver.common.parameters` parsing code to
Lib (#74236 / #74253). Before that migration it used `field->clause`,
which attached both; the refactor to `(lib/ref field)` dropped them.

Without `qp.add.source-table` set to a real table id, BigQuery's `:field`
honeysql method binds `*field-is-from-join-or-source-query?*` to true, so
it skips merging the dataset and table into a single dotted identifier --
emitting `` `dataset`.`table`.`column` `` instead of
`` `dataset.table`.`column` ``. BigQuery then can't resolve the bare
`dataset` name, failing `can-compile-temporal-units-test`.

Restore the two options on the field ref, mirroring `field->field-ref`
(the field-filter equivalent). A temporal-unit param is spliced into
user-written native SQL just like a field filter, so it must be qualified
the same way the user's own FROM clause is.
2026-05-26 11:23:53 -05:00
Nick Fitzpatrick
3eb2e0ccc0 adjust min height of container so that there is no scroll for simple series (#74624) 2026-05-26 21:28:40 +05:30
Nick Fitzpatrick
b68fcb7263 Update persist and refreshModelCache mutation onQueryStarted (#74631) 2026-05-26 21:28:18 +05:30
Riley Thompson
3bdf045bfc Glob for driver plugin manifest file (#74291)
* glob for driver plugin manifest

* prefer new nested manifest location over legacy root location

* kondo

* log chosen manifest

* remove extra file
2026-05-26 15:56:41 +00:00
Alexander Polyankin
beea1993bf Fix search values remap (#74770)
* fix custom values with labels

* fix
2026-05-26 15:43:01 +00:00
Riley Thompson
ae85e3ef01 Add test to verify unable to reproduce (#74606)
* add test

* clean up
2026-05-26 15:08:41 +00:00
Jakub Chodorowicz
feaaafb8e8 Gdgt 2390 add sources for the tgz test plugins (#73819) 2026-05-26 17:04:46 +02:00
Riley Thompson
b8b6b6cb77 Use chr(92) over '\' to work around sql-formatter bug for oracle (#74457)
* use chr(92) over \

* add test

* add test with backslach
2026-05-26 10:35:33 -04:00
Nicola Mometto
7f9d472da2 Security center: send test notifications to unsaved recipients (#74339)
* feat: pass explicit config

* api: build config

* update FE

* Update enterprise/frontend/src/metabase-enterprise/security_center/hooks/use-notification-config.ts

Co-authored-by: Kamil Mielnik <kamil@kamilmielnik.com>

* handle FE review comments

* extend test

---------

Co-authored-by: Kamil Mielnik <kamil@kamilmielnik.com>
2026-05-26 15:15:20 +01:00
Maksym Yakubych
6bc738d93e feat: ui to display data complexity color coding (#74363)
Closes [BOT-1519: UI to display data complexity score and color coding](https://linear.app/metabase/issue/BOT-1519/ui-to-display-data-complexity-score-and-color-coding)

### Description

Adds UI to display data complexity score and color coding.

### How to verify

1. Go to AI usage analytics page.
2. Go to data complexity scores.

### Demo

https://www.loom.com/share/81901a478d1a45a3839d7dd42f3e4ae7?from_recorder=1&focus_title=1

### Checklist

- [x] Tests have been added/updated to cover changes in this PR
- [x] If adding new Loki tests: they pass [stress testing](https://github.com/metabase/metabase/actions/workflows/loki-stress-test-flake-fix.yml)
2026-05-26 11:15:07 -03:00
Sébastien
8f94782867 Default the embed wizard auth mode to SSO when configured, guest otherwise (#74750) 2026-05-26 15:58:05 +02:00
github-automation-metabase
1172e6d22d Auto-update documentation for master (#74736)
Update generated docs (master)

Co-authored-by: github-actions <github-actions@github.com>
2026-05-26 09:22:57 -04:00
Jakub Chodorowicz
addfb5458f gitignore new rspack compilation artifacts (#74755) 2026-05-26 15:19:05 +02:00
Sébastien
b5efc56ecd Show collection picker when saving new question in metabase-browser (#74570) 2026-05-26 15:15:23 +02:00
Sébastien
2bfca17fa3 Embedding wizard: fix docs link and show error when Site URL is misconfigured (#74635) 2026-05-26 14:26:05 +02:00
Nicola Mometto
d2f4caa79a fk deser completeness test (#74502)
* fk deser completeness test

* caught a missing one!

* fix

* exceptions
2026-05-26 09:46:03 +01:00
Nicola Mometto
f028436f26 security center: only include mb email if send to all emails is on (#74334)
only include mb email if send to all emails is on
2026-05-26 09:41:12 +01:00
Nemanja Glumac
f0cbdaf5f8 Upgrade peter evans actions (#74737)
* Upgrade peter-evans/create-pull-request action to v8.1.1

* Replace peter-evans/enable-pull-request-automerge with the gh cli

* Upgrade juliangruber/approve-pull-request-action to v2.1.0
2026-05-26 00:10:00 +02:00
Nemanja Glumac
5a665b0ce8 Upgrade docker-related GitHub actions to their latest versions (#74733)
* Upgrade docker-login actions to v4.2.0

* Upgrade docker/setup-compose action to v2.1.0

* Upgrade docker/build-push-action to v7.2.0

* Upgrade docker/setup-qemu-action to v4.0.0

* Upgrade docker/setup-buildx-action to v4.1.0
2026-05-25 23:34:44 +02:00
Nemanja Glumac
b95da2e563 Upgrade aws actions configure-aws-credentials to v6.1.1 (#74730) 2026-05-25 23:08:59 +02:00
Nemanja Glumac
d03c2eb418 Upgrade actions/cache to v5 (#74715) 2026-05-25 21:54:36 +02:00
Riley Thompson
cefb17e78b Use context-timeouts metric instead of timeouts metric (#74539)
use context-timeouts instead of timeouts
2026-05-25 15:12:47 -04:00
Oleksandr Yakushev
33ada0caee [perf] Don't perform redundant idempotent zone-id conversion in format-rows-xform (#74326) 2026-05-25 20:39:54 +03:00
Alexander Polyankin
42fb68f197 Fix remapping with custom values (#74698)
* fix remapping with custom values

* fix

* fix unit test
2026-05-25 17:56:00 +01:00
Ngoc Khuat
2070f40cbf Fix Snowflake python transform e2e test: explicit target schema (#74716) 2026-05-25 16:36:50 +00:00
lbrdnk
58f267e852 Fix ai proxied /embeddings (#74526)
* Fix url gen for messages and token for embeddings

* Clean up embedding generation

* Clean up auth resolution for proxied anthropic requests

* Add tests for auth resolution

* Update tests for embeddings

* Sentinel for openai token missing

* tests

* whitespace

* revert /messages "fix"

* Docstring on /llm added by hm

* Adjust ex message

* test

* Setting doc

* Use ai-service passthrough when api key is not set

* Revert tests

* test

* whitespace
2026-05-25 18:33:18 +02:00
Timofey Kachalov
49bdda6650 Fix failing Embed Wizard CI (#74709) 2026-05-25 17:34:27 +04:00
Oleh K
080a04582a Fix custom row limit resetting the provided value (#74490)
* Fix custom row limit resetting the provided value (#71276)

Redesign the row-count limit popover into two radios ("Show maximum" and
"Set custom limit") with the input shown only when custom is selected.
Selecting custom auto-focuses and selects the input, and applies the
value (or default 2000); Enter or blurring away applies and closes.
Also fixes the popover not closing on outside click by using onDismiss
instead of onClose for the controlled Mantine Popover.

* Align row limit popover to the right of the target

Use bottom-end positioning so the popover stays anchored to the row
count's right edge, avoiding layout shift between table refreshes.

* Use onKeyDown instead of deprecated onKeyPress in limit popover

* Apply the row limit on input blur and on popover dismiss

Apply the typed value both when the input loses focus and when the
popover is dismissed, guarded by a diff check so it applies once and
only when the value actually changed. Fixes the value being dropped when
clicking outside the input but still inside the popover, and when the
popover closed before the input's blur could fire.

* Apply the row limit only on Enter or popover dismiss

Drop the apply-on-blur path: applying the typed value when the input
lost focus caused a double apply (and a possible race) when editing a
custom value and then clicking "Show maximum". The value now commits on
Enter or when the popover is dismissed, with a diff check so it applies
once and only when changed.

* Remove unnecessary variable

* Use useLatest for popover dismiss refs in LimitPopover

Replaces the manual useRef + mirror-assignment pattern with useLatest for
the refs read in the unmount cleanup. Addresses PR review feedback.

* Always render the custom limit input in LimitPopover

The input is no longer conditionally rendered, removing layout shift when
toggling radios. Custom activation is driven by the input's onFocus, so
clicking the radio focuses the input and clicking/typing directly into it
also activates custom. Restores the brand border when custom is active and
drops the focus useEffect + mount guard. Addresses PR review feedback.

* Defer limit changes until commit; Escape cancels in LimitPopover

Both radios and the typed value are now local-only until the popover is
dismissed. Enter or outside-click commits the selected limit on unmount;
Escape sets a cancel flag so the pending change is discarded instead. This
also stops switching to "Show maximum" from applying immediately. Addresses
PR review feedback about Escape applying the change.

* Cleanup comments
2026-05-25 14:48:22 +02:00
Nemanja Glumac
c207823f25 Upgrade actions/setup-node@v4 to v6.4.0 (#74677) 2026-05-25 14:29:13 +02:00
Nemanja Glumac
dcaaf02905 Upgrade actions/setup-java@v4 to v5.2.0 (#74678) 2026-05-25 13:05:30 +02:00
Nemanja Glumac
52cd1473c6 Upgrade dorny GitHub actions (#74674) 2026-05-25 13:05:06 +02:00
Uladzimir Havenchyk
2814149576 test: get rid of warnings in tests (#74577)
* tests: migrate jest to swc

* fix tests and circular dependency

* fix test

* fix typecheck

* review

* tweak config

* use correct swc plugin

* test: detect memory leak

* test: detect memory leak

* test: fix multiple warnings

* revert

* revert

* revert

* Update frontend/src/metabase/querying/filters/components/FilterPicker/FilterColumnPicker/FilterColumnPicker.unit.spec.tsx

Co-authored-by: Kamil Mielnik <kamil@kamilmielnik.com>

* test: use synchronous act() for synchronous dispatches/blur

Address review feedback: the act() callbacks only perform synchronous
work (store.dispatch, input.blur), so the synchronous act(() => {}) form
is sufficient. dispatchLocationChange is no longer async as a result.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Kamil Mielnik <kamil@kamilmielnik.com>
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-25 12:41:56 +03:00
Ngoc Khuat
6427973e9d Fix NullPointerException in render-minibar with missing range stats (#74572)
* Fix NullPointerException in render-minibar with missing range stats

render-minibar assumed the column fingerprint always supplied min and max
values. When a minibar-enabled column lacks :type/Number fingerprint stats
(e.g. an unsynced or computed column), min and max are nil and (< min 0)
throws a NullPointerException that crashes the entire Slack/email notification
delivery silently. Guard against nil min/max and fall back to rendering the
plain formatted value, consistent with the existing non-numeric fallback.

* Isolate per-card render failures in dashboard subscriptions

A single card that fails to render (e.g. the minibar NPE in #74007, or any
future render error) could take down an entire dashboard subscription: the
per-card try/catch in render-pulse-card-body swaps in a :render-error
placeholder, but render methods build the body as lazy hiccup, so an
exception thrown while building it escapes that catch and only fires later
when the channel realizes the hiccup (email: hiccup/html; Slack: PNG
rendering) — outside any per-card boundary, aborting the whole notification.

Add explicit per-part isolation at the assembly boundary, where each part is
already rendered and realized one at a time:
- email :notification/dashboard reduce wraps each part's render+html
- slack :notification/dashboard mapcat wraps each part->sections!

On failure the part degrades to the existing error placeholder (new
channel.render/error-rendered-part for email; an equivalent section block for
Slack) and the remaining cards still deliver. This catches any part failure,
lazy or not, and leaves the happy path unchanged.

* Simplify min/max nil check in render-minibar
2026-05-25 08:20:38 +00:00
Ngoc Khuat
c607c65bb7 Dispatch transforms within a job run in parallel (#73098) 2026-05-25 07:35:15 +00:00
dpsutton
ad0c7fb28c handle duplicate copies of gson on classpath (#74685)
* handle duplicate copies of gson on classpath

fixes https://github.com/metabase/metabase/issues/73736

The relevant error message for us:

```
java.lang.NoSuchMethodError:
  'com.google.gson.stream.JsonWriter com.google.gson.stream.JsonWriter.value(float)'
  com.google.api.client.json.gson.GsonGenerator.writeNumber(GsonGenerator.java:105)
  com.google.cloud.bigquery.BigQueryRetryAlgorithm.getErrorDescFromResponse(BigQueryRetryAlgorithm.java:227)
  com.google.cloud.bigquery.BigQueryRetryAlgorithm.shouldRetryBasedOnBigQueryRetryConfig(BigQueryRetryAlgorithm.java:115)
  com.google.cloud.bigquery.BigQueryRetryHelper.run(...:108)
  driver.bigquery_cloud_sdk$execute_bigquery(bigquery_cloud_sdk.clj:751)
```

```clojure
user=> (enumeration-seq (.. (Thread/currentThread) getContextClassLoader (getResources "com/google/gson/stream/JsonWriter.class")))
(#object[java.net.URL
         "0x3bc8ea4f"
         "jar:file:/Users/dan/.m2/repository/com/google/code/gson/gson/2.12.1/gson-2.12.1.jar!/com/google/gson/stream/JsonWriter.class"]
 #object[java.net.URL
         "0x50073d1c"
         "jar:file:/Users/dan/.m2/repository/com/vertica/jdbc/vertica-jdbc/23.4.0-0/vertica-jdbc-23.4.0-0.jar!/com/google/gson/stream/JsonWriter.class"])
```

- Vertica bundles: gson 2.8.9 (vendored, unshaded)
- BigQuery demands: gson 2.12.1 (via com.google.code.gson/gson in deps tree — needs JsonWriter.value(float) which was added in 2.9.0)

contents of vertica jar:
 71 com/vertica/io
 70 com/google/gson/internal/bind
 61 com/vertica/utilities/conversion

188 gson class files bundled inside vertica-jdbc, vs ~600 Vertica
classes. They just vendored the entire gson library unshaded into their
fat JAR. Only gson — nothing else third-party.

So mystery solved. We had two copies of gson and vertica could slip
through. Our fix is to ensure that we only accept (or overwrite) when it
comes from a gson lib

```clojure
(def ^:private gson-conflict-handler
  "vertica-jdbc (and potentially other fat JARs) bundle their own copy of gson classes.
   When these overwrite the correct version from com.google.code.gson/gson, BigQuery's
   error-handling path crashes with NoSuchMethodError on JsonWriter.value(float),
   introduced in gson 2.9.0. This handler ensures the pinned gson version always wins
   regardless of JAR processing order. See #73736."
  {"com/google/gson/.*"
   (fn [{:keys [lib path in]}]
     (if (= lib 'com.google.code.gson/gson)
       {:write {path {:stream in}}}
       nil))})
```

* add activation conflict handler as well

```clojure
uberjar-test=> (clojure.test/run-tests)

Testing build.uberjar-test
Clean
  Delete /Users/dan/projects/work/metabase/target/classes
  Delete /Users/dan/projects/work/metabase/target/uberjar/metabase.jar

FAIL in (class-file-conflicts-test) (uberjar_test.clj:55)
No unexpected class file conflicts in the EE uberjar
Unexpected class file conflicts in packages: #{"javax/activation"}
If benign, add to known-conflicting-prefixes with a comment. If dangerous, add a conflict handler in build.uberjar.
expected: (empty? unexpected)
  actual: (not (empty? #{"javax/activation"}))

Ran 1 tests containing 1 assertions.
1 failures, 0 errors.
{:test 1, :pass 0, :fail 1, :error 0, :type :summary}
```

Here's the output when i removed the `activation-conflict-handler` from
the uberjar-er. It hits conflicts on these files.

It doesn't know where the conflicts come from. The api is aware of the
lib trying to write and there's already a conflict. So the order isn't
stable and we must match on the files being written, not the lib
writing. Because lib A might write first so lib B tries to write the
same class we think it's conflict on B. But then B is written first and
now it appears a conflict in A.

also an exclusion in databricks so it doesn't bring in another logger
and we will no longer get this:

```
❯ clj -J"$(llm-repl 6006)" -M:"$ALIASES":build:build-dev:llm-repl
Downloading: vlaaad/reveal/maven-metadata.xml from clojars
WARNING: Specified aliases are undeclared and are not being used: [:performance]
SLF4J(W): Class path contains multiple SLF4J providers.
SLF4J(W): Found provider [org.apache.logging.slf4j.SLF4JServiceProvider@122ea8dc]
SLF4J(W): Found provider [org.slf4j.jul.JULServiceProvider@6cb417fc]
SLF4J(W): See https://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J(I): Actual provider is of type [org.apache.logging.slf4j.SLF4JServiceProvider@122ea8dc]
```

* fixing it up from comments
2026-05-23 10:33:22 -05:00
Cam Saul
cc84d93477 Enable cljfmt fixing comment alignment (#74613)
* Enable cljfmt fixing comment alignment

* Align bb cljfmt version

* Revert cljfmt version bump for now

* Update with latest master
2026-05-22 15:58:22 -07:00
Alexander Polyankin
53b4f86d88 Inactive table field import (#74675)
* fix table/field creation in serdes

* tests

* fix

* fix

* fix the test

* fix

* fix
2026-05-22 18:57:48 -04:00
github-automation-metabase
cda6cc0581 Auto-update documentation for master (#74686)
Update generated docs (master)

Co-authored-by: github-actions <github-actions@github.com>
2026-05-22 17:33:08 -04:00
Oleksandr Solovyov
af995a8dea [BOT-1367] Representations based notebook query tooling (#73052)
* agent-lib: representations YAML parser + schema validator

First step of migrating construct_notebook_query from the sexp-in-array DSL to
the canonical MBQL 5 representations format (see repr-plan.md).

Adds metabase.agent-lib.representations with:

- parse-yaml: wraps metabase.util.yaml with :keywords false so 'lib/type'
  string keys survive intact (we lose the 'lib/' namespace segment if
  clj-yaml keywordizes them). Throws :invalid-representations-yaml with
  :agent-error? false for malformed input.
- validate-query: Malli schema covering the Phase-1 subset (mbql/query top
  level, single-stage mbql.stage/mbql, portable FK vectors for source-table
  and field refs, clause shape [op {opts} ...args]). Structural only; deep
  clause content is validated by lib.schema/Query after resolution.
- parse-and-validate: convenience combinator.

Implementation notes captured in the plan:

- The Malli schema avoids :cat+:ref recursion (triggers
  :malli.core/potentially-recursive-seqex). Clause-shape is encoded as a
  flat [:and vector? [:fn ...]] structural predicate.
- clj-yaml returns #ordered/map (OrderedMap) which satisfies map?
  - no special handling needed in downstream passes.

* models.serialization.resolve: MetadataProvider-backed resolver

Adds metabase.models.serialization.resolve.mp with import-resolver and
export-resolver factories that satisfy SerdesImportResolver /
SerdesExportResolver via a lib.metadata.protocols/MetadataProvider rather
than the application DB.

This lets the representations resolver (next commit) translate portable
FK paths like [Sample, PUBLIC, ORDERS, CREATED_AT] to numeric IDs without
round-tripping through toucan2, so it works for any MetadataProvider the
caller supplies (application DB, mock, saved-question graph, etc.).

Implemented methods (sufficient for Phase 1):
- import-table-fk / export-table-fk
- import-field-fk / export-field-fk  (handles JSON-unfolded nested fields
  by walking :parent-id upward on export and descending name+parent-id
  on import)

Other protocol methods throw :not-implemented-yet so unsupported serdes
entry points fail loudly rather than silently dropping through.

Implementation notes:
- Uses lib.metadata.protocols/table and /field directly, not the mu/defn
  wrappers in lib.metadata - those throw Malli validation errors on nil
  returns; the protocol methods return nil cleanly on a miss, which is
  what we want for 'unknown-table' / 'unknown-field' errors.

* agent-lib: representations resolver (portable YAML -> canonical MBQL 5)

Step 3 of the representations migration. Adds
metabase.agent-lib.representations.resolve with:

- keywordize-query: recursively converts string map keys to keywords
  while preserving string *values* (so 'lib/type: mbql/query' becomes
  :lib/type :mbql/query but portable FK strings like 'ORDERS' stay as
  strings).
- resolve-query: main entry point. Keywordizes -> binds
  resolve/*import-resolver* to the mp-backed resolver -> calls
  resolve/import-mbql -> normalizes through lib.schema/query with the
  MetadataProvider attached.
- parse-validate-resolve: YAML string + MP -> MBQL 5 query in one call.

Also extends .../resolve/mp.clj: implements import-fk-keyed for
:model/Database by name (needed because import-mbql dispatches on the
top-level :database key). Throws :unknown-database with :agent-error?
true when the YAML references a database other than the MetadataProvider's.

Kondo module config updates:
- Adds metabase.agent-lib.representations{.repair,.resolve} to the
  agent-lib module's :api.
- Adds 'models' to agent-lib's :uses (resolver lives under
  metabase.models.serialization.resolve.*).
- Exposes metabase.models.serialization.resolve.mp on the models :api.

Tests cover aggregation+breakout with :temporal-unit, filters+limit+
order-by, fields projection, explicit join with alias+strategy+join-alias,
and three error paths (unknown-table, unknown-field, database-mismatch).

* agent-lib: representations repair pass (fills in missing {} options and lib/type markers)

Step 4 of the representations migration. LLMs reliably drop two things
when writing repr YAML:

1. The empty options map in clauses - writing [count] instead of
   [count, {}], or [field, [Sample, PUBLIC, ORDERS, ID]] with no
   options slot.
2. The lib/type markers on the top-level query and on stages.

Rather than surfacing these as validation errors to the LLM (which wastes
turns and tokens), we normalize them in a repair pass before validation.

metabase.agent-lib.representations.repair composes two idempotent passes:

- ensure-clause-options*: inserts {} at position 2 of clause-like vectors.
  Three cases handled:
    [op]                     -> [op, {}]
    [op, nil, ...]           -> [op, {}, ...]
    [op, <non-nil-non-map>]  -> [op, {}, <non-nil-non-map>]  (insert,
                                                              never replace)
  The clause-like? predicate distinguishes real clauses from portable FK
  paths via looks-like-fk-path? (length >= 3, all string?/nil?). It also
  excludes map-entry? so clojure.walk/postwalk doesn't misfire on map
  traversal.

- ensure-lib-types*: fills 'lib/type': 'mbql/query' on top-level query
  maps and 'lib/type': 'mbql.stage/mbql' on stage-like maps. Uses string
  keys because repair runs pre-keywordization.

Both passes are idempotent - running them twice is equivalent to running
them once. The test suite proves this generatively (defspec, 100
iterations) in addition to 19 example-based assertions.

* metabot: flip construct_notebook_query to representations YAML format

Step 5 of the representations migration. Wires the parse->repair->
validate->resolve pipeline into the construct_notebook_query tool,
replacing the sexp-in-array program format.

metabase.metabot.tools.construct:
- New execute-representations-query: resolves source-entity's database ->
  builds an application-DB-backed MetadataProvider -> runs the repr
  pipeline -> wraps result in the same {:structured-output {...},
  :instructions ...} shape as execute-program. Any ExceptionInfo from the
  pipeline is re-thrown with :agent-error? true so the tool wrapper
  surfaces a helpful message to the LLM instead of a stack trace.
- Tool arg schema: :program is replaced by :query :string (YAML string).
  Kept flat at :string to sidestep MCP flatten-root-schema pitfalls;
  structural validation happens inside execute-representations-query.
- construct-notebook-query-tool now calls execute-representations-query.
- execute-program and construct-program-schema remain in place; they're
  still used by slackbot-construct-notebook-query-tool during the
  migration. Slackbot moves to repr in a later phase.
- resolve-source-database-id is now defn (was defn-) so tests can stub it
  with with-redefs.

metabase.metabot.tools.util:
- ->result-column now emits a :portable_fk key alongside :field_id when
  the column maps to a real database field. Walks the :parent-id chain
  to build the path for JSON-unfolded nested fields. Degrades to nil for
  expression/aggregation columns or when the MetadataProvider can't
  resolve the path. Numeric ids are preserved so existing callers
  (chart creation) keep working.

resources/metabot/prompts/tools/construct_notebook_query.md:
- Fully rewritten for the Phase-1 MVP subset. Documents top-level shape,
  stage shape, the 'options map at position 2 is always present' rule,
  portable FK format (3-element table, 4+ element field, null-schema for
  schemaless DBs, JSON-unfolded nested paths), examples for each MVP op
  (filter incl. 'and', aggregation, breakout with :temporal-unit,
  order-by, limit, fields projection), explicit joins with :join-alias
  propagation, implicit joins via repair, and a Phase-1 scope-out list.

Tests:
- test/metabase/metabot/tools/construct_representations_test.clj: new,
  4 tests / 12 assertions. Happy path (YAML -> fully-resolved MBQL 5),
  malformed YAML surfaces :agent-error?, unknown table surfaces
  :agent-error?, and a broken LLM-style YAML with missing lib/type and
  missing {} options round-trips cleanly through repair.
- test/metabase/metabot/tools_test.clj: migrated
  construct-notebook-query-tool-test from :program / execute-program to
  :query (YAML string) / execute-representations-query.

* metabot: expose portable FK paths in entity_details; disable implicit-joins claim in prompt

Two small follow-ups needed to actually use construct_notebook_query in
its new representations form:

1. entity_details' table-details output now includes:
   - :database_name at the top level
   - :portable_fk: [db-name, schema-or-null, table-name] matching the
     format the representations tool expects for source-table
   Per-field portable FKs were already exposed via ->result-column in
   the previous commit. Without the table-level db-name, the LLM had
   no way to construct the 3-element table FK path.

2. The construct_notebook_query prompt previously promised implicit
   joins via a repair pass ('Metabase will fill in source-field for
   you'). That repair pass isn't implemented yet - it's Phase 1 Step 6.
   Flipped the prompt language to explicitly say implicit joins aren't
   supported yet, and added it to the Phase-1 scope-out list. Users who
   want cross-table fields must write an explicit joins: section.

* metabot: migrate remaining integration tests + document_construct_model_chart to representations

After flipping construct_notebook_query to the representations format,
three other call sites were still sending the legacy :program payload:

1. metabase.metabot.tools.document/document-construct-model-chart-tool
   wraps construct-notebook-query-tool and forwarded its :program arg
   through unchanged. Renamed its schema field :program -> :query and
   updated the inner call. This was an actually-broken call site, not
   just a test.

2. test/metabase/metabot/agent/core_test.clj integration-search-query-
   chart-flow-test drives the real agent loop with a scripted LLM and
   exercises construct_notebook_query end-to-end against the H2 test
   database. Migrated its scripted tool-call to the representations
   YAML format. The test looks up the real database name, schema, and
   table name at runtime to build portable FK paths, so it stays
   resilient to the test DB details. JSON-encoded flow-sequence
   literals embedded in the YAML string (valid YAML since YAML is a
   superset of JSON) avoid the pr-str pitfall where Clojure writes
   [nil ...] instead of [null, ...].

3. test/metabase/metabot/tools/charts/edit_test.clj's
   edit-chart-of-constructed-query-test is another real integration
   test - it calls construct-notebook-query-tool against the products
   table and then exercises edit-chart-tool on the resulting chart.
   Migrated to use the portable_fk fields from entity_details instead
   of numeric field-ids.

4. test/metabase/metabot/tools/document_test.clj's
   document-construct-model-chart-tool-test was a unit test using
   with-redefs; just updated the arg shape to match the new schema.

All three integration tests now pass against the representations
pipeline end-to-end. The last two are especially valuable because they
exercise the full parse -> repair -> validate -> resolve -> QP path
with a real MetadataProvider, which is what will happen in production.

Uses metabase.util.json instead of cheshire directly, per kondo lint.

* agent-lib: auto-wire source-field for implicit joins in representations repair

Phase 1 step 6 of repr-plan.md.

Extends the representations repair pass with a third pass that walks the
first stage's clauses and fills in the `source-field` option on any
`[field, {...}, [db, schema, OTHER_TABLE, col, ...]]` reference whose target
table is reachable from `source-table` via exactly one foreign key. The QP
interprets a field with `source-field` as an implicit join, so the LLM no
longer needs to author an explicit `joins:` entry for the common
single-FK case.

* `repair` now takes a `MetadataProvider` as its second argument. The
  shape passes (ensure clause options, lib/type markers) are unchanged and
  don't need the MP; the new implicit-join pass does. Public surface stays
  one function.
* When the source-table can't be resolved in the MP, the pass silently
  no-ops and leaves the error reporting to the downstream validate /
  resolve stages (which have better messages for that case).
* When there is no FK at all, or more than one FK, the pass throws
  ex-info with `:agent-error? true` and `:error :no-fk-path` or
  `:ambiguous-fk`. `:ambiguous-fk` surfaces the candidate FK portable
  paths so the LLM can pick one and retry with an explicit
  `source-field` option.
* Field clauses that already carry `source-field` or `join-alias` are
  left alone. The `joins` subtree of a stage is not descended \u2014 its
  field clauses live in a join context and are expected to carry a
  `join-alias` already.
* Adds `resolve.mp/outbound-fks-from-table` to return the FK edges
  leaving a given table, so the repair pass can precompute a
  target-table-id -> candidate-FKs map once per stage.
* `tools/util/->result-column` now emits a `:fk_target_portable_fk`
  alongside `:portable_fk` for columns with an `:fk-target-field-id`,
  so the LLM can discover candidate source-fields via `entity_details`.
* Prompt: replaces the "Implicit joins \u2014 not yet supported" section
  with concrete rules and examples; removes the scope-out entry.

Test counts (before \u2192 after):
* agent-lib/representations.repair-test: 14 \u2192 19 tests, 17 \u2192 35 assertions
  (happy-path, skip-on-source-field, skip-on-join-alias, skip-on-source-table,
  no-fk, ambiguous, skip-joins-subtree, idempotence, skip-on-unresolvable-mp)
* serialization/resolve.mp-test: 7 \u2192 10 tests, 34 \u2192 37 assertions
  (outbound-fks happy path, no-FK table, simple MP)
* metabot.tools.construct-representations-test: 4 \u2192 6 tests, 12 \u2192 17
  assertions (implicit-join happy path, ambiguous surfaces agent error)
* All existing metabot/agent-lib tests stay green: 100 tests, 629 assertions.

* agent-lib: rewrite portable database name to match metadata provider in repair pass

Our prompt examples for construct_notebook_query consistently use `Sample`
as the database name. The LLM follows them literally, producing a query like:

    database: Sample
    source-table: [Sample, PUBLIC, ORDERS]

against the real application DB whose name is `Sample Database`. The
metadata-provider-backed resolver requires exact string equality on the DB
name, so the canonical demo flow (e.g. "total revenue per product category")
fails with :unknown-database before the query can even be validated.

Per repr-plan.md step 13, source_entity (and thus the metadata provider) is
the ground truth for the target database -- the YAML's `database:` key and
the DB component of every portable FK are redundant w.r.t. the MP. Add a
new repair pass that overwrites both with the MP's actual DB name before
validation/resolution.

The pass is permissive: any DB name (real, prompt-derived, hallucinated)
gets normalised to the MP's name. Cross-DB queries are not supported by
this tool, so the MP unambiguously identifies the only legal DB.

* Unit tests in repair_test.clj: top-level rewrite, portable-FK rewrite,
  no-op when names already match, no-op when mp=nil, idempotency.
* Reproducer in construct_representations_test.clj: verbatim YAML from the
  bug report against an MP named "Sample Database".
* End-to-end test in charts/edit_test.clj: same YAML against the real
  application sample DB, asserting full chart construction + SQL compile.

* agent-lib: rewrite inline order-by aggregations to aggregation refs in repair pass

When the user asks for something like "top categories by total revenue",
the LLM tends to write `order-by` by re-stating the aggregation expression
inline:

    aggregation: [[sum, {}, [field, {}, [..., TOTAL]]]]
    order-by:    [[desc, {}, [sum, {}, [field, {}, [..., TOTAL]]]]]

In MBQL 5 / legacy MBQL the `order-by` direction must wrap an aggregation
**reference**, not the aggregation expression itself. `lib.normalize`
accepts the inline form, but `lib/->legacy-MBQL` produces a query that
fails legacy-schema validation the moment something downstream re-loads
it (e.g. the chart is opened, the QP round-trips through legacy MBQL and
explodes with `Invalid output: {:query {:order-by ...}}`). The agent
reports success but the user sees a broken chart.

Add a repair pass that walks each stage's `order-by`, structurally
matches each inner clause against the stage's `aggregation` list
(ignoring options-map differences), and rewrites the inline expression
to `["aggregation" {} "<uuid>"]` -- synthesising the matching
aggregation's `lib/uuid` if it doesn't already have one. Idempotent:
existing aggregation references and unmatched expressions are left alone
so validation/normalize can surface real errors.

This does NOT introduce general placeholder-UUID handling (Phase 2 step
10's `@agg-N` references); we only resolve this specific recurring
mistake.

* Update prompt with a canonical "Order by an aggregation" example
  showing that the inner expression must match an entry in
  `aggregation:`.
* Unit tests in repair_test.clj: happy path, multiple aggregations
  matched independently, reuses pre-existing lib/uuid, leaves existing
  aggregation refs alone, leaves field-orderings alone, leaves
  no-match aggregations alone, idempotency.
* Reproducer in construct_representations_test.clj: verifies the
  resolved query round-trips through legacy MBQL cleanly.
* End-to-end test in charts/edit_test.clj: same YAML against the real
  application sample DB, asserting full chart construction + SQL
  execution returning rows in descending sum order.

* agent-lib: drop `source_entity` and `referenced_entities` from construct_notebook_query (repr-plan step 13)

The YAML query is self-describing: it carries `database: <name>` at the
top level and full portable FK paths (`[<db>, <schema>, <table>]`)
everywhere else. The separate `source_entity` parameter was redundant
and could disagree with the YAML, forcing the LLM to keep two things in
sync for no reason.

* `construct-notebook-query-args-schema`: down to `{reasoning?, query,
  visualization?}`. Removed the now-unused `source-entity-schema`.
* `execute-representations-query`: signature is now `[yaml-string]`.
  Database id is derived from the parsed YAML's `database:` field by
  the new public helper `resolve-database-id-from-yaml`. Three
  `:agent-error?` paths:
  - `:missing-database-name` -- YAML lacks a `database:` field.
  - `:unknown-database` -- no `:model/Database` matches that name;
    error message instructs the LLM to use the canonical name reported
    by `entity_details`.
  - `:ambiguous-database-name` -- two databases share that name; LLM
    has no way to disambiguate, so this is a hard error.
* `resolve-source-database-id` kept untouched (still used by
  `execute-program` for slackbot until step 14 migration).
* `document-construct-model-chart-tool` lost its `:source_entity`
  schema entry too -- it's a thin pass-through to the construct tool.

The `rewrite-database-name*` repair pass (Pass 2.5, added during the
step-7 smoke test as a tactical bridge) is now obsolete: the MP is
*built from* the YAML's `database:`, so by construction the names
agree, leaving the pass nothing to do. Per repr-plan.md option (b)
("strict-with-good-error"), the pass and its 5 unit tests are deleted;
the LLM gets a clear error message instead of a silent rewrite. The
`metabase.lib.metadata` import in repair.clj goes with it.

Prompt update (`construct_notebook_query.md`): the `source_entity` /
`referenced_entities` bullets are removed; the `database:` field
description now reads "the exact database name as reported by
`entity_details`" with an explicit warning that misses fail loudly.

Tests:
* All 9 call sites in `construct_representations_test.clj` switched to
  the new single-arg signature. The old DB-name reproducer is rewritten
  to assert the new loud-failure behaviour. Added 3 new tests: missing
  `database:`, non-string `database:`, and `database:` /
  `source-table[0]` mismatch.
* `charts/edit_test.clj`: 4 call sites switched. The DB-name E2E test
  now asserts the agent-targeted error (no chart, just a clear
  `Unknown database` message). Added a sibling
  `construct-notebook-query-llm-uses-canonical-db-name-end-to-end-test`
  for the post-step-13 happy path. The order-by E2E test switched to
  the canonical name (would otherwise short-circuit at the lookup
  before exercising the order-by repair pass).
* `tools_test.clj` and `document_test.clj`: stub signatures and call
  sites updated.
* Removed 5 obsolete `rewrite-database-name-*` unit tests; replaced
  with a comment block pointing at the new test coverage.

86 tests / 529 assertions across all touched namespaces; kondo clean.

* agent-lib: migrate slackbot notebook-query tool to representations pipeline (repr-plan step 14)

The slackbot variant of `construct_notebook_query` now consumes the same
MBQL 5 representations YAML as the main notebook tool, going through
`execute-representations-query` instead of the legacy
`execute-program` + sexp-in-array flow. The slackbot-specific parts
(`:title` and `:display` options, `adhoc_viz` data parts, Slack
visualization link) are unchanged — only the query-construction engine
underneath moved.

Schema change (`slackbot-query-schema`):
* Added: `:query` (a plain `:string`; flat shape for MCP compat — YAML
  validation happens inside `execute-representations-query`).
* Dropped: `:source_entity`, `:referenced_entities`, `:program`. Per
  repr-plan step 13, the YAML is self-describing (`database: <name>`
  at the top level, full portable FK paths everywhere else), so the
  separate entity identifiers are redundant.

Follow-on cleanup in `construct.clj`:
* Deleted `construct-program-schema` — slackbot was its last consumer.
* Marked the legacy sexp-pipeline scaffolding
  (`source-entity->model-str`, `program-source->source-entity`,
  `source-metadata-for`, `surrounding-tables-for`,
  `available-measure-ids`, `build-evaluation-context`, and
  `execute-program` itself) with a section comment naming step 15 as
  the delete point. These functions are kept alive for now because the
  HTTP `/v2/construct-query` endpoint in `metabase.agent-api.api` still
  calls them; they die with the endpoint migration.
* `resolve-source-database-id` stays as-is — `execute-program` uses it,
  and kept in the "live" layer of the namespace rather than the sexp
  block because its signature is friendly enough to potentially live on
  as a public helper after step 15 (TBD).

Tests (`slackbot_query_test.clj`):
* Added `schema-shape-post-step-14-test`: asserts the new keys are
  present and the three legacy keys are gone. Acts as the schema-level
  contract guard.
* Added 4 stubbed-pipeline unit tests for the slackbot wrapper:
  - happy path (YAML pass-through, adhoc_viz data part carries title +
    display + link + query);
  - no-optional-fields path (title/display absent → not included in
    the part, but the link is still built);
  - `:agent-error?` passthrough (bare `:output` with the message,
    no structured-output or data-parts);
  - non-agent exception wrapping (generic `Failed to construct notebook
    query:` prefix so stack traces don't leak to the LLM).
* Added 2 end-to-end tests against the real sample DB:
  - happy path exercising the full repr pipeline + real
    `query->question-url`;
  - unknown-database error path, asserting the step-13
    loud-failure behaviour via the slackbot contract.

Prompt (`construct_notebook_query.md`) is shared between the two tools
and describes `:visualization` (notebook) rather than `:title`/`:display`
(slackbot). This asymmetry is pre-existing — the old slackbot schema
had `:program`, also not documented in the shared prompt — not
introduced by this commit. Left as a follow-up.

All 93 tests / 575 assertions across touched namespaces pass
(representations parser/repair/resolve + construct + slackbot +
charts/edit + document + metabot.tools). Kondo 0/0 on the three
modified files.

* agent-lib: multi-stage representations queries (repr-plan step 8)

Adds a new repair pass that lets LLMs write multi-stage queries the
way they naturally want to: a `[field, {}, "<column-name>"]` clause in
stage N referencing an output of stage N-1, with no `base-type` option
set. The lib-schema requires `base-type` on any string-named field
ref, so these queries used to blow up at `lib/query` time with a
`missing required key, got: nil` error.

## Changes

### `agent_lib/representations/repair.clj`
Adds **Pass 5 — `infer-cross-stage-field-types*`**:
* When `stages.length >= 2`, walk stages left-to-right. Before
  processing stage `i`, build a truncated query of stages[0..i-1],
  resolve it through the existing pipeline, and call
  `lib/returned-columns` on the result. Index those columns by name.
* Walk stage `i` (skipping descent into its `joins` subtree — join
  stages have their own resolution context). Every
  `[field, opts, "<name>"]` clause whose opts lack `base-type` gets
  its opts merged with the inferred `{base-type, effective-type}`.
* Best-effort and idempotent: if the mini-resolve fails, the pass
  logs at debug and lets the downstream resolver surface the real
  error; clauses whose name isn't produced by the previous stage are
  left alone so the lib-schema error points at the typo; clauses that
  already have `base-type` are never overwritten.

Also tightens the docstring on `resolve-implicit-joins*` to explain
why that pass stays stage-0-only (implicit FK joins require a
`source-table`, which only the first stage has).

### `metabot/tools/construct.clj`
Widens the `:agent-error?` catch in `execute-representations-query`:
the previous catch only wrapped parse/repair/validate/resolve, but
cross-stage field-type errors actually surface in
`result-columns-for-query`'s `lib/query` call. The new catch wraps
the entire pipeline from the MP forward, so any lib-schema violation
reaches the LLM with `:agent-error? true` set and no stack trace.

### Prompt (`construct_notebook_query.md`)
* Removes "Phase 1 MVP supports a single stage; multi-stage is coming"
  and the matching "not yet supported" bullet.
* Adds a "Multi-stage queries" section documenting the cross-stage
  ref shape (`[field, {}, <column-name>]` as a string), two concrete
  examples (post-aggregation filter `count > 10`, post-aggregation
  aggregation `avg(daily sum) by month`), and the rules (first-stage
  `source-table`, valid output column names, joins in later stages).

### Tests
* **`repair_test.clj`** — 8 new unit tests: happy path, existing
  `base-type` preserved, breakout-column inheritance, string-FK-path
  isolation, no-previous-stage no-op, unknown-column leave-alone,
  idempotency, repair+resolve smoke.
* **`construct_representations_test.clj`** — 4 new E2E tests: two-stage
  post-aggregation filter, explicit-base-type preservation, three-stage
  chain (filter + order-by both reference the aggregate), and
  unknown-column `:agent-error?` propagation.

All tests pass: 105 tests / 600 assertions across representations
parser/repair/resolve + construct + slackbot + charts/edit + document
+ metabot.tools + serialization resolver. Kondo 0/0 on the 4 modified
source/test files.

## What's NOT changed

* Resolver (`agent_lib/representations/resolve.clj`) — already handled
  multi-stage for free; `import-mbql` walks the structure recursively
  and cross-stage string refs don't match any of its vector-shape
  patterns, so they pass through untouched.
* Representations YAML schema — no new shape was needed; a
  cross-stage ref is just an ordinary `[field, opts, _]` clause with a
  string instead of an FK vector in position 2.
* Implicit-join pass — stays stage-0-only. Implicit FK joins are only
  meaningful from a stage that owns its `source-table`.

* agent-lib: support expressions in representations queries (repr-plan step 9)

- Added pass 1.5 `normalize-expressions-shape*` in repair: accepts the
  LLM-friendly map form `expressions: {Name: clause}` and normalises to the
  canonical MBQL 5 sequential form with `lib/expression-name` stamped from the
  map key. Canonical sequential input passes through unchanged; authored
  `lib/expression-name` in options is never overwritten. Idempotent.

- Resolver unchanged: `["expression" {} "<name>"]` refs are 3-element
  vectors with a string in the last slot and don't match any portable-FK
  pattern, so they pass through the resolver untouched.

- Prompt: new "Expressions (custom columns)" section with arithmetic
  (Subtotal = TOTAL + TAX) and string-concat (FullName) examples, plus a
  rules block on YAML quoting of arithmetic operators and the map-key
  shortcut. Removed expressions from the Phase 1 'not yet supported' list.

- Tests: 7 new unit tests in repair_test (map shape, sequential, sequential
  without name, existing-name preserved, absent, idempotency, multi-stage) +
  3 new end-to-end tests in construct_representations_test (map shape,
  canonical sequential, expression in breakout).

Totals: repair 34\u219241 / 59\u219270 assertions; construct 15\u219218 / 45\u219256.
Kondo 0/0 on touched files.

* agent-lib: resolve integer-indexed aggregation refs to UUID form (repr-plan step 10)

- Added pass 2.8 `resolve-aggregation-ref-indexes*` in repair: walks each
  stage, stamps `lib/uuid` on aggregation clauses that lack one, and rewrites
  every `[aggregation, opts, <int>]` ref to the canonical MBQL 5
  `[aggregation, {lib/uuid, base-type, effective-type, \u2026}, "<uuid>"]` shape.
  0-based integer index mirrors legacy MBQL 4 prior art (`[:aggregation 0]`,
  see `legacy_mbql/schema.cljc`); no new invented sigil.

- Type inference by aggregation head: count / distinct / cum-count /
  count-where \u2192 type/BigInteger; avg / median / stddev / var / share \u2192
  type/Float; sum / sum-where / cum-sum / min / max inherit from the inner
  field's annotated base-type (fallback type/Float); unknown heads \u2192 type/*.
  Authored base-type / effective-type / name / etc. on the ref's options are
  never overwritten.

- Out-of-range index raises `:agent-error?` with code
  `:aggregation-ref-out-of-range` and a message listing every available
  aggregation as `<head> at <index>`. Integer ref in a stage with no
  `aggregation:` raises `:aggregation-ref-no-aggregations`.

- Same-stage only. Cross-stage aggregation refs are out of scope here because
  pMBQL forbids that shape \u2014 the correct downstream form is a cross-stage
  field ref by column name, handled by the existing Pass 4.

- Idempotent: refs whose last slot is already a UUID-shaped string pass
  through untouched. Resolver: no change.

- Prompt: new "Aggregation references" section with the two LLM-facing
  forms (inline vs by-index) and an example. Removed "Aggregation references
  with UUIDs" from the Phase 1 'not yet supported' list.

- Tests: +9 unit tests (happy path, authored options preserved,
  out-of-range agent-error, UUID-string idempotent, no-aggs agent-error,
  no integer refs no-op, full idempotency, type inference table,
  multi-stage same-stage ref) + 2 end-to-end tests (execute-representations-query
  resolving to valid pMBQL; out-of-range surfacing clear error).

Totals: repair 41\u219250 / 70\u219298 assertions; construct 18\u219220 / 56\u219263 assertions.
Kondo 0/0 on touched files.

* resolve/mp: import Card FK by entity_id (repr-plan step 11.1)

The metadata-provider-backed serdes import resolver now handles
`import-fk` for `Card` / `:model/Card`, looking up the saved question /
model by its portable `entity_id` via `serdes/lookup-by-id`. Previously this
threw `:not-implemented-yet` and blocked the agent-lib representations
pipeline from handling `source-card` stages.

Why touch the app DB here (when the rest of the MP resolver is warehouse-only)?
Cards live in the application DB independently of whose warehouse the metadata
provider points at, and the lib metadata protocol's `metadatas` filter does
not support `entity_id` \u2014 a proper implementation would need a new metadata
spec. A one-row DB hit by entity_id is fine in practice.

Cross-database guard: if the resolved card's `database_id` does not match the
metadata provider's current DB, we throw a clear `:agent-error?` with code
`:cross-database-card` so the LLM can recover. Not-found likewise surfaces
as `:agent-error?` with code `:unknown-card`.

Other models (`Segment`, `Metric`, `NativeQuerySnippet`, etc.) continue to
throw `:not-implemented-yet`.

Tests:
- happy path (with-temp Card in current DB) for both 'Card and :model/Card
- nil input returns nil
- unknown entity_id surfaces :unknown-card agent-error
- card belonging to a different database surfaces :cross-database-card
- non-Card models still throw :not-implemented-yet

Totals: mp-test 11\u219215 / 36\u219249 assertions; regressions clean (repair 50/98,
construct 20/63).

* entity_details: expose portable_entity_id for saved questions and models (repr-plan step 11.2)

The agent uses entity_details to discover saved questions and models it can
reference in a representations query via `source-card:`. Previously the
response carried the numeric id but not the portable entity_id, so the LLM
had nothing to put into `source-card:` \u2014 it would guess `card__<id>` or
the numeric id, both of which fail serdes resolution.

This adds `:portable_entity_id` to the card-details payload (both `question`
and `model` variants), analogous to `:portable_fk` on table details.

Tests: +2 (question and model), both use `with-temp Card` and assert the
payload carries the exact entity_id.

* agent-lib: source-card end-to-end (repr-plan step 11.3)

With the MP resolver handling `import-fk` for Card (step 11.1) and
`entity_details` exposing `portable_entity_id` (step 11.2) in place, this
commit wires `source-card:` into the representations pipeline end-to-end.

Representations schema:
- added `source-card :string` to the stage map. Invalid types (e.g. integer)
  are rejected at validation with the standard `:agent-error?` relay.

Repair pipeline:
- new **Pass 5.5 `infer-source-card-field-types*`** stamps `base-type` /
  `effective-type` onto `[field, opts, "<col>"]` refs in a stage whose
  source is a `source-card:` card. Works by mini-resolving a bare
  `{source-card: <eid>}` stage, calling `lib/returned-columns`, and
  indexing the results by column name \u2014 parallel to Pass 5 for multi-stage
  queries. Best-effort no-op when mp is nil or lookup fails, so the real
  error surfaces at resolve/validate time.
- updated the top-level `repair` docstring to describe Pass 5.5 and the
  multi-pass `mp`-dependency contract.

Prompt (`construct_notebook_query.md`):
- new "Querying a saved question or model" section under "Examples for
  each MVP operation" with a filter-and-limit example, rules for column
  references by output name (same shape as cross-stage refs), mutual
  exclusivity with `source-table`, and the same-database constraint.
- "Multi-stage queries" updated to mention `source-card:` as an
  alternative first-stage source.
- removed "source-card" from the Phase-1-not-yet-supported list
  (which is now empty).

Tests (E2E):
- happy path: `source-card: <21-char NanoID>` resolves to numeric id 500;
  field reference `[field, {}, "TOTAL"]` gets `base-type :type/Float`
  stamped from the card's result-metadata.
- unknown entity_id (valid shape) surfaces `:agent-error?` with code
  `:unknown-card`.
- schema rejects non-string `source-card` value (integer) with an agent-
  error.

Totals: construct 20\u219223 / 63\u219271 assertions; regressions all green
(repair 50/98, mp 15/49, entity-details 17/71). Kondo 0/0 on touched files.

Dev-workflow note: Metabase entity_ids are 21-char NanoIDs
(`#"^[A-Za-z0-9_-]{21}$"`). The serdes import resolver's
`{:source-card (id :guard portable-id?)}` pattern only fires when the
id has that shape; malformed ids pass through untouched and blow up at
schema validation (`pos-int?`). Tests that exercise the `:unknown-card`
path must therefore use a valid-shape NanoID that simply doesn't exist,
not a short fake.

* agent-lib: harden source-card prompt + :unknown-card remediation message (repr-plan step 11 follow-up)

Real-world test revealed two cooperating problems:

1. The LLM confidently invents a `source-card:` entity id when it already
   knows the card's numeric id from a prior tool call, rather than calling
   `entity_details` first. This produces `:unknown-card` with a technically-
   correct but un-actionable message ("No saved question or model found
   with entity_id X").

2. When the tool rejects the invented id, the agent falls back to a native-
   SQL workaround like `SELECT * FROM {{#175-orders-count}} AS orders_count`,
   which defeats the point of the representations pipeline.

Fix:

- **Prompt** (`construct_notebook_query.md`): the "Querying a saved question
  or model" section now opens with a mandatory three-step workflow (call
  entity_details \u2192 copy portable_entity_id verbatim \u2192 reference fields by
  their `name`). Explicitly bans inventing, constructing, abbreviating, or
  pattern-matching entity_ids, and bans `SELECT ... {{#<id>-<slug>}}` as a
  fallback. Added a corresponding one-liner in "Rules and common mistakes"
  for repeated exposure.

- **Error message** for `:unknown-card`: now tells the LLM exactly how to
  recover \u2014 call `entity_details` with `entity-type: question` / `model`
  and the numeric id, then copy `portable_entity_id` verbatim. Matches the
  prompt guidance so the agent can self-correct without further hints.

No test impact: existing tests assert on ex-data (`:error :unknown-card`,
`:agent-error? true`), not on the user-facing message text. Totals unchanged:
mp 15/49, construct 23/71, repair 50/98. Kondo clean.

* metabot: expose portable_entity_id on card/model results to unblock source-card (repr-plan step 11 follow-up)

Even with a strict prompt and a helpful `:unknown-card` error message, the
LLM kept inventing entity_ids when asked to reuse a saved question as a
`source-card:` source. Root cause: the tool responses the LLM consults
to identify cards (`search`, `read_resource` for `metabase://question/<id>`
and `metabase://model/<id>`) never actually showed the card's entity_id.
So the agent had a numeric id and a name, but no way to derive the 21-char
NanoID \u2014 and defaulted to fabrication or native-SQL `{{#id-slug}}`
workarounds.

This commit plumbs `portable_entity_id` through to those response shapes
so the agent always has it one copy-paste away from where it needs it.

- **`search` tool** (`metabase.metabot.tools.search`): new
  `enrich-with-portable-entity-ids` post-processing step that, for
  `question` and `model` results, looks up the Card's `entity_id` in bulk
  (one `t2/select-pk->fn` keyed by id) and attaches it as
  `:portable_entity_id`. Tables/dashboards/metrics/transforms are left
  alone \u2014 `source-card:` only accepts cards. Bulk lookup keeps search
  overhead flat regardless of how many cards come back.

- **XML rendering** (`metabase.metabot.tools.shared.llm-representations`
  + `resources/metabot/prompts/llm_representations.selmer`): threaded
  `portable_entity_id` into
  - `search-result->xml` (rendered as a `portable_entity_id="..."`
    attribute on the result tag, emitted only when present so unrelated
    result types stay unchanged)
  - `question->xml` (attribute on `<metabase_question>`)
  - `model->xml` (attribute on `<metabase-model>`)

  `entity_details`' `card-details` already exposed `:portable_entity_id`
  in its structured-output (step 11.2) \u2014 it just wasn't reaching the XML
  the LLM actually reads.

- **Prompt** (`construct_notebook_query.md`): updated the mandatory
  workflow under "Querying a saved question or model" to reflect that
  both `search` and `read_resource` now carry `portable_entity_id`
  directly, so the common path is to reuse it from the already-loaded
  search response rather than calling `entity_details` again.

Tests:
- `enrich-with-portable-entity-ids-test` (search_test): verifies cards
  (question and model variants) pick up `:portable_entity_id`, and that
  dashboards do not.
- New `testing` blocks in `question->xml-test` and `model->xml-test`:
  attribute is emitted when present, omitted when absent.
- `search-result->xml-test` extended with four blocks covering question,
  model, card-without-eid, and table (which must never carry the
  attribute).

Totals: search-test 7\u21928 / 35\u219245, llm-representations-test 19\u219220 /
220\u2192228. All regressions green (mp 15/49, entity-details 17/71, repair
50/98, construct 23/71). Kondo clean.

Dev caveat: Selmer templates are loaded once via
`metabase.metabot.agent.prompts/template-cache`. When iterating on
`llm_representations.selmer` in a running REPL, call
`(metabase.metabot.agent.prompts/clear-cache!)` to pick up edits \u2014
otherwise the old template shape lingers and the new attributes appear to
be silently dropped.

* repair: unwrap nested [field opts [field inner-opts target]] clauses

LLMs occasionally write a field clause wrapping another field clause, e.g.

  ["field" {"temporal-unit" "month"}
   ["field" {} ["Sample" "PUBLIC" "ACCOUNTS" "CANCELED_AT"]]]

reasoning as "apply granularity to this existing field." The intended MBQL
is a single field clause with the options merged. Without repair, the nested
shape caused FK resolution to fail loudly ("Portable field FK ... must have
at least 4 elements"), since the inner [field ...] clause was treated as an
FK path.

Add Pass 1.7 (normalize-shape stage) that collapses nests of any depth into
one flat [field <merged-opts> <target>] clause. Outer options win on key
conflicts (outer is what the LLM thought it was adding). Works for FK
targets and string-column (cross-stage) targets alike.

* metabot: eliminate metric/table/field hallucination surfaces (repr-plan)

Sweeping set of fixes driven by benchmark telemetry — LLM kept inventing
schemas, tables and fields because the payloads it saw did not carry the
identity data it needed. Moves the contract from 'LLM infers from
vibes' to 'LLM copies the portable path verbatim' and adds targeted
agent-facing errors for the remaining slips.

Benchmark result on NLQ Metrics - Categorical Filters: 96% score /
78% passing (up from systemic construct_notebook_query failures);
remaining 2 failures are judge-semantic, not tool-level.

Contract shape (M2, M3, V1, Important A)
- Remove top-level `database:` from the LLM-facing query contract; the
  repair layer stamps it internally from source-table[0] / source-card,
  so the LLM only has to anchor one FK path and cannot contradict itself.
- `<metric>` tags gain `base_table_fully_qualified_name`, wired through
  `enrich-with-metric-base-tables` on search results and directly out of
  `metric-details` for read_resource. LLM now knows which table to put
  in `source-table:` when consuming a metric.
- `<field>` tags gain `fk_target_fully_qualified_name` whenever the
  column carries an FK. Closes the 'write customer.email on the facts
  table' class of hallucinations.
- `<table>`, `<metric>`, `<search-result>` tags carry `database_name`
  consistently so cross-DB portable FKs never have to be guessed.
- `portable_entity_id` surfaced on card/model/metric details.

metric-details dual-shape fix
- `metric-details` is called with two shapes of `card`: a t2 row
  (snake_case) from `get-card`, and a `lib.metadata/card` kebab-case
  SnakeHatingMap forwarded by `convert-metric`. Earlier code read
  `(:dataset-query card) -> :query -> :source-table` which was
  broken twice over (wrong case, legacy MBQL-3 shape); and reading
  `:table_id` on the kebab map threw deprecation errors that surfaced
  to the LLM as empty `<resource> **Error:**` blocks. Now accepts both
  `:table-id`/`:table_id` and `:entity-id`/`:entity_id` and pulls the
  base table off `report_card.table_id` directly.

Targeted agent-facing errors (F, F.2, FK-hints)
- `find-table` in resolve/mp now distinguishes `:unknown-schema` (with
  available schemas), `:unknown-table` wrong-schema variant (lists the
  schemas where the name exists), and `:unknown-table` fuzzy variant
  using token-level Jaccard with a same-schema boost (handles
  `fct_cards -> int_brex_card_dim` via the shared `card` token).
- `find-field` now surfaces FK-reachable candidates when the column
  name exists on an outbound FK target: 'No column "name" on table
  X. It exists on FK-linked tables: [Analytics, schema, target, name].'
  The LLM copies the path and the implicit-join path takes over.
- All agent-facing errors tagged `:agent-error? true` so the tool
  wrapper logs them at debug instead of error-with-stacktrace.

uri-in-source-table fail-fast (M3)
- Detect `source-table: "metabase://..."` / metric URIs early and
  return a targeted hint pointing at read_resource + the right
  base-table path, instead of a generic validation failure.

Prompt updates (construct_notebook_query.md)
- 'Using a metric' rewritten: explicit 'do not invent schema/table
  names; copy base_table_fully_qualified_name verbatim'.
- 'Implicit joins' tip rewritten around the new
  `fk_target_fully_qualified_name` attribute with a concrete
  customerio example.

Log noise (construct.clj)
- Split the tool-level catch by `:agent-error?`: expected agent
  feedback logs at debug with no stacktrace; real errors keep
  `log/error e`.

Tests
- New / updated: metric base-table enrichment, metric XML FQN, field
  XML FK attribute, entity-details base-table regression,
  construct representation error hints, repair unwrap coverage,
  schema/table/field resolver error tiers + FK-candidate hints.
  142 tests / 623 assertions green.

Files:
- resources/metabot/prompts/{llm_representations.selmer,tools/construct_notebook_query.md}
- src/metabase/agent_lib/representations{,/repair}.clj
- src/metabase/metabot/tools/{construct,entity_details,search,util}.clj
- src/metabase/metabot/tools/shared/llm_representations.clj
- src/metabase/models/serialization/resolve/mp.clj
- matching tests under test/

* Slim down LLM entity representations prompt

- Drop redundant attributes from rendered XML (database_name where
  database_id/engine already present, portable_entity_id, FK target
  FQN, base_table FQN in metric/search).
- Remove human-prose definition descriptions for measures/segments;
  keep the formal <definition> only.
- Remove standalone <collection> rendering and the question fields
  block to reduce prompt noise.
- Rename segment_id attribute to id for consistency with other tags.

* Pass resolver explicitly to import-mbql

Use the 2-arity `import-mbql` form instead of `binding` the dynamic
`*import-resolver*`. The dynamic var is intended for the serdes-import
middleware stack, where the resolver is threaded through many internal
calls. Here we have a single, local call site; passing the resolver
as an argument is simpler, easier to test, and avoids relying on
thread-local state.

* Split asset and metadata lookups in serdes mp resolver

The metadata-provider-backed resolver had two implicit responsibilities:
warehouse-schema lookups (databases, tables, fields) backed by the
`MetadataProvider`, and Metabase-content lookups (cards by entity_id)
that quietly went through `serdes/lookup-by-id` and therefore required
the application database.

Make that split explicit by introducing a small `ContentStore`
protocol with a default app-DB-backed implementation. `import-resolver`
now takes an optional `content-store` as its second arg; the 1-arity
form preserves today's behavior. This unlocks reusing the resolver in
contexts without an app DB (the serdes checker, in-memory fixtures,
snapshots) without further refactoring.

Add tests exercising a custom in-memory content store for the happy
path, the unknown-card branch, and the cross-database-card branch.

* Honor extended source-field variants in implicit-join repair

The implicit-join repair pass now leaves any field clause whose options
already carry one of the disambiguating keys alone:

  * source-field             (already supported)
  * source-field-name        (new) - LLM-authored implicit join from a
                                     previous stage's output column
  * source-field-join-alias  (new) - LLM-authored implicit join where
                                     the FK-bearing column lives on an
                                     explicitly-joined table
  * join-alias               (already supported)

We do not auto-fill the two new variants - filling them in correctly
would require resolving the previous stage's returned-columns or an
explicit-join's FK shape, which is out of scope here. The resolver and
`lib.normalize` already pass these strings through unchanged, so the
remaining work is purely "don't second-guess what the LLM authored".

Tests: two new repair-pass leave-alone tests, plus an end-to-end
resolve-test that asserts both keys propagate through to pMBQL with the
expected types (string for the names, numeric id for any
co-existing portable `source-field`).

Prompt: a single rules entry under "Implicit joins" describing each
variant in one line; no extra examples.

* Migrate /v2/construct-query and /v2/query to representations YAML

Both endpoints now accept a flat `{\"query\": \"<yaml-string>\"}` body in the
canonical representations YAML format, the same format already used by the
`construct_notebook_query` / `document_construct_model_chart` /
`slackbot_query` agent tools (after step 14). The old sexp-style
`{source, operations}` envelope is gone.

Why
- The YAML is fully self-describing: the database is derived from the first
  stage's `source-table:` / `source-card:`, all field references are
  portable FKs, and there is no auxiliary `source_entity` /
  `referenced_entities` envelope (consistent with repr-plan step 13).
- Single source of truth for the query format reference: the prompt
  `resources/metabot/prompts/tools/construct_notebook_query.md` documents
  the YAML for the LLM and now also for the HTTP endpoint.
- MCP-friendly: the input schema stays a flat `:string` at the root, so
  inputSchema flattening for the MCP `construct_query` / `query` tools
  Just Works without any `:and`/`:fn` wrappers.

Endpoint changes (`agent_api/api.clj`)
- `::program-request` / `agent-lib/program-schema` re-export gone; the
  `metabase.agent-lib.core` alias is unused and removed from the require.
- New `::construct-query-request` = `{:query NonBlankString}`.
- `::query-request` multi-dispatch: `{continuation_token | query}` (tag
  rename `:program` -> `:fresh`).
- New `evaluate-yaml-to-live-query` / `evaluate-yaml-for-execution` thin
  wrappers around `metabot-construct/execute-representations-query` (the
  pipeline already raises `:agent-error?` ex-data on input failures, which
  the exception middleware surfaces as 400s).

Tests (`agent_api/api_test.clj` + `mcp/api_test.clj`)
- New `orders-yaml` / `orders-field-ref` helpers build a representations
  YAML payload against the live application database, keeping per-test
  bodies compact.
- All previously-program-shaped tests rewritten as YAML-shaped tests with
  the same coverage: simple table query, limit handling, count
  aggregation, filter clause, metric-via-aggregation reference,
  single-page combined query, continuation-token pagination, per-page cap.
- 404-for-unknown-table replaced by 400-with-`:unknown-table`-ex-data
  (the new agent-error path; the old 404 was a side effect of the pre-repr
  source-entity fetch).
- New `construct-query-rejects-empty-query-test` covers schema-level
  rejection of missing/blank `:query`.

Documentation (`reference.md`)
- The big sexp-program section (\~250 lines covering operators, references,
  filter operators, aggregation operators, temporal helpers, four worked
  examples) is replaced by a thin description of the wire protocol plus a
  pointer to the prompt for the format reference. The reference no longer
  duplicates information from the prompt; the prompt is now the single
  source of truth.
- Error-response table reflects the actual error keys the repr pipeline
  raises (`unknown-database`, `unknown-table`, `unknown-field`,
  `unknown-card`, `missing-source-in-first-stage`, `ambiguous-fk` /
  `no-fk-path`, `uri-in-source-table`).

Tests run (clojure-eval): 79 tests / 376 assertions across
`agent_api.api-test`, `mcp.api-test`, and the construct-representations
tool-test suite. All green; clj-kondo 0/0 on the four touched files.

The legacy sexp-pipeline scaffolding in `metabase.metabot.tools.construct`
(`execute-program`, `program-source->source-entity`,
`build-evaluation-context`, `source-entity->model-str`,
`resolve-source-database-id`) is now unused everywhere; deletion is its own
follow-up commit.

* Delete now-unused legacy sexp-pipeline scaffolding from metabot.tools.construct

After step 15 migrated the HTTP /v2/construct-query and /v2/query
endpoints to the representations YAML pipeline, the sexp-program bridge
inside `metabot.tools.construct` has no callers anywhere. Dropping it
removes ~120 lines of code and the last in-tree path that resolves a
database from a single `source_entity.id` rather than from the YAML.

Removed:
  * `resolve-source-database-id` - source_entity -> db-id lookup
  * `source-entity->model-str`   - metabot-string -> agent-lib-string mapper
  * `program-source->source-entity` - HTTP source -> sexp source-entity
  * `source-metadata-for`        - lib metadata for source entity
  * `surrounding-tables-for`     - visible-columns -> surrounding tables
  * `available-measure-ids`      - measures advertised by a source entity
  * `build-evaluation-context`   - assembles the EvaluationContext
  * `execute-program`            - sexp-in-array entry point

Required namespace aliases that became unused are dropped too:
  * `metabase.agent-lib.core`     (only `agent-lib/evaluate-program`)
  * `metabase.lib.metadata`       (only `source-metadata-for`)

Verification: a repo-wide grep for each removed symbol returns no hits
outside `construct.clj`. Tests run via clojure-eval (177 tests / 692
assertions across agent_api.api-test, mcp.api-test,
construct-representations-test, slackbot-query-test, document-test, the
three agent-lib repr test namespaces, and serialization resolve.mp-test)
all green; clj-kondo 0/0 on the touched file.

Note: the underlying agent-lib sexp pipeline (`metabase.agent-lib.core`,
`agent-lib.schema`, `agent-lib.eval/*`, `agent-lib.mbql-integration/*`,
`agent-lib.repair.*`, etc.) is still present and referenced by its own
internal tests \u2014 that deletion is repr-plan step 16.

* Delete agent-lib sexp pipeline (repr-plan step 16)

After steps 14 and 15 migrated every consumer (slackbot, document tools,
construct_notebook_query, the /v2/construct-query and /v2/query HTTP
endpoints, and their MCP equivalents) to the representations YAML pipeline,
the entire structured-program ("sexp-in-array") side of agent-lib has no
remaining callers anywhere in the tree. This commit deletes it.

Verified absence of external callers
- `grep -rn 'metabase.agent-lib' src/ test/ --include='*.clj*'` outside of
  `agent_lib/` itself returns only the three repr requires in
  `metabot/tools/construct.clj` and one prose mention in an api-test
  docstring.
- Inside agent-lib, no non-representations namespace is required by
  `representations*` (the parser, repair, and resolver depend only on
  `metabase.lib.*`, `metabase.models.serialization.resolve*`, and stdlib
  helpers).

Removed (78 files, 7581 lines)
- src/metabase/agent_lib/{core,syntax,refs,join_spec}.clj
- src/metabase/agent_lib/{schema,schema/top_level}.clj
- src/metabase/agent_lib/eval{,/{args,invoke,source,walker}}.clj
- src/metabase/agent_lib/mbql_integration{,/{common,field_resolution,fields,joins,orderables}}.clj
- src/metabase/agent_lib/repair{,/{normalize{,/{forms,top_level}},context{,/passes},stages}}.clj
- src/metabase/agent_lib/runtime{,/{bindings,fields,lookup,transforms}}.clj
- src/metabase/agent_lib/validate{,/{context,cross_checks,operators,program,semantic,walker}}.clj
- src/metabase/agent_lib/capabilities{,/{derived,synthetic,catalog{,/{aggregation,expressions,filtering,joins,ordering,sources,top_level}}}}.clj
- src/metabase/agent_lib/common/{coercions,context,errors,literals}.clj
- All matching tests, the agent-lib test_util, plus fuzz_test and guards_test.

Kept (the entire repr surface)
- src/metabase/agent_lib/representations.clj             (parse + structural validation)
- src/metabase/agent_lib/representations/repair.clj      (idempotent shape passes + implicit joins)
- src/metabase/agent_lib/representations/resolve.clj     (portable -> numeric ID -> normalized pMBQL)
- The three matching test namespaces.

Module-graph adjustments (.clj-kondo/config/modules/config.edn)
- `agent-lib :api`: drop `metabase.agent-lib.core` (no longer exists). The
  repr namespaces remain the public API.
- `agent-api :uses`: drop `agent-lib`. After step 15 `agent_api.api` no
  longer requires anything from `metabase.agent-lib`; it goes through
  `metabot.tools.construct` instead, which itself is the sole user of
  `metabase.agent-lib.representations*`.
- `metabot :uses` still includes `agent-lib` (for the repr namespaces) -
  unchanged.
- `lib :friends` still includes `agent-lib` because the resolver still
  reaches into `metabase.lib.normalize` / `:metabase.lib.schema/query` -
  unchanged.

Verification (clojure-eval against fresh REPL on :49319)
- 181 tests / 719 assertions across:
    metabase.agent-lib.representations-test
    metabase.agent-lib.representations.repair-test
    metabase.agent-lib.representations.resolve-test
    metabase.agent-api.api-test
    metabase.mcp.api-test
    metabase.metabot.tools.construct-representations-test
    metabase.metabot.tools.slackbot-query-test
    metabase.metabot.tools.document-test
    metabase.models.serialization.resolve.mp-test
  All green.
- clj-kondo on src/metabase/agent_lib, src/metabase/agent_api,
  src/metabase/metabot/tools/construct.clj, src/metabase/mcp,
  test/metabase/agent_lib, test/metabase/agent_api, test/metabase/mcp:
  0 errors, 0 warnings.

The agent-lib module is now lean and representations-native, as called
for in repr-plan.md "Phase 3 - cleanup and polish" step 16. Steps 17-20
(rename llm-representations -> llm-shape, normalize `:query-content` to
post-repair YAML, prompt sweep, optional /v1 endpoint cleanup) remain.

* Rename llm-representations -> llm-shape (output-side XML formatters)

Mechanical rename per repr-plan step 17.

Why
----
There are two distinct "representations" in the metabot:

  - **Input-side**: the canonical MBQL 5 YAML format that the LLM authors
    when calling `construct_notebook_query` / `/v2/construct-query`. Owned
    by `metabase.agent-lib.representations.*` and the `construct_notebook_query.md`
    prompt.
  - **Output-side**: the XML shape that we render entity payloads (tables,
    fields, metrics, search results, query metadata, ...) into when handing
    them back to the LLM as tool results / context. Owned by
    `metabase.metabot.tools.shared.llm-representations` and the
    `llm_representations.selmer` template.

Calling both "representations" was already confusing pre-PR; after this
PR's heavy work on the input-side format the symmetry became actively
misleading in code review. This commit renames the output-side surface to
`llm-shape` so the two never get conflated again.

What changed
------------
File renames (git-mv-detected):

  src/metabase/metabot/tools/shared/llm_representations.clj
    -> src/metabase/metabot/tools/shared/llm_shape.clj

  test/metabase/metabot/tools/shared/llm_representations_test.clj
    -> test/metabase/metabot/tools/shared/llm_shape_test.clj

  resources/metabot/prompts/llm_representations.selmer
    -> resources/metabot/prompts/llm_shape.selmer

Symbol updates (no behaviour change):

  - ns:    `metabase.metabot.tools.shared.llm-representations`
        -> `metabase.metabot.tools.shared.llm-shape`
  - alias: `llm-rep` (kept) - the human-readable mental model is still
        "render to LLM-facing shape", we just decoupled the *name of the
        shape* from the input-side representations name.
  - cache key + loader fn names: `llm-representations` -> `llm-shape`
        in `metabase.metabot.agent.prompts`.
  - private `llm-template-name` constant points at the renamed selmer file.
  - one stale prose comment in `agent/user_context.clj`.

Updated alias requires across consumers (no per-file logic change):

  - src/metabase/metabot/tools/{construct,resources,sql,metadata,charts,search}.clj
  - src/metabase/metabot/agent/{prompts,user_context}.clj
  - the lone test that still imported the old alias
    (test/metabase/metabot/agent/user_context_test.clj).

Verification
------------
- clj-kondo on src/metabase/metabot, src/metabase/agent_api, src/metabase/mcp
  and matching test trees: 0 errors / 0 warnings.
- Targeted tests run via clj-nrepl-eval against a freshly-loaded REPL:
  168 / 738 across construct-representations-test, slackbot-query-test,
  document-test, agent.user-context-test, agent_api.api-test, mcp.api-test,
  agent_lib.representations.{repair,resolve}-test - all green.
- `llm-shape-test` itself has 20 pre-existing failures unrelated to this
  rename: I confirmed this by stashing the rename and running the old
  `llm-representations-test` against the same HEAD - same 20 fails.
  Bisecting against `master` shows the regression dates back to commits
  `d47d35b7bec` ("Slim down LLM entity representations prompt") and / or
  the test-restoration commit `87fe407f5f`. The expectations in those
  restored tests assume an older (pre-slim) selmer template. They will be
  refreshed in repr-plan step 19 ("Final prompt cleanup") rather than as
  part of this rename, to keep this commit a pure mechanical rename.

* Emit post-repair representations YAML as :query-content (repr-plan step 18)

What changed
============
`structured->query-data` (in `metabase.metabot.tools.construct`) used to
JSON-encode the *legacy MBQL* form of a constructed query and put that JSON
in `:query-content`. That was a hold-over from the sexp pipeline: the LLM
authored the query in one shape (sexp or YAML) and saw it back in a third
(legacy MBQL JSON), with no overlap to the field paths / aggregation refs
it had just typed. On the next turn it had to re-derive everything.

This commit makes `:query-content` the **post-repair canonical
representations YAML** instead - i.e. the same string-keyed portable shape
the LLM submitted, but with all of the repair-pass normalizations applied:

  * empty `{}` options stamped on every clause that was missing one;
  * `lib/type` markers added to the query and to every stage;
  * `source-field` auto-wired on implicit-join field clauses;
  * `@agg-N` placeholder references resolved to UUID form;
  * inline order-by aggregations rewritten to aggregation-refs;
  * top-level `database:` rewritten to the metadata provider's actual name.

The LLM now sees exactly the normalized form of what it wrote, byte-for-byte
ready to be referenced on the next turn.

Implementation
==============
1. `execute-representations-query` is restructured so the repaired-and-
   validated parsed form is held in a local before resolution. The previous
   `->>` chain (parse / repair / validate / resolve) folded the repaired
   value back into the resolver immediately and discarded it; we now keep it
   so we can serialize it.

2. The repaired value is YAML-encoded with
   `{:flow-style :block}` (multi-line, human-readable - matches what the
   LLM authors and what `construct_notebook_query.md` shows in examples).

3. The serialized string is added to `:structured-output` as `:query-yaml`,
   alongside the existing `:query` (resolved pMBQL) and `:query-id` keys.

4. `structured->query-data` consumes `:query-yaml` and puts it in
   `:query-content`. The legacy-MBQL JSON path is gone (no behavioural
   reason to keep two encodings; the wire shape stays a single XML attribute).

5. `metabase.util.json` is no longer used here; replaced with
   `metabase.util.yaml` in the require block.

Test (`construct-representations-test/happy-path-test`)
========================================================
The existing happy-path test grew an extra `testing` block that:

  * asserts `:query-yaml` is a string in the structured output;
  * round-trips it through `yaml/parse-string ... :keywords false` (matching
    `repr/parse-yaml`'s convention) and verifies the parsed shape is the
    portable string-keyed form (`"lib/type" -> "mbql/query"`,
    portable FK `["Sample", "PUBLIC", "ORDERS"]` for the source-table) -
    i.e. *not* the resolved numeric-id pMBQL;
  * verifies idempotency: feeding the emitted YAML back to
    `execute-representations-query` yields a structured output whose
    `:query-yaml` is byte-equal to the first one. This is the property
    that lets the LLM use the previous turn's response unchanged.

Verification
============
- 24 tests / 94 assertions in `construct-representations-test` - all green.
- 168 tests / 742 assertions across the wider repr-touching suite
  (slackbot-query-test, document-test, agent.user-context-test,
  agent_api.api-test, mcp.api-test, agent_lib.representations.{repair,resolve}-test)
  - all green; no test relied on the old legacy-MBQL JSON shape.
- clj-kondo on the two touched files: 0 errors / 0 warnings.

Steps 19 (final prompt cleanup) and 20 (optional /v1 endpoint removal) are
the only remaining Phase-3 items.

* llm-rep to llm-shape

* Restore prompt-required attributes on llm-shape XML (repr-plan step 19)

Background
==========
Commit `d47d35b7bec` ("Slim down LLM entity representations prompt") trimmed
seven attributes off the rendered `llm_representations.selmer` template at
once, on the rationale that they were redundant given attributes already
present (e.g. `database_id` next to `database_name`, `fully_qualified_name`
next to `base_table_fully_qualified_name`).

Three of those attributes are anything but redundant: the
`construct_notebook_query.md` prompt actively instructs the LLM to read them
when authoring queries:

  * `fk_target_fully_qualified_name` on `<field>` is the *only* anti-
    hallucination signal the LLM has to know that a column lives on a
    related table - the prompt has a worked example showing
    `<field id="892" name="customer_id" fk_target_fully_qualified_name="..."/>`
    and tells the model not to guess that an unknown column is on the
    current table without checking this attribute first;

  * `portable_entity_id` on `<metric>`, `<metabase-model>`,
    `<metabase_question>` and `<{search_tag}>` is the 21-char NanoID the
    prompt requires copied verbatim into `source-card:` (or as the third
    slot of `[metric, {}, ...]`) - without it on the tag, the LLM has to
    do a follow-up `entity_details` call just to read the same value;

  * `base_table_fully_qualified_name` + `database_name` on `<metric>` and
    `<{search_tag}>` is the *only* place the prompt's metric workflow tells
    the model to read the metric's source table from. The corresponding
    "Using a metric" section in `construct_notebook_query.md` instructs:
    "combine [it] with `database_name` to form the portable FK
    [<database_name>, <schema>, <table>]".

The remaining `d47d` trims (`<collection>` standalone, the `Definition: …`
human-prose prefix on measure/segment, the question `<fields>` block, the
`segment_id` -> `id` rename) are *not* prompt-referenced; they stay slim.

The `llm-shape-test` had 20 pre-existing failures because it was restored
in commit `87fe407f5f` ("Restore mistakenly deleted tests") with assertions
asserting the *pre-d47d* template output, against the *post-d47d* template -
i.e. the wrong combination of versions. Step 16-18 work elsewhere (sexp
deletion, rename, repr YAML in `:query-content`) didn't touch this file but
shipped on top of the inconsistent state.

This commit
===========
`resources/metabot/prompts/llm_shape.selmer`: selectively un-trim the four
attributes the prompt actively references:

  - `<field … fk_target_fully_qualified_name="…">` (when present)
  - `<related-table … database_name="…">` (when present; mirrors the search/
    table tags)
  - `<metric … database_name="…" base_table_fully_qualified_name="…"
                portable_entity_id="…">` (each guarded by `{% if %}`)
  - `<table … database_name="…">`
  - `<metabase-model … database_name="…" portable_entity_id="…">`
  - `<metabase_question … portable_entity_id="…">`
  - `<{{search_tag_name}} … database_name="…"
        base_table_fully_qualified_name="…" portable_entity_id="…">`

Plus restore the `is_collection` block - not because it's prompt-mandatory,
but because `metric->xml`, `question->xml` and `dashboard->xml` already feed
the result of `(collection->xml collection)` through the
`{metric,question,dashboard}_collection_xml` slot. With `is_collection` gone
`collection->xml` returned "" and Selmer's `{% if %}` for empty string is
falsy, so the "The metric/question/dashboard is stored in the following
collection: …" block disappeared from the rendered XML even though the
collection data was passed to the template. The block is now visible again
(the dashboard, metric, and question regression tests check for it).

`test/metabase/metabot/tools/shared/llm_shape_test.clj`: align the *six*
assertions that asserted `segment_id="..."` / `<segment segment_id="..."`
or the now-removed `Definition: <description>` prose. The first set is the
`d47d` rename to `id="..."` (consistent with every other tag); the second
just changes from `is/includes` to `is/not-includes` since the prose line
isn't coming back - the structured `<definition>…</definition>` block
already inside the segment / measure carries the same information for the
LLM, more compactly.

Verification (clj-nrepl-eval)
============================
- 20 / 230 in `metabase.metabot.tools.shared.llm-shape-test`: all green.
- 215 / 1096 across the full repr-touching suite (`construct-representations-test`,
  `slackbot-query-test`, `document-test`, `agent.user-context-test`,
  `agent_api.api-test`, `mcp.api-test`, `agent_lib.representations.{repair,
  resolve}-test`, `tools.entity-details-test`, `tools.search-test`,
  `llm-shape-test`): all green.
- clj-kondo on the touched test file: 0/0 (the .selmer file is intentionally
  not Clojure and clj-kondo's parse failures on it are not our problem).

The wider step-19 sweep (`construct_notebook_query.md`, system .selmer
prompts, `agent_api/reference.md`, CLAUDE.md / internal docs) is otherwise
clean: a repo-wide grep for `agent-lib/program`, `source_entity`,
`referenced_entities`, `construct_program`, sexp-tuple `source`/`operations`
shapes, etc., turns up no remaining stale references.

* Repair: rewrite temporal-bucket-extraction aliases to canonical lib names (repr-deletion-followups 1.2)

LLMs naturally pick up English-flavoured names like `dayofweek`,
`day-of-week`, `hour-of-day`, `month-of-year`, `quarter-of-year` for the
temporal-extraction helpers. lib's canonical names (in
`metabase.lib.schema.expression.temporal`) are `get-day-of-week`, `get-hour`,
`get-month`, `get-quarter`. Without this pass the alias forms fail validation
with a generic schema error. Carried over from the old sexp pipeline's
`metabase.agent-lib.repair.normalize.forms/temporal-bucket-extraction-aliases`
(see `repr-deletion-followups.md` § 1.2).

Implementation
==============
`representations/repair.clj`:

- New `temporal-bucket-extraction-aliases` map (5 entries, identical to the
  sexp version).
- New `rewrite-temporal-bucket-aliases*` postwalk that rewrites a clause head
  iff the head is a known alias AND slot 1 is the options map (i.e. it really
  is a clause, not an FK path or a string column inside one).
- Added to the `normalize-shape*` pipeline right after `unwrap-nested-field-
  clauses*` and before `normalize-expressions-shape*` / `ensure-lib-types*`.
  Order: Pass 1 (`ensure-clause-options*`) runs first, so a bare `["day-of-
  week", x]` becomes `["day-of-week", {}, x]` before the alias rewrite sees
  it.

Idempotency
===========
The rewrite only fires when the head is an alias; canonical heads pass through
unchanged. Re-running `repair` on an already-repaired query is a no-op for
this pass.

Tests
=====
`representations/repair_test.clj`: new `rewrite-temporal-bucket-aliases-test`
covers (1) all five aliases mapping to canonical, (2) nested occurrence
inside another clause, (3) canonical heads left alone (idempotency), (4)
missing-options input (Pass 1 + Pass 1.8 chained), (5) FK-path-internal
string "day-of-week" not mistaken for a clause head.

58 / 123 in `representations.repair-test`: all green.

* Repair: rewrite order-by direction aliases to canonical asc/desc (repr-deletion-followups 1.3)

The lib schema accepts only `[asc, {}, <ref>]` / `[desc, {}, <ref>]` as
order-by direction clauses. LLMs sometimes write `[ascending, {}, …]` /
`[descending, {}, …]` (the English long form) and uppercase variants like
`[ASC, {}, …]` / `[Descending, {}, …]` (matching how they would render the
direction in prose). Without this pass any non-canonical spelling fails
schema validation. See `repr-deletion-followups.md` § 1.3.

The sexp pipeline's equivalent (`metabase.agent-lib.common.coercions/
normalize-direction`) only handled case-insensitive `asc`/`desc`. We extend
to `ascending` / `descending` here because the doc explicitly flagged them
as patterns LLMs naturally produce.

Implementation
==============
`representations/repair.clj`:

- New `direction-aliases` map (4 lowercase keys, all mapping to `asc` or
  `desc`).
- New `rewrite-direction-aliases*` postwalk gated by `direction-alias-clause?`,
  which requires a 3-element vector (head + options-map + 1 ref arg) - this
  is the order-by clause shape. The clause-shape gate is what keeps a column
  literally named "asc" inside an FK path safe (it's a 4-element string
  vector, not a clause).
- The head-match itself (`direction-clause-head?`) is case-insensitive but
  excludes the canonical lowercase `asc`/`desc` so that already-canonical
  clauses pass through walk-without-modification - cheap idempotency.
- Added to `normalize-shape*` right after `rewrite-temporal-bucket-aliases*`.

Tests
=====
`representations/repair_test.clj/rewrite-direction-aliases-test`:

- both long-form spellings → canonical (asc, desc);
- four uppercase / mixed-case variants per direction → canonical lowercase;
- already-canonical clauses pass through unchanged (idempotency);
- aliases inside an actual order-by stage are rewritten in place;
- a column literally named "asc" inside an FK path is *not* touched;
- a scalar string "ASC" outside a clause is *not* touched.

59 / 139 in `representations.repair-test`: all green.

* Repair: swap out-of-order between bounds (repr-deletion-followups 1.5)

The lib schema for `:between` does not enforce `min <= max` - there is an
explicit TODO comment in `metabase.lib.schema.filter` flagging this. LLMs
occasionally swap the two bounds, especially for date ranges (e.g.
`between(date, "2024-12-31", "2024-01-01")` for "December → January"). The
query then runs but returns no rows, which is silently misleading - exactly
the failure mode the doc flags as worth catching. See
`repr-deletion-followups.md` § 1.5.

Implementation
==============
`representations/repair.clj`: new `swap-between-bounds*` postwalk that, for
every 5-element `["between" <opts> <expr> <lo> <hi>]` clause, extracts the
two bounds via `between-bound-comparable` and swaps when `lo > hi` and both
are directly comparable scalars of the same kind. Bounds we treat as
literals:

  - numbers (Long, Double, Ratio, …);
  - strings (compared lex - ISO-8601 lex order is chronological);
  - `[absolute-datetime, {}, <iso-str>, <unit>]` clauses (compared by inner
    ISO string).

Anything else (a `field` ref, an expression clause, a `relative-datetime`
clause, a mixed pair like number-vs-string) leaves the clause alone - we
can't compare without execution.

The doc's other 1.5 sub-case ("wrap a single scalar bound as a one-element
vector when the two-arg shape was expected") is intentionally not ported -
`construct_notebook_query.md` is explicit about `between`'s three-arg shape
and the failure mode is unlikely.

Idempotency
===========
After the swap, `lo <= hi` so subsequent passes don't fire. Equal bounds
left alone (already in order). Property test `idempotency-property-test`
continues to pass.

Tests
=====
`representations/repair_test.clj`: 5 new deftests covering numeric, ISO-
string, absolute-datetime, non-literal-bound (untouched), and mixed-kind
(untouched) cases plus an explicit idempotency check on the numeric case.

64 / 151 in `representations.repair-test`: all green.

* Repair: wrap bare ISO-date strings as absolute-datetime in between (repr-deletion-followups 1.6)

LLMs frequently write `between(date_field, "2024-01-01", "2024-12-31")` -
bare strings as the bounds. lib's `:between` schema demands a temporal
expression on each side once any side is temporal; bare strings won't
satisfy `:type/Date` and the query fails validation. Carried over from the
sexp pipeline's `wrap-iso-date-as-absolute-datetime` (see
`repr-deletion-followups.md` § 1.6 - flagged as a high-frequency LLM
pattern).

Implementation
==============
`representations/repair.clj`:

- New `iso-date-pattern` regex covering `yyyy-mm-dd` plus optional
  `Thh:mm[:ss[.fff]][Z|±hh:mm]` time portion.
- `iso-date-string?`, `temporal-shaped-clause?` helpers (the latter
  recognising `absolute-datetime`, `relative-datetime`, `now`, and the
  `get-*` extraction heads).
- `wrap-iso-date-bounds*` postwalk: for every `[between, opts, expr, lo,
  hi]` clause, when at least one of `lo` / `hi` is either already temporal-
  shaped *or* an ISO-date string, wrap the bare ISO bounds as
  `[absolute-datetime, {}, <iso-str>, "day"]`.
- The "or ISO-date" arm extends the sexp behaviour slightly: sexp's
  `repair-between-bounds` only fired when one side was already temporal-
  shaped. We also handle the both-bare-ISO-strings case because it's
  unambiguous (between of two bare yyyy-mm-dd strings is necessarily a
  temporal range; numeric between uses numbers, text between is unusual)
  and otherwise fails validation outright.
- Added to the pipeline before `swap-between-bounds*` (the swap pass knows
  how to compare via the inner ISO string when bounds are wrapped).

Idempotency
===========
After the first wrap, both bounds are vectors and the wrap predicate
(string + ISO regex) no longer matches; second pass is a no-op. The
property test continues to pass.

Compose with 1.5
================
The previously-added `swap-between-bounds-iso-string-test` (commit
cc47c17619f) was updated: bare-ISO inputs now produce wrapped-and-swapped
output (canonical form), not bare-and-swapped. This is a forward step -
the lib schema rejects bare strings, so the previous test's expected
output was actually invalid as a final form.

Tests
=====
`representations/repair_test.clj`: 7 new deftests covering both-bare,
mixed (with absolute-datetime / with relative-datetime), numeric-untouched,
time-portion bounds, idempotency, wrap+swap composition, and non-ISO-string
untouched. Adjusted 3 swap-iso-string assertions in
`swap-between-bounds-iso-string-test` to expect wrapped form.

71 / 159 in `representations.repair-test`: all green.

* Repair: wrap bare 'now' literals in temporal-comparison contexts (repr-deletion-followups 1.7)

LLMs sometimes write `[<, {}, <date-field>, "now"]` or `[between, {}, <field>,
"2024-01-01", "now"]` - using the bare string `"now"` (case-insensitive,
optional whitespace) where lib expects a `["now" {}]` clause. Without this
pass the bare string fails schema validation. Carried over from the sexp
pipeline's `wrap-now-as-expression` (see `repr-deletion-followups.md` § 1.7,
flagged as worth re-adding combined with 1.6).

Implementation
==============
`representations/repair.clj`:

- `now-literal?`: case-insensitive, whitespace-tolerant predicate matching
  `"now"` / `"NOW"` / `" now "` / `"Now\n"`.
- `field-with-temporal-unit?`: a `field` clause whose options carry
  `temporal-unit`. Combined with `temporal-shaped-clause?` (from § 1.6) gives
  `temporal-context-operand?`.
- `temporal-comparison-clause?`: a clause whose head is in
  `#{=, !=, <, <=, >, >=, between}` and which has a real options-map.
- `maybe-wrap-now-operands`: walks the args of a comparison clause, returns
  rewritten args **only if at least one *other* operand in the clause is\n  already temporal-shaped**. The other-operand requirement is what keeps a
  column literally named or containing `"now"` from being silently rewritten
  in non-temporal contexts.
- `wrap-now-literals*` postwalk added to the pipeline right after
  `wrap-iso-date-bounds*` and before `swap-between-bounds*`.

Idempotency
===========
After the first pass, `"now"` is now `["now" {}]` (a vector, not a string),
so `now-literal?` no longer matches. Second pass is a no-op. Property test
continues to pass.

Tests
=====
`representations/repair_test.clj`: two new deftests covering

  - `<`/`>=`/`=` against temporal-bucketed field, absolute-datetime, plus
    case-insensitive / whitespace variants;
  - bare 'now' in a non-temporal comparison left alone;
  - already-canonical ["now" {}] left alone (idempotency);
  - between with mixed ISO + 'now' bounds (Pass 1.86 + 1.865 compose);
  - between of two bare 'now' on a numeric field left alone (no temporal
    sibling).

73 / 169 in `representations.repair-test`: all green.

* Repair: unwrap ["true"/"false" {} x] boolean wrappers (repr-deletion-followups 1.4)

LLMs occasionally write `["true", x]` thinking of it as "wrap this clause
as truthy", or `["false", x]` as "wrap as falsy". Neither has any meaning in
lib; the canonical equivalents are just `x` and `["not", {}, x]`. Carried
over from the sexp pipeline's `syntax/boolean-wrapper-form?` (see
`repr-deletion-followups.md` § 1.4 - flagged as "re-add only if seen";
porting because user requested full Phase A + B).

Implementation
==============
`representations/repair.clj`:

- `boolean-wrapper-clause?` recognises a 3-element vector
  `[<true|false case-insensitive>, <opts-map>, <one arg>]`. Restricted to
  exactly one arg so 0-arg boolean literals (`["true" {}]`,
  `["false" {}]`) and multi-arg uses of these heads are not touched.
- `unwrap-boolean-wrapper` returns `x` for true and `["not" {} x]` for false.
- `unwrap-boolean-wrappers*` postwalk added to the pipeline right after
  Pass 1 (`ensure-clause-options*`), before `unwrap-nested-field-clauses*`,
  so postwalk evaluation order unwraps innermost wrappers first.

Idempotency
===========
After the unwrap, the head is no longer "true" / "false". The 0-arg literal
case is explicitly excluded from the predicate, so `["true" {}]` /
`["false" {}]` pass through untouched. Property test continues to pass.

Tests
=====
`representations/repair_test.clj`: 5 new deftests covering

  - true wrapper unwraps;
  - true wrapper survives bare-clause input (Pass 1 + Pass 1.83 chained);
  - false wrapper rewrites to (not …);
  - 0-arg ["true" {}] / ["false" {}] literals untouched;
  - case-insensitive (TRUE/True/FALSE/False) heads work;
  - nested wrappers unwrap to the innermost expression.

78 / 180 in `representations.repair-test`: all green.

* Repair: normalise alternative case / if argument shapes (repr-deletion-followups 1.8)

lib's canonical `case` shape is
`[case, {}, [[pred1 then1] [pred2 then2] …] default?]` - a single vector of
branch pairs in slot 2 plus an optional default in slot 3. LLMs frequently
emit alternative shapes, especially when transcribing English-flavoured
prose ("if X then 100 else 0"):

  1. Three bare args: `[case, {}, pred, then, else]`
  2. Branch pairs as separate args: `[case, {}, [p1 t1], [p2 t2], default]`
  3. Flat alternating: `[case, {}, p1, t1, p2, t2, default?]`
  4. Trailing `["else" x]` branch inside the pairs vector

Without this pass each alternative fails lib schema validation. `if` is a
documented alias for `case` (lib treats them identically); we normalise
both. Carried over from the sexp pipeline's
`repair/normalize/forms/repair-case-args` (see `repr-deletion-followups.md`
§ 1.8 - flagged "re-add only if seen"; porting because user requested full
Phase A + B).

Implementation
==============
`representations/repair.clj`:

- `branch-pair?`: 2-element vector.
- `else-branch?`: `["else" x]` or its post-Pass-1 form `["else" {} x]`
  (Pass 1 fills `{}` into bare clauses before this pass runs).
- `split-trailing-else`: peels a final else-branch off the pairs vector and
  promotes its payload to the default position.
- `canonical-case-args`: uniform output construction
  `[<vector-of-pairs> default?]`, always stripping a trailing else.
- `classify-case-args`: returns `[branches default]` for any of the shapes
  above (including the canonical one, so that trailing-else stripping is
  centralised). Returns `nil` only for unrecognised shapes.
- `normalise-case-args`: builds the canonical clause and compares against the
  input - returns the input unchanged when the rewritten form is identical
  (so already-canonical input doesn't get re-wrapped, preserving cheap
  idempotency).
- `normalise-case-clauses*` postwalk added to the pipeline right after
  `swap-between-bounds*` and before `normalize-expressions-shape*`.

Idempotency
===========
`canonical-case-args` is a fixed point under repeated application: once
the form is canonical (a single vector of branch-pairs plus optional
default, no trailing else), the next pass produces the identical clause and
`normalise-case-args`'s equality check short-circuits. Property test
continues to pass.

Tests
=====
`representations/repair_test.clj`: 8 new deftests covering each shape
above, the `if` alias, already-canonical inputs (idempotency), and an
explicit idempotency check across all five shapes.

86 / 196 in `representations.repair-test`: all green.

* Repair: normalise fields: to sequential when given a single clause (repr-deletion-followups 1.9)

The repr schema for both stage and join requires
`"fields": [:sequential [:ref ::clause]]` (joins additionally accept enum
strings `"all"` / `"none"`). LLMs occasionally write a single clause as
the value, e.g.

  stages:
  - …
    fields: [field, {}, [Sample, PUBLIC, T, COL]]

rather than the canonical list-of-one. The single-clause shape would
otherwise fail validation. Carried over from the sexp pipeline's
`normalize-with-fields-selection` (see `repr-deletion-followups.md` § 1.9 -
flagged "low risk; the shape is structurally obvious"; porting because user
requested full Phase A + B).

Implementation
==============
`representations/repair.clj`:

- `single-clause-shape?`: a vector with a string head at slot 0 AND a map at
  slot 1. This excludes (a) a sequential-of-clauses (no map at slot 1 of the
  outer vector) and (b) the enum strings `"all"` / `"none"` (not vectors).
- `normalise-fields-key`: when a map's `"fields"` value is single-clause-
  shaped, wrap it in a one-element list.
- `normalise-fields-shape*` postwalk fires on every map containing a
  `"fields"` key. The walker hits both stage maps and join maps; we don't
  need separate stage / join handling because the schema and the LLM
  failure mode are identical.
- Added to the pipeline right after `normalise-case-clauses*` and before
  `normalize-expressions-shape*`.

Idempotency
===========
After the first wrap the value is a list (not a clause-shaped vector) and
the predicate no longer matches. Second pass is a no-op. Property test
continues to pass.

Tests
=====
`representations/repair_test.clj`: 3 new deftests covering

  - stage-level `fields:` single clause wrap;
  - sequential `fields:` left alone (already canonical);
  - join-level `fields:` single clause wrap;
  - join `fields:` enum value `"all"` / `"none"` left alone;
  - explicit idempotency check.

89 / 202 in `representations.repair-test`: all green.

* Repair: splat values-list args of in/=/!=/not-in to flat positional args (repr-deletion-followups 1.10)

lib's canonical shape for `:in` / `:not-in` is **flat**:
`[in, {}, lhs, v1, v2, v3]`. Likewise `:=` / `:!=` take exactly two operand
args. LLMs sometimes wrap the values into a single vector slot:

  [in,     {}, lhs, [v1, v2, v3]]   →  [in,     {}, lhs, v1, v2, v3]
  [not-in, {}, lhs, [v1, v2, v3]]   →  [not-in, {}, lhs, v1, v2, v3]

and occasionally use `=` / `!=` with a list value, expecting list-membership
semantics:

  [=,  {}, lhs, [v1, v2, v3]]       →  [in,     {}, lhs, v1, v2, v3]
  [!=, {}, lhs, [v1, v2, v3]]       →  [not-in, {}, lhs, v1, v2, v3]

Adapted from the sexp pipeline's `in`/`not-in`/`=`/`!=` repair branches in
`repair/normalize/forms.clj` (see `repr-deletion-followups.md` § 1.10). The
sexp form `[in lhs [v1 v2 v3]]` was nested by design; the repr canonical
form is flat, so we splat instead of just rewriting the head.

Implementation
==============
`representations/repair.clj`:

- `scalar-literal?`: a primitive YAML scalar (string, number, boolean, nil).
- `values-list?`: a non-empty vector whose every entry is a scalar-literal.
  This conservative predicate excludes nested clauses (vectors with a
  string head + map options) and FK-paths (which only appear inside a
  `field` clause anyway).
- `list-value-comparison-clause`: returns `[head opts lhs values]` for two
  shapes:
  - `[head, opts-map, lhs, [v1 v2 …]]` (4 elements, options already in place);
  - `[head, lhs, [v1 v2 …]]` (3 elements, options omitted).
  Both shapes are recognised because the pass runs *before*
  `ensure-clause-options*` (Pass 1).
- `normalise-list-value-comparisons*` walks the tree and applies the splat
  for `in`/`not-in` heads, plus the head-rewrite-and-splat for `=`/`!=`.

Pipeline placement
==================
**Critical:** the new pass runs *before* `ensure-clause-options*`. Pass 1
mis-identifies a 2-element values-list (e.g. `["alice" "bob"]`) as a
bare clause and would corrupt it by inserting `{}` between the two scalars,
producing `["alice" {} "bob"]`. Running the splat first means Pass 1 only
ever sees properly-structured clauses where `in`/`not-in`'s args have
already been spread to positional form. (Trying to undo Pass 1's
corruption afterwards would require very heuristic detection of "a
3-element vector with empty-map at slot 1 might actually be a corrupted
values-list".)

Idempotency
===========
After splat, the values are no longer in a vector and the predicate no
longer matches. Already-flat input passes through unchanged. Property test
continues to pass.

Tests
=====
`representations/repair_test.clj`: 5 new deftests covering

  - `in` / `not-in` splat;
  - already-flat in clause unchanged;
  - `=` / `!=` with list value rewrite + splat;
  - scalar-rhs `=` left alone;
  - clause-shaped rhs (e.g. an `absolute-datetime`) left alone;
  - mixed scalar types (numbers, booleans) splat correctly;
  - explicit idempotency check.

181 / 639 across the wider repr-touching suite (`repair-test`,
`resolve-test`, `construct-representations-test`, `agent_api.api-test`,
`mcp.api-test`): all green; 0 regressions.

* Repair: rewrite shell/Python/SQL operator-name aliases to canonical lib heads (repr-deletion-followups 1.1)

LLMs are heavily-trained on shell scripts (`man test`-style `-eq`/`-ne`/`-lt`/
`-gt`/etc.), Python comparison protocols (`__eq__`, `__lt__`, …), and SQL-ish
prose. They occasionally reach for `eq`/`equals`, `gt`/`gte`, etc. as
operator heads instead of the canonical lib symbols (`=`, `!=`, `<`, `<=`,
`>`, `>=`). `repr-deletion-followups.md` § 1.1 originally recommended NOT
re-introducing these aliases, but the cost of supporting them is one extra
hashmap lookup at the start of the pipeline and the failure mode is silent
schema rejection - so the trade-off favours support.

Implementation
==============
`representations/repair.clj`:

- `operator-name-aliases` map covering:
  - shell/Python comparison: `eq`, `ne`, `lt`, `le`/`lte`, `gt`, `ge`/`gte`;
  - verbose comparison: `equals`, `not-equals`;
  - lib renames carried over from the sexp pipeline's
    `syntax/canonical-operator-aliases`: `count-if` → `count-where`,
    `variance` → `var`, `stddev-pop` → `stddev`, `count-distinct` /
    `distinct-count` → `distinct`, `relative-date` → `relative-datetime`,
    `temporal-diff` → `datetime-diff`, `is-not-null` → `not-null`.
- `operator-alias-clause?` predicate: head is a string matching (case-
  insensitively) a known alias, slot 1 is the options-map (i.e. it really is
  a clause, not an FK path string), and head is *not* already canonical
  (cheap idempotency).
- `rewrite-operator-name-aliases*` postwalk added to `normalize-shape*`
  right after `unwrap-nested-field-clauses*` and before
  `rewrite-temporal-bucket-aliases*`.

Notably **not** ported from sexp:
- `if` → `case`: lib accepts both heads as identical (see
  `metabase.lib.schema.expression.conditional` § "`:if` is an alias to `:case`");
- `month` → `get-month`: ambiguous (could be `:temporal-bucket :month`);
- `|` → `or`: too symbolic.

Idempotency
===========
After rewrite the head is canonical, alias map no longer matches. Already-
canonical clauses pass through unchanged. Property test continues to pass.

Tests
=====
`representations/repair_test.clj`: 7 new deftests covering

  - 8 shell-style comparison aliases → canonical;
  - verbose `equals` / `not-equals`;
  - 5 aggregation lib-renames;
  - 3 temporal/null lib-renames;
  - case-insensitive head matching (EQ/Eq/EQUALS/Equals);
  - already-canonical heads untouched (idempotency);
  - alias nested in tree gets rewritten;
  - column literally named "eq" / "gt" inside FK path NOT rewritten.

189 / 670 across the wider repr-touching suite (`repair-test`,
`resolve-test`, `construct-representations-test`, `agent_api.api-test`,
`mcp.api-test`): all green; 0 regressions.

* fix repair.clj

* Repair: auto-split a stage with a post-aggregation filter into two stages (repr-deletion-followups 1.11)

LLMs habitually write SQL HAVING-style post-aggregation filters as a single
stage, e.g.

    stages:
    - source-table: [Sample, PUBLIC, ORDERS]
      aggregation: [[count, {}]]
      breakout:    [[field, {}, [Sample, PUBLIC, ORDERS, PRODUCT_ID]]]
      filters:     [['>', {}, [aggregation, {}, 0], 10]]
      order-by:    [[desc, {}, [aggregation, {}, 0]]]
      limit:       5

In pMBQL this must be expressed as **two stages** - the second consumes the
first's output and references the aggregation's column by name:

    stages:
    - source-table: [Sample, PUBLIC, ORDERS]
      aggregation: [[count, {}]]
      breakout:    [[field, {}, [Sample, PUBLIC, ORDERS, PRODUCT_ID]]]
    - filters:     [['>', {}, [field, {}, count], 10]]
      order-by:    [[desc, {}, [field, {}, count]]]
      limit:       5

lib's schema does not catch the single-stage shape (it's shape-only) - the
broken query passes validate, passes resolve, and only fails at SQL-
generation time or, worse, silently produces wrong results.

Implementation
==============
`representations/repair.clj`:

- `aggregation-column-name-table`: lib-mirroring static map from aggregation
  head to the output column name lib's `column-name-method` produces.
  Covers `count`, `distinct`, `cum-count`, `count-where`, `distinct-where`,
  `avg`, `max`, `median`, `min`, `stddev`, `sum`, `var`, `cum-sum`,
  `percentile`. `metric` is intentionally not in the table - resolving its
  column name needs the metadata-provider, and the auto-split caller emits
  a clean diagnostic when that case is hit.
- `aggregation-clause-column-name`: extracts the column name an aggregation
  produces. An explicit `"name"` in opts (lib's column-name override for
  aggregations) takes precedence over the static-table dispatch.
- `collect-stage-agg-uuids`: set of `lib/uuid` strings from the stage's
  aggregation clauses.
- `form-references-stage-agg?`: postwalk detection of any
  `[aggregation, opts, <uuid>]` clause whose uuid is in the same-stage set.
- `aggregation-uuid->column-name`: builds `{uuid → col-name}` map; returns
  nil if any aggregation can't be named (signal for "emit diagnostic").
- `rewrite-agg-refs-to-cross-stage-fields`: postwalk replaces every
  same-stage aggregation-ref with the cross-stage `[field, {}, <name>]`
  shape, dropping aggregation-specific opts (Pass 5 will re-stamp types).
- `split-post-agg-filter-stage`: detects the trigger and either returns
  `[stage]` (no split) or `[stage' new-stage]` (split). When split is
  required but column-names can't be derived, throws `:agent-error?` with
  a clear instruction.
- `split-post-agg-filters*`: top-level pass; runs after
  `resolve-aggregation-ref-indexes*` (so integer-index refs are already
  UUID-keyed) and before `infer-cross-stage-field-types*` (so the new
  cross-stage field refs get `base-type`/`effective-type` filled in).

Auto-fix policy
===============
When split fires:

- **Pre-agg filters stay in stage 0** so they filter rows before
  aggregation as the LLM intended.
- **Post-agg filters move to stage 1** with refs rewritten to cross-stage.
- **`order-by`, `limit`, `fields` move to stage 1** - the user's intent is
  "of {rows matching the filter}, the top-N sorted by X, projecting Y".
  Leaving these in stage 0 would sort/limit/project before the filter,
  silently producing different (smaller) row sets and the wrong semantics.

Idempotency
===========
After split, stage 0 has no offending agg-ref in its filters and stage 1
has no `aggregation:` vector at all. Re-running the pass finds no
triggering shape and is a no-op. Property test continues to pass.

Property-test fixup
===================
`representations/repair.clj` (`list-value-comparison-clause` from § 1.10):
the property test surfaced an idempotency failure on adversarial inputs
like `["=" nil ["DB" "S" "T"] ["v1" "v2"]]` - on the 1st run nothing
fires (no opts at slot 1, count=4, predicate misses); on the 2nd run with
`{}` stamped at slot 1 the value-list splat mis-classifies the bare
3-string FK-path as the LHS of an `=` comparison and splats. Added a
`comparison-lhs-clause?` requirement: the LHS must itself be a clause
(string head + map opts at slot 1), excluding bare vectors of strings.

Tests
=====
`representations/repair_test.clj`: 9 new deftests covering

  - simple count + post-agg filter splits cleanly;
  - pre-agg filters stay in stage 0, post-agg filters move;
  - order-by, limit, fields move to stage 1;
  - `name` opts override produces a cross-stage ref by that authored name;
  - multiple aggregations each get their own column-name mapping;
  - `metric` agg raises clean `:agent-error?` diagnostic;
  - no trigger when no agg-ref-filter present;
  - no trigger when stage has no aggregations;
  - explicit idempotency on already-split query;
  - `distinct` → "count", `count-where` → "count_where" lib-correct names.

199 / 698 across the wider repr-touching suite (`repair-test`,
`resolve-test`, `construct-representations-test`, `agent_api.api-test`,
`mcp.api-test`): all green; 0 regressions; idempotency property test
passes 100 cases.

* Repair test: expand fuzz coverage with realistic-shape generators (repr-deletion-followups 4.1)

The original `idempotency-property-test` ran 100 cases over a single 'chaos
tree' shape with 18 ops in the head pool. After Phase A + B + 1.1 + 1.11 the
repair pipeline now has 14 normalisation passes plus 7 mp-aware passes, and
each new pass is a potential source of subtle non-idempotency on adversarial
input. The original generator missed several operator heads with repair
passes attached and never produced query-shaped maps with realistic stage
structure.

Changes
=======

`representations/repair_test.clj`:

- **`gen-op-name` expanded to 56 heads** covering every op with a repair
  pass attached: comparison + shell aliases (1.1), list-membership (1.10),
  boolean wrappers (1.4), null checks (1.1), temporal range/literals
  (1.5/1.6/1.7), temporal-bucket extraction + aliases (1.2), direction
  spellings (1.3), aggregations + lib renames (1.1), and `metric` (to
  exercise the 1.11 diagnostic path).

  `case` / `if` / `else` are deliberately omitted from the chaos pool: Pass
  1.84 wraps pairs as `[[pred then]]`, and Pass 1's `clause-like?`
  heuristic mis-classifies a bare 2-element `[s1 s2]` pair as a clause and
  inserts `{}` between the two scalars. In real LLM output `pred` is always
  a clause (e.g. `[\"=\" {} field val]`), so the collision doesn't fire;
  the realistic-query generator below covers canonical case shapes with
  proper clause preds.

- **`idempotency-property-test` bumped to 200 cases** (was 100).

- **New `gen-realistic-query`**: a query-shaped map with a single stage that
  composes:
  - `gen-source-table`: portable FK to a real-looking table;
  - `gen-aggregation-clause`: simple + arg-taking aggs, with LLM-flavoured
    aliases (`count-distinct`, `count-if`, `variance`) sprinkled in;
  - `gen-filter-clause`: covers `=`, `!=`, `eq`/`gt`/`>=` aliases, `in`
    (both flat and wrapped-values shapes), `between` with potentially-
    out-of-order ISO date strings + `now`, and a post-aggregation filter
    that triggers Pass 2.9 split;
  - `gen-order-by-clause`: with mixed-case direction aliases;
  - `gen-expression-clause`: includes `gen-case-clause` (canonical, three-
    bare-args, and pairs-as-args shapes - all with clause preds), arithmetic
    (`+`), and `datetime-diff` (`temporal-diff` alias).

- **`realistic-query-idempotency-property-test`** (200 cases): for each
  generated query, asserts either:
  - `(= (repair q) (repair (repair q)))`, or
  - `repair` throws an `:agent-error?` ex-info (a clean diagnostic path,
    e.g. integer-agg-ref-out-of-range when the random generator emits a
    post-agg filter on a stage with no aggregations).
  Any other thrown exception (NPE, AssertionError, generic ExceptionInfo
  without `:agent-error?`) fails the test.

- **`realistic-query-no-bug-exceptions-test`** (200 cases): asserts that
  no input ever produces a non-`:agent-error?` exception. This is the
  contract of `repair`: it must either succeed or throw with a flag telling
  the agent layer to relay a friendly message to the LLM.

- **`repair-or-agent-error` helper**: classifies a repair call into
  `[:ok r]`, `[:agent-error data]`, or `[:bug ex]`. Centralises the
  exception triage so both new property tests share the same vocabulary.

Bug discovered, scope-documented
================================
While bringing this up the wider `gen-op-name` (with `case`/`if` in scope)
surfaced an interaction between Pass 1 and Pass 1.84 on adversarial inputs
like `[\"case\" {} \" \" \"\"]` (case with bare-string pred and then):

  - Run 1: `[\"case\" {} \" \" \"\"]` → `[\"case\" {} [[\" \" \"\"]]]`
    (Pass 1.84 wraps the two args as a single `[pred then]` pair).
  - Run 2: Pass 1 sees the inner `[\" \" \"\"]` 2-element vector, treats it
    as a clause-like `[head opts? args]` (because the first elem is a
    non-blank string), and inserts `{}`: `[\"case\" {} [[\" \" {} \"\"]]]`.
  - Result: non-idempotent.

This is a fundamental ambiguity between case-pair shape `[pred then]` and
clause shape `[op opts? args]` when `pred` happens to be a string. Real LLM
output never puts a bare string at `pred`, so production is unaffected. The
fix is generator-side: keep `case`/`if` out of the chaos pool and exercise
them only via the realistic generator (which always uses clause preds).
This is documented in-place in the `gen-op-name` literal.

Stability
=========
10 consecutive runs of `repair-test` with fresh seeds: 0 failures, 0 errors.

201 / 700 across the wider repr-touching suite (`repair-test`,
`resolve-test`, `construct-representations-test`, `agent_api.api-test`,
`mcp.api-test`): all green; 0 regressions.

* fix agent.core-test

* Repair: friendly error when a stage's aggregation: contains a [field, ...] entry (E1, repr-deletion-followups 2.3)

Discovery: a query like

    stages:
    - source-table: [Sample, PUBLIC, ORDERS]
      aggregation:
      - [field, {}, [Sample, PUBLIC, ORDERS, TOTAL]]

is silently accepted by lib (shape-only schema), passes resolve, and fails
only at SQL-generation time - or, worse, produces wrong-result SQL. The
`aggregation:` block must contain aggregation functions (`count`, `sum`,
`avg`, `metric`, …), not bare field references.

This commit introduces a new last-pass diagnostic stage in `repair`,
`friendly-errors*`, that runs every error detector in turn after every
shape pass and every mp-aware fill-in is done. Each detector targets a
single LLM-observed pattern and throws `:agent-error?` ex-info with an
LLM-actionable explanation naming the precise location.

Implementation
==============
`representations/repair.clj`:

- New section "Pass 6 - friendly error messages" introducing the pass.
- `aggregation-entry-not-aggregation-error!`: walks each stage's
  `aggregation:` vector and throws when a top-level entry is a
  `[field, ...]` clause. Names the stage and entry index in the data and
  message.
- Critically does NOT recurse into nested clauses: `[sum, {}, [field, ...]]`
  is correct (the outer `sum` is the aggregation; the inner `field` is
  its argument) and is left alone.
- `friendly-errors*` driver appended to the `repair` pipeline as the
  final step.

Tests
=====
`representations/repair_test.clj`: 4 new deftests covering

  - `[field, ...]` as aggregation entry → clean `:agent-error?` with
    `:aggregation-entry-not-aggregation` error code, correct stage/entry
    indices, message mentions `aggregation` and `breakout`;
  - canonical aggregation entries (`count`, `sum`, `avg`, `metric`)
    pass through;
  - `[field, ...]` *nested* inside an aggregation (`[sum, {}, [field, ...]]`)
    is not flagged;
  - multi-stage: error correctly reports the offending stage's index.

205 / 712 across the wider repr-touching suite (`repair-test`,
`resolve-test`, `construct-representations-test`, `agent_api.api-test`,
`mcp.api-test`): all green; 0 regressions; idempotency property tests
(adversarial chaos and realistic-shape, 200 cases each) pass.

* Repair: friendly error when case/if has "default" in opts (E2, repr-deletion-followups 2.3)

LLMs (especially those primed by JSON-Schema-style examples) sometimes
write a `case` clause with the fallback value as a `"default"` key in
the options map:

    [case, {default: "unknown"}, [[pred1 then1] [pred2 then2]]]

lib's canonical shape uses the **third positional argument** as the
fallback:

    [case, {}, [[pred1 then1] [pred2 then2]], "unknown"]

lib silently ignores the unknown `default` opts key and the case clause
evaluates to `null` for any branch miss - a wrong-result class of bug.

Implementation
==============
`representations/repair.clj`:

- `case-default-in-opts-error!`: postwalk detector firing on any `case`
  or `if` clause whose options map contains a `"default"` key. Throws
  `:agent-error?` ex-info with `:case-default-in-opts` error code and
  the offending clause.

- `friendly-errors*` driver: invokes the new detector after the
  per-stage `aggregation-entry-not-aggregation-error!` sweep.

Carried over from the sexp pipeline's
`validate/operators.clj/validate-operator-specific!` `case` branch.

Tests
=====
`representations/repair_test.clj`: 2 new deftests covering

  - `case` with `default` in opts → clean `:agent-error?` with
    `:case-default-in-opts` error code, message mentions "third
    positional argument";
  - `if` with `default` in opts → same error code (alias);
  - canonical `case` with default as 3rd arg passes through;
  - canonical `case` with no default at all passes through.

207 / 718 across the wider repr-touching suite (`repair-test`,
`resolve-test`, `construct-representations-test`, `agent_api.api-test`,
`mcp.api-test`): all green; 0 regressions; idempotency property tests
(adversarial chaos and realistic-shape, 200 cases each) pass.

* Repair: friendly error when sexp-legacy top-level ops appear as clause heads (E3, repr-deletion-followups 2.3)

LLMs trained on the older sexp pipeline still occasionally emit top-level
operations as clause heads:

    aggregation: [[aggregate, {}, [count, {}]]]
    filters:     [[filter, {}, [=, ...]]]

These were verbs at the *program* level in sexp; they have no equivalent
in repr (which uses stage-level blocks: `aggregation:`, `filters:`,
`breakout:`, `order-by:`, `expressions:`, `limit:`, `joins:`). lib
accepts the unknown clause head silently, the resolver passes, and the
QP either produces wrong results or fails at SQL-generation time.

Implementation
==============
`representations/repair.clj`:

- `sexp-legacy-top-level-ops` map: 5 offending heads
  (`aggregate`, `filter`, `order-by`, `breakout`, `limit`) each
  paired with a one-liner pointing at the canonical repr replacement.
- `sexp-legacy-op-as-clause-error!`: postwalk detector firing on any
  vector with one of these heads. Throws `:agent-error?` ex-info with
  `:sexp-legacy-op-as-clause` error code, the offending head, and the
  full clause. The `tru` template carries the head + per-head hint.
- Excludes `map-entry?` nodes critical: postwalk descends into map
  entries and a stage's `{\"breakout\" […]}` entry would otherwise look
  exactly like a `[\"breakout\", …]` clause to the detector and trip.
- Same `map-entry?` guard added to E2 (`case-default-in-opts-error!`)
  for safety.

- `friendly-errors*` driver: invokes the new detector after E1 and E2.

Carried over from the sexp pipeline's
`validate/operators.clj/validate-operator-specific!` top-level-operation
branch.

Tests
=====
`representations/repair_test.clj`: 4 new deftests covering

  - `[aggregate, ...]` clause head → clean `:agent-error?` with
    `:sexp-legacy-op-as-clause` error code, head="aggregate", message
    mentions "top-level operation";
  - `[filter, ...]` clause head → same error code, head="filter";
  - all 5 legacy ops (aggregate/filter/order-by/breakout/limit) trip;
  - canonical stage with `aggregation:`, `filters:`, `breakout:`,
    `order-by:`, `limit:` keys passes through (map-entry guard works).

211 / 729 across the wider repr-touching suite (`repair-test`,
`resolve-test`, `construct-representations-test`, `agent_api.api-test`,
`mcp.api-test`): all green; 0 regressions; idempotency property tests
(adversarial chaos and realistic-shape, 200 cases each) pass.

* Repair: friendly error when sexp-legacy [measure, ...] head appears (E4, repr-deletion-followups 2.3)

`measure` was a sexp-pipeline aggregation helper - a metabase-internal
concept that wrapped a metric and exposed it as an aggregation under a
different identifier. It has no equivalent in repr; metrics are just an
aggregation form: `[metric, <opts>, <portable_entity_id>]`. Without this
diagnostic the head is silently treated as an unknown clause and the
query produces wrong results or fails downstream.

Implementation
==============
`representations/repair.clj`:

- `sexp-legacy-measure-error!`: postwalk detector firing on any vector
  with head `"measure"`. Throws `:agent-error?` ex-info with
  `:sexp-legacy-measure` error code and the offending clause. Excludes
  `map-entry?` nodes (defensive; no realistic stage map carries a
  `measure` key, but consistent with E2/E3).
- `friendly-errors*` driver: invokes the new detector after E1, E2, E3.

Carried over from the sexp pipeline's
`validate/operators.clj/validate-operator-specific!` `measure` branch.

Tests
=====
`representations/repair_test.clj`: 2 new deftests covering

  - `[measure, ...]` clause head → clean `:agent-error?` with
    `:sexp-legacy-measure` error code, message mentions `metric`;
  - canonical `[metric, {}, <portable_entity_id>]` aggregation passes
    through (the entire point of the suggested replacement).

213 / 733 across the wider repr-touching suite (`repair-test`,
`resolve-test`, `construct-representations-test`, `agent_api.api-test`,
`mcp.api-test`): all green; 0 regressions; idempotency property tests
(adversarial chaos and realistic-shape, 200 cases each) pass.

* Repair: friendly error when [expression, opts, name] reference is blank (E5, repr-deletion-followups 2.3)

LLMs occasionally write a reference to a named expression with an empty
or whitespace-only identifier:

    filters: [['=', {}, [expression, {}, ''], 0]]

lib's schema requires a non-blank string identifier here (matching an
entry in some upstream stage's `expressions:` block); without it the
reference resolves to nothing and the query produces wrong results or
fails downstream.

Implementation
==============
`representations/repair.clj`:

- `blank-expression-ref-error!`: postwalk detector firing on any
  `[expression, <opts>, <name>]` (3-element clause with head `expression`
  and map-shaped opts) where the name slot is non-string, missing, or
  blank after trim. Throws `:agent-error?` ex-info with
  `:blank-expression-ref` error code.
- `friendly-errors*` driver: invokes the new detector after E1-E4.

Carried over from the sexp pipeline's
`validate/operators.clj/validate-operator-specific!` `expression-ref`
branch.

Tests
=====
`representations/repair_test.clj`: 4 new deftests covering

  - empty-string ref (`""`) → `:blank-expression-ref`;
  - whitespace-only ref (`"   "`) → trim-aware detection;
  - non-string ref (`nil`) → triggers too;
  - valid non-blank ref (`"my-expr"`) passes through.

215 / 738 across the wider repr-touching suite (`repair-test`,
`resolve-test`, `construct-representations-test`, `agent_api.api-test`,
`mcp.api-test`): all green; 0 regressions; idempotency property tests
(adversarial chaos and realistic-shape, 200 cases each) pass.

* Repair: friendly error when [field, opts, <int>] uses sexp-legacy numeric id (E6, repr-deletion-followups 2.3)

LLMs trained on the older sexp / legacy-MBQL pipeline still occasionally
emit the bare-numeric-id field reference shape:

    filters: [['=', {}, [field, {}, 100], 0]]

repr requires a portable FK in the third slot - either a vector
`[<database>, <schema>, <table>, <column>]` (resolved against the
metadata provider) OR a string column name (cross-stage reference).
lib's schema does reject bare numeric ids, but its error is a generic
shape mismatch ("got integer, expected vector / string") that doesn't
explain the portable-FK syntax. This detector surfaces an LLM-actionable
explanation pointing at the canonical replacement.

Implementation
==============
`representations/repair.clj`:

- `numeric-field-id-error!`: postwalk detector firing on any
  `[field, <opts>, <int>]` clause (3-element, head `field`, opts is a
  map, third slot is an integer). Throws `:agent-error?` ex-info with
  `:numeric-field-id` error code, message points at portable-FK and
  cross-stage column-name.
- `friendly-errors*` driver: invokes the new detector after E1-E5.

Carried over from the sexp pipeline's
`validate/operators.clj/validate-operator-specific!` `field` branch.

Tests
=====
`representations/repair_test.clj`: 3 new deftests covering

  - `[field, {}, 100]` numeric id → clean `:agent-error?` with
    `:numeric-field-id` error code, message mentions "portable FK" +
    "column-name";
  - canonical portable-FK `[field, {}, [Sample, PUBLIC, ORDERS, COL]]`
    passes through;
  - cross-stage string column-name `[field, {}, "count"]` passes
    through.

218 / 744 across the wider repr-touching suite (`repair-test`,
`resolve-test`, `construct-representations-test`, `agent_api.api-test`,
`mcp.api-test`): all green; 0 regressions; idempotency property tests
(adversarial chaos and realistic-shape, 200 cases each) pass.

* fix test

* Enforce permissions in repr query construction

* Fix sample database prompt examples

* Default agent input errors to HTTP 400

* Export final pMBQL back to portable repr YAML

The construct_notebook_query tool now feeds the LLM the **final** normalized
pMBQL exported back to portable representations YAML, rather than the
pre-resolve repaired form. The next LLM turn sees exactly the field paths,
card entity_ids, and aggregation UUIDs that will execute, including
everything `lib.normalize` stamped (uuids, base-types, kebab-cased keys).

Implementation:
- Add `repr.resolve/export-query` — inverse of `resolve-query`. Walks the
  numeric pMBQL through the metadata-provider-backed serdes export resolver,
  then converts the keyworded form back to string-keyed repr data with
  `:lib/metadata` stripped.
- Extend the MP-backed export resolver: `export-fk-keyed` for
  `Database.name`, `export-fk` for `Card` / `:model/Card` by `entity_id`
  (covers both `source-card` and `metric` aggregation refs).
- Fix a latent bug in `serdes.resolve/export-mbql`: the sequential branch
  dropped the explicit resolver, falling back to `*export-resolver*` and
  losing the metadata-provider context for non-clause subterms.

Tests:
- `metabase.models.serialization.resolve.mp-test`: new coverage for
  database-name and Card entity_id export plus an `export-mbql` round-trip
  asserting portable FKs / entity_ids land in the right slots.
- `metabase.agent-lib.representations.resolve-test`: round-trip
  `resolve-query` → `export-query`, including `validate-query` on the
  exported form.
- `metabase.metabot.tools.construct-representations-test`: `query-yaml`
  carries lib-normalized `lib/uuid`s and source-card numeric ids export
  back to their portable entity_id.

repr-plan.md step 18.

* Show existing card / transform queries to LLM as repr YAML

Make the agent's view of *existing* questions, models, and transform
sources symmetric with what construct_notebook_query already returns:
the same canonical representations YAML form, both for structured
(mbql.stage/mbql) and native (mbql.stage/native) queries. Previously
existing cards exposed only metadata (no body), and transform-query
sources fell back to pprint'd pMBQL with numeric ids — the latter
defeating portability for card-id / snippet-id template-tag refs.

* New `metabase.agent-lib.representations.resolve/export-query-yaml`:
  thin wrapper over `export-query` + `yaml/generate-string` that returns
  nil on bad input / export errors so a single broken card can't tear
  down LLM context-building.
* `card-details` now exports the card's own `dataset-query` (not the
  `lib/query` source-card wrapper) and threads it as `:query_yaml`.
  `get-report-details` keeps `:query_yaml` in the slim payload.
* `question->xml` / `model->xml` render the YAML as `<query>...</query>`
  inside `<metabase_question>` / `<metabase-model>`.
* `transform-query-source-text`: structured + native MBQL maps both go
  through repr export; pprint'd pMBQL is now only the last-resort
  fallback. Native legacy `:native :query` strings only fall through
  when there's no `:database` to build a metadata-provider from.

Tests: 9 new across resolve / entity-details / llm-shape / user-context
suites covering the structured + native happy paths, missing/blank
fallbacks, and the slim get-report-details payload. 64 tests / 515
assertions green in the touched suites; 159 / 445 in the wider repr
regression. Kondo 0/0.

* unicode normalization

* Auto-fill source-field-join-alias for explicit-join hops

Pass 3.5 (resolve-source-field-join-alias*) covers the explicit-join half
of repr-plan.md step 12: when a stage[0] field clause's target table is
reachable through exactly one explicit join via exactly one FK on the
joined table, repair fills in both source-field-join-alias and the
matching portable source-field. Mirrors Pass 4 (basic implicit join) but
for the case where the FK column lives on a joins[] table rather than on
source-table itself.

Conservative on purpose: skips clauses that already carry any of the four
disambiguators, defers to Pass 4 when source-table itself can reach the
target, and raises :ambiguous-fk-via-join (agent-error?) when multiple
distinct join aliases are candidates.

The companion source-field-name auto-fill (multi-stage previous-stage
column) is intentionally NOT implemented; rationale documented in
repr-deletion-followups.md (entry 1.10b).

* Close v2 agent request maps and regenerate OpenAPI

The /v2/construct-query and /v2/query request schemas were open maps,
silently accepting (and dropping) the legacy `source_entity` /
`referenced_entities` envelope from before the repr migration. Mark
both as `{:closed true}` so callers (especially the LLM, whose stale
memory keeps reaching for the old shape) get an explicit 400 instead
of having extra keys quietly ignored.

Adds `construct-query-rejects-legacy-envelope-test` covering all three
branches (construct-query, /v2/query fresh, /v2/query continuation),
asserting the 'disallowed key' Malli error for the rejected key.

Regenerates docs/api.json and resources/openapi/openapi.json so the
public spec advertises `additionalProperties: false` and drops stale
references to the removed `metabase.agent-lib.schema.program` schema.

* fix tests

* Migrate agent surface from representations YAML to JSON ::external-query

The metabot/agent pipeline now consumes and emits the canonical portable
MBQL 5 wire format (`::lib.schema/external-query`) as JSON instead of YAML.

Affects three surfaces:

- HTTP `/v2/construct-query` and `/v2/query` accept `{"query": <object>}`
  matching `::lib.schema/external-query`. The defendpoint string-transformer
  keywordizes enum values automatically.
- The `construct_notebook_query` and `slackbot_query` tools take a Clojure
  map (decoded from JSON by the agent layer) instead of a YAML string. Args
  schema is `:map` so the repair pipeline still gets a chance to fix LLM
  shortcuts (missing `{}` options, missing `lib/type` markers).
- LLM-facing query payloads — `:query_yaml` on entity_details card/report
  results, on `<metabase-model>` / `<metabase_question>` XML, on
  `construct_notebook_query` results, and on transform sources in the agent
  context — are now `:query_json` rendered as a triple-backtick `json` code
  block inside the existing XML envelope.

The repair pipeline itself keeps operating on the string-keyed portable
form; conversion happens once at the boundary via the new
`representations/external-query->portable` helper.

`construct_notebook_query.md` rewritten end-to-end with JSON examples.
Reference doc and the auto-generated OpenAPI artifacts regenerated.

* Address PR review feedback

Five fixes prompted by the PR 73052 review (all still relevant after the
rebases and the YAML → JSON migration):

- `construct.clj` `as-agent-input-error`: rename the let-binding so the
  inner `cond->` doesn't shadow its own input (nit from @escherize).
- `resolve.mp/app-db-content-store` and the agent's
  `permission-aware-content-store` now short-circuit non-NanoID
  `source-card` values to `nil`. Without the gate a malformed entity_id
  from the LLM falls through `serdes/lookup-by-id` into
  `find-by-identity-hash`, which scans the entire Card table.
- `resolve.mp/import-card-by-entity-id` reads `:database-id` defensively
  with a snake_case fallback, mirroring `export-card-by-id` so test-side
  content stores can return either shape.
- `try-export-query` logs at `warn` instead of `debug` — the catch
  swallows the failure and drops the query body from the LLM payload, so
  a real export bug would otherwise be silently invisible.
- `/v2/query` continuation-token path re-validates query permissions
  before streaming. The token is client-supplied and a user's data perms
  can change between pages; the QP middleware would catch this anyway,
  but an explicit `api/query-check` produces a cleaner 403 and avoids
  spinning up the streaming response just to abort.

* Allow :model/Table in agent-api module imports

The continuation-token permission re-check added in the prior commit calls
`api/query-check :model/Table` against the source-table id carried inside
the token. `:model/Table` lives in warehouse-schema, so agent-api's
`:model-imports` set in `.clj-kondo/config/modules/config.edn` needs to
list it for `metabase.core.modules-test/model-boundaries-test` to pass.

* Drop leftover sexp-program request defs after rebase

The latest rebase landed an auto-merge that kept both the legacy
`::construct-query-request` (referencing the now-deleted
`agent-lib/program-schema`) and the new JSON `::external-query` version,
plus the no-longer-used `ConstructQueryPrompt` /
`construct-query-prompt-max-length` helpers. The JSON-migration commit's
intent was to delete all of these; this commit reapplies that cleanup
and drops the unused `malli.util` require, restores a compileable
`agent-api/api.clj`, and regenerates the OpenAPI artifacts so they match.

* Promote measures and segments to first-class agent query clauses

The schema and serdes resolve patterns for `[:measure opts <id-or-pid>]`
and `[:segment opts <id-or-pid>]` were already in place but the
metadata-provider-backed ContentStore only implemented the Card lookup
path, so the agent could never invoke them. Benchmark/5 and bench/6
showed every *Tables – Measures and Segments* test stuck at 0.125 — the
rubric's `data_source` metric checks for measure_id / segment_id
references in the final query, which the agent had no way to produce.

Three layered changes:

- Resolver wiring (`resolve/mp.clj`). Extend `ContentStore` with
  measure / segment lookups in both directions (entity-id → row for
  import, id → row for export), implemented on `app-db-content-store`
  and on the construct tool's `permission-aware-content-store`. Wire
  `import-resolver` and `export-resolver` to dispatch on
  `:model/Measure` and `:model/Segment` with table → database cross-DB
  guards (measures and segments are table-scoped, not directly
  database-scoped like cards).

- Drop the E4 `sexp-legacy-measure-error!` repair check. It predated
  the resolve patterns and was actively rejecting the now-canonical
  agent input.

- Surface `portable_entity_id` on `<measure>` / `<segment>` tags and
  through `convert-measure-or-segment`, mirroring the existing
  `<metric>` pattern. The agent now has the opaque NanoID to paste
  verbatim into `["measure", {}, "<pid>"]` /
  `["segment", {}, "<pid>"]` clauses.

`construct_notebook_query.md` is also rewritten end-to-end to the
dense rule-card shape of the older master prompt: operator catalogs
at the end, terse anti-hallucination bullets, and §13 reframed to
treat the opaque-id clause as the preferred form (preserves the
measure/segment's identity in the rendered notebook, propagates
definition updates, surfaces in lineage tooling). The portable-form
definition emit on `convert-measure-or-segment` — added in an earlier
step — stays in place as the compose-on-top fallback for the cases
where the agent wants to extend rather than reuse a measure/segment.

Tests cover the import / export happy paths and unknown-entity errors
for both models in `mp_test`, the construct round-trip with both
clause forms in `entity_details_test`, the new XML attribute in
`llm_shape_test`, and the E4 removal in `repair_test`. Full metabot
module + serialization / agent-lib modules run clean (933 tests, 4156
assertions, 0 failures).

* Declare metabot / models imports of :model/Measure and :model/Segment

The previous commit (4af895fba69) added measure / segment references in
the metabot tools layer (`construct.clj`, `entity_details.clj`) and the
models serdes resolver (`resolve/mp.clj`), tripping
`metabase.core.modules-test/model-boundaries-test`:

- `metabot` references `:model/Measure` and `:model/Segment` — add both
  to `:model-imports`.
- `models` references `:model/Measure` — add it to `:model-imports`
  (`:model/Segment` was already declared there).

No `:uses` changes needed: we only reference the model keywords via
toucan2 and serdes lookups, not by requiring the `measures` / `segments`
namespaces directly.

* Migrate two stragglers from mt/mbql-query to Lib

`mt/mbql-query` and friends were marked `:deprecated "0.61.0"` partway
through this branch's life with the guidance "use Lib to generate MBQL
queries instead of hand-rolling legacy MBQL queries in new tests". Two
test files added during the branch missed the migration — port them
the same way other on-branch tests do (e.g. `metric-query` in
`entity-details-test`):

  (lib/query mp (lib.metadata/table mp (mt/id :orders)))

and `(lib/limit ... 5)` for the `:limit 5` variant. Both files already
required `lib` and `lib.metadata`. Test behaviour unchanged — Cards'
`:dataset_query` and `format-transform-source`'s `:query` both accept
MBQL 5 directly.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* Tighten agent resolver permissions and refuse unsafe repair splits

Five PR-blocking findings on the metabot-repr branch:

* N1 (ACL gap) — agent-side `export-query` defaulted to the unchecked
  `app-db-content-store`, so measure / segment / card refs surfaced
  inside an exported body bypassed `api/read-check`. Reachable from
  `entity_details`, the MCP `get_table` resource, and the ambient
  user-context "user is viewing this table" path (the last two pass
  `with-measures?: true` / `with-segments?: true` unconditionally).

  Introduce `metabase.metabot.tools.shared.content-store` with a
  `read-checked` wrapper that gates every one of the five ContentStore
  methods on `api/*current-user-id*`. When bound (HTTP / tool path) it
  applies `api/read-check` symmetrically across import- and export-
  direction lookups; when unbound (serdes import / background tasks /
  tests / REPL) the row passes through unchanged. `default-store` is
  `unchecked-app-db-content-store` wrapped with `read-checked`.

  `export-query` and `try-export-query` grow a 3-arity that accepts an
  explicit content-store, mirroring `resolve-query`'s shape. The three
  leaky callers — `entity_details.clj:63`, `entity_details.clj:394`,
  `user_context.clj:269` — now pass `shared.content-store/default-store`.

  `construct.clj`'s `permission-aware-content-store` reify (which had
  asymmetric posture — read-checked on import, unchecked on export)
  collapses onto the shared `default-store`. Same posture as before for
  the import direction, plus a now-correct read-check on export.

* M0 (design hygiene) — rename `app-db-content-store` →
  `unchecked-app-db-content-store` so every default-arity caller of
  `import-resolver` / `export-resolver` / `resolve-query` /
  `export-query` is visibly suspect on diff review. Four call sites
  updated. Future N1-shape bugs become much harder to miss.

* S1 (info-leak) — `unknown-table-ex-info` and `unknown-field-ex`
  enumerated schemas / sibling tables / FK-reachable target tables
  against the un-sandboxed `application-database-metadata-provider`,
  then relayed the message verbatim to the user via the `:agent-error?`
  flag. A sandboxed user could enumerate restricted schemas or table
  names just by hallucinating a `source-table:` / `field` reference.

  Strip all suggestion branches. The replacement messages echo only the
  rejected path and point the LLM at `entity_details` for self-
  correction. Drop the now-dead `tokenize` / `candidate-score` /
  `substring-table-candidates` / `fqn` helpers, the matching kondo
  requires (`clojure.set`, `clojure.string`, `metabase.util`), and the
  `outbound-fks-from-table` forward declare that supported the FK-hint
  walker. `outbound-fks-from-table` itself stays — `repair.clj`'s
  implicit-join logic still uses it.

* C2 (silent wrong data) — `aggregation-uuid->column-name` collided on
  same-head aggregations (two `["sum" …]` clauses both mapped to "sum"),
  so the cross-stage filter rewrite emitted `["field" {} "sum"]` for
  both uuids and the QP filtered the wrong column. Lib's real path goes
  through `unique-name-generator` ("sum" / "sum_2") which we can't
  replicate from a static table.

  Refuse the split when output names aren't unique. The existing
  `:post-agg-filter-needs-multi-stage` agent-error fires; the LLM gets
  a diagnostic that names the duplicate-column-name failure mode and
  suggests the `name` opts override as a remedy.

* C3 (silent intent destruction) — `split-post-agg-filter-stage`
  unconditionally moved `order-by` / `limit` / `fields` from stage 0 to
  the new stage 1 whenever a split fired. If those clauses referenced a
  pre-aggregation column that didn't survive the split, the user saw
  either an unhelpful "no matching field" deep in the QP or — worse —
  a different row set with no diagnostic.

  Refuse the split when any of those three trailing clauses is present.
  Throw a new `:post-agg-filter-with-trailing-clauses` agent-error that
  tells the LLM to author two stages explicitly. The relocation code is
  stripped — the auto-fix is now narrow: filters only.

* Reconcile post-merge tests with master and harden the construct_query API

The merge of origin/master into metabot-repr took "ours" on
`src/metabase/agent_api/api.clj` and `test/metabase/mcp/api_test.clj`
to preserve the branch's deletion of the legacy sexp/program pipeline.
That was correct for the source files but accidentally dropped 11
deftest blocks master had added in the same files. It also left in
three test subblocks that master had deleted (the cross-session
isolation assertions that no longer match master's user-scoped handle
design). This commit restores master's test coverage faithfully and
sands a few rough edges on the construct_query API surface.

* Test reconciliation (test/metabase/mcp/api_test.clj):

  - Reverted my temporary 3-arity `mcp-request` / `initialize!` arity
    overload. Restored master's separate `mcp-request-as` and
    `initialize-as!` helpers verbatim.

  - Imported master's `call-tool` MCP-spec contract assertion: every
    successful tool call whose tool declares an `outputSchema` must
    return `:structuredContent` (guards the regression where Claude
    Desktop got 500s on outputSchema/structuredContent mismatches).

  - Removed the three obsolete cross-session subblocks I had rewritten
    as cross-user. Master deletes them entirely; the cross-USER
    boundary is covered by `visualize-query-refuses-cross-user-handle-
    test` (imported below).

  - Imported 11 missing master deftests verbatim, except for 2 that
    referenced the legacy `:source`/`:operations` request shape
    (`construct-query-returns-bare-handle-test`, `execute-query-
    resolves-via-cross-session-fallback-test`). Those 2 were adapted
    to our representations `:query` shape — semantic intent preserved.
    Full set:
      - origin-validation-test
      - tools-list-all-tools-declare-required-hints-test
      - text-content-includes-structured-content-for-maps-test
      - tool-result-emits-structured-content-test
      - tools-list-strict-shape-test
      - tools-expose-output-schema-test
      - construct-query-returns-bare-handle-test
      - visualize-query-resolves-via-cross-session-fallback-test
      - execute-query-resolves-via-cross-session-fallback-test
      - render-drill-through-resolves-via-cross-session-fallback-test
      - visualize-query-refuses-cross-user-handle-test

  - Updated the `visualize_query` "resolves the query stored with a
    construct_query handle" subtest to also assert the prompt round-
    trips through the handle — restoring the semantic master pinned
    via "includes the prompt stored with a construct_query handle".

  - Updated the construct_query description-pointer assertion to
    check for `construct-query.md` (the MCP resource URI both master
    and this branch publish) instead of the file-path string
    `construct_notebook_query`.

  - Fixed the `read-handle` call site in `tools-call-smoke-test` to
    pass the 3rd `user-id` argument master added when making handle
    lookup user-scoped.

* Construct_query API hardening (src/metabase/agent_api/api.clj):

  - Restored the optional `:prompt` field on `::construct-query-
    request` (with master's 10000-char max). The migration to
    representations had inadvertently dropped it, breaking the prompt
    round-trip the MCP layer relies on for `visualize_query` to
    surface the user's original message back to the iframe for
    feedback submission.

  - Wire endpoint now echoes `:prompt` to the response, so the MCP
    body-transform (`make-store-construct-query-result`) can store
    it with the handle.

  - Relaxed `:query` on the wire body from `::lib.schema/external-
    query` to plain `:map`. The deep MBQL-shape validation already
    runs inside the representations pipeline (`repair` → `validate-
    query` → `resolve`), so the boundary check was redundant.
    Importantly, this also dodges master's new
    `tools-manifest/assert-optional-fields-nullable!` lint, which
    walks every map reachable from a tool's input schema and trips
    on `::query`'s `:lib/metadata` / `:database` / `:settings` /
    `:parameters` optional-non-nullable fields (those are correct in
    lib.schema for other reasons; we just don't want them reachable
    from the agent-API tool surface). The outer `:closed true` is
    preserved, so the legacy-envelope-rejection test still passes.

  - Folded a ~40-line condensed inline description back into the
    `construct_query` tool metadata — workflow, universal clause
    shape, top-level / per-stage keys, a minimal worked example, and
    common pitfalls. Strict MCP clients (ChatGPT, Claude Desktop)
    that don't eagerly fetch resources now get format guidance in
    `tools/list` output directly. Full grammar still lives in the
    `metabase://docs/construct-query.md` resource.

* MCP override shape (src/metabase/mcp/tools.clj):

  - Updated `construct-query-mcp-input-malli` and `query-mcp-input-
    malli` from the legacy `{:source, :operations}` shape (no longer
    valid against the wire after the representations migration) to
    the new `{:query <opaque-map>}` shape. The override uses an
    opaque `:map` for `:query` so the manifest's
    `assert-optional-fields-nullable!` lint stays clean.

* Adjusted one wire-validation test on the construct-query endpoint
  (test/metabase/agent_api/api_test.clj): the "non-aggregation in
  aggregation slot" test now asserts the `:aggregation-entry-not-
  aggregation` agent-error code that the repair pipeline emits,
  matching the new boundary where deep validation moved from the
  malli schema into the pipeline. Same HTTP status (400), more
  specific diagnostic.

Test sweep across all touched surfaces (mcp.api-test, agent-api.api-
test, full metabot suite, mcp.session-test / tools-test / resources-
test, all agent_lib representations suites, mp-test, core.modules-
test): 446 tests / 6173 assertions, 0 failures, 0 errors. Master
deftest set in `mcp/api_test.clj` is now fully restored — `comm -23`
of master's deftest names vs HEAD = empty. The 1-line stray
`describe_table.clj` unicode-literal change is from a pre-merge
branch commit and is intentional.

* Strip internal schema names from agent-facing strings; fix construct_query doc pointer

* Fix doc-pointer regression. My earlier reconciliation commit changed
  the `construct_query` tool description to point at
  `metabase://docs/construct-query.md` — matching master's pointer
  string but pointing at content that on this branch still describes
  the legacy program format (`resources/metabase/agent_api/construct_query.md`
  was never rewritten for the representations migration). Restored the
  branch's original pointer at the `construct_notebook_query` tool
  prompt, which IS rewritten for the new format. Test assertion
  updated to match.

* Strip internal Clojure schema concepts from LLM-facing strings.
  The agent only needs to know "Metabase MBQL 5 query as JSON" — names
  like `::lib.schema/external-query` or "portable representations JSON
  format" are implementation jargon and don't help the model produce
  better output. Cleaned in:
   - `mcp/tools.clj` `:tool/description` strings for `construct_query`
     / `query` inputs.
   - `agent_api/api.clj` `:tool/description` on `::construct-query-
     request`'s `:query` field, the `construct_query` tool description,
     and the `/v2/query` tool description.
   - `resources/metabot/prompts/tools/construct_notebook_query.md`'s
     opening line (dropped the schema-name parenthetical).
  Source comments and internal Clojure docstrings keep their technical
  vocabulary — those aren't agent-facing.

* Trimmed the verbose code comment I'd added above the inline tool-
  description string. No "NOT this / NOT that" framing.

* Consolidate construct-query reference to one canonical doc

`resources/metabase/agent_api/construct_query.md` (MCP-served at URI
`metabase://docs/construct-query.md`) still described the legacy
program format. `resources/metabot/prompts/tools/construct_notebook_query.md`
is the file the metabot agent actually loads as its `construct_notebook_query`
tool prompt, and it's already rewritten for the representations format.
Two files, two consumers, drifting content.

Make `construct_notebook_query.md` the single source of truth:

* Point the MCP resource registration's `:render-fn` at the metabot
  prompt file (`metabot/prompts/tools/construct_notebook_query.md`).
* Delete the now-redundant `resources/metabase/agent_api/construct_query.md`
  (and its empty parent dir).
* Update the MCP resource `:description` to drop "Program syntax"
  framing; the file describes the representations format now.
* Mirror the description in `src/metabase/mcp/README.md`.
* Add a single `# Construct Query Reference` heading line to
  `construct_notebook_query.md` — benign for the metabot LLM
  (it's already reading the file as a tool-instruction blob)
  and matches the heading the MCP `resources/read` test asserts.

External MCP clients (ChatGPT, Claude Desktop) fetching the resource
URI now get the same representations-format content the metabot agent
reads internally. Tool descriptions continue to point at the
`construct_notebook_query` tool name; both pointers (the MCP URI and
the tool-prompt path) resolve to the same content.

* Close residual S1 info leaks; catch typo'd stage keys at the agent boundary

Post-merge review surfaced three follow-up items on the agent surface
this commit addresses.

* S1 follow-up — implicit-join and ambiguous-table errors still leaked
  un-sandboxed metadata. The original S1 fix stripped suggestions from
  `unknown-table-ex-info` / `unknown-field-ex` in `mp.clj`, but three
  agent-error paths kept walking the un-sandboxed metadata provider and
  surfacing the results verbatim via `:agent-error? true`:

   - `representations/repair.clj` `:ambiguous-fk` enumerated full
     portable `[db schema table field]` paths for every FK column on
     reachable bridge tables — a sandboxed user could learn the FK
     column names on any FK-reachable table just by hallucinating a
     field reference.
   - `representations/repair.clj` `:ambiguous-fk-via-join` did the same
     for FK columns on joined tables. Alias names (which came from the
     LLM's own `joins:` input) stay surfaced; only the column portables
     are stripped.
   - `mp.clj`'s `:ambiguous-table` echoed `(juxt :schema :name :id)` of
     every matching candidate.

  All three now echo only the rejected input + a prose hint pointing
  the LLM at `entity_details` for the disambiguating columns it has
  permission to see. Existing tests that asserted candidate presence
  were inverted to assert absence (parity with the existing
  `unknown-table-ex-info` / `unknown-field-ex` tests).

* Stage-key typo catch — `lib.schema.mbql-stage/mbql` is not a closed
  map, so a stage with a misspelled key (`aggreagation` for
  `aggregation`) was silently dropped at resolve / `lib.normalize`
  time and the LLM never learned its intent was discarded. The full
  `validate-external-query` walk against `::lib.schema/external-query`
  produces opaque `humanized` output (function-predicate references)
  and pre-empts E1's friendly diagnostic for the aggregation-slot
  case, so a targeted closed-keys check is preferable.

  Added `representations/assert-known-stage-keys!` — walks each
  stage's top-level keys and throws `:unknown-stage-key` with the
  offending key name and the canonical valid-key set in the message.
  Wired into `metabot.tools.construct/execute-representations-query`
  right after the JSON→portable conversion. `validate-external-query`
  is kept as a building block with an updated docstring documenting
  why the boundary now uses the targeted check instead.

* Stale "YAML" wording — the wire flipped to JSON months ago but
  several anchor docstrings still said "YAML". Replaced in:
  `representations/repair.clj` module docstring + `repair` fn
  docstring; `representations/resolve.clj` `export-query` docstring;
  `metabot/tools/construct.clj` `resolve-database-id-from-first-stage`
  docstring. Comments only — no behavior change.

---------

Co-authored-by: Mihael Konjevic <konjevic@gmail.com>
2026-05-22 21:04:55 +02:00
technomancy
9a0bb91109 Stop dropping old datasets from snowflake. (#74663)
* Stop dropping old datasets from snowflake.

There's a couple bugs here; first of all a race condition when
inserting datasets which can cause data to get triple-inserted. We
could improve locking around this, but it shouldn't be happening
anyway since datasets should only be removed when they haven't been
accessed in 5 days.

Local testing in the repl shows the correct behavior, but randomly in
CI sometimes it will decide to drop the datasets anyway. Have not been
able to determine why.

* Remove snowflake from "needs" section of drivers-tests-result.

We want to allow these to run in order to gain data, but we can't
afford to make them disrupt CI.
2026-05-22 11:47:44 -07:00
Nemanja Glumac
0410229822 Upgrade actions artifact (#74634)
* Update `actions/upload-artifact@v4` -> v7

* Upgrade `actions/download-artifact@v4` -> v8
2026-05-22 18:25:27 +00:00
Timothy S. Dean
62bccb2a18 Remove lib UUID from joins (#72790)
`:lib/uuid` keys were leaking into serialized MBQL for column references that use string column names. These UUIDs are non-portable.

This adds matcher clauses for the string-id cases of `:field`/`:metric`/`:segment`/`:measure`, so that `export-mbql-ref` walks them appropriately. UUIDs for aggregation reference targets are preserved.
2026-05-22 17:54:46 +00:00
Rodrigo López Dato
58fe24da85 Enable SMTP debug logs with mail.debug JVM property (#67519)
drewr/postal uses Jakarta Mail, and supports a debug option when sending
messages. However, it requires callers of the library to manually pass the
debug option, and doesn't look at any JVM options.

This change makes it so that setting the mail.debug JVM property to true
enables debug logging for all outgoing SMTP messages. No credentials are
logged.
2026-05-22 14:47:14 -03:00
Timothy S. Dean
1f95beef87 Fix inactive-tables filter on file metadata reimport (#74661)
Reimporting warehouse metadata could fail with a uniqueness constraint violation on `metabase_table`.

The code was matching existing IDs while filtering out inactive tables, which was wrong.

To reproduce, import the metadata, deactivate a table, and then reimport that metadata. A regression test is included.
2026-05-22 11:30:41 -06:00
bryan
6ec2fb2b56 Docs: backfill create_question + create_dashboard MCP tools (#74654) 2026-05-22 16:26:55 +00:00
appleby
3fa1bab551 BOT-1495 Trim all metabot tables to MB_AI_USAGE_MAX_RETENTION_DAYS (#74035)
* BOT-1495 Trim all metabot append-only tables to MB_AI_USAGE_MAX_RETENTION_DAYS

As of BOT-1409 we now trim the ai_usage_log table to prevent unbounded growth. The same arguments for trimming the
ai_usage_log table apply to other metabot append-only tables, like metabot_conversation, metabot_message,
metabot_feedback, and metabot_source_feedback.

- Update the fk on `metabot_message` pointing to `metabot_conversation.id` to `ON DELETE CASCADE`
- Add a trimmer task for `metabot_conversation` to also respect `MB_AI_USAGE_MAX_RETENTION_DAYS`.

Note that if we don't do this, older conversations will not have any source or profile_id associated, since these come
from the ai_usage_log table. This is not the only cause, but one possible cause of issues like BOT-1415

See also

* #73400
* #73518

* Shorten the gap between the ai_usage_log and metabot_conversation trimmer tasks
2026-05-22 09:18:13 -07:00
Alexander Polyankin
d8841313bf Allow custom source for ID params (#74650) 2026-05-22 16:03:15 +00:00
Ryan Laurie
4c551c71fb Metaplow Analytics for Internal Metabase Instance (#72380)
* wip metaplow analytics

# Conflicts:
#	frontend/src/metabase/utils/analytics-untyped.js

* payload fix

* update settings

* fix hostname

* update metaplow

* update mock type

* update types

* add unit tests

* Metaplow/backend events (#74516)

* feat: metabase.analytics.event

Raise track-event! to new namespace matching FE, in preparation to add a second event tracker.

* feat: metabase.analytics.metaplow

Add new analytics namespace to send events to metaplow.

Semantics are similar to snowplow:
- events are sent in up to 50 parallel workers sharing a HTTP connection pool
- event queue can have up to 10k pending events, after which new ones are dropped

Metaplow does not have a batch endpoint currently so that is not used.

* feat: metrics for metaplow errors

* rfct: delay metaplow resources
# Conflicts:
#	src/metabase/analytics/prometheus.clj

* return promise

---------

Co-authored-by: Filipe Silva <filipematossilva@gmail.com>
2026-05-22 10:45:07 -04:00
Nemanja Glumac
6c5473b913 Upgrade actions/github-script@v7 to its latest v9 (#74629) 2026-05-22 16:39:09 +02:00
metamben
8867b796e8 Handle query cache update race (#74587)
Closes #73770
2026-05-22 17:15:32 +03:00
Sébastien
5158913a1b Better copy for embed wizard resource selection for browser (#74638) 2026-05-22 15:27:04 +02:00
Oleksandr Yakushev
d5e910c3ad [perf] Optimize fingerprinters/mode-stats (#74327) 2026-05-22 16:18:18 +03:00
Oleh K
4d3ae02527 Fix adding rows to editable tables with a date column (#74574)
fix: allow adding rows to editable tables with a date column

The date input declared its props as Omit<..., "onChange"> and emitted
onEnter on pick instead of onChange, so a picked date never reached the
create/update form state. With a required (NOT NULL) DATE column the
submit button stayed disabled, blocking record creation (metabase#70647).

Emit onChange (modal form state) and onBlur (inline commit) on pick,
matching the boolean/dropdown selects, and close the calendar after a
pick. Adds an e2e regression covering record creation with a required
date column.
2026-05-22 15:01:27 +02:00
Sébastien
454aff6e7d Integration branch for embed wizard and guide UX improvements (#74497) 2026-05-22 14:10:36 +02:00
Oleh K
c4939f2ad6 Fix double vertical scrollbar in popovers with AccordionList (#74407)
* Fix double vertical scrollbar in filter dropdown

Make the accordion list the scroll container inside clause popovers so the
search bar can stay sticky, instead of the popover dropdown's content box
scrolling around it.

* Fix typo in comment
2026-05-22 13:47:20 +02:00
Romeo Van Snick
6b7b4e2eed Remove tables entity (#74575)
* Add tables redux thunks and retire-ready reducer scaffolding to prepare to remove it entirely

* Migrate simple table consumers off the Tables entity

* Migrate permissions table loading off the Tables entity

* Migrate Tables HOC loaders and list invalidation off the entity

* Remove tables entity

* Match noWrapper semantics

* Avoid mutating redux state in-place

* Avoid double-normalizing the tables data

* Await table metadata fetch

* Add unit test for tables-reducer

* Remove relyance on EntityName for tables

* Avoid forcing request

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>

---------

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
2026-05-22 13:38:47 +02:00
Sébastien
758bd2579b Fix /api/frontend-errors 404 in embedded contexts (#74623) 2026-05-22 11:21:17 +02:00
Oleh K
d96b990019 Rewrite JUnit hook failures so Trunk attributes them to the original test (#74344)
* Add JUnit hook-failure rewrite postprocessor

Adds e2e/support/fix-junit-hooks.ts, a Bun script that rewrites Mocha/Cypress
JUnit XML so failures from before/after hooks are attributed to the underlying
test instead of '<suite> "before each" hook for "<test>"'. This keeps Trunk's
flake tracking keyed on stable test names. Supports directory mode for
performance, --dry-run, and --include substring filtering for gradual rollout.

* Wire JUnit hook-failure rewrite into e2e workflows (dry-run)

* Log per-testcase rewrites under each file's summary line
2026-05-22 11:09:21 +02:00
Ngoc Khuat
e0502bb320 Use transform timeout for queries inside transforms (#72522)
* Use transform timeout for queries inside transforms

Previously, queries executed within a transform were subject to the lower
of MB_DB_QUERY_TIMEOUT_MINUTES (default 20 min) and MB_TRANSFORM_TIMEOUT
(default 4 h). This made MB_TRANSFORM_TIMEOUT effectively unusable for its
stated purpose: long-running transforms always hit the query timeout first.

Bind *query-timeout-ms* to the transform timeout inside
run-cancelable-transform! so transform queries respect MB_TRANSFORM_TIMEOUT,
while regular Metabase queries keep using MB_DB_QUERY_TIMEOUT_MINUTES.

Fixes GDGT-2173.

* Enforce query timeouts per-statement, not via pool leak-detector

The c3p0 unreturnedConnectionTimeout was doing double duty: leak
detection AND query-timeout enforcement. That pool-wide conflation
meant transforms running longer than db-query-timeout always got their
connection destroyed first, defeating the transform-timeout rebinding
introduced earlier in this branch.

This commit:
- Calls Statement.setQueryTimeout uniformly in prepared-statement*/
  statement* so every SQL-JDBC query carries its own server-side
  timeout derived from *query-timeout-ms*. Applied after the multimethod
  returns, so driver overrides that replace the base method also get it.
- Raises the pool's unreturnedConnectionTimeout default to
  max(db-query-timeout, transform-timeout) so the safety net does not
  undercut a legitimate long transform; per-statement timeouts remain
  the real query killer.
- Also rebinds *network-timeout-ms* inside run-cancelable-transform! so
  a single slow socket read does not kill a long transform earlier than
  its own deadline.
- Replaces the prior tautological test with two real ones: the pool
  setting default correctly maxes query-timeout and transform-timeout,
  and a real H2 statement created through statement-or-prepared-statement
  observes the rebound timeout via .getQueryTimeout.

GDGT-2173

* Revert "Enforce query timeouts per-statement, not via pool leak-detector"

This reverts commit 5e01c82f24.

* Reapply "Enforce query timeouts per-statement, not via pool leak-detector"

This reverts commit a9f294247f.

* Simplify: strip ticket refs from source, hoist transform-timeout let

* Stronger tests: proxy-based unit test + multi-driver sweep for setQueryTimeout

* Route transform-timeout into driver pool default via registry, not cross-module lookup

Replaces the requiring-resolve from driver/settings.clj into transforms with a
register-long-running-timeout-provider! registry that transforms self-populates.
Keeps the dependency direction correct (transforms -> driver) and silences the
:metabase/modules kondo check. Also drops ^:parallel from the set-statement-query-timeout!
unit test, which tripped the destructive-fn-in-parallel-test linter.

* Route transforms through a dedicated :transform connection pool

Replaces the cross-feature `register-long-running-timeout-provider!`
registry with a new `:transform` value on the existing `*connection-type*`
enum, so transforms reuse the pool-cache key `[db-id, connection-type]`
mechanism that already isolates `:write-data`. The `:default` pool's
`unreturnedConnectionTimeout` is now unchanged from pre-PR — only the
`:transform` pool sees the longer leak-detector window — addressing
reviewer concerns about widening the safety net for non-transform queries.

* resolve-connection-type: take connection-type as arg, drop dynamic-var read

Pure-function refactor — caller passes *connection-type* (or any value)
explicitly. Easier to test, no implicit dependency on the dynamic binding.

* :transform also merges write-data-details; restore catch Throwable

Transforms route through `with-write-connection` (query_impl.clj:30), so
they need write-data credentials when configured. My earlier refactor's
`with-transform-connection` rebind dropped those credentials by treating
`:transform` as a separate write-context from `:write-data`. Fixed by
having `effective-details` merge `:write-data-details` when *connection-type*
is either `:write-data` or `:transform` — the pool key still differs so
pool properties (unreturnedConnectionTimeout) can be tuned per workload.

Updated `transform-creates-write-pool-test` -> `transform-creates-transform-pool-test`
to assert the new `[db-id :transform]` cache key.

Restored `catch Throwable` in `set-statement-query-timeout!` to match the
original PR — Spark/Hive may throw `AbstractMethodError` (an `Error`,
not `Exception`); the previous PR's `catch Throwable` swallowed it.

* Gate Statement.setQueryTimeout behind :jdbc/set-query-timeout feature flag

SparkSQL's Hive JDBC driver appears to handle Statement.setQueryTimeout
by closing the Thrift transport on the server side; subsequent statement
close() then throws "Failed to close statement" with TTransportException:
Socket is closed by peer. The catch in set-statement-query-timeout!
silences the setQueryTimeout call itself, but the session is already
poisoned and any explicit-joins test errors on .close().

Adds :jdbc/set-query-timeout feature flag (defaults true on :sql-jdbc,
overridden false on :sparksql). set-statement-query-timeout! takes the
driver as an arg and skips the call entirely when the feature is off.

* Skip getQueryTimeout assertion for drivers that opt out of setQueryTimeout

Spark/Hive opts out via :jdbc/set-query-timeout=false, so getQueryTimeout()
returns the JDBC default (0), not the bound *query-timeout-ms*. Branch
the assertion: drivers that support setQueryTimeout still round-trip;
opt-outs only assert a Statement was created.

* Clarify :transform connection-type docstrings per review

The with-transform-connection docstring claimed :transform shares primary
:details with :default, but effective-details merges :write-data-details for
:transform just like :write-data. Reconcile *connection-type*,
with-transform-connection, and resolve-connection-type docstrings to that truth:
:transform resolves details like :write-data but routes through a separate pool
key so c3p0 can carry transform-tolerant properties.
2026-05-22 08:07:32 +00:00
Nick Fitzpatrick
0ff0bc8dcc Add documents to search page (#74595) 2026-05-22 11:59:29 +05:30
Nick Fitzpatrick
2c58286958 Check for active subscriptions when removing parameters (#74591)
* Check for active subscriptions when removing parameters

* invalidate subscriptions on dashboard edit
2026-05-22 11:59:01 +05:30
Alexander Polyankin
1c4cec1540 Pass label_field for parameter card dropdown sources (#74597) 2026-05-22 02:29:36 +00:00
metamben
64d2e589b2 Fix sandbox removed field references (#74537)
Closes QUE2-589
2026-05-22 02:11:40 +03:00
thackervishal
ead94bf562 docs(collections): add updates for missing or undocumented patterns (#74287)
* docs(collections): add updates for missing or undocumented patterns that could lead to customer confusion

1. Curate vs. View relationship was implicit. The View access section didn't
   mention that Curate access is a superset of View. Admins setting up groups
   had to infer this. Added a note to make it explicit.

2. The "Also change sub-collections" toggle was undocumented. The doc said
   granting access to a collection doesn't cascade to sub-collections, but
   never explained how to fix that or what the toggle actually does. Added
   the missing explanation, including that "existing" scopes the toggle to
   sub-collections present at the time permissions are saved — not future ones.

3. The mixed-access edge case was unaddressed. New sub-collections automatically
   inherit their parent's permissions at creation time, regardless of whether
   the toggle was used. This means skipping the toggle can leave a group with
   access to some sub-collections (newly created ones) but not others (existing
   ones that predate the permission grant). Without knowing this, admins have
   no way to diagnose the inconsistency. Added a sentence naming this state and
   implying the fix.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* Update docs/permissions/collections.md

Co-authored-by: Alex Yarosh <alexandra@metabase.com>

---------

Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-authored-by: Alex Yarosh <alexandra@metabase.com>
2026-05-21 20:07:49 +00:00
Aleksandr Lesnenko
0bf62315d3 Workspaces V2 (#73236)
* More perf work

* MySQL workspace fixes: bound-DB preservation + schema-filter for no-schema drivers

Two real bugs surfaced while making the workspace e2e green on MySQL.
Both are general workspace-driver bugs, not test-only adaptations.

1. init-workspace-isolation! :mysql was returning :database_details
   {:user, :password, :db isolation-db}. The workspace config-loader
   merges :database_details over the canonical :details when wiring
   up the workspace user's connection, so the iso DB clobbered the
   canonical :db. Result: connection bound to an empty isolation
   database, sync enumerates zero tables, the workspace can't see
   its own input. Fixed by dropping :db from :database_details
   (matches Postgres' shape — only identity, never location). The
   output DB lives on the WSD row's :output_schema; the rewriter
   prepends it to compiled writes.

2. ws.config/database-entry was unconditionally emitting
   :schema-filters-type "inclusion" + :schema-filters-patterns
   joined input_schemas. On no-schema engines (MySQL,
   ClickHouse-as-1-DB) JDBC reports TABLE_SCHEM as null for every
   row, and metabase.driver.sync/schema-patterns->filter-fn*
   documents that inclusion patterns never match a nil schema.
   Sync silently produced zero Tables. Fixed by gating the
   schema-filter emission on (driver/database-supports? engine
   :schemas db). On schema-having drivers behavior is unchanged.

E2e test (workspace-full-e2e-test):

- Removed three (is (= :fail …)) debug canary stubs that always
  failed — they predated MySQL work and were masking the real
  problems on every driver. Removed the now-unused ordered->plain
  helper and clojure.walk import.
- Added canonical-schema-name / table-row-schema-value /
  create-canonical-schema! / drop-canonical-schema! helpers so the
  test handles MySQL's "schemas are databases" model: reuse the
  bound DB instead of CREATE SCHEMA, expect :model/Table.schema=nil,
  and tear down only the per-run tables instead of the shared DB.
- Added stage-by-stage diagnostic assertions (warehouse table is
  queryable; WSD reaches :provisioned with non-empty :output_schema;
  ws/workspace-mode? after initialize!; loader rewrote :details;
  sync produced rows). Failures now report what step broke instead
  of crashing 100 lines later as (some? nil).
- Added a regression guard: (:db admin-details) must equal
  (:db (:details ws-db)) post-initialize!. Catches any future
  driver that puts :db into :database_details.
- Added a MySQL-aware "no isolation leak" assertion that probes
  information_schema.tables instead of filtering by Table.schema
  (which is always nil on MySQL).

Postgres regression: 22 assertions, 0 failures, 0 errors.
MySQL still has one open issue with the unique-constraint on
:model/Table inserts during transform execute — separate commit.

* Fix unique-constraint collision in transform target sync on no-schema engines

Symptom: on MySQL, transforms.execute/execute! threw a duplicate-key
H2 exception inserting :model/Table:

  Unique index violation: METABASE_TABLE(DB_ID NULLS FIRST,
  NAME NULLS FIRST, UNIQUE_TABLE_HELPER NULLS FIRST)
  VALUES (2, 'x_output_table_…', '')

The collision had two rows in flight:

  row A — synced from the warehouse, :schema nil
          unique_table_helper = COALESCE(nil, '') = ''
  row B — being inserted by sync-table! after the transform run,
          :schema "" (empty-string sentinel from canonicalize-target)
          unique_table_helper = COALESCE('', '')   = ''

Same key, two rows, INSERT rejected.

Cause: canonicalize-target inverts the workspace remap by calling
ws.table-remapping/canonical-schema+name, which returns the storage
row's from_schema. TableRemapping uses "" as an empty-slot sentinel
(so the unique constraint on the row stays enforceable for engines
without schemas). :model/Table.schema instead uses nil for those
engines (that's what JDBC's TABLE_SCHEM reports and what sync writes).
The two formats matter: a select :schema "" doesn't match a row
stored as :schema nil, so target-table missed the existing row and
sync-table! fell through to create-table!, where the unique-helper
COALESCE collapsed both back together.

Fix: in canonicalize-target, denormalize "" → nil for engines where
(driver/database-supports? engine :schemas db) is false. The Table
row format is restored before target-table looks it up. Targeted
change: only crosses the storage→Table boundary for no-schema engines.

Test: add a stage diagnostic that selects :model/Field for the post-
transform output table. If transform-time sync didn't populate
columns, the assertion fires here instead of crashing 100 lines later
in the QP's add-implicit-clauses with "Table X has no Fields".

Also adjusted the e2e to expect the per-driver shape:

- TableRemapping.from_schema is "" on MySQL (zero-slot driver, per
  spec-for-table) and main-schema name on Postgres-family.
- describe-database returns :schema nil on MySQL (JDBC TABLE_SCHEM is
  null for every row); use tbl-schema and probe via
  information_schema.tables on MySQL to assert no iso-DB leak.

Postgres regression: 23 assertions, 0 failures, 0 errors.

MySQL still has one open issue: post-transform sync-fields-for-table!
does not populate Field rows. Root cause is a separate gap — the
workspace QP-side remap (workspace-remap-schema+name) redirects sync
to the iso DB by filling :schema "mb__isolation_…", but MySQL's
qualified-name-components is [] so the SQL compiler emits a bare
identifier and MySQL's describe-fields TABLE_SCHEM filter on a
non-null value matches nothing (TABLE_SCHEM is always null on MySQL).
Resolving that means widening MySQL's qualified-name-components and
the workspace remap lookups to include :db — separate commit.

* more perf experiments

* Symmetric map-shape API for workspace-remap hooks

Change the workspace remap defenterprise hooks from positional
2-tuple returns to a symmetric `{:db :schema :name}` map shape on both
input and output:

  workspace-remap-schema+name : (db-id, from-spec) -> to-spec | nil
  canonical-schema+name       : (db-id, to-spec)   -> from-spec | nil

Three motivations:

1. Vector tuples with unnamed slots are easy to misalign as the
   identifier shape grows. The H7 second half (cross-DB workspaces on
   Snowflake / BigQuery) widens both sides to carry `:db` end-to-end;
   anchoring on a named-slot map now means future widening is just a
   key on the map rather than a length change in a positional tuple.

2. The hooks were each returning the storage row's slot values
   verbatim, which use `""` as the empty-slot sentinel. Consumers had
   to denormalize `""` back to `nil` themselves before using the
   result as a `:model/Table`-row predicate (a select on `:schema ""`
   doesn't match a row stored as `:schema nil`). The previous commit
   added that translation at one call site (canonicalize-target).
   This commit moves the translation into the hook itself via a new
   `store-tuple->table-spec` helper, so consumers receive values
   already in `:model/Table`-row vocabulary.

3. Symmetric shape on both sides of the boundary means the same map
   value can be passed forward through the pipeline without rebuilding
   it at each step.

Call sites updated:

- metabase.sync.fetch-metadata/effective-schema+name now wraps the
  hook in a map-build / destructure pair while keeping its
  `[schema name]` 2-tuple return for backward-compat with existing
  positional destructuring.

- metabase.transforms-base.util/canonicalize-target now takes the map
  directly and merges it into the target. The `""→nil` translation
  it carried in the previous commit is removed — the hook handles it
  now, so the consumer is back to a one-liner.

OSS shim docstrings updated to reflect the new shape.

Postgres regression: 23 assertions, 0 failures, 0 errors.
MySQL e2e unchanged from previous commit (still 1 failure and 1
error from the qualified-name-components MySQL design gap that
follow-up commits will address).

* MySQL workspace: thread iso DB through transform write path

MySQL's `qualified-name-components` was `[]` — meaning workspace remap
had nowhere to record the iso DB name. Compiled SQL was unqualified
and resolved against the connection's bound DB, which is canonical.
Transform writes ended up trying to CREATE TABLE in the canonical DB
(the only DB the SQL identifier could land in), where the workspace
user has only SELECT — so the write failed with a permissions error.

Architectural change: MySQL's `qualified-name-components` -> `[:db]`.
This is purely metadata — MySQL's compiled SELECT SQL is unchanged
because the SQL compiler doesn't read this multimethod, only workspace
remap does. The slot tells consumers "MySQL has a meaningful database
identifier above the table" so workspace remap can record canonical
and iso DB names on `TableRemapping` rows.

Cascade of changes that follow from the slot widening:

* `config.clj/table-namespace`: per-driver routing of the `ns-name`
  parameter to either `:db` or `:schema` based on the driver's
  `qualified-name-components`. MySQL's iso DB name now lives in
  `:db` on the wire format, not `:schema`.

* `transform_hooks.clj/resolve-transform-target`: replace ALL slots
  on the target — `:db`, `:schema`, `:name` — with the to-spec's
  values, instead of only assoc'ing populated ones. The canonical
  target arrives carrying the input namespace in `:schema`; failing
  to clear it leaves the SQL compiler with two competing qualifiers
  and the output lands in the wrong place. Storage's empty-string
  sentinels are denormalized to nil at this boundary.

* `transforms_base/query.clj`: thread `:output-db (:db target)`
  through `transform-details` so downstream compile sites can pick
  up the cross-DB qualifier.

* `driver/sql/query_processor.clj/compile-transform :sql`: when
  `:output-db` is populated and `:output-table` doesn't already
  carry a namespace, emit `(keyword output-db (name output-table))`
  so HoneySQL renders ``iso`.`tbl`` for the CTAS. 3-part writes
  (Snowflake/SQL Server cross-DB) aren't expressible through this
  single-keyword shape; will be addressed when those workspaces land.

* `db-position-value` in both `config.clj` and `table_remapping.clj`
  now includes `:mysql` -> `(:db (:details db))`.

Postgres regression: 23 assertions, 0 failures, 0 errors.
MySQL e2e progresses from "transform write throws permission denied
mid-execute" to "transform write succeeds, downstream sync/read
issues remain". Specifically still open:

  1. Sync's `filter-workspace-side-tables` hook isn't dropping iso-DB
     tables on MySQL (no schema dimension for those rows). They show
     up as :model/Table rows under the canonical Database.

  2. Card MBQL queries against the canonical Table read the iso TABLE
     NAME but with the canonical DB qualifier — `test-data.__x_output_table_X`
     doesn't exist (the iso name lives in the iso DB).

Both follow-ups likely live in the workspace QP middleware where
identifier matching needs to widen to the `:db` slot now that MySQL
populates it. Separate commits.

* Storage layer owns "" <-> nil translation for table remapping

The TableRemapping store uses `""` as an empty-slot sentinel so the
SQL unique constraint `(database_id, db, schema, table_name)` stays
enforceable on engines where NULL is not-equal-to-NULL in indexes.
That's a storage-layer concern. App code wants `nil`.

The read path already played this rule: `store-tuple->table-spec`
denormalizes 3-tuples coming out of the store, and the defenterprise
hooks (`workspace-remap-schema+name`, `canonical-schema+name`) return
denormalized maps. The write path's return value didn't — it handed
callers the storage-shaped `::table-spec` (with `""` sentinels), so
each caller had to denormalize itself before using.

Today the only caller (`transform_hooks/resolve-transform-target`)
had an inline `denorm` lambda doing exactly the same job as the
storage-internal `denormalize-level`. Two translation sites for the
same concern.

Fix: introduce a single `spec->table-spec` projection at the storage
boundary (mirror of `store-tuple->table-spec` for in-memory specs)
and have `add-transform-target-mapping!` return through it. Drop the
inline `denorm` from `transform_hooks` — the hook now `assoc`s the
values directly because they arrive already denormalized.

Net: storage's `""` sentinel never crosses the API boundary upward,
in either direction. Callers see `nil` semantics consistently.

Postgres regression: 23 assertions, 0 failures, 0 errors.
MySQL e2e unchanged shape (18 / 1 / 1) — refactor is behavior-
preserving; remaining MySQL issues are downstream of `:db`-aware
identifier matching in the workspace QP middleware (sync's filter
hooks and the SQLGlot rewriter), separate commits.

* MySQL workspace: scope sync to bound catalog; fix cross-DB lookup

Two principled fixes that follow from MySQL's `qualified-name-components`
being `[:db]`:

1. Sync was leaking iso-DB tables into `:model/Table`
   ----------------------------------------------------
   `active-tables :mysql` was passing `nil` catalog to
   `post-filtered-active-tables`, which makes JDBC's `getTables`
   enumerate every DB the user has privileges on. Workspace MySQL
   users own their iso DB AND have GRANT SELECT on canonical, so
   describe-database surfaced both. Pin the catalog to
   `(.getCatalog conn)` (= the connection's bound DB) so sync only
   sees the canonical input. Strictly more correct for non-workspace
   MySQL too — a user with grants on multiple DBs only configured
   one as the Metabase Database.

2. `canonical-schema+name` lookup was missing on no-schema engines
   ---------------------------------------------------------------
   The hook was passing the input `:schema` (now `nil`-shaped after
   the storage-boundary denormalization commit) directly into the
   index lookup, which stores `""` for absent slots. Result: lookup
   missed for MySQL. Normalize input slots through `normalize-level`
   on the way IN (mirror of how we denormalize on the way OUT) so
   `nil → ""` and the index match succeeds.

MySQL e2e: 18 assertions, 0 failures, 1 error (up from 18/1/1).

The remaining error is the QP read path's Phase 1/Phase 2 coherence
gap on cross-DB workspaces — `:metadata/table` doesn't carry `:db`,
so HoneySQL emits bare identifiers under the canonical bound DB. The
proper fix is structural (carry `:db` on metadata/table, have
HoneySQL read it for engines whose qualified-name-components includes
`:db`); separate commit.

Postgres regression: 23 assertions, 0 failures, 0 errors.

* Centralize "" sentinel coercion at the model boundary

The columns from_db/from_schema/to_db/to_schema on table_remapping use
"" as a storage sentinel meaning "this driver doesn't emit this AST
level" — needed because Postgres/MySQL/H2 treat NULL as not-equal-to-
NULL in unique indexes, so the constraint
(database_id, from_db, from_schema, from_table_name) requires non-null
values to enforce uniqueness for rows with absent slots.

Until now, every writer sprinkled `normalize-level` (nil → "") at
the call site. The middleware also redefined `no-level`, had its own
`prune-no-level`, and used an inline triple-state `norm` predicate
to compare "" / nil / present.

Cleanup, in three pieces:

1. **Model hook owns the coercion** for AppDB writes and lookups.
   `before-insert` and `before-update` fill the four sentinel columns
   with "" if missing or nil. `before-select` translates explicitly-
   passed nil (but doesn't add absent keys — that would over-filter
   broad lookups like (t2/select :model/TableRemapping :database_id 7)).
   App code can pass nil for absent slots:

     (t2/insert! :model/TableRemapping {:database_id 7 :from_table_name "ft" :to_table_name "tt"})
     (t2/select-one :model/TableRemapping :database_id 7 :from_db nil :from_table_name "ft")

   The MapStore impl still needs ""-shaped tuples (raw equality lookup),
   so the explicit `normalize-level` in `spec->tuple` and friends
   stays. The model hook is defense in depth for direct-Toucan callers.

2. **`normalize-level` / `denormalize-level` / `prune-no-level`
   promoted to public** in `table_remapping.clj`. The middleware now
   imports them instead of redefining `no-level` + `prune-no-level`
   locally. One sentinel definition, one place.

3. **`remap-mbql-table-metadata!` uses `denormalize-level`** for the
   nil/"" equivalence comparison instead of the inline triple-state
   `norm` lambda. Fewer sentinel-handling shapes in the codebase.

Postgres regression: 23 assertions, 0 failures, 0 errors.
MySQL e2e: 18 / 0 / 1 — same shape as before this commit.

* Tighten table_remapping model: collapse coerce fns + trim docstring

Two coerce fns differed only in their missing-key branch — collapse
into one with a `fill?` flag. Inline the cache-invalidate body too.
Trim the namespace docstring to the load-bearing facts.

Net: 87 lines removed, 33 added. Same behavior; Postgres 23/0/0.

* Unify around table-spec as the schema for table remapping data

* Another todo

* Another todo

* More TODOs

* Revert test driver filter

* MySQL workspace e2e green: thread :db through metadata + translate to SQLGlot vocab

MySQL e2e now passes 23 assertions, 0 failures, 0 errors. Postgres
unchanged at the same numbers. Three substantive changes plus a small
dev tool.

1. `:metadata/table` carries `:db`
   ----------------------------------
   Schema gets an optional `:db` slot (cross-DB references — BigQuery
   `project.dataset.table`, SQL Server / Snowflake `db.schema.table`,
   MySQL workspace tables routing to a different database). Sync doesn't
   populate it on standard reads; only workspace remap fills it. The
   `[:sql :metadata/table]` ->honeysql handler in
   `driver/sql/query_processor.clj` now reads `:db` and threads it into
   `h2x/identifier`, which drops nil components — so absent-`:db` produces
   the same `schema.table` shape as before.

2. Phase 1 metadata override now writes `:db`
   --------------------------------------------
   `remap-mbql-table-metadata!` previously overrode `:schema` and `:name`
   only. Now also overrides `:db`. Cross-DB workspaces (currently MySQL —
   iso namespace lives in `:db`) routable through Phase 1 without needing
   Phase 2's SQLGlot rewriter to insert a missing qualifier.

3. SQLGlot replacement key uses SQLGlot vocab
   --------------------------------------------
   The empirical bug: SQLGlot's `replace-names` matcher uses `:schema`
   for the leftmost qualifier in `db.tbl` and `:catalog` for the
   outermost in `catalog.db.tbl`. Our codebase's `::table-spec` uses
   `:db` ↔ SQLGlot `Table.catalog` and `:schema` ↔ SQLGlot `Table.db` —
   different vocab. `build-table-replacements` was passing our slot
   names straight through, so on MySQL the `{:db "test-data" :table
   "x"}` key never matched ``\`test-data\`.\`x\``. New
   `table-spec->sqlglot-key` translates at the boundary:

     - 1-slot drivers (MySQL `[:db]`, ClickHouse `[:schema]`):
       collapse to SQLGlot's `:schema`.
     - 2-slot 3-part drivers (Snowflake/SQL Server/BigQuery `[:db :schema]`):
       our `:db` -> SQLGlot's `:catalog`, our `:schema` -> SQLGlot's `:schema`.
     - Postgres-family `[:schema]`: unchanged (our `:schema` ↔ SQLGlot's `:schema`).

   Probed empirically with `metabase.sql-tools.core/replace-names` to
   confirm the slot expectation before writing the translation:

     {:catalog "x" :table "t"} - filters by table only, replaces "t" but
                                  leaves catalog alone.
     {:schema  "x" :table "t"} - matches `x.t`, replaces both.
     {:db      "x" :table "t"} - no match.

4. `dev/src/dev/spy.clj` — tiny dev-only file-based println debugger
   ------------------------------------------------------------------
   `(require '[dev.spy :as spy])` then `(spy/! x)` writes
   `file:line  (form) => <pr-str x>` to `/tmp/spy.log`. Macro form
   captures the source expression via `&form`. `(spy/stack!)` dumps a
   filtered call stack. Loaded lazily via `requiring-resolve` from
   non-dev code so production paths don't pay anything. Used in this
   session to find the SQLGlot vocab mismatch — leaving in tree as
   sharpening tool.

Lifted from this debugging session: the `db.schema.tbl` AST naming
convention is locally consistent but globally divergent from SQLGlot's,
and the boundary is `build-table-replacements`. Worth a follow-up to
make the slot vocab self-documenting (e.g. rename storage's `:db` to
`:catalog` and `:schema` keeps its name, matching SQLGlot exactly). For
now, translate at the boundary.

* dev: add #d debug-print reader (dev.debug)

`#d` is a dev-time tagged literal that logs the source form, line, and value
of any expression to /tmp/<dir>_debug.log, then returns the value unchanged.
Sibling to hashp's `#p`; meant for cases where a file-based log beats
stderr-with-color: cross-thread debugging, test-runner subprocesses, and
agent-driven debugging.

Differences from `#p`:

  - Always writes to a file (per-process default per JVM cwd, so concurrent
    worktrees don't stomp). Override via DEV_DEBUG_LOG env var or
    `(set-file! ...)` / `#d [:set-file! ...]`.
  - Real source line numbers. Hashp reads `:line` from the runtime JVM stack
    trace, which points at the compiled `let` it generates instead of your
    source line. We capture `(meta form)` at READ time so the logged
    location matches what's in the editor.
  - One-line entries via `pr-str` so output stays grep/tail-able. No
    pretty-printing baked in (pipe through `bat -l edn` or similar if you
    want it pretty).
  - Nested `#d` calls are stripped from the logged form-text via
    `strip-inner` walk. The outer entry shows the original source, not the
    inner expansion goo. Mirrors the trick `weavejester/hashp` uses.

Verbs (`#d` with a keyword/vector instead of an expression):

  #d :clear                     ; truncate the log + write a marker line
  #d :where                     ; print the resolved log path
  #d :stack / :stacktrace       ; spit a filtered call stack
  #d [:set-file! \"/tmp/foo\"]    ; redirect output

A bare keyword that matches a known verb is treated as the verb, not as a
value to log. Unknown keywords (e.g. `#d :user-event`) fall through to the
expression branch and log the keyword as a value.

Files:
  dev/src/dev/debug.clj       — the implementation
  dev/src/data_readers.clj    — registers `{d dev.debug/d}` (dev classpath only)
  dev/src/user.clj            — docstring noting both `#p` (hashp) and `#d`

Removed dev/src/dev/spy.clj (the prior name; renamed during iteration).

Tested against:
  - Three-line probe file: each `#d` reports its actual source line.
  - Recursive AVL invariant traversal with nested `#d`: outer entry shows
    `(if (nil? n) 0 (let ...))` cleanly, inner entries show
    `(invariant-ok? (:l n))` etc — no expansion goo leaks across levels.
  - LeetCode dungeon-game DP: `#d` inside `doseq` body, lines 18 and 24 of
    the source correctly distinguished.

* dev.debug: trim noise from #d log lines

- Drop thread-tag from log + stack output. nREPL session names
  (`nREPL-session-<uuid>`) added ~60 chars of identical noise per
  line.
- Strip JVM cwd prefix from logged file path so entries show
  `dev/src/dev/foo.clj` instead of the full absolute path.
- Drop the `:column` suffix; line number alone is enough to find
  the form in an editor.

Worst-case line in a real session: 343 -> 242 chars (~30%).

* permissions

* fix

* Fix compilation errors related to serialization/write-databases-metadata!

* Rewrite native transform SQL through workspace remap

Native transforms bypass the QP middleware pipeline -- their compiled SQL is
handed straight to driver/run-transform!, so Phase 2's apply-workspace-sql-remapping
never runs. A native transform that references a canonical table managed by
another workspace transform fails on the warehouse with "relation does not exist".

Add a transforms-base.workspace-hooks/rewrite-native-sql-for-workspace defenterprise
hook called from transforms-base.query right after compile-source. EE impl reuses
ws.table-remapping/rewrite-sql -- the same primitive Phase 2 uses -- so both paths
emit identical isolation refs.

Move rewrite-sql / build-table-replacements / table-spec->sqlglot-key from
ws.query-processor.middleware to ws.table-remapping (alongside ::table-spec, the
shape they consume) and make them public. Middleware switches to the moved fns.

* e2e: native transform referencing prior MBQL transform's canonical output

Repro for the bug fixed in 680d88ff53. Confirms the workspace native-SQL
rewriter substitutes a canonical table reference inside Transform B's SQL
with the iso-schema name where Transform A actually wrote.

Verified the test fails without the fix (with-redefs on the OSS stub):
  "Error executing raw queries: ERROR: relation
   \"canonical_schema_X.t_a_X\" does not exist"

Postgres-only for now; widen to other workspace-supported drivers once we
have repro coverage.

* e2e: extend native-transform repro test to MySQL

native-transform-references-prior-canonical-output-test now runs on both
:postgres and :mysql. Cross-driver storage shape differences for the
TableRemapping row required two adjustments:

- :from_schema lookup keyword is the empty-string sentinel on MySQL (no
  schema layer) and the canonical schema name on schema-having drivers.
- B's iso table location is `to_db.to_table` on MySQL (iso lives in a
  different DATABASE) versus `to_schema.to_table` on schema-having drivers.

Verified passing on both drivers (4/4 assertions each).

* e2e: drive cross-driver storage shape through spec-for-table

Replace ad-hoc per-driver branching in the native-transform repro test with
ws.table-remapping/spec-for-table -- the canonical translator that maps a
(Database, Table) pair to the storage `(db, schema, table)` shape for that
driver. New drivers added to this deftest's `mt/test-drivers` set will work
without further per-driver branches in the test.

Also use prune-no-level to pick the populated iso namespace slot (schema for
schema-having drivers, db for MySQL) instead of an `if = :mysql` branch.

* add admin connection ui

* tests

* make admin connection use workspaces feature flag

* fix

* m. keep-me

* add todo

* workspaces: rename input_schemas -> input, switch to ::table-namespace shape

Storage, API, wire format, and driver provisioning all now use the same
`{:db ?, :schema ?}` namespace map (Malli `::table-namespace`) instead of
the legacy `[String]` schema-name list.

Why: Snowflake/SQL Server/BigQuery (3-slot drivers) need `:db` to write
`GRANT ... ON SCHEMA <db>.<schema>`. The wire format already used the new
shape (per the 2026-05-04 YAML decision); the model column was the last
thing carrying the legacy `[String]`. Hard cutover (branch hasn't shipped).
Surfaced by the first Snowflake run of `workspace-full-e2e-test`, which
died in `grant-workspace-read-access! :snowflake` on a nil `:name` -> Malli
empty-components rejection.

Changes:

- New migration `062/20260508_workspace_input_rename.yaml` renames the
  JSON column `workspace_database.input_schemas` -> `input`.
- Model `:input_schemas` -> `:input` (deftransform key); `:input_schemas`
  guard in `reject-active-modification!` now compares `:input`.
- API contract changes (FE follow-up needed):
  - POST `/api/ee/workspace-manager/:id/database` body:
    `{database_id, input_schemas: [String]}`
    -> `{database_id, input: [{db?, schema?}]}`.
  - PUT  `/api/ee/workspace-manager/:id/database/:db-id` body:
    `{input_schemas: [String]}` -> `{input: [{db?, schema?}]}`.
  - GET  `/api/ee/workspace-manager/:id` + GET `/api/ee/workspace-manager/`:
    response field `input_schemas` -> `input`.
- Driver `grant-workspace-read-access!` impls now receive
  `[{:db ?, :schema ?}]` consistently. Snowflake / SQL Server / ClickHouse
  / H2 / BigQuery rewritten to do per-namespace (schema-wide / dataset-wide)
  grants — per-table `:name` reads removed everywhere. Postgres / Redshift
  / MySQL: param renamed from `tables` to `input` and helpers updated to
  read the right slot per driver (`:schema` for Postgres-family, `:db` for
  MySQL).
- Snowflake schema grants now emit `GRANT USAGE ON DATABASE` +
  `GRANT USAGE ON SCHEMA` + `GRANT SELECT ON ALL TABLES` +
  `GRANT SELECT ON FUTURE TABLES` per input namespace.
- BigQuery grants moved from per-table `ws-grant-table-read-access!` to
  dataset-level `ws-grant-dataset-acl!` with `roles/bigquery.dataViewer`
  to match Postgres semantics.
- `check-isolation-permissions` callers in `sql_jdbc.clj`, `mysql.clj`,
  `bigquery_cloud_sdk.clj` now hand the namespace shape (single-element
  vector) to `grant-workspace-read-access!`.
- Stale docstring on `driver/qualified-name-components` updated:
  Snowflake / SQL Server / BigQuery are 3-level (`[:db :schema]`),
  not 2-level. Per-driver `defmethod`s were already correct.
- Tests updated: `e2e_test.clj` adds `workspace-input-namespace` helper
  for driver-aware shape; `core_test`, `config_test`, `manager_test`,
  `models/workspace_test`, `models/workspace_database_test`, plus
  `test/util.clj` `with-temp` defaults all use the new shape.
- `dev/src/user.clj` requires `[dev.debug]` so the `#d` reader macro
  resolves at REPL/test JVM bootstrap (data_readers.clj registers the
  symbol but does not auto-require the namespace).

Verification: `./bin/test-agent --drivers=snowflake :only
'[metabase-enterprise.workspaces.e2e-test/workspace-full-e2e-test]'`
returns `23 assertions, 0 failures, 0 errors` in ~57s on the CI
Snowflake account. Postgres + MySQL e2e remain green.

* Split merge-fields by FK status; bundle smaller perf fixes

* sorry cam

* e2e: document redshift skip on native-transform repro

native-transform-references-prior-canonical-output-test stays at
{:postgres :mysql}. Redshift currently fails the underlying
workspace-full-e2e-test because describe-database returns {:tables #{}}
for the workspace user despite USAGE/SELECT being granted (verified
directly through the same connection pool). The native-transform repro
depends on sync surfacing the source table, so it inherits that failure
mode. Suspect snapshot visibility around redshift's catalog
has_table_privilege / has_schema_privilege functions vs. transactional
grants; investigation continues.

* iteration 4.1

* Rewritten migration.

* workspaces e2e: green on Redshift

Three fixes to make `workspace-full-e2e-test` pass on Redshift:

1. **Schema-wide grants in `grant-workspace-read-access! :redshift`.**
   The original per-table loop (commit `3da411a24e2`) read `(:name table)`
   from caller-supplied maps, but the workspaces caller passes per-namespace
   maps `[{:schema s}]` with no `:name` — so the loop was broken on arrival,
   never exercised because Redshift workspace e2e was never run before.
   Replaced with `GRANT SELECT ON ALL TABLES IN SCHEMA + ALTER DEFAULT
   PRIVILEGES IN SCHEMA GRANT SELECT ON TABLES` per source schema, matching
   the Postgres impl's documented "workspace-scoped users receive schema-wide
   SELECT" semantics.

2. **Bypass the test-data describe-database override.**
   `modules/drivers/redshift/test/metabase/test/data/redshift.clj:318` filters
   `describe-database*` to only return tables matching the dataset-name
   prefix pattern. Workspace e2e creates `x_input_table_<run-id>` /
   `x_output_table_<run-id>` which don't conform — the filter silently
   strips them. Added `with-redshift-describe-filter-disabled` test helper
   that unbinds `*override-describe-database-to-filter-by-db-name?*` for the
   duration of the workspace e2e. No-op for non-Redshift drivers.

3. **Cleanup fallback when `ws/delete-workspace!` throws.**
   Redshift's `destroy-workspace-isolation!` rolls the WSD status back to
   `:provisioned` if any cleanup statement fails (`deprovision-workspace-database!`'s
   catch at provisioning.clj:117-120). The outer `delete-workspace!` then
   re-checks status and refuses with 409, blocking `mt/with-temp Database`
   cleanup. Test cleanup catches the throw, force-clears WSD status to
   `:unprovisioned`, and explicitly deletes the Workspace row. The underlying
   Redshift destroy fragility is tracked separately.

Verification: `./bin/test-agent --drivers=redshift :only
'[metabase-enterprise.workspaces.e2e-test/workspace-full-e2e-test]'`
returns `23 assertions, 0 failures, 0 errors` on the CI Redshift cluster.
Snowflake + Postgres + MySQL e2e remain green (helper is no-op for them,
and the Redshift `grant!` rewrite doesn't touch other drivers).

* iteration 4.3

* Iteration 4.4

* Iteration 4.5

* add todo about redshift

* fix metadata endpoint http verb

* Iteration 4.6

* Iteration 4.7

* Iteration 4.8

* Perf fix

* UI wip

* wip UI

* wip UI

* wip UI

* Disable auto-sync

* wip UI

* Remove some obsolete comments

* wip UI

* Cleaning up, running tests

* wip UI

* Clean up migration

* wip UI

* m.

* input_schemas

* m.

* wip UI

* wip UI

* wip UI

* wip UI

* fix

* Cleaning

* wip FE

* wip UI

* wip UI

* Hand-rolled JDBC experiment

* workspaces: native input_schemas + output_schema -> output_namespace

Settled in Slack (Polyankin / Grabow / Bryan): drop the per-row :db slot
from workspace_database.input (the warehouse catalog is already in
Database.details — duplicating it across rows just invites drift), and
rename output_schema -> output_namespace to reflect that the column is
driver-opaque (schema on Postgres-shaped engines, database name on
MySQL, etc.). The table_remapping schema (with from_db/to_db) stays as-is
since those columns already source from Database.details, not from any
workspace input field.

Storage shape end-to-end:
  - workspace_database.input_schemas  text JSON, [String]
  - workspace_database.output_namespace varchar 254, driver-opaque string

Per-driver grant! impls now take a vector of schema-name strings. 3-slot
drivers (Snowflake, SQL Server, BigQuery) derive the catalog from
(:db (:details db)) / (:project-id (:details db)) themselves. MySQL
asserts the input set equals the bound database — silently ignoring
caller intent has bitten this codebase before. Snowflake / SQL Server
throw at grant time if details lacks an explicit db, instead of the
previous (or db default-db) fallback.

Single engine-namespace-positions fn replaces schema-position-value and
db-position-value. One case branch per engine, verbose by design — the
behavior is obvious at a glance now instead of cross-referencing two
helpers. db-position-value is kept as a 1-line accessor over the new fn
for compatibility with existing callers.

The instance-config atom's :output map shape is preserved
({:db ?, :schema ?}). The advanced-config loader's new expand-output
helper does the storage-string -> map expansion once at boot using
engine-namespace-positions, so QP middleware / transform_hooks /
table_remapping/add-transform-target-mapping! consume the same map
shape they always have. Zero churn on the per-query hot path.

The temporary input <-> input_schemas adapter Alexander added in
api/manager.clj is removed; endpoints pass :input_schemas straight
through. The presented WorkspaceDatabaseResponse regains
:output_namespace (it dropped off when the adapter went in).

Migrations: 20260421_workspaces.yaml edited in place to rename the
column; 20260508_workspace_input_rename.yaml deleted entirely (it
existed only to flip the now-reverted shape). Branch is unshipped, so
in-place edits are acceptable per its own changelog comment. Devs on
this branch will need to re-migrate.

* workspace instance UI

* workspaces e2e: drop stale redshift-filter TODO

The with-redshift-describe-filter-disabled wrapper already gates on
(= :redshift driver/*driver*) — non-Redshift drivers no-op through the
else branch. The wrapping behavior the TODO worried about was already
correct, so the comment was just misleading.

* Some fixes

* Disable sync during import tests

* workspaces e2e: add bigquery to the full e2e

The workspaces full-e2e test
(metabase-enterprise.workspaces.e2e-test/workspace-full-e2e-test) ran
against 6 JDBC drivers but skipped BigQuery because BQ uses the BigQuery
Java API instead of JDBC. The BQ-specific driver-isolation test covers
init/grant/destroy + cross-workspace + grant accumulation, but never
exercised the full transform-run + describe-database +
canonical-table-protection loop on BQ. This adds that coverage.

Approach:

- Extract the BQ test helpers from workspace_isolation_test.clj into a
  new non-test ns metabase.driver.bigquery-cloud-sdk.workspace-test-util.
  Both the existing isolation test and the e2e test consume that ns.
  Re-uses the BQ driver's existing private database-details->client and
  get-project-id (via #') instead of duplicating BigQueryOptions/newBuilder
  plumbing.

- Branch each JDBC helper in e2e_test.clj on driver. JDBC drivers go
  through (jdbc/execute! / jdbc/query) on a JDBC db-spec; BigQuery goes
  through bq.util/{execute!,query,create-dataset!,drop-dataset!,list-tables}
  on a BigQuery Java client. The deftest body picks the right
  admin-warehouse value once and threads it through.

- Lazy-load the BQ util ns via requiring-resolve through a small bq-util
  helper. Cross-module test-classpath coupling is the codebase norm
  (matches transforms_python and the redshift filter macro).

- Add :bigquery-cloud-sdk to workspaces-supported-drivers (renamed from
  workspaces-supported-dwh-drivers since 'DWH' no longer carries weight
  with BQ in the set).

- New required fix in workspaces/config.clj: schema-filter-entries used
  to emit :schema-filters-* unconditionally for any engine that
  database-supports? :schemas. BigQuery's list-datasets reads
  :dataset-filters-type / :dataset-filters-patterns, NOT the
  schema-filters keys. Without this fix, a BQ workspace's sync would
  silently return no datasets after boot and the e2e would fail at the
  first describe-database assertion. Added a config_test for the BQ
  branch.

The BQ test only runs in CI where DRIVERS=bigquery-cloud-sdk and the
BigQuery service-account JSON secret is set. Local module-test runs
(./bin/test-agent :module enterprise/workspaces) skip BQ at the
mt/test-drivers gate the same way they skip every other DWH driver.

Out of scope: a BigQuery TableRemapping case-arm in the existing
:from_db/:to_db assertion table, and an :impersonate-service-account
regression guard. Both depend on observing the live BQ run output to
nail down the right expected values; deferred until the first CI BQ
run lands.

* workspaces table_remapping: drop stale brainstorm prose from docstring

The slot-table docstring had a few stray paragraphs about a hypothetical
multi-db Postgres driver that got left in during earlier design notes.
Removed — the table itself already captures the contract.

* instance UI

* Test fixes, PG copy

* Fix regressions from e090449

* FE tests

* workspaces: CI fixups for modules-linter + stats analytics + ClickHouse e2e

Five distinct fixes from the first CI run on PR #73236:

1. modules-test: relocated engine-namespace-positions from
   workspaces.table-remapping into workspaces.core. The advanced-config
   loader's new require of table-remapping triggered the module linter
   (table-remapping is internal API). Promoting the one fn the loader
   needs into .core keeps the surface clean. table-remapping's internal
   callers now reach back through ws/engine-namespace-positions.

2. modules-test: added 'premium-features' to transforms-base :uses in
   .clj-kondo/config/modules/config.edn. transforms-base.workspace-hooks
   requires metabase.premium-features.core; pre-existing on the branch
   but only surfaced now.

3. stats-test/every-feature-is-accounted-for-test: added :workspaces
   to ee-snowplow-features-data, tracked via enable-workspaces?
   feature flag + (t2/exists? :model/Workspace).

4. workspaces e2e: create-canonical-schema! now branches ClickHouse
   onto CREATE DATABASE instead of CREATE SCHEMA. Pre-existing typo
   masked because the ClickHouse e2e job rarely ran in CI; surfaced
   now that the helper layer was reorganized.

5. driver/grant-workspace-read-access! :mysql: removed the
   bound-db-only assertion I added last session. driver-isolation tests
   create per-run input databases that legitimately aren't the bound
   db, and GRANT on those is valid as long as admin has perms.

Plus minor fallout:
- table_remapping_test: stale :input [{:schema}] fixture updated to
  :input_schemas [string].
- corpus_test: ws.table-remapping/engine-namespace-positions ->
  ws/engine-namespace-positions; dropped now-unused require.

Module suite passes (159 tests, 381 assertions). Modules + stats tests
pass directly. Not pushed.

* workspaces: fix four CI-surfaced bugs from PR review

Four mechanical fixes from Dan/Tim Sutton's PR review (gist
30a8827548ee09e2983f5a964cc23038) + drivers-expert analysis of the
connection-test cluster.

1. MySQL check-isolation-permissions passed a `[{:db ...}]` map but
   the multimethod takes `[String]` after the input_schemas migration.
   Caller now passes `[(:schema test-table)]`. Was throwing at runtime
   on every MySQL isolation-permission check.

2. workspaces/api/instance.clj read `:input` from the workspace atom,
   but the atom shape is `:input_schemas`. `(vec (keep :schema (:input
   wsd)))` was always `[]` in production; the matching test fixture in
   instance_test.clj used the wrong shape too, masking the bug. Both
   updated.

3. metabase.driver.connection-test + sql-jdbc.connection-test wrapped
   tests with `(mt/with-premium-features #{:admin-connection})`, but
   commit e090449250 flipped the EE gate from `:admin-connection` to
   `:workspaces` and missed these two test files. Seven `:admin-
   connection` -> `:workspaces` swaps. Connection tests go from red to
   green locally.

4. Sentinel `""` vs `nil` schema mismatch in three table_remapping
   functions (`filter-workspace-side-tables`, `inject-workspace-
   canonical-tuples`, `rewrite-fk-result-canonical`). Storage carries
   `""` for absent slots; MySQL sync tuples carry `nil`. Comparisons
   never matched, causing canonical tables to be retired every sync
   cycle on MySQL, workspace-side tables to leak through, and FK
   rewrites to silently fail. Applied the existing `denormalize-level`
   helper to schema values on both sides of each comparison.

Module suite: 167 tests, 401 assertions, 0 failures locally.

* workspaces: clear TableRemapping rows on deprovision

Closes 'delete-workspace! doesn't clear TableRemapping rows' from the PR
review (table-remapping team's bucket).

Before: deprovisioning a WorkspaceDatabase dropped the warehouse iso
schema but left stale TableRemapping rows behind. A subsequent
re-provision (same workspace) or a fresh workspace targeting the same
canonical tables would inherit those stale rows and route writes /
reads to a schema that no longer exists.

After: deprovision now calls clear-mappings-for-iso! after the driver's
destroy-workspace-isolation! succeeds, deleting every TableRemapping
row on the workspace_database's database_id whose to_* slots match
this workspace's iso namespace.

Scoping by iso namespace (not by database_id alone) is correct because
the unique constraint on (database_id, from_db, from_schema,
from_table_name) already prevents two workspaces on the same metabase
Database from remapping the same canonical table — so an iso-scoped
delete can't accidentally clobber another workspace's rows on the same
DB.

Implementation lives in a new leaf ns,
metabase-enterprise.workspaces.remapping-cleanup, to avoid the
provisioning -> table-remapping -> core -> provisioning require cycle
the in-place fix would have created. The per-engine :db-slot
derivation is duplicated there (kept narrow — only 3-slot engines
need it; rest get the empty-string sentinel); keep it in sync with
table-remapping/engine-namespace-positions.

New tests in remapping-cleanup-test cover:
- iso-scoped delete leaves other workspaces' rows on the same DB alone
- idempotent (returns 0 when no rows match)
- 3-slot engines match on :db too

Module suite: 170 tests, 409 assertions, 0 failures.

* accept bare table reference handling as-is

* Concurrency protection

* workspaces tests: scope TableRemapping queries + MySQL-shape regressions

Closes GHY-3556 + GHY-3553 from the gist-driven PR-review backlog.

GHY-3556 (e2e_test.clj): two `t2/select :model/TableRemapping` calls were
unscoped. Order-dependent: any leftover or parallel-test remap row would
falsify the `=` assertion on line 446 and pick a wrong `to_table_name` at
line 487. Scoped by `:database_id (:id ws-db)`; line 487 also pinned by
`:from_table_name output-table-name` since `select-one` on an unscoped
table is nondeterministic across multiple rows.

GHY-3553 (table_remapping_test.clj): existing tests for
filter-workspace-side-tables, inject-workspace-canonical-tuples, and
rewrite-fk-result-canonical all use Postgres-style `"public"` schemas,
so the sentinel-leak bug fixed in db39a79608 (storage `""` vs sync
`nil` on schema-less drivers) was never exercised. Added three
MySQL-shape regressions that simulate the bug by `add-mapping!`-ing
with `:schema nil` and calling each fn with sync-shape tuples carrying
`:schema nil`. Verified each test bites by REPL-reverting the
`denormalize-level` call site -- pre-fix code fails, post-fix passes.

Module suite: 54 tests, 140 assertions, 0 failures.

* workspaces: engine-namespace-positions throws on unknown 3-slot engine

PR-review gist #6 (dpsutton): the catch-all branch in
`engine-namespace-positions` silently returned `{:db nil :schema
(:schema table)}` for any unrecognized engine. Fine for an unknown
2-slot driver (Postgres-shape is a reasonable default), but a latent
correctness bug for 3-slot or MySQL-style 1-slot-with-db drivers we
forgot to enumerate: every remap row would get `:db nil`, cross-DB
routing would silently break at query time, and the failure surfaces
hundreds of stack frames downstream in the QP.

Split the catch-all by consulting `qualified-name-components`:

  - components includes `:db` -> throw `ex-info` naming the engine and
    its components. Bug is at the caller's enumeration, not at the
    failure site; fail at the boundary.
  - otherwise (2-slot or less) -> keep the existing degrade.

Two new tests in table_remapping_test.clj cover both branches. Verified
the throw test bites by REPL-reverting the production fn to its
pre-fix silent-degrade form -- pre-fix the test fails (no throw),
post-fix it passes.

Module suite: 175 tests, 420 assertions, 0 failures.

* workspaces: gate native-SQL rewriter on :native source type only

DEV-1953. `run-query-transform!` was calling
`rewrite-native-sql-for-workspace` on every compiled query, including
MBQL sources. For an MBQL source `qp.compile/compile` already routes
the query through the full QP pipeline -- including the Phase 2
table-remapping rewriter -- so the second pass is at best a no-op and
at worst a SQLGlot parse failure on emitter-specific SQL (window fns,
dialect-specific CTAS shapes) that would fail an otherwise-valid
transform.

Gate the call on `(-> source :query :type)` being `:native`. Native
sources bypass QP middleware so they genuinely need this rewrite;
MBQL sources don't. Comment makes the rationale explicit so it doesn't
get reintroduced.

Coverage is via the e2e
`native-transform-references-prior-canonical-output-test` which already
exercises both an MBQL transform (transform A, must keep working) and
a native transform that references A's canonical output (transform B,
must continue to get its SQL rewritten).

Module suite: 175 tests, 420 assertions, 0 failures.

* workspaces e2e: document why second e2e excludes Redshift

GHY-3557. `native-transform-references-prior-canonical-output-test`
runs on `#{:postgres :mysql}` while `workspace-full-e2e-test` runs on
the full `workspaces-supported-drivers` set including `:redshift`. The
inconsistency looks accidental but is intentional: the smaller test
depends on workspace-user `describe-database` returning the source
table, and on Redshift the workspace user's `describe-database`
returns `{:tables #{}}` even when USAGE/SELECT grants are intact. The
omission rationale lives in the test docstring; this commit lifts the
warning to the `workspaces-supported-drivers` def so future readers
don't widen the smaller test without first fixing the underlying sync
visibility issue.

* GHY-3549

* workspaces middleware: fail-closed regression tests + dedup helpers

Four bundled PR-review items:

GHY-3575: corpus_test redefined a private `prune-no-level` whose
docstring claimed it mirrored the impl in
`workspaces.query-processor.middleware`, but the canonical fn actually
lives in `workspaces.table-remapping`. Drift risk. Deleted the
re-definition and routed all callers (5 sites) at
`ws.table-remapping/prune-no-level`.

GHY-3568: `phase-2-no-remappings-passthrough-test` and
`phase-2-noop-when-already-remapped-test` only asserted `@called?`,
which is trivially true any time the middleware returns. Replaced
both with assertions that capture the query handed to the next
middleware and compare its SQL to the input -- the actual passthrough
contract. The already-remapped case uses
`sql-tools/referenced-tables-raw` equality rather than string equality
to allow for SQLGlot's whitespace/casing normalization on a round-trip.

GHY-3566 + GHY-3567: added four regression tests for Phase 2's
fail-closed branches -- the security contract documented in the
namespace docstring but previously unverified:

  - `phase-2-fail-closed-on-unparseable-sql-test` -- garbage SQL must
    throw before any driver call.
  - `phase-2-fail-closed-on-db-routing-test` -- db-routing engaged
    throws `ex-info` with `qp.error-type/qp` and message naming the
    incompatibility.
  - `phase-2-fail-closed-on-impersonation-test` -- impersonation-
    enforced for a bound user throws the same shape.
  - `phase-2-impersonation-skipped-without-current-user-test` -- pins
    the existing short-circuit that skips the impersonation predicate
    in sync/transform contexts (no `api/*current-user-id*`).

Verified each fail-closed test bites by REPL-redefining
`apply-workspace-sql-remapping` to a no-op pass-through -- pre-fix
shape FAILS, post-fix shape PASSES.

Module suite: 179 tests, 430 assertions, 0 failures.

* workspaces instance API: rename output_schema -> output_namespace

Wire-name cleanup. Storage column is `output_namespace`, manager API
already calls it `output_namespace`, the runtime atom shape is
`:output {:db :schema}` -- only the instance-API surface was still
calling the flattened schema slot `output_schema`. Inconsistent and
confusing.

Per Alexander's Slack note the FE doesn't consume this field today
(only `name` and database `id`s), so the rename is wire-safe.

Renamed both the schema and the presenter in instance.clj; the
matching assertion in instance_test.clj; and three FE refs that
mirrored the BE field for typing purposes (type def, mock factory,
unit-spec fixture).

Module suite: 179 tests, 430 assertions, 0 failures.
FE type-check: no new errors introduced by this change.

* workspaces e2e: explain disabled bare-table-name card test

GHY-3576. The `#_`-prefixed `mt/with-temp` block looked like an
absent test, but it's deliberate -- bare-table-name SQL
(`SELECT * FROM table`, no schema) isn't supported by the Phase 2
SQLGlot rewriter, which keys on `(schema, table)` pairs.

Replace the terse two-line NOTE with a comment that:
  - states the disabled status up front and links GHY-3576,
  - contrasts with the two qualified-name cases bracketing it (iso
    schema above, canonical schema below), so a reader sees what's
    actually covered,
  - names the rewriter limitation (bare refs lack the schema slot the
    lookup needs),
  - says what re-enabling requires (rewriter attributes bare refs to
    the default schema before lookup),
  - tells the next reader how to re-enable (delete the `#_`).

No code change.

* workspaces e2e: point disabled bare-name test at correct tracking ticket

GHY-3576 was the review ticket about the bad comment itself (now Done
because the comment is fixed). The actual feature work -- "support
bare-table-name SQL refs in the Phase 2 rewriter" -- is tracked at
GHY-3580, which absorbs the older GHY-3533 ("Actually handle bare
table references") so there's a single canonical ticket. Update the
in-source pointers accordingly.

* Set table permissions

* workspaces e2e: drop LIMIT 1 from source-table probe (SQL Server syntax)

SQL Server rejects `LIMIT 1` ("Incorrect syntax near '1'", errorCode 102),
breaking workspace-full-e2e-test under :sqlserver. The probe only needs
>= 1 row; the source table is seeded with 3 rows by create-source-table!
just above, so dropping LIMIT is portable across every supported driver
without needing per-dialect TOP/LIMIT branching.

* Self-review work

* m. testing string

* workspaces: strip caller-name refs from docstrings

Caller names rot - they get renamed, moved, or removed independently of the
function they cite. Per CLAUDE.md ("don't include locations of callers when
documenting an API"), this scrubs the named-caller references from the
workspace remapping API and QP middleware docstrings while keeping the
behavioral / contract notes that explain WHY the function exists.

No code changes.

* dev/user.clj: rewrite stale dev.trace comment block to match dev.debug

GHY-3577. The comment block referenced `dev.trace/d`, `MB_TRACE_LOG`,
`/tmp/trace.log`, and `(dev.trace/tail 50)` -- all artifacts of an
earlier `dev.trace` prototype that no longer exists. The actual
namespace is `dev.debug`, the env var is `DEV_DEBUG_LOG`, the default
path is `/tmp/<cwd-basename>_debug.log`, and there is no `tail` fn
(use shell `tail -f` instead).

Rewrote the block to match `dev.debug`'s real API and env vars:
correct ns, correct env-var name, correct default-path shape, the
actual in-form commands (`:clear` / `:where` / `:stack` /
`[:set-file! ...]`), and the actual log-watch incantation.

* workspaces: fix CI failures from missing :workspaces feature + module config

- Add :workspaces to OSS ee-snowplow-features-data' fallback so
  every-feature-is-accounted-for-test matches the EE side.
- enterprise/workspaces module: add `settings` to :uses (workspaces.settings
  now requires metabase.settings.core) and declare :model/Workspace as an
  exported model.
- enterprise/analytics module: add :model/Workspace to :model-imports for the
  (t2/exists? :model/Workspace) reference in analytics/stats.clj.

* Some kondo fixes

* At least a better init

* Remove parallel

* m. whitespace

* Linting the migration

* delete some dead code

* workspaces config: emit :output_namespace, not expanded :output map

build-workspace-config was emitting the post-load runtime shape
(:output {:db ?, :schema ?}) into the config.yml wire payload, but the
loader spec (advanced-config.file.workspace/::workspace-database-config)
requires :output_namespace as a driver-opaque string. The expansion
:output_namespace -> {:db, :schema} happens in expand-output during
initialize!, not in build-workspace-config.

CI surfaced this as a Spec assertion failure in workspace-full-e2e-test
on Redshift, SQL Server, and (likely) every other workspace driver:
the build -> yaml -> initialize! round-trip aborted with
`(contains? % :output_namespace)`.

The local `table-namespace` helper that built the expanded map is now
dead -- removed, along with its ws.table-remapping require. Docstring
of build-workspace-config updated to reflect the actual emitted shape.

Updated config_test.clj's `build-workspace-config-happy-path-test` and
`build-workspace-config-three-slot-engine-test` assertions to expect
the wire shape (`:output_namespace "string"`) rather than the runtime
shape.

Origin: regression from 0d762c7a06 (output_schema -> output_namespace
rename earlier today; the serializer wasn't updated to match the new
atom shape).

Module suite: 179 tests, 430 assertions, 0 failures.

* workspaces tests: find-sql-exception walks ex-data, not just .getCause

CI surfaced a Redshift-only failure in `grant-accumulation-test`:

   expected SQLException for :select-b-before-grant;
   got class clojure.lang.ExceptionInfo
   expected: (some? sqle)
   actual:   (not (some? nil))

Root cause is in `clojure.java.jdbc`, not in the workspaces driver
or in the test logic. `jdbc/execute!` runs with `:transaction? true`
by default; `db-transaction*` (clojure/java/jdbc.clj L817-831) catches
the SQLException from `permission denied`, attempts a rollback, and
when the rollback itself fails (observed on Redshift's
autocommit-restore path), wraps the *original* SQLException inside
`ex-data :handling` rather than `.getCause`. The thrown ex-info has
no cause set, so the previous chain-walker returned nil and the test
asserted "no SQLException reachable".

Widen `find-sql-exception` to also peek into each level's `ex-data`
for Throwable values, with a visited-set to prevent cycles. Existing
shapes (bare SQLException, ex-info with SQLException as cause) still
resolve the same way; the new shape (ex-info with SQLException in
ex-data) now also resolves.

No driver code changed. Module suite: 179 tests, 430 assertions,
0 failures.

* UI feedback

* fix

* UI feedback

* fix

* fix

* fix

* fix mysql

* fix

* fix

* fixes

* test fix

* fix

* fix

* e2e

* e2e

* appdb portability

* m. honeysql adjustments

* m. test simplification

* workspaces CI fixes: driver-loadable guards + audit perm + temp-db cleanup

Three independent CI failures surfaced on iter 19, all test-side fixes:

1. `advanced-config.file.workspace-test/{snowflake-3-slot,bigquery-3-slot,
   per-driver-fixtures-parse-and-validate}-section-test` errored on Java 21
   Enterprise with "Could not load :snowflake driver". The round-trip step
   calls `apply-workspace-section!` -> `expand-output` ->
   `engine-namespace-positions` -> `driver/qualified-name-components`, which
   triggers driver-load. Snowflake/BigQuery aren't on the default Enterprise
   classpath. Guard each round-trip step with `workspaces.tu/driver-loadable?`
   (same pattern as the addressing-truth-table tests already use). The
   spec-validate steps still run unconditionally.

2. `audit-app.audit-test/audit-db-installation-test` failed across MariaDB
   10.6 / MariaDB Latest / MySQL 8.0 Enterprise. The default user-perms
   map for the audit DB now includes `:perms/workspaces :no` (new
   permission added by the workspaces branch). Added it to the test
   expectation.

3. `workspaces.api.manager-test/add-database-without-schemas-feature-test`
   errored on MySQL with "Cannot delete a Database with active
   workspace_database rows". `mt/with-temp [:model/Database]` cleanup
   fires before `mt/with-model-cleanup [:model/Workspace]` (LIFO), so
   the WSD row referencing the Database is still present when the
   Database delete hook fires. Wrap the body in `try/finally` and
   issue an explicit `DELETE /workspace-manager/:id/database/:db-id`
   to drop the WSD row before the temp-db rollback.

Module suite: 180 tests, 436 assertions, 0 failures.

* redshift workspace destroy: revoke ALTER DEFAULT PRIVILEGES on input schemas

`grant-workspace-read-access! :redshift` issues `ALTER DEFAULT
PRIVILEGES IN SCHEMA <input> GRANT SELECT ON TABLES TO <iso-user>`
on every input schema so the iso-user can see future warehouse tables
without a sync round-trip. Those entries land in `pg_default_acl`,
keyed by `(grantor=admin, schema=input)` with the iso-user as
grantee.

`destroy-workspace-isolation!` was revoking the same kind of
default-priv entry only on the workspace's own output schema, not on
the input schemas where the grant was set up. At `DROP USER` time
Redshift scans every catalog that could hold a dangling reference,
finds the lingering input-schema default-priv rows, and refuses:

  ERROR: user "mb__isolation_..." cannot be dropped because some
    objects depend on it
    Detail: privileges for default privileges on new relations
            belonging to user <grantor> in schema <input-schema>

Add a third statement to the per-granted-schema loop -- symmetric
inverse of the GRANT side -- so `pg_default_acl` is clean before
DROP USER runs.

Origin: the asymmetry was introduced in dc5f212e90 ("workspaces
e2e: green on Redshift") which added the `ALTER DEFAULT PRIVILEGES
GRANT SELECT` line to `grant-workspace-read-access!` but didn't
update the existing teardown (authored by Eric Normand in
ce0c6d6dd3). This commit closes the loop.

Note: `schemas-with-user-grants` queries `svv_relation_privileges`,
which lists schemas where the iso-user has actual relation grants.
In the failing CI path that's sufficient -- the test seeds tables
before granting, so relation grants exist and the schema is
discovered. A more thorough fix would also probe `pg_default_acl`
directly for any schema with dangling default-priv entries for the
iso-user, but that's a separate hardening exercise.

Full diagnosis written up at
`~/dv/vault2/metabase/ai-reports/2026-05-13-redshift-workspace-destroy-default-privs.md`.

* sqlserver: declare schema-filters property in plugin yaml

SQL Server never got a `:schema-filters` entry in its plugin yaml when
the schema-filter UI was rolled out one driver at a time in PR #21022
(Apr 2022). Postgres/Redshift/Snowflake/ClickHouse/BigQuery all got
declarations; SQL Server was simply skipped — no deliberate omission,
no capability gap, no UX rationale (verified via git log on the yaml).

The workspaces config loader writes
`schema-filters-type=inclusion` + `schema-filters-patterns=<input-schemas>`
into a SQL Server Database's `:details` at boot, but the generic
sql-jdbc `describe-database` impl gates the filter behind
`driver.u/find-schema-filters-prop`, which returns nil for drivers
without the declaration. Net effect on SQL Server: the workspace's
isolation schema (e.g. `mb__isolation_92499_1`) leaks into
`describe-database` output, breaking
`workspace-full-e2e-test`'s assertion that no iso-schema table is
described.

MySQL doesn't need this fix: `qualified-name-components = [:db]`
means cross-DB enumeration in `describe-database` is architecturally
impossible — iso tables in a sibling MySQL DB never surface.

Place the declaration after `cloud-ip-address-info` (matching where
Postgres/Redshift put it). Existing customers without filter keys in
`:details` behave identically — patterns are nil, fall through to
the old "enumerate all schemas" path. The connection-edit UI gains a
"Schemas" inclusion/exclusion input matching the other workspace-
supported drivers.

Module suite: 180 tests, 436 assertions, 0 failures.

Full diagnosis at
`~/.claude/plans/we-need-both-to-zazzy-cloud.md`.

* Revert "workspaces: gate native-SQL rewriter on :native source type only"

This reverts the gating logic introduced in c820b95a2e (DEV-1953).
The premise of that fix was that MBQL transforms don't need the
rewriter at this layer because `qp.compile/compile` runs the QP
pipeline including the Phase 2 rewriter. Wrong on two counts:

  1. Phase 2 middleware is installed in `qp.execute/run`'s middleware
     chain (see
     `metabase.query-processor.middleware.enterprise/apply-workspace-sql-remapping-middleware`).
     Transforms route through `run-query-transform!` ->
     `qp.compile/compile` -> `driver/run-transform!`. They bypass
     `qp.execute/run` entirely, so Phase 2 never fires for transforms.
     Both MBQL and native compiled SQL need this hook.

  2. The gate condition was silently always-false. The Transform
     model's :before-insert hook normalizes the source query via
     `lib-be/normalize-query`, which upgrades legacy queries to MBQL 5
     shape (`{:lib/type :mbql/query, :stages [...]}`). After upgrade,
     `(get-in source [:query :type])` is nil for both source types and
     `(= :native nil)` is false. So the rewriter never ran for any
     transform.

CI iter 24 caught it: `native-transform-references-prior-canonical-output-test`
on Postgres Latest failed with "relation canonical_schema.t_a does not
exist" -- transform B's native SQL was never rewritten canonical -> iso.

Restore unconditional rewriting. Updated the comment block to explain
WHY this hook is necessary for both MBQL and native sources.

Linear DEV-1953 will be re-closed as Canceled with this commit as the
reference. The "SQLGlot parse edge on MBQL-emitted SQL" concern that
motivated the gate is theoretical and was not surfacing in practice --
the rewriter has been running unconditionally on every transform since
the start of the workspaces branch without it materializing.

Module suite: 180 tests, 436 assertions, 0 failures.

* wip FE

* wip FE

* workspaces: module-config updates + drop stale comment

Two unrelated fixes bundled per request:

1. `.clj-kondo/config/modules/config.edn`: three new `:uses` entries
   required after the recent OSS-side `metabase.workspaces.core`
   extraction:
   - `workspaces :uses` += `util` (the new core ns requires
     `metabase.util.malli.registry`)
   - `enterprise/workspaces :uses` += `workspaces` (EE workspaces refs
     the new OSS ns)
   - `testing-api :uses` += `workspaces` (testing-api refs the new OSS ns)

   Caught by `metabase.core.modules-test/modules-config-up-to-date-test`
   on both Java 21 EE + Java 21 OSS App DB runs.

2. `src/metabase/transforms_base/query.clj`: drop the historical "an
   earlier attempt to gate on :native-only was reverted" comment block.
   Git history is the system of record for that; in-code prose should
   describe the present invariant, not narrate past edits.

Modules-test: 10 tests, 3862 assertions, 0 failures.

* workspaces e2e: declare ClickHouse MergeTree ORDER BY in helper

ClickHouse requires every MergeTree-family table to declare a storage
engine + ORDER BY (or PRIMARY KEY). The e2e helpers (`create-source-table!`
and `create-output-table!`) issued `CREATE TABLE ... (id INT, v VARCHAR(8))`
with no engine clause on non-BQ drivers. CI iter 33 on ClickHouse
surfaced:

  java.sql.SQLException: Code: 42. DB::Exception:
    ORDER BY or PRIMARY KEY clause is missing.

Lift `create-table-tail` from `driver_isolation_test.clj` into
`e2e_test.clj` (same signature, same behavior) so the e2e CREATE
statements pick up the right engine/ORDER-BY suffix on ClickHouse.
Other drivers get an empty tail and behave identically.

Module suite: 180 tests, 434 assertions, 0 failures.

* mysql workspace init: grant USAGE on bound-db so iso user can connect

The workspace child instance opens its JDBC connection through the
canonical Database row, whose `:details.db` is the bound database
(e.g. `test-data`). MySQL's JDBC driver issues an implicit `USE
<bound-db>` at connect time, and the server refuses with
`ER_DBACCESS_DENIED_ERROR (1044)` unless the user has *some*
privilege row covering that db.

`init-workspace-isolation! :mysql` was granting ALL PRIVILEGES on
the iso db (`<workspace-namespace>`) but issuing no grant against
the bound db. The iso user existed and could authenticate against
the server, but every `getConnection(jdbc:mysql://.../test-data,
iso-user, pw)` failed at the implicit USE before any SQL ran.

`GRANT USAGE ON <bound-db>.* TO <iso-user>@'%'` is MySQL's literal
synonym for "no privileges" -- it just inserts a row in `mysql.db`
so the implicit USE succeeds. Per the MySQL manual:

  USAGE can be specified to mean "no privileges." It is used to
  modify account attributes such as resource limits or SSL
  characteristics without naming specific account privileges.

After this grant the iso user can open connections where the bound
db is the default, but SELECT/INSERT/etc. against bound-db tables
remain denied (ER_TABLEACCESS_DENIED_ERROR / 1142) until
`grant-workspace-read-access!` adds explicit per-db grants.

CI iter 33 caught this on every MySQL/MariaDB job
(`workspaces.driver-isolation-test/{grant-accumulation,
init-handles-pre-existing-namespace, workspace-isolation-perms,
cross-workspace-isolation-perms}-test`). Same root cause across
all four; one statement covers them.

Guarded on `(seq bound-db)` so the no-default-db case (test fixtures
that don't bind a db) doesn't emit a garbage SQL.

Module suite: 180 tests, 434 assertions, 0 failures.

* clean up 10 clj-kondo warnings/errors across workspaces + BQ driver

Linter pass over workspaces-touched files. Net: -72 lines, +35 lines.
Dead-code deletion dominates.

`workspaces/config.clj` + `workspaces/core.clj`: swap deprecated
`driver/database-supports?` -> `driver.util/supports?` (2 call sites
across the two files).

`workspaces/e2e_test.clj`:
  - drop the `bq-util` indirection that constructed a dynamic
    `(symbol "metabase.driver.bigquery-cloud-sdk.workspace-test-util"
    (name sym))` -- clj-kondo's `requiring-resolve` hook rejects
    non-static symbol construction.
  - replace each `(bq-util 'foo)` call with the canonical Metabase
    lazy-load pattern: `((requiring-resolve
    'metabase.driver.bigquery-cloud-sdk.workspace-test-util/foo)
    ...)`. Same pattern used at `test/metabase/test/data/interface.clj:566`
    + `enterprise/backend/test/metabase_enterprise/transforms_python/
    drivers_test.clj:135`. Static symbol prefix -> kondo-clean,
    IDE-traceable. We can't use a direct `:require` because the BQ
    test sources live under `modules/drivers/bigquery-cloud-sdk/test/`
    and aren't on every test-runner classpath (e.g. `./bin/test-agent
    :module enterprise/workspaces`).
  - delete unused `list-warehouse-tables-in-schema` defn (confirmed
    no references elsewhere).

`.clj-kondo/config.edn` + `.clj-kondo/src/hooks/clojure/test.clj`:
the `:metabase/validate-deftest :parallel/safe` whitelist was
unreachable from `use-fixtures` because the hook's `use-fixtures`
arm passed raw top-level config to `warn-about-disallowed-parallel-
forms`, while the `deftest` arm correctly threaded
`(get-in config [:linters :metabase/validate-deftest])`. Fix the
hook to match. Then add 4 entries to the now-reachable whitelist:
`set-instance-workspace!` + `clear-instance-workspace!` under both
the EE ns (`metabase-enterprise.workspaces.core`) and the OSS stub
ns (`metabase.workspaces.core`) so `settings_test.clj`'s `:each`
fixture body doesn't trip the bang-fn check.

`bigquery_cloud_sdk.clj`: delete dead `ws-grant-table-read-access!`
and its only-caller helper `ws-has-table-iam-binding?` (24 lines),
plus the three imports they pulled in (`Identity`, `Role`,
`BigQuery$IAMOption`).

`bigquery_cloud_sdk/workspace_isolation_test.clj`: add missing
`:require [metabase.driver.bigquery-cloud-sdk :as bigquery]` +
`:require [metabase.util :as u]` + `:import (com.google.auth.oauth2
ServiceAccountCredentials)` (the test body already used all three
without the imports/requires), and drop the unused
`BigQuery$DatasetOption` import.

Module suite: 180 tests, 434 assertions, 0 failures.
`./bin/mage kondo` on every touched file: 0 errors, 0 warnings.

* workspaces/mysql: mirror prod config — input == bound DB on MySQL

CI iter on `feature/workspaces-v2` kept failing on MySQL 8.0 / MySQL
Latest / MariaDB 10.6 / MariaDB Latest `workspaces.driver-isolation-test/*`
(7 errors per job × 4 jobs) with:

  java.sql.SQLSyntaxErrorException:
    Access denied for user 'mb__isolation_*'@'%' to database 'test-data'
  SQLState: "42000"  errorCode: 1044

Root cause: the test setup diverged from the production config shape.
The canonical workspace config for MySQL
(`workspace_config_mysql.yml`) lists the bound DB itself as the
`input_schemas` entry — MySQL has no schema layer, so on MySQL the
"input schema" IS a DB name, and the only shipping shape is
`input_schemas = [<bound-db>]`. With that shape,
`grant-workspace-read-access!` runs `GRANT SELECT ON <bound-db>.*`
which writes a `mysql.db` row; the JDBC handshake's DB-access check
finds the row, connection succeeds.

The test instead created a separate `mb_iso_in_<run>` MySQL DB as
input and granted SELECT only on that fresh DB. The connection still
opened with `:db = test-data` (inherited from `mt/db` `:details`).
Iso user had zero privilege row covering `test-data` → handshake
DB-access check failed → 1044, before any query runs.

The fresh-namespace divergence has a legit motivation on Redshift
(`PUBLIC` schema has `CREATE` granted to `PUBLIC` by default, breaking
the input-deny contract). MySQL has no analogous default-grant
problem, so the divergence wasn't needed -- it just broke handshake
on MySQL.

This commit:

1. Reverts the `GRANT USAGE ON <bound-db>.*` from `init-workspace-isolation! :mysql`
   added in 64959c0083. That fix was wrong-premised -- MySQL treats
   db-scoped USAGE as a no-op (the server doesn't write a `mysql.db`
   row when every privilege flag would be N), so it didn't satisfy
   the handshake check. The accompanying comment claimed facts that
   are wrong about MySQL behavior; dropped.

2. Adds two helpers in `driver_isolation_test.clj`:
   - `reuse-bound-db-as-input?` — true on MySQL
   - `input-namespace-name` — returns bound DB on MySQL, fresh-name
     on schema-having drivers
   - `maybe-create-input-namespace!` / `maybe-drop-input-namespace!`
     — skip CREATE DATABASE / DROP DATABASE when the namespace IS the
     bound DB (we don't own it; just per-table cleanup)

3. Routes all 5 single-input test sites through `input-namespace-name`.
   On MySQL `in-schema` is now the bound DB; source tables live there
   with random per-run suffixes; the existing grant on `bound-db.*`
   writes the `mysql.db` row; handshake passes.

4. Excludes MySQL from `cross-workspace-isolation-perms-test`. The
   test premise requires two distinct input namespaces on the same
   Database row, but MySQL can't express that (only one bound DB per
   Database row, and shipping config requires input == bound).
   Cross-Database isolation across two separate MySQL servers is a
   different scenario; not asserted here.

The "ungranted-ns" sub-test inside `workspace-isolation-perms-test`
still creates a fresh DB on MySQL (`mb_iso_nogrant_*`) -- the helper
only skips CREATE/DROP when the namespace equals the bound DB.

Module suite: 180 tests, 434 assertions, 0 failures.

* refactor e2e test by hand

* clj-kondo: drop over-eager use-fixtures destructive-form check

The `use-fixtures` hook in `.clj-kondo/src/hooks/clojure/test.clj` was
calling `warn-about-disallowed-parallel-forms` on every fixture body,
which produces false positives for non-parallel test namespaces that
use destructive ops in `:each` fixtures (a common pattern: `:each` fixture
with `with-redefs`, `with-temporary-setting-values`, `with-model-cleanup`).

Earlier commit 66b99619e0 fixed the hook's config-lookup path so the
warning could actually fire — surfacing 13 pre-existing fixture
warnings across various test namespaces, none of which are inside
`^:parallel`-marked test namespaces.

`deftest-check-parallel` already gates correctly on `^:parallel`
metadata. The fixture-level check would need ns-wide knowledge ("does
any deftest in this ns have ^:parallel?") to fire correctly, which
kondo hooks can't easily do without ns-state tracking. Drop the
fixture-level check as a no-op with a comment explaining the gap.

Net: 13 false positives removed, no real concurrency hazards lost --
deftest-check-parallel still catches `:parallel/unsafe` forms inside
`^:parallel` deftests.

Full kondo run: errors: 0, warnings: 1 (an unrelated unused-binding in
transforms_base/util_test.clj, not introduced by this commit).

* driver-isolation-test: use [[]] for same-ns fn refs in docstrings

Clojure docstring convention for same-project fn/var refs is `[[name]]`,
not `` `name` ``. The `[[]]` syntax is recognized by clojure-lsp,
cider, and codox as a navigable reference; backticks read as opaque
code spans. Switch four sites in the helpers I added in
7ef2ba9f30:

  - `reuse-bound-db-as-input?` docstring: `[[drop-input-namespace-sqls]]`
  - `input-namespace-name` docstring: `[[reuse-bound-db-as-input?]]`
  - `maybe-create-input-namespace!` docstring: `[[reuse-bound-db-as-input?]]`
  - `maybe-drop-input-namespace!` docstring: `[[drop-input-namespace-sqls]]`
    + `[[reuse-bound-db-as-input?]]`

Backtick-wrap is reserved for external libs, parameter names, code
literals (`:details.db`, `[namespace table]`), error messages, file
paths -- but not in-project fn refs.

* e2e-test: clearer parent/child boundary marker in workspace section

Cosmetic: replace the single ASCII-box separator with two boxed
labels and an explicit `;;;` divider between them. Makes the
parent-instance-vs-child-instance scope boundary in the e2e setup
unambiguous to the eye -- the previous shape had Parent and Child
share a single open-ended box that could read as a continuation.

Plain block:

    ;;+---------------------------+
    ;;|       Parent above        |
    ;;+---------------------------+

    ;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;

    ;;+---------------------------+
    ;;|       Child  below        |
    ;;+---------------------------+

No behavior change. No test logic touched.

* driver-isolation-test: extend grant-accumulation MySQL-skip to ClickHouse

ClickHouse, like MySQL, is a database-as-namespace driver: its
`grant-workspace-read-access!` issues `GRANT SELECT ON <db>.*`
(database-wide), which means a single grant on the workspace's input
DB allows reads on every table in that DB -- including the unrelated
src-b table the test was about to assert "is denied."

The existing MySQL-skip at the `:select-b-before-grant` assertion
already documented this is "a grant-granularity question, not an
additivity question." ClickHouse has the same shape, so extend the
skip from `(= driver :mysql)` to `(contains? #{:mysql :clickhouse}
driver)`.

CI iter caught it (`grant-accumulation-test:219` on
`driver-tests / Clickhouse Driver Tests`):

  :clickhouse
   :select-b-before-grant unexpectedly succeeded
   expected: false  actual: false

Module suite: 180 tests, 434 assertions, 0 failures.

Does NOT touch the second ClickHouse e2e failure
(`workspace-full-e2e-test` can't-connect on a workspace-provisioned
ClickHouse Database) -- separate root cause, separate investigation.

* fix

* fix

* tests

* workspaces sync: enrich from-spec with :db slot so MySQL remap lookups hit (GHY-3588)

`metabase.sync.fetch-metadata` passes the hook `{:schema :name}` derived from a
`:model/Table` row — for MySQL that's `{:schema nil :name "orders"}`. The stored
remap row's canonical from-spec carries `{:db <bound-db> :schema nil :table
"orders"}`, so the driver-aware match in `canonical->isolated` (MySQL keys on
`[:table :db]`) false-missed and `table-fields-metadata` queried the driver at
the canonical name, returning no fields. The Transform Table view then surfaced
`Table 'transform_table' has no Fields associated with it.`

Fix: in `remap-table`, fetch the full `:model/Database` row and use
`engine-namespace-positions` to fill any `nil` slot on the caller's from-spec
before projecting and matching. Postgres/Redshift unaffected — their canonical
namespace already comes through `:schema`. MySQL gets `:db (:db (:details db))`,
matching what `spec-for-table` already records on the from-side at write time.

Adds a regression test that exercises the MySQL path via a raw `t2/insert!`
so the post-create sync hook can't try to connect to a non-existent MariaDB.

* lint fixes

* fix mysql

* fix warning

* e2e-test: surface BQ can-connect failures by hitting list-datasets directly

`workspace-full-e2e-test` on BigQuery has been failing at the
`mt/with-temp :model/Database` step with the unhelpful message
"Failed to connect to Database". The real cause is invisible: BigQuery's
`driver/can-connect?` swallows every `Exception` from `list-datasets`,
logs it on the daemon thread, and returns `false`. The test reporter
only sees the rethrow from `metabase.driver.util/can-connect-with-details?`
and the auth/quota/credential exception underneath never reaches the
CI run summary.

Hit `list-datasets` on the test thread before `with-temp` so any
thrown exception bubbles into an `is` assertion and prints. This adds
no behavior in the happy path -- when BQ is healthy, the call succeeds
and the test continues. When it isn't, we'll finally see what's wrong.

* fe fix

* clickhouse fix

* fix clickhouse again

* bigquery workspaces: wait for SA to propagate after create before grants

`init-workspace-isolation!` creates the workspace service account via
`CreateServiceAccount`, then immediately calls `setIamPolicy` (impersonation
grant, then project-level `roles/bigquery.jobUser`). GCP IAM is eventually
consistent: the SA record exists in the control plane as soon as the create
call returns, but the policy-enforcement endpoints may not see it for a few
seconds. The first grant that fires referencing the new SA then throws
`INVALID_ARGUMENT: ... does not exist`, which `ws-grant-project-role!` has
no `catch` for — it propagates and fails the whole `workspace-full-e2e-test`
on BigQuery.

Add `ws-wait-for-service-account!` and call it right after
`ws-create-service-account!`. The wait polls `getServiceAccount` (the same
endpoint `ws-service-account-exists?` already wraps) until the SA is visible.
A single attempt usually suffices; max-attempts/interval mirror the shape of
the existing `ws-wait-for-impersonation-ready!` (60 × 1s vs 120 × 1s — SA
visibility lag is much shorter than full impersonation readiness).

Local repro confirms: the propagation wait succeeds on the first attempt and
the impersonation grant + project grant now run without the race. The test
then proceeds further into the workspace flow.

* edge case bugfix

* bigquery workspaces: set project id on impersonated client + wait for dataset visibility

Two follow-ups to the SA-propagation wait so `workspace-full-e2e-test`
goes green end-to-end on BigQuery (verified locally, 23 assertions
all green).

1) `database-details->client` now calls `.setProjectId` whenever
   `:impersonate-service-account` is set. `ImpersonatedCredentials`
   doesn't carry a project id — it derives identity from the target
   SA, not from a key file — so the SDK threw "A project ID is
   required for this service but could not be determined from the
   builder or the environment" the first time anything tried to use
   the workspace-mode client (the `init-from-config-file!` healthcheck).
   Pull the project id from `:project-id` if present, else from the
   base SA's embedded project id, matching what
   `getOptions.getProjectId` would have returned via the non-
   impersonated path.

2) `grant-workspace-read-access!` now polls until the just-granted
   dataset is visible to the impersonated SA from BOTH `getDataset`
   AND `listDatasets`. BigQuery propagates dataset ACL changes to the
   get and list endpoints on separate clocks: `getDataset` sees the
   grant in ~1s but `listDatasets` (the endpoint `can-connect-with-
   details?` uses, via the inclusion-filter check) can lag tens of
   seconds. Gating on getDataset alone returned the wait too early and
   the next `can-connect` threw `Looks like we cannot find any matching
   datasets`. Each poll iteration builds a fresh impersonated client to
   avoid carrying token / negative-cache state across attempts —
   mirrors the existing `ws-wait-for-impersonation-ready!` pattern.

Test docstring on `workspaces-supported-drivers` updated to point at
the per-driver `:workspace true` flag in `database-supports?` as the
canonical source — set still hardcoded for now but at least the lead
sentence tells future readers where to check when adding a driver.

* small fix and some cleanup

* some cleanup

* workspaces: exclude snowflake from supported driver set

Snowflake is out of scope for v2 per lead-dev call. Flip
`database-supports? :workspace` to false and remove the branch-introduced
snowflake branches from `engine-namespace-positions`, `iso-db-slot`,
the e2e `workspaces-supported-drivers` set, and the branch-new
`qualified-name-components :snowflake` defmethod.

Pre-existing master code (init/destroy/grant isolation defmethods) is
left in place; the feature flag now gates them out.

* workspaces middleware-test: migrate off deprecated mt/mbql-query

`metabase.test.data/mbql-query` is deprecated since 0.61.0 in favor of
Lib-generated MBQL. Kondo started flagging the five remaining usages
in this test ns after the master merge. Three of them build a `:dataset_query`
for `with-temp :model/Card`, which still stores legacy MBQL — those go
through `lib/query` then `lib.convert/->legacy-MBQL`. The other two build
a Phase-1 query for the middleware's MBQL path and just need the Lib map
directly. Behavior unchanged; rerun of the affected three tests stays green
(5 assertions, 0 failures, 0 errors).

* bigquery workspaces: type-hint getDataset varargs to silence reflection warning

Eastwood flagged a reflection call on `BigQuery.getDataset` inside
`ws-wait-for-dataset-visible-via-impersonation!` -- the varargs array
literal `(into-array BigQuery$DatasetOption [])` was inferred as
`unknown`, so the method dispatch couldn't be resolved at compile time.
Add the explicit array-class hint that the rest of this driver uses for
the same overload (see e.g. `get-table*`).

* workspaces e2e: bump db-connection-timeout-ms to 30s for BigQuery cold path

`init-from-config-file!`'s `can-connect-with-details?` is wrapped in a
`with-timeout` that uses `db-connection-timeout-ms`. The test-mode
default is 5s. BigQuery's first call through a freshly-built impersonated
client (OAuth token mint + IAM `generateAccessToken` + cold gRPC channel
to `bigquery.googleapis.com` + `listDatasets`) takes >5s under workspace
mode, so the timeout fires before the workspace-mode read path can warm
up. The previous green local run was lucky; the next CI run reverted to
the same `Timed out after 5.0 s` failure surfaced at
`util.clj:135`.

Wrap the test in `with-temporary-setting-values` to raise the timeout to
30s. Mirrors what `db-connection-timeout-ms` is already bumped to in
`metabase.warehouses.models.database-test` for similar reasons.

Note: this is a bandage. The real fix is to skip the redundant
`can-connect-with-details?` check in `init-from-config-file!` for
workspace databases (provisioning already verified everything that
can-connect would check). Follow-up to be filed once this lands.

* permission e2e tests

* cleanup

* workspaces: drop snowflake from table-remapping + isolation tests

Snowflake is out of scope for v2 (fc478416d5). CI failed on
mysql/postgres app-db tests because the test data still asserted
Snowflake-shaped slot population (`qualified-name-components = [:db :schema]`,
`engine-namespace-positions :snowflake -> :db from details`), which the
previous commit removed.

Fix by removing Snowflake from the test fixtures + helpers:

- table_remapping_test.clj: drop snowflake row from addressing-truth-table
  and switch add-transform-target-mapping!-flows-both-slots-from-namespace-test
  to :sqlserver (still exercises the 3-slot flow).
- remapping_cleanup_test.clj: three-slot-engine-test switched to :sqlserver.
- advanced_config workspace_test.clj: drop snowflake-3-slot-section-test
  and the :snowflake per-driver fixture row.
- config_test.clj: build-workspace-config-three-slot-engine-test switched
  to :sqlserver.
- query_processor/{corpus,cross_driver,driver_corpus}_test.clj: drop snowflake
  rows from per-driver test data and case branches.
- driver_isolation_test.clj: drop :snowflake branches from input-namespace,
  hacker-setup, storage-escape, cross-database, and public-default-grant
  helpers; restrict supports-cross-database-isolation-test? to :sqlserver
  and supports-public-default-grant-test? to :mysql.
- workspace_config_snowflake.yml: deleted.
- test_util.clj: example comment switched to :sqlserver.

Source docstrings/comments in table_remapping.clj, provisioning.clj,
config.clj, and advanced_config/file/workspace.clj updated to stop
listing Snowflake among the 3-slot drivers.

* Metadata file import: add import id and polling, remove env var trigger.

* workspaces config-test: drop stray #p debug reader from build-workspace-config test

Leftover hashp `#p` debug-print reader macro from a REPL session. Clj-kondo
warned `You can use it, but don't commit it.`. No behavior change.

* bigquery workspaces: stop swallowing real IAM failures in ws-grant-impersonation-permission!

The bare `(catch Exception e (log/warn e "…may already exist"))` hid
genuine `PERMISSION_DENIED` (main SA lost serviceAccountTokenCreator
admin grant), `RESOURCE_EXHAUSTED` (IAM quota), `INVALID_ARGUMENT`,
network/`UNAVAILABLE`, and etag-race errors. After a swallowed real
failure, `init-workspace-isolation!` continued on to the project-role
grant (which succeeds independently), then `ws-wait-for-impersonation-
ready!` polled `listDatasets` under the impersonated SA for 120s with
no diagnostic surface pointing back at the real cause.

The pre-check `ws-has-role-binding?` already short-circuits the only
no-op-of-concern (binding already present) before any write, so dropping
the catch keeps the happy path idempotent and lets every other failure
mode propagate immediately.

Verified locally with `workspace-full-e2e-test :bigquery-cloud-sdk`:
23 assertions, 0 failures, 0 errors. Closes GHY-3596.

* m. whitespace

* Missed a config line.

* workspaces sync: swap JDBC connection per iso :db when probing FKs (GHY-3599)

`fetch-metadata/fk-metadata` was the only sync probe that didn't wrap its
driver call in `with-swapped-connection-details`. On MySQL workspaces
with cross-DB swaps (canonical `:db = appdata`, iso `:db = appdata_ws`)
the JDBC connection therefore stayed pointed at the canonical bound DB
during `describe-fks`, missing every FK that lives in the iso DB. Field
metadata, index metadata, and nested-field metadata already had
`do-with-effective-table`; FK metadata was the last gap.

The FK probe takes a set of `:schema-names` rather than a single table,
so the field-level swap shape doesn't transplant directly. New
`defenterprise call-with-fk-probe-iso-dbs` (OSS facade: invokes the
thunk once with no swap, preserving today's behavior) accepts a thunk
and returns a vector of result batches. The EE impl walks the database's
active `TableRemapping` rows, collects distinct iso `:db` values that
differ from canonical, and invokes the thunk once per iso DB inside a
`with-swapped-connection-details` scope. When iso DBs match canonical
(3-slot engines where the workspace stays in the same bound DB) it
collapses to the single-canonical-call path.

`fk-metadata` wraps `describe-fks` in the hook, flattens row batches,
and hands them to `rewrite-fk-result-canonical` unchanged.

Regression test asserts `has-connection-swap?` is true inside the
describe-fks mock when the database has a cross-DB remap, and was
verified to fail before the fix.

  58 table-remapping tests, 117 assertions, 0 failures, 0 errors.
  3 sync-metadata FK tests, 7 assertions, 0 failures, 0 errors.

* workspaces: clear TableRemapping rows even when driver destroy! throws (GHY-3595)

`deprovision-workspace-database!` previously called
`clear-mappings-for-iso!` *after* `destroy!`, then relied on the outer
try/catch to roll status back to `:provisioned` on any throw. The
warehouse-partial-failure case (e.g. BigQuery: dataset dropped, SA
delete throws) skipped the cleanup entirely -- warehouse said
"nothing", app-db said "workspace is provisioned, here are its
TableRemapping rows" -- so the next time anyone queried a canonical
table, the QP rewriter sent the SQL to an iso namespace that no longer
existed and the warehouse returned `Not found`. 5xx in the QP, broken
cards.

Wrap the `destroy!` call in `try/finally` so `clear-mappings-for-iso!`
runs whether or not the warehouse teardown completed. The outer rollback
still flips status back to `:provisioned` on failure (so operators can
retry deprovision), but the workspace can no longer route queries to a
missing iso namespace in the interim. Retry of the deprovision is
idempotent: the destroy multimethods already handle "already gone" for
each resource type they manage.

Regression test asserts: a failing `destroy!` rethrows, but the
TableRemapping rows are cleared before the throw propagates. Verified
the test fails on the pre-fix code path.

  62 tests in workspaces.remapping-cleanup-test + workspaces.table-
  remapping-test, 129 assertions, 0 failures, 0 errors.

* bigquery workspaces-test: split grant-accumulation across two input datasets

`grant-accumulation-bigquery-test` was failing with
"select-b-before-grant unexpectedly succeeded" because both source
tables lived in the same input dataset, and BigQuery's `dataViewer`
ACL is dataset-scoped, not table-scoped — granting on the shared
dataset already gave the workspace SA access to table B before B's
"second grant" call. The additive-grants contract the test pins is
across multiple `grant-workspace-read-access!` *calls*, not across
tables within one dataset.

Split source tables into two separate input datasets (`in-a`, `in-b`).
First grant covers only dataset A → A readable, B denied. Second
grant covers only dataset B → both readable, proving the previous
binding wasn't revoked. Use `expect-denied!` (accepts 4xx) instead of
`expect-write-denied!` (strict 403) because a workspace SA querying a
dataset it has no `bigquery.datasets.get` permission on may surface
the denial as 404 rather than 403.

* docstring cleanup

* Minor cleanup

* BigQuery workspace isolation: retry project-role grant on SA propagation race

After ws-create-service-account! returns, the workspace SA can take a few
seconds to become visible to other GCP APIs. ws-grant-project-role! calls
ProjectsClient.setIamPolicy (Cloud Resource Manager), which validates the
member exists and rejects with INVALID_ARGUMENT: 'Service account ... does
not exist' if propagation hasn't completed. Wrap the get-and-set in a 60s
retry that recognizes this specific error and retries every second.

* workspaces: commit TableRemapping cleanup outside cluster-lock tx

`deprovision-workspace-database!` clears `TableRemapping` rows in a
`finally` so a partial warehouse teardown still leaves app-DB in a
"workspace no longer routes queries" state. But `with-cluster-lock`
wraps the body in `t2/with-transaction`, so when `destroy!` throws the
finally-side DELETE got rolled back along with everything else. App-db
tests across Postgres/MySQL/MariaDB caught this; h2 happened to pass
because its cluster-lock path is an in-process RW lock with no JDBC tx.

Rebind `toucan2.connection/*current-connectable*` to nil for the
cleanup call so the DELETE runs on a fresh autocommit connection
outside the lock tx and survives the rollback. Matches the existing
idiom in `metabase.login-history.record` and `premium-features.token-check`.

Also fix the regression test: it was hitting the same wedge through a
different door -- `metabase.test.redefs` wraps `with-temp` in a
rollback-only transaction, and the new fresh-connection cleanup
couldn't see fixture rows that lived only inside that tx. Bind
`tu.thread-local/*thread-local* false` to disable the wrap for this
case (real callers always persist `TableRemapping` rows before
deprovision runs, so this matches production).

* bigquery cross-workspace test: split source tables across two input datasets

The BigQuery grant-workspace-read-access! impl grants dataViewer at the
*dataset* level, matching the schema-wide semantics of the SQL drivers.
The test was placing both src-a and src-b in a single shared input
dataset, then granting each workspace access to that one dataset --
which (correctly) let either workspace read both tables, so the
':select-other-grant' assertion ('A cannot read a source granted only
to B') failed as a correctly-scoped grant rather than a leak.

Mirror the JDBC sibling's structure: separate in_dataset_a/b per
workspace, src-a in a, src-b in b, grant each workspace only its own.
Verified locally: 19/19 assertions pass.

* bigquery workspaces: verbose timeout error for IAM-impersonation wait

The 120s ceiling in `ws-wait-for-impersonation-ready!` is below Google's
documented worst case for IAM policy propagation (~7min:
https://docs.cloud.google.com/iam/docs/access-change-propagation).
When CI hits that worst-case tail and the wait throws, the old message
was just "Timeout waiting for IAM impersonation to propagate" with the
SA email and attempt count -- not enough to know whether GCP was slow,
the wrong project was targeted, or something else broke entirely.

Capture the underlying GCP exception across attempts, bucket them by
error class (access-denied / deadline / unavailable / unauthenticated /
other), and on timeout throw with:
- total elapsed wall time vs ceiling
- last error message + JVM class
- histogram of error classes seen across all attempts
- project id (so multi-project mixups are obvious)
- pointer to the IAM-propagation doc

Bumping the ceiling itself probably won't help -- it'd still flake -- so
this is purely diagnostic. The happy path also gets a one-line log of
attempt count + elapsed time so CI runs can confirm waits are not
quietly creeping up over time.

* bigquery_cloud_sdk: restore (if ...) shape mangled by autoformatter

`clj-paren-repair` ran via pre-commit hook on 54c5a289a0 and reshaped
the `:when ((if ...) inclusion-patterns ...)` form so the if's closing
paren moved past the inclusion-patterns/exclusion-patterns/dataset-id
args. That gave the `if` 5 arguments and clj-kondo correctly flagged
"Too many arguments to if". Restore the original `(if cond truthy
falsy)` shape with the call args on the outside.

The lint error existed locally before commit too -- I saw it and
dismissed it as pre-existing. It wasn't.

* bigquery_cloud_sdk: drop reflection in timeout-error class lookup

`(some-> last-error (.getClass) (.getName))` from 646bda29d2 triggered
two Eastwood reflection warnings (`.getClass` / `.getName` could not be
type-resolved without a hint on `last-error`).

Use `(some-> last-error class .getName)` -- `clojure.core/class` is
reflection-free and `.getName` on the returned `java.lang.Class` is too.

Verified locally with:
  clojure -X:dev:ee:ee-dev:drivers:drivers-dev:test:eastwood \
    :namespaces '[metabase.driver.bigquery-cloud-sdk]'
=> Warnings: 0. Exceptions thrown: 0

* workspaces config: fix ::non-blank-string spec to reject whitespace-only

The spec was `(s/and string? seq)`, which is a non-empty-string check --
`" "` and `"\t"` slip through. Use `(complement str/blank?)` so the
schema matches what its name promises. This is the same shape used by
the Malli sibling at `src/metabase/lib/schema/common.cljc:91`.

Workspace-database `:output_namespace` flows into SQL identifier
construction; accepting whitespace would have caused either a sync
failure later or a quoted-whitespace identifier in compiled SQL.

Add a focused unit test that pins both the reject set (nil, non-strings,
"", " ", "\t", "\n") and the accept set (strings with at least one
non-whitespace character, including leading/trailing whitespace strings
like " a ").

Resolves PR #73236 review comment #9 from @camsaul.

* serdes yaml: accept all YAML 1.1 plain-float forms

The float regex was `#"-?\d+\.\d+"` -- missed `.5`, `+1.0`, `1.`,
`1e5`, `1.5e-3`, `2E3`. Those scalars silently fell through to the
`:else v` string branch instead of coercing to Double.

Widen to YAML 1.1 plain-float shape:
- mantissa: `<digits>.<digits>?` | `.<digits>` | `<digits>` (bare digits
  only when an exponent follows)
- optional [eE] exponent
- optional leading sign

While in here, also widen the int regex from `#"-?\d+"` to
`#"[-+]?\d+"` so plain `+5` still hits the Long branch instead of
falling into Double. Branch order matters: int branch is first in the
`cond`, so plain digit-only inputs always parse as Long.

Extend `scalar-coercion-rules-test` with all the previously-broken
forms (`+5`, `+1.0`, `1.`, `.5`, `1e5`, `1.5e-3`, `2E3`).

Resolves PR #73236 review comment #12 from @camsaul.

* workspaces: drive engine-namespace-positions via driver multimethods

Replace the hardcoded `case (:engine database)` in
`workspaces.core/engine-namespace-positions` (and the duplicated
abbreviated case in `workspaces.remapping-cleanup/iso-db-slot`) with
two new driver multimethods in `metabase.driver.sql`:

- `table-qualification-style` returns one of
  `:table-qualification-style/{table,schema-table,db-table,db-schema-table}`
  describing the per-driver SQL identifier shape. Default is
  `:schema-table` -- the common case, so Postgres / Redshift / H2 /
  Clickhouse need no override.
- `db-slot-value` returns the `:db` AST slot value for `:db-table` /
  `:db-schema-table` drivers. Default returns nil; overridden by
  MySQL (`(:db (:details db))`), SQL Server (same), BigQuery
  (`(:project-id (:details db))`).

Third-party SQL drivers participate by implementing these instead of
getting a new branch added to the workspace `case`. Resolves Cam's
review comment #1 on PR #73236.

Split into two multimethods (structural vs details-extraction) so the
common case is zero-config and the per-driver `:details` shape stays
encapsulated in the driver that owns it.

Naming: namespaced-on-fn-name (`:table-qualification-style/...`)
rather than the more verbose `:driver.sql.table-qualification-style/...`
or the `:as-alias`-magic form. Matches Bryan's "prefer less magic"
preference: grep-able as a unit, no require ceremony at call sites,
no plain-keyword collision risk.

Drop the throw-on-unknown-3-slot-engine fallback -- with the
multimethod default of `:schema-table`, third-party drivers that
need `:db` slot participation simply override
`table-qualification-style`. Old behavior was "third-party authors
get a runtime exception if their driver declares `:db` in
`qualified-name-components` but we didn't add a case branch."

Tests:
- `engine-namespace-positions-unknown-2-slot-degrades-test` +
  `engine-namespace-positions-unknown-3-slot-throws-test` replaced
  with `engine-namespace-positions-unknown-driver-defaults-to-schema-table-test`
  (single test that pins the no-throw default behavior).
- `three-slot-engine-test` uses a test-local
  `::fake-3-slot` driver (registered via `driver/register!`) +
  per-driver overrides so the test doesn't depend on the sqlserver
  driver module being on the classpath.

Full workspaces module suite: 181 tests / 434 assertions / 0 failures.

REPL-verified dispatch table:
  :postgres / :redshift / :h2 / :clickhouse -> :schema-table (default)
  :mysql                                      -> :db-table
  :sqlserver / :bigquery-cloud-sdk            -> :db-schema-table

(`qualified-name-components` is kept for now -- it's a separate concept
used in non-workspace paths. Its rename to `table-qualification-style`
naming is Cam's #4 and will land in a follow-up commit.)

* fix kondo linter

* docs/driver-changelog: move workspace identifier-shape note to 0.62 + describe the actual API

The 0.61 section had a stale bullet describing
`metabase.driver/qualified-name-components` as the multimethod workspace
remapping consults to decide tuple shape. That hasn't been true since
commit 087b370d73 (PR #73236) -- workspace remap now goes through two
new multimethods in `metabase.driver.sql`:

- `table-qualification-style` (driver) -> identifier-shape enum
- `db-slot-value` (driver, database) -> `:db` AST slot string

Drop the stale 0.61 bullet. Add two new 0.62 bullets describing what
actually landed, including the default behavior (so Postgres /
Redshift / H2 / ClickHouse drivers need zero overrides) and which
existing drivers override what.

Resolves PR #73236 review comments #3 (section placement: belongs in
0.62, not 0.61) and #4 (naming: `table-qualification-style` is the
name Cam suggested; documenting that here).

(`metabase.driver/qualified-name-components` still exists as a separate
multimethod used in non-workspace paths -- its eventual rename to
`table-qualification-style` naming is a separate follow-up.)

* workspaces config: drop manual `keep-me` def

Sibling advanced-config.file.{settings,databases,users,api-keys}
namespaces don't define a `keep-me` var -- the parent's
`(comment <ns>/keep-me)` form references nonexistent vars, and kondo
is configured to tolerate the dangling references inside the comment
block. The def was cargo-cult noise that diverged from the convention.

Drop it. Kondo green on both files post-delete; the parent comment
ref keeps working.

Resolves PR #73236 review comment #8 from @camsaul.

* docstring indent

* serdes file-format dispatcher: case on extension instead of cond on suffix

Replace the three `(str/ends-with? lower ".json")` / `.yaml` / `.yml` branches
with `(case (FilenameUtils/getExtension name) "json" :json ("yaml" "yml") :yaml ...)`.
Apache Commons IO is already on the classpath (see
`src/metabase/geojson/api.clj`, `src/metabase/channel/render/image_bundle.clj`).
The `case` form is cheaper to read and to extend (one literal per branch
instead of a `str/ends-with?` call per format).

`clojure.string` require dropped (no other use in the ns).
6/6 parsers_test assertions pass post-change.

Resolves PR #73236 review comment from @camsaul on
`enterprise/backend/src/metabase_enterprise/serialization/metadata_file_import/parsers.clj`.

* serdes metadata-file-import: rename :kind snake_case keys to kebab-case

Cam flagged `:kind :unknown_format` as un-idiomatic (snake_case keyword
in Clojure code). Same applies to the other `:kind` values in the
parsers subtree -- `:bad_shape`, `:unsupported_alias`, `:missing_key`.

Mass rename to kebab-case:
  :unknown_format    -> :unknown-format
  :bad_shape         -> :bad-shape
  :unsupported_alias -> :unsupported-alias
  :missing_key       -> :missing-key

Also updated tests + docstrings that reference these by name.
27 parser tests / 63 assertions pass post-rename.

Resolves PR #73236 review comment from @camsaul on
`enterprise/backend/src/metabase_enterprise/serialization/metadata_file_import/parsers.clj:27`.

* serdes processors: drop redundant mr/validate + kebab-case :kind values

Two Cam comments rolled into one commit because they touch the same lines:

1. `validate-line!` called `mr/validate` to gate `mr/explain`, but
   `explain` checks validity itself and returns nil for valid inputs.
   Use `(when-let [humanized (me/humanize (mr/explain ...))] ...)` --
   drops one Malli pass per row on the hot drain/merge path.

2. Same `:kind` snake_case -> kebab-case sweep as the parsers subtree:
     :invalid_input              -> :invalid-input
     :file_incomplete            -> :file-incomplete
     :cycle_in_field_graph       -> :cycle-in-field-graph
     :depth_tagging_cap_exceeded -> :depth-tagging-cap-exceeded

Tests/docstrings updated. 12 processors_test + 9 drain_test pass post-change.
REPL-verified validate-line! still throws on invalid + returns nil on valid.

Resolves PR #73236 review comments from @camsaul on
`processors.clj:39` and the broader snake_case keyword theme.

* workspaces api: tighten schemas (db id, status enum, datetime name)

Three Cam comments on the api schemas, all small:

1. `:database_id ms/PositiveInt` -> `::lib.schema.id/database` in
   `workspace_instance.clj` (TableRemapping schema, databases map-of)
   and `workspace_manager.clj` (AddDatabaseParams + WorkspaceDatabaseResponse).
   The id schema documents intent + lets future schema work change the
   ID shape (UUID, opaque string, etc.) without touching every endpoint.

2. `WorkspaceStatus` enum was `[:enum "unprovisioned" ...]` (strings)
   with a `(update :status name)` shim in `present-workspace-database`
   to convert the DB-keyword to a string before validation. Make the
   schema keyword-valued with a `{:decode/api keyword}` rule -- drops
   the manual `name` call, decode rule handles string->kw conversion
   for request params, and responses go through the schema as keywords
   (JSON encoder stringifies at the wire boundary).

3. Rename `Timestamp` -> `DateTimeWithTimeZone`. `Timestamp` is
   ambiguous; `TIMESTAMP` in most SQL dialects means TIMESTAMP WITHOUT
   TIME ZONE. The schema actually requires `OffsetDateTime` or
   `ZonedDateTime` -- the new name says so. Updated the two
   `:created_at` / `:updated_at` references in `WorkspaceResponse`.

13 api tests / 53 assertions pass post-change.

Resolves PR #73236 review comments #9, #10, #11, #12 from @camsaul.

* serdes + workspaces: three low-hanging review nits

Bundle:

1. yaml/coerce-scalar: set-membership over `or` chain of `=` (Cam #11).
   `(or (= v "true") (= v "True") (= v "TRUE"))` -> `(#{"true" "True" "TRUE"} v)`.
   Same for null and false literal sets. Pure readability win, identical
   semantics (set-as-fn returns the matched literal or nil).

2. yaml/skip-value!: `condp instance?` instead of `cond` over `instance?`
   calls (Cam #13). Mirrors the shape of sibling `read-from-event` directly
   above, which already uses `condp instance?`.

3. advanced-config.file.workspace docstring: convert two backticked fn refs
   to `[[wikilinks]]` (Cam #7):
   `advanced-config.file/initialize!` -> `[[metabase-enterprise.advanced-config.file/initialize!]]`
   `workspaces.core/workspace-instance-config` -> `[[metabase-enterprise.workspaces.core/workspace-instance-config]]`

   Per Bryan's docstring-refs convention: backticks for keywords / params /
   external lib code; wikilinks for fn refs within the codebase.

11 yaml-test / 35 assertions pass post-change.

Resolves PR #73236 review comments #7, #11, #13 from @camsaul.

* fix CI fallout from prior commits in this pass

Two regressions surfaced by CI on push 2340ee9f73:

1. `metabase.driver.impl-test/driver-multimethods-in-changelog-test`
   walks every `defmulti` in `metabase.driver` and asserts its `:added`
   version's section in `docs/developers-guide/driver-changelog.md`
   mentions the multimethod by name. `qualified-name-components` was
   `:added "0.61.0"` but the bullet documenting it lived in the 0.61
   section, which earlier commit 0283db68b1 moved out (incorrectly,
   since the multimethod hasn't actually shipped in 0.61.0 -- it's
   landing in 0.62 on this branch).

   Fix: bump `:added` to `"0.62.0"` and add a 0.62 bullet for it
   (alongside the existing `table-qualification-style` + `db-slot-value`
   bullets). Everything ships together.

2. The snake_case -> kebab-case `:kind` rename in commits 79ec803091
   + 9fa9419d5a missed three test files that asserted on the old
   keyword values:
     - serialization/metadata_file_import_test.clj
     - serialization/metadata_file_import/orphan_check_test.clj
     - serialization/metadata_file_import/depth_test.clj
   Rename them now: same sed pipeline as before.

Local runs: meta-test 289/289, three renamed tests 33 tests / 85
assertions, all pass.

* Delete yaml parser

* Create a remapping aware metadata provider

We used to just override metadata in the preexisting metadata provider,
using `lib.metadata.protocols/CachedMetadataProvider/store-metdata!`

```
  (doseq [[from-spec to-spec] remappings
          ,,,
    (lib.metadata.protocols/store-metadata! ;; mutate the existing cache
     metadata-provider
     (assoc table
            :db     (ws.table-remapping/denormalize-level to-db)
            :schema (ws.table-remapping/denormalize-level to-schema)
            :name to-name)))
```

Now instead we map over table metadata results

```clojure
(deftype TableRemappingMetadataProvider [f parent-metadata-provider]
  lib.metadata.protocols/MetadataProvider
  (database [_this]
    (lib.metadata.protocols/database parent-metadata-provider))
  (metadatas [_this {mt :lib/type :as metadata-spec}]
    (cond->> (lib.metadata.protocols/metadatas parent-metadata-provider metadata-spec)
      (= mt :metadata/table)
      (into [] (map f))))
  (setting [_this setting-key]
    (lib.metadata.protocols/setting parent-metadata-provider setting-key)))
```

Notes and concerns:
we clear out the cached values by wrapping with a caching mp which
delegates to the underlying. But this also removes the invocation
tracker. There's no existing mechanism to have a pass through to the old
one. So we could do a few things: implement invocation tracking and
caching, and have the caching apply the remapping as well. But it seems
simplest enough to toss away the tracking and cache so that everything
is blank slate.

The cache seems fine on its own. Invocation tracking affects only
updated card last used at when they are part of the query. This is
considered acceptable for now because this is only happening on the
workspace instance and not the parent. So we don't care about card usage
as this is a development instance.

```clojure
  (let [remapper     (table-remapper remappings)
        remapping-mp (-> (remapping-provider remapper mp)
                         lib.metadata.cached-provider/cached-metadata-provider
                         ;; this creates a new tracker that doesn't carry over previous stats. This is fine for now as any
                         ;; instance using table remapping is a workspace instance and this doesn't matter
                         lib.metadata.invocation-tracker/invocation-tracker-provider)]
    (binding [qp.store/*DANGER-allow-replacing-metadata-provider* true]
      ;; this has no body so it looks like this is a no-op. But the with-metadata-provider sets the metadata provider and then
      ;; doesn't pop it. We could use the private function that this uses: qp.store/set-metadata-provider!, or we could use the
      ;; public function `(qp.store/store-miscellaneous-value! ::qp.store/metadata-provider remapping-mp)`. But the former is
      ;; private and the latter is too general. We want this usage to really stand out.
      (qp.store/with-metadata-provider remapping-mp))
```

```
❯ clj -X:dev:test:ee:ee-dev:drivers:drivers-dev :module enterprise/workspaces
,,,
487 assertions, 0 failures, 0 errors.
{:test 188, :pass 487, :fail 0, :error 0, :type :summary, :duration 37172.476209, :single-threaded 167, :parallel 21}
Ran 21 tests in parallel, 167 single-threaded.
Finding and running tests took 53.1 s.
All tests passed.
Running after-run hooks...
```

* GHY-3646: sweep orphan workspace iso resources on CI test boot

Per-test `finally` blocks destroy workspace iso schemas/users/roles/
datasets/SAs on a clean exit. If the CI job is killed (timeout, OOM,
runner-yanked) between try and finally, the resources orphan on the
shared cloud warehouse and accumulate. Cam called this a no-go for
merge; Bryan added "don't leak isolation users" too.

Extends the existing prefix-and-age scan pattern (already in Redshift +
half-done in Snowflake) to cover the full resource set for all three
shared-cloud drivers. Containerized drivers (Postgres / MySQL /
MariaDB / ClickHouse / SQL Server / H2) need no cleanup — their Docker
containers get torn down with the CI job.

Per-driver changes:

- **Snowflake**: extend `delete-old-test-data!` to also sweep iso
  USERS (`mb__isolation_*`) and ROLES (`MB_ISOLATION_ROLE_*`, note
  different prefix) older than 3h. `SHOW USERS / ROLES LIKE` returns
  `name + created_on`, filtered client-side.

- **Redshift**: extend `delete-old-schemas!` to also drop iso users
  paired with expired iso schemas. Schema-and-user names are identical
  strings by construction (`workspace-isolation-namespace-name` ==
  `workspace-isolation-user-name`), and schema is always created
  BEFORE user, so an orphan user without a paired schema is
  structurally impossible -- no separate age query needed.

- **BigQuery**: new iso dataset sweep (3h via INFORMATION_SCHEMA)
  + new iso SA sweep (3h via SA description-field marker). To support
  the SA age gate without IAM audit logs, the iso SA `description` now
  carries `created-at:<ISO-8601 instant>`. The writer
  (`ws-sa-description`) and reader (`ws-sa-description->created-at`)
  live as a lockstep pair in the production driver ns, with a
  round-trip contract test (`ws-sa-description-roundtrip-test`) that
  fails if either side changes format -- the marker is the only place
  the timestamp is stored, so silent drift would break cleanup.

Each driver's cleanup is split into pure enumerators
(`orphan-isolation-*`) and destructive droppers
(`drop-orphan-isolation-*!`) so REPL-iteration can preview without
side-effects:

    (#'sf-tx/orphan-isolation-users)
    ;; => ("mb__isolation_..." ...)

Per-entry try/catch inside each dropper -- one bad orphan never blocks
the rest.

Resolves GHY-3646. Unblocks one of three Cam merge gates on PR #73236.

* GHY-3489: skip mb__isolation_ schemas in sync

Workspace provisioning creates per-workspace isolation schemas in the
warehouse (`mb__isolation_*` prefix) so local instances can write
transform output without colliding with the canonical namespace stats
reads from. Today, a stats (parent) instance pointing at the same
warehouse with broad `schema-filters-patterns` picks up those isolation
schemas and surfaces them in browse / search / model picker. That's a
leak across the workspace boundary in the wrong direction -- local-side
state showing up on stats.

Add an unconditional skip in `metabase.driver.sync/include-schema?` --
the predicate every SQL-JDBC describe-database path already consults.
Workspace iso only exists for SQL-JDBC drivers (Postgres, Redshift, H2,
ClickHouse, MySQL, SQL Server, BigQuery), so the driver-level filter is
the tightest fit -- single fn, every caller benefits, no per-driver
opt-in needed.

The skip is layered on top of user-configured inclusion/exclusion
patterns: even a broad `mb*` inclusion pattern is overridden, but
customer schemas that happen to start with `mb_` (single underscore)
or `mb__` (different suffix) are NOT affected -- the check uses the
exact `mb__isolation_` prefix via `driver.util/workspace-isolated-schema?`.

Regression test pins all four cases:
- iso schemas skipped with no filters
- iso schemas skipped despite broad `mb*` inclusion
- non-iso `mb_*` schemas pass through
- exact prefix match required (no false positives on `mb_isolation_*`
  or `mb__customer_*`)

8/8 new assertions pass; 3/36 existing sync_test pass.

Resolves GHY-3489. Closes the third of three Cam merge gates on PR #73236.

* GHY-3489 followup: only skip iso schemas on PARENT instances

Previous commit d53ede9496 made `include-schema?` unconditionally skip
`mb__isolation_*` schemas. That broke the child-instance case: workspace
provisioning creates iso schemas precisely so the child can sync them
(with table remapping) -- without sync, workspace-aware queries have
nothing to rewrite against.

Gate on `(metabase.workspaces.core/workspace-mode?)`:
- false (parent / non-workspace EE / OSS): skip iso schemas. Stats's view
  of the warehouse stays clean.
- true (child instance with `:workspace` section in `config.yml`): sync
  iso schemas as before. Remapping middleware downstream handles the
  rewriting.

The `workspace-mode?` fn is a `defenterprise`; OSS fallback returns
false, so OSS instances default to "parent" behavior (correct -- OSS
can't run workspace mode).

Module-linter: `driver` module now `:uses workspaces`. Added with a
one-line comment justifying the dep -- `include-schema?` is the only
caller in the driver namespace and the dep is structural (parent-vs-child
behavior gate), not incidental.

Test changes:
- Renamed `workspace-isolation-schemas-are-always-excluded-test` ->
  `workspace-isolation-schemas-skipped-on-parent-instance-test`. The
  OSS-side `workspace-mode?` returns false, so the existing assertions
  remain valid as parent-instance simulation; no redefs needed.
- New `workspace-isolation-schemas-synced-on-child-instance-test`
  (^:synchronized): uses `with-redefs` to flip `workspace-mode?` to
  true and asserts iso schemas DO sync, that inclusion patterns still
  apply, and that user exclusion can still opt out per-schema.

4 tests / 39 assertions pass.

* review comments + improve docs

* more honest name

the deftype here in the middleware doesn't remap, it takes an `f` that
remaps. Bit more honest and less misleading.

* move table-mapping mp into lib

was fighting with kondo lints about using the non api namespaces for
cache and invocation providers. just tuck this generic provider into lib
metadata and call it a day.

query-platform team will be coming up with a way to get rid of the
[qp.store/*DANGER-allow-replacing-metadata-provider* true] and replacing
metadata provider soon. This should be cleaned up soon™

* fix lint override

* make transforming metadata provider a bit more generic

rather than mapping, let's just have it take an `f` of `(f
metadata-options collection)`.

bit more reusable and not just a table mapper.

* Remove metadata-file-import and disable-auto-sync

* m. Docstring simplification.

* move transforming metadata provider to own namespace

* m. whitespace linting error

* TransformingMetadataProvider: implement CachedMetadataProvider, intercept read paths

The previous impl wrapped the result in a fresh cached-metadata-provider +
invocation-tracker-provider, which threw away the parent's warmed cache --
every workspace query re-fetched everything from the appdb.

The new impl satisfies CachedMetadataProvider directly and delegates the
parent surface, intercepting only the read paths (`metadatas`,
`cached-metadatas`) to apply `f` on the way out. The parent's atom is
never mutated by the wrapper; the same parent can be wrapped, unwrapped,
or read directly with no pollution.

Other CachedMetadataProvider methods (`store-metadata!`, `cache-value!`,
`cached-value`, `clear-cache!`, `has-cache?`) delegate to the parent. The
general-cache slot is shared with the parent; values stored there
reference entities by id, so downstream re-resolution still goes through
the wrapper's `metadatas` and picks up the transform.

`has-cache?` delegating means qp.store/ensure-cached-metadata-provider
recognizes the wrapper as already-cached and doesn't slap another fresh
wrap on top, which would defeat the purpose.

* TransformingMetadataProvider: implement InvocationTracker

Previous commit dropped the `invocation-tracker-provider` wrap from the
factory without satisfying the protocol on the wrapper itself. QP's
`update-used-cards!` middleware calls `(invoked-ids mp :metadata/card)`
on the QP store's MP -- our wrapper threw:

  No implementation of method: :invoked-ids of protocol:
  #'metabase.lib.metadata.protocols/InvocationTracker found for class:
  metabase.lib.metadata.transforming_provider.TransformingMetadataProvider

Add an `InvocationTracker` impl that delegates to the parent when the
parent satisfies it, returning `nil` otherwise (no tracking, but no
crash).

* disable new linter for exceptional case and add context

* Address Cam review comments on TransformingMetadataProvider + BigQuery

- TransformingMetadataProvider: switch `deftype` -> `p/deftype+` so the type
  isn't redefined on every namespace reload (REPL ergonomics in long sessions).
- BigQuery `create-dataset!`: use the new array-type hint syntax
  `^com.google.cloud.bigquery.BigQuery$DatasetOption/1` instead of the
  legacy `^"[Lcom.google.cloud.bigquery.BigQuery$DatasetOption;"`.

* BigQuery: split workspace isolation into metabase.driver.bigquery-cloud-sdk.workspaces

Extract ~660 lines of service-account / IAM / dataset-ACL code into a
sibling namespace under the BigQuery driver module. The main
`bigquery-cloud-sdk` namespace loads it for side-effects so the four
driver multimethods (init / grant / check / destroy
-workspace-isolation!) register normally.

Also move `get-project-id` into `bigquery-cloud-sdk.common` -- it was
the only shared helper between the workspace block and the rest of the
driver. The local `defn-` is gone; all callers now go through the
common ns.

Test files updated to point at the new namespace
(`metabase.driver.bigquery-cloud-sdk.workspaces :as bigquery.ws`).

Verified locally:
- All affected namespaces load + reload cleanly.
- `(get-method driver/init-workspace-isolation! :bigquery-cloud-sdk)` etc.
  resolve.
- `ws-sa-description-roundtrip-test` passes (8 assertions, 0 errors).

Per Cam's review comment on PR #73236.

* BigQuery workspaces: revert array type hint to legacy form for Eastwood

Eastwood's bundled `tools.reader` predates Clojure 1.12 and rejects the
new `^Foo/1` array type hint syntax as a reader error. The hint was
applied per Cam's review on PR #73236 (commit 333918d62d) and traveled
into workspaces.clj when we split the BigQuery namespace.

Revert this one line to the legacy `^"[L...;"` form so Eastwood loads.
We can re-apply the new syntax once Eastwood's tools.reader is updated.

---------

Co-authored-by: Timothy Dean <timothy@timothydean.dev>
Co-authored-by: Bryan Maass <bryan.maass@gmail.com>
Co-authored-by: Ben Grabow <ben.grabow@gmail.com>
Co-authored-by: Alexander Polyankin <alexander.polyankin@metabase.com>
Co-authored-by: Eric Normand <ericwnormand@gmail.com>
Co-authored-by: dan sutton <dan@dpsutton.com>
2026-05-21 15:48:43 -04:00
Timofey Kachalov
5bfdd32be3 Prepare the MCP for ChatGPT app (#74435)
Co-authored-by: Phoomparin Mano <phoomparin@gmail.com>
Co-authored-by: Chris Truter <chris@metabase.com>
2026-05-21 23:23:34 +04:00
Eric Normand
7c7bbb9a95 Guard v60 dependency_status drop changesets against missing source columns (#74594)
Follow-up to #74463. That PR guarded the eight INSERT changesets
(T00:00:03–10) with columnExists/MARK_RAN, but left the dropIndex
(T00:00:11–16) and dropColumn (T00:00:17–24) changesets unguarded. On an
instance whose entity table never had dependency_analysis_version, the
INSERT now skips cleanly, the migration proceeds, and then fails on the
unguarded DROP — e.g. T00:00:23, ALTER TABLE segment DROP COLUMN
dependency_analysis_version, which errors because the column is absent.

Add the same columnExists precondition with onFail: MARK_RAN (plus
validCheckSum: ANY) to all fourteen drop changesets, so every changeset
that touches an entity table's dependency_analysis_version column is
guarded consistently.

The regression test previously ran only the single changeset T00:00:09;
it now runs the full file range T00:00:09–T00:00:24 against the
missing-column state and asserts both the INSERT and the DROP COLUMN are
recorded as MARK_RAN — which is what would have caught this gap.
2026-05-21 14:05:55 -05:00
Eric Normand
019b86391e Remove unused fingerprints (GHY-3636) (#74517)
Remove unused fingerprints

Drop four fingerprint properties with no production consumer:

- :weekday-distribution, :hour-distribution (temporal)
- :min-length, :max-length (text)

Also remove the helpers that only existed to feed the temporal
distributions (temporal->zoned, temporal->weekday, temporal->hour,
bucket-distribution) and the now-unused ChronoField import.

Schema entries removed; fingerprint_version is left alone, so existing
rows keep stale keys until re-fingerprinted -- nothing reads them, so
no functional impact.

Micro-benchmark over 100k synthetic rows: temporal fingerprinting
~17% faster (removing the 7- and 24-bucket reducers); number/text
unchanged or within noise.

GHY-3636
2026-05-21 12:09:57 -05:00
Sébastien
2344a40c70 Warn before converting embedded dashboard questions to SQL (#74494)
Co-authored-by: Jeff Bruemmer <jeff.bruemmer@gmail.com>
2026-05-21 18:33:49 +02:00
Sloan Sparger
0ea554edc0 [Metabot] Show message feedback UI in OSS (#74555)
* show feedback buttons in self-hosted instances

* clean up ishosted flag in setup fn

* clean up
2026-05-21 11:33:16 -05:00
Maksym Yakubych
482e74de55 fix: have metabot fix it button sometimes disappears (#74531)
Closes [BOT-1132: Have Metabot fix it button sometimes disappears on SQL errors](https://linear.app/metabase/issue/BOT-1132/have-metabot-fix-it-button-sometimes-disappears-on-sql-errors)

### Description

The button was only shown when the state was dirty. We need to always show it.

### How to verify

- Create a SQL query with an error
- run the query

### Checklist

- [x] Tests have been added/updated to cover changes in this PR
- [x] If adding new Loki tests: they pass [stress testing](https://github.com/metabase/metabase/actions/workflows/loki-stress-test-flake-fix.yml)
2026-05-21 13:20:12 -03:00
Maksym Yakubych
c1f058ba96 fix: changing db clears drafted inline sql prompt (#74515)
<!-- Added by 'Add Issue References to PR' GitHub Action. To disable linking, add 'no-auto-issue-links' label to your PR. --> closes #72372

<!-- Added by 'Add Issue References to PR' GitHub Action. To disable linking, add 'no-auto-issue-links' label to your PR. --> closes #72372

Closes [BOT-1291: Changing DB clears drafted inline SQL prompt](https://linear.app/metabase/issue/BOT-1291/changing-db-clears-drafted-inline-sql-prompt)

### Description

Changing the DB in the line SQL prompt was clearing and closing the prompt pop-up. Now it doesn't

### How to verify

- Open SQL query page.

- Press Command+Shift+I.
- Type something in the prompt window.
- Change the database to something else.

### Checklist

- [x] Tests have been added/updated to cover changes in this PR
- [x] If adding new Loki tests: they pass [stress testing](https://github.com/metabase/metabase/actions/workflows/loki-stress-test-flake-fix.yml)
2026-05-21 13:03:15 -03:00
Oleh K
6466a6c964 Disable rspack lazy compilation in dev mode (#74589)
Disable rspack lazy compilation in dev to fix broken visualizations

rspack-cli 2.0 auto-enables lazyCompilation for web-only targets. Its
trigger XHR uses a relative URL, so requests hit the backend (:3000)
instead of the dev server (:8080) and 500, leaving lazily-imported
modules uncompiled. Opt out explicitly to restore rspack 1.x behavior.
2026-05-21 15:45:41 +00:00
Nick Fitzpatrick
7d91a4fb45 add dev server for local development (#74588) 2026-05-21 15:19:44 +00:00
Andrei Chernykh
bc6269e364 Fix not-found state after permanently deleting a metric (#74522) 2026-05-21 08:19:28 -07:00
github-automation-metabase
627363c115 Auto-update documentation for master (#74552)
Update generated docs (master)

Co-authored-by: github-actions <github-actions@github.com>
2026-05-21 10:58:02 -04:00
Phoomparin Mano
4f18642557 Revise the MCP question card design and layout (#74332)
* add footer for feedback buttons and explore button

* conditional visualization height

* adjust footer text and padding

* add feedback screen takeover

* layout adjustments

* do not remap border color

* make header a plain 14px bold text

* show time granularity bar at the bottom

* add docstrings for feedback hook

* conditional layout for query explorer bar

* rename to time control bar

* refactor content view

* simplify layout of time control bar

* inline the time control bar

* indicate no whitelabel needed

* fix broken css style

* add hook test for mcp feedback

* simplify time control

* clarify time control height

* inline visualization type

* inline the visibility handler

* lift interfaces up

* fix missing types

* do not create two useMcpApp instances

* address code reviews

* remove border and background color from feedback header

* remove padding for tables per design suggestion

* allow switching back to original visualization

* border tweaks

* do not show viz type selector if nonsensical

* fix selectedChartType type error
2026-05-21 21:09:33 +07:00
Nick Fitzpatrick
8f2cf04842 Uxw 3359 visualizer breaks internal dimension remapping for integer (#74446)
* Fix visualizer dropping internal dimension remapping, make cards combineable

* Fix removing and re-adding integer-category dimensions in the visualizer

* PR Feedback, adjust helper function and variable names
2026-05-21 19:01:10 +05:30
Riley Thompson
61d3fa771f Postgres mbql5 compilation (#72573)
* add sql-mbql5 driver for mbql5 query compilation

* fix e2e aggregation test

* fix kondo, update preprocess doc string

* extract logic into helper fn

* add test for metabase#52611

* replace maybe->pmbql hack with driver multi method dispatch on legacy mbql vs mbql5 lib methods

* fix kondo and field->field-filter-clause-test

* fix substitute-field-filter-test-2 and native-with-param-options-different-than-tag-type-test-2

* require binning feature for aggregation-function-inside-expression-in-aggregation-test

* add mbql5 parent and fix field compilation

* compile value and order by clauses

* compile custom expressions

* remove pmbql

* fix postgres tests

* fix tests

* fix redshift ->honeysql

* fix server-side-relative-datetime-test

* fix aggregation-function-inside-expression-in-aggregation-test for athena with fixed bin width

* add more details in driver log

* fix bad merge

* update qp join test

* use test helper fn

* fix test helper

* Allow :postgres and :postgres-mbql5 drivers to co-exist.

* Update tests to double-run against :postgres and :postgres-mbql5.

* Separate out redshift vs redshift-mbql5 methods.

* Add placeholder test.data namespaces.

* A few more postgres driver fixes.

* Just test the mbql5 version for these two.

* Use snake_case for sql.qp/->honeysql [:postgres-mbql5 :value]

* Fix driver delegation

* Placate kondo

* Use the get-method resolution for postgres-mbql5 methods.

* Make uuid test mbql5-order aware.

* Wrong get-method override.

* keep h2 on mbql4, use get-method instead of manual re-implementation

* use get-method for redshift mbql5

* include postgres in mbql5 experiment

* dont test postgres-mbql5 directly

* fix tests

* fix postgres-mbql5 breakout

* fix canceled-query-no-stacktrace-test

* use lib to construct query

* fix race conditions

* fix test

* fix field->field-filter-clause-test

* fix hardcoded values in ^:parallel field->field-filter-clause-test

* implement postgres-mbql5 methods for action error handling

* self review

---------

Co-authored-by: Phil Hagelberg <phil@hagelb.org>
Co-authored-by: Tamás Benkő <tamas@metabase.com>
2026-05-21 09:29:40 -04:00
Uladzimir Havenchyk
c3b66e89df test: migrate jest test from babel to swc (#74403)
* tests: migrate jest to swc

* fix tests and circular dependency

* fix test

* fix typecheck

* review

* tweak config

* use correct swc plugin
2026-05-21 12:41:55 +00:00
Nicola Mometto
08854a40be fix: add metabase_version to serdes extract (#74476)
* fix: add metabase_version to serdes extract

* feat: silence version mismatch when version is not present

* tests

* the existing test that would've caught it was broken!

* ignore in existing tests
2026-05-21 13:33:33 +01:00
Fraser
e153ff5099 Upgrade rspack and swc (#74564) 2026-05-21 23:14:18 +12:00
Nemanja Glumac
90341cced2 Pin jest-canvas-mock (#74509) 2026-05-21 12:59:27 +02:00
Nick Fitzpatrick
519588bbfa fix: native query editor cuts off last lines when viewing transform sql (#74534) 2026-05-21 12:44:25 +05:30
Anderson
ce331be55f Fix permissions save action when a database was deleted recently (#74355)
* Fix stale database permission saves

* Address pr review
2026-05-20 21:58:23 -03:00
technomancy
1d4448d244 Drivers stability (#74553)
* Increase what counts as an "old" dataset in Snowflake to 5 days.

Deleting a dataset after only 2 days results in way too much churn and causes
instability in CI.

* Allow overall run pass even when bigquery driver tests fail.

We want to re-enable bigquery tests, but they are still unreliable. In order
to address their unreliability, we need to gather data about the failures. We
need the tests to run without impacting our ability to deliver, so we need to
not block the overall PR when they fail.

The bigquery driver tests were disabled over a month ago, so their logs are no
longer available in github or trunk.
2026-05-20 16:48:17 -07:00
Timothy S. Dean
80950071f0 Reproduce and fix deadlock issue with remote sync (#74544)
`load-metabase!` wrapped the entire import in a single transaction, holding row-level locks for the duration. On a live instance, concurrent writers (e.g., view-count batching, result-metadata updates) could deadlock with the import.

I split the transaction into per-entity transactions. Each `serdes/load-one!` call gets one. If an error is thrown, I attempt to check for known transient errors and retry with exponential backoff.
2026-05-20 22:52:57 +00:00
Timothy S. Dean
a563b90b27 Fix async flake: embedding-sdk-info-saves-query-execution-test (#74550)
The test polled for an async `QueryExecution` row with a 2-second timeout, which flaked for me under CI load. I bound `qp.util/*execute-async?*` to `false` so the write is synchronous, and removed the polling. This pattern is used elsewhere in the codebase.
2026-05-20 15:52:54 -06:00
Sloan Sparger
23d6dba5aa [AI Analytics] Make conversation list sortable by more fields (#74380)
* impl

* pr feedback
2026-05-20 21:42:24 +00:00
Nick Fitzpatrick
5f87f066ee Add description to auto sync toggle in remote sync setup (#74507)
* Add description to auto sync toggle in remote sync setup

* Update enterprise/frontend/src/metabase-enterprise/remote_sync/components/RemoteSyncAdminSettings/RemoteSyncSettingsForm.tsx

Co-authored-by: Jeff Bruemmer <jeff.bruemmer@gmail.com>

* adjust unit test

---------

Co-authored-by: Jeff Bruemmer <jeff.bruemmer@gmail.com>
2026-05-21 03:00:20 +05:30
Nick Fitzpatrick
7520bff5a0 Fix error for pivot tables with no dimensions (#74521) 2026-05-21 02:59:43 +05:30
Nick Fitzpatrick
39bc069291 Nick/gdgt 2101 data reference ts (#74267)
* Convert frontend/src/metabase/reference/utils to TypeScript (GDGT-2101)

* Convert frontend/src/metabase/reference/selectors to TypeScript (GDGT-2101)

* Convert frontend/src/metabase/reference/reference to TypeScript (GDGT-2101)

* Convert frontend/src/metabase/reference/components/Detail to TypeScript (GDGT-2101)

* Convert frontend/src/metabase/reference/components/EditableReferenceHeader to TypeScript (GDGT-2101)

* Convert frontend/src/metabase/reference/components/Field to TypeScript (GDGT-2101)

* Convert frontend/src/metabase/reference/components/FieldTypeDetail to TypeScript (GDGT-2101)

* Convert frontend/src/metabase/reference/databases/DatabaseList to TypeScript (GDGT-2101)

* Convert frontend/src/metabase/reference/databases/DatabaseDetail to TypeScript (GDGT-2101)

* Convert frontend/src/metabase/reference/databases/FieldList to TypeScript (GDGT-2101)

* Convert frontend/src/metabase/reference/databases/FieldDetail to TypeScript (GDGT-2101)

* Convert frontend/src/metabase/reference/databases/TableList to TypeScript (GDGT-2101)

* Convert frontend/src/metabase/reference/databases/TableDetail to TypeScript (GDGT-2101)

* Convert frontend/src/metabase/reference/databases/TableQuestions to TypeScript (GDGT-2101)

* Convert frontend/src/metabase/reference/databases/DatabaseListContainer to TypeScript (GDGT-2101)

* Convert frontend/src/metabase/reference/databases/DatabaseDetailContainer to TypeScript (GDGT-2101)

* Convert frontend/src/metabase/reference/databases/FieldListContainer to TypeScript (GDGT-2101)

* Convert frontend/src/metabase/reference/databases/FieldDetailContainer to TypeScript (GDGT-2101)

* Convert frontend/src/metabase/reference/databases/TableListContainer to TypeScript (GDGT-2101)

* Convert frontend/src/metabase/reference/databases/TableDetailContainer to TypeScript (GDGT-2101)

* Convert frontend/src/metabase/reference/databases/TableQuestionsContainer to TypeScript (GDGT-2101)

* Convert frontend/src/metabase/reference/segments/SegmentDetail to TypeScript (GDGT-2101)

* Convert frontend/src/metabase/reference/segments/SegmentFieldDetail to TypeScript (GDGT-2101)

* Convert frontend/src/metabase/reference/segments/SegmentFieldList to TypeScript (GDGT-2101)

* Convert frontend/src/metabase/reference/segments/SegmentQuestions to TypeScript (GDGT-2101)

* Convert frontend/src/metabase/reference/segments/SegmentRevisions to TypeScript (GDGT-2101)

* Convert frontend/src/metabase/reference/segments/SegmentDetailContainer to TypeScript (GDGT-2101)

* Convert frontend/src/metabase/reference/segments/SegmentFieldDetailContainer to TypeScript (GDGT-2101)

* Convert frontend/src/metabase/reference/segments/SegmentFieldListContainer to TypeScript (GDGT-2101)

* Convert frontend/src/metabase/reference/segments/SegmentListContainer to TypeScript (GDGT-2101)

* Convert frontend/src/metabase/reference/segments/SegmentQuestionsContainer to TypeScript (GDGT-2101)

* Convert frontend/src/metabase/reference/segments/SegmentRevisionsContainer to TypeScript (GDGT-2101)

* Refactor reference module types after TS conversion (GDGT-2101)

* removing unused props

* more adjustments

* clean up some casting

* Another cleanup pass

* clean up form types

* e2e adjustments

* getTable fix
2026-05-21 02:59:08 +05:30
Alex Yarosh
7bddd0b135 docs: fix minor typo in SDK Bundle description (#74541)
Fix minor typo in SDK Bundle description
2026-05-20 16:58:08 -04:00
Sloan Sparger
4d39883cd7 [AI Analytics] Date filters respect retention period (#74441)
* min date

* max date

* util clean up

* clean up

* improved code comment

* more unit tests
2026-05-20 20:28:05 +00:00
Fraser
936eb0a1c4 Pass dashboard Id to visualizer (#74465) 2026-05-21 08:14:05 +12:00
Sloan Sparger
e364754e4c [Metabot] Fix AI settings inputs flickering + losing focus (#74527)
* fix model dropdown flicker

* fix provider dropdown flicker

* fix api key input flicker

* fix connect/disconnect/done button flicker

* fix terms checkbox flicker

* drop unused isLoading from MetabotSetupContext

* consolidate BOT-1429 regression tests into one
2026-05-20 14:54:44 -05:00
Timothy S. Dean
2fe12d63ee Metadata File Import (#74524)
Adds a metadata file import path to enterprise serialization. It's a superuser-only HTTP endpoint that streams an exported warehouse-metadata JSON file directly into the app DB, matching existing Databases and creating/updating their Tables and Fields. This is designed to allow workspaces deployments to load warehouse metadata from disk, skipping expensive DB sync at scale. It's the import counterpoint to the existing `POST /metadata/export`.

Changes:
- Importer + endpoints, all streaming (to avoid OOMs) and heavily optimized for 1-1.4 GB JSON files, especially for postgres appdbs.
  - The endpoint requires Content-Type: application/octet-stream to bypass our usual JSON middleware.
- Sync gating
  - Sync cannot run while the file is being imported
  - You can disable auto-syncs entirely
2026-05-20 13:24:15 -06:00
Nick Fitzpatrick
81c537181d Fix setting graph.tooltip_columns warnings (#74237)
* Fix setting graph.tooltip_columns warnings

* create a new helper function for determining available columns for tooltips

* adjusting comment

* Update getReferencedColumns to be breakout aware
2026-05-21 00:12:02 +05:30
Nick Fitzpatrick
dcba346a3b Fix CodeMirror Suggestion hover background (#74505)
fix CodeMirror Suggestion hover background
2026-05-21 00:06:36 +05:30
Eric Normand
4809d20eae Guard v60 dependency_status data migrations against missing source columns (#74463)
* Guard v60 dependency_status data migrations against missing source columns

Changesets v60.2026-04-02T00:00:03 through T00:00:10 copy
dependency_analysis_version from eight entity tables into the new
dependency_status table; later changesets in the same file drop those
source columns. If an instance ever enters a partial-migration state
where T00:00:23 (DROP COLUMN segment.dependency_analysis_version) has
run but T00:00:09 (the corresponding INSERT) has not, init crashes on
the next upgrade with "column dependency_analysis_version does not
exist".

Add a columnExists precondition with onFail: MARK_RAN to each of the
eight INSERT changesets, mirroring the guard already present on the
v58 changeset that adds the column. validCheckSum: ANY is included as
a defensive measure following the precedent in 059_update_migrations.

Adds a schema migration test that reproduces the partial-migration
state on H2 and Postgres and asserts T00:00:09 is recorded as MARK_RAN.

* Fix test changelog table-name lookup for MySQL/MariaDB

The hard-coded "databasechangelog" identifier in the SELECT works on
H2 and Postgres but not MySQL/MariaDB, where Liquibase creates the
table as uppercase DATABASECHANGELOG. Use the cross-DB
liquibase/changelog-by-id helper instead.
2026-05-20 11:35:15 -05:00
technomancy
55426fbde0 Don't drop DB before creating in Snowflake tests. (#74464)
Snowflake seems to be the only driver which drops the database before
creating it. This is a problem because in the function
`metabase.test.data.impl.get-or-create/default-get-or-create-database!` we
call `load-dataset-data-if-needed!` which checks to see if the dataset is
loaded, and if it's not, it calls `create-db!`. If we drop the entire
database every time a single dataset needs to be reinserted, it will
obviously cause a lot of contention.
2026-05-20 09:09:38 -07:00
Brad Anderson
11971cdde8 GRO-370 Embedded metabot upsell (#74109) 2026-05-20 11:25:59 -04:00
Michael Eby
1b423d9eae docs: add docs for scatterplots and bubble chart (#74298)
* docs: add docs for scatterplots and bubble chart

* Add images and examples for additional data shapes

* Additional edits from Alex
2026-05-20 11:25:24 -04:00
Brad Anderson
fd43bd85e3 GRO-411 Corrects transform inspector upsell (#74227) 2026-05-20 11:18:12 -04:00
Sébastien
3922d12cb9 Fix excessive padding in embed wizard (#74488) 2026-05-20 16:53:55 +02:00
Alex Yarosh
fbde606476 docs: transform addons (#74379)
* first draft

* almost there

* transformssss

* Optimized 3 images, saved 147.8 KB (46.2% smaller)

* links

* more links

* Apply suggestions from code review

Co-authored-by: Jeff Bruemmer <jeff.bruemmer@gmail.com>

* Update docs/data-studio/transforms/jobs-and-runs.md

Co-authored-by: Jeff Bruemmer <jeff.bruemmer@gmail.com>

* feedback

* make the top more of a tldr

---------

Co-authored-by: Metabase Automation <github-automation@metabase.com>
Co-authored-by: Jeff Bruemmer <jeff.bruemmer@gmail.com>
2026-05-20 10:36:22 -04:00
Nick Fitzpatrick
280c1fc21a Add no writable databases state to transforms pages (#72880)
* Add no writable databases state to transforms pages

* add manage database permissions check

* only show no writable screen on DB write check, not permissions

* oss e2e adjustment

* handle error and loading states
2026-05-20 19:58:19 +05:30
Sloan Sparger
6c998f5997 [Metabot] Remove old ai_* token features (#74459)
remove old ai_* token features
2026-05-20 14:22:17 +00:00
Uladzimir Havenchyk
09b2415b60 chore: convert Map visualization to typescript (#70374) 2026-05-20 16:12:26 +03:00
Braden Shepherdson
32db915c8a [remote sync] add on-success callback, set git :version on revisions (#74031)
Fixes UXW-3916.

The revision was being written synchronously to *starting* the various
git operations for remote sync, but that means they had `:version nil`,
since the version is only populated once the remote ops are finished.

This change threads a callback function through the async process, and
only adds the revision once the remote operation is completed.
2026-05-20 12:45:04 +00:00
Ngoc Khuat
6f5f2eebaf Fix conditional formatting colors for BigDecimal cells in email renderer (#74475) 2026-05-20 12:14:14 +00:00
Romeo Van Snick
f8b8ebf695 Remove database entity (#74428)
* Route the deprecated database metadata actions through databaseApi

* Reroute use-database-(list-)query hooks through databaseApi

* Reroute admin database create/update/delete thunks through databaseApi

* Replace Databases.loadList/load HOCs with useListDatabasesQuery

* Migrate Databases entity selectors/actions to databaseApi + getMetadata

* Delete the Databases entity

* Use enityCompatibleQuery
2026-05-20 14:52:50 +03:00
Nemanja Glumac
46538ebaa1 Use the latest actions/checkout gh action (#74374) 2026-05-20 12:51:59 +02:00
Nicola Mometto
2e63806240 Add transform as a serdes dependency of table (#74473)
* fix: transform is a dep of table for serdes

* feat: add deser tests for transform tables
2026-05-20 17:27:16 +07:00
Nicola Mometto
6aaf9ed40a Allow serialization to import transform tables after sync (#74474)
fix: relax data source override none->transform during deser
2026-05-20 17:21:30 +07:00
Chris Truter
1e76db8e83 Data Complexity Score rating bands (#74271) 2026-05-20 10:30:44 +02:00
Nick Fitzpatrick
775c990ab3 Handle error response when reverting cards and dashboards (#74145)
* Handle error response when reverting cards and dashboards

* move toast logic into RevisionHistoryTimeline so that error catching applies more broadly to entities

* e2e adjustment
2026-05-20 13:47:57 +05:30
Nick Fitzpatrick
eb58589c76 Add viz settings to click object, apply column formatting to link template (#74137) 2026-05-20 13:30:56 +05:30
Nick Fitzpatrick
aefebb1bad update UX for SCIM page (#74081)
* update UX for SCIM page

* don't show scim token modal if generation fails
2026-05-20 13:30:33 +05:30
Mahatthana (Kelvin) Nomsawadi
83ae53f864 fix: avoid setState during render in SDK ComponentProvider (#74269)
The useInitPlugins hook dispatched setPluginsReady directly during
render, which schedules updates on other components subscribed to the
shared redux store and triggers React's "Cannot update a component
while rendering a different component" warning (e.g. on
GlobalSdkCssVariables). Move the dispatch into useEffect; plugin
registry initialization itself stays synchronous so children still see
plugins on first paint, and PublicComponentWrapper already gates
rendering on pluginsReady.

Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-20 05:18:46 +01:00
Anderson
71cbb7d163 Add migration to fix library root collections with legacy random entity ids (#74178)
* Fix legacy library root collections

* Use SQL migration for legacy Library entity IDs
2026-05-19 20:23:06 -03:00
Fraser
c24724e78f Add affected tests into run tests (#73358) 2026-05-20 09:45:43 +12:00
Eric Normand
36a1c834d2 Skip H2 databases during serdes extract (GHY-3633) (#74429)
* Skip H2 databases during serdes extract

The import path rejects H2 databases via assert-not-h2!, but the export
path included them, so a basic fresh-instance round trip failed
immediately on Sample Database / Internal Metabase Database. Filter
H2 out of extract-query "Database" so the export/import contract is
consistent.

Fixes GHY-3633.

* Gate H2-extract filter on dynamic var so H2-round-trip tests still work

The previous commit unconditionally filtered H2 from the Database
extract-query, which broke the v2.e2e / v2.load / v2.round-trip /
api-test suites that exercise H2 throughout (they already rebind
assert-not-h2! to allow it). Move the filter behind a dynamic var
*include-h2-in-extract?* that those fixtures can flip back on, and
give the API export test an explicit non-H2 database so it still
verifies the data-model export contains a Database entry.

* Drop H2-extract escape hatch from round-trip-test, force :postgres in a few setups

Round-trip-test now reads its baseline directly without bypassing
assert-not-h2! or flipping *include-h2-in-extract?* — the baseline DB
is :postgres, so neither hatch is needed. A handful of e2e/load
fixture databases that were defaulting to :h2 now declare :postgres
explicitly. The e2e and load test fixtures still need both redefs
because ~100 Card with-temps lean on the (data/id) test-data default
(H2); shrinking that surface is for another pass.
2026-05-19 20:50:17 +00:00
Riley Thompson
1a45642180 Fix impersonated model caching (#74303)
* fix impersonated model caching

* fix kondo
2026-05-19 16:41:17 -04:00
Anderson
258dbb310b Fix bar chart dimension click behavior on dashboards (#74094)
* Fix bar chart dimension click behavior on dashboards (#73448)

* Fix SDK internal click behavior filtering

* address Andrei's suggestion

* Fix dashboard click drill test
2026-05-19 16:31:06 -03:00
5874 changed files with 394350 additions and 263881 deletions

View File

@@ -22,7 +22,6 @@ You understand Metabase's driver architecture built on Clojure multimethods with
:bigquery-cloud-sdk, :databricks, :clickhouse, :athena, :sparksql,
:presto-jdbc / :starburst, :vertica, :sqlite, :h2
:mongo (non-SQL, custom protocol)
:druid (non-SQL, REST-based)
```
Key driver extension points you know intimately:

View File

@@ -1 +0,0 @@
Fix flakey test $ARGUMENTS

View File

@@ -44,6 +44,22 @@ This guide covers Clojure and ClojureScript coding conventions for Metabase. See
- Lines ≤ 120 characters
- No blank non-comment lines within definition forms (except pairwise `let`/`cond`)
**Java interop: type-hint parameters in the arglist; prefer `ex-*` helpers:**
```clojure
;; bad — reflection warning, calls Throwable methods directly
(defn root-cause [e]
(if-let [cause (.getCause e)] (recur cause) e))
;; good
(defn- root-cause [^Throwable e]
(if-let [cause (ex-cause e)]
(recur cause)
e))
```
Use `ex-cause`/`ex-message`/`ex-data`/`ex-info` rather than `.getCause`/`.getMessage`/`(.getData ^ExceptionInfo …)` where possible.
## Style Conventions
**Keywords and Metadata:**
@@ -51,6 +67,37 @@ This guide covers Clojure and ClojureScript coding conventions for Metabase. See
- Prefer namespaced keywords for internal use: `:query-type/normal` not `:normal`
- Tag variables with `:arglists` metadata if they're functions but wouldn't otherwise have it
**Time/date: `java-time.api :as t` and `u.date`, not `java.time.*` direct interop:**
```clojure
;; bad — Claude default, requires java.time interop
(:import (java.time LocalDate))
... (LocalDate/now) ...
;; good
(:require
[java-time.api :as t]
[metabase.util.date-2 :as u.date])
... (t/local-date) ...
... (u.date/parse "2024-01-01") ...
```
Only `(:import java.time.X)` when interop with a Java method genuinely needs the class.
**Check non-emptiness with `seq`, not `count`:**
```clojure
;; bad — `count` realizes the whole seq into memory just to test emptiness
(let [prompts (map build-prompt items)]
(when (pos? (count prompts))
(t2/insert! :model/Prompt prompts)))
;; good — `seq` realizes one cell; `insert!` streams the rest
(let [prompts (map build-prompt items)]
(when (seq prompts)
(t2/insert! :model/Prompt prompts)))
```
## Tests
**Organization:**
@@ -62,6 +109,33 @@ This guide covers Clojure and ClojureScript coding conventions for Metabase. See
- Mark pure function tests `^:parallel`
**Redefining `defn`-vars under test:**
Use `mt/with-dynamic-fn-redefs` instead of `with-redefs` when stubbing `defn`/`defn-` vars — plain `with-redefs` is not parallel-test-safe and a clj-kondo hook (`:metabase/prefer-with-dynamic-fn-redefs`, see `.clj-kondo/config.edn`) will warn. `with-redefs` is still fine for `def` / dynamic var rebinding.
```clojure
;; bad — flagged by the linter, not safe under :parallel tests
(with-redefs [my.ns/some-fn (fn [_] :stub)] (do-test))
;; good
(mt/with-dynamic-fn-redefs [my.ns/some-fn (fn [_] :stub)] (do-test))
```
**Async tests: no bare `Thread/sleep` — use `deref` with a timeout sentinel:**
A fixed sleep is the #1 source of CI flake. Tests should fail fast and loud when a contract is violated, not hang or sleep through the bug.
```clojure
;; bad — flaky on slow CI, silent on actual failure
(Thread/sleep 300)
(is (= :expected @seen))
;; good — fails fast with a clear message
(let [v (deref some-promise 1000 ::timeout)]
(is (not= ::timeout v) "promise never delivered")
(is (= :expected v)))
```
## Modules
**OSS Modules:**
@@ -86,6 +160,45 @@ This guide covers Clojure and ClojureScript coding conventions for Metabase. See
- Do not cheat module linters with `:clj-kondo/ignore [:metabase/modules]`
**Enterprise/OSS split: use `defenterprise`, not protocols or runtime feature checks:**
The OSS namespace defines the fallback body and names the EE namespace; the EE namespace overrides with the same fn name and a `:feature` clause naming the premium-features flag that gates the override. Both files use the same docstring shape — describe behavior on both sides.
```clojure
;; src/metabase/remote_sync/core.clj (OSS)
(defenterprise collection-editable?
"Returns whether remote-synced collections are editable.
Always true on OSS; EE consults the remote-sync settings."
metabase-enterprise.remote-sync.core
[_collection]
true)
;; enterprise/backend/src/metabase_enterprise/remote_sync/core.clj (EE)
(defenterprise collection-editable?
"EE override — respects the synced-collections-editable? setting."
:feature :remote-sync
[collection]
(or (synced-collections-editable?) (not (synced? collection))))
```
`defenterprise-schema` is the malli-schema variant. Required from `[metabase.premium-features.core :refer [defenterprise defenterprise-schema]]`. The OSS body is what runs when the feature flag is off — never write `(when (premium/has-feature? ...) ...)` in user code; let `defenterprise` route.
**Security gates fail *open* under `:feature` — use `:feature :none` and check manually.** The router (`dynamic-ee-oss-fn`) calls the EE body only when both the EE namespace is loaded *and* the token currently grants the feature; otherwise it falls back to the OSS body. For a permissive default (`collection-editable?` → `true`) that's fine. But for a *restriction* — sandboxing, impersonation, row-level filtering — the OSS fallback is the unrestricted answer, so a temporarily lost/expired token silently downgrades a sandboxed user to full access. **Fail open.** When the EE override is what *decides whether to restrict*, gate it with `:feature :none` (which always routes to the EE body once EE code is loaded) and check the feature explicitly inside, so token loss fails *closed*:
```clojure
;; enterprise/.../sandbox/api/util.clj — the gate that detects whether a user is sandboxed
(defenterprise enforced-sandboxes-for-user
"..."
;; :none so we can decide whether sandboxing is *configured* even when the feature
;; isn't currently available — letting us BLOCK requests that would otherwise be
;; sandboxed, instead of returning the permissive OSS fallback. Fail closed.
:feature :none
[user-id]
...)
```
Reach for this only for the detection/enforcement decision point. Downstream functions that merely *act* on an already-made decision can keep their normal `:feature :sandboxes` gate.
## REST API Endpoints
**Required Elements:**
@@ -94,6 +207,20 @@ This guide covers Clojure and ClojureScript coding conventions for Metabase. See
- All endpoints need Malli schemas for parameters (detailed and complete)
- All new REST API endpoints MUST HAVE TESTS
**Endpoint shape: `api.macros/defendpoint`, four positional arglist:**
```clojure
(api.macros/defendpoint :get "/:key/callback"
"OIDC callback for a specific provider."
[{provider-key :key} :- [:map [:key ProviderKey]]
_query-params
_body
request]
(oidc-integration/sso-callback provider-key request))
```
The arglist is **always 4 positional bindings** in this order: `route-params`, `query-params`, `body`, `request`. Use `_`-prefixed names for unused ones. Each one gets a Malli schema via `:- <schema>` (omit when `_`). The HTTP verb is a keyword (`:get`/`:post`/`:put`/`:delete`), not a symbol. Required as `[metabase.api.macros :as api.macros]`.
**Naming Conventions:**
- Query parameters use kebab-case
@@ -105,6 +232,20 @@ This guide covers Clojure and ClojureScript coding conventions for Metabase. See
- `GET` endpoints should not have side effects (except analytics)
- `defendpoint` forms should be small wrappers around Toucan model code
**Closed response maps:**
Tag response Malli `:map` schemas with `{:closed true}` so the generated OpenAPI / TypeScript types reject extra keys. Open maps emit `additionalProperties: true`, which produces weak frontend types and makes discriminated unions ineffective. Tighten domain types in the same pass (`pos-int?` rather than `:int` when zero branches elsewhere).
```clojure
;; weak
{:status 200 :body [:map [:status [:= :generated]] [:prompt_count :int]]}
;; tight
{:status 200 :body [:map {:closed true}
[:status [:= :generated]]
[:prompt_count pos-int?]]}
```
## MBQL (Metabase Query Language)
**Restrictions:**
@@ -144,10 +285,215 @@ This guide covers Clojure and ClojureScript coding conventions for Metabase. See
**Linter Suppressions:**
- Use proper format for kondo suppressions
- No `#_:clj-kondo/ignore` (keyword form)
The right shape is a **reader-discard map** placed directly above the form being silenced, listing the rule keywords explicitly:
```clojure
#_{:clj-kondo/ignore [:metabase/validate-defendpoint-route-uses-kebab-case
:metabase/validate-defendpoint-has-response-schema]}
(api.macros/defendpoint :post "/api_key" ...)
```
For a single require entry, hang it as metadata on that entry:
```clojure
(:require
^{:clj-kondo/ignore [:deprecated-namespace :discouraged-namespace]}
[metabase.query-processor.store :as qp.store])
```
Common Metabase-specific rules you'll see: `:metabase/modules`, `:metabase/validate-defendpoint-route-uses-kebab-case`, `:metabase/validate-defendpoint-has-response-schema`, `:metabase/prefer-with-dynamic-fn-redefs`, `:deprecated-namespace`, `:discouraged-namespace`. Don't use the keyword form (`#_:clj-kondo/ignore`); always the map form so it's grep-able by which rule(s) are suppressed.
**Configurable Options:**
- Don't define configurable options that can only be set with environment variables
- Use `:internal` `defsetting` instead
## Metabase Helpers — Quick Reference
Metabase and its libraries have **many** helper functions to make code more readable. Don't hand-roll things that already exist! Reach for the right helper *the first time*.
Organized by **problem you're solving**, not by namespace.
## "I wrote this — use that instead" cheat sheet
*Authors: prefer the right column. Reviewers: flag the left column in PRs.*
| You wrote | Reach for | When NOT to |
| -------------------------------------------------------------------- | ------------------------------------ | ----------------------------------------------------------------- |
| `(vec (keep f xs))` | `(u/keepv f xs)` | — |
| `(let [x (...)] (side-effect!) x)` | `(u/prog1 (...) (side-effect!))` | — (`<>` anaphor binds the result) |
| `(into {} (map (juxt k identity)) xs)` | `(u/index-by k xs)` | — |
| `(into {} (map (juxt k vf)) xs)` | `(u/index-by k vf xs)` | — |
| `#(compare %2 %1)` | `u/reverse-compare` | — |
| `(:col (t2/select-one [Model :col] …))` | `(t2/select-one-fn :col Model …)` | — |
| `(or (:k m) default)` | `(:k m default)` | **The value can be present-but-nil** (hydrated DB null, etc.) |
| `(let [x (...)] (if x then else))` | `(if-let [x (...)] then else)` | — |
| `(filter #(= :a (:type %)) coll)` + same for `:b` … | `(group-by :type coll)` then dispatch | — |
| `(->> xs (map f) (remove g) (mapv h))` | `(into [] (comp (map f) (remove g) (map h)) xs)` | when readability of the threading wins |
| nested `let` / `if` / `if-let` | `b/cond` with `:let` | a 2-level nest may be clearer as-is |
| `(let [{:keys [k]} arg] …)` as first form of body | destructure in the arg list | when the destructure also references something computed in `let` |
| `(or (:k m) (get m (name k)))` per field | normalize once: `(update-keys m keyword)` then plain `:k` | when you genuinely need both shapes per-field |
## `metabase.util` (commonly aliased `[metabase.util :as u]`)
Sequences / collections:
- `(u/keepv f coll)` — like `keep` but returns a vector (eager).
- `(u/index-by kf coll)` / `(u/index-by kf vf coll)` — `{(kf x) (vf x)}` map.
Strings (locale-safe):
- `(u/lower-case-en s)`, `(u/upper-case-en s)` — DO use these instead of `clojure.string/lower-case`/`upper-case` whenever the result feeds into matching/comparison; the locale-default versions break on Turkish `i`.
- `(u/qualified-name x)` — keyword → `"ns/name"` or `"name"`; symbol → `(name x)`; string → identity.
Control flow:
- `(u/prog1 first-form & body)` — anaphoric. Result of `first-form` is bound to `<>`. Returns first-form's value. Use for "do a thing, side-effect using its result, return the thing."
- `(u/the-id x)` — get an entity id from an int or an instance.
Comparators:
- `(u/reverse-compare x y)` — `(compare y x)`. Pass as a sort comparator: `(sort-by f u/reverse-compare coll)`.
Other:
- `(u/conflicting-keys m1 m2)` — keys present in both with different values.
The full `metabase.util.cljc` is ~1300 lines — when in doubt, grep it.
## `metabase.util.namespaces` (aliased `shared.ns`)
- `(shared.ns/import-fns [other-ns f1 f2] [another-ns f3])` — re-export defns from another ns through the current ns. The `:require [other-ns :as other-ns]` alias **must stay** — clj-kondo may flag it as unused but it's needed at macro expansion. Add a `;; needed for import-fns` comment so it doesn't get auto-stripped.
## Toucan 2 (aliased `[toucan2.core :as t2]`)
Single-field reads (avoids fetching whole rows):
- `(t2/select-one-fn :col Model :id 123)` — projection-friendly: when `f` is a keyword, only that column is fetched. Use instead of `(:col (t2/select-one [Model :col] …))`.
- `(t2/select-fn-set f Model & conditions)` — set of `(f row)` values.
- `(t2/select-fn-vec f Model & conditions)` — vector of `(f row)` values.
- `(t2/select-pks-set Model & conditions)` — set of PKs.
Existence / counts:
- `(t2/exists? Model & conditions)` → boolean.
- `(t2/count Model & conditions)` → long.
Writes:
- `(t2/insert-returning-instances! Model row-or-rows)` — returns the rows with PKs set.
- `(t2/insert-returning-pks! Model row-or-rows)` — just the PKs.
- `(t2/update! Model id-or-conditions changes)`.
- `(t2/delete! Model id-or-conditions)`.
Transactions:
- `(t2/with-transaction [t-conn] body…)` — opens a txn on the app DB conn.
Hydration:
- `(t2/hydrate instances :key1 :key2)` — invokes `:batched-hydrate` methods.
- Define via `(methodical/defmethod t2/batched-hydrate [:model/Foo :the-key] [_ k rows] …)`.
Define a model:
- `(methodical/defmethod t2/table-name :model/Foo [_] :foo_table)`
- `(t2/deftransforms :model/Foo {:col mi/transform-json})` — common: `transform-json`, `transform-secret-value` (encrypts), `transform-keyword`.
## Better-cond (`[better-cond.core :as b]`)
`(b/cond …)` like `cond` but supports interleaved `:let [bindings]` and `:when` clauses. Lets you flatten cascading `let`/`if`/`if-let`/`let` into a linear spine.
```clojure
(b/cond
:let [thread (load-thread id)]
(nil? thread) :not-found
:let [exploration (load-exploration (:exploration-id thread))
reason (gate-check exploration)]
reason
(skip! reason)
:let [pool (build-pool thread)]
(empty? pool) :skip-no-data
:else
(run! thread pool))
```
Use it when a function would otherwise be a staircase of nested forms. Don't reach for it for a single `if`/`let`.
## Schemas — `mr/def` for shared, inline for single-use
When a schema is used in 2+ places or has a name worth documenting, extract it into the registry via `mr/def`:
```clojure
(:require [metabase.util.malli.registry :as mr])
(mr/def ::options
"Options for Pulse channel rendering."
[:map
[:channel.render/include-buttons? {:description "default: false" :optional true} :boolean]])
```
`mr/def` keys are always `::namespaced` keywords; downstream callers reference via the fully-qualified keyword. Single-use schemas stay inline in the `mu/defn`'s `:-` annotation — no need to extract.
## Malli — `metabase.util.malli`
- `(mu/defn my-fn :- :string [x :- :int, y :- [:maybe :string]] …)` — inline schemas; runtime validation in dev, no-op in prod by default.
- `(mu/disable-enforcement & body)` — wrap test/perf-sensitive blocks.
- Common malli schemas: `:int`, `:string`, `:boolean`, `:keyword`, `:map`, `[:maybe T]`, `[:sequential T]`, `[:enum :a :b]`, `[:map [:k T]]`, `pos-int?`, `nat-int?`.
## i18n — `[metabase.util.i18n :refer [tru deferred-tru]]`
- `(tru "Hello {0}" name)` — translate at call time. Use in request-handling code.
- `(deferred-tru "…")` — translate at read time. Use in static contexts: `defsetting :description`, `defn` metadata `:description`, top-level `def`s of user-visible labels.
- ICU MessageFormat: doubled apostrophe for a literal: `(tru "it''s ready")`. Numbered placeholders: `{0}`, `{1}`.
- **Don't translate** developer-only debug strings (REPL helper text, internal logs).
## Logging — `[metabase.util.log :as log]`
- Levels: `trace`, `debug`, `info`, `warn`, `error`.
- Two flavors per level:
- `(log/warn "msg")` or `(log/warn e "msg")` — exception first if present.
- `(log/warnf "fmt %s" arg)` or `(log/warnf e "fmt %s" arg)` — printf-style.
- Don't `(log/warn (format "fmt %s" arg))` — use `warnf`.
## Map / key normalization
When parsing loosely-typed external data (LLM JSON, request bodies) that may have **either** string or keyword keys:
- `(update-keys m keyword)` — top-level keys → keywords (Clojure 1.11+, in `clojure.core`).
- `(clojure.walk/keywordize-keys m)` — recursive (use sparingly; can over-keywordize value strings if they happen to be in nested maps).
- After normalizing once, downstream code uses plain `(:k m)` — don't write `(or (:k m) (get m (name k)))` at every use site.
Other useful map ops:
- `(update-vals m f)` — Clojure 1.11+, in `clojure.core`.
- `(medley.core/map-vals f m)`, `(m/map-keys f m)` — older alternatives. Medley pinned at **1.4** in this repo — `m/index-of` and some other newer fns are NOT available.
## Comparators & sorting
- `(sort-by f u/reverse-compare coll)` — descending.
- Vector comparator: `(sort-by (juxt :primary :secondary) coll)`.
- Use `(- x)` on a primary numeric key to invert: `(sort-by (juxt #(- (:score %)) :id) coll)` (handles ties on ascending `:id`).
## Anti-patterns to avoid
- **`(let [x (do-thing!)] (other-thing! x) x)`** — use `u/prog1`.
- **`(vec (keep …))`, `(vec (map …))`** — use `u/keepv` or a transducer `(into [] (comp …) coll)`.
- **Multiple `filter` passes over the same coll for different `:type`s** — one `(group-by :type coll)`.
- **Per-field string/keyword dual-lookup** — normalize once with `update-keys keyword`.
- **Reaching into another module's tables via raw `:left-join` SQL** — expose a function on the owning model namespace and call that.
- **Snake-case keys on a massaged in-memory map** — appdb-row only. Convert at the parse boundary (e.g., when a json response becomes an internal record).
- **`(double (:k m 0.0))` on a hydrated DB column** — present-but-nil NPEs. Use `(double (or (:k m) 0.0))`.
- **`defmulti` with every impl on one page** — extension story is a fiction. Use a private lookup map + a private fn.
- **`(pos? (count xs))` to test a lazy `xs` for non-emptiness** — `count` forces the whole seq into memory. Use `(seq xs)`. (Rebuilding the seq — `(map f xs)` twice — is the only thing that recomputes; a bound seq caches.)
- **Plain `with-redefs` on a `defn` var in a test** — not parallel-safe; the kondo hook will warn. Use `mt/with-dynamic-fn-redefs`.
- **`(Thread/sleep N)` in an async test** — flake source #1. `(deref p timeout-ms ::timeout)` and assert `not= ::timeout`.
- **Hand-rolled `(str/split host #":")` for origin/host comparison** — mis-handles IPv6. Use `mw.security/parse-url` (or `try-parse-url` for client-controlled input).
- **`(subs (pr-str huge) 0 N)` on a log path** — the multi-MB string is already allocated. Bind `*print-length*`/`*print-level*` around `pr-str` instead.
- **New retry loop / new "find this column" helper / new transform-source-type predicate** — search first. `metabase.util.retry/with-retry`, `metabase.lib.equality/find-matching-column`, and `metabase.transforms.feature-gating/any-transforms-enabled?` already exist (along with many others). Reviewers ask "isn't there already an X for this?" more than any single idiom.
- **Plain `defn` when the function has any real contract** — `mu/defn` (or `mu/defn-`) with `:- schema` on the return and each param is the house default; reserve plain `defn` for trivial dev/helper code.
- **`(:import (java.time …))` + `(LocalDate/now)`** in new code — use `[java-time.api :as t]` + `[metabase.util.date-2 :as u.date]`. Direct `java.time` interop only when a Java API genuinely needs the class.
- **Vanilla `defmethod`** — use `methodical/defmethod` (it takes a docstring; the codebase uses it uniformly).
- **`(when (premium/has-feature? :flag) …)` in code** — gate via `defenterprise` so OSS/EE bodies live in their respective namespaces; never sprinkle runtime feature checks.
- **No `(set! *warn-on-reflection* true)` after the `ns` form** in a new `.clj` file — every `.clj` source file in this repo has it (not `.cljc`).
- **`(ns … (:require [a :as a] [b :as b]))` on one line; `(:import java.io.File java.net.URI)` bare** — both wrong shapes. Requires one per line under `:require`, imports as `(Package Class)` lists.
- **Inventing a fresh `:as` alias for `metabase.util`, `metabase.util.malli`, `metabase.util.log`, `java-time.api`, etc.** — use the canonical alias (see the table in Style Conventions). Mixing aliases makes `grep`-based discovery worse.
## When in doubt
- Run `grep -n "defn " src/metabase/util.cljc | head -80` to scan helpers.
- Run `grep -rn "defmacro\|defn-? [a-z-]*\?\b" src/metabase/util.cljc | head` for predicates.
- Check `medley.core/` if it looks "collection-y" — but **Medley is pinned at 1.4** here; new fns aren't available.
- Search the codebase for an existing use of the pattern you're about to write — if no one else does it, there's usually a helper.

View File

@@ -0,0 +1,69 @@
# React, Redux, and Data-Layer Patterns
## API access and RTK Query
- **Use RTK Query for server data.** Add typed endpoints under `metabase/api` or `metabase-enterprise/api`; don't add ad-hoc `fetch`, service wrappers, or new thunks for ordinary API access.
- **Keep API files transport-focused.** Endpoint files should cover request/response typing, URLs, params, light response normalization, and cache tags — not feature business logic.
- **Use canonical API types.** Prefer `metabase-types/api`; fix or derive missing types instead of casting endpoint results. Refer to API endpoint implementation in Clojure when building new endpoint connections.
- **Tag caches deliberately.** Use `listTag`, `idTag`, `provide*Tags`, and `invalidateTags(error, tags)` where appropriate.
- **Guard missing query inputs.** Use `skipToken` or `skip` until all required inputs exist. Build query requirements where the query is created, not in a downstream render path.
- **Use lazy queries only for events.** Data needed to render a screen belongs in a regular query hook at the owner component/hook.
- **Handle mutations with `unwrap()`.** Prefer `await trigger(args).unwrap()` in submit handlers; otherwise branch on `result.error` immediately.
## Where data should live
- **Server data:** RTK Query cache, not duplicated Redux state.
- **Form state:** Form library or local component state, not global Redux.
- **Transient UI state:** Component state or narrow context, not store-wide flags.
- **Cross-route/global tasks:** Redux slice with public selectors.
- **Derived values:** Selectors or pure helpers, not stored copies of computed data. Memoized if identity is important.
## Redux state modeling
- **Avoid using Redux**. Most problems can be solved with local component state or RTK Query.
- **Use Redux only when needed.** Reach for Redux only when distant parts of the app need durable shared state.
- **Model async status explicitly.** Use a discriminated union or explicit `status`/`isLoading`/`error`; don't infer loaded or failed state from truthy/falsy `data`.
- **Store canonical state only.** Derive filtered lists, labels, permissions, booleans, and display variants in selectors.
- **Read through selectors.** Name selectors after what they return (`getCurrentTask`, `getHasPendingMutation`), not how they compute it.
- **Add missing public actions/selectors.** Don't reach into another feature's private state shape because an action or selector is missing.
- **Use `createSelector` for derived state.** Keep selectors pure, cheap, and stable by reference when consumers depend on identity.
## React hooks and components
- **`useMemo` computes render values.** Side effects belong in `useEffect`; event-driven changes belong in event handlers or mutation callbacks.
- **Keep effects narrow and idempotent.** Include all dependencies, guard inside the effect, and clean up timers/subscriptions. Complex logic should be split to multiple effects or custom hooks, also consider extracting functional pieces.
- **Hooks can be used to encapsulate complex logic.** For example, a hook can normalize data, handle form's state and validation, or a hook can manage a table's pagination and sorting. If component's logic becomes too complex, consider extracting a hook.
- **Hooks orchestrate behavior.** Custom hooks should return a small typed result object: data, loading/error state, and callbacks when relevant.
- **Don't mirror props/query data into state** unless the consumer can edit that local copy. Keep derived render data as such.
- **Use `useCallback` for stable identity needs, but deliberately.** Wrap callbacks passed to memoized children, dependency arrays, context values, or hooks that require stable identity — not every local handler.
- **Memoize object/array props only when identity matters.** Prefer simpler plain values otherwise.
- **Stories for presentational components**. Create Storybook stories for presentational components to test different states and props. Stories live next to the component.
## Performance
- **Optimize only where identity or scale matters.** Prefer clear code first, then add `memo`, `useMemo`, `useCallback`, selectors, virtualization, or split components when render cost, prop identity, or list size justifies it.
- **Keep context values stable and avoid frequent updates.** Values updated often should be avoided in context consumed by many components, or stored by reference with a stable context value and explicit subscriptions/selectors.
- **Read global state high in the tree.** Let parent components select context/Redux/API state, derive the view model, and pass narrow props top-down. But avoid passing props too deep (reconsider if more thatn 2 levels).
- **Avoid unstable props in hot paths.** Don't pass fresh object/array/function literals to memoized children; memoize them or move them outside render when identity matters.
- **Render large lists deliberately.** Use pagination, windowing/virtualization, stable keys, and row-level memoization for large or frequently updating lists.
- **Consider memoization for leaf components.** If the source of data updates often, but the component doesn't need to re-render, consider memoizing it.
## Loading, error, and empty states
- **Every async operation needs loading UI.** Use `LoadingAndErrorWrapper` or `DelayedLoadingAndErrorWrapper` for full-page/panel loading; use disabled controls and inline progress for local actions.
- **Every async operation needs an error branch.** Use wrappers or feature-specific alerts for query errors; use `getErrorMessage(error, fallback)` plus a toast/form error for mutation failures.
- **Empty is its own state.** Render no-results, empty-list, and permission-limited states intentionally; don't let them look like unfinished loading.
- **Combine multiple query states explicitly.** Example: `isLoading = a.isLoading || b.isLoading`; `error = a.error || b.error`.
- **Distinguish first load from refresh.** Use `isLoading` for first-load blockers and `isFetching` for background refresh indicators.
- **Test all async branches.** Cover loading, error, empty, and success states when adding or changing data access.
## Review checklist
- Server data uses a typed RTK Query endpoint with correct cache tags.
- Query inputs are guarded with `skipToken`/`skip` until ready.
- Global state is justified, minimal, and read through selectors.
- Context values are stable; frequently changing values don't force broad re-renders.
- Leaf components stay dumb and receive narrow props from parent containers.
- Loading, error, empty, and success branches are visible and tested.
- Hooks are pure during render; side effects are confined to effects or event handlers.
- Mutation failures are surfaced to people using the UI.

View File

@@ -2,8 +2,8 @@
- **Lint:** `bun run lint-eslint-pure`
- Run ESLint on the codebase
- **Format:** `bun run prettier`
- Format code using Prettier
- **Format:** `bun run format`
- Format code with oxfmt (`oxfmt --write`); use `bun run lint-format-pure` to check without writing
- **Type Check:** `bun run type-check-pure`
- Run TypeScript type checking

View File

@@ -25,7 +25,7 @@ Before starting, determine which mode to use:
### When to hand off instead of review
If the spec references an issue (`metabase#NNNNN`) and the user wants to **fix** flakiness or assess whether the test still reproduces the original bug, that's outside the review skill's scope. Point them at `/fix-flakey-test` (or the dedicated flake-fixing workflow), which knows to fetch the issue and the resolving PR's diff. The review skill stays focused on "is this test well-written and conformant" — it doesn't fetch external issue context by default.
If the spec references an issue (`metabase#NNNNN`) and the user wants to **fix** flakiness or assess whether the test still reproduces the original bug, that's outside the review skill's scope. Point them at the dedicated flake-fixing workflow, which knows to fetch the issue and the resolving PR's diff. The review skill stays focused on "is this test well-written and conformant" — it doesn't fetch external issue context by default.
## Review checklist

View File

@@ -20,7 +20,7 @@ For EACH styled component, go through every CSS property and ask: "Can this be a
**Properties that ARE style props** (use these, not CSS modules):
- `display` → `display` prop
- `color` → `c` prop (`c="brand"`, `c="text-primary"`)
- `color` → `c` prop (`c="core-brand"`, `c="text-primary"`)
- `background-color` → `bg` prop (`bg="background-primary"`)
- `font-size` → `fz` prop (`fz="md"`)
- `font-weight` → `fw` prop (`fw="bold"`)
@@ -149,19 +149,19 @@ Read the `.styled.tsx` file AND every component that imports from it. Understand
For each styled component, apply the Mantine-First Decision Gate above. Then determine the migration target:
| Emotion Pattern | Migration Target |
| ------------------------------------------------------------ | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `styled.div` with only layout/spacing/color | `Box`, `Flex`, `Stack`, or `Group` with style props. **NO CSS module needed.** |
| `styled.div` with flexbox column | `Stack` component with style props |
| `styled.div` with flexbox row | `Flex` or `Group` component with style props |
| `styled.span` / `styled.p` with color/weight/size | **`Text component="span"` with style props** (`c`, `fw`, `fz`). NO CSS module. |
| `styled.div` with hover/focus/pseudo-selectors | **Hybrid**: Mantine component with style props for expressible properties + CSS module class for pseudo-selectors only |
| `styled.div` with media queries (simple spacing/sizing) | **Responsive style props**: `p={{ base: "md", lg: "xl" }}`. NO CSS module. |
| `styled.div` with media queries (complex/non-spacing) | CSS module for the media query parts, style props for the rest |
| `styled.div` with animations/keyframes | CSS module for animation, style props for layout |
| `styled(SomeComponent)` with only color/spacing/flex | **Wrap in Mantine component** with style props: `<Box c="brand" flex="0 0 auto"><Icon /></Box>`, or pass style props if the component accepts them |
| `styled(SomeComponent)` with pseudo-selectors/complex styles | CSS module `className` on the component |
| Dynamic props `styled.div<{ isActive: boolean }>` | Mantine style props for simple toggles (`c={active ? "brand" : "text-primary"}`), `cx()` with CSS module classes for complex state combinations involving pseudo-selectors |
| Emotion Pattern | Migration Target |
| ------------------------------------------------------------ | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `styled.div` with only layout/spacing/color | `Box`, `Flex`, `Stack`, or `Group` with style props. **NO CSS module needed.** |
| `styled.div` with flexbox column | `Stack` component with style props |
| `styled.div` with flexbox row | `Flex` or `Group` component with style props |
| `styled.span` / `styled.p` with color/weight/size | **`Text component="span"` with style props** (`c`, `fw`, `fz`). NO CSS module. |
| `styled.div` with hover/focus/pseudo-selectors | **Hybrid**: Mantine component with style props for expressible properties + CSS module class for pseudo-selectors only |
| `styled.div` with media queries (simple spacing/sizing) | **Responsive style props**: `p={{ base: "md", lg: "xl" }}`. NO CSS module. |
| `styled.div` with media queries (complex/non-spacing) | CSS module for the media query parts, style props for the rest |
| `styled.div` with animations/keyframes | CSS module for animation, style props for layout |
| `styled(SomeComponent)` with only color/spacing/flex | **Wrap in Mantine component** with style props: `<Box c="core-brand" flex="0 0 auto"><Icon /></Box>`, or pass style props if the component accepts them |
| `styled(SomeComponent)` with pseudo-selectors/complex styles | CSS module `className` on the component |
| Dynamic props `styled.div<{ isActive: boolean }>` | Mantine style props for simple toggles (`c={active ? "core-brand" : "text-primary"}`), `cx()` with CSS module classes for complex state combinations involving pseudo-selectors |
### Step 3: Create CSS Module (if needed)
@@ -170,14 +170,14 @@ Create `ComponentName.module.css` alongside the component file:
```css
/* Use design system tokens — NEVER raw color/spacing values */
.root {
border: 1px solid var(--mb-color-border);
border: 1px solid var(--mb-color-border-neutral);
border-radius: var(--mantine-radius-md);
background-color: var(--mb-color-background-primary);
}
/* Conditional states as separate classes, combined with cx() */
.active {
background-color: var(--mb-color-brand);
background-color: var(--mb-color-core-brand);
color: var(--mb-color-text-primary-inverse);
}
@@ -191,7 +191,7 @@ Create `ComponentName.module.css` alongside the component file:
cursor: pointer;
&:hover {
color: var(--mb-color-brand);
color: var(--mb-color-core-brand);
background-color: var(--mb-color-background-hover);
}
}
@@ -257,7 +257,7 @@ Remove the old styled file entirely. Remove all imports of it from other files.
- Text: `"text-primary"`, `"text-secondary"`, `"text-tertiary"`
- Background: `"background-primary"`, `"background-secondary"`, `"background-hover"`
- Brand: `"brand"`, `"error"`, `"success"`, `"warning"`
- Brand: `"core-brand"`, `"error"`, `"success"`, `"warning"`
**Typography** (`fz`, `fw`, `lh`, `ta`, `ff`):
@@ -281,8 +281,8 @@ Remove the old styled file entirely. Remove all imports of it from other files.
- `--mb-color-text-primary`, `--mb-color-text-secondary`, `--mb-color-text-tertiary`
- `--mb-color-background-primary`, `--mb-color-background-secondary`, `--mb-color-background-hover`
- `--mb-color-border`, `--mb-color-border-strong`, `--mb-color-border-subtle`
- `--mb-color-brand`, `--mb-color-brand-hover`
- `--mb-color-border-neutral`, `--mb-color-border-neutral-strong`, `--mb-color-border-neutral-subtle`
- `--mb-color-core-brand`, `--mb-color-core-brand-hover`
- `--mb-color-error`, `--mb-color-success`, `--mb-color-warning`
- `--mb-color-shadow`, `--mb-color-focus`
- `--mb-color-accent0` through `--mb-color-accent7`
@@ -400,11 +400,11 @@ export const Item = styled.div<{ isSelected: boolean; disabled: boolean }>`
color: ${(props) =>
props.disabled ? color("text-tertiary") : color("text-primary")};
background-color: ${(props) =>
props.isSelected ? color("brand") : "transparent"};
props.isSelected ? color("core-brand") : "transparent"};
cursor: ${(props) => (props.disabled ? "default" : "pointer")};
&:hover {
color: ${(props) => !props.disabled && color("brand")};
color: ${(props) => !props.disabled && color("core-brand")};
}
`;
```
@@ -418,12 +418,12 @@ export const Item = styled.div<{ isSelected: boolean; disabled: boolean }>`
cursor: pointer;
&:hover {
color: var(--mb-color-brand);
color: var(--mb-color-core-brand);
}
}
.selected {
background-color: var(--mb-color-brand);
background-color: var(--mb-color-core-brand);
}
.disabled {
@@ -457,7 +457,7 @@ When a styled component wraps another component, first check if the styles can b
export const CardIcon = styled(Icon)`
display: block;
flex: 0 0 auto;
color: var(--mb-color-brand);
color: var(--mb-color-core-brand);
`;
export const CardTitle = styled(Ellipsified)`
@@ -471,7 +471,7 @@ export const CardTitle = styled(Ellipsified)`
**After — style props on Mantine wrapper (preferred when ALL properties are expressible):**
```tsx
<Box display="block" flex="0 0 auto" c="brand">
<Box display="block" flex="0 0 auto" c="core-brand">
<Icon {...icon} />
</Box>
<Box c="text-primary" fz="md" fw="bold" ml="md">
@@ -483,13 +483,13 @@ export const CardTitle = styled(Ellipsified)`
```css
.icon {
color: var(--mb-color-brand);
color: var(--mb-color-core-brand);
width: 1.5rem;
height: 1.5rem;
cursor: pointer;
&:hover {
color: var(--mb-color-brand-hover);
color: var(--mb-color-core-brand-hover);
}
}
```
@@ -570,12 +570,12 @@ export const Spinner = styled.div`
### Pattern 6: color() Function → CSS Variables
| Emotion (JS) | CSS Module | Mantine Style Prop |
| ----------------------- | ------------------------------------------------------------ | --------------------------- |
| `color("brand")` | `var(--mb-color-brand)` | `c="brand"` or `bg="brand"` |
| `color("text-primary")` | `var(--mb-color-text-primary)` | `c="text-primary"` |
| `color("border")` | `var(--mb-color-border)` | N/A (use CSS module) |
| `alpha("brand", 0.2)` | `color-mix(in srgb, var(--mb-color-brand), transparent 80%)` | N/A (use CSS module) |
| Emotion (JS) | CSS Module | Mantine Style Prop |
| -------------------------- | ----------------------------------------------------------------- | ------------------------------------- |
| `color("core-brand")` | `var(--mb-color-core-brand)` | `c="core-brand"` or `bg="core-brand"` |
| `color("text-primary")` | `var(--mb-color-text-primary)` | `c="text-primary"` |
| `color("border-neutral")` | `var(--mb-color-border-neutral)` | N/A (use CSS module) |
| `alpha("core-brand", 0.2)` | `color-mix(in srgb, var(--mb-color-core-brand), transparent 80%)` | N/A (use CSS module) |
### Pattern 7: Responsive Spacing/Sizing → Responsive Style Props (NOT CSS Media Queries)
@@ -627,13 +627,13 @@ export const Bar = styled.div<{ width: number }>`
If existing code already uses `CS` classes and you're not migrating that specific code, leave them alone. But when migrating Emotion → Mantine, replace with the proper alternative:
| Instead of `CS.*` | Use |
| ------------------- | ------------------------------------------------------- |
| `CS.overflowHidden` | CSS module: `overflow: hidden` |
| `CS.cursorPointer` | CSS module: `cursor: pointer` |
| `CS.flex1` | Style prop: `flex={1}` |
| `CS.flexNoShrink` | Style prop: `flex="0 0 auto"` |
| `CS.textBrandHover` | CSS module: `&:hover { color: var(--mb-color-brand); }` |
| Instead of `CS.*` | Use |
| ------------------- | ------------------------------------------------------------ |
| `CS.overflowHidden` | CSS module: `overflow: hidden` |
| `CS.cursorPointer` | CSS module: `cursor: pointer` |
| `CS.flex1` | Style prop: `flex={1}` |
| `CS.flexNoShrink` | Style prop: `flex="0 0 auto"` |
| `CS.textBrandHover` | CSS module: `&:hover { color: var(--mb-color-core-brand); }` |
### Pattern 10: Shared Emotion Styles Across Files → Shared CSS Module + cx()
@@ -759,7 +759,7 @@ import S from "./ComponentName.module.css"; // only if CSS module is needed
- [ ] The `.styled.tsx` file is deleted
- [ ] All imports of the deleted styled file are updated
- [ ] No static inline styles — `style={{ }}` used only for truly dynamic runtime values
- [ ] All colors use design tokens (`c="brand"`, `var(--mb-color-brand)`), not raw hex/rgb
- [ ] All colors use design tokens (`c="core-brand"`, `var(--mb-color-core-brand)`), not raw hex/rgb
- [ ] All spacing uses design tokens (`p="md"`, `var(--mantine-spacing-md)`), not arbitrary px values — hardcoded `rem`/`px` literals snapped to the nearest token (see mapping table above)
- [ ] All border-radius values use radius tokens (`var(--mantine-radius-md)`), not hardcoded `0.5rem`
- [ ] All font sizes use font-size tokens (`fz="md"`, `var(--mantine-font-size-md)`), not hardcoded `1rem`

View File

@@ -0,0 +1,244 @@
---
name: metabase-ui-component-from-figma
description: Update or build a Metabase design-system component (frontend/src/metabase/ui — Chip, Badge, Alert, Switch, etc.) to match a Figma spec described in a Linear issue. Use when a ticket asks to restyle an existing `metabase/ui` component or implement a new one from Figma. Covers: checking out the issue, mapping the component (usage/blast radius for an existing one, or scaffolding a new one), building a Storybook showcase matrix (before styling for existing components, after for new ones), styling from exact Figma tokens while iterating with the user, and committing + migrating call sites LAST.
---
# Metabase UI component from Figma
End-to-end workflow for bringing a `metabase/ui` (Mantine) component in line with a Figma design, driven by a Linear ticket.
The golden rule (existing components): **build the Storybook showcase before changing any component styles.** Build it while the old styles are still in place — so the team can see the current component at a glance and get accustomed to it before the migration, then watch it light up as you restyle. You end up with a visual-regression (Loki) candidate either way. A **new** component has nothing to render until it's built, so this rule doesn't apply to create mode — there the order flips: implement and style the component first, then build its showcase to present the result.
This is a frontend task — also load **typescript-write** (and **typescript-review** before handing off).
## Two modes
The ticket is one of two shapes; identify which up front, because Phase 2 differs:
- **Update an existing component** — the component already exists with call sites, either in `metabase/ui` *or* as a legacy component elsewhere (e.g. `frontend/src/metabase/common/components/`, possibly Emotion styled-components) that this ticket also migrates into `metabase/ui`. The risk is *regression* (call sites, behavior), so you map usage and lock behavior with tests before touching it.
- **Create a new component** — nothing equivalent exists anywhere yet. Rarer. The risk is *API design*, so you scaffold the component and its public surface first.
Phases 1, 2, 5 and 6 are shared. The middle two differ in **order**: update mode builds the showcase (Phase 3) *before* styling (Phase 4); create mode does the reverse — implement and style first, then build the showcase as the last step — because there's nothing to render until the component exists.
---
## Phase 1 — Check out the issue
1. Fetch the Linear issue with the Linear MCP `get_issue`.
2. Pull from it:
- **Figma link(s)** — extract the `node-id` (`?node-id=250-13588` → `250-13588`) and `fileKey` (the `/design/<fileKey>/…` segment).
- **`gitBranchName`** — switch to it if not already there.
- **The component's intended API** — variants, sizes, and states the design system supports for this component. This defines the axes of your showcase (Phase 3).
3. Decide the mode (see **Two modes** above): search for an existing component — in `metabase/ui` or as a legacy component elsewhere. Found → **update**; nothing equivalent anywhere → **create**. Confirm with the user when unsure.
4. **Verify a Figma desktop MCP is connected before doing the work** — this skill depends on it for exact tokens in Phase 4 (and styling can't proceed accurately without it). Do a quick read against the issue's node (e.g. `get_metadata`/`get_variable_defs`); if it errors with "nothing selected" or the server isn't connected, prompt the user to enable it (Switch to Dev Mode → MCP section in the right sidebar) and select the relevant layer before continuing.
## Phase 2 — Understand the component
### Mode: update an existing component
1. Locate the component dir (typically containing `Component.config.ts` or `Component.tsx`, `Component.module.css`, `Component.stories.tsx`, `index.ts`). A ticket may also be **migrating a legacy component** — if it still uses Emotion styled-components, reach for the **emotion-migrate** skill.
2. Map every usage across `frontend/src` and `e2e` (JSX `<Component`, sub-components like `Component.Group`, and wrapper components).
3. Write findings to **`local/<TICKET>-<component>-usage-findings.md`**, one entry per usage: code location (`file:line`), short context, and **non-standard usage** (excessive customization, inline styling, invalid/legacy props). Invalid props are a signal that call sites will need migrating in Phase 6. This doc **stays** — the user walks it at the end to verify each real usage still looks right.
4. State the blast radius plainly: how many call sites, where, and whether they're confined.
5. **Lock behavior with tests before changing anything — when warranted.** Only for components that have **custom interactive behavior** (their own event handlers, state, keyboard logic, controlled/uncontrolled wiring). Add unit tests covering that current API/behavior so a regression shows up as a failing test. **Skip tests for thin Mantine wrappers** — don't test built-in Mantine behavior, and a pure visual restyle is covered by the showcase/Loki, not unit tests.
### Mode: create a new component
1. Decide whether it wraps a Mantine primitive (most common — extend it via a `.config.ts` using `<Mantine>.extend({...})` and theme registration) or is bespoke. Prefer wrapping Mantine.
2. Scaffold the component dir + its public surface: `Component.tsx`/`Component.config.ts`, `Component.module.css`, `Component.stories.tsx`, `index.ts`, and wire the export through the group barrel and the `metabase/ui` barrel.
3. There are no call sites to map yet; instead note the intended usage from the ticket so the eventual showcase reflects real props.
4. **Order note:** for a new component, implement and style it (Phase 4) first, *then* build the showcase (Phase 3) to present the finished result — there's nothing to render until it exists. The phases below are written update-first; just run 4 before 3 in create mode.
## Phase 3 — Build the Storybook showcase
Goal: one story that shows **every state at a glance**, themeable, suitable as a Loki visual-regression test. **Update mode: build this before Phase 4, while the old styles are still in place. Create mode: do this after Phase 4**, once the component exists to render.
### Reusable showcase helpers
Use the shared, Storybook-only primitives in **`frontend/src/metabase/ui/stories/showcase/`**: `StoryShowcase` (titled panel), `StorySection`, `StoryRow`, and `StoryJsx` (monospace JSX with light syntax highlighting via `tokenizeJsx`). Extend the set only when a new pattern is genuinely reusable.
### Derive the matrix from the component's full state space
Enumerate **all** the states the component can be in — every supported variant, size, and interaction/selection state (from its props/types and the design-system API, not merely the subset the issue happens to mention) — then cross-reference with the Figma spec and build the matrix on that. Don't omit a supported state because the ticket didn't list it; don't invent an axis Figma doesn't show (e.g. a static component has no hover/pressed).
**Exclude theme from the matrix.** Figma usually models light/dark as a component *property* (because it doesn't use variable modes), but theme is **not** a real component prop in the app — it's global. So never make theme a matrix axis or column; drive light/dark with the global Storybook `theme` toggle instead (see below).
Typical layout (mirrors the Figma component sheet): a `StoryShowcase` title, then a single **CSS-grid matrix** — state labels down the left column, one column per axis-combination (e.g. `variant × size`), each column headed by the real JSX usage via `StoryJsx`. Use `gridTemplateColumns: \`<labelWidth> repeat(N, max-content)\`` with a `React.Fragment` per row.
### Theme: use the global toggle, render ONE copy
Render the showcase once and let Storybook's global `theme` control (defined in `.storybook/preview.tsx`) switch light/dark. Do **not** render light and dark side by side: Metabase bakes per-scheme hex into `--mantine-color-*` at `:root`, and Mantine color tokens follow the global scheme — a nested scheme wrapper won't re-scope them.
### Forcing hover/pressed for the matrix
Use **`storybook-addon-pseudo-states`** (registered in `.storybook/main.ts`); don't add `data-force-*` hooks to component CSS. Set `parameters.pseudo` to force states per cell. Two things make or break it:
1. **Aim the selector at the element the pseudo-class actually sits on**, which for a Mantine component is often an inner slot, not the root. A string/array value applies `.pseudo-<state>` to the matched element, so it must match that slot. Import the CSS-module class (`S.<Slot>`) to target it.
2. **Land your per-cell hook on the root via the factory's root-props slot** (e.g. `wrapperProps`), because Mantine forwards top-level `data-*`/rest props to an inner element, not the root. Check the component's `.d.ts` for the right slot.
The addon toolbar toggle keeps working independently (it forces every rule in the story).
### Example: a finalized matrix story
Illustrative — adapt the axes and states to the component you're working on. This is the shape the pieces above add up to (state labels on the left, one column per `variant × size`, forced hover/pressed via `parameters.pseudo`, the root hook on `wrapperProps`, controls scoped to the meaningful knobs):
```tsx
import type { StoryFn } from "@storybook/react";
import { Fragment } from "react";
import { Box, Chip, type ChipProps, Text } from "metabase/ui";
import { StoryJsx, StoryShowcase } from "metabase/ui/stories/showcase";
import S from "./Chip.module.css";
// Axes derived from the component's API (here: variant × size).
const COLUMNS = [
{ variant: "light", size: "sm" },
{ variant: "light", size: "md" },
{ variant: "filled", size: "sm" },
{ variant: "filled", size: "md" },
] as const;
// Every state the component supports; `id` doubles as the pseudo-state hook.
const STATES = [
{ id: "default", label: "Default" },
{ id: "hover", label: "Hover" },
{ id: "pressed", label: "Pressed" },
{ id: "selected", label: "Selected", checked: true },
{ id: "hover-selected", label: "Hover selected", checked: true },
{ id: "disabled", label: "Disabled", disabled: true },
];
// Force hover/pressed by targeting the slot the pseudo-class sits on (.ChipLabel).
const labelSelector = (id: string) => `[data-state-row="${id}"] .${S.ChipLabel}`;
const Overview: StoryFn<ChipProps> = ({ children }) => (
<StoryShowcase title="Chip">
<Box
style={{
display: "grid",
gridTemplateColumns: `9rem repeat(${COLUMNS.length}, max-content)`,
columnGap: "2rem",
rowGap: "1rem",
alignItems: "center",
}}
>
<div />
{COLUMNS.map(({ variant, size }) => (
<StoryJsx
key={`${variant}-${size}`}
>{`<Chip variant="${variant}" size="${size}" />`}</StoryJsx>
))}
{STATES.map((state) => (
<Fragment key={state.id}>
<Text size="sm" c="text-secondary">
{state.label}
</Text>
{COLUMNS.map(({ variant, size }) => (
<Chip
key={`${variant}-${size}`}
wrapperProps={{ "data-state-row": state.id }} // hook on the ROOT
variant={variant}
size={size}
checked={state.checked ?? false}
disabled={state.disabled ?? false}
onChange={() => {}}
>
{children}
</Chip>
))}
</Fragment>
))}
</Box>
</StoryShowcase>
);
export const OverviewStory = {
render: Overview,
parameters: {
pseudo: {
hover: ["hover", "hover-selected"].map(labelSelector),
active: ["pressed"].map(labelSelector),
},
controls: { include: ["children", "theme"] },
},
};
```
### Story conventions
- Keep an interactive default story plus the matrix story.
- Restrict `argTypes` to the supported API (e.g. only the sizes the DS uses).
- **Scope controls per story.** A static matrix should hide knobs it ignores: `parameters.controls = { include: [...] }` (or `{ disable: true }`). Wire any kept knob (e.g. `children`) through to the cells so it does something.
- `loki.config.js` has a `storiesFilter`; check it before renaming a story that's already covered.
Validate: `bun run lint-eslint-pure -- <files>` and `bun run type-check-pure`.
### Optional: verify in the browser yourself
If a browser-driving MCP is connected (e.g. **chrome-devtools-mcp** or **playwright-mcp**: `navigate_page`, `take_screenshot`, `take_snapshot`, `evaluate_script`, `resize_page`), use it to check your own work against the running Storybook (`http://localhost:6006/iframe.html?id=<story-id>`, the kebab-cased title + story). Screenshot for a visual read; `evaluate_script`/`take_snapshot` to confirm DOM facts (a hook landed on the root, pseudo classes applied, a resolved `--mb-color-*`). This is a **convenience, never a requirement** — without it, rely on the user's visual check (and you may suggest installing the plugin). Storybook is usually already running; don't start it. (Screenshot writes may be sandboxed to the repo root — save under `local/` then `mv`.)
Pause here and let the user look. Iterate on layout/spacing only — **no component style changes yet.**
## Phase 4 — Style the component from Figma
Now change the component's `.config.ts` / `.module.css`. Get **exact tokens from Figma** — never eyeball hex from a screenshot.
### Figma desktop MCP is required for tokens
`get_screenshot` works by `nodeId`, but `get_variable_defs`, `get_design_context`, and `get_metadata` read the **current selection in the Figma desktop app** via the **Dev Mode MCP server**. If they error with "nothing selected" or the server isn't connected, **prompt the user** to enable the Dev Mode MCP server (Switch to Dev Mode → MCP section in right sidebar) and select the relevant layer(s). Don't pixel-sample a screenshot as a substitute.
### Extract and map tokens
1. `get_metadata` on the spec frame for the node id + name of each variant/size/state symbol.
2. `get_variable_defs` per symbol for exact colors, spacing, radius, font.
3. `get_design_context` (forceCode) on one symbol for exact CSS dimensions (padding, gap, icon size). It may ask about Code Connect — skip that.
4. **Use the semantic token names `get_variable_defs` returns.** An up-to-date spec binds each property to a semantic token (a role like `text/hover`, `background/selected`); match it to the corresponding `--mb-color-*` key (key list: `frontend/src/metabase/ui/colors/types/color-keys.ts`) and use it directly — one definition then works in both themes. Sanity-check the resolved shade against `colors/constants/themes/light.ts` / `dark.ts` and `base-colors.ts`. Caveat: returned *values* may be the wrong mode for dark symbols — trust the variable **names**, not the dark hex.
### Color rules — semantic tokens only, NO `color-mix`, NO primitives
Per `docs/developers-guide/frontend.md` ("Colors"): every color must be a **semantic** `--mb-color-*` token. No literal hex, and no `color-mix` in component CSS to fake a shade/alpha.
**Guardrail:** if `get_variable_defs` returns a **primitive ramp** (e.g. `Orion-Alpha/10`, `Ocean/60` — a ramp + number, not a role) bound directly to a property, do **not** translate it to a ramp, hardcode hex, or `color-mix` your way to it. **Stop and flag it to the user** with the specific property/state — it's a design-side gap (bind to a semantic token, or add one). Resume once the spec exposes a semantic token.
### Non-color tokens — flag scale mismatches too
The same discipline applies to **dimensional** values — radius, spacing/padding/gap, elevation/shadow, and typography (font size, line-height, weight). Prefer the codebase's existing scale variable over a literal: `--mantine-radius-*`, `--mantine-spacing-*`, `--mantine-shadow-*`, and the Mantine font tokens.
But Figma's scales don't always line up with the codebase's. When a Figma value **has no matching step** in the corresponding scale — e.g. Figma `radius/md = 12px` while `--mantine-radius-md` is `8px`, or a Figma `xxxs` spacing step with no Mantine equivalent — **don't silently bake in a literal.** Flag it to the user the same way you flag primitive colors: name the property, the Figma value, and the nearest codebase token, so it can be resolved with design / the scale owners (align the value, or add a scale step). Only fall back to a literal if the user confirms it's an accepted one-off, and leave a short comment in the code explaining why.
### Mantine implementation notes (reference)
General gotchas when extending Mantine components — adapt per component:
- **Know which slot each data-attribute lands on.** Mantine puts state attributes on specific slots (e.g. `data-variant` on the root, `data-checked`/`data-disabled` on an inner slot). Scope rules accordingly, usually through a root `classNames.root` class.
- **Out-specify Mantine's built-in variant CSS.** It styles slots at `.m_*:not([data-disabled])` specificity; a descendant combinator through your root class wins. Avoid chained `:not()` (stylelint's `selector-not-notation: complex` rejects it, and combining lowers specificity) — gain specificity via the root class / attributes instead.
- **Size via a `vars` resolver**, returning only Mantine's typed vars for that component (custom var names fail type-check). Map size → vars in `.config.ts`.
- **Built-in icons use `currentColor`**, so they inherit the slot's text color.
- For text on a brand fill, follow the existing convention (`text-primary-inverse`) unless the user wants pure white; flag the dark-theme implication.
After edits: `npx stylelint <css>`, `LINT_CSS_MODULES=true bun run lint-eslint-pure -- <files>`, `bun run type-check-pure`.
## Phase 5 — Iterate with the user
Show the result in the running Storybook and address feedback (color, spacing, radius, shadow, missing/extra state). Surface deliberate deviations explicitly so the user can veto them. If a browser MCP is connected, screenshot the matrix in both themes to catch issues first. **Do not commit until the user is satisfied.**
## Phase 6 — Commit, then migrate call sites
Only after sign-off:
1. Commit when the user explicitly asks; commit only relevant files (exclude build artifacts and the gitignored `local/` doc). Follow the user's commit conventions.
2. **Update mode:** migrate the call sites flagged in Phase 2 if the restyle requires it (e.g. invalid/renamed props). Be surgical with bulk find/replace — a blanket prop rename can hit unrelated components; verify each hit is the target. **Create mode:** confirm the public export is wired through the barrels and used as the ticket intends.
3. Leave the findings doc in place for the user's manual verification pass.
---
## Checklist
- [ ] Linear issue fetched; Figma node/fileKey + `gitBranchName` extracted; on the right branch; mode (update vs create) decided.
- [ ] **Update:** usage + blast radius in `local/<TICKET>-<component>-usage-findings.md`; behavior locked with unit tests **if** the component has custom interactive logic (skip for thin Mantine wrappers / pure restyles). **Create:** component scaffolded with its public surface and barrel exports.
- [ ] Storybook showcase matrix built from the component's full state space (theme excluded — global toggle drives light/dark), single panel, hover/pressed forced via `storybook-addon-pseudo-states`, controls scoped per story; lint + type-check pass. (Update mode: built before styling, **component styles untouched**. Create mode: built after the component is implemented in Phase 4.)
- [ ] Figma desktop MCP available; exact tokens extracted and mapped to semantic `--mb-color-*` (NO `color-mix`, NO primitives) and to existing scale vars for radius/spacing/elevation/type — any color *or* dimensional value with no matching token/scale step flagged to the user, not baked in as a literal.
- [ ] Component styled; stylelint + eslint(+CSS modules) + type-check pass.
- [ ] Iterated with the user until satisfied.
- [ ] Committed (when asked); call sites migrated/verified (update) or export wired (create); findings doc handed back.

View File

@@ -1,20 +1,33 @@
---
name: typescript-review
description: Review TypeScript and JavaScript code changes for compliance with Metabase coding standards, style violations, and code quality issues. Use when reviewing pull requests or diffs containing TypeScript/JavaScript code.
allowed-tools: Read, Grep, Bash, Glob
allowed-tools: Read, Grep, Bash, Glob, Skill
---
# TypeScript/JavaScript Code Review Skill
@./../_shared/typescript-commands.md
@./../_shared/react-redux-patterns.md
## Code Review Guidelines
## Main Focus
Review pull requests with a focus on:
**Primary standard: the [`typescript-write`](../typescript-write/SKILL.md) skill.** Load it first — it defines the authoring rules this review enforces, alongside `frontend/CLAUDE.md` and `docs/developers-guide/frontend.md`.
- Readability and maintainability
- Appropriate test coverage
- Compliance with project coding standards and conventions
- Code quality and best practices
- Type safety and proper TypeScript usage
- React best practices (when applicable)
Adherence to `typescript-write` is the **highest-priority** review dimension: rank any violation of its provisions above all other findings. Treat its **no-`any` hard rule** (no explicit *or* implicit `any` in new code) as **blocking**, and verify it with the LSP rather than by eye.
Review in this priority order:
1. **Violations of [`typescript-write`](../typescript-write/SKILL.md) provisions** — no-`any`, type tightening, type modeling, null/undefined handling, naming, structure, comments. Highest priority; block on the no-`any` rule.
2. Compliance with `frontend/CLAUDE.md`.
3. Readability and maintainability.
4. Appropriate test coverage.
## Blind spots — act as the missing reviewer
These rarely surface in team reviews, so this skill should raise them. They are **additive** — raise them, but rank them below `typescript-write` violations:
- **Accessibility.** Interactive elements need keyboard support, focus management, and accessible names. Flag missing `aria-label`/`aria-labelledby`, non-semantic click targets, modals without focus trap, icon-only buttons without labels, and form inputs without a linked label.
- **Performance.** Flag areas that scale poorly and aren't memoized; inline object/array literals passed to memoized children; effects that fire on every batch of a progressive load; and new dependencies added to hot paths.
- **Security.** Evaluate potential security issues in new code.
- **Bundle size.** Flag new large dependencies, default imports from icon or util libs, and heavy modules imported at route-load time.
- **Analytics.** User-facing flows should emit tracking events. If a PR adds a new flow (button, modal, navigation) without a tracking event, ask whether one is expected.

View File

@@ -7,3 +7,77 @@ description: Write TypeScript and JavaScript code following Metabase coding stan
@./../_shared/development-workflow.md
@./../_shared/typescript-commands.md
@./../_shared/react-redux-patterns.md
## No `any` — hard rule
- **New code must not introduce `any`, explicit or implicit.** No `any` annotations, no `as any` / `as unknown as`, no untyped parameters or returns that infer `any`, no implicitly-`any` destructures or array/object literals.
- **Untyped third-party / boundary values** must be typed at the boundary (a declared type, `unknown` + type guard, or a small typed wrapper) — never let `any` propagate inward.
- **Mandatory LSP verification.** After writing or editing any TS/TSX, inspect the changed symbols with the TypeScript Language Server (hover to read inferred types; `goToDefinition` to confirm sources) and run the project type-check.
## Type tightening
- **Avoid type casts and loose `unknown`** — fix the signature instead.
- **If a function only needs one field of a wide object, accept that field** — not the wide object. The cast often disappears once the signature is right.
- **Reach for `Partial<T>`, `Pick<T, K>`, `Record<K, V>`, and generics** before reaching for a cast.
- **Prefer making props/components generic** (`<T>`) when a value flows through unchanged and the caller knows the type.
- **Prefer `unknown` over loose typing** and narrow before use — an `unknown` value forces a guard at the point of use.
- **`satisfies` for object literals** that must conform without widening (config objects, lookup maps, discriminated literals) — better than `: T` (widens) or `as T` (unsafe).
- **Avoid non-null assertions (`!`)**. Prefer a guard, early return, or `?.`. Use `!` only when non-nullness is provably true and localized, with a comment.
- **No redundant runtime coercion** — don't wrap already-typed values in `Number()` / `String()` / `Boolean()`.
- **Type guards belong in `frontend/src/metabase-types/guards/`**. Do not redefine them locally.
## Type modeling
- **Reuse existing types; don't re-declare them.** Use canonical IDs and domain entity types from `metabase-types/api` (`FieldId`, `TableId`, `ConcreteTableId`, `SchemaName`, …) and key data structures by them (`new Map<ConcreteTableId, …>()`). Don't duplicate generated/API types — compose or derive (`Pick`, `Omit`, indexed access `SomeType["field"]`, `ReturnType`).
- **Use generics to allow TypeScript to infer correct types** when creating functions and components that need to be reusable and type-safe. Don't hesitate to introduce complex generics if they allow to derive types automatically instead of manual narrowing.
- **Model the actual data contract; keep types narrow.** Optional `field?: T` for a key that may be absent, `field: T | undefined` only when the key is always present but the value may be undefined, `| null` for explicit API nulls. Prefer domain unions over broad `string` / `number` / loose `Record`.
- **Refer to API implementation** when defining or refining types to ensure they match the actual data structure. When considering a type cast, first consider if the type should be refined to match the actual data structure.
- **Discriminated unions for variant state, with exhaustive checks.** Model "one of N shapes" as a union with a literal discriminant rather than a bag of optional fields, and exhaust it with ts-pattern's `.exhaustive()` so adding a variant becomes a compile error:
```ts
import { match } from "ts-pattern";
const result = match(status)
.with({ type: "loading" }, () => <Spinner />)
.with({ type: "error", error: P.select() }, (error) => <Error message={error.message} />)
.with({ type: "success", data: P.select() }, (data) => <Content data={data} />)
.exhaustive(); // Compile-time guarantee all cases handled
```
- **Derive union types from constants** (`as const` + `typeof`/`keyof`) so the type and the values can't drift.
- **`readonly` / immutability where mutation isn't intended** — component props, shared constants, exported config. Prefer `readonly T[]` / `ReadonlyArray<T>` for inputs you don't mutate.
- **Type async and error states explicitly** (a discriminated union or the data-layer's typed result) — never leave loading/error/empty implicit.
## Null and undefined
- **Narrow at the source**. If a value is optional only in a corner case, don't thread `undefined` through every layer — guard at the producer.
- **Sensible defaults for optional values**. Use `?.` and `??` at the consumer.
- **Lists should be filtered** before being used in a map or other iteration.
- **Avoid non-strict null comparisons** (`X != null`) when `X` can never be `null` — use a strict check or narrow the type. Use `checkNotNull` where necessary.
- **Check actual nullability against API implementation**. Find the API endpoint implementation and check if the field can actually be null.
## Naming
- **Names describe the entity, not the mechanism**. A name must reflect what the value holds.
- **Align sibling concepts**: keep verb conventions consistent across a related API.
- **No names that encode implementation history** rather than current meaning. Suffixes like `Base`, `New`, `Old`, `Initial` need a real semantic distinction, otherwise drop them.
- **Avoid cryptic identifiers** (`v`, `n`, `$n`) for domain values; short names are fine only in tiny conventional contexts (loop index `i`, coordinates `x`/`y`, generic params `T`/`K`/`V`).
## Code structure and organization
- **Prioritize reusability over duplication**. The codebase already has many utility functions — leverage them. If you introduce duplicated logic, extract it to a shared utility.
- **Generic helpers do not belong in feature folders**. Promote to a shared utility.
- **Keep functions small and single-purpose**. A 100+ line function is hard to review — split into focused named helpers, each with one responsibility and a minimal dependency surface. When necessary, cover with unit tests.
- **Extract distinct complex JSX into named components**. Choose same-file vs separate-file by reuse, coupling, testability, and readability.
## Comments
- **No comments by default**. Well-named identifiers carry the `what`.
- **Comments should be concise**. Add a short, concise comment only when the `why` is non-obvious: a workaround, a hidden invariant, a subtle ordering constraint, a clever reduction. Never document the actual implementation, focus on the intent and the why.
## TypeScript Migration
**When touching existing JavaScript files, propose to convert them to TypeScript first**. Create a separate PR for the conversion, then implement the changes.
## Verify before done
- **Run the project type-check** when finished (see the shared TypeScript commands above).

View File

@@ -61,6 +61,7 @@
:metabase/check-def-check-not-uppercase-name {:level :warning}
:metabase/check-def-no-underscores {:level :warning}
:metabase/check-me-humanize {:level :warning}
:metabase/check-testing-not-empty {:level :warning}
:metabase/defsetting-must-specify-export {:level :warning}
:metabase/defsetting-namespace {:level :warning}
:metabase/discourage-millis-duration {:level :warning}
@@ -178,6 +179,7 @@
clojure.core.async/take!
clojure.core.async/to-chan!
clojure.core.async/to-chan!!
metabase.api.macros.defendpoint.tools-manifest/assert-optional-fields-nullable!
metabase.channel.core/send!
metabase.warehouses.models.database/assert-not-h2!
metabase.driver.sql-jdbc.execute/execute-prepared-statement!
@@ -191,6 +193,10 @@
metabase.lib-metric.ast.plan/validate-arithmetic-ast!
metabase-enterprise.workspaces.isolation/ensure-database-isolation!
metabase-enterprise.workspaces.test-util/create-ready-ws!
metabase-enterprise.workspaces.core/clear-instance-workspace!
metabase-enterprise.workspaces.core/set-instance-workspace!
metabase.workspaces.core/clear-instance-workspace!
metabase.workspaces.core/set-instance-workspace!
;; Safe in tests, where we typically use a private, temporary index per thread.
metabase.search.core/reindex!
metabase.util/run-count!
@@ -200,6 +206,16 @@
{:level :warning
:max-length 150}
;; Warn on long tests (tests over `:min-length`) that look like
;;
;; (deftest my-test
;; (testing ...)
;; (testing ...)
;; (testing ...))
:metabase/validate-deftest-logically-separate-tests
{:level :warning
:min-length 60}
;; this is not enabled here, it is enabled below only for specific namespaces.
:metabase/disallow-hardcoded-driver-names-in-tests
;; don't include `:h2` since that will probably lead to a mountain of tests where we are specifically testing
@@ -254,7 +270,6 @@
(metabase.util.match/match-many)
(metabase.util.match/replace-in)
(metabase.util.match/replace)
(metabase.driver-api.core/match)
(metabase.driver-api.core/match-one)
(metabase.driver-api.core/match-many)
(metabase.driver-api.core/replace)
@@ -429,6 +444,8 @@
instaparse.core/defparser clojure.core/def
metabase-enterprise.action-v2.test-util/with-test-tables! clojure.core/let
metabase-enterprise.content-translation.routes-test/with-new-secret-key! clojure.core/fn
metabase-enterprise.curated-search.reconcile-test/with-isolated-mirror clojure.core/fn
metabase-enterprise.serialization.metadata-file-import.wire-format-test/with-tiny-fixture! clojure.core/fn
metabase-enterprise.serialization.test-util/with-dbs clojure.core/fn
metabase-enterprise.serialization.test-util/with-random-dump-dir clojure.core/let
metabase-enterprise.workspaces.test-util/with-resources! clojure.core/let
@@ -444,6 +461,7 @@
metabase.channel.impl.http-test/with-captured-http-requests clojure.core/fn
metabase.channel.impl.http-test/with-server clojure.core/let
metabase.collections.models.collection-test/with-collection-in-location clojure.core/let
metabase.curated-search.api-test/with-test-entry clojure.core/let
metabase.driver.mongo.connection/with-mongo-client clojure.core/let
metabase.driver.mongo.connection/with-mongo-database clojure.core/let
metabase.driver.mongo.query-processor/mongo-let clojure.core/let
@@ -523,6 +541,7 @@
{:analyze-call
{cljs.test/deftest hooks.clojure.test/deftest
cljs.test/is hooks.clojure.test/is
cljs.test/testing hooks.clojure.test/testing
cljs.test/use-fixtures hooks.clojure.test/use-fixtures
clojure.core/def hooks.clojure.core.def/lint-def
clojure.core/definterface hooks.clojure.core.def/lint-def
@@ -541,6 +560,7 @@
clojure.core/- hooks.metabase.util.timing/discourage-millis-duration
clojure.test/deftest hooks.clojure.test/deftest
clojure.test/is hooks.clojure.test/is
clojure.test/testing hooks.clojure.test/testing
clojure.test/use-fixtures hooks.clojure.test/use-fixtures
malli.error/humanize hooks.malli.error/humanize
metabase-enterprise.advanced-permissions.models.permissions.application-permissions-test/with-new-group-and-current-graph hooks.common/with-two-top-level-bindings
@@ -785,14 +805,7 @@
:name general-qp-and-driver-test-namespaces}
{:pattern "(?:^mage.*)"
:name mage-namespaces}
;;
;; Namespaces where `with-redefs` is still the idiom because they haven't been audited for conversion
;; to `with-dynamic-fn-redefs` yet (enterprise tests, driver tests). The linter's nudge is too noisy
;; here — audit can happen in follow-up PRs.
;;
{:pattern "^(?:metabase-enterprise\\..+-test(?:\\..+)?|metabase\\.driver\\..+-test(?:\\..+)?)$"
:name prefer-with-dynamic-fn-redefs-unaudited}]
:name mage-namespaces}]
:config-in-ns
{test-namespaces
@@ -816,10 +829,6 @@
metabase.util.time-test
{:linters {:metabase/prefer-with-dynamic-fn-redefs {:level :off}}}
;; Enterprise tests and driver tests haven't been audited yet — silence the linter so CI can ship.
prefer-with-dynamic-fn-redefs-unaudited
{:linters {:metabase/prefer-with-dynamic-fn-redefs {:level :off}}}
source-namespaces
{:linters
{:metabase/discourage-millis-duration {:level :warning}
@@ -914,10 +923,7 @@
:deprecated-namespace
{:exclude
[metabase.driver.common.parameters.dates
metabase.driver.common.parameters.operators
metabase.driver.common.parameters.values
metabase.legacy-mbql.normalize
[metabase.legacy-mbql.normalize
metabase.legacy-mbql.schema
metabase.legacy-mbql.schema.helpers
metabase.legacy-mbql.schema.macros

View File

@@ -109,7 +109,8 @@
search
settings
util
warehouses}
warehouses
workspaces}
:model-exports #{:model/Action}
:model-imports #{:model/Card
:model/Collection
@@ -127,6 +128,7 @@
analytics
api
collections
eid-translation
lib
permissions
public-sharing
@@ -157,38 +159,47 @@
{:team "Metabot"
:api #{metabase.agent-api.api
metabase.agent-api.init}
:uses #{agent-lib
api
:uses #{api
auth-identity
channel
collections
dashboards
events
lib
lib-be
llm
metabot
queries
query-permissions
query-processor
request
server
settings
util}
:model-imports #{:model/Card
:model/Collection
:model/Dashboard
:model/DashboardCard
:model/Table
:model/User}}
agent-lib
{:team "Metabot"
:api #{metabase.agent-lib.core}
:api #{metabase.agent-lib.representations
metabase.agent-lib.representations.repair
metabase.agent-lib.representations.resolve}
:uses #{lib
models
util}}
analytics
{:team "UX West"
:api #{metabase.analytics.api
metabase.analytics.api.proxy
metabase.analytics.core
metabase.analytics.event
metabase.analytics.init
metabase.analytics.settings
metabase.analytics.snowplow}
metabase.analytics.settings}
:uses #{analytics-interface
api
app-db
@@ -241,7 +252,6 @@
{:team "Graphy"
:api #{metabase.analyze.core}
:uses #{config
driver
lib
models
query-processor
@@ -323,15 +333,18 @@
:api #{metabase.app-db.cluster-lock ; TODO FIXME
metabase.app-db.core
metabase.app-db.init
metabase.app-db.setup}
;; exposes `query-canceled-exception?*` so the optional H2 driver can register its `:h2` method; H2 classes
;; can't live in core (AOT) namespaces. See `metabase.config.core/h2-available?`.
metabase.app-db.query-cancelation
metabase.app-db.setup
metabase.app-db.transient-error}
:uses #{auth-provider
classloader
config
connection-pool
task
util}
:model-imports #{:model/Database
:model/Field
:model-imports #{:model/Field
:model/QueryCache
:model/Secret
:model/Setting}}
@@ -347,6 +360,7 @@
{:team "UX West"
:api #{metabase.audit-app.core
metabase.audit-app.init}
:friends #{enterprise/memoize-monitor}
:uses #{api
app-db
events
@@ -484,23 +498,29 @@
driver
events
formatter
geojson
lib
lib-be
models
notification
parameters
permissions
pivot
premium-features
query-processor
request
settings
slackbot
system
task
tiles
timeline
types
util}
:model-exports #{:model/Channel :model/ChannelTemplate}
:model-imports #{:model/Dashboard
:model/DashboardCard
:model/DashboardTab
:model/Database
:model/PulseChannel
:model/User}}
@@ -669,6 +689,18 @@
:api #{}
:uses :any}
curated-search
{:team "Metabot"
:api #{metabase.curated-search.api
metabase.curated-search.core}
:uses #{api
models
premium-features
request
util}
:model-exports #{:model/CuratedSearchEntry}
:model-imports #{:model/Card}}
dashboards
{:team "UX West"
:api #{metabase.dashboards.autoplace
@@ -694,8 +726,8 @@
query-processor
search
settings
util
warehouse-schema}
sync
util}
:model-exports #{:model/Dashboard
:model/DashboardCard
:model/DashboardCardSeries
@@ -808,7 +840,9 @@
metabase.driver.common
metabase.driver.connection
metabase.driver.ddl.interface
metabase.driver.h2
;; NOTE: `metabase.driver.h2` is intentionally NOT here. H2 is an optional driver loaded via a runtime
;; `require` (see `metabase.driver.init` / `metabase.config.core/h2-available?`), so nothing references it
;; at compile time and the module checker treats it as extraneous in `:api`.
metabase.driver.impl
metabase.driver.init
metabase.driver.mysql
@@ -837,7 +871,6 @@
config
driver-api
events
legacy-mbql
lib
lib-be
premium-features
@@ -848,7 +881,10 @@
tracing
util
warehouse-schema
warehouses}
warehouses
;; `metabase.driver.sync/include-schema?` skips workspace iso schemas
;; on parent (non-workspace-mode) instances. See GHY-3489.
workspaces}
:model-imports #{:model/Database :model/Field :model/Table}}
driver-api
@@ -884,6 +920,7 @@
metabase.eid-translation.core
metabase.eid-translation.init}
:uses #{api
batch-processing
settings
util}
:model-imports #{:model/Action
@@ -940,7 +977,7 @@
request
tiles
util}
:model-imports #{:model/Card :model/Dashboard :model/EmbeddingTheme}}
:model-imports #{:model/Card :model/Dashboard :model/DashboardCard :model/EmbeddingTheme}}
events
{:team "DevEx"
@@ -981,7 +1018,8 @@
geojson
{:team "UX West"
:api #{metabase.geojson.api
metabase.geojson.init}
metabase.geojson.init
metabase.geojson.settings}
:uses #{api
config
permissions
@@ -1003,6 +1041,17 @@
:api #{metabase.graph.core}
:uses #{util}}
health-inspector
{:team "Querying Platform"
:api #{metabase.health-inspector.api
metabase.health-inspector.init}
:uses #{api
lib
lib-be
settings
task
util}}
indexed-entities
{:team "UX West"
:api #{metabase.indexed-entities.api
@@ -1013,6 +1062,7 @@
legacy-mbql
lib
models
permissions
query-processor
search
sync
@@ -1234,6 +1284,7 @@
channel
collections
config
curated-search
documents
driver
interestingness
@@ -1274,8 +1325,10 @@
:model/Database
:model/Field
:model/Glossary
:model/Measure
:model/ModerationReview
:model/QueryTable
:model/Segment
:model/Table
:model/Transform
:model/User}}
@@ -1323,7 +1376,8 @@
task
task-history
tracing
util}
util
workspaces}
:model-exports #{:model/PersistedInfo}
:model-imports #{:model/Card :model/Database}}
@@ -1335,6 +1389,7 @@
metabase.models.resolution
metabase.models.serialization
metabase.models.serialization.resolve
metabase.models.serialization.resolve.mp
metabase.models.util.spec-update
metabase.models.visualization-settings}
:uses #{api
@@ -1350,6 +1405,7 @@
:model/DashboardTab
:model/Database
:model/Field
:model/Measure
:model/NativeQuerySnippet
:model/Segment
:model/Table
@@ -1381,6 +1437,7 @@
:uses #{analytics
analytics-interface
api
app-db
appearance
auth-identity
channel
@@ -1411,14 +1468,18 @@
:model/Dashboard
:model/DashboardCard
:model/DashboardTab
:model/TaskHistory
:model/TaskRun
:model/User}}
oauth-server
{:team "UX West"
:api #{metabase.oauth-server.api
:api #{metabase.oauth-server.api ;; Special case OAuth standard endpoints under `/`
metabase.oauth-server.api.admin ;; The usual Metabase internal API endpoints under `/api`
metabase.oauth-server.core
metabase.oauth-server.init}
:uses #{api
api-scope
appearance
mcp
models
@@ -1450,8 +1511,7 @@
models
query-processor
util
warehouse-schema
warehouses}
warehouse-schema}
:model-imports #{:model/Card
:model/Dashboard
:model/Field
@@ -1510,7 +1570,8 @@
pivot
{:team "UX West"
:api #{metabase.pivot.core}
:api #{metabase.pivot.core
metabase.pivot.postprocess}
:uses #{models
util}}
@@ -1650,6 +1711,7 @@
dashboards
documents
events
graph
lib
lib-be
metrics
@@ -1663,6 +1725,7 @@
query-processor
request
search
sync
util
view-log
warehouse-schema}
@@ -1745,10 +1808,15 @@
metabase.query-processor.init
metabase.query-processor.interface
metabase.query-processor.metadata
;; exposes the `results-as-bytes` multimethod so the optional H2 driver can register its `JdbcBlob`
;; method; H2 classes can't live in core (AOT) namespaces. See `metabase.config.core/h2-available?`.
metabase.query-processor.middleware.cache-backend.db
metabase.query-processor.middleware.constraints
metabase.query-processor.middleware.enterprise
metabase.query-processor.middleware.permissions
metabase.query-processor.parameters.dates
metabase.query-processor.parameters.operators
metabase.query-processor.parameters.values
metabase.query-processor.pipeline
metabase.query-processor.pivot
metabase.query-processor.preprocess
@@ -1786,15 +1854,15 @@
premium-features
queries
query-permissions
request
server
settings
system
tracing
users
util}
util
workspaces}
:model-imports #{:model/Card
:model/Dashboard
:model/DashboardCard
:model/DashboardCardSeries
:model/Database
:model/QueryCache
@@ -1853,15 +1921,30 @@
:model/Segment
:model/Transform}}
run-tracking
{:team "Gadget"
:api #{metabase.run-tracking.core}
:uses #{analytics-interface
app-db
task
util}}
sample-data
{:team "UX West"
:api #{metabase.sample-data.core
metabase.sample-data.init}
:uses #{plugins
:uses #{config
models
plugins
settings
sync
util}
:model-imports #{:model/Database}}
:model-imports #{:model/Card
:model/Dashboard
:model/DashboardCard
:model/Database
:model/Field
:model/Table}}
search
{:team "UX West"
@@ -1932,6 +2015,7 @@
{:team "UX West"
:api #{metabase.server.core
metabase.server.init
metabase.server.middleware.security
metabase.server.middleware.session
metabase.server.settings
metabase.server.streaming-response} ; TODO -- too many API namespaces
@@ -1958,6 +2042,7 @@
tracing
users
util
enterprise/custom-viz-plugin
enterprise/sso}
:model-imports #{:model/Session}}
@@ -2122,6 +2207,7 @@
{:team "Graphy"
:api #{metabase.sync.api
metabase.sync.core
metabase.sync.field-values
metabase.sync.init
metabase.sync.schedules
metabase.sync.task.sync-databases
@@ -2137,6 +2223,7 @@
events
interestingness
lib
lib-be
models
premium-features
query-processor
@@ -2146,7 +2233,8 @@
tracing
util
warehouse-schema
warehouses}
warehouses
workspaces}
:model-imports #{:model/Database
:model/Field
:model/FieldValues
@@ -2173,16 +2261,17 @@
metabase.task-history.core
metabase.task-history.init}
:uses #{api
app-db
config
models
permissions
premium-features
query-processor
request
run-tracking
task
tracing
util}
:model-exports #{:model/TaskHistory :model/TaskRun}
:model-imports #{:model/Card :model/Dashboard :model/Database}}
tenants
@@ -2219,7 +2308,8 @@
tiles
{:team "UX West"
:api #{metabase.tiles.api
metabase.tiles.init}
metabase.tiles.init
metabase.tiles.settings}
:uses #{api
legacy-mbql
lib
@@ -2227,7 +2317,8 @@
parameters
query-processor
settings
util}}
util}
:model-imports #{:model/Card :model/Dashboard :model/DashboardCard}}
timeline
{:team "UX West"
@@ -2276,6 +2367,7 @@
query-processor
remote-sync
revisions
run-tracking
search
settings
task
@@ -2287,6 +2379,7 @@
enterprise/transforms-python}
:model-exports #{:model/Transform
:model/TransformJob
:model/TransformJobRun
:model/TransformJobTransformTag
:model/TransformRun
:model/TransformTag}
@@ -2305,6 +2398,7 @@
;; :model/TransformRun directly. Remove once the module is agnostic to global
;; versus workspace transforms. See https://linear.app/metabase/issue/BOT-1143
:model-imports #{:model/Database
:model/Field
:model/Table
:model/Transform
:model/TransformRun}
@@ -2313,9 +2407,11 @@
events
lib
lib-be
premium-features
query-processor
sync
util}
util
workspaces}
:friends #{transforms
enterprise/transforms-python}}
@@ -2337,6 +2433,7 @@
util}
:model-imports #{:model/Transform
:model/TransformJob
:model/TransformJobRun
:model/TransformJobTransformTag
:model/TransformRun
:model/TransformTag}}
@@ -2367,12 +2464,30 @@
settings
sync
util
warehouse-schema}
warehouse-schema
workspaces}
:model-imports #{:model/Card
:model/Database
:model/Field
:model/Table}}
usage-metadata
{:team "UX West"
:api #{metabase.usage-metadata.init}
:model-imports #{:model/Card
:model/Field
:model/Query
:model/QueryExecution
:model/Segment
:model/Table}
:uses #{app-db
lib
lib-be
models
settings
task
util}}
user-key-value
{:team "UX West"
:api #{metabase.user-key-value.api
@@ -2478,6 +2593,7 @@
warehouse-schema
{:team "UX West"
:api #{metabase.warehouse-schema.field
metabase.warehouse-schema.field-values.distinct-batch
metabase.warehouse-schema.metadata-from-qp
metabase.warehouse-schema.metadata-queries
metabase.warehouse-schema.models.field
@@ -2601,6 +2717,13 @@
:model/Field
:model/Table}}
workspaces
{:team "Graphy"
:api #{metabase.workspaces.core
metabase.workspaces.table-remapping}
:uses #{premium-features
util}}
xrays
{:team "Graphy"
:api #{metabase.xrays.api
@@ -2645,10 +2768,11 @@
driver
events
lib
lib-be
models
query-processor
util
warehouse-schema}
sync
util}
:model-imports #{:model/Database
:model/Field
:model/FieldValues
@@ -2656,7 +2780,8 @@
enterprise/advanced-config
{:team "UX West"
:api #{metabase-enterprise.advanced-config.api.logs
:api #{metabase-enterprise.advanced-config.api
metabase-enterprise.advanced-config.api.logs
metabase-enterprise.advanced-config.file
metabase-enterprise.advanced-config.init
metabase-enterprise.advanced-config.settings}
@@ -2667,11 +2792,13 @@
lib
permissions
premium-features
sample-data
settings
setup
sync
users
util}
util
enterprise/workspaces}
:model-imports #{:model/ApiKey :model/Database :model/User}}
enterprise/advanced-permissions
@@ -2707,7 +2834,7 @@
enterprise/scim
enterprise/semantic-search
enterprise/sso}
:model-imports #{:model/Database :model/Sandbox}}
:model-imports #{:model/Database :model/Sandbox :model/Workspace}}
enterprise/api
{:team "UX West"
@@ -2829,6 +2956,11 @@
util
enterprise/harbormaster}}
enterprise/connection-overlays
{:team "Graphy"
:api #{}
:uses #{premium-features}}
enterprise/content-translation
{:team "UX West"
:api #{metabase-enterprise.content-translation.routes}
@@ -2853,10 +2985,20 @@
:api #{metabase-enterprise.core.init}
:uses :any}
enterprise/curated-search
{:team "Metabot"
:api #{metabase-enterprise.curated-search.init}
:uses #{premium-features
task
util
enterprise/semantic-search}
:model-imports #{:model/CuratedSearchEntry}}
enterprise/custom-viz-plugin
{:team "Gadget"
:api #{metabase-enterprise.custom-viz-plugin.api
metabase-enterprise.custom-viz-plugin.init}
metabase-enterprise.custom-viz-plugin.init
metabase-enterprise.custom-viz-plugin.settings}
:uses #{api
config
events
@@ -2943,7 +3085,8 @@
settings
util
warehouse-schema
warehouses}
warehouses
workspaces}
:model-exports #{:model/DatabaseRouter}
:model-imports #{:model/Database :model/Transform}}
@@ -2954,8 +3097,7 @@
metabase-enterprise.dependencies.core
metabase-enterprise.dependencies.init
metabase-enterprise.dependencies.models.dependency}
:uses #{analyze
api
:uses #{api
app-db
collections
database-routing
@@ -2966,7 +3108,6 @@
lib
lib-be
models
native-query-snippets
permissions
premium-features
queries
@@ -3029,6 +3170,17 @@
;; make token tracking a hook). Drop this dep once the relocation lands.
:uses #{enterprise/semantic-search}}
enterprise/erd
{:team "Gadget"
:api #{metabase-enterprise.erd.api}
:uses #{api
models
util
warehouse-schema}
:model-imports #{:model/Database
:model/Field
:model/Table}}
enterprise/gsheets
{:team "Cloud"
:api #{metabase-enterprise.gsheets.api
@@ -3079,6 +3231,20 @@
premium-features}
:model-imports #{:model/Card :model/Collection :model/Table}}
enterprise/memoize-monitor
{:team "Graphy"
:api #{metabase-enterprise.memoize-monitor.init}
:uses #{analytics
analytics-interface
audit-app
collections
driver
models
util
warehouse-schema
warehouses
enterprise/serialization}}
enterprise/metabot
{:team "Metabot"
:api #{metabase-enterprise.metabot.api
@@ -3147,6 +3313,7 @@
events
models
premium-features
search
settings
startup
task
@@ -3173,7 +3340,8 @@
enterprise/replacement
{:team "Graphy"
:api #{metabase-enterprise.replacement.api}
:uses #{api
:uses #{analytics
api
app-db
events
lib
@@ -3283,6 +3451,8 @@
{:team "Metabot"
:api #{metabase-enterprise.semantic-search.api
metabase-enterprise.semantic-search.core
metabase-enterprise.semantic-search.db.datasource
metabase-enterprise.semantic-search.embedding
metabase-enterprise.semantic-search.init}
:uses #{activity-feed
analytics
@@ -3301,14 +3471,17 @@
task
tracing
util}
:model-imports #{:model/Card :model/Collection :model/Dashboard}}
:model-imports #{:model/Collection}}
enterprise/serialization
{:team "Graphy"
:api #{metabase-enterprise.serialization.api
metabase-enterprise.serialization.cmd
metabase-enterprise.serialization.core
metabase-enterprise.serialization.init
metabase-enterprise.serialization.schema
metabase-enterprise.serialization.v2.backfill-ids}
:friends #{enterprise/memoize-monitor}
:uses #{analytics
api
app-db
@@ -3324,7 +3497,9 @@
search
server
setup
util}
sync
util
warehouse-schema}
:model-imports :bypass}
enterprise/snippet-collections
@@ -3353,6 +3528,7 @@
settings
sso
system
task
util
enterprise/scim}
:model-imports #{:model/PermissionsGroup :model/Session}}
@@ -3468,14 +3644,30 @@
:model-imports #{:model/Database :model/Table}}
enterprise/workspaces
{:team "Metabot"
:api #{}
:uses #{premium-features}}
enterprise/writable-connection
{:team "Graphy"
:api #{}
:uses #{premium-features}}}
:api #{metabase-enterprise.workspaces.api
metabase-enterprise.workspaces.core
metabase-enterprise.workspaces.init}
:uses #{api
app-db
cache
driver
lib
models
permissions
premium-features
query-processor
sample-data
server
settings
sql-tools
util
workspaces
enterprise/serialization}
:model-exports #{:model/Workspace}
:model-imports #{:model/Database
:model/Table
:model/User}}}
;; namespaces matching these patterns (with `re-find`) are excluded from the module linter. Since regex literals
;; aren't allowed in EDN just used the `(str <regex>)` version i.e. two slashes instead of one.

View File

@@ -21,15 +21,19 @@
(error! form (format "destructive functions like %s are not allowed inside a ^:parallel test or test fixture. If this should be allowed, add it to the whitelist in the Kondo config file [:metabase/validate-deftest]"
qualified-symbol)))))
(walk [form]
(f form)
(doseq [child (:children form)]
(walk child)))]
(when-not (contains? (hooks.common/ignored-linters form) :metabase/validate-deftest)
(f form)
(doseq [child (:children form)]
(walk child))))]
(walk form)))
(defn- deftest-check-parallel
"1. Check if test is marked ^:parallel / ^:synchronized correctly
2. Make sure disallowed forms are not used in ^:parallel tests"
[{[_ test-name & body] :children, :as _node} config]
"1. Check if test is marked ^:parallel / ^:synchronized correctly.
2. Make sure disallowed forms are not used in ^:parallel tests.
3. Disallow ^:parallel tests in namespaces marked ^:synchronous -- the
ns marker is the author's blanket promise that nothing inside runs
in parallel, so an individual ^:parallel deftest contradicts it."
[{[_ test-name & body] :children, :as _node} ns-sym config]
(let [test-metadata (:meta test-name)
metadata-sexprs (map hooks/sexpr test-metadata)
combined-metadata (transduce
@@ -41,11 +45,20 @@
{}
metadata-sexprs)
parallel? (:parallel combined-metadata)
synchronized? (:synchronized combined-metadata)]
synchronized? (:synchronized combined-metadata)
ns-synchronous? (boolean (:synchronous (meta ns-sym)))]
(when (and parallel? synchronized?)
(hooks/reg-finding! (assoc (meta test-name)
:message "Test should not be marked both ^:parallel and ^:synchronized"
:type :metabase/validate-deftest)))
(when (and parallel? ns-synchronous?)
(hooks/reg-finding!
(assoc (meta test-name)
:message (str "Test should not be marked ^:parallel in a namespace marked ^:synchronous -- "
"the ns marker promises no parallel execution. "
"Drop ^:parallel from the deftest, or drop ^:synchronous from the ns. "
"[:metabase/validate-deftest]")
:type :metabase/validate-deftest)))
;; only when the custom `:metabase/deftest-not-marked-parallel-or-synchronized` is enabled: complain if tests are
;; not explicitly marked `^:parallel` or `^:synchronized`. This is mostly to encourage people to mark everything
;; `^:parallel` in places like `metabase.lib` tests unless there is a really good reason not to.
@@ -71,6 +84,36 @@
"Split it up into smaller tests! 🥰")
:type :metabase/i-like-making-cams-eyes-bleed-with-horrifically-long-tests)))))))
(defn- every-top-level-form-in-deftest-is-testing? [{[_deftest _test-name & forms] :children, :as _node}]
(every? (fn [{[first-child] :children, :as form}]
(and (hooks/list-node? form)
(= (hooks.common/node->qualified-symbol first-child)
'clojure.test/testing)))
forms))
(defn- num-top-level-forms-in-deftest [{[_deftest _test-name & forms] :children, :as _node}]
(count forms))
(defn- deftest-check-should-not-be-multiple-separate-tests
"Warn on long tests (tests over `min-length`) that look like
(deftest my-test
(testing ...)
(testing ...)
(testing ...))"
[node {:keys [min-length], :as _config}]
(let [{:keys [row end-row]} (meta node)
test-length (- end-row row)]
(when (and min-length
(> test-length min-length)
(every-top-level-form-in-deftest-is-testing? node)
(> (num-top-level-forms-in-deftest node) 1))
(hooks/reg-finding!
(assoc (meta node)
:message (str "This test looks like it contains several logically separate testing forms... break it"
" out into separate deftests to make it easier to test and debug")
:type :metabase/validate-deftest-logically-separate-tests)))))
(defn- ignore? [node error-type]
(contains? (hooks.common/ignored-linters node) error-type))
@@ -135,13 +178,14 @@
:message (format "All tests must live in a known module; %s is not a known module" (pr-str current-module))
:type :metabase/tests-must-live-in-known-modules)))))))
(defn deftest [{:keys [node cljc lang config], :as input}]
(defn deftest [{:keys [node cljc lang config], ns-sym :ns, :as input}]
;; run [[deftest-check-parallel]] only once... if this is a `.cljc` file only run it for the `:clj` analysis, no point
;; in running it twice.
(when (or (not cljc)
(= lang :clj))
(deftest-check-parallel node (get-in config [:linters :metabase/validate-deftest])))
(deftest-check-parallel node ns-sym (get-in config [:linters :metabase/validate-deftest])))
(deftest-check-not-horrifically-long node (get-in config [:linters :metabase/i-like-making-cams-eyes-bleed-with-horrifically-long-tests]))
(deftest-check-should-not-be-multiple-separate-tests node (get-in config [:linters :metabase/validate-deftest-logically-separate-tests]))
(deftest-check-no-driver-keywords node (get-in config [:linters :metabase/disallow-hardcoded-driver-names-in-tests]))
(deftest-check-in-valid-module input)
input)
@@ -180,11 +224,38 @@
nil)))))))
(defn is [{:keys [node lang]}]
(defn is [{:keys [node lang], :as input}]
(when (= lang :cljs)
(warn-about-missing-test-expr-requires-in-cljs node))
{:node node})
input)
(defn use-fixtures [{:keys [node config]}]
(warn-about-disallowed-parallel-forms node config)
{:node node})
(defn use-fixtures
"Flag `:parallel/unsafe` forms inside fixtures. Skipped when the surrounding
namespace is marked `^:synchronous` -- the ns marker is the author's
explicit opt-in to a single-threaded test ns where destructive setup is
safe. Without that opt-in, kondo can't know whether sibling deftests will
be `^:parallel`, so the conservative default is to flag.
Sister to [[deftest-check-parallel]]: the deftest hook rejects `^:parallel`
tests inside a `^:synchronous` ns, so the two checks together form a
coherent opt-in: either the whole ns is synchronous (fixtures can be
anything; no parallel tests allowed) or it isn't (fixtures must be
parallel-safe)."
[{:keys [node config], ns-sym :ns, :as input}]
(when-not (:synchronous (meta ns-sym))
(let [linter-config (get-in config [:linters :metabase/validate-deftest])]
(doseq [form (rest (:children node))]
(warn-about-disallowed-parallel-forms form linter-config))))
input)
(defn testing
"Check that we don't have an empty `testing` form like
(testing \"message\")"
[{{[_testing _message & body] :children, :as node} :node, :as input}]
(when (empty? body)
(hooks/reg-finding!
(assoc (meta node)
:message "A `testing` form that doesn't wrap anything doesn't do anything"
:type :metabase/check-testing-not-empty)))
input)

View File

@@ -500,8 +500,8 @@
(and resolved (:ns resolved))
(symbol (name (:ns resolved)) (name (:name resolved)))
;; if it wasn't resolved but is still qualified it's probably using the full namespace name rather than an
;; alias.
;; if it wasn't resolved but is still qualified it's probably using the full namespace name rather than an
;; alias.
(qualified-symbol? sexpr)
sexpr)))))
;; some symbols like `*count/Integer` aren't resolvable.

View File

@@ -95,6 +95,8 @@
metabot-feedback-url
metabot-get-prompt-templates-url
metabot-prompt-generator-token-limit
metaplow-tracking-enabled
metaplow-url
multi-setting-read-only
notebook-native-preview-sidebar-width
notification-link-base-url

View File

@@ -28,8 +28,8 @@
(mapcat (fn [[setting-name v]]
(concat
[(with-meta (hooks/token-node '_) (meta setting-name)) v]
;; if the setting name is namespace-qualified add a `_`
;; entry for it too.
;; if the setting name is namespace-qualified add a `_`
;; entry for it too.
(when (namespaced-symbol-node? setting-name)
[(with-meta (hooks/token-node '_) (meta setting-name)) setting-name]))))
(partition 2 (:children bindings))))

View File

@@ -1,7 +1,6 @@
(ns hooks.metabase.util.i18n
(:require
[clj-kondo.hooks-api :as api]
[clojure.string :as str]
[hooks.common]))
(defn- valid-apostrophes?
@@ -13,23 +12,23 @@
(let [c (first chars)]
(if (= c \')
(cond
;; Doubled quotes: skip both
;; Doubled quotes: skip both
(and (next chars) (= (second chars) \'))
(recur (nthrest chars 2))
;; Placeholder escape. We allow for intentional escaping, like either '{0}` or '{{`, but not 'arbitrary
;; strings' So basically, just require the *next* character to be a `{`, find the next single-quote, and
;; continue on from there.
;;
;; Technically a string like "foo bar's happy but i'm not" is totally fine, but we'll say it's invalid
;; because... that inner string literal is probably not what we actually want. So since we've never done
;; this intentionally as far as I can tell, let's just remove the possibility to avoid ambiguity.
;; Placeholder escape. We allow for intentional escaping, like either '{0}` or '{{`, but not 'arbitrary
;; strings' So basically, just require the *next* character to be a `{`, find the next single-quote, and
;; continue on from there.
;;
;; Technically a string like "foo bar's happy but i'm not" is totally fine, but we'll say it's invalid
;; because... that inner string literal is probably not what we actually want. So since we've never done
;; this intentionally as far as I can tell, let's just remove the possibility to avoid ambiguity.
(= (second chars) \{)
(let [at-closing (drop-while #(not= \' %) (next chars))]
(if (seq at-closing)
(recur (next at-closing))
false))
;; Unpaired single quote: invalid
;; Unpaired single quote: invalid
:else
false)
(recur (rest chars)))))))

View File

@@ -49,7 +49,10 @@
:deps
vals
(keep (comp keyword :local/root))
(into #{}))]
(into #{})
;; :h2 is intentionally excluded from the linter set (it's the default test driver, so
;; hardcoding it is fine and ubiquitous), even though it now lives in modules/drivers.
(#(disj % :h2)))]
(doseq [driver driver-modules]
(is (contains? driver-keywords driver)
(format "hooks.clojure.test/driver-keywords should contain %s, please add it" driver))))))

27
.clj-surgeon.edn Normal file
View File

@@ -0,0 +1,27 @@
{:aliases
{;; defendpoint shape: (api.macros/defendpoint METHOD URL DOCSTRING? [args] body)
;; No name slot — synthesize :route [METHOD "path"] from first 2 args.
"defendpoint"
{:fields {:method (fn [z] (some-> z z/down z/right z/sexpr name str/upper-case))
:path (fn [z] (-> z z/down z/right z/right z/sexpr))}}
;; defenterprise shape:
;; (defenterprise NAME DOCSTRING? EE-NAMESPACE [args] body)
"defenterprise"
{:fields {:name ->defn-name
:ee-namespace (fn [z]
;; Skip name + optional docstring; the next
;; symbol is the EE namespace.
(let [name-z (-> z z/down z/right)
after-doc (z/right name-z)
candidate (if (string? (try (z/sexpr after-doc)
(catch Exception _ nil)))
(z/right after-doc)
after-doc)]
(try (let [v (z/sexpr candidate)]
(when (symbol? v) v))
(catch Exception _ nil))))
:arglist ->defn-arg-list}}
"defsetting"
{:fields {:name ->defn-name}}}}

View File

@@ -1,12 +1,10 @@
;; Cljfmt config. See https://github.com/weavejester/cljfmt#formatting-options
{:sort-ns-references?
true
:function-arguments-indentation
:community
:parallel?
true
{:function-arguments-indentation :community
:indent-line-comments? true
:normalize-newlines-at-file-end? true
:parallel? true
:remove-blank-lines-in-forms? true
:sort-ns-references? true
:paths
["src"
@@ -15,11 +13,63 @@
"bin/build"
"bin/lint-migrations-file"
"bin/release-list"
#_"mage/src"
#_"mage/test"
"modules/drivers"
".clj-kondo/src"
".clj-kondo/test"
#_"dev"]
;; a map of symbols that tell cljfmt which forms are allowed to have blank lines inside of them. The value may be
;; either `:all`, which means blank lines are allowed between all elements in the form, e.g. `{cond :all}` to allow
;; blank lines between any of the elements inside cond; or it may be a set of element indexes that are allowed to
;; have blank lines, e.g. `{let #{0}}`, to allow blank lines in the binding of a `let` form.
:extra-blank-line-forms
{are :all
case :all
cond-> :all
cond->> :all
condp :all
clojure.spec.alpha/or :all
clojure.test/are :all
metabase.driver-api.core/match-many :all
metabase.driver-api.core/match-one :all
metabase.driver-api.core/replace :all
metabase.driver-api.core/replace-in :all
metabase.lib.drill-thru.test-util/test-drill-variants-with-merged-args :all
metabase.lib.drill-thru.test-util/test-returns-drill :all
metabase.test/with-temp #{0}
metabase.util.match/match-many :all
metabase.util.match/match-one :all
metabase.util.match/replace :all
metabase.util.match/replace-in :all
;; TODO (Cam 2026-05-21) I'm not really sure about `defprotocol`, `deftype`, `defrecord`, `proxy`, `reify`, etc., I
;; think idiomatically these should probably not have blank lines but we can worry about that later
defprotocol :all
defrecord :all
deftype :all
extend :all
extend-protocol :all
potemkin.types/def-map-type :all
potemkin.types/defprotocol+ :all
potemkin.types/defrecord+ :all
potemkin.types/deftype+ :all
potemkin/def-map-type :all
potemkin/defprotocol+ :all
potemkin/defrecord+ :all
potemkin/deftype+ :all
proxy :all
reify :all
;; TODO (Cam 2026-05-21) -- save these for a follow-on PR
defmacro :all
defmethod :all
defn :all
defn- :all
fn :all
metabase.util.malli/defmethod :all
metabase.util.malli/defn :all
metabase.util.malli/defn- :all}
;; See https://github.com/weavejester/cljfmt/blob/master/cljfmt/resources/cljfmt/indents/clojure.clj for the default
;; definitions
;;
@@ -37,7 +87,7 @@
;; `:inner` indentation is like `:defn` indentation. `[[:inner 0]]` (default for `defn`) means all arguments at depth
;; zero get indented 2 spaces if not on the first line regardless of number of elements on the first line.
:extra-indents
{;; clojure.core stuff
{ ;; clojure.core stuff
fn* [[:inner 0]]
let* [[:block 1]]
with-meta [[:default]]

View File

@@ -1,10 +0,0 @@
# worktrunk project config — https://worktrunk.dev
# Hooks run when switching, creating, merging, or removing worktrees.
# Setup tasks for newly-created worktrees. Run once, blocking, in the
# new worktree's cwd before post-start background tasks kick off.
[pre-start]
# Trust mise config so it loads the project's tool versions without prompting.
trust-mise = "mise trust"
# Install JS deps so husky git hooks get wired up in the new worktree.
bun-install = "bun install"

View File

@@ -1,5 +1,5 @@
> [!IMPORTANT]
> For those employed by Metabase: if you are merging into master, please add either a `backport` or a `no-backport` label to this PR. You will not be able to merge until you do this step. Refer to the section [Do I need to backport this PR?](https://www.notion.so/metabase/Metabase-Branching-Strategy-6eb577d5f61142aa960a626d6bbdfeb3?pvs=4#89f80d6f17714a0198aeb66c0efd1b71) in the Metabase Branching Strategy document for more details. If you're not employed by Metabase, this section does not apply to you, and the label will be taken care of by your reviewer.
> For those employed by Metabase: if you are fixing a P0 or P1 issue, add a `backport` label to this PR. No action is needed otherwise. Refer to the [Backporting](https://app.notion.com/p/metabase/Branching-Strategy-and-Backports-6eb577d5f61142aa960a626d6bbdfeb3?source=copy_link#e71f6e7de2f945d99fbb2d6b764e6f36) section in the Metabase Branching Strategy document for more details. If you're not employed by Metabase, this section does not apply to you, and the label will be taken care of by your reviewer.
> **Warning**
>

View File

@@ -26,7 +26,7 @@ runs:
- name: Upload python-runner logs
if: ${{inputs.upload-logs == 'true' && inputs.logs-artifact-suffix != ''}}
uses: actions/upload-artifact@v4
uses: actions/upload-artifact@v7
with:
name: python-runner-logs-${{inputs.logs-artifact-suffix}}
path: |

View File

@@ -84,18 +84,18 @@ runs:
- name: Auto approve backport PR
if: ${{ steps.create_backport_pull_request.outputs.has-conflicts == 'false' }}
uses: juliangruber/approve-pull-request-action@v1
uses: juliangruber/approve-pull-request-action@68fcc9a5a73b5641cadf757cf99d73720dcb05d0 # v2.1.0
with:
github-token: ${{ inputs.github-token-2 }}
number: ${{ steps.create_backport_pull_request.outputs.backport_pr_number }}
- name: Enable Pull Request Automerge
if: ${{ steps.create_backport_pull_request.outputs.has-conflicts == 'false' }}
uses: peter-evans/enable-pull-request-automerge@v2
with:
token: ${{ inputs.github-token }}
pull-request-number: ${{ steps.create_backport_pull_request.outputs.backport_pr_number }}
merge-method: squash
run: gh pr merge --squash --auto "$PR_NUMBER"
shell: bash
env:
GH_TOKEN: ${{ inputs.github-token }}
PR_NUMBER: ${{ steps.create_backport_pull_request.outputs.backport_pr_number }}
- uses: ./.github/actions/notify-pull-request
if: ${{ steps.create_backport_pull_request.outputs.has-conflicts == 'true' }}

View File

@@ -11,7 +11,7 @@ runs:
steps:
## First we deploy the base branch
- name: Check out base branch
uses: actions/checkout@v4
uses: actions/checkout@v6
with:
ref: ${{ inputs.base-ref }}
@@ -42,7 +42,7 @@ runs:
## Then we deploy the PR branch to ensure it migrates up successfully
- name: Check out pull request branch
uses: actions/checkout@v4
uses: actions/checkout@v6
with:
ref: ${{ inputs.head-ref }}
@@ -81,7 +81,7 @@ runs:
shell: bash
- name: Re-check out base branch
uses: actions/checkout@v4
uses: actions/checkout@v6
with:
ref: ${{ inputs.base-ref }}

View File

@@ -9,7 +9,7 @@ runs:
using: "composite"
steps:
- name: Download previously stored uberjar
uses: actions/github-script@v7
uses: actions/github-script@v9
with:
script: | # js
const fs = require('fs');

View File

@@ -40,10 +40,24 @@ runs:
using: "composite"
steps:
- name: Authenticate to prevent API rate limit
uses: docker/login-action@v3
with:
username: ${{ inputs.username }}
password: ${{ inputs.password }}
shell: bash
env:
DOCKER_USERNAME: ${{ inputs.username }}
DOCKER_PASSWORD: ${{ inputs.password }}
run: |
for attempt in 1 2 3; do
echo "Docker login attempt $attempt of 3..."
if echo "$DOCKER_PASSWORD" | docker login --username "$DOCKER_USERNAME" --password-stdin; then
echo "Docker login succeeded on attempt $attempt"
break
fi
if [ $attempt -eq 3 ]; then
echo "::error::Docker login failed after 3 attempts"
exit 1
fi
echo "::warning::Docker login attempt $attempt failed, retrying in 15s..."
sleep 15
done
- name: Start Containers
shell: bash
@@ -53,16 +67,29 @@ runs:
cd e2e/test/scenarios/
docker compose up \
$([ ${{ inputs.postgres }} = true ] && echo "postgres-sample") \
$([ ${{ inputs.mysql }} = true ] && echo "mysql-sample") \
$([ ${{ inputs.mongo }} = true ] && echo "mongo-sample") \
$([ ${{ inputs.maildev }} = true ] && echo "maildev") \
$([ ${{ inputs.maildev }} = true ] && echo "maildev-ssl") \
$([ ${{ inputs.openldap }} = true ] && echo "ldap") \
$([ ${{ inputs.webhook }} = true ] && echo "webhook-tester") \
$([ ${{ inputs.snowplow }} = true ] && echo "snowplow-micro") \
-d
SERVICES=()
[ "${{ inputs.postgres }}" = true ] && SERVICES+=(postgres-sample)
[ "${{ inputs.mysql }}" = true ] && SERVICES+=(mysql-sample)
[ "${{ inputs.mongo }}" = true ] && SERVICES+=(mongo-sample)
[ "${{ inputs.maildev }}" = true ] && SERVICES+=(maildev maildev-ssl)
[ "${{ inputs.openldap }}" = true ] && SERVICES+=(ldap)
[ "${{ inputs.webhook }}" = true ] && SERVICES+=(webhook-tester)
[ "${{ inputs.snowplow }}" = true ] && SERVICES+=(snowplow-micro)
for attempt in 1 2 3; do
echo "Docker compose up attempt $attempt of 3..."
if docker compose up "${SERVICES[@]}" -d; then
echo "Docker compose up succeeded on attempt $attempt"
break
fi
if [ $attempt -eq 3 ]; then
echo "::error::Docker compose up failed after 3 attempts"
exit 1
fi
echo "::warning::Docker compose up attempt $attempt failed, retrying in 30s..."
docker compose down 2>/dev/null || true
sleep 30
done
if ${{ inputs.maildev }}; then
echo -e "${Y}checking maildev container..." &&

View File

@@ -26,7 +26,7 @@ runs:
steps:
- name: find_release_artifact
id: find_release_artifact
uses: actions/github-script@v7
uses: actions/github-script@v9
with:
result-encoding: string
script: | # js

View File

@@ -34,7 +34,7 @@ runs:
steps:
- name: Determine Docker info
id: docker-info
uses: actions/github-script@v7
uses: actions/github-script@v9
with:
script: | # js
const branch = '${{ inputs.branch }}';

View File

@@ -18,7 +18,7 @@ runs:
using: "composite"
steps:
- name: Remove previous pull-request comment
uses: actions/github-script@v7
uses: actions/github-script@v9
if: ${{ inputs.remove-previous-comment == 'true' && always() && github.event_name == 'pull_request' }}
with:
script: |
@@ -53,7 +53,7 @@ runs:
}
- name: Notify pull-request
uses: actions/github-script@v7
uses: actions/github-script@v9
if: ${{ inputs.add-new-comment == 'true' && always() && github.event_name == 'pull_request' }}
id: notify-pull-request
with:

View File

@@ -13,7 +13,7 @@ runs:
using: "composite"
steps:
- name: Prepare JDK ${{ inputs.java-version }}
uses: actions/setup-java@v4
uses: actions/setup-java@v5.2.0
with:
java-version: ${{ inputs.java-version }}
distribution: "temurin"
@@ -37,14 +37,14 @@ runs:
id: get-cache-keys
- name: Install uv (for Python package installation)
uses: astral-sh/setup-uv@v7
uses: astral-sh/setup-uv@08807647e7069bb48b6ef5acd8ec9567f424441b # v8.1.0
with:
enable-cache: false
version: "0.10.2" # keep in sync with mise.toml
github-token: ${{ github.token }}
- name: Restore M2 cache
uses: actions/cache/restore@v4
uses: actions/cache/restore@v5
with:
path: |
~/.m2

View File

@@ -16,7 +16,7 @@ runs:
using: "composite"
steps:
- name: Install Chrome v144
uses: browser-actions/setup-chrome@v2
uses: browser-actions/setup-chrome@2e1d749697dd1612b833dba4a722266286fbefcd # v2.1.2
with:
chrome-version: 144
id: setup-chrome
@@ -26,7 +26,7 @@ runs:
id: get-cache-keys
- name: Get Cypress cache
uses: actions/cache/restore@v4
uses: actions/cache/restore@v5
with:
path: ~/.cache/Cypress
key: ${{ steps.get-cache-keys.outputs.cypress-cache-key }}

View File

@@ -21,7 +21,7 @@ runs:
id: get-cache-keys
- name: Restore M2 cache
uses: actions/cache/restore@v4
uses: actions/cache/restore@v5
with:
path: |
~/.m2
@@ -39,7 +39,7 @@ runs:
shell: bash
- name: Restore node_modules cache
uses: actions/cache/restore@v4
uses: actions/cache/restore@v5
id: node-modules-cache
with:
path: |

View File

@@ -8,13 +8,13 @@ runs:
using: "composite"
steps:
- name: Prepare Node.js
uses: actions/setup-node@v4
uses: actions/setup-node@v6.4.0
with:
node-version: ${{ inputs.node-version }}
node-version-file: ".nvmrc"
- name: Install bun
uses: oven-sh/setup-bun@v2
uses: oven-sh/setup-bun@0c5077e51419868618aeaa5fe8019c62421857d6 # v2.2.0
with:
bun-version: "1.3.7"
bun-version: "1.3.14"
token: ${{ github.token }}

View File

@@ -4,9 +4,9 @@ description: Set up bun and build release scripts
runs:
using: composite
steps:
- uses: oven-sh/setup-bun@v2
- uses: oven-sh/setup-bun@0c5077e51419868618aeaa5fe8019c62421857d6 # v2.2.0
with:
bun-version: "1.3.7"
bun-version: "1.3.14"
token: ${{ github.token }}
- name: Build release scripts

View File

@@ -54,7 +54,7 @@ runs:
# We have to use the latest version that does not have this issue https://github.com/docker/compose/pull/12752
- name: Set up Docker Compose
uses: docker/setup-compose-action@v1
uses: docker/setup-compose-action@8cccb8c14b6500aaffebff1aa49c502c34d2e5e6 # v2.1.0
with:
version: latest
@@ -67,7 +67,7 @@ runs:
output-name: "metabase.jar"
- name: Retrieve Embedding SDK dist artifact
uses: actions/download-artifact@v4
uses: actions/download-artifact@v8
with:
name: embedding-sdk-${{ inputs.commit }}
path: ${{ github.workspace }}/resources/embedding-sdk
@@ -95,7 +95,7 @@ runs:
- name: Configure AWS credentials
if: ${{ steps.check-sample-app-branch.outputs.sample_app_branch_exists == 'true' }}
uses: aws-actions/configure-aws-credentials@v4
uses: aws-actions/configure-aws-credentials@d979d5b3a71173a29b74b5b88418bfda9437d885
with:
role-to-assume: ${{ inputs.shoppy-iam-role }}
role-session-name: GitHub_to_AWS_via_FederatedOIDC

View File

@@ -17,7 +17,7 @@ runs:
run: sha256sum ./target/uberjar/metabase.jar > SHA256.sum
shell: bash
- name: Upload JARs as artifact
uses: actions/upload-artifact@v4
uses: actions/upload-artifact@v7
with:
name: metabase-${{ inputs.edition }}-${{ github.event.pull_request.head.sha || github.sha }}-uberjar
path: |

View File

@@ -0,0 +1,66 @@
name: Resolve current job id
description: >
Resolve this job's GitHub workflow_jobs.id via the Actions API and export it
as JOB_ID (and JOB_HTML_URL) to $GITHUB_ENV, so any later step can read the
numeric job id without making its own API call. Best-effort: warns and
proceeds on failure so it can never break the run.
runs:
using: composite
steps:
- name: Resolve JOB_ID
shell: bash
env:
# Use the built-in per-job token, not a PAT: this lookup is best-effort
# (a miss just leaves JOB_ID unset, never breaks the run) and other jobs
# depend on the automation PAT's rate-limit budget — we don't want ~70
# e2e shards drawing it down. Worst case under heavy CI load is the
# occasional unresolved job_id, which is acceptable. DEV-2117.
GH_TOKEN: ${{ github.token }}
RUNNER_NAME: ${{ runner.name }}
run: |
# GitHub doesn't expose the current job's numeric id, so we look it up
# from the Actions API. A runner runs exactly one job at a time, so the
# in-progress job with this runner_name within (run_id, run_attempt) is
# unique. We require status == in_progress and no completed_at so a
# reused runner can't match a job it already finished this attempt, with
# max_by(.started_at) as a final tiebreaker.
#
# $GITHUB_RUN_ID is the whole CI run, whose total job count exceeds the
# 100-per-page cap (~185 jobs), so this shard's job is frequently NOT on
# the first page. --paginate --slurp pulls every page and slurps them
# into one array so the filter sees all jobs, not just the first 100 —
# without it the lookup returned null whenever this job sorted past
# position 100 (DEV-2117).
#
# gh rejects `--slurp` together with `--jq` ("the --slurp option is not
# supported with --jq"), so we slurp with gh and run the filter through a
# standalone `jq`. The earlier `--slurp --jq` form errored on every
# attempt — swallowed by 2>/dev/null — leaving JOB_ID permanently unset.
set +e
api_url="/repos/$GITHUB_REPOSITORY/actions/runs/$GITHUB_RUN_ID/attempts/$GITHUB_RUN_ATTEMPT/jobs?per_page=100"
jq_filter="[.[].jobs[] | select(.runner_name == \"$RUNNER_NAME\" and .status == \"in_progress\" and .completed_at == null)] | max_by(.started_at) | \"\(.id) \(.html_url)\""
# The Actions jobs API is eventually consistent: after a runner picks
# up a job, it can take a few seconds for runner_name to appear in the
# response. Retry briefly to absorb that.
out=""
for attempt in 1 2 3 4 5; do
out=$(gh api --paginate --slurp "$api_url" 2>/dev/null | jq -r "$jq_filter" 2>/dev/null)
if [ -n "$out" ] && [ "$out" != "null null" ]; then
break
fi
out=""
echo "Attempt $attempt: job not yet visible in Actions API; retrying..."
sleep 3
done
if [ -n "$out" ]; then
read -r job_id job_url <<<"$out"
echo "JOB_ID=$job_id" >> "$GITHUB_ENV"
echo "JOB_HTML_URL=$job_url" >> "$GITHUB_ENV"
echo "Resolved JOB_ID=$job_id (runner=$RUNNER_NAME)"
else
echo "::warning::Could not resolve current job id after 5 attempts (runner=$RUNNER_NAME)"
fi
exit 0

View File

@@ -39,21 +39,19 @@ runs:
- name: Extract embedding-sdk bundles from current uberjar
shell: bash
run: |
# Try new layout first (app/embedding-sdk/legacy/), fall back to old layout (app/embedding-sdk.js)
unzip -j temp/current-uberjar/metabase.jar \
"frontend_client/app/embedding-sdk/legacy/embedding-sdk.js" \
-d temp/current-bundle 2>/dev/null || \
unzip -j temp/current-uberjar/metabase.jar \
# Extract preserving the jar's internal paths under resources/ so the
# shared measure script sees the same layout as a source build. Both the
# new (embedding-sdk/legacy + chunks) and old (embedding-sdk.js) layouts
# are pulled; whichever exists is what gets measured.
mkdir -p temp/current/resources
unzip -o temp/current-uberjar/metabase.jar \
"frontend_client/app/embedding-sdk/*" \
-d temp/current/resources 2>/dev/null || true
unzip -o temp/current-uberjar/metabase.jar \
"frontend_client/app/embedding-sdk.js" \
-d temp/current-bundle
# Extract chunks directory if it exists (new layout only)
mkdir -p temp/current-chunks
unzip -j temp/current-uberjar/metabase.jar \
"frontend_client/app/embedding-sdk/chunks/*" \
-d temp/current-chunks 2>/dev/null || true
-d temp/current/resources 2>/dev/null || true
- name: Checkout base ref
uses: actions/checkout@v4
uses: actions/checkout@v6
with:
ref: ${{ inputs.base-ref-commit }}
path: temp/base-ref-src
@@ -63,49 +61,16 @@ runs:
cd temp/base-ref-src
bun install
MB_EDITION=${{ inputs.edition }} bun run build-release
- name: Measure current and base bundle sizes
shell: bash
env:
MEASURE_BUNDLES: embedding-sdk-legacy,embedding-sdk-chunked
run: |
MEASURE_ROOT=temp/current node ./.github/scripts/measure-bundle-sizes.js > temp/current-sizes.json
MEASURE_ROOT=temp/base-ref-src node ./.github/scripts/measure-bundle-sizes.js > temp/base-sizes.json
- name: Compare sizes and set output
id: compare
shell: bash
run: |
# --- Legacy (monolithic) bundle comparison ---
current_size=$(wc -c < temp/current-bundle/embedding-sdk.js) || { echo "::error::Failed to read current bundle"; exit 1; }
# Base ref: try new layout first, fall back to old
if [ -f "temp/base-ref-src/resources/frontend_client/app/embedding-sdk/legacy/embedding-sdk.js" ]; then
base_ref_size=$(wc -c < "temp/base-ref-src/resources/frontend_client/app/embedding-sdk/legacy/embedding-sdk.js")
else
base_ref_size=$(wc -c < "temp/base-ref-src/resources/frontend_client/app/embedding-sdk.js")
fi
[ "$current_size" -gt 0 ] && [ "$base_ref_size" -gt 0 ] || { echo "::error::Bundle file is empty"; exit 1; }
diff=$((current_size - base_ref_size))
percent=$(( diff * 100 / base_ref_size ))
echo "=== Legacy (monolithic) bundle ==="
echo "Current size: $current_size bytes"
echo "Base ref size: $base_ref_size bytes"
echo "Difference: $diff bytes ($percent%)"
threshold="${{ inputs.threshold-percent }}"
if [ "$percent" -gt "$threshold" ]; then
status="increased"
elif [ "$percent" -lt "$(( -threshold ))" ]; then
status="decreased"
else
status="stable"
fi
echo "status=$status" >> $GITHUB_OUTPUT
echo "size_change_percent=$percent" >> $GITHUB_OUTPUT
# --- Chunked bundle (informational) ---
if ls temp/current-chunks/*.js 1>/dev/null 2>&1; then
chunks_size=$(cat temp/current-chunks/*.js | wc -c)
echo ""
echo "=== Chunked bundle (bootstrap + chunks) ==="
echo "Total chunked size: $chunks_size bytes"
echo "chunks_size=$chunks_size" >> $GITHUB_OUTPUT
else
echo ""
echo "No chunked bundle found (expected on master)"
fi
env:
THRESHOLD: ${{ inputs.threshold-percent }}
run: node ./.github/actions/sdk-bundle-size-check/compare.js temp/current-sizes.json temp/base-sizes.json

View File

@@ -0,0 +1,54 @@
/* eslint-disable import/no-commonjs */
// Diffs the embedding SDK bundle sizes produced by measure-bundle-sizes.js for
// the current build and the base ref. The gate is driven by the legacy
// (monolithic) bundle's raw bytes; the chunked delivery is reported for context.
const fs = require("fs");
const [, , currentPath, basePath] = process.argv;
const threshold = Number(process.env.THRESHOLD ?? 2);
const current = JSON.parse(fs.readFileSync(currentPath, "utf8"));
const base = JSON.parse(fs.readFileSync(basePath, "utf8"));
const pick = (rows, bundle, kind) => rows.find(row => row.bundle === bundle && row.kind === kind);
const setOutput = (name, value) => {
if (process.env.GITHUB_OUTPUT) {
fs.appendFileSync(process.env.GITHUB_OUTPUT, `${name}=${value}\n`);
}
};
const mb = bytes => (bytes / 1024 / 1024).toFixed(2);
const legacyCurrent = pick(current, "embedding-sdk-legacy", "total");
const legacyBase = pick(base, "embedding-sdk-legacy", "total");
if (!legacyCurrent || !legacyBase || !legacyBase.rawBytes) {
console.error("::error::Could not find embedding-sdk-legacy size in both builds");
process.exit(1);
}
const diff = legacyCurrent.rawBytes - legacyBase.rawBytes;
const percent = Math.trunc((diff * 100) / legacyBase.rawBytes);
const status = percent > threshold ? "increased" : percent < -threshold ? "decreased" : "stable";
console.log("=== Embedding SDK legacy (monolithic) bundle ===");
console.log(`Current: ${legacyCurrent.rawBytes} bytes raw / ${mb(legacyCurrent.gzipBytes)}MB gzip`);
console.log(`Base ref: ${legacyBase.rawBytes} bytes raw / ${mb(legacyBase.gzipBytes)}MB gzip`);
console.log(`Difference: ${diff} bytes (${percent}%), threshold ${threshold}%`);
console.log(`Status: ${status}`);
const chunkedCurrent = pick(current, "embedding-sdk-chunked", "total");
const chunkedBase = pick(base, "embedding-sdk-chunked", "total");
if (chunkedCurrent && chunkedBase) {
const chunkedDiff = chunkedCurrent.rawBytes - chunkedBase.rawBytes;
console.log("");
console.log("=== Embedding SDK chunked (informational) ===");
console.log(`Current: ${chunkedCurrent.rawBytes} bytes raw / ${mb(chunkedCurrent.gzipBytes)}MB gzip`);
console.log(`Base ref: ${chunkedBase.rawBytes} bytes raw / ${mb(chunkedBase.gzipBytes)}MB gzip`);
console.log(`Difference: ${chunkedDiff} bytes`);
}
setOutput("status", status);
setOutput("size_change_percent", percent);

View File

@@ -45,7 +45,7 @@ runs:
# whether to fail or to pass the whole job, based on the quarantine list.
continue-on-error: true
- name: Publish Test Report (JUnit)
uses: dorny/test-reporter@v1
uses: dorny/test-reporter@a43b3a5f7366b97d083190328d2c652e1a8b6aa2 # v3.0.0
# As per the documentation: https://docs.github.com/en/actions/reference/workflows-and-actions/contexts#steps-context
# test.outcome gives us access to the result before `continue-on-error` overrides it and sets it to 'success'.
# We want test reports even if the test step gets cancelled (e.g. times out), hence we match anything that is not a 'success'.
@@ -59,7 +59,7 @@ runs:
fail-on-error: false
- name: Upload Logs on Failure
if: always() && steps.run-test.outcome != 'success'
uses: actions/upload-artifact@v4
uses: actions/upload-artifact@v7
with:
name: ${{ format('test-logs-{0}{1}', github.job, inputs.logs-artifact-suffix && format('-{0}', inputs.logs-artifact-suffix) || '') }}
path: logs/test-log.json

View File

@@ -17,7 +17,7 @@ runs:
steps:
- name: Check if the cache entry exists
id: cache-lookup
uses: actions/cache/restore@v4
uses: actions/cache/restore@v5
with:
path: ${{ inputs.path }}
key: ${{ inputs.key }}
@@ -35,7 +35,7 @@ runs:
GH_TOKEN: ${{ github.token }}
shell: bash
- name: Save the new cache entry
uses: actions/cache/save@v4
uses: actions/cache/save@v5
with:
path: ${{ inputs.path }}
key: ${{ inputs.key }}

View File

@@ -21,12 +21,25 @@ inputs:
default: ""
previous-step-outcome:
required: true
driver-status:
required: false
default: ""
description: >-
Configured status for this driver ("info", "skip", "required", or empty). When
"info", the job is reported as passing regardless of the real test outcome, and
that outcome is surfaced via a job summary and (on PRs) a comment instead.
report-variant:
required: false
default: ""
description: >-
Optional label disambiguating matrix legs of the same job, so the info-driver
job summary and PR comment stay distinct per leg.
runs:
using: "composite"
steps:
- name: Configure AWS credentials
uses: aws-actions/configure-aws-credentials@v4
uses: aws-actions/configure-aws-credentials@d979d5b3a71173a29b74b5b88418bfda9437d885
with:
role-to-assume: ${{ inputs.role-to-assume }}
aws-region: ${{ inputs.aws-region }}
@@ -50,8 +63,11 @@ runs:
# Important!
# 1. This step must always run because it controls job's exit code.
# 2. There's no point in running this outside the organization
# 3. For "info" drivers our own status takes precedence: we let this step pass so
# the job is reported as passing regardless of the real outcome (still uploaded).
if: always() && github.repository_owner == 'metabase'
uses: trunk-io/analytics-uploader@main
continue-on-error: ${{ inputs.driver-status == 'info' }}
uses: trunk-io/analytics-uploader@1198efcc0976501147b10a7e343c3fa069527526 # v2.0.9
env:
TRUNK_REPO_URL: git@github.com:metabase/metabase.git
with:
@@ -62,3 +78,105 @@ runs:
token: ${{ inputs.trunk-api-token }}
use-cache: true
previous-step-outcome: ${{ inputs.previous-step-outcome }}
# For "info" drivers the job is forced green above, so surface the real outcome here —
# but only when it did NOT pass, to avoid spamming summaries/comments for healthy runs.
# Never include logs or stack traces — only the real status and a link to the run.
- name: Report info-level driver result (job summary)
if: ${{ always() && inputs.driver-status == 'info' && inputs.previous-step-outcome == 'failure' }}
shell: bash
env:
OUTCOME: ${{ inputs.previous-step-outcome }}
LABEL: ${{ inputs.output-name }}
VARIANT: ${{ inputs.report-variant }}
run: | # sh
if [ "$OUTCOME" = "success" ]; then EMOJI="✅"; else EMOJI="❌"; fi
TITLE="$LABEL"
[ -n "$VARIANT" ] && TITLE="$LABEL ($VARIANT)"
RUN_URL="$GITHUB_SERVER_URL/$GITHUB_REPOSITORY/actions/runs/$GITHUB_RUN_ID"
{
echo "### ℹ️ Info-level driver: $TITLE"
echo ""
echo "Reported as passing (non-blocking). Real result: $EMOJI \`$OUTCOME\`"
echo ""
echo "[View run]($RUN_URL)"
} >> "$GITHUB_STEP_SUMMARY"
# Maintain ONE PR comment per commit for all info drivers. Each leg upserts its own line
# (keyed by a hidden tag); a passing leg removes its line, and the comment is deleted once
# no failing legs remain. Comments from older commits are cleaned up. Runs on pass and fail
# so recoveries self-heal. Never includes logs or stack traces — only status and a run link.
- name: Sync info-level driver result into the PR comment
if: ${{ always() && inputs.driver-status == 'info' && github.event_name == 'pull_request' }}
uses: actions/github-script@v9
continue-on-error: true
env:
OUTCOME: ${{ inputs.previous-step-outcome }}
LABEL: ${{ inputs.output-name }}
VARIANT: ${{ inputs.report-variant }}
with:
script: | # js
const { OUTCOME, LABEL, VARIANT, GITHUB_SERVER_URL, GITHUB_REPOSITORY, GITHUB_RUN_ID } = process.env;
const pr = context.payload.pull_request;
if (!pr) return;
const { owner, repo } = context.repo;
const sha = pr.head.sha;
// PREFIX matches the current per-commit marker AND the older per-leg one, so legacy
// comments from a previous version get cleaned up on the next sync.
const PREFIX = "<!-- info-driver-result";
const marker = `<!-- info-driver-results:${sha} -->`;
const header = `${marker}\nℹ️ **Info-level drivers** failed (non-blocking) on \`${sha.slice(0, 7)}\` — these do not gate:`;
const lineKey = `${LABEL}:${VARIANT}`;
const title = VARIANT ? `${LABEL} (${VARIANT})` : LABEL;
const runUrl = `${GITHUB_SERVER_URL}/${GITHUB_REPOSITORY}/actions/runs/${GITHUB_RUN_ID}`;
// Only real failures are reported; cancelled/skipped/success remove this leg's line.
// List-item line with the hidden key tag at the END so markdown (links) renders.
const ourLine = OUTCOME === "failure"
? `- **${title}** — ❌ \`${OUTCOME}\` ([Logs](${runUrl})) <!--l:${lineKey}-->`
: null;
const linesOf = (body) => body.split("\n").filter((l) => l.includes("<!--l:"));
const sync = async () => {
const { data: comments } = await github.rest.issues.listComments({
owner, repo, issue_number: pr.number, per_page: 100,
});
const tagged = comments.filter((c) => c.body.includes(PREFIX));
// Drop comments left over from older commits.
for (const c of tagged.filter((c) => !c.body.includes(marker))) {
await github.rest.issues.deleteComment({ owner, repo, comment_id: c.id }).catch(() => {});
}
const mine = tagged
.filter((c) => c.body.includes(marker))
.sort((a, b) => new Date(a.created_at) - new Date(b.created_at));
// Merge any duplicates raced into existence; latest write of each line wins.
const byKey = new Map();
for (const c of mine) {
for (const l of linesOf(c.body)) {
const m = l.match(/<!--l:(.*?)-->/);
if (m) byKey.set(m[1], l);
}
}
if (ourLine) byKey.set(lineKey, ourLine);
else byKey.delete(lineKey);
const lines = [...byKey.values()].sort();
const target = mine[0];
for (const c of mine.slice(1)) {
await github.rest.issues.deleteComment({ owner, repo, comment_id: c.id }).catch(() => {});
}
if (lines.length === 0) {
if (target) await github.rest.issues.deleteComment({ owner, repo, comment_id: target.id }).catch(() => {});
return;
}
const body = [header, "", ...lines].join("\n");
if (target) {
await github.rest.issues.updateComment({ owner, repo, comment_id: target.id, body });
} else {
await github.rest.issues.createComment({ owner, repo, issue_number: pr.number, body });
}
};
// Retry to shrink the window where parallel legs clobber each other's update.
for (let i = 0; i < 4; i++) {
try { await sync(); break; }
catch (e) { if (i === 3) throw e; await new Promise((r) => setTimeout(r, 400 * (i + 1))); }
}

View File

@@ -1,3 +1,8 @@
# Catch-all filter — emits the full changed-files list as `all_changed_files`
# so create-test-plan in run-tests.yml can reuse dorny's diff.
all_changed:
- "**"
default: &default
- "locales/**"
- "bin/**/!(*.md)"
@@ -77,6 +82,19 @@ frontend_loki_ci: &frontend_loki_ci
- ".loki/**"
- ".storybook/**"
frontend_unit_infra: &frontend_unit_infra
- *frontend_configs
- *frontend_ci
- "jest.config.js"
- "jest.tz.unit.conf.json"
- "frontend/test/**"
- "frontend/**/__mocks__/**"
frontend_loki_infra: &frontend_loki_infra
- *frontend_configs
- *shared_ci
- *frontend_loki_ci
backend_ci: &backend_ci
- *shared_ci
- ".github/actions/test-driver/**"
@@ -177,6 +195,9 @@ documentation:
- "docs/**"
- "**/*.md"
docs:
- "docs/**"
embedding_documentation:
- ".typedoc/**"
- "docs/embedding/**"

116
.github/scripts/affected-modules.ts vendored Normal file
View File

@@ -0,0 +1,116 @@
export type ModuleDef = { type: string; pattern: string };
export type Rule = { from: string[]; allow: string[] };
type ModuleNode = { type: string; regex: RegExp };
export type ModuleGraph = {
nodes: ModuleNode[];
dependents: Map<string, Set<string>>;
};
/**
* Compiles a glob pattern to an anchored RegExp.
*/
export function globToRegex(glob: string): RegExp {
const escapeSegment = (seg: string) =>
seg.replace(/[.+?^${}()|[\]\\]/g, "\\$&").replace(/\*/g, "[^/]*");
const body = glob.split("**").map(escapeSegment).join(".*");
return new RegExp(`^${body}$`);
}
export function buildModuleGraph(
elements: ModuleDef[],
rules: Rule[],
): ModuleGraph {
const nodes = elements.map((el) => ({
type: el.type,
regex: globToRegex(el.pattern),
}));
const allTypes = elements.map((e) => e.type);
function expandPattern(pattern: string): string[] {
if (pattern === "*") {
return [...allTypes];
}
if (pattern.endsWith("/*")) {
const prefix = pattern.slice(0, -1);
return allTypes.filter((type) => type.startsWith(prefix));
}
return allTypes.includes(pattern) ? [pattern] : [];
}
// dependents.get(M) is the set of modules that depend on M
// i.e. modules that are affected when M changes
const dependents = new Map<string, Set<string>>(
allTypes.map((type) => [type, new Set()]),
);
for (const rule of rules) {
const fromTypes = rule.from.flatMap(expandPattern);
const allowTypes = rule.allow.flatMap(expandPattern);
for (const target of allowTypes) {
for (const importer of fromTypes) {
if (importer !== target) {
dependents.get(target)?.add(importer);
}
}
}
}
return { nodes, dependents };
}
/**
* Returns the module type that owns the given file.
* First match wins, so element ordering matters for nested patterns.
*/
export function mapFileToModule(
moduleGraph: ModuleGraph,
path: string,
): string | null {
for (const el of moduleGraph.nodes) {
if (el.regex.test(path)) {
return el.type;
}
}
return null;
}
/**
* Returns the set of module types containing any of the changed files.
*/
export function getChangedModules(
moduleGraph: ModuleGraph,
changedFiles: string[],
): Set<string> {
const direct = new Set<string>();
for (const file of changedFiles) {
const module = mapFileToModule(moduleGraph, file);
if (module) {
direct.add(module);
}
}
return direct;
}
/**
* Returns the changed modules and all modules that directly or indirectly depend on them
*/
export function getAffectedModules(
moduleGraph: ModuleGraph,
changedFiles: string[],
): Set<string> {
const direct = getChangedModules(moduleGraph, changedFiles);
const affected = new Set(direct);
const queue = [...direct];
while (queue.length > 0) {
const module = queue.shift()!;
for (const dep of moduleGraph.dependents.get(module) ?? []) {
if (!affected.has(dep)) {
affected.add(dep);
queue.push(dep);
}
}
}
return affected;
}

View File

@@ -0,0 +1,101 @@
import {
buildModuleGraph,
getAffectedModules,
getChangedModules,
mapFileToModule,
} from "./affected-modules";
const ELEMENTS = [
{ type: "lib/utils", pattern: "src/utils/**" },
{ type: "lib/types", pattern: "src/types/**" },
// Nested-pattern pair (mirrors mlv1-inside-mlv2): inner must come first
// so first-match-wins picks it up before the outer pattern.
{ type: "lib/inner", pattern: "src/shared/v1/**" },
{ type: "lib/outer", pattern: "src/shared/**" },
{ type: "feature/foo", pattern: "src/foo/**" },
{ type: "feature/bar", pattern: "src/bar/**" },
{ type: "feature/super", pattern: "src/super/**" },
// Different folder root (mirrors feature/enterprise's enterprise/ path).
{ type: "feature/external", pattern: "external/lib/**" },
// Same type, two patterns (mirrors app/misc per-entry-point declarations).
{ type: "app/main", pattern: "src/app.js" },
{ type: "app/main", pattern: "src/embed.tsx" },
{ type: "shared/other", pattern: "src/*/**" }, // catch-all, must be last
];
const RULES = [
...ELEMENTS.map((el) => ({ from: [el.type], allow: [el.type] })),
{ from: ["lib/*"], allow: ["lib/*"] },
{ from: ["feature/*"], allow: ["lib/*"] },
{ from: ["feature/super"], allow: ["feature/*"] },
{ from: ["app/*"], allow: ["lib/*", "feature/*", "app/*"] },
];
const graph = buildModuleGraph(ELEMENTS, RULES);
describe("affected modules", () => {
describe("mapFileToModule", () => {
it.each([
["src/utils/colors.ts", "lib/utils"],
["src/types/api.ts", "lib/types"],
["src/foo/foo.tsx", "feature/foo"],
["src/bar/bar.tsx", "feature/bar"],
["src/super/index.ts", "feature/super"],
["src/app.js", "app/main"],
["src/randomthing/file.ts", "shared/other"],
["src/shared/v1/expr.ts", "lib/inner"],
["src/shared/v2/expr.ts", "lib/outer"],
["src/shared/index.ts", "lib/outer"],
["external/lib/foo.ts", "feature/external"],
["src/embed.tsx", "app/main"],
])("should map %s to %s", (path, expected) => {
expect(mapFileToModule(graph, path)).toBe(expected);
});
it("should return null for files outside any pattern", () => {
expect(mapFileToModule(graph, "docs/foo.md")).toBe(null);
expect(mapFileToModule(graph, "README.md")).toBe(null);
expect(mapFileToModule(graph, "external/notlib/foo.ts")).toBe(null);
});
});
describe("getChangedModules", () => {
it("should return the unique set of modules whose files changed", () => {
const result = getChangedModules(graph, [
"src/utils/colors.ts",
"src/foo/x.ts",
"src/foo/y.ts",
"docs/unrelated.md",
]);
expect([...result].sort()).toEqual(["feature/foo", "lib/utils"]);
});
it("should return an empty set when no file maps to a module", () => {
expect(getChangedModules(graph, ["docs/foo.md"]).size).toBe(0);
});
});
describe("getAffectedModules", () => {
it("should affect feature/super and app/main but not feature/bar when feature/foo changes", () => {
const result = getAffectedModules(graph, ["src/foo/x.ts"]);
expect([...result].sort()).toEqual([
"app/main",
"feature/foo",
"feature/super",
]);
});
it("should affect many modules when a lib module changes", () => {
const result = getAffectedModules(graph, ["src/utils/colors.ts"]);
expect(result.has("lib/utils")).toBe(true);
expect(result.has("feature/foo")).toBe(true);
expect(result.has("feature/bar")).toBe(true);
expect(result.has("feature/super")).toBe(true);
expect(result.has("app/main")).toBe(true);
});
it("should return empty when no file maps to a module", () => {
expect(getAffectedModules(graph, ["docs/foo.md"]).size).toBe(0);
});
});
});

140
.github/scripts/affected-tests.ts vendored Normal file
View File

@@ -0,0 +1,140 @@
// Per-suite affected-tests logic: combines module-affected closure with
// per-suite "infra-touched" booleans (computed by dorny/paths-filter against
// the `frontend_unit_infra` / `frontend_loki_infra` anchors in file-paths.yaml).
// When infra files change, the whole suite runs in full.
import {
type ModuleDef,
type ModuleGraph,
type Rule,
buildModuleGraph,
getAffectedModules,
getChangedModules,
mapFileToModule,
} from "./affected-modules";
export type TestSuiteDef = {
statsPrefix: string;
runAllTests?: boolean;
};
export type TestPlanStats = {
modules_changed: number;
modules_affected: number;
modules_affected_list: string[];
fe_unit_specs_total: number;
fe_unit_specs_run: number;
fe_unit_specs_skipped: number;
loki_stories_total: number;
loki_stories_run: number;
loki_stories_skipped: number;
e2e_specs_total: number;
e2e_specs_run: number;
e2e_specs_skipped: number;
};
export type TestPlan = {
stats: TestPlanStats;
fe_unit_specs_to_run: string[];
loki_stories_to_run: string[];
e2e_specs_to_run: string[];
};
export const TEST_SUITES = {
unit: {
statsPrefix: "fe_unit_specs",
},
loki: {
statsPrefix: "loki_stories",
},
e2e: {
statsPrefix: "e2e_specs",
// We don't map changed files to e2e tests yet, so for now just run them all
runAllTests: true,
},
} satisfies Record<string, TestSuiteDef>;
export function createTestPlan({
elements,
rules,
testSuites,
changedFiles,
testFilesBySuite,
infraTouchedBySuite,
}: {
elements: ModuleDef[];
rules: Rule[];
testSuites: Record<string, TestSuiteDef>;
changedFiles: string[];
testFilesBySuite: Record<string, string[]>;
infraTouchedBySuite: Record<string, boolean>;
}): TestPlan {
const moduleGraph = buildModuleGraph(elements, rules);
const changedModules = getChangedModules(moduleGraph, changedFiles);
const modulesAffected = getAffectedModules(moduleGraph, changedFiles);
const modulesAffectedList = [...modulesAffected].sort();
const specsToRun: Record<string, string[]> = {};
const stats: Record<string, number | string[]> = {
modules_changed: changedModules.size,
modules_affected: modulesAffected.size,
modules_affected_list: modulesAffectedList,
};
for (const [name, testSuite] of Object.entries(testSuites)) {
const files = testFilesBySuite[name] ?? [];
const tests = createTestPlanForSuite(
moduleGraph,
testSuite,
infraTouchedBySuite[name] ?? false,
changedFiles,
files,
);
specsToRun[name] = tests;
stats[`${testSuite.statsPrefix}_total`] = files.length;
stats[`${testSuite.statsPrefix}_run`] = tests.length;
stats[`${testSuite.statsPrefix}_skipped`] = files.length - tests.length;
}
return {
stats: stats as TestPlanStats,
fe_unit_specs_to_run: specsToRun.unit ?? [],
loki_stories_to_run: specsToRun.loki ?? [],
e2e_specs_to_run: specsToRun.e2e ?? [],
};
}
/**
* Computes the test plan for one test suite
*
* Runs the full suite when `runAllTests` is set or `infraTouched` is true,
* otherwise runs the tests based on affected-modules.
*/
export function createTestPlanForSuite(
moduleGraph: ModuleGraph,
testSuite: TestSuiteDef,
infraTouched: boolean,
changedFiles: string[],
allTestFiles: string[],
): string[] {
if (testSuite.runAllTests || infraTouched) {
return allTestFiles;
}
const affected = getAffectedModules(moduleGraph, changedFiles);
return filterAffectedTests(moduleGraph, affected, allTestFiles);
}
/**
* Filters the test-file list down to tests whose owning module is affected.
*/
export function filterAffectedTests(
moduleGraph: ModuleGraph,
affected: Set<string>,
testFiles: string[],
): string[] {
return testFiles.filter((file) => {
const module = mapFileToModule(moduleGraph, file);
return module !== null && affected.has(module);
});
}

View File

@@ -0,0 +1,190 @@
import { buildModuleGraph } from "./affected-modules";
import {
createTestPlan,
createTestPlanForSuite,
filterAffectedTests,
} from "./affected-tests";
const ELEMENTS = [
{ type: "lib/utils", pattern: "src/utils/**" },
{ type: "feature/foo", pattern: "src/foo/**" },
{ type: "feature/bar", pattern: "src/bar/**" },
{ type: "shared/other", pattern: "src/*/**" },
];
const RULES = [
...ELEMENTS.map((el) => ({ from: [el.type], allow: [el.type] })),
{ from: ["lib/*"], allow: ["lib/*"] },
{ from: ["feature/*"], allow: ["lib/*"] },
];
const TEST_SUITE_DEFS = {
unit: { statsPrefix: "fe_unit_specs" },
loki: { statsPrefix: "loki_stories" },
e2e: { statsPrefix: "e2e_specs", runAllTests: true },
};
const UNIT_FILES = [
"src/foo/foo.unit.spec.ts",
"src/foo/bar.unit.spec.ts",
"src/bar/bar.unit.spec.ts",
"src/utils/utils.unit.spec.ts",
];
const LOKI_FILES = ["src/foo/Foo.stories.tsx", "src/bar/Bar.stories.tsx"];
const E2E_FILES = ["e2e/test/scenarios/a.cy.spec.ts"];
const moduleGraph = buildModuleGraph(ELEMENTS, RULES);
describe("affected tests", () => {
describe("createTestPlanForSuite", () => {
it("should run all tests when infra is touched", () => {
const result = createTestPlanForSuite(
moduleGraph,
TEST_SUITE_DEFS.unit,
true, // infraTouched
["src/foo/x.ts"],
UNIT_FILES,
);
expect(result).toEqual(UNIT_FILES);
});
it("should fall back to affected-tests filtering when infra not touched", () => {
const result = createTestPlanForSuite(
moduleGraph,
TEST_SUITE_DEFS.unit,
false, // infraTouched
["src/foo/x.ts"],
UNIT_FILES,
);
expect(result.sort()).toEqual([
"src/foo/bar.unit.spec.ts",
"src/foo/foo.unit.spec.ts",
]);
});
it("should return nothing when infra not touched and no file maps to a module", () => {
const result = createTestPlanForSuite(
moduleGraph,
TEST_SUITE_DEFS.unit,
false,
["docs/foo.md"],
UNIT_FILES,
);
expect(result).toEqual([]);
});
it("should run all tests for runAllTests suites regardless of inputs", () => {
expect(
createTestPlanForSuite(
moduleGraph,
TEST_SUITE_DEFS.e2e,
false,
["docs/foo.md"],
E2E_FILES,
),
).toEqual(E2E_FILES);
expect(
createTestPlanForSuite(
moduleGraph,
TEST_SUITE_DEFS.e2e,
false,
["src/foo/x.ts"],
E2E_FILES,
),
).toEqual(E2E_FILES);
});
});
describe("createTestPlan", () => {
it("should return run lists and stats together", () => {
const result = createTestPlan({
elements: ELEMENTS,
rules: RULES,
testSuites: TEST_SUITE_DEFS,
changedFiles: ["src/foo/x.ts"],
testFilesBySuite: {
unit: UNIT_FILES,
loki: LOKI_FILES,
e2e: E2E_FILES,
},
infraTouchedBySuite: { unit: false, loki: false, e2e: false },
});
expect(result.fe_unit_specs_to_run.sort()).toEqual([
"src/foo/bar.unit.spec.ts",
"src/foo/foo.unit.spec.ts",
]);
expect(result.loki_stories_to_run).toEqual(["src/foo/Foo.stories.tsx"]);
expect(result.e2e_specs_to_run).toEqual(E2E_FILES);
expect(result.stats).toEqual({
modules_changed: 1,
modules_affected: 1,
modules_affected_list: ["feature/foo"],
fe_unit_specs_total: 4,
fe_unit_specs_run: 2,
fe_unit_specs_skipped: 2,
loki_stories_total: 2,
loki_stories_run: 1,
loki_stories_skipped: 1,
e2e_specs_total: 1,
e2e_specs_run: 1,
e2e_specs_skipped: 0,
});
});
it("should run all unit specs when unit infra is touched", () => {
const result = createTestPlan({
elements: ELEMENTS,
rules: RULES,
testSuites: TEST_SUITE_DEFS,
changedFiles: ["docs/foo.md"],
testFilesBySuite: {
unit: UNIT_FILES,
loki: LOKI_FILES,
e2e: E2E_FILES,
},
infraTouchedBySuite: { unit: true, loki: false, e2e: false },
});
expect(result.stats.fe_unit_specs_run).toBe(UNIT_FILES.length);
expect(result.stats.fe_unit_specs_skipped).toBe(0);
// Only the unit suite was triggered.
expect(result.stats.loki_stories_run).toBe(0);
});
it("should not run anything when no infra is touched and no modules affected", () => {
const result = createTestPlan({
elements: ELEMENTS,
rules: RULES,
testSuites: TEST_SUITE_DEFS,
changedFiles: ["docs/foo.md"],
testFilesBySuite: {
unit: UNIT_FILES,
loki: LOKI_FILES,
e2e: E2E_FILES,
},
infraTouchedBySuite: { unit: false, loki: false, e2e: false },
});
expect(result.stats.fe_unit_specs_run).toBe(0);
expect(result.stats.loki_stories_run).toBe(0);
// e2e is runAllTests, so it still reports run = total.
expect(result.stats.e2e_specs_run).toBe(E2E_FILES.length);
});
});
describe("filterAffectedTests", () => {
it("should keep only tests inside affected modules", () => {
const affected = new Set(["feature/foo", "lib/utils"]);
const tests = [
"src/foo/foo.unit.spec.ts",
"src/utils/colors.unit.spec.ts",
"src/bar/bar.unit.spec.ts",
"docs/unrelated.unit.spec.ts",
];
expect(filterAffectedTests(moduleGraph, affected, tests)).toEqual([
"src/foo/foo.unit.spec.ts",
"src/utils/colors.unit.spec.ts",
]);
});
});
});

67
.github/scripts/create-test-plan.ts vendored Normal file
View File

@@ -0,0 +1,67 @@
// Changed files come from dorny/paths-filter's `all_changed_files` output
import { execFileSync } from "node:child_process";
import { elements, rules } from "../../frontend/lint/module-boundaries.mjs";
import { TEST_SUITES, createTestPlan } from "./affected-tests";
const UNIT_GLOBS = [
"frontend/src/**/*.unit.spec.js",
"frontend/src/**/*.unit.spec.jsx",
"frontend/src/**/*.unit.spec.ts",
"frontend/src/**/*.unit.spec.tsx",
"enterprise/frontend/src/**/*.unit.spec.js",
"enterprise/frontend/src/**/*.unit.spec.jsx",
"enterprise/frontend/src/**/*.unit.spec.ts",
"enterprise/frontend/src/**/*.unit.spec.tsx",
];
const STORY_GLOBS = [
"frontend/**/*.stories.js",
"frontend/**/*.stories.jsx",
"frontend/**/*.stories.ts",
"frontend/**/*.stories.tsx",
"enterprise/frontend/**/*.stories.js",
"enterprise/frontend/**/*.stories.jsx",
"enterprise/frontend/**/*.stories.ts",
"enterprise/frontend/**/*.stories.tsx",
];
const E2E_GLOBS = [
"e2e/test/scenarios/**/*.cy.spec.js",
"e2e/test/scenarios/**/*.cy.spec.jsx",
"e2e/test/scenarios/**/*.cy.spec.ts",
"e2e/test/scenarios/**/*.cy.spec.tsx",
];
function listFiles(globs: string[]): string[] {
return execFileSync("git", ["ls-files", "--", ...globs], { encoding: "utf8" })
.split("\n")
.map((line) => line.trim())
.filter(Boolean);
}
const changedFiles = (process.env.CHANGED_FILES ?? "")
.split(",")
.map((s) => s.trim())
.filter(Boolean);
const testPlan = createTestPlan({
elements,
rules,
testSuites: TEST_SUITES,
changedFiles,
testFilesBySuite: {
unit: listFiles(UNIT_GLOBS),
loki: listFiles(STORY_GLOBS),
e2e: listFiles(E2E_GLOBS),
},
infraTouchedBySuite: {
unit: process.env.UNIT_INFRA_TOUCHED === "true",
loki: process.env.LOKI_INFRA_TOUCHED === "true",
e2e: false,
},
});
process.stdout.write(JSON.stringify(testPlan) + "\n");

View File

@@ -19,23 +19,23 @@ function jsonToCsvFormData(jsonData) {
return formData;
}
function uploadCsvToMb({ baseUrl, tableId, jsonData, mode = 'append' }) {
async function uploadCsvToMb({ baseUrl, tableId, jsonData, mode = 'append' }) {
const formData = jsonToCsvFormData(jsonData);
const operation = mode === 'replace' ? 'replace-csv' : 'append-csv';
return fetch(`${baseUrl}/api/table/${tableId}/${operation}`, {
const response = await fetch(`${baseUrl}/api/table/${tableId}/${operation}`, {
method: 'POST',
headers: {
"x-api-key": process.env.API_KEY
},
body: formData
})
.then(response => {
if (!response.ok) {
throw new Error(`Upload failed`);
}
return { success: true };
});
if (!response.ok) {
const body = await response.text().catch(() => '');
throw new Error(`Upload failed: ${response.status} ${response.statusText} ${body}`.trim());
}
return { success: true };
}
module.exports = { uploadCsvToMb };

207
.github/scripts/measure-bundle-sizes.js vendored Normal file
View File

@@ -0,0 +1,207 @@
/* eslint-disable import/no-commonjs */
const fs = require("fs");
const path = require("path");
const zlib = require("zlib");
// MEASURE_ROOT lets callers point at an extracted tree (e.g. the PR bundle-size
// check measures a current and a base-ref tree); defaults to cwd for the stats
// workflow. MEASURE_BUNDLES restricts which bundles are measured (comma-separated).
const baseDir = process.env.MEASURE_ROOT ? path.resolve(process.env.MEASURE_ROOT) : process.cwd();
const resolve = (...relativeParts) => path.resolve(baseDir, ...relativeParts);
const bundleFilter = process.env.MEASURE_BUNDLES
? new Set(process.env.MEASURE_BUNDLES.split(",").map(bundle => bundle.trim()))
: null;
const isBundleWanted = bundle => !bundleFilter || bundleFilter.has(bundle);
const SDK_ROOT = "resources/frontend_client/app/embedding-sdk";
/**
* Async SDK chunks are unreachable today: the bootstrap manifest lists only the
* chunked entry's INITIAL chunks and output.publicPath is "", so import() chunks
* are emitted but never fetched. So they're excluded from "total" (which equals
* "initial"). There's no reliable way to detect loadability from the build
* artifacts, so this is a manual flag: flip it to true in the same change that
* lands the publicPath fix, and "total" will include the lazy chunks while
* "initial" stays the eager first-load set.
*/
const SDK_ASYNC_CHUNKS_LOADABLE = false;
// The chunk runtime is inlined into the bootstrap, never sent as a standalone file.
const SDK_RUNTIME_RE = /embedding-sdk-chunk-runtime\./;
/**
* Simple targets measured straight from emitted files.
* - `dir`/`file`/`altFile`: what to measure (altFile is a fallback single file).
* - `stats`: entrypoint whose initial assets are the "initial" download.
* - `optional`: skip (don't fail) when the output is absent.
* The hosted SDK is split into embedding-sdk-legacy (the self-contained
* monolith) and embedding-sdk-chunked (handled separately below).
*/
const simpleTargets = [
{
bundle: "app",
dir: "resources/frontend_client/app/dist",
stats: { file: "artifacts/stats-main.json", entrypoint: "app-main" },
},
{
bundle: "embedding-sdk-legacy",
dir: `${SDK_ROOT}/legacy`,
altFile: `${SDK_ROOT}.js`,
optional: true,
},
{
bundle: "embedding-sdk-package",
dir: "resources/embedding-sdk/dist",
optional: true,
},
{
bundle: "embed-js",
file: "resources/frontend_client/app/embed.js",
},
];
function sizeOf(files) {
let rawBytes = 0;
let gzipBytes = 0;
let brotliBytes = 0;
let anyBrotli = false;
for (const file of files) {
const contents = fs.readFileSync(file);
rawBytes += contents.length;
// gzip: recompressed, a proxy for the on-the-fly gzip served on every version
gzipBytes += zlib.gzipSync(contents).length;
// brotli: as-served — count the precompressed .br the build ships; files
// without one (static-viz, sub-threshold chunks) are served compressed on
// the fly, so estimate with brotli. Null when no .br shipped at all.
const brotliPath = `${file}.br`;
if (fs.existsSync(brotliPath)) {
anyBrotli = true;
brotliBytes += fs.statSync(brotliPath).size;
} else {
brotliBytes += zlib.brotliCompressSync(contents).length;
}
}
return {
fileCount: files.length,
rawBytes,
gzipBytes,
brotliBytes: anyBrotli ? brotliBytes : null,
};
}
function collectJsFiles(absolutePath) {
if (fs.statSync(absolutePath).isFile()) {
return [absolutePath];
}
return fs
.readdirSync(absolutePath, { withFileTypes: true })
.flatMap(entry => {
const childPath = path.join(absolutePath, entry.name);
if (entry.isDirectory()) {
return collectJsFiles(childPath);
}
return entry.name.endsWith(".js") ? [childPath] : [];
});
}
function readStats(statsFile) {
const statsPath = resolve(statsFile);
return fs.existsSync(statsPath) ? JSON.parse(fs.readFileSync(statsPath, "utf8")) : null;
}
function entrypointJsAssets(stats, name) {
const entry = stats && stats.entrypoints && stats.entrypoints[name];
if (!entry) {
return null;
}
return entry.assets.map(asset => asset.name || asset).filter(name => name.endsWith(".js"));
}
function initialJsFiles({ file: statsFile, entrypoint, root }, joinRoot) {
const stats = readStats(statsFile);
if (!stats) {
console.warn(`Stats file missing (${statsFile}); reporting initial == total.`);
return null;
}
const assets = entrypointJsAssets(stats, entrypoint);
if (!assets) {
console.warn(`Entrypoint "${entrypoint}" missing in ${statsFile}; reporting initial == total.`);
return null;
}
const base = root ? resolve(root) : joinRoot;
return assets.map(name => path.join(base, name));
}
function measureSimpleTarget({ bundle, dir, file, altFile, stats, optional }) {
const candidates = [dir, file, altFile].filter(Boolean);
const chosen = candidates.find(c => fs.existsSync(resolve(c)));
if (!chosen) {
if (optional) {
console.warn(`Skipping "${bundle}": output not found (${candidates.join(", ")})`);
return [];
}
throw new Error(`Bundle output not found for "${bundle}": ${candidates.join(", ")}`);
}
const root = resolve(chosen);
const total = sizeOf(collectJsFiles(root));
const initialFiles = stats ? initialJsFiles(stats, root) : null;
const initial = initialFiles ? sizeOf(initialFiles) : total;
return [
{ bundle, kind: "initial", ...initial },
{ bundle, kind: "total", ...total },
];
}
/**
* Chunked SDK delivery. "Reachable" = what the bootstrap actually sends: the
* bootstrap (with its inlined runtime) + the chunked entry's initial chunks,
* minus the standalone runtime file. Async chunks are excluded (initial == total)
* unless SDK_ASYNC_CHUNKS_LOADABLE is set, at which point "total" includes them.
*/
function measureChunkedSdk(statsFile) {
const chunksDir = resolve(SDK_ROOT, "chunks");
if (!fs.existsSync(chunksDir)) {
// pre-chunked layout (monolith only) — nothing to measure here
return [];
}
const absRoot = resolve(SDK_ROOT);
const everything = collectJsFiles(chunksDir).filter(f => !SDK_RUNTIME_RE.test(f));
const stats = readStats(statsFile);
const chunked = entrypointJsAssets(stats, "embedding-sdk-chunked");
let reachableFiles;
if (chunked) {
const bootstrap = entrypointJsAssets(stats, "embedding-sdk-bootstrap") || [];
const names = [...new Set([...bootstrap, ...chunked])].filter(n => !SDK_RUNTIME_RE.test(n));
reachableFiles = names.map(n => path.join(absRoot, n));
} else {
console.warn(`embedding-sdk-chunked: entrypoint missing in ${statsFile}; counting whole chunks/ dir`);
reachableFiles = everything;
}
const initial = sizeOf(reachableFiles);
const total = SDK_ASYNC_CHUNKS_LOADABLE ? sizeOf(everything) : sizeOf(reachableFiles);
return [
{ bundle: "embedding-sdk-chunked", kind: "initial", ...initial },
{ bundle: "embedding-sdk-chunked", kind: "total", ...total },
];
}
function measureBundleSizes() {
return [
...simpleTargets.filter(target => isBundleWanted(target.bundle)).flatMap(measureSimpleTarget),
...(isBundleWanted("embedding-sdk-chunked")
? measureChunkedSdk("artifacts/stats-embedding-sdk.json")
: []),
];
}
module.exports = { measureBundleSizes };
if (require.main === module) {
console.log(JSON.stringify(measureBundleSizes(), null, 2));
}

View File

@@ -0,0 +1,41 @@
// This script is used in .github/workflows/run-tests.yml.
// It uploads the test-plan stats row produced by create-test-plan.js to
// the stats.metabase.com tracking table.
// Reads STATS_JSON, PR_NUMBER, HEAD_SHA, BASE_SHA, API_KEY from env.
const { uploadCsvToMb } = require("./csv-to-mb.js");
const stats = JSON.parse(process.env.STATS_JSON);
console.log("stats", stats);
const row = {
Date: new Date().toISOString(),
PR: Number(process.env.PR_NUMBER),
"Head SHA": process.env.HEAD_SHA,
"Base SHA": process.env.BASE_SHA,
"Modules changed": stats.modules_changed,
"Modules affected": stats.modules_affected,
"FE Unit specs total": stats.fe_unit_specs_total,
"FE Unit specs run": stats.fe_unit_specs_run,
"FE Unit specs skipped": stats.fe_unit_specs_skipped,
"Loki stories total": stats.loki_stories_total,
"Loki stories run": stats.loki_stories_run,
"Loki stories skipped": stats.loki_stories_skipped,
"E2E specs total": stats.e2e_specs_total,
"E2E specs run": stats.e2e_specs_run,
"E2E specs skipped": stats.e2e_specs_skipped,
};
uploadCsvToMb({
baseUrl: "https://stats.metabase.com",
tableId: 77008, // "Affected Tests Stats" table on stats.metabase.com
jsonData: [row],
mode: "append",
})
.then(() => {
console.log("Data uploaded successfully");
})
.catch((error) => {
console.error("Error uploading data:", error);
process.exit(1);
});

3
.github/team.json vendored
View File

@@ -120,7 +120,8 @@
"iethree",
"bpander",
"gantoreno",
"losrebellos"
"losrebellos",
"tmashara"
]
}
]

View File

@@ -103,7 +103,7 @@ jobs:
# "be-tests-mariadb-10-6-ee (0)"
name: "${{ matrix.version.name }} ${{ matrix.job.name }}"
steps:
- uses: actions/checkout@v4
- uses: actions/checkout@v6
- name: Test ${{ matrix.version.name }}
id: test-driver
uses: ./.github/actions/test-driver
@@ -226,7 +226,7 @@ jobs:
# "be-tests-mariadb-10-6-ee (0)"
name: "${{ matrix.version.name }} ${{ matrix.job.name }}"
steps:
- uses: actions/checkout@v4
- uses: actions/checkout@v6
- name: Test ${{ matrix.version.name }}
id: test-driver
uses: ./.github/actions/test-driver
@@ -336,7 +336,7 @@ jobs:
POSTGRES_DB: mb_test
POSTGRES_HOST_AUTH_METHOD: trust
steps:
- uses: actions/checkout@v4
- uses: actions/checkout@v6
- name: Test ${{ matrix.version.name }} (${{ matrix.job.name }})
id: test-driver
uses: ./.github/actions/test-driver
@@ -369,7 +369,7 @@ jobs:
if: always() && !cancelled()
steps:
- name: Check job status
uses: actions/github-script@v7
uses: actions/github-script@v9
env:
needs: ${{ toJson(needs) }}
with:

View File

@@ -24,15 +24,15 @@ jobs:
timeout-minutes: 10
steps:
- name: Check out the code
uses: actions/checkout@v4
uses: actions/checkout@v6
- name: Set up Babashka
uses: turtlequeue/setup-babashka@2d4df498c7a578b4f3e906283f9af913590bdec7 # v1.7.0
uses: turtlequeue/setup-babashka@d78ec6570aea3b614bc695cafcceb82eb45c2af9 # v1.8.0
with:
babashka-version: 1.12.197
- name: Set up AWS CLI
uses: aws-actions/configure-aws-credentials@7474bc4690e29a8392af63c5b98e7449536d5c3a # v4
uses: aws-actions/configure-aws-credentials@d979d5b3a71173a29b74b5b88418bfda9437d885 # v6.1.1
with:
role-to-assume: ${{ secrets.CI_MAVEN_IAM_ROLE }}
aws-region: ${{ vars.AWS_REGION }}

View File

@@ -16,11 +16,11 @@ jobs:
app_id: ${{ secrets.METABASE_BOT_APP_ID }}
private_key: ${{ secrets.METABASE_BOT_APP_PRIVATE_KEY }}
- name: Checkout repository
uses: actions/checkout@v4
uses: actions/checkout@v6
- name: Assign author and their team. Returns true if team exists.
id: auto-assign
uses: actions/github-script@v7
uses: actions/github-script@v9
with:
github-token: ${{ steps.generate-token.outputs.token }}
script: | #js

View File

@@ -5,24 +5,101 @@ on:
types: [closed, labeled]
jobs:
create-backport:
# --- Primary path: backport to every supported release line --------------
# Triggered by the plain `backport` label. `contains(array, 'backport')` is an
# exact element match, so this never fires for `double-backport`/`triple-backport`
# — those are handled by the legacy escape hatch at the end of this file.
prepare:
if:
| # the PR must have a backport label when it's merged, or be labeled with a backport label after merge
| # the PR must carry the `backport` label at merge, or be labeled `backport` after merge
github.event.pull_request.merged == true && (
github.event.action == 'closed' && contains(github.event.pull_request.labels.*.name, 'backport') ||
github.event.action == 'labeled' && github.event.label.name == 'backport'
)
runs-on: ${{ vars.DEFAULT_RUNNER_KEY }}
outputs:
commit: ${{ steps.find_commit.outputs.commit }}
targets: ${{ steps.targets.outputs.result }}
steps:
- uses: actions/checkout@v6
with:
fetch-depth: 0
token: ${{ secrets.METABASE_AUTOMATION_USER_TOKEN }}
- uses: ./.github/actions/find-squashed-commit
name: Find commit
id: find_commit
with:
pull-request-number: ${{ github.event.pull_request.number }}
base-ref: ${{ github.event.pull_request.base.ref }}
- name: Prepare release scripts
uses: ./.github/actions/prepare-release-scripts
- name: Compute supported backport targets
id: targets
uses: actions/github-script@v9
env:
AWS_S3_STATIC_BUCKET: ${{ vars.AWS_S3_STATIC_BUCKET }}
AWS_REGION: ${{ vars.AWS_REGION }}
with:
script: | # js
// Backport targets are the supported major lines from version-info.json:
// a line is in support — and a backport target — if and only if its EoL is
// today or later. `getSupportedMajors` encapsulates that rule (and ignores
// the display-only `lts` flag); it is shared with the auto-minor scheduler.
const { getSupportedMajors } = require('${{ github.workspace }}/release/dist/index.cjs');
const targets = await getSupportedMajors();
console.log('supported backport targets', targets);
return targets;
backport:
needs: prepare
runs-on: ${{ vars.DEFAULT_RUNNER_KEY }}
strategy:
fail-fast: false # one conflicting branch must not block the others
matrix:
target: ${{ fromJson(needs.prepare.outputs.targets) }}
steps:
- uses: actions/checkout@v6
with:
fetch-depth: 0
token: ${{ secrets.METABASE_AUTOMATION_USER_TOKEN }}
- name: Backport to v${{ matrix.target }}
uses: ./.github/actions/create-backport
with:
target-version: ${{ matrix.target }}
backport-commit: ${{ needs.prepare.outputs.commit }}
github-token: ${{ secrets.METABASE_AUTOMATION_USER_TOKEN }}
github-token-2: ${{ secrets.GITHUB_TOKEN }}
# --- Legacy escape hatch: explicit depth backports -----------------------
# `single-backport`/`double-backport`/`triple-backport` keep their original behavior —
# backport to CURRENT_VERSION and the 0–2 lines below it — independent of
# `major_version_support`. `single-backport` is a fallback for when the primary path
# can't be used and only CURRENT_VERSION is wanted. Kept for the short term for one-off
# manual depth control. If a branch from the primary path above already exists,
# create-backport simply errors on the duplicate.
legacy-backport:
if:
| # triggered only by the legacy depth labels, at merge or when added afterwards
github.event.pull_request.merged == true && (
github.event.action == 'closed' && (
contains(github.event.pull_request.labels.*.name, 'backport') ||
contains(github.event.pull_request.labels.*.name, 'single-backport') ||
contains(github.event.pull_request.labels.*.name, 'double-backport') ||
contains(github.event.pull_request.labels.*.name, 'triple-backport')
) ||
github.event.action == 'labeled' && (
github.event.label.name == 'backport' ||
github.event.label.name == 'single-backport' ||
github.event.label.name == 'double-backport' ||
github.event.label.name == 'triple-backport'
)
)
runs-on: ${{ vars.DEFAULT_RUNNER_KEY }}
steps:
- uses: actions/checkout@v4
- uses: actions/checkout@v6
with:
fetch-depth: 0
token: ${{ secrets.METABASE_AUTOMATION_USER_TOKEN }}
@@ -35,7 +112,7 @@ jobs:
- name: how many backports?
id: how-many-backports
uses: actions/github-script@v7
uses: actions/github-script@v9
with:
script: | # js
const labels = [
@@ -58,7 +135,7 @@ jobs:
- name: get release versions
id: get-latest-release-version
uses: actions/github-script@v7
uses: actions/github-script@v9
with:
script: | # js
const currentVersion = Number(${{ vars.CURRENT_VERSION }});

View File

@@ -21,7 +21,7 @@ jobs:
outputs:
modules: ${{ steps.get-modules.outputs.modules }}
steps:
- uses: actions/checkout@v4
- uses: actions/checkout@v6
- id: get-modules
run: |
modules=$(ls -1d src/metabase/*/ | xargs -n1 basename | tr '_' '-' | jq -R -s -c 'split("\n") | map(select(length > 0))')
@@ -38,7 +38,7 @@ jobs:
matrix:
modules: ${{ fromJson(needs.setup-oss.outputs.modules) }}
steps:
- uses: actions/checkout@v4
- uses: actions/checkout@v6
- name: Prepare front-end environment
uses: ./.github/actions/prepare-frontend
- name: Prepare back-end environment
@@ -57,7 +57,7 @@ jobs:
- name: Upload coverage to codecov.io
if: always()
uses: codecov/codecov-action@1af58845a975a7985b0beb0cbe6fbbb71a41dbad # v5
uses: codecov/codecov-action@e79a6962e0d4c0c17b229090214935d2e33f8354 # v6.0.1
with:
token: ${{ secrets.CODECOV_TOKEN }}
files: ./target/coverage/codecov.json
@@ -68,7 +68,7 @@ jobs:
outputs:
modules: ${{ steps.get-modules.outputs.modules }}
steps:
- uses: actions/checkout@v4
- uses: actions/checkout@v6
- id: get-modules
run: |
modules=$(ls -1d enterprise/backend/src/metabase_enterprise/*/ | xargs -n1 basename | tr '_' '-' | jq -R -s -c 'split("\n") | map(select(length > 0))')
@@ -85,7 +85,7 @@ jobs:
matrix:
modules: ${{ fromJson(needs.setup-ee.outputs.modules) }}
steps:
- uses: actions/checkout@v4
- uses: actions/checkout@v6
- name: Prepare front-end environment
uses: ./.github/actions/prepare-frontend
- name: Prepare back-end environment
@@ -104,7 +104,7 @@ jobs:
- name: Upload coverage to codecov.io
if: always()
uses: codecov/codecov-action@1af58845a975a7985b0beb0cbe6fbbb71a41dbad # v5
uses: codecov/codecov-action@e79a6962e0d4c0c17b229090214935d2e33f8354 # v6.0.1
with:
token: ${{ secrets.CODECOV_TOKEN }}
files: ./target/coverage/codecov.json

View File

@@ -18,7 +18,7 @@ jobs:
timeout-minutes: 10
name: Clj-Kondo
steps:
- uses: actions/checkout@v4
- uses: actions/checkout@v6
- name: Prepare back-end environment
uses: ./.github/actions/prepare-backend
- name: Run clj-kondo
@@ -30,7 +30,7 @@ jobs:
timeout-minutes: 20
name: Eastwood
steps:
- uses: actions/checkout@v4
- uses: actions/checkout@v6
- name: Prepare back-end environment
uses: ./.github/actions/prepare-backend
- run: clojure -X:dev:ee:ee-dev:drivers:drivers-dev:test:eastwood
@@ -40,10 +40,10 @@ jobs:
# Master / release safety net: lint every backend test namespace.
if: ${{ !inputs.skip && github.event_name == 'push' && (github.ref_name == 'master' || startsWith(github.ref_name, 'release-')) }}
runs-on: ${{ vars.DEFAULT_RUNNER_KEY }}
timeout-minutes: 30
timeout-minutes: 45
name: Eastwood (tests only)
steps:
- uses: actions/checkout@v4
- uses: actions/checkout@v6
- name: Prepare back-end environment
uses: ./.github/actions/prepare-backend
- run: clojure -X:dev:ee:ee-dev:drivers:drivers-dev:test:eastwood/test
@@ -56,7 +56,7 @@ jobs:
timeout-minutes: 15
name: Eastwood (changed tests only)
steps:
- uses: actions/checkout@v4
- uses: actions/checkout@v6
with:
# Enough history for `git merge-base` on a typical PR.
fetch-depth: 200
@@ -167,7 +167,7 @@ jobs:
':mb/old-migrations-test'
}}
steps:
- uses: actions/checkout@v4
- uses: actions/checkout@v6
- name: Prepare front-end environment
uses: ./.github/actions/prepare-frontend
- name: Prepare back-end environment
@@ -203,7 +203,7 @@ jobs:
previous-step-outcome: ${{ steps.run-java-tests.outcome }}
- name: Publish Test Report (JUnit)
uses: dorny/test-reporter@3eeb9fc888e82e8be2fb356bbeec2750231672bc # v1
uses: dorny/test-reporter@a43b3a5f7366b97d083190328d2c652e1a8b6aa2 # v3.0.0
if: failure()
with:
path: "target/junit/**/*_test.xml"
@@ -215,7 +215,7 @@ jobs:
- name: Upload Logs on Failure
if: always() && !success()
uses: actions/upload-artifact@v4
uses: actions/upload-artifact@v7
with:
name: test-logs-${{ matrix.java-version }}-${{ matrix.job.edition }}-${{ github.job }}-${{ matrix.job.test-group-name }}
path: logs/test-log.json
@@ -232,7 +232,7 @@ jobs:
java-version: [21]
name: "Check Java ${{ matrix.java-version }}"
steps:
- uses: actions/checkout@v4
- uses: actions/checkout@v6
- name: Prepare backend
uses: ./.github/actions/prepare-backend
# TODO -- we should probably also check WITHOUT ee -- do a check with only OSS namespaces on the classpath
@@ -245,7 +245,7 @@ jobs:
timeout-minutes: 10
name: Cljfmt
steps:
- uses: actions/checkout@v4
- uses: actions/checkout@v6
- name: Prepare back-end environment
uses: ./.github/actions/prepare-backend
- name: Run cljfmt
@@ -267,7 +267,7 @@ jobs:
needs: ${{ toJson(needs) }}
steps:
- name: Check backend job status
uses: actions/github-script@v7
uses: actions/github-script@v9
env:
needs: ${{ toJson(needs) }}
with:

View File

@@ -2,19 +2,173 @@ name: Backport Reminder
on:
pull_request:
types: [auto_merge_enabled, review_requested, labeled, unlabeled, synchronize]
types: [opened, edited, auto_merge_enabled, review_requested, labeled, unlabeled, synchronize]
branches:
- "master"
merge_group:
concurrency:
group: "backport-reminder-${{ github.event.pull_request.number }}"
cancel-in-progress: true
jobs:
flags:
runs-on: ${{ vars.DEFAULT_RUNNER_KEY }}
outputs:
has_backport_label: ${{ steps.c.outputs.value }}
steps:
- id: c
run: |
echo "value=${{
contains(github.event.pull_request.labels.*.name, 'no-backport') ||
contains(github.event.pull_request.labels.*.name, 'backport') ||
contains(github.event.pull_request.labels.*.name, 'single-backport') ||
contains(github.event.pull_request.labels.*.name, 'double-backport') ||
contains(github.event.pull_request.labels.*.name, 'triple-backport') }}" >> "$GITHUB_OUTPUT"
files-changed:
name: Check which files changed
runs-on: ${{ vars.DEFAULT_RUNNER_KEY }}
timeout-minutes: 3
outputs:
docs: ${{ steps.changes.outputs.docs }}
steps:
- uses: actions/checkout@v6
- name: Test which files changed
uses: dorny/paths-filter@fbd0ab8f3e69293af611ebaee6363fc25e6d187d # v4.0.1
id: changes
with:
token: ${{ github.token }}
filters: .github/file-paths.yaml
backport-reminder:
needs: [ files-changed, flags ]
if: ${{ !cancelled() }}
name: Decide whether to backport or not
runs-on: ${{ vars.DEFAULT_RUNNER_KEY }}
timeout-minutes: 5
permissions: {}
steps:
- uses: jesusvasquez333/verify-pr-label-action@657d111bbbe13e22bbd55870f1813c699bde1401 # v1.4.0
- name: Fail if backport decision label is missing
if: >
github.event_name == 'pull_request' &&
needs.files-changed.outputs.docs == 'true' &&
needs.flags.outputs.has_backport_label == 'false'
run: |
echo "::error::PR has docs changes and is missing a backport decision label. Add one of: no-backport, backport, single-backport, double-backport, or triple-backport"
exit 1
priority-backport-reminder:
name: Remind about backporting P0 + P1 fixes
runs-on: ${{ vars.DEFAULT_RUNNER_KEY }}
timeout-minutes: 5
needs: [flags]
permissions:
contents: read
issues: write
pull-requests: write
steps:
- name: Add, update, or remove the high-priority backport reminder
if: ${{ github.event_name == 'pull_request' }}
uses: actions/github-script@v9
env:
HAS_BACKPORT_LABEL: ${{ needs.flags.outputs.has_backport_label }}
with:
github-token: '${{ secrets.GITHUB_TOKEN }}'
valid-labels: 'no-backport, backport, double-backport, triple-backport'
disable-reviews: true
script: | # js
const { owner, repo } = context.repo;
const prNumber = context.payload.pull_request.number;
// A backport decision has already been made if a decision label is present.
const alreadyDecided = process.env.HAS_BACKPORT_LABEL === "true";
// Issues linked to this PR via closing keywords (fixes/closes/resolves #N),
// together with their labels.
const query = `
query($owner: String!, $repo: String!, $pr: Int!) {
repository(owner: $owner, name: $repo) {
pullRequest(number: $pr) {
closingIssuesReferences(first: 50) {
nodes {
number
title
url
labels(first: 50) { nodes { name } }
}
}
}
}
}`;
const data = await github.graphql(query, { owner, repo, pr: prNumber });
const linkedIssues =
data.repository.pullRequest.closingIssuesReferences.nodes ?? [];
const PRIORITY_LABELS = ["Priority:P0", "Priority:P1"];
const priorityIssues = linkedIssues.filter((issue) =>
issue.labels.nodes.some((l) => PRIORITY_LABELS.includes(l.name)),
);
// Hidden marker so we can find/update/remove our own comment idempotently.
const MARKER = "<!-- backport-priority-reminder -->";
const comments = await github.paginate(
github.rest.issues.listComments,
{ owner, repo, issue_number: prNumber, per_page: 100 },
);
const existing = comments.find(
(c) => c.body && c.body.includes(MARKER),
);
// Remind only while a high-priority issue is closed AND no decision
// has been made yet. Once either stops being true the comment is no
// longer needed, so we remove any stale one below.
const shouldComment = priorityIssues.length > 0 && !alreadyDecided;
if (!shouldComment) {
if (existing) {
await github.rest.issues.deleteComment({
owner,
repo,
comment_id: existing.id,
});
console.log("Removed stale backport-priority reminder.");
} else {
console.log(
"Reminder not needed (no high-priority linked issues, or a backport decision label is already applied).",
);
}
return;
}
const list = priorityIssues
.map((i) => {
const tags = i.labels.nodes
.map((l) => l.name)
.filter((n) => PRIORITY_LABELS.includes(n))
.join(", ");
return `- #${i.number} ${i.title} (${tags})`;
})
.join("\n");
const body = [
MARKER,
"⚠️ This PR closes one or more high-priority issues:",
"",
list,
"",
"Please apply the `backport` label unless you have a very good reason not to.",
].join("\n");
if (existing) {
await github.rest.issues.updateComment({
owner,
repo,
comment_id: existing.id,
body,
});
} else {
await github.rest.issues.createComment({
owner,
repo,
issue_number: prNumber,
body,
});
}

View File

@@ -23,7 +23,7 @@ jobs:
runs-on: ${{ vars.DEFAULT_RUNNER_KEY }}
timeout-minutes: 5
steps:
- uses: actions/checkout@v4
- uses: actions/checkout@v6
with:
sparse-checkout: |
release
@@ -31,7 +31,7 @@ jobs:
- name: Prepare release scripts
uses: ./.github/actions/prepare-release-scripts
- name: Send build start message
uses: actions/github-script@v7
uses: actions/github-script@v9
env:
SLACK_RELEASE_CHANNEL: ${{ vars.SLACK_RELEASE_CHANNEL }}
SLACK_BOT_TOKEN: ${{ secrets.SLACK_BOT_TOKEN }}
@@ -77,7 +77,7 @@ jobs:
echo "It must start with either 'v0.' or 'v1.'."
echo "Please, try again."
exit 1
- uses: actions/checkout@v4
- uses: actions/checkout@v6
with:
sparse-checkout: |
release
@@ -85,7 +85,7 @@ jobs:
- name: Prepare release scripts
uses: ./.github/actions/prepare-release-scripts
- name: Get Release Version
uses: actions/github-script@v7
uses: actions/github-script@v9
id: canonical_version
with:
script: | # js
@@ -120,7 +120,7 @@ jobs:
timeout-minutes: 10
steps:
- name: Check out the code to verify the release branch
uses: actions/checkout@v4
uses: actions/checkout@v6
with:
fetch-depth: 0 # IMPORTANT! to get all the branches
sparse-checkout: |
@@ -129,7 +129,7 @@ jobs:
- name: Prepare release scripts
uses: ./.github/actions/prepare-release-scripts
- name: Get Release Branch
uses: actions/github-script@v7
uses: actions/github-script@v9
id: release_branch
with:
result-encoding: string
@@ -155,7 +155,7 @@ jobs:
node_version: ${{ fromJson(steps.dependencies.outputs.result).node_version }}
build_process: ${{ fromJson(steps.dependencies.outputs.result).build_process }}
steps:
- uses: actions/checkout@v4
- uses: actions/checkout@v6
with:
sparse-checkout: |
release
@@ -163,7 +163,7 @@ jobs:
- name: Prepare release scripts
uses: ./.github/actions/prepare-release-scripts
- name: Get build dependencies
uses: actions/github-script@v7
uses: actions/github-script@v9
id: dependencies
with:
script: | # js
@@ -218,16 +218,16 @@ jobs:
echo "VERSION=$version" >> $GITHUB_ENV
- name: Setup Java ${{ needs.get-build-requirements.outputs.java_version }}
uses: actions/setup-java@v4
uses: actions/setup-java@v5.2.0
with:
distribution: 'temurin'
java-version: ${{ needs.get-build-requirements.outputs.java_version }}
- name: Setup Node.js ${{ needs.get-build-requirements.outputs.node_version }}
uses: actions/setup-node@v4
uses: actions/setup-node@v6.4.0
with:
node-version: ${{ needs.get-build-requirements.outputs.node_version }}.x
- name: Check out the code
uses: actions/checkout@v4
uses: actions/checkout@v6
with:
ref: ${{ github.event.inputs.commit }}
- name: Prepare front-end environment
@@ -252,7 +252,7 @@ jobs:
run: sha256sum ./metabase.jar > SHA256.sum
shell: bash
- name: Upload JARs as artifact
uses: actions/upload-artifact@v4
uses: actions/upload-artifact@v7
with:
name: metabase-release-${{ matrix.edition }}-${{ inputs.commit }}-uberjar
path: |

View File

@@ -12,7 +12,7 @@ jobs:
test-build-scripts:
runs-on: ${{ vars.DEFAULT_RUNNER_KEY }}
steps:
- uses: actions/checkout@v4
- uses: actions/checkout@v6
- name: Prepare back-end environment
timeout-minutes: 5
uses: ./.github/actions/prepare-backend

131
.github/workflows/bundle-size-stats.yml vendored Normal file
View File

@@ -0,0 +1,131 @@
name: Bundle Size Stats
# Reuses the uberjar built on every master merge: downloads the jar (for the
# emitted JS) plus the bundle-stats and embedding-sdk artifacts, measures
# initial/total bundle sizes, and appends them to stats.metabase.com.
on:
workflow_run:
workflows: ["Build Uberjar"]
types: [completed]
branches: [master]
concurrency:
group: bundle-size-stats-${{ github.event.workflow_run.head_sha }}
cancel-in-progress: true
jobs:
bundle-size-stats:
if: ${{ github.event.workflow_run.conclusion == 'success' && github.repository == 'metabase/metabase' }}
runs-on: ${{ vars.DEFAULT_RUNNER_KEY }}
timeout-minutes: 15
env:
HEAD_SHA: ${{ github.event.workflow_run.head_sha }}
steps:
- name: Check out the code
uses: actions/checkout@v6
with:
ref: ${{ github.event.workflow_run.head_sha }}
- name: Download EE uberjar
uses: ./.github/actions/fetch-artifact
with:
commit: ${{ github.event.workflow_run.head_sha }}
edition: ee
extract-path: temp/uberjar
- name: Extract front-end bundles from uberjar
run: |
mkdir -p resources artifacts
unzip -o "temp/uberjar/metabase.jar" \
"frontend_client/app/dist/*" \
"frontend_client/app/embed.js" \
"frontend_client/app/embedding-sdk/*" \
-d resources
unzip -o -j "temp/uberjar/metabase.jar" version.properties -d artifacts
- name: Download bundle stats and SDK package artifacts
uses: actions/github-script@v9
with:
script: | # js
const fs = require("fs");
async function fetchArtifact(name) {
const res = await github.rest.actions.listArtifactsForRepo({
owner: context.repo.owner,
repo: context.repo.repo,
name,
per_page: 1,
});
const artifact = res.data?.artifacts?.[0];
if (!artifact) {
console.log(`Artifact not found: ${name}`);
return "";
}
const download = await github.rest.actions.downloadArtifact({
owner: context.repo.owner,
repo: context.repo.repo,
artifact_id: artifact.id,
archive_format: "zip",
});
fs.writeFileSync(`${name}.zip`, Buffer.from(download.data));
console.log(`Downloaded ${name}`);
return `${name}.zip`;
}
const sha = process.env.HEAD_SHA;
core.exportVariable("STATS_ZIP", await fetchArtifact(`metabase-bundle-stats-ee-${sha}`));
core.exportVariable("PKG_ZIP", await fetchArtifact(`embedding-sdk-${sha}`));
- name: Extract stats and SDK package
run: |
mkdir -p artifacts resources/embedding-sdk
if [ -n "$STATS_ZIP" ]; then unzip -o "$STATS_ZIP" -d artifacts; fi
if [ -n "$PKG_ZIP" ]; then unzip -o "$PKG_ZIP" -d resources/embedding-sdk; fi
- name: Measure bundle sizes
run: node ./.github/scripts/measure-bundle-sizes.js | tee artifacts/bundle-sizes.json
- name: Upload sizes to Stats
uses: actions/github-script@v9
env:
API_KEY: ${{ secrets.STATS_API_KEY }}
with:
script: | # js
const fs = require("fs");
const { uploadCsvToMb } = require("${{ github.workspace }}/.github/scripts/csv-to-mb.js");
const measurements = JSON.parse(
fs.readFileSync("${{ github.workspace }}/artifacts/bundle-sizes.json", "utf8"),
);
const date = new Date().toISOString().slice(0, 10); // YYYY-MM-DD
const commit = process.env.HEAD_SHA.slice(0, 12);
// version.properties (extracted from the uberjar) carries the build's tag.
// Non-release (master) builds report tag=vUNKNOWN, so leave Version empty there.
const versionProps = "${{ github.workspace }}/artifacts/version.properties";
const rawVersion = fs.existsSync(versionProps)
? (fs.readFileSync(versionProps, "utf8").match(/^tag=(.*)$/m)?.[1]?.trim() ?? "")
: "";
const version = rawVersion === "vUNKNOWN" ? "" : rawVersion;
const data = measurements.map(measurement => ({
"Date": date,
"Version": version,
"Commit": commit,
"Bundle": measurement.bundle,
"Kind": measurement.kind, // "initial" or "total"
"Raw bytes": measurement.rawBytes,
"Gzip bytes": measurement.gzipBytes,
"Brotli bytes": measurement.brotliBytes ?? "", // as-served; empty when no .br shipped
"File count": measurement.fileCount,
}));
console.table(data);
uploadCsvToMb({
baseUrl: "https://stats.metabase.com",
tableId: 79417,
jsonData: data,
mode: "append",
}).then(() => {
console.log("Bundle sizes uploaded successfully");
}).catch(error => {
console.error("Error uploading bundle sizes:", error);
process.exit(1);
});

View File

@@ -2,7 +2,7 @@
# It sets the fresh cache each Monday but provides a manual trigger to update it on-demand.
#
# All other jobs only restore from this cache; *they never update it*!
# They should always use `actions/cache/restore@v4` because `actions/cache@v4` would attempt to save the cache.
# They should always use `actions/cache/restore@vX` because `actions/cache@vX` would attempt to save the cache.
#
# Our agreed-upon strategy is to use a deterministic cache key based on the week of the year.
# Jobs can utilize the prior week's cache if the current week's cache is not found.
@@ -42,7 +42,7 @@ jobs:
if: ${{ github.event_name != 'schedule' || github.repository == 'metabase/metabase' }}
runs-on: ${{ vars.DEFAULT_RUNNER_KEY }}
steps:
- uses: actions/checkout@v4
- uses: actions/checkout@v6
- name: Get cache keys
uses: ./.github/actions/get-cache-keys
id: get-cache-keys
@@ -62,16 +62,53 @@ jobs:
CYPRESS_CACHE_KEY: ${{ needs.get-cache-keys.outputs.cypress-cache-key }}
ESLINT_CACHE_KEY: ${{ needs.get-cache-keys.outputs.eslint-cache-key }}
steps:
- uses: actions/checkout@v4
- uses: actions/checkout@v6
# Backend preparation and cache
- name: Prepare backend
uses: ./.github/actions/prepare-backend
# Add or remove aliases to affect the cache size and its contents.
# Adjust to taste, then redeploy this workflow.
# Pre-resolve every Clojure dependency classpath CI actually uses, so the
# saved cache is a SUPERSET of what each job resolves at runtime.
#
# Why a single `-P` is not enough: tools.deps selects dependency *versions*
# globally per classpath, so different alias combinations can resolve the
# same transitive lib to different versions (e.g. `org.clojure/data.json`
# 2.4.0 vs 2.5.x, `org.ow2.asm/asm` 9.2 vs newer). Resolving only one combo
# leaves the others to fetch their version-deltas straight from Maven
# Central / Clojars on every run -> intermittent 403s. See DEV-2094.
#
# Keep this list in sync with the `clojure` invocations across .github/.
# Each entry notes its consumer. We resolve the FULL alias strings (not a
# hand-reduced set) on purpose: if a currently dep-less alias later gains a
# dependency, this still covers it instead of silently drifting back into a
# cache gap.
- name: Pre-resolve Clojure dependencies
run: clojure -A:build:dev:ee:ee-dev:drivers:drivers-dev:cljs -P
shell: bash
run: |
set -euo pipefail
combos=(
":build:dev:ee:ee-dev:drivers:drivers-dev:cljs" # build + uberjar (combined classpath)
":cljs" # shadow-cljs (build:cljs); resolves :cljs alone -> covers asm:9.2
":dev:ee:ee-dev:drivers:drivers-dev:test:eastwood" # backend.yml (eastwood + eastwood/test)
":dev:ci:ee:ee-dev:drivers:drivers-dev:test" # test-driver.yml (ee) -> covers data.json:2.4.0
":dev:ci:oss:oss-dev:drivers:drivers-dev:test" # test-driver.yml (oss)
":dev:ci:test:ee:ee-dev" # backend.yml matrix (ee) + generative-query-test.yml
":dev:ci:test:oss:oss-dev" # backend.yml matrix (oss)
":dev:ci:ee:ee-dev:test:cloverage" # backend-cloverage.yml
":dev:drivers:drivers-dev:test" # reset-cloud-test-dataset.yml + presto script
":dev:drivers:build:build-dev:build-test:ci" # build-scripts.yml
":ee:drivers:check" # backend.yml (check; git dep athos/clj-check)
":ee:doc" # docs-generate-base.yml
":kondo" # resolve-backport-conflicts.yml (replace-deps)
":run:ee" # cross-branch-migration-test
":dev:ee:migrate" # cross-branch-migration-test
)
for combo in "${combos[@]}"; do
echo "::group::clojure -P -A${combo}"
clojure -P -A"${combo}"
echo "::endgroup::"
done
- name: Update M2 cache
uses: ./.github/actions/update-cache

View File

@@ -10,7 +10,7 @@ jobs:
runs-on: ${{ vars.DEFAULT_RUNNER_KEY }}
steps:
- name: Checkout code
uses: actions/checkout@v4
uses: actions/checkout@v6
- name: Check for backport.sh file
run: |
if [ -f "./backport.sh" ]; then

View File

@@ -19,9 +19,9 @@ jobs:
outputs:
frontend_all: ${{ steps.changes.outputs.frontend_all }}
steps:
- uses: actions/checkout@v4
- uses: actions/checkout@v6
- name: Test which files changed
uses: dorny/paths-filter@0bc4621a3135347011ad047f9ecf449bf72ce2bd # v3.0.0
uses: dorny/paths-filter@fbd0ab8f3e69293af611ebaee6363fc25e6d187d # v4.0.1
id: changes
with:
token: ${{ github.token }}
@@ -33,7 +33,7 @@ jobs:
runs-on: ${{ vars.DEFAULT_RUNNER_KEY }}
steps:
- name: Checkout repository
uses: actions/checkout@v4
uses: actions/checkout@v6
with:
fetch-depth: 0
- name: Prepare front-end environment
@@ -43,7 +43,7 @@ jobs:
- name: Compile CLJS
run: NODE_ENV=development bun run build-pure:cljs
- name: Publish to Chromatic
uses: chromaui/action@c03d144dc72e609acbb960489fdd1e1a4a23b34f # v10.2.0
uses: chromaui/action@0b8c883861c1663076150cf1f4592b19d1ff6a54 # v17.0.1
env:
PUBLISH_CHROMATIC: ${{ secrets.CHROMATIC_PROJECT_TOKEN }}
STORYBOOK_BUILD_TIMEOUT: 900000

View File

@@ -19,9 +19,9 @@ jobs:
outputs:
cljs: ${{ steps.changes.outputs.cljs }}
steps:
- uses: actions/checkout@v4
- uses: actions/checkout@v6
- name: Test which files changed
uses: dorny/paths-filter@0bc4621a3135347011ad047f9ecf449bf72ce2bd # v3.0.0
uses: dorny/paths-filter@fbd0ab8f3e69293af611ebaee6363fc25e6d187d # v4.0.1
id: changes
with:
token: ${{ github.token }}
@@ -33,7 +33,7 @@ jobs:
runs-on: ${{ vars.DEFAULT_RUNNER_KEY }}
timeout-minutes: 15
steps:
- uses: actions/checkout@v4
- uses: actions/checkout@v6
- name: Prepare front-end environment
uses: ./.github/actions/prepare-frontend
- name: Prepare back-end environment

View File

@@ -15,9 +15,9 @@ jobs:
outputs:
codeql: ${{ steps.changes.outputs.codeql }}
steps:
- uses: actions/checkout@v4
- uses: actions/checkout@v6
- name: Test which files changed
uses: dorny/paths-filter@0bc4621a3135347011ad047f9ecf449bf72ce2bd # v3.0.0
uses: dorny/paths-filter@fbd0ab8f3e69293af611ebaee6363fc25e6d187d # v4.0.1
id: changes
with:
token: ${{ github.token }}
@@ -28,7 +28,7 @@ jobs:
runs-on: ${{ vars.DEFAULT_RUNNER_KEY }}
steps:
- name: Checkout repository
uses: actions/checkout@v4
uses: actions/checkout@v6
- name: Initialize CodeQL
uses: github/codeql-action/init@v4
with:

View File

@@ -15,7 +15,7 @@ jobs:
contents: write
pull-requests: write
steps:
- uses: actions/checkout@v4
- uses: actions/checkout@v6
- name: Install codespell
run: pip install codespell
@@ -68,7 +68,7 @@ jobs:
- name: Fix spelling errors with Claude Code
if: steps.codespell_check.outcome == 'failure'
uses: anthropics/claude-code-action@3ac52d0da9f8ec9ca7b4dc23bb477e36ef9c77a9 # v1
uses: anthropics/claude-code-action@787c5a0ce96a9a6cfb050ea0c8f4c05f2447c251 # v1.0.133
with:
anthropic_api_key: ${{ secrets.ANTHROPIC_API_KEY }}
github_token: ${{ steps.claude-app-token.outputs.token }}
@@ -83,7 +83,7 @@ jobs:
- name: Create Pull Request
if: steps.codespell_check.outcome == 'failure'
uses: peter-evans/create-pull-request@c5a7806660adbe173f04e3e038b0ccdcd758773c # v6
uses: peter-evans/create-pull-request@5f6978faf089d4d20b00c7766989d076bb2fc7f1 # v8.1.1
with:
token: ${{ secrets.GITHUB_TOKEN }}
branch: claude-fix/codespell-${{ steps.date.outputs.date }}

View File

@@ -53,7 +53,7 @@ jobs:
outputs:
platforms: ${{ steps.platforms.outputs.result }}
steps:
- uses: actions/checkout@v4
- uses: actions/checkout@v6
with:
fetch-depth: 1
# we want the dockerfile from the commit we're building
@@ -70,7 +70,7 @@ jobs:
uses: ./.github/actions/prepare-release-scripts
- name: Get platforms
id: platforms
uses: actions/github-script@v7
uses: actions/github-script@v9
with:
result-encoding: string
script: | # js
@@ -86,19 +86,19 @@ jobs:
- name: Move the Uberjar to the context dir
run: mv ./metabase.jar bin/docker/.
- name: Login to Docker Hub
uses: docker/login-action@c94ce9fb468520275223c153574b00df6fe4bcc9 # v3
uses: docker/login-action@650006c6eb7dba73a995cc03b0b2d7f5ca915bee # v4.2.0
with:
username: ${{ inputs.release == 'true' && secrets.DOCKERHUB_RELEASE_USERNAME || secrets.DOCKERHUB_USERNAME }}
password: ${{ inputs.release == 'true' && secrets.DOCKERHUB_RELEASE_TOKEN || secrets.DOCKERHUB_TOKEN }}
- name: Set up QEMU
if: contains(steps.platforms.outputs.result, 'arm')
uses: docker/setup-qemu-action@c7c53464625b32c7a7e944ae62b3e17d2b600130 # v3
uses: docker/setup-qemu-action@ce360397dd3f832beb865e1373c09c0e9f86d70a # v4.0.0
- name: Set up Docker Buildx
if: contains(steps.platforms.outputs.result, 'arm')
uses: docker/setup-buildx-action@8d2750c68a42422c14e847fe6c8ac0403b4cbd6f # v3
uses: docker/setup-buildx-action@d7f5e7f509e45cec5c76c4d5afdd7de93d0b3df5 # v4.1.0
- name: Build multi-arch container
if: contains(steps.platforms.outputs.result, 'arm')
uses: docker/build-push-action@10e90e3645eae34f1e60eeb005ba3a3d33f178e8 # v6
uses: docker/build-push-action@f9f3042f7e2789586610d6e8b85c8f03e5195baf # v7.2.0
with:
context: bin/docker/.
platforms: ${{ steps.platforms.outputs.result }}
@@ -124,7 +124,7 @@ jobs:
timeout-minutes: 15
steps:
- name: Login to Docker Hub
uses: docker/login-action@c94ce9fb468520275223c153574b00df6fe4bcc9 # v3
uses: docker/login-action@650006c6eb7dba73a995cc03b0b2d7f5ca915bee # v4.2.0
with:
username: ${{ inputs.release == 'true' && secrets.DOCKERHUB_RELEASE_USERNAME || secrets.DOCKERHUB_USERNAME }}
password: ${{ inputs.release == 'true' && secrets.DOCKERHUB_RELEASE_TOKEN || secrets.DOCKERHUB_TOKEN }}
@@ -144,7 +144,7 @@ jobs:
timeout-minutes: 15
steps:
- name: Login to Docker Hub
uses: docker/login-action@c94ce9fb468520275223c153574b00df6fe4bcc9 # v3
uses: docker/login-action@650006c6eb7dba73a995cc03b0b2d7f5ca915bee # v4.2.0
with:
username: ${{ inputs.release == 'true' && secrets.DOCKERHUB_RELEASE_USERNAME || secrets.DOCKERHUB_USERNAME }}
password: ${{ inputs.release == 'true' && secrets.DOCKERHUB_RELEASE_TOKEN || secrets.DOCKERHUB_TOKEN }}

View File

@@ -15,7 +15,7 @@ jobs:
oss: ${{ toJson(fromJson(steps.build-matrix.outputs.result).oss) }}
steps:
- name: Check out the code
uses: actions/checkout@v4
uses: actions/checkout@v6
- name: Get EE Docker info
id: ee-info
uses: ./.github/actions/get-docker-info
@@ -28,7 +28,7 @@ jobs:
edition: oss
- name: Build matrix info
id: build-matrix
uses: actions/github-script@v7
uses: actions/github-script@v9
with:
script: | # js
const editions = JSON.parse('${{ steps.ee-info.outputs.editions }}');
@@ -72,11 +72,11 @@ jobs:
name: Run Trivy vulnerability scanner
steps:
- name: Check out the code
uses: actions/checkout@v4
uses: actions/checkout@v6
with:
ref: ${{ github.event.inputs.commit || github.event.pull_request.head.sha || github.sha }}
- name: Run Trivy vulnerability scanner
uses: aquasecurity/trivy-action@57a97c7e7821a5776cebc9bb87c984fa69cba8f1
uses: aquasecurity/trivy-action@ed142fd0673e97e23eac54620cfb913e5ce36c25 # v0.36.0
env:
TRIVY_OFFLINE_SCAN: true
repo: ${{ fromJson(needs.get-container-info.outputs[matrix.edition]).repo }}
@@ -97,7 +97,7 @@ jobs:
name: Post PR comment
steps:
- name: Check out the code
uses: actions/checkout@v4
uses: actions/checkout@v6
- name: Get Docker info for comment
id: docker-info
uses: ./.github/actions/get-docker-info
@@ -106,7 +106,7 @@ jobs:
edition: ee
- name: Post PR comment
uses: marocchino/sticky-pull-request-comment@773744901bac0e8cbb5a0dc842800d45e9b2b405 # v2
uses: marocchino/sticky-pull-request-comment@0ea0beb66eb9baf113663a64ec522f60e49231c0 # v3.0.4
with:
header: docker-build-status
message: |

View File

@@ -13,9 +13,9 @@ jobs:
outputs:
frontend_embedding_sources: ${{ steps.changes.outputs.frontend_embedding_sources }}
steps:
- uses: actions/checkout@v4
- uses: actions/checkout@v6
- name: Test which files changed
uses: dorny/paths-filter@0bc4621a3135347011ad047f9ecf449bf72ce2bd # v3.0.0
uses: dorny/paths-filter@fbd0ab8f3e69293af611ebaee6363fc25e6d187d # v4.0.1
id: changes
with:
token: ${{ github.token }}
@@ -31,12 +31,12 @@ jobs:
!cancelled() &&
needs.files-changed.outputs.frontend_embedding_sources == 'true'
steps:
- uses: actions/checkout@v4
- uses: actions/checkout@v6
with:
ref: ${{ github.head_ref }}
- name: Check if the PR title is already following Conventional Commits style
uses: actions/github-script@v7
uses: actions/github-script@v9
id: is-conventional-commit
with:
result-encoding: string
@@ -48,7 +48,7 @@ jobs:
- name: Remind PR authors to use conventional commit in the title
if: ${{ steps.is-conventional-commit.outputs.result != 'true' }}
uses: actions/github-script@v7
uses: actions/github-script@v9
with:
script: | #js
const { updateCommentForEmbedding } = require(".github/scripts/conventional-commit-pr-title-reminder/utils.js")

View File

@@ -11,7 +11,7 @@ jobs:
runs-on: ${{ vars.DEFAULT_RUNNER_KEY }}
timeout-minutes: 60
steps:
- uses: actions/checkout@v4
- uses: actions/checkout@v6
- name: Prepare front-end environment
uses: ./.github/actions/prepare-frontend
- name: Prepare back-end environment
@@ -23,7 +23,7 @@ jobs:
--coverage \
--testTimeout=60000
- name: Upload coverage to codecov.io
uses: codecov/codecov-action@1af58845a975a7985b0beb0cbe6fbbb71a41dbad # v5
uses: codecov/codecov-action@e79a6962e0d4c0c17b229090214935d2e33f8354 # v6.0.1
with:
files: ./coverage/lcov.info
flags: front-end

View File

@@ -32,7 +32,7 @@ jobs:
MB_DB_TYPE: h2
steps:
- uses: actions/checkout@v4
- uses: actions/checkout@v6
- uses: ./.github/actions/cross-branch-migration-test
with:
base-ref: ${{ env.BASE_REF }}
@@ -58,7 +58,7 @@ jobs:
POSTGRES_DB: mb_test
POSTGRES_HOST_AUTH_METHOD: trust
steps:
- uses: actions/checkout@v4
- uses: actions/checkout@v6
- uses: ./.github/actions/cross-branch-migration-test
with:
base-ref: ${{ env.BASE_REF }}
@@ -84,7 +84,7 @@ jobs:
POSTGRES_DB: mb_test
POSTGRES_HOST_AUTH_METHOD: trust
steps:
- uses: actions/checkout@v4
- uses: actions/checkout@v6
- uses: ./.github/actions/cross-branch-migration-test
with:
base-ref: ${{ env.BASE_REF }}
@@ -109,7 +109,7 @@ jobs:
MYSQL_ALLOW_EMPTY_PASSWORD: true
MYSQL_DATABASE: mb_test
steps:
- uses: actions/checkout@v4
- uses: actions/checkout@v6
- uses: ./.github/actions/cross-branch-migration-test
with:
base-ref: ${{ env.BASE_REF }}
@@ -135,7 +135,7 @@ jobs:
ports:
- "3306:3306"
steps:
- uses: actions/checkout@v4
- uses: actions/checkout@v6
- uses: ./.github/actions/cross-branch-migration-test
with:
base-ref: ${{ env.BASE_REF }}
@@ -160,7 +160,7 @@ jobs:
MARIADB_ALLOW_EMPTY_ROOT_PASSWORD: true
MARIADB_DATABASE: mb_test
steps:
- uses: actions/checkout@v4
- uses: actions/checkout@v6
- uses: ./.github/actions/cross-branch-migration-test
with:
base-ref: ${{ env.BASE_REF }}
@@ -185,7 +185,7 @@ jobs:
MARIADB_ALLOW_EMPTY_ROOT_PASSWORD: true
MARIADB_DATABASE: mb_test
steps:
- uses: actions/checkout@v4
- uses: actions/checkout@v6
- uses: ./.github/actions/cross-branch-migration-test
with:
base-ref: ${{ env.BASE_REF }}

View File

@@ -30,13 +30,13 @@ jobs:
outputs:
matrix: ${{ steps.set-matrix.outputs.matrix }}
steps:
- uses: actions/checkout@v4
- uses: actions/checkout@v6
with:
sparse-checkout: .github
- name: Generate version matrix
id: set-matrix
uses: actions/github-script@v7
uses: actions/github-script@v9
with:
script: |
const { buildCrossVersionMatrix } = require('${{ github.workspace }}/.github/scripts/build-cross-version-matrix');
@@ -56,7 +56,7 @@ jobs:
fail-fast: false
steps:
- uses: actions/checkout@v4
- uses: actions/checkout@v6
- name: Prepare release scripts
uses: ./.github/actions/prepare-release-scripts
@@ -76,7 +76,7 @@ jobs:
./test.sh --source "${{ matrix.source }}" --target "${{ matrix.target }}"
- name: Upload failure result
uses: actions/upload-artifact@v4
uses: actions/upload-artifact@v7
if: failure()
with:
name: failure-result-${{ matrix.source }}-${{ matrix.target }}
@@ -84,7 +84,7 @@ jobs:
if-no-files-found: ignore
- name: Upload Cypress Artifacts upon failure
uses: actions/upload-artifact@v4
uses: actions/upload-artifact@v7
if: failure()
with:
name: cypress-recording-${{ matrix.source }}-${{ matrix.target }}
@@ -111,7 +111,7 @@ jobs:
runs-on: ${{ vars.DEFAULT_RUNNER_KEY }}
timeout-minutes: 5
steps:
- uses: actions/checkout@v4
- uses: actions/checkout@v6
with:
sparse-checkout: |
release
@@ -121,7 +121,7 @@ jobs:
uses: ./.github/actions/prepare-release-scripts
- name: Download failure results
uses: actions/download-artifact@v4
uses: actions/download-artifact@v8
continue-on-error: true
with:
pattern: failure-result-*
@@ -130,7 +130,7 @@ jobs:
- name: Notify Slack
continue-on-error: true
uses: actions/github-script@v7
uses: actions/github-script@v9
env:
SLACK_BOT_TOKEN: ${{ secrets.SLACK_BOT_TOKEN }}
with:
@@ -189,7 +189,7 @@ jobs:
timeout-minutes: 30
steps:
- uses: actions/checkout@v4
- uses: actions/checkout@v6
- name: Prepare release scripts
uses: ./.github/actions/prepare-release-scripts
@@ -208,7 +208,7 @@ jobs:
./test.sh --source "${{ inputs.source_version }}" --target "${{ inputs.target_version }}"
- name: Upload Cypress Artifacts upon failure
uses: actions/upload-artifact@v4
uses: actions/upload-artifact@v7
if: failure()
with:
name: cypress-recording-${{ inputs.source_version }}-${{ inputs.target_version }}

View File

@@ -9,7 +9,7 @@ jobs:
runs-on: ${{ vars.DEFAULT_RUNNER_KEY }}
timeout-minutes: 5
steps:
- uses: actions/checkout@v4
- uses: actions/checkout@v6
with:
ref: master
sparse-checkout: |
@@ -22,7 +22,7 @@ jobs:
- name: Preflight
id: preflight
uses: actions/github-script@v7
uses: actions/github-script@v9
with:
script: | # js
const { versionRequirements } = require('${{ github.workspace }}/release/dist/index.cjs');
@@ -70,6 +70,10 @@ jobs:
git push -u origin "$BRANCH"
echo "sha=$(git rev-parse HEAD)" >> "$GITHUB_OUTPUT"
# CURRENT_VERSION is an org-level variable — that's the canonical one,
# shared across repos. A repo-level variable of the same name would
# shadow it (repo vars override org vars), so we deliberately update
# the org-level value here and rely on no repo-level override existing.
- name: Bump vars.CURRENT_VERSION
id: bump
continue-on-error: true
@@ -78,7 +82,7 @@ jobs:
run: |
gh variable set CURRENT_VERSION \
--body "${{ steps.preflight.outputs.next }}" \
--repo "${{ github.repository }}"
--org "${{ github.repository_owner }}"
# Pre-create the milestone for the version AFTER this cut, so it's
# already open when next time's backfill needs somewhere to assign
@@ -87,7 +91,7 @@ jobs:
- name: Create 0.${{ steps.preflight.outputs.nextMilestone }} milestone
id: milestone
continue-on-error: true
uses: actions/github-script@v7
uses: actions/github-script@v9
with:
github-token: ${{ secrets.METABASE_AUTOMATION_USER_TOKEN }}
script: | # js
@@ -113,7 +117,7 @@ jobs:
- name: Create Bug:v${{ steps.preflight.outputs.nextMilestone }} label
id: label
continue-on-error: true
uses: actions/github-script@v7
uses: actions/github-script@v9
with:
github-token: ${{ secrets.METABASE_AUTOMATION_USER_TOKEN }}
script: | # js
@@ -145,7 +149,7 @@ jobs:
continue-on-error: true
env:
STATS_API_KEY: ${{ secrets.STATS_API_KEY }}
uses: actions/github-script@v7
uses: actions/github-script@v9
with:
script: | # js
const next = '${{ steps.preflight.outputs.next }}';
@@ -178,7 +182,7 @@ jobs:
console.log(`Created timeline event ${name}`);
- name: Notify Slack
uses: actions/github-script@v7
uses: actions/github-script@v9
env:
SLACK_BOT_TOKEN: ${{ secrets.SLACK_BOT_TOKEN }}
with:
@@ -204,7 +208,7 @@ jobs:
const notes = [
bumpFailed
? `:warning: Auto-bump of \`CURRENT_VERSION\` to *${next}* failed — please update it manually: <https://github.com/${owner}/${repo}/settings/variables/actions|Actions variables>`
? `:warning: Auto-bump of \`CURRENT_VERSION\` to *${next}* failed — please update it manually: <https://github.com/organizations/${owner}/settings/variables/actions|Org Actions variables>`
: `\`CURRENT_VERSION\` bumped to *${next}*.`,
milestoneFailed
? `:warning: Pre-creating milestone \`0.${{ steps.preflight.outputs.nextMilestone }}\` (for the next cut) failed — please create it manually: <https://github.com/${owner}/${repo}/milestones|Repository milestones>`
@@ -233,7 +237,7 @@ jobs:
# Heads-up to the broader eng audience about where backports go now.
- name: Notify the engineering team
uses: actions/github-script@v7
uses: actions/github-script@v9
env:
SLACK_BOT_TOKEN: ${{ secrets.SLACK_BOT_TOKEN }}
with:
@@ -262,7 +266,7 @@ jobs:
# the happy-path message above.
- name: Notify Slack on cut failure
if: failure()
uses: actions/github-script@v7
uses: actions/github-script@v9
env:
SLACK_BOT_TOKEN: ${{ secrets.SLACK_BOT_TOKEN }}
with:

View File

@@ -11,7 +11,7 @@ jobs:
runs-on: ${{ vars.DEFAULT_RUNNER_KEY }}
timeout-minutes: 5
steps:
- uses: actions/setup-node@v4
- uses: actions/setup-node@v6.4.0
with:
node-version: lts/Jod
@@ -19,7 +19,7 @@ jobs:
run: npm install @slack/web-api
- name: Check disabled drivers and notify Slack
uses: actions/github-script@v7
uses: actions/github-script@v9
env:
SLACK_BOT_TOKEN: ${{ secrets.SLACK_BOT_TOKEN }}
with:
@@ -30,15 +30,23 @@ jobs:
const response = await fetch('https://raw.githubusercontent.com/metabase/ci-test-config/refs/heads/master/ci-test-config.json');
const config = await response.json();
const drivers = config.ignored?.drivers || [];
const drivers = config.drivers || [];
const lastUpdated = config.metadata?.last_updated || 'unknown';
const reason = config.metadata?.reason || 'No reason provided';
if (drivers.length === 0) {
console.log('No drivers are currently disabled. Skipping Slack notification.');
const skipped = drivers.filter(d => d.status === 'skip').map(d => d.name);
const info = drivers.filter(d => d.status === 'info').map(d => d.name);
if (skipped.length === 0 && info.length === 0) {
console.log('No drivers are disabled or collecting data. Skipping Slack notification.');
return;
}
const list = (names) => names.map(d => `\`${d}\``).join(' ');
const lines = [];
if (skipped.length) lines.push(`:no_entry: *Disabled (not run):* ${list(skipped)}`);
if (info.length) lines.push(`:information_source: *Collecting data (non-blocking):* ${list(info)}`);
if (!process.env.SLACK_BOT_TOKEN) {
throw new Error('SLACK_BOT_TOKEN is required');
}
@@ -47,13 +55,13 @@ jobs:
await slack.chat.postMessage({
channel: 'engineering-ci',
text: 'CI Disabled Drivers Report',
text: 'CI Driver Status Report',
blocks: [
{
type: 'section',
text: {
type: 'mrkdwn',
text: `:no_entry: *CI Disabled Drivers*\n${drivers.map(d => `\`${d}\``).join(' ')}`
text: `*CI Driver Status*\n${lines.join('\n')}`
},
accessory: {
type: 'button',

View File

@@ -24,7 +24,7 @@ jobs:
timeout-minutes: 30
steps:
- name: Checkout code
uses: actions/checkout@v4
uses: actions/checkout@v6
with:
fetch-depth: 0
token: ${{ secrets.automation_token }}
@@ -34,7 +34,7 @@ jobs:
uses: ./.github/actions/prepare-backend
- name: Set up Babashka
uses: turtlequeue/setup-babashka@2d4df498c7a578b4f3e906283f9af913590bdec7 # v1.7.0
uses: turtlequeue/setup-babashka@d78ec6570aea3b614bc695cafcceb82eb45c2af9 # v1.8.0
with:
babashka-version: 1.12.197

View File

@@ -13,21 +13,21 @@ jobs:
contents: read
steps:
- name: Checkout repository
uses: actions/checkout@v5
uses: actions/checkout@v6
with:
sparse-checkout: |
docs
.lychee
- name: Cache lychee
uses: actions/cache@v3
uses: actions/cache@v5
with:
path: ~/.cache/lychee
key: lychee-${{ runner.os }}-${{ hashFiles('.lychee/config.toml') }}
- name: Run link checker
id: link-check
uses: lycheeverse/lychee-action@f613c4a64e50d792e0b31ec34bbcbba12263c6a6 # v2.3.0
uses: lycheeverse/lychee-action@8646ba30535128ac92d33dfc9133794bfdd9b411 # v2.8.0
with:
fail: true
args: docs --config ./.lychee/config.toml --offline

View File

@@ -8,7 +8,7 @@ on:
target_branch:
description: "Target branch for documentation update"
required: false
default: "release-x.60.x"
default: "release-x.62.x"
type: string
jobs:
@@ -17,6 +17,6 @@ jobs:
name: Call documentation update for release branch
uses: ./.github/workflows/docs-generate-base.yml
with:
target_branch: ${{ inputs.target_branch || 'release-x.60.x' }}
target_branch: ${{ inputs.target_branch || 'release-x.62.x' }}
secrets:
automation_token: ${{ secrets.METABASE_AUTOMATION_USER_TOKEN }}

View File

@@ -4,6 +4,10 @@ on:
workflow_dispatch:
inputs:
test:
# Technically this accepts anything that is accepted by Hawk:
# https://github.com/metabase/hawk/#leiningen-style-test-selection
# For example, to run everything in the tests/ dir you can do
# $ gh workflow run .github/workflows/drivers-stress-test.yml --field 'test=\"test/\" :only-tags [:mb/driver-tests]' --field times=2 --field driver=Snowflake
description: 'The exact test name with its namespace.'
type: string
required: true
@@ -59,7 +63,7 @@ jobs:
MB_ATHENA_TEST_SECRET_KEY: ${{ secrets.MB_ATHENA_TEST_SECRET_KEY }}
MB_ATHENA_TEST_S3_STAGING_DIR: ${{ secrets.MB_ATHENA_TEST_S3_STAGING_DIR }}
steps:
- uses: actions/checkout@v4
- uses: actions/checkout@v6
- name: Test Athena driver
uses: ./.github/actions/test-driver
with:
@@ -79,8 +83,9 @@ jobs:
MB_BIGQUERY_TEST_ACCESS_TOKEN: ${{ secrets.MB_BIGQUERY_TEST_ACCESS_TOKEN }}
MB_BIGQUERY_TEST_REFRESH_TOKEN: ${{ secrets.MB_BIGQUERY_TEST_REFRESH_TOKEN }}
MB_BIGQUERY_CLOUD_SDK_TEST_SERVICE_ACCOUNT_JSON: ${{ secrets.MB_BIGQUERY_CLOUD_SDK_TEST_SERVICE_ACCOUNT_JSON }}
MB_BIGQUERY_CLOUD_SDK_TEST_SERVICE_ACCOUNT_JSON_ROUTING: ${{ secrets.MB_BIGQUERY_CLOUD_SDK_TEST_SERVICE_ACCOUNT_JSON_ROUTING }}
steps:
- uses: actions/checkout@v4
- uses: actions/checkout@v6
- name: Test BigQuery Cloud SDK driver
uses: ./.github/actions/test-driver
with:
@@ -102,7 +107,7 @@ jobs:
env:
CLUSTER_SIZE: nano-quickstart
steps:
- uses: actions/checkout@v4
- uses: actions/checkout@v6
- name: Test Druid driver
uses: ./.github/actions/test-driver
with:
@@ -131,7 +136,7 @@ jobs:
MARIADB_ALLOW_EMPTY_ROOT_PASSWORD: true
MARIADB_DATABASE: mb_test
steps:
- uses: actions/checkout@v4
- uses: actions/checkout@v6
- name: Test MariaDB driver (10.6)
uses: ./.github/actions/test-driver
with:
@@ -160,7 +165,7 @@ jobs:
MARIADB_ALLOW_EMPTY_ROOT_PASSWORD: true
MARIADB_DATABASE: mb_test
steps:
- uses: actions/checkout@v4
- uses: actions/checkout@v6
- name: Test MariaDB driver (latest)
uses: ./.github/actions/test-driver
with:
@@ -182,7 +187,7 @@ jobs:
ports:
- "27017:27017"
steps:
- uses: actions/checkout@v4
- uses: actions/checkout@v6
- name: Test MongoDB driver (4.4)
uses: ./.github/actions/test-driver
with:
@@ -200,7 +205,7 @@ jobs:
MB_MONGO_TEST_PASSWORD: metasample123
MB_TEST_MONGO_REQUIRES_SSL: true
steps:
- uses: actions/checkout@v4
- uses: actions/checkout@v6
- name: Spin up Mongo docker container
run: docker run -d -p 27017:27017 --name metamongo metabase/qa-databases:mongo-sample-4.4 mongod --dbpath /data/db2/ --tlsMode requireTLS --tlsCertificateKeyFile /etc/mongo/metamongo.pem --tlsCAFile /etc/mongo/metaca.crt
- name: Wait until the port 27017 is ready
@@ -238,7 +243,7 @@ jobs:
ports:
- "27017:27017"
steps:
- uses: actions/checkout@v4
- uses: actions/checkout@v6
- name: Test MongoDB driver (5.0)
uses: ./.github/actions/test-driver
with:
@@ -256,7 +261,7 @@ jobs:
MB_MONGO_TEST_PASSWORD: metasample123
MB_TEST_MONGO_REQUIRES_SSL: true
steps:
- uses: actions/checkout@v4
- uses: actions/checkout@v6
- name: Spin up Mongo docker container
run: docker run -d -p 27017:27017 --name metamongo metabase/qa-databases:mongo-sample-5.0 mongod --dbpath /data/db2/ --tlsMode requireTLS --tlsCertificateKeyFile /etc/mongo/metamongo.pem --tlsCAFile /etc/mongo/metaca.crt
- name: Wait until the port 27017 is ready
@@ -297,7 +302,7 @@ jobs:
MONGO_INITDB_ROOT_USERNAME: metabase
MONGO_INITDB_ROOT_PASSWORD: metasample123
steps:
- uses: actions/checkout@v4
- uses: actions/checkout@v6
- name: Test MongoDB driver (latest)
uses: ./.github/actions/test-driver
with:
@@ -326,7 +331,7 @@ jobs:
MYSQL_ALLOW_EMPTY_PASSWORD: true
MYSQL_DATABASE: mb_test
steps:
- uses: actions/checkout@v4
- uses: actions/checkout@v6
- name: Test MySQL driver (8.0)
uses: ./.github/actions/test-driver
with:
@@ -364,7 +369,7 @@ jobs:
ports:
- "3306:3306"
steps:
- uses: actions/checkout@v4
- uses: actions/checkout@v6
- name: Test MySQL driver (latest)
uses: ./.github/actions/test-driver
with:
@@ -390,7 +395,7 @@ jobs:
ports:
- "1521:1521"
steps:
- uses: actions/checkout@v4
- uses: actions/checkout@v6
- name: Test Oracle driver (18.4)
uses: ./.github/actions/test-driver
with:
@@ -425,7 +430,7 @@ jobs:
ports:
- "2484:2484"
steps:
- uses: actions/checkout@v4
- uses: actions/checkout@v6
- name: Test Oracle driver (21.3)
uses: ./.github/actions/test-driver
with:
@@ -455,7 +460,7 @@ jobs:
POSTGRES_DB: mb_test
POSTGRES_HOST_AUTH_METHOD: trust
steps:
- uses: actions/checkout@v4
- uses: actions/checkout@v6
- name: Test Postgres driver (14)
uses: ./.github/actions/test-driver
with:
@@ -488,7 +493,7 @@ jobs:
POSTGRES_DB: mb_test
POSTGRES_HOST_AUTH_METHOD: trust
steps:
- uses: actions/checkout@v4
- uses: actions/checkout@v6
- name: Test Postgres driver (latest)
uses: ./.github/actions/test-driver
with:
@@ -518,7 +523,7 @@ jobs:
env:
JAVA_TOOL_OPTIONS: "-Xmx2g"
steps:
- uses: actions/checkout@v4
- uses: actions/checkout@v6
- name: Wait for port ${{ env.MB_PRESTO_JDBC_TEST_PORT }} to be ready
run: while ! nc -z localhost ${{ env.MB_PRESTO_JDBC_TEST_PORT }}; do sleep 0.1; done
timeout-minutes: 15
@@ -556,7 +561,7 @@ jobs:
MB_REDSHIFT_TEST_HOST: ${{ secrets.MB_REDSHIFT_TEST_HOST }}
MB_REDSHIFT_TEST_PASSWORD: ${{ secrets.MB_REDSHIFT_TEST_PASSWORD }}
steps:
- uses: actions/checkout@v4
- uses: actions/checkout@v6
- name: Test Redshift driver
uses: ./.github/actions/test-driver
with:
@@ -573,11 +578,21 @@ jobs:
MB_SNOWFLAKE_TEST_USER: METABASE CI
MB_SNOWFLAKE_TEST_ACCOUNT: ${{ secrets.MB_SNOWFLAKE_TEST_ACCOUNT }}
MB_SNOWFLAKE_TEST_PASSWORD: ${{ secrets.MB_SNOWFLAKE_TEST_PASSWORD }}
MB_SNOWFLAKE_TEST_PRIVATE_KEY: ${{ secrets.MB_SNOWFLAKE_TEST_PRIVATE_KEY }}
MB_SNOWFLAKE_TEST_WAREHOUSE: ${{ secrets.MB_SNOWFLAKE_TEST_WAREHOUSE }}
MB_SNOWFLAKE_TEST_PK_USER: METABASE PK
MB_SNOWFLAKE_TEST_PK_PRIVATE_KEY: ${{ secrets.MB_SNOWFLAKE_TEST_PK_PRIVATE_KEY }}
MB_SNOWFLAKE_TEST_PK_PRIVATE_KEY_2: ${{ secrets.MB_SNOWFLAKE_TEST_PK_PRIVATE_KEY_2 }}
MB_SNOWFLAKE_TEST_PK_PUBLIC_KEY: ${{ secrets.MB_SNOWFLAKE_TEST_PK_PUBLIC_KEY }}
MB_SNOWFLAKE_TEST_PK_PUBLIC_KEY_2: ${{ secrets.MB_SNOWFLAKE_TEST_PK_PUBLIC_KEY_2 }}
# RSA Role testing:
MB_SNOWFLAKE_TEST_RSA_ROLE_TEST_DEFAULT_USER: RSA_ROLE_TEST_DEFAULT_USER
MB_SNOWFLAKE_TEST_RSA_ROLE_TEST_CUSTOM_USER: RSA_ROLE_TEST_CUSTOM_USER
MB_SNOWFLAKE_TEST_RSA_ROLE_TEST_ROLE: RSA_ROLE_TEST_ROLE
MB_SNOWFLAKE_TEST_RSA_ROLE_TEST_DB: RSA_ROLE_TEST_DB
steps:
- uses: actions/checkout@v4
- uses: actions/checkout@v6
- name: Test Snowflake driver
uses: ./.github/actions/test-driver
with:
@@ -597,7 +612,7 @@ jobs:
ports:
- "10000:10000"
steps:
- uses: actions/checkout@v4
- uses: actions/checkout@v6
- name: Test Spark driver
uses: ./.github/actions/test-driver
with:
@@ -612,7 +627,7 @@ jobs:
CI: 'true'
DRIVERS: sqlite
steps:
- uses: actions/checkout@v4
- uses: actions/checkout@v6
- name: Test SQLite driver
uses: ./.github/actions/test-driver
with:
@@ -639,7 +654,7 @@ jobs:
SA_PASSWORD: 'P@ssw0rd'
MSSQL_MEMORY_LIMIT_MB: 1024
steps:
- uses: actions/checkout@v4
- uses: actions/checkout@v6
- name: Test MS SQL Server 2017 driver
uses: ./.github/actions/test-driver
with:
@@ -666,7 +681,7 @@ jobs:
SA_PASSWORD: 'P@ssw0rd'
MSSQL_MEMORY_LIMIT_MB: 1024
steps:
- uses: actions/checkout@v4
- uses: actions/checkout@v6
- name: Test MS SQL Server 2022 driver
uses: ./.github/actions/test-driver
with:
@@ -689,7 +704,7 @@ jobs:
- "8080:8080"
name: Starburst
steps:
- uses: actions/checkout@v4
- uses: actions/checkout@v6
- name: Make plugins directory
run: mkdir plugins
- name: Test Starburst driver
@@ -707,7 +722,7 @@ jobs:
contents: read
steps:
- name: Configure AWS credentials
uses: aws-actions/configure-aws-credentials@7474bc4690e29a8392af63c5b98e7449536d5c3a # v4
uses: aws-actions/configure-aws-credentials@d979d5b3a71173a29b74b5b88418bfda9437d885 # v6.1.1
with:
role-to-assume: ${{ secrets.CI_IAM_ROLE }}
role-session-name: GitHub_to_AWS_via_FederatedOIDC
@@ -744,7 +759,7 @@ jobs:
username: ${{ needs.login-to-ecr.outputs.docker_username }}
password: ${{ needs.login-to-ecr.outputs.docker_password }}
steps:
- uses: actions/checkout@v4
- uses: actions/checkout@v6
- name: Make plugins directory
run: mkdir plugins
- name: Test Vertica driver

View File

@@ -38,6 +38,24 @@ jobs:
sqlite-should-run: ${{ steps.driver-decisions.outputs.sqlite-should-run }}
sqlserver-should-run: ${{ steps.driver-decisions.outputs.sqlserver-should-run }}
vertica-should-run: ${{ steps.driver-decisions.outputs.vertica-should-run }}
# Per-driver configured status from ci-test-config (skip | info | required)
athena-status: ${{ steps.driver-decisions.outputs.athena-status }}
bigquery-status: ${{ steps.driver-decisions.outputs.bigquery-status }}
clickhouse-status: ${{ steps.driver-decisions.outputs.clickhouse-status }}
databricks-status: ${{ steps.driver-decisions.outputs.databricks-status }}
druid-jdbc-status: ${{ steps.driver-decisions.outputs.druid-jdbc-status }}
mongo-status: ${{ steps.driver-decisions.outputs.mongo-status }}
mongo-ssl-status: ${{ steps.driver-decisions.outputs.mongo-ssl-status }}
mongo-sharded-cluster-status: ${{ steps.driver-decisions.outputs.mongo-sharded-cluster-status }}
mysql-mariadb-status: ${{ steps.driver-decisions.outputs.mysql-mariadb-status }}
oracle-status: ${{ steps.driver-decisions.outputs.oracle-status }}
presto-jdbc-status: ${{ steps.driver-decisions.outputs.presto-jdbc-status }}
redshift-status: ${{ steps.driver-decisions.outputs.redshift-status }}
snowflake-status: ${{ steps.driver-decisions.outputs.snowflake-status }}
sparksql-status: ${{ steps.driver-decisions.outputs.sparksql-status }}
sqlite-status: ${{ steps.driver-decisions.outputs.sqlite-status }}
sqlserver-status: ${{ steps.driver-decisions.outputs.sqlserver-status }}
vertica-status: ${{ steps.driver-decisions.outputs.vertica-status }}
# Per-driver quarantine conflict flags (set when driver is quarantined but has file changes)
athena-quarantine-conflict: ${{ steps.driver-decisions.outputs.athena-quarantine-conflict }}
bigquery-quarantine-conflict: ${{ steps.driver-decisions.outputs.bigquery-quarantine-conflict }}
@@ -59,8 +77,30 @@ jobs:
sqlserver-quarantine-conflict: ${{ steps.driver-decisions.outputs.sqlserver-quarantine-conflict }}
vertica-quarantine-conflict: ${{ steps.driver-decisions.outputs.vertica-quarantine-conflict }}
steps:
# Up front (before any driver job), drop info-driver comments from older commits so a
# superseded comment disappears immediately rather than waiting for this run's drivers.
# Keeps this commit's comment (if any) so a re-run doesn't flicker.
- name: Remove stale info-driver comments
if: ${{ github.event_name == 'pull_request' }}
uses: actions/github-script@v9
continue-on-error: true
with:
script: | # js
const pr = context.payload.pull_request;
if (!pr) return;
const { owner, repo } = context.repo;
const PREFIX = "<!-- info-driver-result";
const keep = `<!-- info-driver-results:${pr.head.sha} -->`;
const { data: comments } = await github.rest.issues.listComments({
owner, repo, issue_number: pr.number, per_page: 100,
});
for (const c of comments) {
if (c.body.includes(PREFIX) && !c.body.includes(keep)) {
await github.rest.issues.deleteComment({ owner, repo, comment_id: c.id }).catch(() => {});
}
}
- name: Checkout repository
uses: actions/checkout@v4
uses: actions/checkout@v6
- name: Fetch base branch
if: github.event_name == 'pull_request' && github.event.pull_request.base.ref != ''
run: |
@@ -121,7 +161,7 @@ jobs:
echo ""
echo "If you merge without testing, you risk breaking the Athena driver!"
exit 1
- uses: actions/checkout@v4
- uses: actions/checkout@v6
- name: Test Athena driver
id: test-driver
uses: ./.github/actions/test-driver
@@ -141,6 +181,8 @@ jobs:
aws-region: ${{ vars.AWS_REGION }}
trunk-api-token: ${{ secrets.TRUNK_API_TOKEN }}
previous-step-outcome: ${{ steps.test-driver.outputs.test-outcome }}
driver-status: ${{ needs.determine-driver-skips.outputs.athena-status }}
report-variant: ${{ matrix.job.name }}
be-tests-bigquery-cloud-sdk-ee:
needs: determine-driver-skips
@@ -151,11 +193,20 @@ jobs:
fail-fast: false
matrix:
job:
- name: Driver Tests
- name: Driver Tests Part 1
test-args: >-
:only-tags [:mb/driver-tests]
:exclude-tags [:mb/transforms-python-test]
:fail-fast? true
:partition/total 2
:partition/index 0
- name: Driver Tests Part 2
test-args: >-
:only-tags [:mb/driver-tests]
:exclude-tags [:mb/transforms-python-test]
:fail-fast? true
:partition/total 2
:partition/index 1
- name: Transforms Python Tests
test-args: >-
:only-tags [:mb/transforms-python-test]
@@ -182,7 +233,7 @@ jobs:
echo ""
echo "If you merge without testing, you risk breaking the BigQuery driver!"
exit 1
- uses: actions/checkout@v4
- uses: actions/checkout@v6
- name: Setup python-runner
if: ${{ matrix.job.needs-python-runner == true }}
uses: ./.github/actions/setup-python-runner
@@ -206,6 +257,8 @@ jobs:
aws-region: ${{ vars.AWS_REGION }}
trunk-api-token: ${{ secrets.TRUNK_API_TOKEN }}
previous-step-outcome: ${{ steps.test-driver.outputs.test-outcome }}
driver-status: ${{ needs.determine-driver-skips.outputs.bigquery-status }}
report-variant: ${{ matrix.job.name }}
- name: Cleanup python-runner
if: ${{ always() && matrix.job.needs-python-runner == true }}
uses: ./.github/actions/cleanup-python-runner
@@ -249,7 +302,7 @@ jobs:
echo ""
echo "If you merge without testing, you risk breaking the ClickHouse driver!"
exit 1
- uses: actions/checkout@v4
- uses: actions/checkout@v6
- name: Setup python-runner
if: ${{ matrix.job.needs-python-runner == true }}
uses: ./.github/actions/setup-python-runner
@@ -259,7 +312,7 @@ jobs:
run: |
sudo echo "127.0.0.1 server.clickhouseconnect.test" | sudo tee -a /etc/hosts
- name: Start ClickHouse in Docker
uses: hoverkraft-tech/compose-action@802a148945af6399a338c7906c267331b39a71af # v2.0.0
uses: hoverkraft-tech/compose-action@d2bee4f07e8ca410d6b196d00f90c12e7d48c33a # v2.6.0
with:
compose-file: "modules/drivers/clickhouse/docker-compose.yml"
down-flags: "--volumes"
@@ -296,54 +349,6 @@ jobs:
upload-logs: ${{ steps.test-driver.outcome == 'failure' }}
logs-artifact-suffix: ${{ matrix.job.name }} ${{ matrix.version.name }}
be-tests-druid-ee:
needs: determine-driver-skips
if: ${{ needs.determine-driver-skips.outputs.druid-should-run == 'true' || needs.determine-driver-skips.outputs.druid-quarantine-conflict == 'true' }}
runs-on: ${{ vars.DEFAULT_RUNNER_KEY }}
timeout-minutes: 40
env:
CI: 'true'
DRIVERS: druid
services:
druid:
image: metabase/druid:0.20.2
ports:
- "8082:8082"
env:
CLUSTER_SIZE: nano-quickstart
name: Druid (Legacy)
steps:
- name: Fail on quarantine conflict
if: ${{ needs.determine-driver-skips.outputs.druid-quarantine-conflict == 'true' }}
run: |
echo "::error::Druid driver is quarantined but you changed files in modules/drivers/druid/"
echo ""
echo "Your changes will NOT be tested because the driver is currently quarantined."
echo "To run Druid tests, add the label: break-quarantine-druid"
echo ""
echo "If you merge without testing, you risk breaking the Druid driver!"
exit 1
- uses: actions/checkout@v4
- name: Test Druid driver
id: test-driver
uses: ./.github/actions/test-driver
with:
junit-name: 'be-tests-druid-ee'
test-args: >-
:only-tags [:mb/driver-tests]
:exclude-tags [:mb/transforms-python-test]
- name: Upload Test Results
uses: ./.github/actions/upload-test-results
if: always()
with:
input-path: ./target/junit/
output-name: ${{ github.job }}
bucket: ${{ vars.AWS_S3_TEST_RESULTS_BUCKET }}
role-to-assume: ${{ secrets.CI_TEST_RESULTS_IAM_ROLE }}
aws-region: ${{ vars.AWS_REGION }}
trunk-api-token: ${{ secrets.TRUNK_API_TOKEN }}
previous-step-outcome: ${{ steps.test-driver.outputs.test-outcome }}
be-tests-druid-jdbc-ee:
needs: determine-driver-skips
if: ${{ needs.determine-driver-skips.outputs.druid-jdbc-should-run == 'true' || needs.determine-driver-skips.outputs.druid-jdbc-quarantine-conflict == 'true' }}
@@ -371,7 +376,7 @@ jobs:
echo ""
echo "If you merge without testing, you risk breaking the Druid JDBC driver!"
exit 1
- uses: actions/checkout@v4
- uses: actions/checkout@v6
- name: Test Druid JDBC driver
id: test-driver
uses: ./.github/actions/test-driver
@@ -419,7 +424,7 @@ jobs:
echo ""
echo "If you merge without testing, you risk breaking the Databricks driver!"
exit 1
- uses: actions/checkout@v4
- uses: actions/checkout@v6
- name: Test Databricks driver
id: test-driver
uses: ./.github/actions/test-driver
@@ -439,6 +444,8 @@ jobs:
aws-region: ${{ vars.AWS_REGION }}
trunk-api-token: ${{ secrets.TRUNK_API_TOKEN }}
previous-step-outcome: ${{ steps.test-driver.outputs.test-outcome }}
driver-status: ${{ needs.determine-driver-skips.outputs.databricks-status }}
report-variant: ${{ matrix.job.name }}
be-tests-databricks-multi-catalog-ee:
needs: determine-driver-skips
@@ -468,7 +475,7 @@ jobs:
echo ""
echo "If you merge without testing, you risk breaking the Databricks driver!"
exit 1
- uses: actions/checkout@v4
- uses: actions/checkout@v6
- name: Test Databricks driver with multi-catalog
id: test-driver
uses: ./.github/actions/test-driver
@@ -488,6 +495,8 @@ jobs:
aws-region: ${{ vars.AWS_REGION }}
trunk-api-token: ${{ secrets.TRUNK_API_TOKEN }}
previous-step-outcome: ${{ steps.test-driver.outputs.test-outcome }}
driver-status: ${{ needs.determine-driver-skips.outputs.databricks-status }}
report-variant: ${{ matrix.job.name }}
be-tests-h2:
needs: determine-driver-skips
@@ -498,7 +507,7 @@ jobs:
CI: 'true'
name: H2
steps:
- uses: actions/checkout@v4
- uses: actions/checkout@v6
- name: Test H2 driver
id: test-driver
uses: ./.github/actions/test-driver
@@ -528,7 +537,7 @@ jobs:
CI: 'true'
name: H2 (OSS)
steps:
- uses: actions/checkout@v4
- uses: actions/checkout@v6
- name: Test H2 driver (OSS)
id: test-driver
uses: ./.github/actions/test-driver
@@ -653,7 +662,7 @@ jobs:
# "be-tests-mariadb-10-2-ee (0)"
name: "${{ matrix.version.name }} ${{ matrix.job.name }}"
steps:
- uses: actions/checkout@v4
- uses: actions/checkout@v6
- name: Setup python-runner
if: ${{ matrix.job.needs-python-runner == true }}
uses: ./.github/actions/setup-python-runner
@@ -735,7 +744,7 @@ jobs:
echo ""
echo "If you merge without testing, you risk breaking the MongoDB driver!"
exit 1
- uses: actions/checkout@v4
- uses: actions/checkout@v6
- name: Setup python-runner
if: ${{ matrix.job.needs-python-runner == true }}
uses: ./.github/actions/setup-python-runner
@@ -796,7 +805,7 @@ jobs:
echo ""
echo "If you merge without testing, you risk breaking the MongoDB SSL driver!"
exit 1
- uses: actions/checkout@v4
- uses: actions/checkout@v6
- name: Spin up Mongo docker container
run: >-
docker run -d
@@ -867,7 +876,7 @@ jobs:
echo ""
echo "If you merge without testing, you risk breaking the MongoDB Sharded Cluster driver!"
exit 1
- uses: actions/checkout@v4
- uses: actions/checkout@v6
- name: Test MongoDB driver (Sharded cluster)
id: test-driver
uses: ./.github/actions/test-driver
@@ -950,7 +959,7 @@ jobs:
echo ""
echo "If you merge without testing, you risk breaking the Oracle driver!"
exit 1
- uses: actions/checkout@v4
- uses: actions/checkout@v6
- name: Test ${{ matrix.version.name }}
id: test-driver
uses: ./.github/actions/test-driver
@@ -1049,7 +1058,7 @@ jobs:
POSTGRES_DB: mb_test
POSTGRES_HOST_AUTH_METHOD: trust
steps:
- uses: actions/checkout@v4
- uses: actions/checkout@v6
- name: Setup python-runner
if: ${{ matrix.job.needs-python-runner == true }}
uses: ./.github/actions/setup-python-runner
@@ -1117,7 +1126,7 @@ jobs:
echo ""
echo "If you merge without testing, you risk breaking the Presto JDBC driver!"
exit 1
- uses: actions/checkout@v4
- uses: actions/checkout@v6
- uses: ./.github/actions/await-port
with:
port: ${{ env.MB_PRESTO_JDBC_TEST_PORT }}
@@ -1220,7 +1229,7 @@ jobs:
echo ""
echo "If you merge without testing, you risk breaking the Redshift driver!"
exit 1
- uses: actions/checkout@v4
- uses: actions/checkout@v6
- name: Setup python-runner
if: ${{ matrix.job.needs-python-runner == true }}
uses: ./.github/actions/setup-python-runner
@@ -1246,6 +1255,8 @@ jobs:
aws-region: ${{ vars.AWS_REGION }}
trunk-api-token: ${{ secrets.TRUNK_API_TOKEN }}
previous-step-outcome: ${{ steps.test-driver.outputs.test-outcome }}
driver-status: ${{ needs.determine-driver-skips.outputs.redshift-status }}
report-variant: ${{ matrix.job.name }}
- name: Cleanup python-runner
if: ${{ always() && matrix.job.needs-python-runner == true }}
uses: ./.github/actions/cleanup-python-runner
@@ -1302,7 +1313,7 @@ jobs:
echo ""
echo "If you merge without testing, you risk breaking the Snowflake driver!"
exit 1
- uses: actions/checkout@v4
- uses: actions/checkout@v6
- name: Setup python-runner
if: ${{ matrix.job.needs-python-runner == true }}
uses: ./.github/actions/setup-python-runner
@@ -1326,6 +1337,8 @@ jobs:
aws-region: ${{ vars.AWS_REGION }}
trunk-api-token: ${{ secrets.TRUNK_API_TOKEN }}
previous-step-outcome: ${{ steps.test-driver.outputs.test-outcome }}
driver-status: ${{ needs.determine-driver-skips.outputs.snowflake-status }}
report-variant: ${{ matrix.job.name }}
- name: Cleanup python-runner
if: ${{ always() && matrix.job.needs-python-runner == true }}
uses: ./.github/actions/cleanup-python-runner
@@ -1358,7 +1371,7 @@ jobs:
echo ""
echo "If you merge without testing, you risk breaking the SparkSQL driver!"
exit 1
- uses: actions/checkout@v4
- uses: actions/checkout@v6
- name: Test Spark driver
id: test-driver
uses: ./.github/actions/test-driver
@@ -1392,14 +1405,14 @@ jobs:
- name: Fail on quarantine conflict
if: ${{ needs.determine-driver-skips.outputs.sqlite-quarantine-conflict == 'true' }}
run: |
echo "::error::SQLite driver is quarantined but you changed files in modules/drivers/sqlite/"
echo "::error::SQLite driver is quarantined but you changed files that affect it"
echo ""
echo "Your changes will NOT be tested because the driver is currently quarantined."
echo "To run SQLite tests, add the label: break-quarantine-sqlite"
echo ""
echo "If you merge without testing, you risk breaking the SQLite driver!"
exit 1
- uses: actions/checkout@v4
- uses: actions/checkout@v6
- name: Test SQLite driver
id: test-driver
uses: ./.github/actions/test-driver
@@ -1471,7 +1484,7 @@ jobs:
echo ""
echo "If you merge without testing, you risk breaking the SQL Server driver!"
exit 1
- uses: actions/checkout@v4
- uses: actions/checkout@v6
- name: Setup python-runner
if: ${{ matrix.job.needs-python-runner == true }}
uses: ./.github/actions/setup-python-runner
@@ -1520,7 +1533,7 @@ jobs:
# - "8080:8080"
# name: Starburst
# steps:
# - uses: actions/checkout@v4
# - uses: actions/checkout@v6
# - name: Make plugins directory
# run: mkdir plugins
# - name: Test Starburst driver
@@ -1550,7 +1563,7 @@ jobs:
contents: read
steps:
- name: Configure AWS credentials
uses: aws-actions/configure-aws-credentials@7474bc4690e29a8392af63c5b98e7449536d5c3a # v4
uses: aws-actions/configure-aws-credentials@d979d5b3a71173a29b74b5b88418bfda9437d885 # v6.1.1
with:
role-to-assume: ${{ secrets.CI_IAM_ROLE }}
role-session-name: GitHub_to_AWS_via_FederatedOIDC
@@ -1598,7 +1611,7 @@ jobs:
echo ""
echo "If you merge without testing, you risk breaking the Vertica driver!"
exit 1
- uses: actions/checkout@v4
- uses: actions/checkout@v6
- name: Make plugins directory
run: mkdir plugins
- name: Test Vertica driver
@@ -1623,11 +1636,19 @@ jobs:
drivers-tests-result:
needs:
# Supplies each driver's configured status (info/skip/required) so this gate can ignore
# "info" drivers regardless of their result -- including cancelled or timed-out runs, which
# the per-job continue-on-error cannot force green because the controlling step never
# completes. Not a driver job; it is filtered out of the pass/fail check below.
- determine-driver-skips
# if you add a driver job, you must add it to this list for it to be a required check
- be-tests-h2
- be-tests-athena-ee
# Listed unconditionally; ci-test-config controls their effect via per-driver status.
# An "info" driver is reported as passing, so it never blocks this check; flip it to
# "required" in the config to make a real failure gate, with no change here.
- be-tests-snowflake-ee
- be-tests-bigquery-cloud-sdk-ee
- be-tests-druid-ee
- be-tests-databricks-ee
- be-tests-druid-jdbc-ee
- be-tests-mysql-mariadb
@@ -1638,7 +1659,6 @@ jobs:
- be-tests-postgres
- be-tests-presto-jdbc-ee
- be-tests-redshift-ee
- be-tests-snowflake-ee
- be-tests-sparksql-ee
- be-tests-sqlite-ee
- be-tests-sqlserver
@@ -1652,21 +1672,50 @@ jobs:
if: always() && !cancelled()
steps:
- name: Check job status
uses: actions/github-script@v7
uses: actions/github-script@v9
env:
needs: ${{ toJson(needs) }}
# Job name -> configured driver status, from determine-driver-skips. An "info" driver is
# data-collection only and must never gate; a cancelled/timed-out info job lands here as
# cancelled/failure (its own continue-on-error can't force it green), so we drop info jobs
# from the gate entirely. Empty/unknown status falls back to "required" (gates as before).
statuses: |
{
"be-tests-athena-ee": "${{ needs.determine-driver-skips.outputs.athena-status }}",
"be-tests-bigquery-cloud-sdk-ee": "${{ needs.determine-driver-skips.outputs.bigquery-status }}",
"be-tests-clickhouse-ee": "${{ needs.determine-driver-skips.outputs.clickhouse-status }}",
"be-tests-databricks-ee": "${{ needs.determine-driver-skips.outputs.databricks-status }}",
"be-tests-druid-jdbc-ee": "${{ needs.determine-driver-skips.outputs.druid-jdbc-status }}",
"be-tests-mongo": "${{ needs.determine-driver-skips.outputs.mongo-status }}",
"be-tests-mongo-ssl": "${{ needs.determine-driver-skips.outputs.mongo-ssl-status }}",
"be-tests-mongo-sharded-cluster-ee": "${{ needs.determine-driver-skips.outputs.mongo-sharded-cluster-status }}",
"be-tests-mysql-mariadb": "${{ needs.determine-driver-skips.outputs.mysql-mariadb-status }}",
"be-tests-oracle": "${{ needs.determine-driver-skips.outputs.oracle-status }}",
"be-tests-presto-jdbc-ee": "${{ needs.determine-driver-skips.outputs.presto-jdbc-status }}",
"be-tests-redshift-ee": "${{ needs.determine-driver-skips.outputs.redshift-status }}",
"be-tests-snowflake-ee": "${{ needs.determine-driver-skips.outputs.snowflake-status }}",
"be-tests-sparksql-ee": "${{ needs.determine-driver-skips.outputs.sparksql-status }}",
"be-tests-sqlite-ee": "${{ needs.determine-driver-skips.outputs.sqlite-status }}",
"be-tests-sqlserver": "${{ needs.determine-driver-skips.outputs.sqlserver-status }}"
}
with:
script: | # js
const needs = JSON.parse(process.env.needs);
const statuses = JSON.parse(process.env.statuses);
const jobs = Object.entries(needs).map(
([jobName, jobValues]) => ({
// Drop the determine-driver-skips bookkeeping job, then drop "info" drivers: they are
// non-gating, so their result (including cancelled/timed-out) must not fail this check.
const jobs = Object.entries(needs)
.filter(([jobName]) => jobName !== 'determine-driver-skips')
.map(([jobName, jobValues]) => ({
name: jobName,
result: jobValues.result
}));
result: jobValues.result,
status: statuses[jobName] || 'required'
}))
.filter(job => job.status !== 'info');
// are all jobs skipped or successful?
if (jobs.every(job => (job.result === 'skipped' || job.result === 'success' || job.name == 'be-tests-starburst-ee'))) {
if (jobs.every(job => (job.result === 'skipped' || job.result === 'success'))) {
console.log("");
console.log(" _------. ");
console.log(" / , \_ ");

View File

@@ -28,16 +28,16 @@ jobs:
regularChunks: ${{ steps.matrix.outputs.regularChunks }}
isDefaultSpecPattern: ${{ steps.matrix.outputs.isDefaultSpecPattern }}
steps:
- uses: actions/checkout@v4
- uses: actions/checkout@v6
with:
sparse-checkout: |
.github
sparse-checkout-cone-mode: false
- uses: actions/setup-node@v4
- uses: actions/setup-node@v6.4.0
with:
node-version: lts/Jod # 22.x.x
- name: Build e2e matrix
uses: actions/github-script@v7
uses: actions/github-script@v9
id: matrix
env:
chunks: ${{ inputs.chunks }}

View File

@@ -36,6 +36,54 @@ on:
type: boolean
required: false
default: false
build_jar:
description: "Build the uberjar from this branch (required to test frontend fixes) instead of downloading a prebuilt one"
type: boolean
required: false
default: false
# Reusable form so other workflows (e.g. the flaky-test-fix PR auto-runner) can fan a
# matrix of changed specs into individual stress runs. Inputs mirror workflow_dispatch;
# both triggers read them via the `inputs` context.
workflow_call:
inputs:
spec:
description: "Relative path of the target spec"
type: string
required: true
burn_in:
description: "Number of times to run the test (e.g. 20)"
type: string
required: false
default: "20"
grep:
description: "Grep and filter tests to run in isolation"
type: string
required: false
mb_edition:
description: 'Set MB_EDITION env var to "ee" (enterprise) or "oss" (open-source)'
type: string
required: false
default: "ee"
qa_db:
description: "Whether to set up and include external QA databases"
type: boolean
required: false
default: false
enable_network_throttling:
description: "Enable network throttling simulation (4g slow like in chrome dev tools)"
type: boolean
required: false
default: true
fail_fast:
description: "Stop running remaining tests after the first failure"
type: boolean
required: false
default: false
build_jar:
description: "Build the uberjar from this branch (required to test frontend fixes) instead of downloading a prebuilt one"
type: boolean
required: false
default: false
jobs:
determine-test-type:
@@ -73,6 +121,7 @@ jobs:
echo '- `qa_db`: "${{ inputs.qa_db }}"' >> $GITHUB_STEP_SUMMARY
echo '- `enable_network_throttling`: "${{ inputs.enable_network_throttling }}"' >> $GITHUB_STEP_SUMMARY
echo '- `fail_fast`: "${{ inputs.fail_fast }}"' >> $GITHUB_STEP_SUMMARY
echo '- `build_jar`: "${{ inputs.build_jar }}"' >> $GITHUB_STEP_SUMMARY
echo '' >> $GITHUB_STEP_SUMMARY
echo 'triggered by: @${{ github.event.sender.login }}' >> $GITHUB_STEP_SUMMARY
@@ -92,7 +141,7 @@ jobs:
MB_EDITION: ${{ inputs.mb_edition }}
CYPRESS_ENABLE_NETWORK_THROTTLING: ${{ inputs.enable_network_throttling }}
steps:
- uses: actions/checkout@v4
- uses: actions/checkout@v6
with:
# Important because we need previous commits hashes to find and download the uberjar!
fetch-depth: 20
@@ -109,16 +158,24 @@ jobs:
mysql: ${{ inputs.qa_db }}
mongo: ${{ inputs.qa_db }}
- name: Download Metabase ${{ matrix.edition }} uberjar
- name: Download Metabase ${{ inputs.mb_edition }} uberjar
if: ${{ !inputs.build_jar }}
uses: ./.github/actions/e2e-download-uberjar
with:
edition: "ee"
edition: ${{ inputs.mb_edition }}
- name: Prepare front-end environment
uses: ./.github/actions/prepare-frontend
- name: Prepare back-end environment
uses: ./.github/actions/prepare-backend
- name: Build Metabase ${{ inputs.mb_edition }} uberjar from this branch
if: ${{ inputs.build_jar }}
run: ./bin/build.sh
env:
# license check is irrelevant for ephemeral e2e jars and slows the build down
SKIP_LICENSES: true
- name: Build Embedding SDK package
if: ${{ needs.determine-test-type.outputs.type == 'component' }}
run: bun run build-embedding-sdk-package
@@ -146,13 +203,13 @@ jobs:
# For all options and fine-grained control, take a look at the documentation
# https://github.com/cypress-io/cypress/tree/develop/npm/grep
- name: Stress-test ${{ github.event.inputs.spec }} ${{ github.event.inputs.burn_in }} times
- name: Stress-test ${{ inputs.spec }} ${{ inputs.burn_in }} times
env:
GREP: ${{ github.event.inputs.grep }}
GREP: ${{ inputs.grep }}
CYPRESS_BROWSER: ${{ steps.cypress-prep.outputs.chrome-path }}
FAIL_FAST: ${{ inputs.fail_fast }}
SPEC: ${{ github.event.inputs.spec }}
BURN_IN: ${{ github.event.inputs.burn_in }}
SPEC: ${{ inputs.spec }}
BURN_IN: ${{ inputs.burn_in }}
run: |
node e2e/runner/run_cypress_ci.js ${{ needs.determine-test-type.outputs.type }} \
--spec "${SPEC}" \
@@ -160,7 +217,7 @@ jobs:
--config-file e2e/support/cypress-stress-test.config.js
- name: Upload Cypress Artifacts upon failure
uses: actions/upload-artifact@v4
uses: actions/upload-artifact@v7
if: failure()
with:
name: cypress-failed-tests-recording

View File

@@ -0,0 +1,88 @@
name: E2E Stress Test (flaky-test-fix PR)
# Auto-runs the e2e stress test on every changed Cypress spec in a PR
on:
pull_request:
# `opened`/`reopened` catch PRs created already labelled; `labeled` catches the label
# being added later; `synchronize` re-burns after new commits while the label is set.
types: [opened, reopened, synchronize, labeled]
concurrency:
# One in-flight set of stress runs per PR branch — new commits cancel the previous burn.
group: ${{ github.workflow }}-${{ github.ref }}
cancel-in-progress: true
jobs:
detect-changed-specs:
name: Detect changed e2e specs
if: contains(github.event.pull_request.labels.*.name, 'flaky-test-fix')
runs-on: ${{ vars.DEFAULT_RUNNER_KEY }}
timeout-minutes: 5
outputs:
# JSON array of spec paths, consumed by the matrix below.
specs: ${{ steps.detect.outputs.specs }}
should_run: ${{ steps.detect.outputs.should_run }}
# burn-50 / burn-100 labels bump the burn-in past 20
burn_in: >-
${{ contains(github.event.pull_request.labels.*.name, 'burn-100') && '100'
|| contains(github.event.pull_request.labels.*.name, 'burn-50') && '50'
|| '20' }}
steps:
- name: Detect changed Cypress spec files
id: detect
env:
GH_TOKEN: ${{ github.token }}
REPO: ${{ github.repository }}
PR_NUMBER: ${{ github.event.pull_request.number }}
run: | # sh
# List every file in the PR, drop deletions, then keep only regular e2e Cypress
# specs: paths under e2e/ ending in .cy.spec.{js,ts,tsx}, excluding the
# embedding-SDK component specs under e2e/test-component/ (those use a different
# runner and are out of scope here). The greps are guarded with `|| true` so an
# empty match doesn't trip `pipefail`.
FILES=$(
gh api "repos/${REPO}/pulls/${PR_NUMBER}/files" \
--paginate \
--jq '.[] | select(.status != "removed") | .filename' \
| { grep -E '^e2e/.*\.cy\.spec\.(js|ts|tsx)$' || true; } \
| { grep -v '^e2e/test-component/' || true; } \
| sort -u
)
if [[ -z "$FILES" ]]; then
echo "No changed e2e Cypress spec files found — nothing to stress test."
echo "should_run=false" >> "$GITHUB_OUTPUT"
exit 0
fi
echo "Changed e2e Cypress specs:"
echo "$FILES" | sed 's/^/ - /'
# Compact JSON array for the matrix, e.g. ["e2e/test/scenarios/a.cy.spec.js"].
SPECS_JSON=$(echo "$FILES" | jq -R . | jq -cs .)
echo "specs=${SPECS_JSON}" >> "$GITHUB_OUTPUT"
echo "should_run=true" >> "$GITHUB_OUTPUT"
echo "Note: you can use the burn-50 or burn-100 labels to increase the burn-in count from the default of 20." >> $GITHUB_STEP_SUMMARY
stress-test:
name: Stress test
needs: detect-changed-specs
if: needs.detect-changed-specs.outputs.should_run == 'true'
strategy:
# Don't let one flaky spec (or one throttling regime) cancel the burn-in of the others.
fail-fast: false
matrix:
spec: ${{ fromJson(needs.detect-changed-specs.outputs.specs) }}
# Burn each spec with and without throttle
enable_network_throttling: [true, false]
uses: ./.github/workflows/e2e-stress-test-flake-fix.yml
secrets: inherit
with:
spec: ${{ matrix.spec }}
burn_in: ${{ needs.detect-changed-specs.outputs.burn_in }}
qa_db: true
mb_edition: ee
build_jar: true
enable_network_throttling: ${{ matrix.enable_network_throttling }}

View File

@@ -37,6 +37,20 @@ env:
HASH: ${{ github.event.pull_request.head.sha || github.sha }}-${{ github.run_attempt }}
PR_NUMBER: ${{ github.event.pull_request.number || '' }}
JOB_NAME: ${{ inputs.name }}
# ci-conductor reporting (DEV-1999). Secret holds the base origin
# (e.g. "https://conductor.<host>"); each consumer appends the path it needs
# (the reporter POSTs to "/webhooks/failed-tests"). Callers pass
# `secrets: inherit`; reporting no-ops when it's unset.
CI_CONDUCTOR_BASE_URL: ${{ secrets.CI_CONDUCTOR_BASE_URL }}
# Shared secret sent as the `x-internal-secret` header to authenticate.
CI_CONDUCTOR_WEBHOOK_SECRET: ${{ secrets.CI_CONDUCTOR_WEBHOOK_SECRET }}
REPO_ID: ${{ github.event.repository.id }}
# Commit + PR target branch under test, so conductor can tie failures to the
# right code. On PRs the ambient GITHUB_SHA is the synthetic merge commit and
# GITHUB_BASE_REF the target, so we pass the PR head sha / base ref explicitly
# (falling back to push-event values). The reporter reads these. DEV-1999.
COMMIT_SHA: ${{ github.event.pull_request.head.sha || github.sha }}
TARGET_BRANCH: ${{ github.event.pull_request.base.ref || github.ref_name }}
jobs:
e2e-tests:
@@ -45,6 +59,11 @@ jobs:
name: e2e-tests-${{ inputs.name }}-${{ inputs.edition }}
env:
MB_EDITION: ${{ inputs.edition }}
# Absolute path for the quarantine failures file (DEV-2082). The after:spec
# recorder runs in Cypress' plugins process, whose cwd differs from the
# workspace where the gate step runs; an absolute path anchored to the
# workspace makes both agree regardless of cwd.
QUARANTINE_FAILURES_FILE: ${{ github.workspace }}/target/quarantine-failures.jsonl
# Sensitive env vars (tokens, secrets) starting with `CYPRESS_` are accessed via async `cy.env()`
# Example: `cy.env(["FOO"]).then(({ FOO }) => ...)`
# For non-sensitive config, use `Cypress.expose()` in cypress config instead (synchronous)
@@ -53,7 +72,7 @@ jobs:
CYPRESS_MB_PRO_CLOUD_TOKEN: ${{ secrets.STAGING_MB_PRO_CLOUD_TOKEN }}
CYPRESS_MB_PRO_SELF_HOSTED_TOKEN: ${{ secrets.STAGING_MB_PRO_SELF_HOSTED_TOKEN }}
steps:
- uses: actions/checkout@v4
- uses: actions/checkout@v6
with:
submodules: true
token: ${{ secrets.METABASE_AUTOMATION_USER_TOKEN }}
@@ -95,7 +114,7 @@ jobs:
uses: ./.github/actions/prepare-cypress
- name: Download failed test artifacts
uses: actions/download-artifact@v4
uses: actions/download-artifact@v8
if: github.run_attempt != '1'
continue-on-error: true
id: failed-specs
@@ -112,6 +131,15 @@ jobs:
- name: Run Metabase
run: node e2e/runner/run_cypress_ci.js start
# Export this job's numeric GitHub job id as $JOB_ID for the Cypress runs
# below (ci-conductor failed-tests reporting reads it in after:spec).
# Snapshot generation is itself a Cypress run that fires after:spec, so
# this must precede it. Placed late, not after checkout: the Actions jobs
# API is eventually consistent, so the longer the job has been alive the
# more reliably the lookup resolves. JOB_ID exported here persists to the
# steps below. DEV-1999.
- uses: ./.github/actions/resolve-job-id
- name: Generate database snapshots
env:
CYPRESS_BROWSER: ${{ steps.cypress-prep.outputs.chrome-path }}
@@ -158,6 +186,19 @@ jobs:
# whether to fail or to pass the whole job, based on the quarantine list.
continue-on-error: true
- name: Rewrite JUnit hook failures
if: always()
continue-on-error: true
timeout-minutes: 1
shell: bash
run: |
if [ -d ./target/junit ]; then
echo "fix-junit-hooks: invoking bun"
bun e2e/support/fix-junit-hooks.ts ./target/junit
else
echo "No ./target/junit directory; nothing to process."
fi
- name: Upload Test Results
uses: ./.github/actions/upload-test-results
if: always()
@@ -173,8 +214,23 @@ jobs:
trunk-api-token: ${{ secrets.TRUNK_API_TOKEN }}
previous-step-outcome: ${{ inputs.specs && steps.tagged-e2e.outcome || steps.split-e2e.outcome }}
# Quarantine gate (DEV-2082). Compare this job's ultimate test failures
# (recorded by after:spec into ./target/quarantine-failures.jsonl) against
# ci-conductor's quarantine list and decide whether the job should pass:
# pass if every failure is quarantined, fail if any isn't. This mirrors
# what Trunk's analytics-uploader does for us today. Runs in dry-run mode
# for now (QUARANTINE_DRY_RUN defaults to true) — it only prints the
# verdict and never fails the job. `continue-on-error` is belt-and-braces
# while it's observational; drop both it and the dry run to enforce.
- name: Quarantine gate (dry run)
if: always()
continue-on-error: true
timeout-minutes: 2
shell: bash
run: bun e2e/support/check-quarantine.ts
- name: Upload Cypress Artifacts upon failure
uses: actions/upload-artifact@v4
uses: actions/upload-artifact@v7
if: |
!cancelled() && (steps.split-e2e.outcome == 'failure' || steps.tagged-e2e.outcome == 'failure')
with:
@@ -185,7 +241,7 @@ jobs:
if-no-files-found: ignore
- name: Upload Failed Tests
uses: actions/upload-artifact@v4
uses: actions/upload-artifact@v7
if: |
!cancelled() && (steps.split-e2e.outcome == 'failure' || steps.tagged-e2e.outcome == 'failure')
with:
@@ -197,7 +253,7 @@ jobs:
id: extract-timings
if: inputs.split_index >= 0 && github.ref == 'refs/heads/master'
continue-on-error: true
uses: actions/github-script@v7
uses: actions/github-script@v9
with:
script: |
const fs = require('fs');
@@ -231,7 +287,7 @@ jobs:
}
- name: Upload Timing Data
uses: actions/upload-artifact@v4
uses: actions/upload-artifact@v7
if: steps.extract-timings.outcome == 'success' && steps.extract-timings.outputs.has-changes == 'true'
with:
name: timing-data-${{ inputs.name }}
@@ -240,7 +296,7 @@ jobs:
- name: Publish Summary
if: failure()
uses: actions/github-script@v7
uses: actions/github-script@v9
with:
script: | #js
const {

View File

@@ -28,7 +28,7 @@ jobs:
timeout-minutes: 5
steps:
- name: Remove test results comment
uses: actions/github-script@v7
uses: actions/github-script@v9
with:
script: | # js
const { owner, repo } = context.repo;
@@ -80,7 +80,7 @@ jobs:
needs: e2e-tests
if: always() && !cancelled() && github.ref == 'refs/heads/master'
steps:
- uses: actions/checkout@v4
- uses: actions/checkout@v6
with:
sparse-checkout: |
.github
@@ -88,7 +88,7 @@ jobs:
sparse-checkout-cone-mode: false
- name: Download all timing artifacts
uses: actions/download-artifact@v4
uses: actions/download-artifact@v8
with:
pattern: timing-data-*
path: timing-files
@@ -98,7 +98,7 @@ jobs:
- name: Merge split timings
id: merge
continue-on-error: true
uses: actions/github-script@v7
uses: actions/github-script@v9
with:
script: |
const fs = require('fs');
@@ -129,7 +129,7 @@ jobs:
- name: Upload merged timing files
if: steps.merge.outcome == 'success'
uses: actions/upload-artifact@v4
uses: actions/upload-artifact@v7
with:
name: merged-e2e-timings
path: e2e/support/runTimings.json

View File

@@ -28,7 +28,7 @@ jobs:
CYPRESS_MB_PRO_SELF_HOSTED_TOKEN: ${{ secrets.STAGING_MB_PRO_SELF_HOSTED_TOKEN }}
steps:
- uses: actions/checkout@v4
- uses: actions/checkout@v6
- name: Prepare Docker containers
uses: ./.github/actions/e2e-prepare-containers
@@ -38,7 +38,7 @@ jobs:
maildev: true
openldap: false
webhook: false
snowplow: false
snowplow: true
postgres: false
mysql: false
mongo: false
@@ -52,7 +52,7 @@ jobs:
output-name: "metabase.jar"
- name: Retrieve Embedding SDK dist artifact
uses: actions/download-artifact@v4
uses: actions/download-artifact@v8
with:
name: embedding-sdk-${{ github.event.pull_request.head.sha || github.sha }}
path: ${{ github.workspace }}/resources/embedding-sdk
@@ -87,6 +87,19 @@ jobs:
CYPRESS_BROWSER: ${{ steps.cypress-prep.outputs.chrome-path }}
run: node e2e/runner/run_cypress_ci.js component
- name: Rewrite JUnit hook failures
if: always()
continue-on-error: true
timeout-minutes: 1
shell: bash
run: |
if [ -d ./target/junit ]; then
echo "fix-junit-hooks: invoking bun"
bun e2e/support/fix-junit-hooks.ts ./target/junit
else
echo "No ./target/junit directory; nothing to process."
fi
- name: Upload Test Results
uses: ./.github/actions/upload-test-results
if: always()
@@ -100,7 +113,7 @@ jobs:
previous-step-outcome: ${{ steps.run-tests.outcome }}
- name: "SDK component tests > Upload Cypress Artifacts upon failure"
uses: actions/upload-artifact@v4
uses: actions/upload-artifact@v7
if: failure()
with:
name: cypress-recording-embedding-sdk-react-${{ matrix.react-version }}
@@ -111,7 +124,7 @@ jobs:
- name: "SDK component tests > Publish Summary"
if: failure()
uses: actions/github-script@v7
uses: actions/github-script@v9
with:
script: | #js
const {
@@ -131,7 +144,7 @@ jobs:
outputs:
target-ref: ${{ steps.get-target-ref.outputs.value }}
steps:
- uses: actions/checkout@v4
- uses: actions/checkout@v6
with:
fetch-depth: 2
@@ -165,7 +178,7 @@ jobs:
steps:
- name: Check out the code from the previous commit
uses: actions/checkout@v4
uses: actions/checkout@v6
with:
ref: ${{ needs.decide-target-ref.outputs.target-ref }}
@@ -177,7 +190,7 @@ jobs:
maildev: true
openldap: false
webhook: false
snowplow: false
snowplow: true
postgres: false
mysql: false
mongo: false
@@ -212,7 +225,7 @@ jobs:
run: ./bin/build.sh
- name: Retrieve Embedding SDK dist artifact built from the current commit N
uses: actions/download-artifact@v4
uses: actions/download-artifact@v8
with:
name: embedding-sdk-${{ github.event.pull_request.head.sha || github.sha }}
path: ${{ github.workspace }}/resources/embedding-sdk
@@ -240,6 +253,19 @@ jobs:
node e2e/runner/run_cypress_ci.js component \
--expose grepTags="-@skip-backward-compatibility"
- name: Rewrite JUnit hook failures
if: always()
continue-on-error: true
timeout-minutes: 1
shell: bash
run: |
if [ -d ./target/junit ]; then
echo "fix-junit-hooks: invoking bun"
bun e2e/support/fix-junit-hooks.ts ./target/junit
else
echo "No ./target/junit directory; nothing to process."
fi
- name: Upload Test Results
uses: ./.github/actions/upload-test-results
if: always()
@@ -253,7 +279,7 @@ jobs:
previous-step-outcome: ${{ steps.run-tests.outcome }}
- name: Upload Cypress Artifacts upon failure
uses: actions/upload-artifact@v4
uses: actions/upload-artifact@v7
if: failure()
with:
name: cypress-recording-embedding-sdk-cross-version
@@ -264,7 +290,7 @@ jobs:
- name: Publish Summary
if: failure()
uses: actions/github-script@v7
uses: actions/github-script@v9
with:
script: | #js
const {

View File

@@ -28,13 +28,13 @@ jobs:
branch_name: ${{ fromJson(steps.build-sample-apps-data.outputs.result).branch_name }}
matrix: ${{ toJSON(fromJson(steps.build-sample-apps-data.outputs.result).matrix) }}
steps:
- uses: actions/checkout@v4
- uses: actions/checkout@v6
- name: Prepare release scripts
uses: ./.github/actions/prepare-release-scripts
- name: Build sample app compatibility data
uses: actions/github-script@v7
uses: actions/github-script@v9
id: build-sample-apps-data
with:
script: | # js
@@ -110,13 +110,13 @@ jobs:
actions: read
steps:
- uses: actions/checkout@v4
- uses: actions/checkout@v6
- uses: ./.github/actions/maximize-disk-space
# We have to use the latest version that does not have this issue https://github.com/docker/compose/pull/12752
- name: Set up Docker Compose
uses: docker/setup-compose-action@2fe291b7677a45ee1269ec56a42604c143505e7e # v1
uses: docker/setup-compose-action@8cccb8c14b6500aaffebff1aa49c502c34d2e5e6 # v2.1.0
with:
version: latest
@@ -136,7 +136,7 @@ jobs:
output-name: "metabase.jar"
- name: Retrieve Embedding SDK dist artifact
uses: actions/download-artifact@v4
uses: actions/download-artifact@v8
with:
name: embedding-sdk-${{ github.event.pull_request.head.sha || github.sha }}
path: ${{ github.workspace }}/resources/embedding-sdk
@@ -180,7 +180,7 @@ jobs:
run: node e2e/runner/run_cypress_ci.js ${{ matrix.test_suite }}
- name: Upload Cypress Artifacts upon failure
uses: actions/upload-artifact@v4
uses: actions/upload-artifact@v7
if: ${{ steps.check-sample-app-branch.outputs.sample_app_branch_exists == 'true' && steps.run-e2e-tests.outcome != 'success' }}
with:
name: cypress-recording-sample-app-${{ matrix.test_suite }}-${{ matrix.environment }}-latest
@@ -219,7 +219,7 @@ jobs:
actions: read
steps:
- uses: actions/checkout@v4
- uses: actions/checkout@v6
- name: Prepare Shoppy E2E environment
uses: ./.github/actions/prepare-shoppy-e2e
@@ -247,7 +247,7 @@ jobs:
run: node e2e/runner/run_cypress_ci.js shoppy-e2e
- name: Upload Cypress Artifacts upon failure
uses: actions/upload-artifact@v4
uses: actions/upload-artifact@v7
if: ${{ steps.shoppy-prep.outputs.sample_app_branch_exists == 'true' && steps.run-e2e-tests.outcome != 'success' }}
with:
name: cypress-recording-sample-app-shoppy-e2e-${{ matrix.environment }}-latest
@@ -284,7 +284,7 @@ jobs:
actions: read
steps:
- uses: actions/checkout@v4
- uses: actions/checkout@v6
- name: Prepare Shoppy E2E environment
uses: ./.github/actions/prepare-shoppy-e2e
@@ -327,7 +327,7 @@ jobs:
run: node e2e/runner/run_cypress_ci.js shoppy-e2e
- name: Upload Cypress Artifacts upon failure
uses: actions/upload-artifact@v4
uses: actions/upload-artifact@v7
if: ${{ steps.shoppy-prep.outputs.sample_app_branch_exists == 'true' && steps.run-e2e-tests.outcome != 'success' }}
with:
name: cypress-recording-shoppy-synthetic-tests-latest
@@ -371,7 +371,7 @@ jobs:
actions: read
steps:
- uses: actions/checkout@v4
- uses: actions/checkout@v6
- name: Retrieve uberjar artifact for ee
uses: ./.github/actions/fetch-artifact
@@ -382,7 +382,7 @@ jobs:
output-name: "metabase.jar"
- name: Retrieve Embedding SDK dist artifact
uses: actions/download-artifact@v4
uses: actions/download-artifact@v8
with:
name: embedding-sdk-${{ github.event.pull_request.head.sha || github.sha }}
path: ${{ github.workspace }}/resources/embedding-sdk
@@ -415,7 +415,7 @@ jobs:
run: node e2e/runner/run_cypress_ci.js ${{ matrix.test_suite }}
- name: Upload Cypress Artifacts upon failure
uses: actions/upload-artifact@v4
uses: actions/upload-artifact@v7
if: ${{ steps.run-e2e-tests.outcome != 'success' }}
with:
name: cypress-recording-host-app-${{ matrix.test_suite }}-${{ matrix.environment }}-latest

View File

@@ -15,9 +15,9 @@ jobs:
outputs:
frontend_embedding_sdk_package_sources: ${{ steps.changes.outputs.frontend_embedding_sdk_package_sources }}
steps:
- uses: actions/checkout@v4
- uses: actions/checkout@v6
- name: Test which files changed
uses: dorny/paths-filter@0bc4621a3135347011ad047f9ecf449bf72ce2bd # v3.0.0
uses: dorny/paths-filter@fbd0ab8f3e69293af611ebaee6363fc25e6d187d # v4.0.1
id: changes
with:
token: ${{ github.token }}
@@ -32,7 +32,7 @@ jobs:
runs-on: ${{ vars.DEFAULT_RUNNER_KEY }}
timeout-minutes: 5
steps:
- uses: actions/checkout@v4
- uses: actions/checkout@v6
- uses: jesusvasquez333/verify-pr-label-action@657d111bbbe13e22bbd55870f1813c699bde1401 # v1.4.0
id: verify-release-notes-labels
with:

View File

@@ -13,7 +13,7 @@ jobs:
runs-on: ${{ vars.DEFAULT_RUNNER_KEY }}
timeout-minutes: 25
steps:
- uses: actions/checkout@v4
- uses: actions/checkout@v6
- name: Prepare front-end environment
uses: ./.github/actions/prepare-frontend
- name: Prepare back-end environment
@@ -25,7 +25,7 @@ jobs:
echo "::remove-matcher owner=tsc::"
bun run build-embedding-sdk-package
- name: Prepare Embedding SDK artifact
uses: actions/upload-artifact@v4
uses: actions/upload-artifact@v7
with:
name: embedding-sdk-${{ github.event.pull_request.head.sha || github.sha }}
path: resources/embedding-sdk

View File

@@ -0,0 +1,45 @@
name: Tag Embedding SDK npm release as latest
on:
workflow_dispatch:
inputs:
version:
description: "npm version to tag as latest, e.g. 0.61.5"
type: string
required: true
workflow_call:
inputs:
version:
description: "npm version to tag as latest, e.g. 0.61.5"
type: string
required: true
concurrency:
group: npm-dist-tag-latest
cancel-in-progress: false
permissions: {}
jobs:
tag-latest:
name: Tag npm release as latest
runs-on: ${{ vars.DEFAULT_RUNNER_KEY }}
timeout-minutes: 5
env:
VERSION: ${{ inputs.version }}
steps:
- name: Setup Node.js
uses: actions/setup-node@v6.4.0
with:
node-version: 24
registry-url: https://registry.npmjs.org
- name: Verify version exists on npm
run: npm view "@metabase/embedding-sdk-react@$VERSION" version
- name: Add `latest` dist-tag to npm release
run: |
npm dist-tag add "@metabase/embedding-sdk-react@$VERSION" latest
echo "Tagged @metabase/embedding-sdk-react@$VERSION as latest" >> "$GITHUB_STEP_SUMMARY"
env:
NODE_AUTH_TOKEN: ${{ secrets.NPM_RELEASE_TOKEN }}

View File

@@ -42,11 +42,11 @@ jobs:
runs-on: ${{ vars.DEFAULT_RUNNER_KEY }}
timeout-minutes: 20
steps:
- uses: actions/checkout@v4
- uses: actions/checkout@v6
- name: Prepare front-end environment
uses: ./.github/actions/prepare-frontend
- name: Retrieve Embedding SDK dist artifact
uses: actions/download-artifact@v4
uses: actions/download-artifact@v8
with:
name: embedding-sdk-${{ github.event.pull_request.head.sha || github.sha }}
path: ${{ github.workspace }}/resources/embedding-sdk
@@ -58,11 +58,11 @@ jobs:
runs-on: ${{ vars.DEFAULT_RUNNER_KEY }}
timeout-minutes: 20
steps:
- uses: actions/checkout@v4
- uses: actions/checkout@v6
- name: Prepare front-end environment
uses: ./.github/actions/prepare-frontend
- name: Retrieve Embedding SDK dist artifact
uses: actions/download-artifact@v4
uses: actions/download-artifact@v8
with:
name: embedding-sdk-${{ github.event.pull_request.head.sha || github.sha }}
path: ${{ github.workspace }}/resources/embedding-sdk
@@ -74,13 +74,13 @@ jobs:
runs-on: ${{ vars.DEFAULT_RUNNER_KEY }}
timeout-minutes: 20
steps:
- uses: actions/checkout@v4
- uses: actions/checkout@v6
with:
ref: ${{ github.event.pull_request.head.sha }}
- name: Prepare front-end environment
uses: ./.github/actions/prepare-frontend
- name: Retrieve Embedding SDK dist artifact
uses: actions/download-artifact@v4
uses: actions/download-artifact@v8
with:
name: embedding-sdk-${{ github.event.pull_request.head.sha || github.sha }}
path: ${{ github.workspace }}/resources/embedding-sdk
@@ -136,7 +136,7 @@ jobs:
env:
BUNDLE_SIZE_THRESHOLD_PERCENT: 2
steps:
- uses: actions/checkout@v4
- uses: actions/checkout@v6
- uses: ./.github/actions/sdk-bundle-size-check
id: compare
with:
@@ -192,7 +192,7 @@ jobs:
needs: ${{ toJson(needs) }}
steps:
- name: Check embedding SDK job status
uses: actions/github-script@v7
uses: actions/github-script@v9
env:
needs: ${{ toJson(needs) }}
with:

View File

@@ -14,7 +14,7 @@ jobs:
runs-on: ${{ vars.DEFAULT_RUNNER_KEY }}
timeout-minutes: 5
steps:
- uses: actions/checkout@v4
- uses: actions/checkout@v6
- name: Get Emotion file count
id: get-emotion-file-count
run: |
@@ -44,7 +44,7 @@ jobs:
echo "EMOTION_FILES=$(($oss+$ee))" >> $GITHUB_OUTPUT
- name: Upload count to Stats
uses: actions/github-script@v7
uses: actions/github-script@v9
env:
EMOTION_FILES: ${{ steps.get-emotion-file-count.outputs.EMOTION_FILES }}
API_KEY: ${{ secrets.STATS_API_KEY }}

View File

@@ -11,15 +11,17 @@ jobs:
timeout-minutes: 10
steps:
- name: Format issue title (escape quotes)
uses: actions/github-script@v7
uses: actions/github-script@v9
id: vars
with:
script: |
core.setOutput('issue_title', ${{ toJson(github.event.issue.title) }}.replaceAll(/"/g, '\\"'));
- name: Send escalated issue ${{ github.event.issue.number }} to Slack
id: slack
uses: slackapi/slack-github-action@007b2c3c751a190b6f0f040e47ed024deaa72844 # v1.23.0
uses: slackapi/slack-github-action@45a88b9581bfab2566dc881e2cd66d334e621e2c # v3.0.3
with:
webhook: ${{ secrets.SLACK_ESCALATIONS_WEBHOOK_URL }}
webhook-type: incoming-webhook
payload: |
{
"blocks": [
@@ -32,6 +34,3 @@ jobs:
}
]
}
env:
SLACK_WEBHOOK_URL: ${{ secrets.SLACK_ESCALATIONS_WEBHOOK_URL }}
SLACK_WEBHOOK_TYPE: INCOMING_WEBHOOK

View File

@@ -38,7 +38,7 @@ jobs:
timeout-minutes: 60
name: Stress test frontend unit test flake fix
steps:
- uses: actions/checkout@v4
- uses: actions/checkout@v6
- name: Prepare front-end environment
uses: ./.github/actions/prepare-frontend
- name: Prepare back-end environment

View File

@@ -37,7 +37,7 @@ jobs:
runs-on: ${{ vars.DEFAULT_RUNNER_KEY }}
timeout-minutes: 20
steps:
- uses: actions/checkout@v4
- uses: actions/checkout@v6
- name: Prepare front-end environment
uses: ./.github/actions/prepare-frontend
- name: Prepare back-end environment
@@ -47,7 +47,7 @@ jobs:
uses: ./.github/actions/get-cache-keys
id: get-cache-keys
- name: Restore ESLint cache
uses: actions/cache/restore@v4
uses: actions/cache/restore@v5
with:
path: ${{ github.workspace}}/.eslintcache
key: ${{ steps.get-cache-keys.outputs.eslint-cache-key }}
@@ -68,7 +68,7 @@ jobs:
runs-on: ${{ vars.DEFAULT_RUNNER_KEY }}
timeout-minutes: 10
steps:
- uses: actions/checkout@v4
- uses: actions/checkout@v6
- name: Prepare front-end environment
uses: ./.github/actions/prepare-frontend
- name: Prepare back-end environment
@@ -91,7 +91,7 @@ jobs:
JEST_JUNIT_OUTPUT_DIR: ./target/junit
JEST_JUNIT_OUTPUT_NAME: test-report-frontend-unit.xml
steps:
- uses: actions/checkout@v4
- uses: actions/checkout@v6
- name: Prepare front-end environment
uses: ./.github/actions/prepare-frontend
- name: Prepare back-end environment
@@ -120,7 +120,7 @@ jobs:
runs-on: ${{ vars.DEFAULT_RUNNER_KEY }}
timeout-minutes: 20
steps:
- uses: actions/checkout@v4
- uses: actions/checkout@v6
- name: Prepare front-end environment
uses: ./.github/actions/prepare-frontend
- name: Run frontend timezones tests
@@ -136,7 +136,7 @@ jobs:
run:
working-directory: enterprise/frontend/src/custom-viz
steps:
- uses: actions/checkout@v4
- uses: actions/checkout@v6
- name: Prepare Node + Bun environment
uses: ./.github/actions/prepare-node-bun
- name: Install dependencies
@@ -164,7 +164,7 @@ jobs:
needs: ${{ toJson(needs) }}
steps:
- name: Check frontend job status
uses: actions/github-script@v7
uses: actions/github-script@v9
env:
needs: ${{ toJson(needs) }}
with:

View File

@@ -35,7 +35,7 @@ jobs:
MB_GENTEST_LIMIT_SECONDS: ${{ inputs.seconds }}
MB_GENTEST_RUN: true
steps:
- uses: actions/checkout@v4
- uses: actions/checkout@v6
with:
ref: ${{ inputs.ref }}
- name: Prepare back-end environment
@@ -47,7 +47,7 @@ jobs:
run: clojure -X:dev:ci:test:ee:ee-dev :only '${{ inputs.test }}'
- name: Publish Test Report (JUnit)
uses: dorny/test-reporter@3eeb9fc888e82e8be2fb356bbeec2750231672bc # v1
uses: dorny/test-reporter@a43b3a5f7366b97d083190328d2c652e1a8b6aa2 # v3.0.0
if: failure()
with:
path: "target/junit/**/*_test.xml"

View File

@@ -19,9 +19,9 @@ jobs:
outputs:
i18n: ${{ steps.changes.outputs.i18n }}
steps:
- uses: actions/checkout@v4
- uses: actions/checkout@v6
- name: Test which files changed
uses: dorny/paths-filter@0bc4621a3135347011ad047f9ecf449bf72ce2bd # v3.0.0
uses: dorny/paths-filter@fbd0ab8f3e69293af611ebaee6363fc25e6d187d # v4.0.1
id: changes
with:
token: ${{ github.token }}
@@ -33,7 +33,7 @@ jobs:
runs-on: ${{ vars.DEFAULT_RUNNER_KEY }}
timeout-minutes: 15
steps:
- uses: actions/checkout@v4
- uses: actions/checkout@v6
- name: Prepare front-end environment
uses: ./.github/actions/prepare-frontend
- name: Prepare back-end environment

View File

@@ -11,7 +11,7 @@ jobs:
runs-on: ${{ vars.DEFAULT_RUNNER_KEY }}
timeout-minutes: 5
steps:
- uses: actions/checkout@v4
- uses: actions/checkout@v6
- name: get JS file count
id: get-js-file-count
run: |
@@ -22,7 +22,7 @@ jobs:
echo "JS_FILES=$(($oss+$ee))" >> $GITHUB_OUTPUT
- name: Upload count to Stats
uses: actions/github-script@v7
uses: actions/github-script@v9
env:
JS_FILES: ${{ steps.get-js-file-count.outputs.JS_FILES }}
API_KEY: ${{ secrets.STATS_API_KEY }}

Some files were not shown because too many files have changed in this diff Show More