Compare commits

...

296 Commits

Author SHA1 Message Date
Nathan Voxland
5ad741c0d2 Cleaning up code 2026-04-01 13:58:39 -05:00
Nathan Voxland
69570bc65f Cleaning up code 2026-03-31 20:15:55 -05:00
Nathan Voxland
b719615aa6 Cleaning up code 2026-03-31 17:09:51 -05:00
Nathan Voxland
afcc49f6f6 Refactor MQ test init: use :mq initialization step instead of is-test? checks
Replace scattered is-test? checks at var definition time with a proper
:mq initialization step in the test startup flow. This mirrors how
production initializes MQ via MqInit in startup/run-startup-logic!.

The :mq step:
- Loads driver.init (so def-listener! namespaces are loaded)
- Calls mq.init/init-for-tests! which sets sync backends, disables
  publish buffering, and registers all listeners

Also:
- Reverts var defaults to production values (appdb backends, 100ms buffer)
- Reverts def-listener! to lazy registration (no eager call)
- Restores with-sync-mq to use fresh *listeners* atom
- Removes all [TZ-DEBUG] diagnostic traces
- :db initialization depends on :mq

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-31 16:00:26 -05:00
Nathan Voxland
d12184f4a0 Fix eager listener registration: call multimethod, not method map lookup
The eager registration code used ((methods def-listener*) channel) which
returns the method fn from the methods map but doesn't invoke it. Changed
to (def-listener* channel) which dispatches through the multimethod and
actually calls the registration function.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-31 14:08:09 -05:00
Nathan Voxland
bde8a43aeb Add tracing to :db init to verify driver.init loading
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-31 13:27:56 -05:00
Nathan Voxland
76f8cb6ce3 Don't rebind *listeners* to empty atom in with-sync-mq
with-sync-mq was binding *listeners* to (atom {}), which created a
fresh empty atom that shadowed the root atom where listeners were
eagerly registered at def-listener! time. Since eftest propagates
bindings to worker threads, parallel tests running on threads spawned
within a with-sync-mq scope would see the empty atom and fail to find
any listeners.

Fix: stop rebinding *listeners*. Listeners are eagerly registered into
the root atom at namespace load time and should remain globally visible.
with-sync-mq still binds backends and buffer settings as needed.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-31 12:55:18 -05:00
Nathan Voxland
31840a1a31 Add atom identity tracing to diagnose listener registration
Trace which *listeners* atom instance is used during registration vs
lookup to detect if a dynamic rebinding is causing listeners to be
registered in one atom but looked up in another.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-31 12:35:43 -05:00
Nathan Voxland
a5968cdf24 Load driver/init.clj during test :db initialization
The MQ listeners defined in driver_notifications.clj (e.g.
connection-pool-invalidated) are only loaded via driver/init.clj, which
is loaded via core/init.clj in production. Tests don't load core/init.clj,
so these listener namespaces are never required and the eager registration
from def-listener! never happens.

Fix: require driver.init during test :db initialization so the listener
namespaces get loaded and listeners are registered.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-31 11:37:56 -05:00
Nathan Voxland
5bcdea9b51 Eagerly register MQ listeners at def-listener! time
def-listener! now immediately registers the listener when the namespace
loads, instead of deferring to register-listeners! called from MqInit.
This ensures listeners are available in test mode where MqInit never runs
(tests use initialize-if-needed!, not startup/run-startup-logic!).

register-listener! is idempotent so the production path (MqInit calling
register-listeners!) still works fine.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-31 10:37:11 -05:00
Nathan Voxland
8eafca6c74 Add tracing to invoke-listener! to diagnose missing listener
The sync backend publish fires but listener output never appears.
Adding trace to see if listener is registered or not.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-31 09:57:50 -05:00
Nathan Voxland
6adf7f2ef8 Default MQ backends to sync in test mode
Same issue as *publish-buffer-ms*: MqInit never runs in tests, so the
alter-var-root that sets backends to sync has no effect. Set defaults
at var definition time based on config/is-test?.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-31 08:59:56 -05:00
Nathan Voxland
2ea7960d5c Default *publish-buffer-ms* to 0 in test mode
MqInit never runs in tests (tests use initialize-if-needed!, not
run-startup-logic!), so the alter-var-root in the previous fix had
no effect. Instead, set the default value at var definition time
based on config/is-test?.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-31 01:26:49 -05:00
Nathan Voxland
66af12b9cd Add more MQ tracing to diagnose CI failures
Traces at buffered-publish!, transport/publish!, sync-backend/publish!,
and MqInit to understand why *publish-buffer-ms* is still 100 in CI
(meaning messages are buffered instead of delivered immediately).

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-31 01:02:26 -05:00
Nathan Voxland
8171f24196 Debugging CI failures 2026-03-31 00:27:49 -05:00
Nathan Voxland
a86dffa344 Fix MQ topic messages silently dropped in test mode
In test mode, MqInit sets sync backends but leaves *publish-buffer-ms*
at its default of 100ms. This causes buffered-publish! to buffer messages
instead of delivering them immediately. Since start-publish-buffer-flush!
is never called in test mode, no background thread drains the buffer,
so topic messages are silently dropped.

This broke report-timezone change notifications — connection pools were
never invalidated when timezone changed, causing stale timezone sessions
on database connections. This manifested as timezone-shifted query results
across 28+ driver tests in CI.

Fix: set *publish-buffer-ms* to 0 in test init path so messages publish
immediately without buffering.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-30 22:56:59 -05:00
Nathan Voxland
db09e70823 Debugging CI failures 2026-03-30 20:27:04 -05:00
Nathan Voxland
407f3bc2df Merge branch 'master' into persistent-queue 2026-03-29 18:22:27 -05:00
Fraser
e9b56f49b5 Allow Enterprise module to import from feature modules (#71468) 2026-03-29 16:26:56 +13:00
Fraser
307b2906b7 Fix module stats upload script (#71385) 2026-03-29 16:26:43 +13:00
Nathan Voxland
5b1181eba5 Fixed module setup 2026-03-28 01:40:24 -05:00
Nathan Voxland
1574a28b4f Fixed tests 2026-03-28 01:26:44 -05:00
lbrdnk
b8d3ac0869 Adjust search schemas (#71572) 2026-03-27 16:13:42 -07:00
Alexander Polyankin
ad4dfb41b1 Fix typo in transform database validation messages (#71596) 2026-03-27 22:34:25 +00:00
Nathan Voxland
ace1e034a2 Fixed tests 2026-03-27 15:50:33 -05:00
Nathan Voxland
9142569d91 Fixed tests 2026-03-27 14:58:23 -05:00
github-automation-metabase
38e28e0ec3 Fix pagerDurty workflow edge case (#71574)
Fix PagerDuty alert broken by single quote in issue title

Use jq --arg to safely construct the JSON payload instead of
interpolating user-controlled data directly into a shell string.
This prevents shell syntax errors when issue titles contain single
quotes or other special characters.

Removes the now-unnecessary 'Setting title' step that only escaped
double-quotes but left single quotes unhandled.
2026-03-27 20:52:32 +01:00
Nathan Voxland
30c6275dc5 Fixed tests 2026-03-27 14:05:41 -05:00
Nathan Voxland
0f2e7ea906 Fixed tests 2026-03-27 13:51:36 -05:00
Nathan Voxland
cff57cbbb5 Merge remote-tracking branch 'origin/master' into persistent-queue
# Conflicts:
#	enterprise/backend/test/metabase_enterprise/agent_api/api_test.clj
2026-03-27 13:38:08 -05:00
Edward Paget
655a3f6ac7 Move agent API tests from enterprise to OSS (#71456) 2026-03-27 18:30:00 +00:00
Nathan Voxland
62c13bb028 Fixed formatting 2026-03-27 13:16:24 -05:00
Chris Truter
3c968befff Improve model encapsulation with serialization bypass (#71553) 2026-03-27 20:11:07 +02:00
Nathan Voxland
8f7064e87f Fixing tests 2026-03-27 12:56:26 -05:00
Sloan Sparger
334ed1fe42 [Metabot] Hide metabot feedback buttons on non-hosted instances (#71539)
hide metabot feedback buttons on non-hosted instances
2026-03-27 12:53:02 -05:00
Sloan Sparger
8e79105b9b [Metabot] Temporary removal of analyze chart button on dashboard page (#71540)
temp removal of analyze chart button on dashboard page
2026-03-27 12:52:53 -05:00
Ryan Laurie
92cd3f6cbf Converts some tests to ts (#71520)
converts tests to ts
2026-03-27 11:23:51 -06:00
Noah Moss
7089932b89 Move slack connect tests over to oss + fix API routes (#71525)
* move slack connect tests over to oss and remove some references to the token

* remove EE routes, always use OSS

* simplify

* undo some of that simplification

* fix tests / lint

* Move one test back to EE since it tests slack-connect's interaction with SAML
2026-03-27 10:50:48 -04:00
Nathan Voxland
27e5f298d8 Fixing tests 2026-03-27 09:27:36 -05:00
Nicolò Pretto
c4a27f18d3 embedding-hub: preserve "Select data" state + reset next steps after changing data segregation strategy (#71368)
* move up the state to persist it across steps

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: reopen create tenants step when changing segregation strategy

When the user changes the data segregation strategy after already
creating tenants, the create tenants step now correctly reopens
instead of skipping to summary.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* address feedback

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-27 15:24:32 +01:00
Phoomparin Mano
811e4466b2 Update in-app JWT onboarding to use snippets instead of iframes (#70475)
* share the same snippets for onboarding flow

* use sdk-docs-snippets namespace

* sync snippets

* fix rspack config

* rspack config: use path

* rspack: simplify resolve aliases

* remove node_modules from react refresh

* add copy button

* cleanup codemirror styling

* future proof: allow importing tsx files as well

* fix eslint pattern

* chore: move eslint ignore

* test to simplify aliases

* add context to ttag strings

---------

Co-authored-by: Nicolò Pretto <info@npretto.com>
2026-03-27 15:24:18 +01:00
Jeff Bruemmer
48c41ec6c8 Docs metabot slack (#71526) 2026-03-27 10:02:47 -04:00
Oleksandr Solovyov
b93319ab57 metabot llm calls: tools are optional/maybe (#71555) 2026-03-27 13:25:04 +00:00
lbrdnk
31905eaba4 Add 60 to version-helpers.ts (#71480)
Update versionRequirements
2026-03-27 11:02:22 +01:00
Nicolò Pretto
a0a14a42d3 scroll to top on page change in admin settings (#71369)
* scroll to top on page change in admin settings

* make tests green

* switch to useRouter instead of hoc
2026-03-27 09:55:04 +01:00
lbrdnk
839784b50c Fix transform writing tools (#71100)
* Add needed capabilities

* Partly fix transforms-write ns

* Partial fix to python transforms

* Add tool prompts

* Cleanup native transforms

* Add parameter schema descriptions

* Python schemas adjustments

* Adjust tool prompts

* Cleanup

* Tests

* Add capabilities test

* tests

* cleanup

* cleanup

* mbql5 and tests

* tests

* tests

* tests

* feature sentinel

---------

Co-authored-by: Tyler Plude <tsplude@gmail.com>
2026-03-27 08:44:57 +01:00
Nemanja Glumac
e9bc4ac447 Notify Slack on cross-version test failures (#71491) 2026-03-27 08:16:10 +01:00
Chris Truter
7180c6ae4b Optimize transform target table backfill migrations (#71487)
Better safe than sorry, given recent problems with expensive migrations.
2026-03-27 06:49:07 +02:00
Nick Fitzpatrick
a57a1c0ab8 preserve navbar status in the url update at the end of handleQBInit (#71505) 2026-03-27 09:59:12 +05:30
Nick Fitzpatrick
bee30e6423 Uxw 3355 page titles not in sync with data studio tabs (#71401)
* Fixing page titles in Data Studio

* shuffling Library files around. Adding library page title to all library subpages
2026-03-27 09:50:52 +05:30
Nick Fitzpatrick
281aa0dc12 decrease font size in click behavior sidebar (#71519) 2026-03-27 09:48:40 +05:30
Nick Fitzpatrick
14bd0853f6 set line height to 'normal' for textareas (#71498) 2026-03-27 09:48:20 +05:30
Romeo Van Snick
8be374db14 Convert query builder redux store to ts (#71063)
* Convert query_builder reducers to js

* Convert query_builder selectors to ts

* Fix types

* Fix types

* Fix types

* Fix isResultDirty selector

* Fix selector

* Move selector tests to TS

* Move reducers-typed into reducers

* Fix types

* Fix types

* Fix object id parsing

* Fix more types

* Fix PKRowIndexmap types

* Make name for EDIT_SUMMARY consistent

* Fix hanging test
2026-03-27 13:59:05 +11:00
Nathan Voxland
c6dff735a9 Fixing tests 2026-03-26 18:13:35 -05:00
Alexander Polyankin
b7e1d1eff8 Serdes for cards, segments, measures, transforms - allow deserializing only with minimal required properties without computed ones (#71464) 2026-03-26 18:51:58 -04:00
github-automation-metabase
3dd4c46871 Auto-update documentation for master (#71533)
Update generated docs (master)

Co-authored-by: github-actions <github-actions@github.com>
2026-03-26 18:49:20 -04:00
Noah Moss
e7ae8072ca Reorganization + consolidation of metabot system prompts (#71506)
* Reorganization + consolidation of metabot system prompts

* More reorganization

* More shared prompt files

* revert search changes included accidentally

* more organization

* Fix dialect resource paths in module drivers

Update llm-sql-dialect-resource defmethods to use correct metabot/prompts/dialects/ path.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* share personality prompt for nlq profile

* bump ci

---------

Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
2026-03-26 17:27:26 -04:00
Riley Thompson
483f2086dd Bump databricks jdbc driver to 3.3.1 (#71513)
* bump databricks jdbc

* trigger ci
2026-03-26 21:17:38 +00:00
Edward Paget
cb9c042404 Fix MCP tool dispatch to pass query params correctly per Ring spec (#71502)
invoke-agent-api-with-params was encoding GET/DELETE query params into
the URI string, but defendpoint reads from :query-params (a parsed
map) on the Ring request. The synthetic request never passes through
wrap-params middleware, so query params in the URI were silently
ignored.

Fix by moving the method-based dispatch into deliver-agent-api-response:
POST params go to :body, GET/DELETE params go to :query-params.
Remove invoke-agent-api-with-params which is no longer needed.
2026-03-26 15:49:09 -05:00
Anderson
1b26236cf1 Do not show JWT shared secret as plaintext after it is saved (#71165)
* Handle JWT shared secret as write-only after initial generation (#UXW-3300)

- Thread generated token through wizard state so AddEndpointStep displays
  the plaintext key without re-reading it from session properties
- FormSecretKey now recognises the backend obfuscated format (**********XX)
  and renders an empty input with 'Already set' placeholder and a
  'Regenerate key' button, rather than showing the masked value

* Add unit tests for FormSecretKey obfuscated value handling (#UXW-3300)

- Tests covering no-value, plaintext, and obfuscated backend secret states
- Verifies 'Already set' placeholder and button labels are correct per state
- Verifies confirmation modal uses caller-supplied title/body
- Verifies token regeneration clears obfuscated value

* Fix prettier formatting in FormSecretKey.tsx

* Update the way we obfuscate the key and handle it on the frontend

* Add be test, improve alert

* Fix ts

* discard unnecessary changes

* Remove section that displayed the JWT secret in embedding set up

* Fix e2e test

* Show dedicated modal when generating new JWT shared key

* Fix unit test

* Fix e2e tests

* Fix latest UI changes

* Address review comments

* Add loading state to secret key modal and assert obfuscated key in E2E test

* Read jwt-shared-secret from /api/setting instead of session-properties / Revert unneeded obfuscated? implementation

* Fix tests

* Replace reads of session properties with api settings to simplify component logic
2026-03-26 16:34:54 -03:00
Chris Truter
14ff22021e Support non-datetime migration ids in tests (#71471)
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-26 21:03:02 +02:00
Nick Fitzpatrick
55df67b632 Update Value normalization / collapsing in ChartSettingInputNumeric.tsx (#71440) 2026-03-27 00:14:41 +05:30
Edward Paget
53b678bcd6 Flatten root-level :or/:and/:multi into flat object schemas for MCP clients (#71494)
* Flatten root-level :or/:and/:multi into flat object schemas for MCP clients

Only the outermost composite schemas are flattened — nested anyOf/allOf
within properties are preserved since LLM clients handle them fine.

- :and → :merge then deref (flattens the [:map {:encode/...}] pattern)
- :or → :union with optional-keys, then deref (merges branches)
- :multi → same as :or, extracts schemas from dispatch triples

Also updates construct_query and query tool descriptions to explain
which fields are mutually exclusive (table_id vs metric_id vs
continuation_token).

* Refactor flatten-root-schema to worklist with cycle detection

Replace the recursive flatten-root-schema with a two-phase approach:
collect-root-maps walks the root composite tree using an iterative
worklist, then flatten-root-schema merges the collected leaves.

This eliminates stack recursion (:or/:multi no longer call
flatten-root-schema on children) and replaces the fuel counter with
a seen set that precisely detects deref cycles. On cycle detection,
emit an empty :map (-> {"type":"object"}) as a safe fallback.

The worklist also fixes :and wrapping :or — both layers now flatten
correctly since all composite nodes are expanded uniformly.

Adds tests for deeply nested composites, nested composites within
properties (preserved as anyOf/allOf), and :multi root flattening.
2026-03-26 17:48:52 +00:00
Nathan Voxland
daaca9ef62 CI fixes 2026-03-26 12:30:15 -05:00
Cam Saul
6a15fee1c5 Test to prove #32068 is fixed 🔧 (#71382)
* Test to prove #32068 is fixed 🔧

* Whoops revert deliberate test breakage

* Simplify test
2026-03-26 16:43:44 +00:00
Nicolò Pretto
92c07ddea1 hide sample db from embedding onboarding picker (#71304)
* hide sample db from embedding onboarding picker

* simplify e2e

* let's see if ci is green

* remove useless line, this is always pg
2026-03-26 17:23:19 +01:00
Sébastien
3dd99b6494 Fixed flaky custom elements api <metabase-question> test (#71500) 2026-03-26 17:20:53 +01:00
Riley Thompson
edb5eaad63 Handle non-dates in sqlite date columns (#71492)
* fix sqlite non date handling

* kondo
2026-03-26 12:14:00 -04:00
Nathan Voxland
ad9b980fef Merge remote-tracking branch 'origin/master' into persistent-queue 2026-03-26 11:12:10 -05:00
Nathan Voxland
96a308a64e CI fixes 2026-03-26 11:11:47 -05:00
Chris Truter
77afdaafc0 MCP further nits (#71478) 2026-03-26 16:08:18 +00:00
metamben
56f04b92ca Render |> as a single operator when formatting SQL (#71435)
Closes #70447
2026-03-26 16:06:37 +00:00
Braden Shepherdson
8020b65546 [MBQL lib] Add user docs for joins to lib.core (#71442)
Part of QUE2-366.
2026-03-26 16:06:19 +00:00
Sébastien
0cc35ac876 Sprinkled upsell gems in embed wizard (#71076) 2026-03-26 16:59:33 +01:00
Kevin Kelleher
617ca0f5f0 Fix boolean cartesian charts defaulting to timeseries axis (#71486) 2026-03-26 11:13:37 -04:00
John Swanson
607fb9ba58 Hide destination DBs from permissions (#71395)
* FE: do not show destination DBs in permissions editor

If you add a destination database, don't show it in the perms editor.

This was a temporary bug (disappears on refresh) BUT if you edit permissions
for the destination database then that is persisted, which is not great. I
will fix that in a separate PR.

* Don't add data permissions rows for destination DBs

- Exclude destination databases from the permissions graph

- Don't add permissions for destination DBs when creating a new group
2026-03-26 06:46:59 -08:00
Nathan Voxland
3c24edbf47 Merge remote-tracking branch 'origin/master' into persistent-queue
# Conflicts:
#	src/metabase/settings/models/setting.clj
2026-03-26 09:33:55 -05:00
Ngoc Khuat
74c8811981 Fix silent data loss when importing unparseable YAML files (#71421)
* Surface ingestion errors instead of silently skipping unparseable YAML files

The serdes ingestion layer was silently dropping YAML files that failed
to parse (introduced in #41887 as defensive coding for editor temp
files). This caused remote sync imports to silently lose entities when
YAML files were malformed or exceeded SnakeYAML's nesting depth limit,
with no error reported to the user.

Add `ingestion-errors` to the Ingestable protocol so the ingestion
layer collects parse failures and surfaces them through `load-metabase\!`.
When `continue-on-error` is false (default, used by remote sync), the
import fails immediately on ingestion errors. When true, errors are
collected and reported alongside load-phase errors.

The dotfile filter from #41567 is preserved — editor temp files are
still silently skipped before parsing is attempted.

Fixes #71257

* Rename ingestion-errors to ingest-errors, extract populate-cache\! helper, add e2e tests

- Rename protocol method to ingest-errors for consistency with ingest-list/ingest-one
- Extract populate-cache\! helper to DRY up duplicated cache-population logic
- Add e2e test for continue-on-error true/false with ingestion errors

* Add tests for ingest-errors on IngestableSnapshot and wrapper delegation

* Fix log/warnf misuse, tighten tests, remove stale comment

- log/warnf -> log/warn to prevent MissingFormatArgumentException
  when error messages contain % characters
- Add tests for ingest-errors ordering contract and multi-file corruption
- Tighten continue-on-error assertion to check exact error count
- Remove stale comment describing pre-fix behavior

* Include file names in ingestion error messages

Wrap parse/read exceptions with ex-info containing the file path,
so the error message shows which files failed, e.g.
"Failed to read 1 file(s) during ingestion: bad_card.yaml"
2026-03-26 23:29:58 +09:00
Oleksandr Solovyov
ce15b2d9f0 fix metabot sql tools incorrectly checked query type (#71428) 2026-03-26 14:06:13 +00:00
github-automation-metabase
e5db491df4 Auto-update documentation for master (#71454)
Update generated docs (master)

Co-authored-by: github-actions <github-actions@github.com>
2026-03-26 09:54:44 -04:00
Chris Truter
fdffe7dc82 Add db fallback for renamed settings (#71470) 2026-03-26 15:42:32 +02:00
lbrdnk
3bd3bf5b6a Edit chart fix (#71424)
* Add edit_chart.md instructions

* Ensure ids of state elements are stringified

* Cleanup

* Test

* add test and do cleanup

* Review comments
2026-03-26 14:00:04 +01:00
Chris Truter
c940bb3935 Add FKs from transforms back to tables (#70617) 2026-03-26 13:40:22 +02:00
Timofey Kachalov
366c74c02e Support entityId for guest embedded questions (#71246) 2026-03-26 14:44:00 +04:00
Chris Truter
0303419907 Increase transform test timeout (#71453) 2026-03-26 12:26:05 +02:00
Sébastien
1511899a7e Implement proper column width measurement for custom font sizes (#71407) 2026-03-26 11:08:04 +01:00
lbrdnk
b1a2673e1a Inject context in user message (#71045)
* Add context injection to user message

* Remove context from internal template

* whitespace

* Adjust tests

* Apply suggestion from @appleby

Co-authored-by: appleby <86076+appleby@users.noreply.github.com>

* use lib

---------

Co-authored-by: appleby <86076+appleby@users.noreply.github.com>
2026-03-26 10:42:20 +01:00
Nicolò Pretto
7074e0236a fix frontmatter in add-tracing skill (#71438) 2026-03-26 09:38:16 +01:00
Nemanja Glumac
ddba65c368 Add E2E (questions) tests on top of the cross-version automation (#70190) 2026-03-26 09:13:57 +01:00
Nathan Voxland
93b375bbe2 Merged from master 2026-03-26 01:33:25 -05:00
Edward Paget
d8bb291939 Inline malli refs and flatten root anyOf in MCP tool schemas (#71457)
* Inline malli refs and flatten root anyOf in MCP tool schemas

MCP clients (Claude API, OpenAI, Gemini, AWS Bedrock) reject or
mishandle $ref and top-level anyOf in tool input schemas. Fix by:

- Walking malli schemas with ::walk-refs and ::walk-schema-refs to
  replace all registered-schema refs with their dereferenced content
  before JSON Schema generation, so mjs/transform never emits $ref
- Flattening root-level anyOf into a merged object schema for
  construct_query (table vs metric request union)
- Expanding construct_query tool description to explain table vs
  metric query differences since the schema no longer structurally
  separates them

Removes all $ref/$defs sanitization code and the defs accumulator
atom that are no longer needed.

* Replace :or/:and with :merge at malli level to eliminate anyOf/allOf

Instead of post-processing JSON Schema to flatten anyOf/allOf, handle
it at the malli schema level: in inline-malli-refs, replace :and with
:merge (eliminates allOf from the encode-map pattern) and :or with
:merge after making all entries optional via mut/optional-keys
(eliminates anyOf, all properties become optional in the merged
output).

This produces clean flat {type: "object", properties: {...}} schemas
that all MCP clients can handle. The construct_query tool now has all
table and metric properties in a single flat schema with none
required, and the description explains which to use.

Removes flatten-any-of and clojure.set dependency.
2026-03-26 06:21:33 +00:00
Nathan Voxland
ca3a2ea136 Merge remote-tracking branch 'origin/master' into persistent-queue
# Conflicts:
#	src/metabase/search/core.clj
#	src/metabase/search/ingestion.clj
#	src/metabase/search/task/search_index.clj
#	src/metabase/testing_api/api.clj
2026-03-25 23:49:31 -05:00
Aleksandr Lesnenko
1132bfd402 New Metric Home page (#71055) 2026-03-26 03:05:51 +00:00
Nathan Voxland
e9f6c93013 Improved performance benchmark and added back skip lock 2026-03-25 21:17:35 -05:00
Maksym Yakubych
37f3a01838 [Metabot] Bring your own keys (#70714)
* move ee/llm into oss

* lots of file movement

* move slack connect to opensource

* ability to have enterprise-only tools

* move tests

* Fix stale metabot-v3 aliases, dangling module refs, disabled linting, broken test paths, and typo

* start using deprecated-name, more ns path fixes

* move llm transforms code to oss

* fix namespaces

* whitespace fix

* checkpoint: switch tools to use mu/defn or defenterprise directly

* compilation fix

* lint fixes

* move around some code to break drivers dependency

* fix tests

* fix tool calls

* sort routes

* fix openapi

* fix module config

* lint

* fix clj-kondo

* fix test

* fix token check tests

* cljfmt

* fix slack test

* maybe fix dashboard tests?

* cljfmt

* is this a fix for dashboard tests

* move metabot models to oss

* untangle module config after rebase

* fix module order

* cleanup

* add back ai service-related settings that are still used

* move some more code to more appropriate places

* more code movement

* better schema names

* clean up mu/defn function name formatting

* update module config

* remove unneeded ai service-related test

* fix module config again

* fix auth in tests by removing more ai service stuff

* enable native agent by default

* test transforms over normal function calls

* requires

* pass through metabot-id to tool calls instead of hardcoding embedded-metabot-id, to fix answer sources test

* fix wrap-tools-with-state calls

* try to fix failing field/dashboard tests by matching the code on master (might have been from a bad rebase)

* Revert "try to fix failing field/dashboard tests by matching the code on master (might have been from a bad rebase)"

This reverts commit cd5ec8b487.

* hard revert parameters.dashboard-test to the version on master after the latest rebase

* mark snippet tests as sequential to try to fix perms test issues, remove a couple stray metabot feature checks

* move entity analysis + sql fixing code to oss

* wrap dashboard tests with with-full-data-perms-for-all-users to isolate the failure

* move ai-entity-analysis and ai-sql-fixer under the metabot module

* fix module test

* remove now-unneeded ai feature falgs

* disable perms cache for these failing dashboard tests

* feat: self-hosted metabot provider settings

* test: patch is-hosted temporarily

* fix: grouping

* move files to the frontend/ folder

* move MetabotProviderSection into MetabotSetup to accomidate design changes

* make anthropic the only allowed provider, updat sonnet 4.6 as default model, small layout change to api key input

* remove PLUGIN_METABOT usage, remove purchase pages, consolidate new connections settings ui

* remove oss impl of sql gen on the FE

* remove MetabotSettingUpModal, add alert for slackbot requiring metabot be setup, remove PLUGIN_METABOT, PLUGIN_METABOT.isCloudManaged checks now check llm-metabot-configured? which breaks things for enterprise for now

* add mock Metabase provider

* fix build

* impl copy feedback

* type-fix

* clean up

* fix unit tests

* fix metabot e2e tests

* type fix

* more test fixes

* fix semantic search tests

* more test fix

* kondo fix

* more test fixes

* copy change

* more kondo fixes

* app db test failure fixes

* fix failing BE tests

* fix e2e tests

* more e2e failure fixes

* fix typo

* remove mocked out endpoint definition

* fix last failing e2e test

---------

Co-authored-by: Oleksandr Solovyov <o@solovyov.net>
Co-authored-by: Noah Moss <noahbmoss@gmail.com>
Co-authored-by: Sloan Sparger <sloansparger@gmail.com>
2026-03-25 23:49:07 +00:00
Chris Truter
87dc660191 Minor improvements to MCP server (#71406)
* Fix token scope logic to match middleware

* Add :secure to CSRF and DRY up

* Add comment to magic number for clarity

* Clean up complex threading

* Don't clobber token-scopes

* Optimize and improve no user-id failure mode

* trim comment fluff

* Simplify and avoid double str call

* Use database time for revoke-token for robustness

* Use long for user-id actually

* Enable refresh token rotation

A stolen refresh token was reusable for its entire 30-day TTL with no
way to detect theft. The oidc-provider library supports rotation — on
each refresh grant the old token is revoked and a new one issued.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* Sanitize brand-color on consent page to prevent CSS injection

The brand-color setting is interpolated via h/raw into a <style> block on
the consent page. A non-hex value could break out of the CSS rule.
Validate defensively at the rendering site rather than changing the setting
setter, to handle already-persisted invalid values gracefully.

* HMAC-bind consent params to CSRF token and sanitize error messages

Two security improvements to the OAuth server:

1. The authorization decision endpoint re-parsed OAuth parameters from
   the POST body without verifying they matched what the user saw on
   the consent page. A modified form could request broader scopes than
   shown. Now the authorize endpoint HMAC-signs the parsed params using
   the CSRF token as key, and the decision endpoint verifies the
   signature against the re-parsed params before proceeding.

2. Error catch blocks on /authorize, /authorize/decision, and /token
   returned raw exception messages that could leak internal details
   (client IDs, redirect URIs, expected vs actual values). Now returns
   generic RFC-compliant error descriptions and logs details server-side.

* Truncate long client name. Show client id

* Non-functional nits

* Clean up

* Align and extract helper

* DRY OCD

---------

Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-25 18:04:40 -04:00
Chris Truter
40691f70d8 Faster search index sync when loading snapshots (#71410) 2026-03-25 21:42:40 +00:00
Noah Moss
f4b0e5946b Consolidate metabot-enabled checks and add them to a number of endpoints (#71366)
* consolidate metabot-enabled checks and add them to a number of endpoints

* switch describe_tests to use mbql-query helper for cards

* Remove metabot.settings namespace import
2026-03-25 21:17:14 +00:00
Noah Moss
f3a2cfab7f Query pagination for MCP (#70971)
* Combined /v1/query endpoint with pagination and continuation tokens

- Add /v1/query that combines query construction and execution in one step
- Implement continuation_token-based pagination for large result sets
- Cap agent query rows at 200 per page; pagination stops at userland limits (2k/10k)
- Refactor construct-query into shared construct-query* helper
- Move agent-api dispatch behind defenterprise seam in EE MCP tools
- Add streaming JSON-RPC support for tool results in MCP layer
- Add json-string-escaping OutputStream for embedding streamed JSON in JSON-RPC
- Move /agent route out of naughty-routes into intentional top-level-routes
- Fix SSO test_setup to handle nil server port in CLI test runner

* Remove streaming JSON-RPC, buffer all tool responses in memory

Strip the streaming code path from the MCP layer so this branch
focuses purely on the query tool and pagination. Buffering ~120KB
for 200 rows is fine for the limits we enforce.

- Remove json-string-escaping OutputStream and streaming-tool-response
- Revert MCP api.clj to synchronous Ring form (no SSE streaming)
- Move StreamingResponse materialization into EE MCP tools layer
- Simplify invoke-agent-api to handle only plain map responses
- Remove streaming-specific tests (escaping, SSE streaming, error)

* a bunch of other changes to try get the rebased build working

* Roll back a bunch of fixes from the oss branch that don't need to be fixed here

* restore code accidentally lost via multiple rebases

* update modules + lint

* strip out dead ee code

* update tool  count test
2026-03-25 16:47:10 -04:00
Nathan Voxland
817eb1d787 Roll back transform perms (#71094) 2026-03-25 15:27:25 -05:00
Nathan Voxland
89786605c1 Fixing tests and created a mq-perf script 2026-03-25 15:19:29 -05:00
Egor Grushin
8b5224136b Cypress bail (#71420)
* add bail feature to the e2e stress test flake workflow

* add bail feature to the e2e stress test flake workflow

* add bail feature to the e2e stress test flake workflow

* Update e2e/support/cypress.js

Co-authored-by: Nemanja Glumac <31325167+nemanjaglumac@users.noreply.github.com>

* renamed BAIL to FAIL_FAST, replace Cypress.env with Cypress.expose

---------

Co-authored-by: Nemanja Glumac <31325167+nemanjaglumac@users.noreply.github.com>
2026-03-25 20:11:22 +00:00
Braden Shepherdson
051c1a6b25 [Params] Always return [] for param /values requests (#71302)
Fixes #71164.

On the BE, always return an empty list rather than nil. Old models with `:id` different from `:field_ref` in their metadata could result in `{:values nil ...}` being returned, which breaks the schema and FE. See #71164 for full details.
2026-03-25 20:11:07 +00:00
Jeff Bruemmer
c3c7feeddf docs - OIDC (#70892) 2026-03-25 13:54:47 -04:00
appleby
274b7de269 Remove dangling reference to use-native-agent setting in e2e tests (#71437)
This defsettings was removed in 311b735ae5
2026-03-25 16:48:29 +00:00
Stas Gavrylov
a0bea40dab GDGT-1755: Migrate common components from deprecated Popover to Mantine Popover (#70503)
* GDGT-1755: Migrate RightClickPopover to use Mantine

* GDGT-1755: Migrate DropdownSidebarFilter to Mantine Popover

* GDGT-1755: Migrate ParameterTargetWidget to Mantine Popover

* GDGT-1755: Make DropdownSidebarFilter close on Esc press

* GDGT-1755: Migrate GroupSelect to Mantine Popover

* GDGT-1755: Migrate deprecated Select to Mantine popover to remove deprecated Popover

* GDGT-1755: Cleanup. Delete unused deprecated components

* Fix prettier

* fix: fix test failures after Popover cleanup

- Remove jest.unmock for deleted Popover module in ChartSkeleton tests
- Add missing data-testid="popover" to DropdownSidebarFilter dropdown

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* GDGT-1755: Cleanup

* GDGT-1755: Address code review comments

* Clean up Popover.module.css: keep only tippy.js global styles

Remove unused module-level classes and obsolete tether-related classes.
Import the remaining global styles in TippyPopover.tsx.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
2026-03-25 18:33:12 +02:00
Nicolò Pretto
2576505465 fix flaky test in embedding hub (#71434) 2026-03-25 17:04:32 +01:00
Nick Fitzpatrick
9f5298fa9a Update TablePicker when users publish/unpublish tables, change name (#70954)
* update TablePicker when users publish/unpublish tables, change name

* refetch tables in useTableLoader

* update table owners
2026-03-25 21:27:23 +05:30
Kevin Kelleher
07f25c0cc7 Allow changing conditional formatting to color range when rule condition is set (#71079) 2026-03-25 10:59:01 -04:00
Nicolò Pretto
cfb3ab406c Implement SSO setup checklist locking for steps 2 and 3 until JWT is configured (#71345)
* Implement SSO setup checklist locking for steps 2 and 3 until JWT is configured

* remove e2e for something already covered by unit

* update checklist default values after merging master
2026-03-25 15:52:01 +01:00
Egor Grushin
a4db64b01b GDGT-2017 write watermark to the DB before setting "succeed" status t… (#71334) 2026-03-25 14:21:55 +00:00
Noah Moss
efcefcad96 Avoid sending only whitespace in a slack API request (#71391)
Don't send only whitespace to the Slack API
2026-03-25 14:09:29 +00:00
Braden Shepherdson
38b071e39d [MBQL lib] Add user docs for filter functions in lib.core (#71035)
Part of QUE2-366.
2026-03-25 14:04:28 +00:00
Braden Shepherdson
46902911c8 [QP] Change the nested query alias from source to __mb_source (#70955)
closes #70224
Fixes QUE2-396.

### Description

This is necessary because `source` is a reasonably common name for a
column, and databases diverge on how they treat a reference like
`"source"."source"`:

- Postgres et al see the relation `source` in scope, and access its
  column called `source`, and all is well.
- Athena et al:
  - See that there is a **column** `source` in scope that doesn't require a prefix.
  - Assume it's a nested column with `ROW` type, and try to access the
    nested `source` field.
  - Type mismatch! It's a string or whatever, not `ROW`.

This PR doesn't *eliminate* the problem, but there's not really anything
we can do to fix Athena's honestly pretty odd choice of how to resolve this
ambiguous parse tree. But by changing our hard-coded alias for all
nested queries from `source` to `__mb_source`, we can make it extremely
unlikely that it will collide with a column that's in scope.
2026-03-25 13:53:00 +00:00
Noah Moss
23f686e698 Include conversation history in slackbot feedback (#71359) 2026-03-25 09:47:42 -04:00
Oleksandr Solovyov
311b735ae5 clean up references to ai-service (#71353) 2026-03-25 15:33:26 +02:00
Ngoc Khuat
e2f1a1a315 Fix flaky schema-change integration test (follow-up to #70863) (#71339) 2026-03-25 21:51:33 +09:00
Alex Yarosh
1f7e4b12f7 docs: update embedding skills (#71415)
Remove development note from SSO for embeds section
2026-03-25 08:30:39 -04:00
Sébastien
178ecbaeef fixed flaky (#71405) 2026-03-25 13:22:25 +01:00
Nick Fitzpatrick
5ac346612f Small positioning adjustments to the data studio nav bar (#71379)
* Small positioning adjustments to the data studio nav bar

* pr feedback
2026-03-25 17:36:41 +05:30
Chris Truter
2ace0b69c1 Built-in MCP server (#70787)
* Add MCP server with agent API integration

Implements an MCP (Model Context Protocol) server for Metabase that exposes
agent API endpoints as MCP tools. Includes an Oauth server, SSE transport, 
session management, scope-based auth, and tool dispatch via defendpoint metadata.

Co-Authored-By: Edward Paget <edpaget@users.noreply.github.com>
Co-Authored-By: Noah Moss <32746338+noahmoss@users.noreply.github.com>
2026-03-25 05:04:46 -04:00
Nick Fitzpatrick
6b8d7846de Update response on set token with network error (#70959)
* Update response on set token with network error

* report error message from checker when setting token

Currently when anything goes wrong during token checking while we're
setting a token, we *always* report that the token is invalid. This is
incorrect.

The token checker already gives us a descriptive `status` for the token,
which can tell the user whether the token is invalid or if we just
weren't able to check it.

One slight change here is to add `:canonical? true` to responses we
receive from MetaStore directly. This allows us to respond with the
correct HTTP status, since if we receive a response from the MetaStore
that tells us the token is invalid, that's a 400 error, where if we
weren't able to get a canonical response at all, that's a 503.

* add unit test

---------

Co-authored-by: John Swanson <john.swanson@metabase.com>
2026-03-25 14:19:32 +05:30
Nathan Voxland
82f5dea54d Code cleanup 2026-03-25 02:28:42 -05:00
Chris Truter
d34d1dc9ff Fix search related testing race conditions (#71290)
Closes [BOT-1105: Search API tests fail with missing temp table](https://linear.app/metabase/issue/BOT-1105/search-api-tests-fail-with-missing-temp-table)
2026-03-25 09:10:23 +02:00
appleby
95f85256c2 BOT-1172 Fix markdown link buffering for slack links (#71388)
Fix markdown link buffering for slack links

find-potential-slack-link-start was looking for a full `<metabase://` prefix which doesn't work if the link prefix is
split across streamed text chunks. Updated it to deal with this case.
2026-03-25 00:37:50 +00:00
appleby
33661d4b6a BOT-1044 Convert links back to metabase:// format before sending to llm (#71125)
* BOT-1044 Convert links back to metabase:// format before sending to llm

This is a re-implementation of https://github.com/metabase/ai-service/pull/597

resolve-links converts metabase:// links into real URLs before sending them back to the client. When the LLM sees
these resolved URLs in the conversation history, it can get confused and start generating resolved links
itself (e.g. /question#<base64>), rather than following instructions to always use metabase:// links. This sometimes
results in wrong/hallucinated base64 repr, especially in longer conversations where many resolved links accumulate in
the history.

With this commit, we add a link-registry to the agent state to record every link rewritten by resolve-links, and use
invert-links to restore them to metabase:// URIs in the message history before sending it to the LLM for completion.

* Simplify state management for link-registry-atom

We don't need to track it separately in the agent state in init-agent. Just extract it from the agent memory in
the loop-step and sync it back after the llm call.

* Only invert urls in markdown links

* Replace handcrafted legacy MBQL in markdown_link_buffer_test.clj

* Mark deftests ^:parallel in markdown_link_buffer_test.clj

* Mark deftests ^:parallel in links_test.clj and breakup into smaller deftests

* Replace handcrafted legacy MBQL in links_test.clj

* Move resolve-slack-links into metabot.agent.links ns

To match existing resolve-links handling for generic metabase links

* Use with-temporary-setting-values to override site-url in markdown_link_buffer_test.clj

To match existing conventions in the codebase.

* Consolidate with-temporary-setting-values and breakup deftests in markdown_link_buffer_test.clj

* Record slack links into the registry map and invert them before sending history back to the llm

See f90619f5e2

* Fix agent memory initialize-test

* review feedback: extract part->invert-links-key helper
2026-03-24 23:41:03 +00:00
github-automation-metabase
d39d8c5800 Auto-update documentation for master (#71389)
Update generated docs (master)

Co-authored-by: github-actions <github-actions@github.com>
2026-03-24 18:06:41 -04:00
Ryan Laurie
60da7517d1 Allow toggling auto releases via variable (#71313)
allow toggling auto releases via variable
2026-03-24 15:58:32 -06:00
appleby
ea43a75160 Fix metabot snowplow test flakes (#71337)
Fix snowplow test flakes

In call-llm-snowplow-test and call-llm-structured-snowplow-test filter the snowplow events to remove potential
new_instance_created events.
2026-03-24 17:23:04 -04:00
Noah Moss
77f28324a0 Add saved question display type to metabot context (#71372)
add saved question display type to metabot context
2026-03-24 20:14:19 +00:00
metamben
778dcf54a1 Introduce config option to specify known hosts for SSH tunnels (#71194)
* Introduce config option to specify known hosts for SSH tunnels

Closes QUE2-424
2026-03-24 22:44:56 +03:00
Alex Yarosh
161ebe81d5 docs: tenant example (#71365)
* Refine tenant attributes and permissions documentation

* Update docs/embedding/tenants.md

Co-authored-by: Jeff Bruemmer <jeff.bruemmer@gmail.com>

* Update docs/embedding/tenants.md

Co-authored-by: Jeff Bruemmer <jeff.bruemmer@gmail.com>

---------

Co-authored-by: Jeff Bruemmer <jeff.bruemmer@gmail.com>
2026-03-24 15:38:54 -04:00
Jeff Bruemmer
1dbfa65885 docs - agent API oss (#71380)
* oss and session

* link

* update
2026-03-24 14:51:59 -04:00
Cam Saul
8cb8e93394 Update GH teams config + modules config for Querying Platform (#71182) 2026-03-24 18:25:45 +00:00
Cam Saul
afc8eb428e Fix Snowflake Netty code scanning alert (#71309)
* Fix Snowflake Netty code scanning alert

* Dummy commit to trigger CI
2026-03-24 18:18:05 +00:00
Anderson
1fbe726c08 Fix ARIA grid role hierarchy in DataGrid for accessibility compliance (#70978)
* Fix aria-expanded on dashboard filter trigger button (#70543)

Closes https://github.com/metabase/metabase/issues/70543

### Description

Move aria-expanded from a wrapper div to the actual trigger button element
for dashboard filter popovers. Mantine's Popover.Target injects
aria-expanded via cloneElement onto its direct child. Previously, a Box
wrapper div was the direct child, so aria-expanded landed on a
non-interactive div instead of the button. This restructures the
component hierarchy so ParameterValueWidgetTrigger (which renders as a
button) is the direct child of Popover.Target, and moves the Sortable
wrapper outside the Popover.

### How to verify

1. Open a dashboard with filter parameters
2. Inspect the filter trigger button element in the DOM
3. Verify aria-expanded is on the button element (not a wrapper div)
4. Click the filter to open the popover and confirm aria-expanded
   toggles to true on the button

### Checklist

- [x] Tests have been added/updated to cover changes in this PR

* Fix ARIA grid role hierarchy in DataGrid for accessibility compliance (#70547)

Closes https://github.com/metabase/metabase/issues/70547

### Description

The DataGrid component had an invalid ARIA role hierarchy: `role="button"`
elements (from dnd-kit's useSortable) were nested directly inside a
`role="grid"` element without proper intermediate roles. Per the WAI-ARIA
grid pattern, the hierarchy must be: grid > rowgroup? > row > gridcell/columnheader.

This commit:
- Adds `role="rowgroup"` to header and body container divs
- Adds `role="row"` to the header row div
- Adds `role="columnheader"` to header cell wrapper divs in DataGrid
- Adds `role="gridcell"` to body cell wrapper divs in DataGrid
- Strips the invalid `role="button"` that dnd-kit injects into SortableHeader
- Removes now-redundant role attributes from HeaderCellWrapper and BodyCell
  (roles are now correctly placed on their parent wrapper elements)

### How to verify

1. Open any dashboard with a table/grid visualization
2. Inspect the DOM of the table-scroll-container element
3. Verify the ARIA role hierarchy follows: grid > rowgroup > row > columnheader/gridcell
4. Run an accessibility audit tool and confirm no `role="button"` violations inside the grid

* Keep button role in sortable header (it's ok since it's not wrapping the columnheader)

* Clean up

* Add unit test

* Fix a few tests

* Fix more e2e tests broken by ARIA role changes

* Fix more e2e tests

* Fix embedding-native.cy

* Another e2e test fix batch

* Fix e2e tests (again)

* fix reproductions-3.cy.spec.js
2026-03-24 15:05:33 -03:00
Jeff Bruemmer
2aec3aa887 Update agent API reference documentation (#71360)
* Update agent API reference documentation

Removed requirement for Metabase Pro/Enterprise from documentation.

* path
2026-03-24 17:36:43 +00:00
Nicolò Pretto
446de63944 feat(embedding-hub): add "copy dashboard to shared tenant collection" step (#70726)
* wip

* wip3

* test ci

* update tests

* remove code from tests

* sort xray dashboards

* cleanup

* clean up tests

* small refactor

* more cleaning up

* keep step disabled if  tenants is not active
2026-03-24 16:18:41 +01:00
Oleksandr Solovyov
08717ff883 closing-connection-native-agent-test: lower amount of chunks we check for (#71354) 2026-03-24 16:38:06 +02:00
Chris Truter
ccef0576f9 Calculate init time reliably (#71356) 2026-03-24 14:16:25 +00:00
github-automation-metabase
1b12502ec3 Revive and merge PR #67694: disable sync_schema side-effect in db-operations-test (#71201)
Fix H2 lock timeout in db-operations-test by mocking sync task

Split the monolithic db-operations-test into three focused tests:
- non-admin-sync-schema-test: mocks quick-task/submit-task! to prevent
  the actual sync job from running, eliminating the race condition with
  DATA_PERMISSIONS cleanup in mt/with-all-users-data-perms-graph!
- non-admin-discard-field-values-test: isolated test for discard_values
- non-admin-rescan-field-values-test: isolated test for rescan_values

Also renames update-database-test, delete-database-test, and fetch-db-test
to non-admin-* for consistency, and adds explicit permission-denial (403)
test cases to each test.

Revives the fix from closed PR #67694.
2026-03-24 14:00:45 +00:00
Braden Shepherdson
b42c2f6605 [QP] Override LIMIT for pivot queries to ensure consistent results (#71033)
Closes #70963
Fixes QUE2-431.

Previously, pivot queries got the same default row limits added as any
aggregated query. But since pivot queries synthesize the results from
several inner queries with different subsets of breakouts, they
have different row counts.

The main ("row details") query the user wrote has all the breakouts,
so it has the most rows. If that query gets truncated, only a subset
of the data is returned, leaving gaps in the pivot table. Worse, the
sub-queries with other subsets of the breakouts, intended to collect
group, column and row subtotals, and the grand total, are inconsistent
with the details in the cells, and with each other, because they can
all end up `SUM`ing (etc.) over different sets of the raw rows.

With this change, a new and much larger row limit is permitted for the
pivot sub-queries. The pivot table viz contains one cell for each
aggregation in the query, multiplied by the rows of the main query.
We budget for **200k cells**, divide by the number of aggregations,
and that becomes the row limit.

If the row details query reaches that limit, the query is rejected:
- Other sub-queries are skipped
- A `pivot-rows-truncated` flag in the results signal the FE
- Which shows an error suggesting the user add a filter or remove
  breakouts to lower the row count.

Some reasonable pivot queries might still be excluded, but now we never
run a pivot query with incorrect data in it. If we execute the query and
render the pivot results, then all the returned data should be correct.
2026-03-24 13:45:34 +00:00
Nick Fitzpatrick
cfa96f3fe0 Catch drag events on Dashcard placeholders (#71274)
* Catch drag events on Dashcard placeholders

* scope drag events, add test
2026-03-24 18:46:31 +05:30
github-automation-metabase
3f55d1b9f4 Use env var indirection in Loki stress-test workflow (#71349)
Fix script injection via env var indirection in loki-stress-test-flake-fix.yml
2026-03-24 13:48:02 +01:00
Nemanja Glumac
acbf9bcc88 Delete Claude bug fix workflow (#70993) 2026-03-24 12:52:30 +01:00
Uladzimir Havenchyk
442cf4abb9 fix: cross branch migration workflow (#71153) 2026-03-24 14:31:51 +03:00
Kamil Mielnik
aee64c6047 GDGT-1869 Remove "as VisualizationSettingsDefinitions" casts (#71217) 2026-03-24 17:30:47 +07:00
Nicolò Pretto
41ca038f67 feat(embedding hub): add ability to hide sidebar for some pages (#71278) 2026-03-24 10:09:27 +01:00
Chris Truter
a0896d1faf Fix clickhouse test flake (#71306) 2026-03-24 10:26:47 +02:00
Nathan Voxland
ad62aed71a Cleaning up code 2026-03-24 02:15:25 -05:00
Nathan Voxland
3ab76adf8d Merged from master 2026-03-24 01:36:35 -05:00
Nathan Voxland
1f019c1e10 Merge remote-tracking branch 'origin/master' into persistent-queue
# Conflicts:
#	enterprise/backend/src/metabase_enterprise/action_v2/data_editing.clj
#	enterprise/backend/test/metabase_enterprise/action_v2/api_test.clj
#	enterprise/backend/test/metabase_enterprise/agent_api/api_test.clj
#	enterprise/backend/test/metabase_enterprise/serialization/v2/e2e_test.clj
#	enterprise/backend/test/metabase_enterprise/serialization/v2/load_test.clj
#	src/metabase/analytics/prometheus.clj
2026-03-24 01:28:52 -05:00
Nathan Voxland
3e4335a798 Merge remote-tracking branch 'origin/persistent-queue' into persistent-queue 2026-03-24 01:19:44 -05:00
Nathan Voxland
70967125aa Code cleanup 2026-03-24 01:16:36 -05:00
Nathan Voxland
5921542894 Code cleanup 2026-03-24 01:05:12 -05:00
Romeo Van Snick
c8cb9c2f04 Remove sync buttons for managed cloud databases (#71203)
* Remove sync buttons for managed cloud databases

* Remove sync settings button for table in attached data warehouse in admin > table metadata and data studio
2026-03-24 14:17:29 +11:00
Fraser
b8c90cdfbe Push shared files down from feature modules (#71137) 2026-03-24 15:24:34 +13:00
Romeo Van Snick
0ab5a0f603 Fix parameter tags reordering in the presence of snippet tags (#71209)
* Disregard snippet tags when reordering parameters

* Add repro for metabase#69160

* Wording

Co-authored-by: Kamil Mielnik <kamil@kamilmielnik.com>

---------

Co-authored-by: Kamil Mielnik <kamil@kamilmielnik.com>
2026-03-24 11:59:52 +11:00
Riley Thompson
8f9282dca1 Don't generate join alias that matches source table name (#70876)
* include source name in taken names

* add test

* Update src/metabase/lib/join.cljc

Co-authored-by: metamben <103100869+metamben@users.noreply.github.com>

* fix tests

* fix test

* Fix test metadata for self-join test

* e2e test fix attempt

* second attempt

---------

Co-authored-by: metamben <103100869+metamben@users.noreply.github.com>
2026-03-24 00:39:09 +00:00
Nathan Voxland
8c572f77f4 Fix race conditions in submit-delivery! and topic offset advancement
- submit-delivery!: atomic busy-check and registration via single swap!
  to prevent concurrent handlers for the same channel
- start-worker-pool!: use compare-and-set! to prevent pool leak on
  concurrent initialization
- Topic backends: only advance offset when submit-delivery! returns true,
  preventing silent message loss when channel is busy
2026-03-23 19:10:23 -05:00
bryan
c8560d8251 Models to transforms (#70735)
Co-authored-by: Alexander Polyankin <alexander.polyankin@metabase.com>
2026-03-23 23:59:16 +00:00
Nathan Voxland
7e5bb21c3d Fix high-priority issues: retry flush failures, non-blocking batch processing, atomic after-commit drain
- Publish buffer: re-buffer messages on flush failure instead of dropping them, with Prometheus counter
- Queue/topic processing: centralized non-blocking dispatch via shared worker pool in mq.impl, with busy-channel tracking so poll threads never block on listener execution
- Queue heartbeats: poll thread updates status_heartbeat every 2min for in-flight batches to prevent stale recovery during long processing
- After-commit callbacks: atomically drain with reset-vals! loop to support callbacks that register new callbacks
- Document all-or-nothing ACK semantics on queue bundle retry in handle! docstring
2026-03-23 18:58:14 -05:00
appleby
74a9d5baeb BOT-1157 Update metabot prometheus and snowplow metrics to include both provider and model (#71259)
We now report the full provider and model rather than just the model, e.g.

    anthropic/claude-haiku-4-5 -> openrouter/anthropic/claude-haiku-4-5
    claude-haiku-4-5           -> anthropic/claude-haiku-4-5

In addition to being useful info to capture, this matches the behavior of the pre-existing ai-service metrics.
2026-03-23 16:42:40 -07:00
bryan
fb428f307e Extend ci-report to accept branch names and commit SHAs (#70742) 2026-03-23 15:50:21 -06:00
Jeff Bruemmer
e50748827a docs - sdk playground (#71314)
* sdk playground link

* edit

* image

* Optimized 1 images, saved 188.7 KB (35.5% smaller)

---------

Co-authored-by: Metabase Automation <github-automation@metabase.com>
2026-03-23 21:48:31 +00:00
Ben Grabow
9ec99afb69 Ghy-3258/table tag dependencies missing in incremental transforms (#71316)
Include table tags as dependencies of native queries
2026-03-23 16:40:06 -05:00
Oleksandr Solovyov
c2ad36cccb move ee/llm into oss (#71068)
* move ee/llm into oss

* lots of file movement

* move slack connect to opensource

* ability to have enterprise-only tools

* move tests

* Fix stale metabot-v3 aliases, dangling module refs, disabled linting, broken test paths, and typo

* start using deprecated-name, more ns path fixes

* move llm transforms code to oss

* fix namespaces

* whitespace fix

* checkpoint: switch tools to use mu/defn or defenterprise directly

* compilation fix

* lint fixes

* move around some code to break drivers dependency

* fix tests

* fix tool calls

* sort routes

* fix openapi

* fix module config

* lint

* fix clj-kondo

* fix test

* fix token check tests

* cljfmt

* fix slack test

* maybe fix dashboard tests?

* cljfmt

* is this a fix for dashboard tests

* move metabot models to oss

* untangle module config after rebase

* fix module order

* cleanup

* add back ai service-related settings that are still used

* move some more code to more appropriate places

* more code movement

* better schema names

* clean up mu/defn function name formatting

* update module config

* remove unneeded ai service-related test

* fix module config again

* fix auth in tests by removing more ai service stuff

* enable native agent by default

* test transforms over normal function calls

* requires

* pass through metabot-id to tool calls instead of hardcoding embedded-metabot-id, to fix answer sources test

* fix wrap-tools-with-state calls

* try to fix failing field/dashboard tests by matching the code on master (might have been from a bad rebase)

* Revert "try to fix failing field/dashboard tests by matching the code on master (might have been from a bad rebase)"

This reverts commit cd5ec8b487.

* hard revert parameters.dashboard-test to the version on master after the latest rebase

* mark snippet tests as sequential to try to fix perms test issues, remove a couple stray metabot feature checks

* move entity analysis + sql fixing code to oss

* wrap dashboard tests with with-full-data-perms-for-all-users to isolate the failure

* move ai-entity-analysis and ai-sql-fixer under the metabot module

* fix module test

* remove now-unneeded ai feature falgs

* disable perms cache for these failing dashboard tests

* simplify test that was still using agent-api feature flag

* memoize spec-index-query by app db to fix failing search tests caused by new test execution order

* LLM OSS FE (#71197)

* bulk move files

* replace usage of /ee/metabot-v3/ to /metabot

* remove PLUGIN_METABOT

* fix broken metabot routes, removes metabot_v3 token feature usage

* remove PLUGIN_AI_*

* bring back enterprise vs oss behavoir, with the code still being OSS

* fix lint + unit tests

* fix embedding sdk stories

* add back missing FixSqlQueryButton.unit.spec.tsx test

* fix last references of `/ee/ai-*` endpoints

* pr feedback

* test fix

---------

Co-authored-by: Noah Moss <noahbmoss@gmail.com>
Co-authored-by: Sloan Sparger <sloansparger@users.noreply.github.com>
2026-03-23 17:30:50 -04:00
Chris Truter
76b01b772b Define additional metrics for release monitoring (#71237) 2026-03-23 17:20:34 -04:00
Noah Moss
733176c925 Add cancelled check for sdk-tests (#71307) 2026-03-24 10:03:27 +13:00
Ryan Laurie
88a87db2b8 Sanitize database details during serialization import (#71311)
[59] Sanitize database details during serialization import (#335)

* Sanitize database details during serialization import

Strip low-level JDBC keys (classname, subprotocol, connection-uri, subname,
init) from database details during serialization import and reject h2 engine.

Also add subname and init to the connection-details dispatch blocklist and
sanitize details in h2 connection-details->spec as defense in depth.

* Use custom runners

* fix error printing and test

* rebind assert-not-h2! during tests

* kondo ignore

---------



# Conflicts:
#	.github/workflows/backend.yml
#	.github/workflows/cross-branch-migrations.yml
#	.github/workflows/tusk-sanity-check.yml

Co-authored-by: Nicola Mometto <nicola.mometto@metabase.com>
Co-authored-by: Ngoc Khuat <qn.khuat@gmail.com>
Co-authored-by: Nemanja <31325167+nemanjaglumac@users.noreply.github.com>
2026-03-23 14:40:06 -06:00
Ryan Laurie
45380db283 Send release jars to cloud bucket (#71093)
* update var

* do it in the tag step
2026-03-23 14:29:58 -06:00
metamben
84b79b9126 Reorganize QP module (#70808)
* Introduce qp.core and qp.test namespaces
2026-03-23 20:14:12 +00:00
Nathan Voxland
2c8c0e30aa Better consolidation of topic vs queue logic 2026-03-23 14:44:03 -05:00
Nicola Mometto
d6d3bbb89a Fix bug with temporal values in bigquery incremental transforms (#71291)
* fix: coerce temporal values used in incremental transforms filters

* fallthrough on LocalDateTime

Co-authored-by: metamben <103100869+metamben@users.noreply.github.com>

---------

Co-authored-by: metamben <103100869+metamben@users.noreply.github.com>
2026-03-23 18:41:40 +00:00
Alex Yarosh
1bb4b60d60 docs: workflow schedule (#71305)
docs workflow schedule
2026-03-23 14:33:38 -04:00
Sébastien
1827c0c37d Fixed dancing MB (#71211) 2026-03-23 16:48:56 +01:00
github-automation-metabase
5dd6f68c46 Fix title escaping for JSON (#71289) 2026-03-23 16:08:50 +01:00
Nemanja Glumac
c85d47365d Remove hard-coded Trivy version (#71287) 2026-03-23 15:03:46 +01:00
github-automation-metabase
17bca77f06 Auto-update documentation for master (#71268)
Update generated docs (master)

Co-authored-by: github-actions <github-actions@github.com>
2026-03-23 09:19:13 -04:00
Jakub Chodorowicz
4c4c049ac8 update copy and styling of table variable select (#71101)
* update copy and styling of table variable select

* extract common LabelWithInfo component

standardize styling
2026-03-23 13:13:42 +01:00
Kevin Kelleher
38fa86db3f Fix scatter plot flicker on hover (#71036) 2026-03-23 07:38:27 -04:00
Nemanja Glumac
ac3a38209f Patch vulnerable Trivy action (#71282)
Resolves DEV-1712

Patches Trivy GitHub action to a version that is confirmed not to be affected by the supply chain attack. We're using SHA-1 instead of a tag for an improved security.

See: https://app.mendral.com/insights/01KM6NFNZPV1JS72KAPCA2K9XH
2026-03-23 12:22:55 +01:00
Chris Truter
c568186ebc Only run Eastwood (test) on master (#71232) 2026-03-23 12:30:13 +02:00
Nicolò Pretto
a49e97237f add ability to hide Our analytics in MiniPicker and hide it in the table picker in the embedding setup (#70920)
* add ability to hide Our analytics in MiniPicker

* remove changes not needed

* update getOurAnalytics with the values from EntityPicker
2026-03-23 11:12:55 +01:00
Fraser
f1c36719aa Move shared query_builder logic to querying (#71129) 2026-03-23 09:22:12 +00:00
Sébastien
292a667717 Fixes a loki flaky (#71225)
* loki flaky

* .

* .

* fonts
2026-03-23 10:02:28 +01:00
Kamil Mielnik
361549c5af GDGT-2024 Consolidate ChartSettingsWidget's noPadding, marginBottom, and borderBottom (#71154) 2026-03-23 14:25:07 +07:00
Romeo Van Snick
c009e25907 Fix timeline form icon input (#71199)
* Add icon to timeline icons

* Use Select from mantine/ui for timeline form

* Add test for icon in timeline creation
2026-03-23 16:21:20 +11:00
Romeo Van Snick
e13fd47144 Convert dataref to ts (#71062)
* Convert dataref to ts

* Fix unit test

* Fix actions

* Fix field reference

* Fix native data reference

* Fix field pane
2026-03-23 13:01:43 +11:00
metabase-bot[bot]
2812c4496d [chore] Update E2E test timings with recent data (#71265)
Update E2E test timings with averaged data

Collected timing data from 7 days of test runs.
All specs with timing data were included (simple averages).

🤖 Generated automatically by update-e2e-timings workflow

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: GitHub Actions <actions@github.com>
2026-03-23 10:11:46 +13:00
Nathan Voxland
28093ce803 Support Group Sync in OIDC (#71176) 2026-03-21 00:24:40 -05:00
Nathan Voxland
a649b08171 Field filter does not show dropdown values (#71114) 2026-03-21 00:23:58 -05:00
Egor Grushin
7a7585be68 Gdgt 1864 Add pinned columns and rows to the DataGrid (#70384) 2026-03-21 02:10:09 +00:00
Anderson
6b241a18fd Fix dashboard tab list items having role="button" instead of role="tab" (#70546) (#70976)
* Fix aria-expanded on dashboard filter trigger button (#70543)

Closes https://github.com/metabase/metabase/issues/70543

### Description

Move aria-expanded from a wrapper div to the actual trigger button element
for dashboard filter popovers. Mantine's Popover.Target injects
aria-expanded via cloneElement onto its direct child. Previously, a Box
wrapper div was the direct child, so aria-expanded landed on a
non-interactive div instead of the button. This restructures the
component hierarchy so ParameterValueWidgetTrigger (which renders as a
button) is the direct child of Popover.Target, and moves the Sortable
wrapper outside the Popover.

### How to verify

1. Open a dashboard with filter parameters
2. Inspect the filter trigger button element in the DOM
3. Verify aria-expanded is on the button element (not a wrapper div)
4. Click the filter to open the popover and confirm aria-expanded
   toggles to true on the button

### Checklist

- [x] Tests have been added/updated to cover changes in this PR

* Fix dashboard tab list items having role="button" instead of role="tab"

Closes https://github.com/metabase/metabase/issues/70546

### Description

The Sortable wrapper around each dashboard tab item was applying a default
`role="button"` to its container div, which overrode the inner TabButton
`role="tab"` from the perspective of screen readers and accessibility tools.
Changed the Sortable wrapper to use `role="presentation"` so it is transparent
to the accessibility tree, allowing the inner TabButton role="tab" to be
properly recognized within the role="tablist" container.

### How to verify

1. Open a dashboard with multiple tabs
2. Inspect the tab items in the browser DevTools
3. Verify that tab items have `role="tab"` (not `role="button"`)
4. Verify the tab list container has `role="tablist"`

### Checklist

- [x] Tests have been added/updated to cover changes in this PR

* Add issue number to test titles (#70546)

* Improve test

* Clean up
2026-03-20 20:19:30 -03:00
Ryan Laurie
b9573fc138 Use Local DigiCert for docker container builds (#71095)
use local cert

# Conflicts:
#	Dockerfile
#	bin/docker/Dockerfile
#	bin/docker/Dockerfile_ubuntu
2026-03-20 21:55:40 +00:00
Ben Grabow
8d41c7d068 Disallow adding new classpath resources as geojson paths (#70745)
* Disallow adding new classpath resources as geojson paths
2026-03-20 21:16:01 +00:00
Ben Grabow
e0365e3297 Revert "Temporarily disable app db version checks" (#71191)
Revert "Temporarily disable app db version checks (#70884)"

This reverts commit c5dcdc3238.
2026-03-20 15:36:19 -05:00
Braden Shepherdson
6b6ea19ab7 [MBQL lib] Add user docs for breakouts functions (#71034)
Part of QUE2-366.
2026-03-20 19:56:00 +00:00
Uladzimir Havenchyk
5a142f6eb1 add a workflow to publish custom viz npm package (#71221)
* add a workflow to publish custom viz npm package

* trigger CI
2026-03-20 12:32:56 -06:00
ignacio
cfacacd556 Update account settings docs (#71240)
This is currently doable in version 59.3.



See this issue: https://github.com/metabase/metabase/issues/63581
2026-03-20 15:22:12 -03:00
Uladzimir Havenchyk
a9b9ad9d85 chore: convert isa.js to typescript (#71014)
* chore: convert isa.js to typescript

* address comments

* address review

* revert back broken checks

* fix review
2026-03-20 20:35:23 +03:00
Chris Truter
68ec84ae4d Make all model imports and exports explicit (#71224) 2026-03-20 09:58:44 -07:00
Jakub Chodorowicz
6724b20a7b enable transform inspector only with advanced (python) transforms (#71067)
hide inspect tab when it's not supported by current features
2026-03-20 17:40:50 +01:00
Edward Paget
dd34d56120 fix: cancel serialization when client disconnects (#69864)
* fix: cancel serialization when client disconnects

Co-Authored-By: John Swanson <john.swanson@metabase.com>

* fix: complete serialization virtual thread streaming response

- Fix channel signal direction: use >!! to write to done-chan instead of <!! (read)
- Fix streaming response: use io/copy instead of .write for File objects
- Set explicit 200 status on success, set-status!/set-content-type! on error path
- Add Content-Disposition header on success for proper download behavior
- Clean up temp files on cancel and use finally for deterministic cleanup
- Remove commented-out old synchronous code
- Use canonical clojure.core.async alias (a instead of async)
- Add server module to enterprise/serialization allowed dependencies
- Relax full_stacktrace test threshold (200 -> 50 lines) to avoid JVM-dependent flakiness

---------

Co-authored-by: John Swanson <john.swanson@metabase.com>
2026-03-20 11:28:41 -05:00
Braden Shepherdson
3559ec98b0 [QP] Do coercions (eg. Unix timestamps) with native sandboxing (RCLS) (#70885)
Fixes QUE2-376.

For MBQL queries, type coercions (like integers to Unix timestamps) only
get applied on the earliest stage where that field is introduced. When
the table the field comes from is sandboxed by a native query, an
extra stage is introduced but it doesn't do the coercion.

Decorates the metadata of any native sandbox column whose table counterpart had coercion defined.

It's unfortunately complicated inside lib since the metadata needs to be propagated through exactly one
stage boundary. If it propagates all the way then we get double-coercion and SQL errors.
2026-03-20 16:24:21 +00:00
Jeff Bruemmer
bc80f935a2 docs - transform inspector (#71188)
* draft

* edits

* edits

* toc

* Optimized 3 images, saved 326.0 KB (47.1% smaller)

* Update docs/data-studio/transforms/transform-inspector.md

Co-authored-by: Alex Yarosh <alexandra@metabase.com>

* edits

---------

Co-authored-by: Metabase Automation <github-automation@metabase.com>
Co-authored-by: Alex Yarosh <alexandra@metabase.com>
2026-03-20 12:19:19 -04:00
github-automation-metabase
d0ac061fee Fix trivy action tag prefix (#71213)
fix: use v prefix on aquasecurity/trivy-action@v0.29.0 tag reference
2026-03-20 09:17:26 -06:00
Jeff Bruemmer
0cdcd2232c docs - agent resources for embedding (#71108) 2026-03-20 10:40:15 -04:00
Sébastien
8de615a69d Fixed empty csv for adhoc pivot tables in SDK (#71141) 2026-03-20 14:28:59 +01:00
Uladzimir Havenchyk
24968aa972 build: use a proper link for cert download (#71064) 2026-03-20 16:22:23 +03:00
Ngoc Khuat
a3ae3ba1f4 Transform inspector reorganization (#71136) 2026-03-20 10:34:24 +00:00
Nicola Mometto
22b6962656 default transforms impls with informative errors (#71145)
feat: provide default transforms impls with informative errors
2026-03-20 10:14:14 +00:00
Nicolò Pretto
94be4bfae3 Emb 1450 simplify sdk e2e test commands and docs (#71168)
* remove the embedding dev command and update the docs

* fix typo
2026-03-20 09:42:10 +01:00
Ngoc Khuat
39bc014f14 Server-side rendering for comment emails (#70938) 2026-03-20 07:55:16 +00:00
Fraser
4864f0ae24 Revert "Further pdf export improvements" (#71208)
Revert "Further pdf export improvements (#71031)"

This reverts commit 536d6bf22f.
2026-03-20 19:45:09 +13:00
Aleksandr Lesnenko
cd7eb82bdb remove emotion from browse pages (#71184) 2026-03-20 06:43:45 +05:30
Nick Fitzpatrick
536d6bf22f Further pdf export improvements (#71031)
* use cssText

* include font props on <text>, <tspan>, and <textPath> nodes
2026-03-20 05:59:35 +05:30
Anderson
ac092618ac Hide 'New metric' and 'Publish a table' actions on Data Studio Library empty state when in read-only mode (#71112)
* Fix: Hide metric creation and table publishing actions in read-only mode

In read-only instances (remote sync read-only mode), the Library still showed:
- 'New metric' action link in the Metrics empty state
- 'Publish a table' action button in the Data empty state
- 'New snippet' action link in the SQL snippets empty state

While the +New button was already hidden in read-only mode, the empty state
action links were not checking for this permission, allowing users to attempt
actions that should be disabled.

Changes:
- Add read-only mode check to EmptyStateAction component in LibrarySectionLayout
- Empty state actions now return null when in remote sync read-only mode
- Add e2e test to verify actions are hidden in read-only mode

* Refactor: use hideAction param and call-site guard instead of returning null from EmptyStateAction

Leverage the existing third param of createEmptyStateItem to suppress
actionUrl for metrics/snippets in read-only mode (consistent with how
the snippet tree handles it). Guard the 'Publish a table' button at
the EmptyStateAction call site in LibrarySectionContent, outside the
component itself. Reorganise the e2e test into the empty state describe
block.
2026-03-19 20:34:20 -03:00
Tyler Plude
03a09e1832 BOT-1127 Metabot Conditional Tool Schema Params (#71050)
* conditional tool schema input params

* simplify feature-available?

* unit tests

* applying to other search tools
2026-03-19 16:01:59 -07:00
Chris Truter
cc74f66a1b Restrict which models are accessible by each module (#71010) 2026-03-19 21:08:20 +00:00
John Swanson
bfa572ec6b Metric viewer can use metrics based on cards (#71107)
[UXW-3260: Metrics from models/table questions are not viewable in the Metrics viewer](https://linear.app/metabase/issue/UXW-3260/metrics-from-modelstable-questions-are-not-viewable-in-the-metrics)
2026-03-19 12:22:17 -08:00
Romeo Van Snick
fb38572a8c Fix flaky test in sidebar content (#71135)
Fix flaky test
2026-03-20 07:11:37 +11:00
lbrdnk
a02acabac1 Add database_engine to enhanced tables (#70968) 2026-03-19 20:59:15 +01:00
Rodrigo López Dato
198cd9a9a3 Remove X-Metabase-Session from autogenerated API docs in favor of X-API-Key (#71155)
* X-API-Key capitalization in docs

* Use X-API-Key instead of X-Metabase-Session in Scalar API docs
2026-03-19 19:32:45 +00:00
Eric Normand
c9c7f230fb Fix string field IDs leaking into bulk-metadata queries (#71167)
Add pos-int? guard to all-implicitly-joined-field-ids pattern match,
matching the guard already used by all-field-ids. String field names
(e.g. from native/model sources) were being collected and passed to
bulk-metadata, causing Postgres type mismatch errors.
2026-03-19 13:58:07 -05:00
Aleksandr Lesnenko
e1608e708d Add skill to migrate emotion to mantine and css modules (#71173)
add skill to migrate emotion to mantine and css modules
2026-03-19 14:21:59 -04:00
Chris Truter
213b1b5905 Add mechanism for more reliably renaming settings (#71088) 2026-03-19 14:12:31 -04:00
Eric Normand
42fdf326b6 Fix StackOverflowError in deduplicated column name resolution (#71096)
Add a dynamic guard `*in-deduplicated-column-resolution?*` and convert the
mutual recursion between `resolve-nonexistent-deduplicated-column-name` and
`resolve-from-previous-stage-or-source` into a loop. Previously, resolving
a field like `name_5000` would create O(N) deep mutual recursion, overflowing
the JVM stack at ~3000-4000 frames.
2026-03-19 12:24:47 -05:00
Nathan Voxland
4ca03e17ef Took out now-unneeded search delay 2026-03-18 02:05:25 -05:00
Nathan Voxland
ec2b0f38ae Merge remote-tracking branch 'origin/master' into persistent-queue
# Conflicts:
#	src/metabase/analytics/prometheus.clj
2026-03-18 01:46:46 -05:00
Nathan Voxland
2bad85564e Publish messages after an appdb transaction. Plus batch them still for 100ms to handle times we don't really have a transaction to avoid so many small batches 2026-03-18 01:45:26 -05:00
Nathan Voxland
d1252c7e8e Fixed merges from master 2026-03-13 11:59:15 -05:00
Nathan Voxland
4a7059cac7 Merge remote-tracking branch 'origin/master' into persistent-queue
# Conflicts:
#	resources/migrations/059_update_migrations.yaml
#	src/metabase/search/appdb/core.clj
#	src/metabase/search/core.clj
#	src/metabase/search/impl.clj
#	src/metabase/search/ingestion.clj
#	src/metabase/search/task/search_index.clj
2026-03-13 11:09:28 -05:00
Nathan Voxland
9bd22bf4b1 Added dedup-fn feature 2026-03-10 02:03:18 -05:00
Nathan Voxland
0462cde56d Fixed kondo 2026-03-09 11:14:43 -05:00
Nathan Voxland
e99312fe64 Renamed def-listener to def-listener! 2026-03-09 10:24:36 -05:00
Nathan Voxland
d8a7cd3501 Fixing tests 2026-03-08 01:41:38 -06:00
Nathan Voxland
2d2c4b35a5 Fixing tests 2026-03-07 01:25:05 -06:00
Nathan Voxland
9e717a6056 Merge remote-tracking branch 'origin/master' into persistent-queue
# Conflicts:
#	resources/migrations/059_update_migrations.yaml
2026-03-07 00:09:19 -06:00
Nathan Voxland
9ed603e6ce Better multi-backend handling 2026-03-06 17:30:39 -06:00
Nathan Voxland
8a5545ad32 Fixed whitespace and migrations 2026-03-06 16:01:30 -06:00
Nathan Voxland
82b89e31fb Fixing tests 2026-03-06 15:09:41 -06:00
Nathan Voxland
607985e054 Fixing tests 2026-03-05 22:15:42 -06:00
Nathan Voxland
a2e43e1996 Fixing tests 2026-03-05 21:23:23 -06:00
Nathan Voxland
3b96cd4619 Deps fix 2026-03-05 15:56:10 -06:00
Nathan Voxland
31790b6052 Added def-listener! and fixing tests 2026-03-05 15:42:23 -06:00
Nathan Voxland
846f38d443 Fixing tests 2026-03-05 02:00:51 -06:00
Nathan Voxland
e7a9e8c100 Removed other settings cache sync code 2026-03-05 01:20:07 -06:00
Nathan Voxland
e1b7b8e912 Use new queue better in search 2026-03-05 00:46:36 -06:00
Nathan Voxland
2480ce058b Support exclusive queue listeners 2026-03-04 11:27:19 -06:00
Nathan Voxland
cf04ba48c2 Immediately deliver topic posts locally without waiting for backend roundtrip 2026-03-04 09:51:00 -06:00
Nathan Voxland
8a853116bf Merge remote-tracking branch 'origin/master' into persistent-queue
# Conflicts:
#	.clj-kondo/config.edn
2026-03-04 01:26:56 -06:00
Nathan Voxland
c54c9ded12 improving code 2026-03-04 01:20:12 -06:00
Nathan Voxland
0fc1a4b8c9 Code improvements and removed postgres topic listener 2026-03-04 00:18:19 -06:00
Nathan Voxland
6ca5625902 Code improvements 2026-02-27 12:47:50 -05:00
Nathan Voxland
a4fdcdef64 Code review fixes 2026-02-21 10:21:20 -06:00
Nathan Voxland
b1fe71ca2f Cleaning up force-sync usage 2026-02-21 02:29:02 -06:00
Nathan Voxland
4daf2cb8a2 Added batch-listen! 2026-02-21 01:52:57 -06:00
Nathan Voxland
4a3d739318 Started to migrate old util.queue code to new mq 2026-02-20 15:41:51 -06:00
Nathan Voxland
ba85c0513c Cleanup failed queue messages 2026-02-19 16:55:00 -06:00
Nathan Voxland
b8c26e3c0c Code review fixes 2026-02-19 16:48:09 -06:00
Nathan Voxland
21b5b3e93a Merge remote-tracking branch 'origin/master' into persistent-queue
# Conflicts:
#	resources/migrations/059_update_migrations.yaml
2026-02-19 16:20:28 -06:00
Nathan Voxland
0a1a13b48b Code cleanup 2026-02-19 16:09:11 -06:00
Nathan Voxland
a18f7a94e2 Code cleanup 2026-02-19 15:34:05 -06:00
Nathan Voxland
067626c71d Added metrics 2026-02-19 10:41:42 -06:00
Nathan Voxland
3bc10ee2e1 Cleanly shut down 2026-02-19 09:53:39 -06:00
Nathan Voxland
b9fba6ab7a Cleaning up topic code 2026-02-19 01:15:32 -06:00
Nathan Voxland
20e5611810 Cleaning up topic code 2026-02-18 16:26:15 -06:00
Nathan Voxland
37d68bca75 Made mq.topic code look like mq.queue code 2026-02-18 15:52:48 -06:00
Nathan Voxland
e58e0b7f80 Queue code improvements 2026-02-18 15:21:22 -06:00
Nathan Voxland
81aab7b8ca Added schemas 2026-02-18 14:15:45 -06:00
Nathan Voxland
bcf8a422f6 Queue code cleanup 2026-02-18 14:03:30 -06:00
Nathan Voxland
ac434a50aa Queue code cleanup 2026-02-18 13:02:49 -06:00
Nathan Voxland
4f050fba0a Use notify on postgres 2026-02-18 01:25:29 -06:00
Nathan Voxland
c36a72146e moved queue and pubsub to .mq 2026-02-18 00:27:53 -06:00
Nathan Voxland
d82ec362ba Added pub/sub messaging 2026-02-17 17:04:11 -06:00
Nathan Voxland
c2172311f5 Better queue testing with memory queue 2026-02-17 16:19:34 -06:00
Nathan Voxland
5a7bcfab93 Better naming for storing messages in the appdb queue 2026-02-17 15:37:43 -06:00
Nathan Voxland
decbc82b1a Code review fixes 2026-02-17 14:56:27 -06:00
Nathan Voxland
bd0f5b3b69 Merge remote-tracking branch 'origin/master' into persistent-queue 2026-02-17 14:33:54 -06:00
Nathan Voxland
7f98271a59 Don't have a queue_message model since it's an implementationd detail 2026-01-26 17:04:26 -06:00
Nathan Voxland
8cc219994d Merge remote-tracking branch 'origin/master' into persistent-queue 2026-01-26 16:44:09 -06:00
Nathan Voxland
84168dbf05 Merge remote-tracking branch 'origin/master' into persistent-queue
# Conflicts:
#	resources/migrations/059_update_migrations.yaml
2026-01-26 15:27:21 -06:00
Nathan Voxland
d6da219a70 Fixes from code review and getting back to project 2026-01-16 16:28:37 -06:00
Nathan Voxland
65426fe653 Merge remote-tracking branch 'refs/remotes/origin/master' into persistent-queue
# Conflicts:
#	resources/migrations/059_update_migrations.yaml
2026-01-16 16:11:19 -06:00
Nathan Voxland
6403d80fea Merge branch 'master' into persistent-queue 2026-01-14 11:11:53 -06:00
Nathan Voxland
bbcba1500c Added queue_message table back 2026-01-12 14:29:48 -06:00
Nathan Voxland
fcf4dbb6e8 Merge branch 'master' into persistent-queue
# Conflicts:
#	.clj-kondo/config/modules/config.edn
#	resources/migrations/056_update_migrations.yaml
2026-01-12 14:29:04 -06:00
Nathan Voxland
3d75754ad8 Added sucess and failure handling, general cleanup 2025-07-18 17:06:05 -05:00
Nathan Voxland
af72309d8e e2e test works again 2025-07-17 22:30:10 -05:00
Nathan Voxland
646b4c4548 Reverted metabase.queue changes, implemented metabase.queue.memory, code cleanup 2025-07-15 21:00:16 -05:00
Nathan Voxland
92ada9186c Code cleanup 2025-07-15 14:33:44 -05:00
Nathan Voxland
e67ce4582e Moved migrations 2025-07-14 23:07:20 -05:00
Nathan Voxland
25e3afcdbb Refactoring and trying to get e2e test to pass again 2025-07-14 23:07:20 -05:00
Nathan Voxland
b31a9a09c5 Configured queue listener to use persistent queue 2025-07-14 23:07:20 -05:00
Nathan Voxland
50614c15aa Initial implementation 2025-07-14 23:07:20 -05:00
2131 changed files with 53065 additions and 39785 deletions

View File

@@ -1,4 +1,4 @@
--b-
---
name: add-tracing
description: Add OpenTelemetry tracing spans to Clojure code following Metabase tracing conventions. Use when instrumenting backend code with trace coverage.
---

View File

@@ -0,0 +1,834 @@
---
name: emotion-migrate
description: Migrate Emotion styled-components to Mantine components with style props and CSS modules. Use when converting .styled.tsx files or removing @emotion imports from components.
---
# Emotion → Mantine + CSS Modules Migration Skill
Migrate Emotion styled-components (`@emotion/styled`, `@emotion/react`) to Mantine layout components with style props and CSS modules. The goal is zero Emotion imports, zero inline styles, and maximum use of design system tokens.
## Priority Order (Strict)
1. **Mantine components + style props** — `Box`, `Flex`, `Stack`, `Group`, `Text`, `Title`, `Card`. This is the DEFAULT. Every CSS property must be checked against style props FIRST.
2. **CSS modules** (`.module.css`) — ONLY for properties that Mantine style props genuinely cannot express: pseudo-selectors (`:hover`, `:focus`, `::before`), `box-shadow`, `border` shorthand, `animation`/`@keyframes`, complex selectors, `cursor`, `pointer-events`, `overflow`, `text-overflow`, `white-space`, `transition`.
3. **Inline styles ONLY for dynamic values** — `style={{ }}` is allowed only for truly dynamic runtime values (e.g., computed widths, positions, data-driven colors). All static styles must use Mantine props or CSS modules.
### Mantine-First Decision Gate (CRITICAL)
For EACH styled component, go through every CSS property and ask: "Can this be a Mantine style prop?" If yes → style prop. If no → CSS module. Do NOT dump an entire component into a CSS module just because one property needs it — split them.
**Properties that ARE style props** (use these, not CSS modules):
- `display` → `display` prop
- `color` → `c` prop (`c="brand"`, `c="text-primary"`)
- `background-color` → `bg` prop (`bg="background-primary"`)
- `font-size` → `fz` prop (`fz="md"`)
- `font-weight` → `fw` prop (`fw="bold"`)
- `line-height` → `lh` prop (`lh="md"`)
- `text-align` → `ta` prop (`ta="center"`)
- `padding` (all variants) → `p`, `px`, `py`, `pt`, `pb`, `pl`, `pr`
- `margin` (all variants) → `m`, `mx`, `my`, `mt`, `mb`, `ml`, `mr`
- `width` → `w`, `min-width` → `miw`, `max-width` → `maw`
- `height` → `h`, `min-height` → `mih`, `max-height` → `mah`
- `flex` → `flex` prop (`flex="0 0 auto"`, `flex={1}`)
- `gap` → `gap` prop (on Flex/Stack/Group)
- `align-items` → `align` prop (on Flex/Stack/Group)
- `justify-content` → `justify` prop (on Flex/Stack/Group)
- `flex-direction` → `direction` prop (on Flex)
- `flex-wrap` → `wrap` prop (on Flex)
- `position` → `pos` prop
- `top/right/bottom/left` → `top`, `right`, `bottom`, `left` props
- `opacity` → `opacity` prop
**Properties that NEED CSS modules** (no style prop equivalent):
- `:hover`, `:focus`, `:active`, `::before`, `::after` (pseudo-selectors)
- `box-shadow`, `border` (shorthand with color), `outline`
- `cursor`, `pointer-events`
- `overflow`, `text-overflow`, `white-space`
- `animation`, `transition`, `transform`
- `@media` queries (UNLESS it's simple responsive spacing/sizing — then use responsive syntax: `p={{ base: "md", lg: "xl" }}`)
**Hybrid approach** — when a component needs BOTH, put style props on the Mantine component AND add a CSS module class for the rest:
```tsx
<Flex
className={S.root} /* for :hover, box-shadow, border */
align="center" /* style prop */
gap="sm" /* style prop */
p="md" /* style prop */
bg="background-primary" /* style prop */
>
```
## CSS Module Conventions (Strict)
### Class Naming: camelCase
All CSS module class names MUST use **camelCase**. This is the dominant convention across the codebase (~830 camelCase vs ~620 PascalCase classes), used consistently in Mantine UI components, and matches standard CSS module conventions.
```css
/* CORRECT */
.root {
}
.settingsSection {
}
.dragHandle {
}
.closeIcon {
}
/* WRONG — do not use PascalCase or kebab-case */
.ItemRoot {
}
.settings-section {
}
```
Modifier/state classes also use camelCase:
```css
.selected {
}
.disabled {
}
.interactive {
}
.draggable {
}
```
### No Cascading — Direct Class Assignment
Cascading selectors are **discouraged**. Instead of styling through parent-child relationships, assign a class directly to the element that needs styling.
```css
/* WRONG — cascading/descendant selectors */
.root > input {
}
.root .label {
}
.container > div > span {
}
/* CORRECT — direct class on the target element */
.input {
}
.label {
}
.title {
}
```
The only acceptable nesting patterns are:
- **Pseudo-selectors on the same element**: `.item { &:hover { } }`
- **Modifier composition**: `.item { &.selected { } }`
- **Hover-reveal patterns** where a parent hover affects a child: `.root:hover .showOnHover { opacity: 1; }` — but only when structurally necessary (the child has no way to know about the parent's hover state)
### Import Alias
Always import the CSS module as `S`:
```tsx
import S from "./ComponentName.module.css";
```
## Step-by-Step Migration Process
### Step 1: Read and Understand
Read the `.styled.tsx` file AND every component that imports from it. Understand:
- Which styled components are used and where
- Which props drive dynamic styles
- Which styles are static vs conditional
- Which styles can map directly to Mantine style props
### Step 2: Classify Each Styled Component
For each styled component, apply the Mantine-First Decision Gate above. Then determine the migration target:
| Emotion Pattern | Migration Target |
| ------------------------------------------------------------ | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `styled.div` with only layout/spacing/color | `Box`, `Flex`, `Stack`, or `Group` with style props. **NO CSS module needed.** |
| `styled.div` with flexbox column | `Stack` component with style props |
| `styled.div` with flexbox row | `Flex` or `Group` component with style props |
| `styled.span` / `styled.p` with color/weight/size | **`Text component="span"` with style props** (`c`, `fw`, `fz`). NO CSS module. |
| `styled.div` with hover/focus/pseudo-selectors | **Hybrid**: Mantine component with style props for expressible properties + CSS module class for pseudo-selectors only |
| `styled.div` with media queries (simple spacing/sizing) | **Responsive style props**: `p={{ base: "md", lg: "xl" }}`. NO CSS module. |
| `styled.div` with media queries (complex/non-spacing) | CSS module for the media query parts, style props for the rest |
| `styled.div` with animations/keyframes | CSS module for animation, style props for layout |
| `styled(SomeComponent)` with only color/spacing/flex | **Wrap in Mantine component** with style props: `<Box c="brand" flex="0 0 auto"><Icon /></Box>`, or pass style props if the component accepts them |
| `styled(SomeComponent)` with pseudo-selectors/complex styles | CSS module `className` on the component |
| Dynamic props `styled.div<{ isActive: boolean }>` | Mantine style props for simple toggles (`c={active ? "brand" : "text-primary"}`), `cx()` with CSS module classes for complex state combinations involving pseudo-selectors |
### Step 3: Create CSS Module (if needed)
Create `ComponentName.module.css` alongside the component file:
```css
/* Use design system tokens — NEVER raw color/spacing values */
.root {
border: 1px solid var(--mb-color-border);
border-radius: var(--mantine-radius-md);
background-color: var(--mb-color-background-primary);
}
/* Conditional states as separate classes, combined with cx() */
.active {
background-color: var(--mb-color-brand);
color: var(--mb-color-text-primary-inverse);
}
.disabled {
color: var(--mb-color-text-tertiary);
pointer-events: none;
}
/* Hover/focus/pseudo-selectors — nest with & */
.interactive {
cursor: pointer;
&:hover {
color: var(--mb-color-brand);
background-color: var(--mb-color-background-hover);
}
}
/* Hover-reveal: acceptable parent→child nesting */
.root:hover .showOnHover {
opacity: 1;
}
/* Responsive styles */
@media (--breakpoint-min-md) {
.root {
padding: var(--mantine-spacing-lg);
}
}
```
### Step 4: Update the Component TSX
```tsx
import cx from "classnames";
import CS from "metabase/css/core/index.css";
import { Box, Flex, Group, Stack, Text } from "metabase/ui";
import S from "./ComponentName.module.css";
// Dynamic props → cx() with conditional classes
<Flex
className={cx(S.root, {
[S.active]: isActive,
[S.disabled]: disabled,
})}
align="center"
gap="sm"
p="md"
w="100%"
>
```
### Step 5: Delete the `.styled.tsx` File
Remove the old styled file entirely. Remove all imports of it from other files.
### Step 6: Verify
- Confirm zero `@emotion/styled` or `@emotion/react` imports remain in the migrated files
- Confirm no static inline styles (dynamic runtime values are fine)
- Confirm all colors use tokens, not raw hex/rgb values
## Design System Token Reference
### Mantine Style Props (use directly on components)
**Spacing** (`p`, `px`, `py`, `pt`, `pb`, `pl`, `pr`, `m`, `mx`, `my`, `mt`, `mb`, `ml`, `mr`):
- `"xs"` = 4px, `"sm"` = 8px, `"md"` = 16px, `"lg"` = 24px, `"xl"` = 32px
- Custom: `rem(48)` for non-standard values (import `rem` from `metabase/ui`)
**Dimensions** (`w`, `h`, `maw`, `mah`, `miw`, `mih`):
- `"100%"`, `"100vh"`, `rem(400)`, etc.
**Colors** (`c`, `bg`):
- Text: `"text-primary"`, `"text-secondary"`, `"text-tertiary"`
- Background: `"background-primary"`, `"background-secondary"`, `"background-hover"`
- Brand: `"brand"`, `"error"`, `"success"`, `"warning"`
**Typography** (`fz`, `fw`, `lh`, `ta`, `ff`):
- Font size: `"xs"` = 11px, `"sm"` = 12px, `"md"` = 14px, `"lg"` = 17px, `"xl"` = 21px
- Font weight: `"bold"`, `"normal"`, or numeric `700`
- Line height: `"xs"` = 100%, `"sm"` = 115%, `"md"` = 122%, `"lg"` = 138%, `"xl"` = 150%
- Text align: `"center"`, `"left"`, `"right"`
**Flexbox** (`align`, `justify`, `gap`, `direction`, `wrap`, `flex`):
- `align="center"`, `justify="space-between"`, `gap="sm"`, `direction="column"`, `wrap="nowrap"`
- `flex={1}`, `flex="0 0 auto"`
**Other**: `pos` (position), `display`
**Responsive syntax**: `px={{ base: "md", md: "lg", lg: rem(48) }}`
### CSS Variable Tokens (use in `.module.css` files)
**Colors** — `var(--mb-color-<name>)`:
- `--mb-color-text-primary`, `--mb-color-text-secondary`, `--mb-color-text-tertiary`
- `--mb-color-background-primary`, `--mb-color-background-secondary`, `--mb-color-background-hover`
- `--mb-color-border`, `--mb-color-border-strong`, `--mb-color-border-subtle`
- `--mb-color-brand`, `--mb-color-brand-hover`
- `--mb-color-error`, `--mb-color-success`, `--mb-color-warning`
- `--mb-color-shadow`, `--mb-color-focus`
- `--mb-color-accent0` through `--mb-color-accent7`
**Mantine spacing** — `var(--mantine-spacing-<size>)`:
- `--mantine-spacing-xs` (4px), `--mantine-spacing-sm` (8px), `--mantine-spacing-md` (16px), `--mantine-spacing-lg` (24px), `--mantine-spacing-xl` (32px)
**Mantine radius** — `var(--mantine-radius-<size>)`:
- `--mantine-radius-xs` (4px), `--mantine-radius-sm` (6px), `--mantine-radius-md` (8px), `--mantine-radius-xl` (40px)
**Mantine font sizes** — `var(--mantine-font-size-<size>)`:
- `--mantine-font-size-xs` (11px), `--mantine-font-size-sm` (12px), `--mantine-font-size-md` (14px), `--mantine-font-size-lg` (17px), `--mantine-font-size-xl` (21px)
**Breakpoints** (in `.module.css` files):
- `@media (--breakpoint-min-sm)` (40em / 640px)
- `@media (--breakpoint-min-md)` (60em / 960px)
- `@media (--breakpoint-min-lg)` (80em / 1280px)
- `@media (--breakpoint-min-xl)` (120em / 1920px)
- Also available: `--breakpoint-max-*` variants
### Snap Hardcoded Literals to Nearest Design Token (CRITICAL)
When migrating, **never carry over hardcoded `rem`/`px` values** from the original Emotion code. Instead, snap them to the nearest design system token. The original values were often arbitrary — the migration is an opportunity to align with the design system.
**Spacing** — snap to the nearest Mantine spacing token:
| Hardcoded value | Nearest token | Style prop | CSS variable |
| --------------------------------- | ------------- | ------------ | ------------------------------ |
| `0.125rem` (2px) | `2px` | `2` (number) | `2px` (keep literal, no token) |
| `0.2rem` (3.2px), `0.25rem` (4px) | **xs** (4px) | `"xs"` | `var(--mantine-spacing-xs)` |
| `0.5rem` (8px) | **sm** (8px) | `"sm"` | `var(--mantine-spacing-sm)` |
| `0.75rem` (12px) | between sm/md | `rem(12)` | `0.75rem` (no exact token) |
| `1rem` (16px) | **md** (16px) | `"md"` | `var(--mantine-spacing-md)` |
| `1.5rem` (24px) | **lg** (24px) | `"lg"` | `var(--mantine-spacing-lg)` |
| `2rem` (32px) | **xl** (32px) | `"xl"` | `var(--mantine-spacing-xl)` |
**Border radius** — snap to the nearest Mantine radius token:
| Hardcoded value | Nearest token | CSS variable |
| ---------------- | ----------------------------- | -------------------------- |
| `0.25rem` (4px) | **xs** (4px) | `var(--mantine-radius-xs)` |
| `0.375rem` (6px) | **sm** (6px) | `var(--mantine-radius-sm)` |
| `0.5rem` (8px) | **md** (8px) | `var(--mantine-radius-md)` |
| `1rem`+ (16px+) | **xl** (40px) or keep literal | `var(--mantine-radius-xl)` |
**Font sizes** — snap to the nearest Mantine font size token:
| Hardcoded value | Nearest token | Style prop | CSS variable |
| -------------------------------- | ------------- | ---------- | ----------------------------- |
| `0.6875rem` (11px) | **xs** (11px) | `fz="xs"` | `var(--mantine-font-size-xs)` |
| `0.75rem` (12px) | **sm** (12px) | `fz="sm"` | `var(--mantine-font-size-sm)` |
| `0.875rem` (14px), `1rem` (16px) | **md** (14px) | `fz="md"` | `var(--mantine-font-size-md)` |
| `1.063rem` (17px) | **lg** (17px) | `fz="lg"` | `var(--mantine-font-size-lg)` |
| `1.3rem` (21px) | **xl** (21px) | `fz="xl"` | `var(--mantine-font-size-xl)` |
**Rules:**
- If the hardcoded value is within ~2px of a token, use the token
- If it falls exactly between two tokens, prefer the smaller one (tighter is safer)
- If no token is close (e.g., `48px` spacing), use `rem(48)` for style props or the literal value in CSS modules
- This applies everywhere: style props, CSS module values, Icon `size` props, etc.
## Common Migration Patterns
### Pattern 1: Static Layout Container → Mantine Component
**Before:**
```tsx
// Component.styled.tsx
export const Container = styled.div`
display: flex;
align-items: center;
gap: 0.5rem;
padding: 1rem;
width: 100%;
`;
```
**After:**
```tsx
<Flex align="center" gap="sm" p="md" w="100%">
```
### Pattern 2: Vertical Stack → Stack Component
**Before:**
```tsx
export const Wrapper = styled.div`
display: flex;
flex-direction: column;
gap: 1.5rem;
padding: 2rem;
`;
```
**After:**
```tsx
<Stack gap="lg" p="xl">
```
### Pattern 3: Dynamic Props → cx() with CSS Module Classes
**Before:**
```tsx
export const Item = styled.div<{ isSelected: boolean; disabled: boolean }>`
color: ${(props) =>
props.disabled ? color("text-tertiary") : color("text-primary")};
background-color: ${(props) =>
props.isSelected ? color("brand") : "transparent"};
cursor: ${(props) => (props.disabled ? "default" : "pointer")};
&:hover {
color: ${(props) => !props.disabled && color("brand")};
}
`;
```
**After (CSS module):**
```css
/* Item.module.css */
.itemRoot {
color: var(--mb-color-text-primary);
cursor: pointer;
&:hover {
color: var(--mb-color-brand);
}
}
.selected {
background-color: var(--mb-color-brand);
}
.disabled {
color: var(--mb-color-text-tertiary);
cursor: default;
&:hover {
color: var(--mb-color-text-tertiary);
}
}
```
**After (TSX):**
```tsx
<Box
className={cx(S.itemRoot, {
[S.selected]: isSelected,
[S.disabled]: disabled,
})}
>
```
### Pattern 4: Extending a Component → Style Props First, CSS Module Only If Needed
When a styled component wraps another component, first check if the styles can be expressed as style props on a Mantine wrapper or on the component itself. Only use a CSS module when properties genuinely need it.
**Before:**
```tsx
export const CardIcon = styled(Icon)`
display: block;
flex: 0 0 auto;
color: var(--mb-color-brand);
`;
export const CardTitle = styled(Ellipsified)`
color: var(--mb-color-text-primary);
font-size: 1rem;
font-weight: bold;
margin-left: 1rem;
`;
```
**After — style props on Mantine wrapper (preferred when ALL properties are expressible):**
```tsx
<Box display="block" flex="0 0 auto" c="brand">
<Icon {...icon} />
</Box>
<Box c="text-primary" fz="md" fw="bold" ml="md">
<Ellipsified>{title}</Ellipsified>
</Box>
```
**After — CSS module (use only when component has pseudo-selectors or non-expressible styles):**
```css
.icon {
color: var(--mb-color-brand);
width: 1.5rem;
height: 1.5rem;
cursor: pointer;
&:hover {
color: var(--mb-color-brand-hover);
}
}
```
```tsx
<Icon className={S.icon} name="warning" />
```
### Pattern 4b: `styled.span` / `styled.p` with Text Styles → `Text` Component
**Before:**
```tsx
export const Label = styled.span`
color: var(--mb-color-text-secondary);
font-weight: bold;
`;
export const Title = styled.span`
color: var(--mb-color-text-primary);
font-size: 1rem;
font-weight: bold;
margin-left: 0.5rem;
`;
```
**After — `Text` component with style props (NO CSS module needed):**
```tsx
<Text component="span" c="text-secondary" fw="bold">
{label}
</Text>
<Text component="span" c="text-primary" fz="md" fw="bold" ml="sm">
{title}
</Text>
```
`Text` renders as `<p>` by default. Use `component="span"` to render inline. All color, typography, and spacing props work directly — no CSS module needed for pure text styling.
### Pattern 5: Keyframes Animation → CSS Module
**Before:**
```tsx
const spin = keyframes`
from { transform: rotate(0deg); }
to { transform: rotate(360deg); }
`;
export const Spinner = styled.div`
animation: ${spin} 1s infinite linear;
`;
```
**After (CSS module):**
```css
@keyframes spin {
from {
transform: rotate(0deg);
}
to {
transform: rotate(360deg);
}
}
.spinner {
animation: spin 1s infinite linear;
}
```
**After (TSX):**
```tsx
<Box className={S.spinner} />
```
### Pattern 6: color() Function → CSS Variables
| Emotion (JS) | CSS Module | Mantine Style Prop |
| ----------------------- | ------------------------------------------------------------ | --------------------------- |
| `color("brand")` | `var(--mb-color-brand)` | `c="brand"` or `bg="brand"` |
| `color("text-primary")` | `var(--mb-color-text-primary)` | `c="text-primary"` |
| `color("border")` | `var(--mb-color-border)` | N/A (use CSS module) |
| `alpha("brand", 0.2)` | `color-mix(in srgb, var(--mb-color-brand), transparent 80%)` | N/A (use CSS module) |
### Pattern 7: Responsive Spacing/Sizing → Responsive Style Props (NOT CSS Media Queries)
When a styled component only changes spacing, sizing, or other style-prop-expressible values at breakpoints, use Mantine's responsive syntax instead of CSS module media queries.
**Before:**
```tsx
export const CaptionRoot = styled.div`
margin-bottom: 1.5rem;
${breakpointMinExtraLarge} {
margin-bottom: 2rem;
}
`;
```
**After — responsive style prop (NO CSS module needed):**
```tsx
<Flex mb={{ base: "lg", xl: "xl" }}>
```
Breakpoint keys: `base` (default), `xs` (40em), `sm` (48em), `md` (60em), `lg` (80em), `xl` (120em).
Only use CSS module `@media` queries when the responsive change involves non-style-prop properties (e.g., `box-shadow`, `border`, `cursor` changes at breakpoints).
### Pattern 8: Dynamic Computed Styles → Inline Styles
Inline styles are allowed **only** for truly dynamic values computed at runtime (e.g., widths, positions, colors from data). Everything else must use Mantine style props or CSS modules.
**Before:**
```tsx
export const Bar = styled.div<{ width: number }>`
width: ${(props) => props.width}%;
`;
```
**After:**
```tsx
<Box style={{ width: `${width}%` }} />
```
### Pattern 9: Core CSS Utility Classes (Discouraged)
**Do NOT introduce new `CS` utility class usage.** Core CSS utilities (`CS` from `metabase/css/core/index.css`) are legacy and discouraged for new code. Prefer Mantine style props or CSS modules instead.
If existing code already uses `CS` classes and you're not migrating that specific code, leave them alone. But when migrating Emotion → Mantine, replace with the proper alternative:
| Instead of `CS.*` | Use |
| ------------------- | ------------------------------------------------------- |
| `CS.overflowHidden` | CSS module: `overflow: hidden` |
| `CS.cursorPointer` | CSS module: `cursor: pointer` |
| `CS.flex1` | Style prop: `flex={1}` |
| `CS.flexNoShrink` | Style prop: `flex="0 0 auto"` |
| `CS.textBrandHover` | CSS module: `&:hover { color: var(--mb-color-brand); }` |
### Pattern 10: Shared Emotion Styles Across Files → Shared CSS Module + cx()
When Emotion exports a shared `css` block (like `animationStyles`) imported by many files, create ONE CSS module with the shared keyframe and a reusable class, then compose via `cx()`.
**Before (Emotion — shared animation used by 13+ skeleton files):**
```tsx
// ChartSkeleton.styled.tsx — defines shared animation
const fadingKeyframes = keyframes`
0% { opacity: 0.0625; }
50% { opacity: 0.125; }
100% { opacity: 0.0625; }
`;
export const animationStyles = css`
opacity: 0.1;
animation: ${fadingKeyframes} 1.5s infinite;
`;
// AreaSkeleton.styled.tsx — consumes it
import { animationStyles } from ".../ChartSkeleton.styled";
export const SkeletonImage = styled.svg`
${animationStyles};
flex: 1 1 0;
margin-top: 1rem;
border-bottom: 1px solid currentColor;
`;
// BarSkeleton.styled.tsx — also consumes it
import { animationStyles } from ".../ChartSkeleton.styled";
export const SkeletonImage = styled.svg`
${animationStyles};
flex: 1 1 0;
margin-top: 1rem;
`;
```
**After — shared CSS module:**
```css
/* ChartSkeleton.module.css */
@keyframes fading {
0% {
opacity: 0.0625;
}
50% {
opacity: 0.125;
}
100% {
opacity: 0.0625;
}
}
.animated {
opacity: 0.1;
animation: fading 1.5s infinite;
}
```
**After — per-component CSS modules with own styles only:**
```css
/* AreaSkeleton.module.css */
.skeletonImage {
flex: 1 1 0;
margin-top: 1rem;
border-bottom: 1px solid currentColor;
}
```
```css
/* BarSkeleton.module.css */
.skeletonImage {
flex: 1 1 0;
margin-top: 1rem;
}
```
**After — component TSX uses cx() to compose:**
```tsx
// AreaSkeleton.tsx
import cx from "classnames";
import ChartSkeletonS from "../ChartSkeleton/ChartSkeleton.module.css";
import S from "./AreaSkeleton.module.css";
const AreaSkeleton = (): JSX.Element => {
return (
<svg
className={cx(ChartSkeletonS.animated, S.skeletonImage)}
xmlns="http://www.w3.org/2000/svg"
viewBox="0 0 371 113"
preserveAspectRatio="none"
>
<path d="..." fill="currentColor" />
</svg>
);
};
```
The shared module owns the `@keyframes` and the `.animated` class. Each consumer composes it with its own module class via `cx()`. No duplication of the keyframe definition.
## Import Template
```tsx
import cx from "classnames"; // only if needed for conditional classes
import { Box, Flex, Group, Stack, Text, rem } from "metabase/ui";
import S from "./ComponentName.module.css"; // only if CSS module is needed
```
## Checklist Before Finishing
- [ ] **MANTINE-FIRST CHECK**: Every CSS property that has a style prop equivalent (`c`, `bg`, `p`, `m`, `fw`, `fz`, `flex`, `gap`, `align`, `w`, `h`, etc.) is expressed as a style prop, NOT in a CSS module
- [ ] `styled.span`/`styled.p` with only color/weight/size are replaced with `Text component="span"` + style props, NOT CSS module classes
- [ ] `styled.div` with only layout props are replaced with `Flex`/`Stack`/`Group`/`Box` + style props, NOT CSS module classes
- [ ] Responsive spacing/sizing uses responsive style props (`mb={{ base: "lg", xl: "xl" }}`), NOT CSS module `@media` queries
- [ ] CSS modules are used ONLY for: pseudo-selectors, box-shadow, border, cursor, overflow, animation, transition, transform, or complex selectors
- [ ] All `@emotion/styled` and `@emotion/react` imports removed from migrated files
- [ ] The `.styled.tsx` file is deleted
- [ ] All imports of the deleted styled file are updated
- [ ] No static inline styles — `style={{ }}` used only for truly dynamic runtime values
- [ ] All colors use design tokens (`c="brand"`, `var(--mb-color-brand)`), not raw hex/rgb
- [ ] All spacing uses design tokens (`p="md"`, `var(--mantine-spacing-md)`), not arbitrary px values — hardcoded `rem`/`px` literals snapped to the nearest token (see mapping table above)
- [ ] All border-radius values use radius tokens (`var(--mantine-radius-md)`), not hardcoded `0.5rem`
- [ ] All font sizes use font-size tokens (`fz="md"`, `var(--mantine-font-size-md)`), not hardcoded `1rem`
- [ ] Layout uses Mantine components (`Flex`, `Stack`, `Group`, `Box`, `Text`) not raw divs/spans with CSS
- [ ] All CSS module class names use camelCase
- [ ] No cascading/descendant selectors targeting raw elements — use direct class assignment instead
- [ ] Component renders identically to the original (visually verify if possible)
## Visual Verification with Screenshots (Optional — Only When Explicitly Asked)
When the user explicitly asks to create before/after screenshots, use the Playwright MCP tools to capture them. This is NOT done by default — only when requested.
Screenshots must be taken in **both light and dark mode** to verify token usage is correct in both themes.
### Switching Color Scheme via API
Toggle between light and dark mode using the Metabase settings API. Use Playwright's `browser_evaluate` or `browser_navigate` with fetch:
```js
// Switch to dark mode
await fetch("/api/setting/color-scheme", {
method: "PUT",
headers: { "Content-Type": "application/json" },
body: JSON.stringify({ value: "dark" }),
});
// Switch to light mode
await fetch("/api/setting/color-scheme", {
method: "PUT",
headers: { "Content-Type": "application/json" },
body: JSON.stringify({ value: "light" }),
});
```
After switching, reload the page for the theme to take effect.
### Process
For each state (**after** = migrated code, **before** = stashed original), capture screenshots in both themes at both viewport widths. This produces **8 screenshots total**.
1. **Take "after" screenshots first** (migrated code is already in place):
- Navigate to the page that renders the migrated components (`browser_navigate`)
- Wait for content to load (`browser_wait_for` with key text, or `textGone: "Loading..."`)
- Close the sidebar if it overlaps content on narrow viewports (`browser_click` the toggle button)
- **Light mode** (set `color-scheme` to `"light"` via API, reload):
- Desktop (1280x900): `after-light-desktop.png`
- Narrow (640x900), close sidebar: `after-light-narrow.png`
- **Dark mode** (set `color-scheme` to `"dark"` via API, reload):
- Desktop (1280x900): `after-dark-desktop.png`
- Narrow (640x900), close sidebar: `after-dark-narrow.png`
2. **Stash changes to capture "before" screenshots**:
- Run `git stash --include-untracked` to temporarily revert to the Emotion version
- Repeat the same 4 screenshots with `before-` prefix:
- `before-light-desktop.png`, `before-light-narrow.png`
- `before-dark-desktop.png`, `before-dark-narrow.png`
- Run `git stash pop` to restore the migrated code
3. **Restore the user's original color scheme** (set back to `"auto"` or whatever it was before).
4. **Display all 8 screenshots** using the `Read` tool so the user can visually compare before/after in both themes at both viewport sizes.
### Key Details
- Save screenshots in the repo root (they will be untracked — user can delete after review)
- The dev server must be running (typically `http://localhost:3000`)
- Use `browser_snapshot` to find element refs when you need to click buttons (e.g., sidebar toggle)
- The greeting text on the home page changes randomly on each load — this is expected, not a regression
- If the page being tested is not the home page, navigate to the correct URL that exercises the migrated components
- Dark mode screenshots are critical for verifying that CSS variable tokens (`var(--mb-color-*)`) are used correctly — hardcoded colors will look wrong in dark mode

View File

@@ -176,8 +176,10 @@
clojure.core.async/to-chan!
clojure.core.async/to-chan!!
metabase.channel.core/send!
metabase.warehouses.models.database/assert-not-h2!
metabase.driver.sql-jdbc.execute/execute-prepared-statement!
metabase.notification.models/create-notification!
metabase.oauth-server.core/reset-provider!
metabase.pulse.send/send-pulse!
metabase-enterprise.transforms-python.python-runner-test/execute!
;; Workspace isolation functions are safe in parallel tests because they operate on
@@ -552,6 +554,7 @@
metabase-enterprise.cache.config-test/with-temp-persist-models hooks.common/with-seven-bindings
metabase-enterprise.dependencies.api-test/with-dependents-test! hooks.common/with-two-bindings
metabase-enterprise.metabot-v3.tools.dependencies-test/with-dependent-transforms! hooks.common/with-two-bindings
metabase-enterprise.metabot.tools.dependencies-test/with-dependent-transforms! hooks.common/with-two-bindings
metabase-enterprise.semantic-search.db.connection/with-migrate-tx hooks.common/let-one-with-optional-value
metabase-enterprise.serialization.api-test/with-serialization-test-data! hooks.common/with-three-bindings
metabase-enterprise.serialization.test-util/with-temp-dpc hooks.toucan2.tools.with-temp/with-temp
@@ -593,6 +596,9 @@
metabase.model-persistence.api-test/with-persistence-setup! hooks.common/with-one-top-level-binding
metabase.model-persistence.api-test/with-setup! hooks.common/with-one-top-level-binding
metabase.model-persistence.test-util/with-persistence-enabled! hooks.metabase.model-persistence.test-util/with-persistence-enabled!
metabase.mq.core/with-queue hooks.common/let-second
metabase.mq.core/with-topic hooks.common/let-second
metabase.mq.core/def-listener! hooks.metabase.mq.impl/def-listener
metabase.models.interface/define-batched-hydration-method hooks.metabase.models.interface/define-hydration-method
metabase.models.interface/define-simple-hydration-method hooks.metabase.models.interface/define-hydration-method
metabase.permissions.core/allow-changing-all-users-group-members hooks.common/do*
@@ -793,7 +799,7 @@
:private-call {:level :off}
:metabase/defsetting-must-specify-export {:level :off}
:metabase/defsetting-namespace {:level :off}
:metabase/test-helpers-use-non-thread-safe-functions {:level :warning} ; more config in `.clj-kondo/config/test-helpers/config.edn
:metabase/test-helpers-use-non-thread-safe-functions {:level :warning} ; more config in `.clj-kondo/config/test-helpers/config.edn`
:discouraged-var
{metabase.driver/database-supports? {:level :off}

File diff suppressed because it is too large Load Diff

View File

@@ -110,6 +110,7 @@
metabase.settings.models.setting-test/test-user-local-allowed-setting!
metabase.settings.models.setting-test/test-user-local-only-setting!
metabase.settings.models.setting.cache/restore-cache!
metabase.settings.models.setting/log-deprecated-env-var-usage!
metabase.settings.models.setting/set!
metabase.settings.models.setting/validate-settings-formatting!
metabase.setup.core/create-token!

View File

@@ -0,0 +1,24 @@
(ns hooks.metabase.mq.impl
(:require
[clj-kondo.hooks-api :as hooks]))
(defn def-listener
"Kondo hook for `mq/def-listener`. Rewrites to a `fn` form so kondo can analyze bindings.
Forms:
(mq/def-listener! :topic/foo [msg] body...)
(mq/def-listener! :queue/bar {:config map} [msgs] body...)"
[{:keys [node]}]
(let [[_ _channel-name & args] (:children node)
;; skip optional config map
args (if (hooks/map-node? (first args))
(rest args)
args)
[bindings & body] args
;; rewrite as: (fn [bindings...] body...)
new-node (hooks/list-node
(list*
(hooks/token-node 'fn)
bindings
body))]
{:node new-node}))

View File

@@ -303,7 +303,6 @@
metabase.util.malli/with [[:default]]
metabase.util.malli/with-api-error-message [[:default]]
metabase.util.namespaces/import-fns [[:default]]
metabase.util.queue-test/await-test-while [[:block 1]]
metabase.util/auto-retry [[:block 1]]
metabase.util/case-enum [[:block 1]]
metabase.util/for-map [[:block 1]]

6
.github/team.json vendored
View File

@@ -1,13 +1,13 @@
{
"teams": [
{
"name": "Querying",
"label": ".Team/Querying",
"name": "Querying Platform",
"label": ".Team/QueryingPlatform",
"members": [
"bshepherdson",
"metamben",
"rileythomp",
"camsual"
"camsaul"
]
},
{

View File

@@ -37,16 +37,16 @@ jobs:
name: Run Eastwood linter
be-linter-eastwood-test:
if: ${{ !inputs.skip }}
if: ${{ !inputs.skip && github.ref_name == 'master' }}
runs-on: ${{ vars.DEFAULT_RUNNER_KEY }}
timeout-minutes: 30
name: Eastwood (test)
name: Eastwood (tests only)
steps:
- uses: actions/checkout@v4
- name: Prepare back-end environment
uses: ./.github/actions/prepare-backend
- run: clojure -X:dev:ee:ee-dev:drivers:drivers-dev:test:eastwood/test
name: Run Eastwood linter on tests
name: Run Eastwood linter on our backend tests
be-tests:
if: ${{ !inputs.skip }}

View File

@@ -1,360 +0,0 @@
name: "Claude Code Bug Fix"
on:
issues:
types:
- labeled
pull_request_review:
types:
- submitted
workflow_dispatch:
inputs:
issue_number:
description: "Issue number to fix"
required: true
type: string
override_employee_check:
description: "Override employee check for issue creator"
required: false
default: false
type: boolean
jobs:
auto-fix-bug:
runs-on: ${{ vars.DEFAULT_RUNNER_KEY }}
timeout-minutes: 30
permissions:
contents: write
issues: write
pull-requests: write
id-token: write
if: |
(github.event_name == 'issues' && github.event.label.name == 'Claude Bug Fix' && github.event.issue.state == 'open') ||
(github.event_name == 'workflow_dispatch') ||
(github.event_name == 'pull_request_review' && github.event.review.state == 'changes_requested' && startsWith(github.event.pull_request.head.ref, 'claude-fix/'))
steps:
- name: Checkout repository
uses: actions/checkout@v4
with:
fetch-depth: 0
token: ${{ secrets.GITHUB_TOKEN }}
- name: Get issue data and check employee status
id: get_issue
uses: actions/github-script@v7
with:
github-token: ${{ secrets.GITHUB_TOKEN }}
script: |
const { getIssueData } = require('${{ github.workspace }}/.github/scripts/get-issue-data.js');
try {
const result = await getIssueData({
context,
github,
manualIssueNumber: '${{ github.event.inputs.issue_number }}'
});
const { issueNumber, issueData, employeeComments, prReviews, triggerType, isIssueAuthorEmployee } = result;
// Check employee status and fail if not authorized
if (!result.isIssueAuthorEmployee && '${{ github.event.inputs.override_employee_check }}' !== 'true') {
core.setFailed(`User ${result.issueData.user.login} is not a Metabase employee. Skipping bug fix automation.`);
}
core.setOutput('issue_number', issueNumber);
core.setOutput('issue_data', JSON.stringify(issueData));
core.setOutput('issue_comments', JSON.stringify(employeeComments));
core.setOutput('pr_reviews', JSON.stringify(prReviews));
core.setOutput('trigger_type', triggerType);
} catch (error) {
core.setFailed(error.message);
}
- name: Find linked Linear issue
id: find_linear
continue-on-error: true
uses: actions/github-script@v7
env:
LINEAR_API_KEY: ${{ secrets.LINEAR_API_KEY }}
ISSUE_NUMBER: ${{ steps.get_issue.outputs.issue_number }}
with:
script: |
const { findLinearIssue } = require('${{ github.workspace }}/.github/scripts/find-linear-issue.js');
const githubIssueUrl = `https://github.com/${context.repo.owner}/${context.repo.repo}/issues/${process.env.ISSUE_NUMBER}`;
const result = await findLinearIssue({
githubIssueUrl,
linearApiKey: process.env.LINEAR_API_KEY
});
if (result.success) {
console.log(`Found Linear issue: ${result.issue}`);
core.setOutput('linear_issue', result.issue);
} else {
console.log(`Could not find Linear issue: ${result.message}`);
}
- name: Set up git configuration
run: |
git config --global user.name "Claude Code[bot]"
git config --global user.email "noreply@anthropic.com"
- name: Prepare issue context and bug fix prompt
id: prepare_prompt
uses: actions/github-script@v7
env:
ISSUE_DATA: ${{ steps.get_issue.outputs.issue_data }}
ISSUE_COMMENTS: ${{ steps.get_issue.outputs.issue_comments }}
PR_REVIEWS: ${{ steps.get_issue.outputs.pr_reviews }}
TRIGGER_TYPE: ${{ steps.get_issue.outputs.trigger_type }}
with:
script: |
const issueData = JSON.parse(process.env.ISSUE_DATA);
const issueComments = JSON.parse(process.env.ISSUE_COMMENTS);
const prReviews = JSON.parse(process.env.PR_REVIEWS);
const triggerType = process.env.TRIGGER_TYPE;
const issueNumber = '${{ steps.get_issue.outputs.issue_number }}';
// Convert issue title to kebab case and create branch name
const issueTitle = issueData.title || 'untitled';
const kebabTitle = issueTitle
.toLowerCase()
.replace(/[^a-z0-9]+/g, '-')
.replace(/^-+|-+$/g, '')
.replace(/-+/g, '-')
.substring(0, 50);
const branchName = `claude-fix/issue-${issueNumber}-${kebabTitle}`;
// Format data for bash environment
const formattedComments = issueComments.length > 0
? issueComments.map(comment =>
`**Comment by @${comment.user.login} on ${comment.created_at}:**\n${comment.body}`
).join('\n\n---\n\n')
: '';
const formattedReviews = prReviews.length > 0
? prReviews.map(review =>
`**Review by @${review.user.login} on ${review.submitted_at} (Changes Requested):**\n${review.body || 'No review comment provided'}`
).join('\n\n---\n\n')
: '';
core.exportVariable('BRANCH_NAME', branchName);
core.exportVariable('ISSUE_NUMBER', issueNumber);
core.exportVariable('ISSUE_TITLE', issueData.title);
core.exportVariable('ISSUE_AUTHOR', issueData.user.login);
core.exportVariable('ISSUE_CREATED', issueData.created_at);
core.exportVariable('ISSUE_BODY', issueData.body || 'No description provided');
core.exportVariable('ISSUE_COMMENTS_FORMATTED', formattedComments);
core.exportVariable('PR_REVIEWS_FORMATTED', formattedReviews);
core.exportVariable('TRIGGER_TYPE', triggerType);
- name: Generate bug fix prompt
run: |
cat << 'EOF' >> $GITHUB_ENV
BUG_FIX_PROMPT<<PROMPT_END
You are helping to fix a bug reported in GitHub issue #${{ env.ISSUE_NUMBER }}.
## Bug Report Details
**Title:** ${{ env.ISSUE_TITLE }}
**Reporter:** ${{ env.ISSUE_AUTHOR }}
**Created:** ${{ env.ISSUE_CREATED }}
**Description:**
${{ env.ISSUE_BODY }}
**Comments:**
${{ env.ISSUE_COMMENTS_FORMATTED }}
EOF
# Add review feedback section only if this is a review trigger
if [ "${{ env.TRIGGER_TYPE }}" = "pull_request_review" ]; then
cat << 'EOF' >> $GITHUB_ENV
## Previous Fix Attempt - Review Feedback (MUST ADDRESS)
Your previous automated fix attempt received the following feedback that needs to be addressed:
${{ env.PR_REVIEWS_FORMATTED }}
EOF
fi
# Add the rest of the prompt (always the same)
cat << 'EOF' >> $GITHUB_ENV
## Instructions
1. **Analyze the issue description and comments** - Understand what the issue is describing
a. Pay particular attention to reproduction steps and if there's an agreed upon fix.
2. **Search the codebase** - Use Grep and Glob tools to find relevant files
3. **Implement a fix** - Use the appropriate write skill based on the file types:
- Use clojure-write for Clojure/ClojureScript files
- Use typescript-write for TypeScript/JavaScript files
- Follow the coding standards and patterns in each skill
4. **Test the fix** - Run appropriate tests as specified in the skills
5. **Verify the fix** - Ensure tests pass and code follows conventions
## Important Guidelines
- Use the appropriate write skill for the language you're working in
- Write failing tests first, then fix them
- Work in small, testable increments
- Don't commit changes - leave that for the workflow
- IMPORTANT: Focus only on fixing the reported bug, don't make unrelated changes
## Scratchpad Directory
A `.claude-scratchpad/` directory has been created for your temporary files:
- Use this directory for ALL temporary verification files. However, prefer using production test files rather than adhoc ones.
- Only modify actual production code in the main repository directories
- The scratchpad will be cleaned up automatically - nothing from it will be committed
Start by understanding the bug and then systematically work toward a solution.
PROMPT_END
EOF
- name: Determine branch strategy
run: |
BRANCH_NAME="${{ env.BRANCH_NAME }}"
if git branch -r --list "origin/${BRANCH_NAME}" | grep -q .; then
echo "Branch already exists: $BRANCH_NAME"
echo "Will push to existing branch and skip PR creation"
echo "BRANCH_EXISTS=true" >> $GITHUB_ENV
else
echo "Creating new branch: $BRANCH_NAME"
echo "Will create new PR"
echo "BRANCH_EXISTS=false" >> $GITHUB_ENV
fi
# Export the branch name
echo "BRANCH_NAME=$BRANCH_NAME" >> $GITHUB_ENV
- name: Create scratchpad directory for Claude
run: |
# Create a scratchpad directory for Claude to use for temporary files
mkdir -p .claude-scratchpad
echo "Scratchpad directory created for Claude's temporary verification files"
- name: Prepare front-end environment
uses: ./.github/actions/prepare-frontend
- name: Prepare back-end environment
uses: ./.github/actions/prepare-backend
- name: Generate GitHub App Token for Claude
uses: actions/create-github-app-token@v1
id: claude-app-token
with:
app-id: ${{ secrets.METABASE_BOT_APP_ID }}
private-key: ${{ secrets.METABASE_BOT_APP_PRIVATE_KEY }}
- name: Generate Bug Fix with Claude Code
uses: anthropics/claude-code-action@v1
with:
anthropic_api_key: ${{ secrets.ANTHROPIC_API_KEY }}
github_token: ${{ steps.claude-app-token.outputs.token }}
prompt: ${{ env.BUG_FIX_PROMPT }}
claude_args: |
--allowedTools "Bash(bun run test-unit*),Bash(bun run lint-eslint-pure),Bash(./bin/mage*),Bash(clojure*),Bash(find*),Bash(ls*),Bash(grep*),Write,Read,Glob,Grep,Edit,MultiEdit"
--disallowedTools "Bash(git commit*),Bash(git push*),Bash(gh*)"
- name: Clean up scratchpad directory
run: |
# Remove the scratchpad directory so its contents don't get committed
rm -rf .claude-scratchpad/
# Remove output file generated by Claude Code action
rm -f output.txt
echo "Scratchpad directory and action output files cleaned up"
- name: Check for changes and analysis
id: check_changes
run: |
# Now check for actual code changes (excluding analysis file)
if [ -n "$(git status --porcelain)" ]; then
echo "has_changes=true" >> $GITHUB_OUTPUT
echo "Changes detected:"
git status --short
else
echo "has_changes=false" >> $GITHUB_OUTPUT
echo "No changes made by Claude."
fi
- name: Get issue team labels and prepare reviewers
if: steps.check_changes.outputs.has_changes == 'true'
id: prepare_reviewers
uses: actions/github-script@v7
env:
ISSUE_DATA: ${{ steps.get_issue.outputs.issue_data }}
with:
script: |
const issueData = JSON.parse(process.env.ISSUE_DATA);
const issueLabels = issueData.labels.map(label => label.name);
const teamLabels = issueLabels.filter(label => label.startsWith('.Team/'));
const issueCreator = issueData.user.login;
core.setOutput('team_labels', teamLabels.join(','));
core.setOutput('reviewers', issueCreator);
- name: Create Pull Request
id: create_pr
if: steps.check_changes.outputs.has_changes == 'true'
uses: peter-evans/create-pull-request@v6
with:
token: ${{ secrets.GITHUB_TOKEN }}
branch: ${{ env.BRANCH_NAME }}
base: master
commit-message: |
Fix: ${{ env.ISSUE_TITLE }}
Fixes #${{ env.ISSUE_NUMBER }}
🤖 Generated with [Claude Code](https://claude.ai/code)
Co-Authored-By: Claude <noreply@anthropic.com>
committer: "Claude Code[bot] <noreply@anthropic.com>"
author: "Claude Code[bot] <noreply@anthropic.com>"
title: "Fix: ${{ env.ISSUE_TITLE }}"
body: |
This PR automatically fixes the bug reported in #${{ env.ISSUE_NUMBER }}.
${{ steps.find_linear.outputs.linear_issue && format('Closes {0}', steps.find_linear.outputs.linear_issue) || '' }}
## Bug Report
**Original Issue:** #${{ env.ISSUE_NUMBER }}
**Reporter:** @${{ env.ISSUE_AUTHOR }}
**Issue Title:** ${{ env.ISSUE_TITLE }}
## Automated Fix
🤖 This fix was generated automatically by Claude Code based on the issue description.
**Please review carefully:**
- [ ] The fix addresses the reported issue
- [ ] Tests pass and code follows conventions
- [ ] No unrelated changes were introduced
- [ ] The solution is appropriate and maintainable
## Next Steps
- Review the changes and test thoroughly
- If the fix looks good, approve and merge
- If issues are found, either fix them or close this PR
---
🤖 Generated with [Claude Code](https://claude.ai/code)
Co-Authored-By: Claude <noreply@anthropic.com>
labels: |
${{ steps.prepare_reviewers.outputs.team_labels }}
automated-fix
ci skip
reviewers: ${{ steps.prepare_reviewers.outputs.reviewers }}
assignees: ${{ github.actor }}
- name: Log existing branch update
if: steps.check_changes.outputs.has_changes == 'true' && env.BRANCH_EXISTS == 'true'
uses: actions/github-script@v7
with:
script: |
console.log('🔄 Updated existing branch: ${{ env.BRANCH_NAME }}');
console.log('The changes have been pushed to the existing PR for this issue.');
- name: Log analysis for no changes
if: steps.check_changes.outputs.has_changes == 'false'
uses: actions/github-script@v7
with:
script: |
console.log('🤖 No code changes were made by Claude.');

View File

@@ -76,7 +76,7 @@ jobs:
with:
ref: ${{ github.event.inputs.commit || github.event.pull_request.head.sha || github.sha }}
- name: Run Trivy vulnerability scanner
uses: aquasecurity/trivy-action@0.29.0
uses: aquasecurity/trivy-action@57a97c7e7821a5776cebc9bb87c984fa69cba8f1
env:
TRIVY_OFFLINE_SCAN: true
repo: ${{ fromJson(needs.get-container-info.outputs[matrix.edition]).repo }}
@@ -85,7 +85,6 @@ jobs:
image-ref: docker.io/${{ env.repo }}:${{ env.tag }}
format: sarif
output: trivy-results.sarif
version: "v0.57.1"
- name: Upload Trivy scan results to GitHub Security tab
uses: github/codeql-action/upload-sarif@v3
with:

View File

@@ -154,7 +154,6 @@ jobs:
services:
mariadb:
image: metabase/mariadb:10.6 # https://github.com/metabase/metabase-mariadb-docker-image
image: ${{ matrix.version.image }}
ports:
- "3306:3306"
env:

View File

@@ -61,13 +61,38 @@ jobs:
- name: Prepare release scripts
uses: ./.github/actions/prepare-release-scripts
- name: Prepare front-end environment
uses: ./.github/actions/prepare-frontend
- name: Prepare Cypress environment
uses: ./.github/actions/prepare-cypress
- name: Run cross-version test
env:
CURRENT_VERSION: ${{ vars.CURRENT_VERSION }}
RESULT_FILE: ${{ runner.temp }}/cross-version-result-${{ matrix.source }}-${{ matrix.target }}.json
run: |
cd cross-version
./test.sh --source "${{ matrix.source }}" --target "${{ matrix.target }}"
- name: Upload failure result
uses: actions/upload-artifact@v4
if: failure()
with:
name: failure-result-${{ matrix.source }}-${{ matrix.target }}
path: ${{ runner.temp }}/cross-version-result-${{ matrix.source }}-${{ matrix.target }}.json
if-no-files-found: ignore
- name: Upload Cypress Artifacts upon failure
uses: actions/upload-artifact@v4
if: failure()
with:
name: cypress-recording-${{ matrix.source }}-${{ matrix.target }}
path: |
./cypress
./logs/test.log
if-no-files-found: ignore
- name: Show logs on failure
if: failure()
run: |
@@ -75,6 +100,84 @@ jobs:
docker compose logs metabase || true
docker compose logs postgres || true
# Notify Slack on nightly failures
notify-on-failure:
if: always() && github.event_name == 'schedule' && needs.test-matrix.result == 'failure'
needs: [test-matrix]
runs-on: ${{ vars.DEFAULT_RUNNER_KEY }}
timeout-minutes: 5
steps:
- uses: actions/checkout@v4
with:
sparse-checkout: |
release
.github
- name: Prepare release scripts
uses: ./.github/actions/prepare-release-scripts
- name: Download failure results
uses: actions/download-artifact@v4
continue-on-error: true
with:
pattern: failure-result-*
path: failure-results
merge-multiple: true
- name: Notify Slack
continue-on-error: true
uses: actions/github-script@v7
env:
SLACK_BOT_TOKEN: ${{ secrets.SLACK_BOT_TOKEN }}
with:
script: |
const fs = require('fs');
const path = require('path');
const { buildFailureResults, sendCrossVersionSlackNotification } = require('${{ github.workspace }}/release/dist/index.cjs');
// Get all failed jobs from the GitHub API
const { owner, repo } = context.repo;
const jobInfo = await github.rest.actions.listJobsForWorkflowRun({
owner,
repo,
run_id: context.runId,
per_page: 100,
});
const failedJobs = jobInfo.data.jobs
.filter(job => job.status === 'completed' && job.conclusion === 'failure' && job.name.startsWith('test-matrix'))
.map(job => ({ name: job.name, url: job.html_url }));
if (failedJobs.length === 0) {
console.log('No failed test-matrix jobs found');
return;
}
// Load artifact data for enrichment (phase)
const resultsDir = 'failure-results';
const artifactData = new Map();
if (fs.existsSync(resultsDir)) {
for (const file of fs.readdirSync(resultsDir)) {
if (!file.endsWith('.json')) continue;
try {
const result = JSON.parse(fs.readFileSync(path.join(resultsDir, file), 'utf8'));
artifactData.set(`${result.source}-${result.target}`, result);
} catch (e) {
console.log(`Skipping malformed result: ${file}`);
}
}
}
const failures = buildFailureResults(failedJobs, artifactData);
const runUrl = `${process.env.GITHUB_SERVER_URL}/${process.env.GITHUB_REPOSITORY}/actions/runs/${context.runId}`;
const slackBotToken = process.env.SLACK_BOT_TOKEN;
if (!slackBotToken) {
console.log('SLACK_BOT_TOKEN is not set; skipping Slack notification.');
return;
}
await sendCrossVersionSlackNotification(failures, runUrl, slackBotToken);
console.log('Slack notification sent');
# Manual test for workflow_dispatch
test-manual:
if: github.event_name == 'workflow_dispatch'
@@ -87,6 +190,12 @@ jobs:
- name: Prepare release scripts
uses: ./.github/actions/prepare-release-scripts
- name: Prepare front-end environment
uses: ./.github/actions/prepare-frontend
- name: Prepare Cypress environment
uses: ./.github/actions/prepare-cypress
- name: Run cross-version test
env:
CURRENT_VERSION: ${{ vars.CURRENT_VERSION }}
@@ -94,6 +203,16 @@ jobs:
cd cross-version
./test.sh --source "${{ inputs.source_version }}" --target "${{ inputs.target_version }}"
- name: Upload Cypress Artifacts upon failure
uses: actions/upload-artifact@v4
if: failure()
with:
name: cypress-recording-${{ inputs.source_version }}-${{ inputs.target_version }}
path: |
./cypress
./logs/test.log
if-no-files-found: ignore
- name: Show logs on failure
if: failure()
run: |

View File

@@ -2,7 +2,7 @@ name: Auto-update documentation (master)
on:
schedule:
- cron: "0 1 * * 1-5" # Every Monday through Friday at 1 AM UTC
- cron: "0 21 * * 1-5" # Every Monday through Friday at 9 PM UTC
workflow_dispatch:
jobs:

View File

@@ -2,7 +2,7 @@ name: Auto-update documentation (release)
on:
schedule:
- cron: "0 2 * * 1-5" # Every Monday through Friday at 2 AM UTC
- cron: "0 22 * * 1-5" # Every Monday through Friday at 10 PM UTC
workflow_dispatch:
inputs:
target_branch:

View File

@@ -31,6 +31,11 @@ on:
type: boolean
required: false
default: true
fail_fast:
description: "Stop running remaining tests after the first failure"
type: boolean
required: false
default: false
jobs:
determine-test-type:
@@ -67,6 +72,7 @@ jobs:
echo '- `grep`: "${{ inputs.grep }}"' >> $GITHUB_STEP_SUMMARY
echo '- `qa_db`: "${{ inputs.qa_db }}"' >> $GITHUB_STEP_SUMMARY
echo '- `enable_network_throttling`: "${{ inputs.enable_network_throttling }}"' >> $GITHUB_STEP_SUMMARY
echo '- `fail_fast`: "${{ inputs.fail_fast }}"' >> $GITHUB_STEP_SUMMARY
echo '' >> $GITHUB_STEP_SUMMARY
echo 'triggered by: @${{ github.event.sender.login }}' >> $GITHUB_STEP_SUMMARY
@@ -144,6 +150,7 @@ jobs:
env:
GREP: ${{ github.event.inputs.grep }}
CYPRESS_BROWSER: ${{ steps.cypress-prep.outputs.chrome-path }}
FAIL_FAST: ${{ inputs.fail_fast }}
run: |
node e2e/runner/run_cypress_ci.js ${{ needs.determine-test-type.outputs.type }} \
--spec '${{ github.event.inputs.spec }}' \

View File

@@ -88,22 +88,30 @@ jobs:
timeout-minutes: 5
steps:
- name: Generate workflow summary
env:
BRANCH_NAME: ${{ github.head_ref || github.ref_name }}
STORY_FILES: ${{ needs.detect-changed-stories.outputs.story_files }}
BURN_IN: ${{ inputs.burn_in || 50 }}
EVENT_NAME: ${{ github.event_name }}
PR_NUMBER: ${{ github.event.pull_request.number }}
PR_ACTOR: ${{ github.event.pull_request.user.login }}
ACTOR: ${{ github.actor }}
run: |
echo '## Loki Stress Test' >> $GITHUB_STEP_SUMMARY
echo '' >> $GITHUB_STEP_SUMMARY
echo '**Branch:** `${{ github.head_ref || github.ref_name }}`' >> $GITHUB_STEP_SUMMARY
echo "**Branch:** \`${BRANCH_NAME}\`" >> $GITHUB_STEP_SUMMARY
echo '' >> $GITHUB_STEP_SUMMARY
echo '**Story files:**' >> $GITHUB_STEP_SUMMARY
echo '${{ needs.detect-changed-stories.outputs.story_files }}' | tr ',' '\n' | while read -r file; do
echo "${STORY_FILES}" | tr ',' '\n' | while read -r file; do
echo "- \`$file\`" >> $GITHUB_STEP_SUMMARY
done
echo '' >> $GITHUB_STEP_SUMMARY
echo '**Burn-in:** ${{ inputs.burn_in || 50 }} runs' >> $GITHUB_STEP_SUMMARY
echo "**Burn-in:** ${BURN_IN} runs" >> $GITHUB_STEP_SUMMARY
echo '' >> $GITHUB_STEP_SUMMARY
if [[ "${{ github.event_name }}" == "pull_request" ]]; then
echo 'Triggered by PR #${{ github.event.pull_request.number }} from @${{ github.event.pull_request.user.login }}' >> $GITHUB_STEP_SUMMARY
if [[ "${EVENT_NAME}" == "pull_request" ]]; then
echo "Triggered by PR #${PR_NUMBER} from @${PR_ACTOR}" >> $GITHUB_STEP_SUMMARY
else
echo 'Triggered manually by @${{ github.actor }}' >> $GITHUB_STEP_SUMMARY
echo "Triggered manually by @${ACTOR}" >> $GITHUB_STEP_SUMMARY
fi
stress-test-loki:

View File

@@ -20,6 +20,7 @@ jobs:
NAMED_MODULES=$(node --input-type=module -e "
import { elements } from './frontend/lint/module-boundaries.mjs';
const named = elements.filter(e => !['shared/other', 'app/misc', 'other'].includes(e.type));
console.log(named.length);
")
NAMED_DIRS=$(node --input-type=module -e "
@@ -29,6 +30,7 @@ jobs:
const m = el.pattern.match(/^frontend\/src\/metabase\/(\w[\w-]*)\//)
if (m) dirs.add(m[1]);
}
for (const d of dirs) console.log(d);
")
TOTAL_DIRS=$(ls -d frontend/src/metabase/*/ | xargs -I{} basename {} | wc -l | tr -d ' ')
NAMED_COUNT=$(echo "$NAMED_DIRS" | grep -c . || true)

View File

@@ -13,32 +13,35 @@ jobs:
(github.event.label.name == 'Priority:P0' || github.event.label.name == 'Priority:P1' || github.event.label.name == '.Escalation') &&
contains(github.event.issue.labels.*.name, '.Team/Querying')
steps:
- name: Setting title
uses: actions/github-script@v7
id: vars
with:
script: |
core.setOutput('issue_title', ${{ toJson(github.event.issue.title) }}.replaceAll(/"/g, '\\"'));
- name: Trigger PagerDuty Incident
env:
ROUTING_KEY: ${{ secrets.QUERYING_PAGERDUTY_INTEGRATION_KEY }}
ISSUE_TITLE: ${{ github.event.issue.title }}
LABEL_NAME: ${{ github.event.label.name }}
ISSUE_NUMBER: ${{ github.event.issue.number }}
ISSUE_URL: ${{ github.event.issue.html_url }}
REPO: ${{ github.repository }}
ALL_LABELS: ${{ join(github.event.issue.labels.*.name, ', ') }}
run: |
payload=$(jq -n \
--arg rk "$ROUTING_KEY" \
--arg dk "github-issue-$ISSUE_NUMBER" \
--arg sum "[$LABEL_NAME] $ISSUE_TITLE" \
--arg src "GitHub Issues - $REPO" \
--arg lbl "$ALL_LABELS" \
--arg url "$ISSUE_URL" \
'{
routing_key: $rk,
event_action: "trigger",
dedup_key: $dk,
payload: {
summary: $sum,
source: $src,
severity: "critical",
custom_details: { Labels: $lbl }
},
links: [{ href: $url, text: "View GitHub Issue" }]
}')
curl -X POST https://events.pagerduty.com/v2/enqueue \
-H 'Content-Type: application/json' \
-d '{
"routing_key": "${{ secrets.QUERYING_PAGERDUTY_INTEGRATION_KEY }}",
"event_action": "trigger",
"dedup_key": "github-issue-${{ github.event.issue.number }}",
"payload": {
"summary": "[${{ github.event.label.name }}] ${{ steps.vars.outputs.issue_title }}",
"source": "GitHub Issues - ${{ github.repository }}",
"severity": "critical",
"custom_details": {
"Labels": "${{ join(github.event.issue.labels.*.name, ', ') }}"
}
},
"links": [
{
"href": "${{ github.event.issue.html_url }}",
"text": "View GitHub Issue"
}
]
}'
-d "$payload"

View File

@@ -0,0 +1,68 @@
name: Release Custom Viz Package
run-name: Release @metabase/custom-viz v${{ inputs.version }} from ${{ inputs.branch }}
on:
workflow_dispatch:
inputs:
branch:
description: "Branch to release from"
type: string
required: true
default: "custom-viz"
version:
description: "Version to publish (e.g. 0.1.0)"
type: string
required: true
npm_tag:
description: "NPM dist-tag"
type: choice
required: true
default: "latest"
options:
- latest
- next
- canary
jobs:
build-and-publish:
runs-on: ubuntu-22.04
timeout-minutes: 20
defaults:
run:
working-directory: frontend/src/custom-viz
steps:
- name: Check out the code
uses: actions/checkout@v4
with:
ref: ${{ inputs.branch }}
- name: Prepare front-end environment
uses: ./.github/actions/prepare-frontend
- name: Setup npm registry auth
uses: actions/setup-node@v4
with:
node-version-file: ".nvmrc"
registry-url: "https://registry.npmjs.org"
- name: Set package version
run: npm version ${{ inputs.version }} --no-git-tag-version
- name: Build package
run: bun run build
- name: Publish to NPM
run: npm publish --tag ${{ inputs.npm_tag }}
env:
NODE_AUTH_TOKEN: ${{ secrets.NPM_RELEASE_TOKEN }}
- name: Generate workflow summary
run: |
tee -a $GITHUB_STEP_SUMMARY << EOF
## Published @metabase/custom-viz
- **Version**: ${{ inputs.version }}
- **Tag**: ${{ inputs.npm_tag }}
- **Branch**: ${{ inputs.branch }}
- **Triggered by**: @${{ github.event.sender.login }}
EOF

View File

@@ -13,13 +13,13 @@ on:
type: string
required: false
default: ""
# schedule:
# - cron: "0 21 * * 1-5" # every weekday at 4 pm EST/5 pm EDT
schedule:
- cron: "0 21 * * 1-5" # every weekday at 4 pm EST/5 pm EDT
jobs:
auto-patch-trigger:
# don't run this workflow on a cron for forks
if: ${{ github.event_name != 'schedule' || github.repository == 'metabase/metabase' }}
if: ${{ github.event_name != 'schedule' || (github.repository == 'metabase/metabase' && vars.ENABLE_AUTO_PATCH == 'true') }}
runs-on: ${{ vars.DEFAULT_RUNNER_KEY }}
timeout-minutes: 5
steps:

View File

@@ -110,7 +110,7 @@ jobs:
sdk-tests:
needs: [uberjar, embedding-sdk-package, files-changed]
if: always()
if: always() && !cancelled()
uses: ./.github/workflows/embedding-sdk.yml
secrets: inherit
with:

View File

@@ -192,6 +192,42 @@ jobs:
./COMMIT-ID
./SHA256.sum
upload-to-cloud:
runs-on: ${{ vars.DEFAULT_RUNNER_KEY }}
needs: tag-uberjar-for-release
timeout-minutes: 15
steps:
- name: Configure AWS credentials
uses: aws-actions/configure-aws-credentials@v4
with:
aws-access-key-id: ${{ secrets.AWS_S3_RELEASE_ACCESS_KEY_ID }}
aws-secret-access-key: ${{ secrets.AWS_S3_RELEASE_SECRET_ACCESS_KEY }}
aws-region: ${{ vars.AWS_REGION }}
- uses: actions/checkout@v4
with:
sparse-checkout: .github
- name: Retrieve release ee uberjar artifact
uses: ./.github/actions/fetch-artifact
with:
commit: ${{ inputs.commit }}
edition: ee
release: "true"
- name: Determine the upload path
uses: actions/github-script@v7
id: version_path
with:
result-encoding: string
script: | # js
const version = '${{ inputs.version }}';
// only EE version
const version_path = 'release/' + version.replace(/^v0\./, "v1.");
console.log("sending to", version_path);
return version_path;
- name: Upload to S3
run: |
aws s3 cp ./metabase.jar s3://${{ secrets.AWS_S3_CLOUD_DEPLOY_BUCKET }}/${{ steps.version_path.outputs.result }}/metabase.jar
run-pre-release-tests:
name: Run pre-release tests for ${{ inputs.version }}
needs: tag-uberjar-for-release

View File

@@ -19,7 +19,7 @@ jobs:
id: vars
with:
script: |
core.setOutput('issue_title', ${{ toJson(github.event.issue.title) }}.replaceAll(/"/g, '\\"'));
core.setOutput('issue_title', JSON.stringify(${{ toJson(github.event.issue.title) }} + '\n' + ${{ toJson(github.event.issue.html_url) }}));
- name: Extracting team name
id: team_name
run: |
@@ -39,7 +39,7 @@ jobs:
"type": "section",
"text": {
"type": "mrkdwn",
"text": "${{ steps.vars.outputs.issue_title }}\n${{ github.event.issue.html_url }}"
"text": ${{ steps.vars.outputs.issue_title }}
}
}
]

3
.gitignore vendored
View File

@@ -96,7 +96,7 @@ coverage-summary.json
junit.xml
pom.xml
pom.xml.asc
profiles.clj
/profiles.clj
target/checksum.txt
xcshareddata
xcuserdata
@@ -176,3 +176,4 @@ CLAUDE.local.md
/resources/python-sources/sqlglot-*.dist-info/
/resources/python-sources/.lock
__pycache__/
.mcp.json

Binary file not shown.

Before

Width:  |  Height:  |  Size: 67 KiB

After

Width:  |  Height:  |  Size: 68 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 67 KiB

After

Width:  |  Height:  |  Size: 68 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 60 KiB

After

Width:  |  Height:  |  Size: 60 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 60 KiB

After

Width:  |  Height:  |  Size: 60 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 66 KiB

After

Width:  |  Height:  |  Size: 67 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 66 KiB

After

Width:  |  Height:  |  Size: 67 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 59 KiB

After

Width:  |  Height:  |  Size: 59 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 59 KiB

After

Width:  |  Height:  |  Size: 59 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 66 KiB

After

Width:  |  Height:  |  Size: 66 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 66 KiB

After

Width:  |  Height:  |  Size: 67 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 59 KiB

After

Width:  |  Height:  |  Size: 59 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 59 KiB

After

Width:  |  Height:  |  Size: 59 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 84 KiB

After

Width:  |  Height:  |  Size: 84 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 78 KiB

After

Width:  |  Height:  |  Size: 77 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 74 KiB

After

Width:  |  Height:  |  Size: 73 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 75 KiB

After

Width:  |  Height:  |  Size: 75 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 90 KiB

After

Width:  |  Height:  |  Size: 90 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 78 KiB

After

Width:  |  Height:  |  Size: 79 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 86 KiB

After

Width:  |  Height:  |  Size: 85 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 72 KiB

After

Width:  |  Height:  |  Size: 72 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 77 KiB

After

Width:  |  Height:  |  Size: 77 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 76 KiB

After

Width:  |  Height:  |  Size: 76 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 75 KiB

After

Width:  |  Height:  |  Size: 76 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 77 KiB

After

Width:  |  Height:  |  Size: 77 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 75 KiB

After

Width:  |  Height:  |  Size: 76 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 72 KiB

After

Width:  |  Height:  |  Size: 72 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 72 KiB

After

Width:  |  Height:  |  Size: 72 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 75 KiB

After

Width:  |  Height:  |  Size: 75 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 83 KiB

After

Width:  |  Height:  |  Size: 84 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 77 KiB

After

Width:  |  Height:  |  Size: 76 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 73 KiB

After

Width:  |  Height:  |  Size: 73 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 74 KiB

After

Width:  |  Height:  |  Size: 75 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 90 KiB

After

Width:  |  Height:  |  Size: 89 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 78 KiB

After

Width:  |  Height:  |  Size: 79 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 85 KiB

After

Width:  |  Height:  |  Size: 85 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 71 KiB

After

Width:  |  Height:  |  Size: 71 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 76 KiB

After

Width:  |  Height:  |  Size: 77 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 75 KiB

After

Width:  |  Height:  |  Size: 75 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 74 KiB

After

Width:  |  Height:  |  Size: 75 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 76 KiB

After

Width:  |  Height:  |  Size: 77 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 74 KiB

After

Width:  |  Height:  |  Size: 75 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 71 KiB

After

Width:  |  Height:  |  Size: 71 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 71 KiB

After

Width:  |  Height:  |  Size: 71 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 75 KiB

After

Width:  |  Height:  |  Size: 74 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 108 KiB

After

Width:  |  Height:  |  Size: 109 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 107 KiB

After

Width:  |  Height:  |  Size: 107 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 127 KiB

After

Width:  |  Height:  |  Size: 128 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 112 KiB

After

Width:  |  Height:  |  Size: 113 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 86 KiB

After

Width:  |  Height:  |  Size: 86 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 85 KiB

After

Width:  |  Height:  |  Size: 85 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 114 KiB

After

Width:  |  Height:  |  Size: 113 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 116 KiB

After

Width:  |  Height:  |  Size: 115 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 114 KiB

After

Width:  |  Height:  |  Size: 114 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 114 KiB

After

Width:  |  Height:  |  Size: 113 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 74 KiB

After

Width:  |  Height:  |  Size: 74 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 10 KiB

After

Width:  |  Height:  |  Size: 10 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 10 KiB

After

Width:  |  Height:  |  Size: 10 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 167 KiB

After

Width:  |  Height:  |  Size: 167 KiB

View File

@@ -6,7 +6,7 @@ For detailed guidance on writing and reviewing code and documentation, see the s
## Clojure
- **[clojure-eval](.claude/skills/clojure-eval/SKILL.md)** - Always use this to evaluate Clojure code, **run tests**, and verify edits/compile. Prefer this over `clojure -X:...` shell commands.
- **[clojure-eval](.claude/skills/clojure-eval/SKILL.md)** - Always use this to evaluate Clojure code, **run tests**, and verify edits/compile. Prefer this over shell commands.
- **[clojure-write](.claude/skills/clojure-write/SKILL.md)** - Clojure/ClojureScript development with REPL-driven workflow and coding conventions
- **[clojure-review](.claude/skills/clojure-review/SKILL.md)** - Clojure/ClojureScript code review guidelines and style enforcement

View File

@@ -42,6 +42,9 @@ FROM eclipse-temurin:21-jre-alpine AS runner
ENV FC_LANG=en-US LC_CTYPE=en_US.UTF-8
# copy certs before the RUN so keytool can import them
COPY bin/docker/DigiCertGlobalRootG2.crt.pem /app/certs/DigiCertGlobalRootG2.crt.pem
# dependencies
RUN apk add -U bash fontconfig curl font-noto font-noto-arabic font-noto-hebrew font-noto-cjk java-cacerts && \
apk upgrade && \
@@ -49,7 +52,6 @@ RUN apk add -U bash fontconfig curl font-noto font-noto-arabic font-noto-hebrew
mkdir -p /app/certs && \
curl https://truststore.pki.rds.amazonaws.com/global/global-bundle.pem -o /app/certs/rds-combined-ca-bundle.pem && \
/opt/java/openjdk/bin/keytool -noprompt -import -trustcacerts -alias aws-rds -file /app/certs/rds-combined-ca-bundle.pem -keystore /etc/ssl/certs/java/cacerts -keypass changeit -storepass changeit && \
curl https://cacerts.digicert.com/DigiCertGlobalRootG2.crt.pem -o /app/certs/DigiCertGlobalRootG2.crt.pem && \
/opt/java/openjdk/bin/keytool -noprompt -import -trustcacerts -alias azure-cert -file /app/certs/DigiCertGlobalRootG2.crt.pem -keystore /etc/ssl/certs/java/cacerts -keypass changeit -storepass changeit && \
mkdir -p /plugins && chmod a+rwx /plugins

12
bb.edn
View File

@@ -303,17 +303,21 @@
:task (task! (doctor/doctor!))}
ci-report
{:doc "Generate a CI failure report for a PR"
:examples [["./bin/mage ci-report" "Check current branch, find its PR, and generate report"]
{:doc "Generate a CI failure report for a PR, branch, or commit"
:examples [["./bin/mage ci-report" "Check current branch (PR or branch CI)"]
["./bin/mage ci-report 68292" "Generate report for PR #68292"]
["./bin/mage ci-report --detailed 68292" "Generate detailed report with full FAIL/ERROR blocks"]
["./bin/mage ci-report source-replacement" "Report for a branch by name"]
["./bin/mage ci-report abc123def" "Report for a specific commit SHA"]
["./bin/mage ci-report --detailed 68292" "Detailed report with full FAIL/ERROR blocks"]
["./bin/mage ci-report --detailed > ci_report.md"
"Save detailed report to markdown file and print progress."]]
:options [["-d" "--detailed" "Include full FAIL/ERROR blocks with expected/actual diffs"]]
:usage-fn (fn [_]
(str "Fetches CI check status and failed job logs from GitHub Actions.\n"
"Parses Trunk test reports to extract test failure details.\n\n"
"With no arguments, looks up the PR for your current git branch.\n\n"
"Accepts a PR number/URL, branch name, or commit SHA.\n"
"With no arguments, uses the current branch's PR if one exists,\n"
"otherwise falls back to the current branch's latest CI run.\n\n"
"Use --detailed to get full test failure output with expected/actual diffs.\n\n"
"Requires GitHub CLI (gh) to be installed and authenticated.\n"
"If you see 'gh: command not found' or auth errors, run: gh auth login"))

View File

@@ -0,0 +1,22 @@
-----BEGIN CERTIFICATE-----
MIIDjjCCAnagAwIBAgIQAzrx5qcRqaC7KGSxHQn65TANBgkqhkiG9w0BAQsFADBh
MQswCQYDVQQGEwJVUzEVMBMGA1UEChMMRGlnaUNlcnQgSW5jMRkwFwYDVQQLExB3
d3cuZGlnaWNlcnQuY29tMSAwHgYDVQQDExdEaWdpQ2VydCBHbG9iYWwgUm9vdCBH
MjAeFw0xMzA4MDExMjAwMDBaFw0zODAxMTUxMjAwMDBaMGExCzAJBgNVBAYTAlVT
MRUwEwYDVQQKEwxEaWdpQ2VydCBJbmMxGTAXBgNVBAsTEHd3dy5kaWdpY2VydC5j
b20xIDAeBgNVBAMTF0RpZ2lDZXJ0IEdsb2JhbCBSb290IEcyMIIBIjANBgkqhkiG
9w0BAQEFAAOCAQ8AMIIBCgKCAQEAuzfNNNx7a8myaJCtSnX/RrohCgiN9RlUyfuI
2/Ou8jqJkTx65qsGGmvPrC3oXgkkRLpimn7Wo6h+4FR1IAWsULecYxpsMNzaHxmx
1x7e/dfgy5SDN67sH0NO3Xss0r0upS/kqbitOtSZpLYl6ZtrAGCSYP9PIUkY92eQ
q2EGnI/yuum06ZIya7XzV+hdG82MHauVBJVJ8zUtluNJbd134/tJS7SsVQepj5Wz
tCO7TG1F8PapspUwtP1MVYwnSlcUfIKdzXOS0xZKBgyMUNGPHgm+F6HmIcr9g+UQ
vIOlCsRnKPZzFBQ9RnbDhxSJITRNrw9FDKZJobq7nMWxM4MphQIDAQABo0IwQDAP
BgNVHRMBAf8EBTADAQH/MA4GA1UdDwEB/wQEAwIBhjAdBgNVHQ4EFgQUTiJUIBiV
5uNu5g/6+rkS7QYXjzkwDQYJKoZIhvcNAQELBQADggEBAGBnKJRvDkhj6zHd6mcY
1Yl9PMWLSn/pvtsrF9+wX3N3KjITOYFnQoQj8kVnNeyIv/iPsGEMNKSuIEyExtv4
NeF22d+mQrvHRAiGfzZ0JFrabA0UWTW98kndth/Jsw1HKj2ZL7tcu7XUIOGZX1NG
Fdtom/DzMNU+MeKNhJ7jitralj41E6Vf8PlwUHBHQRFXGU7Aj64GxJUTFy8bJZ91
8rGOmaFvE7FBcf6IKshPECBV1/MUReXgRPTqh5Uykw7+U0b6LJ3/iyK5S9kJRaTe
pLiaWN0bfVKfjllDiIGknibVb63dDcY3fe0Dkhvld1927jyNxF1WW6LZZm6zNTfl
MrY=
-----END CERTIFICATE-----

View File

@@ -5,6 +5,9 @@ ENV FC_LANG=en-US LC_CTYPE=en_US.UTF-8
ARG GIT_COMMIT_SHA
ENV GIT_COMMIT_SHA=${GIT_COMMIT_SHA}
# copy cert before RUN so keytool can import it
COPY DigiCertGlobalRootG2.crt.pem /app/certs/DigiCertGlobalRootG2.crt.pem
# dependencies
RUN apk add -U bash fontconfig curl font-noto font-noto-arabic font-noto-hebrew font-noto-cjk font-noto-thai java-cacerts && \
apk upgrade && \
@@ -12,7 +15,6 @@ RUN apk add -U bash fontconfig curl font-noto font-noto-arabic font-noto-hebrew
mkdir -p /app/certs && \
curl https://truststore.pki.rds.amazonaws.com/global/global-bundle.pem -o /app/certs/rds-combined-ca-bundle.pem && \
/opt/java/openjdk/bin/keytool -noprompt -import -trustcacerts -alias aws-rds -file /app/certs/rds-combined-ca-bundle.pem -keystore /etc/ssl/certs/java/cacerts -keypass changeit -storepass changeit && \
curl https://cacerts.digicert.com/DigiCertGlobalRootG2.crt.pem -o /app/certs/DigiCertGlobalRootG2.crt.pem && \
/opt/java/openjdk/bin/keytool -noprompt -import -trustcacerts -alias azure-cert -file /app/certs/DigiCertGlobalRootG2.crt.pem -keystore /etc/ssl/certs/java/cacerts -keypass changeit -storepass changeit && \
mkdir -p /plugins && chmod a+rwx /plugins

View File

@@ -5,13 +5,15 @@ ENV FC_LANG=en-US LC_CTYPE=en_US.UTF-8
ARG GIT_COMMIT_SHA
ENV GIT_COMMIT_SHA=${GIT_COMMIT_SHA}
# copy cert before RUN so it's available without a network fetch
COPY DigiCertGlobalRootG2.crt.pem /usr/local/share/ca-certificates/DigiCertGlobalRootG2.crt.pem
# Dependencies
RUN apt-get update && \
apt-get upgrade -y && \
apt-get install -y ca-certificates ca-certificates-java fonts-noto && \
apt-get clean && \
curl https://truststore.pki.rds.amazonaws.com/global/global-bundle.pem -o /usr/local/share/ca-certificates/rds-combined-ca-bundle.pem && \
curl https://cacerts.digicert.com/DigiCertGlobalRootG2.crt.pem -o /usr/local/share/ca-certificates/DigiCertGlobalRootG2.crt.pem && \
update-ca-certificates && \
mkdir -p /plugins && chmod a+rwx /plugins && \
keytool -list -cacerts

View File

@@ -10,15 +10,30 @@
# Usage:
# bin/test-agent :only '[metabase.foo-test]'
# bin/test-agent :only '[metabase.foo-test/some-test]'
# bin/test-agent --oss :only '[metabase.foo-test]' # run without EE extensions
# bin/test-agent # runs all tests (rarely what you want)
#
# Any extra arguments are forwarded directly to `clojure -X:dev:ee:ee-dev:test`.
# Options:
# --oss Run without enterprise extensions (uses :oss:oss-dev instead of :ee:ee-dev)
#
# Any extra arguments are forwarded directly to the clojure test command.
# The :test alias uses port 0 (OS-assigned random port) so parallel runs never collide.
set -euo pipefail
export HAWK_MODE=cli/ci
# Check for --oss flag
ALIASES=":dev:ee:ee-dev:test"
args=()
for arg in "$@"; do
if [[ "$arg" == "--oss" ]]; then
ALIASES=":dev:ci:test:oss:oss-dev"
else
args+=("$arg")
fi
done
# Filter noise and strip ANSI escape sequences
clean_output() {
grep -v '^Downloading: ' \
@@ -28,4 +43,4 @@ clean_output() {
| sed $'s/\033[\[(][0-9;]*[A-Za-z]//g; s/\033[^[]*//g'
}
exec clojure -X:dev:ee:ee-dev:test "$@" 2>&1 | clean_output
exec clojure -X"${ALIASES}" "${args[@]}" 2>&1 | clean_output

View File

@@ -41,7 +41,7 @@
"@react-oauth/google": "^0.11.1",
"@reduxjs/toolkit": "^2.5.0",
"@snowplow/browser-tracker": "^3.1.6",
"@tanstack/react-table": "^8.21.2",
"@tanstack/react-table": "^8.21.3",
"@tanstack/react-virtual": "^3.13.2",
"@tippyjs/react": "^4.2.6",
"@tiptap/core": "^3.2.0",
@@ -1510,7 +1510,7 @@
"@szmarczak/http-timer": ["@szmarczak/http-timer@1.1.2", "", { "dependencies": { "defer-to-connect": "^1.0.1" } }, "sha512-XIB2XbzHTN6ieIjfIMV9hlVcfPU26s2vafYWQcZHWXHOxiaRZYEDKEwdl129Zyg50+foYV2jCgtrqSA6qNuNSA=="],
"@tanstack/react-table": ["@tanstack/react-table@8.21.2", "", { "dependencies": { "@tanstack/table-core": "8.21.2" } }, "sha512-11tNlEDTdIhMJba2RBH+ecJ9l1zgS2kjmexDPAraulc8jeNA4xocSNeyzextT0XJyASil4XsCYlJmf5jEWAtYg=="],
"@tanstack/react-table": ["@tanstack/react-table@8.21.3", "", { "dependencies": { "@tanstack/table-core": "8.21.3" }, "peerDependencies": { "react": ">=16.8", "react-dom": ">=16.8" } }, "sha512-5nNMTSETP4ykGegmVkhjcS8tTLW6Vl4axfEGQN3v0zdHYbK4UfoqfPChclTrJ4EoK9QynqAu9oUf8VEmrpZ5Ww=="],
"@tanstack/react-virtual": ["@tanstack/react-virtual@3.13.2", "", { "dependencies": { "@tanstack/virtual-core": "3.13.2" } }, "sha512-LceSUgABBKF6HSsHK2ZqHzQ37IKV/jlaWbHm+NyTa3/WNb/JZVcThDuTainf+PixltOOcFCYXwxbLpOX9sCx+g=="],
@@ -5718,8 +5718,6 @@
"@svgr/plugin-svgo/deepmerge": ["deepmerge@4.3.1", "", {}, "sha512-3sUqbMEc77XqpdNO7FRyRog+eW3ph+GYCbj+rK+uYyRMuwsVy0rMiVtPn+QJlKFvWP/1PYpapqYn0Me2knFn+A=="],
"@tanstack/react-table/@tanstack/table-core": ["@tanstack/table-core@8.21.2", "", {}, "sha512-uvXk/U4cBiFMxt+p9/G7yUWI/UbHYbyghLCjlpWZ3mLeIZiUBSKcUnw9UnKkdRz7Z/N4UBuFLWQdJCjUe7HjvA=="],
"@testing-library/jest-dom/aria-query": ["aria-query@5.0.0", "", {}, "sha512-V+SM7AbUwJ+EBnB8+DXs0hPZHO0W6pqBcc0dW90OwtVG02PswOu/teuARoLQjdDOH+t9pJgGnW5/Qmouf3gPJg=="],
"@testing-library/jest-dom/chalk": ["chalk@3.0.0", "", { "dependencies": { "ansi-styles": "^4.1.0", "supports-color": "^7.1.0" } }, "sha512-4D3B6Wf41KOYRFdszmDqMCGq5VV/uMAB273JILmO+3jAlh8X4qDtdtgCR3fxtbLEMzSx22QdhnDcJvu2u1fVwg=="],

1
cross-version/.gitignore vendored Normal file
View File

@@ -0,0 +1 @@
.xv-*

View File

@@ -1,55 +1,70 @@
# Cross-Version Migration Testing
Tests Metabase database migrations across version upgrades and downgrades.
Tests that Metabase data survives database migrations across version upgrades and downgrades.
## Quick Start
Uses Cypress e2e tests to create data on a **source** version, then verify it's intact on a **target** version after migration. The application database is PostgreSQL (matching production), and state is preserved between versions via a Docker volume.
## Scripts
### `test.sh` — CI / production runs
Runs the full migration test end-to-end. Starts source, runs `@source` specs, stops it, starts target, runs `@target` specs.
```bash
# Test upgrade (OSS)
./test.sh --source v0.58.6 --target v0.58.7
# Test downgrade (OSS)
./test.sh --source v0.58.7 --target v0.58.6
# Test Enterprise edition (use v1.x.x)
./test.sh --source v1.58.6 --target v1.58.7
./test.sh --source v1.57.6 --target v1.58.3 # upgrade
./test.sh --source v1.58.3 --target v1.57.6 # downgrade
```
Edition is inferred from version prefix: `v0.x.x` = OSS, `v1.x.x` = EE.
We only test Enterprise Edition (`v1.x`) for now. Direction is inferred from version comparison.
## Requirements
For **downgrades**, the script expects the target to refuse startup, then runs cascading `migrate down` (one step per major version) before retrying.
- Docker
- Docker Compose v2
- Bun (for version helpers CLI)
### `dev.sh` — local development
## How It Works
Starts a single Metabase version in Docker with H2 and opens Cypress interactively. Snapshot/restore works via `/api/testing/*` endpoints, controlled by the `CROSS_VERSION_DEV_MODE` env var.
### Upgrade Test
1. Start PostgreSQL container
2. Start SOURCE Metabase version, wait for healthy
3. Stop SOURCE container (PostgreSQL persists)
4. Start TARGET version - migrations run automatically
5. Verify health check passes
```bash
./dev.sh --version v1.57.6
./dev.sh --version v1.58.3 --port 3001
```
### Downgrade Test
1. Start PostgreSQL container
2. Start SOURCE (newer) Metabase version, wait for healthy
3. Stop SOURCE container
4. Start TARGET (older) version - expects refusal
5. Run `migrate down` command
6. Start TARGET version again
7. Verify health check passes
## E2E test specs
## Environment Variables
Specs live in `e2e/cross-version/` and are organized by major version:
```
e2e/cross-version/
56/ # specs for v56
57/ # specs for v57
latest/ # specs for HEAD / newest versions
```
Each spec file uses `@source` and `@target` tags (via `cypress-grep`) to separate setup from verification. Each folder also contains a `helpers.ts` with version-specific utility functions used by the specs.
### When to create a new versioned folder
The `latest/` folder is the working copy — it targets the current development branch and is used for local dev with `dev.sh`. When a Metabase release introduces a breaking change that makes existing specs fail on older versions (e.g., a renamed UI element, a changed API response), copy `latest/` into a new numbered folder (e.g., `58/`). That snapshot covers all future versions until the next breaking change requires a new folder.
In short: versioned folders are frozen snapshots, `latest/` is the living version.
### Spec resolution
Both `test.sh` and `dev.sh` resolve which spec folder to use for a given version:
1. **Exact match** — `e2e/cross-version/{major}/` if it exists
2. **Closest newer** — the lowest numbered folder >= the major version
3. **Fallback** — `e2e/cross-version/latest/`
## Environment variables
| Variable | Default | Description |
|----------|---------|-------------|
| `METABASE_PORT` | 3000 | Port to expose Metabase |
| `HEALTH_TIMEOUT` | 120 | Seconds to wait for health check |
| `METABASE_PORT` | `3000` (`3077` in dev) | Port to expose Metabase |
| `HEALTH_TIMEOUT` | `120` | Seconds to wait for health check |
| `CROSS_VERSION_DEV_MODE` | — | Set automatically by `dev.sh`; enables snapshot/restore helpers in specs |
## CI
## Requirements
The GitHub workflow (`.github/workflows/cross-version.yml`) supports:
- **Manual trigger**: Specify source/target versions
- **Nightly schedule**: Runs predefined version matrix
- Docker and Docker Compose v2
- Bun (for `cli.ts` version helpers)
- Node.js with `bunx` (for Cypress)

167
cross-version/dev.sh Executable file
View File

@@ -0,0 +1,167 @@
#!/usr/bin/env bash
#
# Interactive dev environment for writing cross-version e2e tests.
#
# Starts an older Metabase version in Docker with H2 and opens Cypress.
# Uses H2 with a shared volume so snapshot/restore works via test endpoints.
#
# Resolves which spec folder to use:
# e2e/cross-version/{major}/ if it exists, otherwise e2e/cross-version/latest/
#
# Usage:
# ./dev.sh --version v1.57.6
# ./dev.sh --version v1.58.7 --port 3001
#
set -euo pipefail
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
REPO_ROOT="$(cd "$SCRIPT_DIR/.." && pwd)"
VERSION=""
PORT="${METABASE_PORT:-3077}"
RED='\033[0;31m'
GREEN='\033[0;32m'
NC='\033[0m'
log() { echo -e "${GREEN}[xv-dev]${NC} $*"; }
error() { echo -e "${RED}[xv-dev]${NC} $*" >&2; }
usage() {
cat <<EOF
Usage: $0 --version VERSION [--port PORT]
Options:
--version VERSION Metabase version to run (e.g., v1.57.6, v1.58.7)
--port PORT Port to expose (default: 3077)
--help Show this help
Examples:
$0 --version v1.57.6
$0 --version v1.58.7 --port 3001
EOF
exit 1
}
while [[ $# -gt 0 ]]; do
case $1 in
--version) VERSION="$2"; shift 2 ;;
--port) PORT="$2"; shift 2 ;;
--help) usage ;;
*) error "Unknown option: $1"; usage ;;
esac
done
if [[ -z "$VERSION" ]]; then
error "--version is required"
usage
fi
# CLI wrapper for version helpers
cli() {
bun "$SCRIPT_DIR/cli.ts" "$@"
}
IMAGE=$(cli image "$VERSION")
MAJOR=$(cli major "$VERSION")
H2_DIR="$SCRIPT_DIR/.xv-h2/v${MAJOR}"
LOG_FILE="$SCRIPT_DIR/.xv-metabase.log"
# Resolve spec folder: exact match, then closest older-version folder, then latest
SPECS_DIR="$REPO_ROOT/e2e/cross-version/${MAJOR}"
if [[ ! -d "$SPECS_DIR" ]]; then
closest=""
closest_dist=999
for dir in "$REPO_ROOT"/e2e/cross-version/[0-9]*/; do
v=$(basename "$dir")
(( v < MAJOR )) && continue
dist=$(( v - MAJOR ))
if (( dist < closest_dist )); then
closest_dist=$dist
closest="$dir"
fi
done
if [[ -n "$closest" ]]; then
SPECS_DIR="${closest%/}"
else
SPECS_DIR="$REPO_ROOT/e2e/cross-version/latest"
fi
fi
log "============================================"
log "Cross-Version Dev Environment"
log "============================================"
log "Version: $VERSION"
log "Image: $IMAGE"
log "Port: $PORT"
log "H2 dir: $H2_DIR"
log "Logs: $LOG_FILE"
log "Specs: $SPECS_DIR"
log "============================================"
# Clean up previous session
rm -f "$LOG_FILE"
# Start with a clean app db every time
# H2 files are owned by the Docker container's user, so use Docker to clean up
if [[ -d "$H2_DIR" ]]; then
docker run --rm -v "$(dirname "$H2_DIR"):/data" alpine rm -rf "/data/$(basename "$H2_DIR")"
fi
mkdir -p "$H2_DIR"
cleanup() {
log "Stopping Metabase..."
docker rm -f xv-dev-metabase 2>/dev/null || true
}
trap cleanup EXIT
# Remove any leftover container from a previous run (e.g., after SIGKILL)
docker rm -f xv-dev-metabase 2>/dev/null || true
log "Starting Metabase ${VERSION}..."
docker run -d \
--name xv-dev-metabase \
-p "${PORT}:3000" \
-v "${H2_DIR}:/metabase.db" \
-e MB_ENABLE_TEST_ENDPOINTS=true \
-e MB_DANGEROUS_UNSAFE_ENABLE_TESTING_H2_CONNECTIONS_DO_NOT_ENABLE=true \
"$IMAGE"
# Stream backend logs to file for debugging (docker logs xv-dev-metabase)
docker logs -f xv-dev-metabase > "$LOG_FILE" 2>&1 &
# Create snapshots dir inside the container for /api/testing/snapshot and /api/testing/restore
docker exec xv-dev-metabase sh -c "mkdir -p /e2e/snapshots && chmod 777 /e2e/snapshots"
log "Waiting for Metabase to be ready..."
TIMEOUT=120
START=$(date +%s)
while true; do
ELAPSED=$(( $(date +%s) - START ))
if (( ELAPSED >= TIMEOUT )); then
error "Timed out after ${TIMEOUT}s"
docker logs xv-dev-metabase
exit 1
fi
if curl -sf "http://localhost:${PORT}/api/health" 2>/dev/null | grep -q '"ok"'; then
break
fi
sleep 2
done
log "Metabase is ready at http://localhost:${PORT}"
log "Saving blank snapshot..."
curl -sf -X POST "http://localhost:${PORT}/api/testing/snapshot/blank"
log ""
log "Opening Cypress..."
cd "$REPO_ROOT"
CROSS_VERSION_DEV_MODE=true \
CYPRESS_BASE_URL="http://localhost:${PORT}" \
bunx cypress open \
--e2e \
--config-file "e2e/cross-version/cypress.config.js" \
--config "specPattern=${SPECS_DIR}/**/*.cy.spec.ts"

View File

@@ -11,6 +11,7 @@
set -euo pipefail
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
REPO_ROOT="$(cd "$SCRIPT_DIR/.." && pwd)"
cd "$SCRIPT_DIR"
# Defaults
@@ -18,6 +19,7 @@ SOURCE_VERSION=""
TARGET_VERSION=""
METABASE_PORT="${METABASE_PORT:-3000}"
HEALTH_TIMEOUT="${HEALTH_TIMEOUT:-120}"
RESULT_FILE="${RESULT_FILE:-/tmp/cross-version-result.json}"
# Colors for output
RED='\033[0;31m'
@@ -29,6 +31,17 @@ log() { echo -e "${GREEN}[cross-version]${NC} $*"; }
warn() { echo -e "${YELLOW}[cross-version]${NC} $*"; }
error() { echo -e "${RED}[cross-version]${NC} $*" >&2; }
# Write a structured result file on failure so CI can categorize the failure
write_failure() {
local phase="$1" # "migration" or "e2e"
jq -n \
--arg phase "$phase" \
--arg source "$SOURCE_VERSION" \
--arg target "$TARGET_VERSION" \
'{phase:$phase, source:$source, target:$target}' > "$RESULT_FILE"
log "Wrote failure result to $RESULT_FILE"
}
usage() {
cat <<EOF
Usage: $0 --source VERSION --target VERSION
@@ -66,6 +79,41 @@ cli() {
bun "$SCRIPT_DIR/cli.ts" "$@"
}
run_e2e() {
local phase="$1"
local version="$2"
local major specs_dir
major=$(cli major "$version")
specs_dir="$REPO_ROOT/e2e/cross-version/${major}"
if [[ ! -d "$specs_dir" ]]; then
local closest="" closest_dist=999
for dir in "$REPO_ROOT"/e2e/cross-version/[0-9]*/; do
local v=$(basename "$dir")
(( v < major )) && continue
local dist=$(( v - major ))
if (( dist < closest_dist )); then
closest_dist=$dist
closest="$dir"
fi
done
if [[ -n "$closest" ]]; then
specs_dir="${closest%/}"
log "No exact specs for v${major} — using closest: ${specs_dir##*/}/"
else
specs_dir="$REPO_ROOT/e2e/cross-version/latest"
log "No exact specs for v${major} — falling back to latest/"
fi
else
log "Found exact specs for v${major}"
fi
log "Running @${phase} e2e tests for ${version} from: e2e/cross-version/${specs_dir##*/}/"
CYPRESS_SPEC_PATTERN="${specs_dir}/**/*.cy.spec.ts" \
"$REPO_ROOT/e2e/cross-version/run.sh" --phase "$phase"
}
wait_for_health() {
local timeout="$1"
local start_time=$(date +%s)
@@ -194,6 +242,11 @@ migrate_down_step() {
error "migrate down encountered errors (check logs above)"
return 1
fi
if echo "$output" | grep -q "not rolled back"; then
error "migrate down had changesets that were not rolled back (check logs above)"
return 1
fi
}
# Cascading migrate down: rolls back multiple major versions one at a time
@@ -319,17 +372,25 @@ main() {
if ! wait_for_health "$HEALTH_TIMEOUT"; then
error "❌ SOURCE version ($SOURCE_VERSION) failed health check"
docker compose logs metabase
write_failure "migration"
exit 1
fi
log "✅ SOURCE version ($SOURCE_VERSION) is healthy"
log ""
log "Step 2: Stopping SOURCE version ($SOURCE_VERSION)..."
log "Step 2: Running e2e tests (@source)..."
if ! run_e2e source "$SOURCE_VERSION"; then
write_failure "e2e"
exit 1
fi
log ""
log "Step 3: Stopping SOURCE version ($SOURCE_VERSION)..."
stop_metabase
log ""
log "Step 3: Starting TARGET version ($TARGET_VERSION)..."
log "Step 4: Starting TARGET version ($TARGET_VERSION)..."
start_metabase "$target_image"
if [[ "$direction" == "upgrade" ]]; then
@@ -337,15 +398,24 @@ main() {
if ! wait_for_health "$HEALTH_TIMEOUT"; then
error "❌ TARGET version ($TARGET_VERSION) failed health check after upgrade"
docker compose logs metabase
write_failure "migration"
exit 1
fi
log "✅ UPGRADE successful - TARGET version ($TARGET_VERSION) is healthy"
log ""
log "Step 5: Running e2e tests (@target)..."
if ! run_e2e target "$TARGET_VERSION"; then
write_failure "e2e"
exit 1
fi
else
# Downgrade: should refuse to start, then we run migrate down
if ! check_downgrade_refused; then
error "❌ TARGET version ($TARGET_VERSION) did not properly detect downgrade"
docker compose logs metabase
write_failure "migration"
exit 1
fi
@@ -353,25 +423,32 @@ main() {
stop_metabase
log ""
log "============================================"
log "Step 4: Rolling back database ($SOURCE_VERSION → $TARGET_VERSION)..."
log "============================================"
log "Step 5: Rolling back database ($SOURCE_VERSION → $TARGET_VERSION)..."
if ! cascading_migrate_down "$SOURCE_VERSION" "$TARGET_VERSION"; then
error "❌ migrate down failed"
write_failure "migration"
exit 1
fi
# Try starting again
log ""
log "Step 5: Starting TARGET version ($TARGET_VERSION) after migrate down..."
log "Step 6: Starting TARGET version ($TARGET_VERSION) after migrate down..."
start_metabase "$target_image"
if ! wait_for_health "$HEALTH_TIMEOUT"; then
error "❌ TARGET version ($TARGET_VERSION) failed health check after migrate down"
docker compose logs metabase
write_failure "migration"
exit 1
fi
log "✅ DOWNGRADE successful - TARGET version ($TARGET_VERSION) is healthy after migrate down"
log ""
log "Step 7: Running e2e tests (@target)..."
if ! run_e2e target "$TARGET_VERSION"; then
write_failure "e2e"
exit 1
fi
fi
log ""

View File

@@ -106,6 +106,7 @@
methodical/methodical {:mvn/version "1.0.127"} ; drop-in replacements for Clojure multimethods and adds several advanced features
metosin/malli {:git/url "https://github.com/metabase/malli" ; Data-driven Schemas for Clojure/Script and babashka
:git/sha "b4a2d78647a049870b316753d7aae7fb4ade3ae5"} ; patched to resolve an issue causing OOM
net.carcdr/oidc-provider {:mvn/version "0.6.2"} ; OAuth/OIDC server abstract implementation
net.cgrand/macrovich {:mvn/version "0.2.2"} ; utils for writing macros for both Clojure & ClojureScript
net.clojars.wkok/openai-clojure {:mvn/version "0.23.0"
:exclusions [lambdaisland/uri]} ; OpenAI

View File

@@ -0,0 +1,213 @@
# Toucan2 Model Boundary Enforcement
## What it does
The module system controls which *namespaces* can be imported between modules.
Model boundary enforcement extends this to `:model/X` keywords — controlling
which Toucan2 models each module can reference.
Two optional config keys per module in
`.clj-kondo/config/modules/config.edn`:
| Key | Default | Meaning |
|------------------|---------|---------|
| `:model-exports` | `#{}` | Set of `:model/X` keywords this module allows **others** to use. |
| `:model-imports` | `#{}` | Set of `:model/X` keywords **this module** may use from other modules. |
When omitted, no models are exported/imported (closed by default). `:any`
opens all restrictions.
`:model-imports` also supports `:bypass` for cross-cutting modules that
reference nearly every model (e.g. `cmd`, `enterprise/serialization`). A
bypassed module can reference any model — even unexported ones. Models that
are *only* referenced by bypass modules do not need `:model-exports` entries
(the staleness test will flag them if added).
## Why it exists
Two core ideas:
1. Some models need encapsulation.
2. Some modules should be storage-agnostic.
This makes both enforceable.
**Encapsulation.** Some domain models span multiple tables with complex
invariants. `enterprise/workspaces`, for example, involves `WorkspaceInput`,
`WorkspaceGraph`, `WorkspaceTransform`, and others whose internal relationships
should be managed exclusively by that module. `:model-exports` lets a module
declare a public surface area (e.g. only `#{:model/Workspace
:model/WorkspaceMergeTransform}`) while keeping internal models private.
**Testability and performance.** Restricting model access makes it easier to
reason about where blocking IO happens. Some models are more expensive than
others (large tables, complex hooks). Modules that receive data as arguments
rather than querying models directly are easier to test and easier to reason
about performance-wise.
**Jekyll mode.** In Jekyll mode, artifact definitions live in files on disk
rather than being mutated directly in the app-database. The less a module
depends on direct model access today, the less work needed to make it
compatible. Restricting model imports nudges modules in this direction
incrementally.
**Shared logic across storage backends.** Workspaces use parallel models
(`:model/Transform` / `:model/WorkspaceTransform`) backed by separate tables.
This design may change in future. While it exists, core algorithms like
dependency ordering and query compilation need to work identically for both.
If `transforms-base` restricts its model imports, it *must* be written in a
storage-agnostic way — accepting data as arguments rather than hardcoding model
references. Adding this validation caught a real regression where new code in
`transforms-base` had introduced direct `:model/Transform` references.
## How it works
Model ownership is derived automatically by scanning source files for
`(*/defmethod t2/table-name :model/X ...)` calls (any symbol ending in
`/defmethod`). No manual ownership configuration is needed.
At test time (`metabase.core.modules-test`), the system:
1. Builds a map of `:model/X` to owning module by scanning all source files
2. For each source file, finds all `:model/X` keyword usages
3. Checks each reference against:
- The defining module's `:model-exports` (is this model exported?)
- The using module's `:model-imports` (is this module allowed to use it?)
- The model definition exists somewhere (unknown models are always a violation)
### Violation types
| Type | Meaning |
|-------------------|---------|
| `:not-exported` | The defining module restricts `:model-exports` and this model is not in the set |
| `:not-imported` | The using module restricts `:model-imports` and this model is not in the set |
| `:unknown-model` | The model definition was not found anywhere (always a violation) |
### Exemptions
- `metabase.models.resolution` is exempt (it intentionally references all
models as a registry)
- Test namespaces are excluded by the existing `ignored-namespace-patterns`
- Same-module references are always allowed
## How to use it
### Restricting a module's exports
Add `:model-exports` to the module's config entry:
```clojure
enterprise/workspaces
{:team "Metabot"
:model-exports #{:model/Workspace :model/WorkspaceMergeTransform}
...}
```
Only the listed models can be referenced from other modules. All other models
defined in this module become internal.
For example, if a module exports only `:model/Workspace`, then a use of
`:model/WorkspaceInput` from another module will fail with `:not-exported`.
### Restricting a module's imports
Add `:model-imports` to the module's config entry:
```clojure
transforms-base
{:team "Querying"
:model-imports #{:model/Database :model/DatabaseRouter :model/Field :model/Table}
...}
```
The module can only reference these models from other modules. Any other
`:model/X` keyword in the module's source files will cause a test failure.
Unknown models (not defined anywhere) are also violations.
### Bypassing import restrictions
Some modules are inherently cross-cutting and import nearly every model. Use
`:bypass` instead of an explicit set:
```clojure
cmd
{:team "UX West"
:model-imports :bypass
...}
```
A bypassed module can reference any model, even unexported ones. This means
bypass modules don't drive `:model-exports` — if a model is *only* used
outside its home module by bypass modules, it doesn't need to be exported.
Use sparingly. Prefer explicit sets where feasible.
### Verifying changes
```bash
./bin/test-agent :only '[metabase.core.modules-test]'
```
Or at the REPL:
```clojure
(dev.deps-graph/model-ownership) ;; see which module owns each model
(dev.deps-graph/model-boundary-violations) ;; find any violations
```
The test also validates that configured `:model-exports` / `:model-imports`
only mention known models, that exported models are actually owned by the
module exporting them, and that no stale entries exist (exports not referenced
externally, imports not referenced in the module's source files).
### Computing model boundaries
Use the dev namespace to see what the config should contain:
```clojure
(dev.model-boundary-config/compute-model-boundaries) ;; returns data
(dev.model-boundary-config/update-config!) ;; rewrite config.edn
```
## Limitations and future directions
Each limitation below has a natural extension.
**Keyword scanning isn't airtight.** Code can access the app-database without
`:model/X` keywords — via non-namespaced keywords like `:transform`, raw SQL
through `t2/query`, or table-name strings. The enforcement catches conventional
usage but not determined workarounds. A stronger approach: ban `toucan2.core`
entirely for pure modules via a `:banned-requires` config key, closing all
loopholes at once.
**Binary access control.** A module either can or cannot reference a given
model. A natural refinement: distinguish read-only vs read-write access, which
is particularly relevant for Jekyll mode where reads (from the app-database as
cache) are acceptable but authoritative writes are not.
**No transitive checking.** Only keywords appearing directly in a module's
source files are checked. If module A calls a function in module B that
internally references `:model/Transform`, this is not flagged against A. A
future extension could check transitive model access through module
dependencies, possibly with different restrictions for direct vs transitive
references.
**Static-only.** Static analysis operates at the module level and can't
distinguish code paths within a module. Runtime enforcement could provide
stronger guarantees that certain code paths are pure, complementing the
read-only/read-write distinction above.
## Design decisions
- **Test-time enforcement, not compile-time.** Keeps the implementation simple
(source scanning with rewrite-clj) and avoids adding overhead to normal
development. The tradeoff: violations are only caught when tests run.
- **Closed by default.** Both `:model-exports` and `:model-imports` default
to `#{}`. New modules must explicitly declare their model dependencies.
All existing modules have explicit config entries populated from a scan of
actual usage (see `dev.model-boundary-config`).
- **Ownership is derived, not configured.** Instead of manually mapping models
to modules, ownership is determined by scanning for `t2/table-name` defmethod
calls. Avoids config duplication and stays in sync automatically.

View File

@@ -10,7 +10,7 @@
[metabase.api.routes.common]
[metabase.channel.render.core :as channel.render]
[metabase.channel.urls :as urls]
[metabase.query-processor :as qp]
[metabase.query-processor.core :as qp]
[metabase.query-processor.middleware.permissions :as qp.perms]
[metabase.util.malli.schema :as ms])
(:import

View File

@@ -744,3 +744,179 @@
;; should only include tests for `settings-rest`, `api-routes`, `core`, and the handful of random modules that use
;; `settings-rest` (21 files as of 2025-11-26)
(source-filenames->relevant-test-filenames ["src/metabase/settings_rest/api.clj"]))
;;;; Model boundary enforcement
(mu/defn find-model-keywords :- [:set :keyword]
"Find all `:model/X` keywords referenced in a source file, ignoring comments."
[file]
(try
(let [models (atom #{})]
(loop [stack [(z/of-node (r.parser/parse-file-all file))]]
(when-let [zloc (peek stack)]
(let [stack' (pop stack)
stack' (if-let [right (z/right zloc)]
(conj stack' right)
stack')]
(if (comment-loc? zloc)
(recur stack')
(do
(when (and (= (z/tag zloc) :token)
(keyword? (z/sexpr zloc))
(= "model" (namespace (z/sexpr zloc)))
(Character/isUpperCase ^char (first (name (z/sexpr zloc)))))
(swap! models conj (z/sexpr zloc)))
(recur (if-let [child (z/down zloc)]
(conj stack' child)
stack')))))))
@models)
(catch Throwable e
(throw (ex-info (format "Error scanning model keywords in %s: %s" (str file) (ex-message e))
{:file file}
e)))))
(mu/defn find-model-definitions :- [:set :keyword]
"Find all models with their `t2/table-name` defined in this file."
[file]
(let [models (atom #{})]
(loop [stack [(z/of-node (r.parser/parse-file-all file))]]
(when-let [zloc (peek stack)]
(let [stack' (pop stack)
stack' (if-let [right (z/right zloc)]
(conj stack' right)
stack')]
(if (comment-loc? zloc)
(recur stack')
(do
(when (= (z/tag zloc) :list)
(let [first-child (z/down zloc)]
(when (and first-child
(= (z/tag first-child) :token)
(some-> (:string-value (z/node first-child)) (str/ends-with? "/defmethod")))
(let [second-zloc (z/right first-child)
third-zloc (some-> second-zloc z/right)]
(when (and second-zloc
(= (z/sexpr second-zloc) 't2/table-name)
third-zloc
(keyword? (z/sexpr third-zloc))
(= "model" (namespace (z/sexpr third-zloc))))
(swap! models conj (z/sexpr third-zloc)))))))
(recur (if-let [child (z/down zloc)]
(conj stack' child)
stack')))))))
@models))
(defn model-ownership
"Scan all source files via [[find-model-definitions]], building a map of `:model/X` => module symbol.
The module is derived from the defining namespace via [[module]]."
[]
(into (sorted-map)
(for [file (find-source-files)
:let [ns-symb (-> (ns.file/read-file-ns-decl file)
ns.parse/name-from-ns-decl)
mod (module ns-symb)
models (find-model-definitions file)]
:when mod
model models]
[model mod])))
(def ^:private model-boundary-exempt-namespaces
"Namespaces that are exempt from model boundary checking. These are 'glue' namespaces that intentionally reference
all models."
#{'metabase.models.resolution})
(defn- model-reference-violations
"Return violation types for a single model reference. Pure function — no IO.
When `model-imports` is `:bypass`, the using module is exempt from all model boundary checks —
it may reference any model regardless of whether it is exported."
[model defining-mod model-exports model-imports]
(if (= model-imports :bypass)
[]
(cond-> []
(nil? defining-mod)
(conj :unknown-model)
(and defining-mod
(not= model-exports :any)
(not (contains? model-exports model)))
(conj :not-exported)
(and defining-mod
(not= model-imports :any)
(not (contains? model-imports model)))
(conj :not-imported))))
(defn model-references-by-module
"Scan all source files and build a map of `{module => #{:model/X ...}}` — the set of model keywords
referenced in each module's source files. Exempt namespaces (e.g. `metabase.models.resolution`) are excluded.
Includes all modules (including bypass modules) — callers filter as needed."
[]
(reduce
(fn [acc file]
(try
(let [ns-symb (-> (ns.file/read-file-ns-decl file)
ns.parse/name-from-ns-decl)
mod (module ns-symb)]
(if (and mod (not (contains? model-boundary-exempt-namespaces ns-symb)))
(let [models (find-model-keywords file)]
(if (seq models)
(update acc mod (fnil into (sorted-set)) models)
acc))
acc))
(catch Throwable e
(throw (ex-info (format "Error scanning model references in %s" (str file))
{:file file}
e)))))
(sorted-map)
(find-source-files)))
(defn model-boundary-violations
"Find all model boundary violations across the codebase.
For each source file, checks that:
1. The defining module's `:model-exports` allows the model (`:any` or set containing it)
2. The using module's `:model-imports` allows the model (`:any` or set containing it)
3. The model's definition exists somewhere (`:unknown-model` is always a violation)
Modules with `:model-imports :bypass` are exempt from all checks — they may reference any model,
even unexported ones.
Returns a sequence of violation maps with `:file`, `:module`, `:model`, `:defining-module`, `:violation-type`."
([]
(model-boundary-violations (kondo-config)))
([kondo-config]
(let [ownership (model-ownership)]
(into []
(comp
(mapcat
(fn [file]
(try
(let [ns-symb (-> (ns.file/read-file-ns-decl file)
ns.parse/name-from-ns-decl)
mod (module ns-symb)]
(when (and mod
(not (contains? model-boundary-exempt-namespaces ns-symb)))
(let [model-imports (get-in kondo-config [mod :model-imports] #{})
models (find-model-keywords file)
rel-path (file->path-relative-to-project-root file)]
(for [model models
:let [defining-mod (get ownership model)]
:when (not= defining-mod mod)
:let [model-exports (when defining-mod
(get-in kondo-config [defining-mod :model-exports] #{}))]
violation-type (model-reference-violations
model defining-mod model-exports model-imports)]
{:file rel-path
:module mod
:model model
:defining-module defining-mod
:violation-type violation-type}))))
(catch Throwable e
(throw (ex-info (format "Error checking model boundaries in %s" (str file))
{:file file}
e)))))))
(find-source-files)))))
(comment
(model-ownership)
(model-boundary-violations (kondo-config)))

View File

@@ -0,0 +1,141 @@
(ns dev.model-boundary-config
"Compute `:model-exports` and `:model-imports` for every module.
Usage at the REPL:
(dev.model-boundary-config/compute-model-boundaries)
;; => {:model-exports {module => #{:model/X ...}}
;; :model-imports {module => #{:model/X ...}}}
To update config.edn with computed model boundaries:
(dev.model-boundary-config/update-config!)"
(:require
[clojure.string :as str]
[dev.deps-graph :as deps-graph]
[rewrite-clj.parser :as r.parser]
[rewrite-clj.zip :as z]))
(set! *warn-on-reflection* true)
(defn compute-model-boundaries
"Compute the required `:model-exports` and `:model-imports` for every module.
Returns `{:model-exports {module => sorted-set-of-models}
:model-imports {module => sorted-set-of-models}}`
`:model-exports` for module M = models owned by M that are referenced by at least one non-bypass module.
`:model-imports` for module M = models owned by other modules that M references.
Modules with `:model-imports :bypass` are excluded: they don't need computed imports, and their
references don't drive exports (models used only by bypass modules need not be exported)."
[]
(let [config (deps-graph/kondo-config)
ownership (deps-graph/model-ownership)
module-refs (deps-graph/model-references-by-module)
all-modules (set (keys config))
bypass-modules (into #{}
(keep (fn [[mod mod-config]]
(when (= (:model-imports mod-config) :bypass)
mod)))
config)
;; {:model/X => #{modules that reference it}} — inverted index for fast export lookups
;; Only non-bypass modules drive exports.
model->referencing-modules
(reduce-kv (fn [acc mod models]
(if (contains? bypass-modules mod)
acc
(reduce (fn [acc model]
(update acc model (fnil conj #{}) mod))
acc
models)))
{}
module-refs)]
{:model-exports
(into (sorted-map)
(for [mod all-modules
:let [owned (into #{} (comp (filter (fn [[_ owner]] (= owner mod))) (map key)) ownership)]
:when (seq owned)]
[mod (into (sorted-set)
(for [model owned
:let [refs (get model->referencing-modules model)]
:when (some #(not= % mod) refs)]
model))]))
:model-imports
(into (sorted-map)
(for [mod all-modules
:when (not (contains? bypass-modules mod))
:let [imported (into (sorted-set)
(for [model (get module-refs mod)
:let [defining-mod (get ownership model)]
:when (and defining-mod (not= defining-mod mod))]
model))]
:when (seq imported)]
[mod imported]))}))
(def ^:private config-path ".clj-kondo/config/modules/config.edn")
(def ^:private set-indent
"Indentation for multi-line model sets — aligns with existing config.edn style."
(apply str (repeat 20 \space)))
(defn- model-set-str
"Build a string representation of a sorted set of model keywords.
Short sets (<=3 items) stay on one line; longer sets get one item per line."
[models]
(let [sorted (sort models)]
(if (<= (count sorted) 3)
(str "#{" (str/join " " sorted) "}")
(str "#{" (str/join (str "\n" set-indent) sorted) "}"))))
(defn- find-module-config
"Navigate from `root` to the config map for `module-sym`. Returns a zipper or nil."
[root module-sym]
(let [mods-zloc (-> (z/of-node root)
(z/find-value z/next :metabase/modules)
z/right)]
(z/find-value (z/down mods-zloc) z/right module-sym)))
(defn- find-key-value
"Find the value zipper for keyword `k` inside a map zipper, or nil."
[map-zloc k]
(some-> (z/find-value (z/down map-zloc) z/right k)
z/right))
(defn update-config!
"Compute model boundaries and update config.edn with `:model-exports` and `:model-imports` for all modules.
Only updates keys that already exist in the config — does not add new keys.
Uses rewrite-clj to manipulate the AST directly, preserving formatting."
[]
(let [boundaries (compute-model-boundaries)
root-zloc (z/of-node (r.parser/parse-file-all config-path))
modules-zloc (-> root-zloc
(z/find-value z/next :metabase/modules)
z/right)
module-syms (loop [zloc (z/down modules-zloc), acc []]
(if-not zloc
acc
(recur (some-> zloc z/right z/right)
(conj acc (z/sexpr zloc)))))
updated-root
(reduce
(fn [root module-sym]
(reduce
(fn [root config-key]
(let [mod-cfg (some-> (find-module-config root module-sym) z/right)
val-zloc (when mod-cfg (find-key-value mod-cfg config-key))]
;; Only update keys that already exist and are sets (skip :bypass, :any, etc.)
(if (and val-zloc (set? (z/sexpr val-zloc)))
(let [computed (get-in boundaries [config-key module-sym] #{})]
(if (seq computed)
(z/root (z/replace val-zloc (r.parser/parse-string (model-set-str computed))))
;; Empty — remove the key-value pair
(-> val-zloc z/remove z/remove z/root)))
root)))
root
[:model-exports :model-imports]))
(z/root root-zloc)
module-syms)]
(spit config-path (z/root-string (z/of-node updated-root)))
(println "Updated" config-path "with :model-exports and :model-imports.")))

Some files were not shown because too many files have changed in this diff Show More