This launches a metabase dev repl.
Configurable:
- database type
- database edition
- backend port
- config file
- token
- metabase edition (oss vs ee)
You can just run `./bin/mage dev-repl` and get an interactive CLI that
will ask you about these settings, launch the database, start the
frontend (on a random port), and start Metabase configured to talk to
that database/frontend.
Or you can specify options from the command line, like
`./bin/mage dev-repl -e ee -t all-features -d postgres`.
* Add tests for source-model-card
source-model-card is only called from one function, card-returned-columns.
So we have to test source-model-card (which is private) through card-returned-columns.
This kills all mutants but 3, which have to do with the self-reference check in source-model-card. Is it really
necessary? It is an impossible branch to reach.
* new pickers
implement all pickers
remove test code
remove dead code
type fixes
reorganize
type fixes
data picker plus plus
more fixes
type fixes
fix column test ids
type and testfixes
some e2e test updates and kick off ci again
improved default states
more e2e test updates
test updates
better loading state
lint fixes
better root namespace handling
fix more root collection nonsense
improve backparsing logic
fix closing multiple modals on esc or enter
more e2e test updates
e2e test updates 4
fix (some) backparsing and add tests
fix spacing
e2e test updates 5
more backparse fixes
weird snippet deal
more tenant shenanigans
sweet sweet unit tests
add back NewDashboardDialog
more e2e tests and start NewDashboardDialog unit test
lint fixes
fix infinite render loop 😅
various fixes
bug fixes
search improvements
better cache hitting
dialog unit tests
test fixes
mucho test fixes
* fix conflict resolution errors
* test and type fixes
* bug and test fixes
* New folder button for snippets and transforms, e2e tests
* fix some dashboard cases
* test fixes
* entity picker e2e tests
* much fixo
* more e2e fixes
* fix some flakes
* prettier
* fixes
* more e2e test updates
* fix sdk component tests
* some stabilization
* fix dashboard saving bug
* prettier
* address review comments
* test fixes
* fix sandbox tests
* obey the linter
* cant pass null namespace
* oh types
* timeline
* fix default expanded 😫
---------
Co-authored-by: Kevin Kelleher <kevin.kelleher@metabase.com>
* Experiment with communicating with LLMs in Clojure
* Libor comments
* switch to new openai /v1/responses api, check for structured output support
* support for claude to show how different adapters are
* linter fixes
* update llm config edn
* init chat completion endpoint and env vars
* basic sqlgen BE endpoint
* BE test script
* basic table schema context extraction
* temp hack EE inline entrypoint to use OSS sql gen
* BE OSS async & streaming support
* BE OSS async & streaming support
* structured json response
* dialect instructions and sys prompt mustache
* hacky disable file logging via env var
* improved dialect mapping
* improved table metadata
* fix merge conflict
* abstract out sql generation to a seperate hook
* move MetabotPromptInput and MetabotInlineSQLPrompt components into OSS land
* always register metabot context rather than doing it conditionally in enterprise
* move OSS implementation into a seperate file
* oss implementation first pass
* add setting to know if the feature is enabled or not, optionally show ui based on if it is enabled, consolidate state into a single hook call to fix OSS issues, gets everything working
* clean up
* add test coverage
* revert metabot context changes, they did not end up being needed
* restrict @mentions to tables in OSS and models, tables, questions in EE
* send current sql value and no longer send buffer id
* add anthropic provider
* dumb module resolution workaround to get unit tests working correctly
* if there is only one search model provided preselect it
* tweak sys prompt; add table descriptions
* improved table & field metadata
* tweak model env var
* add openai settings
* fix CSP tiptap nonce console error
* fix oss not having access to default import of use-metabot-sql-suggestion hook in prod builds
* add backend support for source_sql
* OSS SQL Generation v1.6 (#68584)
* update layout
* impl table input bar
* persist state on open / close of inline sql prompt input
* ui improvements, referenced_entities now in req/res body
* support referenced_entities param in /generate-sql; add extract-tables endpoint
* use search, fix some ux/ui issue with dropdown
* fix sizing issue
* simplify generate-sql response format; properly union the referenced entities
* return table model spec in response
* updated backend tests
* improve table selection logic using new endpoint changes w/ richer data
* rate limiting; throttling; http timeouts; max resp tokens
---------
Co-authored-by: Sloan Sparger <sloansparger@gmail.com>
* small spacing / sizing corrections
* fix ee extracting tables when it should not, add test cases
* handle disabeld state and enter press better
* cleanup
* fix modules
* add init; fix kondo
* test fix
* fix modules
* Add snowplow events to oss sqlgen endpoints (#68822)
* Copy token_usage snowplow schema from ai-service
* Add :snowplow/token_usage to metabase.analytics.snowplow/schema->version
* Add basic anthropic cost estimation based on static tables
* Return :usage and :duration-ms from llm.anthropic/chat-completion
* Add snowplow token_usage tracking to oss /generate-sql endpoint
* Add snowplow token_usage tracking to oss /generate-sql-streaming endpoint
* Add snowplow simple_event tracking for sqlgen success and failure events
* Consolidate snowplow tracking tests
* Appease whitespace linter
* Don't use non-api analytics.setting namespace
* Ensure duration_ms is an integer in snowplow events
* Add note about oss instances to hashed_metabase_license_token schema description
Co-authored-by: Thomas Schmidt <thomas@metabase.com>
* Fix snowplow lint by adding old versions of token_usage schemas
The snowplow linter was complaining that these older schemas were missing. These versions had been deleted from the
ai-service repo, so weren't copied over. Restore them from the last version in ai-service prior to being deleted.
---------
Co-authored-by: Thomas Schmidt <thomas@metabase.com>
* Add README reminder to token_usage snowplow schema
See also https://github.com/metabase/ai-service/pull/605
* cleanup streaming
* rm stream test
* pr feedback & test fixes
* test fix
* [Metabot] OSS SQL generation admin UI changes (#68824)
* wip
* load models async, move model input to a select, show error if deprecated model
* move to password input for api key
* handle case where env vars have been set via env vars and can not be changed
* clean up oss view
* remove token checks for deciding to upsell metabot
* be tests for new models endpoint
* fix bug
* fix broken api tests
* ui feedback, token validation, style fixes
* unit tests
* fix cypress
* fix e2e test tag
* type fixes
* condo / eastwood fixes
* fix unit tests
* fix e2e test
* fix stale css var reference
* pr feedback
* fix var
* fix n+1 query; improve metadata fetching
* driver multimethod for dialect instruction resource; review PR cleanups & fixes
* pr feedback
* fix added in driver multimethod
* whitespace lint
* address my own comments
---------
Co-authored-by: Oleksandr Solovyov <o@solovyov.net>
Co-authored-by: Sloan Sparger <sloansparger@gmail.com>
Co-authored-by: appleby <86076+appleby@users.noreply.github.com>
Co-authored-by: Thomas Schmidt <thomas@metabase.com>
Co-authored-by: Sloan Sparger <sloansparger@users.noreply.github.com>
Co-authored-by: Noah Moss <noahbmoss@gmail.com>
* replace moment with dayjs in cljs
* Fix CLJS compilation with DayJS port of metabase.util.time
* fix some tests
* fix time parsing in drill chart
* formatting
* make a better fix
* kondo
* fix sdk test
* indent
* drop moment from locales
* reorder imports
* ran bin/mage cljfmt-all to indent
* Add an ignore for 1 unsorted namespace
- cljfmt makes the namespaces unordered -> kondo complains
* typo
* attampt to move logic to clojurescript
* one more thing to remove business logic from metabase-lib
* address cljs review
---------
Co-authored-by: Braden Shepherdson <braden@metabase.com>
Co-authored-by: Bryan Maass <bryan.maass@gmail.com>
* feat: add library/transforms toggles to modal
In the 'Set up git sync' modal in data studio this adds pre-selected
toggles to sync the library and transforms. This also creates the
library if you select it to sync, but it has not yet been created.
* fixup! push the library/transforms behavior into CollectionList
* fixup! better types to reduce casting
* fixup! missing mock setup
* Update enterprise/frontend/src/metabase-enterprise/remote_sync/components/RemoteSyncAdminSettings/RemoteSyncSettingsForm.tsx
Co-authored-by: Anderson <ands.alves.nunes@gmail.com>
* fixup! separate library sync component
* fixup! prettier
---------
Co-authored-by: Anderson <ands.alves.nunes@gmail.com>
* add fix and test
* dont drop bigquery tables and add test
* use atomic renames when possible
* use helper functions
* use existing wait-for-table
* use shared temp-table-name
* small change
* remove bigquery hack, make more similar to python impl
* address comments
* use random suffix and table name for temp tables
* use table hash
* update temp-table-name
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
* mocked test to verify strategy selection
* fix test
* test description
* remove test
---------
Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
Adds new spec-driven methods to detect writability of remote-sync
controlled methods. These use the existing sync-eligibility settings so
in the future we can straightforwardly apply these settings to
remote-sync controlled models (TODO: follow up to build them for the
other models).
Adds batched-versions of these checks to avoid n+1 queries when loading
can_write for models.
Adds tests of remote-sync checks for snippets.
* Split out more granular 400 vs 404 error codes for metabot tools
* a couple new tests
* fix test
* fix tests, cleanup
* discrete error handling in resolve-datasource
* refactor
* undo some refactor to fix tests
* Add an alerts button on the SDK (#67796)
* Prepare to add alerts to SDK
* Remove unrelated prop from storybook
* Add basic alerts button
* Only show alerts button with the proper flag
* Prevent download button from being shown without proper flag
* use CSS to hide things instead of JS
* Properly position all question buttons
* Use JS to hide empty divs as pure CSS is impossible
* Render alert modal ✌️
* Fix type
* Add todo
* Structure the components better
* Control alert modal state
* Add interactive question storybook
* Make interactive and static question button styles consistent
* Add alerts modal to interactive question
* Make alert button works with custom layouts
* Fix tests
* Fix unit tests
* Fix import order issue
* Remove outdated comment
* Add the same condition to display alert button from the core app
* Polish
* Modify question storybooks to show a question instead of a model
* Hide empty elements more reliably
* Add more conditions to display alert button
* Remove unused `classNames` prop
* Use the same style as the interactive question one
* Remove outdated diff
* Fix alerts button not working in custom layouts
* Remove another unused code
* Add a basic test
* Add unit tests
* fix visibility logic
* Fix E2E tests
* Make assertion on static question tests better
* Improve assertions
* Add interactive question unit tests
* Add e2e test for static question
* Add interactive question e2e tests
* Fix subcomponent
* clarify comment
* Hide other channels than email on the alert modal (#68184)
* Hide channel options on alerts modal
* Add unit tests
* Make the test name clearer
* Fix can't view created notifications the first time (#68197)
* Fix can't view created notifications
* Add tests
* Remove unnecessary return type
* Add a safety guard
* Hide Alert (and download) button on edit mode (#68189)
Hide some buttons on edit mode as it makes no sense there
* Alerts polishing (#68477)
* Misc. improvements
* Make it typesafe
* Add alerts to modular embedding (#68286)
* Add the option in modular embedding wizard
* Support `withAlerts` in modular embedding
* Close the modal after hiding the alert button
* Add E2E tests
* Fix type
* Fix `withAlerts` not working on modular embedding with `drills: false`
* It's 2026!
* 1 plugin to rule 'em all!
* Fix E2E after merging master
* Fix test name
* Add alerts analytics (#68531)
* Add alert setup analytics
* Track withAlerts on modular embedding
* Update snowplow schema
* Fix E2E tests
* Fix schema version
* Remove links from alerts test email (#68547)
* Remove links from alerts template
* Remove links from alert email computed part
* Add tests
* Make embedding util easier to use
* Make alerts and subscriptions payload lighter
* Remove unexpected pprint
* Remove links from alert email via tasks (#68610)
* Save notification card's disable_links when creating
* Add tests
* brand key adjustments
* loki tests
* handle the forgotten brand-dark
* Update Loki Snapshots
* removing brand-hover from dynamic-css-vars
* Adjustments to calendar picker colors
* small adjustments on the permissions pages
* sdk: remove background-hover tests from sdk as dynamic css derives are replaced with color-mix
* stupid spelling
* building shared resources
* Adjusting unit test, loki images
* rebasing adjustments
* Update Loki Snapshots
* small color fixes while I wait for loki
* re-adding highlighted class
---------
Co-authored-by: Metabase Automation <github-automation@metabase.com>
Co-authored-by: Phoomparin Mano <phoomparin@gmail.com>
* Data Analyst CRUD on measures/segments
Data analysts with unrestricted view-data permissions on a table can now
create, update, and delete segments and measures for that table
(previously superuser-only).
`can-read?` was already granted by the `manage-table-metadata` permission.
Add permission tests for `can-write?` and `can-create?` methods.
* feat: enable transform setting on first import
When a we import a remote that has transforms set the
`remote-sync-transforms` setting to true.
Update the remote sync object table clean up to be driven by the spec
configuration.
Add tests showing that transforms are consistently synced and erased
according to the following rules:
1. if the instance has `remote-sync-transforms` off and local transforms
and the remote does not have transforms content they are preserved after
an import
2. if the instance has `remote-sync-transforms` off and local transforms
and the remote has transforms content, the local transforms are
completely replaced by the remote.
3. if the instance has `remote-sync-transforms` on and the remote has
transforms content any local transforms are completely replaced by the
remote.
4. if the instance has `remote-sync-transforms` on has local transforms,
but the remote has no transforms, the local transforms are removed on import.
* fixup! relocate settings update
* fixup! don't let python libraries clash on import
* fixup! transactions for mysql
* feat: transform setting into collection list
Move the setting for syncing transforms into the list of collections
available to remote sync, so conceptually the transforms root collection
looks like a standard syncable collection.
* Clean up FE a little bit
* undo unneeded changes
---------
Co-authored-by: Anderson <anderson@metabase.com>
* temp
* hook up mode switcher in other apps
* Ci issues
* adding get help, fixing e2e tests, other adjustments
* most unit tests
* rename to profile link?
* adjusting unit tests with routing
* partial feedback
* rename profile link
* base current app off routes array
* adjustments
* add non-admin user state, fix unit tests
* PR feedback
* fixing test
* fixing e2e
* only 1 app switcher in admin navbar
* e2e adjustments
In the case statement, the default case did not TIMEZONE() to alter the timezone. The cases with TIMEZONE() returned a timestamp. The default case without TIMEZONE() returned timestamptz. Postgres reconciled the different types by converting all of them to timestamptz. So the timestamp was altered (by TIMEZONE()), but a new UTC timestamp was added (by the cast).
When the post-processor formats the rows, it checks that the metadata for the column has :converted_timezone. If it has that, it is formatted in the converted timezone, and since it's UTC, it will convert it again, a second time, which is incorrect.
The fix here is to check if the returned column is a timestamptz and strip it before formatting. We assume the database has already done the conversion. This avoids the second conversion.
* data studio upsell
* refactor: fix route guards
* fix: lint comments
* fix: data studio layout tests
* fix: remove local stub for api
* fix: moar lint errors
* fix: comments
---------
Co-authored-by: Gabriel Moreno <gabriel@metabase.com>
* fix: clean up style issues in new remote-sync code
Fixes a grab bag of small style/docstring issues I found after the
parent PRs were merged.
* fixup! lint
* fixup! keep the newsnippetpage as always dirty
* Fix migration for data studio
We usually auto-populate some other fields in permissions based on the value of object - perm_value, perm_type, and collection_id.
This obviously doesn't happen in a migration! So set them manually here.
* just roll forward
* Push the analysis of the dependents to the job
Fixes QUE-3001
The analysis findings of dependents are marked as stale (if they exists) and the analysis job picks them up with
higher priority. Also, the job is triggered right after the analysis of the dependency and doesn't stop until all
stale findings are refreshed.
* Fix boolean type in migration
* Set *warn-on-reflection* to true
* Stick to SQL change
* Fix tests
* Remove race condition
* Process review feedback
* fixes
* fix tests
---------
Co-authored-by: Alexander Polyankin <alexander.polyankin@metabase.com>
The revert operation triggers a chain: revert API -> reload card page.
Test was only waiting for the revert API, then sometimes it would manage
to click the History tab *before* the page reload. Then the page reloads
and we're not on the History tab anymore.
Waiting for the card query request to finish means that we've reloaded
the page and ready to roll.
* New endpoint implementations
* add new schemas for new endpoints
* new field values endpoints
* simple search endpoint + bastic test
* WIP - query construction endpoint
* Query construction tests, convert :output errors to real 404
* Basic query execution endpoint that just wraps qp/process-query
* Add query response schema
* Add descriptions to malli specs
* Add more tests, simplify some schemas
* Misc fixes, malli description improvements, try to fix decoding/encoding stuff but will need to revisit
* fix whitespace
* Greatly simplify schema sharing w/existing tool endpoints
* cleanup ns
* Fix some tests
* Fix up metric field values test
* add agent-client test helper, and new agent-api-user-binding-test
* simplify base64 query encoding
* small cleanups
* Address feedback
* add comment
* bump ci
* bump ci
* fix modules
* switch to agent-api feature flag
* Ensure we catch semantic search errors before RRF + add a new prometheus metric
* Revert "Ensure we catch semantic search errors before RRF + add a new prometheus metric"
This reverts commit d972fdf6df.
* Move fallback logic to semantic-search.core to not be metabot-specific, add test
* off grid detection
* woof
* fixing bad counts
* adjusting no data points check
* expose initalUiState in visualizer
* cleaning up dashcard functions
* adjust broken behavior for existing tests
* e2e tests
* PR feedback
* remove buttons to open settings and auto adjust. Adding tests for pinned cards and documents
* removing unneeded code, pr feedback, and adding loki screenshots
* fix silly thing
Add support for syncing PythonLibrary entities (Python transform libraries)
with a Git repository when remote-sync-transforms setting is enabled.
Implements serdes serialization for pythonlibraries using `path` as the
entity id.
Adds FE changes to refresh on dirty state and show changes in the AllChangesView
* Add support for versioned endpoints to deftool
* Create new API namespace + add custom jwt middleware
* add kondo module dependency
* revert the deftool stuff which isn't needed now
* fix lint issues
* fix jwt case insensitivity issue; more structured errors for auth issues
* small jwt cleanups
* Add support for session token exchange
* cleanup, lint, tests
* more test cases, cleanup
* bump ci
* Restructure session logic to use normal middleware, add test for session expiration
* fix up module config
* share more code with jwt sso module + remove some now-redundent tests
* try to fix tests by using existing SSO JWT test setup
* fix whitespace
* fix up module config
1. if the `log-artifact-suffix` is empty, just do
`test-logs-{github-job}`.
2. include a `log-artifact-suffix` argument for the driver tests
This way we don't get 409 conflict errors when uploading logs on test
failures for driver tests.
* refactor: remove avatars and "no owner" fallback from transforms list
* refactor: right-align rows column on data structure
* fix: search_check icon fill color
Say you are doing an implicit join from orders to products in order to summarize by products.category. Previously, with implicit joins, we join with the actual products table instead of a select blah from products wrapper, and so the various fields returned by the join were returned under their original name. However, the code would abbreviate excessively long column names from the joined table and break the query.
That difference was due to add-implicit-fields running before add-implicit-joins. Explicit joins got implicit fields added, while implicit joins didn't get those fields. This pr runs add-implicit-fields a second time after add-implicit-joins, which makes implicit and explicit joins compile to the same (correct) sql.
The test was missing an explicit database selection step which sometimes
resulted in the Save button being disabled.
Also removed the unnecessary sync_schema call — the syncable_schemas
endpoint queries Postgres directly via the driver, so no metadata
sync is needed.
* Do not delete built-in transform tags on sync
I found that some tests were failing intermittently - sometimes
TransformTags were present, and sometimes they were missing. I tracked
it down to this test, which enables and then disables remote syncing
transforms.
The issue is that if we sync transforms (... and transform tags), then
unsync them, then we delete everything that was synced - including the
built-in tags.
We should exclude these from deletion.
* Fix serialization API tests
logs:
```
2026-01-27 15:02:23,442 INFO util.queue :: Starting listener field-value-invalidate with 1 threads 🎧
2026-01-27 15:02:23,448 INFO util.queue :: Starting listener search-index-update with 1 threads 🎧
2026-01-27 15:02:23,448 INFO core.core :: Metabase Initialization COMPLETE
2026-01-27 15:02:23,453 INFO core.core :: Metabase Initialization COMPLETE in 4.8 s (JVM uptime: 57.8 s)
```
metrics:
```
❯ http get :9191/metrics | grep startup
metabase_startup_jvm_to_complete_millis 57764.0
metabase_startup_init_duration_millis 4816.0
```
Note that there two different notions of startup time: from jvm
instantiation, and from before init! was called. This maintains the two
separately so we can diagnose if class load time gets longer
independently of amount of work we do during startup itself
* feat: support remote sync for NativeQuerySnippets
When the Library collection is remote-synced, all NativeQuerySnippets
and their collections (snippets namespace) are automatically tracked
for remote sync. No separate setting is needed - syncing is tied
directly to the Library collection's is_remote_synced status.
Update the FE to use the declarative approach to refresh dirty state
when snippets are modified and update the library dirty state indicator
to turn on when any snippet or snippet collection is modified.
* fix: deletion of snippets/transforms collections
Fixes a bug where collections in the special namespaces were not cleaned
up after changing branches.
* fixup! don't double create entries
* fixup! transaction scope for test on mysql
* Show empty schemas when picking a transform target
* Use listSyncableDatabaseSchemas endpoint in data structure
* Handle empty schemas
* Add repro for metabase#68378
* Move transforms tests to new folder
* Split off transforms reproductions into separate specs
* Better logging around token checks
We used to just log we were checking but not give error responses. So it
would look like a ton of token checks for no real reason.
Now includes error responses if any, and also include logs when engaging
the token check circuit breaker, and when it is "half open", when it
lets one real token check happen every 30 seconds to see if the service
is back.
```
2026-01-23 12:26:47,175 INFO premium-features.token-check :: Checking with the MetaStore to see whether token 'f325...d01a' is valid...
2026-01-23 12:26:47,186 INFO premium-features.token-check :: Reporting Metabase stats: {:embedding-dashboard-count 0, :enabled-embedding-sdk false, :internal-users 1, :embedding-question-count 0, :query_executions_static_embed 0, :query_executions_public_link 0, :domains 1, :metabot-usage-date 2026-01-22, :enabled-embedding-static false, :enabled-embedding-simple false, :metabot-queries 0, :external-users 0, :metabot-tokens 0, :query_executions_simple_embed 0, :enabled-embedding-interactive false, :query_executions_internal 0, :metabot-users 0, :query_executions_sdk_embed 0, :query_executions_interactive_embed 0, :users 1}
2026-01-23 12:26:49,190 INFO premium-features.token-check :: Error checking token: Connect timed out †
2026-01-23 12:26:49,194 INFO premium-features.token-check :: Checking with the MetaStore to see whether token 'f325...d01a' is valid...
2026-01-23 12:26:49,201 INFO premium-features.token-check :: Reporting Metabase stats: {:embedding-dashboard-count 0, :enabled-embedding-sdk false, :internal-users 1, :embedding-question-count 0, :query_executions_static_embed 0, :query_executions_public_link 0, :domains 1, :metabot-usage-date 2026-01-22, :enabled-embedding-static false, :enabled-embedding-simple false, :metabot-queries 0, :external-users 0, :metabot-tokens 0, :query_executions_simple_embed 0, :enabled-embedding-interactive false, :query_executions_internal 0, :metabot-users 0, :query_executions_sdk_embed 0, :query_executions_interactive_embed 0, :users 1}
2026-01-23 12:26:51,205 INFO premium-features.token-check :: Engaging circuit breaker in token check ††
2026-01-23 12:26:51,206 INFO premium-features.token-check :: Error checking token: Connect timed out
2026-01-23 12:27:44,548 INFO premium-features.token-check :: In token check circuit breaker but attempting check
2026-01-23 12:27:44,551 INFO premium-features.token-check :: Checking with the MetaStore to see whether token 'f325...d01a' is valid...
2026-01-23 12:27:44,565 INFO premium-features.token-check :: Reporting Metabase stats: {:embedding-dashboard-count 0, :enabled-embedding-sdk false, :internal-users 1, :embedding-question-count 0, :query_executions_static_embed 0, :query_executions_public_link 0, :domains 1, :metabot-usage-date 2026-01-22, :enabled-embedding-static false, :enabled-embedding-simple false, :metabot-queries 0, :external-users 0, :metabot-tokens 0, :query_executions_simple_embed 0, :enabled-embedding-interactive false, :query_executions_internal 0, :metabot-users 0, :query_executions_sdk_embed 0, :query_executions_interactive_embed 0, :users 1}
2026-01-23 12:27:46,574 INFO premium-features.token-check :: Engaging circuit breaker in token check
2026-01-23 12:27:46,575 INFO premium-features.token-check :: Error checking token: Connect timed out
...
2026-01-23 12:32:50,432 INFO premium-features.token-check :: In token check circuit breaker but attempting check †††
2026-01-23 12:32:50,433 INFO premium-features.token-check :: Checking with the MetaStore to see whether token 'f325...d01a' is valid...
2026-01-23 12:32:50,444 INFO premium-features.token-check :: Reporting Metabase stats: {:embedding-dashboard-count 0, :enabled-embedding-sdk false, :internal-users 1, :embedding-question-count 0, :query_executions_static_embed 0, :query_executions_public_link 0, :domains 1, :metabot-usage-date 2026-01-22, :enabled-embedding-static false, :enabled-embedding-simple false, :metabot-queries 0, :external-users 0, :metabot-tokens 0, :query_executions_simple_embed 0, :enabled-embedding-interactive false, :query_executions_internal 0, :metabot-users 0, :query_executions_sdk_embed 0, :query_executions_interactive_embed 0, :users 1}
2026-01-23 12:32:52,450 INFO premium-features.token-check :: Engaging circuit breaker in token check
2026-01-23 12:32:52,450 INFO premium-features.token-check :: Error checking token: Connect timed out
```
The highlights are now we include:
- † `Error checkign token: Connect timed out` the error response when
appropriate
- †† `Engaging circuit breaker in token check` when we engage the
circuit breaker to no longer do real token checks
- ††† `In token check circuit breaker but attempting check` indicating
we are doing a real one from the circuit breaker to determine if the
service is restored
* Whew. Lots of stuff
add a namespace to the circuit breaker keyword:
`:token-check/circuit-breaker`. Change the keyword holding
`:token-check/app-db-not-ready` to cause. I had used reason but nothing
checked that.
this is because want to log when a token check fails, but not when it is
- failure due to circuit breaker
- failure due to appdb not ready yet.
token checks can happen anywhere and with an enormous frequency. they
fail before the appdb is setup (we can't check settigns yet) and of
course they fail during the circuit breaker opening. that's the point of
the circuit breaker. So we don't have to log these.
We _do_ want to log other failures.
We also want to do some prometheus metrics. Here's a journey:
You previously couldn't import the prometheus analytics namespace
because analytics.settings required appearance.settings. It does so
because a docstring in analytics settings used to include Metabase in
the string. I removed that so now it says "in order to help us
improve". And then things would be smooth sailing.
Except we had an issue.
```
026-01-23 20:50:03,514 INFO analytics.prometheus :: Starting prometheus metrics collector
2026-01-23 20:50:03,539 INFO analytics.prometheus :: Starting prometheus metrics web-server on port 9191
thread229238 we made the system!
thread229238 and now we are done with the lock!
2026-01-23 20:50:03,607 INFO analytics.prometheus :: Starting prometheus metrics web-server on port 9191
2026-01-23 20:50:03,611 ERROR premium-features.token-check :: Error validating token: Failed to initialize Prometheus on port 9,191
2026-01-23 20:50:03,611 INFO analytics.prometheus :: Starting prometheus metrics web-server on port 9191
2026-01-23 20:50:03,612 INFO analytics.prometheus :: Starting prometheus metrics web-server on port 9191
2026-01-23 20:50:03,614 INFO analytics.prometheus :: Starting prometheus metrics web-server on port 9191
2026-01-23 20:50:03,616 INFO analytics.prometheus :: Starting prometheus metrics web-server on port 9191
2026-01-23 20:50:03,617 INFO analytics.prometheus :: Starting prometheus metrics web-server on port 9191
2026-01-23 20:50:03,618 INFO analytics.prometheus :: Starting prometheus metrics web-server on port 9191
2026-01-23 20:50:03,619 INFO analytics.prometheus :: Starting prometheus metrics web-server on port 9191
2026-01-23 20:50:03,619 INFO analytics.prometheus :: Starting prometheus metrics web-server on port 9191
2026-01-23 20:50:03,653 INFO core.core :: Setting up prometheus metrics
```
We have locking around the system but it is starting tons of times! Lots
of debugging, swearing that this can't happen. but then i finally threw
in my favorite trick: `(clojure.repl/pst (Exception. "how did we get
here"))`. And it turns out we are doing this:
1. token check fails, so we call `prometheus/inc!`
2. inc is helpful, sees system isn't yet setup, and sets it up for us
3. Setup metric system
4. setup metrics in system
5. we hit the following code which gets initial values for labels
```clojure
(defn- initial-labelled-metric-values []
(for [metric (keys (methods known-labels))
labels (known-labels metric)]
{:metric metric
:labels labels
:value (initial-value metric labels)}))
```
6. We hit the following search code about which search we have
```clojure
(defmethod analytics/initial-value :metabase-search/engine-active
[_ {:keys [engine]}]
(if (search.engine/supported-engine? (keyword "search.engine" engine))
1 0))
```
7. This takes us to semantic search:
```clojure
(defmethod search.engine/supported-engine? :search.engine/semantic [_]
(try
(supported?)
(catch Exception e
(log/warn e "Semantic search engine not supported")
false)))
```
8. `supported?` here checks to see if you support a particular search
impelementation, which of course depends on _YOUR FEATURES_.
9. Checking features fails, so we should increment our prometheus
counter
10. the system is not yet initialized yet. Let's initialize it. (this is
also step 2)
So I have added a simple `inc-if-initialized!` function. It's not
important if the system is down. It's ok if these events are lost. The
system will be initialized in core anyways.
```
❯ http get :9191/metrics | grep metabase_token
HELP metabase_token_check_attempt_total Total number of token checks. Includes a status label.
TYPE metabase_token_check_attempt_total counter
metabase_token_check_attempt_total{status="failure",} 8.0
```
logs:
```
2026-01-23 22:58:26,668 INFO premium-features.token-check :: Checking with the MetaStore to see whether token '82bf...c3fe' is valid...
2026-01-23 22:58:28,706 INFO premium-features.token-check :: Error checking token: Connect timed out
2026-01-23 22:58:28,750 INFO core.core :: Setting up prometheus metrics
2026-01-23 22:58:28,751 INFO analytics.prometheus :: Starting prometheus metrics collector
2026-01-23 22:58:28,789 INFO premium-features.token-check :: Checking with the MetaStore to see whether token '82bf...c3fe' is valid...
2026-01-23 22:58:30,793 INFO premium-features.token-check :: Error checking token: Connect timed out
2026-01-23 22:58:30,798 INFO analytics.prometheus :: Starting prometheus metrics web-server on port 9191
...
2026-01-23 22:58:53,441 INFO premium-features.token-check :: Checking with the MetaStore to see whether token '82bf...c3fe' is valid... {mb-quartz-job-type=SyncAndAnalyzeDatabase}
...
2026-01-23 22:58:55,445 INFO premium-features.token-check :: Engaging circuit breaker in token check {mb-quartz-job-type=SyncAndAnalyzeDatabase}
2026-01-23 22:58:55,445 INFO premium-features.token-check :: Error checking token: Connect timed out {mb-quartz-job-type=SyncAndAnalyzeDatabase}
```
```
Exception how here??
metabase.analytics.prometheus/setup! (prometheus.clj:584)
metabase.analytics.prometheus/setup! (prometheus.clj:576)
metabase.analytics.prometheus/inc! (prometheus.clj:668)
metabase.analytics.prometheus/inc! (prometheus.clj:658)
metabase.analytics.prometheus/inc! (prometheus.clj:665)
metabase.analytics.prometheus/inc! (prometheus.clj:658)
metabase.premium-features.token-check/error-catching-token-checker/reify--98262 (token_check.clj:431)
metabase.premium-features.token-check/check-token (token_check.clj:488)
metabase.premium-features.token-check/check-token (token_check.clj:484)
metabase.premium-features.token-check/check-token (token_check.clj:486)
metabase.premium-features.token-check/check-token (token_check.clj:484)
metabase.premium-features.token-check/eval98325/*token-features*98348--98349 (token_check.clj:534)
metabase.premium-features.token-check/eval98325/fn--98353 (token_check.clj:529)
metabase.premium-features.token-check/has-feature? (token_check.clj:575)
metabase.premium-features.token-check/has-feature? (token_check.clj:569)
clojure.lang.Var.invoke (Var.java:386)
metabase.premium-features.defenterprise/check-feature (defenterprise.clj:41)
metabase.premium-features.defenterprise/check-feature (defenterprise.clj:35)
metabase.premium-features.defenterprise/dynamic-ee-oss-fn/fn--71956 (defenterprise.clj:53)
metabase.search.semantic.core/eval180600/fn--180601 (core.clj:58)
clojure.lang.MultiFn.invoke (MultiFn.java:229)
metabase.search.core/eval139302/fn--139304 (core.clj:77)
clojure.lang.MultiFn.invoke (MultiFn.java:234)
metabase.analytics.prometheus/initial-labelled-metric-values/iter--97802--97808/fn--97809/iter--97804--97810/fn--97811 (prometheus.clj:521)
clojure.lang.LazySeq.force (LazySeq.java:50)
clojure.lang.LazySeq.realize (LazySeq.java:89)
clojure.lang.LazySeq.seq (LazySeq.java:106)
clojure.lang.RT.seq (RT.java:555)
clojure.core/seq--5488 (core.clj:139)
clojure.core/concat/fn--5579 (core.clj:727)
clojure.lang.LazySeq.force (LazySeq.java:50)
clojure.lang.LazySeq.realize (LazySeq.java:89)
clojure.lang.LazySeq.seq (LazySeq.java:106)
clojure.lang.Cons.next (Cons.java:41)
clojure.lang.RT.next (RT.java:733)
clojure.core/next--5472 (core.clj:64)
clojure.core/next--5472 (core.clj:64)
metabase.analytics.prometheus/setup-metrics! (prometheus.clj:546)
metabase.analytics.prometheus/setup-metrics! (prometheus.clj:533)
metabase.analytics.prometheus/make-prometheus-system (prometheus.clj:60)
metabase.analytics.prometheus/make-prometheus-system (prometheus.clj:55)
metabase.analytics.prometheus/setup!/fn--97866 (prometheus.clj:591)
metabase.analytics.prometheus/setup! (prometheus.clj:590)
metabase.analytics.prometheus/setup! (prometheus.clj:576)
metabase.analytics.prometheus/inc! (prometheus.clj:668)
metabase.analytics.prometheus/inc! (prometheus.clj:658)
metabase.analytics.prometheus/inc! (prometheus.clj:665)
metabase.analytics.prometheus/inc! (prometheus.clj:658)
metabase.premium-features.token-check/error-catching-token-checker/reify--98262 (token_check.clj:431)
metabase.premium-features.token-check/check-token (token_check.clj:488)
```
* module config fixup
There's apparently an instance on cloud that is trying to remove 70k queries from the cache at once, and postgres
doesn't allow 70k params to a query. Batching the deletes in groups of 1000 should fix that.
* Rename file containing jsx to .jsx
* Add e2e helpers to build metadata provider
* Split up metadata helpers
* WIP
* Add more direct createTestQuestion helper
* Replace test
* Build cljs for tests
* Export type
* Only require entities in the state for getMetadata
* Just pass state to getMetadata
* Move EntitiesSchema to metabase/schema
* Wrap mbql e2e helpers in promise
* Split off getMetadataWithoutSettings
* Use getMetadataWithoutSettings to get metadata from state
* Build entities state manually
* Simplify wrapper
* Fix metadataProvider
* Add createTestJsQuery wrapper for cypress
* Use plain promises and fetch
* Allow passing GetMetadataOpts to createTest(Native)JsQuery directly
* Improve createTestQuestion types
* Make use of createTestJsQuery in test
* Reduce nest in createTestQuestion
* Use simplified createTestQuestion
* Split up mbql helpers
* Fix card metadata call
* Use correct card endpoint
* Pass missing options
* Convert another more complex test to createTestQuestion
* Fix types
* Use static ids for defaults to avoid import loop
* Fetch more tables by default
* Build cljs in embedding tests
* Build cljs in component tests
* Prepare backend env
* Add plugin so we can avoid building CLJS everywher
* Avoid building cljs where not needed
* Fix types
* Rename query helpers
* Split up helpers
* Compile cljs in stress tests
Update E2E test timings with averaged data
Collected timing data from 7 days of test runs.
All specs with timing data were included (simple averages).
🤖 Generated automatically by update-e2e-timings workflow
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: GitHub Actions <actions@github.com>
* Respect date_abbreviate when formatting date
* Respect abbreviate setting in month format too
* Add unit tests for abbreviated formats
* Add unit test for empty formatting settings
* Add repro for metabase#68574
* Add formatting settings unit tests
The existing transform dependencies code only worked with query-based transforms (mbql or native). This extends that
code to work with python transforms as well.
* Read-only: Adjust transform header menu options / Remove edit definition button / Run tab updates
* Make Settings tab read-only
* fix remote sync setup in e2e test
* Do not allow creating a new transform
* Make Python library not editable
* Hide edit definition in mbql transform view page
* Fix e2e
* Group e2e tests to improve performance
Closes DEV-1289
### Description
Within the last day or two, `metabase.query-processor-test.remapping-test/self-referencing-test` began failing on MySQL Latest. I hypothesize this has to do with the [latest release](https://dev.mysql.com/doc/relnotes/mysql/9.6/en/news-9-6-0.html) fixing an unknown bug around batch inserts in InnoDB and the deprecation of a recusive [sic] fk-related flag.
The error is thrown not in the test conditions, but in the test setup -- MySQL apparently now checks foreign key constraints row-per-row even on a batch insert.
This adds a flag that the test can set which changes the test data-loading behavior to disable the FK check before insertion and re-enable it after.
* fix(UXW-2752): remove unwanted padding on data structure fields tab group
* fix(UXW-2707): remove double border on job transforms section
* feat(UXW-2718): add horizontal layout for schedule controls
* feat(UXW-2709): fix spacing in the segments empty state
* fix(UXW-2780): long text overflow in user input
* fix(UXW-2710): set proper placeholder text on empty python library first load
* feat(UXW-2767): rename 'New collection' to 'Transform folder'
* refactor: blow grid & use `Ellipsified` instead
Closes QUE-2984.
Adds an event handler for when a DWH sync is completed, which checks all its tables and fields for
their latest update times, and re-analyzes all cards which depend on tables updated since the last analysis.
* feat: support remote-syncing measures
Add remote-sync support for measures, following the same pattern as segments:
- Add :model/Measure spec to remote-sync-specs with hybrid identity,
parent-table eligibility, and event tracking
- Update measure permissions to check remote-sync/table-editable? for
can-write? and can-create? (replacing superuser-only derive)
- Add batched hydration for :can_write on measures
- Add measures to serdes/descendants for Table so they export with parent
- Add tests for measure import/export tracking in impl_test.clj
On the Frontend creates a new data-driven approach to setting up the
middleware listener for changes that cause us to refresh dirty-state.
* fixup! typecheck
* Hoist macaw impl from query analyzer
* Use macaw-opts in bigquery module
* Use macaw-options in sqlserver
* Use macaw-options in sql driver
* Remove require
* Comment
* Test
* Update test
* Update macaw version
* Best effort support for bigquery
* Make test driver specific
* Remove final-not-reserved
* Not allowed on clickhouse only
* Clickhouse exception test
* best effort opts on api
* Remove final-non-reserved
* Remove requires
* driver.u/parsed-query
* Use parsed-query
* parsed-query
* ch test
* throw
* Lint against incorrect Macaw usage (#68568)
* Remove now unused function
* Shutdown loud exceptions for now
* Shutdown ch test
* Update test
---------
Co-authored-by: Chris Truter <crisptrutski@users.noreply.github.com>
Fixes https://github.com/metabase/metabase/issues/68578
This issue is not about the stack overflow per-se but blowing up on a
string includes:
```javascript
{
"app": "metabase",
"time": "2026-01-22T12:08:53.660601485Z",
"stream": "stderr",
"app_version": "v1.57.10-X01",
"message": "Exception in thread \"Thread-41\" java.lang.NullPointerException: Cannot invoke \"Object.toString()\" because \"s\" is null
at clojure.string$includes_QMARK_.invokeStatic(string.clj:377)
at clojure.string$includes_QMARK_.invoke(string.clj:373)
at metabase.app_db.cluster_lock$retryable_QMARK_.invokeStatic(cluster_lock.clj:27)
at metabase.app_db.cluster_lock$retryable_QMARK_.invoke(cluster_lock.clj:19)
at metabase.app_db.cluster_lock$do_with_cluster_lock.invokeStatic(cluster_lock.clj:87)
at metabase.app_db.cluster_lock$do_with_cluster_lock.invoke(cluster_lock.clj:65)
at metabase.app_db.cluster_lock$do_with_cluster_lock.invokeStatic(cluster_lock.clj:79)
at metabase.app_db.cluster_lock$do_with_cluster_lock.invoke(cluster_lock.clj:65)
at metabase.search.task.search_index$init_BANG_.invokeStatic(search_index.clj:36)
at metabase.search.task.search_index$init_BANG_.invoke(search_index.clj:32)
at clojure.lang.AFn.run(AFn.java:22)
at java.base/java.lang.Thread.run(Unknown Source)",
"namespace": "hosting-eb9b3875e12d465a",
"container": "metabase",
"cluster": "metabase-1",
"_p": "F"
}
```
The error message itself comes from a stackoverflow which perhaps does
not have a message? Either way, it's possible for this to be missing so
be a bit defensive
* Forcing overflow policy on ChartTypeDropdown
* tpyo
* refactoring to enable testing this with loki
* fixed story
* loki
* screenshot
* removed useless unit test
Filter extract-columns endpoint to only return numeric and temporal columns
The extract-columns-from-query function is used to populate the incremental
filter dropdown for native SQL transforms. Previously it returned all columns
without filtering on type, leading to undefined behaviour when unsupported
column types were selected.
This change filters the results to only include columns with types supported
for incremental checkpointing:
- Temporal types (timestamp, timestamp with timezone)
- Numeric types (integer, float, decimal)
Text, boolean, and other unsupported types are now filtered out, matching
the behaviour of MBQL and Python transforms which already filter columns
client-side.
Updated documentation to clarify this endpoint is specifically for incremental
transform checkpoint column selection.
Also validates that transform POST/PUT conform to this rule and you are unable to update the transform to have
a filter column of an unsupported type.
Reverts the revert of https://github.com/metabase/metabase/pull/67686
* Rename Data Studio Library +New menu action labels
Update the menu item labels in the Data Studio Library's +New button:
- "Publish a table" → "Published table"
- "New snippet" → "Snippet"
- "New snippet folder" → "Snippet folder"
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
* prettier
---------
Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
* feat: support remote sync transforms in FE
1. Updates the DataStudioLayout to show transform dirty state
We get the orange dot next to Transforms if transforms or transform
collections are dirty.
2. Update the all changes view to include transforms
Also includes their collection heirarchy if the transforms sync
setting is enabled
3. Add the transforms sync setting
To both the remote sync admin page and the Set up git sync modal in
the data studio
* fixup! refresh transforms after sync
* fixup! data studio test feedback
* fixup! refactor failing test to troubleshoot debugging
* feat: add Slack Connect SSO provider
Add enterprise Slack Connect SSO integration using the OIDC infrastructure:
- Slack Connect provider: OAuth2/OIDC flow for Slack workspace authentication
- Settings: configuration options for Slack client ID, secret, and auth modes
- User provisioning: automatic user creation from Slack identity
- API endpoints: SSO initiation and callback handling
- Analytics: usage tracking for Slack Connect feature
Slack Connect allows users to authenticate via their Slack workspace,
with support for both SSO-only and link-account authentication modes.
* fix: session reset after auth
* fixup! avoid email clash
* fixup! wrong value in sso settings
* fixup! cleanup tests
* fixup! fix test
---------
Co-authored-by: John Swanson <john.swanson@metabase.com>
Update llms.txt and llms-full.txt (master)
Regenerated llms.txt to include all markdown files from the docs/ directory.
Also generated llms-full.txt files for the embedding section.
🤖 Generated automatically by llms-txt-generate workflow
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: GitHub Actions <actions@github.com>
Previously even when autoprovisioning was on, you could not log into a
deactivated tenant (either as a new user or an existing user with that
tenant).
This changes the behavior to be more consistent. Just as a deactivated
user can log in when provisioning is on (and their user gets
reactivated), a deactivated tenant's users can still log in, but doing
so reactivates the tenant.
* add data studio library filtering events
* add bulk properties and unpublish event
* add bulk sync events
* add search event
* add assertions on events in e2e tests
* add search debounce
* remove unncessery comment
* update event names
* track filter searches with data_studio_table_picker_search_performed
* refactor tracking function
* guard and log event only for ConcreteTableId
* log search when started fetching
* fix events not fired
* fix event not fired on first search
* Add AnchorLinkButton component for document anchor links
- Add AnchorLinkButton component that copies URL to clipboard on click
- Shows 'Copied!' tooltip feedback using Mantine useClipboard hook
- Follows same pattern as CommentsButton for consistent styling
- Add unit tests for the component
* Add AnchorLinkMenu component for left-side anchor link hover
- Add AnchorLinkMenu floating menu for document blocks
- Uses createPortal for positioning (same pattern as CommentsMenu)
- Positioned on left side of blocks using floating-ui
- Shows/hides based on hover state via show prop
- Add unit tests for the component
* Add AnchorLinkMenu to document node views
- Add documentWithAnchor URL helper to generate shareable anchor URLs
- Update Heading, Paragraph, and Blockquote node views to render AnchorLinkMenu
- AnchorLinkMenu appears on the left side on hover
- CommentsMenu on the right side remains unchanged
- Each node view now uses two useFloating instances for positioning
* Add scroll-to-anchor for document URLs with hash
- Add E2E tests for anchor links functionality
- Implement scroll-to-anchor when navigating to document with #blockId hash
- Tests cover: hover shows anchor icon, copy to clipboard, scroll to block
- Regression test ensures CommentsMenu still works on right side
* Fix scroll-to-anchor dependency arrays for navigation edge cases
- Add documentId to scroll effect dependencies to re-run when navigating
between documents with the same anchor hash
- Add location.hash to reset effect dependencies to properly reset flag
when navigating via back button or changing anchors
* Fix accidental logo replacement and E2E blockquote test
- Restore original Metabase logo SVG (was replaced with red star debug graphic)
- Fix E2E test: use find('blockquote') instead of findByRole('blockquote')
since native blockquote elements have no implicit ARIA role
* Fix scroll-to-anchor reliability and empty line behavior
- Implement retry mechanism for scroll-to-anchor to ensure target element exists
- Add highlight animation when scrolling to anchor
- Hide anchor link and comment menus on empty lines (Heading, Paragraph, Blockquote)
* Fix selector mismatch for scroll-to-anchor
- Add data-node-id attribute to NodeViewWrappers in Heading, Paragraph, and Blockquote
extensions so they can be queried by DocumentPage
- Update DocumentPage to query using [data-node-id] selector instead of internal _id attribute
- Remove unused ID_ATTRIBUTE_NAME import in DocumentPage
* Fix highlight animation color variable
- Replace non-existent --mb-color-brand-alpha-25 with --mb-color-brand-light
- Ensure highlight animation is visible when scrolling to anchor
* added anchor links to lists and code blocks. some bugs fixes.
* make sure scroll-to-anchor observer is correctly initialized
- Refactor scroll-to-anchor logic to ensure smoother navigation
- Optimize highlight animation visibility during scrolling
- Adjust anchor link behavior on empty lines for better user experience
* small fixes
* clean up
* fixes and cleanup
* e2e test fixes
* fixed e2e test
* Leave anchor links only for header blocks
* Fix alignment
* Review fixes
* Fix unit tests
---------
Co-authored-by: Denis Berezin <denis.berezin@metabase.com>
https://github.com/metabase/metabase/pull/68190 did:
> Treats a 0 value of n_live_tup as NULL, due to confusion reported when
> auto analyze thresholds have not been met on a table.
I think what's happening is this is trying to suss out "i don't know"
and not give misleading information. When there are no stats pg would
report there are 0 rows rather than saying i don't know. So we toss away
these unreliable 0 count estimations.
This test flakes 7% of the time. Our hypothesis is that stats are empty
7% of the time so this row count estimation now gets removed.
By default the schema is `:estimated_row_count (mt/malli=? [:maybe
:int])` which was being clobbered with
`{:estimated_row_count (mt/malli=? int?)}` So the fix is to just not
clobber
* feat: support transforms in remote sync
1. Serialize Transforms as Collection Items
Transforms were previously serialized as a top-level item, but this
moves them to be serialized as a child of the collection they are
member of. This also fixes an error in their serdes/dependencies
implementation where it did not include the parent collection leading
to deserialization issues.
2. Enforce Remote-Sync write rules
Adds implementations of the model interface permissions methods for
transforms that enforce the can-write? = false and can-create? =
false rules when remote-sync is in read-only mode.
3. Add a setting to control whether transforms are synced
Transforms are globally synced not synced via collections, so this
adds a new boolean setting that controls whether they are synced on
instances in read-write mode. Like all other content transforms are
always synced from the remote source if the instance is in read-only
mode.
4. Add a new spec-based configuration system for models
Models we want to make remote-syncable have a complicated, but still
data-driven system to allow you to specify their remote-sync behavior
and have that automatically setup in a single place the
`metabase-enterprise.remote-sync.spec` namespace.
* fixup! test with wrong folder
* fixup! not handling archived transform collections
* fixup! clean up aliases
* basic analyst role
* analyst role
* Backend for data analyst role
* Fixup frontend to work with Backend
The dedicated analyst endpoints have been removed:
- `GET /api/user/analysts` -> Use `GET /api/user?is_data_analyst=true`
- `POST /api/user/analysts/:id` -> Use `PUT /api/user/:id` with `{ is_data_analyst: true }`
- `DELETE /api/user/analysts/:id` -> Use `PUT /api/user/:id` with `{ is_data_analyst: false }`
These hooks no longer exist:
- useListAnalystsQuery
- useAddAnalystMutation
- useRemoveAnalystMutation
New Patterns:
Listing analysts:
```
// Before
const { data } = useListAnalystsQuery();
// After
const { data } = useListUsersQuery({ is_data_analyst: true });
```
Adding an analyst:
```
// Before
await addAnalyst(userId).unwrap();
// After
await updateUser({ id: userId, is_data_analyst: true }).unwrap();
```
Removing an analyst:
```
// Before
await removeAnalyst(userId).unwrap();
// After
await updateUser({ id: userId, is_data_analyst: false }).unwrap();
```
Type Changes:
The field is_analyst has been renamed to is_data_analyst everywhere:
- User.is_data_analyst (was is_analyst)
- UpdateUserRequest.is_data_analyst (was is_analyst)
- ListUsersRequest.is_data_analyst (new field for filtering)
* kondo
* circular dep
* fix backend not compiling
* use the actual role
* analyst management, glossary access for admins
* various data studio access
* access to library
* drop custom analyst seats management ui
* automatic perms group
* fix tests
* specs
* permissions
* fix lints, add migration rollbacks
* data analysts can ALWAYS view table metadata
implemented for:
- search
- `table/can-read?`
* Also remove `can_access_data_model`
* Revert "Also remove `can_access_data_model`"
This reverts commit 03af432ccb.
* fix more tests
* more tests
* snapshots
* remove redundant checks
* everyone can see
* hide data analyst group for OSS
* Make data analysts a normal group on OSS
* Automatically provide data analysts with perms
Data analysts automatically get manage-table-metadata permissions on
every table/database.
* only disable setting some perms for data analysts
* kondo
* oops, one more kondo
* test fixes
* specs, fixes
* more specs, fixes
* cleanup
* more test fixes
* fix module config
* ci
* doh, fix test
* more test whack-a-mole
* move data analyst group to the top of perms views
* review
* ci
* Turn DA into a regular group when feature missing
instead of just on OSS.
The goal is:
- OSS: a previously existing Data Analysts group is a regular group with
configurable permissions. The real Data Analysts group is invisible and
has no members.
- EE without the token: same as above
- EE with the token: the real Data Analysts group is visible.
* oops, remove unnecessary require
* another one
* fix membership API
* fix a couple more tests
* macro gets a !
* fix `fetch-groups-test`
* ci
* ci
* ci
* ci
* fix collection perm in migration
---------
Co-authored-by: John Swanson <john.swanson@metabase.com>
Add :fail-fast? true to BigQuery and Snowflake test-args, matching the
existing Redshift configuration. This stops test execution on first
failure, saving CI time and resources.
If the first table mentioned by a dashboard has no fields (presumably because the table and its field are all
inactive), /api/dashboard/<id>/query_metadata would fail due to a toucan2 bug. This bumps toucan2 to fix the bug and
adds a test to verify that this case now works.
Add ref schema caching
Introduces a custom :ref IntoSchema that permits
reuse of Schema objects for references when safe to do so.
Early tests indicate a dramatic reduction in heap retention, after
loading a single dashboard, I see a 10x resident heap reduction from ~300MB to ~30MB.
It should also act as knot tie / runtime combinatorics limiter, so we should not see 'leaky' growth at runtime.
More work could be done as there are still _some_ gaps here.
- Bare pointers (keywords) can still lead to exponential allocation
- We do not cache when :documentation is present on the ref
(as it would specialise the Schema object delivered for that specific reference)
* Desrtucture all this.props
* Convert to function component
* Remove the wrapper
* Forward ref in topbar
* Limit resize height
* Pass availableHeight to native query editor
* Move var to constants
* Add more padding
* Rename viewHeight to availableHeight
* Handle unset availableHeight
* Add availableHeight to QueryEditor
* Calculate availble height above the fold
* Resize editor when available size shrinks
* Remove layer of nesting
* Rename calc to get
* Pass availableHeight to sql editor in modal
* Remove resizableBoxProps prop
* Do not resize when closing editor
* Use runQuery from props when availabe
* Sort props and remove unused props
* Clean up setIsNativeEditorOpen
* Remove dispatch from NativeQueryEditor completely
* Move state change out of useEffect
* Pull out resizable box into separate component
* Simplify availableHeight prop
* Move resizable logic into separate component
* Clean up code
* Handle availableHeight being 0 initially
* Make minHeight take preference
* Avoid setting height to Infinity
* Add repro for metabase#59075
* Fix e2e test
* Do not use default export
* Add type to NativeQueryEditor
* Remove unused databases loader
* Remove unused ExplicitSize wrapper
* Remove enity loader wrappers
* Flatten props
* Fix setDatabaseQuery type
* Fix types
* Add className to NativeQueryEditor
* Fix types
* Pull out handler
* Manually update metadata when updating snippet
* Preserve sidebar state if need be
* Fix e2e test
* Fix mock exports
* Fix e2e test
* Do not overwrite availableHeight
* Remove viewHeight prop
* Increase limit
* Fix broken test
* Remove unneeded useHeightAboveFold helper
* Fix Horror Show fingerprint OOMs
This may likely fix many OOMs that others are seeing. Anyone with large
schema will likely benefit from this.
```clojure
(let [database (toucan2.core/select-one :model/Database :id 2)]
(metabase.sync.analyze.fingerprint/fingerprint-fields-for-db!
database (fn [s t] (println "synced " s "for " (:name t)))))
```
This is how we fingerprint fields. DB 2 here is "Horror Show", a
database with 200 tables, each of which has 900 fields in it. This
database has zero rows in it, it it just a large schema.
Syncing this recently has caused huge cpu spikes and then OOM errors
until the instance is killed.
Profiling locally with
```
MB_JETTY_PORT=3091 java "$(socket-repl 6091)" -Xmx2g -jar metabase.jar
```
and visual vm hooked up, I run
```clojure
(let [database (toucan2.core/select-one :model/Database :id 2)]
(metabase.sync.analyze.fingerprint/fingerprint-fields-for-db!
database (fn [s t] (println "synced " s "for " (:name t)))))
```
And observe this behavior:
The ram usage approaches the 2gb limit and then bounces there. The CPU
begins to spike erratically and spend 10% of its time in GC.
Taking a heap dump we can see that we have 2.8 million ResultSetMetaData
objects occuping 183 Mb.
The ResultSetMetaData are closed over and kept in the metadata of the
snake hating maps stored in the cached metadata provider.
This brings into stark relief one simple way to fix this: don't use a
caching metadata provider. This PR just raising the body of the
fingerprint-tables-for-db function to not include
`(qp.store/with-metadata-provider (u/the-id database)` . There is no
benefit to caching here as this is an enumeration of the database. It is
by design not meant to reuse anything. If we didn't OOM we would end up
with 200 tables and 200*900 = 180,000 fields in the cache, each of which
was accessed only once.
That is what this PR fixes.
I also followed one other thread: why are we holding so many
ResultSetMetaData objects in memory. This seems to come about from a
change in https://github.com/metabase/metabase/pull/62330 specifically
this line:
https://github.com/metabase/metabase/blob/master/src/metabase/lib_be/metadata/jvm.clj#L63
```clojure
(mu/defn instance->metadata
"Convert a (presumably) Toucan 2 instance of an application database model with `snake_case` keys to a MLv2 style
metadata instance with `:lib/type` and `kebab-case` keys."
[instance :- :map
metadata-type :- :keyword]
(let [normalize (if-let [schema (get metadata-type->schema metadata-type)]
(fn [instance]
(lib.normalize/normalize schema instance))
identity)]
(-> instance
(perf/update-keys memoized-kebab-key)
(assoc :lib/type metadata-type)
normalize
u.snake-hating-map/snake-hating-map
;; the line below here captures the result coming back from
;; toucan and has a closure over
;; toucan2.jdbc.read/make-column-thunk which has a closure over
;; (.getMetaData rset)
(vary-meta assoc :metabase/toucan-instance instance))))
```
As it stands without any changes, the instance gets quite unstable and
falls over. If you comment out the `var-meta` form above, the graph
returns to a very normal sawtooth graph of memory usage and
reclaimation. But central to this is that it does not need to retain any
of this memory at all. But without this metadata containing the instance
and db classes, we only cache the table and field data and the
fingerprinting can finish. When we cache this stuff and also
inadvertently keep these db classes, we quite quickly run out of
headroom and the instance becomes unstable and falls over.
* fix kondo lint (removes a legacy usage as well :) 💰
* bit more cleanup
kept having a few flakes, couldn't consistently get it to pass 45 times
in a row locally. So just simplify it. Don't worry about "in process"
stats. Trying to get the webserver paused in the middle of a request was
occasionally deadlocking, often flaking.
So just calculate the simple ones and it's fine
```clojure
statistics-handler-test=> (clojure.test/run-test server-stats-test)
Testing metabase.server.statistics-handler-test
Ran 1 tests containing 13 assertions.
0 failures, 0 errors.
{:test 1, :pass 13, :fail 0, :error 0, :type :summary}
statistics-handler-test=> (dotimes [_ 100](server-stats-test))
nil
```
Also got rid of the macro that really only existed to collect the forms
into an anonymous function. Everything in-lined and now don't need a
kondo hook, etc
* remove unused alias
* kondo the refers
* reintroduce in-flight metrics
these:
```
(is (prometheus-test/approx= 1 (value! :jetty/requests-active)))
(is (prometheus-test/approx= 1 (value! :jetty/dispatched-active)))
```
Require a mechanism that can pause a request while active, verify that
the requests-active and dispatched active are incremented, and then
carry on. This is all kinda trivial except getting it wrong can
deadlock. Here are 500 successful runs of this test:
```
statistics-handler-test=> (dotimes [_ 100] (server-stats-test))
nil
statistics-handler-test=> (dotimes [_ 100] (server-stats-test))
nil
statistics-handler-test=> (dotimes [_ 100] (server-stats-test))
nil
statistics-handler-test=> (dotimes [_ 100] (server-stats-test))
nil
statistics-handler-test=> (dotimes [_ 100] (server-stats-test))
nil
```
Filter extract-columns endpoint to only return numeric and temporal columns
The extract-columns-from-query function is used to populate the incremental
filter dropdown for native SQL transforms. Previously it returned all columns
without filtering on type, leading to undefined behaviour when unsupported
column types were selected.
This change filters the results to only include columns with types supported
for incremental checkpointing:
- Temporal types (timestamp, timestamp with timezone)
- Numeric types (integer, float, decimal)
Text, boolean, and other unsupported types are now filtered out, matching
the behaviour of MBQL and Python transforms which already filter columns
client-side.
Updated documentation to clarify this endpoint is specifically for incremental
transform checkpoint column selection.
Also validates that transform POST/PUT conform to this rule and you are unable to update the transform to have
a filter column of an unsupported type.
* feat: add owner fields
* feat: hydrate owner property on transforms
* feat: extend `start-run` function to accept `user_id` and use it on scheduled transform runs
* feat: implement 'Ownership' section in transform settings page
* test: implement unit specs
* test: add e2e spec
* feat: show owner avatar in transforms list
* Minor clojure improvements
* test: simplify & extend e2e spec
* refactor: simplify avatar usage & change unassigned copy to "No owner"
* refactor: de-duplicate success & error messages
* test: add missing serialization test transform attributes
* test: fix e2e spec by scrolling into view first
* refactor: attribute manual runs to current user rather than owner
* refactor: cleanup
* tests: add missing user id parameter for jobs unit spec
---------
Co-authored-by: Nathan Voxland <nathan@metabase.com>
Ensure client-full-response uses future so that interrupts are isolated from caller
An intentional server side .interrupt might leak and cause subsequent
code to fail.
Prior to https://github.com/metabase/metabase/pull/65872 the
client-full-response used a with-timeout, which incidentally put the
http handler in a future (isolating the interrupt).
This commit restores the previous behaviour, in a more intentional way.
Update llms.txt and llms-full.txt (master)
Regenerated llms.txt to include all markdown files from the docs/ directory.
Also generated llms-full.txt files for the embedding section.
🤖 Generated automatically by llms-txt-generate workflow
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: GitHub Actions <actions@github.com>
* fix(uxw-2711): dependency graph corner radius
* fix(uxw-2706): give snippets a default name
* feat(uxw-2750): use actual link elements for transform list page
* feat(uxw-2750): use actual links for library section layout items
* fix(uxw-2751): prevent transform cancel handler from wiping changes
* .
* .
* .
* edit is actually working
* add now works too
* fix
* added e2e test
* doc
* .
* added a wait to see if that helps
* better waits
* .
* addressed comments
* Apply suggestions from code review
Co-authored-by: Alex Yarosh <alexandra@metabase.com>
* test
* test
---------
Co-authored-by: Alex Yarosh <alexandra@metabase.com>
* Add test helper definition and implement joins
* Add support for fields clause
* Be explicit with types
* Pull out findJoinStrategy helper
* Pull out appenTestQueryStage
* Simplify helpers
* Move helper
* Allow disamiguating column name
* Fix rhs columns
* pull out createTestJoinClause
* Add filters
* Add aggregations
* Add breakouts
* Add expressions
* Add ordering
* Add limiting rows
* Make source optional
* Make provider the first argument
* Expand example in tests
* Automatically pick the base column if no table was specified
* Remove unused sourceMetadata argument where not needed
* Fix join metadata
* Improve types
* Add createNativeQuery and createNativeJsQuery helpers
* Use column groups to disambiguate instead of table name
* Test the results
* Deprecate createQuery
* Replace createQueryWithClauses with createTestQuery
* Add SAMPLE_PROVIDER
* Add helpers to build metadata provider
* Use displayName for groupName
* Remove raw mbql in test and replace it with Lib inspections
This introduces a pre-commit hook that automatically removes unused require
statements detected by clj-kondo, preventing CI failures.
Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
`command` is a shell builtin that doesn't exist as a standalone binary on
Linux (only macOS has /usr/bin/command as a wrapper). When babashka tries
to execute `command -v git` directly via process/shell, it fails on Linux
because there's no `command` binary to run.
Using `which` instead works on both platforms since it's an actual binary.
* feat: add generic OIDC provider infrastructure
Add OSS infrastructure for OIDC-based authentication providers:
- OIDC discovery: automatic endpoint discovery from issuer URL
- Token handling: ID token validation, parsing, and claim extraction
- State management: encrypted state cookies for CSRF protection
- Schema validation: Malli schemas for OIDC configuration and tokens
- Common utilities: shared HTTP and request utilities
This provides a foundation for enterprise SSO integrations that use
OIDC/OAuth2 flows (e.g., Slack Connect, generic OIDC providers).
* fixup! dead code clean up
There was a bunch of speculative code we didn't end up using when this
change from a 'configurable OIDC' to a just support slack course
* fixup! ttls for caches for jwks/discovery
fixup! check for open redirect when creating oidc state
* fixup! use valid-hosts? to check before making jwks/discovery requests
* fixup! don't use hardcoded jwt algo
* fixup! use java-time
* fixup! kondo
* fixup! use system.core not system.settings
* Update IAM authentication details for RDS instances
Clarified the types of RDS instances that support IAM authentication.
* Update docs/databases/connections/aws-rds.md
* Update docs/databases/connections/aws-rds.md
Previously, when running a transform job (e.g., daily), it would fail if a single transform in the run failed. This PR makes it so that the job continues to run transforms that don't depend on any failed transform. As previously, if any transform in the run fails, the job is marked as failed.
Addresses #67704.
When a UNIX timestamp is accessed through an implicit join, the coercion to DateTime wasn't being applied. The `allow-casting?` check requires `pos-int?` on `::source-table` to apply coercion. For implicit joins, `::source-table` is set to a string, so it was being skipped.
Fix: in `add-alias-info/add-source-to-field-ref`, wet the flag `:qp/allow-coercion-for-columns-without-integer-qp.add.source-table`. I cribbed this approach from the SparkSQL driver, when it has to rewrite a numeric ID with a string.
* feat: configure initial read-only mode for python transforms
* refactor: hide UI elements in readonly mode
* refactor: move database selector to the topbar
* style: add in/out animations for bottom panel
* refactor: move common library controls to the topbar
* test: add unit specs for python transform editor sections
* test: add e2e spec
* fix: show correct run query button state after edits
* refactor: general cleanup & test trimming
* refactor: remove source controls
* refactor: trim down test mocks
* fix: pass missing `editable` flag to transform editor
* fix: types
* fix: e2e data source selector
* refactor: use `isEditMode` instead of `readOnly`
* adding segment editor to scalar
* temp
* adding tooltip
* pretend i'm a clojure dev, disable tooltip when there are no segments
* adding unit test and story
* loki tests
* silly me
* I am a clojure god
* adjustments
* adding docstring, adjusting loki test
* combining scalar loki tests
* CardSegment -> ScalarSegment
* BE update, filter unvalid segments
* Require free text for ui-bug and other feedback types
Add Yup validation to MetabotFeedbackModal that requires free text feedback when issue type is ui-bug or other. Issue types requiring free text are maintained in a constant array for easy extension.
* Address PR feedback: improve type safety and UX in feedback modal
- Replace 'as any' with proper TypeScript type assertion
- Fix type inconsistency between schema and initial value for issue_type
- Add dynamic label showing '(required)' when ui-bug or other is selected
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
* pr feedback
---------
Co-authored-by: Claude Sonnet 4.5 <noreply@anthropic.com>
Co-authored-by: Sloan Sparger <sloansparger@gmail.com>
* initial work
* fix test
* fix another test
* source entity malli schema
* clean up and add test
* add tests
* clean up
* fix tests
* drop finding_details column
* fix analysis version
* Update enterprise/backend/src/metabase_enterprise/dependencies/models/analysis_finding.clj
Co-authored-by: Timothy S. Dean <timothy.dean@metabase.com>
* use transaction
* use node-errors refactor suggestion
* use maybe-resolve-join
* add schemas
* add comment
---------
Co-authored-by: Timothy S. Dean <timothy.dean@metabase.com>
This is the permission we're correctly checking with the below
`api/check-403` but by using `warehouses/get-database` (which, btw, has
the MOST confusing possible arguments I've ever seen) we were throwing a
403 if we didn't have read access to the whole database (which
checks *create-queries* perms on the entire database).
We still need to exclude mirror databases. In the medium term there
might be a better way to do this - maybe we could somehow have two
different models, so `(t2/select :model/Database)` would always exclude
mirror databases, while `(t2/select
:model/DatabaseIncludingMirrorDatabase)` would return them? I dunno.
Native sql validation is a bit too unreliable for general deployment at the moment, so we are disabling it for check_* endpoints and the broken entities list unless/until we can make it more reliable.
Co-authored-by: Alexander Polyankin <alexander.polyankin@metabase.com>
Treats 0 n_live_tup as NULL, due to confusion reported when auto analyze
thresholds have not been met on a table.
To justify:
- You have a non-zero estimate (No change)
- A zero estimate but actually have rows (We now hide the estimate)
- A zero estimate and actually have zero rows (We now hide the estimate,
unfortunately)
This behaviour is considered on balance preferable to the behaviour now
where users will likely be confused, even thought we explicitly label
the field 'estimated'.
Other options considered that we might explore in the future:
- Forcing ANALYZE, e.g. on sync
- Bounded/LIMIT count
- Some policy based on last analyze timestamps
- Better tooltips or explaination on frontend
We should revisit in the future once we have a better idea for how
estimates should work across drivers (as its only postgres for now).
When a user creates a question based on a model, `source_card_id` was set to the model's ID. If that question was later converted to native SQL, `source_card_id` wouldn't get cleared. This means that deleting the model would cause the question to be deleted even though they no longer have any relationship to one another.
The code fix won't help anyone whose data is already incorrect, so this adds a migration to correct the inconsistent data. I added an assertion to the t2 hook to catch this at dev time in the future. And ultimately, the delete cascade seems overkill, so there's a migration to replace that with SET NULL instead.
* Implement read-only on Library page
* Add some gorgeous unit tests
* Make segment and measures read-only / Add tests
* Apply read-only logic to measure list and segment list pages
A recent change to search permissions queries in [#67055](https://github.com/metabase/metabase/pull/67055) will cause H2 to use SYS locks during search operations. This is because H2 uses temporary tables to implement CTE's, and their creation/dropping requires global locks.
This commit removes some ^:parallel meta in an attempt to avoid test
flakes due to lock timeout.
This change is a bit speculative:
- It is not 100% confirmed that parallel tests are the only problem (e.g. some background task might also hold a lock)
- The tests changed are based on a cursory scan for search tests that
hold transactions (mt/with-temp). There may be other tests.
The impact of the CTE lock behaviour in production searches is likely
minimal because:
a. Search queries will typically run in auto commit sessions and so lock
lifetime will be short
b. concurrent multi-user deployments are most likely not on H2.
Should keep an eye on both tests and any reports in production of lock
timeout errors / slow searches when using H2.
* Why is it a warning when it should be an error?
* Fix error logging format in notification worker
* Improve error logging in notification send process
Fix silly mistakes that I do when I'm sleepy
When a card is updated, look for all mbql child cards and recalculate their result metadata. With models, calculate
what their "correct" result metadata is pre-update, use that to calculate user overrides, and then reapply those
overrides when computing the new result metadata.
This should fix a variety of bugs around users updating a model and then not seeing those change downstream.
- Add max-height and vertical scrolling to input area
- Fix newline preservation in multi-line messages
- Add horizontal scrolling for wide code blocks in messages
- Style code blocks with proper background, border, and custom scrollbar
- Reduce code font size to 0.6875rem for better proportion
- Increase user message bubble width to 90%
- Add inline code styling with background and border
* Add /has-remote-changes endpoint to remote-sync API
Adds a new endpoint GET /api/ee/remote-sync/has-remote-changes that checks
if there are changes on the remote (Git) repository that can be pulled.
Response format:
{
"has_changes": true,
"remote_version": "abc123...",
"local_version": "789xyz...",
"cached": false
}
* Disable git pull button when there are no changes to pull
* Fix unit test
* Fix e2e
* Adjust conflict modal 'push' option to always force
* Fix conflict resolution options when switching branches
* Fix modal title not wrapping properly
* Fix long long modal title e2e issue
---------
Co-authored-by: Edward Paget <ed.paget@gmail.com>
* Add /has-remote-changes endpoint to remote-sync API
Adds a new endpoint GET /api/ee/remote-sync/has-remote-changes that checks
if there are changes on the remote (Git) repository that can be pulled.
Response format:
{
"has_changes": true,
"remote_version": "abc123...",
"local_version": "789xyz...",
"cached": false
}
* Update api_test.clj
* Increase verbosity in the caching middleware
* Fix linter
* Make it simpler
* Broke so many tests
* Make the linter happy
* Fix tests (going back to old implementation)
Fully support remote-sync for data studio and make it easier to extend support for remote-sync to other models in the future.
Supports serializing tables with their fields and segments via remote sync if they are published to a remote-synced collection.
Supports dirty state tracking of tables, fields, and segments.
Refetches dirty state when tables, fields, and segments are created/updated/deleted in the data studio
Make sure the data studio still displays content when remote-sync is in read-only mode
enforce that tables, fields, and segments published to remote-synced collections are not editable when remote-sync is in read-only mode.
Deletes, tables, fields, and segments from the remote-sync repo when they are removed or archived.
This also creates two small abstractions to make it easier to add new models (like measures) to remote-sync when they are ready. The remote-sync.events models adds a defmodel-change-handler macro that setups handlers for -create, -update, -delete, events models can publish that the dirty state tracking will follow.
Right now the UI does not change when the data studio is in read-only mode, but all create/edit operations correctly return 403 if you try to modify things. I have a follow up branch to support disabling UI elements.
* feat: add dirty tracking for model changes
* feat: add table, field, segement to dirty state query
* feat: extract fields with tables and segments
when exporting a collection with a table we want to include its children
like fields and segments
* feat: allow importing/deleting tables
and segments and fields
* feat: add dirty tracking for field/segment
* feat: show path in git sync controls
* fix: skip archived segments in table descendants
* fix: view library when in read-only remote-sync
Allows users to still see library folder they can't edit when that
condition is caused by the library being remote-sync enabled and the
instance being in remote-sync mode.
* feat: use remote-sync status in table edit perms
* fix: don't use include_editable_data_model param
...in data studio so these will still load when remote sync is read-only.
* fix: always show dirty library state on removed
We can't track what collections a table was removed from so we just want
to show the library as dirty if any item has a 'removed' dirty state.
* chore: update modules config
* chore: fix kondos
* fix: only publish table update events on ee
otherwise a handler doesn't exist which causes them to throw
* fixup! whitespace
* refactor: make the table changes rendering simpler
* fixup! missing when
* fixup! tests shouldn't mock modules
* fix: make dirty tracking work with new implementation
* refactor: make the hydration step in events cleaner
* fix: make sure dirty returns lowercase
* fix: delete segments correctly
* fixup! remove extra module
* fixup! correct migration order
* fixup! FE lints and typecheck
* fixup! remove correct extraneous module
* Update enterprise/frontend/src/metabase-enterprise/remote_sync/components/ChangesLists/EntityLink.tsx
Co-authored-by: Anderson <ands.alves.nunes@gmail.com>
* fixup! PR feedback and tsc
* fixup! segment failing to redirect
* fixup! better fix for navigate after save
---------
Co-authored-by: Anderson <ands.alves.nunes@gmail.com>
For the data warehouses we can safely set a fixed
unreturnedConnectionTimeout to a fairly low number (5 minute default)
because we also have a default maximum query length for data warehouses,
which can't be exceeded anywhere.
For the application DB though, this isn't safe, as operations against it
can legitimately hold connections for extended periods - for example,
dump-to-h2 on large databases streams data for as long as it takes to
copy all rows.
Bumping this to 1 hour by default (configurable via
`MB_APPLICATION_DB_UNRETURNED_CONNECTION_TIMEOUT`).
* simpler tests? you decide
* ignore discouraged var. just tests. don't need api docs
* avec-server -> with-server
originally there was another `with-server` and i didn't want to clobber
it until i was ready
* bit of cleanup
* cljfmt
* bit of linter appeasement
* Support measures and metrics for Metabot
* Add segment support to metric querying
* Minor clojure tweaks for metabot measures and segments poc (#67425)
* Handle measures and segments like metrics in table-details and card-details
Move these into the existing m/assoc-some and handle them in the same way as metrics
* Remove query->output-format
This was just a wrapper around lib/->pMBQL, but all these queries are already MBQL5 queries
* Metabot measures and segments cleanup and tests (#67488)
* some refactor
* draft some happy path tests
* use lib for creating measure and segment clauses
* fix tests
* fix some filter tests to use mbql5
* update more tests
* fix more tests
* fix model query syntax
* lint
* remove normalize call again
* add back normalize
* minor test cleanup
* address comments
* tweak another test
* add a query= helper that strips uuids, update a number of tests to use it
* add basic error handing for bad ids
* Minor cleanup
* address mike's comment
---------
Co-authored-by: appleby <86076+appleby@users.noreply.github.com>
Co-authored-by: Noah Moss <32746338+noahmoss@users.noreply.github.com>
Co-authored-by: Noah Moss <noahbmoss@gmail.com>
* fix: don't unset remote-sync settings on api req
...if request is partial. Fixes an issue where the token could be unset
when switching between read-write and read-only modes.
* add debugging
* remove debugging
* fixup! handle extra collection
* fixup! update syntax
Update llms.txt and llms-full.txt (master)
Regenerated llms.txt to include all markdown files from the docs/ directory.
Also generated llms-full.txt files for the embedding section.
🤖 Generated automatically by llms-txt-generate workflow
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: GitHub Actions <actions@github.com>
* Fix: In Safari, Reducing the Tile Size of a Number viz Might Result in the Number Being truncated like a String Rather than Rounded
Fixes#51082🤖 Generated with [Claude Code](https://claude.ai/code)
Co-Authored-By: Claude <noreply@anthropic.com>
* UXW-46 refactor claude solution, implement proper fix
* Update Loki Snapshots
* Fix e2e tests
---------
Co-authored-by: Claude Code[bot] <noreply@anthropic.com>
Co-authored-by: Denis Berezin <denis.berezin@metabase.com>
Co-authored-by: Metabase Automation <github-automation@metabase.com>
* feat: split can-read from can-query in data model
Adds a new kind of permission type `can-query` that describes when a
user is allowed to query the data model vs `can-read` which encompasses
when a user can see the metadata of a query. This makes api calls to
read table metadata more straightforward and the eventual integration
with remote sync easier.
* feat: update api to use can-edit/can-query params
* fixup! allow can-write-metadata queries
* fixup! field test
* fixup! whitespace
* fixup! returning empty lists instead of 403
* fixup! schema response
* fixup! enforce query permissions where necessary
* fixup! add last tests
* fixup! can query filter before saved questions
* use ActionIcon for data studio toggle icon
* Use ActionIcon on snippet tree row ellipsis menu
* Change link colors in breadcrumbs on hover
* Use ActionIcon for tree expand icon
* Update hover state on transforms runs filters
* Fix data studio nav collapsed logo hover
* Match hover state on snippets collection and snippets folders
* Fix ts
* fix: remote-sync origin url errors
Fixes issues where the tmp directory for the remote-sync repo can get
into a bad state where it doesn't have an origin URL. This calls a
function that explicitly sets our desired origin URL and updates all
calls to use the more specific :remote-url keyword rather than
inconsistently using :url vs :remote-url.
* Update git_test.clj
* Extract cycle detection algo and fix available-measures
* Rremove duplicated code.
* Add more serialization tests
* Fix CLJS
* Remove MBQL4 support for measures at the model level
We can't hydrate unbounded collections of cards with the
can_run_adhoc_query hydration since it linearly adds db queries to the
request. Users with > 10000k can have severe CPU/memory issues
processing this.
* Fix heading level for multi-tenant strategy section
* Correct 'Admin setting' to 'Admin settings'
Updated references from 'Admin setting' to 'Admin settings' for consistency throughout the document.
* more edits
* typos
* feat: include query type on dependencies api response
* feat: show icons based on query type
* test: add unit spec for dependency graph utils
* refactor: restore changes after merge conflict
Fixes#67667.
`lib.equality` has logic to do "fuzzy" matching of field refs with
the corresponding columns. This logic exists to support somewhat
malformed older queries with impoverished field refs. Refs in healthy
recent queries are fully specified and there's nothing ambiguous to
required the fuzzy matching.
The bug #67667 caught a case where this fuzzy matching could find a
column with the same name on the RHS of a join, if the original target
column has disappeared from an upstream query.
This change causes the QP to fail in exactly this case: when a would-be
LHS ref in a join condition ends up pointing at the RHS of the same
join. That would produce a strange join condition like
`LEFT JOIN Orders ON Orders.ID = Orders.PRODUCT_ID`
which produces incoherent results, and could result in a very costly
Cartesian join.
This is a user-level error, since the query is broken and needs repair.
Basically, the issue is that trying to implicit join using the instance analytics content model's made_public_by_user field made qp throw an error. The underlying problem is that the field in question wasn't defined as a foreign key, even though the model's result metadata claimed it was a foreign key. Updating the field to actually be a foreign key fixed the problem. This also applies to the content model's question_database_id field, which was also fixed.
* use source alias if we have a fk-field-id
* try only on single stages
* try to identify implicit joins
* try fix
* Clean up
* move test and test all drivers
* move test
* only test drivers with left join feature
* adjust length for certain drivers
* smaller for mysql
* fix starts-with filter
* improve test
* case insensitive filter for h2
* move let bindings up
* Use instance settings for date filters in parameters
* Fix test instance to use default time format
* Add unit tests for date paramater formatting with custom settings
* let users with data model permission access data studio
* respect manage table metadata permission in data studio
* fix ci
* fix flaky specs
* review
* review
* Add e2e test for missing field mapping
* Make validateTemplateTags public
* Render errors in run button tooltip
* Pass errors to run button
* Fix selector in test
* Simplify test
* Try with realClick
* Add Lib implementation of validateTemplateTags
* Also validate display-name
* Use Lib implementation of validateTemplateTags
* Localize error messages
* fixup! Add Lib implementation of validateTemplateTags
* Add Lib.ValidationError type
* Remove unused code
* Also validate temporal-units dimension
* Use validate-template-tags in can-run
* Add tag-name to template tag errors
* Fix malli types
* Add tests for validate-template-tags
* Add tooltip to save button too
* Also test time grouping and save button
* Remove more unused code
* Remove simple wrapper
* Use cond-> pattern instead of filter identity
* Also test coerced datetime column
* Add e2e reproduction for opening the datetime filter for coerced fields
* Also check :effective-type in original-isa
* Fix test
* init pass at adding an error boundary to metabot
* update the illustration used for error boundary and remove simulated error ui
* add unit tests
* split up the mega metabot.unit.spec.tsx unit test file
* fix lint
Previously if you had:
```
A (curate)
B (curate)
C (no permissions)
D (curate)
```
Then if C was archived, you would be unable to move or archive A or
B.
* Fix `text-secondary` not working on dark theme on charts
* Support all colors so it's easier to reason about
* Add tests
* Fix type error
* Revert part of the code that's not used anymore
* Add test snapshots
* Make overriding font more consistency across environments
e.g. From storybook to SDK to the main app.
* Fix formatting
* Skips cloud driver tests on PRs unless files changed
Cloud drivers (Athena, BigQuery, Databricks, Redshift, Snowflake) now
only run on PRs when their files changed or 'ci:all-cloud-drivers' label
is present. Full coverage maintained on master/release branches.
* [do not merge] add failing databricks test
* Revert "[do not merge] add failing databricks test"
This reverts commit de5b6fa602.
* empty commit
* Fix cloud driver skip logic to respect ci:all-cloud-drivers label
The previous condition had three checks that could block cloud drivers:
1. !inputs.skip - false when no backend files changed
2. skip != 'true' - from mage can-skip-driver-tests
3. X-should-run == 'true' - the per-driver decision
The problem: checks 1 and 2 would override check 3, so the label had no
effect. Even with ci:all-cloud-drivers, cloud drivers were skipped.
We don't need inputs.skip or can-skip-driver-tests for cloud drivers
because the X-should-run output already encapsulates the full decision:
- master/release branch → run
- ci:all-cloud-drivers label → run
- driver files changed → run
- otherwise → skip
The label is an explicit override, so it should work even on PRs with no
backend changes. Removing the redundant checks lets the label work.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
* mage -driver-decisions + roll in module analysis
* Remove quarantine check from test-driver action
Quarantine logic is now consolidated in mage -driver-decisions.
The check happens earlier in determine-driver-skips, so quarantined
drivers get should-run=false and their jobs never start at all.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
* Fail CI if quarantined driver has file changes without break-quarantine label
If you change files in a quarantined driver's module (e.g., modules/drivers/snowflake/)
but don't add the break-quarantine-<driver> label, a new check-quarantine-conflicts
job will fail with a clear error message.
This prevents accidentally merging changes to quarantined drivers without running
their tests.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
* Add --github-output-only flag to mage driver-decisions
Instead of using grep to filter mage output, the task now supports two modes:
- Default: prints human-readable analysis + GITHUB_OUTPUT preview
- --github-output-only: prints only clean key=value lines for CI
The workflow now calls mage twice: once for debugging visibility,
once with --github-output-only redirected to $GITHUB_OUTPUT.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
* Add quarantine conflict checks to all driver jobs
Extends the quarantine conflict detection to ALL drivers, not just cloud
drivers. When a driver is quarantined but has file changes in
modules/drivers/<driver>/, the job will run but immediately fail with
a clear error message explaining how to add the break-quarantine label.
Changes:
- Add quarantine conflict check steps to presto-jdbc, sparksql, sqlite,
sqlserver, and vertica jobs
- Update login-to-ecr job condition for vertica quarantine conflicts
- Fix verbose output suppression in --github-output-only mode by adding
verbose? parameter to changes-important-file-for-drivers?
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
* Add driver decision priority documentation and tests
Adds comprehensive documentation to driver-decision function showing the
priority order as a table, plus explanations of what triggers "driver
module affected" and the difference between cloud vs self-hosted drivers.
Adds mage.modules-test with 14 tests (78 assertions) covering:
- Quarantine logic and break-quarantine labels
- Global skip behavior
- H2 always-runs behavior
- Master/release branch behavior
- Driver module affected behavior
- Cloud driver special rules (labels, file changes)
- Self-hosted driver skip logic
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
* Comment out starburst from driver-directory mapping
Starburst tests are currently disabled in drivers.yml, so the mapping
entry was inconsistent with all-drivers list. Commented out with a note
explaining why, so it can be easily re-enabled when starburst tests
come back.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
* silence messages when in github-output-only mode
* remove redundant cloud-driver change detection
- this happens inside the script instead
* get rid of redundant rules definitions
- now only print how we make decisions in help for the task
* make a temporary redshift change
* Revert "make a temporary redshift change"
This reverts commit 3d93c60368.
* format usage fn code
* light refactor
* Rename driver decision variables for clarity and flatten decision tree
- driver-module-affected? → driver-deps-affected? (module dependency graph)
- drivers-changed → particular-driver-changed? (specific driver's files)
- Reorder priority: skip/H2/master now before quarantine
- Flatten nested cond for cloud driver rules
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
* Quarantine now respected on master/release branches
- Move quarantine check before master/release in driver-decision priority
- Quarantined drivers are now skipped even on master/release (unless break-quarantine label)
- Update tests and priority table documentation to reflect new order
- Cloud drivers don't benefit from driver-deps-affected (only label or file changes)
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
* Fix parsing of :ERROR chunks in metabase-enterprise.metabot-v3.util/aisdk->messages
There was a typo in the case expr where :ERROR chunks were being handled like :DATA chunks, but in fact should be
handled like :TEXT chunks. This was resulting in an error like
1. Unhandled java.lang.ClassCastException
class java.lang.String cannot be cast to class
clojure.lang.Associative (java.lang.String is in module java.base
of loader 'bootstrap'; clojure.lang.Associative is in unnamed
module of loader 'app')
RT.java: 847 clojure.lang.RT/assoc
core.clj: 193 clojure.core/assoc
core.clj: 192 clojure.core/assoc
util.clj: 56 metabase-enterprise.metabot-v3.util/aisdk->messages/fn/fn
...
whenever an :ERROR chunk was encountered as we tried to ASSOC a string where we were expecting a map.
https://v4.ai-sdk.dev/docs/ai-sdk-ui/stream-protocol#text-parthttps://v4.ai-sdk.dev/docs/ai-sdk-ui/stream-protocol#error-part059d904d6e/ai_service/ai_sdk.py (L50)059d904d6e/ai_service/ai_sdk.py (L203)059d904d6e/ai_service/ai_sdk.py (L324)
* Add some error chunk parsing tests to aisdk-line-parse-test
* Mark aisdk-line-parse-test as ^:parallel
This does io to read some files, but should be threadsafe as no other tests read or modify those files.
* Add make-mock-error-stream-response test util
And rename make-mock-stream-response to make-mock-text-stream-response
* Add client-error-test to test streaming-request with error chunks in response
We have a grace periods and circuit breakers for token checks. But one
issue is that if the token check doesn't throw an error we can't catch
it.
An easy way to see this is to use a local network address that won't
respond like 192.168.random.random (i used 192.168.40.40) then see the
behavior. Without the timeouts, the threads all just get stuck here and
the page never loads. With timeouts, you'll see a few stutters until the
circuit breaker steps in, at which point you have a non-enterprise
version.
https://www.loom.com/share/6ae056dc126b48c9abe8a7ad9366fd05
* remove api "uses" dependency on api-routes
api-routes can use all modules to expose their routes. ie it uses
timeline module, etc. It has `:uses :any`.
```clojure
mage.modules=> (let [deps (dependencies)
all-deps (keys deps)]
(reduce (fn [c module]
(update c (if (skip-driver-tests? deps [module])
:skip :run) inc))
{:skip 0 :run 0}
all-deps))
{:skip 0, :run 153}
```
This is shown in this path here:
```clojure
mage.modules=> (let [ds (dependencies)
ds (into {}
(map (fn [[k v]] (if (= v :any)
[k (-> ds keys set)]
[k v])))
ds)]
(letfn [(driver-path [to]
(loop [q '[[driver]]
seen #{}]
(let [p (first q)]
(when-let [node (peek p)]
(if (= node to)
p
(recur (into (vec (rest q))
(for [adjacent (remove seen (ds node))]
(conj p adjacent)))
(conj seen node)))))))]
(driver-path 'enterprise/remote-sync)))
[driver warehouse-schema api api-routes enterprise/remote-sync]
```
driver uses arehouse-schema uses api uses api-routes which uses
any. Therefore all changes will trigger driver tests.
So after these changes we observe:
- the path above is empty (drivers don't transitively use remote-sync
module) and therefore would not trigger driver tests
- only 76 modules trigger driver tests
```clojure
mage.modules=> (let [deps (dependencies)
all-deps (keys deps)]
(reduce (fn [c module]
(update c (if (skip-driver-tests? deps [module])
:skip :run) inc))
{:skip 0 :run 0}
all-deps))
{:skip 77, :run 76}
```
What else did i try?
1. add a `#_{:clj-kondo/ignore [:metabase/modules]}` to the ns
I didn't love this because it felt a bit like cheating. All of this is
kinda annoying because it's fundamentally about changes during dev time.
2. remove the ns require and add a requiring-resolve
This actually doesn't break the module usage analysis. It's smart enough
to see this so this just makes it more tenuous and easy to miss if we
change anything while also still leaving in place the module
entanglement.
So I just fire the event from the api module when rebuilding routes and
handle the event from the api-routes module. The trick is that the api
module sends along the function to rebuild the docs
* handle cycling ref
* remove reference to deleted file
* our linter is scared of an extra line
* use an events namespace
not doing an init namespace as that's similar but not the same. Those
are for settings and models. It's fine to just require the event
namespace in the core namespace. it's a bit of ceremony but the pattern
is useful
* include random event correctly
used to be just `(first (descendants :metabase/event))`. This was
fragile and just now started to fail:
```clojure
models-test=> (first (descendants :metabase/event))
:metabase.api-routes.events/api-events
```
because of course do we things like `(derive ::api-events
:metabase/event)`.
```clojure
models-test=> (filter (comp (complement #{"event"}) namespace)
(descendants :metabase/event))
(:metabase.api-routes.events/api-events
:metabase.audit-app.events.audit-log/card-event
:metabase.audit-app.events.audit-log/cloud-add-on-event
:metabase-enterprise.dependencies.events/transform-delete
:metabase.activity-feed.events.recent-views/legacy-card-event
:metabase.view-log.events.view-log/collection-read-event
:metabase.audit-app.events.audit-log/notification-handler-event
:metabase.audit-app.events.audit-log/user-update-event
:metabase-enterprise.dependencies.events/transform-run
:metabase.revisions.events/transform-event
...
)
```
This has been lucky for a long time
* Update Remote Sync branch picker to match new designs
* Move git sync controls next data studio logo
* Fix unit test
* Fix e2e test
* Address Claude's review
* Remove console.error
* Use css modules for the chevron icon transition
* Use popover instead of findByRole('presentation')
* disable progress bar in whitespace linter github actions workflow
* add extra whitespace to demonstrate failure
* Revert "add extra whitespace to demonstrate failure"
This reverts commit ba34650efd.
* feat: add git settings modal to data studio
* Refactor GitSettingsModal to reuse existing settings form / Adjust some styles
* Fix and improve unit tests
* Adjust font size according to variant
* fix unit test
* fixup! mock API not other plugins in this layout
* Split unit test files and improve endpoints setup
* Implement PLUGIN_REMOTE_SYNC.GitSyncSetupMenuItem with internal rendering condition
---------
Co-authored-by: Anderson <anderson@metabase.com>
Fixes#65838.
This issue was about dashboard filters showing the wrong value when
the filter was bound to an FK field from a table with multiple FKs to
the same foreign table. The dashboard parameters logic on the BE was
choosing which FK to use for the filter values arbitrarily, meaning that
binding the filter to one of them would work fine, but the other would
give incoherent results.
The fix is to be deliberate about selecting the parameter's column. That
doesn't solve all ambiguous cases, but we really can't help if there's
a multi-hop join that has parallel edges in the middle!
* Add field grouping of host and port
* Add host and port settings to postgres, mysql
* Add host and port grouping to db drivers
* Add database story with host and port group
* Add groupFields unit tests
* Fix typo
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
* Fix missing tunnel parameter value
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
* Refactor and simplify field grouping function
* feat: add nested fields support for db format
* feat: remove id field, adapt new format foy postgres, mongo, mysql, remove leftovers
* feat: support nested fields, add backdrop component
* fix: group inputs loosing focus on type/render
* refactor: simplify custom containers logic
* style: adjust backdrop container style
* refactor: field mappers for db forms
* refactor: move host and port config to the new format
* refactor: cleanup old grouping approach
* refactor: remove unneeded property
* fix: typo in the html prop name
* internal: validate container style comping from BE
this value can be set be external drivers so it's better to validate it
Signed-off-by: Jakub Chodorowicz <jakub.chodorowicz@metabase.com>
* fix: type errors
* fix: driver translation test
* refator: use existing getFlattenedFields function
* fix: flatten properties for connection property uniquness
* backtrack from ssl grouping for mongo
---------
Signed-off-by: Jakub Chodorowicz <jakub.chodorowicz@metabase.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
* Acquire locks before changing data permissions
Data permission have a read-modify-write cycle:
- we read the existing permissions to see what's currently set
- then we decide what to do (e.g. delete the existing DB level
permissions to write a granular table-level permission)
- then we do the write
This has a race condition if multiple Metabase instances are modifying
permissions at the same time.
For example, say we have:
Initial state: `{:db_id 1 :table_id nil :perm_value :unrestricted}`
Instance A: set table 1 to :blocked
Instance B: set database to :blocked
T1: A reads state, sees db-level permission, plans to expand to table-level
T2: B reads state, sees db-level permission, plans to replace it
T3: A deletes db-level, inserts table-level rows for all tables
T4: B deletes db-level (no-op), inserts new db-level :blocked
Result (CORRUPT):
```
{:db_id 1 :table_id 1 :perm_value :blocked} ;; from A
{:db_id 1 :table_id 2 :perm_value :unrestricted} ;; from A
{:db_id 1 :table_id nil :perm_value :blocked} ;; from B
```
Both db-level AND table-level permissions now exist.
For the permissions API, we take a cluster lock, but sync operations
bypassed this cluster lock. This PR creates a lower-level cluster lock
per `[database, group, perm-type]` so only one instance/thread can do
the read/decide/write cycle for a particular db/group/perm at a time.
* Add a test for race conditions setting data perms
This took a while before I was able to track down the fact that using
`(mt/with-temp )` caused both threads to share the same connection, so
they could obtain the lock simultaneously 🤦
* Use a slightly coarser lock
I realized that if an instance has a huge number of groups, we'll obtain
a lock and ask Postgres to maintain an active savepoint for each group,
which might be untenable.
I think the best solution here is to use slightly coarser locks. This
might increase contention slightly, since we won't be able to change
permissions for `db 1, group 1, perm type view-data` at the same time
that we change permissions for `db 1, group 2, perm type view-data`. But
I think the tradeoff here is necessary.
* Prevent unnecessary height calculation
* Prioritize local layout from `react-grid-layout` for height calculation
* Refix the issue for real
* Add tests
* fix: hide untested and unsupported db engines in table editing section
* move allowed engines to constants file
* test: add hidden editing toggle test case
* fix: avoid persisting settings that are hidden from dashboard UI
* test: add `sanitizeDashcardSettings` unit spec
* refactor: use `_.pick` for sanitizing dashcard settings
* Fix pie chart flickering on toggling legend / Convert ChartWithLegend to ts
* Convert test to ts
* Resolve flickering when toggling visualization config sidebar
* Fix regression with map chart (only apply svg centralization to pie charts)
* Apply PieChartContainer classname to inner component
* Default to base-type when it differs from effective type
* Make 'er parallel
* Does this clause make it better?
* Remove parallel.
* New test and fix for two literals
* Use y-type for effective type
* Update test to try number
* Don't expose `question` in SDK's `entityTypes`
Because it's only applicable to staged picker
* Fix type
* Clarify the use of `question` in SDK's `entityTypes`
* Fix SDK not building
* Prevent test email from including Metabase links (#67068)
* Send a new flag from FE if testing subscription from modular embedding
* Pass :disable_links to dashboard subscription email template
* Remove links from dashboard subscription email template
* Remove link from pulse cards
* Remove dangling divider line
* Add test
* Fix tests
* Use the same logic as in the actual subscription tasks
* Prepare to use a new column name from pulse table
* Remove embedding subscriptions email links (#67181)
* Add a column to pulse to save when a subscription is created on embedding
* Save disable_links to "pulse" table
* Fix tests
* Fix more tests
* Update column name
* Add tests
* Make code less confusing after embedding name change
* Fix liquibase boolean type
* Fix tests
* Fix more tests
* Fix formatting
* Fix wrong test names
* Make function comment styles consistent
* Make the test more solid
* Add modular embedding SDK tests
* Don't render the dashboard subscription button when there's no data cards (#67501)
* fix: respect time format in exlude date picker filter QUE-2611
* rework: move constant out of function body
* test: fix failing tests due to lack of redux provider
* fix: add missing redux provider in story
* test: improve testing logic for ExcludeDatePicker
* internal: use specialized useSetting hook
* test: adjust looping with async action
The root cause of the flakiness was that the save modal sometimes picks the root collection, while at other times it picks the current user's personal collection. The selector was trying to find "Our analytics" in the breadcrumbs, which is why it was failing every time the actual collection is personal collection.
The second issue was that trying to (1) clear and then (2) type the metric name was extremely unreliable.
* Add failing test for #63822, DEV-1190
* Add new feature flag
* Update FE config
* Fix tests 🔧
* Disable for 7 more drivers
* Fix MySQL + MariaDB
* Fix TypeScript
* Check that the number was parsed correctly
* Don't parse int columns that are remapped
* oops, remove old change
* Better fix and specific test
* kick ci
* kick ci
* Make sure it still hits the bigint branch
* Move check into parseNumber()
* Move check back to value.tsx
* Single describe in tests
* Add blank lines (fe-lint)
Log each entity (card) as its dependencies get backfilled by the job. This is to be able to see if there is a particular entity that's causing the OOM.
* prototype
* open chat sidebar on navigate
* update copy for new route
* use nlq profile and make sticky, redirect to empty query builder if no navigate_to is recieved
* responsiveness
* move code to enterprise folder
* imple error states, better loading state
* handle user leaving the page while a response is being loaded
* lint fix, change inline sql shortcut key, disable prompt suggestions and fade out once a prompt has been sent
* fix e2e test
* add test coverage, update copy
* fix failing e2e tests
* attempt to fix e2e flake
* UXW-2449 fix pasting mixed content containing cardEmbed
* Fix issue with cardEmbed loading forever
* Fix issues with smartLink and caedEmbed paste to a list
* Fix typecheck
* Remove openapi.json from repo and disable CI check
The committed openapi.json was causing friction:
- Cascade CI failures when master got out of sync
- Merge conflicts on a 36k line generated file
- Phantom local changes when switching branches
The OpenAPI generation still works - run `yarn generate-openapi` when needed.
We're just not committing the artifact for now.
* Check for Node.js 22+ before generating OpenAPI types
The previous check for `yarn` wasn't sufficient - the system may have
yarn installed with an older Node.js version. Now we explicitly check
that Node.js 22+ is available before running the generation.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
* Revert client-side OpenAPI types generation
* Remove types generation from postinstall
* Restore package.json newline
* Restore prepare-backend action
---------
Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
Co-authored-by: Stas Gavrylov <stasgavrylov@gmail.com>
Co-authored-by: Stas Gavrylov <stas@metabase.com>
* Allow clicking dates with hour and minutes granularity to set parameter
* Add repro for metabase#58556
* Fix unit tests
* Add repro for metabase#66277
* Allow selecting date ranges with hours and minutes
* accept optional profile-id in search tool request and use it to determine collection filtering
* cleanup
* try to fix tests
* fix build
* kondo
* fix toucan syntax
* address comment
* oops, fix build
* remove some more use_case code on the FE that slipped through the revert via messy rebase
* set network timeout in connection options
* add test
* kondo
* fix test
* Add try catch
* use network timeout setting
* use exception chain util
* use cached thread pool, and timeout to call connection.close
* just do setnetworktimeout
* address comments
* Fix: Cannot add filters on boolean MySQL columns when the data source is a model
Fixes#65826🤖 Generated with [Claude Code](https://claude.ai/code)
Co-Authored-By: Claude <noreply@anthropic.com>
* Introduce multimethod
* Improve docstring
* Add db-type-name to changelog
* Make tests work
* improve changelog entry
---------
Co-authored-by: Claude Code[bot] <noreply@anthropic.com>
Co-authored-by: Eric Normand <ericwnormand@gmail.com>
* Fix: Redshift external tables have unknown types for string/enum columns on sync
Fixes#63025🤖 Generated with [Claude Code](https://claude.ai/code)
Co-Authored-By: Claude <noreply@anthropic.com>
* Add more Iceberg types to Redshift external table types
---------
Co-authored-by: Claude Code[bot] <noreply@anthropic.com>
Co-authored-by: Eric Normand <ericwnormand@gmail.com>
- when a spec is registered, store the raw form in an atom. This is so
things like `search.spec/explode-raw-case` can be stored as raw symbols
rather than functions. (Do note that this is technically a bug in a rare
edge case, where the definition of `search.spec/explode-raw-case`
changes and the search index should technically be invalidated - but I
think this is rare and low-impact enough that we can just not care.)
- when we access the current search index version, canonicalize the
stored specs, plus default attributes and attribute types, and sha256
hash the result
* UXW-2576 show tenant collections in the trash
I also deleted the `audit-namespace-clause` function. It felt like it
didn't do enough to justify its existence - every caller needed to
modify the list of namespaces. Could probably DRY things up a bit still,
but think we should change the approach.
* Check `use-tenants` before returning tenant colls
* s/audit-namespace-clause/namespace-clause helper
`namespace-clause` takes the ns column, ns value, and an optional
argument for whether *this* particular result should include the tenant
collections in the response (since sometimes we want 'em, sometimes we
don't.)
* kondo
* Prevent updating tenant specific root collections
* Localize the name of dedicated tenant collections
* rename dedicated tenant collections 'Our Data'
When displayed on the FE for an admin, they can be renamed to something
like "Foobar's tenant collection"
* Internal users see different names
* prevent tenant users to edit their dedicated tenant collection
* Revert "prevent tenant users to edit their dedicated tenant collection"
This reverts commit 08d2d722be.
* Rename `Our Data` to `Our data`
Co-authored-by: Nicolò Pretto <info@npretto.com>
---------
Co-authored-by: Nicolò Pretto <info@npretto.com>
* Demo of only importing the relevant plugin in unit test
* Optimize enterprise plugin unit tests with targeted setupEnterpriseOnlyPlugin calls
* fixed tests
* fixup
* .
* fixed test
* removed hasEnterprisePlugin
* fixup
* removed remaining hasEnterprisePlugins and unnecessary setupEnterprise fn
* specificPlugins -> enterprisePlugins
* .
* fixed type
* docs
* Add effective type to expression when resolving from ref
* Add repro for QUE-2567
* Fallback to base type when checking effective types
* Avoid shadowing var
* Add tests for effective-type fallback
* Fix normalization of MBQL 4 `:aggregation-options` with a `nil` options map
* Update tests
* Fix normalization of Template Tag :options
* Allow raw values as expression definitions
* Automatically remove :ident from joins
* Properly handle aggregations expressions that contain `:if` (alias for `:case`)
* Allow `:coalesce`, `:if`, and `:case` as datetime exprs in MBQL 4
* Test & Kondo fixes 🔧
* WIP
* Working test! (it reproduces)
* Fhew! I found it!
* Remove printlns
* Also remove sensitive fks in calculation
* Remove stray marker comment
* Move tests to fields-test
* Exclude :retired too
* Move tests and add one for retired
* Add another retired
* Better tests
* Appease the kondo
* Fix broken test
### Description
We want to communicate per-billing-period quantities with customer, instead of always communicate monthly quantities. Invoiced totals will stay the same.
### How to verify
Describe the steps to verify that the changes are working as expected.
1. Ensure you have a token with the `offer-metabase-ai-tiered` feature, but without the `metabot-v3` feature.
2. Navigate to `/admin/metabot` and see the quantities as per billing period, and not per month.
### Checklist
- [x] Tests have been added/updated to cover changes in this PR
Fixes: 2b76c38e1f
Closes: https://metaboat.slack.com/archives/C07RMUQ0GE5/p1765907134400039
* Update default value locations in comments
* Update snowplow schema for modular embedding analytics
* Add more type safety
* fix 'dashboard.drills' and 'question.drills' default values for guest embeds
* Fix auth type
* Test guest embed authentication
* Log `with-subscriptions`
* Strengthen the type
* Hide external collections sidebar for internal users without tenant collection accesses (#67033)
* show the add button if user can write to root
* update unit tests for shared collections nav
* simplify test mocks
* getIsTenantUser selector
* flatten navbar for tenant users
* move up state and use plugin, keep "Collections" name when the external
collection section is not shown
* polish
* throw new error
* fix unit test
---------
Co-authored-by: Phoomparin Mano <poom@metabase.com>
* add openapi-typescript schema generation
* chore: add live type generation
* Temp fix for self ref schema
* nemanja wants avaible locales schema
* generate openapi.json file on dev or when updating the apis
* now it e2e. you update an api schema => generate openapi.json => hook to update d.ts
* add hey api
* update user profiles
* proper types now
* fix: remove leftover folders and adjust imports
* chore: Claude guide to generate MALI schemas
* feature: add schema for users API
* wip add user type to profile
* remove the ref of the old lib. all hey-api now
* WIP
* Update FE unit mock
* fetch openapi.json from file system instead of the api
* Add in-browser overlay with errors
* disable error overlay
* feat: script to generate openapi spec
* experiments
* do not clean cwd
* Enable pretty-printing for OpenAPI json
* Add OpenAPI spec keys sorting
* Move generated openapi.json to resources/openapi
* Commit generated typings with sorted keys. Add types.get.ts to eslintignore because it has incorrectly linted typing
* wip
* Fix nullable types
* test fixes
* revert hack for self ref schemas as the new lib cna generate it
* Move TS generated types to gitignore. Cleanup.
* revert uncessary schema changes and also fix indentation
* Add types generation to TS CI job
* Update CI to generate and cache TS types
* Fix CI types generation
* Cleanup FE
* reorganized the modules and remove the requiring-resolve
* Fix unit test
* fix: wrong function call to regenerate api spec
* chore: upgrade hey-api (fixes self referencing hey_api type)
* feat: disable type generation for MetabaseLegacyMbqlSchema
* chore: regenerate openapi definition
* chore: regenerate openapi.json
* feat: add generate-openapi-spec command
* build: use generate-openapi-spec in yarn scripts
* build: add script that verifies the that openapi spec is uptodate
* build: add CI check for openapi schema being uptodate
* build: fix wrong filename of api check script
* internal: use yarn script for schema generation
* internal: generate version in the api schema generation step
* internal: install node 22 in the api generation job
* internal: fix hey api running twice on build hot
* internal: debounce ts api types generation
* bring back debounce logic
* revert changes in users schema
* internal: add malli docs instructions in skills format
* internal: add details to defendpoint+malli skill
* Fix incorrect types:watch initialization in build-hot script
* Remove version field from generated OpenAPI JSON to allow proper versions comparison on CI
* Update OpenAPI JSON schema
* a nice solution for the dynamic timezone enum
* update openapi json
* test fixes
* update openapi json with latest apis
* resolve tips when this happens again
* fix kondo test and revert the locale type in properties api
* update spec to remove locales
* Revert enum usage on UI
* build: types as part of postinstall
* types: replace hardcoded log type values
* build: format types with prettier - compatible with embedding type transforms
* Update OpenAPI schema
* Update OpenAPI schema
* Revert types cache generation
* Address PR comments
* Reset spec format
* Reset spec again
* Merge master and update OpenAPI spec
* Update package.json
Co-authored-by: Nemanja Glumac <31325167+nemanjaglumac@users.noreply.github.com>
* feedback
* resolve comments
* Update OpenAPI spec
* Update OpenAPI spec
* Update OpenAPI spec
* Update OpenAPI spec
* follow derive conventions
* Remove deprecated rspack type check deps
* Add eslint rule to restrict imports from metabase-types/openapi
* Update OpenAPI schema
---------
Co-authored-by: Nicolo-Maria <nicolo.mezzopera@metabase.com>
Co-authored-by: Ngoc Khuat <qn.khuat@gmail.com>
Co-authored-by: Nemanja <31325167+nemanjaglumac@users.noreply.github.com>
Co-authored-by: Stas Gavrylov <stasgavrylov@gmail.com>
Co-authored-by: Uladzimir Havenchyk <uladzimir@metabase.com>
Co-authored-by: Oleksandr Solovyov <o@solovyov.net>
Co-authored-by: Stas Gavrylov <stas@metabase.com>
Co-authored-by: Bryan Maass <bryan.maass@gmail.com>
The savingModel prop allow "collections" to be written to the
shared-tenant-collection namespace root while other model types are disallowed.
Fixes how this prop is calculated because the original implementation
left off other types of cards. This prop is now just "collection" | null
since collection is the only model type that matters here.
* Change url from checkout to login
* Use store login url when pro-self-hosted plan
* Unit test WIP
* source_plan query param added
* Cleanups
* Move useGetStoreUrl to hooks file
* Unit tests added
* Misc
* Misc
* Remove hooks
* Removed login path page
* Param renamed to `migration-source-plan`
* Unit tests updated
This pull request removes the user setting that persisted the open/closed state of the native query preview sidebar in the notebook editor. The sidebar's visibility is now managed entirely in the UI state (not persisted per user), simplifying the logic for showing or hiding the sidebar and cleaning up related code and tests.
1. opening a notebook editor will *always* start from a closed native preview sidebar
2. state is now managed exclusively by Redux (sidebar state might be preserved until the page reload, but it is not persisted on the backend)
3. We're changing a previous behavior where, if a native preview sidebar was open and we're resizing the window to something small, the sidebar was automatically closed. This PR changes that behavior. i.e. There is no special resize handling anymore. Whatever the (Redux) state of the sidebar is, it will stay like that no matter the window size.
The catch block in startContainers was swallowing errors - it would
log and tear down containers but never re-throw. This meant when
health checks failed, the script would print "All done!" and exit 0,
then Cypress would run and fail with a confusing "could not verify
server is running" error.
Now we re-throw after cleanup so the actual failure is surfaced.
* .
* .
* .
* .
* .
* trying without one timeout
* trying without another timeout
* trying without another other timeout
* trying without yet another timeout
* .
* Do not allow picking unsupported databases in Python data picker
* Do not auto-select disabled items
* Add e2e test for empty data selector
* Simplify ugly if condition
* Fix e2e test
* Do not open datapicker wen going to next step
* broad strokes
* clean up snippets and metrics pages
* tons of PR feedback
* Hi Dan
* test adjustments
* more adjustments
* fix card issue
* removing text on runs list page
* fixing glossary tests
* PR Feedback
* remove header prop
* page spacing adjustmnets
* Center Viz Empty State, fix python library page
* fix python transform page
* scrolling behavior on Job and Transform list pages
* last bits of shame
* fix: druid breakout and agg names must be unique
Fixes#10670
* Fix CI
* cljfmt
* format
* format rows
* make it work with druid jdbc
* address comments
* use seq over some?
---------
Co-authored-by: Riley Thompson <riley@metabase.com>
Co-authored-by: Riley Thompson <rileythompson99@gmail.com>
* WIP UXW-2451 Fixes issue where changing light/dark mode messes up color variables by removing nested providers in useBodyCellMeasure which probably breaks other things
* Update Loki Snapshots
---------
Co-authored-by: Metabase Automation <github-automation@metabase.com>
This generally indicates an MBQL 4 -> 5 conversion issue, and we need
the signal in the 57 rollout.
Also improves the detail of the logging for when `:dataset_query {}`.
* Add Australia (Sydney) to available regions
Added Australia (Sydney) to the list of regions.
* Add Asia Pacific (Sydney) IP addresses to whitelist
Added IP addresses for the Asia Pacific (Sydney) region.
* Reorder South America IP addresses section
* Update cloud region list in change-region.md
Removed duplicate entry for Australia (Sydney) and updated Asia Pacific region.
* Filter Popover close on click outside
* scroll on mount for field and preview panels, tests
* ci bandaid
* UXW-2510 Fix background color mismatch in table metadata panels
* PR Feedback
* shameful fix
---------
Co-authored-by: Brad Anderson <brad@metabase.com>
* Fix check for card changes
The query coming from the FE looks like:
```
{:aggregation [["count"]],
:breakout [["field"
42
{:base-type "type/DateTime",
:temporal-unit "month"}]],
:source-table 7}
```
(note strings like `"count"` and `"month"` and, most importantly, `"type/DateTime"`.)
The card from the DB looked like:
```
{:aggregation [[:count]],
:breakout [[:field
42
{:base-type :type/DateTime,
:temporal-unit :month}]],
:source-table 7}
```
When we compare the query to the old version to see if it's changed (and
thus, we should invalidate the verified status) we walk it first,
converting all keywords to strings:
```
(let [prepare (fn prepare [card] (walk/prewalk (fn [x] (if (keyword? x)
(name x)
x))
```
... but this is not the correct way to do this conversion, because
`:type/DateTime` becomes `"DateTime"`. We need to include the namespace
if it's present.
* Compare normalized queries
* Modify `changed?` to only compare updated keys
In the end we're doing a `t2/update!` with the updates passed in. If a
key isn't in the updates, it's not going to be updated.
Simplify `changed?` to just take two arguments, `before` and `after`,
and compare the keys in `after` (after doing the necessary
normalization).
If a key is in `after` but *not* before we can't compare it and should
throw an exception. For example, if we passed a card like
`(t2/select-one [:model/Card :id :name])`, we should throw.
* More sane way to check if a card has changed
Instead of ad-hoc updating keys, use the toucan transforms to just
canonicalize all the values before we compare.
* kondo
We currently have the "revision number", but:
- the intent there is slightly different; we want to prohibit one user
from making changes to the permissions graph before they *see* changes
another user might have made concurrently, not just sequence the
concurrent changes, and
- we allow API users to pass the `force` parameter to skip the revision
check entirely
Note that the reason we have `force` available is to allow people to
change permissions without first making a request to *GET* the
permissions graph (which can in some cases be extremely large). It is
not intended to allow concurrent updates to permissions.
https://github.com/metabase/metabase/actions/runs/20267445215/job/58194458921 failed because snapshots were running for QA databases, and we don't have those Docker containers up for this job.
Since we don't need them, I have adjusted the snapshot step to only run the default snapshots. There's a comment what to do if we change our mind and want to run external database checks in the future.
The summary: we are offering drill-throughs on custom columns defined after aggregation, but those custom columns don't exist in the base query. The error presented in such a way that `lib.underlying/top-level-column` for a dimension was returning `nil`, and we were happily putting that `nil` into the query we constructed.
I created a utility to filter dimensions by whether or not the top-level-column can be resolved, and used this for the underlying-records drill-through (from the original issue) as well as the pivot drill-through (which behaved similarly). If no dimensions have a top-level column, the drill-through is not available. If some do, we offer the partial drill-through.
`source` is a legacy key on the frontend that gets translated to `lib/source`. Some maps are using both `source` and `lib/source`, and the translation layer translates keys and values in completely separate steps. When the time comes to translate the legacy value of `source` into a valid value for `lib/source`, the code can't tell what the key was originally named. So already-valid values were going through the translation process, producing invalid values (e.g., `:source/source/aggregations`).
This adds a step to object parsing to detect that both keys are present; when they are, it sets an option to simply skip `source` during the parsing; secondly, it adds a step in the value parsing to check whether the value is already a keyword namespaced by `source/`.
Theoretically the BE could provide the email if you send in an expired
token, but it seems slightly less controversial to just provide a link
like `/auth/reset_password/the-very-long-token?email=some%40email.com`.
This way having the expired token itself can't reveal any information,
but if you click on an expired link, then it's just 2 clicks to send
yourself a fresh token instead of click -> type email -> click.
Fixes [UXW-658: New user invites expire quickly, adds friction to new user experience](https://linear.app/metabase/issue/UXW-658/new-user-invites-expire-quickly-adds-friction-to-new-user-experience)
`util.perf/mapv` checks the `count` of all its collection args, so it's
not safe to call with an infinite `seq`.
`lib.metadata.result-metadata/merge-cols` was doing exactly that in the
case where there was a mismatch between the column counts from lib and
driver. It was passing `(concat short-list (repeat nil))` to `mapv` and
causing an infinite loop!
* Correctly handle non-string `@tenants` in JWTs
Previously we threw exceptions if we received an invalid non-string
`@tenant`, like `123` or `false` or `{:foo "bar"}`.
Now we:
- convert integers to strings,
- take strings as-is, and
- throw a meaningful exception for anything else
`:dataset_query {}` should not happen, and indicates a bug in the
MBQL 4 normalization. On reading any card, if its `:dataset_query`
is `{}`, add the ID to an in-memory set and update a Prometheus
metric.
This reverts commit 9191713854.
used to do this in-house but now we use trunk for this. Old way has
ultimate flexibility. But trunk has some really nice workflows built
in (reporting, creating issues in linear, etc).
* rename to full app and modular embedding
* Update docs/embedding/sdk/quickstart-with-sample-app.md
* Update docs/embedding/full-app-ui-components.md
* Apply suggestions from code review
---------
Co-authored-by: Jeff Bruemmer <jeffbruemmer@gmail.com>
* Add DB infra for metabot use cases
* add enabled? use case setting and hook it up end to end for transforms
* remove transforms use case toggle
* add back migrations, number of tests
* add more missing migrations; lint
* some simplification
* fix some tests
* pull use-case out of tool call request header & fix last test
* more test fixes
* add back commits, move use_case->profile mapping to the appdb
* clean ns
* bump ci
* just store use case names in metabot_message instead of fks
* update test
* serialization fix; refactor
* lint
* restore migrations, address code review
* add back migrations again
* Tenants, External Users, and tenant based JWT with Audit Events
Allow the creation of tenant users
While working on this, I realized that my approach to the "All External
Users" group and renaming the "All Users" group was untenable.
I had been planning to just update these things in the database (create
AEU, rename AU to AIU) in the setter function for the `:use-tenants`
setting. The issue here is caching: we don't want to need to actually
reach out to the database every time we need to fetch the All Users or
All External Users, but it would also be quite unfortunate if one
Metabase instance in a cluster had an outdated view of, e.g. the ID of
"All External Users".
(Although - I'm not really sure how much we use the *name* of "All
Users" when we fetch `(all-users)`, so that might be worth a look.)
For now, I've modified things to just update things dynamically in the
`GET /api/permissions/group` and `GET /api/permissions/group/:id`
endpoints: we show/hide the "All External Users" group, and rename the
"All Users" group, based on whether Tenants are enabled.
Allow the creation of tenants
Listing tenants / tenant members will be a followup PR
lint
doh missing require
fix tests
add a test for enabling tenants changing groups
fix tests, slight change to tenant creation API
kick ci
fix another test
give minimum permissions to ext groups
fix more tests
impl user stategy setting ui
more API endpoints
add more tests, fix a very silly migration
slug validation
fix mariadb/mysql
lint migrations file
kondo
fix test
replace tenant members API with `/user?tenant_id=`
also realized that the `/user/recipients` endpoint needed to be modified
for tenants: tenant users should not see recipients that are not part of
the same tenant. (reminder for later: we will need to add safeguards
around actually creating those pulses, not just assume the API won't
send bad recipients!!)
IF you are an admin, `/api/user` respects a `tenant_id` query param and returns:
- "normal" (non-tenant) users if it's `nil`
- specific-tenant users if not
If you are *not* an admin, `/api/user` ignores the `tenant_id` passed
and only returns users in the same tenant.
`/api/user/recipients` will:
- return ALL users if you are a superuser (both tenant and non-tenant)
- return only users in the SAME tenant as you if not
doh
tenant crud wip
integrate endpoints, fix type errors
Allow getting all external users
previously you could do only
```
GET /api/user => internal users
GET /api/user?tenant_id=null => internal users
GET /api/user?tenant_id=123 => users in tenant 123
```
now you can do
```
GET /api/user => internal users
GET /api/user?tenant_id=null => internal users
GET /api/user?tenant_id=123 => users in tenant 123
GET /api/user?tenancy=all => internal + external users
GET /api/user?tenancy=internal => internal users
GET /api/user?tenancy=external => external users (in all tenants)
```
If you specify both `tenancy` and `tenant_id` (regardless of the values
passed) you'll get a 400.
fix my bad rebase
fix bad rebase more
remove #p
mark tenants as `is_active` or not
initial external users management
remove permissions check on the trash endpoint
I don't know why this was added in the first place (I mean, I think I
added it, but I don't know why I did).
The trash is always accessible to everyone. We gate the visibility of
things *in* the trash, but the trash itself is always readable.
no personal collection for tenant users
remove tenant query inside oss
tenant user routes
make it so that you cannot remove users from the external users group
do not send email to tenant users
no login for inactive tenants
disable collection permissions, don't show sent email modal on user success with SMTP
remove all users group has higher permissions warnings for external users group
moving is external checks behind plugins
unit test cleanup, add simple test
read only application permissions for external users
set tenant on login with JWT
incredibly hacky tenant attribute
separate `:attributes` from `:login_attributes`
thinking here is that `login_attributes` are the things you set on the
user, where `attributes` are all the attributes of the user including
things like tenant slug or maybe someday locale etc.
fix build
fix token check
create a tenant collection in a migration
this works, but the tenant collection is always visible to admins, even
if the setting is off.
hide tenant collection from normal users
fixing build and FE rebase
Delete stray "howdy"
Lol, sorry about that.
fix tests
throw instead of silent failure on no tenant
superuser can't be tenant
fix up tenant collections
tenant collection now includes subcollections.
Tenant collections are hidden if the feature isn't enabled.
tenant collections marked in API
bugfix and current user can write any coll
don't add `attributes` key to all users
just the current user on login
invalidate groups when changing user strategy
read only user
{{ tenant_slug }} user attribute
fix be test
rip out tenant collection stuff on BE
fix BE tests
lint
fix magic string
no tenantifying existing users for now
also clean up a complicated, confusing fn.
starting e2e
fixing e2e, adjusting @tenant user attribute
fix permissions editor, add tenant.name e2e test
allow more than one user per tenant
🤦
allow getting only active tenants
*actually* don't replace existing values with nil
getting e2e ready for tenant user provisioning
tenants management ux
e2e coverage on sandboxing, deactivating tenants
replace {{tenant_slug}} with `@tenant.slug`
filter out other-tenant users from pulse creation
rename any existing `@.+` user attributes
prohibit new user attributes starting with `@`
fix migration
ugh, forgot to actually commit this file
add more tests
kondo
use the `tenants` feature
fix a test - missed one `:advanced-permissions`
e2e test coverage, updating tenant attribute to @tenant.slug
Partial Type Fixes
types green
unit tests
adding groups to e2e test, maybe fixing tenant feature check?
Add tenants to tracked snowplow features
Fixing Admin ui
Modernizing tenants page
separate external group in permissions sidebar
permissions editor updates
hardcode that *everyone* has trash access
shame... bad imports
adjusting Main Navbar. using globe icon
CR response
fix tests
fix attributes when tenants are disabled
lock up the 'all external users' collection
tenants route guard
add tests for missing feature and oss
fixing some unit tests, and type errors
set token features in default.cy.snap
unit and e2e test adjustment
another pass on e2e
fix e2e and build
fix a bad merge
Green CI?
reset token after restoring DB
move a test to fix the test
kondo
Move `/api/ee/tenants` to `/api/ee/tenant`
PR Feedback, part 1
move route functions into ee
ADM-1105 permissions editor adjustments for external users
ADM 1104 do not show invite button on external users when there is no tenant
adjusting unit tests
fix no results message for deactivated users page
center no results message
Fix migrations IDs / related tests
adjusting no results message for tehants
fixing unit tests, adding e2e test for permissions
chore: add timestamp columns to tenants (#60575)
* chore: add timestamp columns to tenants
* fixup! add hooks
* kick ci
Auto provision tenants, respect tenants feature (#60571)
deactivate tenant users along with their tenants (#60552)
* deactivate tenant users along with their tenants
Tenant users from deactivated tenants were already prevented from
logging in (we checked the activation status of their tenant) but for
bookkeeping it's probably best to actually deactivate the users
themselves along with their tenants, and reactivate them if the tenant
is reactivated.
* invalidate users when updating a tenant, adjust external users list screen
* e2e test
---------
Co-authored-by: Nick Fitzpatrick <nickfitz.582@gmail.com>
audit events for tenants (#60647)
* Revert "Delete unused string util from metabase lib (#64389)"
This reverts commit 0af38ef8f0.
* fix the build
* fix tests
* fix rebase error in invite button
* fix more tests
* fix more tests
* fix more tests
* fix more tests
* fix more tests
* fix more tests
* fix another test
* Fix another couple tests
* and more
* and more
* Close the modal to create shared tenant collection after it is created (#66049)
* Fix more tests, nearly all e2e tests passing
* a couple more tests
* couple more tests
* last (???) few tests
* Remove edit permissions in ellipsis menu when viewing shared tenant collection (#66042)
* Suggest external users only when adding tenant group members (#66046)
* feat: separate namespace for tenant collections (#66102)
* fixup! FE build
* feat: add remote-sync support for tenant collections (#66272)
* feat: add remote-sync support for tenant collections
* Update permissions.clj
* fixup! tests
* fixup! enterprise tests
* fixup! mysql/h2
* EMB-1011: add default read perms for namespace
* Fix tests (#66403)
* Fix tests
Not exactly sure how these got merged, I think maybe they were hidden by
GH issues.
* Fix kondo
* fix apparent rebase error
* fixup! EMB-1065 fix namespace filtering
* fixup! some bad perms tests
* fixup! remote sync dep checking across collections
* fixup! remove blank test
* fixup! missing tenant setting
* fixup! add premium feature declaration
* fixup! get test working with h2
* fixup! revert h2 fix
* fixup! revert h2 fix
* fixup! real h2 fix
* Dedicated Tenant Collections
* More fixes
* Tests and more fixes
* Fix rebase error
* rename
* Fix query for shared tenant collections
* fixup! UXW-2386 don't pass `tables` model into tenant root
* fixup! UXW-2436 build correct path for namespaced collections
* Remove test of non-existent fn
* Fix failing frontend unit and e2e tests in tenants feature branch (#66243)
Co-authored-by: John Swanson <john.d.swanson@gmail.com>
* fix failing frontend ESLint errors
* Prefix users and groups as "Internal" in admin settings when tenants are active (#66169)
* Update design of the user strategy modal for tenants (#66182)
Co-authored-by: Mahatthana (Kelvin) Nomsawadi <me@bboykelvin.dev>
* Add multi-tenancy setup guide to embedding hub (#66113)
* Update design of the tenants listing and internal user page (#66193)
* Give tenant users a personal collection
* Fix BE tests
* another be test
* Add specific error messages for JWT tenant claim errors (#66172)
* kondo/fmt
* fix running jwt tests standalone
* feat: per collection setting (#66475)
* feat: per collection setting
* feat: remove unsynced collections
* fixup! tests
* fixup! lints
* fixup! test
* Remove duplicated tenant permissions
* handle disabling feature more cleanly
when tenants is turned off:
- disable all tenant users
- reject any logins from tenant users
- reject any JWTs that specify a tenant
* Add tenant entity picker e2e tests. Fix one bug surfaced
* uxw-2456: remove drop shadows from dedicated tenant collection cards
* Hide tenant collections permissions tab when tenants are disabled (#66537)
* Hide create shared tenant collection button on sidebar for non-admin users (#66538)
* Hide tenant collections sidebar section when no collections exist for non-admins (#66539)
* Fix dedicated tenant collection root in collection picker as tenant user (#66549)
* fix frontend type-check: only set namespace if it is defined
* EMB-1093: Fix not passing a tenant attribute
Fixes [EMB-1093: If you enable multi-tenants and don't add JWT
attribute, it will
break](https://linear.app/metabase/issue/EMB-1093/if-you-enable-multi-tenants-and-dont-add-jwt-attribute-it-will-break)
* EMB-2459: Fix restrict where collections can be moved to
* Fixup working e2e entity picker tests
* fixup! mysql namespace tests
* fixup! tenant group mapping tests
* fixup! ws
* tree endpoint allows specifying multiple nses
* fix broken test
* Show dedicated tenant collections in collection pickers for admins (#66583)
* feat(sdk-bundle): add tenants ee plugin (#66586)
* fixup! bad ns clause audit feature is off
* fixup! UXW-2473 support non-string attributes
* Hide tenant collections section when there are no visible shared collections (#66597)
* Add tenant cards to search
* fixup! tenant collection cleanup
* EMB-1082 rename strings in the ui (#66603)
* make tenant field required (#66516)
* change `tenant` key to `@tenant`
- if the feature is off, you can still log in if you have the `@tenant`
key set. It will be silently ignored, but a warning will be logged.
- if the feature is on, `@tenant` is a drop-in replacement for `tenant`.
- we'd previously prevented setting user attributes that start with `@`.
This updates `remove-invalid-attributes` to do the same with JWT/SAML
attributes.
* add SAML tests
* Expose SAML attribute for tenant
* External users can't use MB if feature disabled
Plus add tests for:
- tenant user, feature disabled
- tenant user, feature unavailable
* Add missing test files
* Multi-stage embedding data picker should query models and questions from tenant collections (#66598)
* Fix external group tooltip referencing All Internal Users (#66654)
* Allow conversion of tenant <-> internal users
Upon conversion, all previous group memberships are destroyed and the
user is added to either the "All External Users" or "All Users" groups
accordingly.
* feat: implementing new tenant/remote-sync designs (#66567)
* fix missing dep in test file
and a tiny removal of an unnecessary `dissoc`
* fixup!: broken embedding page due to bad rebase conflict resolution
* fixup!: fix type errors
* fixup!: fix frontend jest unit test errors
* fixup! Move search tests requiring EE to EE code
* Fix copy test
* add confirmation warning when disabling tenants (#66697)
* fix test
* more renaming
* fixup!: failing tenants.cy.spec.js test and rename "All tenant users" migration
* fixup! whitespace fixes
* Remove serialization for dropped column
* fixup! missing migration
* fix other wrongly ordered migration
* fixup! Ordering more migrations
* fixup! other tests
* fix test
* fix another test
* api response schemas for tenant endpoints
* kondo
* one more kondo
* fixup! bad test query
* fixup! popover to use the right way of passing args
* Fix push changes modal
* fixup! correct token for entity picker tests
* Fix tenants e2e test with correct attribute
* fixup! tenants tests
* fixup! remove test for old behavior
* fix type on events, removed unused string union values
* update MainNavSharedCollections.unit.spec.tsx to use the new name
* fix linting issue unused import
* remove test file not on master anymore
* fix permissions select
* fix dashboard filters autowiring e2e test
* fix formating of module.edn
* remove useless test file
* add enum for namespaces
* fixup! remove toggle
* Update EntityPickerModal.tsx
* Remove commented tests
* Update e2e test based on feedback
* Split add-attributes-test
* Update test/metabase/collections_rest/api_test.clj
* Fix copy cmd model list
* Remove :put handling code for is_remote_synced
* Update src/metabase/premium_features/settings.clj
* Update resources/migrations/056_update_migrations.yaml
* Remove extraneous regex in api def
* Update frontend/src/metabase-types/analytics/event.ts
* UXW-2536 popup remote-sync modal after request
* Remove the "sidebar" event type
* Improve some backend tests
* Fix remote-sync e2e tests
* refactor: remove useAdminSetting and use settings already in the
component
* same for the jwt form
* move error message to Messages
* variant filled for apply button
* convert attributes to strings on read/write
In practice there is no difference between the value `true` and the
value `"true"`, so let's keep things simple by making all attributes
maps of string->string.
Fixes [UXW-2550: Cannot edit tenant users with non-string attributes](https://linear.app/metabase/issue/UXW-2550/cannot-edit-tenant-users-with-non-string-attributes)
* fw={500} => medium
* add c() to t`Sync {collection.name}`
* useSetting("use-tenants") in MainNavSharedCollections
* fixup extra dialog on permspagelayout
* fix broken tests
* fix the other broken test
* Remove tenant collection app permission
* Fix embedding sdk collection type
* Convert tenants e2e test to typescript
* move setupRemoteSyncEndpoints to server-mocks folder
* actually remove it from test-utils
* Small BE fixes
* use usePrevious from react-use
* Remove custom root tenant collection hook
* Change group detail to use text
* fix sdk type
* Redid git controls with button.group
* fixup! bad test
* fixup! tenant logic and components into plugins
* Update frontend/src/metabase/lib/string.js
Co-authored-by: Ryan Laurie <30528226+iethree@users.noreply.github.com>
* fixup! bad merge
* Rearrange last unit test for CollectionPicker
* fixup! PR feedback
* fixup! handle nils and seqs in attributes
* fixup! kondos
* fixup! better stringify
* Add missing maybe to login attributes schema
* fixup! yaml tests
* fixup! sequential check
* fixup! Stringify ldap attributes
* Fix ldap tests
---------
Co-authored-by: John Swanson <john.swanson@metabase.com>
Co-authored-by: John Swanson <john.d.swanson@gmail.com>
Co-authored-by: Phoomparin Mano <phoomparin@gmail.com>
Co-authored-by: Phoomparin Mano <poom@metabase.com>
Co-authored-by: Mahatthana (Kelvin) Nomsawadi <me@bboykelvin.dev>
Co-authored-by: Nicolò Pretto <info@npretto.com>
Co-authored-by: Ryan Laurie <iethree@gmail.com>
Co-authored-by: Ryan Laurie <30528226+iethree@users.noreply.github.com>
OOMs in cloud are linked to this weak map retaining too much. The
exported heap dump reports are suggestive but not conclusive, and I
can't reproduce locally.
Adds `:metabase-query-processor/computed-weak-map-queries` to measure
the size of the weak map. That should sawtooth with GCs as it
accumulates queries and then discards them, but we'll see if it rises
forever on the problem instances.
This also adjusts the caching for a `MetadataProvider`-level entity
(`:metadata/card`) from the weak map to the `MetadataProvider`'s
internal cache. That's speculative, since there's no evidence to link
it to the issue, but it should be at least as effective a cache and
there's less risk of accidentally retaining too much.
* Closes EMB-990 add description to columns in CollectionBrowser
* keep the existing behavior with tooltip unless description is explicitely set to be visible
* .
* addressed comments
* cleaned up unit tests
* Ignore collection for unpublished tables
Unpublished tables have no collection and the `collection_id` means
nothing, and should be ignored.
* Add a test
* Fix legacy search permissions
Legacy search ignored the collection permissions for published tables,
so it was possible to get a table back from search even though you
didn't have permission to see it because it was published to a table.
* segments: published table segment list, revision history, dependency graph, breadcrumbs, move preview to the menu
* specs
* fix specs
* dedupe tests of segments pages
* fix avatars
* fix rebase
* visuals
* update specs
* final refactoring
* polish
* review
* drop react context although it adds a bit of dupe
For those of us who use clojure-mcp (https://github.com/bhauman/clojure-mcp), our Claude configuration directly contradicts instructions we're giving it. This PR only adds to our `CLAUDE.md` and our skills a note that **if** clojure-mcp is available, it should prefer it over shell-based methods (like `./bin/mage -repl`).
docs scoop up all settings and has ways to avoid. In
`metabase.cmd.env-var-dox/avoid?` This had a docstring and then an extra
docstring. We can just copy the nice docstring to the docstring slot and
put `:doc false`.
```clojure
(defn- avoid?
"Used to filter out environment variables with high foot-gun indices."
[env-var]
(or (false? (:doc env-var))
(false? (:can-read-from-env? env-var))
;; Ideally, we'd move off of this list completely,
;; but not all environment variables are defsettings.
(contains? env-vars-not-to-mess-with (format-prefix env-var))))
```
* Show breadcrumbs on published table page (Library > Data > <table>)
* UXW-2524 Makes table breadcrumbs dynamic
* Adjust e2e test
---------
Co-authored-by: Brad Anderson <brad@metabase.com>
* UXW-2527 Disallows Library as an option in the snippet collection modal
* UXW-2534 Renames root sql snippet folder from "Top folder" to "SQL snippets" globally
this file got removed during a master->uxw-394-sub-collections-permissions-toggle merge on the PR #65928
the file had some changes, and removing it was not intended.
the version restored is this one, extracted from the same PR:
ab62f89a2c
motivating example:
we ran the redshift tests, and had three failures: redshift 1, redshift
2, and redshift transforms.
redshifts 1 and 2 timedout at 60 minutes. But the log upload failed
because the python transform tests failed in 24 minutes and then
uploaded:
from the transform failure run:
```
Run actions/upload-artifact@v4
with:
name: test-logs-be-tests-redshift-ee
path: logs/test-log.json
retention-days: 1
if-no-files-found: warn
compression-level: 6
overwrite: false
include-hidden-files: false
env:
CI: true
...
With the provided path, there will be 1 file uploaded
Artifact name is valid!
Root directory input is valid!
Beginning upload of artifact content to blob storage
Uploaded bytes 1072768
Finished uploading artifact content to blob storage!
SHA256 digest of uploaded artifact zip is 267d4b085d49ca4d682fedca97832fecb603bbb5a52c0e1d99bd4daa49984ba3
Finalizing artifact upload
Artifact test-logs-be-tests-redshift-ee.zip successfully finalized. Artifact ID 4816135520
Artifact test-logs-be-tests-redshift-ee has been successfully uploaded! Final size is 1072768 bytes. Artifact ID is 4816135520
```
And then from the redshift tests:
```
Run actions/upload-artifact@v4
with:
name: test-logs-be-tests-redshift-ee
path: logs/test-log.json
retention-days: 1
if-no-files-found: warn
compression-level: 6
overwrite: false
include-hidden-files: false
env:
CI: true
...
With the provided path, there will be 1 file uploaded
Artifact name is valid!
Root directory input is valid!
Error: Failed to CreateArtifact: Received non-retryable error: Failed request: (409) Conflict: an artifact with this name already exists on the workflow run
```
The same happens on redshift part 1.
```
Error: Failed to CreateArtifact: Received non-retryable error: Failed
request: (409) Conflict: an artifact with this name already exists on
the workflow run
```
This just introduces on the driver action to look for a
logs-artifact-suffix and use it if present. And on driver jobs that use
a matrix include the matrix.job.name (or matrix.version.name depending
on the exact structure).
* Fix query normalization issues in v57
* Remain strict when saving a query
* replace bad dataset_query with legacy_query
* Test fixes 🔧
* Test/Kondo fix 🔧
---------
Co-authored-by: dan sutton <dan@dpsutton.com>
### Description
In https://github.com/metabase/metabase/pull/65343 we added a page to purchase Metabase AI (Metabot). Now we are improving the text to:
* Show information about trial only if Metabot was not already trialed
* Improve wording regarding multiple questions / messages in chat
### How to verify
1. Ensure you have a token with the `offer-metabase-ai-tiered` feature, but without the `metabot-v3` feature.
2. Check that the subscription associated with the token had either with (test case 1) or without (test case 2) the `metabase-ai` or the `metabase-ai-tiered` add-on.
3. Navigate to `/admin/metabot` and see the page look like in the mock ups.
### Checklist
- [x] Tests have been added/updated to cover changes in this PR
Closes: https://linear.app/metabase/issue/CLO-4673/show-different-copy-on-metabot-ai-purchase-page-after-trial
Closes: https://linear.app/metabase/issue/CLO-4688/copy-correction-in-the-admin-panel
* #66585 Updates static embeds to omit color-scheme settings to preserve legacy transparent iframe behavior
* support public embed too (#66693)
* support public embed too
* Imports isPublicEmbedding to prevent app crash
---------
Co-authored-by: Brad Anderson <brad@metabase.com>
---------
Co-authored-by: Nicolò Pretto <info@npretto.com>
If you have a query like "WITH query_a AS (...), query_b AS (select ... from query_a) SELECT ... from query_b",
native query parsing understood that query_b in the final query referred to query_b in the WITH, but it didn't realize
that query_a in query_b referred to query_a in the WITH. Now, it should understand these sorts of depedent ctes.
* Move documents feature to OSS
* Remove plugin
* Move comments, remove public sharing plugin
* Fix some CI issues
* Move documents BE to OSS
* Move documents BE to OSS
* Move documents BE to OSS
* Move documents BE to OSS
* Move documents BE to OSS
* Fix bug with state
* Move rest enterprise imports
* Remove token checks from apis related to documents
* Fix CI
* Fix CI
* Fix CI
* Fix CI
* Fix BE tests
* Fix CI
* Fix CI
* Fix bookmarking a document
* Add branding footer for public document
* Add e2e tests
* Fix CI
* Fix CI
* Fix CI
* Fix CI
* BE fixes for documents to oss (#66511)
* reinstate sandbox test
* refactor circular deps
* fix: copy documents/comments
* fix path
* remote-sync tests are ee only
* ee only
* documents are not just ee
* kondo
* missing mapping
* set site-url
* fix broken tests
* Fix metabot plugin
* Fix issue where having a metabot block inside a document prevents it from loading when publicly shared (#66564)
* Prevents public document view from crashing when document contains a metabot node. Uses fallback similar to print version.
* Omits SupportingText drag handle when in read-only mode
* Adds e2e test to verify metabot blocks don't crash public documents and aren't editable within them
* Fix accidental spacing change
* Fix unit tests, cleanup e2e test
* fix after merge
---------
Co-authored-by: Nicola Mometto <nicola.mometto@metabase.com>
Co-authored-by: Brad Anderson <brad@metabase.com>
* add events when making cards/dash pubic or retracting
https://github.com/metabase/metabase/issues/64417
When we make a card public, we put a uuid on it and include
`made_public_by_id`. But when you retract the public link, we remove
both bits of information. This is a nice invariant but it removes
information about who publicized a card.
Imagine a card is discovered by have been public. Of course you want to
immediately remove it. But this erases the audit trail (so to speak) of
who made it public. It is not a great idea to delay remediation during
the investigation. Now you can just consult the audit log
* make events
* formatting
* clean up tests a bit
tests were a bit wack. We should have utility functions to get audit
events. Also noticed we are going off of the object id and ignoring the
object. Could lead to gross issues if there's dashboard with id 4 in
there and also card with id 4. so require model on new one
* fmt
* Remove models
* Remove models
* Move
* Move
* Drop columns for model publishing
* Drop columns for model publishing
* Drop columns for model publishing
* Drop columns for model publishing
* Rename library-models to library-data
* Rename library-models to library-data
* Rename library-models to library-data
* Rename library-models to library-data
* Fix tests
* Fix table URLs
Comment out MB_PREMIUM_EMBEDDING_TOKEN in .env.example to prevent
conflicts with Cypress tests that need to start from a clean state, and
set tokens programmatically.
Fixes#66464.
This was a corner case in `lib.field.resolution` and `qp.util.add-alias-info`
that regressed in 57. It made the QP less smart about repairing a certain
class of broken queries, which regrettably exist in the wild.
The breakage in question was a `:field` ref in later stage, which had a
numeric field ID and a `:join-alias`. If that field ID was not actually
returned as part of the join anymore, fallback metadata would be
generated by looking up the field by ID. That "works" for non-joined
fields but for an explicitly joined field it would use
`source."SOME_FIELD"` rather than `"The Join Alias"."SOME_FIELD"`, which
causes SQL compilation to fail.
### Description
In https://github.com/metabase/metabase/pull/65343 we added a page to purchase Metabase AI (Metabot). Now we are improving the layout and content to better present the product.
### How to verify
1. Ensure you have a token with the `offer-metabase-ai-tiered` feature, but without the `metabot-v3` and `offer-metabase-ai` features.
2. Navigate to `/admin/metabot` and see the page look like in the mock ups.
### Checklist
- [x] Tests have been added/updated to cover changes in this PR
Closes: https://linear.app/metabase/issue/CLO-4669/make-small-changes-to-ai-page-in-admin
* Combine metabot search queries using OR
* Fix zero-terms edge case
* Point out a critical defect
* Urgent typo
* Add docstring to join-results-by-rrf function
Co-authored-by: crisptrutski <881351+crisptrutski@users.noreply.github.com>
* Use non-semantic fallback engine for term-queries
* Group experimental options together
* Small fixes and testing
* Workaround for semantic search matching unwatned stuff
* Renaming and wiring
* Remove unused import
* Remove sharp edge around 0 queries
* Fix bug in disjunction for legacy search. Test coverage for it.
* Complete comment
* Whitespace lint
* Try to fix new tests - run in semantic search test group, simplify test configuration
* More test fixes
* More test fixes
* Whitespace lint
* Make appdb search check more explicit
* don't run metabot semantic search tests on h2 because they rely on the appdb backend being available
* address test comments
* remove experimental flags; make this the normal behavior
* fix test
* fix disjunction on h2
* try again to fix search-test
* move search-test to metabase-enterprise.metabot-v3.tools.semantic-search-test since it now requires semantic search to be available
---------
Co-authored-by: Chris Truter <chris@metabase.com>
Co-authored-by: crisptrutski <881351+crisptrutski@users.noreply.github.com>
* Add a subscription button for modular embeddings (#65704)
* Add an entry to the dashboard subscriptions on modular embeddings
* Hide the subscription when email hasn't been set up
* Add tests
* Add subscription button to interactive and editable dashboard components
* Make dashboard button styles consistent + expose subscriptions button
* Fix types
* Fix wrong default `aria-label` for `ToolbarButton`
* Support `withSubscriptions` prop
* Add tests
* Make it so that modifying tests would be easier
* clarify missing assertions with a comment
* Fix types
* Update Loki Snapshots
* Add `withSubscriptions` to storybook
* Remove duplicate test
* Fix comment date format
* Tighthen ToolbarButton prop type
* Avoiding cramping all test combinations into a single test
* Move shared tests to the correct directory
* Make the function parameter order makes more sense
* Simplify SDK dashboard button visibility logic
* Make shared tests more structured
---------
Co-authored-by: Metabase Automation <github-automation@metabase.com>
* Avoid showing other subscription channels (#66126)
* Make text search easier on an enum
* Prepare state for upcoming change
* Avoid showing the NEW_PULSE screen on SDK
* Add tests
* Fix tests
* Make mocking module more idiomatic
* Update subscription sidebar email edit view content (#66258)
* Update subscription sidebar email edit view content
* Add tests
* Improve test related variable name
* Properly memoize parent callback
* Add dashboard subscriptions option to EAJS dashboards (#66309)
* Expose `withSubscriptions` to EAJS dashboards
* Close subscription sidebar when unchecking `withSubscriptions` checkbox on EAJS
* Add `withSubscriptions` to EAJS dashboards
* Add tests + adjust no email setup tooltip condition
* Add missing unit tests mock
* Only closes sidebar when subscriptions sidebar is open
* Make guest embed warning tooltip aligned with its label
* Fix a test after merging guest embeds
* Update tooltip design from using hovercard to tooltip
* Close the subscription sidebar when the last pulse is archived (#66496)
* close the subscription sidebar when the last pulse is archived
* Ensure we only close the sidebar on the last subscription deletion
* Remove unused code
* simplify mocking
---------
Co-authored-by: Mahatthana (Kelvin) Nomsawadi <me@bboykelvin.dev>
* Added one happy path e2e test (#66506)
* Added one happy path e2e test
* .
* .
* Pass tests on CI with SMTP server
* Remove unused interceptions
* Use explicit assertions
---------
Co-authored-by: Mahatthana (Kelvin) Nomsawadi <me@bboykelvin.dev>
* Run maildev in CI
* Fix admins aren't shown subscription list page after creating first subscription (#66372)
* Run maildev in CI
* Fix admins aren't shown subscription list page after creating first one
* Fix test name typo
* Apply suggestions from code review
Co-authored-by: metabase-bot[bot] <109303359+metabase-bot[bot]@users.noreply.github.com>
* Fix tests after merging another PR
---------
Co-authored-by: metabase-bot[bot] <109303359+metabase-bot[bot]@users.noreply.github.com>
* Make subscriptions on SDK/EAJS a Pro feature (#66392)
* Make subscriptions on SDK a Pro feature
* Add subscription only to EAJS on pro plan
* Correct directory name
* Fix InteractiveDashboard unit tests to use EE token
* Fix StaticDashboard tests to use EE token feature
* Fix EditableDashboard tests to use EE token feature
* Remove OSS tests because there's no SDK on OSS version
* Update Eajs related terms
* Allow subscriptions SDK plugin to be reinitialized
* Standardize plugin enabled property name for EAJS plugins
* Add embed wizard tests
* Add EAJS tests
* Fix OSS test
* Add missing assertions on Enterprise tests
* Optimize API calls on SDK/EAJS
* Clarify comments
* Use initialize plugin pattern for EAJS
* Fix test name
* Fix assertion because upstream content change
* Auto update EAJS subscriptions options after configure email
* Update no email setup tooltip
* Remove debugging code
* Try to fix 1 nasty E2E test
* Fix the samn test again
* Fix the E2E again™️
---------
Co-authored-by: Metabase Automation <github-automation@metabase.com>
Co-authored-by: Sébastien <sebastien@metabase.com>
Co-authored-by: metabase-bot[bot] <109303359+metabase-bot[bot]@users.noreply.github.com>
* When setting settings, include 5 minute cookie of updated timestamp
if other instances have outdated settings cache, this cookie will let
them know what the requestor has seen. The instance can then know it is
out of date. This won't help for instances that aren't hit by the person
setting the setting.
Motivating example is someone who is doing an embedding onboarding
checklist. And then hits a new instance that hasn't yet loaded the "show
embedding homepage" setting so it flips back to the regular
homepage. The hope here is the person doing this checklist will let the
other instance know it is out of date, reload the settings cache and
then see the embedding setting.
* Interesting strategy here for cookies.
I hated that the route would would set a cookie at a particular key, do
timestamp stuff, etc. I know ring is a map and abstraction, but i added
a new paradigm:
```clojure
(defn- add-settings-last-updated-cookie
"Add a cookie with the current settings-last-updated timestamp to the response."
[response]
(assoc-in response [:mb/cookies :cookie/settings-cache-timestamp] true))
```
This means that just the middleware is responsible for checking and
setting this key. This was largely driven by two different namespaces
having a private def of:
```clojure
(def ^:private settings-last-updated-cookie-name "metabase.SETTINGS_LAST_UPDATED")
```
And it's nice if the ring map can indicate to this middleware to handle
all the key and timestamp stuff and it just has to indicate to set this
with `cookie/settings-cache-timestamp`.
So the request looks like this:
```clojure
{:status 204
:headers {"X-Frame-Options" "DENY"
"X-XSS-Protection" "1; mode=block"
"Last-Modified" "Mon, 1 Dec 2025 17:09:59 GMT"
"Strict-Transport-Security" "max-age=31536000"
"X-Permitted-Cross-Domain-Policies" "none"
"Cache-Control" "max-age=0, no-cache, must-revalidate, proxy-revalidate"
"X-Content-Type-Options" "nosniff"
"Content-Security-Policy" "...<lots of junk>..."
"Expires" "Tue, 03 Jul 2001 06:00:00 GMT"}
:body nil
:mb/cookies {:cookie/settings-cache-timestamp true} ;; <---- our new addition
:cookies {"metabase.DEVICE" {:value "60ac9454-9dd1-439a-afa1-921ff029686d"
:http-only true
:path "/"
:expires "Fri, 1 Dec 2045 17:09:59 GMT"
:same-site :lax}}}
```
And then in practice, it looks like this:
Set the setting and get a cookie with the timestamp in response:
```
❯ http put :3000/api/setting/site-name value=foo x-api-key:$KEY
HTTP/1.1 204 No Content
...
Set-Cookie: metabase.SETTINGS_LAST_UPDATED=2025-12-01+17%3A28%3A15.450873%2B00; Path=/; Max-Age=300; SameSite=Lax
...
```
Simulate sending to another instance with a timestamp that exceeds the
currently know timestamp (i'll increment the hour based sent back in
this response)
```
❯ http get :3000/api/user/current x-api-key:$KEY Cookie:metabase.SETTINGS_LAST_UPDATED="2025-12-01+18%3A23%3A51.059155%2B00" | jq '.date_joined'
"2025-12-01T16:51:08.137822Z"
```
(note the 18 vs the 17)
And the logs are this:
```
INFO middleware.settings-cache :: Settings cookie indicates cache is out of date. Refreshing...
DEBUG middleware.log :: GET /api/user/current 200 19ms (11 DB calls) App DB connections: 0/4 Jetty threads: 5/50 (5 idle, 0 queued) (245 total active threads) Queries in flight: 0 (0 queued) {:metabase-user-id 346}
```
* fixes:
- logged known timestamp not weird cookie value
- timestamp is optional, so handle that
- typo in log konwn -> known
* Add tests, add functionality
tests:
ensure it (a) sets the cookie, (b) honors the cookie, and when it
restores the cache, to set the cookie so it doesn't keep thrashing. But
presumably it won't since it's up to date. My "in the future" version is
a bit synthetic.
Functionality wise, updating the cache now returns and this will saet
the header with the most recent cache setting
* fix: show confirmation on escape, clickoutside and browser's back actions for SidebarCacheForm (#53825)
additional: refactor useConfirmOnRouteLeave hook, add an ability to get router across application without property drilling using hook instead
# Conflicts:
# frontend/src/metabase/plugins/index.ts
* fix: small adjustment
* fix: small adjustment
* fix: tests
* fix: tests
* fix: created useStackedSidesheets hook to help control stacked sidesheets
* adds a test for useStackedSidesheets that covers #53825 issue
* adds tests for useConfirmOnRouteLeave that cover #53825 issue
* rename useStackedSidesheets to useStackedModals. Add backward compatibility prop opened
* fix: add history provider as well for the future purposes.
* fix typings
* fix code based on comments from PR
* fix tests
* fix tests
* fix tests
* remove useStackedModals. Write e2e for all "closing" actions
* remove useStackedModals. Write e2e for all "closing" actions
* adjusted code according to PR comments
* fix tests
* make tests that asserts the confirmation modal showing if there are dirty form explicit in order to see which of them is broken in CI
* fix tests
* remove "forward" browser's button testing on trying to close a dirty form
* fix tests
* fix tests
* Fix sub-collections toggle to stick when permission seletor is closed / Handle dirty state gracefully
* Add e2e tests
* Fix outstanding e2e tests
* Remove e2e test that was expecting the previous undesired behavior
* Update test that accounts for issue #30494
* Add clarifying comment
* data studio app permission
* actually transforms are for admins
* check permission for open in data studio
* Auto-set library permissions on data studio permission changes
* Fixed module references
* initial library collection perms
* Fixed module references
* Fixed module references
* Fixed lint errors
* perms
* specs, fixes
* ci
* data studio permission
* Added data-studio permission check to glossary
---------
Co-authored-by: Nathan Voxland <nathan@metabase.com>
* IAM auth note on postgresql
* move to rds
* rds
* mysql note on verify-ca
* Update docs/databases/connections/aws-rds.md
Co-authored-by: Alex Yarosh <alexandra@metabase.com>
---------
Co-authored-by: Alex Yarosh <alexandra@metabase.com>
* Add display_name filtering to table list
* Re-introduce mixed case LIKE for mysql/maria
Must have been lost in a merge conflict
* Escaping rules
* Kondo
* Add Metabase Sample Database to editable tables support
Updated the list of databases that support editable tables to include Metabase Sample Database.
* Update docs/data-modeling/editable-tables.md
* Update docs/data-modeling/editable-tables.md
- If you decide to build the backend, you should run it
- There are no more derived options - everything is explicit
- Running Docker containers is a non-negotiable
- Tests need these supporting services to run
- If you really insist on not running them, either comment out those lines temporarily or do not use this script to set up the e2e environment
- Because of this, there are no more custom envs `QA_DB_ENABLED` and `CYPRESS_QA_DB_MONGO`
- Alternative location (directory) for the e2e copy of the sample-application.db is now set to be `e2e/tmp` since it's already gitignored
* UXW-2209 Updates datamodel tracking triggered_from to be either "data_studio" or "admin" depending on where the metadata is being edited
* Updates e2e tests
* UXW-2352 Fix issue where CopyCardForm resets when the destination collection is changed
* Unskips now-passing e2e tests
* Fix issue where Duplicate modal doesn't close on success
The previous version of the build script (`build-for-test`) had a weird quirk that could be both confusing and dangerous. It was building a backend-only JAR in the same location and under same name as the full-blown, all-in-one Metabase JAR. This means that, by running Cypress locally, one could easily override the previously built healthy JAR. It's not catastrophic but it was confusing.
This PR makes it perfectly clear what we're doing, and there should be no conflicts between the JARs any longer.
* library collection search
* Fix e2e tests from group-16
* Fix a couple tests from group 42
* Fix e2e tests from group-17
* Fix e2e tests from group-18
* tests: add bulk publishing scenario
fix wrong collection link in case of root collection
* UXW-2315, UXW-2314
* UXW-2294
* more UXW-2315, UXW-2314
* more UXW-2315, UXW-2314
* correct collection types
* Remove Browsing Memory Feature
* Revert "Revert "Don't return library collections by default from APIs""
This reverts commit 814c49473b.
* Revert "Revert "Fixed null type filtering""
This reverts commit 25e0d78408.
* fetch full tree to have library collections visible
* show library in permissions
* Fixing tests
* fix infinite re-render in tree
* rewrite unit tests and fix max depth problem
* fix: publish property not validated in the tree view after table publishing
* Move bulk editing apis to enterprise (#65968)
* Make sidebar sticky
* test: bulk attribute actions for tables, schemas and dbs
* fix more unit tests
* Copy preview section tests
* Copy error handling tests
* Copy undos tests
* Update test
* test: update new ee endpoints paths
* Make new bulk table APIs ee-only (fe)
- Closes GDGT-1251
* Fix global H.DataModel.visit overrides
* Remove redundant cy.reload
* Remove redundant cy.reload
* add select/deselect tests
* use helper
* data picker using proper library endpoint. Fixing types I broke. Endless Shame
* Fix Tree skeleton resizing multiple times during loading
- Closes GDGT-1253
* Improve icon
* Fill content
* don't call getCollectionItems on database collection
* add search tests and support shift click in search
* default table publish collection
* fox: reset bulk form values when selection changes
* Fix e2e tests from group-19
* fix: fix missing dependencies in useEffect gdgt-1245
* test: refactor test setup action order
* Fix e2e tests from group-20
* Fix e2e tests from group-22
* Fix e2e tests from group-23
* test: restore default db as the first step
* Kondo the kondo config
* default collections for metrics and models outside of data studio
* Move edit test to ee
* Fix list filtering now default is bronze
* Move bulk-edit-test to ee
* Fix table metadata for test
* Revert "Added `collection` argument to search"
This reverts commits:
- 21999e01
- b0f71c03ea
- 4fa0c3b422
* Fixed OSS loading of tables
* Fixed OSS loading of tables
* add tests for filtering
* Fix e2e tests from group 41 / joins-repros
* more tests
* Fix e2e tests from group-20
* Fix e2e tests from group-23
* Fix e2e tests from group-24
* Fix e2e tests from group-26
* Fix e2e tests from group-27
* Fix e2e tests from group-28
* Fix e2e tests from group-30
* constuct initial paths from library collections
* segments editor, hasEnabledNestedQueries
* Fix a couple more (something between groups 40 and 43)
* Fixed module config
* keyboard navigation
* Added library/tree API
* Fix some e2e tests from groups 40 and 42
* Fixing tests
* disable scoped search on the 'database collection', models.cy.spec should be passing now
* Fix e2e tests from group-33
* Minor cleanup
* Fix e2e tests from group-34
* Fixing tests
* Fix e2e tests from group 37 (except one depending on Recents tab)
* Starts fixing e2e tests from oss group
* cleanup
* wait to focus
* hide our analytics in the data studio
* Added library/tree API
* merge library and data studio
* change backend library flag validation to data-studio
* tidy things up
* Make bulk edit inputs non-clearable
- This prevents sending "empty" requests when clearing values
- And clearing does not really make sense here
* fix: test: table rescan and discard mocks are wrong
* fix: test: wrong mock in oss sync table schema
* Remove ^:parallel from bulk edit tests
They can contend and cause deadlocks
* fix: test: failure due to repeated mocking of same route
* test: refactor: cypress table tests
* Update e2e test
* Update e2e test
* Update e2e test
* fix browse nav section test
* Add missing token
* add check icon for published tables
* Removed unused import
* Updates some e2e tests from oss group, removes some now-invalid tests
* test: use default external db, skip transform test
* Starts fixing e2e tests from mongo group
* remove background from not active table in search
* specs
* Fix e2e tests from group-15
* hack fix ci while i'm on meetings
* scoped search is back
* Fix some e2e tests from group-32
* table picker fixes
* Fixed merge
* Fix e2e test from group-42
* merge fixes
* re-enable the recents tab on the data picker modal
* Avoid race condition on list-table-filtering-test
* fix embedding types
* mini picker unit tests
* Fixing tests
* ci, fixes
* fix library collection bugs
* cleanup data studio plugin
* fix lots of unit tests
* fix import
* fix
* Fixed unused import
* Fixed accidental #p
* ci
* revert util
* specs
* specs
* specs
* fix bug / bad type in SchemaFormSelect (#65995)
* Use actual anti join in GET /table query
* Remove old comments
* Activate token after restore
* test: unskip table metadata transform test
* test: fix search tables e2e test
* fix: test: selection in table picker
* Fix Source input disabled when filtering by Metabase transform
- Closes GDGT-1259
* test: fix: filter unused table
* Fix e2e test
* fix: test: datamodel toasts and errors
* Fix e2e test
* Fix e2e tests from group-9
* Fix flaky tests
* GDGT-1244: Fix DB tree virtualization (#66125)
* GDGT-1244: Fix tree virtualization. Replace top with translateY to improve performance
* GDGT-1244: Fix scroll bug. Refactor border styling
* bring back modeling empty state
* test: adjust publishing test
* Fix e2e test from group-22
* Fix e2e test from group-25
* fix allowing cross DB joins
* properly disable invalid items in MiniPicker
* Fix e2e test from group-26
* Fix straggler from oss group
* Starts fixing e2e tests from group-32
* unit test adjustments
* Fix remaining issue with e2e group 37
* fix item filtering
* Update Loki Snapshots
* add recent views to mini picker
* test recents and backparsing
* additional filtering test
* fix minipicker shows items it should not show
* Fixed :here in library API
* Fixed :here in library API
* UXW-2358 publishing outside of the data studio
* Fixed :here in library API
* snapshot
* Fix most e2e tests from group-32
* Lint fixes
* Fixed failing test
* revert fix
* Reapply "Added `collection` argument to search"
This reverts commit f28d9d5af9.
* Quick fix in CollectionPicker/utils.ts
* Fix flaky Remote Sync test (#65862)
* Clean up backend-test matrix (#66158)
* Fix timeseries value detection (#66161)
* Prevent driver tests (#66156)
Co-authored-by: Bryan Maass <bryan.maass@gmail.com>
* Fetch uberjar using a proper action (#66165)
* fix embedding column picking
* Trying CI with search_index.verified
* Fixed failing test
* Translate label
* Fixed failing test
* fix: published info not updated in the tree picker after deleting model
* fix: refactor: publish collection link tests
* Fixed failing test
* Fix remaining e2e tests from group-32
* FileUploadStatus unit test, quesiton reproductions
* create data studio plugin file
* fix hidden items some more
* Fix issue where confirmation modal closes then reopens again then closes again when you archive a dashboard
* fix dark mode
* UXW-2367
* UXW-2378
* Fixed merge issues
* this element is not visible because you can't even open incompatible sample database db
* Fix e2e test from group-14
* fix embedding data picker clicks
* filtering fixes for transforms
* UXW-2372
* update mini picker specs after a fix
* fix flaky join test
* Fixing in-place support for collections
* fix: cannot scroll search results
* ui: tweak search results checkbox spacing to match tree results
* fix: scroll position jump when entering field sort mode gdgt-1269
* Revert "fix: scroll position jump when entering field sort mode gdgt-1269"
This reverts commit bfecdec661.
* UXW-2279 Changes Data Studio's Glossary page background color to background-secondary
* Fix linters
* Fix e2e tests
* fixes for sqlserver
* fix driver changelog
* fix
* more fix
* fix: support params in bigq
* fix: inline params for sqlserver
* fix test
* fix: we do want the unique key
* fix: missing nil in test
* fix clickhouse
---------
Co-authored-by: Ryan Laurie <iethree@gmail.com>
Co-authored-by: Brad Anderson <brad@metabase.com>
Co-authored-by: Anderson <anderson@metabase.com>
Co-authored-by: Jakub Chodorowicz <jakub@chodorowicz.com>
Co-authored-by: Aleksandr Lesnenko <alxnddr@gmail.com>
Co-authored-by: Nathan Voxland <nathan@metabase.com>
Co-authored-by: Uladzimir Havenchyk <uladzimir@metabase.com>
Co-authored-by: Uladzimir Havenchyk <125459446+uladzimirdev@users.noreply.github.com>
Co-authored-by: Ngoc Khuat <qn.khuat@gmail.com>
Co-authored-by: Kamil Mielnik <kamil@kamilmielnik.com>
Co-authored-by: Nick Fitzpatrick <nickfitz.582@gmail.com>
Co-authored-by: Dan Stone <wotbrew@gmail.com>
Co-authored-by: Sloan Sparger <sloansparger@users.noreply.github.com>
Co-authored-by: Aleksandr Lesnenko <alxnddr@users.noreply.github.com>
Co-authored-by: Metabase Automation <github-automation@metabase.com>
Co-authored-by: John Swanson <john.swanson@metabase.com>
Co-authored-by: Nemanja Glumac <31325167+nemanjaglumac@users.noreply.github.com>
Co-authored-by: Denis Berezin <denis.berezin@metabase.com>
Co-authored-by: dpsutton <dan@dpsutton.com>
Co-authored-by: Bryan Maass <bryan.maass@gmail.com>
Co-authored-by: Nicola Mometto <nicola.mometto@metabase.com>
This pull request refactors and streamlines the configuration for CI workflows and file path groupings in .github/file-paths.yaml and updates related workflow logic in .github/workflows/run-tests.yml. The changes focus on consolidating shared configuration, improving maintainability, and removing redundant or unused path groups.
* Add :database_type to metabot tools that return column metadata
Add :database_type to metabot tools that return column metadata in order to give metabot information about the real
database type when generating SQL.
Examples of places the :database_type now appears:
- :used_tables in :user_is_viewing context
- {table,card,metric}-details
- get-query-details
- enhanced-database-tables
* Fix test after rebase onto #66271
Fix get-table-details-database-type-test after rebase (1) use new field-id formatting (2) get-table-details no longer returns implicitly-joinable fields
in the top-level `:fields`.
* More flexible approach to resolving columns from metabot field_ids
* fix one field-stats-test
* fix field-stats-test
* support nano-ids in field ids (?)
* remove some dead code
* Exclude implicitly joinable fields when fetching cols for table
* try switching field format to use dash
* fix field IDs in a number of tests
* Kondo
* add some unit tests for new field parsing & column lookup helper fns
* Fix more tests
* Fix more tests
* fix regex
* New approach -- use wide field IDs in related_tables and add a related_by field for each table
* Fix some tests
* remove unneeded table
* cleanup, new test
* Revert to passing in & matching on expected prefix
* Address tyler's comments
* accept a regex for the prefix as well
* add back small malli group-by fix from the last PR - removed unintentionally during revision
* Make uberjar (artifact) file name configurable via env var
Add the ability to customize the output filename for the uberjar build
via the MB_JAR_FILENAME environment variable. This allows backend-only
test builds to use a separate filename (e.g., metabase-backend.jar)
to avoid overwriting the regular all-in-one metabase.jar.
* fix: add missing channels to copy & rendering logic
Mentioned webhooks and Slack where necessary to stay consistent with the
admin UI
* fix: standardize lowercase 'admin' & prefer 'email' over 'SMTP'
* fix: remove extra comma on wording
* test: adjust e2e test assertions
* test: add `DashboardSubscriptionMenuItem.unit.spec.tsx`
* test: extract single case from `describe` block
* test: fix subscriptions e2e case & text lookup on alert spec
Fixes#66199.
`metabase.legacy-mbql.schema`, now used for normalizing MBQL 4 in
place of the old bespoke normalization in earlier versions, regressed in
this weird but legitimate use case: Writing a custom aggregation like
`MAX("String Literal")`.
That's an odd thing to do, but it can be useful to provide a label in an
aggregated query. This is the MBQL equivalent of
`(SELECT ..., 'A' as label FROM ...) UNION ALL (SELECT ..., 'B' as label ...)`
This pull request refactors the Cypress runner utility to simplify error handling and improve consistency in how test failures are reported. The main change is that runCypress now throws errors on test failures instead of relying on an external exit or cleanup function, which streamlines its usage across different scripts.
This ensures consistent error propagation and simplifies the control flow for callers.
These tools already included :database_id in the results; we now include the :database_engine as well, to give metabot
a reminder about which SQL dialect it should generate.
* Bump org.apache.arrow/arrow-memory-netty from 17.0.0 to 18.3.0
Inspect dependency changes here:
* Upgrade arrow-memory-core to version 18.3.0
Updated the version of arrow-memory-core from 17.0.0 to 18.3.0.
* Consolidate changes
* Fix formatting
---------
Co-authored-by: bryan <bryan.maass@gmail.com>
Co-authored-by: Luis Paolini <paoliniluis@gmail.com>
Co-authored-by: Cam Saul <github@camsaul.com>
Co-authored-by: Cam Saul <1455846+camsaul@users.noreply.github.com>
* Fix some e2e tests from group-9
* test: add bulk sync action e2e test
* fix column selection
* Fixed module config
* Fix a few e2e tests from oss group
* Fix e2e tests from groups 46 and 47
* Fix e2e tests from groups 44 and 45
* Fix group-3 straggler, some of group-8
* Potentially disastrous change (increases item-picker-level of SchemaList and TableList) to fix test in group-8
* Fix group-6 straggler, some more of group-8
* Updates some more e2e tests from group-9 (some actual failures left)
* Fix e2e tests from group-14
* do enterprise code correctly
* library collection search
* Fix e2e tests from group-16
* Fix a couple tests from group 42
* Fix e2e tests from group-17
* Fix e2e tests from group-18
* tests: add bulk publishing scenario
fix wrong collection link in case of root collection
* UXW-2315, UXW-2314
* UXW-2294
* more UXW-2315, UXW-2314
* more UXW-2315, UXW-2314
* correct collection types
* Remove Browsing Memory Feature
* Revert "Revert "Don't return library collections by default from APIs""
This reverts commit 814c49473b.
* Revert "Revert "Fixed null type filtering""
This reverts commit 25e0d78408.
* fetch full tree to have library collections visible
* show library in permissions
* Fixing tests
* fix infinite re-render in tree
* rewrite unit tests and fix max depth problem
* fix: publish property not validated in the tree view after table publishing
* Move bulk editing apis to enterprise (#65968)
* Make sidebar sticky
* test: bulk attribute actions for tables, schemas and dbs
* fix more unit tests
* Copy preview section tests
* Copy error handling tests
* Copy undos tests
* Update test
* test: update new ee endpoints paths
* Make new bulk table APIs ee-only (fe)
- Closes GDGT-1251
* Fix global H.DataModel.visit overrides
* Remove redundant cy.reload
* Remove redundant cy.reload
* add select/deselect tests
* use helper
* data picker using proper library endpoint. Fixing types I broke. Endless Shame
* Fix Tree skeleton resizing multiple times during loading
- Closes GDGT-1253
* Improve icon
* Fill content
* don't call getCollectionItems on database collection
* add search tests and support shift click in search
* default table publish collection
* fox: reset bulk form values when selection changes
* Fix e2e tests from group-19
* fix: fix missing dependencies in useEffect gdgt-1245
* test: refactor test setup action order
* Fix e2e tests from group-20
* Fix e2e tests from group-22
* Fix e2e tests from group-23
* test: restore default db as the first step
* Kondo the kondo config
* default collections for metrics and models outside of data studio
* Move edit test to ee
* Fix list filtering now default is bronze
* Move bulk-edit-test to ee
* Fix table metadata for test
* Revert "Added `collection` argument to search"
This reverts commits:
- 21999e01
- b0f71c03ea
- 4fa0c3b422
* Fixed OSS loading of tables
* Fixed OSS loading of tables
* add tests for filtering
* Fix e2e tests from group 41 / joins-repros
* more tests
* Fix e2e tests from group-20
* Fix e2e tests from group-23
* Fix e2e tests from group-24
* Fix e2e tests from group-26
* Fix e2e tests from group-27
* Fix e2e tests from group-28
* Fix e2e tests from group-30
* constuct initial paths from library collections
* segments editor, hasEnabledNestedQueries
* Fix a couple more (something between groups 40 and 43)
* Fixed module config
* keyboard navigation
* Added library/tree API
* Fix some e2e tests from groups 40 and 42
* Fixing tests
* disable scoped search on the 'database collection', models.cy.spec should be passing now
* Fix e2e tests from group-33
* Minor cleanup
* Fix e2e tests from group-34
* Fixing tests
* Fix e2e tests from group 37 (except one depending on Recents tab)
* Starts fixing e2e tests from oss group
* cleanup
* wait to focus
* hide our analytics in the data studio
* Added library/tree API
* merge library and data studio
* change backend library flag validation to data-studio
* tidy things up
* Make bulk edit inputs non-clearable
- This prevents sending "empty" requests when clearing values
- And clearing does not really make sense here
* fix: test: table rescan and discard mocks are wrong
* fix: test: wrong mock in oss sync table schema
* Remove ^:parallel from bulk edit tests
They can contend and cause deadlocks
* fix: test: failure due to repeated mocking of same route
* test: refactor: cypress table tests
* Update e2e test
* Update e2e test
* Update e2e test
* fix browse nav section test
* Add missing token
* add check icon for published tables
* Removed unused import
* Updates some e2e tests from oss group, removes some now-invalid tests
* test: use default external db, skip transform test
* Starts fixing e2e tests from mongo group
* remove background from not active table in search
* specs
* Fix e2e tests from group-15
* hack fix ci while i'm on meetings
* scoped search is back
* Fix some e2e tests from group-32
* table picker fixes
* Fixed merge
* Fix e2e test from group-42
* merge fixes
* re-enable the recents tab on the data picker modal
* Avoid race condition on list-table-filtering-test
* fix embedding types
* mini picker unit tests
* Fixing tests
* ci, fixes
* fix library collection bugs
* cleanup data studio plugin
* fix lots of unit tests
* fix import
* fix
* Fixed unused import
* Fixed accidental #p
* ci
* revert util
* specs
* specs
* specs
* fix bug / bad type in SchemaFormSelect (#65995)
* Use actual anti join in GET /table query
* Remove old comments
* Activate token after restore
* test: unskip table metadata transform test
* test: fix search tables e2e test
* fix: test: selection in table picker
* Fix Source input disabled when filtering by Metabase transform
- Closes GDGT-1259
* test: fix: filter unused table
* Fix e2e test
* fix: test: datamodel toasts and errors
* Fix e2e test
* Fix e2e tests from group-9
* Fix flaky tests
* GDGT-1244: Fix DB tree virtualization (#66125)
* GDGT-1244: Fix tree virtualization. Replace top with translateY to improve performance
* GDGT-1244: Fix scroll bug. Refactor border styling
* bring back modeling empty state
* test: adjust publishing test
* Fix e2e test from group-22
* Fix e2e test from group-25
* fix allowing cross DB joins
* properly disable invalid items in MiniPicker
* Fix e2e test from group-26
* Fix straggler from oss group
* Starts fixing e2e tests from group-32
* unit test adjustments
* Fix remaining issue with e2e group 37
* fix item filtering
* Update Loki Snapshots
* add recent views to mini picker
* test recents and backparsing
* additional filtering test
* fix minipicker shows items it should not show
* Fixed :here in library API
* Fixed :here in library API
* UXW-2358 publishing outside of the data studio
* Fixed :here in library API
* snapshot
* Fix most e2e tests from group-32
* Lint fixes
* Fixed failing test
* revert fix
* Reapply "Added `collection` argument to search"
This reverts commit f28d9d5af9.
* Quick fix in CollectionPicker/utils.ts
* Fix flaky Remote Sync test (#65862)
* Clean up backend-test matrix (#66158)
* Fix timeseries value detection (#66161)
* Prevent driver tests (#66156)
Co-authored-by: Bryan Maass <bryan.maass@gmail.com>
* Fetch uberjar using a proper action (#66165)
* fix embedding column picking
* Trying CI with search_index.verified
* Fixed failing test
* Translate label
* Fixed failing test
* fix: published info not updated in the tree picker after deleting model
* fix: refactor: publish collection link tests
* Fixed failing test
* Fix remaining e2e tests from group-32
* FileUploadStatus unit test, quesiton reproductions
* create data studio plugin file
* fix hidden items some more
* Fix issue where confirmation modal closes then reopens again then closes again when you archive a dashboard
* fix dark mode
* UXW-2367
* UXW-2378
* Fixed merge issues
* this element is not visible because you can't even open incompatible sample database db
* Fix e2e test from group-14
* fix embedding data picker clicks
* filtering fixes for transforms
* UXW-2372
* update mini picker specs after a fix
* fix flaky join test
* Fixing in-place support for collections
* fix: cannot scroll search results
* ui: tweak search results checkbox spacing to match tree results
* fix: scroll position jump when entering field sort mode gdgt-1269
* Revert "fix: scroll position jump when entering field sort mode gdgt-1269"
This reverts commit bfecdec661.
* UXW-2279 Changes Data Studio's Glossary page background color to background-secondary
---------
Co-authored-by: Nathan Voxland <nathan@metabase.com>
Co-authored-by: Brad Anderson <brad@metabase.com>
Co-authored-by: Jakub Chodorowicz <jakub@chodorowicz.com>
Co-authored-by: Ryan Laurie <iethree@gmail.com>
Co-authored-by: Anderson <anderson@metabase.com>
Co-authored-by: Aleksandr Lesnenko <alxnddr@gmail.com>
Co-authored-by: Uladzimir Havenchyk <uladzimir@metabase.com>
Co-authored-by: Uladzimir Havenchyk <125459446+uladzimirdev@users.noreply.github.com>
Co-authored-by: Ngoc Khuat <qn.khuat@gmail.com>
Co-authored-by: Kamil Mielnik <kamil@kamilmielnik.com>
Co-authored-by: Nick Fitzpatrick <nickfitz.582@gmail.com>
Co-authored-by: Dan Stone <wotbrew@gmail.com>
Co-authored-by: Sloan Sparger <sloansparger@users.noreply.github.com>
Co-authored-by: Stas Gavrylov <stas@metabase.com>
Co-authored-by: Aleksandr Lesnenko <alxnddr@users.noreply.github.com>
Co-authored-by: Metabase Automation <github-automation@metabase.com>
Co-authored-by: John Swanson <john.swanson@metabase.com>
Co-authored-by: Nemanja Glumac <31325167+nemanjaglumac@users.noreply.github.com>
Co-authored-by: Denis Berezin <denis.berezin@metabase.com>
Co-authored-by: dpsutton <dan@dpsutton.com>
Co-authored-by: Bryan Maass <bryan.maass@gmail.com>
* GDGT-1252: Make list view consider mini-bar-chart setting
* GDGT-1252: Fix incorrect model settings update when changing columns settings
* GDGT-1252: e2e test
* GDGT-1252: Fix unit test for ListViewItem
* GDGT-1252: Address PR comments. Minor refactoring
* GDGT-1252: Refactor DatasetEditor to get rid of modelSettings hack
* GDGT-1252: Update e2e test to cover settings persistence
* Update Loki Snapshots
* Revert Loki update
---------
Co-authored-by: Metabase Automation <github-automation@metabase.com>
### Description
Missing billing information is treated as an error for purposes of
displaying the purchase page with prices. Non-store users cannot
retrieve billing information. Thus, previously, they would always see
an error page: "Error fetching information about available add-ons."
With this change, they see the "Please ask a Metabase Store Admin" page
as intended.
We achieve this by splitting the page content for store users from that
of non-store users. Only the former needs to load data (billing and
add-on information), so only that page needs `LoadingAndErrorWrapper`.
For store users we show no dynamic content that could fail to load
(apart from a store user's email address, which we gracefully omit if
it is missing).
### How to verify
1. Ensure you have a token with the `offer-metabase-ai-tiered`
feature, but without the `metabot-v3` and `offer-metabase-ai`
features.
2. As a Metabase user with an email address that *does not* match any
of the token's organization's store user email addresses, navigate
to `/admin/metabot` and see "Please ask a Metabase Store Admin".
3. As a Metabase user with an email address that *does* match any
of the token's organization's store user email addresses, navigate
to `/admin/metabot` and see the purchase page with tier selector.
### Checklist
- [x] Tests have been added/updated to cover changes in this PR
Fixes: 2b76c38e1f
Closes: https://metaboat.slack.com/archives/C07RMUQ0GE5/p1763644357893079
This PR adds a non-destructive dev script whose only job is to detect and report on your local development environment setup.
## What it does
**`./bin/dev-requirements`** detects:
- Version managers:
- nvm, fnm, n (Node.js version managers)
- volta, asdf, mise (multi-language support)
- jenv, sdkman (Java version managers)
- Installed development tools and their versions
- Whether your setup meets Metabase's requirements
It cross-references detected versions against project configuration files:
- Node.js version from `.nvmrc`
- Clojure version from `deps.edn`
- Java 21 Temurin distribution requirement
## What it doesn't do
- No installations
- No modifications
- No assumptions about your preferences
The script is purely informational - it shows you what you have, what's required, and recommends mise for version management. Perfect for new contributors or when troubleshooting setup issues.
* feat: use boolean flag for remote-sync status
It turns out type is too overloaded to have a 'remote-synced' collection
type. This is behavior that we might want to share around other typed
collections, so this migrates us to use a boolean column for this value instead.
* fixup! e2e test find synced collection
* fixup! e2e collection fixture has correct values
* fixup! make sure flag is set on insert/update
* bump databricks
This moves over to a thin jar (or regular jar) which provides jdbc
classes and lists its dependencies. We can now bump or pin as we
like. The previous versions were uberjars that had their dependencies
packaged inside.
```clojure
clj -Sdeps '{:deps {com.databricks/databricks-jdbc-thin {:mvn/version "3.0.5"}}}'
Clojure 1.12.1
user=> (Class/forName "com.databricks.client.jdbc.Driver")
com.databricks.client.jdbc.Driver
```
```clojure
clj -Xdeps tree :extra '{:deps {com.databricks/databricks-jdbc-thin {:mvn/version "3.0.5"}}}'
org.clojure/clojure 1.12.1
. org.clojure/spec.alpha 0.5.238
. org.clojure/core.specs.alpha 0.4.74
com.databricks/databricks-jdbc-thin 3.0.5
. com.databricks/databricks-sdk-java 0.69.0
X org.slf4j/slf4j-api 2.0.9 :older-version
. org.apache.commons/commons-configuration2 2.11.0 :newer-version
X org.apache.commons/commons-lang3 3.14.0 :older-version
. org.apache.commons/commons-text 1.12.0
X org.apache.commons/commons-lang3 3.14.0 :older-version
. commons-logging/commons-logging 1.3.2 :newer-version
X com.fasterxml.jackson.core/jackson-databind 2.15.2 :older-version
...
```
contrasted with
```clojure
❯ clj -Xdeps tree :extra '{:deps {com.databricks/databricks-jdbc {:mvn/version "3.0.1"}}}'
org.clojure/clojure 1.12.1
. org.clojure/spec.alpha 0.5.238
. org.clojure/core.specs.alpha 0.4.74
com.databricks/databricks-jdbc 3.0.1
```
* bump to run drivers
* include test namespace change to trigger driver tests
* reorg
* added reinit fns to oss plugins
* ಠ_ಠ
* Added doc and eslint to prevent import of individual reinitialize functions
* added short doc to reinitialize functions to indicate they're internal
### Don't: Use `ms/TemporalString` for Java Time objects in response schemas
```clojure
;; WRONG - Java Time objects aren't strings yet
[:date_joinedms/TemporalString]
;; RIGHT - schemas validate BEFORE JSON serialization
[:date_joined:any];; Java Time object, serialized to string by middleware
[:last_login[:maybe:any]];; Java Time object or nil
```
**Why:** Response schemas validate the internal Clojure data structures BEFORE they are serialized to JSON. Java Time objects like `OffsetDateTime` get converted to ISO-8601 strings by the JSON middleware, so the schema needs to accept the raw Java objects.
### Don't: Use `[:sequential X]` when the data is actually a set
```clojure
;; WRONG - group_ids is actually a set
[:group_ids{:optionaltrue}[:sequentialpos-int?]]
;; RIGHT - matches the actual data structure
[:group_ids{:optionaltrue}[:maybe[:setpos-int?]]]
```
**Why:** Toucan hydration methods often return sets. The JSON middleware will serialize sets to arrays, but the schema validates before serialization.
### Don't: Create anonymous schemas for reused structures
Use `mr/def` for schemas used in multiple places:
```clojure
(mr/def::User
[:map
[:idpos-int?]
[:emailstring?]
[:namestring?]])
```
## Finding Return Types
1.**Look at the function being called**
```clojure
(api.macros/defendpoint:get"/:id"
[{:keys[id]}]
(t2/select-one:model/Field:idid));; Returns a Field instance
```
2.**Check Toucan models for structure**
Look in `src/metabase/*/models/*.clj` for model definitions.
metabase.lib.convert/->legacy-MBQL{:message"Conversion to legacy MBQL is discouraged, please use Lib to manipulate MBQL queries"}
metabase.lib.core/->legacy-MBQL{:message"Conversion to legacy MBQL is discouraged, please use Lib to manipulate MBQL queries"}
metabase.lib.equality/find-column-indexes-for-refs{:message"Use lib.equality/closest-matches-in-metadata or lib.equality/find-closest-matches-for-refs instead of lib.equality/find-column-indexes-for-refs"}
metabase-enterprise.transforms.interface/execute!{:message"Use metabase-enterprise.transforms.execute/execute! instead of metabase-enterprise.transforms.interface/execute!"}
next.jdbc/active-tx?{:message"Use metabase.app-db.core/in-transaction? to see if we are in a transaction"}
toucan2.core/debug{:message"Don't commit code using t2/debug"}
toucan2.tools.debug/debug{:message"Don't commit code using t2/debug"}}
toucan2.tools.debug/debug{:message"Don't commit code using t2/debug"}
macaw.core/parsed-query{:message"Use metabase.driver.util/parsed-query instead of macaw.core/parsed-query"}}
:discouraged-namespace
{camel-snake-kebab.core{:message"CSK is not Turkish-safe, use the versions in metabase.util instead."}
toucan2.tools.with-temp/with-temp{:message"Use lib.tu/mock-metadata-provider in QP tests. Exceptions: tests for code that doesn't use MPs or models not supported by MPs"}}
> Watch out for these deprecated props and gotchas for Metabase 57 onwards, for modular embedding:
>
> 1. \`config\` prop on MetabaseProvider no longer exist - it is replaced by \`authConfig\`.
> 2. \`authProviderUri\` field no longer exist.
> 3. \`jwtProviderUri\` optional field only exists in v58+. This is used to make JWT auth faster by skipping the \`GET /auth/sso\` discovery request. Not needed for initial implementation.
> 4. \`fetchRequestToken\` is not needed by default. This is only used to customize how the SDK fetches the request token.
> 5. Numeric IDs must be integers not strings, e.g. \`dashboardId={1}\`. When the ID is retrieved from the URL and it is numeric, convert it to an integer via \`parseInt\` before passing it to the SDK. IDs can also be strings for entity ids, so you should not parse all IDs as numbers if entity ids are also to be expected.
`;
}
/**
* Generate llms-full.txt file for a specific section
* @param {string} branch - The branch name
* @param {Object} config - Section config object with section name and optional getPreamble function
* @param {Array} files - Array of all markdown file objects
* @param {string} baseUrl - Base URL for raw GitHub links
Some files were not shown because too many files have changed in this diff
Show More
Reference in New Issue
Block a user
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.