Update E2E test timings with averaged data
Collected timing data from 7 days of test runs.
All specs with timing data were included (simple averages).
🤖 Generated automatically by update-e2e-timings workflow
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: GitHub Actions <actions@github.com>
Fixes QUE2-383.
Uses `:lib/card-id` in metadata to check if a column came from a card at any point.
If so, it should use a name-based `:field` ref rather than an ID-based one.
This makes the refs to joined columns consistent with those to main source columns,
based on whether the source is a table or card.
* Tenant attributes and collections improvements
First round of edits to add the ability to create tenant attributes using SSO.
* Revise tenant-specific access for non-admin internal users
* Edits based on Alberto's comments
* Move tenant claims info to claims section, link to it from attributes
* Apply suggestion from @alexyarosh
Co-authored-by: Alex Yarosh <alexandra@metabase.com>
* Moved tenant attribute logic and simplified sentence
* Fix a sentence, improve code example, and clean up permissions bullet points
---------
Co-authored-by: Alex Yarosh <alexandra@metabase.com>
As of Cypress 15.10.0, `Cypress.env()` is deprecated. This commit migrates
all usages to the new recommended APIs:
- Non-sensitive config values → `Cypress.expose()` (synchronous)
- Sensitive token values → `cy.env()` (asynchronous)
Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
* revert sorting by id on parameters serialization, parameters ordering is significant and must be preserved
* revert, we're storing position
* feat: add :position to parameters, backfill
* feat: backfill position on imported parameters
* chore: tests
* FE: update position
* chore: gitignore python-sources.lock
* chore: ensure we get position
* restrict to dashboard params
* tests
* simpler approach: just sort by position on serialization export/import
The legacy .insertAll API (and newer Storage API) do not work well with
tables that are created/dropped frequently. This is because their
metadata caches can take a long time to up date.
Python transforms use the drop-recreate fallback strategy for bigquery
so we observe this inconsistency for every run after the first, even
with absurd multi minute timeouts.
The SQL API does not have a problem, so we'll use that. We expect a
performance regression but its better to work slowly than not work at
all.
Other changes:
- Removed now redundant bigquery retry loop
- Dropped dictionary support, we cannot support it easily and it cannot be
considered higher prio than bigquery working at all.
- Added correct parameter typing for JSON.
- Fixed ordering issues in tests (assumed row order when it is
non-deterministic)
* fix: real error codes in streaming responses
* fixup! try faking old errors deeper in the stack
* fixup! wrap fetch endpoint in embedding too
* fixup! treat string only errors differently
* fixup! handle errors in a couple other places
* fixup! hard to figure out what kind of error it is
* fixup! typecasts
* fixup! test arity
* fixup! remove blanket 403 page from dataset
* fixup! simplify backend 403 responses
* fixup! a couple missed tests
* fixup! add mockresponse for mockclient
* fixup! other failing tests
* fixup! last failing test
* fixup! add another OSS check
* fixup! one more oss test
* fixup! update status codes on some more tests
* fixup! use committed function
- Was using wrong project / creds
- Using two different sha names only worked when shas were not used as
qualifiers in the query (they are for bigquery)
Added the ability to override the hash-key used for tx/hash-dataset to
support forcing a hash collision (as is required by routing).
Many uses of `lib.util/source-card-id` and `source-table-id` were
followed by calling `lib.metadata/card` or `lib.metadata/table`
respectively.
This PR adds two new functions `lib.metadata.calculation/primary-source-card`
and `primary-source-table` which look up the card/table directly.
It also adds `lib.metadata.calculation/primary-source` for the handful
of places where whichever of `primary-source-card` or `primary-source-table`
returns non-nil gets used, such as in some drill-thrus.
- Adds a fully automated `run!` entry point to `dev.mutation-testing` that orchestrates the entire mutation testing workflow: generate report → create Linear project → group functions → write tests via `claude -p` → verify → retry →
commit → create draft PRs
- Replaces the previous manual, step-by-step SKILL.md instructions (which required the operator to execute each step) with Clojure functions that handle the full pipeline automatically
- Key new functions: `parse-namespace`, `group-functions`, `build-test-prompt`, `invoke-claude!`, `verify-and-retry!`, `process-group!`, `print-summary!`
## Motivation
The mutation testing workflow previously required ~20 manual steps per namespace, with the Claude operator interpreting skill instructions and running each step. This was slow, error-prone, and required constant supervision. Now you just
call `(mut-test/run! 'metabase.lib.order-by)` and walk away.
## How it works
1. **`parse-namespace`** — derives source/test paths, short names, auto-detects `.cljc` vs `.clj`
2. **`group-functions`** — clusters private functions with their public entry points based on test overlap, skips fully-covered functions
3. **`build-test-prompt`** — assembles a prompt with function source, existing tests, and surviving mutations
4. **`invoke-claude!`** — shells out to `claude -p` with scoped tool access (`Edit`, `Read`, `Bash(clj-nrepl-eval*)`)
5. **`verify-and-retry!`** — checks which mutations were killed, retries up to 2x with targeted prompts for survivors
6. **`process-group!`** — full per-group pipeline: branch → invoke Claude → verify → commit & push → Linear issue → draft PR
7. **`run!`** — ties it all together, catches errors per group so one failure doesn't stop the batch
Given browser or API access to a Metabase instance, the tools in this PR
can recursively fetch cards and all the metadata (tables, fields, etc.)
required to run them through QP preprocessing and compilation.
Of course this cannot actually _execute_ the compiled form, since you
don't have access to the user's DWH, but it enables debugging of
hard-to-reproduce user bug reports in your local REPL.
This works by making API calls like `/api/card` and
`/api/card/query_metadata` until a complete picture of the metadata is
acquired. This is poured into a `MetadataProvider`, which is all the QP
needs to analyze and compile the query.
Update E2E test timings with averaged data
Collected timing data from 7 days of test runs.
All specs with timing data were included (simple averages).
🤖 Generated automatically by update-e2e-timings workflow
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: GitHub Actions <actions@github.com>
Adds Box Plot viz and a Scalar viz with conditional colors to the Example dashboard.
The resources/sample-content.edn was regenerated from the live instance content.
* chore: convert dashboard reducer to typescript
* convert to rtk
* use immer when possible
* fixup
* Apply suggestion from @kamilmielnik
I checked it locally and it is indeed unnecessary guard
Co-authored-by: Kamil Mielnik <kamil@kamilmielnik.com>
---------
Co-authored-by: Kamil Mielnik <kamil@kamilmielnik.com>
Part of the mutation testing project for metabase.lib.field.
Closes QUE2-338
Functions
Stats
Surviving mutations before: 16
Tests added: 1
Mutations killed by this PR: 16
Mutations remaining: 0
Mutations killed
Replace fingerprint with nil
Replace :type with :type__
Replace :type/DateTime with :DateTime__
Replace let with comment
Replace earliest with nil
Replace latest with nil
Replace when-not with when
Replace days with nil
Replace > with >=
Replace > with nil
Replace days with nil
Replace 1 with 0
Replace :minute with :minute__
Replace 31 with 0
Replace 365 with 0
Replace :week with :week__
Mutations not killed (with rationale)
Replace :day with :day__ — available-temporal-buckets-for-type has its own internal default of :day for datetime types, so the :day return value from fingerprint-based-default-unit is redundant with the fallback
* [Mutation Testing] Add tests for metabase.lib.field/fieldable-columns
* test: add mutation tests for column-metadata-effective-type and nest-display-name
Adds tests that exercise:
- type-of fallback to :base-type when :effective-type is absent
- type-of preference for :effective-type over :base-type
- temporal extraction returning :type/Integer
- temporal truncation preserving effective type
- nested field with nil parent display-name graceful fallback
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Fixes QUE2-350.
There are two changes here:
- Exporting some legit public functions from `lib.core`
- "Befriending" the `enterprise/sandbox` and `lib-be` modules, which are
deeply coupled to lib and MBQL in any case.
That allows us to remove several exported namespaces:
- `lib.binning`
- `lib.convert`
- `lib.field`
- `lib.metadata.cached-provider`
- `lib.metadata.column`
- `lib.metadata.invocation-tracker`
- `lib.metadata.result-metadata`
- `lib.normalize`
- `lib.query`
- `lib.temporal-bucket`
- `lib.walk`
The functions newly exported from `lib.core` are:
- `lib.field/infer-has-field-values`, called from some BE API handlers
and AppDB readers, producing a "cooked" value the FE wants rather than
exposing details of how `:model/FieldValues` is stored. Single-use,
but a legitimate export.
- `lib.field/json-field?`, checks if the passed in field is a JSON field,
that's a valid export used in a few places.
- `lib.query/native?`, checks if the first stage of this query is native,
that's another legit export used in a few places.
`lib.normalize/normalize` and `lib.query/query` were already exported, I
updated some usages of those functions to `lib/foo` instead.
* Convert `admin/permissions/utils/urls` to TypeScript
Part of the Accelerate TypeScript Migration project.
Closes DEV-1510
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* improve type guard
---------
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Co-authored-by: Uladzimir Havenchyk <uladzimir@metabase.com>
Move some logging-only computations into the `(log/* ...)` call so it won't be executed
unless the log level is sufficiently verbose for the log to actually be output.
* GDGT-1755: Migrate Text to TS
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
* GDGT-1755: Migrate Label to TS
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
* GDGT-1755: Migrate Subhead to TS
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
* GDGT-1755: Migrate IconWrapper to TS
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
* GDGT-1755: Migrate LabelIcon to TS
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
* GDGT-1755: Migrate use-scroll-on-mount to TS
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
* GDGT-1755: Missed TS fixes
* GDGT-1755: Migrate QueryButton to TS
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
* GDGT-1755: Migrate IconBorder to TS
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
* GDGT-1755: Remove Text.tsx wrapper, import from metabase/ui directly
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
* GDGT-1755: Adjust line height in places using new Text component
* GDGT-1755: Fix styling inconsistencies
* Update frontend Claude guidelines for typecheck/linting to use bun
* GDGT-1755: Address PR comments
---------
Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
These tests were temporarily disabled on 2025-09-26 due to flakes.
Re-enabling them now to restore full test coverage.
Lets monitor and see what happens
* wip: transform inspector BE
* WIP: inspect tables
* wip
* format join info as code
* wip: join info
* style
* add join stats type and display match rate
* chore: row
* chore: remove limit
* wip
* handle differerent cases for match rate and row count
* add joins table display and icons
* fix copy
* fix missing join table names
* cleanup
* cleanup
* extend match
* fix missing source table type
* add join tooltip
* chore: with join info
* wip
* wip
* add field statistics dummy component
* small ui adjustments
* small ui adjustments
* add filled rows
* order input columns like joins
* update visualization card styling
* wip: update join calc to be iterative
* add target and source stats
* add target and source stats, a filter by output fields and a tab for unmapped fields
* add target and source stats, a filter by output fields and a tab for unmapped fields
* chore: sort by order
* fix: inverse map
* add cross product warning
* add null join warning
* small QoL UI improvements
* wip: render output_row_count
* wip: native joins
* fix a bug with conditional rendering
* wip: try to pull out useful fields
* WIP: claude'd UI selection of visited_fields
* use default automatic display selection logic
* add transform mock page with mock summary section
* add mock api console log
* fix fetch wrong query metadata
* add mock lenses
* add mock lenses api
* REVERTME: ignore for deploy
* feat: schemas
* require
* feat: degeneracy check
* WIP cljc api
* wip context
* wip lenses
* update comments
* chore: visited-fields
* wip
* wip: base
* wip: triggered lens
* wip: lib
* distinguish drill lenses
* fix: return visited-fields
* wip: v2 api
* WIP: vibe'd FE
* tabs/remove mock
* account for params
* params
* ui improvements
* fix
* show fingerprinting
* fingerprint layout
* use BE fallback
* handle more complex joins
* fallback to interesting fields
* fix trigger via cljc
* remove debug
* different text
* compute-card-results in cljs
* condition trigger in cljc
* semi working
* fix card_id correlation
* remove dead code
* rename BE to just inspector
* switch to new
* ensure transform is deleted
* chore: cleanup schema
* bring back the field fingerprint info
* move cljs out of lib
* fix: schema
* minor cleanup
* appease kondo
* kondo
* remove dead code
* feat: centralize native analysis in query_analisis
* chore: remove #p
* clean up
* Gdgt 1663 move fingerprints out of the lens to data inspect (#68996)
* [transforms inspector] depend on trunk macaw (#69022)
depend on trunk macaw
* fix lint and type issues (#69017)
* remove unused lib function
* fix infinite rendering loop
* fix type lib.interestingFields
* fix
* fix: ui bug
* chore: disable visited fields for now
* chore: better unmatched-rows
* chore: exclude not nulls
* minor tweaks
* minor tweaks
* rename
* chore: tighter contract
* minor optimisation
* Gdgt 1669 make cards clickable (#69116)
* Add card link
* link to hash of question
* add clickable title
* remove tooltip
* Use MBQL info to match columns whe npossible (#69120)
use mbql info for column matching
* Gdgt 1675 disable unmatched rows view details button in inspector (#69152)
* remove row index column
* adapt viz UI, hide row index column
* Unmatched rows lens: return cards for both LHS or RHS null scenarios (#69123)
* feat: handle both cases of no join match
* feat: simple degeneracy
* export function
* remove dead fields
* only propose drill lenses that are applicable (#69169)
* Transforms inspector consistent casing for FE (#69174)
* chore: consistency
* chore: some docstrings updats
* feat: snake_case for the FE
* feat: pass visitedFields to interestingFields, remove ad-hoc logic in FE (#69175)
feat: pass visitedFields to interestingFields, remove ad-hoc logic in the FE
* feat: namespace card-ids with params (#69180)
* add estimated lens complexity (hardcoded for now) (#69181)
* add estimated lens complexity (hardcoded for now)
* refactor: use with-metadata
* add metadata to triggers, not just cards (#69182)
* appease kondo
* chore: don't depend on git sha macaw
* reinstate metabase-enterprise.transforms.core from merge conflict
* Gdgt 1722 new UI for data summary new (#69220)
* Transforms inspector BE tests (#69225)
* feat: inspector tests
* fix: impls
* fix: nil safe
* feat: integration tests
* make it work for both FE and BE
* fix card-results for non oute rjoin
* e2e test
* fixes
* fix: quoting
* fix for mysql/bigquery/redshift
* gdgt-1721 rework lenses navigation (#69205)
* Transforms inspector module (#69278)
* split transforms inspector into its own module, introduce transforms-rest
* handle kondo modules warnings
* fix: shadow_cljs
* kondo: fixes
* Transforms inspector be review (#69339)
* chore: align
* remove useless comments
* feat: add schema to context
* feat: simplify column-matching
* fix: visited-fields
* fix test
* remove unused
* tighter schema
* extrac-sources test
* fix: missing metadata-provider
* Gdgt 1744 combine separate averages columns into one for data summary (#69355)
* gdgt-1723 New UI for Joins Analysis (#69325)
* Gdgt 1725 allow running transform from empty screen (#69394)
* chore: move crud functions on transforms, avoid -rest modules dependencies
* fix tests
* appease kondo
* fix modules
* fix: small UI bug
* Transforms orphaned columns (#69497)
* add orphaned columns to unmatched-rows analysis
* chore: update tests
* Transforms inspector analytics (#69484)
* feat: flow discover/lens through transform events
* track transform_id in query execution
* feat: add transform-inspector ids/params to query info
* feat: add inspector specific /query endpoint
* feat: hook FE to new api
* feat: add lens_id/params to view
* feat: add them to ua
* fix: pass params too
* fix: use text.type
* fix test
* chore: remove unused requires
* chore: events
* fix FE part
---------
Co-authored-by: Egor Grushin <egor@metabase.com>
* fix: use lib functions instead of manual mbql construction (#69519)
* fix: use lib functions instead of manual mbql construction
* chore: refactor utils
* cleanup
* more manipulation with lib
* bugfix
* strip fields/filters early
* do it in util function
* fix: bad merge resolution
* kondo
* another bad merge resolution
* chore: indent
* Gdgt 1754 handle slow loading lens indicator (#69334)
* lens loading indicator
# Conflicts:
# frontend/src/metabase/transforms/pages/TransformInspectPage/TransformInspectPage.tsx
# frontend/src/metabase/transforms/pages/TransformInspectPage/components/LensContent/LensContent.tsx
* add onCardLoaded handler to context
* Gdgt 1724 new UI column (#69282)
* Gdgt 1783 fix wrong args passed to lens query (#69546)
* fix wrong args being passed to lens queries
* improve naming of query parameters
* remove whitespaces from merge resolution
* add transforms api test
* feat: i18n the inspector strings (#69565)
* fix dark mode color issues (#69570)
fix bg fixed to white
* Gdgt 1737 use card metadata instead of parsing card ids (#69583)
* use card metadata instead of parsing ids
* improve way of getting the table name from cards
* use macaw still, TODO
* port some query-analysis to sql-tools (#69732)
* chore: port some query-analysis to sql-tools
* remove sql-tools from transforms
* chore: vendor macaw functions, appease kondo
* refactor
* chore: refactor
* side agnostic
* revert manual delete, closes https://linear.app/metabase/issue/GDGT-1730/pre-merge-blocker-revert-t2delete-on-transform-once-uxw-3021-is-fixed
* fix: always sync audit db on change
* dead code
* fix: use honeysql instead of building sql manually (#69774)
* GDGT-1791 Maximum depth error in console (#69754)
GDGT-1791 fix and refactor useCardLoadingTracker
* Gdgt 1651 add frontend e2e tests for transform inspector UI (#69677)
* GDGT-1823 make guarantee that LensContent is always stores one lens data (#69800)
* fix: 404 on lens not found (#69838)
* fix: 404 on lens not found
* chore: don't pmap
* fix: remove nil values from statistics
* fix
* Transforms inspector self review be (#69843)
* style
* chore: reuse resolve-join-sides
* add from-table-id for mbql queries too
* split context in mbql/native, don't union
* whitespace
* fix: preprocess native queries
* fix: update tests
* fix
* revert unrelated commit that made in the branch erroneously
* Gdgt 1812 better handle missing sources in lens inspector (#69848)
* Gdgt 1817 add snowplow product events fe (#69801)
* Add analytics events
# Conflicts:
# frontend/src/metabase/transforms/analytics.ts
* add analytics tests
* simplify and improve lens loaded tracking event
* Move the idempotence check to useLensLoadedTracking
* GDGT-1831 replace unstyled button with a box to get rid of button in button error
* chore: transform lens_params
* GDGT-1786: Parametric lens navigation improvement (#69821)
* GDGT-1757: Refactor Column Comparison utils grouping and scoring logic (#69853)
* Improve types with entity ids (#69891)
* add concrete type declarations and usage for InspectorLensId, InspectorCardId, InspectorSectionId
* add better type to lens params
* cleanup of utils
* improve type for displaySettings
* use entity aliases in more places
* use existing type aliases for source and target
* add comment about InspectorTriggerCondition type
* unify prefix for inspector types
* remove spurious catch
* deduplicate types between metabase-api and lib
* Remove unused query
* fix provided tags for transform inspector
* add alerts and drill buttons test for default lens sections (#69899)
* restrict join analysis to single condition joins
* fix: bad merge reoslution
* fix types in lens test
* use isa function to detect type
* use concrete table id type alias
* use Urls.openInNewTab util
* remove empty mocks file
* use concrete table id type
* move lib for transform inspector to src/transforms/lib
* remove superflous log in e2e tests
* add proper SchemaName type
* remove superflous export
* revert move of isFetchBaseQueryError
* remove obsolete test
* use correct SchemaName type alias
* use correct VisualizationDisplay type
* remove leftover unused type
* remove superflous export
* Use CardId type alias
* discern lens also by based on params when tracking loading
* GDGT-1839 addressing the PR comments
* GDGT-1839 addressing the PR comments
* GDGT-1839 addressing the PR comments
* GDGT-1839 move getLensKey and LensHandle from api/tags directory
---------
Co-authored-by: Egor Grushin <egor@metabase.com>
Co-authored-by: Jakub Chodorowicz <jakub.chodorowicz@metabase.com>
Co-authored-by: Jakub Chodorowicz <jakub@chodorowicz.com>
* Convert `app-embed` to TypeScript
Part of the Accelerate TypeScript Migration project.
Closes DEV-1516
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* Show Python upsell on self hosted
* Add e2e tests
* Fix unit test
* Simplify TransformsUpsellPage a bit more, fix tests
* Simplify logic
* Fix e2e and unit test
* Update self-hosted python upsell as per new design / Code clean up
* Fix unit test and also add a new one for useTransformsBilling
* Remove e2e tests to be moved to a new PR
* Fix cta link and bullet points icon alignment
Our frontend code that prevents sending unchanged collections missed
tenant collections being added to the payload. This fixes both the
frontend to make sure we don't send when there are no changed
collections and it updates the backend actually check if collections are
being changed rather than just checking if the collections map exists.
* fix: transforms collections set setting on import
Also turns off the transforms sync setting if there are not transform
related models in the import
* fix: transform/snippet settings on collection sync
Updates our check for 'did transforms or snippets change on remote' to
examine collections in the respective namespaces so an import with only
transform collections (or snippet collection) creates the correct
settings on the importing instance.
* fixup! better handling for custom vs builtin tags
* fixup! better conflict checking of namespaced cols
* fixup! explicit connectionb for token cache
* fixup! PR feedback
* Adding dataPicker to embedding-data-picker redux store
* Refactor: move `dataPicker` redux
Move from interactive embedding store to embedding data picker store
* Add `dataPicker` type for the SDK
* Fix storybook id value
* Accept `dataPicker` in modular embedding SDK
* Make the doc more suitable for markdown table in the docs site
* Add test for InteractiveQuestion
* Fix drillThroughQuestionProps type
* Add tests for drillThroughQuestionProps.dataPicker
* Fix SDK components can't specify different `entityTypes` or `dataPicker`
* Fix doc error
* Fix comment grammar
* Fix wrong file name
* Clarify redux slice usage with a comment
* Make context more specific to avoid misuse
* Add `question` back to SDK's `entityTypes` on the API doc
* feat(dashboard): move subscriptions to action menu
* fix unit tests
* fix e2e tests
* fix console warning about uncontrolled component
* fix more tests
* fix remaining test
* bring test back
* Validate transform queries can be preprocessed
* remove prn
* update backend error messages
* Add support for custom toggleTemplateTagEditor
* Allow passing setTemplateTagConfig to TagEditorParam
* Add support for storing parameter values when question changes
* Add template-tags tab
* Add template tag sidebar
* Allow saving optional template tags or tags with a default value
* Show error message on save
* Add new spec for transform template tags
* Avoid sharing state when navigating
* Add combination test for template tag types in transforms
* Add definitionTab helper
* Add test for multiple template tags
* Fix error message
* Fix tooltips in test
* Remove tooltip check in tests
* Use getErrorMessage to get error message in toast
* Only return message if it is not empty
* Wrap error message in form
* Use H.createTestNativeQuery helper
* Move variableTemplateTags and isVariableTemplateTag to Lib
* Use more conventional key to reset uistate
* update backend error messages again
* Revert "Wrap error message in form"
This is now handled in the api
This reverts commit 9efecaf261.
* Fix test
---------
Co-authored-by: William Zimrin <william@metabase.com>
When an artifact doesn't exist (expected for new commits), the
fetch-artifact action was using throw new Error() which causes
GitHub Actions to report "Unhandled error" - confusing and noisy.
Using core.setFailed() cleanly marks the step as failed without
the scary error message.
Fixes: DEV-1581
Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
Prints the mage command and args before execution so we can quickly
see what labels/flags were passed when debugging CI driver issues.
Closes: DEV-1577
Fixes QUE2-353.
`mi/normalize-result-metadata-column` called some internals to modernize
a possibly-lib, possibly-legacy column from `:result_metadata` on a
card. Pull that logic into Lib as a new exported function.
* Remove dead code for actions
* Disable HTTP action type at API level
* clarify thrown exceptions
* Remove HTTP action tests
HTTP actions are now blocked at the API level; these tests cover
functionality that's no longer accessible.
* Remove remaining HTTP action references from api_test
The previous commit missed several HTTP action usages:
- all-actions-default helper function
- list-actions-test action-1 binding
- action-parameters-test (now uses query action)
We want to support transforms with template tags, as long as those tags aren't required or have default values. However, if we do that, the backend should actually verify that we do in fact have values for all relevant template tags. This pr adds basic validation logic to transform creation and update routes -- essentially, we just run qp.preprocess/preprocess and see if we get an error.
* Fix downloading currency values in narrowSymbol format (#69779)
* Add tests for narrowSymbol currency style in downloads
---------
Co-authored-by: Will Bryant <will.bryant@gmail.com>
Problem: When a database is configured for both SQL access and Transforms, any feature with SQL access (like raw SQL questions) inherits the elevated permissions (CREATE, DROP, INSERT, etc.) that Transforms need.
Solution: Admins can _optionally_ configure write credentials alongside existing read credentials. One logical database, two connection types. Specific code paths opt in to using write credentials; by default everything uses a read-only connection.
How: This PR adds a new `write_data_details` column on `Database`. If `write_data_details` isn't configured, everything will still use `details` as before. When it is configured, it acts as a set of overrides on top of `details`, but only in code paths that require write permissions. There should be no behavioral change for existing deployments.
* Explain how DB routing and query caching work together
* Update docs/configuring-metabase/caching.md
Co-authored-by: Jeff Bruemmer <jeff.bruemmer@gmail.com>
---------
Co-authored-by: Jeff Bruemmer <jeff.bruemmer@gmail.com>
It makes sense to use strings in http tests because that is what the
http-client will accept as json.
If a string target :type is supplied, we convert it to a keyword to
ensure we can dispatch to the intended method.
Fixes QUE2-354.
`lib/with-card-clean-hook` can be used to bind a callback function to
any material changes made by the `lib.convert/clean` process, which
generally indicates errors.
* fix: cancelling remote sync tasks dismisses modal
Previously we didn't close this modal until the task reached a
'cancelled' state which could take several seconds.
* fixup! PR feedback
Closes QUE2-285
The original idea was making the :strategy key required and leaving the setting of the value to the normalization, but
I decided not to go down that road. Here's why: there are multiple places on the path of constructing a join where we
can specify the join strategy but don't have to. These steps are not going through normalization and introducing
normalization would be inefficient and feel strange. Also, it would need to be done at several points, so it doesn't
seem a gain in the end. Because of this, all this PR does is naming the default strategy, so that changing it should
be possible at a single point. (Doing this would be a bad idea, so this PR is meant to be a documentation
improvement.)
* Convert Heading to TypeScript
Closes DEV-1409
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
* Delete heading and replace with title
---------
Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
* Convert `lib/schema_metadata` to TypeScript
Part of the Accelerate TypeScript Migration project.
Closes DEV-1506
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* Map Snowflake big integers to :type/BigInteger (fix#63600)
* Test fixes 🔧
* Test fixes 🔧
* Test fix 🔧
* More test fixes 🔧
* Another round of test fixes 🔧
* (Hopefully) final test fix 🔧
* Code cleanup
* Revert changes that broke Clickhouse test
* Test fix 🔧
* Convert `common/hooks/use-scroll-on-mount` to TypeScript
Part of the Accelerate TypeScript Migration project.
Closes DEV-1518
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* Convert `lib/encoding` to TypeScript
Part of the Accelerate TypeScript Migration project.
Closes DEV-1490
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* Add events for transform run tags and dependency diagnostics row selection
* Fix e2e test
* Adjust event as per review / Fix e2e tests
* Add tracking events for when Dependency Graph is opened for an entity
* Add remaining e2e tests
* Update simple_event/jsonschema/1-0-0
Update E2E test timings with averaged data
Collected timing data from 7 days of test runs.
All specs with timing data were included (simple averages).
🤖 Generated automatically by update-e2e-timings workflow
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: GitHub Actions <actions@github.com>
Fixes QUE-3044.
The QP will ignore refs in `:fields` clauses which cannot be resolved, since they're output only and not critical to the meaning of the query. This change makes the deps analysis treat these "soft" refs the same way - they are not considered to break the queries.
I added some commented tests that are now captured as part of QUE-3081 rather than part of this PR.
Fixes QUE2-349.
This removes several namespaces from the export list for `lib` which were only used by `legacy-mbql`.
- `metabase.lib.filter.desugar.jvm`
- `metabase.lib.hierarchy`
- `metabase.lib.schema.binning`
- `metabase.lib.schema.constraints`
- `metabase.lib.schema.middleware-options`
- `metabase.lib.schema.settings`
- `metabase.lib.schema.window`
* Ensure race free waiting for overlapping transforms
Previously transforms could contend for the is_active slot.
A common outcome (observed with two overlapping schedules) was that one transform would crash (due to the duplicate key violation).
This commit ensures a collision will fallback to waiting to try again rather than crashing (and alerting the user).
Introduces a unified `sql-tools` API for SQL parsing with swappable backends:
- Macaw (Java, existing) — moved from `metabase.driver.sql`, behavior preserved
- SQLGlot (Python, new) — broader dialect support via GraalVM Polyglot
Architecture: callers go through `sql-tools` which dispatches to either backend
via the `MB_SQL_TOOLS_PARSER_BACKEND` setting (default: sqlglot). The old
Macaw calls (parsed-query -> ast -> field-references) are replaced
with a `sql-tools` API (e.g. `sql-tools/referenced-fields`,
`sql-tools/validate-query`).
SQLGlot runs Python inside GraalVM Polyglot contexts managed by a reusable pool
to amortize initialization cost. The Python source and SQLGlot library are
bundled into the uberjar via a dedicated build step.
* record group membership removals in activity log
Previously the `PUT /api/membership/:group-id/clear` and
`DELETE /api/membership/:id` endpoints did not emit group membership
deletion events.
In the long run it would be better if `t2` were the interface for
PermissionsGroupMembership, so users didn't need to call these "blessed"
functions. But for now, let's go along with the existing pattern here,
have some blessed functions for doing things, and throw exceptions if
someone does a direct `t2/delete!` on a `:model/PermissionsGroupMembership`.
* use macro for more obvious workaround
The ci-test-config.json uses driver directory names (e.g., "bigquery-cloud-sdk")
but the mage code uses internal driver keywords (e.g., :bigquery). This mismatch
meant quarantined drivers like bigquery were not being recognized.
Now uses the existing driver-directory->drivers mapping to translate config names
to internal keywords, with fallback to direct keyword conversion for names that
match (like "databricks").
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
* Add schema for test-native-query-spec
* Implement test-native-query helper
* Add js wrapper for test-native-query
* Add ts wrapper for test-native-query
* Make common parts optional
* Fix types
* Fix temporal-unit test spec
* Handle snippet tags
* Fix temporal-unit tags
* Fix card tag spec
* Simplify schema
* Add unit tests for createNativeQuery
* Add tests for test-native-query
* Add missing options
* Fix types
* Actually resolve field ref
* Code style
* Add native-query api endpoint
* Add h.createTestNativeQuery helper
* Add test for required, widget-type and default options
* Move parsing into test-spec module as it is part of the implementation
* Add example e2e test use of H.createTestNativeQuery
* Use coercer instead of decoder to validate spec too
* Fix lint
* Move adjust to update-vals
Co-authored-by: Braden Shepherdson <braden@metabase.com>
* Match threading style
* Add test for optional template tagsfg
---------
Co-authored-by: Braden Shepherdson <braden@metabase.com>
* show both transforms tiers even when on trial
* Show upgrade modal in place when clicking Upgrade to Pro
* Improve comments
* Show upsell transforms trial variant when available for instance not on trial
* Fix ts
* Say 'Start trial' when trial is available
* Use correct type for order-bys
* Move parser for test-query-spec into implementation
* Be more strict in test
* Convert another e2e test to prove the parsing works
* Make args optional in test-query-spec
* Use threading instead of partial
Co-authored-by: Braden Shepherdson <braden@metabase.com>
---------
Co-authored-by: Braden Shepherdson <braden@metabase.com>
* move app switcher into Dependency Graph
* adjusting unit test
* remove z-index nonsense
* moving the app switcher up to DependencyGraphPage, small change to SelectItem
* switch function for metabot fetching field values
* refactor tests
* fix lint
* fix tests - keep existing behavior of auto-creating field values if needed
* [Lib] remove deprecated/confusing `:source-alias` key from col metadata
* Metadata schema cleanup
* Test fixes 🔧
* New test fix 🔧
* Ban legacy keys in column metadata
* Fix most tests
* More test fixes 🔧
* Kondo
* Test fix 🔧
* Add TypeDoc-friendly interfaces for EAJS web component documentation
* Add TypeDoc config for EAJS web component docs generation
* Generate EAJS web component props tables via TypeDoc
* Replace manual parameter lists with auto-generated props tables
* Fixed file references in components.md and enforced types in embed.ts
* .
* .
* .
* added detail on a prop and uncommited *.md files
* added info about attributes needing to be stringified
* .
Jest v30 validates that snapshot files have the current documentation URL.
Updated the header from the old shortened URL to the new official URL.
Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
* Fix virtualized list items rendering
* Add a repro for GDGT-1776
* Fix GDGT-1776
* Remove isHidden prop from MiniPickerItem and instead just do not render items that should be hidden
* Clean up
* Fix var access
Use prepare-release-scripts action in release-log workflow
Simplify the workflow by using the shared action instead of
manually setting up bun and building release scripts.
This works because the workflow always checks out master,
so the action from master (which uses bun) is always used.
Fixes: DEV-1544
Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
* Make ci-report work without GNU timeout
* Destroy process on timeout
* Cherry-pick b9ff360230
---------
Co-authored-by: Bryan Maass <bryan.maass@gmail.com>
The previous fix ran `cd release && bun install --yarn` which changed the
working directory for subsequent commands. The sed command then tried to
access `release/yarn.lock` from within the `release/` directory, resulting
in "sed: can't read release/yarn.lock: No such file or directory".
Split into separate steps with explicit working-directory for clarity.
The sed commands now continue on error to let Snyk report parsing issues
directly rather than failing on the workaround.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Premium feature token checks are cached per-instance in memory with a 12h TTL. During rolling deploys after a plan upgrade, different instances can disagree about which features are available:
1. Request hits instance A (old cache) → reports old features
2. Request hits instance B (freshly started, no cache) → reports new features
3. Frontend says "upgrade successful!"
4. Request hits instance A again → old features are back
This flapping means the frontend can't reliably detect when an upgrade has landed.
## Solution
### DB-hash-aware cross-instance cache
Replace the per-instance in-memory TTL cache with a two-layer caching scheme coordinated through the DB:
- A **local in-memory cache** holds the full token status (features, validity, etc.)
- A **DB table** (`premium_features_token_cache`) stores only a SHA-256 hash of the token status, keyed by a hash of the token itself. The DB never stores actual features — just enough to coordinate freshness across instances.
When a token check comes in:
- If the local cache matches the DB hash and is < 12h old: serve it immediately
- If it matches the DB hash and is < 36h old: serve the stale data and refresh asynchronously in the background
- If the DB hash doesn't match (another instance refreshed), or the cache is > 36h old, or this is a cold start: synchronous MetaStore fetch
This ensures all instances converge quickly after a plan change while keeping MetaStore traffic low. Cold-starting instances always hit the MetaStore — there's no way to bootstrap features from the DB alone, since it only stores hashes.
### Checker stack (inner to outer)
1. **`store-and-airgap-token-checker`** — hits MetaStore or validates airgap token
2. **`circuit-breaker-token-checker`** — circuit breaking + timeout
3. **`db-hash-aware-token-checker`** — the core caching layer (described above)
4. **`local-cached-token-checker`** — 5s TTL memoization to avoid hitting the DB on every request
5. **`error-catching-token-checker`** — wraps exceptions into `{:valid false}` responses
The 5s local cache means we hit the DB at most once every 5s, even during the grace period.
### Force-refresh endpoint
`POST /api/premium-features/token/refresh` clears all caches and synchronously re-checks against the MetaStore. The frontend uses this when polling for new features after a purchase (`useTokenRefreshUntil` in `UpgradeModalLoading`, `MetabotSettingUpModal`, `TransformsSettingUpModal`). A cookie-based mechanism propagates the invalidation to other instances so they clear their local caches on the next request.
Root Cause:
Bun's `bun install --yarn` command has a bug where it normalizes version
ranges inconsistently in the generated yarn.lock:
- Alias entries normalize ranges: "safer-buffer@>= 2.1.2 < 3"
(converts "< 3.0.0" to "< 3")
- Dependency specs keep original: safer-buffer ">= 2.1.2 < 3.0.0"
(preserves the ".0.0")
Snyk performs exact string matching when resolving dependencies to their
lock file entries. When it sees `safer-buffer@>= 2.1.2 < 3.0.0` as a
dependency, it looks for that exact string in the alias list, but only
finds `safer-buffer@>= 2.1.2 < 3`, causing the error:
"Dependency safer-buffer@>= 2.1.2 < 3.0.0 was not found in yarn.lock"
Solution:
Add a post-processing sed command that normalizes version ranges in the
dependency specifications to match the aliases. This transforms:
- "< 3.0.0" → "< 3"
- "< 3.0" → "< 3"
This ensures Snyk can match dependencies to their resolutions.
Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
Sorts parameters by their uuid to ensure that we maintain an order for
parameters across exports no matter the order they are serialized to the
database in.
Luckily a bug saved us from *actually* creating revisions here (the
`before-update` hook doesn't get an actual instance in this case). But
... we shouldn't be attempting it.
Offer to install essential build tools during setup, which are required
for compiling native extensions in development dependencies.
macOS:
- Detects Xcode Command Line Tools via xcode-select -p
- Triggers native installer dialog and polls for completion
- 10-minute timeout with manual installation fallback
Linux (multi-distro support):
- Debian/Ubuntu/Mint/Pop: build-essential
- Fedora/RHEL/CentOS/Rocky/Alma: Development Tools group
- Arch/Manjaro/EndeavourOS: base-devel
- openSUSE/SLES: devel_basis pattern
- Unknown distros: manual installation guidance
The check runs as part of step 1 (prerequisites) and is idempotent -
existing installations are detected and skipped with a success message.
* Add progress indicator during Xcode CLT installation wait
Show dots every 10 seconds while polling for installation completion,
so users know the script is still running during the wait.
Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
This completes the removal of `@peculiar/webcrypto` from the codebase,
following the Jest setup cleanup in #69466.
Changes:
- Update `polyfills.cljc` to use Node's native `crypto.webcrypto`
- Remove `@peculiar/webcrypto` from package.json
The previous comment about needing Node v23+ was outdated:
- `crypto.webcrypto` available since Node 15
- `globalThis.crypto` available since Node 19
- Node 22 (current LTS) fully supports it
Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
* Use native NodeJS cryppolyfill in fe unit tests
* Force jose library to use Node.js runtime in Jest tests
The jose library's browser runtime expects CryptoKey to be globally
available, which jsdom doesn't provide. This caused tests using JWT
signing (via jose) to fail with "CryptoKey is not defined" after
switching from @peculiar/webcrypto to Node's native webcrypto.
By mapping jose to its Node.js runtime via moduleNameMapper, we avoid
the CryptoKey dependency since the Node runtime uses require('crypto')
directly.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
* Remove old and unused props
* Support `targetCollection` in the CreateDashboardModal
* Add tests
* Handle special collection name e.g. `root`, `personal`
* Address review
* Update test names
* Report conflicts on transforms/snippets/lib
[UXW-2743: Warn before remote sync overwrites local library/transforms](https://linear.app/metabase/issue/UXW-2743/warn-before-remote-sync-overwrites-local-librarytransforms)
Co-authored-by: edpaget<ed@metabase.com>
* fixup! store conflicts in remote-sync-task table
* fixup! make conflict checking spec driven
* Restore initial implementation (implement conflict modal when pulling for the first time)
* Fix modal not opening when in data studio
* Don't set version in task when it fails with conflict / Adjust conflict options / Start e2e tests
* Add e2e tests
* Add some unit tests
* Fix remote sync settings update when creating new branch
* Isolate git config for initial import conflict test cases
* Report conflicts on transforms/snippets/lib
[UXW-2743: Warn before remote sync overwrites local library/transforms](https://linear.app/metabase/issue/UXW-2743/warn-before-remote-sync-overwrites-local-librarytransforms)
Co-authored-by: edpaget<ed@metabase.com>
* fixup! store conflicts in remote-sync-task table
* fixup! make conflict checking spec driven
* fixup! tests failures
* fixup! lints
* fixup! exclude tags from conflict
* Fix unintended changes from merge in RemoteSyncSettingsForm.tsx
* Address PR comments
---------
Co-authored-by: John Swanson <john.swanson@metabase.com>
Co-authored-by: Edward Paget <ed.paget@gmail.com>
* Do not allow publishing tables without query perms
Publishing a table changes query permissions. So you shouldn't be able
to publish a table that you don't have query permissions on, and then be
able to query it.
* fns for `can-publish?`, `can-publish-all-table?`
* switch to `library` ff
* Unify table visibility with can-query semantics
Table visibility checks had drifted across different APIs.
- `(can-query? table)` checked whether you had view access
and create-queries access OR the table was published. This is incorrect:
the table is queryable if you have view access AND either the table is
published OR you have create-queries.
- the collections API included tables that were not queryable.
- the search API allowed you to view tables if a) you had data
permissions on the table, AND b) the table was unpublished or published
in a collection you had access to. This is incorrect, since publishing a
table shouldn't hide a previously visible table from search results.
Search should return tables for which you have view-data permissions
PLUS the table is either published or you have create-queries
permissions.
Fundamentally this change:
- correctly defines what `(can-query? table)` means, and
- uses that new definition of `can-query` to determine table visibility
across the collection, search, and table APIs.
Note that I did not change some existing usages of
`(mi/visible-filter-clause :model/Table ...)`, which will continue to
exclude tables for which the user does not have explicit data
permissions set (but could otherwise query due to being published). That
should be a simple followup PR.
- align SQL table visibility with mi/can-query? semantics for search paths
- add enterprise `published-table-visible-clause` and wire optional
- `published-via-collection` support into `(mi/visible-filter-clause :model/Table ...)`
- update search table fallback permission check to use canonical `mi/can-query?`
- fix collection table children filtering to include `published-via-collection` queryability
- make `(mi/can-query? :model/Table)` return explicit booleans
- add/adjust EE tests for:
- published-table SQL clause behavior
- collection items visibility under view/query perms
- search visibility when user lacks query perms
- search visibility for unpublished tables when user has data perms
- stabilize appdb negation test by avoiding brittle exact count for "customer" in shared index data
* don't use `published-tables` directly
and fix kondo config
* correct feature for library
* kondo
* ensure we bind `*current-user-id*` in search
fallbacks rely on functions that use dynamic vars to check permissions.
Search is designed to use only the search context (with its
`current-user-id` and `current-user-perms`) and binds those dynamic vars
accordingly.
Previously we just needed to bind `current-user-permissions-set` but now
we should also bind `current-user-id` as well.
* fix tests
* correct feature for library
two changes:
- chris changed a file in a workspaces test file and no backend tests
ran at all:
```
=== Module Analysis ===
Changed modules: #{}
Driver module affected: false
Important file changed: false
Drivers with file changes: #{}
=== Driver Decisions ===
h2 RUN - H2/Postgres always run
athena SKIP - no relevant changes for cloud driver
bigquery SKIP - no relevant changes for cloud driver
clickhouse SKIP - driver module not affected
databricks SKIP - no relevant changes for cloud driver
druid SKIP - driver module not affected
druid-jdbc SKIP - driver module not affected
mongo SKIP - driver module not affected
mongo-ssl SKIP - driver module not affected
mongo-sharded-cluster SKIP - driver module not affected
mysql-mariadb SKIP - driver module not affected
oracle SKIP - driver module not affected
postgres RUN - H2/Postgres always run
presto-jdbc SKIP - driver module not affected
redshift SKIP - no relevant changes for cloud driver
snowflake SKIP - no relevant changes for cloud driver
sparksql SKIP - driver module not affected
sqlite SKIP - driver module not affected
sqlserver SKIP - driver module not affected
vertica SKIP - driver module not affected
```
So we want the workspace module to be marked dirty by this change. It's
currently filtered out by `remove-non-driver-test-namespaces`.
But a follow-on: workspaces run against different dbs and we therefore
want to ensure this not only triggers driver tests, but also cloud db
tests. So a few tests change to ensure this happens, consequent of
adding `enterprise/workspaces` to `modules-triggering-cloud-drivers`.
Better display for transform sync changes where we treat toggling the
`remote-sync-transforms` as though we are toggling the
`is_remote_synced` setting on Root transforms collection. This then
creates a 'collection' typed remote sync object entry for the Root
collection and uses that to drive dirty state display (or non-display)
on the frontend.
This object also drives how items are deleted from the sync so when this
'collection' is in a deleted state it removes all transforms related
content from the sync but preserves it locally.
* ease how we generate nano-ids
Fixes https://github.com/metabase/metabase/issues/68130
nano-ids are not needed to be secure, they just need to be
distinct. They hide no secrets but assign identity.
In https://github.com/metabase/metabase/pull/66413 we replaced our usage
of https://github.com/zelark/nano-id with encore. These are almost the
same except that encore uses `secure?` to be true by default. In
practice this is:
```clojure
(if secure?
(com.taoensso.encore.Ids/genNanoId (.get com.taoensso.encore.Ids/SRNG_STRONG) len)
(com.taoensso.encore.Ids/genNanoId len))
```
And that `com.taoensso.encore.Ids/SRNG_STRONG` is
```java
public static final ThreadLocal<SecureRandom> SRNG = ThreadLocal.withInitial(SecureRandom::new);
public static final ThreadLocal<SecureRandom> SRNG_STRONG = ThreadLocal.withInitial(() -> {
try { return SecureRandom.getInstanceStrong(); }
catch (java.security.NoSuchAlgorithmException e) {
throw new RuntimeException("Failed to initialize strong SecureRandom", e);
}});
```
the implmentation in the old library was
```clojure
(defonce *secure-random (delay (SecureRandom.)))
```
So passing `false` for the encore version of the nanoid restores the
previous behavior.
CALLOUT:
One thing to call out is that this did slightly change how we generate
nano-ids based on a seed. We used this behavior to backfill
nano-ids. That let us try to ensure that the same thing in multiple
systems would have the same nano-id, thus identifying them across
systems.
We are now generating different nanoids for the same inputs.
```clojure
❯ java -cp $JARS/1.45.4.jar clojure.main
Clojure 1.11.1
user=> (doto 'metabase.util require in-ns)
metabase.util
metabase.util=> (generate-nano-id "cafebabe")
"VlXdQMzunVywNw_SArkKX"
metabase.util=>
```
This is a consistent nanoid all the way up to 57:
```clojure
❯ java -cp $JARS/1.57.10.jar clojure.main
Clojure 1.12.2
user=> (doto 'metabase.util require in-ns)
metabase.util
metabase.util=> (generate-nano-id "cafebabe")
"VlXdQMzunVywNw_SArkKX"
metabase.util=>
```
And now is different in 58:
```clojure
❯ java -cp $JARS/1.58.3.jar clojure.main
Clojure 1.12.3
user=> (doto 'metabase.util require in-ns)
metabase.util
metabase.util=> (generate-nano-id "cafebabe")
"xNzFsobWPxaYpY0ucTMmz"
```
This is not a problem because we backfilled these nanoids in ~41 or so
and no longer allow anyone to save a model without one. But is worth a
callout that 58 changed this.
* fix drivers team attribution (#69323)
causes module config tests to fail
* Use bun in package.json
* Update workflows
* Update more workflows
* Kick ci
* Fix yaml
* Fix type errors
* Compat with snyk
* Fix jest resolutions
* Add another esm package
* Fix aliases and next js missing deps
* Update next js embed tests
* Try fix embed test config
* Fix
* Fix requires in e2e support
* Use bun not yarn in embed bin
* Revert config changes
* Use bun in more scripts and use prepare-release action
* Replace more yarn usages
* Remove overrides from bunfig move to package json
* Revert e2e support changes
* Add back comment
* [ci skip] comment typo
* Update bun lock
* Update to reference bun
* Update to reference bun
* Update license disclaimer
* Tidy tests
* Update to use bun in workflows
* Change aliases
* Update codemirrort selector
* Tidy license generation
* Remove .only
* Fix yarn references
* Checkout .github for release scripts
* Update lockfile
* Re-add package
* update release bunlock
* fix checkout check
---------
Co-authored-by: Ryan Laurie <iethree@gmail.com>
Fixes#67075.
`:starts-with`, `:contains` and `:ends-with`, which use SQL `LIKE`
clauses on SQL databases, were previously using the literal string from
the user directly, without escaping the `LIKE` metacharacters `%`, `_`
and backslash.
This change adds a new `sql.qp` multimethod to the driver API, with a
basic implementation that uses backslashes and works for Postgres,
MySQL/Maria and H2. SQL Server supports escaping via regex-style `[%]`
character classes.
Oracle and SQLite need an explicitly set `LIKE '...' ESCAPE '\'` clause.
I'm leaving that for a future PR since it requires refactoring how the
`LIKE` clauses are constructed more deeply.
Updates the driver changelog, though fortunately the default case should
work for most drivers. Not to be backported, because of the driver API
change.
The workflow was using `github.sha` (the merge commit SHA) when fetching
artifacts, but the uberjar workflow uploads artifacts using the PR head
commit SHA (`github.event.pull_request.head.sha`). This mismatch caused
the containerize workflow to fail with "artifact not found" errors.
Added `github.event.pull_request.head.sha` to the fallback chain in three
places: the artifact fetch commit, the GIT_COMMIT_SHA build arg, and the
Trivy scanner checkout ref.
Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
* wip
* Revert "wip"
This reverts commit a4dbdef16c.
* cleanup
* fix bug when clicking on breadcrumbs
* clean up old type
* Update frontend/src/embedding-sdk-bundle/components/private/SdkInternalNavigation/SdkInternalNavigationProvider.tsx
Co-authored-by: Phoomparin Mano <poom@metabase.com>
* jsdoc for virtual
---------
Co-authored-by: Phoomparin Mano <poom@metabase.com>
* Update Claude instructions
* Some DashboardSubscription type usage refactoring
* Migrate SendTestPulse to TS. Introduce DashboardSubscriptionData. Related refactoring
* Migrate ArchivePulseModal to TS
* Migrate UnsubscribePulseModal to TS
* Migrate ArchiveModal and test
* Migrate to TS: UnsubscribeModal. Related refactoring
* Migrate to TS: Notifications actions and routes
* Migrate to TS: ModalRoute refinement
* Fix eslint
* Adjust types of ModalRoutecomponents
* Fixes following bot code review
* Some more type refinement
* Address PR comments
* More fixes/refactoring
* Update route files. Convert to components
* Rename ArchiveNotificationModal to avoid confusion with collections. Fix export
* Fix more exports. Remove type casting. Add Response type guard
* Refactor routes back to getter functions
* Add early return for modals rendering. Add isObject in type-guard
* Refine Response type-guard
* Add auto-detection of changed Loki stories for stress testing
Enable the Loki stress test workflow to automatically trigger on PRs
that modify .stories.tsx files and detect which stories to test based
on git diff. Manual workflow_dispatch still works with optional filter
override.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
* enable page
* update permission functions, add transformsEnabled check
* hook up button
* temp
* add unit tests for transform layout page
* adjusting unit tests for permissions graph
* adjust e2e tests
* temp
* updating checks in e2e tests
* adding missing route mocks to unit test
* now the test should work
* do not export transforms-enabled
* adjust transforms e2e test
* update selectors to be a bit more clear
* additional tests
Update E2E test timings with averaged data
Collected timing data from 7 days of test runs.
All specs with timing data were included (simple averages).
🤖 Generated automatically by update-e2e-timings workflow
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: GitHub Actions <actions@github.com>
* Guest Embedding should use the primary database for queries
* make sure user is set to nil for guest embeds
* test
* test
* .
* .
* fixed existing clj test
* fixed existing clj test
* lint
* address comments on unit tests
* using with-database-routing-off
* kondo
* whitespace
* modules order
* addressed comments on tests
* SDK internal navigation: main implementation with context (#68900)
* squashed everything
* Trigger CI
* test ci
* fix accidentally enabling navigation by default
* fix tests
* clean up
* clean up and revert
* fix eslint rule after v9 migration
* idon'trememberanymore
* fix stale closure
* fix button position and more cleanup
* rename to virtual
* fix virtual
* more renaming
* more tests
* more fixes
* more fixes
* more reverts and cleaning up
* remove unused type
* internal navigation: change back button cover on hover (#69122)
* add support for ct to sdk back button (#69219)
add support for tc in back button
* support enable-entity-navigation in EAJS (#69185)
* tests, fixes
* do a real test for guest on a dashboard
* add "enableEntityNavigation" to types for browser
* Update frontend/src/embedding-sdk-bundle/components/private/SdkInternalNavigation/SdkInternalNavigationProvider.tsx
Co-authored-by: Sébastien <sebastien@metabase.com>
---------
Co-authored-by: Sébastien <sebastien@metabase.com>
* export back button, test that it shows up with title=false, align button (#69222)
* export back button and test it, also test that it shows up with
title={false}
* EMB-1275 map question props to question
* EMB-1278 add last remaining test
* EMB-1299: align back button with save button when drilling
* revert back button changes to keep backward compatibilty
* removed test
---------
Co-authored-by: Sébastien <sebastien@metabase.com>
* feat(eajs): support handleLink function
* fix unit tests
* rename, remove timeout which caused bugs if host app used alert() or other blocking things
* nest in pluginsConfig as is in the sdk
* put token feature guard
* Revert " put token feature guard"
This reverts commit 4f368ab41e.
* remove isWithinIframe
* Make eslint rule easier to read
* Prevent future misuse of the Metabase logo in the SDK
* Fix wrong Metabase icon on the SDK when using EE build with whitelabel feature
* fix eslint error message
* Allow internal users to have perms on tenant colls
Notably, this required allowing internal users to list tenants. I think
this is acceptable?
* CR response
* more cr response
* access
* sneak in some other edits
* Apply suggestions from code review
Co-authored-by: Jeff Bruemmer <jeff.bruemmer@gmail.com>
---------
Co-authored-by: Jeff Bruemmer <jeff.bruemmer@gmail.com>
* trigger cloud driver tests on certain modules
Motivating example:
https://github.com/metabase/metabase/pull/68256/changes added some
schemas to transform apis. Transforms run when drivers run. But it
didn't trigger cloud dbs. And of course, there was an issue with
redshift transform tests.
https://github.com/metabase/metabase/actions/runs/21718378450/job/62645866080
is the determine driver skips link:
```
=== Module Analysis ===
Changed modules: #{models transforms}
Driver module affected: true
Important file changed: false
Drivers with file changes: #{}
=== Driver Decisions ===
h2 RUN - H2/Postgres always run
athena SKIP - no relevant changes for cloud driver
bigquery SKIP - no relevant changes for cloud driver
clickhouse RUN - driver module affected by shared code changes
databricks SKIP - no relevant changes for cloud driver
druid RUN - driver module affected by shared code changes
druid-jdbc RUN - driver module affected by shared code changes
mongo RUN - driver module affected by shared code changes
mongo-ssl RUN - driver module affected by shared code changes
mongo-sharded-cluster RUN - driver module affected by shared code changes
mysql-mariadb RUN - driver module affected by shared code changes
oracle RUN - driver module affected by shared code changes
postgres RUN - H2/Postgres always run
presto-jdbc RUN - driver module affected by shared code changes
redshift SKIP - no relevant changes for cloud driver
snowflake SKIP - no relevant changes for cloud driver
sparksql RUN - driver module affected by shared code changes
sqlite RUN - driver module affected by shared code changes
sqlserver RUN - driver module affected by shared code changes
vertica RUN - driver module affected by shared code changes
```
Important part is `Changed modules: #{models transforms}` and `redshift
SKIP - no relevant changes for cloud driver`. This intends to change that.
* update comment
If you have an expression with an if statement inside an if statement, the outer if would get desugared to a case
statement, but the inner if would be left untouched. This makes desugaring also desugar the inner if.
Fixes QUE-3070.
This splits the vague, dual-purpose `supported-entities` into two:
- those entities which can be depended on, whose dependents get marked
stale when they're updated (eg. a table during sync)
- those entities which we can analyze, and should check for staleness
I added `:table` to the old `supported-entities` since they're a valid
*upstream* dependency that might needs its downstream dependents to be
marked as stale, and it failed the schema.
The trouble is that we don't run analysis on tables, but the same set of
"supported" types was used to check for what entities need reanalysis.
That led to a chain like this:
- Something changed, and the entity-check job is scheduled immediately.
- It runs, and checks each `supported-entities` for any entity with
a missing, `stale` or outdated analysis finding.
- That list now includes tables, and since tables are not analyzed none
of them have entries - so there's apparently lots of work to do!
- On trying to analyze tables, it fails because they're not supported,
and it silently does nothing.
- But the batch draining logic still thinks the whole batch was used, so
it reschedules.
- No real progress is getting made, since the tables cannot really be
analyzed, therefore they never get marked as done.
- Infinite loop spinning forever!
* Create a metabase_table.transform_id column instead of dynamically populating it from the dependencies
* Fixing tests
* Include transform in table dependencies
* Moved test after merge
* Moved test after merge
* transforms are now oss
* Fixed failing tests
* Review fixes
* Hide transforms from metabot if source is not readable
* test simplification
* fix test nesting
* dedupe transform-ids earlier
* Fix module interface
* Fix rebase - transforms were moved to OSS
seeing error in CI:
```
ERROR in metabase.upload.db-test/current-database-test (DbException.java:502)
Uncaught exception, not in assertion.
clojure.lang.ExceptionInfo: Table "METABASE_DATABASE" not found (this database is empty); SQL statement:
SELECT * FROM "METABASE_DATABASE" WHERE "UPLOADS_ENABLED" = TRUE [42104-214]
```
https://github.com/metabase/metabase/actions/runs/21711162702/job/62620032674
we just need to ensure that the db is initialized first
On Apple Silicon Macs, some webpack dependencies (webpack-notifier)
require Rosetta 2 to run x86_64 binaries. Without it, the webpack dev
server fails with "Error: spawn Unknown system error -86".
This adds a check during prerequisites that detects if Rosetta is
missing and offers to install it via softwareupdate --install-rosetta.
Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
Our migrations can race with writers, this leads to situations
where we might still see values for e.g. an enum according to some prior
schema.
We saw on stats a sync occurred concurrently with the migrations and caused a record with medallion data_layer values to appear _after_ the migration.
Should follow up with a longer term fix without leaving such cruft in the code.
* Remove link from broken dashboard notification
* Remove links from alerts setup email
* Add tests
* Remove links from unsubscribe email
* Remove links from card change or archived emails
* add tests for card change email notifications
* add tests to the broken dashboard email
* Format
* Remove unused comments
* Improve test util names
* Format again
* Fix name
* Fixing tests
* Fixing tests
* Fixing tests
* Fixing tests
* Fixing tests
* Fixing tests
* Fixing test
* Added source_readable to transform APIs
* Fixing tests
* Starts implementing read-only state for job detail page
* Stubs out potential solution for preventing non-admins from putting jobs into a state where they can no longer edit them
* Troubleshooting failing test
* Shows permissions error on transform page when user doesn't have source-read permissions. Disables run button when user lacks write access.
* Troubleshooting failing test
* Makes TagSection readOnly on job page if user lacks write access on any transforms that'd be run by the job
* Removes unnecessarily duplicated code from ScheduleSection
* Troubleshooting failing test
* Troubleshooting failing test
* Troubleshooting failing test
* Troubleshooting failing test
* Cleanup
* Fix unit tests
* Updates e2e test (group-15)
* Updates e2e tests (transforms-permissions.cy.spec.ts)
* Fixed transform-job permission check
* Only require transform write access for tag selector on job page
* Tweak tag tooltip text
* Prevents deleting/editing tag when it runs a job the user doesn't have transform write access on
* Adds a couple more transforms-permissions e2e tests
* Build transform permission off table can-query
* Added :can_run to transform-tag API
* Updates FE to use can_run on TransformTag
* Code cleanup
* Updates transforms list to disable transform rows the user doesn't have read access for
* Code cleanup
* Fixing test failures
* Fixing test failures
* Fixing test failures
* Fixing test failures
* Merged from master
* Test troubleshooting
* Resolved conflicts
* fixup: Reimplements row disabling in TransformListPage
* fixup: Adds readOnly prop to new IncrementalTransformSettingsWrapper component
* fixup: Reimplements readOnly state for PythonTransformEditor
* Adds read-only state for transform owner section
* Updates recently-moved canAccessDataStudio with canAccessTransforms check
* Updates permission-management unit tests
* Adds tooltip to transform column in permissions table
* Hides tooltip in Transforms column in permissions table if selected group is admin or analysts
* Fix unit test
* Updates some e2e tests that deal with non-admins using transforms to make sure they belong to the analyst group
* Adjusting backend permissions for new logic: all analysists can see transforms they have db access to, transform-permission is for writes
* Code improvements
* Code improvements
* Fixed test
* Fixing tests
* Troubleshooting tests
* Troubleshooting tests
* Fixing tests
* Update e2e tests
* Brought back can_access_transforms
* Fixing tests
* Better python permission checking
* Start, WIP
* Moved Library back to EE
# Conflicts:
# enterprise/frontend/src/metabase-enterprise/data-studio/library/LibrarySectionLayout/PublishTableModal.tsx
# frontend/src/metabase/data-studio/data-model/components/TableSection/TableAttributesEditBulk.tsx
* Continue moving Library back to EE
* Hotfix
* Fixing permissions
* Fixing permissions
* Fixing tests
* Fixing tests
* move out transforms api
* Fixing tests
* move transforms to oss
* actually it is true
* cleanup
* Fixing tests
* Fix post-merge ts error
* Close library functionality in OSS, fix some units
* Move oss table api endpoints for data structure, fix more unit tests
* Move oss table api endpoints for data structure, fix more unit tests
* Updates transforms list page so "New" is hidden and "Python library" is disabled if user has no transform permission for any database
* Fix typecheck issues
* Move non-python transform APIs to OSS
* Fix kondo
* Fix EE BE
* Code review fixes
* Code review fixes
* Merge remote-tracking branch 'origin/master' into transforms-permissions
# Conflicts:
# .clj-kondo/config/modules/config.edn
# enterprise/backend/test/metabase_enterprise/remote_sync/transforms_test.clj
# enterprise/frontend/src/metabase-enterprise/transforms/pages/TransformListPage/TransformListPage.tsx
* Code review fixes
* Fixed merge conflict
* Fixed merge conflict
* Fix lint and TS error from merge
* Fix merge issues
* Fix be issues
* Fix some unit tests
* Fix be tests issues
* Fix be tests issues
* Code review fixes
* Allow people with transform permission to see syncable schemas on a db
* code cleanup
* Fix be tests issues
* Fixing permissions
* Fix be tests issues
* Fix prettier
* wip: oss transforms
* fix module order
* fix tests for OSS transforms
* cleanup
* cleanup
* Fix some unit tests
* Fix library for OSS
* fix some more tests
* cljfmt
* add missing placeholder file
* fix more tests
* more tests
* more tests
* deftest-oss into `metabase.test`
* Fix some unit tests
* Fix some unit tests
* Fix transforms upsell for oss
* fix search filter
* fix transforms tables
* fix cycles
* feat: transform setting into collection list (#68758)
* feat: transform setting into collection list
Move the setting for syncing transforms into the list of collections
available to remote sync, so conceptually the transforms root collection
looks like a standard syncable collection.
* Clean up FE a little bit
* undo unneeded changes
---------
Co-authored-by: Anderson <anderson@metabase.com>
* Remove Security and Appearance from embedding settings related links on OSS (#68151)
* feat: enable transform setting on first import (#68807)
* feat: enable transform setting on first import
When a we import a remote that has transforms set the
`remote-sync-transforms` setting to true.
Update the remote sync object table clean up to be driven by the spec
configuration.
Add tests showing that transforms are consistently synced and erased
according to the following rules:
1. if the instance has `remote-sync-transforms` off and local transforms
and the remote does not have transforms content they are preserved after
an import
2. if the instance has `remote-sync-transforms` off and local transforms
and the remote has transforms content, the local transforms are
completely replaced by the remote.
3. if the instance has `remote-sync-transforms` on and the remote has
transforms content any local transforms are completely replaced by the
remote.
4. if the instance has `remote-sync-transforms` on has local transforms,
but the remote has no transforms, the local transforms are removed on import.
* fixup! relocate settings update
* fixup! don't let python libraries clash on import
* fixup! transactions for mysql
* fix: y axis not shown even if it would fit (#68861)
* Deps breaking broken (#68866)
* Add rename /broken to /breaking and add new /broken
---------
Co-authored-by: Alexander Polyankin <alexander.polyankin@metabase.com>
* Analysts can do crud on segments and measures (#68501)
* Data Analyst CRUD on measures/segments
Data analysts with unrestricted view-data permissions on a table can now
create, update, and delete segments and measures for that table
(previously superuser-only).
`can-read?` was already granted by the `manage-table-metadata` permission.
Add permission tests for `can-write?` and `can-create?` methods.
* Remove `brand-light` and `brand-lighter` (#68290)
* brand key adjustments
* loki tests
* handle the forgotten brand-dark
* Update Loki Snapshots
* removing brand-hover from dynamic-css-vars
* Adjustments to calendar picker colors
* small adjustments on the permissions pages
* sdk: remove background-hover tests from sdk as dynamic css derives are replaced with color-mix
* stupid spelling
* building shared resources
* Adjusting unit test, loki images
* rebasing adjustments
* Update Loki Snapshots
* small color fixes while I wait for loki
* re-adding highlighted class
---------
Co-authored-by: Metabase Automation <github-automation@metabase.com>
Co-authored-by: Phoomparin Mano <phoomparin@gmail.com>
* Fix empty Python Library not showing placeholder/comment (#68826)
* CI: query-processor module should trigger driver tests (#68877)
* Delete fuzzing workflow (#68880)
* fix(sdk): fix failing tests for white-labeled colors (#68790)
* Dependency diagnostics sidebar updates (#68825)
* Use getter function to move element (#68714)
* Fix cloud implicit join tests (#68762)
A previous pr broke some tests on master, and this fixes them.
* feat(sdk, embed-js): Add an alerts support to modular embeddings (#68777)
* Add an alerts button on the SDK (#67796)
* Prepare to add alerts to SDK
* Remove unrelated prop from storybook
* Add basic alerts button
* Only show alerts button with the proper flag
* Prevent download button from being shown without proper flag
* use CSS to hide things instead of JS
* Properly position all question buttons
* Use JS to hide empty divs as pure CSS is impossible
* Render alert modal ✌️
* Fix type
* Add todo
* Structure the components better
* Control alert modal state
* Add interactive question storybook
* Make interactive and static question button styles consistent
* Add alerts modal to interactive question
* Make alert button works with custom layouts
* Fix tests
* Fix unit tests
* Fix import order issue
* Remove outdated comment
* Add the same condition to display alert button from the core app
* Polish
* Modify question storybooks to show a question instead of a model
* Hide empty elements more reliably
* Add more conditions to display alert button
* Remove unused `classNames` prop
* Use the same style as the interactive question one
* Remove outdated diff
* Fix alerts button not working in custom layouts
* Remove another unused code
* Add a basic test
* Add unit tests
* fix visibility logic
* Fix E2E tests
* Make assertion on static question tests better
* Improve assertions
* Add interactive question unit tests
* Add e2e test for static question
* Add interactive question e2e tests
* Fix subcomponent
* clarify comment
* Hide other channels than email on the alert modal (#68184)
* Hide channel options on alerts modal
* Add unit tests
* Make the test name clearer
* Fix can't view created notifications the first time (#68197)
* Fix can't view created notifications
* Add tests
* Remove unnecessary return type
* Add a safety guard
* Hide Alert (and download) button on edit mode (#68189)
Hide some buttons on edit mode as it makes no sense there
* Alerts polishing (#68477)
* Misc. improvements
* Make it typesafe
* Add alerts to modular embedding (#68286)
* Add the option in modular embedding wizard
* Support `withAlerts` in modular embedding
* Close the modal after hiding the alert button
* Add E2E tests
* Fix type
* Fix `withAlerts` not working on modular embedding with `drills: false`
* It's 2026!
* 1 plugin to rule 'em all!
* Fix E2E after merging master
* Fix test name
* Add alerts analytics (#68531)
* Add alert setup analytics
* Track withAlerts on modular embedding
* Update snowplow schema
* Fix E2E tests
* Fix schema version
* Remove links from alerts test email (#68547)
* Remove links from alerts template
* Remove links from alert email computed part
* Add tests
* Make embedding util easier to use
* Make alerts and subscriptions payload lighter
* Remove unexpected pprint
* Remove links from alert email via tasks (#68610)
* Save notification card's disable_links when creating
* Add tests
* make bin/mage run-tests able to run a single test (#67450)
* Add content translations for aggregations (#68733)
* fix: typecheck
* oops, add missing file
* fix upload impl tests
* Move upsells to ee for transforms
* fix search tests
* fix mage tests
* fix audit perms
* fix `user->tenant-collection-and-descendant-ids`
* reorganize transform permissions
* fix cyclic dep
* add forgotten test file
* fix modules tests / kondo
* another module config fix
* Fix python transforms upsell in oss, fix unit tests
* fix more tests
* one more test
* one more
* Fix e2e tests
* one more
* and a few more
* oops, syntax error
* fix more tests
* kondoooooo
* dumb test: try increasing latch timeout
* hopefully actually fix h2 oss test
* Fix flaky e2e tests
* Fix lint
* Trigger rebuild
* Fix CI
* Fix CI
* Fix failing e2e test
* fix check for is-data-analyst
* Gate transforms based on `:hosting` flag
* move noop events
* Remove unnecessary `check-permissions-for-model`
* use transform feature gating in `can-read?`
* cr resp: moving tests around
* add `enabled-source-types-for-user` and use it
* remove last `deftest-oss` usage
also clean up kondo config:
- no longer define custom expansions for old macros that no longer exist
- remove `deftest-oss`
* change owner to UX west
* use `is-data-analyst?` and `check-data-analyst`
... in a few more places, instead of manual `(or api/*is-superuser?* api/*is-data-analyst?*)`
* Fixes for BE after merge
* Integrate Workspaces code
* Fixes for BE
* Fixes for BE
* Fixes
* remove unnecessary `when-ee-evailable`
also that typo is driving me up the wall and I'm going to need to fix
it :grimace:
* transforms doesn't use config
* missing import
* fix tests
* kondoooo
* last be test failure?
* fix feature gating for workspaces transforms API
why do we have a separate workspaces transforms API
* kondo
* two more tests
* tests
* couple more tests
* almost... there
* fix a flaky transform run test
this was failing locally because I already had enough transform runs to
fill up the page. simple helper to clean up the transform runs that
exist before running the test, then restore them.
* fix data-model.shared.cy.spec.ts
---------
Co-authored-by: Nathan Voxland <nathan@metabase.com>
Co-authored-by: Brad Anderson <brad@metabase.com>
Co-authored-by: Aleksandr Lesnenko <alxnddr@gmail.com>
Co-authored-by: John Swanson <john.swanson@metabase.com>
Co-authored-by: Edward Paget <edpaget@users.noreply.github.com>
Co-authored-by: Anderson <anderson@metabase.com>
Co-authored-by: kevinkelleher12 <kevin.kelleher@metabase.com>
Co-authored-by: Maksym Yakubych <undrfined@gmail.com>
Co-authored-by: metamben <103100869+metamben@users.noreply.github.com>
Co-authored-by: Alexander Polyankin <alexander.polyankin@metabase.com>
Co-authored-by: Nick Fitzpatrick <nick@metabase.com>
Co-authored-by: Metabase Automation <github-automation@metabase.com>
Co-authored-by: Phoomparin Mano <phoomparin@gmail.com>
Co-authored-by: Anderson <ands.alves.nunes@gmail.com>
Co-authored-by: bryan <bryan.maass@gmail.com>
Co-authored-by: Fraser <fraser@metabase.com>
Co-authored-by: Phoomparin Mano <poom@metabase.com>
Co-authored-by: Romeo Van Snick <romeo@romeovansnick.be>
Co-authored-by: William <william@metabase.com>
Co-authored-by: Mahatthana (Kelvin) Nomsawadi <me@bboykelvin.dev>
Co-authored-by: Oleksandr Solovyov <alexander@solovyov.net>
Co-authored-by: Timofey Kachalov <timofei@metabase.com>
I think this permission makes more sense and is probably what we
intended. Note that users without `manage-table-metadata` but with table
permission can already read this information from the
`/api/query_metadata` endpoint.
* module test runner
support for `:module` and `:modules` syntax for our test runner
When using multiple with `:modules` need to put them in a vector, and
for shell reasons need to quote it. So there's a convenience method
where you just pass an individual:
```shell
clj -X:dev:ee:ee-dev:test :modules '[tiles enterprise/agent-api]'
clj -X:dev:ee:ee-dev:test :module enterprise/agent-api
```
REPL
===
```clojure
test-runner=> (metabase.test-runner/find-and-run-tests-repl
{:modules '[tiles enterprise/agent-api]})
Running tests with options {:mode :repl, :namespace-pattern #"^(?:(?:metabase.*)|(?:hooks\..*))", :exclude-directories [".clj-kondo/src" "classes" "dev" "enterprise/backend/src" "local" "resources" "resources-ee" "src" "target" "test_config" "test_resources"], :test-warn-time 3000, :only ("test/metabase/tiles" "enterprise/backend/test/metabase_enterprise/agent_api")}
Running tests in ("test/metabase/tiles" "enterprise/backend/test/metabase_enterprise/agent_api")
Looking for test namespaces in directory test/metabase/tiles
Looking for test namespaces in directory enterprise/backend/test/metabase_enterprise/agent_api
Finding tests took 159.0 ms.
Running 32 tests
Running before-run hooks...
LONG TEST in metabase-enterprise.agent-api.api-test/get-metric-details-test
Test took 3.447 seconds seconds to run
Ran 32 tests in 6.506 seconds
129 assertions, 0 failures, 0 errors.
{:test 32, :pass 129, :fail 0, :error 0, :type :summary, :duration 6505.825875, :single-threaded 20, :parallel 12}
Ran 12 tests in parallel, 20 single-threaded.
Finding and running tests took 6.7 s.
All tests passed.
Running after-run hooks...
Running QP tests against these drivers: #{:h2}
{:test 32,
:pass 129,
:fail 0,
:error 0,
:type :summary,
:duration 6505.825875,
:single-threaded 20,
:parallel 12}
```
CLI
===
multiple:
```shell
❯ clj -X:dev:ee:ee-dev:test :modules '[tiles enterprise/agent-api]'
Warning: environ value /Users/dan/.sdkman/candidates/java/current for key :java-home has been overwritten with /Users/dan/.sdkman/candidates/java/21.0.2-tem
Reflection warning, clojurewerkz/quartzite/triggers.clj:85:6 - call to method forJob on org.quartz.TriggerBuilder can't be resolved (argument types: unknown).
Reflection warning, clojurewerkz/quartzite/scheduler.clj:254:3 - call to method checkExists on org.quartz.Scheduler can't be resolved (argument types: unknown).
Reflection warning, macaw/walk.clj:47:5 - call to method walk on com.metabase.macaw.AstWalker can't be resolved (argument types: unknown).
Running tests with options {:mode :cli/local, :namespace-pattern #"^(?:(?:metabase.*)|(?:hooks\..*))", :exclude-directories [".clj-kondo/src" "classes" "dev" "enterprise/backend/src" "local" "resources" "resources-ee" "src" "target" "test_config" "test_resources"], :test-warn-time 3000, :exclude-tags [:metabot-v3/e2e], :only ("test/metabase/tiles" "enterprise/backend/test/metabase_enterprise/agent_api")}
Running tests in ("test/metabase/tiles" "enterprise/backend/test/metabase_enterprise/agent_api")
Looking for test namespaces in directory test/metabase/tiles
Looking for test namespaces in directory enterprise/backend/test/metabase_enterprise/agent_api
Finding tests took 11.8 s.
Running 32 tests
Running before-run hooks...
0/32 0% [ ] ETA: --:--Placeholder until other changesets
LONG TEST in metabase-enterprise.agent-api.api-test/get-metric-details-test
Test took 10.536 seconds seconds to run
32/32 100% [==================================================] ETA: 00:00
Ran 32 tests in 18.556 seconds
129 assertions, 0 failures, 0 errors.
{:test 32, :pass 129, :fail 0, :error 0, :type :summary, :duration 18555.859333, :single-threaded 20, :parallel 12}
Ran 12 tests in parallel, 20 single-threaded.
Finding and running tests took 30.4 s.
All tests passed.
Running after-run hooks...
Running QP tests against these drivers: #{:h2}
```
individual:
```
❯ clj -X:dev:ee:ee-dev:test :module enterprise/agent-api
Warning: environ value /Users/dan/.sdkman/candidates/java/current for key :java-home has been overwritten with /Users/dan/.sdkman/candidates/java/21.0.2-tem
Reflection warning, clojurewerkz/quartzite/triggers.clj:85:6 - call to method forJob on org.quartz.TriggerBuilder can't be resolved (argument types: unknown).
Reflection warning, clojurewerkz/quartzite/scheduler.clj:254:3 - call to method checkExists on org.quartz.Scheduler can't be resolved (argument types: unknown).
Reflection warning, macaw/walk.clj:47:5 - call to method walk on com.metabase.macaw.AstWalker can't be resolved (argument types: unknown).
Running tests with options {:mode :cli/local, :namespace-pattern #"^(?:(?:metabase.*)|(?:hooks\..*))", :exclude-directories [".clj-kondo/src" "classes" "dev" "enterprise/backend/src" "local" "resources" "resources-ee" "src" "target" "test_config" "test_resources"], :test-warn-time 3000, :exclude-tags [:metabot-v3/e2e], :module enterprise/agent-api, :only ("enterprise/backend/test/metabase_enterprise/agent_api")}
Running tests in ("enterprise/backend/test/metabase_enterprise/agent_api")
Looking for test namespaces in directory enterprise/backend/test/metabase_enterprise/agent_api
Finding tests took 12.6 s.
Running 19 tests
Running before-run hooks...
0/19 0% [ ] ETA: --:--Placeholder until other changesets
LONG TEST in metabase-enterprise.agent-api.api-test/get-metric-details-test
Test took 11.252 seconds seconds to run
19/19 100% [==================================================] ETA: 00:00
Ran 19 tests in 19.122 seconds
105 assertions, 0 failures, 0 errors.
{:test 19, :pass 105, :fail 0, :error 0, :type :summary, :duration 19122.404791, :single-threaded 19}
Ran 0 tests in parallel, 19 single-threaded.
Finding and running tests took 31.8 s.
All tests passed.
Running after-run hooks...
Running QP tests against these drivers: #{:h2}
```
* very opinionated
* add onboarding accordion
* add storybook for onboarding accordion
* fix the hover color of accordion
* chore: remove unused scrollListItemIntoView
* cleanup accordion
* add onboarding stepper instead
* retain checked icon in selected steps
* remove active step
* add unit tests for accordion
* rename step label to step number
* remove stepper root
* remove explicit ref
* refactor: simplify styles
* simplify storybook
* auto advance stepper
* support locked steps
* auto-infer step number
* remove forwardRef
* do not provide defaults for completed and locked steps
* add horizontal padding to match 64px in design
* style fixes to match design
* address reviews on scroll refs
* fix inconsistent scroll state
Athena's JDBC driver returns type names with the same capitalization as
the user types into the admin interface, but the
`sql-jdbc.sync/database-type->base-type` implementation for Athena only
recognized lower-case names (which are the defaults, I gather).
I think most databases don't have this problem, since they ignore the
case you typed in e.g. `CREATE TABLE` statements and have the JDBC
driver always return the canonical spelling of the type names.
Fixes QUE-3000.
When either the table metadata or field metadata is edited, this runs
the analysis. Since only some metadata changes can impact the analysis,
this is over-sensitive and runs sometimes when it doesn't need to.
Update llms.txt and llms-full.txt (master)
Regenerated llms.txt to include all markdown files from the docs/ directory.
Also generated llms-full.txt files for the embedding section.
🤖 Generated automatically by llms-txt-generate workflow
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: GitHub Actions <actions@github.com>
Change medallion values to hidden/internal/published. Hidden inherits visbility_type = hidden behaviour, internal/final are label only for now.
---------
Co-authored-by: Dan Stone <wotbrew@gmail.com>
Jest's haste map uses filenames (without extensions) as identifiers for
manual mocks. Both ExplicitSize and NativeQueryEditor had `__mocks__/index.ts`
barrel files, causing a collision on the name "index".
Fix by removing the barrel files and using explicit require() calls in
register-visualizations.js to point directly to the uniquely-named mock
files (ExplicitSize.tsx and NativeQueryEditor.jsx).
Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
The fix was changing the mock from returning a function directly (which
would be a default export) to returning an object with a Link property
(named export), matching how the actual module exports the component.
* fix kerberos tests
in CI we're running:
```
[2/3] RUN ( curl -fsSL https://deb.nodesource.com/setup_22.x | bash ) && export DEBIAN_FRONTEND=noninteractive && apt-get update && apt-get -y install --no-install-recommends nodejs yarn rlwrap fontconfig
...
3.579 Hit:1 http://deb.debian.org/debian trixie InRelease
(#6 3.579 Hit:2 http://deb.debian.org/debian trixie-updates InRelease
(#6 3.588 Hit:3 http://deb.debian.org/debian-security trixie-security InRelease
(#6 3.651 Hit:4 https://deb.nodesource.com/node_22.x nodistro InRelease
(#6 3.661 Reading package lists...
(#6 4.185 Reading package lists...
(#6 4.691 Building dependency tree...
(#6 4.834 Reading state information...
Dockerfile:5
--------------------
4 |
5 | >>> RUN ( curl -fsSL https://deb.nodesource.com/setup_22.x | bash ) \
6 | >>> && export DEBIAN_FRONTEND=noninteractive \
7 | >>> && apt-get update && apt-get -y install --no-install-recommends nodejs yarn rlwrap fontconfig
8 |
--------------------
failed to solve: process "/bin/sh -c ( curl -fsSL https://deb.nodesource.com/setup_22.x | bash ) && export DEBIAN_FRONTEND=noninteractive && apt-get update && apt-get -y install --no-install-recommends nodejs yarn rlwrap fontconfig" did not complete successfully: exit code: 100
(#6 4.850 E: Unable to locate package yarn
(#6 ERROR: process "/bin/sh -c ( curl -fsSL https://deb.nodesource.com/setup_22.x | bash ) && export DEBIAN_FRONTEND=noninteractive && apt-get update && apt-get -y install --no-install-recommends nodejs yarn rlwrap fontconfig" did not complete successfully: exit code: 100
------
> [2/3] RUN ( curl -fsSL https://deb.nodesource.com/setup_22.x | bash ) && export DEBIAN_FRONTEND=noninteractive && apt-get update && apt-get -y install --no-install-recommends nodejs yarn rlwrap fontconfig:
3.517 2026-02-03 18:01:51 - To install N|solid Runtime, run: apt install nsolid -y
3.517
3.579 Hit:1 http://deb.debian.org/debian trixie InRelease
3.579 Hit:2 http://deb.debian.org/debian trixie-updates InRelease
3.588 Hit:3 http://deb.debian.org/debian-security trixie-security InRelease
3.651 Hit:4 https://deb.nodesource.com/node_22.x nodistro InRelease
4.850 E: Unable to locate package yarn
```
and it seems like debian doesn't host yarn (anymore?) which is
understandable. So we just do:
```
&& npm install --global yarn
```
And we should be good.
LLM diagnosed this. The setup here is a bit strange to me but it cats
it's resulting dockerfile:
```yaml
- name: Print compose config
run: cat mba-compose.yml
```
which was helpful. That dockerfile references:
```yaml
metabase:
build:
context: /home/runner/work/metabase/metabase/.devcontainer/
dockerfile: Dockerfile
```
And it's that devcontainer we are fixing here
* trigger kerberos tests
it needs files to be changed:
```yaml
run-presto-kerberos-test:
needs: files-changed
if: needs.files-changed.outputs.backend_presto_kerberos == 'true'
```
which is configured in `.github/file-paths.yaml`
```yaml
backend_presto_kerberos:
- "**/presto_jdbc/**"
- "**/presto_jdbc.clj"
- ".github/workflows/presto-kerberos-integration-test.yml"
```
so i add a `^:synchronized` teset to a file that should trigger this
files changed check and we get CI
* fix how we detect presto changes
usually hyphens are underscores, so it was looking for
`"**/presto_jdbc/**"` but the driver module is not a classpath root and
it uses a hyphen. Also it excluded the test file. so just manually add
that in, although it would be hit by the hyphen rule that i added
* correct typo in test filename
* touch test file
* Update oss sqlgen snowplow tests to use snowplow-test/with-fake-snowplow-collector
To follow existing snowplow test patterns from other modules.
* Move chat-completion-returns-usage-test next to other chat-completion tests
* Rename estimate-sanity-test to estimate-simplified-models-test
And use "raw" model names by default
* Update model-pricing with all models returned by list-models
Also add a comment with code for updating the model-pricing table.
* Ensure estimated_costs_usd is both number? and pos?
to ensure you still get a readable diff when the predicate fails rather than a hard to debug stacktrace.
* Use mt/with-random-premium-token! to get valid test token
* feat: send current user email on cloud reqs
* feat: upgrade modal
* refactor: simplify upsell banner integration & tests
* test: metabase-enterprise.cloud-proxy.api-test
* fix: error stats for upgrade modal
* etc: whitespace
* fix: upgrade modal should only show up for upgrade url
* feat: refresh page after upsell flow for module reload
* refactor: use global modals instead of localized jsx
* refactor: remove unnecessary mock data and convert to localized strings
* test: add hm.client interceptor tests
* constants to types
---------
Co-authored-by: Gabriel Moreno <gabriel@metabase.com>
Co-authored-by: Ryan Laurie <iethree@gmail.com>
* Misc agent API improvements + llms-agent-api.txt
* add llms-agent-api.txt
* Add more specific query execution response schema, fix some tests, and update llms.txt again
* Simplify count aggregation schema
* address comments, some other tweaks
* rename & move agent API llms.txt
* bump ci
* bump ci
* small change: ensure we record query execution properly with :agent context
* fix test
BigQuery can't do case statements with different types in each branch. Other dbs can. This fixes a test that ran a case statement with different types to skip BigQuery, but it adds a new assertion to check correct behavior without it.
This also corrects a fault in the SnowFlake driver that incorrectly assumed that convertTimezone() returned a timestamp_ntz but it returns a timestamp_tz. This wraps it in a cast back to timestamp_ntz.
* Add `autoRefreshInterval` to the SDK (#68729)
* Fix new callback don't get registered with our own `useInterval`
* wip
* Handle negative number
* Add autoRefreshInterval to StaticDashboard storybook
* Add autoRefreshInvertal to EditableDashboard storybook
* Add tests
* Make test order more sensible
* Add more test coverage
* Update types
* Update test format for consistency
* Make the condition clearer
* Fix wrong user-facing comment
* Fix wrong test name
* use the same jest fake timer configuration for all tests
* Address review
* Add support for `auto-refresh-interval` in modular embedding (#68830)
* Support `auto-refresh-interval` in modular embedding
* Make the test names more sensible
* Add tests
* Add an auto-refresh indicator (#68847)
* Add a dummy refresh indicator
* Add the actual icon
* Pass tests
* Add tests
* Fix another test
Move reindex connection management to use a connection only around the writes, not try to use a connection across all writes, which may commit and therefore close server-side cursors still being used to read the next batch.
* Add tests for source-model-card
source-model-card is only called from one function, card-returned-columns.
So we have to test source-model-card (which is private) through card-returned-columns.
This kills all mutants but 3, which have to do with the self-reference check in source-model-card. Is it really
necessary? It is an impossible branch to reach.
* new pickers
implement all pickers
remove test code
remove dead code
type fixes
reorganize
type fixes
data picker plus plus
more fixes
type fixes
fix column test ids
type and testfixes
some e2e test updates and kick off ci again
improved default states
more e2e test updates
test updates
better loading state
lint fixes
better root namespace handling
fix more root collection nonsense
improve backparsing logic
fix closing multiple modals on esc or enter
more e2e test updates
e2e test updates 4
fix (some) backparsing and add tests
fix spacing
e2e test updates 5
more backparse fixes
weird snippet deal
more tenant shenanigans
sweet sweet unit tests
add back NewDashboardDialog
more e2e tests and start NewDashboardDialog unit test
lint fixes
fix infinite render loop 😅
various fixes
bug fixes
search improvements
better cache hitting
dialog unit tests
test fixes
mucho test fixes
* fix conflict resolution errors
* test and type fixes
* bug and test fixes
* New folder button for snippets and transforms, e2e tests
* fix some dashboard cases
* test fixes
* entity picker e2e tests
* much fixo
* more e2e fixes
* fix some flakes
* prettier
* fixes
* more e2e test updates
* fix sdk component tests
* some stabilization
* fix dashboard saving bug
* prettier
* address review comments
* test fixes
* fix sandbox tests
* obey the linter
* cant pass null namespace
* oh types
* timeline
* fix default expanded 😫
---------
Co-authored-by: Kevin Kelleher <kevin.kelleher@metabase.com>
* Experiment with communicating with LLMs in Clojure
* Libor comments
* switch to new openai /v1/responses api, check for structured output support
* support for claude to show how different adapters are
* linter fixes
* update llm config edn
* init chat completion endpoint and env vars
* basic sqlgen BE endpoint
* BE test script
* basic table schema context extraction
* temp hack EE inline entrypoint to use OSS sql gen
* BE OSS async & streaming support
* BE OSS async & streaming support
* structured json response
* dialect instructions and sys prompt mustache
* hacky disable file logging via env var
* improved dialect mapping
* improved table metadata
* fix merge conflict
* abstract out sql generation to a seperate hook
* move MetabotPromptInput and MetabotInlineSQLPrompt components into OSS land
* always register metabot context rather than doing it conditionally in enterprise
* move OSS implementation into a seperate file
* oss implementation first pass
* add setting to know if the feature is enabled or not, optionally show ui based on if it is enabled, consolidate state into a single hook call to fix OSS issues, gets everything working
* clean up
* add test coverage
* revert metabot context changes, they did not end up being needed
* restrict @mentions to tables in OSS and models, tables, questions in EE
* send current sql value and no longer send buffer id
* add anthropic provider
* dumb module resolution workaround to get unit tests working correctly
* if there is only one search model provided preselect it
* tweak sys prompt; add table descriptions
* improved table & field metadata
* tweak model env var
* add openai settings
* fix CSP tiptap nonce console error
* fix oss not having access to default import of use-metabot-sql-suggestion hook in prod builds
* add backend support for source_sql
* OSS SQL Generation v1.6 (#68584)
* update layout
* impl table input bar
* persist state on open / close of inline sql prompt input
* ui improvements, referenced_entities now in req/res body
* support referenced_entities param in /generate-sql; add extract-tables endpoint
* use search, fix some ux/ui issue with dropdown
* fix sizing issue
* simplify generate-sql response format; properly union the referenced entities
* return table model spec in response
* updated backend tests
* improve table selection logic using new endpoint changes w/ richer data
* rate limiting; throttling; http timeouts; max resp tokens
---------
Co-authored-by: Sloan Sparger <sloansparger@gmail.com>
* small spacing / sizing corrections
* fix ee extracting tables when it should not, add test cases
* handle disabeld state and enter press better
* cleanup
* fix modules
* add init; fix kondo
* test fix
* fix modules
* Add snowplow events to oss sqlgen endpoints (#68822)
* Copy token_usage snowplow schema from ai-service
* Add :snowplow/token_usage to metabase.analytics.snowplow/schema->version
* Add basic anthropic cost estimation based on static tables
* Return :usage and :duration-ms from llm.anthropic/chat-completion
* Add snowplow token_usage tracking to oss /generate-sql endpoint
* Add snowplow token_usage tracking to oss /generate-sql-streaming endpoint
* Add snowplow simple_event tracking for sqlgen success and failure events
* Consolidate snowplow tracking tests
* Appease whitespace linter
* Don't use non-api analytics.setting namespace
* Ensure duration_ms is an integer in snowplow events
* Add note about oss instances to hashed_metabase_license_token schema description
Co-authored-by: Thomas Schmidt <thomas@metabase.com>
* Fix snowplow lint by adding old versions of token_usage schemas
The snowplow linter was complaining that these older schemas were missing. These versions had been deleted from the
ai-service repo, so weren't copied over. Restore them from the last version in ai-service prior to being deleted.
---------
Co-authored-by: Thomas Schmidt <thomas@metabase.com>
* Add README reminder to token_usage snowplow schema
See also https://github.com/metabase/ai-service/pull/605
* cleanup streaming
* rm stream test
* pr feedback & test fixes
* test fix
* [Metabot] OSS SQL generation admin UI changes (#68824)
* wip
* load models async, move model input to a select, show error if deprecated model
* move to password input for api key
* handle case where env vars have been set via env vars and can not be changed
* clean up oss view
* remove token checks for deciding to upsell metabot
* be tests for new models endpoint
* fix bug
* fix broken api tests
* ui feedback, token validation, style fixes
* unit tests
* fix cypress
* fix e2e test tag
* type fixes
* condo / eastwood fixes
* fix unit tests
* fix e2e test
* fix stale css var reference
* pr feedback
* fix var
* fix n+1 query; improve metadata fetching
* driver multimethod for dialect instruction resource; review PR cleanups & fixes
* pr feedback
* fix added in driver multimethod
* whitespace lint
* address my own comments
---------
Co-authored-by: Oleksandr Solovyov <o@solovyov.net>
Co-authored-by: Sloan Sparger <sloansparger@gmail.com>
Co-authored-by: appleby <86076+appleby@users.noreply.github.com>
Co-authored-by: Thomas Schmidt <thomas@metabase.com>
Co-authored-by: Sloan Sparger <sloansparger@users.noreply.github.com>
Co-authored-by: Noah Moss <noahbmoss@gmail.com>
* replace moment with dayjs in cljs
* Fix CLJS compilation with DayJS port of metabase.util.time
* fix some tests
* fix time parsing in drill chart
* formatting
* make a better fix
* kondo
* fix sdk test
* indent
* drop moment from locales
* reorder imports
* ran bin/mage cljfmt-all to indent
* Add an ignore for 1 unsorted namespace
- cljfmt makes the namespaces unordered -> kondo complains
* typo
* attampt to move logic to clojurescript
* one more thing to remove business logic from metabase-lib
* address cljs review
---------
Co-authored-by: Braden Shepherdson <braden@metabase.com>
Co-authored-by: Bryan Maass <bryan.maass@gmail.com>
* feat: add library/transforms toggles to modal
In the 'Set up git sync' modal in data studio this adds pre-selected
toggles to sync the library and transforms. This also creates the
library if you select it to sync, but it has not yet been created.
* fixup! push the library/transforms behavior into CollectionList
* fixup! better types to reduce casting
* fixup! missing mock setup
* Update enterprise/frontend/src/metabase-enterprise/remote_sync/components/RemoteSyncAdminSettings/RemoteSyncSettingsForm.tsx
Co-authored-by: Anderson <ands.alves.nunes@gmail.com>
* fixup! separate library sync component
* fixup! prettier
---------
Co-authored-by: Anderson <ands.alves.nunes@gmail.com>
* add fix and test
* dont drop bigquery tables and add test
* use atomic renames when possible
* use helper functions
* use existing wait-for-table
* use shared temp-table-name
* small change
* remove bigquery hack, make more similar to python impl
* address comments
* use random suffix and table name for temp tables
* use table hash
* update temp-table-name
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
* mocked test to verify strategy selection
* fix test
* test description
* remove test
---------
Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
Adds new spec-driven methods to detect writability of remote-sync
controlled methods. These use the existing sync-eligibility settings so
in the future we can straightforwardly apply these settings to
remote-sync controlled models (TODO: follow up to build them for the
other models).
Adds batched-versions of these checks to avoid n+1 queries when loading
can_write for models.
Adds tests of remote-sync checks for snippets.
* Split out more granular 400 vs 404 error codes for metabot tools
* a couple new tests
* fix test
* fix tests, cleanup
* discrete error handling in resolve-datasource
* refactor
* undo some refactor to fix tests
* Add an alerts button on the SDK (#67796)
* Prepare to add alerts to SDK
* Remove unrelated prop from storybook
* Add basic alerts button
* Only show alerts button with the proper flag
* Prevent download button from being shown without proper flag
* use CSS to hide things instead of JS
* Properly position all question buttons
* Use JS to hide empty divs as pure CSS is impossible
* Render alert modal ✌️
* Fix type
* Add todo
* Structure the components better
* Control alert modal state
* Add interactive question storybook
* Make interactive and static question button styles consistent
* Add alerts modal to interactive question
* Make alert button works with custom layouts
* Fix tests
* Fix unit tests
* Fix import order issue
* Remove outdated comment
* Add the same condition to display alert button from the core app
* Polish
* Modify question storybooks to show a question instead of a model
* Hide empty elements more reliably
* Add more conditions to display alert button
* Remove unused `classNames` prop
* Use the same style as the interactive question one
* Remove outdated diff
* Fix alerts button not working in custom layouts
* Remove another unused code
* Add a basic test
* Add unit tests
* fix visibility logic
* Fix E2E tests
* Make assertion on static question tests better
* Improve assertions
* Add interactive question unit tests
* Add e2e test for static question
* Add interactive question e2e tests
* Fix subcomponent
* clarify comment
* Hide other channels than email on the alert modal (#68184)
* Hide channel options on alerts modal
* Add unit tests
* Make the test name clearer
* Fix can't view created notifications the first time (#68197)
* Fix can't view created notifications
* Add tests
* Remove unnecessary return type
* Add a safety guard
* Hide Alert (and download) button on edit mode (#68189)
Hide some buttons on edit mode as it makes no sense there
* Alerts polishing (#68477)
* Misc. improvements
* Make it typesafe
* Add alerts to modular embedding (#68286)
* Add the option in modular embedding wizard
* Support `withAlerts` in modular embedding
* Close the modal after hiding the alert button
* Add E2E tests
* Fix type
* Fix `withAlerts` not working on modular embedding with `drills: false`
* It's 2026!
* 1 plugin to rule 'em all!
* Fix E2E after merging master
* Fix test name
* Add alerts analytics (#68531)
* Add alert setup analytics
* Track withAlerts on modular embedding
* Update snowplow schema
* Fix E2E tests
* Fix schema version
* Remove links from alerts test email (#68547)
* Remove links from alerts template
* Remove links from alert email computed part
* Add tests
* Make embedding util easier to use
* Make alerts and subscriptions payload lighter
* Remove unexpected pprint
* Remove links from alert email via tasks (#68610)
* Save notification card's disable_links when creating
* Add tests
* brand key adjustments
* loki tests
* handle the forgotten brand-dark
* Update Loki Snapshots
* removing brand-hover from dynamic-css-vars
* Adjustments to calendar picker colors
* small adjustments on the permissions pages
* sdk: remove background-hover tests from sdk as dynamic css derives are replaced with color-mix
* stupid spelling
* building shared resources
* Adjusting unit test, loki images
* rebasing adjustments
* Update Loki Snapshots
* small color fixes while I wait for loki
* re-adding highlighted class
---------
Co-authored-by: Metabase Automation <github-automation@metabase.com>
Co-authored-by: Phoomparin Mano <phoomparin@gmail.com>
* Data Analyst CRUD on measures/segments
Data analysts with unrestricted view-data permissions on a table can now
create, update, and delete segments and measures for that table
(previously superuser-only).
`can-read?` was already granted by the `manage-table-metadata` permission.
Add permission tests for `can-write?` and `can-create?` methods.
* feat: enable transform setting on first import
When a we import a remote that has transforms set the
`remote-sync-transforms` setting to true.
Update the remote sync object table clean up to be driven by the spec
configuration.
Add tests showing that transforms are consistently synced and erased
according to the following rules:
1. if the instance has `remote-sync-transforms` off and local transforms
and the remote does not have transforms content they are preserved after
an import
2. if the instance has `remote-sync-transforms` off and local transforms
and the remote has transforms content, the local transforms are
completely replaced by the remote.
3. if the instance has `remote-sync-transforms` on and the remote has
transforms content any local transforms are completely replaced by the
remote.
4. if the instance has `remote-sync-transforms` on has local transforms,
but the remote has no transforms, the local transforms are removed on import.
* fixup! relocate settings update
* fixup! don't let python libraries clash on import
* fixup! transactions for mysql
* feat: transform setting into collection list
Move the setting for syncing transforms into the list of collections
available to remote sync, so conceptually the transforms root collection
looks like a standard syncable collection.
* Clean up FE a little bit
* undo unneeded changes
---------
Co-authored-by: Anderson <anderson@metabase.com>
* temp
* hook up mode switcher in other apps
* Ci issues
* adding get help, fixing e2e tests, other adjustments
* most unit tests
* rename to profile link?
* adjusting unit tests with routing
* partial feedback
* rename profile link
* base current app off routes array
* adjustments
* add non-admin user state, fix unit tests
* PR feedback
* fixing test
* fixing e2e
* only 1 app switcher in admin navbar
* e2e adjustments
In the case statement, the default case did not TIMEZONE() to alter the timezone. The cases with TIMEZONE() returned a timestamp. The default case without TIMEZONE() returned timestamptz. Postgres reconciled the different types by converting all of them to timestamptz. So the timestamp was altered (by TIMEZONE()), but a new UTC timestamp was added (by the cast).
When the post-processor formats the rows, it checks that the metadata for the column has :converted_timezone. If it has that, it is formatted in the converted timezone, and since it's UTC, it will convert it again, a second time, which is incorrect.
The fix here is to check if the returned column is a timestamptz and strip it before formatting. We assume the database has already done the conversion. This avoids the second conversion.
* data studio upsell
* refactor: fix route guards
* fix: lint comments
* fix: data studio layout tests
* fix: remove local stub for api
* fix: moar lint errors
* fix: comments
---------
Co-authored-by: Gabriel Moreno <gabriel@metabase.com>
* fix: clean up style issues in new remote-sync code
Fixes a grab bag of small style/docstring issues I found after the
parent PRs were merged.
* fixup! lint
* fixup! keep the newsnippetpage as always dirty
* Fix migration for data studio
We usually auto-populate some other fields in permissions based on the value of object - perm_value, perm_type, and collection_id.
This obviously doesn't happen in a migration! So set them manually here.
* just roll forward
* Push the analysis of the dependents to the job
Fixes QUE-3001
The analysis findings of dependents are marked as stale (if they exists) and the analysis job picks them up with
higher priority. Also, the job is triggered right after the analysis of the dependency and doesn't stop until all
stale findings are refreshed.
* Fix boolean type in migration
* Set *warn-on-reflection* to true
* Stick to SQL change
* Fix tests
* Remove race condition
* Process review feedback
* fixes
* fix tests
---------
Co-authored-by: Alexander Polyankin <alexander.polyankin@metabase.com>
The revert operation triggers a chain: revert API -> reload card page.
Test was only waiting for the revert API, then sometimes it would manage
to click the History tab *before* the page reload. Then the page reloads
and we're not on the History tab anymore.
Waiting for the card query request to finish means that we've reloaded
the page and ready to roll.
* New endpoint implementations
* add new schemas for new endpoints
* new field values endpoints
* simple search endpoint + bastic test
* WIP - query construction endpoint
* Query construction tests, convert :output errors to real 404
* Basic query execution endpoint that just wraps qp/process-query
* Add query response schema
* Add descriptions to malli specs
* Add more tests, simplify some schemas
* Misc fixes, malli description improvements, try to fix decoding/encoding stuff but will need to revisit
* fix whitespace
* Greatly simplify schema sharing w/existing tool endpoints
* cleanup ns
* Fix some tests
* Fix up metric field values test
* add agent-client test helper, and new agent-api-user-binding-test
* simplify base64 query encoding
* small cleanups
* Address feedback
* add comment
* bump ci
* bump ci
* fix modules
* switch to agent-api feature flag
* Ensure we catch semantic search errors before RRF + add a new prometheus metric
* Revert "Ensure we catch semantic search errors before RRF + add a new prometheus metric"
This reverts commit d972fdf6df.
* Move fallback logic to semantic-search.core to not be metabot-specific, add test
* off grid detection
* woof
* fixing bad counts
* adjusting no data points check
* expose initalUiState in visualizer
* cleaning up dashcard functions
* adjust broken behavior for existing tests
* e2e tests
* PR feedback
* remove buttons to open settings and auto adjust. Adding tests for pinned cards and documents
* removing unneeded code, pr feedback, and adding loki screenshots
* fix silly thing
Add support for syncing PythonLibrary entities (Python transform libraries)
with a Git repository when remote-sync-transforms setting is enabled.
Implements serdes serialization for pythonlibraries using `path` as the
entity id.
Adds FE changes to refresh on dirty state and show changes in the AllChangesView
* Add support for versioned endpoints to deftool
* Create new API namespace + add custom jwt middleware
* add kondo module dependency
* revert the deftool stuff which isn't needed now
* fix lint issues
* fix jwt case insensitivity issue; more structured errors for auth issues
* small jwt cleanups
* Add support for session token exchange
* cleanup, lint, tests
* more test cases, cleanup
* bump ci
* Restructure session logic to use normal middleware, add test for session expiration
* fix up module config
* share more code with jwt sso module + remove some now-redundent tests
* try to fix tests by using existing SSO JWT test setup
* fix whitespace
* fix up module config
1. if the `log-artifact-suffix` is empty, just do
`test-logs-{github-job}`.
2. include a `log-artifact-suffix` argument for the driver tests
This way we don't get 409 conflict errors when uploading logs on test
failures for driver tests.
* refactor: remove avatars and "no owner" fallback from transforms list
* refactor: right-align rows column on data structure
* fix: search_check icon fill color
Say you are doing an implicit join from orders to products in order to summarize by products.category. Previously, with implicit joins, we join with the actual products table instead of a select blah from products wrapper, and so the various fields returned by the join were returned under their original name. However, the code would abbreviate excessively long column names from the joined table and break the query.
That difference was due to add-implicit-fields running before add-implicit-joins. Explicit joins got implicit fields added, while implicit joins didn't get those fields. This pr runs add-implicit-fields a second time after add-implicit-joins, which makes implicit and explicit joins compile to the same (correct) sql.
The test was missing an explicit database selection step which sometimes
resulted in the Save button being disabled.
Also removed the unnecessary sync_schema call — the syncable_schemas
endpoint queries Postgres directly via the driver, so no metadata
sync is needed.
* Do not delete built-in transform tags on sync
I found that some tests were failing intermittently - sometimes
TransformTags were present, and sometimes they were missing. I tracked
it down to this test, which enables and then disables remote syncing
transforms.
The issue is that if we sync transforms (... and transform tags), then
unsync them, then we delete everything that was synced - including the
built-in tags.
We should exclude these from deletion.
* Fix serialization API tests
logs:
```
2026-01-27 15:02:23,442 INFO util.queue :: Starting listener field-value-invalidate with 1 threads 🎧
2026-01-27 15:02:23,448 INFO util.queue :: Starting listener search-index-update with 1 threads 🎧
2026-01-27 15:02:23,448 INFO core.core :: Metabase Initialization COMPLETE
2026-01-27 15:02:23,453 INFO core.core :: Metabase Initialization COMPLETE in 4.8 s (JVM uptime: 57.8 s)
```
metrics:
```
❯ http get :9191/metrics | grep startup
metabase_startup_jvm_to_complete_millis 57764.0
metabase_startup_init_duration_millis 4816.0
```
Note that there two different notions of startup time: from jvm
instantiation, and from before init! was called. This maintains the two
separately so we can diagnose if class load time gets longer
independently of amount of work we do during startup itself
* feat: support remote sync for NativeQuerySnippets
When the Library collection is remote-synced, all NativeQuerySnippets
and their collections (snippets namespace) are automatically tracked
for remote sync. No separate setting is needed - syncing is tied
directly to the Library collection's is_remote_synced status.
Update the FE to use the declarative approach to refresh dirty state
when snippets are modified and update the library dirty state indicator
to turn on when any snippet or snippet collection is modified.
* fix: deletion of snippets/transforms collections
Fixes a bug where collections in the special namespaces were not cleaned
up after changing branches.
* fixup! don't double create entries
* fixup! transaction scope for test on mysql
* Show empty schemas when picking a transform target
* Use listSyncableDatabaseSchemas endpoint in data structure
* Handle empty schemas
* Add repro for metabase#68378
* Move transforms tests to new folder
* Split off transforms reproductions into separate specs
* Better logging around token checks
We used to just log we were checking but not give error responses. So it
would look like a ton of token checks for no real reason.
Now includes error responses if any, and also include logs when engaging
the token check circuit breaker, and when it is "half open", when it
lets one real token check happen every 30 seconds to see if the service
is back.
```
2026-01-23 12:26:47,175 INFO premium-features.token-check :: Checking with the MetaStore to see whether token 'f325...d01a' is valid...
2026-01-23 12:26:47,186 INFO premium-features.token-check :: Reporting Metabase stats: {:embedding-dashboard-count 0, :enabled-embedding-sdk false, :internal-users 1, :embedding-question-count 0, :query_executions_static_embed 0, :query_executions_public_link 0, :domains 1, :metabot-usage-date 2026-01-22, :enabled-embedding-static false, :enabled-embedding-simple false, :metabot-queries 0, :external-users 0, :metabot-tokens 0, :query_executions_simple_embed 0, :enabled-embedding-interactive false, :query_executions_internal 0, :metabot-users 0, :query_executions_sdk_embed 0, :query_executions_interactive_embed 0, :users 1}
2026-01-23 12:26:49,190 INFO premium-features.token-check :: Error checking token: Connect timed out †
2026-01-23 12:26:49,194 INFO premium-features.token-check :: Checking with the MetaStore to see whether token 'f325...d01a' is valid...
2026-01-23 12:26:49,201 INFO premium-features.token-check :: Reporting Metabase stats: {:embedding-dashboard-count 0, :enabled-embedding-sdk false, :internal-users 1, :embedding-question-count 0, :query_executions_static_embed 0, :query_executions_public_link 0, :domains 1, :metabot-usage-date 2026-01-22, :enabled-embedding-static false, :enabled-embedding-simple false, :metabot-queries 0, :external-users 0, :metabot-tokens 0, :query_executions_simple_embed 0, :enabled-embedding-interactive false, :query_executions_internal 0, :metabot-users 0, :query_executions_sdk_embed 0, :query_executions_interactive_embed 0, :users 1}
2026-01-23 12:26:51,205 INFO premium-features.token-check :: Engaging circuit breaker in token check ††
2026-01-23 12:26:51,206 INFO premium-features.token-check :: Error checking token: Connect timed out
2026-01-23 12:27:44,548 INFO premium-features.token-check :: In token check circuit breaker but attempting check
2026-01-23 12:27:44,551 INFO premium-features.token-check :: Checking with the MetaStore to see whether token 'f325...d01a' is valid...
2026-01-23 12:27:44,565 INFO premium-features.token-check :: Reporting Metabase stats: {:embedding-dashboard-count 0, :enabled-embedding-sdk false, :internal-users 1, :embedding-question-count 0, :query_executions_static_embed 0, :query_executions_public_link 0, :domains 1, :metabot-usage-date 2026-01-22, :enabled-embedding-static false, :enabled-embedding-simple false, :metabot-queries 0, :external-users 0, :metabot-tokens 0, :query_executions_simple_embed 0, :enabled-embedding-interactive false, :query_executions_internal 0, :metabot-users 0, :query_executions_sdk_embed 0, :query_executions_interactive_embed 0, :users 1}
2026-01-23 12:27:46,574 INFO premium-features.token-check :: Engaging circuit breaker in token check
2026-01-23 12:27:46,575 INFO premium-features.token-check :: Error checking token: Connect timed out
...
2026-01-23 12:32:50,432 INFO premium-features.token-check :: In token check circuit breaker but attempting check †††
2026-01-23 12:32:50,433 INFO premium-features.token-check :: Checking with the MetaStore to see whether token 'f325...d01a' is valid...
2026-01-23 12:32:50,444 INFO premium-features.token-check :: Reporting Metabase stats: {:embedding-dashboard-count 0, :enabled-embedding-sdk false, :internal-users 1, :embedding-question-count 0, :query_executions_static_embed 0, :query_executions_public_link 0, :domains 1, :metabot-usage-date 2026-01-22, :enabled-embedding-static false, :enabled-embedding-simple false, :metabot-queries 0, :external-users 0, :metabot-tokens 0, :query_executions_simple_embed 0, :enabled-embedding-interactive false, :query_executions_internal 0, :metabot-users 0, :query_executions_sdk_embed 0, :query_executions_interactive_embed 0, :users 1}
2026-01-23 12:32:52,450 INFO premium-features.token-check :: Engaging circuit breaker in token check
2026-01-23 12:32:52,450 INFO premium-features.token-check :: Error checking token: Connect timed out
```
The highlights are now we include:
- † `Error checkign token: Connect timed out` the error response when
appropriate
- †† `Engaging circuit breaker in token check` when we engage the
circuit breaker to no longer do real token checks
- ††† `In token check circuit breaker but attempting check` indicating
we are doing a real one from the circuit breaker to determine if the
service is restored
* Whew. Lots of stuff
add a namespace to the circuit breaker keyword:
`:token-check/circuit-breaker`. Change the keyword holding
`:token-check/app-db-not-ready` to cause. I had used reason but nothing
checked that.
this is because want to log when a token check fails, but not when it is
- failure due to circuit breaker
- failure due to appdb not ready yet.
token checks can happen anywhere and with an enormous frequency. they
fail before the appdb is setup (we can't check settigns yet) and of
course they fail during the circuit breaker opening. that's the point of
the circuit breaker. So we don't have to log these.
We _do_ want to log other failures.
We also want to do some prometheus metrics. Here's a journey:
You previously couldn't import the prometheus analytics namespace
because analytics.settings required appearance.settings. It does so
because a docstring in analytics settings used to include Metabase in
the string. I removed that so now it says "in order to help us
improve". And then things would be smooth sailing.
Except we had an issue.
```
026-01-23 20:50:03,514 INFO analytics.prometheus :: Starting prometheus metrics collector
2026-01-23 20:50:03,539 INFO analytics.prometheus :: Starting prometheus metrics web-server on port 9191
thread229238 we made the system!
thread229238 and now we are done with the lock!
2026-01-23 20:50:03,607 INFO analytics.prometheus :: Starting prometheus metrics web-server on port 9191
2026-01-23 20:50:03,611 ERROR premium-features.token-check :: Error validating token: Failed to initialize Prometheus on port 9,191
2026-01-23 20:50:03,611 INFO analytics.prometheus :: Starting prometheus metrics web-server on port 9191
2026-01-23 20:50:03,612 INFO analytics.prometheus :: Starting prometheus metrics web-server on port 9191
2026-01-23 20:50:03,614 INFO analytics.prometheus :: Starting prometheus metrics web-server on port 9191
2026-01-23 20:50:03,616 INFO analytics.prometheus :: Starting prometheus metrics web-server on port 9191
2026-01-23 20:50:03,617 INFO analytics.prometheus :: Starting prometheus metrics web-server on port 9191
2026-01-23 20:50:03,618 INFO analytics.prometheus :: Starting prometheus metrics web-server on port 9191
2026-01-23 20:50:03,619 INFO analytics.prometheus :: Starting prometheus metrics web-server on port 9191
2026-01-23 20:50:03,619 INFO analytics.prometheus :: Starting prometheus metrics web-server on port 9191
2026-01-23 20:50:03,653 INFO core.core :: Setting up prometheus metrics
```
We have locking around the system but it is starting tons of times! Lots
of debugging, swearing that this can't happen. but then i finally threw
in my favorite trick: `(clojure.repl/pst (Exception. "how did we get
here"))`. And it turns out we are doing this:
1. token check fails, so we call `prometheus/inc!`
2. inc is helpful, sees system isn't yet setup, and sets it up for us
3. Setup metric system
4. setup metrics in system
5. we hit the following code which gets initial values for labels
```clojure
(defn- initial-labelled-metric-values []
(for [metric (keys (methods known-labels))
labels (known-labels metric)]
{:metric metric
:labels labels
:value (initial-value metric labels)}))
```
6. We hit the following search code about which search we have
```clojure
(defmethod analytics/initial-value :metabase-search/engine-active
[_ {:keys [engine]}]
(if (search.engine/supported-engine? (keyword "search.engine" engine))
1 0))
```
7. This takes us to semantic search:
```clojure
(defmethod search.engine/supported-engine? :search.engine/semantic [_]
(try
(supported?)
(catch Exception e
(log/warn e "Semantic search engine not supported")
false)))
```
8. `supported?` here checks to see if you support a particular search
impelementation, which of course depends on _YOUR FEATURES_.
9. Checking features fails, so we should increment our prometheus
counter
10. the system is not yet initialized yet. Let's initialize it. (this is
also step 2)
So I have added a simple `inc-if-initialized!` function. It's not
important if the system is down. It's ok if these events are lost. The
system will be initialized in core anyways.
```
❯ http get :9191/metrics | grep metabase_token
HELP metabase_token_check_attempt_total Total number of token checks. Includes a status label.
TYPE metabase_token_check_attempt_total counter
metabase_token_check_attempt_total{status="failure",} 8.0
```
logs:
```
2026-01-23 22:58:26,668 INFO premium-features.token-check :: Checking with the MetaStore to see whether token '82bf...c3fe' is valid...
2026-01-23 22:58:28,706 INFO premium-features.token-check :: Error checking token: Connect timed out
2026-01-23 22:58:28,750 INFO core.core :: Setting up prometheus metrics
2026-01-23 22:58:28,751 INFO analytics.prometheus :: Starting prometheus metrics collector
2026-01-23 22:58:28,789 INFO premium-features.token-check :: Checking with the MetaStore to see whether token '82bf...c3fe' is valid...
2026-01-23 22:58:30,793 INFO premium-features.token-check :: Error checking token: Connect timed out
2026-01-23 22:58:30,798 INFO analytics.prometheus :: Starting prometheus metrics web-server on port 9191
...
2026-01-23 22:58:53,441 INFO premium-features.token-check :: Checking with the MetaStore to see whether token '82bf...c3fe' is valid... {mb-quartz-job-type=SyncAndAnalyzeDatabase}
...
2026-01-23 22:58:55,445 INFO premium-features.token-check :: Engaging circuit breaker in token check {mb-quartz-job-type=SyncAndAnalyzeDatabase}
2026-01-23 22:58:55,445 INFO premium-features.token-check :: Error checking token: Connect timed out {mb-quartz-job-type=SyncAndAnalyzeDatabase}
```
```
Exception how here??
metabase.analytics.prometheus/setup! (prometheus.clj:584)
metabase.analytics.prometheus/setup! (prometheus.clj:576)
metabase.analytics.prometheus/inc! (prometheus.clj:668)
metabase.analytics.prometheus/inc! (prometheus.clj:658)
metabase.analytics.prometheus/inc! (prometheus.clj:665)
metabase.analytics.prometheus/inc! (prometheus.clj:658)
metabase.premium-features.token-check/error-catching-token-checker/reify--98262 (token_check.clj:431)
metabase.premium-features.token-check/check-token (token_check.clj:488)
metabase.premium-features.token-check/check-token (token_check.clj:484)
metabase.premium-features.token-check/check-token (token_check.clj:486)
metabase.premium-features.token-check/check-token (token_check.clj:484)
metabase.premium-features.token-check/eval98325/*token-features*98348--98349 (token_check.clj:534)
metabase.premium-features.token-check/eval98325/fn--98353 (token_check.clj:529)
metabase.premium-features.token-check/has-feature? (token_check.clj:575)
metabase.premium-features.token-check/has-feature? (token_check.clj:569)
clojure.lang.Var.invoke (Var.java:386)
metabase.premium-features.defenterprise/check-feature (defenterprise.clj:41)
metabase.premium-features.defenterprise/check-feature (defenterprise.clj:35)
metabase.premium-features.defenterprise/dynamic-ee-oss-fn/fn--71956 (defenterprise.clj:53)
metabase.search.semantic.core/eval180600/fn--180601 (core.clj:58)
clojure.lang.MultiFn.invoke (MultiFn.java:229)
metabase.search.core/eval139302/fn--139304 (core.clj:77)
clojure.lang.MultiFn.invoke (MultiFn.java:234)
metabase.analytics.prometheus/initial-labelled-metric-values/iter--97802--97808/fn--97809/iter--97804--97810/fn--97811 (prometheus.clj:521)
clojure.lang.LazySeq.force (LazySeq.java:50)
clojure.lang.LazySeq.realize (LazySeq.java:89)
clojure.lang.LazySeq.seq (LazySeq.java:106)
clojure.lang.RT.seq (RT.java:555)
clojure.core/seq--5488 (core.clj:139)
clojure.core/concat/fn--5579 (core.clj:727)
clojure.lang.LazySeq.force (LazySeq.java:50)
clojure.lang.LazySeq.realize (LazySeq.java:89)
clojure.lang.LazySeq.seq (LazySeq.java:106)
clojure.lang.Cons.next (Cons.java:41)
clojure.lang.RT.next (RT.java:733)
clojure.core/next--5472 (core.clj:64)
clojure.core/next--5472 (core.clj:64)
metabase.analytics.prometheus/setup-metrics! (prometheus.clj:546)
metabase.analytics.prometheus/setup-metrics! (prometheus.clj:533)
metabase.analytics.prometheus/make-prometheus-system (prometheus.clj:60)
metabase.analytics.prometheus/make-prometheus-system (prometheus.clj:55)
metabase.analytics.prometheus/setup!/fn--97866 (prometheus.clj:591)
metabase.analytics.prometheus/setup! (prometheus.clj:590)
metabase.analytics.prometheus/setup! (prometheus.clj:576)
metabase.analytics.prometheus/inc! (prometheus.clj:668)
metabase.analytics.prometheus/inc! (prometheus.clj:658)
metabase.analytics.prometheus/inc! (prometheus.clj:665)
metabase.analytics.prometheus/inc! (prometheus.clj:658)
metabase.premium-features.token-check/error-catching-token-checker/reify--98262 (token_check.clj:431)
metabase.premium-features.token-check/check-token (token_check.clj:488)
```
* module config fixup
There's apparently an instance on cloud that is trying to remove 70k queries from the cache at once, and postgres
doesn't allow 70k params to a query. Batching the deletes in groups of 1000 should fix that.
* Rename file containing jsx to .jsx
* Add e2e helpers to build metadata provider
* Split up metadata helpers
* WIP
* Add more direct createTestQuestion helper
* Replace test
* Build cljs for tests
* Export type
* Only require entities in the state for getMetadata
* Just pass state to getMetadata
* Move EntitiesSchema to metabase/schema
* Wrap mbql e2e helpers in promise
* Split off getMetadataWithoutSettings
* Use getMetadataWithoutSettings to get metadata from state
* Build entities state manually
* Simplify wrapper
* Fix metadataProvider
* Add createTestJsQuery wrapper for cypress
* Use plain promises and fetch
* Allow passing GetMetadataOpts to createTest(Native)JsQuery directly
* Improve createTestQuestion types
* Make use of createTestJsQuery in test
* Reduce nest in createTestQuestion
* Use simplified createTestQuestion
* Split up mbql helpers
* Fix card metadata call
* Use correct card endpoint
* Pass missing options
* Convert another more complex test to createTestQuestion
* Fix types
* Use static ids for defaults to avoid import loop
* Fetch more tables by default
* Build cljs in embedding tests
* Build cljs in component tests
* Prepare backend env
* Add plugin so we can avoid building CLJS everywher
* Avoid building cljs where not needed
* Fix types
* Rename query helpers
* Split up helpers
* Compile cljs in stress tests
Update E2E test timings with averaged data
Collected timing data from 7 days of test runs.
All specs with timing data were included (simple averages).
🤖 Generated automatically by update-e2e-timings workflow
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: GitHub Actions <actions@github.com>
* Respect date_abbreviate when formatting date
* Respect abbreviate setting in month format too
* Add unit tests for abbreviated formats
* Add unit test for empty formatting settings
* Add repro for metabase#68574
* Add formatting settings unit tests
The existing transform dependencies code only worked with query-based transforms (mbql or native). This extends that
code to work with python transforms as well.
* Read-only: Adjust transform header menu options / Remove edit definition button / Run tab updates
* Make Settings tab read-only
* fix remote sync setup in e2e test
* Do not allow creating a new transform
* Make Python library not editable
* Hide edit definition in mbql transform view page
* Fix e2e
* Group e2e tests to improve performance
Closes DEV-1289
### Description
Within the last day or two, `metabase.query-processor-test.remapping-test/self-referencing-test` began failing on MySQL Latest. I hypothesize this has to do with the [latest release](https://dev.mysql.com/doc/relnotes/mysql/9.6/en/news-9-6-0.html) fixing an unknown bug around batch inserts in InnoDB and the deprecation of a recusive [sic] fk-related flag.
The error is thrown not in the test conditions, but in the test setup -- MySQL apparently now checks foreign key constraints row-per-row even on a batch insert.
This adds a flag that the test can set which changes the test data-loading behavior to disable the FK check before insertion and re-enable it after.
* fix(UXW-2752): remove unwanted padding on data structure fields tab group
* fix(UXW-2707): remove double border on job transforms section
* feat(UXW-2718): add horizontal layout for schedule controls
* feat(UXW-2709): fix spacing in the segments empty state
* fix(UXW-2780): long text overflow in user input
* fix(UXW-2710): set proper placeholder text on empty python library first load
* feat(UXW-2767): rename 'New collection' to 'Transform folder'
* refactor: blow grid & use `Ellipsified` instead
Closes QUE-2984.
Adds an event handler for when a DWH sync is completed, which checks all its tables and fields for
their latest update times, and re-analyzes all cards which depend on tables updated since the last analysis.
* feat: support remote-syncing measures
Add remote-sync support for measures, following the same pattern as segments:
- Add :model/Measure spec to remote-sync-specs with hybrid identity,
parent-table eligibility, and event tracking
- Update measure permissions to check remote-sync/table-editable? for
can-write? and can-create? (replacing superuser-only derive)
- Add batched hydration for :can_write on measures
- Add measures to serdes/descendants for Table so they export with parent
- Add tests for measure import/export tracking in impl_test.clj
On the Frontend creates a new data-driven approach to setting up the
middleware listener for changes that cause us to refresh dirty-state.
* fixup! typecheck
* Hoist macaw impl from query analyzer
* Use macaw-opts in bigquery module
* Use macaw-options in sqlserver
* Use macaw-options in sql driver
* Remove require
* Comment
* Test
* Update test
* Update macaw version
* Best effort support for bigquery
* Make test driver specific
* Remove final-not-reserved
* Not allowed on clickhouse only
* Clickhouse exception test
* best effort opts on api
* Remove final-non-reserved
* Remove requires
* driver.u/parsed-query
* Use parsed-query
* parsed-query
* ch test
* throw
* Lint against incorrect Macaw usage (#68568)
* Remove now unused function
* Shutdown loud exceptions for now
* Shutdown ch test
* Update test
---------
Co-authored-by: Chris Truter <crisptrutski@users.noreply.github.com>
Fixes https://github.com/metabase/metabase/issues/68578
This issue is not about the stack overflow per-se but blowing up on a
string includes:
```javascript
{
"app": "metabase",
"time": "2026-01-22T12:08:53.660601485Z",
"stream": "stderr",
"app_version": "v1.57.10-X01",
"message": "Exception in thread \"Thread-41\" java.lang.NullPointerException: Cannot invoke \"Object.toString()\" because \"s\" is null
at clojure.string$includes_QMARK_.invokeStatic(string.clj:377)
at clojure.string$includes_QMARK_.invoke(string.clj:373)
at metabase.app_db.cluster_lock$retryable_QMARK_.invokeStatic(cluster_lock.clj:27)
at metabase.app_db.cluster_lock$retryable_QMARK_.invoke(cluster_lock.clj:19)
at metabase.app_db.cluster_lock$do_with_cluster_lock.invokeStatic(cluster_lock.clj:87)
at metabase.app_db.cluster_lock$do_with_cluster_lock.invoke(cluster_lock.clj:65)
at metabase.app_db.cluster_lock$do_with_cluster_lock.invokeStatic(cluster_lock.clj:79)
at metabase.app_db.cluster_lock$do_with_cluster_lock.invoke(cluster_lock.clj:65)
at metabase.search.task.search_index$init_BANG_.invokeStatic(search_index.clj:36)
at metabase.search.task.search_index$init_BANG_.invoke(search_index.clj:32)
at clojure.lang.AFn.run(AFn.java:22)
at java.base/java.lang.Thread.run(Unknown Source)",
"namespace": "hosting-eb9b3875e12d465a",
"container": "metabase",
"cluster": "metabase-1",
"_p": "F"
}
```
The error message itself comes from a stackoverflow which perhaps does
not have a message? Either way, it's possible for this to be missing so
be a bit defensive
* Forcing overflow policy on ChartTypeDropdown
* tpyo
* refactoring to enable testing this with loki
* fixed story
* loki
* screenshot
* removed useless unit test
Filter extract-columns endpoint to only return numeric and temporal columns
The extract-columns-from-query function is used to populate the incremental
filter dropdown for native SQL transforms. Previously it returned all columns
without filtering on type, leading to undefined behaviour when unsupported
column types were selected.
This change filters the results to only include columns with types supported
for incremental checkpointing:
- Temporal types (timestamp, timestamp with timezone)
- Numeric types (integer, float, decimal)
Text, boolean, and other unsupported types are now filtered out, matching
the behaviour of MBQL and Python transforms which already filter columns
client-side.
Updated documentation to clarify this endpoint is specifically for incremental
transform checkpoint column selection.
Also validates that transform POST/PUT conform to this rule and you are unable to update the transform to have
a filter column of an unsupported type.
Reverts the revert of https://github.com/metabase/metabase/pull/67686
* Rename Data Studio Library +New menu action labels
Update the menu item labels in the Data Studio Library's +New button:
- "Publish a table" → "Published table"
- "New snippet" → "Snippet"
- "New snippet folder" → "Snippet folder"
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
* prettier
---------
Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
* feat: support remote sync transforms in FE
1. Updates the DataStudioLayout to show transform dirty state
We get the orange dot next to Transforms if transforms or transform
collections are dirty.
2. Update the all changes view to include transforms
Also includes their collection heirarchy if the transforms sync
setting is enabled
3. Add the transforms sync setting
To both the remote sync admin page and the Set up git sync modal in
the data studio
* fixup! refresh transforms after sync
* fixup! data studio test feedback
* fixup! refactor failing test to troubleshoot debugging
* feat: add Slack Connect SSO provider
Add enterprise Slack Connect SSO integration using the OIDC infrastructure:
- Slack Connect provider: OAuth2/OIDC flow for Slack workspace authentication
- Settings: configuration options for Slack client ID, secret, and auth modes
- User provisioning: automatic user creation from Slack identity
- API endpoints: SSO initiation and callback handling
- Analytics: usage tracking for Slack Connect feature
Slack Connect allows users to authenticate via their Slack workspace,
with support for both SSO-only and link-account authentication modes.
* fix: session reset after auth
* fixup! avoid email clash
* fixup! wrong value in sso settings
* fixup! cleanup tests
* fixup! fix test
---------
Co-authored-by: John Swanson <john.swanson@metabase.com>
Update llms.txt and llms-full.txt (master)
Regenerated llms.txt to include all markdown files from the docs/ directory.
Also generated llms-full.txt files for the embedding section.
🤖 Generated automatically by llms-txt-generate workflow
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: GitHub Actions <actions@github.com>
Previously even when autoprovisioning was on, you could not log into a
deactivated tenant (either as a new user or an existing user with that
tenant).
This changes the behavior to be more consistent. Just as a deactivated
user can log in when provisioning is on (and their user gets
reactivated), a deactivated tenant's users can still log in, but doing
so reactivates the tenant.
* add data studio library filtering events
* add bulk properties and unpublish event
* add bulk sync events
* add search event
* add assertions on events in e2e tests
* add search debounce
* remove unncessery comment
* update event names
* track filter searches with data_studio_table_picker_search_performed
* refactor tracking function
* guard and log event only for ConcreteTableId
* log search when started fetching
* fix events not fired
* fix event not fired on first search
* Add AnchorLinkButton component for document anchor links
- Add AnchorLinkButton component that copies URL to clipboard on click
- Shows 'Copied!' tooltip feedback using Mantine useClipboard hook
- Follows same pattern as CommentsButton for consistent styling
- Add unit tests for the component
* Add AnchorLinkMenu component for left-side anchor link hover
- Add AnchorLinkMenu floating menu for document blocks
- Uses createPortal for positioning (same pattern as CommentsMenu)
- Positioned on left side of blocks using floating-ui
- Shows/hides based on hover state via show prop
- Add unit tests for the component
* Add AnchorLinkMenu to document node views
- Add documentWithAnchor URL helper to generate shareable anchor URLs
- Update Heading, Paragraph, and Blockquote node views to render AnchorLinkMenu
- AnchorLinkMenu appears on the left side on hover
- CommentsMenu on the right side remains unchanged
- Each node view now uses two useFloating instances for positioning
* Add scroll-to-anchor for document URLs with hash
- Add E2E tests for anchor links functionality
- Implement scroll-to-anchor when navigating to document with #blockId hash
- Tests cover: hover shows anchor icon, copy to clipboard, scroll to block
- Regression test ensures CommentsMenu still works on right side
* Fix scroll-to-anchor dependency arrays for navigation edge cases
- Add documentId to scroll effect dependencies to re-run when navigating
between documents with the same anchor hash
- Add location.hash to reset effect dependencies to properly reset flag
when navigating via back button or changing anchors
* Fix accidental logo replacement and E2E blockquote test
- Restore original Metabase logo SVG (was replaced with red star debug graphic)
- Fix E2E test: use find('blockquote') instead of findByRole('blockquote')
since native blockquote elements have no implicit ARIA role
* Fix scroll-to-anchor reliability and empty line behavior
- Implement retry mechanism for scroll-to-anchor to ensure target element exists
- Add highlight animation when scrolling to anchor
- Hide anchor link and comment menus on empty lines (Heading, Paragraph, Blockquote)
* Fix selector mismatch for scroll-to-anchor
- Add data-node-id attribute to NodeViewWrappers in Heading, Paragraph, and Blockquote
extensions so they can be queried by DocumentPage
- Update DocumentPage to query using [data-node-id] selector instead of internal _id attribute
- Remove unused ID_ATTRIBUTE_NAME import in DocumentPage
* Fix highlight animation color variable
- Replace non-existent --mb-color-brand-alpha-25 with --mb-color-brand-light
- Ensure highlight animation is visible when scrolling to anchor
* added anchor links to lists and code blocks. some bugs fixes.
* make sure scroll-to-anchor observer is correctly initialized
- Refactor scroll-to-anchor logic to ensure smoother navigation
- Optimize highlight animation visibility during scrolling
- Adjust anchor link behavior on empty lines for better user experience
* small fixes
* clean up
* fixes and cleanup
* e2e test fixes
* fixed e2e test
* Leave anchor links only for header blocks
* Fix alignment
* Review fixes
* Fix unit tests
---------
Co-authored-by: Denis Berezin <denis.berezin@metabase.com>
https://github.com/metabase/metabase/pull/68190 did:
> Treats a 0 value of n_live_tup as NULL, due to confusion reported when
> auto analyze thresholds have not been met on a table.
I think what's happening is this is trying to suss out "i don't know"
and not give misleading information. When there are no stats pg would
report there are 0 rows rather than saying i don't know. So we toss away
these unreliable 0 count estimations.
This test flakes 7% of the time. Our hypothesis is that stats are empty
7% of the time so this row count estimation now gets removed.
By default the schema is `:estimated_row_count (mt/malli=? [:maybe
:int])` which was being clobbered with
`{:estimated_row_count (mt/malli=? int?)}` So the fix is to just not
clobber
* feat: support transforms in remote sync
1. Serialize Transforms as Collection Items
Transforms were previously serialized as a top-level item, but this
moves them to be serialized as a child of the collection they are
member of. This also fixes an error in their serdes/dependencies
implementation where it did not include the parent collection leading
to deserialization issues.
2. Enforce Remote-Sync write rules
Adds implementations of the model interface permissions methods for
transforms that enforce the can-write? = false and can-create? =
false rules when remote-sync is in read-only mode.
3. Add a setting to control whether transforms are synced
Transforms are globally synced not synced via collections, so this
adds a new boolean setting that controls whether they are synced on
instances in read-write mode. Like all other content transforms are
always synced from the remote source if the instance is in read-only
mode.
4. Add a new spec-based configuration system for models
Models we want to make remote-syncable have a complicated, but still
data-driven system to allow you to specify their remote-sync behavior
and have that automatically setup in a single place the
`metabase-enterprise.remote-sync.spec` namespace.
* fixup! test with wrong folder
* fixup! not handling archived transform collections
* fixup! clean up aliases
* basic analyst role
* analyst role
* Backend for data analyst role
* Fixup frontend to work with Backend
The dedicated analyst endpoints have been removed:
- `GET /api/user/analysts` -> Use `GET /api/user?is_data_analyst=true`
- `POST /api/user/analysts/:id` -> Use `PUT /api/user/:id` with `{ is_data_analyst: true }`
- `DELETE /api/user/analysts/:id` -> Use `PUT /api/user/:id` with `{ is_data_analyst: false }`
These hooks no longer exist:
- useListAnalystsQuery
- useAddAnalystMutation
- useRemoveAnalystMutation
New Patterns:
Listing analysts:
```
// Before
const { data } = useListAnalystsQuery();
// After
const { data } = useListUsersQuery({ is_data_analyst: true });
```
Adding an analyst:
```
// Before
await addAnalyst(userId).unwrap();
// After
await updateUser({ id: userId, is_data_analyst: true }).unwrap();
```
Removing an analyst:
```
// Before
await removeAnalyst(userId).unwrap();
// After
await updateUser({ id: userId, is_data_analyst: false }).unwrap();
```
Type Changes:
The field is_analyst has been renamed to is_data_analyst everywhere:
- User.is_data_analyst (was is_analyst)
- UpdateUserRequest.is_data_analyst (was is_analyst)
- ListUsersRequest.is_data_analyst (new field for filtering)
* kondo
* circular dep
* fix backend not compiling
* use the actual role
* analyst management, glossary access for admins
* various data studio access
* access to library
* drop custom analyst seats management ui
* automatic perms group
* fix tests
* specs
* permissions
* fix lints, add migration rollbacks
* data analysts can ALWAYS view table metadata
implemented for:
- search
- `table/can-read?`
* Also remove `can_access_data_model`
* Revert "Also remove `can_access_data_model`"
This reverts commit 03af432ccb.
* fix more tests
* more tests
* snapshots
* remove redundant checks
* everyone can see
* hide data analyst group for OSS
* Make data analysts a normal group on OSS
* Automatically provide data analysts with perms
Data analysts automatically get manage-table-metadata permissions on
every table/database.
* only disable setting some perms for data analysts
* kondo
* oops, one more kondo
* test fixes
* specs, fixes
* more specs, fixes
* cleanup
* more test fixes
* fix module config
* ci
* doh, fix test
* more test whack-a-mole
* move data analyst group to the top of perms views
* review
* ci
* Turn DA into a regular group when feature missing
instead of just on OSS.
The goal is:
- OSS: a previously existing Data Analysts group is a regular group with
configurable permissions. The real Data Analysts group is invisible and
has no members.
- EE without the token: same as above
- EE with the token: the real Data Analysts group is visible.
* oops, remove unnecessary require
* another one
* fix membership API
* fix a couple more tests
* macro gets a !
* fix `fetch-groups-test`
* ci
* ci
* ci
* ci
* fix collection perm in migration
---------
Co-authored-by: John Swanson <john.swanson@metabase.com>
Add :fail-fast? true to BigQuery and Snowflake test-args, matching the
existing Redshift configuration. This stops test execution on first
failure, saving CI time and resources.
If the first table mentioned by a dashboard has no fields (presumably because the table and its field are all
inactive), /api/dashboard/<id>/query_metadata would fail due to a toucan2 bug. This bumps toucan2 to fix the bug and
adds a test to verify that this case now works.
Add ref schema caching
Introduces a custom :ref IntoSchema that permits
reuse of Schema objects for references when safe to do so.
Early tests indicate a dramatic reduction in heap retention, after
loading a single dashboard, I see a 10x resident heap reduction from ~300MB to ~30MB.
It should also act as knot tie / runtime combinatorics limiter, so we should not see 'leaky' growth at runtime.
More work could be done as there are still _some_ gaps here.
- Bare pointers (keywords) can still lead to exponential allocation
- We do not cache when :documentation is present on the ref
(as it would specialise the Schema object delivered for that specific reference)
* Desrtucture all this.props
* Convert to function component
* Remove the wrapper
* Forward ref in topbar
* Limit resize height
* Pass availableHeight to native query editor
* Move var to constants
* Add more padding
* Rename viewHeight to availableHeight
* Handle unset availableHeight
* Add availableHeight to QueryEditor
* Calculate availble height above the fold
* Resize editor when available size shrinks
* Remove layer of nesting
* Rename calc to get
* Pass availableHeight to sql editor in modal
* Remove resizableBoxProps prop
* Do not resize when closing editor
* Use runQuery from props when availabe
* Sort props and remove unused props
* Clean up setIsNativeEditorOpen
* Remove dispatch from NativeQueryEditor completely
* Move state change out of useEffect
* Pull out resizable box into separate component
* Simplify availableHeight prop
* Move resizable logic into separate component
* Clean up code
* Handle availableHeight being 0 initially
* Make minHeight take preference
* Avoid setting height to Infinity
* Add repro for metabase#59075
* Fix e2e test
* Do not use default export
* Add type to NativeQueryEditor
* Remove unused databases loader
* Remove unused ExplicitSize wrapper
* Remove enity loader wrappers
* Flatten props
* Fix setDatabaseQuery type
* Fix types
* Add className to NativeQueryEditor
* Fix types
* Pull out handler
* Manually update metadata when updating snippet
* Preserve sidebar state if need be
* Fix e2e test
* Fix mock exports
* Fix e2e test
* Do not overwrite availableHeight
* Remove viewHeight prop
* Increase limit
* Fix broken test
* Remove unneeded useHeightAboveFold helper
* Fix Horror Show fingerprint OOMs
This may likely fix many OOMs that others are seeing. Anyone with large
schema will likely benefit from this.
```clojure
(let [database (toucan2.core/select-one :model/Database :id 2)]
(metabase.sync.analyze.fingerprint/fingerprint-fields-for-db!
database (fn [s t] (println "synced " s "for " (:name t)))))
```
This is how we fingerprint fields. DB 2 here is "Horror Show", a
database with 200 tables, each of which has 900 fields in it. This
database has zero rows in it, it it just a large schema.
Syncing this recently has caused huge cpu spikes and then OOM errors
until the instance is killed.
Profiling locally with
```
MB_JETTY_PORT=3091 java "$(socket-repl 6091)" -Xmx2g -jar metabase.jar
```
and visual vm hooked up, I run
```clojure
(let [database (toucan2.core/select-one :model/Database :id 2)]
(metabase.sync.analyze.fingerprint/fingerprint-fields-for-db!
database (fn [s t] (println "synced " s "for " (:name t)))))
```
And observe this behavior:
The ram usage approaches the 2gb limit and then bounces there. The CPU
begins to spike erratically and spend 10% of its time in GC.
Taking a heap dump we can see that we have 2.8 million ResultSetMetaData
objects occuping 183 Mb.
The ResultSetMetaData are closed over and kept in the metadata of the
snake hating maps stored in the cached metadata provider.
This brings into stark relief one simple way to fix this: don't use a
caching metadata provider. This PR just raising the body of the
fingerprint-tables-for-db function to not include
`(qp.store/with-metadata-provider (u/the-id database)` . There is no
benefit to caching here as this is an enumeration of the database. It is
by design not meant to reuse anything. If we didn't OOM we would end up
with 200 tables and 200*900 = 180,000 fields in the cache, each of which
was accessed only once.
That is what this PR fixes.
I also followed one other thread: why are we holding so many
ResultSetMetaData objects in memory. This seems to come about from a
change in https://github.com/metabase/metabase/pull/62330 specifically
this line:
https://github.com/metabase/metabase/blob/master/src/metabase/lib_be/metadata/jvm.clj#L63
```clojure
(mu/defn instance->metadata
"Convert a (presumably) Toucan 2 instance of an application database model with `snake_case` keys to a MLv2 style
metadata instance with `:lib/type` and `kebab-case` keys."
[instance :- :map
metadata-type :- :keyword]
(let [normalize (if-let [schema (get metadata-type->schema metadata-type)]
(fn [instance]
(lib.normalize/normalize schema instance))
identity)]
(-> instance
(perf/update-keys memoized-kebab-key)
(assoc :lib/type metadata-type)
normalize
u.snake-hating-map/snake-hating-map
;; the line below here captures the result coming back from
;; toucan and has a closure over
;; toucan2.jdbc.read/make-column-thunk which has a closure over
;; (.getMetaData rset)
(vary-meta assoc :metabase/toucan-instance instance))))
```
As it stands without any changes, the instance gets quite unstable and
falls over. If you comment out the `var-meta` form above, the graph
returns to a very normal sawtooth graph of memory usage and
reclaimation. But central to this is that it does not need to retain any
of this memory at all. But without this metadata containing the instance
and db classes, we only cache the table and field data and the
fingerprinting can finish. When we cache this stuff and also
inadvertently keep these db classes, we quite quickly run out of
headroom and the instance becomes unstable and falls over.
* fix kondo lint (removes a legacy usage as well :) 💰
* bit more cleanup
kept having a few flakes, couldn't consistently get it to pass 45 times
in a row locally. So just simplify it. Don't worry about "in process"
stats. Trying to get the webserver paused in the middle of a request was
occasionally deadlocking, often flaking.
So just calculate the simple ones and it's fine
```clojure
statistics-handler-test=> (clojure.test/run-test server-stats-test)
Testing metabase.server.statistics-handler-test
Ran 1 tests containing 13 assertions.
0 failures, 0 errors.
{:test 1, :pass 13, :fail 0, :error 0, :type :summary}
statistics-handler-test=> (dotimes [_ 100](server-stats-test))
nil
```
Also got rid of the macro that really only existed to collect the forms
into an anonymous function. Everything in-lined and now don't need a
kondo hook, etc
* remove unused alias
* kondo the refers
* reintroduce in-flight metrics
these:
```
(is (prometheus-test/approx= 1 (value! :jetty/requests-active)))
(is (prometheus-test/approx= 1 (value! :jetty/dispatched-active)))
```
Require a mechanism that can pause a request while active, verify that
the requests-active and dispatched active are incremented, and then
carry on. This is all kinda trivial except getting it wrong can
deadlock. Here are 500 successful runs of this test:
```
statistics-handler-test=> (dotimes [_ 100] (server-stats-test))
nil
statistics-handler-test=> (dotimes [_ 100] (server-stats-test))
nil
statistics-handler-test=> (dotimes [_ 100] (server-stats-test))
nil
statistics-handler-test=> (dotimes [_ 100] (server-stats-test))
nil
statistics-handler-test=> (dotimes [_ 100] (server-stats-test))
nil
```
Filter extract-columns endpoint to only return numeric and temporal columns
The extract-columns-from-query function is used to populate the incremental
filter dropdown for native SQL transforms. Previously it returned all columns
without filtering on type, leading to undefined behaviour when unsupported
column types were selected.
This change filters the results to only include columns with types supported
for incremental checkpointing:
- Temporal types (timestamp, timestamp with timezone)
- Numeric types (integer, float, decimal)
Text, boolean, and other unsupported types are now filtered out, matching
the behaviour of MBQL and Python transforms which already filter columns
client-side.
Updated documentation to clarify this endpoint is specifically for incremental
transform checkpoint column selection.
Also validates that transform POST/PUT conform to this rule and you are unable to update the transform to have
a filter column of an unsupported type.
* feat: add owner fields
* feat: hydrate owner property on transforms
* feat: extend `start-run` function to accept `user_id` and use it on scheduled transform runs
* feat: implement 'Ownership' section in transform settings page
* test: implement unit specs
* test: add e2e spec
* feat: show owner avatar in transforms list
* Minor clojure improvements
* test: simplify & extend e2e spec
* refactor: simplify avatar usage & change unassigned copy to "No owner"
* refactor: de-duplicate success & error messages
* test: add missing serialization test transform attributes
* test: fix e2e spec by scrolling into view first
* refactor: attribute manual runs to current user rather than owner
* refactor: cleanup
* tests: add missing user id parameter for jobs unit spec
---------
Co-authored-by: Nathan Voxland <nathan@metabase.com>
Ensure client-full-response uses future so that interrupts are isolated from caller
An intentional server side .interrupt might leak and cause subsequent
code to fail.
Prior to https://github.com/metabase/metabase/pull/65872 the
client-full-response used a with-timeout, which incidentally put the
http handler in a future (isolating the interrupt).
This commit restores the previous behaviour, in a more intentional way.
Update llms.txt and llms-full.txt (master)
Regenerated llms.txt to include all markdown files from the docs/ directory.
Also generated llms-full.txt files for the embedding section.
🤖 Generated automatically by llms-txt-generate workflow
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: GitHub Actions <actions@github.com>
* fix(uxw-2711): dependency graph corner radius
* fix(uxw-2706): give snippets a default name
* feat(uxw-2750): use actual link elements for transform list page
* feat(uxw-2750): use actual links for library section layout items
* fix(uxw-2751): prevent transform cancel handler from wiping changes
* .
* .
* .
* edit is actually working
* add now works too
* fix
* added e2e test
* doc
* .
* added a wait to see if that helps
* better waits
* .
* addressed comments
* Apply suggestions from code review
Co-authored-by: Alex Yarosh <alexandra@metabase.com>
* test
* test
---------
Co-authored-by: Alex Yarosh <alexandra@metabase.com>
* Add test helper definition and implement joins
* Add support for fields clause
* Be explicit with types
* Pull out findJoinStrategy helper
* Pull out appenTestQueryStage
* Simplify helpers
* Move helper
* Allow disamiguating column name
* Fix rhs columns
* pull out createTestJoinClause
* Add filters
* Add aggregations
* Add breakouts
* Add expressions
* Add ordering
* Add limiting rows
* Make source optional
* Make provider the first argument
* Expand example in tests
* Automatically pick the base column if no table was specified
* Remove unused sourceMetadata argument where not needed
* Fix join metadata
* Improve types
* Add createNativeQuery and createNativeJsQuery helpers
* Use column groups to disambiguate instead of table name
* Test the results
* Deprecate createQuery
* Replace createQueryWithClauses with createTestQuery
* Add SAMPLE_PROVIDER
* Add helpers to build metadata provider
* Use displayName for groupName
* Remove raw mbql in test and replace it with Lib inspections
This introduces a pre-commit hook that automatically removes unused require
statements detected by clj-kondo, preventing CI failures.
Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
`command` is a shell builtin that doesn't exist as a standalone binary on
Linux (only macOS has /usr/bin/command as a wrapper). When babashka tries
to execute `command -v git` directly via process/shell, it fails on Linux
because there's no `command` binary to run.
Using `which` instead works on both platforms since it's an actual binary.
* feat: add generic OIDC provider infrastructure
Add OSS infrastructure for OIDC-based authentication providers:
- OIDC discovery: automatic endpoint discovery from issuer URL
- Token handling: ID token validation, parsing, and claim extraction
- State management: encrypted state cookies for CSRF protection
- Schema validation: Malli schemas for OIDC configuration and tokens
- Common utilities: shared HTTP and request utilities
This provides a foundation for enterprise SSO integrations that use
OIDC/OAuth2 flows (e.g., Slack Connect, generic OIDC providers).
* fixup! dead code clean up
There was a bunch of speculative code we didn't end up using when this
change from a 'configurable OIDC' to a just support slack course
* fixup! ttls for caches for jwks/discovery
fixup! check for open redirect when creating oidc state
* fixup! use valid-hosts? to check before making jwks/discovery requests
* fixup! don't use hardcoded jwt algo
* fixup! use java-time
* fixup! kondo
* fixup! use system.core not system.settings
* Update IAM authentication details for RDS instances
Clarified the types of RDS instances that support IAM authentication.
* Update docs/databases/connections/aws-rds.md
* Update docs/databases/connections/aws-rds.md
Previously, when running a transform job (e.g., daily), it would fail if a single transform in the run failed. This PR makes it so that the job continues to run transforms that don't depend on any failed transform. As previously, if any transform in the run fails, the job is marked as failed.
Addresses #67704.
When a UNIX timestamp is accessed through an implicit join, the coercion to DateTime wasn't being applied. The `allow-casting?` check requires `pos-int?` on `::source-table` to apply coercion. For implicit joins, `::source-table` is set to a string, so it was being skipped.
Fix: in `add-alias-info/add-source-to-field-ref`, wet the flag `:qp/allow-coercion-for-columns-without-integer-qp.add.source-table`. I cribbed this approach from the SparkSQL driver, when it has to rewrite a numeric ID with a string.
* feat: configure initial read-only mode for python transforms
* refactor: hide UI elements in readonly mode
* refactor: move database selector to the topbar
* style: add in/out animations for bottom panel
* refactor: move common library controls to the topbar
* test: add unit specs for python transform editor sections
* test: add e2e spec
* fix: show correct run query button state after edits
* refactor: general cleanup & test trimming
* refactor: remove source controls
* refactor: trim down test mocks
* fix: pass missing `editable` flag to transform editor
* fix: types
* fix: e2e data source selector
* refactor: use `isEditMode` instead of `readOnly`
* adding segment editor to scalar
* temp
* adding tooltip
* pretend i'm a clojure dev, disable tooltip when there are no segments
* adding unit test and story
* loki tests
* silly me
* I am a clojure god
* adjustments
* adding docstring, adjusting loki test
* combining scalar loki tests
* CardSegment -> ScalarSegment
* BE update, filter unvalid segments
* Require free text for ui-bug and other feedback types
Add Yup validation to MetabotFeedbackModal that requires free text feedback when issue type is ui-bug or other. Issue types requiring free text are maintained in a constant array for easy extension.
* Address PR feedback: improve type safety and UX in feedback modal
- Replace 'as any' with proper TypeScript type assertion
- Fix type inconsistency between schema and initial value for issue_type
- Add dynamic label showing '(required)' when ui-bug or other is selected
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
* pr feedback
---------
Co-authored-by: Claude Sonnet 4.5 <noreply@anthropic.com>
Co-authored-by: Sloan Sparger <sloansparger@gmail.com>
* initial work
* fix test
* fix another test
* source entity malli schema
* clean up and add test
* add tests
* clean up
* fix tests
* drop finding_details column
* fix analysis version
* Update enterprise/backend/src/metabase_enterprise/dependencies/models/analysis_finding.clj
Co-authored-by: Timothy S. Dean <timothy.dean@metabase.com>
* use transaction
* use node-errors refactor suggestion
* use maybe-resolve-join
* add schemas
* add comment
---------
Co-authored-by: Timothy S. Dean <timothy.dean@metabase.com>
This is the permission we're correctly checking with the below
`api/check-403` but by using `warehouses/get-database` (which, btw, has
the MOST confusing possible arguments I've ever seen) we were throwing a
403 if we didn't have read access to the whole database (which
checks *create-queries* perms on the entire database).
We still need to exclude mirror databases. In the medium term there
might be a better way to do this - maybe we could somehow have two
different models, so `(t2/select :model/Database)` would always exclude
mirror databases, while `(t2/select
:model/DatabaseIncludingMirrorDatabase)` would return them? I dunno.
Native sql validation is a bit too unreliable for general deployment at the moment, so we are disabling it for check_* endpoints and the broken entities list unless/until we can make it more reliable.
Co-authored-by: Alexander Polyankin <alexander.polyankin@metabase.com>
Treats 0 n_live_tup as NULL, due to confusion reported when auto analyze
thresholds have not been met on a table.
To justify:
- You have a non-zero estimate (No change)
- A zero estimate but actually have rows (We now hide the estimate)
- A zero estimate and actually have zero rows (We now hide the estimate,
unfortunately)
This behaviour is considered on balance preferable to the behaviour now
where users will likely be confused, even thought we explicitly label
the field 'estimated'.
Other options considered that we might explore in the future:
- Forcing ANALYZE, e.g. on sync
- Bounded/LIMIT count
- Some policy based on last analyze timestamps
- Better tooltips or explaination on frontend
We should revisit in the future once we have a better idea for how
estimates should work across drivers (as its only postgres for now).
When a user creates a question based on a model, `source_card_id` was set to the model's ID. If that question was later converted to native SQL, `source_card_id` wouldn't get cleared. This means that deleting the model would cause the question to be deleted even though they no longer have any relationship to one another.
The code fix won't help anyone whose data is already incorrect, so this adds a migration to correct the inconsistent data. I added an assertion to the t2 hook to catch this at dev time in the future. And ultimately, the delete cascade seems overkill, so there's a migration to replace that with SET NULL instead.
* Implement read-only on Library page
* Add some gorgeous unit tests
* Make segment and measures read-only / Add tests
* Apply read-only logic to measure list and segment list pages
A recent change to search permissions queries in [#67055](https://github.com/metabase/metabase/pull/67055) will cause H2 to use SYS locks during search operations. This is because H2 uses temporary tables to implement CTE's, and their creation/dropping requires global locks.
This commit removes some ^:parallel meta in an attempt to avoid test
flakes due to lock timeout.
This change is a bit speculative:
- It is not 100% confirmed that parallel tests are the only problem (e.g. some background task might also hold a lock)
- The tests changed are based on a cursory scan for search tests that
hold transactions (mt/with-temp). There may be other tests.
The impact of the CTE lock behaviour in production searches is likely
minimal because:
a. Search queries will typically run in auto commit sessions and so lock
lifetime will be short
b. concurrent multi-user deployments are most likely not on H2.
Should keep an eye on both tests and any reports in production of lock
timeout errors / slow searches when using H2.
* Why is it a warning when it should be an error?
* Fix error logging format in notification worker
* Improve error logging in notification send process
Fix silly mistakes that I do when I'm sleepy
When a card is updated, look for all mbql child cards and recalculate their result metadata. With models, calculate
what their "correct" result metadata is pre-update, use that to calculate user overrides, and then reapply those
overrides when computing the new result metadata.
This should fix a variety of bugs around users updating a model and then not seeing those change downstream.
- Add max-height and vertical scrolling to input area
- Fix newline preservation in multi-line messages
- Add horizontal scrolling for wide code blocks in messages
- Style code blocks with proper background, border, and custom scrollbar
- Reduce code font size to 0.6875rem for better proportion
- Increase user message bubble width to 90%
- Add inline code styling with background and border
* Add /has-remote-changes endpoint to remote-sync API
Adds a new endpoint GET /api/ee/remote-sync/has-remote-changes that checks
if there are changes on the remote (Git) repository that can be pulled.
Response format:
{
"has_changes": true,
"remote_version": "abc123...",
"local_version": "789xyz...",
"cached": false
}
* Disable git pull button when there are no changes to pull
* Fix unit test
* Fix e2e
* Adjust conflict modal 'push' option to always force
* Fix conflict resolution options when switching branches
* Fix modal title not wrapping properly
* Fix long long modal title e2e issue
---------
Co-authored-by: Edward Paget <ed.paget@gmail.com>
* Add /has-remote-changes endpoint to remote-sync API
Adds a new endpoint GET /api/ee/remote-sync/has-remote-changes that checks
if there are changes on the remote (Git) repository that can be pulled.
Response format:
{
"has_changes": true,
"remote_version": "abc123...",
"local_version": "789xyz...",
"cached": false
}
* Update api_test.clj
* Increase verbosity in the caching middleware
* Fix linter
* Make it simpler
* Broke so many tests
* Make the linter happy
* Fix tests (going back to old implementation)
Fully support remote-sync for data studio and make it easier to extend support for remote-sync to other models in the future.
Supports serializing tables with their fields and segments via remote sync if they are published to a remote-synced collection.
Supports dirty state tracking of tables, fields, and segments.
Refetches dirty state when tables, fields, and segments are created/updated/deleted in the data studio
Make sure the data studio still displays content when remote-sync is in read-only mode
enforce that tables, fields, and segments published to remote-synced collections are not editable when remote-sync is in read-only mode.
Deletes, tables, fields, and segments from the remote-sync repo when they are removed or archived.
This also creates two small abstractions to make it easier to add new models (like measures) to remote-sync when they are ready. The remote-sync.events models adds a defmodel-change-handler macro that setups handlers for -create, -update, -delete, events models can publish that the dirty state tracking will follow.
Right now the UI does not change when the data studio is in read-only mode, but all create/edit operations correctly return 403 if you try to modify things. I have a follow up branch to support disabling UI elements.
* feat: add dirty tracking for model changes
* feat: add table, field, segement to dirty state query
* feat: extract fields with tables and segments
when exporting a collection with a table we want to include its children
like fields and segments
* feat: allow importing/deleting tables
and segments and fields
* feat: add dirty tracking for field/segment
* feat: show path in git sync controls
* fix: skip archived segments in table descendants
* fix: view library when in read-only remote-sync
Allows users to still see library folder they can't edit when that
condition is caused by the library being remote-sync enabled and the
instance being in remote-sync mode.
* feat: use remote-sync status in table edit perms
* fix: don't use include_editable_data_model param
...in data studio so these will still load when remote sync is read-only.
* fix: always show dirty library state on removed
We can't track what collections a table was removed from so we just want
to show the library as dirty if any item has a 'removed' dirty state.
* chore: update modules config
* chore: fix kondos
* fix: only publish table update events on ee
otherwise a handler doesn't exist which causes them to throw
* fixup! whitespace
* refactor: make the table changes rendering simpler
* fixup! missing when
* fixup! tests shouldn't mock modules
* fix: make dirty tracking work with new implementation
* refactor: make the hydration step in events cleaner
* fix: make sure dirty returns lowercase
* fix: delete segments correctly
* fixup! remove extra module
* fixup! correct migration order
* fixup! FE lints and typecheck
* fixup! remove correct extraneous module
* Update enterprise/frontend/src/metabase-enterprise/remote_sync/components/ChangesLists/EntityLink.tsx
Co-authored-by: Anderson <ands.alves.nunes@gmail.com>
* fixup! PR feedback and tsc
* fixup! segment failing to redirect
* fixup! better fix for navigate after save
---------
Co-authored-by: Anderson <ands.alves.nunes@gmail.com>
For the data warehouses we can safely set a fixed
unreturnedConnectionTimeout to a fairly low number (5 minute default)
because we also have a default maximum query length for data warehouses,
which can't be exceeded anywhere.
For the application DB though, this isn't safe, as operations against it
can legitimately hold connections for extended periods - for example,
dump-to-h2 on large databases streams data for as long as it takes to
copy all rows.
Bumping this to 1 hour by default (configurable via
`MB_APPLICATION_DB_UNRETURNED_CONNECTION_TIMEOUT`).
* simpler tests? you decide
* ignore discouraged var. just tests. don't need api docs
* avec-server -> with-server
originally there was another `with-server` and i didn't want to clobber
it until i was ready
* bit of cleanup
* cljfmt
* bit of linter appeasement
* Support measures and metrics for Metabot
* Add segment support to metric querying
* Minor clojure tweaks for metabot measures and segments poc (#67425)
* Handle measures and segments like metrics in table-details and card-details
Move these into the existing m/assoc-some and handle them in the same way as metrics
* Remove query->output-format
This was just a wrapper around lib/->pMBQL, but all these queries are already MBQL5 queries
* Metabot measures and segments cleanup and tests (#67488)
* some refactor
* draft some happy path tests
* use lib for creating measure and segment clauses
* fix tests
* fix some filter tests to use mbql5
* update more tests
* fix more tests
* fix model query syntax
* lint
* remove normalize call again
* add back normalize
* minor test cleanup
* address comments
* tweak another test
* add a query= helper that strips uuids, update a number of tests to use it
* add basic error handing for bad ids
* Minor cleanup
* address mike's comment
---------
Co-authored-by: appleby <86076+appleby@users.noreply.github.com>
Co-authored-by: Noah Moss <32746338+noahmoss@users.noreply.github.com>
Co-authored-by: Noah Moss <noahbmoss@gmail.com>
* fix: don't unset remote-sync settings on api req
...if request is partial. Fixes an issue where the token could be unset
when switching between read-write and read-only modes.
* add debugging
* remove debugging
* fixup! handle extra collection
* fixup! update syntax
Update llms.txt and llms-full.txt (master)
Regenerated llms.txt to include all markdown files from the docs/ directory.
Also generated llms-full.txt files for the embedding section.
🤖 Generated automatically by llms-txt-generate workflow
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: GitHub Actions <actions@github.com>
* Fix: In Safari, Reducing the Tile Size of a Number viz Might Result in the Number Being truncated like a String Rather than Rounded
Fixes#51082🤖 Generated with [Claude Code](https://claude.ai/code)
Co-Authored-By: Claude <noreply@anthropic.com>
* UXW-46 refactor claude solution, implement proper fix
* Update Loki Snapshots
* Fix e2e tests
---------
Co-authored-by: Claude Code[bot] <noreply@anthropic.com>
Co-authored-by: Denis Berezin <denis.berezin@metabase.com>
Co-authored-by: Metabase Automation <github-automation@metabase.com>
* feat: split can-read from can-query in data model
Adds a new kind of permission type `can-query` that describes when a
user is allowed to query the data model vs `can-read` which encompasses
when a user can see the metadata of a query. This makes api calls to
read table metadata more straightforward and the eventual integration
with remote sync easier.
* feat: update api to use can-edit/can-query params
* fixup! allow can-write-metadata queries
* fixup! field test
* fixup! whitespace
* fixup! returning empty lists instead of 403
* fixup! schema response
* fixup! enforce query permissions where necessary
* fixup! add last tests
* fixup! can query filter before saved questions
* use ActionIcon for data studio toggle icon
* Use ActionIcon on snippet tree row ellipsis menu
* Change link colors in breadcrumbs on hover
* Use ActionIcon for tree expand icon
* Update hover state on transforms runs filters
* Fix data studio nav collapsed logo hover
* Match hover state on snippets collection and snippets folders
* Fix ts
* fix: remote-sync origin url errors
Fixes issues where the tmp directory for the remote-sync repo can get
into a bad state where it doesn't have an origin URL. This calls a
function that explicitly sets our desired origin URL and updates all
calls to use the more specific :remote-url keyword rather than
inconsistently using :url vs :remote-url.
* Update git_test.clj
* Extract cycle detection algo and fix available-measures
* Rremove duplicated code.
* Add more serialization tests
* Fix CLJS
* Remove MBQL4 support for measures at the model level
We can't hydrate unbounded collections of cards with the
can_run_adhoc_query hydration since it linearly adds db queries to the
request. Users with > 10000k can have severe CPU/memory issues
processing this.
* Fix heading level for multi-tenant strategy section
* Correct 'Admin setting' to 'Admin settings'
Updated references from 'Admin setting' to 'Admin settings' for consistency throughout the document.
* more edits
* typos
* feat: include query type on dependencies api response
* feat: show icons based on query type
* test: add unit spec for dependency graph utils
* refactor: restore changes after merge conflict
Fixes#67667.
`lib.equality` has logic to do "fuzzy" matching of field refs with
the corresponding columns. This logic exists to support somewhat
malformed older queries with impoverished field refs. Refs in healthy
recent queries are fully specified and there's nothing ambiguous to
required the fuzzy matching.
The bug #67667 caught a case where this fuzzy matching could find a
column with the same name on the RHS of a join, if the original target
column has disappeared from an upstream query.
This change causes the QP to fail in exactly this case: when a would-be
LHS ref in a join condition ends up pointing at the RHS of the same
join. That would produce a strange join condition like
`LEFT JOIN Orders ON Orders.ID = Orders.PRODUCT_ID`
which produces incoherent results, and could result in a very costly
Cartesian join.
This is a user-level error, since the query is broken and needs repair.
Basically, the issue is that trying to implicit join using the instance analytics content model's made_public_by_user field made qp throw an error. The underlying problem is that the field in question wasn't defined as a foreign key, even though the model's result metadata claimed it was a foreign key. Updating the field to actually be a foreign key fixed the problem. This also applies to the content model's question_database_id field, which was also fixed.
* use source alias if we have a fk-field-id
* try only on single stages
* try to identify implicit joins
* try fix
* Clean up
* move test and test all drivers
* move test
* only test drivers with left join feature
* adjust length for certain drivers
* smaller for mysql
* fix starts-with filter
* improve test
* case insensitive filter for h2
* move let bindings up
* Use instance settings for date filters in parameters
* Fix test instance to use default time format
* Add unit tests for date paramater formatting with custom settings
* let users with data model permission access data studio
* respect manage table metadata permission in data studio
* fix ci
* fix flaky specs
* review
* review
* Add e2e test for missing field mapping
* Make validateTemplateTags public
* Render errors in run button tooltip
* Pass errors to run button
* Fix selector in test
* Simplify test
* Try with realClick
* Add Lib implementation of validateTemplateTags
* Also validate display-name
* Use Lib implementation of validateTemplateTags
* Localize error messages
* fixup! Add Lib implementation of validateTemplateTags
* Add Lib.ValidationError type
* Remove unused code
* Also validate temporal-units dimension
* Use validate-template-tags in can-run
* Add tag-name to template tag errors
* Fix malli types
* Add tests for validate-template-tags
* Add tooltip to save button too
* Also test time grouping and save button
* Remove more unused code
* Remove simple wrapper
* Use cond-> pattern instead of filter identity
* Also test coerced datetime column
* Add e2e reproduction for opening the datetime filter for coerced fields
* Also check :effective-type in original-isa
* Fix test
* init pass at adding an error boundary to metabot
* update the illustration used for error boundary and remove simulated error ui
* add unit tests
* split up the mega metabot.unit.spec.tsx unit test file
* fix lint
Previously if you had:
```
A (curate)
B (curate)
C (no permissions)
D (curate)
```
Then if C was archived, you would be unable to move or archive A or
B.
* Fix `text-secondary` not working on dark theme on charts
* Support all colors so it's easier to reason about
* Add tests
* Fix type error
* Revert part of the code that's not used anymore
* Add test snapshots
* Make overriding font more consistency across environments
e.g. From storybook to SDK to the main app.
* Fix formatting
* Skips cloud driver tests on PRs unless files changed
Cloud drivers (Athena, BigQuery, Databricks, Redshift, Snowflake) now
only run on PRs when their files changed or 'ci:all-cloud-drivers' label
is present. Full coverage maintained on master/release branches.
* [do not merge] add failing databricks test
* Revert "[do not merge] add failing databricks test"
This reverts commit de5b6fa602.
* empty commit
* Fix cloud driver skip logic to respect ci:all-cloud-drivers label
The previous condition had three checks that could block cloud drivers:
1. !inputs.skip - false when no backend files changed
2. skip != 'true' - from mage can-skip-driver-tests
3. X-should-run == 'true' - the per-driver decision
The problem: checks 1 and 2 would override check 3, so the label had no
effect. Even with ci:all-cloud-drivers, cloud drivers were skipped.
We don't need inputs.skip or can-skip-driver-tests for cloud drivers
because the X-should-run output already encapsulates the full decision:
- master/release branch → run
- ci:all-cloud-drivers label → run
- driver files changed → run
- otherwise → skip
The label is an explicit override, so it should work even on PRs with no
backend changes. Removing the redundant checks lets the label work.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
* mage -driver-decisions + roll in module analysis
* Remove quarantine check from test-driver action
Quarantine logic is now consolidated in mage -driver-decisions.
The check happens earlier in determine-driver-skips, so quarantined
drivers get should-run=false and their jobs never start at all.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
* Fail CI if quarantined driver has file changes without break-quarantine label
If you change files in a quarantined driver's module (e.g., modules/drivers/snowflake/)
but don't add the break-quarantine-<driver> label, a new check-quarantine-conflicts
job will fail with a clear error message.
This prevents accidentally merging changes to quarantined drivers without running
their tests.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
* Add --github-output-only flag to mage driver-decisions
Instead of using grep to filter mage output, the task now supports two modes:
- Default: prints human-readable analysis + GITHUB_OUTPUT preview
- --github-output-only: prints only clean key=value lines for CI
The workflow now calls mage twice: once for debugging visibility,
once with --github-output-only redirected to $GITHUB_OUTPUT.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
* Add quarantine conflict checks to all driver jobs
Extends the quarantine conflict detection to ALL drivers, not just cloud
drivers. When a driver is quarantined but has file changes in
modules/drivers/<driver>/, the job will run but immediately fail with
a clear error message explaining how to add the break-quarantine label.
Changes:
- Add quarantine conflict check steps to presto-jdbc, sparksql, sqlite,
sqlserver, and vertica jobs
- Update login-to-ecr job condition for vertica quarantine conflicts
- Fix verbose output suppression in --github-output-only mode by adding
verbose? parameter to changes-important-file-for-drivers?
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
* Add driver decision priority documentation and tests
Adds comprehensive documentation to driver-decision function showing the
priority order as a table, plus explanations of what triggers "driver
module affected" and the difference between cloud vs self-hosted drivers.
Adds mage.modules-test with 14 tests (78 assertions) covering:
- Quarantine logic and break-quarantine labels
- Global skip behavior
- H2 always-runs behavior
- Master/release branch behavior
- Driver module affected behavior
- Cloud driver special rules (labels, file changes)
- Self-hosted driver skip logic
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
* Comment out starburst from driver-directory mapping
Starburst tests are currently disabled in drivers.yml, so the mapping
entry was inconsistent with all-drivers list. Commented out with a note
explaining why, so it can be easily re-enabled when starburst tests
come back.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
* silence messages when in github-output-only mode
* remove redundant cloud-driver change detection
- this happens inside the script instead
* get rid of redundant rules definitions
- now only print how we make decisions in help for the task
* make a temporary redshift change
* Revert "make a temporary redshift change"
This reverts commit 3d93c60368.
* format usage fn code
* light refactor
* Rename driver decision variables for clarity and flatten decision tree
- driver-module-affected? → driver-deps-affected? (module dependency graph)
- drivers-changed → particular-driver-changed? (specific driver's files)
- Reorder priority: skip/H2/master now before quarantine
- Flatten nested cond for cloud driver rules
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
* Quarantine now respected on master/release branches
- Move quarantine check before master/release in driver-decision priority
- Quarantined drivers are now skipped even on master/release (unless break-quarantine label)
- Update tests and priority table documentation to reflect new order
- Cloud drivers don't benefit from driver-deps-affected (only label or file changes)
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
* Fix parsing of :ERROR chunks in metabase-enterprise.metabot-v3.util/aisdk->messages
There was a typo in the case expr where :ERROR chunks were being handled like :DATA chunks, but in fact should be
handled like :TEXT chunks. This was resulting in an error like
1. Unhandled java.lang.ClassCastException
class java.lang.String cannot be cast to class
clojure.lang.Associative (java.lang.String is in module java.base
of loader 'bootstrap'; clojure.lang.Associative is in unnamed
module of loader 'app')
RT.java: 847 clojure.lang.RT/assoc
core.clj: 193 clojure.core/assoc
core.clj: 192 clojure.core/assoc
util.clj: 56 metabase-enterprise.metabot-v3.util/aisdk->messages/fn/fn
...
whenever an :ERROR chunk was encountered as we tried to ASSOC a string where we were expecting a map.
https://v4.ai-sdk.dev/docs/ai-sdk-ui/stream-protocol#text-parthttps://v4.ai-sdk.dev/docs/ai-sdk-ui/stream-protocol#error-part059d904d6e/ai_service/ai_sdk.py (L50)059d904d6e/ai_service/ai_sdk.py (L203)059d904d6e/ai_service/ai_sdk.py (L324)
* Add some error chunk parsing tests to aisdk-line-parse-test
* Mark aisdk-line-parse-test as ^:parallel
This does io to read some files, but should be threadsafe as no other tests read or modify those files.
* Add make-mock-error-stream-response test util
And rename make-mock-stream-response to make-mock-text-stream-response
* Add client-error-test to test streaming-request with error chunks in response
We have a grace periods and circuit breakers for token checks. But one
issue is that if the token check doesn't throw an error we can't catch
it.
An easy way to see this is to use a local network address that won't
respond like 192.168.random.random (i used 192.168.40.40) then see the
behavior. Without the timeouts, the threads all just get stuck here and
the page never loads. With timeouts, you'll see a few stutters until the
circuit breaker steps in, at which point you have a non-enterprise
version.
https://www.loom.com/share/6ae056dc126b48c9abe8a7ad9366fd05
* remove api "uses" dependency on api-routes
api-routes can use all modules to expose their routes. ie it uses
timeline module, etc. It has `:uses :any`.
```clojure
mage.modules=> (let [deps (dependencies)
all-deps (keys deps)]
(reduce (fn [c module]
(update c (if (skip-driver-tests? deps [module])
:skip :run) inc))
{:skip 0 :run 0}
all-deps))
{:skip 0, :run 153}
```
This is shown in this path here:
```clojure
mage.modules=> (let [ds (dependencies)
ds (into {}
(map (fn [[k v]] (if (= v :any)
[k (-> ds keys set)]
[k v])))
ds)]
(letfn [(driver-path [to]
(loop [q '[[driver]]
seen #{}]
(let [p (first q)]
(when-let [node (peek p)]
(if (= node to)
p
(recur (into (vec (rest q))
(for [adjacent (remove seen (ds node))]
(conj p adjacent)))
(conj seen node)))))))]
(driver-path 'enterprise/remote-sync)))
[driver warehouse-schema api api-routes enterprise/remote-sync]
```
driver uses arehouse-schema uses api uses api-routes which uses
any. Therefore all changes will trigger driver tests.
So after these changes we observe:
- the path above is empty (drivers don't transitively use remote-sync
module) and therefore would not trigger driver tests
- only 76 modules trigger driver tests
```clojure
mage.modules=> (let [deps (dependencies)
all-deps (keys deps)]
(reduce (fn [c module]
(update c (if (skip-driver-tests? deps [module])
:skip :run) inc))
{:skip 0 :run 0}
all-deps))
{:skip 77, :run 76}
```
What else did i try?
1. add a `#_{:clj-kondo/ignore [:metabase/modules]}` to the ns
I didn't love this because it felt a bit like cheating. All of this is
kinda annoying because it's fundamentally about changes during dev time.
2. remove the ns require and add a requiring-resolve
This actually doesn't break the module usage analysis. It's smart enough
to see this so this just makes it more tenuous and easy to miss if we
change anything while also still leaving in place the module
entanglement.
So I just fire the event from the api module when rebuilding routes and
handle the event from the api-routes module. The trick is that the api
module sends along the function to rebuild the docs
* handle cycling ref
* remove reference to deleted file
* our linter is scared of an extra line
* use an events namespace
not doing an init namespace as that's similar but not the same. Those
are for settings and models. It's fine to just require the event
namespace in the core namespace. it's a bit of ceremony but the pattern
is useful
* include random event correctly
used to be just `(first (descendants :metabase/event))`. This was
fragile and just now started to fail:
```clojure
models-test=> (first (descendants :metabase/event))
:metabase.api-routes.events/api-events
```
because of course do we things like `(derive ::api-events
:metabase/event)`.
```clojure
models-test=> (filter (comp (complement #{"event"}) namespace)
(descendants :metabase/event))
(:metabase.api-routes.events/api-events
:metabase.audit-app.events.audit-log/card-event
:metabase.audit-app.events.audit-log/cloud-add-on-event
:metabase-enterprise.dependencies.events/transform-delete
:metabase.activity-feed.events.recent-views/legacy-card-event
:metabase.view-log.events.view-log/collection-read-event
:metabase.audit-app.events.audit-log/notification-handler-event
:metabase.audit-app.events.audit-log/user-update-event
:metabase-enterprise.dependencies.events/transform-run
:metabase.revisions.events/transform-event
...
)
```
This has been lucky for a long time
* Update Remote Sync branch picker to match new designs
* Move git sync controls next data studio logo
* Fix unit test
* Fix e2e test
* Address Claude's review
* Remove console.error
* Use css modules for the chevron icon transition
* Use popover instead of findByRole('presentation')
* disable progress bar in whitespace linter github actions workflow
* add extra whitespace to demonstrate failure
* Revert "add extra whitespace to demonstrate failure"
This reverts commit ba34650efd.
* feat: add git settings modal to data studio
* Refactor GitSettingsModal to reuse existing settings form / Adjust some styles
* Fix and improve unit tests
* Adjust font size according to variant
* fix unit test
* fixup! mock API not other plugins in this layout
* Split unit test files and improve endpoints setup
* Implement PLUGIN_REMOTE_SYNC.GitSyncSetupMenuItem with internal rendering condition
---------
Co-authored-by: Anderson <anderson@metabase.com>
Fixes#65838.
This issue was about dashboard filters showing the wrong value when
the filter was bound to an FK field from a table with multiple FKs to
the same foreign table. The dashboard parameters logic on the BE was
choosing which FK to use for the filter values arbitrarily, meaning that
binding the filter to one of them would work fine, but the other would
give incoherent results.
The fix is to be deliberate about selecting the parameter's column. That
doesn't solve all ambiguous cases, but we really can't help if there's
a multi-hop join that has parallel edges in the middle!
* Add field grouping of host and port
* Add host and port settings to postgres, mysql
* Add host and port grouping to db drivers
* Add database story with host and port group
* Add groupFields unit tests
* Fix typo
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
* Fix missing tunnel parameter value
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
* Refactor and simplify field grouping function
* feat: add nested fields support for db format
* feat: remove id field, adapt new format foy postgres, mongo, mysql, remove leftovers
* feat: support nested fields, add backdrop component
* fix: group inputs loosing focus on type/render
* refactor: simplify custom containers logic
* style: adjust backdrop container style
* refactor: field mappers for db forms
* refactor: move host and port config to the new format
* refactor: cleanup old grouping approach
* refactor: remove unneeded property
* fix: typo in the html prop name
* internal: validate container style comping from BE
this value can be set be external drivers so it's better to validate it
Signed-off-by: Jakub Chodorowicz <jakub.chodorowicz@metabase.com>
* fix: type errors
* fix: driver translation test
* refator: use existing getFlattenedFields function
* fix: flatten properties for connection property uniquness
* backtrack from ssl grouping for mongo
---------
Signed-off-by: Jakub Chodorowicz <jakub.chodorowicz@metabase.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
* Acquire locks before changing data permissions
Data permission have a read-modify-write cycle:
- we read the existing permissions to see what's currently set
- then we decide what to do (e.g. delete the existing DB level
permissions to write a granular table-level permission)
- then we do the write
This has a race condition if multiple Metabase instances are modifying
permissions at the same time.
For example, say we have:
Initial state: `{:db_id 1 :table_id nil :perm_value :unrestricted}`
Instance A: set table 1 to :blocked
Instance B: set database to :blocked
T1: A reads state, sees db-level permission, plans to expand to table-level
T2: B reads state, sees db-level permission, plans to replace it
T3: A deletes db-level, inserts table-level rows for all tables
T4: B deletes db-level (no-op), inserts new db-level :blocked
Result (CORRUPT):
```
{:db_id 1 :table_id 1 :perm_value :blocked} ;; from A
{:db_id 1 :table_id 2 :perm_value :unrestricted} ;; from A
{:db_id 1 :table_id nil :perm_value :blocked} ;; from B
```
Both db-level AND table-level permissions now exist.
For the permissions API, we take a cluster lock, but sync operations
bypassed this cluster lock. This PR creates a lower-level cluster lock
per `[database, group, perm-type]` so only one instance/thread can do
the read/decide/write cycle for a particular db/group/perm at a time.
* Add a test for race conditions setting data perms
This took a while before I was able to track down the fact that using
`(mt/with-temp )` caused both threads to share the same connection, so
they could obtain the lock simultaneously 🤦
* Use a slightly coarser lock
I realized that if an instance has a huge number of groups, we'll obtain
a lock and ask Postgres to maintain an active savepoint for each group,
which might be untenable.
I think the best solution here is to use slightly coarser locks. This
might increase contention slightly, since we won't be able to change
permissions for `db 1, group 1, perm type view-data` at the same time
that we change permissions for `db 1, group 2, perm type view-data`. But
I think the tradeoff here is necessary.
* Prevent unnecessary height calculation
* Prioritize local layout from `react-grid-layout` for height calculation
* Refix the issue for real
* Add tests
* fix: hide untested and unsupported db engines in table editing section
* move allowed engines to constants file
* test: add hidden editing toggle test case
* fix: avoid persisting settings that are hidden from dashboard UI
* test: add `sanitizeDashcardSettings` unit spec
* refactor: use `_.pick` for sanitizing dashcard settings
* Fix pie chart flickering on toggling legend / Convert ChartWithLegend to ts
* Convert test to ts
* Resolve flickering when toggling visualization config sidebar
* Fix regression with map chart (only apply svg centralization to pie charts)
* Apply PieChartContainer classname to inner component
* Default to base-type when it differs from effective type
* Make 'er parallel
* Does this clause make it better?
* Remove parallel.
* New test and fix for two literals
* Use y-type for effective type
* Update test to try number
* Don't expose `question` in SDK's `entityTypes`
Because it's only applicable to staged picker
* Fix type
* Clarify the use of `question` in SDK's `entityTypes`
* Fix SDK not building
* Prevent test email from including Metabase links (#67068)
* Send a new flag from FE if testing subscription from modular embedding
* Pass :disable_links to dashboard subscription email template
* Remove links from dashboard subscription email template
* Remove link from pulse cards
* Remove dangling divider line
* Add test
* Fix tests
* Use the same logic as in the actual subscription tasks
* Prepare to use a new column name from pulse table
* Remove embedding subscriptions email links (#67181)
* Add a column to pulse to save when a subscription is created on embedding
* Save disable_links to "pulse" table
* Fix tests
* Fix more tests
* Update column name
* Add tests
* Make code less confusing after embedding name change
* Fix liquibase boolean type
* Fix tests
* Fix more tests
* Fix formatting
* Fix wrong test names
* Make function comment styles consistent
* Make the test more solid
* Add modular embedding SDK tests
* Don't render the dashboard subscription button when there's no data cards (#67501)
* fix: respect time format in exlude date picker filter QUE-2611
* rework: move constant out of function body
* test: fix failing tests due to lack of redux provider
* fix: add missing redux provider in story
* test: improve testing logic for ExcludeDatePicker
* internal: use specialized useSetting hook
* test: adjust looping with async action
The root cause of the flakiness was that the save modal sometimes picks the root collection, while at other times it picks the current user's personal collection. The selector was trying to find "Our analytics" in the breadcrumbs, which is why it was failing every time the actual collection is personal collection.
The second issue was that trying to (1) clear and then (2) type the metric name was extremely unreliable.
* Add failing test for #63822, DEV-1190
* Add new feature flag
* Update FE config
* Fix tests 🔧
* Disable for 7 more drivers
* Fix MySQL + MariaDB
* Fix TypeScript
* Check that the number was parsed correctly
* Don't parse int columns that are remapped
* oops, remove old change
* Better fix and specific test
* kick ci
* kick ci
* Make sure it still hits the bigint branch
* Move check into parseNumber()
* Move check back to value.tsx
* Single describe in tests
* Add blank lines (fe-lint)
Log each entity (card) as its dependencies get backfilled by the job. This is to be able to see if there is a particular entity that's causing the OOM.
* prototype
* open chat sidebar on navigate
* update copy for new route
* use nlq profile and make sticky, redirect to empty query builder if no navigate_to is recieved
* responsiveness
* move code to enterprise folder
* imple error states, better loading state
* handle user leaving the page while a response is being loaded
* lint fix, change inline sql shortcut key, disable prompt suggestions and fade out once a prompt has been sent
* fix e2e test
* add test coverage, update copy
* fix failing e2e tests
* attempt to fix e2e flake
* UXW-2449 fix pasting mixed content containing cardEmbed
* Fix issue with cardEmbed loading forever
* Fix issues with smartLink and caedEmbed paste to a list
* Fix typecheck
* Remove openapi.json from repo and disable CI check
The committed openapi.json was causing friction:
- Cascade CI failures when master got out of sync
- Merge conflicts on a 36k line generated file
- Phantom local changes when switching branches
The OpenAPI generation still works - run `yarn generate-openapi` when needed.
We're just not committing the artifact for now.
* Check for Node.js 22+ before generating OpenAPI types
The previous check for `yarn` wasn't sufficient - the system may have
yarn installed with an older Node.js version. Now we explicitly check
that Node.js 22+ is available before running the generation.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
* Revert client-side OpenAPI types generation
* Remove types generation from postinstall
* Restore package.json newline
* Restore prepare-backend action
---------
Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
Co-authored-by: Stas Gavrylov <stasgavrylov@gmail.com>
Co-authored-by: Stas Gavrylov <stas@metabase.com>
* Allow clicking dates with hour and minutes granularity to set parameter
* Add repro for metabase#58556
* Fix unit tests
* Add repro for metabase#66277
* Allow selecting date ranges with hours and minutes
* accept optional profile-id in search tool request and use it to determine collection filtering
* cleanup
* try to fix tests
* fix build
* kondo
* fix toucan syntax
* address comment
* oops, fix build
* remove some more use_case code on the FE that slipped through the revert via messy rebase
* set network timeout in connection options
* add test
* kondo
* fix test
* Add try catch
* use network timeout setting
* use exception chain util
* use cached thread pool, and timeout to call connection.close
* just do setnetworktimeout
* address comments
* Fix: Cannot add filters on boolean MySQL columns when the data source is a model
Fixes#65826🤖 Generated with [Claude Code](https://claude.ai/code)
Co-Authored-By: Claude <noreply@anthropic.com>
* Introduce multimethod
* Improve docstring
* Add db-type-name to changelog
* Make tests work
* improve changelog entry
---------
Co-authored-by: Claude Code[bot] <noreply@anthropic.com>
Co-authored-by: Eric Normand <ericwnormand@gmail.com>
* Fix: Redshift external tables have unknown types for string/enum columns on sync
Fixes#63025🤖 Generated with [Claude Code](https://claude.ai/code)
Co-Authored-By: Claude <noreply@anthropic.com>
* Add more Iceberg types to Redshift external table types
---------
Co-authored-by: Claude Code[bot] <noreply@anthropic.com>
Co-authored-by: Eric Normand <ericwnormand@gmail.com>
- when a spec is registered, store the raw form in an atom. This is so
things like `search.spec/explode-raw-case` can be stored as raw symbols
rather than functions. (Do note that this is technically a bug in a rare
edge case, where the definition of `search.spec/explode-raw-case`
changes and the search index should technically be invalidated - but I
think this is rare and low-impact enough that we can just not care.)
- when we access the current search index version, canonicalize the
stored specs, plus default attributes and attribute types, and sha256
hash the result
* UXW-2576 show tenant collections in the trash
I also deleted the `audit-namespace-clause` function. It felt like it
didn't do enough to justify its existence - every caller needed to
modify the list of namespaces. Could probably DRY things up a bit still,
but think we should change the approach.
* Check `use-tenants` before returning tenant colls
* s/audit-namespace-clause/namespace-clause helper
`namespace-clause` takes the ns column, ns value, and an optional
argument for whether *this* particular result should include the tenant
collections in the response (since sometimes we want 'em, sometimes we
don't.)
* kondo
* Prevent updating tenant specific root collections
* Localize the name of dedicated tenant collections
* rename dedicated tenant collections 'Our Data'
When displayed on the FE for an admin, they can be renamed to something
like "Foobar's tenant collection"
* Internal users see different names
* prevent tenant users to edit their dedicated tenant collection
* Revert "prevent tenant users to edit their dedicated tenant collection"
This reverts commit 08d2d722be.
* Rename `Our Data` to `Our data`
Co-authored-by: Nicolò Pretto <info@npretto.com>
---------
Co-authored-by: Nicolò Pretto <info@npretto.com>
* Demo of only importing the relevant plugin in unit test
* Optimize enterprise plugin unit tests with targeted setupEnterpriseOnlyPlugin calls
* fixed tests
* fixup
* .
* fixed test
* removed hasEnterprisePlugin
* fixup
* removed remaining hasEnterprisePlugins and unnecessary setupEnterprise fn
* specificPlugins -> enterprisePlugins
* .
* fixed type
* docs
* Add effective type to expression when resolving from ref
* Add repro for QUE-2567
* Fallback to base type when checking effective types
* Avoid shadowing var
* Add tests for effective-type fallback
* Fix normalization of MBQL 4 `:aggregation-options` with a `nil` options map
* Update tests
* Fix normalization of Template Tag :options
* Allow raw values as expression definitions
* Automatically remove :ident from joins
* Properly handle aggregations expressions that contain `:if` (alias for `:case`)
* Allow `:coalesce`, `:if`, and `:case` as datetime exprs in MBQL 4
* Test & Kondo fixes 🔧
* WIP
* Working test! (it reproduces)
* Fhew! I found it!
* Remove printlns
* Also remove sensitive fks in calculation
* Remove stray marker comment
* Move tests to fields-test
* Exclude :retired too
* Move tests and add one for retired
* Add another retired
* Better tests
* Appease the kondo
* Fix broken test
### Description
We want to communicate per-billing-period quantities with customer, instead of always communicate monthly quantities. Invoiced totals will stay the same.
### How to verify
Describe the steps to verify that the changes are working as expected.
1. Ensure you have a token with the `offer-metabase-ai-tiered` feature, but without the `metabot-v3` feature.
2. Navigate to `/admin/metabot` and see the quantities as per billing period, and not per month.
### Checklist
- [x] Tests have been added/updated to cover changes in this PR
Fixes: 2b76c38e1f
Closes: https://metaboat.slack.com/archives/C07RMUQ0GE5/p1765907134400039
* Update default value locations in comments
* Update snowplow schema for modular embedding analytics
* Add more type safety
* fix 'dashboard.drills' and 'question.drills' default values for guest embeds
* Fix auth type
* Test guest embed authentication
* Log `with-subscriptions`
* Strengthen the type
* Hide external collections sidebar for internal users without tenant collection accesses (#67033)
* show the add button if user can write to root
* update unit tests for shared collections nav
* simplify test mocks
* getIsTenantUser selector
* flatten navbar for tenant users
* move up state and use plugin, keep "Collections" name when the external
collection section is not shown
* polish
* throw new error
* fix unit test
---------
Co-authored-by: Phoomparin Mano <poom@metabase.com>
* add openapi-typescript schema generation
* chore: add live type generation
* Temp fix for self ref schema
* nemanja wants avaible locales schema
* generate openapi.json file on dev or when updating the apis
* now it e2e. you update an api schema => generate openapi.json => hook to update d.ts
* add hey api
* update user profiles
* proper types now
* fix: remove leftover folders and adjust imports
* chore: Claude guide to generate MALI schemas
* feature: add schema for users API
* wip add user type to profile
* remove the ref of the old lib. all hey-api now
* WIP
* Update FE unit mock
* fetch openapi.json from file system instead of the api
* Add in-browser overlay with errors
* disable error overlay
* feat: script to generate openapi spec
* experiments
* do not clean cwd
* Enable pretty-printing for OpenAPI json
* Add OpenAPI spec keys sorting
* Move generated openapi.json to resources/openapi
* Commit generated typings with sorted keys. Add types.get.ts to eslintignore because it has incorrectly linted typing
* wip
* Fix nullable types
* test fixes
* revert hack for self ref schemas as the new lib cna generate it
* Move TS generated types to gitignore. Cleanup.
* revert uncessary schema changes and also fix indentation
* Add types generation to TS CI job
* Update CI to generate and cache TS types
* Fix CI types generation
* Cleanup FE
* reorganized the modules and remove the requiring-resolve
* Fix unit test
* fix: wrong function call to regenerate api spec
* chore: upgrade hey-api (fixes self referencing hey_api type)
* feat: disable type generation for MetabaseLegacyMbqlSchema
* chore: regenerate openapi definition
* chore: regenerate openapi.json
* feat: add generate-openapi-spec command
* build: use generate-openapi-spec in yarn scripts
* build: add script that verifies the that openapi spec is uptodate
* build: add CI check for openapi schema being uptodate
* build: fix wrong filename of api check script
* internal: use yarn script for schema generation
* internal: generate version in the api schema generation step
* internal: install node 22 in the api generation job
* internal: fix hey api running twice on build hot
* internal: debounce ts api types generation
* bring back debounce logic
* revert changes in users schema
* internal: add malli docs instructions in skills format
* internal: add details to defendpoint+malli skill
* Fix incorrect types:watch initialization in build-hot script
* Remove version field from generated OpenAPI JSON to allow proper versions comparison on CI
* Update OpenAPI JSON schema
* a nice solution for the dynamic timezone enum
* update openapi json
* test fixes
* update openapi json with latest apis
* resolve tips when this happens again
* fix kondo test and revert the locale type in properties api
* update spec to remove locales
* Revert enum usage on UI
* build: types as part of postinstall
* types: replace hardcoded log type values
* build: format types with prettier - compatible with embedding type transforms
* Update OpenAPI schema
* Update OpenAPI schema
* Revert types cache generation
* Address PR comments
* Reset spec format
* Reset spec again
* Merge master and update OpenAPI spec
* Update package.json
Co-authored-by: Nemanja Glumac <31325167+nemanjaglumac@users.noreply.github.com>
* feedback
* resolve comments
* Update OpenAPI spec
* Update OpenAPI spec
* Update OpenAPI spec
* Update OpenAPI spec
* follow derive conventions
* Remove deprecated rspack type check deps
* Add eslint rule to restrict imports from metabase-types/openapi
* Update OpenAPI schema
---------
Co-authored-by: Nicolo-Maria <nicolo.mezzopera@metabase.com>
Co-authored-by: Ngoc Khuat <qn.khuat@gmail.com>
Co-authored-by: Nemanja <31325167+nemanjaglumac@users.noreply.github.com>
Co-authored-by: Stas Gavrylov <stasgavrylov@gmail.com>
Co-authored-by: Uladzimir Havenchyk <uladzimir@metabase.com>
Co-authored-by: Oleksandr Solovyov <o@solovyov.net>
Co-authored-by: Stas Gavrylov <stas@metabase.com>
Co-authored-by: Bryan Maass <bryan.maass@gmail.com>
The savingModel prop allow "collections" to be written to the
shared-tenant-collection namespace root while other model types are disallowed.
Fixes how this prop is calculated because the original implementation
left off other types of cards. This prop is now just "collection" | null
since collection is the only model type that matters here.
* Change url from checkout to login
* Use store login url when pro-self-hosted plan
* Unit test WIP
* source_plan query param added
* Cleanups
* Move useGetStoreUrl to hooks file
* Unit tests added
* Misc
* Misc
* Remove hooks
* Removed login path page
* Param renamed to `migration-source-plan`
* Unit tests updated
This pull request removes the user setting that persisted the open/closed state of the native query preview sidebar in the notebook editor. The sidebar's visibility is now managed entirely in the UI state (not persisted per user), simplifying the logic for showing or hiding the sidebar and cleaning up related code and tests.
1. opening a notebook editor will *always* start from a closed native preview sidebar
2. state is now managed exclusively by Redux (sidebar state might be preserved until the page reload, but it is not persisted on the backend)
3. We're changing a previous behavior where, if a native preview sidebar was open and we're resizing the window to something small, the sidebar was automatically closed. This PR changes that behavior. i.e. There is no special resize handling anymore. Whatever the (Redux) state of the sidebar is, it will stay like that no matter the window size.
The catch block in startContainers was swallowing errors - it would
log and tear down containers but never re-throw. This meant when
health checks failed, the script would print "All done!" and exit 0,
then Cypress would run and fail with a confusing "could not verify
server is running" error.
Now we re-throw after cleanup so the actual failure is surfaced.
* .
* .
* .
* .
* .
* trying without one timeout
* trying without another timeout
* trying without another other timeout
* trying without yet another timeout
* .
* Do not allow picking unsupported databases in Python data picker
* Do not auto-select disabled items
* Add e2e test for empty data selector
* Simplify ugly if condition
* Fix e2e test
* Do not open datapicker wen going to next step
* broad strokes
* clean up snippets and metrics pages
* tons of PR feedback
* Hi Dan
* test adjustments
* more adjustments
* fix card issue
* removing text on runs list page
* fixing glossary tests
* PR Feedback
* remove header prop
* page spacing adjustmnets
* Center Viz Empty State, fix python library page
* fix python transform page
* scrolling behavior on Job and Transform list pages
* last bits of shame
* fix: druid breakout and agg names must be unique
Fixes#10670
* Fix CI
* cljfmt
* format
* format rows
* make it work with druid jdbc
* address comments
* use seq over some?
---------
Co-authored-by: Riley Thompson <riley@metabase.com>
Co-authored-by: Riley Thompson <rileythompson99@gmail.com>
* WIP UXW-2451 Fixes issue where changing light/dark mode messes up color variables by removing nested providers in useBodyCellMeasure which probably breaks other things
* Update Loki Snapshots
---------
Co-authored-by: Metabase Automation <github-automation@metabase.com>
This generally indicates an MBQL 4 -> 5 conversion issue, and we need
the signal in the 57 rollout.
Also improves the detail of the logging for when `:dataset_query {}`.
* Add Australia (Sydney) to available regions
Added Australia (Sydney) to the list of regions.
* Add Asia Pacific (Sydney) IP addresses to whitelist
Added IP addresses for the Asia Pacific (Sydney) region.
* Reorder South America IP addresses section
* Update cloud region list in change-region.md
Removed duplicate entry for Australia (Sydney) and updated Asia Pacific region.
* Filter Popover close on click outside
* scroll on mount for field and preview panels, tests
* ci bandaid
* UXW-2510 Fix background color mismatch in table metadata panels
* PR Feedback
* shameful fix
---------
Co-authored-by: Brad Anderson <brad@metabase.com>
* Fix check for card changes
The query coming from the FE looks like:
```
{:aggregation [["count"]],
:breakout [["field"
42
{:base-type "type/DateTime",
:temporal-unit "month"}]],
:source-table 7}
```
(note strings like `"count"` and `"month"` and, most importantly, `"type/DateTime"`.)
The card from the DB looked like:
```
{:aggregation [[:count]],
:breakout [[:field
42
{:base-type :type/DateTime,
:temporal-unit :month}]],
:source-table 7}
```
When we compare the query to the old version to see if it's changed (and
thus, we should invalidate the verified status) we walk it first,
converting all keywords to strings:
```
(let [prepare (fn prepare [card] (walk/prewalk (fn [x] (if (keyword? x)
(name x)
x))
```
... but this is not the correct way to do this conversion, because
`:type/DateTime` becomes `"DateTime"`. We need to include the namespace
if it's present.
* Compare normalized queries
* Modify `changed?` to only compare updated keys
In the end we're doing a `t2/update!` with the updates passed in. If a
key isn't in the updates, it's not going to be updated.
Simplify `changed?` to just take two arguments, `before` and `after`,
and compare the keys in `after` (after doing the necessary
normalization).
If a key is in `after` but *not* before we can't compare it and should
throw an exception. For example, if we passed a card like
`(t2/select-one [:model/Card :id :name])`, we should throw.
* More sane way to check if a card has changed
Instead of ad-hoc updating keys, use the toucan transforms to just
canonicalize all the values before we compare.
* kondo
We currently have the "revision number", but:
- the intent there is slightly different; we want to prohibit one user
from making changes to the permissions graph before they *see* changes
another user might have made concurrently, not just sequence the
concurrent changes, and
- we allow API users to pass the `force` parameter to skip the revision
check entirely
Note that the reason we have `force` available is to allow people to
change permissions without first making a request to *GET* the
permissions graph (which can in some cases be extremely large). It is
not intended to allow concurrent updates to permissions.
https://github.com/metabase/metabase/actions/runs/20267445215/job/58194458921 failed because snapshots were running for QA databases, and we don't have those Docker containers up for this job.
Since we don't need them, I have adjusted the snapshot step to only run the default snapshots. There's a comment what to do if we change our mind and want to run external database checks in the future.
The summary: we are offering drill-throughs on custom columns defined after aggregation, but those custom columns don't exist in the base query. The error presented in such a way that `lib.underlying/top-level-column` for a dimension was returning `nil`, and we were happily putting that `nil` into the query we constructed.
I created a utility to filter dimensions by whether or not the top-level-column can be resolved, and used this for the underlying-records drill-through (from the original issue) as well as the pivot drill-through (which behaved similarly). If no dimensions have a top-level column, the drill-through is not available. If some do, we offer the partial drill-through.
`source` is a legacy key on the frontend that gets translated to `lib/source`. Some maps are using both `source` and `lib/source`, and the translation layer translates keys and values in completely separate steps. When the time comes to translate the legacy value of `source` into a valid value for `lib/source`, the code can't tell what the key was originally named. So already-valid values were going through the translation process, producing invalid values (e.g., `:source/source/aggregations`).
This adds a step to object parsing to detect that both keys are present; when they are, it sets an option to simply skip `source` during the parsing; secondly, it adds a step in the value parsing to check whether the value is already a keyword namespaced by `source/`.
Theoretically the BE could provide the email if you send in an expired
token, but it seems slightly less controversial to just provide a link
like `/auth/reset_password/the-very-long-token?email=some%40email.com`.
This way having the expired token itself can't reveal any information,
but if you click on an expired link, then it's just 2 clicks to send
yourself a fresh token instead of click -> type email -> click.
Fixes [UXW-658: New user invites expire quickly, adds friction to new user experience](https://linear.app/metabase/issue/UXW-658/new-user-invites-expire-quickly-adds-friction-to-new-user-experience)
`util.perf/mapv` checks the `count` of all its collection args, so it's
not safe to call with an infinite `seq`.
`lib.metadata.result-metadata/merge-cols` was doing exactly that in the
case where there was a mismatch between the column counts from lib and
driver. It was passing `(concat short-list (repeat nil))` to `mapv` and
causing an infinite loop!
* Correctly handle non-string `@tenants` in JWTs
Previously we threw exceptions if we received an invalid non-string
`@tenant`, like `123` or `false` or `{:foo "bar"}`.
Now we:
- convert integers to strings,
- take strings as-is, and
- throw a meaningful exception for anything else
`:dataset_query {}` should not happen, and indicates a bug in the
MBQL 4 normalization. On reading any card, if its `:dataset_query`
is `{}`, add the ID to an in-memory set and update a Prometheus
metric.
This reverts commit 9191713854.
used to do this in-house but now we use trunk for this. Old way has
ultimate flexibility. But trunk has some really nice workflows built
in (reporting, creating issues in linear, etc).
* rename to full app and modular embedding
* Update docs/embedding/sdk/quickstart-with-sample-app.md
* Update docs/embedding/full-app-ui-components.md
* Apply suggestions from code review
---------
Co-authored-by: Jeff Bruemmer <jeffbruemmer@gmail.com>
* Add DB infra for metabot use cases
* add enabled? use case setting and hook it up end to end for transforms
* remove transforms use case toggle
* add back migrations, number of tests
* add more missing migrations; lint
* some simplification
* fix some tests
* pull use-case out of tool call request header & fix last test
* more test fixes
* add back commits, move use_case->profile mapping to the appdb
* clean ns
* bump ci
* just store use case names in metabot_message instead of fks
* update test
* serialization fix; refactor
* lint
* restore migrations, address code review
* add back migrations again
* Tenants, External Users, and tenant based JWT with Audit Events
Allow the creation of tenant users
While working on this, I realized that my approach to the "All External
Users" group and renaming the "All Users" group was untenable.
I had been planning to just update these things in the database (create
AEU, rename AU to AIU) in the setter function for the `:use-tenants`
setting. The issue here is caching: we don't want to need to actually
reach out to the database every time we need to fetch the All Users or
All External Users, but it would also be quite unfortunate if one
Metabase instance in a cluster had an outdated view of, e.g. the ID of
"All External Users".
(Although - I'm not really sure how much we use the *name* of "All
Users" when we fetch `(all-users)`, so that might be worth a look.)
For now, I've modified things to just update things dynamically in the
`GET /api/permissions/group` and `GET /api/permissions/group/:id`
endpoints: we show/hide the "All External Users" group, and rename the
"All Users" group, based on whether Tenants are enabled.
Allow the creation of tenants
Listing tenants / tenant members will be a followup PR
lint
doh missing require
fix tests
add a test for enabling tenants changing groups
fix tests, slight change to tenant creation API
kick ci
fix another test
give minimum permissions to ext groups
fix more tests
impl user stategy setting ui
more API endpoints
add more tests, fix a very silly migration
slug validation
fix mariadb/mysql
lint migrations file
kondo
fix test
replace tenant members API with `/user?tenant_id=`
also realized that the `/user/recipients` endpoint needed to be modified
for tenants: tenant users should not see recipients that are not part of
the same tenant. (reminder for later: we will need to add safeguards
around actually creating those pulses, not just assume the API won't
send bad recipients!!)
IF you are an admin, `/api/user` respects a `tenant_id` query param and returns:
- "normal" (non-tenant) users if it's `nil`
- specific-tenant users if not
If you are *not* an admin, `/api/user` ignores the `tenant_id` passed
and only returns users in the same tenant.
`/api/user/recipients` will:
- return ALL users if you are a superuser (both tenant and non-tenant)
- return only users in the SAME tenant as you if not
doh
tenant crud wip
integrate endpoints, fix type errors
Allow getting all external users
previously you could do only
```
GET /api/user => internal users
GET /api/user?tenant_id=null => internal users
GET /api/user?tenant_id=123 => users in tenant 123
```
now you can do
```
GET /api/user => internal users
GET /api/user?tenant_id=null => internal users
GET /api/user?tenant_id=123 => users in tenant 123
GET /api/user?tenancy=all => internal + external users
GET /api/user?tenancy=internal => internal users
GET /api/user?tenancy=external => external users (in all tenants)
```
If you specify both `tenancy` and `tenant_id` (regardless of the values
passed) you'll get a 400.
fix my bad rebase
fix bad rebase more
remove #p
mark tenants as `is_active` or not
initial external users management
remove permissions check on the trash endpoint
I don't know why this was added in the first place (I mean, I think I
added it, but I don't know why I did).
The trash is always accessible to everyone. We gate the visibility of
things *in* the trash, but the trash itself is always readable.
no personal collection for tenant users
remove tenant query inside oss
tenant user routes
make it so that you cannot remove users from the external users group
do not send email to tenant users
no login for inactive tenants
disable collection permissions, don't show sent email modal on user success with SMTP
remove all users group has higher permissions warnings for external users group
moving is external checks behind plugins
unit test cleanup, add simple test
read only application permissions for external users
set tenant on login with JWT
incredibly hacky tenant attribute
separate `:attributes` from `:login_attributes`
thinking here is that `login_attributes` are the things you set on the
user, where `attributes` are all the attributes of the user including
things like tenant slug or maybe someday locale etc.
fix build
fix token check
create a tenant collection in a migration
this works, but the tenant collection is always visible to admins, even
if the setting is off.
hide tenant collection from normal users
fixing build and FE rebase
Delete stray "howdy"
Lol, sorry about that.
fix tests
throw instead of silent failure on no tenant
superuser can't be tenant
fix up tenant collections
tenant collection now includes subcollections.
Tenant collections are hidden if the feature isn't enabled.
tenant collections marked in API
bugfix and current user can write any coll
don't add `attributes` key to all users
just the current user on login
invalidate groups when changing user strategy
read only user
{{ tenant_slug }} user attribute
fix be test
rip out tenant collection stuff on BE
fix BE tests
lint
fix magic string
no tenantifying existing users for now
also clean up a complicated, confusing fn.
starting e2e
fixing e2e, adjusting @tenant user attribute
fix permissions editor, add tenant.name e2e test
allow more than one user per tenant
🤦
allow getting only active tenants
*actually* don't replace existing values with nil
getting e2e ready for tenant user provisioning
tenants management ux
e2e coverage on sandboxing, deactivating tenants
replace {{tenant_slug}} with `@tenant.slug`
filter out other-tenant users from pulse creation
rename any existing `@.+` user attributes
prohibit new user attributes starting with `@`
fix migration
ugh, forgot to actually commit this file
add more tests
kondo
use the `tenants` feature
fix a test - missed one `:advanced-permissions`
e2e test coverage, updating tenant attribute to @tenant.slug
Partial Type Fixes
types green
unit tests
adding groups to e2e test, maybe fixing tenant feature check?
Add tenants to tracked snowplow features
Fixing Admin ui
Modernizing tenants page
separate external group in permissions sidebar
permissions editor updates
hardcode that *everyone* has trash access
shame... bad imports
adjusting Main Navbar. using globe icon
CR response
fix tests
fix attributes when tenants are disabled
lock up the 'all external users' collection
tenants route guard
add tests for missing feature and oss
fixing some unit tests, and type errors
set token features in default.cy.snap
unit and e2e test adjustment
another pass on e2e
fix e2e and build
fix a bad merge
Green CI?
reset token after restoring DB
move a test to fix the test
kondo
Move `/api/ee/tenants` to `/api/ee/tenant`
PR Feedback, part 1
move route functions into ee
ADM-1105 permissions editor adjustments for external users
ADM 1104 do not show invite button on external users when there is no tenant
adjusting unit tests
fix no results message for deactivated users page
center no results message
Fix migrations IDs / related tests
adjusting no results message for tehants
fixing unit tests, adding e2e test for permissions
chore: add timestamp columns to tenants (#60575)
* chore: add timestamp columns to tenants
* fixup! add hooks
* kick ci
Auto provision tenants, respect tenants feature (#60571)
deactivate tenant users along with their tenants (#60552)
* deactivate tenant users along with their tenants
Tenant users from deactivated tenants were already prevented from
logging in (we checked the activation status of their tenant) but for
bookkeeping it's probably best to actually deactivate the users
themselves along with their tenants, and reactivate them if the tenant
is reactivated.
* invalidate users when updating a tenant, adjust external users list screen
* e2e test
---------
Co-authored-by: Nick Fitzpatrick <nickfitz.582@gmail.com>
audit events for tenants (#60647)
* Revert "Delete unused string util from metabase lib (#64389)"
This reverts commit 0af38ef8f0.
* fix the build
* fix tests
* fix rebase error in invite button
* fix more tests
* fix more tests
* fix more tests
* fix more tests
* fix more tests
* fix more tests
* fix another test
* Fix another couple tests
* and more
* and more
* Close the modal to create shared tenant collection after it is created (#66049)
* Fix more tests, nearly all e2e tests passing
* a couple more tests
* couple more tests
* last (???) few tests
* Remove edit permissions in ellipsis menu when viewing shared tenant collection (#66042)
* Suggest external users only when adding tenant group members (#66046)
* feat: separate namespace for tenant collections (#66102)
* fixup! FE build
* feat: add remote-sync support for tenant collections (#66272)
* feat: add remote-sync support for tenant collections
* Update permissions.clj
* fixup! tests
* fixup! enterprise tests
* fixup! mysql/h2
* EMB-1011: add default read perms for namespace
* Fix tests (#66403)
* Fix tests
Not exactly sure how these got merged, I think maybe they were hidden by
GH issues.
* Fix kondo
* fix apparent rebase error
* fixup! EMB-1065 fix namespace filtering
* fixup! some bad perms tests
* fixup! remote sync dep checking across collections
* fixup! remove blank test
* fixup! missing tenant setting
* fixup! add premium feature declaration
* fixup! get test working with h2
* fixup! revert h2 fix
* fixup! revert h2 fix
* fixup! real h2 fix
* Dedicated Tenant Collections
* More fixes
* Tests and more fixes
* Fix rebase error
* rename
* Fix query for shared tenant collections
* fixup! UXW-2386 don't pass `tables` model into tenant root
* fixup! UXW-2436 build correct path for namespaced collections
* Remove test of non-existent fn
* Fix failing frontend unit and e2e tests in tenants feature branch (#66243)
Co-authored-by: John Swanson <john.d.swanson@gmail.com>
* fix failing frontend ESLint errors
* Prefix users and groups as "Internal" in admin settings when tenants are active (#66169)
* Update design of the user strategy modal for tenants (#66182)
Co-authored-by: Mahatthana (Kelvin) Nomsawadi <me@bboykelvin.dev>
* Add multi-tenancy setup guide to embedding hub (#66113)
* Update design of the tenants listing and internal user page (#66193)
* Give tenant users a personal collection
* Fix BE tests
* another be test
* Add specific error messages for JWT tenant claim errors (#66172)
* kondo/fmt
* fix running jwt tests standalone
* feat: per collection setting (#66475)
* feat: per collection setting
* feat: remove unsynced collections
* fixup! tests
* fixup! lints
* fixup! test
* Remove duplicated tenant permissions
* handle disabling feature more cleanly
when tenants is turned off:
- disable all tenant users
- reject any logins from tenant users
- reject any JWTs that specify a tenant
* Add tenant entity picker e2e tests. Fix one bug surfaced
* uxw-2456: remove drop shadows from dedicated tenant collection cards
* Hide tenant collections permissions tab when tenants are disabled (#66537)
* Hide create shared tenant collection button on sidebar for non-admin users (#66538)
* Hide tenant collections sidebar section when no collections exist for non-admins (#66539)
* Fix dedicated tenant collection root in collection picker as tenant user (#66549)
* fix frontend type-check: only set namespace if it is defined
* EMB-1093: Fix not passing a tenant attribute
Fixes [EMB-1093: If you enable multi-tenants and don't add JWT
attribute, it will
break](https://linear.app/metabase/issue/EMB-1093/if-you-enable-multi-tenants-and-dont-add-jwt-attribute-it-will-break)
* EMB-2459: Fix restrict where collections can be moved to
* Fixup working e2e entity picker tests
* fixup! mysql namespace tests
* fixup! tenant group mapping tests
* fixup! ws
* tree endpoint allows specifying multiple nses
* fix broken test
* Show dedicated tenant collections in collection pickers for admins (#66583)
* feat(sdk-bundle): add tenants ee plugin (#66586)
* fixup! bad ns clause audit feature is off
* fixup! UXW-2473 support non-string attributes
* Hide tenant collections section when there are no visible shared collections (#66597)
* Add tenant cards to search
* fixup! tenant collection cleanup
* EMB-1082 rename strings in the ui (#66603)
* make tenant field required (#66516)
* change `tenant` key to `@tenant`
- if the feature is off, you can still log in if you have the `@tenant`
key set. It will be silently ignored, but a warning will be logged.
- if the feature is on, `@tenant` is a drop-in replacement for `tenant`.
- we'd previously prevented setting user attributes that start with `@`.
This updates `remove-invalid-attributes` to do the same with JWT/SAML
attributes.
* add SAML tests
* Expose SAML attribute for tenant
* External users can't use MB if feature disabled
Plus add tests for:
- tenant user, feature disabled
- tenant user, feature unavailable
* Add missing test files
* Multi-stage embedding data picker should query models and questions from tenant collections (#66598)
* Fix external group tooltip referencing All Internal Users (#66654)
* Allow conversion of tenant <-> internal users
Upon conversion, all previous group memberships are destroyed and the
user is added to either the "All External Users" or "All Users" groups
accordingly.
* feat: implementing new tenant/remote-sync designs (#66567)
* fix missing dep in test file
and a tiny removal of an unnecessary `dissoc`
* fixup!: broken embedding page due to bad rebase conflict resolution
* fixup!: fix type errors
* fixup!: fix frontend jest unit test errors
* fixup! Move search tests requiring EE to EE code
* Fix copy test
* add confirmation warning when disabling tenants (#66697)
* fix test
* more renaming
* fixup!: failing tenants.cy.spec.js test and rename "All tenant users" migration
* fixup! whitespace fixes
* Remove serialization for dropped column
* fixup! missing migration
* fix other wrongly ordered migration
* fixup! Ordering more migrations
* fixup! other tests
* fix test
* fix another test
* api response schemas for tenant endpoints
* kondo
* one more kondo
* fixup! bad test query
* fixup! popover to use the right way of passing args
* Fix push changes modal
* fixup! correct token for entity picker tests
* Fix tenants e2e test with correct attribute
* fixup! tenants tests
* fixup! remove test for old behavior
* fix type on events, removed unused string union values
* update MainNavSharedCollections.unit.spec.tsx to use the new name
* fix linting issue unused import
* remove test file not on master anymore
* fix permissions select
* fix dashboard filters autowiring e2e test
* fix formating of module.edn
* remove useless test file
* add enum for namespaces
* fixup! remove toggle
* Update EntityPickerModal.tsx
* Remove commented tests
* Update e2e test based on feedback
* Split add-attributes-test
* Update test/metabase/collections_rest/api_test.clj
* Fix copy cmd model list
* Remove :put handling code for is_remote_synced
* Update src/metabase/premium_features/settings.clj
* Update resources/migrations/056_update_migrations.yaml
* Remove extraneous regex in api def
* Update frontend/src/metabase-types/analytics/event.ts
* UXW-2536 popup remote-sync modal after request
* Remove the "sidebar" event type
* Improve some backend tests
* Fix remote-sync e2e tests
* refactor: remove useAdminSetting and use settings already in the
component
* same for the jwt form
* move error message to Messages
* variant filled for apply button
* convert attributes to strings on read/write
In practice there is no difference between the value `true` and the
value `"true"`, so let's keep things simple by making all attributes
maps of string->string.
Fixes [UXW-2550: Cannot edit tenant users with non-string attributes](https://linear.app/metabase/issue/UXW-2550/cannot-edit-tenant-users-with-non-string-attributes)
* fw={500} => medium
* add c() to t`Sync {collection.name}`
* useSetting("use-tenants") in MainNavSharedCollections
* fixup extra dialog on permspagelayout
* fix broken tests
* fix the other broken test
* Remove tenant collection app permission
* Fix embedding sdk collection type
* Convert tenants e2e test to typescript
* move setupRemoteSyncEndpoints to server-mocks folder
* actually remove it from test-utils
* Small BE fixes
* use usePrevious from react-use
* Remove custom root tenant collection hook
* Change group detail to use text
* fix sdk type
* Redid git controls with button.group
* fixup! bad test
* fixup! tenant logic and components into plugins
* Update frontend/src/metabase/lib/string.js
Co-authored-by: Ryan Laurie <30528226+iethree@users.noreply.github.com>
* fixup! bad merge
* Rearrange last unit test for CollectionPicker
* fixup! PR feedback
* fixup! handle nils and seqs in attributes
* fixup! kondos
* fixup! better stringify
* Add missing maybe to login attributes schema
* fixup! yaml tests
* fixup! sequential check
* fixup! Stringify ldap attributes
* Fix ldap tests
---------
Co-authored-by: John Swanson <john.swanson@metabase.com>
Co-authored-by: John Swanson <john.d.swanson@gmail.com>
Co-authored-by: Phoomparin Mano <phoomparin@gmail.com>
Co-authored-by: Phoomparin Mano <poom@metabase.com>
Co-authored-by: Mahatthana (Kelvin) Nomsawadi <me@bboykelvin.dev>
Co-authored-by: Nicolò Pretto <info@npretto.com>
Co-authored-by: Ryan Laurie <iethree@gmail.com>
Co-authored-by: Ryan Laurie <30528226+iethree@users.noreply.github.com>
OOMs in cloud are linked to this weak map retaining too much. The
exported heap dump reports are suggestive but not conclusive, and I
can't reproduce locally.
Adds `:metabase-query-processor/computed-weak-map-queries` to measure
the size of the weak map. That should sawtooth with GCs as it
accumulates queries and then discards them, but we'll see if it rises
forever on the problem instances.
This also adjusts the caching for a `MetadataProvider`-level entity
(`:metadata/card`) from the weak map to the `MetadataProvider`'s
internal cache. That's speculative, since there's no evidence to link
it to the issue, but it should be at least as effective a cache and
there's less risk of accidentally retaining too much.
* Closes EMB-990 add description to columns in CollectionBrowser
* keep the existing behavior with tooltip unless description is explicitely set to be visible
* .
* addressed comments
* cleaned up unit tests
* Ignore collection for unpublished tables
Unpublished tables have no collection and the `collection_id` means
nothing, and should be ignored.
* Add a test
* Fix legacy search permissions
Legacy search ignored the collection permissions for published tables,
so it was possible to get a table back from search even though you
didn't have permission to see it because it was published to a table.
* segments: published table segment list, revision history, dependency graph, breadcrumbs, move preview to the menu
* specs
* fix specs
* dedupe tests of segments pages
* fix avatars
* fix rebase
* visuals
* update specs
* final refactoring
* polish
* review
* drop react context although it adds a bit of dupe
For those of us who use clojure-mcp (https://github.com/bhauman/clojure-mcp), our Claude configuration directly contradicts instructions we're giving it. This PR only adds to our `CLAUDE.md` and our skills a note that **if** clojure-mcp is available, it should prefer it over shell-based methods (like `./bin/mage -repl`).
docs scoop up all settings and has ways to avoid. In
`metabase.cmd.env-var-dox/avoid?` This had a docstring and then an extra
docstring. We can just copy the nice docstring to the docstring slot and
put `:doc false`.
```clojure
(defn- avoid?
"Used to filter out environment variables with high foot-gun indices."
[env-var]
(or (false? (:doc env-var))
(false? (:can-read-from-env? env-var))
;; Ideally, we'd move off of this list completely,
;; but not all environment variables are defsettings.
(contains? env-vars-not-to-mess-with (format-prefix env-var))))
```
* Show breadcrumbs on published table page (Library > Data > <table>)
* UXW-2524 Makes table breadcrumbs dynamic
* Adjust e2e test
---------
Co-authored-by: Brad Anderson <brad@metabase.com>
* UXW-2527 Disallows Library as an option in the snippet collection modal
* UXW-2534 Renames root sql snippet folder from "Top folder" to "SQL snippets" globally
this file got removed during a master->uxw-394-sub-collections-permissions-toggle merge on the PR #65928
the file had some changes, and removing it was not intended.
the version restored is this one, extracted from the same PR:
ab62f89a2c
motivating example:
we ran the redshift tests, and had three failures: redshift 1, redshift
2, and redshift transforms.
redshifts 1 and 2 timedout at 60 minutes. But the log upload failed
because the python transform tests failed in 24 minutes and then
uploaded:
from the transform failure run:
```
Run actions/upload-artifact@v4
with:
name: test-logs-be-tests-redshift-ee
path: logs/test-log.json
retention-days: 1
if-no-files-found: warn
compression-level: 6
overwrite: false
include-hidden-files: false
env:
CI: true
...
With the provided path, there will be 1 file uploaded
Artifact name is valid!
Root directory input is valid!
Beginning upload of artifact content to blob storage
Uploaded bytes 1072768
Finished uploading artifact content to blob storage!
SHA256 digest of uploaded artifact zip is 267d4b085d49ca4d682fedca97832fecb603bbb5a52c0e1d99bd4daa49984ba3
Finalizing artifact upload
Artifact test-logs-be-tests-redshift-ee.zip successfully finalized. Artifact ID 4816135520
Artifact test-logs-be-tests-redshift-ee has been successfully uploaded! Final size is 1072768 bytes. Artifact ID is 4816135520
```
And then from the redshift tests:
```
Run actions/upload-artifact@v4
with:
name: test-logs-be-tests-redshift-ee
path: logs/test-log.json
retention-days: 1
if-no-files-found: warn
compression-level: 6
overwrite: false
include-hidden-files: false
env:
CI: true
...
With the provided path, there will be 1 file uploaded
Artifact name is valid!
Root directory input is valid!
Error: Failed to CreateArtifact: Received non-retryable error: Failed request: (409) Conflict: an artifact with this name already exists on the workflow run
```
The same happens on redshift part 1.
```
Error: Failed to CreateArtifact: Received non-retryable error: Failed
request: (409) Conflict: an artifact with this name already exists on
the workflow run
```
This just introduces on the driver action to look for a
logs-artifact-suffix and use it if present. And on driver jobs that use
a matrix include the matrix.job.name (or matrix.version.name depending
on the exact structure).
* Fix query normalization issues in v57
* Remain strict when saving a query
* replace bad dataset_query with legacy_query
* Test fixes 🔧
* Test/Kondo fix 🔧
---------
Co-authored-by: dan sutton <dan@dpsutton.com>
### Description
In https://github.com/metabase/metabase/pull/65343 we added a page to purchase Metabase AI (Metabot). Now we are improving the text to:
* Show information about trial only if Metabot was not already trialed
* Improve wording regarding multiple questions / messages in chat
### How to verify
1. Ensure you have a token with the `offer-metabase-ai-tiered` feature, but without the `metabot-v3` feature.
2. Check that the subscription associated with the token had either with (test case 1) or without (test case 2) the `metabase-ai` or the `metabase-ai-tiered` add-on.
3. Navigate to `/admin/metabot` and see the page look like in the mock ups.
### Checklist
- [x] Tests have been added/updated to cover changes in this PR
Closes: https://linear.app/metabase/issue/CLO-4673/show-different-copy-on-metabot-ai-purchase-page-after-trial
Closes: https://linear.app/metabase/issue/CLO-4688/copy-correction-in-the-admin-panel
* #66585 Updates static embeds to omit color-scheme settings to preserve legacy transparent iframe behavior
* support public embed too (#66693)
* support public embed too
* Imports isPublicEmbedding to prevent app crash
---------
Co-authored-by: Brad Anderson <brad@metabase.com>
---------
Co-authored-by: Nicolò Pretto <info@npretto.com>
If you have a query like "WITH query_a AS (...), query_b AS (select ... from query_a) SELECT ... from query_b",
native query parsing understood that query_b in the final query referred to query_b in the WITH, but it didn't realize
that query_a in query_b referred to query_a in the WITH. Now, it should understand these sorts of depedent ctes.
* Move documents feature to OSS
* Remove plugin
* Move comments, remove public sharing plugin
* Fix some CI issues
* Move documents BE to OSS
* Move documents BE to OSS
* Move documents BE to OSS
* Move documents BE to OSS
* Move documents BE to OSS
* Fix bug with state
* Move rest enterprise imports
* Remove token checks from apis related to documents
* Fix CI
* Fix CI
* Fix CI
* Fix CI
* Fix BE tests
* Fix CI
* Fix CI
* Fix bookmarking a document
* Add branding footer for public document
* Add e2e tests
* Fix CI
* Fix CI
* Fix CI
* Fix CI
* BE fixes for documents to oss (#66511)
* reinstate sandbox test
* refactor circular deps
* fix: copy documents/comments
* fix path
* remote-sync tests are ee only
* ee only
* documents are not just ee
* kondo
* missing mapping
* set site-url
* fix broken tests
* Fix metabot plugin
* Fix issue where having a metabot block inside a document prevents it from loading when publicly shared (#66564)
* Prevents public document view from crashing when document contains a metabot node. Uses fallback similar to print version.
* Omits SupportingText drag handle when in read-only mode
* Adds e2e test to verify metabot blocks don't crash public documents and aren't editable within them
* Fix accidental spacing change
* Fix unit tests, cleanup e2e test
* fix after merge
---------
Co-authored-by: Nicola Mometto <nicola.mometto@metabase.com>
Co-authored-by: Brad Anderson <brad@metabase.com>
* add events when making cards/dash pubic or retracting
https://github.com/metabase/metabase/issues/64417
When we make a card public, we put a uuid on it and include
`made_public_by_id`. But when you retract the public link, we remove
both bits of information. This is a nice invariant but it removes
information about who publicized a card.
Imagine a card is discovered by have been public. Of course you want to
immediately remove it. But this erases the audit trail (so to speak) of
who made it public. It is not a great idea to delay remediation during
the investigation. Now you can just consult the audit log
* make events
* formatting
* clean up tests a bit
tests were a bit wack. We should have utility functions to get audit
events. Also noticed we are going off of the object id and ignoring the
object. Could lead to gross issues if there's dashboard with id 4 in
there and also card with id 4. so require model on new one
* fmt
* Remove models
* Remove models
* Move
* Move
* Drop columns for model publishing
* Drop columns for model publishing
* Drop columns for model publishing
* Drop columns for model publishing
* Rename library-models to library-data
* Rename library-models to library-data
* Rename library-models to library-data
* Rename library-models to library-data
* Fix tests
* Fix table URLs
Comment out MB_PREMIUM_EMBEDDING_TOKEN in .env.example to prevent
conflicts with Cypress tests that need to start from a clean state, and
set tokens programmatically.
Fixes#66464.
This was a corner case in `lib.field.resolution` and `qp.util.add-alias-info`
that regressed in 57. It made the QP less smart about repairing a certain
class of broken queries, which regrettably exist in the wild.
The breakage in question was a `:field` ref in later stage, which had a
numeric field ID and a `:join-alias`. If that field ID was not actually
returned as part of the join anymore, fallback metadata would be
generated by looking up the field by ID. That "works" for non-joined
fields but for an explicitly joined field it would use
`source."SOME_FIELD"` rather than `"The Join Alias"."SOME_FIELD"`, which
causes SQL compilation to fail.
### Description
In https://github.com/metabase/metabase/pull/65343 we added a page to purchase Metabase AI (Metabot). Now we are improving the layout and content to better present the product.
### How to verify
1. Ensure you have a token with the `offer-metabase-ai-tiered` feature, but without the `metabot-v3` and `offer-metabase-ai` features.
2. Navigate to `/admin/metabot` and see the page look like in the mock ups.
### Checklist
- [x] Tests have been added/updated to cover changes in this PR
Closes: https://linear.app/metabase/issue/CLO-4669/make-small-changes-to-ai-page-in-admin
* Combine metabot search queries using OR
* Fix zero-terms edge case
* Point out a critical defect
* Urgent typo
* Add docstring to join-results-by-rrf function
Co-authored-by: crisptrutski <881351+crisptrutski@users.noreply.github.com>
* Use non-semantic fallback engine for term-queries
* Group experimental options together
* Small fixes and testing
* Workaround for semantic search matching unwatned stuff
* Renaming and wiring
* Remove unused import
* Remove sharp edge around 0 queries
* Fix bug in disjunction for legacy search. Test coverage for it.
* Complete comment
* Whitespace lint
* Try to fix new tests - run in semantic search test group, simplify test configuration
* More test fixes
* More test fixes
* Whitespace lint
* Make appdb search check more explicit
* don't run metabot semantic search tests on h2 because they rely on the appdb backend being available
* address test comments
* remove experimental flags; make this the normal behavior
* fix test
* fix disjunction on h2
* try again to fix search-test
* move search-test to metabase-enterprise.metabot-v3.tools.semantic-search-test since it now requires semantic search to be available
---------
Co-authored-by: Chris Truter <chris@metabase.com>
Co-authored-by: crisptrutski <881351+crisptrutski@users.noreply.github.com>
* Add a subscription button for modular embeddings (#65704)
* Add an entry to the dashboard subscriptions on modular embeddings
* Hide the subscription when email hasn't been set up
* Add tests
* Add subscription button to interactive and editable dashboard components
* Make dashboard button styles consistent + expose subscriptions button
* Fix types
* Fix wrong default `aria-label` for `ToolbarButton`
* Support `withSubscriptions` prop
* Add tests
* Make it so that modifying tests would be easier
* clarify missing assertions with a comment
* Fix types
* Update Loki Snapshots
* Add `withSubscriptions` to storybook
* Remove duplicate test
* Fix comment date format
* Tighthen ToolbarButton prop type
* Avoiding cramping all test combinations into a single test
* Move shared tests to the correct directory
* Make the function parameter order makes more sense
* Simplify SDK dashboard button visibility logic
* Make shared tests more structured
---------
Co-authored-by: Metabase Automation <github-automation@metabase.com>
* Avoid showing other subscription channels (#66126)
* Make text search easier on an enum
* Prepare state for upcoming change
* Avoid showing the NEW_PULSE screen on SDK
* Add tests
* Fix tests
* Make mocking module more idiomatic
* Update subscription sidebar email edit view content (#66258)
* Update subscription sidebar email edit view content
* Add tests
* Improve test related variable name
* Properly memoize parent callback
* Add dashboard subscriptions option to EAJS dashboards (#66309)
* Expose `withSubscriptions` to EAJS dashboards
* Close subscription sidebar when unchecking `withSubscriptions` checkbox on EAJS
* Add `withSubscriptions` to EAJS dashboards
* Add tests + adjust no email setup tooltip condition
* Add missing unit tests mock
* Only closes sidebar when subscriptions sidebar is open
* Make guest embed warning tooltip aligned with its label
* Fix a test after merging guest embeds
* Update tooltip design from using hovercard to tooltip
* Close the subscription sidebar when the last pulse is archived (#66496)
* close the subscription sidebar when the last pulse is archived
* Ensure we only close the sidebar on the last subscription deletion
* Remove unused code
* simplify mocking
---------
Co-authored-by: Mahatthana (Kelvin) Nomsawadi <me@bboykelvin.dev>
* Added one happy path e2e test (#66506)
* Added one happy path e2e test
* .
* .
* Pass tests on CI with SMTP server
* Remove unused interceptions
* Use explicit assertions
---------
Co-authored-by: Mahatthana (Kelvin) Nomsawadi <me@bboykelvin.dev>
* Run maildev in CI
* Fix admins aren't shown subscription list page after creating first subscription (#66372)
* Run maildev in CI
* Fix admins aren't shown subscription list page after creating first one
* Fix test name typo
* Apply suggestions from code review
Co-authored-by: metabase-bot[bot] <109303359+metabase-bot[bot]@users.noreply.github.com>
* Fix tests after merging another PR
---------
Co-authored-by: metabase-bot[bot] <109303359+metabase-bot[bot]@users.noreply.github.com>
* Make subscriptions on SDK/EAJS a Pro feature (#66392)
* Make subscriptions on SDK a Pro feature
* Add subscription only to EAJS on pro plan
* Correct directory name
* Fix InteractiveDashboard unit tests to use EE token
* Fix StaticDashboard tests to use EE token feature
* Fix EditableDashboard tests to use EE token feature
* Remove OSS tests because there's no SDK on OSS version
* Update Eajs related terms
* Allow subscriptions SDK plugin to be reinitialized
* Standardize plugin enabled property name for EAJS plugins
* Add embed wizard tests
* Add EAJS tests
* Fix OSS test
* Add missing assertions on Enterprise tests
* Optimize API calls on SDK/EAJS
* Clarify comments
* Use initialize plugin pattern for EAJS
* Fix test name
* Fix assertion because upstream content change
* Auto update EAJS subscriptions options after configure email
* Update no email setup tooltip
* Remove debugging code
* Try to fix 1 nasty E2E test
* Fix the samn test again
* Fix the E2E again™️
---------
Co-authored-by: Metabase Automation <github-automation@metabase.com>
Co-authored-by: Sébastien <sebastien@metabase.com>
Co-authored-by: metabase-bot[bot] <109303359+metabase-bot[bot]@users.noreply.github.com>
* When setting settings, include 5 minute cookie of updated timestamp
if other instances have outdated settings cache, this cookie will let
them know what the requestor has seen. The instance can then know it is
out of date. This won't help for instances that aren't hit by the person
setting the setting.
Motivating example is someone who is doing an embedding onboarding
checklist. And then hits a new instance that hasn't yet loaded the "show
embedding homepage" setting so it flips back to the regular
homepage. The hope here is the person doing this checklist will let the
other instance know it is out of date, reload the settings cache and
then see the embedding setting.
* Interesting strategy here for cookies.
I hated that the route would would set a cookie at a particular key, do
timestamp stuff, etc. I know ring is a map and abstraction, but i added
a new paradigm:
```clojure
(defn- add-settings-last-updated-cookie
"Add a cookie with the current settings-last-updated timestamp to the response."
[response]
(assoc-in response [:mb/cookies :cookie/settings-cache-timestamp] true))
```
This means that just the middleware is responsible for checking and
setting this key. This was largely driven by two different namespaces
having a private def of:
```clojure
(def ^:private settings-last-updated-cookie-name "metabase.SETTINGS_LAST_UPDATED")
```
And it's nice if the ring map can indicate to this middleware to handle
all the key and timestamp stuff and it just has to indicate to set this
with `cookie/settings-cache-timestamp`.
So the request looks like this:
```clojure
{:status 204
:headers {"X-Frame-Options" "DENY"
"X-XSS-Protection" "1; mode=block"
"Last-Modified" "Mon, 1 Dec 2025 17:09:59 GMT"
"Strict-Transport-Security" "max-age=31536000"
"X-Permitted-Cross-Domain-Policies" "none"
"Cache-Control" "max-age=0, no-cache, must-revalidate, proxy-revalidate"
"X-Content-Type-Options" "nosniff"
"Content-Security-Policy" "...<lots of junk>..."
"Expires" "Tue, 03 Jul 2001 06:00:00 GMT"}
:body nil
:mb/cookies {:cookie/settings-cache-timestamp true} ;; <---- our new addition
:cookies {"metabase.DEVICE" {:value "60ac9454-9dd1-439a-afa1-921ff029686d"
:http-only true
:path "/"
:expires "Fri, 1 Dec 2045 17:09:59 GMT"
:same-site :lax}}}
```
And then in practice, it looks like this:
Set the setting and get a cookie with the timestamp in response:
```
❯ http put :3000/api/setting/site-name value=foo x-api-key:$KEY
HTTP/1.1 204 No Content
...
Set-Cookie: metabase.SETTINGS_LAST_UPDATED=2025-12-01+17%3A28%3A15.450873%2B00; Path=/; Max-Age=300; SameSite=Lax
...
```
Simulate sending to another instance with a timestamp that exceeds the
currently know timestamp (i'll increment the hour based sent back in
this response)
```
❯ http get :3000/api/user/current x-api-key:$KEY Cookie:metabase.SETTINGS_LAST_UPDATED="2025-12-01+18%3A23%3A51.059155%2B00" | jq '.date_joined'
"2025-12-01T16:51:08.137822Z"
```
(note the 18 vs the 17)
And the logs are this:
```
INFO middleware.settings-cache :: Settings cookie indicates cache is out of date. Refreshing...
DEBUG middleware.log :: GET /api/user/current 200 19ms (11 DB calls) App DB connections: 0/4 Jetty threads: 5/50 (5 idle, 0 queued) (245 total active threads) Queries in flight: 0 (0 queued) {:metabase-user-id 346}
```
* fixes:
- logged known timestamp not weird cookie value
- timestamp is optional, so handle that
- typo in log konwn -> known
* Add tests, add functionality
tests:
ensure it (a) sets the cookie, (b) honors the cookie, and when it
restores the cache, to set the cookie so it doesn't keep thrashing. But
presumably it won't since it's up to date. My "in the future" version is
a bit synthetic.
Functionality wise, updating the cache now returns and this will saet
the header with the most recent cache setting
* fix: show confirmation on escape, clickoutside and browser's back actions for SidebarCacheForm (#53825)
additional: refactor useConfirmOnRouteLeave hook, add an ability to get router across application without property drilling using hook instead
# Conflicts:
# frontend/src/metabase/plugins/index.ts
* fix: small adjustment
* fix: small adjustment
* fix: tests
* fix: tests
* fix: created useStackedSidesheets hook to help control stacked sidesheets
* adds a test for useStackedSidesheets that covers #53825 issue
* adds tests for useConfirmOnRouteLeave that cover #53825 issue
* rename useStackedSidesheets to useStackedModals. Add backward compatibility prop opened
* fix: add history provider as well for the future purposes.
* fix typings
* fix code based on comments from PR
* fix tests
* fix tests
* fix tests
* remove useStackedModals. Write e2e for all "closing" actions
* remove useStackedModals. Write e2e for all "closing" actions
* adjusted code according to PR comments
* fix tests
* make tests that asserts the confirmation modal showing if there are dirty form explicit in order to see which of them is broken in CI
* fix tests
* remove "forward" browser's button testing on trying to close a dirty form
* fix tests
* fix tests
* Fix sub-collections toggle to stick when permission seletor is closed / Handle dirty state gracefully
* Add e2e tests
* Fix outstanding e2e tests
* Remove e2e test that was expecting the previous undesired behavior
* Update test that accounts for issue #30494
* Add clarifying comment
* data studio app permission
* actually transforms are for admins
* check permission for open in data studio
* Auto-set library permissions on data studio permission changes
* Fixed module references
* initial library collection perms
* Fixed module references
* Fixed module references
* Fixed lint errors
* perms
* specs, fixes
* ci
* data studio permission
* Added data-studio permission check to glossary
---------
Co-authored-by: Nathan Voxland <nathan@metabase.com>
* IAM auth note on postgresql
* move to rds
* rds
* mysql note on verify-ca
* Update docs/databases/connections/aws-rds.md
Co-authored-by: Alex Yarosh <alexandra@metabase.com>
---------
Co-authored-by: Alex Yarosh <alexandra@metabase.com>
* Add display_name filtering to table list
* Re-introduce mixed case LIKE for mysql/maria
Must have been lost in a merge conflict
* Escaping rules
* Kondo
* Add Metabase Sample Database to editable tables support
Updated the list of databases that support editable tables to include Metabase Sample Database.
* Update docs/data-modeling/editable-tables.md
* Update docs/data-modeling/editable-tables.md
- If you decide to build the backend, you should run it
- There are no more derived options - everything is explicit
- Running Docker containers is a non-negotiable
- Tests need these supporting services to run
- If you really insist on not running them, either comment out those lines temporarily or do not use this script to set up the e2e environment
- Because of this, there are no more custom envs `QA_DB_ENABLED` and `CYPRESS_QA_DB_MONGO`
- Alternative location (directory) for the e2e copy of the sample-application.db is now set to be `e2e/tmp` since it's already gitignored
* UXW-2209 Updates datamodel tracking triggered_from to be either "data_studio" or "admin" depending on where the metadata is being edited
* Updates e2e tests
* UXW-2352 Fix issue where CopyCardForm resets when the destination collection is changed
* Unskips now-passing e2e tests
* Fix issue where Duplicate modal doesn't close on success
The previous version of the build script (`build-for-test`) had a weird quirk that could be both confusing and dangerous. It was building a backend-only JAR in the same location and under same name as the full-blown, all-in-one Metabase JAR. This means that, by running Cypress locally, one could easily override the previously built healthy JAR. It's not catastrophic but it was confusing.
This PR makes it perfectly clear what we're doing, and there should be no conflicts between the JARs any longer.
* library collection search
* Fix e2e tests from group-16
* Fix a couple tests from group 42
* Fix e2e tests from group-17
* Fix e2e tests from group-18
* tests: add bulk publishing scenario
fix wrong collection link in case of root collection
* UXW-2315, UXW-2314
* UXW-2294
* more UXW-2315, UXW-2314
* more UXW-2315, UXW-2314
* correct collection types
* Remove Browsing Memory Feature
* Revert "Revert "Don't return library collections by default from APIs""
This reverts commit 814c49473b.
* Revert "Revert "Fixed null type filtering""
This reverts commit 25e0d78408.
* fetch full tree to have library collections visible
* show library in permissions
* Fixing tests
* fix infinite re-render in tree
* rewrite unit tests and fix max depth problem
* fix: publish property not validated in the tree view after table publishing
* Move bulk editing apis to enterprise (#65968)
* Make sidebar sticky
* test: bulk attribute actions for tables, schemas and dbs
* fix more unit tests
* Copy preview section tests
* Copy error handling tests
* Copy undos tests
* Update test
* test: update new ee endpoints paths
* Make new bulk table APIs ee-only (fe)
- Closes GDGT-1251
* Fix global H.DataModel.visit overrides
* Remove redundant cy.reload
* Remove redundant cy.reload
* add select/deselect tests
* use helper
* data picker using proper library endpoint. Fixing types I broke. Endless Shame
* Fix Tree skeleton resizing multiple times during loading
- Closes GDGT-1253
* Improve icon
* Fill content
* don't call getCollectionItems on database collection
* add search tests and support shift click in search
* default table publish collection
* fox: reset bulk form values when selection changes
* Fix e2e tests from group-19
* fix: fix missing dependencies in useEffect gdgt-1245
* test: refactor test setup action order
* Fix e2e tests from group-20
* Fix e2e tests from group-22
* Fix e2e tests from group-23
* test: restore default db as the first step
* Kondo the kondo config
* default collections for metrics and models outside of data studio
* Move edit test to ee
* Fix list filtering now default is bronze
* Move bulk-edit-test to ee
* Fix table metadata for test
* Revert "Added `collection` argument to search"
This reverts commits:
- 21999e01
- b0f71c03ea
- 4fa0c3b422
* Fixed OSS loading of tables
* Fixed OSS loading of tables
* add tests for filtering
* Fix e2e tests from group 41 / joins-repros
* more tests
* Fix e2e tests from group-20
* Fix e2e tests from group-23
* Fix e2e tests from group-24
* Fix e2e tests from group-26
* Fix e2e tests from group-27
* Fix e2e tests from group-28
* Fix e2e tests from group-30
* constuct initial paths from library collections
* segments editor, hasEnabledNestedQueries
* Fix a couple more (something between groups 40 and 43)
* Fixed module config
* keyboard navigation
* Added library/tree API
* Fix some e2e tests from groups 40 and 42
* Fixing tests
* disable scoped search on the 'database collection', models.cy.spec should be passing now
* Fix e2e tests from group-33
* Minor cleanup
* Fix e2e tests from group-34
* Fixing tests
* Fix e2e tests from group 37 (except one depending on Recents tab)
* Starts fixing e2e tests from oss group
* cleanup
* wait to focus
* hide our analytics in the data studio
* Added library/tree API
* merge library and data studio
* change backend library flag validation to data-studio
* tidy things up
* Make bulk edit inputs non-clearable
- This prevents sending "empty" requests when clearing values
- And clearing does not really make sense here
* fix: test: table rescan and discard mocks are wrong
* fix: test: wrong mock in oss sync table schema
* Remove ^:parallel from bulk edit tests
They can contend and cause deadlocks
* fix: test: failure due to repeated mocking of same route
* test: refactor: cypress table tests
* Update e2e test
* Update e2e test
* Update e2e test
* fix browse nav section test
* Add missing token
* add check icon for published tables
* Removed unused import
* Updates some e2e tests from oss group, removes some now-invalid tests
* test: use default external db, skip transform test
* Starts fixing e2e tests from mongo group
* remove background from not active table in search
* specs
* Fix e2e tests from group-15
* hack fix ci while i'm on meetings
* scoped search is back
* Fix some e2e tests from group-32
* table picker fixes
* Fixed merge
* Fix e2e test from group-42
* merge fixes
* re-enable the recents tab on the data picker modal
* Avoid race condition on list-table-filtering-test
* fix embedding types
* mini picker unit tests
* Fixing tests
* ci, fixes
* fix library collection bugs
* cleanup data studio plugin
* fix lots of unit tests
* fix import
* fix
* Fixed unused import
* Fixed accidental #p
* ci
* revert util
* specs
* specs
* specs
* fix bug / bad type in SchemaFormSelect (#65995)
* Use actual anti join in GET /table query
* Remove old comments
* Activate token after restore
* test: unskip table metadata transform test
* test: fix search tables e2e test
* fix: test: selection in table picker
* Fix Source input disabled when filtering by Metabase transform
- Closes GDGT-1259
* test: fix: filter unused table
* Fix e2e test
* fix: test: datamodel toasts and errors
* Fix e2e test
* Fix e2e tests from group-9
* Fix flaky tests
* GDGT-1244: Fix DB tree virtualization (#66125)
* GDGT-1244: Fix tree virtualization. Replace top with translateY to improve performance
* GDGT-1244: Fix scroll bug. Refactor border styling
* bring back modeling empty state
* test: adjust publishing test
* Fix e2e test from group-22
* Fix e2e test from group-25
* fix allowing cross DB joins
* properly disable invalid items in MiniPicker
* Fix e2e test from group-26
* Fix straggler from oss group
* Starts fixing e2e tests from group-32
* unit test adjustments
* Fix remaining issue with e2e group 37
* fix item filtering
* Update Loki Snapshots
* add recent views to mini picker
* test recents and backparsing
* additional filtering test
* fix minipicker shows items it should not show
* Fixed :here in library API
* Fixed :here in library API
* UXW-2358 publishing outside of the data studio
* Fixed :here in library API
* snapshot
* Fix most e2e tests from group-32
* Lint fixes
* Fixed failing test
* revert fix
* Reapply "Added `collection` argument to search"
This reverts commit f28d9d5af9.
* Quick fix in CollectionPicker/utils.ts
* Fix flaky Remote Sync test (#65862)
* Clean up backend-test matrix (#66158)
* Fix timeseries value detection (#66161)
* Prevent driver tests (#66156)
Co-authored-by: Bryan Maass <bryan.maass@gmail.com>
* Fetch uberjar using a proper action (#66165)
* fix embedding column picking
* Trying CI with search_index.verified
* Fixed failing test
* Translate label
* Fixed failing test
* fix: published info not updated in the tree picker after deleting model
* fix: refactor: publish collection link tests
* Fixed failing test
* Fix remaining e2e tests from group-32
* FileUploadStatus unit test, quesiton reproductions
* create data studio plugin file
* fix hidden items some more
* Fix issue where confirmation modal closes then reopens again then closes again when you archive a dashboard
* fix dark mode
* UXW-2367
* UXW-2378
* Fixed merge issues
* this element is not visible because you can't even open incompatible sample database db
* Fix e2e test from group-14
* fix embedding data picker clicks
* filtering fixes for transforms
* UXW-2372
* update mini picker specs after a fix
* fix flaky join test
* Fixing in-place support for collections
* fix: cannot scroll search results
* ui: tweak search results checkbox spacing to match tree results
* fix: scroll position jump when entering field sort mode gdgt-1269
* Revert "fix: scroll position jump when entering field sort mode gdgt-1269"
This reverts commit bfecdec661.
* UXW-2279 Changes Data Studio's Glossary page background color to background-secondary
* Fix linters
* Fix e2e tests
* fixes for sqlserver
* fix driver changelog
* fix
* more fix
* fix: support params in bigq
* fix: inline params for sqlserver
* fix test
* fix: we do want the unique key
* fix: missing nil in test
* fix clickhouse
---------
Co-authored-by: Ryan Laurie <iethree@gmail.com>
Co-authored-by: Brad Anderson <brad@metabase.com>
Co-authored-by: Anderson <anderson@metabase.com>
Co-authored-by: Jakub Chodorowicz <jakub@chodorowicz.com>
Co-authored-by: Aleksandr Lesnenko <alxnddr@gmail.com>
Co-authored-by: Nathan Voxland <nathan@metabase.com>
Co-authored-by: Uladzimir Havenchyk <uladzimir@metabase.com>
Co-authored-by: Uladzimir Havenchyk <125459446+uladzimirdev@users.noreply.github.com>
Co-authored-by: Ngoc Khuat <qn.khuat@gmail.com>
Co-authored-by: Kamil Mielnik <kamil@kamilmielnik.com>
Co-authored-by: Nick Fitzpatrick <nickfitz.582@gmail.com>
Co-authored-by: Dan Stone <wotbrew@gmail.com>
Co-authored-by: Sloan Sparger <sloansparger@users.noreply.github.com>
Co-authored-by: Aleksandr Lesnenko <alxnddr@users.noreply.github.com>
Co-authored-by: Metabase Automation <github-automation@metabase.com>
Co-authored-by: John Swanson <john.swanson@metabase.com>
Co-authored-by: Nemanja Glumac <31325167+nemanjaglumac@users.noreply.github.com>
Co-authored-by: Denis Berezin <denis.berezin@metabase.com>
Co-authored-by: dpsutton <dan@dpsutton.com>
Co-authored-by: Bryan Maass <bryan.maass@gmail.com>
Co-authored-by: Nicola Mometto <nicola.mometto@metabase.com>
This pull request refactors and streamlines the configuration for CI workflows and file path groupings in .github/file-paths.yaml and updates related workflow logic in .github/workflows/run-tests.yml. The changes focus on consolidating shared configuration, improving maintainability, and removing redundant or unused path groups.
* Add :database_type to metabot tools that return column metadata
Add :database_type to metabot tools that return column metadata in order to give metabot information about the real
database type when generating SQL.
Examples of places the :database_type now appears:
- :used_tables in :user_is_viewing context
- {table,card,metric}-details
- get-query-details
- enhanced-database-tables
* Fix test after rebase onto #66271
Fix get-table-details-database-type-test after rebase (1) use new field-id formatting (2) get-table-details no longer returns implicitly-joinable fields
in the top-level `:fields`.
* More flexible approach to resolving columns from metabot field_ids
* fix one field-stats-test
* fix field-stats-test
* support nano-ids in field ids (?)
* remove some dead code
* Exclude implicitly joinable fields when fetching cols for table
* try switching field format to use dash
* fix field IDs in a number of tests
* Kondo
* add some unit tests for new field parsing & column lookup helper fns
* Fix more tests
* Fix more tests
* fix regex
* New approach -- use wide field IDs in related_tables and add a related_by field for each table
* Fix some tests
* remove unneeded table
* cleanup, new test
* Revert to passing in & matching on expected prefix
* Address tyler's comments
* accept a regex for the prefix as well
* add back small malli group-by fix from the last PR - removed unintentionally during revision
* Make uberjar (artifact) file name configurable via env var
Add the ability to customize the output filename for the uberjar build
via the MB_JAR_FILENAME environment variable. This allows backend-only
test builds to use a separate filename (e.g., metabase-backend.jar)
to avoid overwriting the regular all-in-one metabase.jar.
* fix: add missing channels to copy & rendering logic
Mentioned webhooks and Slack where necessary to stay consistent with the
admin UI
* fix: standardize lowercase 'admin' & prefer 'email' over 'SMTP'
* fix: remove extra comma on wording
* test: adjust e2e test assertions
* test: add `DashboardSubscriptionMenuItem.unit.spec.tsx`
* test: extract single case from `describe` block
* test: fix subscriptions e2e case & text lookup on alert spec
Fixes#66199.
`metabase.legacy-mbql.schema`, now used for normalizing MBQL 4 in
place of the old bespoke normalization in earlier versions, regressed in
this weird but legitimate use case: Writing a custom aggregation like
`MAX("String Literal")`.
That's an odd thing to do, but it can be useful to provide a label in an
aggregated query. This is the MBQL equivalent of
`(SELECT ..., 'A' as label FROM ...) UNION ALL (SELECT ..., 'B' as label ...)`
This pull request refactors the Cypress runner utility to simplify error handling and improve consistency in how test failures are reported. The main change is that runCypress now throws errors on test failures instead of relying on an external exit or cleanup function, which streamlines its usage across different scripts.
This ensures consistent error propagation and simplifies the control flow for callers.
These tools already included :database_id in the results; we now include the :database_engine as well, to give metabot
a reminder about which SQL dialect it should generate.
* Bump org.apache.arrow/arrow-memory-netty from 17.0.0 to 18.3.0
Inspect dependency changes here:
* Upgrade arrow-memory-core to version 18.3.0
Updated the version of arrow-memory-core from 17.0.0 to 18.3.0.
* Consolidate changes
* Fix formatting
---------
Co-authored-by: bryan <bryan.maass@gmail.com>
Co-authored-by: Luis Paolini <paoliniluis@gmail.com>
Co-authored-by: Cam Saul <github@camsaul.com>
Co-authored-by: Cam Saul <1455846+camsaul@users.noreply.github.com>
* Fix some e2e tests from group-9
* test: add bulk sync action e2e test
* fix column selection
* Fixed module config
* Fix a few e2e tests from oss group
* Fix e2e tests from groups 46 and 47
* Fix e2e tests from groups 44 and 45
* Fix group-3 straggler, some of group-8
* Potentially disastrous change (increases item-picker-level of SchemaList and TableList) to fix test in group-8
* Fix group-6 straggler, some more of group-8
* Updates some more e2e tests from group-9 (some actual failures left)
* Fix e2e tests from group-14
* do enterprise code correctly
* library collection search
* Fix e2e tests from group-16
* Fix a couple tests from group 42
* Fix e2e tests from group-17
* Fix e2e tests from group-18
* tests: add bulk publishing scenario
fix wrong collection link in case of root collection
* UXW-2315, UXW-2314
* UXW-2294
* more UXW-2315, UXW-2314
* more UXW-2315, UXW-2314
* correct collection types
* Remove Browsing Memory Feature
* Revert "Revert "Don't return library collections by default from APIs""
This reverts commit 814c49473b.
* Revert "Revert "Fixed null type filtering""
This reverts commit 25e0d78408.
* fetch full tree to have library collections visible
* show library in permissions
* Fixing tests
* fix infinite re-render in tree
* rewrite unit tests and fix max depth problem
* fix: publish property not validated in the tree view after table publishing
* Move bulk editing apis to enterprise (#65968)
* Make sidebar sticky
* test: bulk attribute actions for tables, schemas and dbs
* fix more unit tests
* Copy preview section tests
* Copy error handling tests
* Copy undos tests
* Update test
* test: update new ee endpoints paths
* Make new bulk table APIs ee-only (fe)
- Closes GDGT-1251
* Fix global H.DataModel.visit overrides
* Remove redundant cy.reload
* Remove redundant cy.reload
* add select/deselect tests
* use helper
* data picker using proper library endpoint. Fixing types I broke. Endless Shame
* Fix Tree skeleton resizing multiple times during loading
- Closes GDGT-1253
* Improve icon
* Fill content
* don't call getCollectionItems on database collection
* add search tests and support shift click in search
* default table publish collection
* fox: reset bulk form values when selection changes
* Fix e2e tests from group-19
* fix: fix missing dependencies in useEffect gdgt-1245
* test: refactor test setup action order
* Fix e2e tests from group-20
* Fix e2e tests from group-22
* Fix e2e tests from group-23
* test: restore default db as the first step
* Kondo the kondo config
* default collections for metrics and models outside of data studio
* Move edit test to ee
* Fix list filtering now default is bronze
* Move bulk-edit-test to ee
* Fix table metadata for test
* Revert "Added `collection` argument to search"
This reverts commits:
- 21999e01
- b0f71c03ea
- 4fa0c3b422
* Fixed OSS loading of tables
* Fixed OSS loading of tables
* add tests for filtering
* Fix e2e tests from group 41 / joins-repros
* more tests
* Fix e2e tests from group-20
* Fix e2e tests from group-23
* Fix e2e tests from group-24
* Fix e2e tests from group-26
* Fix e2e tests from group-27
* Fix e2e tests from group-28
* Fix e2e tests from group-30
* constuct initial paths from library collections
* segments editor, hasEnabledNestedQueries
* Fix a couple more (something between groups 40 and 43)
* Fixed module config
* keyboard navigation
* Added library/tree API
* Fix some e2e tests from groups 40 and 42
* Fixing tests
* disable scoped search on the 'database collection', models.cy.spec should be passing now
* Fix e2e tests from group-33
* Minor cleanup
* Fix e2e tests from group-34
* Fixing tests
* Fix e2e tests from group 37 (except one depending on Recents tab)
* Starts fixing e2e tests from oss group
* cleanup
* wait to focus
* hide our analytics in the data studio
* Added library/tree API
* merge library and data studio
* change backend library flag validation to data-studio
* tidy things up
* Make bulk edit inputs non-clearable
- This prevents sending "empty" requests when clearing values
- And clearing does not really make sense here
* fix: test: table rescan and discard mocks are wrong
* fix: test: wrong mock in oss sync table schema
* Remove ^:parallel from bulk edit tests
They can contend and cause deadlocks
* fix: test: failure due to repeated mocking of same route
* test: refactor: cypress table tests
* Update e2e test
* Update e2e test
* Update e2e test
* fix browse nav section test
* Add missing token
* add check icon for published tables
* Removed unused import
* Updates some e2e tests from oss group, removes some now-invalid tests
* test: use default external db, skip transform test
* Starts fixing e2e tests from mongo group
* remove background from not active table in search
* specs
* Fix e2e tests from group-15
* hack fix ci while i'm on meetings
* scoped search is back
* Fix some e2e tests from group-32
* table picker fixes
* Fixed merge
* Fix e2e test from group-42
* merge fixes
* re-enable the recents tab on the data picker modal
* Avoid race condition on list-table-filtering-test
* fix embedding types
* mini picker unit tests
* Fixing tests
* ci, fixes
* fix library collection bugs
* cleanup data studio plugin
* fix lots of unit tests
* fix import
* fix
* Fixed unused import
* Fixed accidental #p
* ci
* revert util
* specs
* specs
* specs
* fix bug / bad type in SchemaFormSelect (#65995)
* Use actual anti join in GET /table query
* Remove old comments
* Activate token after restore
* test: unskip table metadata transform test
* test: fix search tables e2e test
* fix: test: selection in table picker
* Fix Source input disabled when filtering by Metabase transform
- Closes GDGT-1259
* test: fix: filter unused table
* Fix e2e test
* fix: test: datamodel toasts and errors
* Fix e2e test
* Fix e2e tests from group-9
* Fix flaky tests
* GDGT-1244: Fix DB tree virtualization (#66125)
* GDGT-1244: Fix tree virtualization. Replace top with translateY to improve performance
* GDGT-1244: Fix scroll bug. Refactor border styling
* bring back modeling empty state
* test: adjust publishing test
* Fix e2e test from group-22
* Fix e2e test from group-25
* fix allowing cross DB joins
* properly disable invalid items in MiniPicker
* Fix e2e test from group-26
* Fix straggler from oss group
* Starts fixing e2e tests from group-32
* unit test adjustments
* Fix remaining issue with e2e group 37
* fix item filtering
* Update Loki Snapshots
* add recent views to mini picker
* test recents and backparsing
* additional filtering test
* fix minipicker shows items it should not show
* Fixed :here in library API
* Fixed :here in library API
* UXW-2358 publishing outside of the data studio
* Fixed :here in library API
* snapshot
* Fix most e2e tests from group-32
* Lint fixes
* Fixed failing test
* revert fix
* Reapply "Added `collection` argument to search"
This reverts commit f28d9d5af9.
* Quick fix in CollectionPicker/utils.ts
* Fix flaky Remote Sync test (#65862)
* Clean up backend-test matrix (#66158)
* Fix timeseries value detection (#66161)
* Prevent driver tests (#66156)
Co-authored-by: Bryan Maass <bryan.maass@gmail.com>
* Fetch uberjar using a proper action (#66165)
* fix embedding column picking
* Trying CI with search_index.verified
* Fixed failing test
* Translate label
* Fixed failing test
* fix: published info not updated in the tree picker after deleting model
* fix: refactor: publish collection link tests
* Fixed failing test
* Fix remaining e2e tests from group-32
* FileUploadStatus unit test, quesiton reproductions
* create data studio plugin file
* fix hidden items some more
* Fix issue where confirmation modal closes then reopens again then closes again when you archive a dashboard
* fix dark mode
* UXW-2367
* UXW-2378
* Fixed merge issues
* this element is not visible because you can't even open incompatible sample database db
* Fix e2e test from group-14
* fix embedding data picker clicks
* filtering fixes for transforms
* UXW-2372
* update mini picker specs after a fix
* fix flaky join test
* Fixing in-place support for collections
* fix: cannot scroll search results
* ui: tweak search results checkbox spacing to match tree results
* fix: scroll position jump when entering field sort mode gdgt-1269
* Revert "fix: scroll position jump when entering field sort mode gdgt-1269"
This reverts commit bfecdec661.
* UXW-2279 Changes Data Studio's Glossary page background color to background-secondary
---------
Co-authored-by: Nathan Voxland <nathan@metabase.com>
Co-authored-by: Brad Anderson <brad@metabase.com>
Co-authored-by: Jakub Chodorowicz <jakub@chodorowicz.com>
Co-authored-by: Ryan Laurie <iethree@gmail.com>
Co-authored-by: Anderson <anderson@metabase.com>
Co-authored-by: Aleksandr Lesnenko <alxnddr@gmail.com>
Co-authored-by: Uladzimir Havenchyk <uladzimir@metabase.com>
Co-authored-by: Uladzimir Havenchyk <125459446+uladzimirdev@users.noreply.github.com>
Co-authored-by: Ngoc Khuat <qn.khuat@gmail.com>
Co-authored-by: Kamil Mielnik <kamil@kamilmielnik.com>
Co-authored-by: Nick Fitzpatrick <nickfitz.582@gmail.com>
Co-authored-by: Dan Stone <wotbrew@gmail.com>
Co-authored-by: Sloan Sparger <sloansparger@users.noreply.github.com>
Co-authored-by: Stas Gavrylov <stas@metabase.com>
Co-authored-by: Aleksandr Lesnenko <alxnddr@users.noreply.github.com>
Co-authored-by: Metabase Automation <github-automation@metabase.com>
Co-authored-by: John Swanson <john.swanson@metabase.com>
Co-authored-by: Nemanja Glumac <31325167+nemanjaglumac@users.noreply.github.com>
Co-authored-by: Denis Berezin <denis.berezin@metabase.com>
Co-authored-by: dpsutton <dan@dpsutton.com>
Co-authored-by: Bryan Maass <bryan.maass@gmail.com>
* GDGT-1252: Make list view consider mini-bar-chart setting
* GDGT-1252: Fix incorrect model settings update when changing columns settings
* GDGT-1252: e2e test
* GDGT-1252: Fix unit test for ListViewItem
* GDGT-1252: Address PR comments. Minor refactoring
* GDGT-1252: Refactor DatasetEditor to get rid of modelSettings hack
* GDGT-1252: Update e2e test to cover settings persistence
* Update Loki Snapshots
* Revert Loki update
---------
Co-authored-by: Metabase Automation <github-automation@metabase.com>
### Description
Missing billing information is treated as an error for purposes of
displaying the purchase page with prices. Non-store users cannot
retrieve billing information. Thus, previously, they would always see
an error page: "Error fetching information about available add-ons."
With this change, they see the "Please ask a Metabase Store Admin" page
as intended.
We achieve this by splitting the page content for store users from that
of non-store users. Only the former needs to load data (billing and
add-on information), so only that page needs `LoadingAndErrorWrapper`.
For store users we show no dynamic content that could fail to load
(apart from a store user's email address, which we gracefully omit if
it is missing).
### How to verify
1. Ensure you have a token with the `offer-metabase-ai-tiered`
feature, but without the `metabot-v3` and `offer-metabase-ai`
features.
2. As a Metabase user with an email address that *does not* match any
of the token's organization's store user email addresses, navigate
to `/admin/metabot` and see "Please ask a Metabase Store Admin".
3. As a Metabase user with an email address that *does* match any
of the token's organization's store user email addresses, navigate
to `/admin/metabot` and see the purchase page with tier selector.
### Checklist
- [x] Tests have been added/updated to cover changes in this PR
Fixes: 2b76c38e1f
Closes: https://metaboat.slack.com/archives/C07RMUQ0GE5/p1763644357893079
This PR adds a non-destructive dev script whose only job is to detect and report on your local development environment setup.
## What it does
**`./bin/dev-requirements`** detects:
- Version managers:
- nvm, fnm, n (Node.js version managers)
- volta, asdf, mise (multi-language support)
- jenv, sdkman (Java version managers)
- Installed development tools and their versions
- Whether your setup meets Metabase's requirements
It cross-references detected versions against project configuration files:
- Node.js version from `.nvmrc`
- Clojure version from `deps.edn`
- Java 21 Temurin distribution requirement
## What it doesn't do
- No installations
- No modifications
- No assumptions about your preferences
The script is purely informational - it shows you what you have, what's required, and recommends mise for version management. Perfect for new contributors or when troubleshooting setup issues.
* feat: use boolean flag for remote-sync status
It turns out type is too overloaded to have a 'remote-synced' collection
type. This is behavior that we might want to share around other typed
collections, so this migrates us to use a boolean column for this value instead.
* fixup! e2e test find synced collection
* fixup! e2e collection fixture has correct values
* fixup! make sure flag is set on insert/update
* bump databricks
This moves over to a thin jar (or regular jar) which provides jdbc
classes and lists its dependencies. We can now bump or pin as we
like. The previous versions were uberjars that had their dependencies
packaged inside.
```clojure
clj -Sdeps '{:deps {com.databricks/databricks-jdbc-thin {:mvn/version "3.0.5"}}}'
Clojure 1.12.1
user=> (Class/forName "com.databricks.client.jdbc.Driver")
com.databricks.client.jdbc.Driver
```
```clojure
clj -Xdeps tree :extra '{:deps {com.databricks/databricks-jdbc-thin {:mvn/version "3.0.5"}}}'
org.clojure/clojure 1.12.1
. org.clojure/spec.alpha 0.5.238
. org.clojure/core.specs.alpha 0.4.74
com.databricks/databricks-jdbc-thin 3.0.5
. com.databricks/databricks-sdk-java 0.69.0
X org.slf4j/slf4j-api 2.0.9 :older-version
. org.apache.commons/commons-configuration2 2.11.0 :newer-version
X org.apache.commons/commons-lang3 3.14.0 :older-version
. org.apache.commons/commons-text 1.12.0
X org.apache.commons/commons-lang3 3.14.0 :older-version
. commons-logging/commons-logging 1.3.2 :newer-version
X com.fasterxml.jackson.core/jackson-databind 2.15.2 :older-version
...
```
contrasted with
```clojure
❯ clj -Xdeps tree :extra '{:deps {com.databricks/databricks-jdbc {:mvn/version "3.0.1"}}}'
org.clojure/clojure 1.12.1
. org.clojure/spec.alpha 0.5.238
. org.clojure/core.specs.alpha 0.4.74
com.databricks/databricks-jdbc 3.0.1
```
* bump to run drivers
* include test namespace change to trigger driver tests
* reorg
* added reinit fns to oss plugins
* ಠ_ಠ
* Added doc and eslint to prevent import of individual reinitialize functions
* added short doc to reinitialize functions to indicate they're internal
* Try to fix model references in transforms generated by metabot
* Add an e2e test
* move test later in file
* simplify implementation
* address bot comments
* add comment
...migrations.
Handles cases where emails were set to longer than the valid length of
254 bytes. Those emails would fail on this migration previously. We're
not currently using auth-identity to lookup email addresses so this
won't cause any incorrect behavior, but also if any emails like this
exist they weren't valid anyway.
Also sets the validCheckSum to ANY like some other SQL migrations to
handle cases where this migration has already been run successfully, as
on stats.
* feat: add api for manage support access grants
* Setup frontend endpoint logic and start work on UI
* Implement create access grant; revoke; UI clean up
* Move section to tools > help / Adjust main UI
* feat: migration to new auth-identity module
* Fix schema migration tests
* Adjust backend fields structure on the backend / Update types and table in the UI
* Implement invitation mechanism and auth provider
* Create a route for the access grant modal
* Fix fe types / Fix clj spaces
* Add some test placeholders
* Remove stray #p
* Add missing index
* refix broken migration
* Change the way we temporarily make the new feature always enabled
* Add email sending on grant creation
* Add login and tests
* Make auths expirable
* Remove extraneous test
* Fix bad syntax in test
* Split two column migration into two migrations
* Fix test syntax issue
* Better expires at test
* Create e2e tests / Split troubleshooting.cy.spec and move to ts
* Add fe unit tests
* Add fe unit tests
* Fix kondo
* Fix config file and settings require
* Revert added disallowed exported namespaces
* Correctly test expiration
* Add tests for revoked at behavior
* Better better expires_at test
* Fix help.cy.spec.ts / Self review updates
* Fixup last issues with module config
* Make password reset is disabled for support users
Improve some email copy
* Real token check
* Move test relying on EE-code out of OSS code
* Send invitation notification after sso user creations
* Fix module config
* fixup! BE feedback
* fixup! altered not found msg
* Use user email as fallack when name is not set on support grants table
---------
Co-authored-by: Edward Paget <ed.paget@gmail.com>
Co-authored-by: Edward Paget <edpaget@users.noreply.github.com>
* fix cut not removing text from input
* ensure we fetch all data for all possible models for smart links
* remove unused comment
* fixes smart links in metabot input sometimes degrading into their text representation and fixes some cases where wrong model name ("model" instead of "dataset") was used for metabot links
* fix unit tests
* ui for cancelling a request, handles state updates for canceled requests
* press esc to cancel metabot request, fix state being incorrect on requests after an aborted request
* add test coverage for ui, update test utils to enable aborting requests in unit tests, small bits of clean up
* add tests to processStream function
* update snapshot
Fixes#65714.
These are sent as `:number/<=` or `:number/>=` and the validation was
too strict from 57 onward. Added a test for none, min, max, and both
ends of a Between filter.
* Metabot tool endpoitns should support MBQL5
* Update filter tool to work with MBQL 5
* Fix find-outliers-wrong-query-test
Was attempting to assoc-in/dissoc-in MBQL4 query keys on an MBQL5 query.
---------
Co-authored-by: Mike Appleby <86076+appleby@users.noreply.github.com>
Running Cypress tests doesn't have to be hard.
This PR is moving us in a direction where (1) there's some logic that sets up the environment for e2e testing, and then (2) there's a simple command to invoke cypress (aka "decoupling" or "separation of concerns")
Cypress only needs to know which config to look at. It contains all the information it needs to run.
There are two ways to invoke Cypress:
1. Cypress cli
2. Cypress Module API (programmatic entry point)
They should be compatible and interchangeable. Anything you pass to the node module as options can be passed as individual flags to the cli.
```
npx cypress run --config-file path/to/some/config --spec path/to/spec.cy.spec.ts
```
is equivalent to
```
cypress.run({ configFile: "path/to/some/config", spec: "path/to/spec.cy.spec.ts" });
```
This PR removes all abstractions from the actual node module runner.
* Turn off db routing for clickhouse and it's trick.
DB routing needs a "degree of freedom" to run the exact same query
against a different schema/db/instance whatever. Clickhouse only has
database.table. so to db route on the same clickhouse instance you have
to omit the database and let it resolve during query time.
`select * from db.table` can't work. Have to do `select * from table`
and let the connection dictate db1 or db2.
In 56, tables look like this:
```clojure
{:name "upload_records_10k_20251118164620", :schema "db_eafc4346850f435e"}
```
In 57, now they look like this for single db clickhouse things:
```clojure
{:name "upload_records_10k_20251118164620", :schema nil}
```
There are two critical problems here:
1. When uploading, it's not aware to omit the schema. So a table is
created with that schema. During the next sync that table is archived
and a new one with a nil'd schema is created. So the model created for
the uploaded CSV would break as it points to an inactive table. This has
a further compounding issue because now there "appear" to be two schema:
the faked nil one and also a new one like `db_eafc...`.
2. But _all_ tables will be archived in the same manner which were
initially found in 56 or below. These tables will be marked as inactive
and new ones created. Meaning all mbql queries based on clickhouse will
fail.
I wanted to revert this change but there's actually a bit more in the PR
and some conflicts. It also does more than just enabling db-routing so I
think we can perhaps undo the damage in this way more easily.
* remove outdated test
* fix: allow analytics collection perms to be set
In #65098, we did not fully replicate the behavior of the old graph
endpoint where collection namespacing rules aren't applied to the
"analytics" namespace. Collections in this namespace can be modified as
though they are part of the root `nil` namespace. The bug was in
removing collections not part of the targetted namespace of the
`update-graph!` funciton from the graph to update.
This restores the old behavior by not excluding collection permissions
updates for collections in the "analytics" namespace in the
`remove-collections-from-other-namespaces` function.
it also updates our tests for the collection graph to make sure mock
analytics collections have the right namespace set rather than a `nil` namespace.
* fixup! reset revisions
* fixup! tests to EE module fix tests in EE module
* Update Table Editing e2e tests to use Postgres instead of SampleDB
* Update table editing e2e tests to include different column types
* Address PR comments
* move the actual sdk folders
* update rspack config and eslint config
* fix relative import in CommonSdkStoryWrapper
* allow default exports in preview.tsx
* update eslint config to allow imports from frontend/src/embedding-sdk-shared
* remove legacy eslint disable directive
* temporarily 'disable' eslint-disable-next-line no-external-references-for-sdk-package-code that has not been ported yet
* update rspack config to use new paths
* update jest config to use new paths
* add `embedding-sdk-package` to restricted imports
* copy eslint files to apply to the new sdk folders
* add eslint-disable-next-line no-restricted-imports to some places where we need emotion
* fix jest setup
* add and use setupEmbeddingSdkEnterprisePlugins
* use `useSetting` instead of `getApplicationColors` to remove enterprise import
* move defineMetabaseAuthConfig to embedding-sdk-shared
* move MetabotProvider to embedding-sdk-bundle
* fix docs generation issue about types
* fix prettier ignore file
* move back tests to ee folder
* remove ../../../ from imports
* fix eslint issues
* fix, again, jest config :sad:
* Split warehouses and warehouses-rest
* Split warehouse-schema and warehouse-schema-rest
* Split actions and actions-rest
* Split collections and collections-rest
* Split `dashboards` and `dashboards-rest`
* Split public-sharing and public-sharing-rest
* Split embedding and embedding-rest; eliminated driver deps: xrays segments sample-data tiles
* Split users and users-rest
* Split setup and setup-rest
* Some more dev functions
* Update Kondo modules config
* Fix test namespace name
* Fix test failure 🔧
* MAGE modules
* Trivial change
* Trivial change 2
* Add note
* Let's try this
* Don't rely on team label
* Update GH Action
* Fix shell script syntax
* Use same runner type for all jobs in drivers workflow
* We can skip Postgres tests too
* Fix Mage command??
* debug github output
* Test ignoring tests in Lib
* Can we skip fetching the base ref??
* Fix test-ignoring logic
* Ok so maybe we do need to fetch the base ref
* Try git-fetch --depth=1
* Fix drivers not waiting for job
* Remove TODO
* Cache Babashka dependencies
* Update step name
* Make sure Babashka deps are actually saved
* Flip the status codes around
* PR feedback and cleanup
Fixes#64135.
Previously this code would throw if multiple dashboard and/or card
params were bound to the same SQL variable. Now multiple links are
allowed, provided that only one is actually given a value.
Actually, this logic de-duplicates the values, so if multiple filters
all set the same value, that also works.
* feat: migration to new auth-identity module
* Fix schema migration tests
* Make auths expirable
* Remove extraneous test
* Fix bad syntax in test
* Split two column migration into two migrations
* Fix test syntax issue
* Better expires at test
* Better better expires_at test
* Send invitation notification after sso user creations
* Fix module config
* UXW-1866: Make MultiAutocomplete dropdown preserve open state when selecting options
* UXW-1866: Fix e2e test
* Add note on running e2e tests for translation functionality
* Fix e2e test
* Fix e2e test
Removed multiple tests related to info popovers for field picker options and saved questions.
This behavior is already covered with a unit test at:
`frontend/src/metabase/common/components/MetadataInfo/ColumnInfoIcon/ColumnInfoIcon.unit.spec.tsx`
* Fix the update keys for postgres search idx upsert
The upsert keys were calculated by looking at the first entry we were
inserting and removing `:id`, `:model`, and `:model_id`.
But the entries are not necessarily homogeneous. In this case, the first
entry is a dashboard (has no `:dashboard_id`) and the second entry is a
card (yes `:dashboard_id`).
This meant we never actually updated the dashboard_id when updating a card.
I attempted to add a test for this:
```
(deftest updates-work-properly
(mt/with-test-user :rasta
(mt/with-temp [:model/Dashboard {dash-id :id} {}]
(let [card-name (str (random-uuid))
card-id-1 (:id (mt/user-http-request :rasta :post 200 "card/" {:name card-name
:collection_id nil
:database_id 1
:display "table"
:creator_id (mt/user->id :rasta)
:visualization_settings {}
:dataset_query {:type "native" :native {:query "select 123"}
:database 1}}))
card-id-2 (:id (mt/user-http-request :rasta :post 200 "card/" {:name card-name
:collection_id nil
:database_id 1
:display "table"
:creator_id (mt/user->id :rasta)
:visualization_settings {}
:dataset_query {:type "native" :native {:query "select 123"}
:database 1}}))]
(mt/user-http-request :rasta :put 200 (str "card/" card-id-1) {:dashboard_id dash-id})
(is (= [card-name]
(map :name (:data (mt/user-http-request :rasta :get 200 "search" :q card-name)))))))))
```
The good news is that this test passes after my change. The bad news is
that it also passes without my change. I think what's going on is that
the behavior is different with `*force-sync*` on/off: in async mode, we
get the full batch that contains the updates for `[dashboard, card]`.
In synchronous mode, we get separate batches for `[dashboard]` and then
`[card]`.
I think we might want to rework tests somehow - it would be quite nice
to be able to still run things asynchronously but drain the queue before
making assertions, so the behavior wouldn't be different between
tests/prod.
* add a test
* replace password with key pair auth
* re enable snowflake tests
* add private key env var
* update private key details for temp dbs
* small changes
* fix describe database test
* fix multi method params
* update test
* update test
* try fix
* check CI
i noticed that this tests pass if we don't put a connection-uri in
there. I'm not sure what the original intent was, but the following
works just fine:
```
{:url "jdbc❄️//<account>.snowflakecomputing.com/?schema=INFORMATION_SCHEMA",
:properties {"use-password" "false",
"connection-uri" "jdbc❄️//<account>.snowflakecomputing.com/?user=<user>CI&private_key_file=%2Fprivate%2Fvar%2Ffolders%2F1d%2F3ns5s1gs7xjgb09bh1yb6wpc0000gn%2FT%2Fmetabase-secret_8808579641027884666.tmp&schema=INFORMATION_SCHEMA",
"client_session_keep_alive" "true",
"timezone" "UTC",
"warehouse" "<warehouse>",
"user" "<user>",
"ssl" "true",
"private_key_file" "/var/folders/1d/3ns5s1gs7xjgb09bh1yb6wpc0000gn/T/metabase-secret_8808579641027884666.tmp",
"week_start" "7",
"application" "Metabase_Metabase",
"account" "<account>",
"db" "\"<db>\"",
"time_output_format" "HH24:MI:SS.FF3",
"client_metadata_request_use_connection_ctx" "true"}}
```
When it goes through clojure.java.jdbc. Somehow we are getting a
connection string that doesn't play nice. I _think_ it may just be an
issue with snowflake but i'm not positive.
* remove parallel test markers
* remove parallel marker
when running tests locally with these removed:
```clojure
(run/find-and-run-tests-repl {:only 'metabase.driver.snowflake-test})
...
Ran 49 tests in 116.525 seconds
502 assertions, 1 failure, 0 errors.
{:test 49, :pass 501, :fail 1, :error 0, :type :summary, :duration 116525.006833, :single-threaded 32, :parallel 17}
```
yields only one expected error. Previously using regular stuff also
worked
```clojure
snowflake-test=> (run-tests)
Testing metabase.driver.snowflake-test
Ran 49 tests containing 502 assertions.
1 failures, 0 errors.
{:test 49, :pass 501, :fail 1, :error 0, :type :summary}
```
(again the single test that failed). but using the CLI tool i would get
errors when running them in parallel.
* fixup stupid snowflake
1. i don't know why we are running into multithreading issues now. But
i've removed those tags and the issues are gone.
2.
```clojure
snowflake-test=> (let [spec {:connection-uri "jdbc❄️//account.snowflakecomputing.com/?schema=INFORMATION_SCHEMA&user=<user>&password=<password>&db=%22v5_airports%22"}]
[(jdbc/query spec ["select CURRENT_DATABASE() db"])
(jdbc/query spec ["select CURRENT_SCHEMA() db"])])
[({:db "v5_airports"}) ({:db "INFORMATION_SCHEMA"})]
snowflake-test=> (let [spec {:connection-uri "jdbc❄️//account.snowflakecomputing.com/?schema=INFORMATION_SCHEMA&user=<user>&password=<password>&db=v5_airports"}]
[(jdbc/query spec ["select CURRENT_DATABASE() db"])
(jdbc/query spec ["select CURRENT_SCHEMA() db"])])
[({:db nil}) ({:db nil})]
```
In the above, note that putting the db in the connection string _and
url-encoding it_ (it's already quoted) is necessary.
* conditionally add db
in tests there's not a database so just be smart about it:
```clojure
{:warehouse "<warehouse>",
:additional-options nil,
:timezone "UTC",
:db nil,
:account "<account>",
:use-password false,
:time_output_format "HH24:MI:SS.FF3",
:user "<user>",
:subname "//<account>.snowflakecomputing.com/"}
```
* address pr comments
* remove outdated comment
pretty freaky comment eh? I added this when i commented out the
`handle-conn-uri` call underneath it and it caused tests to pass. That
was helpful in identifying the mechanism, but not a good solution. Put a
comment here so that that was not a final state, forgot to remove
comment when we got to a proper final state and `handle-conn-uri` was
fixed.
---------
Co-authored-by: dan sutton <dan@dpsutton.com>
Fixes#65726.
If you implicitly joined a field that happened to be the target of a
remapping, the QP was labeling it as `:remapped_from` the FK even
though (a) it wasn't really remapped and (b) there was no
corresponding FK column marked as `:remapped_to` the target column.
(Eg. `Orders.PRODUCT_ID` is remapped to `Products.TITLE`. Then I
implicitly join `Products.TITLE` directly and the QP marks it as
remapped from `Orders.PRODUCT_ID` even though it wasn't.)
That caused the FE to do odd things like drop and relabel columns.
This change adjusts the QP so it will ignore plausible remappings that
don't actually get returned from the last stage.
We get a bunch of this useless stuff in CI:
```
Placeholder until other changesets
UPDATE SUMMARY
Run: 641
Previously run: 0
Filtered out: 69
-------------------------------
Total change sets: 710
FILTERED CHANGE SETS SUMMARY
DBMS mismatch: 69
```
Looking for this in liquibase this is not logged but printed to the
console :(.
It goes through this:
```java
private static void writeMessage(String message, UpdateSummaryOutputEnum showSummaryOutput, OutputStream outputStream) throws LiquibaseException {
switch (showSummaryOutput) {
case CONSOLE:
writeToOutput(outputStream, message);
break;
case LOG:
writeToLog(message);
break;
default:
writeToOutput(outputStream, message);
writeToLog(message);
}
}
```
So we want that enum to be LOG.
```clojure
liquibase=> (clojure.reflect/reflect liquibase.Liquibase)
{:bases #{java.lang.Object java.lang.AutoCloseable},
:flags #{:public},
:members #{ ,,,
#clojure.reflect.Method{:name setShowSummaryOutput,
:return-type void,
:declaring-class liquibase.Liquibase,
:parameter-types [liquibase.UpdateSummaryOutputEnum],
:exception-types [],
:flags #{:public}}
,,,}}
```
So easy fix. Set this enum and hopefully it shuts up.
* claude fix
* simplify
* handle nested fields for athena
* nfc have position 0
* set non leafs as details only
* clean up parser and add tests
* whitespace
* disable nested fields
* kondo
* kondo
* disable nested schema test
* implement plugins for auth
* move some stuff to the oss folder as those file create issues later on
* make embedding-sdk:docs:generate pass
* move metabot to use plugins
* remove imports from package from other folders
* Revert "Adjust cards-with-filters to work with action cards (#65805)"
This reverts commit d6b857a2c6.
* Revert "Fix mapping of params to later query stages (#65651)"
This reverts commit c98e640426.
* Revert "Compute param values from card results (#64318)"
This reverts commit 2693173e7f.
* Fix SDK save dashboard question to not show collection/dashboard picker
* Fix EditableDashboard sometimes doesn't show the correct tab on first render
* Add an assertion
* left some breadcrumbs
* Update unit tests to be async
* Fix code that populates :used_tables in metabot context
This was causing :used_tables to be absent in the :user_is_viewing
context, limiting metabot's ability to get sql fixing right without
information about the available tables and their fields.
Call lib-be/normalize-query and lib/native-only-query? in
metabase-enterprise.metabot-v3.context/query-for-sql-parsing and
likewise use lib funcs to get template-tags in
metabase-enterprise.metabot-v3.query-analyzer.parameter-substitution/replace-tags
* Update ai-sql-fixer and tests to work with MBQL5 queries
* Fix metabot-v3.table-utils-test to use MBQL5 queries
* Add click behavior setup information to introduction
* Update docs/embedding/sdk/introduction.md
Co-authored-by: Jeff Bruemmer <jeff.bruemmer@gmail.com>
---------
Co-authored-by: Jeff Bruemmer <jeff.bruemmer@gmail.com>
* Update screen reader description to match the new menu copy
* Update frontend/src/metabase/embedding/components/ArchiveButton.tsx
Co-authored-by: metabase-bot[bot] <109303359+metabase-bot[bot]@users.noreply.github.com>
---------
Co-authored-by: metabase-bot[bot] <109303359+metabase-bot[bot]@users.noreply.github.com>
When we introduced a new e2e run mode, we didn't realize this piece of code needs to be updated as well.
For people who were running the backend from source (but using e2e mode) some drivers might not have worked. Example:
```
"message": "No method in multimethod 'connection-properties' for dispatch value: :sqlite"
```
This commit fixes that.
Cloud never used this to pre-fill the user on MB setup.
We have the same functionality with way less code available today, either way.
Instead of:
```
/setup/user_defaults#123456
```
do this:
```
/setup?first_name=John&last_name=Doe
```
### Description
In #62513 we added a page to offer a Metabase AI (Metabot) trial. Now
we want to offer to purchase the paid Metabot add-on.
### How to verify
1. Ensure you have a token with the `offer-metabase-ai-tiered`
feature, but without the `metabot-v3` and `offer-metabase-ai`
features.
2. Ensure you have a Metabase API key linked to that same token.
3. Navigate to `/admin/metabot` and see a page offering you to purchase
different quantities of Metabot queries.
4. Select a quantity (or go with the default of 500), check "I agree",
then click "confirm purchase".
5. Confirm that your token now has the `metabot-v3` feature and the
associated subscription has an appropriate Metabase AI product.
### Checklist
- [x] Tests have been added/updated to cover changes in this PR
### Notes
* Rename MetabotPurchasePage to MetabotTrialPage
- and add the page for the purchase of the paid add-on as
`MetabotPurchasePage`, to make the name more clearly represent the
purpose (paid vs. trial).
* Move frontend client for /api/ee/cloud-add-ons into metabase-enterprise/api
* Use ts-pattern instead of hand-crafted type guards in Metabot trial page
* Don't swallow errors when submitting Metabot purchase form
* Create Redux selector to derive isStoreUser
* Test failures to purchase Metabot trial
* Adjust implementation of POST /api/ee/cloud-add-ons/:product-type to dashes instead of underscores
* Require accepting terms of service for Metabot trial
* Move RTK and Formik error handling for Metabot into utils.ts
* Avoid use of any type in Metabot Formik error handling
* Improve control flow with side effects
* Fix typo in log message
Closes: https://linear.app/metabase/issue/CLO-4423/allow-to-buy-metabase-ai-from-the-admin-under-a-token-feature
References: https://github.com/metabase/metabase/pull/62513
Co-authored-by: Benoit Vinay <ben@metabase.com>
* Update the delete button copy for better UX
* Apply suggestions from code review
Co-authored-by: Phoomparin Mano <poom@metabase.com>
---------
Co-authored-by: Phoomparin Mano <poom@metabase.com>
* Convert plugin system to deferred initialization with undo capability
* fixed wrong rebase
* better initialization
* .
* fixed unit tests
* .
* duh
* added back whitelabel overrides for SDK
* fixup
* fixed unit test
* cleanup
* UXW-2235 Prevents 403s from SmartLinks from preventing the whole document from showing (SmartLink renders with "No access" version instead)
* UXW-2235 Adds e2e tests
* Add /livez and /readyz endpoints
* nit comment
* Update tests with new endpoints
* Update src/metabase/server/routes.clj
Co-authored-by: metabase-bot[bot] <109303359+metabase-bot[bot]@users.noreply.github.com>
* bad bot
* fix tests
* fix license test failure
https://github.com/metabase/metabase/pull/65034 bumped some jetty deps
from 12.0.24 to 12.1.3. Along for the ride came a new org:
```
org.eclipse.jetty.ee9.websocket/jetty-ee9-websocket-jetty-server 12.1.3
X org.eclipse.jetty.ee9/jetty-ee9-servlet 12.1.3 :use-top
. org.eclipse.jetty.ee9/jetty-ee9-webapp 12.1.3
. org.eclipse.jetty/jetty-xml 12.1.3
. org.eclipse.jetty/jetty-util 12.1.3
X org.slf4j/slf4j-api 2.0.17 :use-top
. org.eclipse.jetty.ee/jetty-ee-webapp 12.1.3 <--- new
```
This has a new org of `org.eclipse.jetty.ee` and no discernable license
in the pom or the jar.
Refresher we look in the pom for things like
```
<licenses>
<license>
<name>The Apache Software License, Version 2.0</name>
<url>http://www.apache.org/licenses/LICENSE-2.0</url>
<distribution>repo</distribution>
</license>
</licenses>
```
or in the jar for some common names of licenses, like in this mysql
connector jar:
```
-rw-r--r-- 134 18-Sep-2023 19:11:56 INFO_SRC
-rw-r--r-- 71049 18-Sep-2023 19:11:56 LICENSE
-rw-r--r-- 1624 18-Sep-2023 19:11:56 README
-rw-r--r-- 846 18-Sep-2023 19:12:04 META-INF/INDEX.LIST
```
I didn't see a license at either of these spots, but consulted
https://mvnrepository.com/artifact/org.eclipse.jetty.ee/jetty-ee-webapp
to see it listed as Apache 2.0 and EPL 2.0
---------
Co-authored-by: metabase-bot[bot] <109303359+metabase-bot[bot]@users.noreply.github.com>
Co-authored-by: dan sutton <dan@dpsutton.com>
Part of #64774.
Replaces quadratic (refs * columns) search in `add-alias-info` with map lookups
by ID and `:lib/source-column-alias`. They don't always succeed, but I ran the tests with a fallback case
that checked the old quadratic way too - in no case does the new match fail but the old way succeed.
- Speeds up the basic query of wide tables (see `qp.perf-test`) about 7x, from 8825ms to 1435ms.
- Reduces the big user query from #64774 from 205ms to 145ms on my machine.
Transfer-Encoding: chunked will cause Apache HttpClient to wrap body in ContentLengthInputStream,
which, in turn, quote: "NEVER closes the underlying stream".
So 'with-open' closing the stream was never propagated. And the new test with transfer-encoding
header captures that.
Part of #64774.
Now that most of the query processor machinery has been ported to use MBQL 5 and the `metabase.lib.core`
functionality, there were several needless round trips between MBQL 4 and 5.
The `driver.sql.qp/preprocess` function was converting queries from MBQL 5 to 4 up front,
and then each of three inner operations (now implemented in MBQL 5) were wrapped to convert 4 to 5 and back.
Removing this wasted effort reduced the time on my machine for the big user query from #64774 from 205ms to 145ms.
* fix cut not removing text from input
* ensure we fetch all data for all possible models for smart links
* remove unused comment
* fixes smart links in metabot input sometimes degrading into their text representation and fixes some cases where wrong model name ("model" instead of "dataset") was used for metabot links
* fix unit tests
* fix phantom selection when cutting value from input (context: https://github.com/metabase/metabase/pull/65255\#pullrequestreview-3436250849)
* ditch our manual batches of rows for bufferedwriter
Q: Why did we use a manual batch at first?
A: i noticed it was super slow and worked around this. Then realized i
was gzipping it so that was the actual slowdown. The batching was
already done and it didn't hurt anything.
Q: Why get rid of it then?
A: want to move to "cell" based counting. Rows are fine until they get
really wide and then 300 columns * 5,000 rows = 1,500,000 cells in
memory at once. Put some strings in there and we blow up. So lets switch
to cell counting rather than row counting. This is way easier if we only
care about number of rows or cells when holding them in memory. Don't
have to count the batch ourself when streaming. So if that goes row by
row, we just watch for the number of cells as the trigger to switch to
streaming and we are done.
```clojure
temp-storage-test=> (time
(mt/test-driver :postgres
(select-keys (mt/process-query (mt/native-query {:query "SELECT
row_num AS id,
-- Random Integer between 1 and 100000
floor(random() * 100000 + 1)::INTEGER AS random_int,
-- Random Float
(random() * 1000)::NUMERIC(10, 4) AS random_float,
-- Random Date within the last 5 years
(current_date - (random() * 5 * 365)::INTEGER)::DATE AS random_date,
-- Random Timestamp within the last 30 days
(NOW() - (random() * 30 * '1 day'::interval)) AS random_timestamp,
-- Random String: Generates a random alphanumeric string about 1000 characters long
(
SELECT string_agg(
chr(
(CASE WHEN random() < 0.5 THEN 65 ELSE 97 END) + (random() * 25)::INTEGER
),
''
)
FROM generate_series(1, 1000)
) AS random_string
FROM
generate_series(1, 500000) AS row_num;"})
(temp-storage/notification-rff 2000 :testing))
[:row_count :status :data.rows-file-size :notification/truncated?])))
"Elapsed time: 1457.921083 msecs"
{:row_count 15184,
:status :completed,
:data.rows-file-size 16369108,
:notification/truncated? true}
```
* Switch to cell based counts, not rows
We found in practice some people might have queries with 300
columns. When we keep 5,000 of those in memory, we end up having
1,500,000 cells in memory at once. We then are open again to OOM
issues. So this swaps over to a cell basec count rather than rows to be
a bit more granular.
* Change from 2,000 rows to 20,000 cells
the thinking here is 2,000 rows with 10 values. It's pretty arbitrary though
* rename var for cells to disk threshold
* i have to delete this test because i used `time` in a comment block
* formatting fix
If you have an empty first stage followed by "real" later stages, the frontend produces duplicate buttons in the query
builder. Cleaning those stages up removes those buttons.
* BE modularization: decouple `server` from `core`
* Split up enterprise/api and enterprise/api-routes
* Split up `settings` and `settings-rest`
* Dox for new util
* Add another util function
* Split `permissions` and `permissions-rest`
* Code fixes and additional utils
* Improved deps graph dox
When a join's conditions came from multiple tables, we used the first table as the table name, even though some of the cols came from a different table. We now display "Previous results" to denote that the lhs columns are from different tables.
* fix: update harbormaster API use
The Harbormaster API was updated to use snake_case in all params
and returns in https://github.com/metabase/harbormaster/pull/6663.
There's an backwards compatible interim period where it will support both kebab-case and snake_case in params, and keep existing returns.
The changes in this PR will make Metabase forwards and backwards compatible with the Harbormaster API changes.
After the interim period, the Harbormaster API will only take and return snake_case data.
* feat: support missing api key on hm.client/call
The OpenAPI spec is now public and the api key is not mandatory.
* new click and link behavior
* Apply suggestions from code review
Co-authored-by: Jeff Bruemmer <jeff.bruemmer@gmail.com>
* handle handlelink
* fix typo
---------
Co-authored-by: Jeff Bruemmer <jeff.bruemmer@gmail.com>
This has been a very valuable benchmark for processing complex queries,
especially large `:case` statements and nested aggregations.
Adding it with some REPL-able benchmarks in `query-processor.perf-test`.
* reverts the color-scheme meta tag
* adds the color scheme meta tag if entry point is not embed-sdk
* add backend tests for color-scheme meta tags
* add a cypress test for color-scheme
- catalog metadata is not transactional in redshift
- calls like pg_catalog.has_schema_privilege when using a string argument will resolve the oid of the schema/table etc
- if the oid cannot be resolved (e.g. table/schema does not exist) an error is thrown
We occasionally see sync failed due to relation does not exist, one cause might be concurrent test runs / cleanup deleting tables/schemas while running.
I have fixed the first half of the union to use the oid form of these calls which will result in NULL instead of an error while racing with DDL in other runs.
Need to research on external tables before changing the external table sub-query: probably want to join to svv_external_schemas and use the oid there, but not 100% yet.
* Fix docs
it throws an error in the js console about webassembly not having the
right permissions
Two ways to fix: give permissions, or rollback to a version that works.
I'm pinning to a version that works since we are close to a release.
39.2 breaks, 39.1 works.
I'm also including the recommended stylesheets
In the meantime, i'm going to investigate
resources/openapi/index_external.html.mustache as i'm not sure what uses
this and if it needs to be updated or deleted. Perhaps used by `clojure
-M:ee:doc api-documentation`?
* switch template to pinned 39.1 version
* Add breaking changes detection information
Added section on checking for breaking changes when saving questions, models, or metrics.
* Add breaking changes check to model editing documentation
Added section on checking for breaking changes when saving a model.
* Apply suggestions from code review
Co-authored-by: Jeff Bruemmer <jeff.bruemmer@gmail.com>
---------
Co-authored-by: Jeff Bruemmer <jeff.bruemmer@gmail.com>
* add aws jdbc wrapper
* add aws-iam option to auth-provider-options
* WIP: hook ssl iam in postgres
* wip
* working
* style
* postgres appdb aws iam support
* auth-provider-options-only-aws
* aws iam for mysql
* ssl
* warnings
* as error
* support appdb uri
* add unit tests
* warning
* don't automatically include useSSL
* support ssl-cert configs for mysql appdb
* more unit tests
* refactor
* integration test
* fix: always set useSSL for aws-iam in mysql appdb
* fix mysql uri
* chore
* style
* fix
* connection tests
* setup for CI
* temporarily disable until secrets are setup
* chore: set
* add warning
* Revert "temporarily disable until secrets are setup"
This reverts commit 4ca6eb4607.
* show new library icon
* UXW-2115 Address PR feedback - makes it so only the Library uses the synced collection icon, not the nested subdirectories
* UXW-2115 Fix getCollectionType condition
* UXW-2115 Removes unnecessary type changes
* UXW-2115 Reverts changes for synced collection icon
---------
Co-authored-by: Brad Anderson <brad@metabase.com>
* refactor HomeGreeting and convert to use css module over styled components
* update colors for metabot illustration
* bad ai
* rename
* pr feedback
* small adjustments
* spark failure fix
I cannot run spark locally so from CI:
```
FAIL in metabase.driver.sparksql-test/friendly-inline-strings-in-convert-to-sql-test (sparksql_test.clj:67)
:sparksql
The native query returned in query results should use user-friendly splicing
expected: "SELECT COUNT(*) AS `count` FROM `test_data`.`venues` AS `t1` WHERE `t1`.`name` = 'wow'"
actual: "SELECT COUNT(*) AS `count` FROM `test_data`.`venues` AS `t1` WHERE `t1`.`name` = 'wow'"
diff:
expected: "SELECT COUNT(*) AS `count` FROM `test_data`.`venues` AS `t1` WHERE `t1`.`name` = 'wow'"
actual: "SELECT COUNT(*) AS `count` FROM `test_data`.`venues` AS `t1` WHERE `t1`.`name` = decode(unhex('776f77'), 'utf-8')"
```
Locally hit this on an m1 mac:
```
impl.get-or-create :: create-database! failed; destroying :sparksql database “test-data” {mb-test=metabase.driver.sparksql-test/friendly-inline-strings-in-convert-to-sql-test}
java.util.concurrent.TimeoutException: Timed out after 30.0 mins
```
* fix license test failure
https://github.com/metabase/metabase/pull/65034 bumped some jetty deps
from 12.0.24 to 12.1.3. Along for the ride came a new org:
```
org.eclipse.jetty.ee9.websocket/jetty-ee9-websocket-jetty-server 12.1.3
X org.eclipse.jetty.ee9/jetty-ee9-servlet 12.1.3 :use-top
. org.eclipse.jetty.ee9/jetty-ee9-webapp 12.1.3
. org.eclipse.jetty/jetty-xml 12.1.3
. org.eclipse.jetty/jetty-util 12.1.3
X org.slf4j/slf4j-api 2.0.17 :use-top
. org.eclipse.jetty.ee/jetty-ee-webapp 12.1.3 <--- new
```
This has a new org of `org.eclipse.jetty.ee` and no discernable license
in the pom or the jar.
Refresher we look in the pom for things like
```
<licenses>
<license>
<name>The Apache Software License, Version 2.0</name>
<url>http://www.apache.org/licenses/LICENSE-2.0</url>
<distribution>repo</distribution>
</license>
</licenses>
```
or in the jar for some common names of licenses, like in this mysql
connector jar:
```
-rw-r--r-- 134 18-Sep-2023 19:11:56 INFO_SRC
-rw-r--r-- 71049 18-Sep-2023 19:11:56 LICENSE
-rw-r--r-- 1624 18-Sep-2023 19:11:56 README
-rw-r--r-- 846 18-Sep-2023 19:12:04 META-INF/INDEX.LIST
```
I didn't see a license at either of these spots, but consulted
https://mvnrepository.com/artifact/org.eclipse.jetty.ee/jetty-ee-webapp
to see it listed as Apache 2.0 and EPL 2.0
* Make Trunk analytics always run
This prevents false positives because Trunk analytics actio will now
always analyze test results, and compare them with their own test
quarantine list. It uses this information to know if it should fail the
job or let it pass.
* Introduce dry-run
When a PR is in a draft mode, we don't want to upload results but still
have to quarantine tests.
* Update wording for subscriptions and alerts section
* Update docs/permissions/application.md
Co-authored-by: Jeff Bruemmer <jeff.bruemmer@gmail.com>
---------
Co-authored-by: Jeff Bruemmer <jeff.bruemmer@gmail.com>
* toggle dark mode from command palette
* I'm sorry I built this in a dumb way Ryan
---------
Co-authored-by: Nick Fitzpatrick <nickfitz.582@gmail.com>
* upgrade to echarts v6
* Update Loki Snapshots
* Fix BE test
* update gridlines helper function
* adjust test
---------
Co-authored-by: Metabase Automation <github-automation@metabase.com>
Co-authored-by: Nick Fitzpatrick <nickfitz.582@gmail.com>
* Make sure /dirty is called when moving an entity out of the library
* Untouch embedding code / Clean up / Add some tests
* Check cache from both entities and RTK / Undo unnecessary changes
* Make sure getCollectionFromCollectionsTree gets collection from latest query
* ci: allow running stress tests on sdk tests
* type choice
* yeah this should work 🤞
* ok we should also build the sdk
* maybe this time
* do we need to setup the BE?
* Revert "do we need to setup the BE?"
This reverts commit 7e9092e2c3.
* trying the auto infer thing out
* refactor and move the logic up to dedup it
* rename output
* move up env?
* Revert "move up env?"
This reverts commit f78f72e396.
* increase timeout to make stress test with throttling pass
* increase the timeout even more :shrugs:
Part of #64774.
Hopefully this is sufficient to unblock users seeing the high CPU times
like in #64774. I discovered other, less significant performance issues
during this investigation that I'll be addressing in follow-up PRs.
### Investigation Notes
The problem query and relevant table metadata from a helpful customer
was taking 950ms to preprocess and compile on my 2022 M1 Max, running
Metabase in dev mode but with Malli schema enforcement disabled.
The query is pretty big! It uses 3 tables with 200-300 columns each,
has several stages including some from a joined card, and ends up with
several stages doing two levels of aggregation. It also includes
several *huge* `:case` expressions.
The optimizations below have reduce the time required from 950ms to 310ms
on my machine. A 3x improvement, but I think there's more to be gained in
the follow-up PRs I mentioned.
### Optimizations
Main optimizations:
- `lib.util/find-stage-index-and-clause-by-uuid` pre-computed and cached
on each stage, saved 120ms on the above query.
- `lib.metadata.cached-provider/fields` attempts to pre-load the cache
of individual `lib.metadata/field` calls, having just fetched them
all, but it used the wrong key and they were all misses instead!
- `expression-metadata` and `aggregation-metadata` are recomputed over
and over again by their stage and later stages; cache them by value
- `returned-columns` was cached at the top level and frequently missed
because of dumb details like explicit stage 1 vs. -1 for last stage.
- Switched it to caching on individual `returned-columns-method`s
with more precise cache keys.
#### New Caching Approach
Both of the latter two are using a new caching approach: a weak map
using queries as keys. Since CLJ(S) maps are immutable, any change to
the query means a new pointer.
That's still far more eviction than optimal! Most edits to a query
don't actually invalidate most of the cached values! But it's a lot
better than what we had before, and because the map's keys are weak
references, the queries and their cached data are not retained any
longer than the query itself.
### New Benchmarking Tools
I also added a `qp.perf-test` namespace with some metadata providers
with 10-, 100-, 1000- and 10,000-column tables for testing.
So far the tests it performs are quite basic, though. I'll be adapting
the user-provided query into a test case in the future, since it's an
excellent real-world example of a complex but not pathological query.
```
Updating table metadata for Table 5465 ''properati_ar.properties_rent_201607'' {mb-quartz-job-type=SyncAndAnalyzeDatabase}
:is_writable of Table 5465 ''properati_ar.properties_rent_201607'' changed from null to null {mb-quartz-job-type=SyncAndAnalyzeDatabase}
Updating table metadata for Table 5485 ''properati_ar.properties_sell_201706'' {mb-quartz-job-type=SyncAndAnalyzeDatabase}
:is_writable of Table 5485 ''properati_ar.properties_sell_201706'' changed from null to null {mb-quartz-job-type=SyncAndAnalyzeDatabase}
Updating table metadata for Table 5468 ''properati_ar.properties_rent_201602'' {mb-quartz-job-type=SyncAndAnalyzeDatabase}
:is_writable of Table 5468 ''properati_ar.properties_rent_201602'' changed from null to null {mb-quartz-job-type=SyncAndAnalyzeDatabase}
Updating table metadata for Table 5511 ''properati_ar.properties_rent_201612'' {mb-quartz-job-type=SyncAndAnalyzeDatabase}
:is_writable of Table 5511 ''properati_ar.properties_rent_201612'' changed from null to null {mb-quartz-job-type=SyncAndAnalyzeDatabase}
Updating table metadata for Table 5431 ''properati_ar.properties_sell_201802'' {mb-quartz-job-type=SyncAndAnalyzeDatabase}
:is_writable of Table 5431 ''properati_ar.properties_sell_201802'' changed from null to null {mb-quartz-job-type=SyncAndAnalyzeDatabase}
Updating table metadata for Table 5441 ''properati_ar.properties_sell_201707'' {mb-quartz-job-type=SyncAndAnalyzeDatabase}
:is_writable of Table 5441 ''properati_ar.properties_sell_201707'' changed from null to null {mb-quartz-job-type=SyncAndAnalyzeDatabase}
Updating table metadata for Table 5467 ''personal_bq_repro.events_20230519'' {mb-quartz-job-type=SyncAndAnalyzeDatabase}
:is_writable of Table 5467 ''personal_bq_repro.events_20230519'' changed from null to null {mb-quartz-job-type=SyncAndAnalyzeDatabase}
Updating table metadata for Table 5484 ''properati_ar.properties_sell_201708'' {mb-quartz-job-type=SyncAndAnalyzeDatabase}
:is_writable of Table 5484 ''properati_ar.properties_sell_201708'' changed from null to null {mb-quartz-job-type=SyncAndAnalyzeDatabase}
Updating table metadata for Table 5434 ''properati_ar.resultados_apto_credito'' {mb-quartz-job-type=SyncAndAnalyzeDatabase}
:is_writable of Table 5434 ''properati_ar.resultados_apto_credito'' changed from null to null {mb-quartz-job-type=SyncAndAnalyzeDatabase}
Updating table metadata for Table 5501 ''properati_ar.properties_sell_201704'' {mb-quartz-job-type=SyncAndAnalyzeDatabase}
:is_writable of Table 5501 ''properati_ar.properties_sell_201704'' changed from null to null {mb-quartz-job-type=SyncAndAnalyzeDatabase}
Updating table metadata for Table 5433 ''properati_ar.properties_rent_201604'' {mb-quartz-job-type=SyncAndAnalyzeDatabase}
:is_writable of Table 5433 ''properati_ar.properties_rent_201604'' changed from null to null {mb-quartz-job-type=SyncAndAnalyzeDatabase}
Updating table metadata for Table 5478 ''properati_ar.properties_sell_201702'' {mb-quartz-job-type=SyncAndAnalyzeDatabase}
:is_writable of Table 5478 ''properati_ar.properties_sell_201702'' changed from null to null {mb-quartz-job-type=SyncAndAnalyzeDatabase}
Updating table metadata for Table 5498 ''properati_ar.properties_sell_201512'' {mb-quartz-job-type=SyncAndAnalyzeDatabase}
:is_writable of Table 5498 ''properati_ar.properties_sell_201512'' changed from null to null {mb-quartz-job-type=SyncAndAnalyzeDatabase}
Updating table metadata for Table 5447 ''properati_ar.properties_rent_201709'' {mb-quartz-job-type=SyncAndAnalyzeDatabase}
:is_writable of Table 5447 ''properati_ar.properties_rent_201709'' changed from null to null {mb-quartz-job-type=SyncAndAnalyzeDatabase}
Updating table metadata for Table 5463 ''properati_ar.properties_rent_201609'' {mb-quartz-job-type=SyncAndAnalyzeDatabase}
:is_writable of Table 5463 ''properati_ar.properties_rent_201609'' changed from null to null {mb-quartz-job-type=SyncAndAnalyzeDatabase}
Updating table metadata for Table 5448 ''properati_ar.properties_rent_201509'' {mb-quartz-job-type=SyncAndAnalyzeDatabase}
:is_writable of Table 5448 ''properati_ar.properties_rent_201509'' changed from null to null {mb-quartz-job-type=SyncAndAnalyzeDatabase}
```
We unconditionally say we are updating table metadata, but then when we
actually change anything we of course include the table name.
Note here there is some kind of bug that we are changing the
table (which property??) from null to null.
* Updates 35 SVG illustrations
The same asset now works well for both light and dark mode.
* branding OnboardingIllustration, adding cloug and bug metabot to metabot logo
* PR Feedback
---------
Co-authored-by: Nick Fitzpatrick <nickfitz.582@gmail.com>
* Annotate draft transforms in user_is_viewing context with source type when possible
* add unit tests
* Move transform-source-type helper to transform utils; some refactor
* switch to use lib/native-only-query; change default transform query in tests to use mbql5
* switch batch to using lib.query/native?
* switch back to native-only-query?
* clean ns
* Compute param values from card results
seq for bool
Tests, multi-stage queries
Compute values also for params on nested stages
whitespace
Unused ns
Last stage check first
Docstring
Drop base query's last stage clauses
Address review remarks
Apply suggestions from code review
Co-authored-by: metamben <103100869+metamben@users.noreply.github.com>
Remove redundant ops
Adjust for test
Search for col also in aggregated result columns
Look for breakoutable column only if source-uuid is present
* symb
* Rebase fix
* fix rebase, remove unused module
* post rebase, fix getting database id
* kondo
* Revamp
* limit
* Adjust the solution for impl joins
* Fix e2e
* Comment
* Cleanup
* Cleanup
* comment
* comment
* Update src/metabase/parameters/dashboard.clj
Co-authored-by: metamben <103100869+metamben@users.noreply.github.com>
* indent
---------
Co-authored-by: metamben <103100869+metamben@users.noreply.github.com>
* UXW-1991 Fixes various issues both new and old with the Branding page like the colors not updating until refresh and swatches not updating when setting colors back to their defaults
* UXW-1991 Uses hex format for brand colors placeholders
* UXW-1991 Fix persistence issues with setting/unsetting accent colors
we want to not hammer the network when things are down. But during
startup we don't need to circuit break these errors that we throw before
the app-db is read
* Enhance glossary section with context for definitions
This is a smaller change than I expected, since glossary terms don't show up in searches yet. Might just cut this if it doesn't add anything useful. I also wasn't sure if we should mention Metabot here.
* Add dark mode options
Adds the available options and explains that you don't have to reload the page
* Add a little clarification to the computed goal example
This confused me, so I'm adding this little clarification
* Jeff's suggested change
Co-authored-by: Jeff Bruemmer <jeff.bruemmer@gmail.com>
* Remove the bit about LLMs again
---------
Co-authored-by: Jeff Bruemmer <jeff.bruemmer@gmail.com>
* fix: allow stacktraces to be suppressed in API
Allows administrators to configure their metabase instance using
MB_HIDE_STACKTRACES to prevent stacktraces from appearing in API
response. For standard responses it replaces unhandled exceptiosn with a
generic message and 500 response. For streaming responses it leaves the
error message and strips out the trace/via keys to prevent exception
info from leaking.
* fixup! dissoc traces from map errors
* Prevent trashed documents from being edited
* Add test
* Soften the test name tone
* Disallow even updating document when archived via API
* Add BE test
* Fix Cypress test issue reference
* Update tests after merging master
* Only check for archived when not unarchiving
* Don't check archived documents when archiving or unarchiving them
* Add missing condition from merging
* Fix logs upload for drivers
When we introduced `continue-on-error` to the "test" step, we had to
update the conditionals for the test report steps as well. I've
introduced a bug by checking for a failure, instead of saying - upload
test reports whenever the job is not successful.
Hint: test.outcome means "step status BEFORE continue-on-error overrides
the exit code"
Collected timing data from 7 days of test runs.
All specs with timing data were included (simple averages).
🤖 Generated automatically by update-e2e-timings workflow
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: GitHub Actions <actions@github.com>
* add debug message type for tool calls
* rename toolCalls -> activeToolCalls
* Basic debug mode slash command to hide/show tool calls
* ToolCallDetailsModal + better formatting
* Fix type
* Fix types, some simplification
* Address minor bot comments
* small changes
* move AgentToolCallMessage into its own file and add copy buttons + tool call id to modal
* fix copied value
* add unit tests
* move failure case further downstream to mitigate any possible customer issue
* ci fixes
---------
Co-authored-by: Sloan Sparger <sloansparger@gmail.com>
* fix: improve collections graph performance
Fixes performance generating and updating a collection graph by avoiding
the cross join of all negative permissions (blocks) between groups and
collections. Instead it builds sparse graph where missing entries for
group/collection paris can be assumed to be :none permissions.
In practice this drops a collection permission update on a graph of 5K
groups and 5K collections from a greater than 10min operation on my
local machine against MySQL to a .5s operation.
* fixup! explicilty disj :root in other places
* fixup! improve some docs
* fixup! improve more docs
* fixup! seq checks
* fixup! blank lines
* fixup! convert raw sql concat into honeysql
* fixup! e2e test
* Update error message for remote sync dep checking
Try to make these error messages less jargony because they are presented
by the user interface when dependency checking content fails.
Closes UXW-2142 and UXW-1879.
* Update some missed tests
* Add a grace period to token checks
There's more work on this namespace to be done i think, but this is a
first cut at it. There's a lot of control flow by exceptions which is
how we jump around in here. Fundamentally this should be far
simpler. Will take a crack at that shortly. I also don't love that this
just hammers the `fetch-token-status` function and then memoizes inside
of that. Would prefer simpler layout of `token-features` function and
that can be aware of population issues.
Adds a simple protocol: GracePeriod, which has two functions: save! and
retrieve. On successful token check, we `save!` the response. On errors,
we `retrieve` previous responses. This obviously can be nil in the case
we have never had a successful response, and will become nil once the
grace period's internal notion of time expires.
This implementation is based on guava's cache which has a nifty
expireafterwrite functionality.
```
(*token-features*) ;; call this function to simulate the check of features
Checking with the MetaStore to see whether token 'f325...d01a' is valid...
Reporting Metabase stats: {... user counts etc}
-> #{"features come back"}
;; meanwhile clear cache to simulate elapsing of 12 hours
(*token-features*) ;; call function again to simulate check of features
2025-10-22 21:30:26,620 INFO premium-features.token-check :: Checking with the MetaStore to see whether token 'f325...d01a' is valid...
2025-10-22 21:30:26,622 ERROR premium-features.token-check :: Error fetching token status from https://token-check.metabase.com:
clojure.lang.ExceptionInfo: network issues ;; hit network issue
2025-10-22 21:30:26,632 INFO premium-features.token-check :: Using token from grace period
->> #{graceperiod network features}
...
;; here after 5 seconds (in real life will be 36 hours)
2025-10-22 21:31:00,064 WARN premium-features.token-check :: Removing token: f325...d01a from grace period cache {mb-quartz-job-type=Cache}
```
* tests that updating value populates grace period cache
* couple changes
protect against storing nil value. guava cache doesn't like it so gone
it is.
added some tests:
```clojure
(testing "Setting a new value populates the grace period cache"
(let [new-token (tu/random-token)]
(binding [http/request (fn [& _] {:status 200
:body "{\"valid\":true,\"status\":\"fake\",\"features\":[\"fake\",\"features\"]}"})]
(mt/discard-setting-changes [premium-embedding-token]
(premium-features.settings/premium-embedding-token! new-token)
(is (= #{"fake" "features"} (token-check/*token-features*)))
(is (= {:valid true :status "fake" :features ["fake" "features"]}
(token-check/retrieve token-check/grace-period new-token)))))))
```
But this ended up leaving a setting table value for this value. i don't
feel like looking into discard setting changes so i'm gonna just remove it
* attempt to simplify the api a bit
the thee important functions are now `token-information`, `harness` and
`decode-token`.
Everyone should use token-information. Harness hooks up the grace period
and retry strategies, and `decode-token` only cares about decoding a
token. It should return the token-status map {:valid true ...} or throw.
* fixup test
* clean up tests and rename a bit
* remove comment block
* remove commented out grace period
* bit of cleanup
* fix with-random-premium-token! macro
* function composes them instead of caller
rather than a caller needing to know all of the layers, i thread them
for you and let you provide all arugments in one function call.
dynamic var for needing to customize layers in tests. largely for
removing the circuit-breaker which is stateful and hard to restart if
you want to simulate errors in tests
* rename test
* Add documents to content view
* add document to audit_log
* added comment events to the audit log, publish events
* wip: transform events
* glossary
* reversion the content migrations
* include transforms in content
* remote sync audit events
* wip: action v2 event
* kondo
* fix action tests
* transform run event
* add tests for new events
* bump
* tests
* task data
* let's not change the old migration in case we might break the hashsum
* typo
* more typos
* fix bugs in h2/mysql
* add :previous-object
* fix tests
* move branch into details
* fix test
* fix: run tests only on ee
---------
Co-authored-by: Nicola Mometto <nicola.mometto@metabase.com>
* Normalize parameter type category/= to category
Stats had exactly one dashboard with one such parameter and that dashboard was immutable because the
before-update hook failed before any update could happen.
* - Improve remote-sync error messages
- Do a fetch when checking git settings to ensure the current version can be read and is being checked
* Validate git url
* Validate branch name in production mode
* Validate branch name in production mode
* Removed unneeded assoc
* Fixed formatting
* Code improvements
* Code improvements
* Allow direct path URLs
* Allow direct path URLs
* E2e Fixes
* E2e Fixes
* Allow empty branch to be passed
* Legacy MBQL normalization overhaul
* Kondo + test fixes 🔧
* Cleanup and test fixes 🔧
* Code cleanup
* Test fixes 🔧
* Extra test 🔧
* Some test fixes 🔧
* Test fixes 🔧
* Test fixes 🔧
* Test fixes 🔧
* Fix SerDes test failure
* cljfmt
* Code cleanup
* Fix normalizing :datetime with bytes modes
* Fix legacy MBQL normalization of :datetime
* Handle broken case clauses
* Set Card metadata to nil when invalid
* Fixx
* Fix Kondo
* Fix/unskip some tests / try to make flaky FE test to pass in CI
* adding branching tests, unskipping admin tests
* PR Feedback
* PR Feedback
---------
Co-authored-by: Anderson <anderson@metabase.com>
* Add type field to transforms with temporary backfill job
* add a default value for tests
* add some new tests, fix serdes
* lint
* remove debugging defs
* revert serdes spec change
* Revert "remove transforms from serdes baseline (?)"
This reverts commit e8fc8fe80bfe15e00eb031b4050877f271b8081f.
* try again to fix serialization tests
* address bot comments
* Revert "try again to fix serialization tests"
This reverts commit 13d78664b79b7f13406d89e1f8f910610ea2cce9.
* Fix search ingestion functions to use :type field + some cleanup
* Update baseline to fix serialization test
* Revert "Update baseline to fix serialization test"
This reverts commit 60cb7909e5a73d04c42d36c915a32bb28d684e37.
* fix up comment
* reset serialization_baseline to version on master
* let's try this again: manually update baseline to add type field
* fix search tests by adding correct types to transfomrs
* update metabot tool apis
* fix revision test
* simplify test
* address feedback - move type setting to the model layer
* simplify tests
* Fix up transform serdes tests: remove top-level type fields and updated_at from being serialized, and fix shape of transforms in extraction tests
* set :type after select just in case; fix serdes tests again
* guard against empty source on select
* remove redundant search spec for transforms (introduced via a previous merge)
* switch from type to source_type to avoid conflict with type field on the frontend
* add better error handling
Basically, the same case found two unrelated bugs. For one, we don't support table functions like `select * from
my_function()`. For another, `select orders from orders` is also unsupported but apparently legal (at least in
postgres). This updates sql parsing/validation to support both things.
This pull request updates the configuration and environment handling for E2E and compatibility tests, mainly to support running tests against a staging license/token server and to improve test reliability. The changes affect GitHub Actions workflows, environment variables, and some test and backend logic.
*Backend Logic for E2E Mode*
Added `is-e2e?` flag in core.clj to distinguish E2E test runs from dev/prod/test modes, allowing for custom behavior in E2E scenarios.
Modified token check logic in token_check.clj to allow E2E tests to use a staging server for license validation.
Changed store API settings in settings.clj to use staging endpoints when in dev or E2E mode, ensuring tests interact with the correct backend.
* poc of handleLink
* safer implementation
* make it return a boolean to prevent default
* fix types?
* fix static viz build
* adds tests
* make tests more consistents with fns
* test return value undefined
* Update enterprise/frontend/src/embedding-sdk-shared/lib/sdk-global-plugins.ts
Co-authored-by: metabase-bot[bot] <109303359+metabase-bot[bot]@users.noreply.github.com>
* refactor plugin call
* fix types
* removed reduntant test
* change the api to use `{handled: boolean}`
* support links rendered as anchor tags, refactor the logic and tests
* add story
---------
Co-authored-by: metabase-bot[bot] <109303359+metabase-bot[bot]@users.noreply.github.com>
* init branch
* FE - add codemirror diffing, add diff support to native query editor, fake a proposed query
* Add /ee/metabot-tools/get-transforms endpoint
Returns a sequence of all python or native query transforms.
BOT-391 [Backend] Create transform index tool API
* Add test that get-transforms does not include non-native MBQL transforms
* Add /ee/metabot-tools/get-transform-details endpoint
* allow opening metabot on transforms admin pages
* use transforms_codegen profile for metabot on transforms page
* send transform in context
* show metabot on edit + create transforms page
* accept tranform suggestions from metabot
* Cleanup malli schemas for get-transforms and get-transform-details
* add transform context on edit query page
* use transforms instead of data queries
* Parse out tables from transforms in metabot user context
* wip on creation flow
* hack in a message to metabot on save
* quick fix preview error not displaying correctly
* misc. fe improvements
* fix transform context on subsequent edits for the same transform
* half hacky way to use current editor state for context in transform editing
* Switch /ee/metabot-v3/get-transforms from :get to :post
This endpoint doesn't expect args of it's own in the body, but does accept the standard metabot conversation_id that
it echoes back.
* Show 'Apply Changes' button when a transform change is generated by the agent
* handle new transforms better
* Ensure we have a fallback for context items that don't have SQL to parse
* persist diff of transform to the conversation history
* temp - make metabot always visible on transforms admin pages
* Change button label to "Proposed changes to <transform>"
* copy change for create transform buttons in chat
* better hidden message ui
* hack - better emtpy convo text
* hide suggested prompts
* Add basic snippets tools
* ensure proposed queries go through nativeQuery.setQueryText so that template tags are parsed
* better todos
* even better todo list ui
* crude sidebar ui for query transforms diffs
* better suggestion chat ui
* good'nuff @mentions
* debug logging for search table bug
* further logging
* Ensure we parse template tags for transform edits in Metabot sidebar (#64001)
* cleaner, more copy/pastable, better flowing @mentions in metabot
* Fix metabot get-transforms api test to :post not :get
* Update metabot tools test with basic python transform tests
* Ensure transform :source and :target keys get snake cased when returned from metabot tool apis
* Add get-transform-python-library-details metabot tool api endpoint
* Enhance context with :used_tables for python transforms
* surface transforms in @-mention search
* some type fixes
* python FE progress
* python fe improvements + edit experience now works
* clear proposed on cancel
* fix accept in sidebar not saving
* better diff colors
* smaller todos
* cleaner code for todos
* improve ranking of transforms in search by hardcoding a last_viewed_at time in user-recency-expr
* first pass at nested @mentions
* Conditionally add transforms to model->db-model
* Add transforms-python to module deps for metabot
* Move native-query-transform? into transforms.util and re-use existing utils
metabot-v3.tools.transforms defined a couple of helper functions that were nearly identical to ones that have since
been added to transforms.util ns. Move some things around and make use of the existing transforms.util predicates,
which are now exported from the module's core namespace.
* lots of fe clean up
* prevent rewinding conversation for admin pages
* @mention solution that plays nicely with documents
* fix search bug
* fix type issues and few small feedback things
* Add transforms to search for Metabot (#64134)
* index transforms content
* Add support for transforms & query search in metabot search tool API
* fixup whitespace
* update to use new :fields syntax
* Fix up serach-native-query malli types
* Fix the 15-day interval for transforms to be db-agnostic
* move engine loading to metabase.search.init to break circular dep
* fix kondo
* Revert "Fix the 15-day interval for transforms to be db-agnostic"
This reverts commit 11b34e4cd4a8a5a0cf8041a1d82d487ed8c81025.
* add transform to set of appdb scorer models; unrevert db type fix
* fix kondo
* implement requisite multimethods for transforms not to error in in-place search engine
* ensure we enforce metabot/unknown contexts for transforms in in-place search backend as well
* fix filter test
* transform->postprocess to reduce naming ambiguity; add a basic transform ingestion test
* add a metabot result postprocessing test
* add transforms scoring test
* expand ingestion tests
* fix tests + kondo
* address feedback
* make suggestion models configurable for a Metabot instance
* fix @mention recents in documents, update suggestion sidebar component to not accept / reject items + calculate isViewing better
* quick fixes
* for rye guy
* Handful of fixes for transforms search (#64287)
* Add both search and search_v2 endpoints, and fix bug with search-native-query option
* add a docstring
* add a basic unit test for search-native-query option
* address comments
* better state management for suggested transofrms, move css around, clean up suggestion message code
* Update the list of source tables in Python editor when metabot changes it (#64258)
If metabot proposes changes to the list of input tables for a Python transform, update the PythonDataPicker component
to keep it in sync with the text of the transform source.
* python tables reference fix
* bug fixes and better state organization
* better draft support
* fix types
* add on leave modal, clear transforms on route leave, calc dirty state, fix onChange but for native query editor, better logic for showing cta text for transform suggestions
* Ensure transforms get deleted synchronously from the search index (#64382)
* Delete transforms from search index in t2/define-before-delete
* kondo
* fix up tests
* Stop pretending to snake_case transform source and target keys
The malli schemas were wrong, so these weren't actually getting snake_cased anyway.
See also:
https://github.com/metabase/ai-service/pull/387#discussion_r2414569482
* Fix failing test & kondo warning
* Add transforms dependency check API for metabot (#64542)
* WIP - add dependency check API for metabot
* omit bad_cards for now
* update return format
* add tests
* Fix fn call
* revert unneeded refactor
* Fix search transform-ingestion-test to use mt/native-query
* switch test to mt/native-query
* Fix module usage test
* fix up tests to use lib/native-query instead
* try to fix last test
* switch native-query-transform? to use lib.query/native?
* more mbql 5/metabase lib updates
* revert the 1-pixel fix
* Subtract a pixel when computing metabot chat filler height to avoid scrollbar flicker (#64690)
* Fix MetabotAgentSuggestionMessage only show "create" for new transforms (#64598)
https://metaboat.slack.com/archives/C09CR3MT1KR/p1760034396345789
* Surface broken cards from Metabot dependencies check API (#64781)
* fix missing sidebar diff after mbql5 change
* surface broken cards
* add test
* fix kondo
* remove unused code
* Fix up dep check question terminology & schema
* fix up test
* fix up another test
* address feedback
* Update the transform source schema to use ::queries.schema/query
* remove unused require
* address FE feedback
* fix modules config test
* @mentions tests and improvements
* couple of e2e tests fixes
* prevent leave confirmation when saving
* fix most of unit tests
* add back searchable models in metabot @mentions
* More tests for transform search + transform dependencies tool, and some new future-proofing read checks (#64896)
* add some future-proofing read checks to transform deps checks + related tests
* add some more transform search tests
* remove 404 check; move read check later in impl
* convert from toucan transform instance to metadata
* clean ns
* Move honeysql helper to simplify module dependencies
* fix more function calls in tests
* fix more calls
* fix (hopefully) last two calls
* kondo fix
* fix module config
* Fix detection of requires in threading macros
* Add metabase.driver.sql.query-processor back as a driver API namespace
* Add tests for metabot tools snippets and check-transform-dependencies endpoints (#64894)
* Add tests for metabot tools snippets API endpoints
* Add test for metabot tools check-transform-dependencies endpoint
* Remove trailing blank line to appease linter
* copy over appdb implementations of add-interval-honeysql-form to h2x; add back the sql.qp version for other drivers
* switch other driver methods back to sql.qp
* update relevant callsites
* move some test logic
* fix up error
* restore :added meta
* fix tests, move over one more pg timestamp helper
* oops, update callsites
* make function public
* add MetabotChatEditor.unit.spec.tsx
* add links.unit.spec.ts
* add AIMarkdown.unit.spec.tsx
* add MetabotAgentTodoMessage.unit.spec.tsx
* add MetabotAgentSuggestionMessage.unit.spec.tsx
* add metabot/state/reducer.unit.spec.ts w/ tests for suggested transforms state
* add e2e test coverage for create + update flow for both native sql + python transforms
* FE pr feedback and lint fix
* unit test fixes
* fix search test
* add temp metabot toggle button in admin nav
* fix a couple of failing FE tests
* attempt fix ci flake issue with metabot.unit.spec.tsx
* remove metabot-only restriction on transform search
* remove redundant logs in transform api endpoints to address ngoc's comment
* fix a number of transform search tests
* fix another test
* fix hopefully last search test
* bot feedback
* bot feedback
* fix function name
---------
Co-authored-by: Sloan Sparger <sloansparger@gmail.com>
Co-authored-by: Mike Appleby <86076+appleby@users.noreply.github.com>
Co-authored-by: Mihael Konjevic <konjevic@gmail.com>
Co-authored-by: Tyler Plude <tsplude@gmail.com>
Make native validation allow using current-level aliases in other clauses
`select a as b from table where b > 1` wasn't passing validation because the b alias wasn't in scope in the where.
This should now be fixed.
Also, native validation keeps more context when generating references, which should allow for better error messages.
Instrumentation adds a lot of overhead. Unfortunately, even 30 seconds timeout wasn't enough for some of our unit tests. We're doubling that value in order to minimize the chances of this workflow halting/failing again.
* Add workflow_dispatch trigger
* Rename workflow to 'Frontend Test Coverage'
* Fix remote sync settings test
* Fix sync type setting validation test
* Fix check sync task type is production
* Update plain type to collection_type
* Cascade changes to collection type
* Update deps checks when collection type changes
* Fix task type check in export
* move setting check to api for better status report
* add tests for sync-type
* Don't mark newly created entities as updated or removed UXW-1806
* UXW-1821 enabled badge
* UXW-1818: hide branch picker and push button for non-admins
* cleanup and UXW-1814
* remove border from the branch picker button
* cleanup
* UXW-1829 update diff modal ui
* UXW-1807: token should appear masked
* Allow setting of nil token
* UXW-1833: show backend error when saving invalid git settings
* Add new collection update/create events
* Remove per collection imp/exp; change dirty status table
* Redo event handling add removed status
* Sync object table on syncs
* Fix status names
* Added /current-task/cancel API and track version in remote_sync_task
* Add is_remote_synced to api responses
* Fix adding is_remote_sync hydration
* Fixed compilation error
* remove hashp
* Fix loading is_remote_sync hydration
* Resolve ambiguous reference
* Fix n+1 detector
if it gets a heterogeneous set of things we should hydrate each type
separately.
* Add can-create? handlers
* remote sync: a few bugfixes and improvements from my bugbash (#64372)
* Set `collection.type` on archive/unarchive
When unarchiving or archiving collections that have the `remote-synced`
type, we need to set their new type appropriately.
When archiving, we should give them a normal (nil) type.
When unarchiving any collection *to* a remote-synced collection, we
should set the type to `remote-synced`.
* Allow serdes without archived items
and use it in remote sync export.
During dependency checking, we do *not* want to exclude archived items:
if card A uses card B, and card B is archived, we need to include Card B
in the list of dependencies! Otherwise we'll say that it's fine to
remote sync card A when it's not.
* Add tests
* Documents are descendants of collections
* don't include cards as document descendants
* fix tests, dashboard descendants args
* remove #p and unnecessary query
* remove another `#p`
* Update auto sync token to enable import job
* More is_remote_sync hydration fixes
* Fix lints and document oss serdes
* Fix cards to be valid for update events
* Don't actually import if nothing has changed in the remote repo
* Remove hashp
* UXW-1817: change branches should update content
* UXW-1813 ensure sync dirty state is properly invalidated
* UXW-1864 fix how collections changes appear in the diff
* Fix ee-available? check
* Improve is_remote_synced hydration
* Fix some more oss/ee issues
finish remote ChangeLog model
* fix invalidation on removal
* fix missing require
* Don't need to handle transforms directories specially
* fix tests to make sure state is fully reset
* remove duplicate require
* Fix deadlock on import
* Fixing tests
* Fix document tests
* Fix kondos
* handle removed sync_status
* UXW-1851: show moving out of synced collection error
* feature flag, misc fixes
* dirty state tracking for snippets
* Fix ns typo
* Add remote-sync token and config
* enforce remote_sync token feature on the FE
* allow cancelling the remote-sync
* Track timeline
* Force re-import when configuring for production mode
* Fixed compilation error
* Force re-import when configuring for production mode only when auto-sync is enabled
* Throwa 400 error if there are dirty objects when doing an import
* force import button
* Better archiving checks with remote synced collections
* Better tests for documents and remote-sync deps
* Remove hashp
* remote hashp
* Remove other hashp
* More consistent collection write checks
* Fix broken collection tests
* Fix broken documents tests
* improve imports/exports frontend
* Implement can-create? for timeline-event
* tweak copy
* Always update branch setting
* Initial Commit
* Fix more tests
* Remove bad parallel annotations
* Show error toast when adding a collection that has references outside of the repo
* reasonable structure, error handling
* timelines editing is less frequent, we can invalidate dirty state on any edit
* Fix dirty state api tests for mysql
* bring error details back
* Fix collection move tests
* Fix remove parallel from can_restore tests
Splitting this into smaller tests seems to introduce some kind of state
that causes tests to fail in parallel
* Fix make logs use unique names for be tests
* Bump toucan2 versions
* Fix var name
* Fix check that there are exportable collections
* Add test for exportable check
* Started on e2e test
* Fix issue with merge
* Fix arguments to card descendants
* Handle new ingestable interface
* Put back changes in cypress-runner
* Fix remaining failing enterprise tests
* Fix one set of app tests
* Clean remote sync state in obj tests
* single collection
* Fix unit test - enterprise getIcon function only applied when new feature is enabled (ResultItem.unit.spec.tsx)
* Fix ts issues / Clean up
* FE fixes
* Remove parallel annotations
* Fix missing import from merging master
* Fix the fix
* always create collection to sync
* no unsyncing
* Remove associating location to trash from search
* Prevent users from switching branches with unsynced changes + show warning
* Create remote-sync collection when it doesn't exist
* Prevent changing to production with unsynced changes
* Make sure dirty state refetches on admin settings load
* single synced collection ui update
* Cannot change to production mode when there are unsynced changes
* Fix failing query
* unquote branches
* fix eastwood
* fix timeline event publishing
* fix import
* Move dashboard remote sync api tests into ee
...where they can reference the setting
* Created /stash API and renamed create branch api to /create-branch
* Fix settings issue
* Fix remaining permissions message tests
* Make sure drop target always works for trash
* Add checks for if we're in trash elsewhere
* cleanup
* git-sync to remote-sync
* more git-sync to remote-sync
* Don't allow export if it overwrites an external commit
* - Don't assume a "main" branch
- Attempt an import when no remote-sync branch exists when enabling dev mode
* Fix creating remote sync library
* refactor to middleware
* hack: add a delay for refetching instance settings or dirty state after remote sync task finishes
* UXW-1953: confirmation flow when switching branches in prod
* Update modal when switching branches with dirty state / Component refactorings
* Create a branch off the last-used commit, not the branch itself
* Add options modal when trying to push while remote is ahead
* Support force option on export
* Update branch setting and complete task in txn
* Let export! set branch in stash and fix tests
* Fix PR feedback nits
* Update enterprise/backend/src/metabase_enterprise/remote_sync/api.clj
* Consistent documentation style in remote-sync
* Add option to pull from remote / Adjust push/pull icons / Adjust out-of-sync options modal
* modals UI polish
* constants clean up
* Add api schemas and make error messages consistent
* fix check-403 to write-check
* Apply suggestions from code review
Co-authored-by: metabase-bot[bot] <109303359+metabase-bot[bot]@users.noreply.github.com>
* Clean up a couple components / Move things around
* Address renaming suggestions
* Fix var names and bad merge
* double->float in schema
* Share error handling
* Update enterprise/backend/src/metabase_enterprise/remote_sync/settings.clj
* Update enterprise/backend/src/metabase_enterprise/remote_sync/settings.clj
* Update enterprise/backend/src/metabase_enterprise/remote_sync/settings.clj
* Addressing small PR comments
* Refetch properties when branch is created (current branch changed)
* Make sure /current-task is only called when import and export are not in progress
* Replace unset color variables
* Improve element padding on side panel
* Remove a couple trailing spaces reported by clj whitespace linter
* Show error when settings request fails / Make sure current-task also waits for the PUT settings mutation
* Fix document perm tests
* Allow empty remote-url, moved import and export calls to impl
* Fix minor prettier issue
* Fix another lil lint issue
* temp
* e2e push / pull
* Unify more of the import implementation
* fix CI failures
* fix mysql ci failures
* Address metabot critiques
* Address additional metabot feedback
* last round of docstring feedback
* library admin settings e2e
* Add attribution to TODO
* fix ws
* fix missed commiting file
* fix another missing file
* production switch branches
* More clojure like docstrings
* More docstring fixes
* adding branch switching test
* Address Nick's PR review comments (most of them)
* Fix more nits
* Resolving Nick's comments part 2
* Improving remote-sync settings handling
* attempt to repair synced_folder icon
* Fail if attempting to connect to an uninitialized repo
* correct the name of synced_collection.svg
* add new repository icon
* Add default-branch to source protocol
* Update Git Sync to Remote Sync in Admin settings (#64993)
* Make api docstrings and errors more generic
* Use check-400 to simplify api
* fix ws
* Rollback BaseSMTPConnectionForm.tsx (fixes a e2e test)
* fix e2e tests?
* Fix small merge issue
* Always use `last-version` instead of caring about import vs. export version
* starting warning modal tests
* Fetch source at the beginning of import/export operations
* Fix broken test
* Delete branches that were deleted on remote
* Fix missing test suffix
* Fix eastwood
* Adjust branching e2e test, skip failing tests
---------
Co-authored-by: Edward Paget <ed.paget@gmail.com>
Co-authored-by: Nathan Voxland <nathan@metabase.com>
Co-authored-by: Aleksandr Lesnenko <alxnddr@gmail.com>
Co-authored-by: Edward Paget <edpaget@users.noreply.github.com>
Co-authored-by: Aleksandr Lesnenko <alxnddr@users.noreply.github.com>
Co-authored-by: Anderson <anderson@metabase.com>
Co-authored-by: eliotmetabase <eliot@metabase.com>
Co-authored-by: metabase-bot[bot] <109303359+metabase-bot[bot]@users.noreply.github.com>
Co-authored-by: Nick Fitzpatrick <nickfitz.582@gmail.com>
Co-authored-by: Maz Ameli <maz@metabase.com>
* init branch
* FE - add codemirror diffing, add diff support to native query editor, fake a proposed query
* Add /ee/metabot-tools/get-transforms endpoint
Returns a sequence of all python or native query transforms.
BOT-391 [Backend] Create transform index tool API
* Add test that get-transforms does not include non-native MBQL transforms
* Add /ee/metabot-tools/get-transform-details endpoint
* allow opening metabot on transforms admin pages
* use transforms_codegen profile for metabot on transforms page
* send transform in context
* show metabot on edit + create transforms page
* accept tranform suggestions from metabot
* Cleanup malli schemas for get-transforms and get-transform-details
* add transform context on edit query page
* use transforms instead of data queries
* Parse out tables from transforms in metabot user context
* wip on creation flow
* hack in a message to metabot on save
* quick fix preview error not displaying correctly
* misc. fe improvements
* fix transform context on subsequent edits for the same transform
* half hacky way to use current editor state for context in transform editing
* Switch /ee/metabot-v3/get-transforms from :get to :post
This endpoint doesn't expect args of it's own in the body, but does accept the standard metabot conversation_id that
it echoes back.
* Show 'Apply Changes' button when a transform change is generated by the agent
* handle new transforms better
* Ensure we have a fallback for context items that don't have SQL to parse
* persist diff of transform to the conversation history
* temp - make metabot always visible on transforms admin pages
* Change button label to "Proposed changes to <transform>"
* copy change for create transform buttons in chat
* better hidden message ui
* hack - better emtpy convo text
* hide suggested prompts
* Add basic snippets tools
* ensure proposed queries go through nativeQuery.setQueryText so that template tags are parsed
* better todos
* even better todo list ui
* crude sidebar ui for query transforms diffs
* better suggestion chat ui
* good'nuff @mentions
* debug logging for search table bug
* further logging
* Ensure we parse template tags for transform edits in Metabot sidebar (#64001)
* cleaner, more copy/pastable, better flowing @mentions in metabot
* Fix metabot get-transforms api test to :post not :get
* Update metabot tools test with basic python transform tests
* Ensure transform :source and :target keys get snake cased when returned from metabot tool apis
* Add get-transform-python-library-details metabot tool api endpoint
* Enhance context with :used_tables for python transforms
* surface transforms in @-mention search
* some type fixes
* python FE progress
* python fe improvements + edit experience now works
* clear proposed on cancel
* fix accept in sidebar not saving
* better diff colors
* smaller todos
* cleaner code for todos
* improve ranking of transforms in search by hardcoding a last_viewed_at time in user-recency-expr
* first pass at nested @mentions
* Conditionally add transforms to model->db-model
* Add transforms-python to module deps for metabot
* Move native-query-transform? into transforms.util and re-use existing utils
metabot-v3.tools.transforms defined a couple of helper functions that were nearly identical to ones that have since
been added to transforms.util ns. Move some things around and make use of the existing transforms.util predicates,
which are now exported from the module's core namespace.
* lots of fe clean up
* prevent rewinding conversation for admin pages
* @mention solution that plays nicely with documents
* fix search bug
* fix type issues and few small feedback things
* Add transforms to search for Metabot (#64134)
* index transforms content
* Add support for transforms & query search in metabot search tool API
* fixup whitespace
* update to use new :fields syntax
* Fix up serach-native-query malli types
* Fix the 15-day interval for transforms to be db-agnostic
* move engine loading to metabase.search.init to break circular dep
* fix kondo
* Revert "Fix the 15-day interval for transforms to be db-agnostic"
This reverts commit 11b34e4cd4a8a5a0cf8041a1d82d487ed8c81025.
* add transform to set of appdb scorer models; unrevert db type fix
* fix kondo
* implement requisite multimethods for transforms not to error in in-place search engine
* ensure we enforce metabot/unknown contexts for transforms in in-place search backend as well
* fix filter test
* transform->postprocess to reduce naming ambiguity; add a basic transform ingestion test
* add a metabot result postprocessing test
* add transforms scoring test
* expand ingestion tests
* fix tests + kondo
* address feedback
* make suggestion models configurable for a Metabot instance
* fix @mention recents in documents, update suggestion sidebar component to not accept / reject items + calculate isViewing better
* quick fixes
* for rye guy
* Handful of fixes for transforms search (#64287)
* Add both search and search_v2 endpoints, and fix bug with search-native-query option
* add a docstring
* add a basic unit test for search-native-query option
* address comments
* better state management for suggested transofrms, move css around, clean up suggestion message code
* Update the list of source tables in Python editor when metabot changes it (#64258)
If metabot proposes changes to the list of input tables for a Python transform, update the PythonDataPicker component
to keep it in sync with the text of the transform source.
* python tables reference fix
* bug fixes and better state organization
* better draft support
* fix types
* add on leave modal, clear transforms on route leave, calc dirty state, fix onChange but for native query editor, better logic for showing cta text for transform suggestions
* Ensure transforms get deleted synchronously from the search index (#64382)
* Delete transforms from search index in t2/define-before-delete
* kondo
* fix up tests
* Stop pretending to snake_case transform source and target keys
The malli schemas were wrong, so these weren't actually getting snake_cased anyway.
See also:
https://github.com/metabase/ai-service/pull/387#discussion_r2414569482
* Fix failing test & kondo warning
* Add transforms dependency check API for metabot (#64542)
* WIP - add dependency check API for metabot
* omit bad_cards for now
* update return format
* add tests
* Fix fn call
* revert unneeded refactor
* Fix search transform-ingestion-test to use mt/native-query
* switch test to mt/native-query
* Fix module usage test
* fix up tests to use lib/native-query instead
* try to fix last test
* switch native-query-transform? to use lib.query/native?
* more mbql 5/metabase lib updates
* revert the 1-pixel fix
* Subtract a pixel when computing metabot chat filler height to avoid scrollbar flicker (#64690)
* Fix MetabotAgentSuggestionMessage only show "create" for new transforms (#64598)
https://metaboat.slack.com/archives/C09CR3MT1KR/p1760034396345789
* Surface broken cards from Metabot dependencies check API (#64781)
* fix missing sidebar diff after mbql5 change
* surface broken cards
* add test
* fix kondo
* remove unused code
* Fix up dep check question terminology & schema
* fix up test
* fix up another test
* address feedback
* Update the transform source schema to use ::queries.schema/query
* remove unused require
* address FE feedback
* fix modules config test
* @mentions tests and improvements
* couple of e2e tests fixes
* prevent leave confirmation when saving
* fix most of unit tests
* add back searchable models in metabot @mentions
* More tests for transform search + transform dependencies tool, and some new future-proofing read checks (#64896)
* add some future-proofing read checks to transform deps checks + related tests
* add some more transform search tests
* remove 404 check; move read check later in impl
* convert from toucan transform instance to metadata
* clean ns
* Add tests for metabot tools snippets and check-transform-dependencies endpoints (#64894)
* Add tests for metabot tools snippets API endpoints
* Add test for metabot tools check-transform-dependencies endpoint
* Remove trailing blank line to appease linter
* add MetabotChatEditor.unit.spec.tsx
* add links.unit.spec.ts
* add AIMarkdown.unit.spec.tsx
* add MetabotAgentTodoMessage.unit.spec.tsx
* add MetabotAgentSuggestionMessage.unit.spec.tsx
* add metabot/state/reducer.unit.spec.ts w/ tests for suggested transforms state
* add e2e test coverage for create + update flow for both native sql + python transforms
* FE pr feedback and lint fix
* unit test fixes
* fix search test
* add temp metabot toggle button in admin nav
* fix a couple of failing FE tests
* attempt fix ci flake issue with metabot.unit.spec.tsx
* remove metabot-only restriction on transform search
* remove redundant logs in transform api endpoints to address ngoc's comment
* fix a number of transform search tests
* fix another test
* fix hopefully last search test
---------
Co-authored-by: Sloan Sparger <sloansparger@gmail.com>
Co-authored-by: Mike Appleby <86076+appleby@users.noreply.github.com>
Co-authored-by: Noah Moss <noahbmoss@gmail.com>
Co-authored-by: Mihael Konjevic <konjevic@gmail.com>
Co-authored-by: Noah Moss <32746338+noahmoss@users.noreply.github.com>
Due to `useEngineDocMarkdownContent` changes the dynamic imports are needed for `import(`docs/databases/connections/${docFileName}.md`)`
Co-authored-by: Jonathan Sheely <jsheely@thirtytech.net>
* Collections are delete-able
The `can_delete` flag for collections was the only missing bit to have
collections be deletable, since I added an API endpoint to delete
collections a while ago! I thought FE work was required, doh.
Fixes UXW-1090
* Fix e2e tests
* Rework X-Rays to use Lib
* Test change
* Fix the last few failing tests
* Remove dead code
* PR feedback
* Claude updates
* Fix typos
* Oops
* e2e test fix 🔧
* Assert premium features for some middleware
* Fix an existing bug
These tests started failing here, which revealed that they formerly
never actually were testing what they thought they were.
A few reasons:
- they turned on `advanced-permissions` feature by itself, so sandboxing
was always off here
- they set up sandboxing in a nonstandard way
- one had a typo so we weren't even wrapping the body with
`with-download-perms!` (it was a separate block entirely)
This hid the bug that some of the tests were intended to reproduce - if
a card used for sandboxing had a native query, then we required that you
have download permissions on the entire database, which is wrong.
* fix test
* make assertions with/without tokens for sandboxing
* Add comments about why `:feature :none`
* Update enterprise/backend/test/metabase_enterprise/sandbox/query_processor/middleware/sandboxing_test.clj
Co-authored-by: Cam Saul <1455846+camsaul@users.noreply.github.com>
* add a nicer docstring and change fn name
replace `any-native-unsandboxed-stage?` with
`any-native-stage-not-introduced-by-sandbox?`
* Block without feature enabled
* CR response
---------
Co-authored-by: Cam Saul <1455846+camsaul@users.noreply.github.com>
* Allow any scheme for URLs, not just http/https
* Explicitly allow app:// and capacitor:// schemes
* fix flakiness in e2e test
* Only accept HTTP/HTTPS when no protocol is specified
---------
Co-authored-by: Uladzimir Havenchyk <uladzimir@metabase.com>
* ugly solution #1
* revert the ugly part
* refactor and tests
* fix(modular): make links/click behavior that link to the same origin as the host app work and open them on a new tab
* update tests to expect a click behavior
* clean up tests
* tests for the same domain issue
* add also basic support for links, just in case
* Update e2e/test-component/scenarios/embedding-sdk/dashboard-click-behavior.cy.spec.tsx
* refactor tests as per feedback to use mockIsEmbeddingSdk
* don't remove column settings
* Remove unnecessary default value
* Remove unnecessary export
* Show move to trash icon in CollectionBrowser
* Make the trash button easier to work with
* Add a dropdown
* Fix icon
* Archive resources in the collection browser
* Add tests
* Update UX
* Prevent external type change
* Remove custom style
* subtler add data button in side nav
* fix tests
* update test
* fix unrelated broken test
* fix another unrelated broken test
* try fixing test again
* undo changes to unrelated tests
* Cursor's attempt at fixing related test
* Extend db connection timeout in tests
We allow for 10 seconds in the app, and 3 seconds in test. I extend that
3 seconds to 5 seconds. This should really only affect bigquery.
In bigquery, we test if you "can connect" by connecting to the project
and then attmepting to find any datasets that you are able to see. This
is important because each test run connects to the same project but is
scoped down to fewer datsets. It's possible we could change this
behavior and return if you can list datasets at all, but it is nice if
you can see at least one dataset based on your inclusion and exclusion
filters.
For reference, the filters for my current CI run are:
```clojure
bigquery-cloud-sdk=> (update details :service-account-json (constantly "<<redacted>>"))
{:project-id nil,
:service-account-json "<<redacted>>",
:dataset-filters-type "inclusion",
:dataset-filters-patterns "sha_4c3ca6238018e2b60025297f3e5c39671588adf2_QLGKZKIAOSQXOGGDURQY",
:include-user-id-and-hash true,
:project-id-from-credentials "metabase-bigquery-driver"}
```
In my repl, i'm seeing the following behavior:
```clojure
bigquery-cloud-sdk=> (time (let [client (database-details->client details)
project-id (get-project-id details)
datasets (.listDatasets client project-id (u/varargs BigQuery$DatasetListOption))
]
(vec (for [^Dataset dataset (.iterateAll datasets)
:let [dataset-id (.. dataset getDatasetId getDataset)]
]
dataset-id))))
"Elapsed time: 2692.292292 msecs"
["BQ_tupac_sightings"
"Shakespeare"
...
"v6_sample_dataset"]
bigquery-cloud-sdk=> (count *1)
273
```
So just enumerating all datasets takes ~2.7 seconds so we are flirting
with the fail limit of 3 seconds.
I extend this to 5 seconds and this should only give a bit of breathing
room to bigquery and everything else remains the same.
* fix flakiness in e2e test
---------
Co-authored-by: Uladzimir Havenchyk <uladzimir@metabase.com>
* Add migrations
* Add new columns to serde spec
* Update serde baseline
Will change again in follow up once sql-jdbc behaviour is ported
* Extend sql-jdbc describe-table
* Sync new field columns
* Extend field our-metadata to the new sync columns
* Fix tests
* Add postgres support new field columns
* Add generic driver tests for nullability and defaults
* Extend clickhouse test driver to :not-null? :default field defs
* Add new columns to mysql describe-fields-sql
* Add new features for better test specification
- :describe-is-nullable
- :describe-default-expr
In both cases if the feature is unsupported, the driver must return `nil` for the respective field metadata keyword.
* Add :database-is-generated test and driver feature
* Fix load-data inserts when rows empty
* Do not return generated-column-sql for redshift
* Fix snowflake-test/describe-table-test
* Fix bigquery load-tabledef when table has no rows
* Use generation_expression rather than :extra for mysql/mariadb
* Fixup driver features
* Amend pk-sync-test to verify the new :database_is_pk behaviour more explicitly as the semantic type changes
* Add sync tests for default/generated/nullable
* Formatting
* Add SQLite support for generated/nullable/default
* Add SQL-Server support for generated/nullable/default
* Only test dynamic dataset databases for now
* For druid JDBC all columns (apart from __time) are nullable
* Vertica does not support generated columns
* Starburst does not support generated columns
* Presto does not support generated columns
* Revert accidental database-required behaviour change
* Assert table synced
* Use def'd datasets rather than inline
* Oracle features
* Vertica features
* Opt starburst, presto out of defaults
* Add :test/create-table-without-data to deal with mongo
* Add some new lines to mig yml
* Add note about sync to driver test
* Do not spam logs for pk changes on upgrade
* Further logging guard refinement
* Fix migration conflicts
* [Search] Separate out embeddable_text from searchable_text and use labeled format for embedding
* add embeddable_text to test data
* update mock-embeddings in tests to use new format for embeddable strings
* Fix more tests
* Fix more tests
* Fix last tests
* Fix last test
* improve comments & add a test that transform fns are not applied
Avoids FE ordering depending on hash iteration order on initial sync - in particular avoids
breakage in tests relying on id positionally if hashes change.
* Remove double beforeEach setup probably from a mistake
* Fix the flaky test. I hope...
* Attempt to fix the test that failed the interception
* Revert change
* Fix the flake on Cypress CI
Closes QUE-2613.
`convertTimezone` now checks the type of the field ref, since we might not know to make the compiled
column a timezone-backed column when the previous stage is a native query.
* Remove double border
* Remove overflowing corners on functionbrowser
* Fix HelpText header background in dark mode
* Remove dot in customexpression editor
* Fix dark mode completions in native editor
* Move color fixes to CodeMirror component
* Move CodeMirror cursor fix in darkmode to shared CodeMirror module
* Move completion styles to shared CodeMirror component
* Enable highlighting of partials in Python completions
* Remove more shared CM code
* Fix e2e test
* Fix timout in e2e test
This addresses most of QUE-2686 #64774 but a second PR is required to fix a
56-specific issue that doesn't apply to `master`.
The FE was not using the new
`lib.metadata.cached-provider/cached-metadata-provider` except where
it gets added by `lib/query`. Previously, the FE constructed the
*uncached* MP once (ish) and then passed it to `lib/query` many times.
Each of those queries got its MP wrapped with `cached-metadata-provider`
but the next time the query got converted via `lib/query` its cached
values (eg. `visible-columns`, `returned-columns`) were lost.
This PR wraps the `lib.js.metadata/metadata-provider*` with
`cached-metadata-provider` from the start, so the cached values have
the same lifespan as the `MetadataProvider`, rather than each individual
query.
I also noticed that `/query_metadata` requests don't use
`lib-be.metadata.jvm/with-metadata-provider-cache`, so they end up
recreating `MetadataProvider`s for the same underlying database over and
over. That increases AppDB traffic and CPU usage to recompute
`returned-columns` etc. repeatedly. Both of those scale fast with the
complexity of the query.
* disable excessive spam in the prs
* to-revert: change source code to trigger the build
* test2
* :shrugs:
* Revert "to-revert: change source code to trigger the build"
This reverts commit 37f6abb085.
* clean up and comment
This fixup broke because QP started returning the db-shaped
viz-settings (e.g. `:table.columns`) instead of the normalized
viz-settings (e.g.
`:metabase.models.visualization-settings/table-columns `) under
`[:json_query :viz-settings]`.
The existing test for this fix did not catch the regression because the
shape of viz settings was faked out for testing, and did not get
transformed like real viz settings. `{:ver 2}` stayed as `{:ver 2}`,
while `{:table.columns []}` is transformed into
`{::visualization-settings/table-columns []}`
* Fix date range picker text inputs not in sync with calendar inputs (#64602)
* Unskip and update test
* Add repro for #64602
* Add unit tests
* Add issue tags
* Rename date to displayedDate
* Update the last instance of deprecated `FormRadioWidget`
This commit replaces a deprecated `FormRadioWidget` with the
`FormRadioGroup` component (which is a light wrapper around
the plain Radio.Group from Mantine).
* Remove the deprecated `FormRadio` component
* Revise alert creation instructions in documentation
Updated instructions for creating alerts, including clearer steps and options for different alert types. Adjusted formatting for consistency and added details on editing and managing alerts.
* Enhance clarity in delete and restore documentation
Updated the documentation for deleting and restoring items to clarify the impact on related content and improve phrasing.
* Apply suggestions from code review
Co-authored-by: Jeff Bruemmer <jeff.bruemmer@gmail.com>
---------
Co-authored-by: Jeff Bruemmer <jeff.bruemmer@gmail.com>
We call native-query-deps on query save on mongo queries. This ensure that doesn't error, while still flagging if we don't implement native-query-deps for a driver that is supposed to support it.
* use disk at query time of notification payload creation
We have some nice work to dump query results to disk if they are larger
than a row-count threshold. But this presupposes that results can fit
into memory. It's completely possible, and relatively easy, to do some
kind of "select *" operation in a dashboard where the only limit that
prevents billions of rows coming back is the excel limit we add in.
This tries to optimistically use the disk. If creates a new rff that has
two modes: in-memory and a gzipped dataoutput stream. The first is in
operation for the first 2,000 rows. If we never exceed this limit it
remains a regular reducing function. Once this threshold is exceeded, we
open a new file and stuff the rows in there and don't keep them in
memory any longer. This prevents us from having to hold the entire
result set in memory at once.
On the consumption side, we have a threshold such that if a file is
_larger_ than this, we won't attempt to load it and render it for static
viz. This is what lets us defensively decline to do an operation that
would reasonably lead to an OOM.
Optimizations for the future:
- give up on file creation once it exceeds the threshold size. We can
easily monitor the size of the file as we go and stop the transduction
or results.
- clear out files more frequently. This will probably litter lots of
temp files, some possibly quite large, on teh disk. We need to more
frequently clear them up. The directory is marked as deleteOnExit, but
we need to ensure we don't put lots of files in there during process
lifetime.
- better error rendering of "too large" error. Right now it hits the
default error rendering (previously it would oom). We can make this
better.
- explore how parallel we can get now. Presumably we are now quite
memory effecient. Can we go back to 3, or 10, worker threads?
* fix typo and reintroduce constant for row threshold
* more references to typo constant
* missed one somehow?
* include pulses (alerts) for autocommit off for downloads
we want to stream results. and now we do into a file so this can respect
the page size which only happens in pg when using autocommit false
* special error msg for too large results/dont attach large results
feedback that query results were too large
alerts also by default include results. let's check for the
render/too-large and not include them in that case. Was actually getting
a seq error on the deftype that holds the file. maybe want to give that
a seqable method?
* fix tests
we had an unrealized temp storage file going through
* unification all to streaming before moving to temp-storage
* move streaming to the temp-storage namespace
* don't deref results if no one wants attachments
* don't log the error here. it's pretty normal
* doc strings
* better logging and truncate results above threshold
once we are above the size that we won't read the file, we don't have to
keep piping results to it.
Concretely, what does this get us? If someone absentmindedly did
`select * from transactions` it could be 100s of GBs or more. Without
this, we would fill up the disk for the instance even though we don't
try to render it in the email. But once we know that there's no way we
will read the contents of the file, we really don't have to write to it
any longer.
* Make settings for size limit and whether to enforce
maybe someone has a beefy machine and they don't mind chugging through
843mb of query results to get a graph. they can easily set
`MB_ENFORCE_NOTIFICATION_TEMP_FILE_SIZE_LIMIT` to false and the old
behavior will be restored. If someone doesn't like the 10mb limit and
prefers a 50mb limit they can do
`MB_NOTIFICATION_TEMP_FILE_SIZE_MAX_BYTES=52428800` and then they will
have the higher limit
* Huge performance improvements
The real gist here is removing the gzip on the output. This was crushing
our performance. 18 seconds instead of ~4 seconds. Real bad. And it's
better to not gzip anyways. We really want to be tied pretty closely to
the size of the results. GZiping uses a ton of cpu and gets it very
compact. But we want a number that is pretty representative of the
results. Repeated values get gzipped down to almost nothing and then
they could still OOM the instance. But 50mb of results might get
squashed down to ~8mb of text. We want to be tethered close the size in
a csv or in memory.
Also changed the serialization to write chunks of rows and not a single
row. Its totally possible that the gzip fix did all of the work and
chunks or individual rows all get put together into the
bufferedoutputstream so it doesn't matter anyways. But the good thing is
that the format of this file is entirely the work of this namespace and
no one else gets to see it.
* Check for fidelity of regular results and disk based
timing wise: running the same card 356 (a join of people, products,
orders) and verify timing stays consistent (we are only 100ms longer
with disk)
```clojure
temp-storage=> (time
(-> (toucan2.core/select-one-fn :dataset_query :model/Card :id 356)
metabase.query-processor/process-query
:data :rows count))
"Elapsed time: 740.709583 msecs"
18760
temp-storage=>
(time (->
(metabase.notification.payload.execute/execute-card 1 356)
:result
(select-keys [:notification/truncated?
:status :running_time
:data.rows-file-size
:data])
(update :data (fn [d] (select-keys d [:rows])))))
"Elapsed time: 840.608 msecs"
{:status :completed,
:running_time 724,
:data.rows-file-size 8056845,
:data {:rows #object[metabase.notification.payload.temp_storage.StreamingTempFileStorage
"0x3336077d"
{:status :pending, :val nil}]}}
```
Compare in memory results:
```clojure
temp-storage=> (let [card-id 356
regular-rows (-> (toucan2.core/select-one-fn :dataset_query :model/Card :id card-id)
(metabase.query-processor/process-query)
:data :rows)
disk-rows (-> (metabase.notification.payload.execute/execute-card 1 356)
:result :data :rows)
dereffed @disk-rows]
(doseq [[f results] [["/tmp/results.from-disk" dereffed]
["/tmp/results.from-qp" regular-rows]]]
(with-open [w (clojure.java.io/writer f)]
(doseq [row results]
(io/copy (str (pr-str row) \newline) w))))
[(= regular-rows dereffed) (= (count regular-rows) (count dereffed))])
[true true]
```
and then compare the files written above:
```shell
/tmp
❯ diff results.from-disk results.from-qp
/tmp
❯ echo $?
0
```
* weird postgres types
* postgres 12 didn't like gen_random_uuid so just hardcode one
everyone else promised they wouldn't use
7e3cd49d-bfe1-4620-83dd-0c163719175c for anything important
* formatting, lints
* alignment
* reformat, remove unused aliases
* remove errant comment
* cleanup logic and add some comments
unconditionally add to rowcount and in-memory collection of rows. For
two of three branches, this is "done": (1) collecting in memory, and (2)
streaming to disk but our "batch" isn't full. Only when the batch is a
multiple of 5,000 rows (number completely picked at random, balancing
this) we can then write the row block to disk.
Totally possible this can switch to just writing to disk on each row and
rely on the buffered writer to handle batching. But the use of the
in-memory collection being similar across almost all of the branches
makes this solution feel a bit cleaner for the moment.
also does things like inverting `if-not` so the short branch is at the
top and not hanging at the end where it's esay to miss.
* unify two file size settings
used to have some notion of `enforce` and `file-size-limits` Rather than
juggle both, you can just set the filesize to 0 to make it unbounded
* reuse human-readable-size in more places, fix some typos
* typos, fixes
* prometheus metrics
* followup to metrics
I’ve added a new metric
```clojure
(prometheus/counter :metabase-notification/temp-storage
{:description "Number and type of temporary storage uses"
;; memory, disk, above-threshold, truncated, not-limited
:labels [:storage]})
```
to record the different scenarios of query results:
- stayed in `memory`
- wrote them to `disk` but remained under the threshold
- wrote to disk and they went `above-threshold`
- wrote to disk and they went so far above threshold that we abandoned the query so the results were `truncated`
- wrote to disk but the limit was set to 0 so the results were `not-limited`
and these look like: (replaced # with ♯ so git message will include)
```
❯ http get :9191/metrics | grep temp
♯ HELP metabase_notification_temp_storage_total Number and type of temporary storage uses
♯ TYPE metabase_notification_temp_storage_total counter
metabase_notification_temp_storage_total{storage="memory",} 2.0
metabase_notification_temp_storage_total{storage="disk",} 2.0
metabase_notification_temp_storage_total{storage="truncated",} 2.0
```
* make predicates a bit more clojurey
* missed a refactor somehow
* better styling
thread the human readable size through
* styling changes
* bit of formatting
---------
Co-authored-by: Alexander Polyankin <alexander.polyankin@metabase.com>
* Add currency label style options to formatting.md
Added options for displaying currency labels in the formatting section.
* Clarify currency label style options
Updated the currency label style section to clarify options for displaying currency labels, including examples for Canadian dollars.
* Apply suggestions from code review
Co-authored-by: Jeff Bruemmer <jeff.bruemmer@gmail.com>
---------
Co-authored-by: Jeff Bruemmer <jeff.bruemmer@gmail.com>
* subtler background lines on cartesian charts
* increase font size on cartesian charts and legends to 13px
* improve size and spacing of legends for pies and choropleths
* change legend color to text-primary
* change case and color of pie chart center subtext
* change small funnel text to 13px
* tweak the number of y-axis tick marks when set to auto for small charts
* Update Loki Snapshots
* Update Loki Snapshots
* update specs, polish
* specs
* specs
* cleanup
* Update Loki Snapshots
* subtler background lines on cartesian charts
* increase font size on cartesian charts and legends to 13px
* improve size and spacing of legends for pies and choropleths
* change legend color to text-primary
* change case and color of pie chart center subtext
* change small funnel text to 13px
* tweak the number of y-axis tick marks when set to auto for small charts
* Update Loki Snapshots
merge in updated loki tests
* update specs, polish
* specs
* specs
* cleanup
* Update Loki Snapshots
merge master's loki snapshots
* Update Loki Snapshots
* update be specs
* move border-subtle definition from colors.module to colors.ts
* table border color fixes
* fix overly thick dashboard heading border-bottom
* Cursor's attempt at fixing related tests
* Cursor again attempts to fix tests
* specs
* cleanup
* another attempt to fix specs
* Update Loki Snapshots
* fix cartesian chart and pivot table lines in dark mode
* Update Loki Snapshots
* attempt to fix chart test
* spec
* unrelated test fix but I need to merge this PR
---------
Co-authored-by: Metabase Automation <github-automation@metabase.com>
Co-authored-by: Aleksandr Lesnenko <alxnddr@gmail.com>
Co-authored-by: Aleksandr Lesnenko <alxnddr@users.noreply.github.com>
* command palette update
Updated keyboard shortcuts for opening the chat sidebar in Metabot documentation.
* Simplify custom action creation instructions
Removed instructions for creating a custom action from the command palette.
* Update appearance settings reference to 'Branding'
* Remove Metabot reference from exploration.md
Removed mention of asking Metabot in the exploration section.
* Add 'boolean' to filters page
Added boolean expressions support to custom filter documentation.
* Add boolean keywords to expressions doc
Updated section on conditional operators to include comparison and boolean logic operators.
* Fix anchor so old links work
* Fix missing period
Co-authored-by: Alex Yarosh <alexandra@metabase.com>
---------
Co-authored-by: Alex Yarosh <alexandra@metabase.com>
When using the `underlying-records` drill ("See these Orders") on a bucketed temporal value, such as a data point in
a time series, use a `:between` filter with the endpoints of the bucket, rather than `:=`.
The queries work with `:=` (because the QP expands them to `:between`) but some parts of the filter UI
render it badly. With `:between`, the filter is rendered properly as a date range.
* Make Metabot logo gray when brand color customization is defined
* Add MetabotLogo stories and test them with Loki
Signed-off-by: Jakub Chodorowicz <jakub@chodorowicz.com>
---------
Signed-off-by: Jakub Chodorowicz <jakub@chodorowicz.com>
Fixes QUE-2561
For example, in our Sample Database, `Orders.PRODUCT_ID` is remapped to
`Products.TITLE` by default. If you go to the **Table Metadata** in admin
and set `Products.TITLE` to "Do not include"
(which is `:visibility-type :sensitive` internally) it should not be
used for remapping.
* Add useEffectOnceIf hook
* Fix entity picker list not scrolling to chosen element
* Remove explicit dependencies
* Move useEffectOnceIf to VirtualizedList folder
We had tried this previously, but the athena driver has these libraries
uberjar'd into it. So we need this to be in lockstep. See
https://github.com/metabase/metabase/pull/63894 for more info.
but i'll include some info here:
there are two copies on the classpath:
```clojure
user=> (enumeration-seq (.. (Thread/currentThread) getContextClassLoader (getResources "io/netty/buffer/ByteBufAllocator.class")))
(#object[java.net.URL 0x3bb34fe "jar:file:/private/tmp/fixed-athena/plugins/bigquery-cloud-sdk.metabase-driver.jar!/io/netty/buffer/ByteBufAllocator.class"]
#object[java.net.URL 0x4ac1867d "jar:file:/private/tmp/fixed-athena/plugins/athena.metabase-driver.jar!/io/netty/buffer/ByteBufAllocator.class"])
```
one in the bigquery driver jar and one in the athena driver jar. The
bigquery driver jar has an explicit dep and we can control it
independently. The athena jar has it uberjar'd in and we can't control
it independently. So we want to make sure that we are relying on the
version that is (unfortunately) baked into athena.
For this we can see the athena 3.6.0 jar has the following
pom.properties file:
`META-INF/maven/io.netty/netty-buffer/pom.properties`
```properties
artifactId=netty-buffer
groupId=io.netty
version=4.1.124.Final
```
So we will pin that version in bigquery as well so we only have a
single version (but still two copies) on the classpath.
This way whichever classfile is resolved (and this can vary depending on
whether an athena db or bigquery db is synced first, or health checked)
it should behave the same
* fix long snippet names overflow
* add percent width so it uses most space both in SQL models and SQL editor
* fix sidebar width in transforms page
* for SnippetRow, use flex instead of width percentages, fix collection row
* fix snippetrow background so it expands snippet instead of inserting it into the editor
When investigating a separate issue I've noticed that we're logging "Loading ..." even for entities that are not actually being loaded. That's because we were printing this line when something is a dependency - even if it exists in the database and is not provided in the export.
This changes the approach somewhat, we will still have some indication we need some entity, but only at DEBUG level, and instead the "Loading ..." will appear only if we're genuinely about to read something from an export and put it in the database.
Not only it's easier to understand what's going on, it's going to be way less noisy.
* WRK-903 Update command palette modal (#64039)
* Remove metabot from command palette
* Remove footer
* Style search input
* Icon color
* Remove actions section when there is no search query
* Rename 'recent items' to 'recents'
* Show 10 recents by default
* Style item
* Update copy (Search results -> Results)
* Use project react import convention
* Move link to search results to group header
* Close palette on results link click
* Fix types
* Use disabled prop in useCommandPalette
* Update item bg
* Align search icon
* Update color whem is active
* Update table icon
* Fix wrong item being selected
* Use stack
* Fix icons shrinking
* Hide subtext on overflow
* Update unit tests
* Fix types
* Clean up
* Hide actions in a different way
* Update helper
* Remove redundant changes
* Update unit tests
* Update unit tests
* Update helper
* Update test
* Update test
* Update tests
* Update test
* Update test
* Update tests
* Add a border to search input
* Update test
* Simplify text overflow handling
* Avoid parent().parent()
* ESLint
* Improve assertion
* Improve assertion
* Remove borders between result groups in command palette
* update styles of the app bar (#64105)
* update styles of the app bar
* rework profilelink to ts
* fix some tests
* fix another e2e test
* update search style
* fix shortcuts conflict
* unify search buttons between oss and enterprise
* don't translate metakey
* fix colors
* let's have two keydown handlers
* fix test
* fix typecheck
* address review comments
* fix showplow event in test
* show metabot icon on small screens
* fix test
* do not show metabot button within iframe
* attempt to fix flaky test
* lint
* address review comments
* address review comments
* add a unit test
* update colors
* address comments
* update skeleton
---------
Co-authored-by: Kamil Mielnik <kamil@kamilmielnik.com>
* support returning one action with "onClick" to call it immediately
* feat(sdk): add question to the clicked object, part of EMB-796
* feat(sdk): clean up the data we pass as `clicked`, part of EMB-796
makes export types checker happy
remove source
* add stories
* only require `onClick` for the "immediate" actions
* address feedback: fix type of onClick and add warning in case of wrong
shape
* allow cc review to be skipped
* Update .github/workflows/claude-code-review.yaml
Co-authored-by: metabase-bot[bot] <109303359+metabase-bot[bot]@users.noreply.github.com>
* trying to retrigger when labels change
---------
Co-authored-by: metabase-bot[bot] <109303359+metabase-bot[bot]@users.noreply.github.com>
* Claude code review (experimental)
* Run my job PLEASE!
* Fix syntax
* More guidelines
* Minor change to see if we can get some useful feedback
* Seriously run my damn workflow
* Add GH token
* Remove paths
* Appease Claude
* Tell Claude how to get the Clojure Style Guide
* Attempt Clojure style guide again
* Give Claude ReadFile perms for the style guide
* Move info about community style guide to CLAUDE.md
* Try again
* Another pass
* Seriously please follow Community style guidelines
* More advice.
* LEARN ABOUT TEST NAMING CLAUDE!!!!
* More advice
* Appease Kondo
* I have a lot to say I guess
* Iterate on CLAUDE.md
* Clarify module linter cheating rule
* CLAUDE u iz dumb
* Chaos
* Dumb as a rocc
* EVEN. MORE. GUIDELINES.
* Clarify semicolon rules more
* Threaten our AI overlords
* Are you dense?
* Annoyed
* BE SUPER DUPER EXPLICIT
* Fun time
* fun
* More dumb comment examples
* Clarify defmethod rules
* SERIOUSLY!!!
* I don't think Claude can count
* More examples of what to avoid
* U not listening
* Clarify
* Use emojys
* Parens
* Be. Even. More. Explicit. About. Semicolons.
* Be SUPER EXPLICIT
* Clarify
* Discourage `qp.store/with-metadata-provider`
* Move SmartScalar subcomponents to separate files
* Update getValueHeight to support multiple comparisons
* Fix dashcard header height
* Fix overflow with cheap tactics
* Hide all but 1 comparisons if they don't fit
* Fix line height
* Move PreviousValueComparison to its own directory
* Extract DetailCandidate
* Extract VariationPercent
* Extract DetailCandidate
* Remove inTooltip prop in VariationPercent
* Add hidden comparisons to the tooltip
* Update constant
* Add ellipsis badge
* Do not use " • " as text separator
* Make tooltip text white
* Wrap text in the tooltip
* Fix text alignment
* Revert static viz change
* Add e2e test
* Update tests
* Update Loki Snapshots
* Remove separator in static viz
* Update Loki Snapshots
---------
Co-authored-by: Metabase Automation <github-automation@metabase.com>
* Allow passing error/friendly in malli schema's to declare them to be be safe for users to see
* Add a user-friendly error message when aggregations don't contain aggregations
* Allow :error/message to be a thunk
* Add repro for metabase#61893
* Use deferred-tru for error message
* Remove deferred-tru
* Fix regression with :async? requests
- callbacks were not passed and clj-http would throw
* Fix data correctness issue if cancelled during table read
Fixes an issue where cancellation during the read stage
could cause the python run to receive (and output) truncated/empty incorrect data and write back / succeed anyway.
* Add cancellation test
* Give script a bit more time for CI
* Disable cancellation test for problematic drivers
* Also disable redshift
* Also disable mongo
* Fix regression due to merge conflict
* disable comments in an iframe
* Remove comments from UI for embedded scenario
* Make isWithinIframe check external for CommentsMenu
* Add e2e test to check that comments are not available in embedded app
* Update Loki Snapshots
---------
Co-authored-by: stasgavrylov <stasgavrylov@gmail.com>
Co-authored-by: Metabase Automation <github-automation@metabase.com>
* Add missing class to ClickActionIcon
* Remove superflous transition
* Move ClickActionsTextIcon to css
* Move ClickActionControl to css modules and metabase/ui
* Tweak css to match prevouis button height
* Split fields returned for cards/models into core fields and related_tables
* Update tests, and fix implementation to always include :related_tables
* typo
* fix kondo
* add separate with_related_tables flag to get-table-details API
* add test for new flag
* fix malli recursive schema
* add back query-needed?
* support related-tables when fetching table details as well, not just models
* add get-table-details-related-tables-test
* don't overwrite tyler's 404 fix
* Extract lazy-evaluation of currencyMapCache to an iife
* Add getCurrencySymbolNative
* Add narrow symbol support
* Update unit tests
* Add test
* Add e2e test
* Fix issue with current value not being shown
* Fix#54317: Object detail navigation state doesn't reset
Added a verification that caps the selected object index.
* Fix#54317: Changed useState and reseted index.
Now using useEffect and reseting to the first index.
* Fix#54317: Added an e2e test
* Adjust test to new filters
---------
Co-authored-by: Sébastien <sebastien@metabase.com>
Co-authored-by: Jakub Chodorowicz <jakub.chodorowicz@metabase.com>
Co-authored-by: Jakub Chodorowicz <jakub@chodorowicz.com>
* js source of truth for colors
* use more betterer colors
* keyboard shortcut for toggling dark mode
* Adds colorConfig for defining a light and dark color for each color variable
* Copies over dark colors from `dark-mode` branch (excluding variables that don't exist in colorConfig`
* Ports other style changes from `dark-mode` branch
* Revert 3301ed93693
* Ports mantine theme logic from `dark-mode` but uses SSOT colorConfig instead of SEMANTIC_COLOR_SCHEMES
* Fix TS errors
* Fix text color and random white bg on permissions table
* Removing a bunch of usages of JS color... probably breaking things
* Updates mantine color scheme when resolvedColorScheme changes
* a bunch more
* fix progress bars. shame on Nick
* Stubs out forcing light mantine theme when isEmbedding (since it has its own dark mode)
* Fix hard-to-read dark mode straggler caused by one-off bg color
* add orionAlphaInverse colors
* make lighthouse illustration work with CSS vars
* Fixes hover states of query builder header buttons in dark mode
* Updates PivotTables in dark mode to behave similarly to night mode
* Fix upsell CTA hover color in dark mode
* Tweaks admin nav dark mode color
* some small fixes
* Fixes some incorrect brand color values when white-labeling
* Tweaks admin nav bar colors and uses same colors on dark mode as light mode (if it ain't broke...)
* Fix gauge visualization text/input colors in dark mode
* Makes things legible and more consistent in query builder in dark mode
* Fixes a few filter chip stragglers in dark mode
* metabot visits the dark side
* Use ColorSchemeToggle for dashboard night mode button
* Prevents dashboard from using night mode when theme=night is set in the hash
* Update color.ts
* Revert "Prevents dashboard from using night mode when theme=night is set in the hash"
This reverts commit 06a504c088e31c60aca7c7550aa24a0f32edba6c.
* set color scheme for scrollbars
* hard to see toggle in database info
* Embedding and other fixed
* fix blue brand
* Uses canonical dark mode for dashboard night mode and static embedding's dark mode
* Fixes issue where Static Embedding preview pane doesn't show checkerboard background when "Dashboard background" is turned off
* Updates find-never-defined-css-variables.ts so it looks for defined variables in colors.ts. Removes references to now-removed base colors.
* Fixes some TS errors
* happy-ish static viz
* only resolve to color var if name is known
* type corrections, removing auto color scheme
* Passes storybook theme arg to ThemeProvider
* add konami code shortcut for Vamsi
* Fix dashboard context TS errors
* Fix runtime warning
* Prevents whitelabel colors from being used in colors.ts when token lacks whitelabel feature. Updates `filter` and `summarize` colors to use whitelabel value if present.
* re-enable updateColors
* happy types
* Updates storybook renderer to work with stories that set the theme via theme: "dark"
* Fix issue where DatesRangeSdk story doesn't pick up brand color
* green unit tests?
* Reverts unnecessary change carried over from original `dark-mode` POC which changed some loki/storybook backgrounds
* Fix issue where theme switching themes in Question embed previews doesn't appear to do anything
* handfull of e2e suites
* Revives some `theme` infrastructure needed to support `theme=transparent`
* Restores SdkError colors on dark mode (#64306)
* happy unit and e2e tests?
* Fix lint errors
* feat(sdk-bundle): use existing tooltip configuration for dark mode (#64325)
* fix card shadow crash
* backdrop color fix
* Refines ColorSchemeToggle. Color scheme is "auto" by default. When "auto", color scheme is automatically updated when browser/device settings is changed. Switching to the theme that's defined as the browser/device preference will set the color scheme to "auto".
* Adds clarifying "(system preference)" to color scheme toggle tooltip
* embedding e2e, unit tests
* Fix TS error
* update colorConfig for sdkThemeProvider
* force e2e to run in light mode
* Fix embedding-sdk:docs:generate error
* Updates ThemeProvider to ignore from hash if we're outside of static embeds. This prevents old night mode dashboard urls from messing with the new dark mode.
* Fix issue where dark mode shortcut isn't pulling in the latest values (not toggling right)
* Replace hardcoded value with already-defined constant
* fix quotes in css
Co-authored-by: cubic-dev-ai[bot] <191113872+cubic-dev-ai[bot]@users.noreply.github.com>
* use firefox compatible scale
Co-authored-by: cubic-dev-ai[bot] <191113872+cubic-dev-ai[bot]@users.noreply.github.com>
* fix css quotes
* fix color scheme toggle
* fix test
* fix conflict
* update test
* PR Feedback
* fix embedding homepage flicker
* e2e adjustment
* adding text-hover, fixing checkbox border
* setup.cy green plz
* e2e test for toggling color scheme
* e2e plz
* type check
* e2e adjustment
* Update Loki Snapshots
* test colors in RGB
* OS check for shortcut
---------
Co-authored-by: Ryan Laurie <iethree@gmail.com>
Co-authored-by: Nick Fitzpatrick <nickfitz.582@gmail.com>
Co-authored-by: Brad Anderson <brad@metabase.com>
Co-authored-by: AlessioLaiso <7041012+AlessioLaiso@users.noreply.github.com>
Co-authored-by: Phoomparin Mano <poom@metabase.com>
Co-authored-by: Ryan Laurie <30528226+iethree@users.noreply.github.com>
Co-authored-by: cubic-dev-ai[bot] <191113872+cubic-dev-ai[bot]@users.noreply.github.com>
Co-authored-by: Metabase Automation <github-automation@metabase.com>
Instead of hard-coding a "metabase-ai" as the only add-on,
we're now parametrizing this endpoint to accept different
product types, which translates into buying different add-ons.
* Alternative fix for internal queries
reenable the admin screen for http://localhost:3000/admin/tools/errors
I know we aren't supposed to look at any piece of a query, including
type here, but there are a few reasons i'm doing so.
1. i don't know of any predicates in mbql lib for this
metabase.lib.core doesn't have a `native?` predicate, nor an `mbql?`
predicate. It does seem have predicates about stages but that doesn't
seem applicable here
2. even if it _did_, the internal query is not recognized by lib, so we
then have to teach it internal queries which means the other PR (#64375)
has to be implemented either way, defeating the purpose of this very
simple PR.
```clojure
user=> (metabase.lib.core/any-native-stage? {:limit 1,
:type :internal,
:fn "metabase-enterprise.audit-app.pages.queries/bad-table",
:args [nil nil nil "last_run_at" "desc"],
:offset 0,
:parameters []})
Execution error (ExceptionInfo) at metabase.lib.walk.util/fn$fn (util.cljc:72).
Invalid input: {:lib/type ["missing required key, got: nil"], :stages ["missing required key, got: nil"]}
```
Why aren't there tests?
🤠
Want to contrast the two approaches first. Happy to backfill tests once
the approach is agreeable. This approach here is _very_ simple and lets
the working qp approach continue working without disturbing
metabase.lib. This seems pretty nice because ultimately these internal
queries should/will be replaced. But in the meantime, do we want to
teach lib about them, or just skirt it altogether?
* test that api works for these queries
* enable feature in test
The python code is executed by the runner but its output is not saved back. The endpoint blocks
until the output is received.
Any objects in S3 should be deleted before the request returns (or will otherwise be collected via retention)
Limits:
- Run timeout is set to 60 seconds
- Samples up to 100 rows from each input table.
- Outputs up to 100 rows from the python output.
Fixes#63245.
Add `:reversed? true` option to `lib.walk/walk-stages` that reverses the order.
Then use that option for `qp.middleware.add-implicit-joins/second-pass`, since that `second-pass`
is about making sure if stage N needs an implicitly joined column from the previous stage, that the
previous stage actually returns it. #63245 happens when the join clause and usage are 2+ stages apart.
Flipping the iteration order will push the need for the implicitly joined column through as many stages
as necessary.
* use local datetime instead of zoned datetime when creating exclusion filters from parameters
* use local datetime instead of zoned datetime when creating exclusion filters from parameters
* Configure ESLint to prevent test files going into SDK build
* Fix SDK build to not include test files
* Fix the config to allow __support__/metadata
* Revert a file I thought was a test file
* Make rules consistent
* Clean up tsconfig pattern
* Make the error message less repeatable
* Fix incorrect separator on FE and add guard to setting
* Reproduction
* when-some
* Add FE test
* naming
* Update frontend/src/metabase/admin/settings/components/widgets/FormattingWidget.unit.spec.tsx
Co-authored-by: Ryan Laurie <30528226+iethree@users.noreply.github.com>
---------
Co-authored-by: Ryan Laurie <30528226+iethree@users.noreply.github.com>
* WRK-905: Allow commenting on an unsaved document
* Fix e2e tests
* Remove 'disabled' prop from CommentsMenu. Remove force click in comments test
* Allow editing document when comments sidebar is opened
* Fix linting
* Fix e2e tests
* Address PR comments. Allow charts editing when comments sidebar is open
* store transform revisions
* Add tests
* remove unneeded arity
* fix whitespace
* include impl.transform in revisions.init
* move transform test to enterprise folder
* use =? and check unneeded keys were removed
* revert =? use
* dissoc source from transform comparison
* Allow token without exp, but raise a warning
* Fix typing
* Fix wrong doc link anchor
* Fix doc link accessible label
* Add a test
* Fix sso endpoint calling repeating when jwt doesn't contain `exp`
* Fix unit tests
* Fix test for real!
* Fix test name typo
Co-authored-by: Nicolò Pretto <info@npretto.com>
* Use correct terminology "session"
Co-authored-by: Nicolò Pretto <info@npretto.com>
* Fix showing no EXP claim warning when the token is being fetched
Co-authored-by: Nicolò Pretto <info@npretto.com>
---------
Co-authored-by: Nicolò Pretto <info@npretto.com>
Fixes https://github.com/metabase/metabase/issues/62179
Before (and bad) (wordwrap introduced for clarity):
```
2025-08-14 16:17:36,662 WARN app-db.liquibase :: The following
changesets were not rolled back. Likely because they are not in the
changelog file: v56.2025-06-05T16:48:48, v56.2025-06-06T20:11:53,
v56.2025-06-06T20:11:54, v56.2025-06-06T20:11:55,
v56.2025-06-06T20:11:56, v56.2025-06-06T20:11:58,
v56.2025-06-13T15:00:00, v56.2025-07-07T08:02:43,
v56.2025-07-17T20:11:55, v56.2025-07-17T20:11:56 2025-08-14 16:17:36,662
INFO liquibase.logging :: Successfully released change log lock ```
```
And now:
```
2025-10-01 16:00:19,326 INFO app-db.liquibase :: Rolling back app database schema to version 55
2025-10-01 16:00:19,326 INFO app-db.liquibase :: Going to roll back changeset v56.2025-07-17T20:11:56
Rolling Back Changeset: migrations/056_update_migrations.yaml::v56.2025-07-17T20:11:56::edpaget
2025-10-01 16:00:19,328 ERROR app-db.liquibase :: Error rolling back migration v56.2025-07-17T20:11:56
liquibase.exception.RollbackFailedException: liquibase.exception.DatabaseException: ERROR: column "jwt_attributes" does not exist
Position: 41 [Failed SQL: (0) UPDATE core_user SET login_attributes = jwt_attributes WHERE sso_source = 'jwt']
at liquibase.changelog.ChangeSet.rollback(ChangeSet.java:1041)
...
Caused by: liquibase.exception.DatabaseException: ERROR: column "jwt_attributes" does not exist
Position: 41 [Failed SQL: (0) UPDATE core_user SET login_attributes = jwt_attributes WHERE sso_source = 'jwt']
at liquibase.executor.jvm.JdbcExecutor$ExecuteStatementCallback.doInStatement(JdbcExecutor.java:520)
... omitted
... 65 more
Caused by: org.postgresql.util.PSQLException: ERROR: column "jwt_attributes" does not exist
Position: 41
at org.postgresql.core.v3.QueryExecutorImpl.receiveErrorResponse(QueryExecutorImpl.java:2734)
... 70 more
2025-10-01 16:00:19,330 WARN app-db.liquibase :: The following
changesets were not rolled back. Likely because there were errors in
rollback (v56.2025-07-17T20:11:56): v56.2025-06-06T20:11:53,
v56.2025-06-06T20:11:54, v56.2025-06-06T20:11:55,
v56.2025-06-06T20:11:56, v56.2025-06-06T20:11:58,
v56.2025-06-13T15:00:00, v56.2025-07-07T08:02:43,
v56.2025-07-17T20:11:55, v56.2025-07-17T20:11:56 2025-10-01 16:00:19,330
INFO liquibase.logging :: Successfully released change log lock
```
How i recreated:
connected to a jar:
```
MB_DB_CONNECTION_URI="postgres://dan:password@localhost:5432/downgrade-checks" MB_JETTY_PORT=2999 java "$(socket-repl 6000)" -jar $JARS/1.56.6.2.jar
```
```sql
downgrade-checks=# alter table core_user drop column jwt_attributes;
ALTER TABLE
```
this will make a downgrade fail:
```yaml
- changeSet:
id: v56.2025-07-17T20:11:56
author: edpaget
comment: |
Add sync login_attributes with jwt_attributes, we need to replace jwt_attributes
with login attributes for users with sso_source jwt so that they will get the new
behavior where login_attributes can override jwt_attributes
changes:
- sql: >-
UPDATE core_user
SET jwt_attributes = login_attributes, login_attributes = '{}'
WHERE sso_source = 'jwt';
rollback:
- sql: >-
UPDATE core_user
SET login_attributes = jwt_attributes
WHERE sso_source = 'jwt';
```
This will fail as the `jwt_attributes` column has been removed.
```
Error rolling back migration v56.2025-07-17T20:11:56
liquibase.exception.RollbackFailedException: liquibase.exception.DatabaseException: ERROR: column "jwt_attributes" does not exist
Position: 41 [Failed SQL: (0) UPDATE core_user SET login_attributes = jwt_attributes WHERE sso_source = 'jwt']
```
And then connect to the jar again, redefine the changes here, and run
```clojure
(let [data-source (metabase.app-db.core/data-source)]
(with-open [conn (.getConnection ^javax.sql.DataSource data-source)]
(with-liquibase [liquibase conn]
(rollback-major-version! conn liquibase false))))
```
* Fix `u/slugify` function's max-length option
`u/slugify` url-encoded non-ASCII characters, then took `max-length`
chars off the result.
Both make sense in isolation - but if we're slugifying, e.g., the string
勇士, with max-length 10, this results in:
- first, url encode non-ASCII characters => "%E5%8B%87%E5%A3%AB"
- then, take 10 characters => "%E5%8B%87%"
But a bare `%` character, as in the trailing character above, is not
valid URI syntax.
Discovered in [UXW-666: Users can't download results for questions with
long cyrillic names, error "URL
malformed"](https://linear.app/metabase/issue/UXW-666/users-cant-download-results-for-questions-with-long-cyrillic-names)
but there are probably more cases where this has caused mischief.
* fix test
These log lines will now be debug level warnings rather warning level warnings.
```
<lib.normalize> Error normalizing MBQL 5
<lib.field.resolution> We tried every trick we could think
<lib.join> Failed to find matching column in join
```
Intrepid readers will also notice that moved "Resolved field %s to an
expression. ..." to debug as well.
* Fix downloading tables grouped by datetime->int
The Linear task has a slightly more involved setup. I found that I could
replicate the issue with a slightly simpler setup:
- set up a simple query:
- select COUNT from ORDERS
- group by the day of the month
- log in as the sandboxed user
- try to download the card results (as CSV, XLSX, etc)
- see the error
Why does this happen? I found that, for a sandboxed user, the query
results show the "day of the month" column as having
```
:base_type :type/DateTime,
:database_type "TIMESTAMP",
:effective_type :type/Integer,
```
while an admin user sees the same column as having
```
:base_type :type/Integer,
:database_type "INTEGER",
:effective_type :type/Integer,
```
This seems bizarre and worth looking more into, but honestly shouldn't
really matter - if the effective type is an integer, we certainly
shouldn't be formatting it as a datetime.
`field-is-type?` is only used for checking whether a field is temporal
for formatting purposes, so I think this is a relatively low risk
change, and note that `metabase.lib.types.isa/temporal?` (the
replacement suggested by the deprecation notice for
`metabase.types.core/temporal-field?`) returns `false` for both
cases.
[UXW-660: Pivot tables with integer breakout columns fail to download as formatted XLSX for sandboxed tables](https://linear.app/metabase/issue/UXW-660/pivot-tables-with-integer-breakout-columns-fail-to-download-as)
* Write a test
* add python runner timeout as setting
* set timeout status
* add timeout status test
* appease kondo
* proper script
* Fix if: expr in upload runner logs
* Fix python runner log name
* Remove setting qualifier
* Add python-runner-timeout-seconds to FE settings type
* Ensure premium feature enabled so setting can be changed
---------
Co-authored-by: Dan Stone <wotbrew@gmail.com>
* Add context to Save verb
* Add key
* Extract button from nested t
* Find other instances of nested c(...).jt`...${<x>...</x>}...` calls and unnest them
* Remove extra interval tick when there is only one interval
* Add repro for metabase#63671
* Update Loki Snapshots
---------
Co-authored-by: Metabase Automation <github-automation@metabase.com>
* Fix can't drill SQL questions in embedding SDK
* Add tests
* Fix the audit DB should not be able to see SQL
* Fix every CI check failing from updating from master
* Address code feedback, make it more clean
* Fix transducer order
* pin shape
* Update docs/questions/visualizations/map.md
Co-authored-by: Jeff Bruemmer <jeff.bruemmer@gmail.com>
* wording
Clarified limitations on data points for maps and added details about pin types.
* image
---------
Co-authored-by: Jeff Bruemmer <jeff.bruemmer@gmail.com>
Very silly bug, we were just populating the ID with the wrong
thing (`(:database-id table)` instead of `(:db_id table)`), so the FE
couldn't build links correctly.
There was a mismatch between `recent_views` and `popular_items`
endpoints. They have different queries, but are using the same
postprocessing function. So I needed to unify them: both now use
`:db_id`.
* Fix ssh tunneling for clickhouse
* Remove questionmark
* Use kebab case keys
* Fix kondo;
This is also fixed in https://github.com/metabase/metabase/pull/63645,
so there may be a conflict.
* Don't squash exceptions.
* remove some tests, fix some tests
I removed some tests. Let's be very clear why this is happening. Push
back if you disagree:
```clojure
(deftest clickhouse-connection-fails-test
(mt/test-driver :clickhouse
(mt/with-temp [:model/Database db {:details (assoc (:details (mt/db)) :password "wrongpassword") :engine :clickhouse}]
(testing "`driver/database-supports? :uploads` does not throw even if the connection fails."
(is (false? (driver/database-supports? :clickhouse :uploads db)))
(is (false? (driver/database-supports? :clickhouse :uploads nil))))
(testing "`driver/database-supports? :connection-impersonation` does not throw even if the connection fails."
(is (false? (driver/database-supports? :clickhouse :connection-impersonation db)))
(is (false? (driver/database-supports? :clickhouse :connection-impersonation nil))))
(testing (str "`sql-jdbc.conn/connection-details->spec` does not throw even if the connection fails, "
"and doesn't include the `select_sequential_consistency` parameter.")
(is (nil? (:select_sequential_consistency (sql-jdbc.conn/connection-details->spec :clickhouse db))))
(is (nil? (:select_sequential_consistency (sql-jdbc.conn/connection-details->spec :clickhouse nil))))))))
```
This test is _GONE_ because it's insane that we are testing that
`conneciton-details->spec` and `database-supports?` is resilient to
opening a connection to a server. That's nuts. We don't do that any
more. Should be the same as if checking `(+ 1 1)` doesn't open a network
connection. We get version information in
```clojure
(def ^:private ^{:arglists '([database])} get-clickhouse-version
(memoize/ttl
^{:clojure.core.memoize/args-fn (fn [[database]] (:details database))}
(fn [database]
(sql-jdbc.execute/do-with-connection-with-options
:clickhouse
(sql-jdbc.conn/db->pooled-connection-spec database)
nil
(fn [^java.sql.Connection conn]
(with-open [ver-stmt (.createStatement conn)
ver-rset (.executeQuery ver-stmt clickhouse-version-query)
cloud-stmt (.createStatement conn)
cloud-rset (.executeQuery cloud-stmt "SELECT value='1' FROM system.settings WHERE name='cloud_mode'")]
(cond-> nil
(.next ver-rset)
(assoc :version (.getString ver-rset 1)
:semantic-version {:major (.getInt ver-rset 2)
:minor (.getInt ver-rset 3)})
(.next cloud-rset)
(assoc :cloud (.getBoolean cloud-rset 1)))))))
:ttl/threshold default-cache-ttl))
(defmethod driver/dbms-version :clickhouse
[_driver db]
(get-clickhouse-version db))
```
We don't need to run this query every time you get a connection spec or
ask if the database supports something. We have the version and if it's
cloud and that's sufficient. The only reason it was opening a connection
was to run the cloud check (`select value=1 from system.settings where
name=cloud_mode`) above.
Also removed:
```clojure
(deftest impersonation-support-check-over-ssh-tunnel-test
(testing "driver-supports? :connection-impersonation should work through an SSH tunnel (#62377)"
;; Use an in-memory SSH server that forwards to the correct port through localhost.
;; Since the real database is still reachable locally, it's possible for the driver to ignore the SSH tunnel and
;; still connect to the DB. So this test runs twice, once with all correct and once with an incorrect SSH
;; password. The latter check should fail (returning false for `driver-supports?`) if the SSH tunnel is being
;; respected.
(mt/test-driver :clickhouse
(let [username "username", password "password"]
(with-open [ssh-server (driver.tu/basic-auth-ssh-server username password)]
(doseq [[correct-password? ssh-password] [[true password] [false "wrong-password"]]]
(memoize/memo-clear! @#'clickhouse.version/get-clickhouse-version)
(let [ssh-port (.getPort ssh-server)]
(with-ssh-tunnel! {:tunnel-enabled true
:tunnel-host "localhost"
:tunnel-auth-option "password"
:tunnel-port ssh-port
:tunnel-user username
:tunnel-pass ssh-password}
(testing (str "using " ssh-password)
(is (= correct-password?
(driver/database-supports? :clickhouse :connection-impersonation (mt/id)))))))))))))
```
Again, this is verifying connection resiliency in a call
to `(driver/database-supports? :clickhouse
:connection-impersonation (mt/id))` and ensuring that it uses the ssh
tunnel. This entire concern is gone, and a red flag that it ever
existed.
Changes:
a few tests used the `db` from `with-temp` to do stuff, but now that we
require them to have dbms_version i populated that
in the version check i thought i saw one in test going through with
`dbms-version` and not `dbms_version` so i'm checking both just to be
sure. it's a bit sloppy
* small changes and add back test
* handle dbms-version vs. dbms_version
* fix kondo
---------
Co-authored-by: Riley Thompson <riley@metabase.com>
Co-authored-by: dan sutton <dan@dpsutton.com>
Co-authored-by: Riley Thompson <rileythompson99@gmail.com>
* Factor out setup/log actions for python runner, use in driver tests
* Remove redundant cd
Co-authored-by: Nicolò Maria Mezzopera <nicolo.mezzopera@metabase.com>
---------
Co-authored-by: Nicolò Maria Mezzopera <nicolo.mezzopera@metabase.com>
* Pass className to native editor
* Add missing border on top of native editor
* Set minimal editor height so the results panel is always visible
* Move initial height calculation to hook
* Add missing import
* Use late-binding for handlers so they can change without having to recreate markers
* Allow isEditing props to re-render Map component
* Allow disabling zoom control on maps
* Allow clicking the Map action buttons
* Disable zoom and hover when in editing mode
* Add repro for metabase#64138
* Fix e2e tests
* UXW-1766 Fixes issue where getDashboardHeaderParameters looks at all cached dashcards rather than just the ones from the currently viewed dashboard
* UXW-1766 Adds/updates unit tests
We're introducing a new endpoint in metabase/metabase. It sends the feedback to the backend which then (1) fetches the premium token and (2) constructs the correct store api URL with the same feedback as the payload. Our backend acts as a proxy because the api url contains the token and we can't send it from the FE.
* Extract native query content for search indexing rather than the entire dataset_query structure
* remove inline def
* Fix doc string
* Update search spec field-fields logic to support the :fn syntax to fix test
* Remvoe query_type from expected card spec fields as we're no longer using it to detect native query changes
* Address feedback on :fn semantics, and add a more detailed docstring to define-spec macro
* feat: use new detail view in metadata preview
* drop relationships
* fix e2e helper
* use correct '
* make table responsive
* update helper
* adjust values in tests
* use useSampleData
* rework approach
* update helper
* oops, lost changes
* restore test
* let's add table just in case
* another attempt
* responsive
* restore indexes
* rename index -> rowNumber
* show details group in more cases
* remove force: true
* fix index
* When deleting DB, hide cards from search
We have a current issue with search: deleting items does not actually
remove them from search.
This is a particularly nasty version of this, because while usually a
delete follows an update (that marks the card as archived,
which *does* remove it from search), in this case we're just deleting
the items directly as part of an `ON DELETE CASCADE`.
This is a bit hacky, but for now we can just mark them as `archived`
before we delete them to hide them from search.
[UXW-602: Search API returns inexisting cards](https://linear.app/metabase/issue/UXW-602/search-api-returns-inexisting-cards)
* Add i18n / formatting to existing log calls
* Cleanup log conveyance to failure :message
* Improve error feedback capability
- Ensure exception messages do not leak to user logs by default 'Something went wrong'
- Exceptions can carry a `transform-message` key to provide a custom failure message (with i18n)
- Defined messaging for:
- Error during table copy
- Error during execution
- Error during writeback
I would like to follow up with additional context in the form of message log entries for further progress / context of any failure.
Logs can be correlated to metabase logs for stack traces or further debugging. Python output remains presented as-is (traceback etc).
* [match] Prevent match-lite from extracting common result that depends on uncommon bindings
* [match] Fail if match-lite's :guard is provided with incorrect predicate syntax
Also fix its treatment of named functions passed as predicates.
* Inline AdminPaneLayout
* Do not show invite button on deactivated tab
* Simplify
* Use modern button
* Add tabs in People page
* Move mb
* Fix typo
* Make people search input full width
* Move tabs to the top
* UXW-333 Prevents `updateDashboardAndCards` from being called when discarding changes (still called when clicking "Save changes")
* UXW-333 Adds e2e test
* for fetching tiles encode field refs as safe base64
* clojure implementation
* update public api
* pass lat and lon fields in query params
* specs
* fix specs
* update preview endpoints and tests
* clean up previous url safe base64 implementation
* fix specs
* UXW-709 When drilling into a dashboard card that had a filter applied, the resulting query can now be saved outside of the original dashboard. "New Question"/"New SQL query" via dashboard edit screen behavior is unaffected (still forces you to save to that dashboard).
* UXW-709 Updates unit tests
* Allow redirect url from password reset page
* Add await when inviting user so that errors will be displayed
* Allow for different email content when inviting user from setup / Set returnUrl param
* Code refinements
* Add backend tests
* Add frontend tests
* Run formatter
* handle email content when invitor's first name is empty/unset
* Allow running dev on custom MB_FRONTEND_DEV_PORT
Signed-off-by: Jakub Chodorowicz <jakub@chodorowicz.com>
* Unify port env var naming between JS and Clojure
* Add dev server port info
* Update devenv docs
Co-authored-by: Sébastien <sebastien@metabase.com>
* update docs
---------
Signed-off-by: Jakub Chodorowicz <jakub@chodorowicz.com>
Co-authored-by: Sébastien <sebastien@metabase.com>
* Add working table with 2 columns
* Table layout and dnd polishing
* ResizeNode
* Refactor to use grid container
* Add Card node title ellipsification
* cleanup, add min height
* Fix FlexContainer to work with ResizeNode
* Fix empty ResizeNode after card dnd
* Document layout no dnd kit (#63363)
* using tiptap drag and drop
* Fix some issues with dnd
* Add support for resize node, fix styles
* Remove comments
---------
Co-authored-by: Nick Fitzpatrick <nickfitz.582@gmail.com>
* Fixes after merge comments
* Fix issues with dnd logic
* Fix issue where rearranging columns within a flexContainer didn't respect which side of a cardEmbed we dropped; it always inserted dragged column to the left.
* Fix issue where moving a cardEmbed out of a flexContainer and placing it right after doesn't wrap the card in a resizer
* Fix issues with dropzones and card id
* Fix issue with column resize handlers
* Fix issues with cards previews on dnd
* Feat: UXW-717 Reconcile comments with document layout (#63593)
* UXW-717 Add comment option to cardEmbed menu
* UXW-717 Wires up comment option in cardEmbed menu
* UXW-717 Stubs out CommentsMenu reorganization so logic can be reused inside cardEmbeds (button and menu item)
* UXW-717 Design tweaks per design feedback (and fixes ts error)
* UXW-717 Updates hover state of cardEmbed ellipsis menu trigger per design feedback
* Fix issues with funnel chart and resizer
* Fix issues with column rearrange
* Fix adding third column position
* UXW-739 Adds ResizeNode wrapper to charts added by metabot
* UXW-749 [1/2] Fix: Cutting/copying a chart from a group then pasting it (cmd+v) results in a chart without a resizeNode
* UXW-749 [2/2] Fix: Copying and pasting from one chart to another doesn't preserve the structure correctly
* Refactor dnd logic - consolidate all use cases in HandleEditorDrop
* remove redundant message and add_message icons
* actually remove redundant message icons
* replace references to old icons with new icons
* Fix issue with dropping a card into existing flexContainer
* Fix issue with moving a card from one flexContainer into another
* UXW-748 - Fix broken document when typing with a card selected
* Handle dropping cards onto paragraphs (#63754)
* Handle dropping cards onto paragraphs
* PR Feedback
* cleanup flex containers
---------
Co-authored-by: Denis Berezin <denis.berezin@metabase.com>
* small feedbacks
* breaking up document drop handler
* UXW-689 - add basic e2e tests
* type fix
* loki diff
* Add more e2e tests
* Fix typecheck
* happy types
* passing document.cy.spec
* Fix e2e tests
* e2e test for card resizing
* flex container column resize tests
* Add node deletion e2e tests
* Fix review issues
* Try to fix typecheck
* Refactor useCommentsButtonProps to a react component
* final PR feedback
* fix removing a card via menu
---------
Co-authored-by: Nick Fitzpatrick <nickfitz.582@gmail.com>
Co-authored-by: Brad Anderson <brad@metabase.com>
Co-authored-by: Maz Ameli <maz@metabase.com>
Co-authored-by: Nick Fitzpatrick <nick@metabase.com>
Co-authored-by: Kamil Mielnik <kamil@kamilmielnik.com>
Previously `extract-query "NativeQuerySnippet"` would use standard method for exporting entities, which checks for
collection_id in entity and prevents its export if it's outside requested collections.
While that makes sense for Cards (and we do escape analysis for those), snippets live in their own separate collection
namespace and this logic is not relevant to them. So now we export either those in requested collections _or_
dependencies of cards that are being exported.
* cherry pick driver code (#63628)
* cherry pick driver code
* update changelog
* add type->database-type unit tests
* kondo
* create/insert table
* insert-from-jsonl test
* missing rename-tables for snowflake
* extend rename-tables-test to more drivers
* line breaks
* add driver python transform feature
* handle review comments
* kondo
* feat: replace string->val with a more generic insert-col->val
* remove non transactional impls
* chore: docsstrings
* TODO
* extra test
* fix extra test
* test all base-types
* fix sqls
* setup ci properly (#63693)
* S3 transport layer for python-runner (#63697)
* Add s3 dependencies (#63706)
* port prometheus/instrumentation (#63631)
* port prometheus metrics
* port instrumentation
* instrument existing transform paths
* wider buckets
* remove nil handling
* warn on nil bytes/rows
* add stage-type mapping
* typo
* split CI into separate jobs (#63746)
* Add python_runner.clj (#63747)
* Add python runner
* Kondo
* Traceback fix
* Remove dead code
* start the runner by default
* Cancellation not used yet
* Remove unused keys from fields-metadata
* cherry pick driver fixes
* start for postgres and mysql as well
* clickhouse fixes
* relax for snowflake
* typo
* exclude transform-python-test from appdb
* exclude transforms-python-test as well
* try exclude-tag here too
---------
Co-authored-by: Ngoc Khuat <qn.khuat@gmail.com>
Co-authored-by: Nicola Mometto <nicola.mometto@metabase.com>
* execute.clj (#63755)
* execute.clj
* with drivers test and execute test
* with wait tables
* missing utils
* /s/open/start
* fix failing tests due to getting the wrong source and target
* kondo
* move cancel-python-code-http-call to python-runner
* Dan's feedback
* we have 2 implementstation of cancel
* and 2 db-routing-enabled?
---------
Co-authored-by: Nicola Mometto <nicola.mometto@metabase.com>
* Transforms python APIs (#63762)
* transforms python api
* new tests pass locally
* clean up the db id utils
* move to util
* fix: missing refer
* chore: missing tests/kondo
* kondo
* more kondo
* fix: make function resilient
* eh, validate on old
* on new
* Add python transform support for jobs (#63764)
* Add python transforms job support
* Fix whitespace in when
* Avoid churn due to new local
* Add feature test
* Add ignore for logs (there might be away to avoid this)
* Use new target-database-id util
* Remove duplicated fn
---------
Co-authored-by: Nicola Mometto <nicola.mometto@metabase.com>
Co-authored-by: Dan Stone <wotbrew@gmail.com>
* Move transform ordering tests to the python module (#63827)
* Add transform ordering tests to the python module
- Consistent with api tests for python transforms being in the python
module
- In practice transforms module 'knows' about python, so this is
organizational only
- Copied test helpers for now, not sure if functions strong enough to be
shared
* Check :transforms/python
* Proj python/table name len (#63842)
* feat: ensure temp-table-name respects max length
* add unit test
* add limit for missing drivers
* depend on it
* fix: use db instead of db-id
* Proj ptyhon/table swap (#63878)
* add rename feature, rename-table! multimethod
* add table swap with atomic or rename or fallback
* changelog
* drop requirement
* ensure we're testing replace flow
* wait for completion, table already exists
* wip
* add rename-table-test
* fix for snowflake
* fix: support bigints (#63898)
* fix: support bigints
* merge meta
* Frontend for Python Transforms (#63699)
* cherry pick driver code (#63628)
* cherry pick driver code
* update changelog
* add type->database-type unit tests
* kondo
* create/insert table
* insert-from-jsonl test
* missing rename-tables for snowflake
* extend rename-tables-test to more drivers
* line breaks
* add driver python transform feature
* handle review comments
* kondo
* feat: replace string->val with a more generic insert-col->val
* remove non transactional impls
* chore: docsstrings
* TODO
* extra test
* fix extra test
* test all base-types
* fix sqls
* setup ci properly (#63693)
* feat: add Python transform frontend pages
* Integerate with FE
* Fix wrong source field name used for showing script in transform view
* Revert "Fix wrong source field name used for showing script in transform view"
This reverts commit c7dfb541f3.
* Fix edit button not doing anything for python scripts
Closes QUE-2300
* call api proxy instead of internal call, also have settings page for transform
* python transforms now properly being scheduled
* FE select table when creating python transform
* multiple tables
* ensure tables are unique in tables selector
* source alignments
* run test
* dead code
* Better default schema, better docstring
* fixex test run
* add cancel button
* better endpoint name
* add cancel button to test python endpoint
* working button
* Auto-update Python transform function signature when table aliases change
- Automatically updates def transform(...) signature when aliases are modified
- Updates docstring Args section with aligned descriptions and table names
- Adds reset button to restore default slugified aliases
- Validates alias input to only allow valid Python identifiers
- Handles multi-line signatures for many parameters (>80 chars)
- Shows visual distinction between custom and auto-generated aliases
- Fixes code editor layout to prevent overlap with Run button
* feat: settings page with s3 configs
* feat: split settings
* remove placeholders
* eh, rename to appease the munging
* Logs are viewable inline during execution of python transforms
* Some minor fixes around docstring rewriting
* solve more fe lints
* fix type errors
* more type check fixes - including some smelly optional override stuff
* fix delete transform + table
* fix: don't remove indentation before """
* chore: allow empty schemas
* Critical fix: docstring indentation
* Separate file / s3 lifetimes / writes / reads from http calls (#63294)
- python-runner no longer simulates a single http api that does
everything (incl s3 interaction)
- Separates file lifetimes (shared storage) from execution
- Easier to see how to avoid reading into strings (e.g. for piping into
tmp files, follow up)
Earlier on we had a service that managed much of the I/O, metabase now
coordinates it so it makes little sense pretending there is still
an overall http api.
# Conflicts:
# enterprise/backend/src/metabase_enterprise/transforms/api.clj
# enterprise/backend/src/metabase_enterprise/transforms/execute.clj
# enterprise/backend/src/metabase_enterprise/transforms/python_runner.clj
# enterprise/backend/test/metabase_enterprise/transforms/python_runner_test.clj
* Add first-class PythonTransformSource type
# Conflicts:
# enterprise/frontend/src/metabase-enterprise/transforms/components/QueryEditor/EditorBody/EditorBody.tsx
* Lay python editor out in two columns
* Make python editor resizable
* Move parseCSV to utils
* Split off results into separate component
* Add empty and loading state components
* Split up ExecutionResult component
* Move python execution to separate hook
* Remove color in editor gutter
* Increase indentUnit for python
* Allow extension to be passed to CodeEditor
* Add pandas completions
* Use DataGrid for displaying results
* Python library APIs (#63378)
Co-authored-by: Romeo Van Snick <romeo@romeovansnick.be>
# Conflicts:
# enterprise/backend/src/metabase_enterprise/api/routes.clj
# enterprise/backend/src/metabase_enterprise/transforms/python_runner.clj
# enterprise/backend/test/metabase_enterprise/transforms/python_runner_test.clj
# enterprise/frontend/src/metabase-enterprise/api/tags.ts
# resources/migrations/057_update_migrations.yaml
* transforms python module (#63503)
# Conflicts:
# enterprise/backend/src/metabase_enterprise/transforms/api.clj
# enterprise/backend/src/metabase_enterprise/transforms/execute.clj
# enterprise/backend/src/metabase_enterprise/transforms/jobs.clj
# enterprise/backend/src/metabase_enterprise/transforms/util.clj
# enterprise/backend/src/metabase_enterprise/transforms_python/python_runner.clj
# enterprise/backend/test/metabase_enterprise/transforms_python/api_test.clj
# enterprise/backend/test/metabase_enterprise/transforms_python/e2e_test.clj
# enterprise/backend/test/metabase_enterprise/transforms_python/python_runner_test.clj
* Use normal title case
* Handle json-line output
* Remove unused props
* Rename PythonEditor to PythonQueryEditor
* Move script rewriter to utils
* Flatten Python transform page
* Rename shared to common
* Remove dead code
* Reduce specialization in Transform editor
* Remove unused props
* Move PythonQueryEditor to top-level components
* Nest Python editor components in PythonTransformEditor
* Restructure Python transform editor
* Split up component more
* Add helpers for python editor
* Disable default schema
* Add test ids
* Fix import path
* Reuse transform save modal
* Fix parameter
* Add e2e tests for python common lib
* Move python transform api to general transforms folder
* Move type to metabase-api/types
* Disable debugger for python transforms for now, allow to open it with query param
* Disable debugger hotkey for python transforms
* Add syntax highlighting to Python script on Transform page
* Remove cancelation code for now
# Conflicts:
# enterprise/frontend/src/metabase-enterprise/transforms/components/RunButton/RunButton.tsx
# enterprise/frontend/src/metabase-enterprise/transforms/pages/TransformPage/RunSection/RunSection.tsx
* Add more generic RunInfo
* Remove unused LogOutput
* Set up python runner in CI (#63566)
* Check out with submodules in CI
* Include python-runner in docker compose for tests
* Set up python runner in CI
* Fix import
* Update python-runner
# Conflicts:
# python-runner
* Fix font sizes
* Fix table layout
* Revert "Remove unused LogOutput"
This reverts commit 90c0704344.
* Add LogOutput back to runsection
* Remove duplicate schema field
* Use css instead of styled components
* Fix LogOutput styling
* Add transforms/python feature
* Do not allow picking unsupported databases
* Use slugify from metabase/lib/formatting
* Add type for transform alias map
* Fix some type imports
* Refactor PythonDataPicker
# Conflicts:
# enterprise/frontend/src/metabase-enterprise/transforms/components/PythonTransformEditor/PythonDataPicker/PythonDataPicker.tsx
* Further refactors
* Use data selector for database picker
* Use data picker to pick tables
* Remove unused var
* Improve tests for python transform
* Fix duplicate imports
* Use DatabaseId in python transform
* Remove source/draft ambiguity in types
* Move library types to metabase-types/api
* Remove api transforms
* Remove vibe-comment
* Remove unused component
* Validate source more
* Move TableInput to separate file
* Remove TableOption type
* Simplify props
* Move AliasInput to separate file
* Remove unused props
* Add border to empty results
* Fix import
* Remove extension
* Move draft back to new python transform page
* Fix new python transform page nesting
* Simpify new transform page structure
* Add analytics to history revision and data reference (#63587)
* Transform run cancellation (#62387)
* Add cancellation to transform api
* Use effect to set polling state
* Add cancellation button to run button
* Set up cancellation on transform page
* Use confirmation modal to cancel transform
* Add run cancel button to run list
* Make api endpoint take transform id instead of run id.
* Fix cancelation api
* Add canceled state to TransformRunStatus
* Add canceled state to transform page
* Add polling to RunListPage
* Remove comment in code
* Set status to canceling when you click cancel
* Add canceling status
* Poll when the transform is canceling
* Add copy for runs that are being canceled
* Manually track fetching status for running and canceling
* Handle canceling status in utils
* Avoid wrapping cancel button in run list
* Add test for canceling run on transform page
* Add e2e for canceling test from runs page
* Fix prop type
* Change copy to Canceling
* Add custom color to canceled state
* Remove mock code
* Increase cancelation tiemout
* Fix copy in text
* Fix snowplow event
* Remove void specifier and close modal in handler directyle
* Add optimistic update for run cancelation
* Avoid manually tracking cancelation status
* Avoid useEffect when changing isPolling
* Add in missing helper
* Remove another useEffect
* Add placeholder run when starting
---------
Co-authored-by: Eric Normand <ericwnormand@gmail.com>
Co-authored-by: Alexander Polyankin <alexander.polyankin@metabase.com>
* Fix command z not working in connection string input (#63689)
- Stop preventing default event handling onPaste.
- Improve Cypress paste emulation to be compatible with React way of handling paste events.
* Transforms prepare for python (#63698)
* pull out transform run utils to prepare for python transforms
* fix: check running-run on transform-id, not run-id
* chore: put on a new line
* Fix#63629: Filter on cards causes an unnecessary horizontal scroll bar for public/static embedding (#63650)
* Closes EMB-50 tabs dimmed out on iOS (#63690)
* S3 transport layer for python-runner (#63697)
* Add s3 dependencies (#63706)
* port prometheus/instrumentation (#63631)
* port prometheus metrics
* port instrumentation
* instrument existing transform paths
* wider buckets
* remove nil handling
* warn on nil bytes/rows
* add stage-type mapping
* typo
* Implement data picker according to designs
* Further adjust to design file
* Fix some e2e tests
* Clone python runner when running CI
* Avoid starting the container in e2e tests twice
* Hide python runner admin route when user does not have the feature for it
* Remove unused variables
* Move Python runner amdin page to transforms plugin
* Do not show Python library link when the user does not have python transforms support
* Hide Python transforms features when the user does not have transforms-python token
* Use useHasTokenFeature hook where possible
* Add transforms-python feature
* Fix e2e tests
* Do not include python-runner in test docker-compose yet
* Tag tests for python transforms with @transforms-python
* Add helper to update Python library
* Add check for python output
* Fix Revert on python library when it does not exist
* Fix e2e test helper
* Add python runner settings in CI
* WIP
* Translate pandas definitions
* Add AnsiLogs component
* Use AnsiLogs in Logs components
* Use AnsiLogs in LogOutput component
* Remove token feature gate in enterprise folder
* Return alias instead of throwing
* Check type in isPythonTransform assertion
* Use isResourceNotFoundError
* Make transforms-python it's own plugin
* /s/localhost/localstack
* Add test for editing Python transform
* Verify that running the transform with the library works
* Add missing transforms-python feature to mocks
* Simplify source layout
* Change setting type of python runner token
* Fix tests
* Use ConcreteTableIds in python transforms only
* Deduplicate codemirror helpers
* Ignore errors in non-parseable JSON lines
* Use useLocation hook to enable debugger
* Use useErrorMessage helper
* Move SourceSection to transforms-python plugin
* Fix codemirror helpers
* Remove undefined color
* Fix types
* Fix codemirror helpers more
* Move transform ordering tests to the python module (#63827)
* Add transform ordering tests to the python module
- Consistent with api tests for python transforms being in the python
module
- In practice transforms module 'knows' about python, so this is
organizational only
- Copied test helpers for now, not sure if functions strong enough to be
shared
* Check :transforms/python
* Proj python/table name len (#63842)
* feat: ensure temp-table-name respects max length
* add unit test
* add limit for missing drivers
* depend on it
* fix: use db instead of db-id
* Proj ptyhon/table swap (#63878)
* add rename feature, rename-table! multimethod
* add table swap with atomic or rename or fallback
* changelog
* drop requirement
* ensure we're testing replace flow
* wait for completion, table already exists
* wip
* add rename-table-test
* fix for snowflake
* Extract sourceDatabaseId helper
* Fix e2e helpers
* Fix error in table picker
* Add missing test-id
* Fix transform tests
* Validate python source
* Fix issues after merge
* Fix tag test
* Fix other tag test
---------
Co-authored-by: Nicola Mometto <nicola.mometto@metabase.com>
Co-authored-by: Ngoc Khuat <qn.khuat@gmail.com>
Co-authored-by: Nicolo-Maria <nicolo.mezzopera@metabase.com>
Co-authored-by: Dan Stone <wotbrew@gmail.com>
Co-authored-by: Chris Truter <chris@metabase.com>
Co-authored-by: Nemanja Glumac <31325167+nemanjaglumac@users.noreply.github.com>
Co-authored-by: Eric Normand <ericwnormand@gmail.com>
Co-authored-by: Alexander Polyankin <alexander.polyankin@metabase.com>
Co-authored-by: Jakub Chodorowicz <jakub.chodorowicz@metabase.com>
Co-authored-by: Brad Anderson <brad@metabase.com>
Co-authored-by: Sébastien <sebastien@metabase.com>
Co-authored-by: Chris Truter <crisptrutski@users.noreply.github.com>
* Proj python fix tests (#63970)
* add premium feature
* fix tests that didn't run previously
* exclude from vertica
* remove internal id from mongo table-rows
* fix: require transforms/python db feature for python transforms
* missing in stats
* fix for mongo
* whitespaces to appease the linter
* more more whitespace for the linter gods
* disambiguate for the e2e tests
* Some small type fixes for Python transforms (#64021)
* Remove duplicate helper
* Add updateEnterpriseSetting helper for type-correctness
* Add missing python runner settings
* Use correct setting name
---------
Co-authored-by: Ngoc Khuat <qn.khuat@gmail.com>
Co-authored-by: Chris Truter <crisptrutski@users.noreply.github.com>
Co-authored-by: Dan Stone <wotbrew@gmail.com>
Co-authored-by: Romeo Van Snick <romeo@romeovansnick.be>
Co-authored-by: Nicolo-Maria <nicolo.mezzopera@metabase.com>
Co-authored-by: Chris Truter <chris@metabase.com>
Co-authored-by: Nemanja Glumac <31325167+nemanjaglumac@users.noreply.github.com>
Co-authored-by: Eric Normand <ericwnormand@gmail.com>
Co-authored-by: Alexander Polyankin <alexander.polyankin@metabase.com>
Co-authored-by: Jakub Chodorowicz <jakub.chodorowicz@metabase.com>
Co-authored-by: Brad Anderson <brad@metabase.com>
Co-authored-by: Sébastien <sebastien@metabase.com>
* Allow passing language as a plugin to CodeEditor
* Pull out NativeQueryPreview
* Render NativeQueryPreview in the native transform editor
* Relax prop types
* Add e2e test for query preview in transforms
* Fix e2e tests
* Pass query instead of question
* Use CopyButton instead of bespoke component
* Fix unit test
Attempting to use a snippet template-tag with a trailing space in the snippet name (e.g., `{{snippet: foobar }}`) caused an exception to be thrown when running the native query (missing required parameter).
The regex used for identifying snippet tags will now strip trailing spaces appropriately.
* Misc Lib improvements
* Restore deleted function... for now
* Test fix 🔧
* Mark more stuff in Legacy MBQL utils as deprecated
* Use Lib for query->source-card
* Update modules config
* Update tests 🔧
* Fix the X-Rays
* Druid test fix 🔧
* Appease Kondo
* docs: db routing with static embedding
Clarified limitations of database routing in Metabase.
* Update docs/permissions/database-routing.md
Co-authored-by: Jeff Bruemmer <jeff.bruemmer@gmail.com>
---------
Co-authored-by: Jeff Bruemmer <jeff.bruemmer@gmail.com>
* [perf] Suggest drop-in replacements for clojure.core functions from metabase.util.performance
* Fix walk for nil
* [perf] Make metabase.util.performance/reduce compatible with non-iterables (like strings)
Changes to any user attribute for sso users were failing preventing them
from being to set their user locale. This was due to us not checking if
`email` is present in the body of the update request before failing a
`check-403` call if the user has an sso source.
rather than 4.2.4.Final, they will now be "io.netty/netty-buffer
4.1.118.Final" which puts it the same as that in 3.5.0.
The issue is that both the bigquery-cloud-sdk driver and athena driver
get put onto the classpath and it depends on which order the drivers are
added for resolving which version of this netty buffer dep you get.
```clojure
user=> (metabase.classloader.core/the-classloader)
user=> (enumeration-seq (.. (Thread/currentThread) getContextClassLoader (getResources "io/netty/buffer/ByteBufAllocator.class")))
(#object[java.net.URL 0x3bb34fe "jar:file:/private/tmp/fixed-athena/plugins/bigquery-cloud-sdk.metabase-driver.jar!/io/netty/buffer/ByteBufAllocator.class"]
#object[java.net.URL 0x4ac1867d "jar:file:/private/tmp/fixed-athena/plugins/athena.metabase-driver.jar!/io/netty/buffer/ByteBufAllocator.class"])
```
One annoying bit to keep track of this is that bigquery uses a regular
jar that specifies its dependencies, so you can you regular tooling:
```
clj -X:deps tree :aliases '[:drivers]' > /tmp/deps.with.netty.downgrade
```
And then see those deps:
```
. metabase/bigquery-cloud-sdk /Users/dan/projects/work/metabase/modules/drivers/bigquery-cloud-sdk
. com.google.cloud/google-cloud-bigquery 2.54.2
. com.google.cloud/google-cloud-core 2.60.0
. ... [lots omitted]
. org.apache.arrow/arrow-memory-netty 17.0.0
X org.apache.arrow/arrow-memory-core 17.0.0 :older-version
. org.apache.arrow/arrow-memory-netty-buffer-patch 17.0.0
X org.apache.arrow/arrow-memory-core 17.0.0 :older-version
. io.netty/netty-buffer 4.1.110.Final
```
This version was created without any pinning of netty. Checking the
athena version is a bit more tricky. We use a fat jar. But you can
download the source from amazon as a jar with deps:
```
athena-jdbc-3.5.0-lean-jar-and-separate-dependencies-jars
```
https://docs.aws.amazon.com/athena/latest/ug/jdbc-v3-driver-previous-versions.html
This zip includes `runtime-dependencies` which includes
`netty-buffer-4.1.jar`
The pom inside of this jar specifies
```xml
<parent>
<groupId>io.netty</groupId>
<artifactId>netty-parent</artifactId>
<version>4.1.118.Final</version>
</parent>
```
And `META-INF/maven/io.netty/netty-buffer/pom.properties` specifies
```
artifactId=netty-buffer
groupId=io.netty
version=4.1.118.Final
```
So i am confident now that with the pinning in bigquery and this version
here that regardless of class resolution from
"jar:file:/private/tmp/fixed-athena/plugins/bigquery-cloud-sdk.metabase-driver.jar!/io/netty/buffer/ByteBufAllocator.class"
or
"jar:file:/private/tmp/fixed-athena/plugins/athena.metabase-driver.jar!/io/netty/buffer/ByteBufAllocator.class"
you will get the same class and athena shall work again.
* ensure type/* for base-type if otherwise not known
Using the follow query that returns pg arrays:
```sql
SELECT *
FROM (
VALUES
(1, ARRAY['https://example.com', 'https://example.org']),
(2, ARRAY['https://test.com']),
(3, ARRAY['https://foo.com', 'https://bar.com', 'https://baz.com'])
) AS adhoc_table (id, urls);
```
In 55 this returned a nil base-type
```clojure
user=> (metabase.query-processor.metadata/legacy-result-metadata
{:database 2
:type :native,
:native {:template-tags {},
:query "SELECT *\nFROM (\n VALUES\n (1, ARRAY['https://example.com', 'https://example.org']),\n (2, ARRAY['https://test.com']),\n (3, ARRAY['https://foo.com', 'https://bar.com', 'https://baz.com'])\n) AS adhoc_table (id, urls);"}}
1)
[{:display_name "id",
:source :native,
:field_ref [:field "id" {:base-type :type/Integer}],
:ident "native[]__id",
:name "id",
:database_type "int4",
:base_type :type/Integer,
:semantic_type :type/PK}
{:display_name "urls",
:source :native,
:field_ref [:field "urls" {:base-type nil}],
:ident "native[]__urls",
:name "urls",
:database_type "_text",
:base_type nil}]
```
In master, this just fails. If you remove the type checking in places,
it will eventually return similar.
This base-type is computed in sql-jdbc/metadata
```clojure
(mu/defn query-result-metadata :- [:sequential driver-api/schema.metadata.column]
"Default implementation of [[metabase.driver/query-result-metadata]] for JDBC-based drivers. Gets metadata without
actually running a query."
([driver :- :keyword
query :- :map]
(driver-api/with-qp-setup [query query]
(let [database (driver-api/database (driver-api/metadata-provider))
{:keys [query params]} (driver-api/compile query)]
(query-result-metadata driver database query params))))
([driver :- :keyword
database :- driver-api/schema.metadata.database
^String sql :- :string
params :- [:maybe [:sequential :any]]]
(sql-jdbc.execute/do-with-connection-with-options
driver
database
nil
(fn [^java.sql.Connection conn]
(with-open [stmt (sql-jdbc.execute/prepared-statement driver conn sql params)]
(let [rsmeta (.getMetaData stmt)]
(mapv (fn [i]
(let [database-type (.getColumnTypeName rsmeta i)]
{:lib/type :metadata/column
:name (.getColumnLabel rsmeta i)
:database-type database-type
:base-type (or (sql-jdbc.sync.interface/database-type->base-type driver (keyword database-type))
:type/*)}))
(range 1 (inc (.getColumnCount rsmeta))))))))))
```
Before this change, there was no `or` and
`sql-jdbc.sync.interface/database-type->base-type` would just return
nil.
```clojure
user=> (metabase.driver.sql-jdbc.sync/database-type->base-type :postgres (keyword "_text"))
nil
```
Unfortunately this is a multimethod so we can't easily update it in one
spot to fall back to type/*. Each implementation must take care to do
this. A few do (clickhouse does) but a few dont (snowflake does not). So
we update this call site to ensure we have something from the
multimethod or add our own type/*
Why did this start happening now?
> I think it's because I added :metadata/model-metadata to :metabase.lib.schema.info/info at some point
Query info stashes the model metadata which is
```clojure
[:metadata/model-metadata {:optional true} [:maybe [:sequential ::lib.schema.metadata/lib-or-legacy-column]]]
```
We had been throwing errors asynchronously when trying to save metadata,
but now we have on-thread type mismatch for model metadata, which is
what is in the ticket.
* add test to ensure we can handle existing base_type nil's
this got annoying because legacy metadata from the app-db comes from a
different location. I was adding a card schema migration but this is
better here.
The previous commit makes sure we don't put out more bad data, but this
commit here ensures that we can update the card.
* Move cron info up into cron section
* Move schedule explanation logic to utils
* Allow ReactNode as explain message
* Render explain message in cron component
* Add status to transform job page
* Add e2e tests for job error message
* Add assertion to lookup
* Add extract shared logic for run status
* Add missing test-ids
* Select correct section
* Don't send new user login for SDK or Embedded Analytics JS
* Update tests
* Stop using hard-coded string to detect modular embedding headers
* Conform to spacing rule
* add isClosed impls to proxys
* use mt/id in test
* test all drivers
* check if statements are supported, refactor statement-supported? multimethod to drive feature
* add namespace to statements feature
* Only report analytics when not in the preview
This would avoid confusing preview usage with the actual usage
* Ensure analytics would be report even if iframe isn't ready
* Add E2E test
* Make the test pass
* Stop relying on redux store to fetch `site-url`
* Clarify test with comment
* Add tests
* Fix linter
I was naming a file wrong, so it doesn't match unit test spec. This rule
apparently doesn't take tests into account, which is great.
* Add search endpoint for Metabot and adjusted metadata
* More tests and better split out of logic into separate files
* dedupe search terms and description if they happen to be the same
* fix bad merge conflict
* Answer blending sql querying via search tables (#62415)
* Only put tables used in SQL into context
* Change format of injected database content by reusing existing models
* Create separate function for new schema enchancement
Co-authored-by: Sloan Sparger <sloansparger@gmail.com>
* Add table reference for implicitly joinable fields in context injection (#62753)
* Update Metabot profile IDs (#62864)
* [Answer blending] Updates to the metabot admin configs (#62551)
* remove ability to configure the selected collection for default metabot
* add verified toggle
* add migrations for metabot collection configuration changes
* add metabot PUT endpoint, rip out metabot entities endpoints and usages, convert FE over to new FE, fix tests
* remove metabot_entity table, model, usage and refactor code to use updated metabot and metabot_prompts schemas
* enforce verification and regenerate prompts on verification + collection id change
* fix bad references
* fix bad regexp for fetch-mock collections by id endpoints breaking unit tests
* fix kondo
* fix + increase tests for enterprise/backend/test/metabase_enterprise/metabot_v3/tools/api_test.clj
* fix migration boolean default value syntax
* maybe fix be tests
* fix some tests
* enforce verified filter on the search endpoint
* factor in verified check into find-metric
* hide verification setting behind token feature check, fix migration lint, do not migrate collection id for default metabot, fix kondo errors
* fix core tests
* fix kondo
* fix most of the failing serialization tests
* fix migration lints
* fix one test file
* maybe fix migration failures
* move test resource file
* prevent default metabot from getting a collection id configured
* yolo
* add cascade delete to entity drop table migration
* increase unit tests for verification config UI
* remove extra add column
* fix migration... i think for real this time
* for real for real fix migrations
* no really
* tests fixes and add preventing verification toggle if the instance does not have the content_verification feature
* fix failing feature tests in ci
* fix one more
* fix cljfmt issue
---------
Co-authored-by: Tyler Plude <tsplude@gmail.com>
* [Metabot] Quick slash command + dynamic profile ui (#62909)
* quick slash command + dyanmic profile ui
* add back code i did not mean to remove
* fix failing test
* Revert "fix failing test"
This reverts commit b1c7e9efb9.
* [BOT-284] BE for notebook query tool table support (#62710)
* init
* moved query-table endpoint to query-datasource; add support for both model and table ids
* unit test fixes
* use existing card-query and table-query utils
* adding unit tests
* [Answer blending] Respect collection id in search (#63150)
* remove find metric tool
* adjustments to search endpoint
* add tests
* fix failing test in ci
* add way to override the metabot id
* add way to override metabot request id
* fix whitespace lint
* missing paren...
* fix failing test
* New BE `/search` endpoint for ai-service `SearchTool` (#63154)
* new metabot search tool BE endpoint
* remove debug prints
* parallel searches; incomplete effort to return more fields for results
* Simplify result transforming & response schemas
* unit tests
* kondo fixes; pr feedback
* fallback search model list
* remove references to MetabotEntity model which got re-added incorrectly in rebase
* Make internal metabot use verified content by default (#63529)
Update linter for usage of task in metabot-v3
Move suggested-prompts to separate namespace
Add collection condition only when current-user-id
Add startup task
migration
Update tests
Verified cards only
Test
Docstrings
Limit number of entities
Review feedback
* [METABOT] Default use_verified_content to false; tweak prompts (#63843)
* default use_verified_content false; limit amount of prompts
* reworked unit test to not depend on ai-service or appdb contents
* PR feedback
* remove mock of get metrics and models in favor of mt/with-empty-h2-app-db
* Small refactors & simplifications to metabot PUT endpoint and suggested
prompts job
* small simplification, use m/distinct-by for deduping tables
* rm deprecated search_data_sources
* Cleanup and prioritize verified content (#63877)
* Remove now redundant content verification guard
* Prioritize verified content
* Redundant merge
* Test
* ensure field IDs in user_is_viewing context are snake_cased
* Use proper source column alias as name for the model / table fields (#63826)
* Use proper source column alias as name for the model / table fields
* Add name as fallback
* test fix
* switch to using :lib/desired-column-alias, revert test fix
---------
Co-authored-by: Tyler Plude <tsplude@gmail.com>
Co-authored-by: Noah Moss <noahbmoss@gmail.com>
* Fix some tests: consistent use of snake_case, and a :lib/source-column-alias fallback for model field :name
* fix more field_id snake_case issues
* fix another snake case issue - revert result_columns back to result-columns
---------
Co-authored-by: Thomas Schmidt <thomas.heinz.schmidt@gmail.com>
Co-authored-by: Sloan Sparger <sloansparger@gmail.com>
Co-authored-by: Thomas Schmidt <thomas@metabase.com>
Co-authored-by: Sloan Sparger <sloansparger@users.noreply.github.com>
Co-authored-by: Noah Moss <noahbmoss@gmail.com>
Co-authored-by: lbrdnk <lbrdnk@users.noreply.github.com>
* fix: jetty stats handler tests flake
These tests could hang indefinitely on the `(.join server)` statement
because the core.async/go block handling async requests was incorrectly
doing a blocking take `<!!`. This also updates our :dev alias to enable
core.async to print warnings when using blocking operations in `go` blocks.
* fixup! oncomplete dispatch in go block
* fixup! using alts!! in go block
* WRK-893: Update API to store comment html content
* WRK-891: Add HTML content sending in comments. Add proper SmartLink serialization to HTML.
* WRK-891: Update comment email template to display comment content
* WRK-891: Fixes for comment email template. Add space insertion after smartLink
* trigger comment notification by id
* put old api behavior back and cleanup comment notification context
* whitespace
* fix tests
* Fix changeset author
* WRK-891: Fix smart links serialization. Add parent comment display in email
* Add style payload for comments email template
* fix types
* 🙄
* test email changes
* test email again
* make email formatting looking perfect
* cleanup
* require html content to be provided when posting a comment
* fix e2e test
* fix another e2e test
* pass 0 as tiptap placeholder when no label available
---------
Co-authored-by: Oleksandr Solovyov <o@solovyov.net>
Co-authored-by: Uladzimir Havenchyk <uladzimir@metabase.com>
Co-authored-by: Uladzimir Havenchyk <125459446+uladzimirdev@users.noreply.github.com>
* Adjust DB form page width and form fields spacing
* Extract layout field settings to one file
* Use lg spacing between form fields
* Conditionally add labelProps property
* Fix typo
* Refactor deprecated default export
* Simplify responsive max width
* Simplify DB page responsive styles
* Add card background to add db form
having issues with classpath. I _think_ that athena is finding the
bigquery provided version of a netty class and that is failing. but it's
a bit difficult to follow.
downgrading now until we can take time to understand and sort it.
* Remove the +New option for creating new models
* Bring back the from-scratch approach
Based on Alex's requested change, but also realized that you can use the command palette
* Move note about models getting pinned back into the new model section
* Format UI terms using bold instead of italics
* docs: send only attachements
* Apply suggestions from code review
Co-authored-by: Jeff Bruemmer <jeff.bruemmer@gmail.com>
---------
Co-authored-by: Jeff Bruemmer <jeff.bruemmer@gmail.com>
* update test and implementation
* update models query to make more sense
* flake fix: try adding timeouts to Done
* remove cy.wait from tests
* wait for model creation
* modify create model tests
* Don't force reindex on serialization load to be async
* Allow reindex to be controlled when calling load
* Keep load reindex sync for now
* Make reindex async again
* Add a controlled version of NotebookNativePreview
* Use ReactNode instead of JSX.Element
* Add a button to toggle native sql
* Add sql preview to transform query editor
* Pull out resizehandle component
* Add e2e test for converting an MBQL transform to a native transform
* Move native query toggle out of header
* Move Native editor sidebar to separate component
* Fix overflow in native sidebar
* Fix copy on button
* Move connected NotebookNativePreview to query_builder
* Remove canConvertToNative prop
* Rename prop
* Move styles to css
* Closes EMB-816 misleading text for number of rows
* updated e2e tests
* marked a test as flaky until we figure out what is up with it
* addressed legit comment
* Update Loki Snapshots
* loki
---------
Co-authored-by: Metabase Automation <github-automation@metabase.com>
* Add Embedded Analytics JS usage snowplow schema
* Add analytics
* Clarify code with comments
* Fix analytics logic + fix typing
* Add unit tests
* Make E2E embedded analytics JS util supports multiple elements
* Fix code not working in Cypress for some reason
* Add E2E test suite
* Fix import order
* Remove extra line
* Fix E2E from util API change
* Update snowplow/iglu-client-embedded/schemas/com.metabase/embedded_analytics_js/jsonschema/1-0-0
Co-authored-by: Phoomparin Mano <poom@metabase.com>
* Update e2e/test/scenarios/embedding/sdk-iframe-embedding/sdk-iframe-embedding.cy.spec.ts
Co-authored-by: cubic-dev-ai[bot] <191113872+cubic-dev-ai[bot]@users.noreply.github.com>
* Update e2e/support/helpers/e2e-embedding-iframe-sdk-helpers.ts
Co-authored-by: cubic-dev-ai[bot] <191113872+cubic-dev-ai[bot]@users.noreply.github.com>
* Fix E2E by updating a function API
* Pass snowplow schema lint
---------
Co-authored-by: Phoomparin Mano <poom@metabase.com>
Co-authored-by: cubic-dev-ai[bot] <191113872+cubic-dev-ai[bot]@users.noreply.github.com>
* Add cancellation to transform api
* Use effect to set polling state
* Add cancellation button to run button
* Set up cancellation on transform page
* Use confirmation modal to cancel transform
* Add run cancel button to run list
* Make api endpoint take transform id instead of run id.
* Fix cancelation api
* Add canceled state to TransformRunStatus
* Add canceled state to transform page
* Add polling to RunListPage
* Remove comment in code
* Set status to canceling when you click cancel
* Add canceling status
* Poll when the transform is canceling
* Add copy for runs that are being canceled
* Manually track fetching status for running and canceling
* Handle canceling status in utils
* Avoid wrapping cancel button in run list
* Add test for canceling run on transform page
* Add e2e for canceling test from runs page
* Fix prop type
* Change copy to Canceling
* Add custom color to canceled state
* Remove mock code
* Increase cancelation tiemout
* Fix copy in text
* Fix snowplow event
* Remove void specifier and close modal in handler directyle
* Add optimistic update for run cancelation
* Avoid manually tracking cancelation status
* Avoid useEffect when changing isPolling
* Add in missing helper
* Remove another useEffect
* Add placeholder run when starting
---------
Co-authored-by: Eric Normand <ericwnormand@gmail.com>
Co-authored-by: Alexander Polyankin <alexander.polyankin@metabase.com>
* use iframe for changelog
* add query params and tabs
* always show update info
it's useful even if you're on the latest version
* remove metabase as configurable host
* always allow changelog/releases in iframes
* fix tests
* fix tests
* update tests
* use root metabase domains
* fix test
---------
Co-authored-by: John Swanson <john.d.swanson@gmail.com>
* Send parse connection string parsing events only on blur
* Use synchronous last parsing result for monitoring
* Improve test assertions for connection string events
* Make sure we don' track the same parsing event again
* Refactor ts-match to simple if conditions
* Make lastStatus have the same success value depending on non undefined values
* Improve the snowplow Cypress test setup for connection string
* [Admin settings] Show help documentation to connect DB in a side panel
* Add unit tests
* Add more tests / tweaks
* Add e2e tests
* Make sure help content is not rendered if not available
* Self-review tweaks
* Use useDocsUrl in DatabaseHelpSidePanel.tsx
* Use aside element for the side panel
* Use await findBy when possible
* Use JSDoc / Remove unnecessary comment
* Add some style to blockquote
* Add test file for useHelpLink
* Fix failing test
* Add analytics event
* Document password handling for special characters
* Add section on special character passwords in MariaDB
Added information about handling passwords with special characters in MariaDB connection strings.
* Apply suggestions from code review
Co-authored-by: cubic-dev-ai[bot] <191113872+cubic-dev-ai[bot]@users.noreply.github.com>
* Update docs/databases/connections/mysql.md
---------
Co-authored-by: Alex Yarosh <alexandra@metabase.com>
Co-authored-by: cubic-dev-ai[bot] <191113872+cubic-dev-ai[bot]@users.noreply.github.com>
* Fix vertical legend position for cartesian charts
* Set dashcard title font size to 14px
* Decrease legend dot size
* Decrease value labels density for cartesian charts
* Use normal font weight for axis names, labels, etc
* Make y axis split lines solid vs dashed
* Show scalar card title in dashcard header
* Always show pie chart total
* Hide axis names on small dashcards
* Update screenshots
* Fix tests
* Fix number cards font size
* Fix hiding cartesian chart axis
* Fix font styles in other legend components
* Fix e2e tests
* Hide cartesian chart axes independently
* Use different legend style only in dashboards
* Apply min spacing to labels per series
* Fix legend dot size in edit mode
* Update label sizes for funnels and sankey charts
* Use bold font for data labels
* Update screenshots
* prettier
* Update Loki Snapshots
---------
Co-authored-by: Metabase Automation <github-automation@metabase.com>
* Trivial preprocessing middleware => Lib
* Handle already-native MBQL 5 queries
* substitute-persisted-query => Lib
* A bunch of post-processing middleware is actually MBQL-5 ready
* Fixx 🔧
* Test fixes 🔧
* More fixes 🔧
* Fix more stuff
* Wow even more test fixes =(
* (Hopefully) final round of test fixes
* Kondo
* More test fixes 🔧
* Fixes 🔧
* Swap cancel and save button
* Render sidebar in native query editor for transforms
* use toggleSnippetSidebar from props
* Fix snippets sidebar
* Export SelectionRange
* Add aria label for data reference button
* Open database on mount
* Add aria-label to snippets button
* Add e2e tests for sidebar in SQL transforms
* Rename prop for clarity
* Return instead of throwing
* Use Lib.rawNativeQuery
* Pass selected text to native editor
* Fix types
* Move longer handlers to hooks in utils
* Handle empty text and reshuffle code
* add-implicit-clauses middleware => Lib
* Test fixes 🔧
* Some lib.convert fixes from #61398
* Test fixes 🔧
* More test fixes
* Test fixes 🔧
* Handful of test fixes 🔧
* Discourage use of legacy MBQL utils in QP
* Integrate the platinum metadata provider
* Only one failing test left
* Test fix 🔧
* Kondo
* Test fixes 🔧
* Test fix 🔧
* Test fixes 🔧
* Handle legacy pivot viz settings where `base-type` might not be present
* Handle legacy pivot viz settings where `base-type` might not be present
* Handle legacy pivot viz settings where `base-type` might not be present
* Handle legacy pivot viz settings where `base-type` might not be present
* Handle legacy pivot viz settings where `base-type` might not be present
* Handle legacy pivot viz settings where `base-type` might not be present
* Handle legacy pivot viz settings where `base-type` might not be present
* Handle legacy pivot viz settings where `base-type` might not be present
* Handle legacy pivot viz settings where `base-type` might not be present
* Test fix 🔧
* Test fix 🔧
* Kondo
* Test fix
* Test fix 🔧
* Skip failing transforms tests
* Mark failing tests flaky
* Setup sidebar and onboarding page for embedding hub (#62517)
* Add the add data and new dashboard modals to embedding hub (#62528)
* mark sandboxed step as done (#62758)
* Auto-mark 'secure your embeds' step as done when either JWT or SAML is configured (#62736)
Co-authored-by: Nicolò Pretto <info@npretto.com>
* Add UTM tags to documentation links in the embedding hub (#62747)
* Add a wizard to select a table to x-ray in embedding hub (#62715)
* Mark the create embed steps in embedding hub as done from embed flow (#62759)
* Add a dropdown to switch embedding solutions in embedding hub (#62870)
* fix(embed-hub): adds key prop to component passed to jt (#63060)
* feat(embed-hub): checklist endpoint (#62875)
* feat(embed-hub): EMB-770 and EMB-742, add checklist endpoint with
'create-dashboard'
* feat(embed-hub): EMB-740 auto mark "Add data" as done
* Revert "mark sandboxed step as done (#62758)"
This reverts commit be775aceed.
* feat(embed-hub): EMB-768 move sandbox check to the BE
* EMB-772: move all "done" checks on the be, move endpoints to ee
* cleanup
fmt
configure enterprise module to make linting pass
test: activate license to make e2e tests pass again
remove fe unit test now that all the logic is on the be
add doc to the endpoint to maybe the tests will pass?
whitespace?
fixes
idk anymore how to make all the linters be happy
last try
refactor has-configured-sso to a function
revert visibility to see if tests pass
sort config.edn
* tests: remove cy.wait from embed hub tests (#63066)
* Bit of cleanup
- remove warn-on-reflection; there is no interop
- clean up a bit of duplication in sql clause generation
- remove unncessary try/catch around t2 access
- remove a few boolean calls on boolean functions
---------
Co-authored-by: dan sutton <dan@dpsutton.com>
* fix(embed-js): disable the next button when embedded js is disabled (#63134)
* Replace the accordion in embedding hub with steppers (#63271)
* Move embedding settings into its own top-level menu (#63339)
* Add related setting cards to modular embedding and setup guide (#63340)
* Remove the Embed menu from the New menu (#63385)
* Completing later steps on embedding hub should not tick off the former steps (#63393)
* Modular embedding toggle label should depend on toggle states (#63391)
* Add the interactive embedding upsell to the settings page (#63399)
* Lock embed in production step in embedding hub checklist until sso configured (#63449)
* Use ee plugins to check if embedding is enabled in nav (#63446)
* Check for CSV uploads in embedding-hub checklist endpoint (#63501)
* update embedded-analytics-js.md to have consistent quickstart pages
---------
Co-authored-by: Nicolò Pretto <info@npretto.com>
Co-authored-by: dan sutton <dan@dpsutton.com>
Co-authored-by: Alexander Polyankin <alexander.polyankin@metabase.com>
Fixes issue where undo history is deleted on save. Also fixes issue where CommentsSidesheet stays open (big unclosable grey box) when navigating to a different document when it was open.
* Move sqlserver description to extra info tooltip
* Fix mongo additional-options not appearing in advanced-options section
# Conflicts:
# e2e/test/scenarios/admin/databases.cy.spec.js
* Add new DB engine icons, add icon to regular add DB flow
* Refactor old export style
* update h2 icon
* Fix layout of browse databases cards
* Adjust padding of browse databases cards
* Remove empty comment
* Render default db icon in case of empty engine key
* Fall back to DESCRIBE if Athena JDBC reports duplicate column names
* Make test fail another way
* Fix test
* Verify that the hijacked function gets called
* Metadata Provider Platinum Edition
* Revert some formatting changes
* Revert changes to QP tests
* Revert NOCOMMIT changes
* Fix missing ns alias
* Pull in missing changes from other branches
* Misc fixes and changes
* Fixes
* Fix bugs in JVM metadatas
* Fix composed provider not propagating inactive
* JS fixes 🔧
* Test fix 🔧
* Test fixes 🔧
* Test fixes 🔧
* Another JS test fix 🔧
* More FE fixes 🔧
* Don't be too strict
* Set database
* Fix merge conflict
* Fix kondo
---------
Co-authored-by: Alexander Polyankin <alexander.polyankin@metabase.com>
* Sandboxing middleware => Lib
* Test fix 🔧
* Remove some unused stuff
* rename `row-level-restricitions` middleware => `sandboxing`
* Remove TODO
* :model/GroupTableAccessPolicy => :model/Sandbox
* Test fixes 🔧
* Test fix and mark more stuff deprecated
* Test and Kondo fixes 🔧
* "Fix" sandboxes with extra columns because I don't want to update e2e tests
* cljfmt
* docs - add modular embedding
Also adjust headers a bit because ToC is difficult to scan
* Update docs/embedding/securing-embeds.md
* Update docs/embedding/securing-embeds.md
* Implement new DatabaseFormError component & break DatabaseForm component down
* Implement scrolling actions / Avoid 'div in p' console error
* Fix typing check issues / Add 'Read the docs' link url
* Define 'invite teammate' and 'contact support' urls
* add more tests / further refactoring
* css tweaks
* Simplify tests a bit
* Remove comment / Fix translation cases
* Refactor tests
* Adjust 'Check Host and Port settings' button / Fix css lint
* Fix test
* Add target=_blank to doc links / Use existing color for the error container border
* Use cloud-gateway-ips setting when showing IPs to whitelist / Move some helpers to a local utils.tsx
* Use 'be.visible' over 'exist' in cypress test
* Use useDisclosure
* Make Alert title have dark text color by default / Add storybook case to show warning color variant
Trino's Hive plugin shows the tables in question in its metadata, but if you try to get a handle on these tables, it will always throw an exception with error code `UNSUPPORTED_TABLE_TYPE`. The Iceberg plugin has a similar exception for such circumstances.
For us, there doesn't seem to be much difference between not being able to query a table because it's an unsupported type and not being able to query because we don't have permissions, so`have-select-privilege?` is modified to return false in these cases. We can examine the vendor-specific error code on the SQLException to detect them.
* Update Start new thread UI
* Refactor
* Lint migrations
* Lint migrations
* Lint
* Fix comments count
* Thread submit input improvements
* Align 'Start new thread' button
* mention metabase entities
* Stupid link copying
* Deep linking to comments
* fix: sidebar style improvements
* Fix line breaking bug on submit
* Maz Comments styling tweaks (#62964)
* add tabs hiding/showing logic
* only show persistent button for open comments
* move comment buttons to the right
* attempt to fix tests
* always display the add-a-comment input in the sidebar
* Remove unused attr
* Placeholders without flags
* Refactor comments' tabs markup
* Refactor CommentsMenu
---------
Co-authored-by: Kamil Mielnik <kamil@kamilmielnik.com>
Co-authored-by: Stas Gavrylov <stasgavrylov@gmail.com>
* Fix incorrect counter for unresolved comments
* Fix warning
* Fix key warning in Discussions
* Make document read-only when comments sidebar is open
* Refactor URL generation
* Automatically open correct tab for the comment targeted by URL hash
* Clean up
* Use createPortal instead of Portal from metabase/ui to fix CommentsMenu appearing above CommentsSidesheet when visiting a specific comment via URL
* Remove more/less feature
* Set max-height when editing comments, fix scrolling area
* Extra padding
* Make it possible to mention yourself
* Add ability to resolve deleted threads
* Remove ability to comment on resolved threads
* Remove ability to add new comments on Resolved tab
* allow using @ character without showing mention list
* Fix type errors
* Fix types
* Resolve/unresolve deleted comments
* Remove DocumentContext
* Fix inserting mentions on Enter press
* Minor styling fixes for mentions
* Remove unused endpoint
* Clean up
* Remove unused component
* Remove stories
* Fix storybook
* Remove TODO
* Remove unused file
* Remove outdated comment
* Avoid cast
* Remove redundant code
* Deal with any
* Don't trim empty nodes
* Simplify Discussions component
* Align avatar, submit button and code editor
* Do not show last divider if not necessary
* Get rid of currentIndex
* Fix typo
* Remove hack
* Check only unresolved comments when deciding whether to autofocus comment editor when sidebar opens
* Enable inline_code styling in comments
* Rename css file
* Fix cast
* Use top instead of transform
* Use a flag
* Remove commented-out code
* Move getCommentsUrl to utils
* Use Menu
* Add Avatar to metabase/ui
* Bring back accidentally removed code
* Resolve CSS duplication
* Use void
* Add comment
* Add all comments sidebar
* Use createSelector
* Fix no results styling
* Autofocus editor when editing comments
* Disable menu when editing is disabled
* Add a toast when copying the link
* Prevent deleting and editing someone else's comments
* Fix types
* Update copy
* Cleanup all comments sidebar
* Extract isMetabotBlock
* Use CustomStarterKit
* Copy
* feat: add Emoji picker (#62991)
* feat: add Emoji picker
* fix: styles
* fix: class for loading label
* Add ability to inline emojis in CommentEditor
* fix: keyboard events for emoji picker
* fix: focus visible for emoji picker viewport
* Implement proper keyboard support for inline emojis
* Temporarily hide emoji reaction button
* Improve UX around selecting first emoji
---------
Co-authored-by: Stas Gavrylov <stasgavrylov@gmail.com>
* Pass editable
* Use mentions plugin
* Fix z-index
* Add reactions BE
* Improve list paddings
* Add users support to SmartLink and MentionExtension
* Remove old mentions
* use semantic colors for Avatar
similar logic to what do for users in admin
* show add comment input when we switch from resolved tab
* Limit number of user results
* feat: add comment reactions (#63072)
* feat: add comment reactions
* fix: typescript
* fix: click to react
* fix: styling
* tweak spacing between replies (#63100)
* fix: reply styles
* Update submit shortcut to Cmd-Enter. Fix emoji picking behavior
* fix: do not close discussion sidesheet on some escape events
* fix: cancel editing on escape press
* Make DocumentPage 400px narrower when comments modal is opened
* Extract shared tiptap-related code
* Move EmojiPicker to common/components
* Fix DiscussionComment error when creator is null
* Fix missing formatting menu in CommentEditor
* Add tests boilerplate
* Allow many reactions on a comment
* Hydrate creator info on deleted comments
* Add first test and helpers
* Add tests for more types of nodes
* Add comment
* Improve test name
* Test highlighting
* Add test for splitting paragraphs
* fix: blur comment editor on submit
* Add schema upgrade test
* Add test for creating comments and threads
* Add tests for deleting comments
* Editing a comment
* Test for Esc
* fix: limit editable functionality and add comment styles (#63161)
* Fix test
* Update test
* Fix flaky test
* Fix tooltip
* fix: do not react on deleted comments
* fix: show resolved threads
* fix duplicate replies display
* Add mentions tests
* Make suggestions taller to accommodate users search results without a vertical scrollbar
* don't count deleted comments for the comment badge
* don't show threads if all comments deleted
* Add emoji tests
* fix: can react true by default
* Fix floating menu positioning in CommentEditor
* fix document / menu selected item styling
* Sort comment reactions by created_at ASC
* Make Esc close emoji popup
* first pass at comments api
* Fix reactions ascending order
* WIP
* fix linting
* model resolution
* update test setup to include _id, so equality check works correctly
* replace is_deleted with deleted_at to detect deleted comments
* fix types
* remove ?new=true from url after adding a new comment
* Fix tests
* Improve arrow navigation when emoji popup is opened. Allow selecting with Tab
* Add test
* Refactor
* Update test
* Fix emoji popup Esc handler removal
* Add formatting tests
* Rename test
* Add e2e API helpers to create and delete comments
* Update test name
* Refactor
* Fix types
* update is not deleted eh
* Add test for the comment button
* Add test
* Add a test
* Add test
* test: basic resolve/unresolve tests (#63243)
* test: add resolve other person thread test
* Add test for disabling document editing when comments are open
* Add test for unsaved document changes
* Add test for show all comments button and fix the issue
* Add tests for deleting/editing other people's comments
* Add e2e tests for comment links
* add e2e tests for all comments
* way better/less verbose permission checks
* test: cannot reply to resolved threads
* ESLint
* Update assertion
* Fix unresolved comments count
* e2e tests for comments reactions
* first iteration of notifications
* fix migration lint errors
* fix module tests
* Add label attribute for SmartLinks to simplify server-side notifications rendering
* add e2e test for removing ?new=true after creating a comment
* Fix
* fix clj syntax
* Remove commented-out code
* Use H.readClipboard()
* Refactor
* Update reaction icon
* Move util
* Lint
* Fix
* Lift conditional rendering, move Reaction component to a separate file
* Refactor
* Do not render invalid links
* Refactor
* Update new comment notification template
* add e2e test for top blocks
* fix: comment date to respect browser locale
* Update reactions icon
* Add 1st test for email subscriptions
* Update email template to match figma, remove the comment content
* Add document icon to email template
* Update test
* Remove protocol from the link
* Extract verifyEmail helper
* fix migrations because mysql is majorly unhappy about those rollbacks
* correct urls for comments, notifications on mentions
* Update assertion
* Insert both comment URL and document URL into the email template
* Fix url
* fix copy tests
* Add a setup for next test
* Implement assertions
* 2nd email variant
* Add another test
* Add an assertion
* add more tests for email template
* Ignore whitespace
* Error handling for mutations - toasts
* add more tests
* feedback: drop blue border from paragraph
* feedback: placeholder
* fix e2e test to pass
* Update setup
* Comments design polish (#63352)
* use new Send icon instead of up_arrow
* remove extra space below a Discussion
* restyle Discussions
* fix CommentEditor spacing
* restyle current user reaction
* only show No Comments message on the All Comments sheet
* add on-hover highlighting for All Comments section
* fix for fe-type-check
* fix test
* Cleanup
---------
Co-authored-by: Kamil Mielnik <kamil@kamilmielnik.com>
* put user permissions check back and new api for mentions
* pagination
* Use new GET /api/ee/comment/mentions in frontend
* Use new GET /api/ee/comment/mentions in smart link node
* paid feature in tests
* seed notifications
* oh
* clj-kondo 🤦
* port numbers
* Try to fix flaky test
* attempt to fix flaky e2e
* disable returning users for mentions
* disable mention related tests
---------
Co-authored-by: Stas Gavrylov <stasgavrylov@gmail.com>
Co-authored-by: Anton Kulyk <kuliks.anton@gmail.com>
Co-authored-by: Konstantin Darutkin <spalt08@mail.ru>
Co-authored-by: Maz Ameli <maz@metabase.com>
Co-authored-by: Uladzimir Havenchyk <uladzimir@metabase.com>
Co-authored-by: Oleksandr Solovyov <o@solovyov.net>
Fixes#62492.
A literal UUID was reaching the driver as
`[:value "uuid-string" {:base_type :type/UUID}]`
but this was not recognized as a UUID-valued field because `:value`
clauses have `:base_type` and not `:base-type` like `:field` clauses.
Therefore the `:sql` core driver was rewriting this clause to cast the
UUID field to a string and compare that against the literal string.
With that logic now correctly recognizing such a `:value` clause, the
UUID fields are compared directly. This is much better for indexing,
but also works with *nullable* UUID columns, where the
`CAST(my_uuid AS text)` would cause SQL compilation errors.
* Make with-index! use new, clean, separate database per test
* Add with-test-db! setup modes
* Comment
* Adjust with-test-db! and related
* Drop with-index! in favor of modal with-test-db! in first test
* semantic.tu/db removal
* Drop use of semantic.tu/db in favor of get-pgvector-datasource!
* Drop semantic.tu/db from tu
* Remove once-fixture
* Remove ensure-no-migration-table-fixture
* Comment
* Comment
* Cleanup
* Make the rest of the query-test ns use with-test-db!
* Remove redundant test
* Remove remaining usage of with-index!
* Drop remnants of with-index! and make linter happy
* Cleanup
* Comment
* Remove with-mock-index-metadata! in favor of with-temp-db!
* Adjust with-index-contents!
* Fixture to avoid running app-db tests
* Fixture
* Require
* whitespace
Problem: Perf tests were failing because they couldn't find the uberjar artifact.
The wait_for_uberjar mechanism was trying to wait for a job named 'build (ee)'
but the actual job name is 'Build MB ee'. Additionally, this wait mechanism was
unnecessary complexity.
Solution: Align with pr-env-labeled.yml pattern:
1. Build uberjar first when 'Run-perf' label is added
2. Run perf test after uberjar is built
3. Remove unnecessary wait_for_uberjar mechanism
Changes:
- perf-test-labeled.yml: Add uberjar build job before running perf test
- perf-test.yml: Remove wait_for_uberjar parameter and 56-line wait mechanism
This simplifies the workflow and ensures the uberjar artifact exists before
the perf test tries to fetch it, following the same proven pattern used by
pr-env-labeled.yml.
🤖 Generated with [Claude Code](https://claude.ai/code)
Co-authored-by: Claude <noreply@anthropic.com>
* Make dataset-query->dimensions more robust
* *disable-updates*
* Add macro impls for easier debugging
* Make dataset-query->dimensions more resilient
* Whitespace
* fix: don't ignore db sync/rescan promise
When the call returns non-200, the button still says it's fine instead of showing a failure.
You can test this by trying to sync "Metabase Cloud Storage" DWH.
See https://metaboat.slack.com/archives/CKZEMT1MJ/p1749830993738189
* add unit tests
* fix mocks
---------
Co-authored-by: Ryan Laurie <iethree@gmail.com>
* Ensure results are realized when falling back to in-place results in semantic search
Unlike the semantic and appdb backends, the in-place backend returns a reducible result set from
search.engine/results. Make sure we realize the results before attempting to pass them to `m/indexed-by` to dedup
them. This avoid a "Don't know how to convert reducible to ISeq" exception when falling back to in-place results.
* Add comment
want to check that the new table exists before adding it into the
app-db. The old way worked for all databases but is pretty dumb: it
enumerates all tables in the db (as it would for regular sync) and then
checks if the new table is in there.
We can be better with the `driver/table-exists?` function.
The shape of data coming back from describe-data is the following:
```clojure
notify=> (let [database (toucan2.core/select-one :model/Database :id 1)]
(driver/describe-database :h2 database))
{:tables #{{:name "ORDERS",
:schema "PUBLIC",
:description nil,
:is_writable true}
{:name "PEOPLE",
:schema "PUBLIC",
:description nil,
:is_writable true}
{:name "FEEDBACK",
:schema "PUBLIC",
:description nil,
:is_writable true}
...
```
So we can create the same shape (i omit description and is_writable to
seemingly no deleterious effect).
reports that a customer has around 200k tables spread across 300
schemas. the fields query by default would just get a single cursor into
these fields, passing all schema into the query.
This can take quite a long time. Our own horror show schema would take
~45 minutes to sync. But holding a cursor open this long just seems to
invite bad times.
I've taken the logic and just called it multiple times, once per schema.
* WRK-874: Populate preview for List View customization with sample values
* WRK-874: Set default and min size for list view on dashboard
* WRK-874: Fix List view card size on the dashboard
* WRK-874: Add e2e test for empty column placeholder
* Fix lint errors
* Add horizontal padding for List view
* Refactor generatePreviewSample
* Stretch join data sections when needed
* Tweak padding to avoid jump
* Center 'on' in join condition
* Make gap consistent
* Tighten margins to avoid jump
* Tweak padding to make all wells the same height
* BOT-310 lower the weight on the :rrf score for semantic search results
This seems to give a better blend between the :rrf score coming from the semantic search hybrid query and the other
post-search metadata scorers.
* Add with-weights and with-only-semantic-weights test util macros
* Use semantic.tu/with-weights in scoring tests
* Update query tests to use semantic.tu/with-only-semantic-weights
This prevents other metadata scorers from overriding the expected rankings for these tests. The query tests are
largely focused on making sure the basic machinery of semantic queries are working. Tests exercising individual
scorers are in scoring_test.clj
* Add tests for :official_collection and :verified "premium" scorers
Fixes#35561
We currently don't allow filtering on null foreign key values, but that is a reasonable thing to want to do. This extends the foreign key drill to support null values.
🤖 Generated with [Claude Code](https://claude.ai/code)
Co-authored-by: Claude Code[bot] <noreply@anthropic.com>
Co-authored-by: William Zimrin <william@metabase.com>
* Unpolished WIP hackathon code for additional driver support
* fix kondo
* add transforms/table featuer for supported drivers
* fix transforms for dbs with no schema
* rename transform datasets to prevent foreign key overlap
* Add clickhouse support
* fix clickhouse tests
* fix redshift tests
* fix bigquery failures
* kondo and filter tests by feature
* fix transform dataset for bigquery
* fix sqlserver transforms with joins
* fix clickhouse order by when no non nullable field is available
* add more complex test with join, filter, aggregation and ordering
* use different metadata provider
* filter for left join feature in test
* improve sqlserver compile-transform
* done use legacy mbql in test, refactor normalize-name
* clean up
* add sqlserver error test
* Squashed commit of the following:
commit a11b602e19
Author: Romeo Van Snick <romeo@romeovansnick.be>
Date: Thu Aug 21 17:44:13 2025 +1000
Add message when a new transform will be created
commit 2cfdbfaba3
Author: Romeo Van Snick <romeo@romeovansnick.be>
Date: Thu Aug 21 17:37:36 2025 +1000
Allow creating values in schema form select
commit 17ff4dcb3f
Author: Romeo Van Snick <romeo@romeovansnick.be>
Date: Thu Aug 21 16:51:13 2025 +1000
Allow selecting schemas always, not only when there are schemas
* create schemas
* fix redshift
* add snowflake
* add sqlserver
* add clickhouse
* add bigquery
* Add test
* add default impl
* fix redshift tests
* Move SchemaFormSelect to components folder
* Allow creating schemas in the target update modal
* Allow initial value in SchemaFormSelect
* Only render schema field when the database supports :schemas
* Do not render new schema item if value is empty
* Add unit tests for SchemaFormSelect
* Add e2e tests for schema creation
* check schema exists first
* fix sql jdbc schema exists and add create schema permissions test
* update changelog
* parse query and add into for sqlserver
* add type hints for linter
* Address comments
* realize bigquery execute-raw-queries
* Disable target schema link if it does not exists (yet)
* Add tooltip when schema is new
* Use hydrated table from transform
* Add e2e tests
* Allow existing schema link to render as enabled even when table does not exist yet
* Do not add default to databaseId
* Add tests for existing schema with new table
* Fix e2e test
* address comments
* rename connection-details to conn-spec
* Compare against null not empty string
* use find first
* Await request to avoid flakiness in hover
---------
Co-authored-by: Timothy Dean <timothy@timothydean.dev>
Co-authored-by: Romeo Van Snick <romeo@romeovansnick.be>
* Add last_run, next_run and tag id filters
Closes QUE-2156
* Placate kondo
* Test with scheduled jobs and filtering by tags
* Fix next_run_start_time test
* Return run timestamps as timezone localized strings
* Handle Zulu
* Prepare job list table for next_run and transforms
* Add JobListParams type and use it in getJobListUrl
* Add new parameters to listTransformJobs handler
* Parse params and pass them down on job list page
* Add filters to job list page
* Add tooltipPosition prop to Icon
* Add TimezoneIndicator component
* Allow rendering React nodes in table headers
* Add timezone indicators to job list and transform list
* Add timezoneToUTCOffset helper
* Add timezone to cron explanation
* Format timestamps as ISO-8601 strings with timezone
* Add next run timestamp
* Hide transform count in list for now
* Add next_run to TransformJob
* Add timezone indicators to run list
* Avoid wrapping timestamps in run list
* Move shared widgets to transforms/components
* Allow passing available shortcuts to filter picker
* Disable shortcuts that are in the past for Next run, which is always in the future
* Add e2e test for filters
* Add UTC offset in tests
* Report run timestamps in UTC and expose system timezone
The system timezone is exposed in /api/session/properties as "system-timezone".
* Expose system-timezone to admins only
* Fix setting docstring
* Fix serdes baseline-comparison-test
* Make transform_job API return run timestamps in UTC
* Placate kondo
* Timezone
* Use system-timezone instead of report-timezone
* Add tests
* Fix test string
---------
Co-authored-by: Romeo Van Snick <romeo@romeovansnick.be>
Co-authored-by: Alexander Polyankin <alexander.polyankin@metabase.com>
The Store API's OpenAPI description (OAD) can change over time, but
Metabase's Store client would not notice. For example, the client
would fail a HTTP request, not because the server rejected it, but
because the client "believes" it needs to send a field that the server
no longer requires.
Instead of caching the client, and thus OAD, for the whole lifetime of
the Metabase instance, cache it only for 5min.
Fixes: 1dd56e5f14
References: https://metaboat.slack.com/archives/C010ZSXQY87/p1757002462314229
* add base color rule to stylelint
* allow in colors file
* lint in CI
* better highlighting
* css fixes
* fix all improper color use
* Update frontend/lint/stylelint-rules/no-base-colors.js
Co-authored-by: Phoomparin Mano <poom@metabase.com>
* Update Loki Snapshots
* fix color name
* Update Loki Snapshots
---------
Co-authored-by: Phoomparin Mano <poom@metabase.com>
Co-authored-by: Metabase Automation <github-automation@metabase.com>
* Add transform dependencies and job transforms endpoints
* Add API handler for /api/ee/transform/:id/dependencies
* Add API handler for /api/ee/transform-job/:id/transforms
* Add description to TitleSection
* Add Dependencies section to transform page
* Add transform order to job page
* Invalidate transform jobs when adding tags to transforms
* Add empty state for job transforms
* Add e2e test for transform and job dependencies
* Fix type for job dependencies component
* transform ordering is well defined
* don't sort whole list when we just need the first element
* make new function private
* Use neg?
* Remove unused require
* use a sorted map instead of scanning for the earliest transform each time
* extend docstring
* use ->
* update docstring again
Co-authored-by: metamben <103100869+metamben@users.noreply.github.com>
---------
Co-authored-by: Romeo Van Snick <romeo@romeovansnick.be>
Co-authored-by: William Zimrin <william@metabase.com>
* Update embedded analytics JS docs to include browser
* small update
* new appearance setting
* Update docs/embedding/embedded-analytics-js.md
Co-authored-by: Jeff Bruemmer <jeff.bruemmer@gmail.com>
* Apply suggestion from @jeff-bruemmer
Co-authored-by: Jeff Bruemmer <jeff.bruemmer@gmail.com>
* Apply suggestion from @jeff-bruemmer
Co-authored-by: Jeff Bruemmer <jeff.bruemmer@gmail.com>
* update on the browser
---------
Co-authored-by: Jeff Bruemmer <jeff.bruemmer@gmail.com>
* Update pgvector to pg17 in semantic search docker-compose.yml
This matches the version that will be used in prod and staging deployments.
* Update to pg17 in semantic-search.yml CI workflow
* Remove enable-ssl-tests env var from semantic-search.yml workflow
* Move validate.js to ts and remove unused validators
* Remove any and unknown for typing parameter values
* Use ParameterValuesMap where possible
* Fix parameter-parsing
* Fix tests
* Fix cast to empty string
metabase.session.models.session-test/send-email-on-first-login-from-new-device-test
test is failing because sometimes there's an extra email in there.
metabase-enterprise.impersonation.driver-test/nested-do-with-connection-with-options-test
is failing because it deadlocks all the app-db connections
* Also skip the :bookmarked scorer on mysql
MySQL does not like the CASTs in the search-index-query
* Fix appdb-available-with-semantic test for mysql appdb
When the appdb is :mysql, we should fallback to the in-place engine rather than appdb
* Move body of search.engine/results :search.engine/in-place into a func for easy mocking
* Fix api-engine-switching-test for mysql appdb
* Fix test-semantic-search-fallback-supplementation and test-semantic-search-max-combined-results-limit for mysql appdb
* Update semantic-search CI job to run against all supported appdbs
* Implement last_run_... filters for transform runs
* Change filters to start_time, end_time, run_method
* Use timestamp literals instead of strings in test
* Add start_time, end_time and run_method to transform run query params in api
* Parse start_time, end_time and run_method params on run list page
* Allow custom react elements in displayValue prop
* Add filters for start_time and end_time to run list page
* Allow filtering on multiple run methods
* Add run method filter widget
* Close time filter when changing value
* Add e2e test for start at, end at and run method filters
* Allow multiple values for run_method filter
* Add test with start_time=2025-08-22~2025-08-23
* Fix e2e test
* Rename the run_method parameter to run_methods
* Remove "local" timestamps
* Return transform run timestamps in the system timezone
* Stop expecting a last_run with a nil value
* Rename run_method to run_methods to match api
---------
Co-authored-by: Romeo Van Snick <romeo@romeovansnick.be>
Co-authored-by: Alexander Polyankin <alexander.polyankin@metabase.com>
* Create standalone add DB page
* Use full page view for editing db connection
* Fix title on edit database page
* fix dataspec cypress test
* fix missing formLocation prop
* fix add initial data cypress test
* Fix add db test not waiting for db connected
* Fix default schedule cypress test
* Remove unnecessary nullish coalescing and route
* Extract common logic to use-database-connection hook
* Adjust Add DB page width
* Adjust padding for smaller devices
* Use FormLocation type in prop
* Add repair-index! API with temp table mechanism for detecting deletions
* Fix up tests, bug fixes
* Explicitly disable reindex! and reset-tracking
* Add repair index task, and cleanup job in case there are orphans
* fix tests
* Test - comment out new tests to see if it fixes CI failures
* switch with-index! macro to use init! instead of reindex
* try again to fix with-index
* fix kondo
* ensure drop table table names get quoted
* switch to using gate table for the anti-join
* add missing assertion in test
* add more error handling in find-lost-deletes, reorganize a function
* make sure index cleanup task is initialized
* make sure index repair task is initialized
* Fix time-interval filters
Reported on stats that the examples dashboard was breaking when you used
some relative timestamps. There it was a bit cryptic with
```
class java.lang.Integer cannot be cast to class
clojure.lang.IPersistentMap (java.lang.Integer is in module java.base of
loader 'bootstrap'; clojure.lang.IPersistentMap is in unnamed module of
loader 'app')
```
Locally in dev, we get:
```
Error processing query: Invalid output: ["invalid tuple size 4, expected 5, got: [:time-interval [:field {:base-type :type/DateTime, :lib/uuid \"69655229-95a0-4bfb-9d3a-7675b1111ec4\"} 13] :current :day]"]
```
And it's a failure to have the time-interval options
before:
`[:time-interval field-clause :last (keyword unit)]`
after:
`[:time-interval interval-options field-clause :last (keyword unit)]`
spec:
```
(mbql-clause/define-mbql-clause :time-interval :- :type/Boolean
;; TODO -- we should probably further constrain this so you can't do weird stuff like
;;
;; [:time-interval {} <time> :current :year]
;;
;; using units that don't agree with the expr type
[:tuple
[:= {:decode/normalize common/normalize-keyword} :time-interval]
[:merge ::common/options ::time-interval-options] ;; <---- missing bits
#_expr [:ref ::expression/temporal]
#_n [:multi
{:dispatch (some-fn keyword? string?)}
[true [:enum {:decode/normalize common/normalize-keyword} :current :last :next]]
;; I guess there's no reason you shouldn't be able to do something like 1 + 2 in here
[false [:ref ::expression/integer]]]
#_unit [:ref ::temporal-bucketing/unit.date-time.interval]])
```
Switching them to lib calls and then i noticed there were no tests.
With fix:
```clojure
dates=> (date-string->filter
"lastmonth"
[:field {:lib/uuid "00000000-0000-0000-0000-000000000000", :base-type :type/DateTime} "field"])
[:time-interval
{:lib/uuid "bbca7a83-feb9-4259-a4db-bbb79687eb4e"}
[:field
{:lib/uuid "00000000-0000-0000-0000-000000000000",
:base-type :type/DateTime}
"field"]
:last
:month]
```
In a 56.3.4 repl:
```
user=> (metabase.driver.common.parameters.dates/date-string->filter "lastmonth" [:field {:lib/uuid "00000000-0000-0000-0000-000000000000", :base-type :type/DateTime} "field"])
[:time-interval [:field {:lib/uuid "00000000-0000-0000-0000-000000000000", :base-type :type/DateTime} "field"] :last :month]
user=> (pp)
[:time-interval
[:field {:lib/uuid "00000000-0000-0000-0000-000000000000", :base-type :type/DateTime} "field"]
:last
:month]
nil
```
Note the missing options in the second position
* test e2e dashboard filters
takes a sampling of a few different parameter types and different
parsers. At one point i was doing something like:
```clojure
(concat (for [unit [:day :week :month :quarter :year]
relative [:this :last]]
(str (name relative) (name unit)))
(for [unit '[days weeks months quarters years]
include? [true false]
relative '[past next]]
(str relative "3" (name unit) (when include? "~"))))
```
But that seems overkill. This test is kinda seeing if the query
processor throws an error (or at least that's where most errors
happen). So those checks can be more exhaustive and against some
in-memory metadata providers
* two fixes, compaction and when to run
Test logs are super, duper awesome. We upload them on failure since we
don't presumably need to see healthy logs.
There are two issues:
1. We upload them on `failure()`, but timeouts don't seem to trigger
this condition. So now upload on `always() && !success()` Why not just
`!success()`? An LLM suggests that timeouts are a bit special and go
into another mode. The always() will run and then we qualify it. Let's
see.
2. The logs are all on a single line.
We have a few axes here: compact and eventEol.
I have three files with these. If you want to grep for logs during a
particular test, here's what you get:
Compact: (current behavior)
```
❯ ag "metabase.util.queue-test/result-listener-test" logs/test-log.json.compact
<not reproducing here, because it's the whole file>
```
compact false
```
❯ ag "metabase.util.queue-test/result-listener-test" logs/test-log.json.not-compact
251: "mb-test" : "metabase.util.queue-test/result-listener-test"
268: "mb-test" : "metabase.util.queue-test/result-listener-test"
414: "mb-test" : "metabase.util.queue-test/result-listener-test"
```
It's _too_ expanded.
```
{
"instant" : {
"epochSecond" : 1756844736,
"nanoOfSecond" : 357005000
},
"thread" : "main",
"level" : "INFO",
"loggerName" : "metabase.util",
"message" : "Maximum memory available to JVM: 8.0 GB",
"contextMap" : { },
"endOfBatch" : false,
"loggerFqcn" : "org.apache.logging.log4j.spi.AbstractLogger",
"threadId" : 1,
"threadPriority" : 5
}
{
"instant" : {
"epochSecond" : 1756844739,
"nanoOfSecond" : 802772000
},
```
Compact true, eventEol (this PR sets this)
```
❯ ag "metabase.util.queue-test/result-listener-test" logs/test-log.json
17:{"instant":{"epochSecond":1756844991,"nanoOfSecond":537601000},"thread":"main","level":"INFO","loggerName":"metabase.util.queue","message":"Starting listener \u001B[32mtest-result-listener\u001B[0m with 1 threads 🎧","contextMap":{"mb-test":"metabase.util.queue-test/result-listener-test"},"endOfBatch":false,"loggerFqcn":"org.apache.logging.log4j.spi.AbstractLogger","threadId":1,"threadPriority":5}
18:{"instant":{"epochSecond":1756844991,"nanoOfSecond":572361000},"thread":"main","level":"INFO","loggerName":"metabase.util.queue","message":"Stopping listener test-result-listener...","contextMap":{"mb-test":"metabase.util.queue-test/result-listener-test"},"endOfBatch":false,"loggerFqcn":"org.apache.logging.log4j.spi.AbstractLogger","threadId":1,"threadPriority":5}
20:{"instant":{"epochSecond":1756844991,"nanoOfSecond":585407000},"thread":"main","level":"INFO","loggerName":"metabase.util.queue","message":"Stopping listener test-result-listener...done","contextMap":{"mb-test":"metabase.util.queue-test/result-listener-test"},"endOfBatch":false,"loggerFqcn":"org.apache.logging.log4j.spi.AbstractLogger","threadId":1,"threadPriority":5}
```
You get the whole log lines. super helpful. pipe these through a
processing pipeline, use jq, etc.
Also, these files are ~817MB when running in a test suite. I'm doing these with a single namespace
```
❯ clj -X:dev:test :only metabase.util.queue-test
❯ ls -lh logs
13K Sep 2 15:29 test-log.json
13K Sep 2 15:24 test-log.json.compact
16K Sep 2 15:26 test-log.json.not-compact
```
* stacktraces when we don't get returned connections
* use new hawk version. waiting on release
* quick checkin on CI stuff
* load file to ignore vars
```clojure
;; json file with a list of vars
test-runner=> (println (slurp "/tmp/config.json"))
{
"ignored": {
"vars": [
"metabase.util.queue-test/bounded-transfer-queue-test",
"metabase.util.queue-test/take-batch-test"
]
}
}
nil
test-runner=> (find-tests (build-final-options {:ci-config-file "/tmp/config.json"
:only 'metabase.util.queue-test}))
Running tests with options {:namespace-pattern #"^(?:(?:metabase.*)|(?:hooks\..*))", :exclude-directories [".clj-kondo/src" "classes" "dev" "enterprise/backend/src" "local" "resources" "resources-ee" "src" "target" "test_config" "test_resources"], :test-warn-time 3000, :ignored {:vars #{"metabase.util.queue-test/bounded-transfer-queue-test" "metabase.util.queue-test/take-batch-test"}}, :ci-config-file "/tmp/config.json", :only metabase.util.queue-test}
Running tests in metabase.util.queue-test
Finding tests took 1000.0 µs.
Running 4 tests
(#'metabase.util.queue-test/non-delayed-take-batch-test
#'metabase.util.queue-test/result-listener-test
#'metabase.util.queue-test/listener-handler-test
#'metabase.util.queue-test/multithreaded-listener-test)
;; only found 4 tests
;; no ignored vars
test-runner=> (println (slurp "/tmp/empty-config.json"))
{
"ignored": {
"vars": []
}
}
nil
;; finds 6 tests
test-runner=> (find-tests (build-final-options {:ci-config-file "/tmp/empty-config.json"
:only 'metabase.util.queue-test}))
Running tests with options {:namespace-pattern #"^(?:(?:metabase.*)|(?:hooks\..*))", :exclude-directories [".clj-kondo/src" "classes" "dev" "enterprise/backend/src" "local" "resources" "resources-ee" "src" "target" "test_config" "test_resources"], :test-warn-time 3000, :ignored {}, :ci-config-file "/tmp/empty-config.json", :only metabase.util.queue-test}
Running tests in metabase.util.queue-test
Finding tests took 0.0 ns.
Running 6 tests
(#'metabase.util.queue-test/non-delayed-take-batch-test
#'metabase.util.queue-test/take-batch-test
#'metabase.util.queue-test/result-listener-test
#'metabase.util.queue-test/listener-handler-test
#'metabase.util.queue-test/bounded-transfer-queue-test
#'metabase.util.queue-test/multithreaded-listener-test)
```
* Fetch ci-test-config.json from master on each test run
This affects all BE tests, driver and app-db. Don't pass an explicit arg
and just let the default pick it up.
```bash
❯ clj -X:dev:test :only metabase.util.queue-test
Running tests with options {:mode :cli/local, :namespace-pattern #"^(?:(?:metabase.*)|(?:hooks\..*))", :exclude-directories [".clj-kondo/src" "classes" "dev" "enterprise/backend/src" "local" "resources" "resources-ee" "src" "target" "test_config" "test_resources"], :test-warn-time 3000, :ignored {}, :exclude-tags [:metabot-v3/e2e], :only metabase.util.queue-test}
Running tests in metabase.util.queue-test
Finding tests took 7.5 s.
Running 6 tests
Running before-run hooks...
LONG TEST in metabase.util.queue-test/take-batch-test
Test took 4.720 seconds seconds to run
LONG TEST in metabase.util.queue-test/bounded-transfer-queue-test
Test took 7.518 seconds seconds to run
6/6 100% [==================================================] ETA: 00:00
Ran 6 tests in 12.421 seconds
53 assertions, 0 failures, 0 errors.
{:test 6, :pass 53, :fail 0, :error 0, :type :summary, :duration 12421.435583, :single-threaded 6}
Ran 0 tests in parallel, 6 single-threaded.
Finding and running tests took 20.0 s.
All tests passed.
Running after-run hooks...
Running QP tests against these drivers: #{:h2}
metabase on quick-CI-test-disabling [$!?] via ☕ v21.0.2 on ☁️ metabase-query took 48s
❯ clj -X:dev:test :only metabase.util.queue-test :ci-config-file '"/tmp/config.json"'
Loading CI config file from /tmp/config.json # <------ new output
Running tests with options {:mode :cli/local, :namespace-pattern #"^(?:(?:metabase.*)|(?:hooks\..*))", :exclude-directories [".clj-kondo/src" "classes" "dev" "enterprise/backend/src" "local" "resources" "resources-ee" "src" "target" "test_config" "test_resources"], :test-warn-time 3000, :ignored {:vars #{"metabase.util.queue-test/bounded-transfer-queue-test" "metabase.util.queue-test/take-batch-test"}}, :exclude-tags [:metabot-v3/e2e], :only metabase.util.queue-test, :ci-config-file "/tmp/config.json"}
Running tests in metabase.util.queue-test
Finding tests took 6.9 s.
Running 4 tests
Running before-run hooks...
4/4 100% [==================================================] ETA: 00:00
Ran 4 tests in 0.133 seconds
42 assertions, 0 failures, 0 errors.
{:test 4, :pass 42, :fail 0, :error 0, :type :summary, :duration 133.16525, :single-threaded 4}
Ran 0 tests in parallel, 4 single-threaded.
Finding and running tests took 7.1 s.
All tests passed.
Running after-run hooks...
Running QP tests against these drivers: #{:h2}
```
Note that it runs 4 tests instead of 6. The config passed here was:
``javascript
{
"version": "1.0.0",
"ignored": {
"vars": [
"metabase.util.queue-test/bounded-transfer-queue-test",
"metabase.util.queue-test/take-batch-test"
]
},
"metadata": {
"last_updated": "2025-08-02",
"updated_by": "engineering-team",
"reason": "Testing CI config implementation"
}
}
```
* empty ci-test-config
this will be the file when this merges
* print config when it has vars that we are ignoring
* PR review feedback
typo in example json
set continue-on-error: true. if network fails, let's not fail our
build. it's just a little file. carry on soldier. worst case scenario a
test known to flake might flake. might not.
* Add repro for metabase#52872
* Wrap long titles without spaces in saved questions
* Add repro for metabase#52872 for dashboards
* Fix width of dashboard with long name
* Use max-width instead of width
* etc: kickoff comments
* get quotas statuses from token-check (#61560)
* get quotas statuses from token-check
* Check quota and estimated-row-count of dbs before replication
* Evict token-check cache to get updated quotas
* Turn the MB pg replication button into a modal (#61701)
Closes: https://linear.app/metabase/issue/CLO-4308/turn-the-mb-pg-replication-button-into-a-modal
* Fix schema-filters-patterns split
* split token-check row estimates counts (#61783)
* Split token-check row estimates counts
* add preview endpoint
* Align replication schema-filters with HM
Fixes: 805f4cafcf
Depends-on: https://github.com/metabase/harbormaster/pull/6445
* preview endpoint changed
* temp frontend modal
* After pairing with Raimon
* Fix lint and fmt errors
* pg replication modal (#61913)
* Success modal on starting replication
* Visual improvement to DatabaseReplicationForm
* made non-pk-tables list prettier
---------
Co-authored-by: Dennis Schridde <dennis@metabase.com>
* Run prettier and linter on DatabaseReplicationForm
* Turn DatabaseReplicationModal into a multi step thing and add backend API to support this
* Workaround broken dirty bit
* Call setFieldValue("schemaFields", "") when schemaSelect=all
* fix enabling of the replication button
* UI updates
* UI updates
* removed package-lock file
* Don't wait for replication to become "active" in "setting up" modal
The progression from "initializing" to "active" can take arbitrary
amounts of time. Instead we should show the "setting up" modal while
creating the replication, which is synchronous and has an upper bound
of 40s.
Fixes: 6f2ba1852e
References: https://metaboat.slack.com/archives/C079ZS06ERJ/p1754940482733049
* Loading for preview method
* Error handling
* Modal padding and title updates
* Remove info about future email from form
* Misc
* Misc
* DatabaseReplicationStatusInfoProps cleanup
* Misc
* Status modal content padding
* Modal properties
* fix: throw harbormaster call error body in ex-data
* Show which tables are excluded from replication because of owner mismatch
Closes: https://linear.app/metabase/issue/CLO-4346/preview-should-also-include-schemas-that-cant-be-synced-due-to-owner
* Align text of replication modal more with Figma
* UI updates 1
* Replication stats spacing updates
* Misc
* Tables listing icons
* Alignment updates
* Layout updates for tables listing
* Misc
* Colours updates
* List updates
* Misc
* Misc text alignment
* Misc
* Scrollable tables list
* Fix tables listing
* Comma separated DB WIP
* Misc
* Fix margins
* Ui updates
* More on setting up and success layout
* Test for spinner
* More on spinner
* Colour update
* Box shadow for spinner
* Spinner size updated
* Spinner padding updated
* Spinner added to setting up step
* Misc
* Sizes updated
* Fix for unit tests
* Debounce for schema filters
* Schema filters onChange
* Remove loading in button is no schema filters
* Loader for preview
* Misc
* feat: check replication table ownership
Same as how HM does it.
* Moved laoder next to the submit button
* Make DatabaseReplicationForm look more "loading..." during preview
* Remove setFieldValue hack
Was obsoleted by 9cdbeaec7a
* Also show skeletons when data is not fresh (i.e. when loading)
* Fix linter: Change set IDs are not in order
* Build uberjar container also when pushing a PR branch
Before it would only build when the `build-docker-uberjar` label was
added. So to build the latest branch head you'd have to remove and
re-add the label.
Fixes: 22507a963a
* Link to postgres docs
* SUrface "incorrect" errors from HM response to connection creation
* Misc
* Modal layout
* Error modal UI updates
* Misc
* More on layout + alignment fixed
* More on error layout
* Fix for modal closing
* rfct: use pg conn preview
* revert owner sync
* rfct: use existing schema filter format
* lint: fix fe
* feat: also show tables that will be replicated
* test: update be tests
* rfct: minor renames for consistency
* fix: cannot replicate 0 tables
* style: fix
* fix: copy
* test: update body prop
* test: add form tests
* fix: progress indicator on bottom
* fix: card styles
* fix: error message when no tables to replicate
* fix: easy stuff from jakub's review
* fix: error state for preview too
* claude: fast FE unit tests
* rfct: use fetchMock in FE test
* fix: remove small error step text
* fix: mocal close should reset setup step
* fix: translate noSyncReason
* fix: catch invalid schema pattern as 400
* rfct: use plain fn for onSubmit
* rfct: raise inline fn
* rfct: use correct token-check alias
* rfct: remove unused preview fields
* rfct: raise free-quota fn
* fix: throw on tunnel or certs
* rfct: remove silly when
* rfct: testable preview-replication and errors map
* test: granular fe error tests
* test: fix BE tests
* test: preview-replication unit tests
* fix: test redef name
* rfct: style and db as dbname
---------
Co-authored-by: Raimon Grau <raimon@metabase.com>
Co-authored-by: Dennis Schridde <63082+devurandom@users.noreply.github.com>
Co-authored-by: Dennis Schridde <dennis@metabase.com>
Co-authored-by: Raimon Grau <raimonster@gmail.com>
Co-authored-by: Arkadi Yaroslavtsev <arkadi@metabase.com>
Co-authored-by: losrebellos <ben@benoitvinay.com>
* Fix question download filenames to use question name instead of generic 'query_result'
Resolves: https://github.com/metabase/metabase/issues/61708
Previously, when downloading questions from the SDK (and public links), files were
named with a generic "query_result_{timestamp}.{format}" pattern even when the
question had a meaningful name.
This fix ensures that:
- Questions with valid names use the slugified question name as filename
- Questions with nil/empty/blank names fall back to "question" instead of "query_result"
- Both regular card downloads (/api/card/:id/query/:format) and public link downloads are fixed
Changes:
- Modified process-query-for-card-default-run-fn in query_processor/card.clj
- Modified process-query-for-card-with-id-run-fn in public_sharing/api.clj
- Added proper fallback logic using str/blank? to handle edge cases
Expected behavior:
- "My Awesome Question" → my_awesome_question_{timestamp}.csv
- nil/empty name → question_{timestamp}.csv
🤖 Generated with [Claude Code](https://claude.ai/code)
Co-Authored-By: Claude <noreply@anthropic.com>
* Cleanup and added tests
* Make whitespace linter happy
* Fix silly oversight
* Set warn-on-reflection
* Fix issues
---------
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: Timothy Dean <timothy@timothydean.dev>
* Hide CSV panel with no meaningful uploadable dbs
* Hide add data when no meaningful actions can be performed
* Simplify conditional for shouldShowUploadPanel test
Co-authored-by: Anderson <ands.alves.nunes@gmail.com>
* Show CSV tab also when user can upload data
User with upload permissions, but without admin rights, cannot see uploadable dbs in the api response
* Add e2e test for hidden add data for regular users without upload permissions
---------
Co-authored-by: Anderson <ands.alves.nunes@gmail.com>
* Show a special error screen when you don't have permissions to run the query
* Show a special error screen when you don't have permissions to run the query
* Fix tests
* Unpolished WIP hackathon code for additional driver support
* fix kondo
* add transforms/table featuer for supported drivers
* fix transforms for dbs with no schema
* rename transform datasets to prevent foreign key overlap
* Add clickhouse support
* fix clickhouse tests
* fix redshift tests
* fix bigquery failures
* kondo and filter tests by feature
* fix transform dataset for bigquery
* fix sqlserver transforms with joins
* fix clickhouse order by when no non nullable field is available
* add more complex test with join, filter, aggregation and ordering
* use different metadata provider
* filter for left join feature in test
* improve sqlserver compile-transform
* done use legacy mbql in test, refactor normalize-name
* clean up
* add sqlserver error test
* parse query and add into for sqlserver
* add type hints for linter
* Address comments
---------
Co-authored-by: Timothy Dean <timothy@timothydean.dev>
Previously, if you had city, state, latitude, and longitude breakouts, you'd get zoom in (city), zoom in (state), zoom in (lat/lon for lat), zoom in (default binning for lat), zoom in (lat/lon for lon), and zoom in (default binning for lon). Those would all be called "zoom in", so you'd just see half a dozen drills with the same title. This hides the default binning drills if we are showing lat/lon, and gives the assorted geographic zoom-ins different titles.
* `wrap-value-literals` => Lib
* Mostly working nicely
* Clean up other middleware + test fix
* Finish it up
* Code cleanup
* Kondo
* Fix test 🔧
* Work around https://github.com/clj-kondo/clj-kondo/issues/2610
* Test fixes; PR feedback
* Fix walk recursion into `:case` and `:if`
* code cleanup
* Oracle test fix 🔧
* Another test fix 🔧
Fixes#62377.
Correctly get a connection to the database while checking the ClickHouse version
(used to check if it supports `:connection-impersonation`). The driver was using the
internal, raw form of getting a connection, which didn't apply the SSH tunnel properly.
* Boilerplate
* Add list view toggle control to ViewFooter
* Remove processRemappedColumns, add layout
* Navigation
* Back button,breadcrumbs
* TODO
* Layout
* Layout
* Refactor
* Column detection logic
* Inline
* Fix
* Extract Breadcrumb
* Fix
* Entity name in nav
* Condition
* Fix
* Header
* Style
* Remove
* Do not render header if no column can be used there
* Body
* Add empty list viz
* Update VizStyleToggle to use list display type
* Sidebar
* Style
* Add simple list view
* Add virtualization for ListViz view. Optimize rendering
* Copy
* Hover style
* Add object detail modal opening on list item click
* Remove back button
* Overflows
* a11y
* Redundant css
* Fix
* Fix rendering
* Style links
* Cleanup
* FK pill
* Render images
* Prevent showing old images while new are loading
* Take currency_in_header into account
* Extract `ColumnHeader`
* Cleanup
* Fix spacing, colors
* Refactor relationships
* Update PK drill
* PK drill
* Fix
* Fix
* Improve layout of list items
* Center cells content
* Add sorting
* Fix virtualized behavior
* Fix copy if row name is not available
* Max height for values
* Update ViewStyleToggle rendering rule to fix incorrect list view icon
* Styling fixes
* Make whole list view scrollable
* Hide truncated data warning for list view
* Fix PK drilll
* Don't memo formatted rows
* Fix virtualization
* Use drills for sorting
* Fix virtualization flickering
* Fix drills
* Disable viz settings for list view
* List view visual enhancements (#62124)
* spaecing and font styling
* add entity_type icons to listitems
* display 4 columns instead of 5
* change icon for transaction tables
* add circle for boolean fields
* add badge style to category columns
* add colors to badges, incl Score cols
* Save list questions as tables
* Footer
* Padding and scrolling
* Styling
* Styling
* Remove icon
* Fix useObjectDetail call for ListView
* Fix list view columns alignment
* Move detail view breadcrumbs to AppBar
* Style breadcrumbs
* Fix scrolling
* Show icon if avatar is missing
* Fix react key warning
* Align breadcrumbs with detail view page
* Remove arrow navigation and footer
* Styling
* Error handling
* Styling
* Make Detail View breadcrumbs preserve card settings
* Make DetailView breadcrumbs return to previous ListView state
* Add company, currency, and receipt icons (#62196)
* add company currency and receipt icons
* add new icons to IconName and Icons obj
* Remove duplicate 'currency' icon declaration
---------
Co-authored-by: Stas Gavrylov <stasgavrylov@gmail.com>
* Update icons for TransactionTable and CompanyTable
* Make titleColumn always present. Update categoryValue styling
* ListView: Extract rendering logic to ColumnValue component
* Add tooltips for large column values
* Make list item grid always have at list 1 column
* Refactor DatasetEditorInner to typescript
* Rename DatasetEditor 'metadata' mode to 'columns'. Add new 'Metadata' tab for list view
* List view customization styling fixes (#62546)
* fix font sizes and spacing
* styling on ReorderableTagsInput
* make the viewsidebar 400px
* Make ReorderableTagsInput dropdown behave like multi-select
* Fix saving. Remove ViewStyleToggle
* Refactor List view to proper viz settings. Fix bugs. Remove TabHint popup
* Remove TabHintToast completely
* Fix container highlight when dragging between inputs
* Fix column settings in ListViz. Fix eslint
* Remove legacy DetailView code
* PR Cleanup
* Fix settings in list item preview
* Fix sorting. Replace custom column items with ColumnItem
* Fix saving models with list display type
* Fix incorrect condition to save list view models as tables
* Fix incorrect columns param in ListViewColumnsSidebar
* Fix list entity icon hover
* Tests – Models List View (#62836)
* Add storybook for ReorderableTagsInput
* wip
* Unit tests for ReorderableTagsInput
* Fix models-metadata related e2e tests
* e2e tests for List View
* Fix eslint
* Unit tests fixes
---------
Co-authored-by: Kamil Mielnik <kamil@kamilmielnik.com>
Co-authored-by: Anton Kulyk <kuliks.anton@gmail.com>
Co-authored-by: Maz Ameli <maz@metabase.com>
* Populate provider_name field on startup health check
* Serve provider patterns from BE side
* cleanup tests and expand clojure host provider detection
* Fix provider detection test
* Fix regexes for supabase and timescale
* adjust module api exports to standards
* mark private function ins provider-detection namespace
* extend provider values to null and empty, skip backfilling on empty provider_name
* Don't hardcode postgres in health check provider detection
* Make providers api dynamic and ready to support more engines
* Sort lines providers and engines
* Move hostname field tests to a test folder
* memoize detected provider
* Fix for tests for private functions
* Fix import of DatabaseProviderName type
* Use update function for setting provider_name on health check
* Fix typo in test case
* Remove duplicate entry
* simplify detect-provider-from-database call
* Move postgres extra-info to postgres driver module
* Simplify provider update check on startup, use null value for unidentified
* match provider also using engine
* bit of cleanup
- just move the regexes into the postgres driver namespace
- remove regexes and turn them into strings since most databases will put
them in yaml
- just test the public api
- simplify all the host and details checks. Database is a very standard
shape on the backend
* allowing setting null to provider match is null, cleanup unused imports
* move postgres tests to postgres driver; add a few tests for yaml
# Conflicts:
# test/metabase/driver/postgres_test.clj
---------
Co-authored-by: dan sutton <dan@dpsutton.com>
* Potential fix for code scanning alert no. 512: Insecure randomness
Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com>
* Handle max = 1 case
---------
Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com>
* Do not apply update-keys to state
* Add comment documenting the intentional order
* remove recursive snake case update
* apply fix to both streaming and non streaming
* adding kebab->snake helpers that ignore camelcase
* Fix kondo
---------
Co-authored-by: Thomas Schmidt <thomas@metabase.com>
This is based on this discussion with Marcos: https://metaboat.slack.com/archives/C01R1JY03J4/p1756173436062039
It appears that we do set a reporting time zone for SQL queries, but it did not have an effect in my own testing on my local MariaDB. However, it works just fine when running the "official" Docker version of MariaDB.
* allow specifying athena hostname
* use hostname
* fix params
* add test and clarify hostname field
* update helper text
* Apply suggestion from @jeff-bruemmer
Co-authored-by: Jeff Bruemmer <jeff.bruemmer@gmail.com>
* update hostname field
* apply suggestion
* improve test
---------
Co-authored-by: Jeff Bruemmer <jeff.bruemmer@gmail.com>
* Add dead letter queue for the semantic search indexer
The DLQ system handles failed semantic search indexing operations if they are causing the indexer to stall.
- Applies back-off policies for retries (hard coded for now).
- Prevents batch poisoning from blocking indexer progress.
* Add the jitter factor I mistakenly deleted
* Doc fixes
* Slightly better stall recovery log
* Kondo
* Index attempt_at on DLQ table
* Add some more logging
* Init includes DLQ creation
* Fix some rebase collisions
* Formatting
* Add some asserts to show dlq creation during init
* Name test fixed with hashing
* Inline table-name-prefix
* Bump index meta version
* Prefer hex to b64 for table/index names they remain usable unescaped.
* Fix next-dlq-run test in on-indexing-idle
The test was backwards, so the loop only idled when `next-dlq-run` was due, rather than avoiding idle in that case.
* Ensure exit-early-due-to-cold-data? even if we have no last-seen-change
Ensure we still yield back to quartz if more than exit-early-cold-duration seconds have elapsed since the loop start
and we have no last-seen-change. This can happen in cases where no docs were updated since the loop start if our
watermark is already at the end of the gate table.
* Reduce log level to :trace for "nothing to do" case in indexing-step
* Whitespace cleanup
* Reduce ee-search-indexer-exit-early-cold-duration from 30s to 5s
30s is longer than the quartz interval for the indexer job, so we'll just get rescheduled immediately after yielding
control.
* Increase the run-frequency for the SemanticSearchIndexer job from every 10s to every 20s
* Move table-exists? into shared util ns and reuse
* Simplify honeysql qualified col reference in add-entries!
* Fix comment typo: s/indexes/indexer/
* Add missing seq in try-batch!
* Change ee-search-indexer-exit-early-cold-duration back to 30s
This value needs to be higher than the dlq-frequency, otherwise the dlq never runs.
---------
Co-authored-by: Mike Appleby <86076+appleby@users.noreply.github.com>
in dev, the default value was the staging store. When not in dev, the
default value of this url was "false"
```
❯ http get http://localhost:3056/api/session/properties Cookie:"metabase.SESSION=d10a4524-837e-4aaf-84a3-b2f99bc9ecb2" | jq '."store-url"'
"false"
```
In javascript, most of the store url stuff is through
```javascript
export function useStoreUrl(storePath: StorePaths = "") {
return useSelector((state) => getStoreUrl(state, storePath));
}
```
which is based on `getStoreUrl` which is
```
const DEFAULT_STORE_URL = "https://store.metabase.com/";
...
export function getStoreUrl(state: State, path: StorePaths = "") {
try {
const storeUrl = getSetting(state, "store-url");
const url = new URL(path, storeUrl);
return url.toString();
} catch {
return DEFAULT_STORE_URL;
}
}
```
So what happens normally is:
1. in dev, we usually have no setting set for this, so the `when-some`
hits nil. If you do have something there, it is honored.
2. In prod, we read false for the config/is-dev and we use that as the
setting value
```clojure
settings=> (with-redefs [config/is-dev? false]
(store-url-default))
"false"
```
3. There are lots of usages of this on the frontend. But they are all
protected by the try/catch and returning the default store. This is LOAD
BEARING in production.
```javascript
new URL("false", null)
Uncaught TypeError: Failed to construct 'URL': Invalid base URL
```
4. Cloud migration does something slightly different and so fails to get
this load bearing try/catch. It takes a `checkoutUrl` in its props:
```javascript
export const MigrationSuccess = ({
migration,
restartMigration,
isRestarting,
checkoutUrl,
}: MigrationSuccessProps) => {
```
This manually constructs a url:
```javascript
const checkoutUrl = useMemo(() => {
return migration
? `${storeUrl}/checkout?migration-id=${migration.external_id}`
: `${storeUrl}/checkout`;
}, [migration, storeUrl]);
```
And now we don't have the benefit of the `URL(base, extension)` which
would have (dumbly) failed closed. So now we end up with
`false/checkout?...` and then get a "We're a bit lost" locally
So to recap:
we do the right thing in dev, and then production is broken. Most
pathways were just catching this error, one way constructed a url
fragment and therefore was susceptible.
```clojure
settings=> (with-redefs [config/is-dev? false]
(store-url-default))
"false"
settings=> (store-url-default)
"https://store.staging.metabase.com"
```
* Do not pass Sortable attributes when element is not sortable
* Render parameter value trigger as button
* Add repro for metabase#52674
* Use UnstyledButton
* Do not render parameter widget as button when it does not have a popover
* Update Loki Snapshots
* Use UnstyledButton and Box
* Fix syntax error
---------
Co-authored-by: Metabase Automation <github-automation@metabase.com>
This was working for `:base_type`s but not `:effective_type`s.
The `:result_metadata` for a native Mongo query always had
`:effective_type :type/*` even when we correctly inferred the
`:base_type`.
Fixes#62397.
* initial commit
* enable multi db by default for tests
* multi db as default, set inclusion filters for tests
* update introspection tests
* enable routing
* set enable multi db true as default
* use redef for spec
* clean up
* add more generic test
* update tests
* remove unused code
* clean up
* handle encrypted migration
* fix test and migration id
* move created and updated at fields
* more tests
* mark tests parallel
* fix whitespace
* fix athena test
* Update Debian/systemd docs
* Remove references to Debian
* Use H2 database in .env file by default
* Add journalctl command to follow logs
---------
Co-authored-by: Jeff Bruemmer <jeff.bruemmer@gmail.com>
* Refactor appdb-based scorers in the semantic search backend
Refactor the appdb-based scorers in the semantic search backend in preparation for adding the bookmarked scorer in
BOT-312. This refactor has several benefits: (1) makes the appdb-based scorers work more like the existing index-based
scorers; (2) allows sharing the exact same honeysql expressions for the user-recency-expr and model-rank-expr between
the appdb and semantic backends, reducing code dup; (3) makes it easier to add the bookmarked scorer since it will be
able to reuse the appdb backend's honeysql expr for bookmarked scores as well.
* Move user-recency-expr into search.scoring and use it in appdb and semantic backends
* Move model-rank-expr into search.scoring and use it in appdb and semantic backends
* Move mysql check from with-appdb-scores into appdb-scorers
Again in preprartion for adding the appdb-based bookmarked scorer later.
* Fix whitespace
I noticed that
https://github.com/metabase/metabase/pull/62513/files?file-filters%5B%5D=.clj&show-viewed-files=true
that the third tier drivers were running. Checking the action output
https://github.com/metabase/metabase/actions/runs/17220604727/job/48854867380?pr=62513
i see :
```
Is master or release: false
Has '.Team/Drivers' or '.Team/Querying' or '.Team/SemanticLayer' label: false
Checking for changes in relevant folders...
Changes detected in the following folders: src/metabase/lib src/metabase/query_processor.
Tier 3 driver tests will run.
```
So it thinks that something changed in src/metabase/lib/* and
src/metabase/query_processor/*
This branch
ds/clo-4370-button-in-metabase-admin-section-to-purchase-metabot-ai-add
targets master. And the diff is checking the diff of files between the
state of master and the PR. Which means if any changes have landed in
master affecting those areas this diff will report positive.
At the time of this writing this is:
```
❯ git diff --name-only c6a9193bff462d83c5c2 | wc -l
1259
```
Where c6a9 is master and 462 is the branch.
A better option is to find where the branch and the merge target share a
common ancestor and then check for changes from there.
```
❯ git diff --name-only $(git merge-base c6a9193bff462d83c5c2) 462d83c5c2 | wc -l
32
```
So we use `git merge-base <master> <branch>` and then diff that sha
against the branch.
```
git diff --name-only (git merge-base <base> <pr-branch>) <pr-branch>
```
And we get:
```
❯ git merge-base c6a9193bff462d83c5c237e6eee311
❯ git show 37e6eee311
commit 37e6eee311
Author: Phoomparin Mano <poom@metabase.com>
Date: Wed Aug 20 01:40:19 2025 +0700
Add the metabase-browser custom element to Embed JS (#60828)
Co-authored-by: cubic-dev-ai[bot]
<191113872+cubic-dev-ai[bot]@users.noreply.github.com>
```
Which is the common root. Then the diff from there:
```
❯ git diff --name-only $(git merge-base c6a9193bff462d83c5c2) 462d83c5c2
.clj-kondo/config/modules/config.edn
enterprise/backend/src/metabase_enterprise/api/routes.clj
enterprise/backend/src/metabase_enterprise/cloud_add_ons/api.clj
enterprise/backend/src/metabase_enterprise/harbormaster/client.clj
enterprise/backend/test/metabase_enterprise/api/session_test.clj
enterprise/backend/test/metabase_enterprise/cloud_add_ons/api_test.clj
enterprise/frontend/src/metabase-enterprise/metabot/components/MetabotAdmin/MetabotPurchasePage.tsx
enterprise/frontend/src/metabase-enterprise/metabot/components/MetabotAdmin/MetabotPurchasePage.unit.spec.tsx
enterprise/frontend/src/metabase-enterprise/metabot/components/MetabotAdmin/MetabotPurchaseSettingUpModal.tsx
enterprise/frontend/src/metabase-enterprise/metabot/index.tsx
frontend/src/metabase-types/api/metabot.ts
frontend/src/metabase-types/api/mocks/settings.ts
frontend/src/metabase-types/api/settings.ts
frontend/src/metabase/admin/app/reducers.ts
frontend/src/metabase/admin/routes.jsx
frontend/src/metabase/api/metabot.ts
frontend/src/metabase/api/utils/use-token-refresh.ts
frontend/src/metabase/common/components/AdminLayout/AdminSettingsLayout.tsx
frontend/src/metabase/plugins/index.ts
resources/frontend_client/app/assets/img/metabot-cloud-96x96.svg
src/metabase/cloud_migration/core.clj
src/metabase/cloud_migration/models/cloud_migration.clj
src/metabase/cloud_migration/settings.clj
src/metabase/core/init.clj
src/metabase/premium_features/core.clj
src/metabase/premium_features/settings.clj
src/metabase/premium_features/token_check.clj
src/metabase/server/middleware/security.clj
src/metabase/store_api/core.clj
src/metabase/store_api/init.clj
src/metabase/store_api/settings.clj
test/metabase/analytics/stats_test.clj
```
And now we have a correct list of files changed in that PR, not just
files different between current master and current PR.
* Convert revision usage
* convert pivot test + metabot usages
* convert more tests
* migrate final clj-time usage in logger.core
* switch quartzite to a git sha to get most recent commit w/o clj-time
* Revert "switch quartzite to a git sha to get most recent commit w/o clj-time"
This reverts commit 393bc56aad.
* Make readOnly notebook completely non-interactive
* Add readOnly prop to PopoverBackButton
* Add readOnly to FilterPickerHeader
* Add readOnly prop to FilterPickers
* Allow making filter body readOnly
* Pass readOnly to FilterPicker from FilterStep
* Add disabled prop to ClausePopover
* Disable step popovers when readOnly (except for filters)
* Do not disable pointer-events for notebook preview
* Disable popover when button is disabled
* Render readOnly expression popover in readOnly mode
* Make limit step readOnly when readOnly is set
* Add test for readonly notebook
* Disable tooltip on join picker when read-only
* Disable join picker popover when read-only
* Hide back button on date picker
* Simplify backbutton props
* Do not allow going back in readOnly ExpressionWidget
* Allow readOnly AggregationPicker
* Allow opening aggregation popover when it is an expression
* Render aggregation popover when it is a custom expression
* Fix types
* Remove renderPopoverWhenReadOnly flag
* Fix cursor logic on SortStep
* Do not render pointer cursor when step does not have popover
* Render custom expression popover for non-custom aggregations in read-only mode
== Goal ==
In case the Metabase instance does not have the Metabot AI add-on, we
want to show a page in the admin section to purchase it.
== How to test ==
1. Pick a valid token from staging with the `offer-metabase-ai` feature, but without the `metabot-v3` feature (`metabase-ai` add-on)
2. Pick the token's instance's Metabase API key
3. Pick an email address that matches a store user of the token's organization
4. Spin up a Metabase instance with that API key, token and user
5. Navigate to the admin section, AI tab
6. See the new page to purchase the add-on and click the button
Closes: https://linear.app/metabase/issue/CLO-4370/button-in-metabase-admin-section-to-purchase-metabot-ai-add-on
* UXW-503 Triggers error state when underlying card has been deleted
* UXW-503 Makes CardEmbed error state selectable, bordered
* UXW-503 Updates error state per design feedback (less of a jump between loading and error state, fewer different error visuals)
* UXW-503 Adds e2e test
* UXW-503 Removes useMemo per PR feedback
* UXW-503 Simplifies an e2e test, makes it less prone to flakiness
* Add gate table tombstone cleanup to index cleanup job
* add partial index on gated_at for just the tombstone rows
* fix up index creation syntax
* WIP - adding check that indexer ran within last 24 hours
* fix tests
* fix whitespace, adjust log/docstring
* kondo
* fix build, maybe broken from bad rebase?
* fix: handle more header names in translation csv
Handle the locale being specified as 'locale', 'locale code', 'language'
also make the check for header a little looser so we can report errors
in the header row more accurately.
* fixup! have sample CSV match copy
* Call createUser action when inviting new user on the /setup page
* Add e2e test case
* Add invite_sent event / Check user went to next step in e2e test
* Let the backend handle the invite_sent snowplow event
* Change approach to set source value
* Fix unit tests
* Add unwrap() to createUser dispatch
* Remove unused invite param from the /setup endpoint
* Remove unneeded frontend code
* Fix kondo issues
* track search_engine
* track request id
* record request id on search_query event
* fix bug with search_click position not factoring in pagination
* report offset on search_query
* report on the entity model
* move changes into new schema file and update referenced schema
* add back view_more event
* track hashed + salted search term on search_click + search_query events
* refactor trackSearchClick to accept object param instead of long list of args
* report search term for internal purposes only
* pr feedback
* add test coverage for new search reporting attributes
* maybe fix e2e in ci
* fix failing e2e
* Localize rows of initial tags and initial jobs in db on setup or startup
* Also translate job descriptions.
* Use fewer hits to the db on startup
* Use site locale
* kick ci
* Remove extraneous arg
* Remove extraneous strs
* Different translation model (on select)
* setup no longer needed from transforms
* Fix
* Test transform tag translation
* Test transform job initial translations
* Fix runtime crash on field visibility change with preview open
* Update Loki Snapshots
---------
Co-authored-by: Metabase Automation <github-automation@metabase.com>
* Query builder breadcrumbs can link back to archived models
* Update Loki Snapshots
---------
Co-authored-by: Metabase Automation <github-automation@metabase.com>
Closes#62099
When a Field has a coercion strategy (say, `UnixSeconds->DateTime`) set in the `Table Metadata`,
and then that field is **joined**, it will get double-coerced. The first time in the inner query for the join,
the second at the top level `SELECT`.
Since after the first coercion it will be a `TIMESTAMP` instead of and `INTEGER`, the second coercion is a SQL error.
The solution is to add `:qp/ignore-coercion true` in the `resolve-joins` middleware, when the `:fields` of the outer
query are being populated with joined fields. That prevents the QP from re-applying type coercions and date truncations.
* Add user-recency scorer to the semantic search backend
* Only query the recent_views table for models it knows about
* Replace group-by with m/index-by in user-recency scorer
* Cleanup comments
* Improve prometheus metrics for user-recency and add tests
Permission filtering is already recorded in :metabase-search/permission-filtering-ms so no need for a new metric that
records it + collection filtering. Instead, add a new metric :metabase-search/semantic-collection-filter-ms so there
is no overlap.
Also remove embedding label from the collection filter and appdb scorer metrics, as these don't currently depend on
the embedding model in any meaningful way.
* Rename :metabase-search/permission-filtering-ms to :metabase-search/semantic-permission-filter-ms
To match the naming convention of other metrics exposed by the semantic search backend.
Fixes a regression where making sure the autocommit was correctly set to
enable reading streaming result sets disallowed ddl and other
non-readonly changes in postgresql due to the interaction of the
'readOnly', 'readOnlyMode', and 'autoCommit' jdbc driver settings.
* UXW-502 Makes sql error scrollable. Also makes it vertically centered when not scrollable.
* UXW-506 Hides some UI from Download as PDF
* UXW-506 Prevents metabot blocks from getting split across pages in generated PDF
* UXW-506 Adds print-only Metabot icon to Metabot blocks in documents
solves https://github.com/metabase/metabase/security/code-scanning/493
but this is actually not an issue at all. When we build the driver jars
we remove deps provided by metabase itself. Metabase itself relies on
```
org.apache.commons/commons-lang3 {:mvn/version "3.18.0"}
```
```clojure
(defn- remove-provided-libs [basis driver edition]
(let [provided-lib->provider (into {}
(filter (fn [[lib]]
(get-in basis [:libs lib])))
(provided-libs driver edition))]
;; log which libs we're including and excluding.
(doseq [lib (sort (keys (:libs basis)))]
(u/announce (if-let [provider (get provided-lib->provider lib)]
(format "SKIP %%s (provided by %s)" provider)
"INCLUDE %s")
(colorize/yellow lib)))
;; now remove the provide libs from `:classpath`, `:classpath-roots`, and `:libs`
(let [provided-libs-set (into #{} (keys provided-lib->provider))
provided-paths-set (into #{} (mapcat #(get-in basis [:libs % :paths])) provided-libs-set)]
(-> basis
(update :classpath-roots #(vec (remove provided-paths-set %)))
(update :libs #(into {} (remove (fn [[lib]] (provided-libs-set lib))) %))
(update :classpath #(into {} (remove (fn [[path]] (provided-paths-set path))) %))))))
```
That output looks like:
```
Write :clickhouse :ee uberjar -> /home/runner/work/metabase/metabase/resources/modules/clickhouse.metabase-driver.jar
SKIP org.clojure/clojure (provided by metabase-core)
SKIP org.clojure/core.specs.alpha (provided by metabase-core)
SKIP org.clojure/spec.alpha (provided by metabase-core)
INCLUDE com.clickhouse/clickhouse-client
INCLUDE com.clickhouse/clickhouse-data
INCLUDE com.clickhouse/clickhouse-http-client
INCLUDE com.clickhouse/clickhouse-jdbc
INCLUDE com.clickhouse/client-v2
INCLUDE com.clickhouse/jdbc-v2
SKIP com.fasterxml.jackson.core/jackson-core (provided by metabase-core)
SKIP com.google.errorprone/error_prone_annotations (provided by metabase-core)
SKIP com.google.guava/failureaccess (provided by metabase-core)
SKIP com.google.guava/guava (provided by metabase-core)
SKIP com.google.guava/listenablefuture (provided by metabase-core)
SKIP com.google.j2objc/j2objc-annotations (provided by metabase-core)
SKIP commons-codec/commons-codec (provided by metabase-core)
SKIP commons-io/commons-io (provided by metabase-core)
SKIP org.apache.commons/commons-compress (provided by metabase-core)
SKIP org.apache.commons/commons-lang3 (provided by metabase-core) <------------------
SKIP org.apache.httpcomponents.client5/httpclient5 (provided by metabase-core)
SKIP org.apache.httpcomponents.core5/httpcore5 (provided by metabase-core)
SKIP org.apache.httpcomponents.core5/httpcore5-h2 (provided by metabase-core)
SKIP org.clojure/clojure (provided by metabase-core)
SKIP org.clojure/core.specs.alpha (provided by metabase-core)
SKIP org.clojure/spec.alpha (provided by metabase-core)
SKIP org.jspecify/jspecify (provided by metabase-core)
INCLUDE org.lz4/lz4-java
SKIP org.ow2.asm/asm (provided by metabase-core)
INCLUDE org.roaringbitmap/RoaringBitmap
INCLUDE org.roaringbitmap/shims
SKIP org.slf4j/slf4j-api (provided by metabase-core)
```
But this is included here in the hopes that it appeases snyk
* Correctly invalidate transform tags
* Remove Undo button from toast when undoing a tag
* Avoid wrapping in certain columns for transform lists
* Add test for tag removal invalidation
* Fix unit tests
* Use helpers in test
* Prove #27521 is fixed
* Fix the test from #60583
* Code cleanup
* I have fixed#39033
* Enable e2e test
* Add ref for #62383
* Ok so I guess I have fixed#47734 now too
* Test update 🔧
* Update the test again
* Test fix 🔧
* Test fix 🔧
* QP limit middleware => Lib
* Preprocess returns MBQL 5
* Split off changes to limit middleware
* Fix Kondo warnings
* Revert changes from Limit PR
* Code cleanup
* Preprocess returns MBQL 5
* Kondo cleanup
* Rework some tests in `wrap-value-literals-test` to use mock metadata providers
* Update more tests to use mock metadata providers
* Use even less with-temp
* Remove with-temp in QP middleware viz settings tests
* Rework metrics-test to never use with-temp
* Remove more with-temps
* Update case-test to use mock metadata providers
* Even more with-temps
* Update even more tests
* Fix two more
* Nice
* Revert stricter schema for query-hash
* Finish rewriting tests to use Mock MPs instead of with-temp or add ignores
* cljfmt
* Test fix 🔧
* Fix permissions
* Split off malli improvements into #62523
* PR cleanup
* PR feedback
* PR feedback
`{:status 500}` is rendered as a response with status 200 and a body of
`{"status": 500}`, wheras `{:status 500 :body ""}` is rendered as a
response with status 500 with empty body.
Fixes: b7253af4f2
* chore: don't run migrations on reindex!, just ensure index is initialised
* feat: support :force-reset?, to support `/re-init` endpoint
* fix test
* chore: check for index-created? in test
* undo
* feat: include index-created-at
* feat: support lazy gating + stale index behavior
* fix tests
* fix
* fix test
* return new index from record-new-index-table!
* chore: don't pipe opts to create-index-table
* feat: randomize index name
* force new index
* shorter name
* suffix as part of qualify
* re-init test
* rename find-best-index to find-compatible-index
* introduce create-new-index-spec, split out of find-compatible-index
* pick last index
* docstring
* add test
* split into determine-target-index/switch-to-index
* remove index-created? opt
* ensure table always exists
* back out including id/index_created_at in index
* move model-table-suffix out of qualify-index
* fix kondo warnings
* Add PostgreSQL provider icons to db form
fix types and opacity
# Conflicts:
# frontend/src/metabase/databases/components/DatabaseDetailField/DatabaseDetailField.tsx
* Store provider name field in form context and send it to the db create endpoint
* store provider name in the app db
* Use the non-deprecated form field for DB provider field
* Allow setting null value to provider name field
* Fix db tests
fix test
fix test
* Add db submit integration test
fix type error
* Rename mock file name
* Simplify postgesql json form mock
* Set provider_name in the same db update as rest of the fields
* Use select-keys-when for DB put update handler
* wip
* fix unit test and fix utm tags on link
* update docs link
* Update frontend/src/metabase/home/components/EmbedHomepage/tests/EmbedHomepage.unit.spec.tsx
Co-authored-by: Phoomparin Mano <poom@metabase.com>
* Update frontend/src/metabase/home/components/EmbedHomepage/EmbedJsContent.tsx
Co-authored-by: Phoomparin Mano <poom@metabase.com>
* address feedback
* swap static and sdk as per feedback in the pr
---------
Co-authored-by: Phoomparin Mano <poom@metabase.com>
* Fix duplicated table column viz settings
* Add BE logic
* Fix
* Review fixes
* Review fixes
* fixup! use linear time deduplication
---------
Co-authored-by: Edward Paget <ed.paget@gmail.com>
* Updates page title to be document's name
* Memoizes SetTitle so it doesn't perform a bunch of calculations on every render when the title hasn't changed
* fix: adjust schemas for inactive tables too
When toggling the multi-catalog setting for a database, we rename
existing schemas to match. For example, if you turn multi-catalog on and
have the existing schema `foo`, we'll rename it to `mycatalog.foo`.
Similarly, if you have the existing schema `mycatalog.foo` and turn
multi-catalog *off*, we'll rename it to `foo`.
We were ignoring inactive schemas here, which meant that if you turned
on multi-catalog and some of your tables became inactive (quite possibly
because you didn't update the schema filter to match, a major footgun
for the user), we would:
- update the tables going one way (`foo` to `mycatalog.foo`) and
deactivate for not matching the schema filter, then
- never update the tables again, because now they're inactive
Instead we should update the schemas for inactive tables as well.
Related to [UXW-375: Databricks multiple catalogs - Table
"database.catalog.schema.table" is
inactive](https://linear.app/metabase/issue/UXW-375/databricks-multiple-catalogs-table-databasecatalogschematable-is) / https://github.com/metabase/metabase/issues/61492
I don't think this is a complete fix though - we may need to rethink the
feature, and possibly just not allow you to change the setting for
existing databases.
* Add a test
* Allow rendering databases as disabled in DataSelector
* Allow rendering databases as disabled in DataSourceSelectors
* Allow rendering databases as disabled in NativeQueryEditor
* Add features for transforms/* to type
* Disable unsupported source tables in the native queyr transform builder
* Allow disabling databases in DataPicker
* Allow passing shouldDisableItem from Notebook to DataStep
* Disable unsupported databases in datapicker for transforms
* Remove type error in filter
* Pass shouldDisableItem to more pickers
* Add missing database to item
* Filter out unsupported questions too
* Remove database property again from DataPickerItem
* Fix item type
* Rename db_id to database_id
* Pass database_id in collection picker
* Disable unsupported tables, metrics and dashboards
* Hide recent items that have been disabled
* Padd shouldDisableItem to more sub-pickers
* Add database metadata to table
* Add shouldDisableItem to TablePickers
* Rename helper
* Simplify types
* Add hook to avoid using metadata
* Disable questions based on unsupported databases in question picker
* Allow disambiguating RecentItem from TableItem
* Disallow transforms on audit databases
* Add tests for unsupported databases
* Add unit tests for disabling items
* Disable dashboards in Question picker
* Use plain ORDERS_ID
* Move database support helper to utils and pull data fetching out of hook
* Pull out helper into separate definiton
* Add test about creating transforms from metrics
* Remove transforms/view and transforms/materialized-view from api types
* Use Loader from metabase/ui
* Fix types
* fix: handle pivot-grouping as a BigDecimal
Some database drivers (Oracle) seem to convert the pivot-grouping
integer into a BigDecimal; this attempts to safely convert it back to an
integer we can perform bitwise operations on, using .longValueExtract
which will throw an exception if the number is not losslessly
convertible to a long.
* fixup! throw if conversion fails
In `enforced-sandboxes-for-tables` we were only checking the
permission in the `data_permissions` table, without special-casing
administrators like we do elsewhere.
So when checking to see if an administrator in `All Users` (sandboxed)
and `Administrators` had an enforced sandbox for a table, we were
checking to see whether there were rows in `data_permissions` providing
`Administrators` with `:unrestricted` data access to the table - which
will never be the case, because Administrators is special and never gets
any rows there.
Instead, special-case the super user case like we do elsewhere. Don't
bother getting the groups they're a part of - we know the set of tables
they're sandboxed on is empty.
* fix: handle whitespaces in the connection string, decode username and password
* Trim the connection URI at first step
* Add safe uri decode, edge case test and remove redundant trim
* wait up to 10 seconds for worker to stop
in CI getting lots of flakes. Here's reproduction locally:
```
queue-test=> (dotimes [_ 20] (listener-handler-test))
FAIL in (listener-handler-test) (queue_test.clj:178) ;; 1
Standard behavior with a handler
expected: (not (thread-name-running? thread-name))
actual: (not (not true))
FAIL in (listener-handler-test) (queue_test.clj:178) ;; 2
Standard behavior with a handler
expected: (not (thread-name-running? thread-name))
actual: (not (not true))
,,,
FAIL in (listener-handler-test) (queue_test.clj:178) ;; 18
Standard behavior with a handler
expected: (not (thread-name-running? thread-name))
actual: (not (not true))
nil
```
and after:
```
queue-test=> (dotimes [_ 20] (listener-handler-test))
nil
```
* eastwood doesn't like new qualified method stuff
* Attempt to fix flaky tests
emails can end up in here from other tests as well. So this takes two
tracks:
first, i want to just check all emails in the box for this
user. Sometimes there are multiple emails in here and the one we want
isn't first. and second, better error messages. This will show us all
email message subjects. Hopefully that lets us identify what else might
be going wrong.
Next step if this is still failing it to attach which test sent the
message using clojure.test/*testing-vars*.
Adds a simple function and a macro to `metabase.util.log` to allow
propagation of ThreadContext across thread boundaries.
`(with-context-meta some-map)` adds the `mb-` prefixed items in the
`ThreadContext` to the metadata of that map.
`(with-restored-context-from-meta some-map)` takes a map presumably
resulting from calling `with-context-meta`, and sets the current
`ThreadContext` accordingly.
I changed the dispatch-fn and the worker consumption functions in
`metabase.notification.send` to use these to propagate the ThreadContext
to the worker thread.
Sending a notification like
```
(log/with-context {:aoooooook "hey"} (dispatch! {:flippity "ofosidjffoo"}))
```
now results in:
```
2025-08-18 14:42:45,716 ERROR notification.send :: Error in notification worker {mb-aoooooook=hey}
clojure.lang.ExceptionInfo: Invalid input: {:payload_type ["invalid dispatch value, got: {:flippity \"ofosidjffoo\"}"]}
```
* Improve log context for exceptions
This adds a feature to our log context that I've used and loved in the
past.
For illustration, let's say I do this:
```
(defn my-fn []
(log/with-context {:user 123 :database 5}
(/ 1 0)))
(try (my-fn)
(catch Exception e (log/error e)))
```
With the current code, we'll run `my-fn`, throw an exception,
then *exit* the `log/with-context` block. When we eventually log the
exception, the context will be empty.
This is annoying, because when the exception was thrown, we had context
that a developer thought would be useful. But just when it would
be *most* useful to have that information - an uncaught exception we
didn't think about bubbles up to the HTTP request logging middleware or
whatever - it's thrown away.
This change essentially wraps `with-context` calls with a `try/catch`
that catches any exceptions and re-throws them, annotated with the
original context.
This way, when we actually log the exception, we can swap back the
original context. In the above example, `{:user 123 :database 5}` would
be logged with the exception.
* Move log context ex-data to special key
When the exception middleware catches an exception it does some fancy
logic about whether to just let the body be the exception message (like
"Not found") or a blob of data with the message, stacktrace, data, etc.
Including the log context in the ex-data itself caused this to get
confused (because some additional data was present in the `ex-data`) and
spit out the blob where previously we'd just presented a user-friendly
error message.
To avoid doing this accidentally in the future, I just stuck all the log
context data under a special key, `:metabase.util.log/context`, in the
ex-data, which we can explicitly ignore when deciding whether to include
the whole exception in the error response.
* Fix failing test
* Fix conversion of joins with a native source query to legacy
* Don't do nasty stuff to parameters when converting from legacy
* Move parameters to correct location when converting joins
* Fix conversion of joins with filters
* Splice join parameters together
* QP params => Lib
* Fix Kondo warnings
* Simplify PR [ci skip]
* Beautiful QP middleware
* Beautiful QP middleware.
* Add notez
* Test fixes 🔧
* JVM metadata provider fix 🔧
* Kondo & cljfmt
* Test fixes 🔧
* kondo
* Delete the Lib-style rewrites of the parameters code, I decided I didn't like them
* Fix the schema
* Revert unneeded changes
* Fix busted tests 🔧
* Fix two more busted tests.
* Fix even more broken tests
* Fix broken tests 🔧
* Improved normalization for :value clauses
* Test fix 🔧
* Test fix 🔧
* Fix resolution of field name refs in first stage
* Test fixes 🔧
* Another round of test fixes 🔧
* Fix :expression ref parameters
* Maybe fix flakes
* Test fixes 🔧
* cljfmt/kondo
* Reduce coupling between semantic.index and semantic.scoring
Previously ->col-expr was defined in the scoring namespace and had to "know" about hybrid search query table aliases, which is gross.
Instead, define a hybrid-coalesce func in the semantic.index namespace and pass it into the semantic-scorers function. Also use it in hybrid-search-query, to ensure hybrid column references are constructed in the same way in both places.
* Add semantic-scorers->col-expr-is-respected-test
* Remove ->col-expr and just re-project cols from hybrid query with well-known names
* Add an API endpoint to delete collections
This permanently deletes a collection and anything contained inside it.
One "exception" that's not really an exception: say you have A > B > C > D,
you trash C, then trash A.
Now you have:
```
Trash > A > B
Trash > C > D
```
(because C was trashed directly).
But internally, we still represent D's location as `/A/B/C/`.
Now you permanently delete A. We need to make sure D's and C's locations
get fixed. Therefore, before deleting A, we go through all
the *directly*-archived children of A, and move *them* to A's parent.
Moving a collection also moves its children, so we'll move the whole
`C > D` structure out of `B` into the root collction.
Then we'll delete `A > B`, leaving `C > D` behind.
* Don't get the parent for no reason
the children-location of my parent is... my location. :doh:
* Ensure we cleanup the index metadata tables as well as the index table in with-index!
Otherwise, we wind up with many "duplicate" docs in the gate table which results in duplicates in the index. These are
not true duplicates; they have fresh model ids from the with-temp'ed mock docs we create, but they have identical
content, so they have identical embeddings and tsvectors. Those documents then wind up with identical text and
semantic scores when querying the index, and since we ORDER BY the scores (with no tie breaker secondary ordering),
this creates non-determinism in the semantic and keyword rankings, making test assertions wonky.
* Tighten up test assertions in native-query-test
These were relaxed to top-three? in #62101, but since the query results are no longer non-deterministic, it should be
safe to switch back to requiring the match to be the first result.
* feat(mage): add port option to mage nrepl
Adds an option to the mage nrepl command to customize the port mage's
nrepl starts on. This is mostly so I don't need to modify my clojure-mcp
configuration to point to a different port when working on mage stuff.
* feat(mage): add port option to mage nrepl
Adds an option to the mage nrepl command to customize the port mage's
nrepl starts on. This is mostly so I don't need to modify my clojure-mcp
configuration to point to a different port when working on mage stuff.
* feat(mage): print better argument usage help
* make semantic search resilient to errors on fallback engine
* sort active-engines by default-engine first
* more logging around appdb init
* add test
* update the dashboard form to use mantine components
* committing forgotten files
* e2e adjustments and fixes
* test adjustments
* remove shameful hardcoded font sizes
* fix font size on switches
* adjusting unit test
* Fix OSS/EE artifact conflict in uberjar workflow
Problem: When fetching both OSS and EE artifacts in check-existing-artifacts job,
unzip was prompting for confirmation to replace COMMIT-ID file, causing the
workflow to fail with 'EOF or read error, treating as [N]one' error.
Solution: Use different extract paths for OSS and EE artifacts to avoid conflicts:
- EE artifact extracts to 'ee-artifact/' directory
- OSS artifact extracts to 'oss-artifact/' directory
This allows both artifacts to coexist without file conflicts, fixing the
artifact recycling issue for both OSS and EE versions.
🤖 Generated with [Claude Code](https://claude.ai/code)
Co-Authored-By: Claude <noreply@anthropic.com>
* fix(ci): resolve OSS/EE artifact conflicts by extracting to separate paths
Fixes COMMIT-ID file conflicts when fetching both OSS and EE uberjar artifacts
by extracting each artifact directly to its designated directory path instead
of the current directory. Simplifies fetch-artifact action logic while
maintaining full backwards compatibility.
Changes:
- Extract artifacts directly to specified extract-path
- Eliminate file conflicts between OSS and EE artifacts
- Simplify processing logic by unifying unzip and rename steps
- Maintain backwards compatibility for all existing workflow usages
🤖 Generated with [Claude Code](https://claude.ai/code)
Co-Authored-By: Claude <noreply@anthropic.com>
* fix(action): correct error message for missing artifacts to use input name
* fix(ci): optimize uberjar concurrency to prevent duplicate builds per commit
Changes concurrency grouping from branch-based to commit-based to allow
multiple workflows to call uberjar for the same commit while preventing
duplicate builds. The existing artifact checking logic handles deduplication.
- Group by commit SHA instead of branch ref
- Disable cancel-in-progress to avoid wasting build work
- Allow concurrent calls from different workflows (pr-env, perf-test, etc.)
- First caller builds, subsequent callers reuse existing artifacts
🤖 Generated with [Claude Code](https://claude.ai/code)
Co-Authored-By: Claude <noreply@anthropic.com>
* fix(action): use temp directory to avoid file conflicts during artifact extraction
The fetch-artifact action was failing when multiple artifacts were extracted
to the same directory due to file conflicts (e.g., COMMIT-ID files). The unzip
command would prompt to replace existing files, causing CI failures.
This change extracts artifacts to unique temporary directories first, then
moves only the JAR file to the final location, eliminating conflicts while
maintaining full backwards compatibility.
🤖 Generated with [Claude Code](https://claude.ai/code)
Co-Authored-By: Claude <noreply@anthropic.com>
* fix(workflow): remove source_changed input and simplify build conditions
* fix(action): remove unnecessary blank line in fetch-artifact action
---------
Co-authored-by: Claude <noreply@anthropic.com>
* Fix regression - small cta for large upsell banner and big card
* Extract upsell styles to per component css modules
* Regression - bring back margins to cta in large upsellbigcard
* Add upsell components to loki visual tests
* Fix too big margin in upsell large variant
* Align storybook story with the real usage
* Update Loki Snapshots
* Allow custom button styling on UpsellCard, move custom styles card instance
---------
Co-authored-by: Metabase Automation <github-automation@metabase.com>
* Update writing-sql.md
Adds a section below "How Metabase executes SQL queries" noting that we don't officially support multi-statement queries or DDL (and recommend against use of procedures and functions).
* Apply suggestions from code review
Co-authored-by: Jeff Bruemmer <jeff.bruemmer@gmail.com>
---------
Co-authored-by: Jeff Bruemmer <jeff.bruemmer@gmail.com>
* Make Lib schemas better about catching MBQL-only keys in native stages or joins
* Replacing a field ref in join :fields should set the correct join alias
* Add view_count scorer to semantic search backend
* Get pgvector db from global env in semantic.scoring/view-count-percentiles*
Get the db data-source directly rather than passing it in as an argument to this function to side-step potential
issues with using the db as a cache key for the memoized version.
https://github.com/metabase/metabase/pull/62086#discussion_r2272790618
Adds an option to the mage nrepl command to customize the port mage's
nrepl starts on. This is mostly so I don't need to modify my clojure-mcp
configuration to point to a different port when working on mage stuff.
* remove references to static questions
* Ditch overall sidebar for now (#61523)
* Remove list of cards sidebar
* Some type fixes
* Review fixes
* feat: associate in_report cards w/ report_document
* feat: associate cards with reports in create
* better markdown serde
* switch to prosemirror ast
* chore: add enterprise card test
* remove snapshots
* less redux
* send used card ids
* drop flushSync error
* remove replace question, drop refresh question
* fix backend not starting
* saving new report should not show LeaveRouteConfirmModal
* polish
* Insert into reports from metabot (#61642)
* insert from metabot
* Metabot plugin and hook
* make stuff nicer
* cleanup
* no more snapshots
* run build and docker uberjar on push
* remove trigger for uberjar
* Print Report
* cleanup
* always create a card clone when embedding
* always clone non in_report cards
* disable metabot sidebar on mob+b
* stop header bouncing
* jumping layout when loading
* why scroll to bottom when opened
* gross emojis
* cleaning up emoji ui a bit
* icons
* fe: reports -> documents
* update metabot question creation parsing
* keep report cards and collections in sync (#61686)
* do not leak prosemirror styles into charts
* chore: rename everything to document (#61743)
strip out snapshotting code
* fix FE usage of documents api
* fix embedding dashboard-only cards to documents
* uxw-366: Navigation back to the report from a drill through can be difficult - can we add the back button like for dashboards
* mention all entities
* split mention vs command, allow mentioning other entities
* paste smartlinks
* missing import
* fix links to tables
* fix layout
* auto focus on title on new documents
* UXW-404 Improves image formatting
* Tweaks metabot block styling, wires up "stop generating" functionality
* Fix issues with multiple in-flight requests
* Adds subtle animation to "Working on it..." message
* Document Metadata
* Fix some CI issues
* Persist cards when saving document (#61818)
* chore: remove document version model
* persist cards only when saving documents
* chore: remove in_document type
* feat: return document creator
* feat: save cards with parameters pass in as cards
in POST and PUT requests to documents
* chore: exclude doc cards from where shouldn't be
search, recents, collection items
* chore: fixup search and card tests
* contract
* feat: update document ast with new card ids
* fix: report -> document in collection api
* fix: ws
* feat: clone cards not associated with document
* chore: add tests for document permissions
* fix contract
* fetch root collection
* update message
* must be working now
* fix sidebar
---------
Co-authored-by: Edward Paget <ed.paget@gmail.com>
* wip but working sql query editor in documents
* feat: revision history for documents
* feat: include revision info in collection tree res
* document entity
* Fix issue where temporary implementation of metabot block can't create questions
* feat: allow documents to be archived
* Document Collection Item Caching
* Fix issue where submitting empty metabot prompt (by hitting "Run" or cmd+enter) makes it spin forever
* fix UXW-412 again
* feat: allow documents to be bookmarked
* fix: hide documents in collection items if not EE
* fix: add perms info to documents in collections
* fix: always add documents to bookmark lookup
* feat: add document delete endpoint
* fix: require non-blank string for name
* App Bar Breadcrumbs for documents
* fix: tests and lints
* fix: ordering of migrations for mysql
* fix: for OSS tests
* fix: tests and kondos
* fix: module sort
* New metabot documents API
* UXW-400: formatting floating menu
* UXW-416: replace question should replace only 1 question
* swap icon
* l10n
* fix: bad var reference in tests
* undo moves should work once archived is boolean
* Show tooltip with precise time in document metadata
* Updates MetabotEmbed to work with upcoming BE changes
* Updates MetabotEmbed to always use draft cards
* Updates MetabotEmbed "stop generating" functionality to work with regular (non-streaming) api call
* Updates MetabotEmbed to use most up-to-date api contract
* Stubs out the necessary updates to make MetabotEmbed work with an UnsavedCard from the BE rather than a full Card
* Updates MetabotEmbed to use most up-to-date api contract
* fixed drills & query editing for metabot cards
* fix: foreign key indexes
* Replaces hard-coded metabot-generated card title with title from the /generate-content response (or translates a default title if none provided)
* fix: add entity id column to documents
* add title to untitled docs
* UXW-453 Fix 404 when clicking draft card title
* Handle URL slugs, remove version code from document page, handle 404 and 403
* UXW-459 Fix issue where creating a new document when starting from another document overwrites the existing document (and shows previous document's metadata in new document)
* hide object detail row in document tables
* UXW-459 Fix issues with LeaveRouteConfirmModal and new documents
* UXW-461 Prevents jump when Save button appears/disappears. Also adds subtle fade-in transition when it appears.
* UXW-461 Save button always shows up for new documents (and user has necessary permissions)
* improved mention and command
* clean
* fix replace question
* do not show bubble menu on disallowed nodes
* MetabotEmbed polish:
* Prevents prompt from highlighting when you hit "Run" (potentially bringing up the editor bubble menu).
* Prevents useEffect hook from running way more than it needs to.
* Adds margin below MetabotEmbed.
* Hitting escape with nothing in the textarea will now remove the MetabotEmbed block.
* Makes layout less janky when Metabot responds with a long error message.
* Localizes "Created with Metabot" text
* drop emojis
* no emojis and markdown downloading
* fix metadata fetching
* initial e2e tests
* fix: set null on collection cascade for documents
mirrors behavior for dashboards and cards
* fix: remove documents token hack
* feat: documents in search
Also supports view_counts and last_viewed_at for documents
* fix: emit documentment read on api get
* fix: in_place search fixes and test fixes
* fix: migration order
* fix: set type to :question dashboard_id to nil
* fix: more in-place search fixes for documents
* fix: final in-place search test fixes
* fix: handle inadvertent documents in oss tests
* mention + command feedback
* placeholder
* fix command search
* Fix issue where charts sometimes don't get added via metabot when inserted in the middle of a document
* fix command
* feat: support pinning documents
* fixup! pinning reconcilation with documents
* feat: support serdes on documents (#62032)
* fix: kondo warning on unused var
* updated styles
* bump padding
* Markdown Loki Test
* UXW-462 Adds database picker in metabot block
* UXW-462 Fix issue where sometimes custom tiptap node tries to re-render too quickly resulting in an error
* refactor
* feedback on the command
* UXW-462 Fix issue where you can write markdown in metabot blocks
* improve state management UXW-463 UXW-450
* Updates "Created by Metabot" line based on design/user feedback
* feat: add documents to recents
* fix: whitespace and round-trip serdes test
* fix: whitespace exclusion path
* fix: token features in recent views test
* Able to generate sql charts
* fix: casing in tests
* Changes how mention text is referenced in the documents metabot prompt serializer. It uses a cache in the documents slice now. It was using a node attribute before but that was hacky and it messed with the "is dirty" calculation.
* more e2e tests
* Fix issue where metabot nodes sometimes yield a false positive in hasUnsavedChanges because they updated their own attributes
* Update menu items (#62072)
* Update commands and remove descriptions
* Removes unused code
---------
Co-authored-by: Brad Anderson <brad@metabase.com>
* feat: return can_restore, can_delete for docs
* Updates Document type with newly added properties
* Fix issue where "Delete permanently" errors out instead of deleting
* Move, Archive, and Bookmark in documents page
* bury shame
* UXW-472 Adds ArchivedEntityBanner for documents
* fix: remove extraneous precondition migration
* UXW-451 Adds some light dataset error handling
* Reverts probably-accidental change
* feat: check for name length
* Reports fix fe ci (#62046)
* Fix some CI issues
* Fix some CI issues
* Fix typecheck issues
* Fix lint
* Fix unit tests
* Fix e2e tests
* fix pivot query editing in documents
* better command popover positioning
* UXW-473 fix layout
* fix z-index
* only include tables you can use for native query builder
* Fix typecheck
* Light documents metabot cleanup
* Fix issue where browsers spell check areas that shouldn't be spellchecked (e.g. "Ask Metabot ...")
* Removes package-lock.json that was accidentally added in 63a94c8788 and should've been removed in 68366d7d37
* cleanup
* attempting to get a card embed in storybook
* Fix documents stories
* fix: bookmarks fail if token exists but ee doesn't
* fix: bookmark inside a collection error
* Removes now-unused extensions
* chore: tests for metabot tools
* Add token check on the FE
* fix: remove unused ns from test
* Review fixes
* Revert token check
* Add token check on the FE
* fix: correctly serialize token status to FE
* Fix e2e tests (#62113)
* Fix e2e tests
* Fix e2e
* fix: use adhoc metadata if card id is negative
* chore: add tests for isSavedCard guard
* UXW-489: replace shows entity picker, UXW-507: don't highlight chart after insertion
* Fix typecheck
* fix: token feature tests
* fix: add docs to stats ping
* Fix typecheck
* Fix e2e
* Reverts now-unnecessary changes
* Reverts more now-unnecessary changes
* Address cubic PR feedback (type guards)
* Fix issue where landscape-orientation images larger than the max-width get stretched
* Fix e2e
* PR feedback (code cleanliness)
* PR feedback (simplify custom node definition)
* adding loki references
* delete loki reference files
* adding missing loki reference files
* auto-fix generated sql (#62167)
---------
Co-authored-by: Edward Paget <edpaget@users.noreply.github.com>
* PR feedback (unit test + remove unused classes)
* addressing some minor feedback
* chore: add table utils tests
* review
* Cleans up types
* Removes /documents/redux-utils. Uses plain useDispatch/useSelector hooks instead.
* review
* cleanup documents selectors
* remove unnecessary type cast in visualizationUtils.ts
* remove versioning mention in comments
* use _.isEqual instead of json serialization and string compare
* Code review fixes
* remove inline styles from FormatButton
* fix bubble menu does not immediately show what formatting is applied
* remove inline styles
* drop unused suggestion styles
* consistent hook names
* drop unused dep
* remove any from DocumentPage
* clean up change callbacks of the editor component
* drop unused onClode prop in EmbedQuestionSettingsSidebar
* provide correct reportTimezone to ModifyQuestionModal
* make runQuestionQuery optional
* NativeQueryModal cleanup
* clean up NativeQueryModal
* drop unused code in useDocumentState
* unnecessary casting
* reset error state
* drop unused prop
* bubble menu disallowed nodes
* dedupe similar hooks
* review
* fix: kondos
* Update e2e/support/helpers/api/createDocument.ts
Co-authored-by: cubic-dev-ai[bot] <191113872+cubic-dev-ai[bot]@users.noreply.github.com>
* fix: PR comments
* fix: remove schema ns
* Potential fix for e2e test failing on CI only
---------
Co-authored-by: Nick Fitzpatrick <nickfitz.582@gmail.com>
Co-authored-by: Denis Berezin <denis.berezin@metabase.com>
Co-authored-by: Edward Paget <ed.paget@gmail.com>
Co-authored-by: Ryan Laurie <30528226+iethree@users.noreply.github.com>
Co-authored-by: Luiz Arakaki <luiz.arakaki@metabase.com>
Co-authored-by: Ryan Laurie <iethree@gmail.com>
Co-authored-by: Edward Paget <edpaget@users.noreply.github.com>
Co-authored-by: Brad Anderson <brad@metabase.com>
Co-authored-by: Nathan Voxland <nathan@metabase.com>
Co-authored-by: eliotmetabase <eliot@metabase.com>
Co-authored-by: cubic-dev-ai[bot] <191113872+cubic-dev-ai[bot]@users.noreply.github.com>
* Remove feature-visualizer and nm-semantic-search-in-metabase from the Build Uberjar workflow
The feature-visualizer branch was removed in 3e605c6d87, but came back in (presumably due to bad merge conflict
resolution) from the semantic search branch.
* Fix typo in gate test: documen -> document
* Make cause third arg to ex-info in semantic-search /status endpoint
* Create HeaderLinkGroup with new links and add to ExistingDatabaseHeader.tsx
* Don't show database added popup when database is created
* Ensure 'Browse data' button link does nothing when disabled
* Fix e2e tests
* Fix e2e tests part 2
* Adjust 'Browse data' link to be an actual button when disabled
* Fix padding (keep button padding & adjust flex gap)
* copy over relevant code from semantic search service
* copy docker compose, add defenterprise integration
* WIP getting the integration working
* fix up reindex! call
* fix whitespace
* kick ci
* remove ^:const to try to fix compilation errorin tests
* Adds mi/can-read? checks to search results (#61060)
* Add a couple of helpful comments for manual testing
* Add ability to change db-url at the repl
* Add can-read? checks for search results
* Semantic search engine should not be used if the db is not setup
* sort kondo modules
* fix up syntax in metabase.search.impl and whitespace
* remove a blank line
* fix remaining clj-kondo issues
* Add semantic search test workflow (#61065)
* Add new workflow for semantic search
* Add a quick example test to see if things work
* Have the new tests only run if db-url is defined
* Sparks of joy
* Add a delay to appease the kondo gods
Not sure this rule should apply to once-fixture, but for now lets fix
it.
* Semantic search filtering, basic querying tests, connection pooling, etc (#61098)
* fix results postprocessing, add verified field to search index
* scaffold a basic query test
* index filtering tests
* quicky and dirty implementation of the requested ui changes
* Add basic connection pooling + tests
* Fix test indentation
* add test for verified content, refactor tests a bit
* address comments
---------
Co-authored-by: Sloan Sparger <sloansparger@gmail.com>
* Add nm-semantic-search-in-metabase to uberjar workflow and remove metabot-v3-main (#61142)
In preparation to deploy the nm-semantic-search-in-metabase branch to stats.
* Use EE feature for semantic search defenterprises (#61139)
* switch defenterprise features from :none to :semantic-search
* update tests
* Register semantic-search premium feature
* FE - default to app db search, enable semantic in command palette and search page, add disable semantic search on search page
* Test collection based permission checks apply to search results (#61152)
* Fix t2-instance derivation from search docs
Lookup in database as the search document model is lossy
* Test collection based permission checks apply to search results
* Swappable embedding models + semantic search kill switch (#61176)
* first stab at swappable embedding models + setting to disable semantic search globally
* fixups
* fixups
* fix up error in tests
* add semantic-search to stats ping to quiet test
* quiet another test
* fix whitespace
* fix kondo warnings
* fix up and simplify embedding tests
* fix core init test
* fix more tests
* tweak setting defaults + comments, remove debugging code
* fix compilation
* change oss version of supported? to return false instead of throwing
* fix lint issues
* Batched index updates for semantic search (#61117)
* kick ci
* Add comment to trigger reindex job
* Fix docstring typo
* Batch index updates for semantic search
* Wrap {create,drop}-index-table! in mt/as-admin in with-temp-index-table!
* Move binding for *vector-dimensions* into with-mock-embeddings
* Create metabase-enterprise.semantic-search.test-util and move once-fixture there
* Move other test helpes and macros into the semantic.test-util ns
* Factor out with-indexable-documents! from with-index!
* Move binding of search.ingestion/*force-sync* into with-temp-index-table!
Questionable, but realistically can't see tests wanting to use a temp index table without forcing sync reindex.
* Add table-exists-in-db?! to semantic.test-util
* Add some basic tests for funcs in the semantic.index namespace
* Add tests for batched index updates
* Disable flaky tests
* Fix up issues post-rebase
---------
Co-authored-by: Noah Moss <noahbmoss@gmail.com>
* Fix accidental blocking search index init (#61200)
* kick ci
* Fix accidental eager index initialisation
---------
Co-authored-by: Noah Moss <noahbmoss@gmail.com>
* Batched requests to OpenAI for fetching vector embeddings (#61217)
* Batch calls to openai according to token counts
* refactor
* mock the new batch embedding function
* mock both batched and unbatched embedding functions in tests
* a little code consolidation
* address comments
* Init db connection in reindex! if needed
* Add more DB existance checks for safety
* fix fe tests
* Lower batch size, improve logging, and make it configurable via env var (#61274)
* Lower batch size, improve logging, and make it configurable via env var
* fix whitespace
* Remove transaction from batched semantic search index updates (#61229)
* Drop uneccessary ! from table-exists-in-db?!
* Don't wrap batched index updates in a transaction
https://github.com/metabase/metabase/pull/61117#discussion_r2214828302
* Filter indexed entities from semantic search index
* Create HNSW index on semantic search embeddings column (#61238)
* Create HNSW index on semantic search embeddings column
* Add test for HNSW index creation
* Ensure identifiers are quoted when formatting honey sql in semantic search index
The nano-ids used in temp table names when testing (and various other places) might contain uppercase chars or hyphens which need to be quoted.
* Fix batch-delete! log message
* alter create-index-table! to not wipe the index if it already exists, and use the upsert API during index population (stop-gap changes to try to get the staging instance working)
* fix document filtering
* Only create HNSW embeddings index :if-not-exists (#61305)
Since init! no longer wipes the index table if it already exists, we also need to only create the embeddings HNSW
index if it does not exist.
* Better batching for semantic search (#61299)
* streaming embedding results to the index-updating code
* rework
* suppress kondo
* Parameterization refactor (#61328)
Adds the `index` and `embedding-model` parameters to functions in `index` and `embedding` respectively. This allows a degree more flexibility and less coupling to global variables which restrict how indexing functions can be used.
The goal is this PR supports changes regarding active/pending and the embedding model being something you can play with at runtime.
* Add embedding-model parameter
* Restore previous top-level transaction behaviour on init/reindex
* Reduced redundancy
* Embedding test fixes
* Fixup comments
* Fix rebase goofs
* Fix tests again
* Joy
* Add `index` parameter, composed of table-name and embedding model
* Keep index name in the index structure
* English is a hard language
* Remove `:redef` we no longer want
* Another merge goof fix
* Fix bug in associating batched embeddings with original documents (#61347)
* Fix bug in associating batched embeddings with original documents
* tweak
* add log
* Quick follow-up optimization to prior PR: don't iterate over all of filtered-documents in each batch
* [Metabot] Hybrid search w/ RRF (#61353)
* rrf based hybrid search implemented
* keyword results weighted more heavily by name, corrected some issues with the rrf calc
* fix final result ranking
* move tsvector-related util functions into search.util and expose in the search interface to fix kondo errors
* Taking a stab at fixing SQL injection risks following existing postgres tsvector querying pattern
* Move postgres tsvector util tests
* gracefully handle nil names
* switch back to raw embedding literals but validate their contents
* add back perm checks (whoops)
* add back str/replace for ts-search-expr, bump up cosine distance cutoff
* tweak cutoff again, fix create index test
* remove str/replace usage bc it seems unneeded
* fix rrf calc
---------
Co-authored-by: Noah Moss <noahbmoss@gmail.com>
* Remove indexed-entities from the semantic search model-set and documents reducible (#61304)
* Remove indexed-entities from the semantic search model-set
* Add comment about filter-read-permitted and indexed-entity docs
* Add prometheus metrics for semantic search backend (#61309)
* Add :metabase-search/semantic-index-size prometheus metric
And update it on upsert-index! / delete-from-index!
This metric mirrors the corresponding :metabase-search/appdb-index-size
* Add :metabase-search/semantic-embedding-tokens prometheus metric
* Make log levels / messages more consistent
* Remove vestigial populate! reference
* Record actual token usage as reported by OpenAI API response
* BOT-255 Index metadata system (#61379)
* Add index-metadata system
Adds the necessary indirection to support more advanced index table
management.
- Allows multiple indexes to exist
- Indexes are tracked by a new index_metadata table
- A single active pointer determines which index to use for the current
search engine backend
- It is assumed table == logical index and there is no other
partitioning, hence the unique constraint
- Some small simplifications to index.clj (drop index-name, assume names
are prefixed with the table name)
- The top level orchestration code happens in a new pgvector-api
namespace to avoid bloating the enterprise hooks (which assume a global
environment), and allow the api level to be properly parameterized.
* Kondo
* Appease kondo with require for next.jdbc in comment block
---------
Co-authored-by: Mike Appleby <86076+appleby@users.noreply.github.com>
* Enable fts on native query contents in the semantic search backend (#61576)
* Enable fts on native query contents in the semantic search backend
Update the semantic search backend to handle fts of native query contents in exactly the same way as the appdb
backend, namely: all keyword search content gets indexed into two separate columns, one with and one without native
query contents. At runtime, we decide which column to query against based on the user's search setting.
For now, this only modifies the keyword search side of the hybrid search result. The semantic search on the vector
embeddings still does not include native query contents, but any keyword matches can still get blended into the result
set by the RRF ranking.
* Ensure index names do not exceed postgres max of 63 characters
* Allow to toggle NativeQuery independent of DisableSemanticSearch in SearchSidebar.tsx
* Cleanup whitespace
* Add native-query-test
* Add :native_query to semnatic.tu/mock-documents
* Add upsert-index!-tsvectors-test
* Remove unecessary :raw wrapper from column names
* Remove unused =? alias
* [Semantic Search] Embedding generation support for OpenAI compatible APIs (#61641)
* make open base url configurable, add arctic-embed support, sanitize index table name for models with slashes + commas + periods
* make table names slightly shorter
* Update enterprise/backend/src/metabase_enterprise/llm/settings.clj
Co-authored-by: appleby <86076+appleby@users.noreply.github.com>
* Update enterprise/backend/src/metabase_enterprise/semantic_search/index.clj
Co-authored-by: appleby <86076+appleby@users.noreply.github.com>
* Update enterprise/backend/src/metabase_enterprise/semantic_search/index.clj
Co-authored-by: appleby <86076+appleby@users.noreply.github.com>
* split off provider name to make index shorter
* Fix tests failing due to removing text-embedding-3-large model
* Abbreviate model names to ensure we do not exceed postgres identifier name limit
---------
Co-authored-by: appleby <86076+appleby@users.noreply.github.com>
* Ensure that semantic search entrypoints are safe to call without init! (#61438)
* Ensure that semantic search entrypoints are safe to call without init!
* Move more helpers into semantic.test-util namespace
* Add tests for calling semantic search entrypoints prior to calling init!
* Adjust fix and tests to work better with new index-metadata apis
And also log a warning and return nil if anyone calls into update-index! or delete-from-index! prior to calling init!
* set default for ee-openai-api-base-url setting
* BOT-258 semantic indexed entities (#61527)
* kick ci
* Semantic search filtering, basic querying tests, connection pooling, etc (#61098)
* fix results postprocessing, add verified field to search index
* scaffold a basic query test
* index filtering tests
* quicky and dirty implementation of the requested ui changes
* Add basic connection pooling + tests
* Fix test indentation
* add test for verified content, refactor tests a bit
* address comments
---------
Co-authored-by: Sloan Sparger <sloansparger@gmail.com>
* FE - default to app db search, enable semantic in command palette and search page, add disable semantic search on search page
* Init permission support for indexed entities
* fix kondo
* Fix rebase errors
* Fix rebase errors
* pr feedback
* unit test index entity perm filter
---------
Co-authored-by: Noah Moss <noahbmoss@gmail.com>
Co-authored-by: Noah Moss <32746338+noahmoss@users.noreply.github.com>
Co-authored-by: Sloan Sparger <sloansparger@gmail.com>
* BOT-259 Semantic search filter parity (#61396)
* kick ci
* Semantic search filtering, basic querying tests, connection pooling, etc (#61098)
* fix results postprocessing, add verified field to search index
* scaffold a basic query test
* index filtering tests
* quicky and dirty implementation of the requested ui changes
* Add basic connection pooling + tests
* Fix test indentation
* add test for verified content, refactor tests a bit
* address comments
---------
Co-authored-by: Sloan Sparger <sloansparger@gmail.com>
* FE - default to app db search, enable semantic in command palette and search page, add disable semantic search on search page
* Remaining search filter support minus has_temporal_dimensions
* ignore public API warning about metabase.search.filter
* Fix collection filter impl
* fix kondo
* [no ci] remove comments
* More efficient querying
* Fix rebase errors
* fix merge
---------
Co-authored-by: Noah Moss <noahbmoss@gmail.com>
Co-authored-by: Noah Moss <32746338+noahmoss@users.noreply.github.com>
Co-authored-by: Sloan Sparger <sloansparger@gmail.com>
* try something else
* Remove semantic HNSW index breaking where clause (#61765)
* Adjust semantic search query to favor HNSW index over seq scans (#61768)
* Adjust semantic search query to favor HNSW index over seq scans
* remove limit and SET random_page_cost
---------
Co-authored-by: Sloan Sparger <sloansparger@gmail.com>
* [SEMANTIC] base64 embeddings and better logging (#61809)
* Improved logging; base64 embedding
* trim double
* Use parse-double over parseDouble
* fix search.engine/init! return (#61825)
* fix: return model frequencies from search.engine/init!
* fix: document return type of init!, like per delete! and update!
* fix: calc for batches
* semantic search instrumentation (#61790)
* intrument semantic search
* test for analytics
* typo
* fix: id is numeric
* move test next to metrics one
* Support switching between :semantic and :appdb engines (#61826)
* add test
* lift util
* feat: add full api test
* fix: appdb available when search-engine is semantic
* Fallback to appdb search if semantic search doesn't return enough results (#61815)
* Fallback to appdb or in-place search if not enough results are returned from semantic search
* Dedupe combined results
* Bump up hybrid search query limit + make it configurable
* tweak setting doc, fix reflection warning in test code
* clean up tests
* move fallback implementation into the EE namespace, add a new setting for min results threshold
* fix lint issues
* ensure tests set semantic-search feature
* Do not realise document reducibles (#61860)
* wip
* batched streaming upsert-index!
* fix: actually dedupe searchable_text
* Revert "try something else"
This reverts commit eec29881c5.
* can still inner seq
* push indexed-entities last
* fix: semsearch delete
* lift delete-from-index-batch-sql
* test
* dedupe test
* style
* comment
* indexed entities last in search
* Semantic search test fixes (#61884)
* Fix api-engine-switching-test
* Test - switch test fixture to :each to see if it removes a race condition
* Revert "Test - switch test fixture to :each to see if it removes a race condition"
This reverts commit 33386c602b.
* another test - ensure init-delay is derefed before getting datasource
* remove compose-fixtures
* Revert "remove compose-fixtures"
This reverts commit 16c882a599.
* another test
* test
* add back some of core-test
* comment out all code but leave in ns
* add back fixtures
* add back one test
* replace with stub test
* remove compose-fixtures
* fix tests with a try/finally in db_test.clj
* oops, add back :once
* also revert back fixture order
* more cleanup
* use with-index! in appdb-available-with-semantic-test rather than trying to generate real embeddings
* Fix db test
* Only fetch embeddings when needed (#61918)
* chore: use appropriate builder-fn instead of unqualify-keys
* feat: cache on already seen embeddings
* add test
* distinct-on content
* better logging
* reuse test impl
* note
* repl utils
* we have if-not-exists semantics, so no point in not invoking it on init.
We need to invoke it else the new index won't get picked up.
This will change anyway once we have proper migrations with #BOT-278
* fix tests
* Incorporate metadata into semantic search result ranking (#61906)
* Move common code into search.scoring and add search.engine/scorers for backend-specific scorers
* Refactor to remove search.engine/scorers
* Add new index columns required by scoring expressions
* Add metadata score to semantic search backend
* Give up on reusing scorer expressions between appdb and semantic search
* Add :all-scores stats to semantic search query results
* Fix failing pgvector query test
* Adjust rrf weight to fix failing query tests
* Add premium scorers for verified and official_collection
* Get rid of single-use test helper dog-training-native-query-json
* Add scoring tests
* Fix for permissions issue in test
* Remove extra whitespace in bindings
* [Semantic Search] Add AI Service as an embeddings providers (#61850)
* wip
* clean up and better abbrev logic
* fix provider references
* remove magic int
* do not munge model name if not needed
* add / fix tests
* make mock index have a shorter name
* shorten nanoid
* math was not mathing
* fix kondo error
* death by lints
* [BOT-260] sem search consistency, gating and indexer loops (#61602)
* Add document gating
Adds a document gating mechanism and async indexing job.
- Allows indexing to recover if it encounters a transient error.
- Avoids data loss during indexing due to process death
- Provides an ordering which is more closely related to logical record
validity times rather than write time into the index.
* Fix some test docstrings
* Fix some loop tests
* Schedule indexer only if the instance has the :semantic-search feature
* Avoid redundant replay of the very last gate entry when the quartz job fires
* Add a quick candidate-filter byte equality test
* Refine loop interrupt/error test (get rid of old gunk)
* Use shared `tu/spy` instead of copy-pasta
* Replace candidate filter with simple set of last seen
Use TEXT for hash for eq convenience.
* Avoid future foot guns with a (letfn) refactor of indexing-step
Avoids issues that might later be introduced by the 'accidental' closure of (actual) variables that might later be changed during this side-effecting operation. In particular the move-to-next-watermark local function.
^:once was suggested but a hoisted letfn that can only close over args (which keeps the mutable indexing-state at the forefront of ones mind) keeps us honest.
* Check if engine enabled and supported during indexer task
Previously was only checking if the db was configured.
* Be more careful about Instant and timestamp representation.
Preferred is sql.Timestamp for now due to the default setup of postgres / JDBC.
* Fix up some asserts that still reflected byte hashes
* Add a quick e2e index everything test.
Find horrific pagination bug. Fix that.
We did not account for having enough ties that our buffer was full. This was likely in tests even with the high precision of the pg clock because:
- we coerced stuff to Instant which lost precision
- we generally did not account for tie breaks in pagination (bad form)
So we are paginating properly using the secondary id as a tie break. Within the confidence time only, as otherwise this would be dangerous (it is right to stall if you see too much until the lag time has elapsed). This stops stalls if lots of documents share a buffer.
Will follow up with another test that forces time collisions to assert this tie break behaviour independently.
* Test fixes
* Joy
* Add indexer docstring
* Thread pool for semantic search index batch updates (#61947)
* parallel batch processing for index updates
* tweak setting docs
* switch back to transducing version
* Revert "switch back to transducing version"
This reverts commit 4a99c3c6869e28f56d794d8057f006700cfc8432.
* fix setting name
* revamp approach - use a ThreadPoolExecutor with a custom rejection policy
* fix tests
* add concurrency test
* Fix up tests by adding a serial mode that disables thread pool usage
* Fix final test -- ensure we spy on & call only on the 4-arity version of upsert-documents
* Switch to custom retry handler
* Still wrap executor in a delay for lazy initialization
* Fix upt ests
* Fix final tests
* [Semantic Search] Remove disable semantic search filter (#61973)
* remove disable semantic search filter
* fix failing test
* Collapse indexed-entity ids in semantic search results (#61952)
* Collapse indexed-entity ids in semantic search results
The indexed-entity results have composite ids like model-index-id:model-pk. We need to extract the model-pk and attach
that as the :id on the results. Otherwise, the links generated by the frontend are invalid like
/model/12345-blah/1234:62681
* Add indexed-entity-id->model-{index-id,pk} to search.util
And use indexed-entity-id->model-index-id in filter-can-read-indexed-entity
* Fix test failure due to overly long index name
* Use dimensions param in OpenAI api calls (#62010)
* Use dimensions param in OpenAI api calls
* Use starts-with?
* Improved fallback logic & prometheus metrics for semantic search (#62024)
* Add prometheus metrics to track appdb search fallback logic
* undo broken function call
* Update logic to not fallback if no results were filtered out
* fix more tests
* Tweaks
* Try again to fix tests
* Address comments
* update docstring
* Fix non-batched get-embedding for openai and ai-service providers (#62050)
* Fix non-batched get-embedding for openai and ai-service providers
We were passing a single text string rather than a seq of texts, resulting in an error when we tried to get the token
count.
ERROR semantic-search.embedding :: Failed to generate OpenAI embedding for text of length: 8
java.lang.ClassCastException: class java.lang.Character cannot be cast to class java.lang.String
* Add test for extract-base64-response-embeddings
* Add test-get-embedding
* Semantic search indexing status endpoint (#62082)
* add search-items-count function
* Add endpoint for reporting indexed count + total estimate
* try to fix test
* Temp commit to debug CI failure
* Add honeysql classpath to debug output
* Revert "Add honeysql classpath to debug output"
This reverts commit 4395f420f7.
* Revert "Temp commit to debug CI failure"
This reverts commit 36fbf047d6.
* Fix honeysql create-index :raw forms (#62144)
Something changed with honeysql 2.7.1340 and :raw forms in the tail of a create-index now get unwrapped and need to be
nested in another vector.
For example, in earliers versions
(sql/format
(sql.helpers/create-index
[:foo-idx :if-not-exists]
[:foo-table :using-hnsw [:raw "embedding vector_cosine_ops"]]))
produced an sql string like
CREATE INDEX IF NOT EXISTS foo_idx ON foo_table USING HNSW (embedding vector_cosine_ops)
but on 2.7.1340 you get
CREATE INDEX IF NOT EXISTS foo_idx ON foo_table USING HNSW (raw EMBEDDING VECTOR_COSINE_OPS)
Unclear if this is a honeysql bug or we're doing something unsupported, but fix it with a workaround for now by
wrapping the :raw in an extra vector like [[:raw "foo"]], in order to get CI green.
* Move reindex/index to gate the documents (#62101)
* switch init/reindex to gated updates
* nil stats
* sync index in test
* non deterministic search result
* Semantic search migration (#62043)
* Migration
* Rebase follow-up
* [Semantic Search] Add admin setting to enable semantic search (#61622)
* add basic setting switch for enabling semantic search, upsell component, logic for falling back to default search in settings, and basic e2e test
* fix type issue
* Fix serialization baseline test
* finished
* small fixes
* do not animate progress bar when progress === 100
* impl PR feedback
---------
Co-authored-by: Tyler Plude <tsplude@gmail.com>
* revert e2e diffs (#62173)
* Re-order semantic search index columns to test schema migration (#62172)
* Re-order semantic search index columns
Mostly to test schema migration, but this also makes the table contents easier to scan in PgAdmin / psql imho by
putting ids next to ids, timestamps next to timestamps, etc.
* migration 1
* cleanup
---------
Co-authored-by: lbrdnk <lbrdnk@users.noreply.github.com>
* fix reflection warnings in tests (#62151)
* fix reflection warnings in test
* just use bound-fn*
---------
Co-authored-by: Dan Stone <wotbrew@gmail.com>
Co-authored-by: Sloan Sparger <sloansparger@gmail.com>
Co-authored-by: appleby <86076+appleby@users.noreply.github.com>
Co-authored-by: Sloan Sparger <sloansparger@users.noreply.github.com>
Co-authored-by: Tyler Plude <tsplude@gmail.com>
Co-authored-by: Ryan Laurie <iethree@gmail.com>
Co-authored-by: Nicola Mometto <nicola.mometto@metabase.com>
Co-authored-by: Mihael Konjević <konjevic@gmail.com>
Co-authored-by: lbrdnk <lbrdnk@users.noreply.github.com>
* API Keys module cleanup
* Test fixes 🔧
* Fix Kondo errors
* Unmark these tests as `^:parallel` for now because I do not have the energy for dealing with MySQL
* I don't have time to fight MySQL
* Test fix 🔧
* add-alias-info => MLv2
* Restore deleted test
* WIP
* WIP [ci skip]
* Ensure we're actually using a cached metadata provider in tests
* Remove profiling code
* Restore accidentally-deleted `native-query-snippet` fn
* Fix accidental edits
* Make `visible-columns` stage-only
* More minor caching improvements
* cljfmt & kondo
* More minor caching improvements
* Fix#35252
* cljfmt & kondo
* Another round of test fixes and lint fixes 🔧
* Test fixes 🔧
* Test fix 🔧
* PR feedback
* Test & Lint fixes 🔧
* Test fix 🔧
* Kondo/cljfmt
* Appease Eastwood
* cljfmt run against everything
* Revert changes to deps.edn
* Did I fixx??
* cljfmt/kondo
* Fix Oracle test failure
* Fix nested fields (MongoDB)
* cljfmt
* Nested column name fixes
* Fix Mongo for reals
* Revert BigQuery breaking stuff
* Fix equality for binned columns
* Test fix 🔧
* Remove dead :ident keys
* Fix is_verified field refs
* Fix is_verified field refs
* Some sanity checking
* Fix typo
* Fixx
* Ok, don't be super fussy about bad code using :type/Text as a parameter
* Fix validation of legacy expression refs inside stage parameters
* Fix aggregation names not being deduplicated
* :as-alias
* replace saigon with bangkok
* use ho chi minh as equivalent replacement
* use bangkok for sqlserver
* update us/pacific to america/los_angeles
* Test debugging
* Fix debugging
* Test fixes 🔧
* Fix busted hardcoded e2e query and add disallowed-keys helper
* cljfmt
* Update deprecation notices 56 => 57
* Kick the can down the road
* Fix another busted test 🔧
* Improve field resolution even more 😒
* cljfmt
* Fix sandboxing with bad field refs.
* Ok I give up I guess I just accidentally fixed sandboxing with custom columns
* cljfmt
* I fixed it
* Appease the linter
* Test fixes 🔧
* OOOPS
* Fix test
* prettier
---------
Co-authored-by: Riley Thompson <rileythompson99@gmail.com>
* UI for metabot feedback
* copy and spacing tweaks (#62049)
* switch to download based approach
* test
---------
Co-authored-by: Maz Ameli <maz@metabase.com>
* BOT-285: ensure field statistics are populated
* chore: test
* extend field-values treatment to dummy tools
* no more need for ensure-field-values!
* chore: use get-or-create
* style
* remove unused require
* fix
* nil safety
* extract
* wip
* wip2
* fix and wip
* wip
* fiiiix
* use the component name inside the iframe
* fix tests
* use custom elements attributes in the preview
* clean up
* clean up
* merge question with new question in the iframe code
* remove SdkIframeEmbedTagSettings
* styles -> S
* restore defaultTheme functionality that was merged on master
* fix test, now the iframe is not reset and we don't need to select the table again
* Fix containerize job to only run when uberjar succeeds
The containerize job now properly checks that the uberjar job succeeded
before attempting to containerize. This prevents containerization from
being triggered on documentation-only changes where the uberjar job
is skipped.
Added: needs.uberjar.result == 'success' condition
* Set default output name for fetched artifacts to "metabase.jar"
- Add :database-feature to SettingDefinition schema to specify required driver features
- Update validate-settable-for-db\! to check database driver feature support
- Add validation to ensure :database-feature is only used with database-local settings
- Fix validation logic to throw when driver does NOT support required feature
- Add comprehensive tests for database feature validation
- Update defsetting macro documentation with :database-feature option
This allows database-local settings like database-enable-actions and
database-enable-table-editing to require specific driver features
(e.g. :actions, :actions/data-editing) before they can be enabled.
* Fix uberjar containerization and implement flexible fetch-artifact action
Resolves issues where uberjars weren't getting containerized when adding the
build-docker-uberjar label to PRs or when manually dispatching workflows.
Major changes:
- Create containerize-uberjar.yml workflow for centralized containerization logic
- Create uberjar-containerize-labeled.yml for PR label-triggered builds
- Extract containerization from uberjar.yml to avoid 3-level workflow nesting
- Add containerization to run-tests.yml for master branch pushes
- Implement flexible extract-path parameter in fetch-artifact action
- Replace preserve-structure boolean with string-based path specification
- Update all workflows to use appropriate extract-path values
The fetch-artifact action now supports:
- extract-path: "." (current directory with custom output-name)
- extract-path: "target/uberjar" (maintains original structure)
- extract-path: "custom/path" (any directory structure)
Containerization now works in all scenarios:
- Master push/merge → run-tests.yml → containerize-uberjar.yml
- PR with build-docker-uberjar label → uberjar-containerize-labeled.yml
- Manual dispatch → containerize-uberjar.yml
🤖 Generated with [Claude Code](https://claude.ai/code)
Co-Authored-By: Claude <noreply@anthropic.com>
* Fix fetch-artifact action to avoid moving file to itself
Prevents "mv: source and destination are the same file" error when
extract-path matches the existing jar location (e.g. target/uberjar).
🤖 Generated with [Claude Code](https://claude.ai/code)
Co-Authored-By: Claude <noreply@anthropic.com>
* Refactor fetch-artifact action to be more DRY
- Consolidate JAR path logic into a single function
- Export JAR_PATH to GITHUB_ENV for use in subsequent steps
- Remove repetitive path determination code
- Reduce lines of code from 113 to 108
🤖 Generated with [Claude Code](https://claude.ai/code)
Co-Authored-By: Claude <noreply@anthropic.com>
* Add Docker Hub link and pull command to PR comment
Enhanced the PR comment after containerization to include:
- Docker tag name (branch with slashes replaced by dashes)
- Direct link to Docker Hub repository tag page
- Ready-to-use docker pull command
This makes it easier for developers to find and pull the built image.
🤖 Generated with [Claude Code](https://claude.ai/code)
Co-Authored-By: Claude <noreply@anthropic.com>
* Add get-docker-info action and update workflows to utilize it for Docker image retrieval
* Refactor post-comment job to streamline Docker info retrieval and PR comment posting in containerize workflow
* Remove unnecessary blank lines in fetch-artifact and get-docker-info actions for cleaner code
* Update containerize job conditions to trigger on pull request labels for Docker builds
* revert using fetch-artifact for release artifacts
* add migration
* add table before update to correctly set deactivated_at
* no need for schema
* archive tables
* tests
* note
* include active to index
* guard against nil changes
* doc
* guard against archival
* note
* prper count
* serdes/test
* extra safety guard
* more accurate logging
* use epocSecond
* more error validation
* change suffix
* add rollback mig
* move migrations to v56
* safer
* test(embed-js): migrate tests to use custom element api
* use CreateApiKeyResponse
* Update e2e/test/scenarios/embedding/sdk-iframe-embedding/sdk-iframe-embedding.cy.spec.ts
Co-authored-by: Phoomparin Mano <poom@metabase.com>
---------
Co-authored-by: Phoomparin Mano <poom@metabase.com>
Cards created before v0.55 lack the `:card_schema` field in their revision history. When loading these old revisions, the system throws "Cannot SELECT a Card without including :card_schema" because the Card model's `after-select` expects this field to always be present.
Existing code adds the legacy default `card_schema` value (20) in `revert-to-revision!`. This PR adds the value (20) to old revisions in `revision.clj`'s `after-select`.
This allows users to view revision history and revert cards that were created before the v0.55 upgrade.
* Updates the info and info_filled icons
* use currentcolor fill instead
* remove fixed width and height and rely on viewBox
* tweak svg paths slightly
* fix y position
---------
Co-authored-by: Phoomparin Mano <phoomparin@gmail.com>
Co-authored-by: Maz Ameli <maz@metabase.com>
* Ensure we're actually using a cached metadata provider in tests
* Remove profiling code
* Restore accidentally-deleted `native-query-snippet` fn
* Fix accidental edits
* Fix metadata calculation for joins that contain joins
* All Lib tests are passing
* The PR that fixes everything [WIP] [ci skip]
* Test fix 🔧
* Sort namespaces
* Docstring formatting
* [Lib tests] 100 failures, 33 errors.
* [Lib] 98 failures, 24 errors.
* [Lib] 76 failures, 17 errors.
* [Lib] 49 failures, 4 errors
* [Lib] 46 failures, 2 errors.
* [Lib] 15 failures, 0 errors.
* [Lib] 13 failures, 0 errors.
* All Lib tests passing
* [QP tests] 2 failures, 1 error.
* Revert :encode/cache-key
* Minor caching performance improvements
* Fix nest breakouts/cumulative aggregations
* Add some more stuff to lib.core
* Test fixes/cljfmt/kondo
* Test fix 🔧
* Test fixes and fix some busted tests
* OMG CACHE SO MUCH FASTER!!!!!
* Fix MongoDB
* Fix tests and support mock segments
* Code cleanup and test fixes 🔧
* Test fix 🔧
* Fix set versus map
* PR feedback
* cljfmt
* More PR feedback and tighten up the field resolution code a bit
* Some schema cleanup
* PR feedback
* PR feedback
* Add TODO
* PR feedback
* Remove implementation of join `visible-columns` which is the same as :default
* Remove unused Card visible columns
* Restore the Card implementation of visible columns in THIS PR.
Missed this from the big remerge, where login attributes were being
reset to an empty value on jwt login. Also fixed not including the
jwt-group-sync on the initial formik state.
* Added to_session endpoint
* Added to_session endpoint
* Added to_session endpoint
* Added to_session endpoint
* Added to_session endpoint
* Some small cleanup
- remove unnecessary try/catch clauses
- rename/rework unnecessary public wrapper fn
- make `jwt` required for the endpoint
- only return session token, not any JWT data
* Test fixes
---------
Co-authored-by: John Swanson <john.swanson@metabase.com>
* wip: add connection string form prefill feature;
add tests
* Refactor deprecated export format
* Skip jdbc:prefix when parsing
* Add snowflake support
* Add success message
* Add a feedback message when pasting a connection string
* Refactor field mapping
* wip use regexes to parse connection strings
* make port optional
* Add snowflake
* Add bigquery and clickhouse
* Add Athena support
* Add Amazon Redshift
* Improve clickhouse
* Add failure message
* Add databricks and druid
* Add MySQL
* Add presto
* Adapt snowflake, create objectToString helper
* Add Apache Spark
* Add SQL server
* Add Oracle
* Add Starburst
* Add vertica
* Reset connection string input on change of key
* Map only when select db and protocol matches
* Fix and refactor semicolon params parsing
* Add tracking to database string uri parsing
* create mapping config with placeholders
use placeholders also in tests
* Create engineKeys types
* Create Cypress paste command using synthetic react events
* Use fade transition for feedback animation
* Add e2e tests for connection strings form filling
* fill in display name from connection string
* Add succes failure text test
* Validate form after filling in fields
* Use more standard fallback for paste
* Remove unused regex parsing implementation
* use mass setFields for fomik setting values
* Add test case for clearing existing values, including engine select field
* Fix form fields being overwritten when pasting connection string
* Add onPaste behavior and use simpler paste mechanism in Cypress
* Fix typo
* Extract and use engineKey type and isEngineKey guard
* Make connection string field controlled by formik
Fix submit values
Fix type error
* Remove unneeded prop justify
* Run regex just for currently selected db
* Fix naming of the test file
* Add e2e db connection test
* Fix missing path in th rspack emedding setup
* Explain why connection string is not sent to BE
* use pojo instead of map for db engines config
* Revert to * export to maintain consistency
* fix connection string test
* create a more meaningful type name
* Add result_metadata perms check to qp
* Check result metadata view data on card put
* Throw the exception
* Test
* Add perm check for ad-hoc referencing cards
* Docstrings
* Use seq
* Order of checks
* Apply suggestions from code review
Co-authored-by: metamben <103100869+metamben@users.noreply.github.com>
* Simplify into
---------
Co-authored-by: metamben <103100869+metamben@users.noreply.github.com>
* add table link to metadata page
* Table
* Add a e2e test
* Add a e2e test
---------
Co-authored-by: Alexander Polyankin <alexander.polyankin@metabase.com>
When streaming data for a query, we were only including columns if a)
the viz settings for the column were completely empty, or if b) the viz
settings for the columns included specifically enabling that column.
The problem occurs most clearly for things like native SQL queries.
If you construct a query like
```
SELECT id FROM venues;
```
and then set the column viz settings to *something* non-empty (most easy
way: just hide the `id` column and then show it again), and then save
the card, then change the query to:
```
SELECT id, name FROM venues;
```
then if you download the CSV for this card (even after saving it!)
you'll get a result that only contains `id` and not `name`.
A few tests needed to change here because they were implicitly assuming
the old behavior, but I don't think that's entirely intentional. This
behavior was different than the behavior in the Metabase application
proper, where UI results would show all the columns (in the above
example, they'd immediately show `id` *and* `name`).
* jwt edits
* Optimized 2 images, saved 188.8 KB (46.0% smaller)
* why are y'all 'waiting for the status to be reported'
---------
Co-authored-by: Metabase Automation <github-automation@metabase.com>
* Create Upsell CTA component
* Replace pro labe with gem in embed upsell
* Create and use UpsellCta in Embedding Upsell
* Rebuiltd the logic for showing links and upsells in embedding modal
* Add UpsellCta tests
* Remove unneeded hook
Fix test and URL rendering for hosted
* Remove unneeded hook useInteractiveEmbeddingCta
* Fix test not running in target build (OSS)
* Fix embedded modal ee end premium cloud tests
* Add missing translation tag
* Use single upsellcta styling
* feat: promote uberjar as the single uberjar creation workflow
* chore: remove useless reuse-existing-artifacts
* refactor: streamline PR environment workflow by removing wait_for_uberjar input and simplifying uberjar retrieval
* fix: ensure default editions are used if container info retrieval fails
* chore: don't build uberjar outside of run-tests
* fix: update uberjar artifact name to use the current SHA instead of PR head SHA
* fix: update uberjar artifact names to use PR head SHA when available
* fix: update uberjar workflow to simplify input parameters and improve conditional checks
* fix: add concurrency settings to uberjar workflow
* fix: update documentation condition in run-tests workflow
* fix: add default node version
* fix: update conditions for container info retrieval and containerization steps
* fix: update default node version to 22 in uberjar workflow
* Update .github/workflows/uberjar.yml
* Update .github/workflows/uberjar.yml
---------
Co-authored-by: Uladzimir Havenchyk <125459446+uladzimirdev@users.noreply.github.com>
was blocking on startup
stacktrace below is representative. has a few other things like reveal
but shows that we get into a huge loop over all cards and you end up
waiting for the instance to be ready for a long time
```
at metabase.lib.card$fn__77290$fn__77297.invoke(card.cljc:71)
at metabase.lib.card$fn__77302$__GT_card_metadata_columns77301__77303$fn__77304.invoke(card.cljc:141)
at clojure.core$mapv$fn__8569.invoke(core.clj:7059)
at clojure.lang.PersistentVector.reduce(PersistentVector.java:418)
at clojure.core$reduce.invokeStatic(core.clj:6964)
at clojure.core$mapv.invokeStatic(core.clj:7050)
at clojure.core$mapv.invoke(core.clj:7050)
at metabase.lib.card$fn__77302$__GT_card_metadata_columns77301__77303.invoke(card.cljc:141)
at metabase.lib.card$fn__77302$fn__77308.invoke(card.cljc:120)
at metabase.lib.card$fn__77316$card_cols_STAR_77315__77317.invoke(card.cljc:163)
at metabase.lib.card$fn__77316$fn__77322.invoke(card.cljc:157)
at metabase.lib.card$fn__77350$card_metadata_columns77349__77351.invoke(card.cljc:223)
at metabase.lib.card$fn__77350$fn__77354.invoke(card.cljc:217)
at metabase.lib.card$eval77367$_AMPERSAND_f__77368.invoke(card.cljc:257)
at metabase.lib.card$eval77367$fn__77372.invoke(card.cljc:242)
at clojure.lang.MultiFn.invoke(MultiFn.java:244)
at metabase.lib.metadata.calculation$fn__74776$returned_columns74775__74777$fn__74778.invoke(calculation.cljc:524)
at metabase.lib.metadata.cache$fn__74548$do_with_cached_value74547__74549.invoke(cache.cljc:104)
at metabase.lib.metadata.cache$fn__74548$fn__74555.invoke(cache.cljc:92)
at metabase.lib.metadata.calculation$fn__74776$returned_columns74775__74777.invoke(calculation.cljc:523)
at metabase.lib.metadata.calculation$fn__74776$fn__74781.invoke(calculation.cljc:503)
at metabase.lib.metadata.calculation$fn__74776$returned_columns74775__74777.invoke(calculation.cljc:516)
at metabase.lib.metadata.calculation$fn__74776$fn__74781.invoke(calculation.cljc:503)
at metabase.lib.field.resolution$fn__80775$resolve_in_card_returned_columns80774__80776.invoke(resolution.cljc:352)
at metabase.lib.field.resolution$fn__80775$fn__80784.invoke(resolution.cljc:346)
at metabase.lib.field.resolution$fn__80788$resolve_in_card_or_stage_metadata80787__80789.invoke(resolution.cljc:365)
at metabase.lib.field.resolution$fn__80788$fn__80801.invoke(resolution.cljc:358)
at metabase.lib.field.resolution$resolve_from_previous_stage_or_source_STAR_.invokeStatic(resolution.cljc:410)
at metabase.lib.field.resolution$resolve_from_previous_stage_or_source_STAR_.invoke(resolution.cljc:390)
at metabase.lib.field.resolution$fn__80844$resolve_from_previous_stage_or_source80843__80845.invoke(resolution.cljc:440)
at metabase.lib.field.resolution$fn__80844$fn__80861.invoke(resolution.cljc:434)
at metabase.lib.field.resolution$fn__80865$resolve_field_ref80864__80867.invoke(resolution.cljc:471)
at metabase.lib.field.resolution$fn__80865$fn__80878.invoke(resolution.cljc:457)
at metabase.lib.field$eval80922$_AMPERSAND_f__80923.invoke(field.cljc:62)
at metabase.lib.field$eval80922$fn__80925.invoke(field.cljc:60)
at clojure.lang.MultiFn.invoke(MultiFn.java:239)
at metabase.lib.metadata.calculation$fn__74679$metadata74678__74680$fn__74681.invoke(calculation.cljc:287)
at metabase.lib.metadata.cache$fn__74548$do_with_cached_value74547__74549.invoke(cache.cljc:104)
at metabase.lib.metadata.cache$fn__74548$fn__74555.invoke(cache.cljc:92)
at metabase.lib.metadata.calculation$fn__74679$metadata74678__74680.invoke(calculation.cljc:286)
at metabase.lib.metadata.calculation$fn__74679$fn__74684.invoke(calculation.cljc:275)
at metabase.lib.breakout$fn__80328$breakouts_metadata80327__80329$fn__80331.invoke(breakout.cljc:43)
at clojure.core$mapv$fn__8569.invoke(core.clj:7059)
at clojure.lang.PersistentVector.reduce(PersistentVector.java:418)
at clojure.core$reduce.invokeStatic(core.clj:6964)
at clojure.core$mapv.invokeStatic(core.clj:7050)
at clojure.core$mapv.invoke(core.clj:7050)
at metabase.lib.breakout$fn__80328$breakouts_metadata80327__80329.invoke(breakout.cljc:42)
at metabase.lib.breakout$fn__80328$fn__80334.invoke(breakout.cljc:35)
at metabase.lib.stage$fn__81784$breakouts_columns81783__81785.invoke(stage.cljc:59)
at metabase.lib.stage$fn__81784$fn__81787.invoke(stage.cljc:55)
at metabase.lib.stage$fn__81843$summary_columns81842__81844.invoke(stage.cljc:92)
at metabase.lib.stage$fn__81843$fn__81846.invoke(stage.cljc:86)
at metabase.lib.stage$eval81986$_AMPERSAND_f__81988.invoke(stage.cljc:290)
at metabase.lib.stage$eval81986$fn__81996.invoke(stage.cljc:283)
at clojure.lang.MultiFn.invoke(MultiFn.java:244)
at metabase.lib.metadata.calculation$fn__74776$returned_columns74775__74777$fn__74778.invoke(calculation.cljc:524)
at metabase.lib.metadata.cache$fn__74548$do_with_cached_value74547__74549.invoke(cache.cljc:104)
at metabase.lib.metadata.cache$fn__74548$fn__74555.invoke(cache.cljc:92)
at metabase.lib.metadata.calculation$fn__74776$returned_columns74775__74777.invoke(calculation.cljc:523)
at metabase.lib.metadata.calculation$fn__74776$fn__74781.invoke(calculation.cljc:503)
at metabase.lib.metadata.calculation$fn__74776$returned_columns74775__74777.invoke(calculation.cljc:516)
at metabase.lib.metadata.calculation$fn__74776$fn__74781.invoke(calculation.cljc:503)
at metabase.lib.metadata.calculation$fn__74776$returned_columns74775__74777.invoke(calculation.cljc:513)
at metabase.lib.metadata.calculation$fn__74776$fn__74781.invoke(calculation.cljc:503)
at metabase.lib.metadata.calculation$fn__74776$returned_columns74775__74777.invoke(calculation.cljc:510)
at metabase.lib.metadata.calculation$fn__74776$fn__74781.invoke(calculation.cljc:503)
at metabase.queries.models.card$dataset_query__GT_dimensions.invokeStatic(card.clj:1270)
at metabase.queries.models.card$dataset_query__GT_dimensions.invoke(card.clj:1262)
at metabase.queries.models.card$extract_non_temporal_dimension_ids.invokeStatic(card.clj:1277)
at metabase.queries.models.card$extract_non_temporal_dimension_ids.invoke(card.clj:1274)
at metabase.search.ingestion$execute_function_attr.invokeStatic(ingestion.clj:68)
at metabase.search.ingestion$execute_function_attr.invoke(ingestion.clj:63)
at metabase.search.ingestion$execute_all_function_attrs$fn__131886.invoke(ingestion.clj:80)
at clojure.lang.PersistentHashMap$NodeSeq.kvreduce(PersistentHashMap.java:1309)
at clojure.lang.PersistentHashMap$BitmapIndexedNode.kvreduce(PersistentHashMap.java:804)
at clojure.lang.PersistentHashMap$NodeSeq.kvreduce(PersistentHashMap.java:1314)
at clojure.lang.PersistentHashMap$BitmapIndexedNode.kvreduce(PersistentHashMap.java:804)
at clojure.lang.PersistentHashMap.kvreduce(PersistentHashMap.java:238)
at clojure.core$fn__8559.invokeStatic(core.clj:6987)
at clojure.core$fn__8559.invoke(core.clj:6967)
at clojure.core.protocols$fn__8287$G__8282__8296.invoke(protocols.clj:174)
at clojure.core$reduce_kv.invokeStatic(core.clj:6998)
at clojure.core$reduce_kv.invoke(core.clj:6989)
at metabase.search.ingestion$execute_all_function_attrs.invokeStatic(ingestion.clj:76)
at metabase.search.ingestion$execute_all_function_attrs.invoke(ingestion.clj:73)
at metabase.search.ingestion$__GT_document.invokeStatic(ingestion.clj:87)
at metabase.search.ingestion$__GT_document.invoke(ingestion.clj:85)
at clojure.core$map$fn__5952$fn__5953.invoke(core.clj:2759)
at medley.core$distinct_by$fn__15232$fn__15233.invoke(core.cljc:290)
at clojure.core$map$fn__5952$fn__5953.invoke(core.clj:2759)
at clojure.core$completing$fn__8562.invoke(core.clj:7010)
at clojure.core$map$fn__5952$fn__5953.invoke(core.clj:2759)
at toucan2.pipeline$with_init$fn__51829.invoke(pipeline.clj:327)
at clojure.core$completing$fn__8562.invoke(core.clj:7010)
at toucan2.jdbc.result_set$reduce_result_set.invokeStatic(result_set.clj:156)
at toucan2.jdbc.result_set$reduce_result_set.invoke(result_set.clj:125)
at toucan2.jdbc.query$reduce_jdbc_query.invokeStatic(query.clj:51)
at toucan2.jdbc.query$reduce_jdbc_query.invoke(query.clj:22)
at toucan2.jdbc.pipeline$transduce_execute_with_connection_primary_method_java_sql_Connection_default_default.invokeStatic(pipeline.clj:19)
at toucan2.jdbc.pipeline$transduce_execute_with_connection_primary_method_java_sql_Connection_default_default.invoke(pipeline.clj:9)
at methodical.impl.combo.common$partial_STAR_$fn__22829.invoke(common.clj:15)
at methodical.impl.combo.threaded$combine_methods_thread_last$fn__23165$combined_method_thread_last__23166.invoke(threaded.clj:64)
at methodical.util.FnWithMeta.invoke(util.clj:46)
at methodical.impl.standard$invoke_multifn.invokeStatic(standard.clj:65)
at methodical.impl.standard$invoke_multifn.invoke(standard.clj:47)
at methodical.impl.standard.StandardMultiFn.invoke(standard.clj:216)
at toucan2.pipeline$transduce_execute$with_connection_STAR___51727.invoke(pipeline.clj:78)
at toucan2.connection$bind_current_connectable_fn$fn__51305.invoke(connection.clj:104)
at toucan2.connection$bind_current_connectable_fn$fn__51305.invoke(connection.clj:104)
at toucan2.jdbc.connection$do_with_connection_primary_method_java_sql_Connection.invokeStatic(connection.clj:13)
at toucan2.jdbc.connection$do_with_connection_primary_method_java_sql_Connection.invoke(connection.clj:11)
at methodical.impl.combo.common$partial_STAR_$fn__22829.invoke(common.clj:12)
at methodical.util.FnWithMeta.invoke(util.clj:46)
at toucan2.connection$do_with_connection_around_method_toucan2_connection_default.invokeStatic(connection.clj:118)
at toucan2.connection$do_with_connection_around_method_toucan2_connection_default.invoke(connection.clj:106)
at methodical.impl.combo.common$partial_STAR_$fn__22829.invoke(common.clj:12)
at methodical.util.FnWithMeta.invoke(util.clj:46)
at methodical.impl.standard$invoke_multifn.invokeStatic(standard.clj:55)
at methodical.impl.standard$invoke_multifn.invoke(standard.clj:47)
at methodical.impl.standard.StandardMultiFn.invoke(standard.clj:210)
at toucan2.connection$do_with_connection_primary_method_.invokeStatic(connection.clj:204)
at toucan2.connection$do_with_connection_primary_method_.invoke(connection.clj:194)
at methodical.impl.combo.common$partial_STAR_$fn__22829.invoke(common.clj:12)
at methodical.util.FnWithMeta.invoke(util.clj:46)
at toucan2.connection$do_with_connection_around_method_toucan2_connection_default.invokeStatic(connection.clj:118)
at toucan2.connection$do_with_connection_around_method_toucan2_connection_default.invoke(connection.clj:106)
at methodical.impl.combo.common$partial_STAR_$fn__22829.invoke(common.clj:12)
at methodical.util.FnWithMeta.invoke(util.clj:46)
at methodical.impl.standard$invoke_multifn.invokeStatic(standard.clj:55)
at methodical.impl.standard$invoke_multifn.invoke(standard.clj:47)
at methodical.impl.standard.StandardMultiFn.invoke(standard.clj:210)
at toucan2.pipeline$transduce_execute.invokeStatic(pipeline.clj:77)
at toucan2.pipeline$transduce_execute.invoke(pipeline.clj:64)
at clojure.lang.Var.invoke(Var.java:401)
at toucan2.pipeline$transduce_compiled_query.invokeStatic(pipeline.clj:244)
at toucan2.pipeline$transduce_compiled_query.invoke(pipeline.clj:240)
at toucan2.pipeline$transduce_built_query.invokeStatic(pipeline.clj:252)
at toucan2.pipeline$transduce_built_query.invoke(pipeline.clj:246)
at toucan2.pipeline$transduce_query_primary_method_default.invokeStatic(pipeline.clj:272)
at toucan2.pipeline$transduce_query_primary_method_default.invoke(pipeline.clj:269)
at methodical.impl.combo.common$partial_STAR_$fn__22829.invoke(common.clj:15)
at methodical.impl.combo.threaded$combine_methods_thread_last$fn__23165$combined_method_thread_last__23166.invoke(threaded.clj:64)
at methodical.util.FnWithMeta.invoke(util.clj:46)
at methodical.impl.standard$invoke_multifn.invokeStatic(standard.clj:65)
at methodical.impl.standard$invoke_multifn.invoke(standard.clj:47)
at methodical.impl.standard.StandardMultiFn.invoke(standard.clj:216)
at toucan2.pipeline$transduce_query_STAR_.invokeStatic(pipeline.clj:278)
at toucan2.pipeline$transduce_query_STAR_.invoke(pipeline.clj:274)
at toucan2.pipeline$transduce_with_model.invokeStatic(pipeline.clj:293)
at toucan2.pipeline$transduce_with_model.invoke(pipeline.clj:280)
at toucan2.pipeline$transduce_parsed.invokeStatic(pipeline.clj:309)
at toucan2.pipeline$transduce_parsed.invoke(pipeline.clj:295)
at clojure.lang.AFn.applyToHelper(AFn.java:160)
at clojure.lang.AFn.applyTo(AFn.java:144)
at clojure.core$apply.invokeStatic(core.clj:669)
at clojure.core$apply.invoke(core.clj:662)
at toucan2.pipeline$reducible_fn$reify__51862.reduce(pipeline.clj:404)
at clojure.core$transduce.invokeStatic(core.clj:7025)
at clojure.core.Eduction.reduce(core.clj:7874)
at clojure.core$reduce.invokeStatic(core.clj:6964)
at clojure.core$reduce.invoke(core.clj:6947)
at metabase.util$rconcat$reify__16182.coll_reduce(util.cljc:1188)
at clojure.core$transduce.invokeStatic(core.clj:7026)
at clojure.core.Eduction.reduce(core.clj:7874)
at clojure.core$transduce.invokeStatic(core.clj:7025)
at clojure.core$transduce.invokeStatic(core.clj:7021)
at clojure.core$transduce.invoke(core.clj:7012)
at metabase.search.appdb.index$index_docs_BANG_.invokeStatic(index.clj:279)
at metabase.search.appdb.index$index_docs_BANG_.invoke(index.clj:275)
at metabase.search.appdb.core$populate_index_BANG_.invokeStatic(core.clj:169)
at metabase.search.appdb.core$populate_index_BANG_.invoke(core.clj:168)
at metabase.search.appdb.core$eval132669$fn__132671.invoke(core.clj:182)
at clojure.lang.MultiFn.invoke(MultiFn.java:234)
at metabase.search.core$init_index_BANG_$iter__133794__133798$fn__133799.invoke(core.clj:91)
at clojure.lang.LazySeq.force(LazySeq.java:50)
at clojure.lang.LazySeq.realize(LazySeq.java:89)
at clojure.lang.LazySeq.seq(LazySeq.java:106)
at clojure.lang.RT.seq(RT.java:555)
at clojure.core$seq__5488.invokeStatic(core.clj:139)
at clojure.core.protocols$seq_reduce.invokeStatic(protocols.clj:24)
at clojure.core.protocols$fn__8266.invokeStatic(protocols.clj:74)
at clojure.core.protocols$fn__8266.invoke(protocols.clj:74)
at clojure.core.protocols$fn__8207$G__8202__8220.invoke(protocols.clj:13)
at clojure.core$reduce.invokeStatic(core.clj:6965)
at clojure.core$reduce.invoke(core.clj:6947)
at metabase.search.core$init_index_BANG_.invokeStatic(core.clj:88)
at metabase.search.core$init_index_BANG_.doInvoke(core.clj:80)
at clojure.lang.RestFn.invoke(RestFn.java:411)
at metabase.search.task.search_index$init_BANG_$fn__173098.invoke(search_index.clj:38)
at metabase.app_db.cluster_lock$do_with_cluster_lock_STAR_$with_connection_STAR___126394$with_transaction_STAR___126395.invoke(cluster_lock.clj:61)
at toucan2.connection$bind_current_connectable_fn$fn__51305.invoke(connection.clj:104)
at metabase.app_db.connection$do_transaction$thunk__61025.invoke(connection.clj:145)
at metabase.app_db.connection$do_transaction.invokeStatic(connection.clj:164)
at metabase.app_db.connection$do_transaction.invoke(connection.clj:141)
at metabase.app_db.connection$do_with_transaction_primary_method_java_sql_Connection.invokeStatic(connection.clj:201)
at metabase.app_db.connection$do_with_transaction_primary_method_java_sql_Connection.invoke(connection.clj:174)
at methodical.impl.combo.common$partial_STAR_$fn__22829.invoke(common.clj:13)
at methodical.util.FnWithMeta.invoke(util.clj:46)
at toucan2.connection$do_with_transaction_around_method_toucan2_connection_default.invokeStatic(connection.clj:249)
at toucan2.connection$do_with_transaction_around_method_toucan2_connection_default.invoke(connection.clj:245)
at methodical.impl.combo.common$partial_STAR_$fn__22829.invoke(common.clj:13)
at methodical.util.FnWithMeta.invoke(util.clj:46)
at methodical.impl.standard$invoke_multifn.invokeStatic(standard.clj:58)
at methodical.impl.standard$invoke_multifn.invoke(standard.clj:47)
at methodical.impl.standard.StandardMultiFn.invoke(standard.clj:212)
at metabase.app_db.cluster_lock$do_with_cluster_lock_STAR_$with_connection_STAR___126394.invoke(cluster_lock.clj:53)
at toucan2.connection$bind_current_connectable_fn$fn__51305.invoke(connection.clj:104)
at toucan2.connection$bind_current_connectable_fn$fn__51305.invoke(connection.clj:104)
at toucan2.connection$bind_current_connectable_fn$fn__51305.invoke(connection.clj:104)
at toucan2.jdbc.connection$do_with_connection_primary_method_javax_sql_DataSource.invokeStatic(connection.clj:18)
at toucan2.jdbc.connection$do_with_connection_primary_method_javax_sql_DataSource.invoke(connection.clj:15)
at methodical.impl.combo.common$partial_STAR_$fn__22829.invoke(common.clj:12)
at methodical.util.FnWithMeta.invoke(util.clj:46)
at toucan2.connection$do_with_connection_around_method_toucan2_connection_default.invokeStatic(connection.clj:118)
at toucan2.connection$do_with_connection_around_method_toucan2_connection_default.invoke(connection.clj:106)
at methodical.impl.combo.common$partial_STAR_$fn__22829.invoke(common.clj:12)
at methodical.util.FnWithMeta.invoke(util.clj:46)
at methodical.impl.standard$invoke_multifn.invokeStatic(standard.clj:55)
at methodical.impl.standard$invoke_multifn.invoke(standard.clj:47)
at methodical.impl.standard.StandardMultiFn.invoke(standard.clj:210)
at metabase.app_db.connection$do_with_connection_primary_method_default.invokeStatic(connection.clj:132)
at metabase.app_db.connection$do_with_connection_primary_method_default.invoke(connection.clj:130)
at methodical.impl.combo.common$partial_STAR_$fn__22829.invoke(common.clj:12)
at methodical.util.FnWithMeta.invoke(util.clj:46)
at toucan2.connection$do_with_connection_around_method_toucan2_connection_default.invokeStatic(connection.clj:118)
at toucan2.connection$do_with_connection_around_method_toucan2_connection_default.invoke(connection.clj:106)
at methodical.impl.combo.common$partial_STAR_$fn__22829.invoke(common.clj:12)
at methodical.util.FnWithMeta.invoke(util.clj:46)
at methodical.impl.standard$invoke_multifn.invokeStatic(standard.clj:55)
at methodical.impl.standard$invoke_multifn.invoke(standard.clj:47)
at methodical.impl.standard.StandardMultiFn.invoke(standard.clj:210)
at toucan2.connection$do_with_connection_primary_method_.invokeStatic(connection.clj:204)
at toucan2.connection$do_with_connection_primary_method_.invoke(connection.clj:194)
at methodical.impl.combo.common$partial_STAR_$fn__22829.invoke(common.clj:12)
at methodical.util.FnWithMeta.invoke(util.clj:46)
at toucan2.connection$do_with_connection_around_method_toucan2_connection_default.invokeStatic(connection.clj:118)
at toucan2.connection$do_with_connection_around_method_toucan2_connection_default.invoke(connection.clj:106)
at methodical.impl.combo.common$partial_STAR_$fn__22829.invoke(common.clj:12)
at methodical.util.FnWithMeta.invoke(util.clj:46)
at methodical.impl.standard$invoke_multifn.invokeStatic(standard.clj:55)
at methodical.impl.standard$invoke_multifn.invoke(standard.clj:47)
at methodical.impl.standard.StandardMultiFn.invoke(standard.clj:210)
at metabase.app_db.cluster_lock$do_with_cluster_lock_STAR_.invokeStatic(cluster_lock.clj:53)
at metabase.app_db.cluster_lock$do_with_cluster_lock_STAR_.invoke(cluster_lock.clj:51)
at metabase.app_db.cluster_lock$fn__126406$do_with_cluster_lock126405__126407$do_with_cluster_lock_STAR__STAR___126409.invoke(cluster_lock.clj:78)
at clojure.lang.AFn.applyToHelper(AFn.java:152)
at clojure.lang.AFn.applyTo(AFn.java:144)
at clojure.core$apply.invokeStatic(core.clj:667)
at clojure.core$apply.invoke(core.clj:662)
at metabase.util.retry$decorate$fn$reify__126370.call(retry.clj:110)
at io.github.resilience4j.retry.Retry.lambda$decorateCallable$5(Retry.java:306)
at metabase.util.retry$decorate$fn__126369.doInvoke(retry.clj:111)
at clojure.lang.RestFn.invoke(RestFn.java:400)
at metabase.util.retry$fn__126375$make126374__126376$fn__126377.invoke(retry.clj:120)
at metabase.app_db.cluster_lock$fn__126406$do_with_cluster_lock126405__126407.invoke(cluster_lock.clj:82)
at metabase.app_db.cluster_lock$fn__126406$fn__126412.invoke(cluster_lock.clj:63)
at metabase.app_db.cluster_lock$fn__126406$do_with_cluster_lock126405__126407.invoke(cluster_lock.clj:75)
at metabase.app_db.cluster_lock$fn__126406$fn__126412.invoke(cluster_lock.clj:63)
at metabase.search.task.search_index$init_BANG_.invokeStatic(search_index.clj:37)
at metabase.search.task.search_index$init_BANG_.invoke(search_index.clj:33)
at metabase.search.task.search_index$eval173156$fn__173157.invoke(search_index.clj:46)
at metabase.startup.core$run_startup_logic_BANG_$fn__173079.invoke(core.clj:30)
at metabase.startup.core$run_startup_logic_BANG_.invokeStatic(core.clj:28)
at metabase.startup.core$run_startup_logic_BANG_.invoke(core.clj:24)
at metabase.core.core$init_BANG__STAR_.invokeStatic(core.clj:181)
at metabase.core.core$init_BANG__STAR_.invoke(core.clj:117)
at metabase.core.core$init_BANG_.invokeStatic(core.clj:194)
at metabase.core.core$init_BANG_.invoke(core.clj:189)
at dev$init_BANG_.invokeStatic(dev.clj:143)
at dev$init_BANG_.invoke(dev.clj:139)
at dev$start_BANG_.invokeStatic(dev.clj:180)
at dev$start_BANG_.invoke(dev.clj:176)
at dev$eval206099.invokeStatic(NO_SOURCE_FILE:1)
at dev$eval206099.invoke(NO_SOURCE_FILE:1)
at clojure.lang.Compiler.eval(Compiler.java:7739)
at clojure.lang.Compiler.eval(Compiler.java:7729)
at clojure.lang.Compiler.eval(Compiler.java:7694)
at clojure.core$eval.invokeStatic(core.clj:3232)
at clojure.core$eval.invoke(core.clj:3228)
at vlaaad.reveal.repl$wrap_eval$fn__37945.invoke(repl.clj:90)
at clojure.main$repl$read_eval_print__9248$fn__9251.invoke(main.clj:437)
at clojure.main$repl$read_eval_print__9248.invoke(main.clj:437)
at clojure.main$repl$fn__9257.invoke(main.clj:459)
at clojure.main$repl.invokeStatic(main.clj:459)
at clojure.main$repl.doInvoke(main.clj:368)
at clojure.lang.RestFn.applyTo(RestFn.java:140)
at clojure.core$apply.invokeStatic(core.clj:667)
at clojure.core$apply.invoke(core.clj:662)
```
Previously we were treating all currency formatting as if we were in a
table, and only adding the currency symbol if `currency-in-header` were
explicitly set to `false` for the column.
When formatting a scalar value, though, we want to always add the
currency symbol inline (since there is no header in this case).
I found that `metabase.channel.render.body/format-cell` and
`metabase.formatter.core/format-number` were actually only used to
format *scalar* cells and *scalar* numbers (as opposed to cells in a
table), so I renamed them for a bit more clarity and added a new
optional argument, `scalar?`, to
`metabase.formatter.core/number-formatter`. When passed (and truthy) the
currency symbol will be splatted into the formatted number even if
`:currency-in-header` is not `false`.
* minor edits
* Update docs/permissions/row-and-column-security.md
Co-authored-by: Alex Yarosh <alexandra@metabase.com>
---------
Co-authored-by: Alex Yarosh <alexandra@metabase.com>
instead just pass `(not-empty q)` to the search subsystem, which will
convert it to `nil` if it's empty and pass it unchanged if it's not.
I still don't think this is probably the ideal user experience when you
navigate to `/search` but it's not an error at least.
* UXW-382 Updates "Public link" menu item for admins when public links aren't enabled so it doesn't look obviously broken
* UXW-382 Updates tests
* UXW-382 Updates "enable public links" menu item to be disabled per PR feedback
* UXW-382 Swaps core styles for mantine props per PR feedback
* fix: issues generating valid urls for map tiles
Fixes to issues generating map tiles, one is on x-ray dashboards we need
to check that we're not in a real dashboard and send an ad hoc tile
request.
Second if the field ref sent in the request has options we need to
remove the 'base-type' option because the type will be encoded as
something like test%2FFloat and the way Jetty decodes URLs before
parsing turns it into a '/' in a position a path separator cannot be:
https://github.com/jetty/jetty.project/issues/7713
* fixup! typeguard dashbaord id
* Update maps.cy.spec.js
* fixup! better handle null field opts
* Re-implementation of `metabase.driver.common.parameters.*` that uses Lib
* Fix Kondo warnings
* Move tests that have nothing to do with drivers to QP
* Test fixes 🔧
* Test fixes 🔧
* Try simplifying the PR
* db routing for redshift
* simplify test
* dataset name multimethod
* small change
* use delayed defs
* add redshift ci routing env var
* small fix
* update dataset definitions
* change back to original data
* feat: driver/do-with-resilient-connection
* feat: :keep-open? option to do-with-connection-with-options
* feat: attempt describe-database with resilient connection
* feat: execute jdbc query with retry conn
* test
* avoid recursive issues
* simplify
* changelog
* type hint
* force local conn, so the connection being closed won't close outer unrealized result sets
* working test
* more test
* opts
* style
* add test for connection options
* notes
* fix test
* docstring
* Update src/metabase/driver/sql_jdbc/execute.clj
Co-authored-by: lbrdnk <lbrdnk@users.noreply.github.com>
* logs
* better logs
* try to close conn when it's no longer valid
* pull out logic to is-conn-open
* is-conn-open? test
* clarify
* Update docs/developers-guide/driver-changelog.md
Co-authored-by: lbrdnk <lbrdnk@users.noreply.github.com>
* update version
* chore: move tests to right namespace
* retry have select privilege on error
* add test
* improve test
* remove let
* separate connection for each test
* clean up test
* try druid jdbc ci fix
* close connection during test
* use do with resilient connection in test for druid
* formatting
* always set role if supported
* Fix clickhouse and sqlserver
* impersonation test
* use normal-driver-select
* address comments
* kondo
* remove debug
---------
Co-authored-by: Nicola Mometto <nicola.mometto@metabase.com>
Co-authored-by: lbrdnk <lbrdnk@users.noreply.github.com>
* UXW-28 Adds confirmation modal when deleting a tab that contains dashboard-only cards
* UXW-28 Makes confirmation modal smarter; uses short version if all questions on a tab are dashboard questions, lists affected questions if it's a mix
* UXW-28 Adds tests
* feat: driver/do-with-resilient-connection
* feat: :keep-open? option to do-with-connection-with-options
* feat: attempt describe-database with resilient connection
* feat: execute jdbc query with retry conn
* test
* avoid recursive issues
* simplify
* changelog
* type hint
* force local conn, so the connection being closed won't close outer unrealized result sets
* working test
* more test
* opts
* style
* add test for connection options
* notes
* fix test
* docstring
* Update src/metabase/driver/sql_jdbc/execute.clj
Co-authored-by: lbrdnk <lbrdnk@users.noreply.github.com>
* logs
* better logs
* try to close conn when it's no longer valid
* pull out logic to is-conn-open
* is-conn-open? test
* clarify
* Update docs/developers-guide/driver-changelog.md
Co-authored-by: lbrdnk <lbrdnk@users.noreply.github.com>
* update version
* chore: move tests to right namespace
* impersonation test
* use normal-driver-select
* trigger CI
---------
Co-authored-by: lbrdnk <lbrdnk@users.noreply.github.com>
* feat(simple-embed): parse attributes with json5 to be more permissive with quotes
* Update package.json
Co-authored-by: cubic-dev-ai[bot] <191113872+cubic-dev-ai[bot]@users.noreply.github.com>
* fix tests
* Apply suggestion from @heypoom
Co-authored-by: Phoomparin Mano <poom@metabase.com>
---------
Co-authored-by: cubic-dev-ai[bot] <191113872+cubic-dev-ai[bot]@users.noreply.github.com>
Co-authored-by: Phoomparin Mano <poom@metabase.com>
* Remove debugging stuff
* 14 failures, 1 error.
* 10 failures, 1 error.
* 10 failures, 0 errors.
* Better log messages
* Faster field ref resolution
* Test fixes 🔧
* Some test fixes 🔧
* Remove resolve-joined-fields middleware
* Fix lib.underlying calculation of fields with join aliases
* Fix busted pivot tests
* Remove unused fn
* More resolution fixes
* drill-thru use vars so we can get updated fns in the REPL
* Consolidate the lib metadata => legacy metadata functions
* Add a pile of additional tests
* Migrate the `resolve-joined-fields` middleware (#39884)
* Support automatically removing duplicate refs in `:fields` and `:breakouts` during normalization
* More cleanup and extra tests 🔧
* Add ::mbql.s/OrderBys
* Mess with logging in resolution even more
* Order bys should be distinct ignoring keys like :base-type or namespaced keys
* I have fixed#60587
* One last round of test fixes??
* Test fix
* Code cleanup
* Fix test failures 🔧
* MongoDB test fix
* Fix Oracle test 🔧
* MongoDB test fix 🔧
* cljfmt
* e2e test fix?
* Test fix 🔧
* Less noisy logging
* Improve field resolution for implicit joins; PR feedback (part 1)
* PR feedback (part 2) and extra dox
* Run tests if files in lib/drivers/qp are changes
check src/metabase/lib, src/metabase/query_processor, modules/drivers
we key off of team label but this is hopefully close enough to what we
really want
* run an action linter
```
❯ actionlint /Users/dan/projects/work/metabase/.github/workflows/drivers.yml
.github/workflows/drivers.yml:440:71: property "exclude-tag" is not defined in object type {name: string; test-args: string} [expression]
.github/workflows/drivers.yml:537:15: description is required in metadata of "Wait for localhost port to be ready" action at "/Users/dan/projects/work/metabase/.github/actions/await-port/action.yml" [action]
|
537 | - uses: ./.github/actions/await-port
| ^~~~~~~~~~~~~~~~~~~~~~~~~~~~
.github/workflows/drivers.yml:587:13: description is required in metadata of "Test database driver" action at "/Users/dan/projects/work/metabase/.github/actions/test-driver/action.yml" [action]
|
587 | uses: ./.github/actions/test-driver
| ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~
.github/workflows/drivers.yml:743:71: property "exclude-tag" is not defined in object type {name: string; test-args: string} [expression]
```
acitonlint seems pretty nice
not fixing the exclude-tag since that seems to work just fine
* fix syntax
* fetch base branch for file checks
to see if we changed files, need the base
* indentation is fun. super fast iteration
* resequence if before run
* run third tier on devex team label
* empty to poke CI
* make prepare-statement private
probably good for this to be private either way, but this is done now to
kick CI and run tests that aren't running in CI
* close single quote
We're using a setting as a half-baked way to track whether we've ever
created a non-table chart. We do this regardless of whether you're an
admin or not, but the endpoint responds with a 403 if you're not an
admin.
Honestly motivation here was fixing [UXW-152: 403 thrown for non-admins
when viewing question re:
non-table-chart-generated](https://linear.app/metabase/issue/UXW-152/403-thrown-for-non-admins-when-viewing-question-re-non-table-chart)
but it feels a little silly. But... it's not *completely* unreasonable.
* dev: add the ability to run native ddl as part of dataset definitions
* Fixing tests
* Fix kondo
* Fix tests and kondo
* Fix clickhouse tests
* Fix another clickhouse test
* Fix bigquery test
* Fix bigquery test
* Trigger Build
* fix: filter-out non-string attributes
We expected login_attributes to only be strings. When they are created
through SAML or JWT logins we don't actually enforce this, so this
filters out non-string attributes printing a warning message if they are
passed for these login types.
* feat: auto assign jwt users to groups by name
When group sync is turned on and there are no group mappings defined for
the jwt integration.
* fixup! remove tenant user group
* fixup! correct bad merge
* fixup! remove some tenant_ids in tests
* fixup! more test fixes
* feat: make jwt group attribute admin configurable
* feat: update group mapping text for jwt behavior
* feat(new-embed): log an error when the parsing of a json attribute fails
* Update enterprise/frontend/src/metabase-enterprise/embedding_iframe_sdk/webcomponents.ts
Co-authored-by: Phoomparin Mano <poom@metabase.com>
---------
Co-authored-by: Phoomparin Mano <poom@metabase.com>
* fix: allow scalar user-attributes
Only drop collections which cannot be converted to strings when
generating database queries.
* fixup! remove unused namespace
* fixup! use keep to retain false booleans
* fix: filter-out non-string attributes
We expected login_attributes to only be strings. When they are created
through SAML or JWT logins we don't actually enforce this, so this
filters out non-string attributes printing a warning message if they are
passed for these login types.
* feat: auto assign jwt users to groups by name
When group sync is turned on and there are no group mappings defined for
the jwt integration.
* fixup! remove tenant user group
* fixup! correct bad merge
* fixup! remove some tenant_ids in tests
* fixup! more test fixes
* feat: make jwt group attribute admin configurable
The workflow in the `metabase-ee-extra` repository could run before the
container image tags were created by the workflow in the `metabase`
repository, causing the former to fail. Fix by making the image push
an explicit dependency of the ee-extra trigger.
This works by just totally removing the permission grants created by
sandboxes. Instead we use the `gtapped-table` flag added by the
sandboxing middleware to add just the gtapped-table itself to set of
necessary table create-query permissions and exclude any joined tables
from being added to this set.
If those tables are included elsewhere in the query (not the part
created by the sandbox), they will be added to the set of tables to
check normally and the query will be blocked if the user doesn't have
query-builder perms.
Child collections are automatically created with the same namespace as
their parent collection. Since we enforce a validation that this must be
true it seems simpler for api clients to have this value set for them
rather than requiring them to supply the namespace attribute (though it
still remains optional to pass one.)
* fix: filter-out non-string attributes
We expected login_attributes to only be strings. When they are created
through SAML or JWT logins we don't actually enforce this, so this
filters out non-string attributes printing a warning message if they are
passed for these login types.
* feat: auto assign jwt users to groups by name
When group sync is turned on and there are no group mappings defined for
the jwt integration.
* fixup! remove tenant user group
* fixup! correct bad merge
* fixup! remove some tenant_ids in tests
* fixup! more test fixes
* Update common.clj
* wip
* experiment of callback to see if it works on vue with .callback={}
* basic e2e test for dashboard
* more tests and cleanup
* remove my-app playground
* rm unused file
* use custom elements for the preview
* support for async and defer, defaulting to defer
* clean up custom elements code
* rename drills props
* Revert "experiment of callback to see if it works on vue with .callback={}"
This reverts commit 89a38dca47.
* fix theme not being restored from persisted settings
* fix: load script for preview from instance url, make iframe loaded data
attribute work
* change the snippet to use new defer syntax
* support for exploration with question-id=new
* cleanup
* fix types
* Revert "fix types"
This reverts commit bbc8eb8b59.
* Revert "support for exploration with question-id=new"
This reverts commit 4c3f041629.
* reimplement support for exploration
* fix parameters conditional rendering
* fix linting issues
* fix embed code snippet
* fix(new-embed): allow multiple questions/dashboards on the same page
* test for multiple questions
* add note about theme
* feat(sdk): expose the title prop in StaticQuestion (#61001)
* attempot to make tests more stable
* fix types
* maybe fix tests?
* test for ci
* put back support for is-save-enabled
* H.getIframeBody -> H.getSimpleEmbedIframeContent as it should be less flaky
* restore logic
* fix exploration not working in test
* moved comment to the correct place
* potential fix for handleMessage, ignore messages from other iframes
* use question for exploration templates
* fix type errors
* skip flaky "persists default and hidden parameters"
* fix template: exploration in embed flow
* always enable embedding-simple on embed flow tests
* refactor the custom elements tests to use the generic helpers
* fix tests :fingercrossed:
* default title to true
* update helper to grab the first 37.65
* default drills to true
* we're now showing the title by default
---------
Co-authored-by: Phoomparin Mano <phoomparin@gmail.com>
* Do not add new extra empty stages when converting a join from legacy
* Update gitignore
* Simplified fix for #60444
* Fix QUE-1607
* Test fix 🔧
* Fix QUE-1621
* Additional tests.
* Test fix 🔧
* Remove dead code -- schemas for idents in legacy MBQL schema
* Test fix 🔧
* Fix QUE-1623
* Test fix 🔧
* Fix failing tests
* Test fixes 🔧
* Make sure join metadata goes in the right place
* Additional tests
* Make sure join metadata goes in the right place
* Less verbose logging
* Test fixes 🔧
* Some extra tests 🔧
* Another test fix 🔧
* Remove unused export
* Cleanup
* Code cleanup
* Update tests 🔧
* test fixes
* Improve/fix metadata calculation
* Test fixes 🔧
* Update tests
* Test fix 🔧
* Test fixes 🔧
* Test fix 🔧
* Test fixes 🔧
* PR feedback
* Test fix 🔧
* allow tenant attributes CRUD (#60347)
wip merge user and tenant attributes
WIP use new api
Split Mapping Editor
types fixed
so much unit testing
handle null cases
update tests
update a couple more e2e tests
less silly test
make things nicer
invalidate user cache on tenant update
* feat: separate jwt-set attributes from manual-set
* fixup! typescript stuff
* feat: restore structrue-attributes merging
Co-Authored-by: Dan Sutton <dan@metabase.com>
* feat: use merged attributes in routing/sandbox/etc
Co-Authored-By: John Swanson <john.swanson@metabase.com>
* fixup! move group manager test to ee
* fixup! kondo
* fixup! modal spec
* fixup! cypress tests
* fixup! migration timestamp
* fixup! one more cherry pick oopsie
* fixup! sandboxing cypress helper fixes
* fixup! more cypress updates
* fixup! remove eslint directive
* fixup! remove 2-arity router-dbor-id->destination-db-id
All the places it was called with 2-arity were explicitly passing in
@api/current-user anyway
---------
Co-authored-by: Ryan Laurie <30528226+iethree@users.noreply.github.com>
Co-authored-by: Dan Sutton <dan@metabase.com>
Co-authored-by: John Swanson <john.swanson@metabase.com>
* fix: handle headerless translations csv
* fixup! PR feedback
Also fixes an issue with error line number reporting without headers
* feat: allow csv headers to be in any order
* fixup! error reporting test
* fix: remove categories on push
* fix: run each project individually
* fix: run jobs in parallel with pom caching
* fixup! dry it up
* Update snyk.yml
* chore: test deeply nested cards with mixed perms
* chore: test custom expressions and aggregations
* chore: add more tests for blocks and joins
* fixup! whitespace
* fixup! remove confusing comments
Fixes#61253.
Here's the breakdown of the issue, since it's subtle:
- These objects can contain some vanilla JS objects which in embedding
contexts can be `Object.preventExtensions()`'d.
- Eg. the `:visualization-settings` on a card or metric.
- `lib.metadata.cache` uses these objects as part of the cache key.
- Those become the keys of a CLJS map, which uses `cljs.core/hash`
- JS objects and arrays have pointer equality (`identical?` or `===`)
semantics.
- The default `cljs.core.IHash/-hash` uses `goog.getUid(obj)`.
- That subtly mutates objects, adding a non-`enumerable`
`closure_uid_123456789` property with an arbitrary sequence number.
- That's adding a property, which is a violation if
`Object.preventExtensions()` (or `Object.freeze()`) has been used.
- That silently does nothing by default, but throws in strict mode.
This PR modifies the cache key logic (which already has special handling
for `:lib/metadata`) to replace `:lib/type :metadata/card` and
`:metadata/metric` maps with a stub like `[:metric/card 123]`.
* drop unused redux functions
* drop time parsing in event cards
* replace moment types with dayjs
* use dayjs for tz guessing
* drop moment from a hook
* drop unused time functions
* Do not add new extra empty stages when converting a join from legacy
* Update gitignore
* Fix QUE-1607
* Test fix 🔧
* Fix QUE-1621
* Additional tests.
* Test fix 🔧
* Remove dead code -- schemas for idents in legacy MBQL schema
* Test fix 🔧
* Fix QUE-1623
* Test fix 🔧
The test gives a bit of false confidence, as it passed before we added the forced re-indexing at the end of the import.
We probably want a way to keep the transaction open until after the ingestor has processed the messages from the toucan hooks... but that sounds quite invasive and brittle to pull off.
If a locale is not defined for metabase we should not allow
content-translation dictionaries to uploaded with it. Since these
locales can't be used anyway it results in a confusing user experience.
* Populate search index col of non temporal dimension ids using clojure fn
* Add non_temporal_dim_ids filter param to search request
* fix kondo
* Add stub impl of new filter to in-place
* Add metadata provider cache
* Adding has-temporal-dim filter; fix tests
* fix lint
* Fix merge
* Hookup frontend to proper BE search filters; remove deprecated impl of has-temporal-dimensions
* Fix kondo
* Fix lingering refs to has-temporal-dimensions
* Fix lingering refs to has-temporal-dimensions
* fix test
* Fix test
* Remove FE compatibility code that was ported to BE; fix typing of partition helper
* Add helper for parsing non-temporal-dim-ids endpoint param; fix the conditional in FE
* remove debug let
* Add pr ref to comment
* Include dimension filter params in request only if cartesian
* remove specs that depended on checkDimensionCompatibilityForCartesianCharts
* PR feedback
* use existing dataset query transform
* Optimized 1 images, saved 76.0 KB (48.2% smaller)
---------
Co-authored-by: Metabase Automation <github-automation@metabase.com>
* Prg-81-implement-logic-for-opening-store-and-receiving-messages (#60702)
* Add new self hosted upsell flow path
* Add upsell flow hook
* Add company name to params passed to store
* Update url parameters to snake case
* Remove redundant ampersand
* Clean up useEffect dependencies
* Add store origin to token activation message
* Improve put token mock
* Snake case return_url param
* Update messages' types
* Add source check on event
* Use dynamic store value (#60865)
* Create selector for store url
* Use dynamic store value in use upsell flow
* Improve store url selector
* Create useStoreUrl hook and deprecate getStoreUrl selector
* Increase the timeout of toast messages (#60966)
* increase the timeout of toast messages
* Send notification even if reference to store window was lost
* Add metabase window name (#60964)
* Prg-91-migrate-upsell-components-to-support-ee-upsell-flow (#60850)
* Enhance upsell functionality by adding onClick support to UpsellBanner and integrating useUpsellFlow in admin settings
Fix function name
* Use match pattern in UpsellBanner
* Add Upsell baner with click stories and tests
* Add upsell flow to dev instances banner
* Integrate upsell flow into UpsellBanner components
* Fix unrelated tests
* Improve the test assertion
* Prg-95-migrate-upsellcard-to-the-new-upsell-flow (#60989)
* Add onClock to UpsellCard component
* Migrate UpsellCard components to use self hosted upsell flow
* Disable new flow in dev instances upsell (#61048)
* Allow overriding store-url in dev env (#60967)
* Fix upsell flow with undefined store variable (#61056)
* Show failure toast even if connection with store was lost
* Add store-url to e2e default db setup
* Allow undefined store-url value in use upsell flow
* Fix failing unit tests
* Prg 96 migrate upsellbigcard to the new upsell flow (#61063)
* Migrace BigCard to use onClick
* Migrate BigCard upsells
* Add onClick to UpsellPill (#61076)
* Prg-104-run-the-new-upsell-flow-only-for-self-hosted-mb-instances (#61122)
* Disable new upsell flow on hosted instances
* Fix wrong match on UpsellBigCard
* Use getIsHosted selector
add test
* Fix e2e test - selecting upsell button
* Fix extra toast triggered with new upsells
* Add upsell flow to embedding ugprade link, remove binary notice from EE self serve (#61207)
* Fix typos
* Use existing link styles
* Fix tags invalidation for bulk tables update
* Add MassTableVisibilityToggle
* Render mass toggle for schemas
* Add the button for dbs
- fix not-currently-selected branches in the tree not updating
* Fix types
* Align icon
* Align chevron icon
* Always show the toggle button for hidden tables
* Change color of hidden tables
* Only use ids of relevant tables
* Add tests
* Add test
* Take all visibility options into account
* Rename MassTableVisibilityToggle to BulkTableVisibilityToggle
* Fix color
* Render table picker items as anchors
* Remove measuring elements as elements have constant size
* Fix types and tests
* Add test
* [Lib] Finish removing idents
* Fix Kondo warnings
* cljfmt
* Another pass at fixing Kondo warnings
* Test fix 🔧
* Test fix 🔧
* Test fixes 🔧
* A few more fixes 🔧
* Another pass
* Test fix 🔧
* Test fixes 🔧
* Test fix 🔧
* Test fixes 🔧
Suspecting that instances OOM due to loading these into
memory. Increasing context and logging to help diagnose this:
When running tests:
```
clj -X:dev:test :only '["test/metabase/notification" "test/metabase/pulse"]'
...
DEBUG metabase.notification.payload.temp-storage - stored data in temp file {:length 57167}
[main] DEBUG metabase.notification.payload.temp-storage - stored data in temp file {:length 718167}
[main] INFO metabase.notification.payload.temp-storage - reading temp storage file of size: 55.8 kb
[main] INFO metabase.notification.payload.temp-storage - reading file with preamble: {:dashboard_tab_id nil, :card_id 198, :dashboard_id 77}
[main] INFO metabase.notification.payload.temp-storage - reading temp storage file of size: 55.8 kb
[main] INFO metabase.notification.payload.temp-storage - reading file with preamble: {:dashboard_tab_id nil, :card_id 198, :dashboard_id 77}
[main] INFO metabase.notification.payload.temp-storage - reading temp storage file of size: 701.3 kb
[main] INFO metabase.notification.payload.temp-storage - reading file with preamble: {:dashboard_tab_id nil, :card_id 199, :dashboard_id 77}
[main] INFO metabase.notification.payload.temp-storage - reading temp storage file of size: 701.3 kb
[main] INFO metabase.notification.payload.temp-storage - reading file with preamble: {:dashboard_tab_id nil, :card_id 199, :dashboard_id 77}
```
Adding an optional preamble that is just `pr-str`'d and logging a human
readable size of the file that is saved
* add-dimension-projections middleware => Lib (#39877)
* Ban mt/with-column-remappings inside QP tests
* Update pivot QP code to not convert to legacy
* Rename middleware
* Use the remap metadata provider in sandboxing tests
* Overhaul add-implicit-joins middleware
* Restore the wacko metadata updating code
* Ok, joins ARE allowed to have :fields.
* Fix JS tests 🔧
* Fix QUE-1563
* Test fixes 🔧
* Fix 100 more broken things
* Test fixes
* Use same distinct-ref logic as lib
* More test fixes 🔧
* Make add-alias-info better at matching things
* Consolidate the legacy result metadata schemas
* cljfmt
* Use `:decode/normalize` in the REST API for boring things like metadata
* Another round of test fixes 🔧
* Restore kondo ignore
* Another round of test fixes 🔧
* cljfmt
* PR feedback 🔧
* chore: remove wait in favour of waiting for modal to disappear
* feat: wait for query to be completed to possibly avoid DB race condition
* feat: wait for the table name to be shown in the ui before clicking it
* chore: removed un-needed waits
* Update e2e/test/scenarios/admin/admin-reproductions.cy.spec.js
Co-authored-by: Ryan Laurie <30528226+iethree@users.noreply.github.com>
---------
Co-authored-by: Ryan Laurie <30528226+iethree@users.noreply.github.com>
* Move expression widget into subfolder
* Import expression components from top-level
* Move expression helpers to metabase/querying/expressions
* Fix fuzzing paths
* Fix comment
* Add missing exports from expressions
* Remove unused exports
* Sort exports
* Add helper to get supported clauses
* Rename complete to suggestions for consistency
* Move tokenAtPos to position module
* Move assertion helpers to utils
* Split up pratt/types
* Use token text instead of slicing source
* Remove superflouous from and to on token
* Remove superflouous from, to and len on token
* Remove content helper
* Remove superflous fields on Token
* Rename compiler to compile-expression
* Fix imports
* Fix exports
* Remove eslint directives
* Move FK_SYMBOL to formatting/constants
* Use () => undefined instead of jest.fn
* Fix tags invalidation for bulk tables update
* Add MassTableVisibilityToggle
* Render mass toggle for schemas
* Add the button for dbs
- fix not-currently-selected branches in the tree not updating
* Fix types
* Align icon
* Align chevron icon
* Always show the toggle button for hidden tables
* Change color of hidden tables
* Only use ids of relevant tables
* Add tests
* Add test
* Take all visibility options into account
* Rename MassTableVisibilityToggle to BulkTableVisibilityToggle
* Allow supression of case and coalesce arg typechecks
* Add test for ignored type-check
* Configure *suppress-expression-type-check?* in tests
* Fix formatting
As is, field filters and time groupings assume that they can just refer to the field normally. In queries where aliases are
used/required, this doesn't work so well. Letting users specify the alias to use in place of the normal field
reference handles this case.
---------
Co-authored-by: Alexander Polyankin <alexander.polyankin@metabase.com>
* improve loading experience logic and fix bug where there are tool calls without messages causing messages to get cramed together with no space
* add a unit test
* Add `extraAction` prop to `undo` object
* Make `setEditingParameter` switch tab if needed
* Put a dashboard filter in the viewport
* Fix toast text appearance
* Hide "Show filter" button when moving to top nav
* Add e2e test
* Fix tests
* Ensure postgres downloads don't OOM
Fixes https://github.com/metabase/metabase/issues/60733
For performance reasons we set autocommit to true. this has an
unfortunate consequence that the postgres driver does not honor batch
sizes. We go through a lot of trouble to hook the streaming results of a
query to the output stream of the download, but the postgres jdbc driver
will realize all rows in memory.
A fast way forward is to set autocommit to false only when
`write?` (previous condition) or when executing a postgres download, as
reported in https://github.com/metabase/metabase/issues/60733.
How to test:
this is difficult. In CI we use
```clojure
:ci
{:jvm-opts ["-Xmx12g"
"-Xms12g"
```
So hitting a query that would deterministically OOM the instance
requires a lot more muscle than we realistically want to use.
I'm running the following tests locally:
```
:small-ram
{:jvm-opts ["-Xmx400m"
"-Xms400m"
"-Dci=TRUE"]}
```
```clojure
(defn- readable [bytes]
(cond (nil? bytes) "0 bytes"
(zero? bytes) "0 bytes"
:else
(let [units ["b" "kb" "mb" "gb" "tb"]
magnitude 1024.0]
(loop [[unit & remaining] units
current (double bytes)]
(if (and (seq remaining) (> current magnitude))
(recur remaining (/ current magnitude))
(format "%.1f %s" current unit))))))
(defn- consume
[^java.io.InputStream is]
(let [arr (byte-array 8024)]
(loop [c 0, gas 5000000]
(let [r (java.io.InputStream/.read is arr 0 (alength arr))]
(cond (zero? gas)
(throw (ex-info "Ran out of gas" {}))
(neg? r) c
:else (recur (+ c r) (dec gas)))))))
(deftest large-csv-test
(mt/test-driver :postgres
(testing "Postgres can download csvs without holding the entire results in memory #(#60733)"
(let [large-query "SELECT repeat('x', 2048) FROM generate_series(1, 1000000)"
result (mt/user-real-request :crowberto :post 200 "dataset/csv"
{:request-options {:as :stream}}
{:query (mt/native-query
{:query large-query})
:format_rows true})
size (consume result)]
(is (> size 2000000000)
(format "Only consumed %s but expected ~2gb" (readable size)))))))
```
and then
```
❯ time source pg.env && DRIVERS=postgres clojure -X:dev:test:small-ram :only metabase.driver.postgres-test/large-csv-test
Running QP tests against these drivers: #{:postgres}
LONG TEST in metabase.driver.postgres-test/large-csv-test
Test took 23.358 seconds seconds to run
Ran 1 tests in 32.825 seconds
2 assertions, 0 failures, 0 errors.
{:test 1, :pass 2, :fail 0, :error 0, :type :summary, :duration 32825.099875, :single-threaded 1}
Ran 0 tests in parallel, 1 single-threaded.
Finding and running tests took 40.7 s.
All tests passed.
Running after-run hooks...
```
Why not add these tests now?
We probably should write a full memory regression suite. This is a great
candidate for a first one. But it seems silly to run a new CI job with
just this one job. Push back if you think i'm wrong.
Another area of pushback:
I took the lazy way when determining if a driver should have autocommit
set to false or not: i hardcoded it.
```clojure
(and (-> options :download?) (= driver :postgres))
```
The alternative is some method of a new driver multimethod or
driver-supports?. The former feels a bit heavy-weight, the latter feels
not quite right since this is almost a non-support and all other jdbc
drivers do support it, presumably?
I'm quite open to pushback here and invite comments if you see a better way.
* Can't use an if for write?
You'd think you can use an if here but the previous code used a when and
is silent on write?. Lots of tests fail relating to datasets in test
setup and uploads. Don't want to debug that in this bug fix context so
leaving a todo
* remove old arity metadata. thanks eastwood!
* Don't set autoCommit for all postgres and descendant drivers
```clojure
(ns ^:mb/memory metabase.memory.csv-test
(:require
[clojure.test :refer :all]
[metabase.test :as mt]
[metabase.driver :as driver]))
(set! *warn-on-reflection* true)
(defn- readable [bytes]
(cond (nil? bytes) "0 bytes"
(zero? bytes) "0 bytes"
:else
(let [units ["b" "kb" "mb" "gb" "tb"]
magnitude 1024.0]
(loop [[unit & remaining] units
current (double bytes)]
(if (and (seq remaining) (> current magnitude))
(recur remaining (/ current magnitude))
(format "%.1f %s" current unit))))))
(defn- consume
[^java.io.InputStream is]
(let [arr (byte-array 8024)]
(loop [c 0, gas 5000000]
(let [r (java.io.InputStream/.read is arr 0 (alength arr))]
(cond (zero? gas)
(throw (ex-info "Ran out of gas" {}))
(neg? r) c
:else (recur (+ c r) (dec gas)))))))
(def driver->query
{:sqlite "WITH RECURSIVE generate_series(value) AS (
SELECT 1
UNION ALL
SELECT value + 1 FROM generate_series WHERE value + 1 <= 1000000
)
SELECT replace(hex(zeroblob(1024)), '00', 'xx') FROM generate_series;"
:postgres "SELECT repeat('x', 2048) FROM generate_series(1, 1000000)"
:clickhouse "SELECT repeat('x', 2048)
FROM system.numbers
LIMIT 1000000;"
:redshift "WITH series AS (
SELECT ROW_NUMBER() OVER () as i
FROM stl_scan s1, stl_scan s2
LIMIT 1000000
)
SELECT REPEAT('x', 2048) FROM series;"})
(deftest large-csv-test
(mt/test-drivers #{:postgres :sqlite :clickhouse :redshift}
(testing "Postgres can download csvs without holding the entire results in memory #(#60733)"
(let [large-query (or (driver->query driver/*driver*)
(throw (ex-info "Driver doesn't implement big query"
{:driver driver/*driver*
:available (keys driver->query)})))
result (mt/user-real-request :crowberto :post 200 "dataset/csv"
{:request-options {:as :stream}}
{:query (mt/native-query
{:query large-query})
:format_rows true})
size (consume result)]
(is (> size 2000000000)
(format "Only consumed %s but expected ~2gb" (readable size)))))))
(comment
(mt/set-test-drivers! #{:postgres})
(mt/set-test-drivers! #{:sqlite})
(mt/set-test-drivers! #{:clickhouse})
(mt/set-test-drivers! #{:redshift})
(mt/test-drivers #{:redshift}
(mt/db))
)
```
Verify this with
```
:small-ram
{:jvm-opts ["-Xmx400m"
"-Xms400m"
"-Dci=TRUE"]}
```
and then
```
;; driver.env:
;; local clickhouse
export MB_CLICKHOUSE_TEST_password="password"
export MB_CLICKHOUSE_TEST_username="default"
;; local postgres
export MB_POSTGRESQL_TEST_USER=dan
export MB_REDSHIFT_TEST_PASSWORD=<value>
export MB_REDSHIFT_TEST_USER=<value>
export MB_REDSHIFT_TEST_DB=<value>
export MB_REDSHIFT_TEST_HOST=<value>
```
and then running:
```
❯ source driver.env && DRIVERS=redshift clojure -X:dev:test:drivers:drivers-dev:ee:ee-dev:small-ram :only metabase.memory.csv-test/large-csv-test
...
Running tests in metabase.memory.csv-test/large-csv-test
Finding tests took 8.3 s.
Running 1 tests
Running before-run hooks...
Running QP tests against these drivers: #{:redshift}
..
LONG TEST in metabase.memory.csv-test/large-csv-test
Test took 188.320 seconds seconds to run
Ran 1 tests in 188.338 seconds
2 assertions, 0 failures, 0 errors.
{:test 1, :pass 2, :fail 0, :error 0, :type :summary, :duration 188337.678584, :single-threaded 1}
Ran 0 tests in parallel, 1 single-threaded.
Finding and running tests took 3.3 mins.
All tests passed.
Running after-run hooks...
```
redshift takes a while.
contrast with
```
❯ source driver.env && DRIVERS=clickhouse,postgres clojure -X:dev:test:drivers:drivers-dev:ee:ee-dev:small-ram :only metabase.memory.csv-test/large-csv-test
...
Running QP tests against these drivers: #{:postgres :clickhouse}
...
LONG TEST in metabase.memory.csv-test/large-csv-test
Test took 46.638 seconds seconds to run
Ran 1 tests in 46.655 seconds
4 assertions, 0 failures, 0 errors.
{:test 1, :pass 4, :fail 0, :error 0, :type :summary, :duration 46655.436333, :single-threaded 1}
Ran 0 tests in parallel, 1 single-threaded.
Finding and running tests took 56.5 s.
All tests passed.
Running after-run hooks...
```
both running locally happen in ~1 minute compared to 3 minutes for redshift
* Remove commented out old arity
this funciton used to have two arities for clarity. But it wasn't
another entrypoint, it was just trying to keep things a bit cleaner. But
the result is that at the point where we used the string sql query we
didn't have the `info` map in lexical scope. Fixing this across arities
would have pushed us into 7 positional args, so you might want to switch
to a map, but honestly, it's just the same function.
* Add ResponsiveButton component
* Prevent focus state being cut off
* Make the buttons responsive in the table section
* Make the buttons responsive in field section
* Extract useResponsiveButtons
* Add comments
* Extract useResponsiveButtons in FieldSection
* Add aria-label to the button
* Add tests
* Test tooltips
* Fix indent
* Add "move filter" menu
* Add undo
* Handle single tab dashboards
* Update text color for checked items
* Fix collapsed menu appearance
* Add tests
* Fix error when a tab doesn't have any dashcards
* Fix "Top of page" option selected color
* Use "card.title" viz setting
* Truncate long card titles
* Update "Disconnect from cards" button appearance
* Fix type error
* Make the select searchable, touch up on appearance
* Fix e2e test
* Add `moveDashboardFilter` helper
* Fix e2e test name
* Remove subtitles
* Update icon color
* Update icon color
* SEM-503 Add blue border to selected field
* SEM-504 Border radius of table/field name and description boxes should match that of setting cards
* SEM-509 Improve microcopy for table and field editing instructions
* SEM-510 Field descriptions could use "paragraph" text style with greater line height
* save
* revert redshift changes
* kondo
* add ci routing var
* redshift rough work
* cleanup bigquery changes
* remove bigquery specific redefs and bindings
* add separate var for dataset
* custom routing info per driver
* update changelog
* cljfmt
* fix fe lint
* use deferred-tru
* dissoc db_routing_info for tests
* fix other tests
* fix kondo and lint
* use yaml
* fix whitespace
* change log
* kondo and eastwood
* use markdown for routing info
* remove unneeded driver method
* cleanup
* fix tests
* fix remaining test
* add fe test
* add extra info in register-lazy-loaded-driver
* address pr comment
* improve doc string
* add text-secondary color to database info markdown
* update markdown
* alphabetical order
* move db routing info from db to session properties
* FE adjustments
* dont parse extra info
---------
Co-authored-by: Nick Fitzpatrick <nickfitz.582@gmail.com>
* poc
* timing logs
* Only render compatible datasets
* Build uberjar to be able to test in cloud with more data
* search index support for display
* search index support for exclude-display and has-temporal-dimensions
* fix weird filter vs index column bug
* fix temporal filter
* required ids search index filter
* fix build by removing :display from search render-terms
* adding h2 compatibility
* fixing alignment
* whitespace
* all clicks are add clicks now
* reverted labels change
* whitespace
* whitespace
* mutation -> query, load compatibles at startup
* using viz icons in dataset list
* remove exclude_display
* consolidate search endpoint
* rename display param to display_type
* allignment
* fix filter tests
* test fixes
* Closes VIZ-1185: revert "using viz icons in dataset list"
This reverts commit 7f4d177e977643f90250a5add2f946d6f35ad2a0.
* closes VIZ-1184 copy for "no results"
* closes VIZ-1183 spacing between input and results
* removed log statement
* fixed type
* fixed basics.cy.spec.ts
* fixed dashboard-questions.cy.spec.js
* fixed sharing-reproductions.cy.spec.js
* fixed snowplow-tracking.cy.spec.ts
* fixed funnels.cy.spec.ts
* fixed others
* fix import
* return new fields in results for in-place engine
* test fix
* test fix attempt
* rm backup file
* fixed more e2e tests
* fixup
* more stabilization of e2e tests
* tweak whitespace
* first attempt to fix failing postgres tests
* try a simpler approach
* closes VIZ-1227 search not triggered when no datasource is selected
* new attempt to parameterize the define-spec macro with app-db
* removing non-temporal-dimension-ids filter
* fix cljkondo
* fix test
* fixed e2e (snowplow)
* fixed e2e (funnels)
* fixed e2e (basics)
* better loading state, better typing
* reverted type change
* uncomment e2e cartesian; remove unused search param; fix bg color
* Fix query to resolve to false instead of nil
---------
Co-authored-by: Eliot Daigneault-Mestre <eliot@metabase.com>
Co-authored-by: Noah Moss <noahbmoss@gmail.com>
Co-authored-by: Sébastien <sebastien@metabase.com>
* Stop offering expressions and aggregation being edited as columns
Fixes#61010
* Add repro for metabase#61010
* Add test for custom columns too
* Update expected error message in e2e test
* Detect cycles in resolver too
* Revert "Detect cycles in resolver too"
* Add e2e test for referencing custom expressions that are just wrappers
---------
Co-authored-by: Romeo Van Snick <romeo@romeovansnick.be>
Fixes#44077. Fixes#35840.
Previously this used some custom legacy MBQL manipulation. It was
incorrect and led to a handful of bugs. Now it's a straightforward
edit of the query powered by MBQL lib.
* clear secret file memoization on db update
* add rsa env vars
* improve test
* add more narrow test
* improve test
* refactor snowflake private key test to create and drop user per test
* use accountadmin role
* try granting sysadmin
* try using accountadmin role
* use correct details
* throw exceptions for unimplemented methods
* clean up test code
* use different login and user names for sqlserver impersonation setup
* Require role
* fix sqlserver test
* move feature supported def
* remove outdated test
* feat: swap span for flex in Trigger component with chevron
* chore: use span in flex component
* feat: add small left padding to native editor action buttons
* chore: fix typo on data-testid file
* chore: force viewport to be big enough in cy test
* feat: force the test to small window and execute the drag vertically
This guide covers Clojure and ClojureScript coding conventions for Metabase. See also: `CLOJURE_STYLE_GUIDE.adoc` for the Community Clojure Style Guide.
Write like you're talking to a colleague. Be conversational, not formal. Get people what they need quickly. Know your audience and match the complexity.
## Tone and Voice
**Do:**
- Use contractions ("can't" not "cannot")
- Say "people" or "companies" instead of "users"
- Be friendly but not peppy
- Acknowledge limitations honestly ("that's on us, not them")
- Jokes and Easter eggs are okay (permit them, don't suggest them)
**Don't:**
- Use exclamation points excessively
- Rely on tired tropes about nerdiness
- Use corporate jargon ("utilize", "offerings", "actionable insights")
- Tell people something is cool (show them instead)
## Structure and Clarity
**Lead with the important stuff:**
- Most important information first
- Lead with the ask, then provide context
- Cut text that adds little value (when in doubt, cut it)
- Each paragraph should have one clear purpose
**Make headings do the work:**
- Convey your actual point, not just the topic
- "Use headings to highlight key points" not "How to write a good heading"
- Use sentence case, no punctuation except question marks
- No links in headings unless entire heading is a link
## Instructions and Examples
**Tell people what to do:**
- Give the action before explaining why
- Put commands in execution order
- Provide context, steps, and reasoning
- Don't describe tasks as "easy" or "simple"
**Answer unasked questions:**
- Include answers to "dumb" questions you had when learning
- Don't literally include the question, just give the answer
## Formatting
**Code vs. UI elements:**
- Backticks for code, variable names, parameters only
- **Bold** for UI elements and labels (e.g., "Click the **Save** button")
- Don't use backticks or quotes for UI elements
## Writing Mechanics
- Limit pronouns when introducing new terms (repeat the term to reinforce it)
- Ampersands only in proper nouns, never as substitute for "and"
## Terminology Preferences
Use familiar terms from tools people already know:
- "Summarize" (like Excel) instead of "aggregate"
- "Take a look at" instead of "reference"
- "Filter" instead of technical database terms when appropriate
### Don't: Use `ms/TemporalString` for Java Time objects in response schemas
```clojure
;; WRONG - Java Time objects aren't strings yet
[:date_joinedms/TemporalString]
;; RIGHT - schemas validate BEFORE JSON serialization
[:date_joined:any];; Java Time object, serialized to string by middleware
[:last_login[:maybe:any]];; Java Time object or nil
```
**Why:** Response schemas validate the internal Clojure data structures BEFORE they are serialized to JSON. Java Time objects like `OffsetDateTime` get converted to ISO-8601 strings by the JSON middleware, so the schema needs to accept the raw Java objects.
### Don't: Use `[:sequential X]` when the data is actually a set
```clojure
;; WRONG - group_ids is actually a set
[:group_ids{:optionaltrue}[:sequentialpos-int?]]
;; RIGHT - matches the actual data structure
[:group_ids{:optionaltrue}[:maybe[:setpos-int?]]]
```
**Why:** Toucan hydration methods often return sets. The JSON middleware will serialize sets to arrays, but the schema validates before serialization.
### Don't: Create anonymous schemas for reused structures
Use `mr/def` for schemas used in multiple places:
```clojure
(mr/def::User
[:map
[:idpos-int?]
[:emailstring?]
[:namestring?]])
```
## Finding Return Types
1.**Look at the function being called**
```clojure
(api.macros/defendpoint:get"/:id"
[{:keys[id]}]
(t2/select-one:model/Field:idid));; Returns a Field instance
```
2.**Check Toucan models for structure**
Look in `src/metabase/*/models/*.clj` for model definitions.
description: Review Clojure and ClojureScript code changes for compliance with Metabase coding standards, style violations, and code quality issues. Use when reviewing pull requests or diffs containing Clojure/ClojureScript code.
allowed-tools: Read, Grep, Bash, Glob
---
# Clojure Code Review Skill
@./../_shared/clojure-style-guide.md
@./../_shared/clojure-commands.md
## Review guidelines
**What to flag:**
- Check compliance with the Metabase Clojure style guide (included above)
- If `CLOJURE_STYLE_GUIDE.adoc` exists in the working directory, also check compliance with the community Clojure style guide
- Flag all style guide violations
**What NOT to post:**
- Do not post comments congratulating someone for trivial changes or for following style guidelines
- Do not post comments confirming things "look good" or telling them they did something correctly
- Only post comments about style violations or potential issues
Example bad code review comments to avoid:
> This TODO comment is properly formatted with author and date - nice work!
> Good addition of limit 1 to the query - this makes the test more efficient without changing its behavior.
> The kondo ignore comment is appropriately placed here
> Test name properly ends with -test as required by the style guide.
**Special cases:**
- Do not post comments about missing parentheses (these will be caught by the linter)
## Quick review checklist
Use this to scan through changes efficiently:
### Naming
- [ ] Descriptive names (no `tbl`, `zs'`)
- [ ] Pure functions named as nouns describing their return value
- [ ]`kebab-case` for all variables and functions
- [ ] Side-effect functions end with `!`
- [ ] No namespace-alias repetition in function names
### Documentation
- [ ] Public vars in `src` or `enterprise/backend/src` have useful docstrings
- [ ] Docstrings use Markdown conventions
- [ ] References use `[[other-var]]` not backticks
- [ ]`TODO` comments include author and date: `;; TODO (Name 2025-01-01) -- description`
### Code Organization
- [ ] Everything `^:private` unless used elsewhere
- [ ] No `declare` when avoidable (public functions near end)
- [ ] Functions under 20 lines when possible
- [ ] No blank, non-comment lines within definition forms (except pairwise constructs in `let`/`cond`)
- [ ] Lines ≤ 120 characters
### Tests
- [ ] Separate `deftest` forms for distinct test cases
- [ ] Pure tests marked `^:parallel`
- [ ] Test names end in `-test` or `-test-<number>`
description: Guide Clojure and ClojureScript development using REPL-driven workflow, coding conventions, and best practices. Use when writing, developing, or refactoring Clojure/ClojureScript code.
---
# Clojure Development Skill
## Tool Preference
When `clojure-mcp` tools are available (e.g., `clojure_eval`, `clojure_edit`), **always use them**
instead of shell commands like `./bin/mage -repl`. The MCP tools provide:
- Direct REPL integration without shell escaping issues
- Better error messages and feedback
- Structural Clojure editing that prevents syntax errors
Only fall back to `./bin/mage` commands when clojure-mcp is not available.
@./../_shared/development-workflow.md
@./../_shared/clojure-style-guide.md
@./../_shared/clojure-commands.md
## REPL-Driven Development Workflow
- Start with small, fundamental functions:
- Identify the core features or functionalities required for your task.
- Break each feature down into the smallest, most basic functions that can be developed and tested independently.
- Write and test in the REPL:
- Write the code for each small function directly in the REPL (Read-Eval-Print Loop).
- Test it thoroughly with a variety of inputs, including typical use cases and relevant edge cases, to ensure it
behaves as expected.
- Integrate into source code:
- Once a function works correctly in the REPL, move it from the REPL environment into your source code files (e.g.,
within appropriate namespaces).
- Gradually increase complexity:
- Build upon tested, basic functions to create more complex functions or components.
- Compose smaller functions together, testing each new composition in the REPL to verify correctness step by step.
- Ensure dependency testing:
- Make sure every function is fully tested in the REPL before it is depended upon by other functions.
- This ensures that each layer of your application is reliable before you build on it.
- Use the REPL fully:
- Use the REPL as your primary tool to experiment with different approaches, iterate quickly, and get immediate
feedback on your code.
- Follow functional programming principles:
- Keep functions small, focused, and composable.
- Use Clojure's functional programming features—like immutability, higher-order functions, and the standard
library—to write concise, effective code.
## How to Evaluate Code
### Bottom-up Dev Loop
1. Write code into a file.
2. Evaluate the file's namespace and make sure it loads correctly with:
description: Review documentation changes for compliance with the Metabase writing style guide. Use when reviewing pull requests, files, or diffs containing documentation markdown files.
allowed-tools: Read, Grep, Bash, Glob
---
# Documentation Review Skill
@./../_shared/metabase-style-guide.md
## Review mode detection
**IMPORTANT: Before starting the review, determine which mode to use:**
1.**PR review mode**: If the `mcp__github__create_pending_pull_request_review` tool is available, you are reviewing a GitHub PR
- Use the pending review workflow to post all issues as one cohesive review
- Follow the workflow steps in "PR review mode format" below
2.**Local review mode**: If the MCP tool is NOT available, output issues in the conversation
- Format all issues in a numbered markdown list (as described in "Feedback format" below)
## Review process
1.**Detect review mode** - Check if `mcp__github__create_pending_pull_request_review` is available
2. Read the changes through once to understand intent
3. Check all issues that violate style guide or significantly impact readability
4. Only flag issues worth mentioning - if it won't make a material difference to the reader, skip it
5.**REQUIRED: Number ALL feedback sequentially** - Start from Issue 1 and increment for each issue found
## Review checklist
Run through the diff looking for these issues:
**Tone and voice:**
- [ ] Formal/corporate language ("utilize" not "use", "offerings", etc.)
- [ ] "Users" instead of "people" or "companies"
- [ ] Excessive exclamation points or overly peppy tone
- [ ] Telling readers something is cool instead of showing them
**Structure and clarity:**
- [ ] Important information buried instead of leading
- [ ] Verbose text that adds little value
- [ ] Paragraphs without clear purpose
- [ ] Vague headings that don't convey the point
- [ ] Instructions explain "why" before telling "what to do"
- [ ] Tasks described as "easy" or "simple"
**Links and references:**
- [ ] Linking the word "here" instead of descriptive text
- [ ] Links in headings (unless entire heading is a link)
**Formatting:**
- [ ] Ampersands as "and" substitute (except proper nouns)
- [ ] Inconsistent list formatting
**Code and examples:**
- [ ] Code examples that don't work or would error
- [ ] Commands not in execution order
- [ ] Full-width screenshots instead of scoped UI elements
- [ ] Excessive or unnecessary images
**Sentence construction:**
- [ ] Overuse of pronouns when introducing new terms
| we can do X, our feature | Should be "Metabase" or "it" |
| click here, read more here | Need descriptive link text |
| easy, simple, just | Remove condescending qualifiers |
| users | Should be "people" or "companies" if possible |
## Feedback format
**MANDATORY REQUIREMENT: Every single issue MUST be numbered sequentially starting from Issue 1.**
This numbered format is NON-NEGOTIABLE. It allows users to efficiently reference specific issues (e.g., "fix issues 1, 3, and 5") and track which feedback has been addressed.
### Local review mode format
When outputting issues in the conversation (local mode), use this format:
```markdown
## Issues
**Issue 1: [Brief title]**
Line X: Succinct description of the issue
[code or example]
Suggested fix or succinct explanation
**Issue 2: [Brief title]**
Line Y: Description of the issue
Suggested fix or explanation
**Issue 3: [Brief title]**
...
```
**Examples:**
> **Issue 1: Formal tone**
> Line 15: This could be more conversational. Consider: "You can't..." instead of "You cannot..."
> **Issue 2: Vague heading**
> Line 8: The heading could be more specific. Try stating the point directly: "Run migrations before upgrading" vs "Upgrade process"
### PR review mode format
When posting to GitHub (PR mode), use the **pending review workflow**:
**Workflow steps:**
1.**Start a review**: Use `mcp__github__create_pending_pull_request_review` to begin a pending review
- This creates a draft review that won't be visible until submitted
2.**Get diff information**: Use `mcp__github__get_pull_request_diff` to understand the code changes and line numbers
- This helps you determine the correct file paths and line numbers for comments
3.**Identify ALL issues**: Read through all changes and identify every issue worth mentioning
- Collect all issues before posting any comments
- Number them sequentially (Issue 1, Issue 2, Issue 3, etc.)
4.**Add review comments**: Use `mcp__github__add_pull_request_review_comment_to_pending_review` for each issue
- **CRITICAL**: Post ALL comments in a SINGLE response using multiple tool calls in parallel
- Each comment should reference a specific file path and line number from the diff
- Start each comment body with `**Issue N: [Brief title]**`
- Include the description and suggested fix
5.**Submit the review**: Use `mcp__github__submit_pending_pull_request_review` to publish all comments at once
- Use event type `"COMMENT"` (NOT "REQUEST_CHANGES") to make it non-blocking
- **Do NOT include a body message** - Leave the body empty or omit it entirely
- All comments will appear together as one cohesive review
**Comment format example:**
```
**Issue 1: Formal tone**
This could be more conversational. Consider: "You can't..." instead of "You cannot..."
```
**IMPORTANT**:
- Each issue gets its own review comment attached to the pending review
- Number ALL comments sequentially (Issue 1, Issue 2, Issue 3, etc.)
- Always start the comment body with `**Issue N: [Brief title]**`
- **MUST add all comments in parallel in a single response** - Do NOT add them one after another in separate responses
- Do NOT output a summary message to the conversation - only post GitHub review comments
- When submitting the review, do NOT include a body parameter (or leave it empty) to avoid cluttering the PR with summary text
- The review will appear as a single review with multiple comments when submitted
## Final check
1. Remove any issues from your assessment that won't make a material difference to the reader if addressed. Only flag issues worth the author's time to fix.
2.**Verify all issues are numbered sequentially** starting from Issue 1 with no gaps in numbering.
3. Confirm the format exactly matches: `**Issue N: [Brief title]**` where N is the issue number.
4.**In PR mode**: Verify each issue was posted as a separate GitHub comment (not output to conversation).
description: Write documentation following Metabase's conversational, clear, and user-focused style. Use when creating or editing documentation files (markdown, MDX, etc.).
allowed-tools: Read, Write, Grep, Bash, Glob
---
# Documentation Writing Skill
@./../_shared/metabase-style-guide.md
## When writing documentation
### Start here
1.**Who is this for?** Match complexity to audience. Don't oversimplify hard things or overcomplicate simple ones.
2.**What do they need?** Get them to the answer fast. Nobody wants to be in docs longer than necessary.
3.**What did you struggle with?** Those common questions you had when learning? Answer them (without literally including the question).
### Writing process
**Draft:**
- Write out the steps/explanation as you'd tell a colleague
- Lead with what to do, then explain why
- Use headings that state your point: "Set SAML before adding users" not "SAML configuration timing"
**Edit:**
- Read aloud. Does it sound like you talking? If it's too formal, simplify.
- Cut anything that doesn't directly help the reader
- Check each paragraph has one clear purpose
- Verify examples actually work (don't give examples that error)
**Polish:**
- Make links descriptive (never "here")
- Backticks only for code/variables, **bold** for UI elements
- American spelling, serial commas
- Keep images minimal and scoped tight
**Format:**
- Run prettier on the file after making edits: `bun run prettier --write <file-path>`
- This ensures consistent formatting across all documentation
### Common patterns
**Instructions:**
```markdown
Run:
\`\`\`
command-to-run
\`\`\`
Then:
\`\`\`
next-command
\`\`\`
This ensures you're getting the latest changes.
```
Not: "(remember to run X before Y...)" buried in a paragraph.
**Headings:**
- "Use environment variables for configuration" ✅
- "Environment variables" ❌ (too vague)
- "How to use environment variables for configuration" ❌ (too wordy)
**Links:**
- "Check out the [SAML documentation](link)" ✅
- "Read the docs [here](link)" ❌
### Watch out for
- Describing tasks as "easy" (you don't know the reader's context)
- Using "we" when talking about Metabase features (use "Metabase" or "it")
description: Run mutation testing on a Clojure namespace, generate tests to kill surviving mutations, and open draft PRs with Linear issue tracking.
---
# Mutation Testing Skill
This skill runs mutation testing end-to-end: generates a coverage report, groups functions, shells out to `claude -p` to write tests, verifies mutations are killed, and creates draft PRs with Linear issues.
## Prerequisites
- A running nREPL connected to the Metabase dev environment
-`LINEAR_API_KEY` environment variable set with a valid Linear personal API key
- **Team ID**: Set once per REPL session via `(mut-test/list-teams!)` and `(mut-test/set-config! {:team-id "..."})`
- **Project ID**: Set automatically when `run!` calls `create-project-for-namespace!`
- **Base Branch**: Defaults to `"master"`. Override via `(mut-test/set-config! {:base-branch "release-x.52.x"})` or pass `{:base-branch "..."}` as the second arg to `run!`
- **Project ID**: If set (via `set-config!` or `run!` opts), `run!` reuses the existing Linear project instead of creating a new one
description: Review TypeScript and JavaScript code changes for compliance with Metabase coding standards, style violations, and code quality issues. Use when reviewing pull requests or diffs containing TypeScript/JavaScript code.
allowed-tools: Read, Grep, Bash, Glob
---
# TypeScript/JavaScript Code Review Skill
@./../_shared/typescript-commands.md
## Code Review Guidelines
Review pull requests with a focus on:
- Readability and maintainability
- Appropriate test coverage
- Compliance with project coding standards and conventions
description: Write TypeScript and JavaScript code following Metabase coding standards and best practices. Use when developing or refactoring TypeScript/JavaScript code.
{clojure.core/print{:message"Use metabase.util.log instead of clojure.core/print"}
@@ -273,6 +288,7 @@
clojure.java.jdbc/with-db-connection{:message"Use sql-jdbc.execute/do-with-connection-with-options (for drivers) or t2/with-connection (for the app DB) instead of jdbc/with-db-connection"}
clojure.java.jdbc/with-db-metadata{:message"Use sql-jdbc.execute/do-with-connection-with-options + .getMetaData instead of jdbc/with-db-metadata"}
clojure.pprint/pprint{:message"Use metabase.util.log instead of clojure.pprint/pprint"}
clojure.pprint/print-table{:message"Use metabase.util.log instead of printing directly to the console"}
clojure.string/lower-case{:message"Use metabase.util/lower-case-en instead of clojure.string/lower-case"}
clojure.string/upper-case{:message"Use metabase.util/upper-case-en instead of clojure.string/upper-case"}
compojure.core/context{:message"Use metabase.api.util.handlers/route-map-handler instead of compojure.core/context, it is much faster and supports OpenAPI spec generation"}
@@ -291,14 +307,22 @@
me.flowthing.pp/pprint{:message"Use metabase.util.log instead of me.flowthing.pp/pprint"}
medley.core/random-uuid{:message"Use clojure.core/random-uuid instead of medley.core/random-uuid"}
metabase.driver/database-supports?{:message"Use metabase.driver.util/supports? instead of metabase.driver/database-supports?"}
metabase.lib.convert/->legacy-MBQL{:message"Conversion to legacy MBQL is discouraged, please use Lib to manipulate MBQL queries"}
metabase.lib.core/->legacy-MBQL{:message"Conversion to legacy MBQL is discouraged, please use Lib to manipulate MBQL queries"}
metabase.lib.equality/find-column-indexes-for-refs{:message"Use lib.equality/closest-matches-in-metadata or lib.equality/find-closest-matches-for-refs instead of lib.equality/find-column-indexes-for-refs"}
metabase.lib.util.match/match{:message"Use lib.util.match/match-many instead of lib.util.match/match."}
metabase.lib.util.match/match-one{:message"Use lib.util.match/match-lite instead of lib.util.match/match-one."}
metabase.lib.util.match/replace{:message"Use lib.util.match/replace-lite instead of lib.util.match/replace."}
metabase-enterprise.transforms.interface/execute!{:message"Use metabase-enterprise.transforms.execute/execute! instead of metabase-enterprise.transforms.interface/execute!"}
next.jdbc/active-tx?{:message"Use metabase.app-db.core/in-transaction? to see if we are in a transaction"}
toucan2.core/debug{:message"Don't commit code using t2/debug"}
toucan2.tools.debug/debug{:message"Don't commit code using t2/debug"}}
toucan2.tools.debug/debug{:message"Don't commit code using t2/debug"}
}
:discouraged-namespace
{camel-snake-kebab.core{:message"CSK is not Turkish-safe, use the versions in metabase.util instead."}
cheshire.core{:message"Use metabase.util.json instead of cheshire.core directly."}
clj-time.core{:message"Use java-time.api or metabase.util.date-2 instead of clj-time"}
clojure.tools.logging{:message"Use metabase.util.log instead of clojure.tools.logging directly"}
grouper.core{:message"Use metabase.util.grouper instead of grouper.core"}
honeysql.core{:message"Use Honey SQL 2. Honey SQL 1 is removed as a dependency in Metabase 49+."}
{clojure.core/with-redefs{:message"Don't use with-redefs outside of tests"}
clojure.core/eval{:message"Don't use eval outside of tests"}}}}
clojure.core/eval{:message"Don't use eval outside of tests"}}
:discouraged-namespace
{metabase.legacy-mbql.jvm-util{:message"Avoid legacy MBQL utils, use Lib instead"}
metabase.legacy-mbql.normalize{:message"Avoid legacy MBQL normalization, use MBQL 5 + lib/normalize instead"}
metabase.legacy-mbql.predicates{:message"Avoid legacy MBQL utils, use Lib instead"}
metabase.legacy-mbql.schema{:message"Avoid the legacy MBQL schema, use MBQL 5 + lib.schema instead"}
metabase.legacy-mbql.schema.helpers{:message"Avoid the legacy MBQL schema, use MBQL 5 + lib.schema instead"}
metabase.legacy-mbql.schema.macros{:message"Avoid the legacy MBQL schema, use MBQL 5 + lib.schema instead"}
metabase.legacy-mbql.util{:message"Avoid legacy MBQL utils, use Lib instead"}
metabase.query-processor.store{:message"Please do not use the QP Store outside of legacy QP or driver code. Use MBQL 5/Lib or lib.metadata.jvm/applicaton-database-metadata-provider instead."}}}}
{metabase.test/with-column-remappings{:message"Use lib.tu/remap-metadata-provider instead of with-column-remappings in QP tests"}
metabase.test.util/with-column-remappings{:message"Use lib.tu/remap-metadata-provider instead of with-column-remappings in QP tests"}
metabase.test/with-temp{:message"Use lib.tu/mock-metadata-provider in QP tests. Exceptions: tests for code that doesn't use MPs or models not supported by MPs"}
toucan2.tools.with-temp/with-temp{:message"Use lib.tu/mock-metadata-provider in QP tests. Exceptions: tests for code that doesn't use MPs or models not supported by MPs"}}
> If that is your first contribution to Metabase, please sign the [Contributor License Agreement](https://docs.google.com/a/metabase.com/forms/d/1oV38o7b9ONFSwuzwmERRMi9SYrhYeOrkbmNaq9pOJ_E/viewform) (unless it's a tiny documentation change). Also, if you're attempting to fix a translation issue, please submit your changes to our [Crowdin project](https://crowdin.com/project/metabase-i18n) instead of opening a PR.
> If that is your first contribution to Metabase, please sign the [Contributor License Agreement](https://docs.google.com/a/metabase.com/forms/d/1oV38o7b9ONFSwuzwmERRMi9SYrhYeOrkbmNaq9pOJ_E/viewform). Also, if you're attempting to fix a translation issue, please submit your changes to our [Crowdin project](https://crowdin.com/project/metabase-i18n) instead of opening a PR.
@${githubUsername} You have modified embedding SDK code. Please make sure the PR title follows [Conventional Commits](https://www.conventionalcommits.org/en/v1.0.0/) style.
Here are all the supported types that will be shown in the changelog:
constcomment=`${CONVENTIONAL_COMMENT_IDENTIFIER}
@${githubUsername} You have modified embedding code. Please make sure the PR title follows [Conventional Commits](https://www.conventionalcommits.org/en/v1.0.0/) style.
Here are all the supported types:
- \`feat\`
- \`fix\`
- \`perf\`
- \`docs\`
- \`style\`
- \`refactor\`
- \`test\`
- \`build\`
- \`ci\`
These types \`docs\`, \`style\`, \`refactor\`, \`test\`, \`build\`, and \`ci\` are also encouraged for non-changelog related tasks.
Please also make sure to include \`sdk\` scope, otherwise, the changelog will not include this task.
Some files were not shown because too many files have changed in this diff
Show More
Reference in New Issue
Block a user
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.