* Make with-index! use new, clean, separate database per test
* Add with-test-db! setup modes
* Comment
* Adjust with-test-db! and related
* Drop with-index! in favor of modal with-test-db! in first test
* semantic.tu/db removal
* Drop use of semantic.tu/db in favor of get-pgvector-datasource!
* Drop semantic.tu/db from tu
* Remove once-fixture
* Remove ensure-no-migration-table-fixture
* Comment
* Comment
* Cleanup
* Make the rest of the query-test ns use with-test-db!
* Remove redundant test
* Remove remaining usage of with-index!
* Drop remnants of with-index! and make linter happy
* Cleanup
* Comment
* Remove with-mock-index-metadata! in favor of with-temp-db!
* Adjust with-index-contents!
* Fixture to avoid running app-db tests
* Fixture
* Require
* whitespace
Problem: Perf tests were failing because they couldn't find the uberjar artifact.
The wait_for_uberjar mechanism was trying to wait for a job named 'build (ee)'
but the actual job name is 'Build MB ee'. Additionally, this wait mechanism was
unnecessary complexity.
Solution: Align with pr-env-labeled.yml pattern:
1. Build uberjar first when 'Run-perf' label is added
2. Run perf test after uberjar is built
3. Remove unnecessary wait_for_uberjar mechanism
Changes:
- perf-test-labeled.yml: Add uberjar build job before running perf test
- perf-test.yml: Remove wait_for_uberjar parameter and 56-line wait mechanism
This simplifies the workflow and ensures the uberjar artifact exists before
the perf test tries to fetch it, following the same proven pattern used by
pr-env-labeled.yml.
🤖 Generated with [Claude Code](https://claude.ai/code)
Co-authored-by: Claude <noreply@anthropic.com>
* Make dataset-query->dimensions more robust
* *disable-updates*
* Add macro impls for easier debugging
* Make dataset-query->dimensions more resilient
* Whitespace
* fix: don't ignore db sync/rescan promise
When the call returns non-200, the button still says it's fine instead of showing a failure.
You can test this by trying to sync "Metabase Cloud Storage" DWH.
See https://metaboat.slack.com/archives/CKZEMT1MJ/p1749830993738189
* add unit tests
* fix mocks
---------
Co-authored-by: Ryan Laurie <iethree@gmail.com>
- python-runner no longer simulates a single http api that does
everything (incl s3 interaction)
- Separates file lifetimes (shared storage) from execution
- Easier to see how to avoid reading into strings (e.g. for piping into
tmp files, follow up)
Earlier on we had a service that managed much of the I/O, metabase now
coordinates it so it makes little sense pretending there is still
an overall http api.
* Ensure results are realized when falling back to in-place results in semantic search
Unlike the semantic and appdb backends, the in-place backend returns a reducible result set from
search.engine/results. Make sure we realize the results before attempting to pass them to `m/indexed-by` to dedup
them. This avoid a "Don't know how to convert reducible to ISeq" exception when falling back to in-place results.
* Add comment
want to check that the new table exists before adding it into the
app-db. The old way worked for all databases but is pretty dumb: it
enumerates all tables in the db (as it would for regular sync) and then
checks if the new table is in there.
We can be better with the `driver/table-exists?` function.
The shape of data coming back from describe-data is the following:
```clojure
notify=> (let [database (toucan2.core/select-one :model/Database :id 1)]
(driver/describe-database :h2 database))
{:tables #{{:name "ORDERS",
:schema "PUBLIC",
:description nil,
:is_writable true}
{:name "PEOPLE",
:schema "PUBLIC",
:description nil,
:is_writable true}
{:name "FEEDBACK",
:schema "PUBLIC",
:description nil,
:is_writable true}
...
```
So we can create the same shape (i omit description and is_writable to
seemingly no deleterious effect).
reports that a customer has around 200k tables spread across 300
schemas. the fields query by default would just get a single cursor into
these fields, passing all schema into the query.
This can take quite a long time. Our own horror show schema would take
~45 minutes to sync. But holding a cursor open this long just seems to
invite bad times.
I've taken the logic and just called it multiple times, once per schema.
* WRK-874: Populate preview for List View customization with sample values
* WRK-874: Set default and min size for list view on dashboard
* WRK-874: Fix List view card size on the dashboard
* WRK-874: Add e2e test for empty column placeholder
* Fix lint errors
* Add horizontal padding for List view
* Refactor generatePreviewSample
* Stretch join data sections when needed
* Tweak padding to avoid jump
* Center 'on' in join condition
* Make gap consistent
* Tighten margins to avoid jump
* Tweak padding to make all wells the same height
* BOT-310 lower the weight on the :rrf score for semantic search results
This seems to give a better blend between the :rrf score coming from the semantic search hybrid query and the other
post-search metadata scorers.
* Add with-weights and with-only-semantic-weights test util macros
* Use semantic.tu/with-weights in scoring tests
* Update query tests to use semantic.tu/with-only-semantic-weights
This prevents other metadata scorers from overriding the expected rankings for these tests. The query tests are
largely focused on making sure the basic machinery of semantic queries are working. Tests exercising individual
scorers are in scoring_test.clj
* Add tests for :official_collection and :verified "premium" scorers
Fixes#35561
We currently don't allow filtering on null foreign key values, but that is a reasonable thing to want to do. This extends the foreign key drill to support null values.
🤖 Generated with [Claude Code](https://claude.ai/code)
Co-authored-by: Claude Code[bot] <noreply@anthropic.com>
Co-authored-by: William Zimrin <william@metabase.com>
* Unpolished WIP hackathon code for additional driver support
* fix kondo
* add transforms/table featuer for supported drivers
* fix transforms for dbs with no schema
* rename transform datasets to prevent foreign key overlap
* Add clickhouse support
* fix clickhouse tests
* fix redshift tests
* fix bigquery failures
* kondo and filter tests by feature
* fix transform dataset for bigquery
* fix sqlserver transforms with joins
* fix clickhouse order by when no non nullable field is available
* add more complex test with join, filter, aggregation and ordering
* use different metadata provider
* filter for left join feature in test
* improve sqlserver compile-transform
* done use legacy mbql in test, refactor normalize-name
* clean up
* add sqlserver error test
* Squashed commit of the following:
commit a11b602e19
Author: Romeo Van Snick <romeo@romeovansnick.be>
Date: Thu Aug 21 17:44:13 2025 +1000
Add message when a new transform will be created
commit 2cfdbfaba3
Author: Romeo Van Snick <romeo@romeovansnick.be>
Date: Thu Aug 21 17:37:36 2025 +1000
Allow creating values in schema form select
commit 17ff4dcb3f
Author: Romeo Van Snick <romeo@romeovansnick.be>
Date: Thu Aug 21 16:51:13 2025 +1000
Allow selecting schemas always, not only when there are schemas
* create schemas
* fix redshift
* add snowflake
* add sqlserver
* add clickhouse
* add bigquery
* Add test
* add default impl
* fix redshift tests
* Move SchemaFormSelect to components folder
* Allow creating schemas in the target update modal
* Allow initial value in SchemaFormSelect
* Only render schema field when the database supports :schemas
* Do not render new schema item if value is empty
* Add unit tests for SchemaFormSelect
* Add e2e tests for schema creation
* check schema exists first
* fix sql jdbc schema exists and add create schema permissions test
* update changelog
* parse query and add into for sqlserver
* add type hints for linter
* Address comments
* realize bigquery execute-raw-queries
* Disable target schema link if it does not exists (yet)
* Add tooltip when schema is new
* Use hydrated table from transform
* Add e2e tests
* Allow existing schema link to render as enabled even when table does not exist yet
* Do not add default to databaseId
* Add tests for existing schema with new table
* Fix e2e test
* address comments
* rename connection-details to conn-spec
* Compare against null not empty string
* use find first
* Await request to avoid flakiness in hover
---------
Co-authored-by: Timothy Dean <timothy@timothydean.dev>
Co-authored-by: Romeo Van Snick <romeo@romeovansnick.be>
* Add last_run, next_run and tag id filters
Closes QUE-2156
* Placate kondo
* Test with scheduled jobs and filtering by tags
* Fix next_run_start_time test
* Return run timestamps as timezone localized strings
* Handle Zulu
* Prepare job list table for next_run and transforms
* Add JobListParams type and use it in getJobListUrl
* Add new parameters to listTransformJobs handler
* Parse params and pass them down on job list page
* Add filters to job list page
* Add tooltipPosition prop to Icon
* Add TimezoneIndicator component
* Allow rendering React nodes in table headers
* Add timezone indicators to job list and transform list
* Add timezoneToUTCOffset helper
* Add timezone to cron explanation
* Format timestamps as ISO-8601 strings with timezone
* Add next run timestamp
* Hide transform count in list for now
* Add next_run to TransformJob
* Add timezone indicators to run list
* Avoid wrapping timestamps in run list
* Move shared widgets to transforms/components
* Allow passing available shortcuts to filter picker
* Disable shortcuts that are in the past for Next run, which is always in the future
* Add e2e test for filters
* Add UTC offset in tests
* Report run timestamps in UTC and expose system timezone
The system timezone is exposed in /api/session/properties as "system-timezone".
* Expose system-timezone to admins only
* Fix setting docstring
* Fix serdes baseline-comparison-test
* Make transform_job API return run timestamps in UTC
* Placate kondo
* Timezone
* Use system-timezone instead of report-timezone
* Add tests
* Fix test string
---------
Co-authored-by: Romeo Van Snick <romeo@romeovansnick.be>
Co-authored-by: Alexander Polyankin <alexander.polyankin@metabase.com>
The Store API's OpenAPI description (OAD) can change over time, but
Metabase's Store client would not notice. For example, the client
would fail a HTTP request, not because the server rejected it, but
because the client "believes" it needs to send a field that the server
no longer requires.
Instead of caching the client, and thus OAD, for the whole lifetime of
the Metabase instance, cache it only for 5min.
Fixes: 1dd56e5f14
References: https://metaboat.slack.com/archives/C010ZSXQY87/p1757002462314229
* add base color rule to stylelint
* allow in colors file
* lint in CI
* better highlighting
* css fixes
* fix all improper color use
* Update frontend/lint/stylelint-rules/no-base-colors.js
Co-authored-by: Phoomparin Mano <poom@metabase.com>
* Update Loki Snapshots
* fix color name
* Update Loki Snapshots
---------
Co-authored-by: Phoomparin Mano <poom@metabase.com>
Co-authored-by: Metabase Automation <github-automation@metabase.com>
* Add transform dependencies and job transforms endpoints
* Add API handler for /api/ee/transform/:id/dependencies
* Add API handler for /api/ee/transform-job/:id/transforms
* Add description to TitleSection
* Add Dependencies section to transform page
* Add transform order to job page
* Invalidate transform jobs when adding tags to transforms
* Add empty state for job transforms
* Add e2e test for transform and job dependencies
* Fix type for job dependencies component
* transform ordering is well defined
* don't sort whole list when we just need the first element
* make new function private
* Use neg?
* Remove unused require
* use a sorted map instead of scanning for the earliest transform each time
* extend docstring
* use ->
* update docstring again
Co-authored-by: metamben <103100869+metamben@users.noreply.github.com>
---------
Co-authored-by: Romeo Van Snick <romeo@romeovansnick.be>
Co-authored-by: William Zimrin <william@metabase.com>
* Update embedded analytics JS docs to include browser
* small update
* new appearance setting
* Update docs/embedding/embedded-analytics-js.md
Co-authored-by: Jeff Bruemmer <jeff.bruemmer@gmail.com>
* Apply suggestion from @jeff-bruemmer
Co-authored-by: Jeff Bruemmer <jeff.bruemmer@gmail.com>
* Apply suggestion from @jeff-bruemmer
Co-authored-by: Jeff Bruemmer <jeff.bruemmer@gmail.com>
* update on the browser
---------
Co-authored-by: Jeff Bruemmer <jeff.bruemmer@gmail.com>
* Update pgvector to pg17 in semantic search docker-compose.yml
This matches the version that will be used in prod and staging deployments.
* Update to pg17 in semantic-search.yml CI workflow
* Remove enable-ssl-tests env var from semantic-search.yml workflow
* Move validate.js to ts and remove unused validators
* Remove any and unknown for typing parameter values
* Use ParameterValuesMap where possible
* Fix parameter-parsing
* Fix tests
* Fix cast to empty string
metabase.session.models.session-test/send-email-on-first-login-from-new-device-test
test is failing because sometimes there's an extra email in there.
metabase-enterprise.impersonation.driver-test/nested-do-with-connection-with-options-test
is failing because it deadlocks all the app-db connections
* Also skip the :bookmarked scorer on mysql
MySQL does not like the CASTs in the search-index-query
* Fix appdb-available-with-semantic test for mysql appdb
When the appdb is :mysql, we should fallback to the in-place engine rather than appdb
* Move body of search.engine/results :search.engine/in-place into a func for easy mocking
* Fix api-engine-switching-test for mysql appdb
* Fix test-semantic-search-fallback-supplementation and test-semantic-search-max-combined-results-limit for mysql appdb
* Update semantic-search CI job to run against all supported appdbs
* Implement last_run_... filters for transform runs
* Change filters to start_time, end_time, run_method
* Use timestamp literals instead of strings in test
* Add start_time, end_time and run_method to transform run query params in api
* Parse start_time, end_time and run_method params on run list page
* Allow custom react elements in displayValue prop
* Add filters for start_time and end_time to run list page
* Allow filtering on multiple run methods
* Add run method filter widget
* Close time filter when changing value
* Add e2e test for start at, end at and run method filters
* Allow multiple values for run_method filter
* Add test with start_time=2025-08-22~2025-08-23
* Fix e2e test
* Rename the run_method parameter to run_methods
* Remove "local" timestamps
* Return transform run timestamps in the system timezone
* Stop expecting a last_run with a nil value
* Rename run_method to run_methods to match api
---------
Co-authored-by: Romeo Van Snick <romeo@romeovansnick.be>
Co-authored-by: Alexander Polyankin <alexander.polyankin@metabase.com>
* Create standalone add DB page
* Use full page view for editing db connection
* Fix title on edit database page
* fix dataspec cypress test
* fix missing formLocation prop
* fix add initial data cypress test
* Fix add db test not waiting for db connected
* Fix default schedule cypress test
* Remove unnecessary nullish coalescing and route
* Extract common logic to use-database-connection hook
* Adjust Add DB page width
* Adjust padding for smaller devices
* Use FormLocation type in prop
* Add repair-index! API with temp table mechanism for detecting deletions
* Fix up tests, bug fixes
* Explicitly disable reindex! and reset-tracking
* Add repair index task, and cleanup job in case there are orphans
* fix tests
* Test - comment out new tests to see if it fixes CI failures
* switch with-index! macro to use init! instead of reindex
* try again to fix with-index
* fix kondo
* ensure drop table table names get quoted
* switch to using gate table for the anti-join
* add missing assertion in test
* add more error handling in find-lost-deletes, reorganize a function
* make sure index cleanup task is initialized
* make sure index repair task is initialized
* Fix time-interval filters
Reported on stats that the examples dashboard was breaking when you used
some relative timestamps. There it was a bit cryptic with
```
class java.lang.Integer cannot be cast to class
clojure.lang.IPersistentMap (java.lang.Integer is in module java.base of
loader 'bootstrap'; clojure.lang.IPersistentMap is in unnamed module of
loader 'app')
```
Locally in dev, we get:
```
Error processing query: Invalid output: ["invalid tuple size 4, expected 5, got: [:time-interval [:field {:base-type :type/DateTime, :lib/uuid \"69655229-95a0-4bfb-9d3a-7675b1111ec4\"} 13] :current :day]"]
```
And it's a failure to have the time-interval options
before:
`[:time-interval field-clause :last (keyword unit)]`
after:
`[:time-interval interval-options field-clause :last (keyword unit)]`
spec:
```
(mbql-clause/define-mbql-clause :time-interval :- :type/Boolean
;; TODO -- we should probably further constrain this so you can't do weird stuff like
;;
;; [:time-interval {} <time> :current :year]
;;
;; using units that don't agree with the expr type
[:tuple
[:= {:decode/normalize common/normalize-keyword} :time-interval]
[:merge ::common/options ::time-interval-options] ;; <---- missing bits
#_expr [:ref ::expression/temporal]
#_n [:multi
{:dispatch (some-fn keyword? string?)}
[true [:enum {:decode/normalize common/normalize-keyword} :current :last :next]]
;; I guess there's no reason you shouldn't be able to do something like 1 + 2 in here
[false [:ref ::expression/integer]]]
#_unit [:ref ::temporal-bucketing/unit.date-time.interval]])
```
Switching them to lib calls and then i noticed there were no tests.
With fix:
```clojure
dates=> (date-string->filter
"lastmonth"
[:field {:lib/uuid "00000000-0000-0000-0000-000000000000", :base-type :type/DateTime} "field"])
[:time-interval
{:lib/uuid "bbca7a83-feb9-4259-a4db-bbb79687eb4e"}
[:field
{:lib/uuid "00000000-0000-0000-0000-000000000000",
:base-type :type/DateTime}
"field"]
:last
:month]
```
In a 56.3.4 repl:
```
user=> (metabase.driver.common.parameters.dates/date-string->filter "lastmonth" [:field {:lib/uuid "00000000-0000-0000-0000-000000000000", :base-type :type/DateTime} "field"])
[:time-interval [:field {:lib/uuid "00000000-0000-0000-0000-000000000000", :base-type :type/DateTime} "field"] :last :month]
user=> (pp)
[:time-interval
[:field {:lib/uuid "00000000-0000-0000-0000-000000000000", :base-type :type/DateTime} "field"]
:last
:month]
nil
```
Note the missing options in the second position
* test e2e dashboard filters
takes a sampling of a few different parameter types and different
parsers. At one point i was doing something like:
```clojure
(concat (for [unit [:day :week :month :quarter :year]
relative [:this :last]]
(str (name relative) (name unit)))
(for [unit '[days weeks months quarters years]
include? [true false]
relative '[past next]]
(str relative "3" (name unit) (when include? "~"))))
```
But that seems overkill. This test is kinda seeing if the query
processor throws an error (or at least that's where most errors
happen). So those checks can be more exhaustive and against some
in-memory metadata providers
* two fixes, compaction and when to run
Test logs are super, duper awesome. We upload them on failure since we
don't presumably need to see healthy logs.
There are two issues:
1. We upload them on `failure()`, but timeouts don't seem to trigger
this condition. So now upload on `always() && !success()` Why not just
`!success()`? An LLM suggests that timeouts are a bit special and go
into another mode. The always() will run and then we qualify it. Let's
see.
2. The logs are all on a single line.
We have a few axes here: compact and eventEol.
I have three files with these. If you want to grep for logs during a
particular test, here's what you get:
Compact: (current behavior)
```
❯ ag "metabase.util.queue-test/result-listener-test" logs/test-log.json.compact
<not reproducing here, because it's the whole file>
```
compact false
```
❯ ag "metabase.util.queue-test/result-listener-test" logs/test-log.json.not-compact
251: "mb-test" : "metabase.util.queue-test/result-listener-test"
268: "mb-test" : "metabase.util.queue-test/result-listener-test"
414: "mb-test" : "metabase.util.queue-test/result-listener-test"
```
It's _too_ expanded.
```
{
"instant" : {
"epochSecond" : 1756844736,
"nanoOfSecond" : 357005000
},
"thread" : "main",
"level" : "INFO",
"loggerName" : "metabase.util",
"message" : "Maximum memory available to JVM: 8.0 GB",
"contextMap" : { },
"endOfBatch" : false,
"loggerFqcn" : "org.apache.logging.log4j.spi.AbstractLogger",
"threadId" : 1,
"threadPriority" : 5
}
{
"instant" : {
"epochSecond" : 1756844739,
"nanoOfSecond" : 802772000
},
```
Compact true, eventEol (this PR sets this)
```
❯ ag "metabase.util.queue-test/result-listener-test" logs/test-log.json
17:{"instant":{"epochSecond":1756844991,"nanoOfSecond":537601000},"thread":"main","level":"INFO","loggerName":"metabase.util.queue","message":"Starting listener \u001B[32mtest-result-listener\u001B[0m with 1 threads 🎧","contextMap":{"mb-test":"metabase.util.queue-test/result-listener-test"},"endOfBatch":false,"loggerFqcn":"org.apache.logging.log4j.spi.AbstractLogger","threadId":1,"threadPriority":5}
18:{"instant":{"epochSecond":1756844991,"nanoOfSecond":572361000},"thread":"main","level":"INFO","loggerName":"metabase.util.queue","message":"Stopping listener test-result-listener...","contextMap":{"mb-test":"metabase.util.queue-test/result-listener-test"},"endOfBatch":false,"loggerFqcn":"org.apache.logging.log4j.spi.AbstractLogger","threadId":1,"threadPriority":5}
20:{"instant":{"epochSecond":1756844991,"nanoOfSecond":585407000},"thread":"main","level":"INFO","loggerName":"metabase.util.queue","message":"Stopping listener test-result-listener...done","contextMap":{"mb-test":"metabase.util.queue-test/result-listener-test"},"endOfBatch":false,"loggerFqcn":"org.apache.logging.log4j.spi.AbstractLogger","threadId":1,"threadPriority":5}
```
You get the whole log lines. super helpful. pipe these through a
processing pipeline, use jq, etc.
Also, these files are ~817MB when running in a test suite. I'm doing these with a single namespace
```
❯ clj -X:dev:test :only metabase.util.queue-test
❯ ls -lh logs
13K Sep 2 15:29 test-log.json
13K Sep 2 15:24 test-log.json.compact
16K Sep 2 15:26 test-log.json.not-compact
```
* stacktraces when we don't get returned connections
* use new hawk version. waiting on release
* quick checkin on CI stuff
* load file to ignore vars
```clojure
;; json file with a list of vars
test-runner=> (println (slurp "/tmp/config.json"))
{
"ignored": {
"vars": [
"metabase.util.queue-test/bounded-transfer-queue-test",
"metabase.util.queue-test/take-batch-test"
]
}
}
nil
test-runner=> (find-tests (build-final-options {:ci-config-file "/tmp/config.json"
:only 'metabase.util.queue-test}))
Running tests with options {:namespace-pattern #"^(?:(?:metabase.*)|(?:hooks\..*))", :exclude-directories [".clj-kondo/src" "classes" "dev" "enterprise/backend/src" "local" "resources" "resources-ee" "src" "target" "test_config" "test_resources"], :test-warn-time 3000, :ignored {:vars #{"metabase.util.queue-test/bounded-transfer-queue-test" "metabase.util.queue-test/take-batch-test"}}, :ci-config-file "/tmp/config.json", :only metabase.util.queue-test}
Running tests in metabase.util.queue-test
Finding tests took 1000.0 µs.
Running 4 tests
(#'metabase.util.queue-test/non-delayed-take-batch-test
#'metabase.util.queue-test/result-listener-test
#'metabase.util.queue-test/listener-handler-test
#'metabase.util.queue-test/multithreaded-listener-test)
;; only found 4 tests
;; no ignored vars
test-runner=> (println (slurp "/tmp/empty-config.json"))
{
"ignored": {
"vars": []
}
}
nil
;; finds 6 tests
test-runner=> (find-tests (build-final-options {:ci-config-file "/tmp/empty-config.json"
:only 'metabase.util.queue-test}))
Running tests with options {:namespace-pattern #"^(?:(?:metabase.*)|(?:hooks\..*))", :exclude-directories [".clj-kondo/src" "classes" "dev" "enterprise/backend/src" "local" "resources" "resources-ee" "src" "target" "test_config" "test_resources"], :test-warn-time 3000, :ignored {}, :ci-config-file "/tmp/empty-config.json", :only metabase.util.queue-test}
Running tests in metabase.util.queue-test
Finding tests took 0.0 ns.
Running 6 tests
(#'metabase.util.queue-test/non-delayed-take-batch-test
#'metabase.util.queue-test/take-batch-test
#'metabase.util.queue-test/result-listener-test
#'metabase.util.queue-test/listener-handler-test
#'metabase.util.queue-test/bounded-transfer-queue-test
#'metabase.util.queue-test/multithreaded-listener-test)
```
* Fetch ci-test-config.json from master on each test run
This affects all BE tests, driver and app-db. Don't pass an explicit arg
and just let the default pick it up.
```bash
❯ clj -X:dev:test :only metabase.util.queue-test
Running tests with options {:mode :cli/local, :namespace-pattern #"^(?:(?:metabase.*)|(?:hooks\..*))", :exclude-directories [".clj-kondo/src" "classes" "dev" "enterprise/backend/src" "local" "resources" "resources-ee" "src" "target" "test_config" "test_resources"], :test-warn-time 3000, :ignored {}, :exclude-tags [:metabot-v3/e2e], :only metabase.util.queue-test}
Running tests in metabase.util.queue-test
Finding tests took 7.5 s.
Running 6 tests
Running before-run hooks...
LONG TEST in metabase.util.queue-test/take-batch-test
Test took 4.720 seconds seconds to run
LONG TEST in metabase.util.queue-test/bounded-transfer-queue-test
Test took 7.518 seconds seconds to run
6/6 100% [==================================================] ETA: 00:00
Ran 6 tests in 12.421 seconds
53 assertions, 0 failures, 0 errors.
{:test 6, :pass 53, :fail 0, :error 0, :type :summary, :duration 12421.435583, :single-threaded 6}
Ran 0 tests in parallel, 6 single-threaded.
Finding and running tests took 20.0 s.
All tests passed.
Running after-run hooks...
Running QP tests against these drivers: #{:h2}
metabase on quick-CI-test-disabling [$!?] via ☕ v21.0.2 on ☁️ metabase-query took 48s
❯ clj -X:dev:test :only metabase.util.queue-test :ci-config-file '"/tmp/config.json"'
Loading CI config file from /tmp/config.json # <------ new output
Running tests with options {:mode :cli/local, :namespace-pattern #"^(?:(?:metabase.*)|(?:hooks\..*))", :exclude-directories [".clj-kondo/src" "classes" "dev" "enterprise/backend/src" "local" "resources" "resources-ee" "src" "target" "test_config" "test_resources"], :test-warn-time 3000, :ignored {:vars #{"metabase.util.queue-test/bounded-transfer-queue-test" "metabase.util.queue-test/take-batch-test"}}, :exclude-tags [:metabot-v3/e2e], :only metabase.util.queue-test, :ci-config-file "/tmp/config.json"}
Running tests in metabase.util.queue-test
Finding tests took 6.9 s.
Running 4 tests
Running before-run hooks...
4/4 100% [==================================================] ETA: 00:00
Ran 4 tests in 0.133 seconds
42 assertions, 0 failures, 0 errors.
{:test 4, :pass 42, :fail 0, :error 0, :type :summary, :duration 133.16525, :single-threaded 4}
Ran 0 tests in parallel, 4 single-threaded.
Finding and running tests took 7.1 s.
All tests passed.
Running after-run hooks...
Running QP tests against these drivers: #{:h2}
```
Note that it runs 4 tests instead of 6. The config passed here was:
``javascript
{
"version": "1.0.0",
"ignored": {
"vars": [
"metabase.util.queue-test/bounded-transfer-queue-test",
"metabase.util.queue-test/take-batch-test"
]
},
"metadata": {
"last_updated": "2025-08-02",
"updated_by": "engineering-team",
"reason": "Testing CI config implementation"
}
}
```
* empty ci-test-config
this will be the file when this merges
* print config when it has vars that we are ignoring
* PR review feedback
typo in example json
set continue-on-error: true. if network fails, let's not fail our
build. it's just a little file. carry on soldier. worst case scenario a
test known to flake might flake. might not.
* Add repro for metabase#52872
* Wrap long titles without spaces in saved questions
* Add repro for metabase#52872 for dashboards
* Fix width of dashboard with long name
* Use max-width instead of width
- Measure time properly
- Make endpoint config optional, so it's skipped in staging
- Make auth key config option, so it's skipped in staging
- Skip dev-only settings defaults in staging
- Add prefix, so staging permissions work
- Use short names for Java classes (Cursive lint)
- Remove trivial comments, answer some others
- Fix test-id that broke malli
* etc: kickoff comments
* get quotas statuses from token-check (#61560)
* get quotas statuses from token-check
* Check quota and estimated-row-count of dbs before replication
* Evict token-check cache to get updated quotas
* Turn the MB pg replication button into a modal (#61701)
Closes: https://linear.app/metabase/issue/CLO-4308/turn-the-mb-pg-replication-button-into-a-modal
* Fix schema-filters-patterns split
* split token-check row estimates counts (#61783)
* Split token-check row estimates counts
* add preview endpoint
* Align replication schema-filters with HM
Fixes: 805f4cafcf
Depends-on: https://github.com/metabase/harbormaster/pull/6445
* preview endpoint changed
* temp frontend modal
* After pairing with Raimon
* Fix lint and fmt errors
* pg replication modal (#61913)
* Success modal on starting replication
* Visual improvement to DatabaseReplicationForm
* made non-pk-tables list prettier
---------
Co-authored-by: Dennis Schridde <dennis@metabase.com>
* Run prettier and linter on DatabaseReplicationForm
* Turn DatabaseReplicationModal into a multi step thing and add backend API to support this
* Workaround broken dirty bit
* Call setFieldValue("schemaFields", "") when schemaSelect=all
* fix enabling of the replication button
* UI updates
* UI updates
* removed package-lock file
* Don't wait for replication to become "active" in "setting up" modal
The progression from "initializing" to "active" can take arbitrary
amounts of time. Instead we should show the "setting up" modal while
creating the replication, which is synchronous and has an upper bound
of 40s.
Fixes: 6f2ba1852e
References: https://metaboat.slack.com/archives/C079ZS06ERJ/p1754940482733049
* Loading for preview method
* Error handling
* Modal padding and title updates
* Remove info about future email from form
* Misc
* Misc
* DatabaseReplicationStatusInfoProps cleanup
* Misc
* Status modal content padding
* Modal properties
* fix: throw harbormaster call error body in ex-data
* Show which tables are excluded from replication because of owner mismatch
Closes: https://linear.app/metabase/issue/CLO-4346/preview-should-also-include-schemas-that-cant-be-synced-due-to-owner
* Align text of replication modal more with Figma
* UI updates 1
* Replication stats spacing updates
* Misc
* Tables listing icons
* Alignment updates
* Layout updates for tables listing
* Misc
* Colours updates
* List updates
* Misc
* Misc text alignment
* Misc
* Scrollable tables list
* Fix tables listing
* Comma separated DB WIP
* Misc
* Fix margins
* Ui updates
* More on setting up and success layout
* Test for spinner
* More on spinner
* Colour update
* Box shadow for spinner
* Spinner size updated
* Spinner padding updated
* Spinner added to setting up step
* Misc
* Sizes updated
* Fix for unit tests
* Debounce for schema filters
* Schema filters onChange
* Remove loading in button is no schema filters
* Loader for preview
* Misc
* feat: check replication table ownership
Same as how HM does it.
* Moved laoder next to the submit button
* Make DatabaseReplicationForm look more "loading..." during preview
* Remove setFieldValue hack
Was obsoleted by 9cdbeaec7a
* Also show skeletons when data is not fresh (i.e. when loading)
* Fix linter: Change set IDs are not in order
* Build uberjar container also when pushing a PR branch
Before it would only build when the `build-docker-uberjar` label was
added. So to build the latest branch head you'd have to remove and
re-add the label.
Fixes: 22507a963a
* Link to postgres docs
* SUrface "incorrect" errors from HM response to connection creation
* Misc
* Modal layout
* Error modal UI updates
* Misc
* More on layout + alignment fixed
* More on error layout
* Fix for modal closing
* rfct: use pg conn preview
* revert owner sync
* rfct: use existing schema filter format
* lint: fix fe
* feat: also show tables that will be replicated
* test: update be tests
* rfct: minor renames for consistency
* fix: cannot replicate 0 tables
* style: fix
* fix: copy
* test: update body prop
* test: add form tests
* fix: progress indicator on bottom
* fix: card styles
* fix: error message when no tables to replicate
* fix: easy stuff from jakub's review
* fix: error state for preview too
* claude: fast FE unit tests
* rfct: use fetchMock in FE test
* fix: remove small error step text
* fix: mocal close should reset setup step
* fix: translate noSyncReason
* fix: catch invalid schema pattern as 400
* rfct: use plain fn for onSubmit
* rfct: raise inline fn
* rfct: use correct token-check alias
* rfct: remove unused preview fields
* rfct: raise free-quota fn
* fix: throw on tunnel or certs
* rfct: remove silly when
* rfct: testable preview-replication and errors map
* test: granular fe error tests
* test: fix BE tests
* test: preview-replication unit tests
* fix: test redef name
* rfct: style and db as dbname
---------
Co-authored-by: Raimon Grau <raimon@metabase.com>
Co-authored-by: Dennis Schridde <63082+devurandom@users.noreply.github.com>
Co-authored-by: Dennis Schridde <dennis@metabase.com>
Co-authored-by: Raimon Grau <raimonster@gmail.com>
Co-authored-by: Arkadi Yaroslavtsev <arkadi@metabase.com>
Co-authored-by: losrebellos <ben@benoitvinay.com>
* Fix question download filenames to use question name instead of generic 'query_result'
Resolves: https://github.com/metabase/metabase/issues/61708
Previously, when downloading questions from the SDK (and public links), files were
named with a generic "query_result_{timestamp}.{format}" pattern even when the
question had a meaningful name.
This fix ensures that:
- Questions with valid names use the slugified question name as filename
- Questions with nil/empty/blank names fall back to "question" instead of "query_result"
- Both regular card downloads (/api/card/:id/query/:format) and public link downloads are fixed
Changes:
- Modified process-query-for-card-default-run-fn in query_processor/card.clj
- Modified process-query-for-card-with-id-run-fn in public_sharing/api.clj
- Added proper fallback logic using str/blank? to handle edge cases
Expected behavior:
- "My Awesome Question" → my_awesome_question_{timestamp}.csv
- nil/empty name → question_{timestamp}.csv
🤖 Generated with [Claude Code](https://claude.ai/code)
Co-Authored-By: Claude <noreply@anthropic.com>
* Cleanup and added tests
* Make whitespace linter happy
* Fix silly oversight
* Set warn-on-reflection
* Fix issues
---------
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: Timothy Dean <timothy@timothydean.dev>
- Automatically updates def transform(...) signature when aliases are modified
- Updates docstring Args section with aligned descriptions and table names
- Adds reset button to restore default slugified aliases
- Validates alias input to only allow valid Python identifiers
- Handles multi-line signatures for many parameters (>80 chars)
- Shows visual distinction between custom and auto-generated aliases
- Fixes code editor layout to prevent overlap with Run button
* Hide CSV panel with no meaningful uploadable dbs
* Hide add data when no meaningful actions can be performed
* Simplify conditional for shouldShowUploadPanel test
Co-authored-by: Anderson <ands.alves.nunes@gmail.com>
* Show CSV tab also when user can upload data
User with upload permissions, but without admin rights, cannot see uploadable dbs in the api response
* Add e2e test for hidden add data for regular users without upload permissions
---------
Co-authored-by: Anderson <ands.alves.nunes@gmail.com>
* Show a special error screen when you don't have permissions to run the query
* Show a special error screen when you don't have permissions to run the query
* Fix tests
* Unpolished WIP hackathon code for additional driver support
* fix kondo
* add transforms/table featuer for supported drivers
* fix transforms for dbs with no schema
* rename transform datasets to prevent foreign key overlap
* Add clickhouse support
* fix clickhouse tests
* fix redshift tests
* fix bigquery failures
* kondo and filter tests by feature
* fix transform dataset for bigquery
* fix sqlserver transforms with joins
* fix clickhouse order by when no non nullable field is available
* add more complex test with join, filter, aggregation and ordering
* use different metadata provider
* filter for left join feature in test
* improve sqlserver compile-transform
* done use legacy mbql in test, refactor normalize-name
* clean up
* add sqlserver error test
* parse query and add into for sqlserver
* add type hints for linter
* Address comments
---------
Co-authored-by: Timothy Dean <timothy@timothydean.dev>
Previously, if you had city, state, latitude, and longitude breakouts, you'd get zoom in (city), zoom in (state), zoom in (lat/lon for lat), zoom in (default binning for lat), zoom in (lat/lon for lon), and zoom in (default binning for lon). Those would all be called "zoom in", so you'd just see half a dozen drills with the same title. This hides the default binning drills if we are showing lat/lon, and gives the assorted geographic zoom-ins different titles.
* `wrap-value-literals` => Lib
* Mostly working nicely
* Clean up other middleware + test fix
* Finish it up
* Code cleanup
* Kondo
* Fix test 🔧
* Work around https://github.com/clj-kondo/clj-kondo/issues/2610
* Test fixes; PR feedback
* Fix walk recursion into `:case` and `:if`
* code cleanup
* Oracle test fix 🔧
* Another test fix 🔧
Fixes#62377.
Correctly get a connection to the database while checking the ClickHouse version
(used to check if it supports `:connection-impersonation`). The driver was using the
internal, raw form of getting a connection, which didn't apply the SSH tunnel properly.
* Boilerplate
* Add list view toggle control to ViewFooter
* Remove processRemappedColumns, add layout
* Navigation
* Back button,breadcrumbs
* TODO
* Layout
* Layout
* Refactor
* Column detection logic
* Inline
* Fix
* Extract Breadcrumb
* Fix
* Entity name in nav
* Condition
* Fix
* Header
* Style
* Remove
* Do not render header if no column can be used there
* Body
* Add empty list viz
* Update VizStyleToggle to use list display type
* Sidebar
* Style
* Add simple list view
* Add virtualization for ListViz view. Optimize rendering
* Copy
* Hover style
* Add object detail modal opening on list item click
* Remove back button
* Overflows
* a11y
* Redundant css
* Fix
* Fix rendering
* Style links
* Cleanup
* FK pill
* Render images
* Prevent showing old images while new are loading
* Take currency_in_header into account
* Extract `ColumnHeader`
* Cleanup
* Fix spacing, colors
* Refactor relationships
* Update PK drill
* PK drill
* Fix
* Fix
* Improve layout of list items
* Center cells content
* Add sorting
* Fix virtualized behavior
* Fix copy if row name is not available
* Max height for values
* Update ViewStyleToggle rendering rule to fix incorrect list view icon
* Styling fixes
* Make whole list view scrollable
* Hide truncated data warning for list view
* Fix PK drilll
* Don't memo formatted rows
* Fix virtualization
* Use drills for sorting
* Fix virtualization flickering
* Fix drills
* Disable viz settings for list view
* List view visual enhancements (#62124)
* spaecing and font styling
* add entity_type icons to listitems
* display 4 columns instead of 5
* change icon for transaction tables
* add circle for boolean fields
* add badge style to category columns
* add colors to badges, incl Score cols
* Save list questions as tables
* Footer
* Padding and scrolling
* Styling
* Styling
* Remove icon
* Fix useObjectDetail call for ListView
* Fix list view columns alignment
* Move detail view breadcrumbs to AppBar
* Style breadcrumbs
* Fix scrolling
* Show icon if avatar is missing
* Fix react key warning
* Align breadcrumbs with detail view page
* Remove arrow navigation and footer
* Styling
* Error handling
* Styling
* Make Detail View breadcrumbs preserve card settings
* Make DetailView breadcrumbs return to previous ListView state
* Add company, currency, and receipt icons (#62196)
* add company currency and receipt icons
* add new icons to IconName and Icons obj
* Remove duplicate 'currency' icon declaration
---------
Co-authored-by: Stas Gavrylov <stasgavrylov@gmail.com>
* Update icons for TransactionTable and CompanyTable
* Make titleColumn always present. Update categoryValue styling
* ListView: Extract rendering logic to ColumnValue component
* Add tooltips for large column values
* Make list item grid always have at list 1 column
* Refactor DatasetEditorInner to typescript
* Rename DatasetEditor 'metadata' mode to 'columns'. Add new 'Metadata' tab for list view
* List view customization styling fixes (#62546)
* fix font sizes and spacing
* styling on ReorderableTagsInput
* make the viewsidebar 400px
* Make ReorderableTagsInput dropdown behave like multi-select
* Fix saving. Remove ViewStyleToggle
* Refactor List view to proper viz settings. Fix bugs. Remove TabHint popup
* Remove TabHintToast completely
* Fix container highlight when dragging between inputs
* Fix column settings in ListViz. Fix eslint
* Remove legacy DetailView code
* PR Cleanup
* Fix settings in list item preview
* Fix sorting. Replace custom column items with ColumnItem
* Fix saving models with list display type
* Fix incorrect condition to save list view models as tables
* Fix incorrect columns param in ListViewColumnsSidebar
* Fix list entity icon hover
* Tests – Models List View (#62836)
* Add storybook for ReorderableTagsInput
* wip
* Unit tests for ReorderableTagsInput
* Fix models-metadata related e2e tests
* e2e tests for List View
* Fix eslint
* Unit tests fixes
---------
Co-authored-by: Kamil Mielnik <kamil@kamilmielnik.com>
Co-authored-by: Anton Kulyk <kuliks.anton@gmail.com>
Co-authored-by: Maz Ameli <maz@metabase.com>
* Populate provider_name field on startup health check
* Serve provider patterns from BE side
* cleanup tests and expand clojure host provider detection
* Fix provider detection test
* Fix regexes for supabase and timescale
* adjust module api exports to standards
* mark private function ins provider-detection namespace
* extend provider values to null and empty, skip backfilling on empty provider_name
* Don't hardcode postgres in health check provider detection
* Make providers api dynamic and ready to support more engines
* Sort lines providers and engines
* Move hostname field tests to a test folder
* memoize detected provider
* Fix for tests for private functions
* Fix import of DatabaseProviderName type
* Use update function for setting provider_name on health check
* Fix typo in test case
* Remove duplicate entry
* simplify detect-provider-from-database call
* Move postgres extra-info to postgres driver module
* Simplify provider update check on startup, use null value for unidentified
* match provider also using engine
* bit of cleanup
- just move the regexes into the postgres driver namespace
- remove regexes and turn them into strings since most databases will put
them in yaml
- just test the public api
- simplify all the host and details checks. Database is a very standard
shape on the backend
* allowing setting null to provider match is null, cleanup unused imports
* move postgres tests to postgres driver; add a few tests for yaml
# Conflicts:
# test/metabase/driver/postgres_test.clj
---------
Co-authored-by: dan sutton <dan@dpsutton.com>
* Potential fix for code scanning alert no. 512: Insecure randomness
Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com>
* Handle max = 1 case
---------
Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com>
* Do not apply update-keys to state
* Add comment documenting the intentional order
* remove recursive snake case update
* apply fix to both streaming and non streaming
* adding kebab->snake helpers that ignore camelcase
* Fix kondo
---------
Co-authored-by: Thomas Schmidt <thomas@metabase.com>
This is based on this discussion with Marcos: https://metaboat.slack.com/archives/C01R1JY03J4/p1756173436062039
It appears that we do set a reporting time zone for SQL queries, but it did not have an effect in my own testing on my local MariaDB. However, it works just fine when running the "official" Docker version of MariaDB.
* allow specifying athena hostname
* use hostname
* fix params
* add test and clarify hostname field
* update helper text
* Apply suggestion from @jeff-bruemmer
Co-authored-by: Jeff Bruemmer <jeff.bruemmer@gmail.com>
* update hostname field
* apply suggestion
* improve test
---------
Co-authored-by: Jeff Bruemmer <jeff.bruemmer@gmail.com>
* Add dead letter queue for the semantic search indexer
The DLQ system handles failed semantic search indexing operations if they are causing the indexer to stall.
- Applies back-off policies for retries (hard coded for now).
- Prevents batch poisoning from blocking indexer progress.
* Add the jitter factor I mistakenly deleted
* Doc fixes
* Slightly better stall recovery log
* Kondo
* Index attempt_at on DLQ table
* Add some more logging
* Init includes DLQ creation
* Fix some rebase collisions
* Formatting
* Add some asserts to show dlq creation during init
* Name test fixed with hashing
* Inline table-name-prefix
* Bump index meta version
* Prefer hex to b64 for table/index names they remain usable unescaped.
* Fix next-dlq-run test in on-indexing-idle
The test was backwards, so the loop only idled when `next-dlq-run` was due, rather than avoiding idle in that case.
* Ensure exit-early-due-to-cold-data? even if we have no last-seen-change
Ensure we still yield back to quartz if more than exit-early-cold-duration seconds have elapsed since the loop start
and we have no last-seen-change. This can happen in cases where no docs were updated since the loop start if our
watermark is already at the end of the gate table.
* Reduce log level to :trace for "nothing to do" case in indexing-step
* Whitespace cleanup
* Reduce ee-search-indexer-exit-early-cold-duration from 30s to 5s
30s is longer than the quartz interval for the indexer job, so we'll just get rescheduled immediately after yielding
control.
* Increase the run-frequency for the SemanticSearchIndexer job from every 10s to every 20s
* Move table-exists? into shared util ns and reuse
* Simplify honeysql qualified col reference in add-entries!
* Fix comment typo: s/indexes/indexer/
* Add missing seq in try-batch!
* Change ee-search-indexer-exit-early-cold-duration back to 30s
This value needs to be higher than the dlq-frequency, otherwise the dlq never runs.
---------
Co-authored-by: Mike Appleby <86076+appleby@users.noreply.github.com>
in dev, the default value was the staging store. When not in dev, the
default value of this url was "false"
```
❯ http get http://localhost:3056/api/session/properties Cookie:"metabase.SESSION=d10a4524-837e-4aaf-84a3-b2f99bc9ecb2" | jq '."store-url"'
"false"
```
In javascript, most of the store url stuff is through
```javascript
export function useStoreUrl(storePath: StorePaths = "") {
return useSelector((state) => getStoreUrl(state, storePath));
}
```
which is based on `getStoreUrl` which is
```
const DEFAULT_STORE_URL = "https://store.metabase.com/";
...
export function getStoreUrl(state: State, path: StorePaths = "") {
try {
const storeUrl = getSetting(state, "store-url");
const url = new URL(path, storeUrl);
return url.toString();
} catch {
return DEFAULT_STORE_URL;
}
}
```
So what happens normally is:
1. in dev, we usually have no setting set for this, so the `when-some`
hits nil. If you do have something there, it is honored.
2. In prod, we read false for the config/is-dev and we use that as the
setting value
```clojure
settings=> (with-redefs [config/is-dev? false]
(store-url-default))
"false"
```
3. There are lots of usages of this on the frontend. But they are all
protected by the try/catch and returning the default store. This is LOAD
BEARING in production.
```javascript
new URL("false", null)
Uncaught TypeError: Failed to construct 'URL': Invalid base URL
```
4. Cloud migration does something slightly different and so fails to get
this load bearing try/catch. It takes a `checkoutUrl` in its props:
```javascript
export const MigrationSuccess = ({
migration,
restartMigration,
isRestarting,
checkoutUrl,
}: MigrationSuccessProps) => {
```
This manually constructs a url:
```javascript
const checkoutUrl = useMemo(() => {
return migration
? `${storeUrl}/checkout?migration-id=${migration.external_id}`
: `${storeUrl}/checkout`;
}, [migration, storeUrl]);
```
And now we don't have the benefit of the `URL(base, extension)` which
would have (dumbly) failed closed. So now we end up with
`false/checkout?...` and then get a "We're a bit lost" locally
So to recap:
we do the right thing in dev, and then production is broken. Most
pathways were just catching this error, one way constructed a url
fragment and therefore was susceptible.
```clojure
settings=> (with-redefs [config/is-dev? false]
(store-url-default))
"false"
settings=> (store-url-default)
"https://store.staging.metabase.com"
```
* Do not pass Sortable attributes when element is not sortable
* Render parameter value trigger as button
* Add repro for metabase#52674
* Use UnstyledButton
* Do not render parameter widget as button when it does not have a popover
* Update Loki Snapshots
* Use UnstyledButton and Box
* Fix syntax error
---------
Co-authored-by: Metabase Automation <github-automation@metabase.com>
This was working for `:base_type`s but not `:effective_type`s.
The `:result_metadata` for a native Mongo query always had
`:effective_type :type/*` even when we correctly inferred the
`:base_type`.
Fixes#62397.
* initial commit
* enable multi db by default for tests
* multi db as default, set inclusion filters for tests
* update introspection tests
* enable routing
* set enable multi db true as default
* use redef for spec
* clean up
* add more generic test
* update tests
* remove unused code
* clean up
* handle encrypted migration
* fix test and migration id
* move created and updated at fields
* more tests
* mark tests parallel
* fix whitespace
* fix athena test
* Update Debian/systemd docs
* Remove references to Debian
* Use H2 database in .env file by default
* Add journalctl command to follow logs
---------
Co-authored-by: Jeff Bruemmer <jeff.bruemmer@gmail.com>
* Refactor appdb-based scorers in the semantic search backend
Refactor the appdb-based scorers in the semantic search backend in preparation for adding the bookmarked scorer in
BOT-312. This refactor has several benefits: (1) makes the appdb-based scorers work more like the existing index-based
scorers; (2) allows sharing the exact same honeysql expressions for the user-recency-expr and model-rank-expr between
the appdb and semantic backends, reducing code dup; (3) makes it easier to add the bookmarked scorer since it will be
able to reuse the appdb backend's honeysql expr for bookmarked scores as well.
* Move user-recency-expr into search.scoring and use it in appdb and semantic backends
* Move model-rank-expr into search.scoring and use it in appdb and semantic backends
* Move mysql check from with-appdb-scores into appdb-scorers
Again in preprartion for adding the appdb-based bookmarked scorer later.
* Fix whitespace
I noticed that
https://github.com/metabase/metabase/pull/62513/files?file-filters%5B%5D=.clj&show-viewed-files=true
that the third tier drivers were running. Checking the action output
https://github.com/metabase/metabase/actions/runs/17220604727/job/48854867380?pr=62513
i see :
```
Is master or release: false
Has '.Team/Drivers' or '.Team/Querying' or '.Team/SemanticLayer' label: false
Checking for changes in relevant folders...
Changes detected in the following folders: src/metabase/lib src/metabase/query_processor.
Tier 3 driver tests will run.
```
So it thinks that something changed in src/metabase/lib/* and
src/metabase/query_processor/*
This branch
ds/clo-4370-button-in-metabase-admin-section-to-purchase-metabot-ai-add
targets master. And the diff is checking the diff of files between the
state of master and the PR. Which means if any changes have landed in
master affecting those areas this diff will report positive.
At the time of this writing this is:
```
❯ git diff --name-only c6a9193bff462d83c5c2 | wc -l
1259
```
Where c6a9 is master and 462 is the branch.
A better option is to find where the branch and the merge target share a
common ancestor and then check for changes from there.
```
❯ git diff --name-only $(git merge-base c6a9193bff462d83c5c2) 462d83c5c2 | wc -l
32
```
So we use `git merge-base <master> <branch>` and then diff that sha
against the branch.
```
git diff --name-only (git merge-base <base> <pr-branch>) <pr-branch>
```
And we get:
```
❯ git merge-base c6a9193bff462d83c5c237e6eee311
❯ git show 37e6eee311
commit 37e6eee311
Author: Phoomparin Mano <poom@metabase.com>
Date: Wed Aug 20 01:40:19 2025 +0700
Add the metabase-browser custom element to Embed JS (#60828)
Co-authored-by: cubic-dev-ai[bot]
<191113872+cubic-dev-ai[bot]@users.noreply.github.com>
```
Which is the common root. Then the diff from there:
```
❯ git diff --name-only $(git merge-base c6a9193bff462d83c5c2) 462d83c5c2
.clj-kondo/config/modules/config.edn
enterprise/backend/src/metabase_enterprise/api/routes.clj
enterprise/backend/src/metabase_enterprise/cloud_add_ons/api.clj
enterprise/backend/src/metabase_enterprise/harbormaster/client.clj
enterprise/backend/test/metabase_enterprise/api/session_test.clj
enterprise/backend/test/metabase_enterprise/cloud_add_ons/api_test.clj
enterprise/frontend/src/metabase-enterprise/metabot/components/MetabotAdmin/MetabotPurchasePage.tsx
enterprise/frontend/src/metabase-enterprise/metabot/components/MetabotAdmin/MetabotPurchasePage.unit.spec.tsx
enterprise/frontend/src/metabase-enterprise/metabot/components/MetabotAdmin/MetabotPurchaseSettingUpModal.tsx
enterprise/frontend/src/metabase-enterprise/metabot/index.tsx
frontend/src/metabase-types/api/metabot.ts
frontend/src/metabase-types/api/mocks/settings.ts
frontend/src/metabase-types/api/settings.ts
frontend/src/metabase/admin/app/reducers.ts
frontend/src/metabase/admin/routes.jsx
frontend/src/metabase/api/metabot.ts
frontend/src/metabase/api/utils/use-token-refresh.ts
frontend/src/metabase/common/components/AdminLayout/AdminSettingsLayout.tsx
frontend/src/metabase/plugins/index.ts
resources/frontend_client/app/assets/img/metabot-cloud-96x96.svg
src/metabase/cloud_migration/core.clj
src/metabase/cloud_migration/models/cloud_migration.clj
src/metabase/cloud_migration/settings.clj
src/metabase/core/init.clj
src/metabase/premium_features/core.clj
src/metabase/premium_features/settings.clj
src/metabase/premium_features/token_check.clj
src/metabase/server/middleware/security.clj
src/metabase/store_api/core.clj
src/metabase/store_api/init.clj
src/metabase/store_api/settings.clj
test/metabase/analytics/stats_test.clj
```
And now we have a correct list of files changed in that PR, not just
files different between current master and current PR.
* Convert revision usage
* convert pivot test + metabot usages
* convert more tests
* migrate final clj-time usage in logger.core
* switch quartzite to a git sha to get most recent commit w/o clj-time
* Revert "switch quartzite to a git sha to get most recent commit w/o clj-time"
This reverts commit 393bc56aad.
* Make readOnly notebook completely non-interactive
* Add readOnly prop to PopoverBackButton
* Add readOnly to FilterPickerHeader
* Add readOnly prop to FilterPickers
* Allow making filter body readOnly
* Pass readOnly to FilterPicker from FilterStep
* Add disabled prop to ClausePopover
* Disable step popovers when readOnly (except for filters)
* Do not disable pointer-events for notebook preview
* Disable popover when button is disabled
* Render readOnly expression popover in readOnly mode
* Make limit step readOnly when readOnly is set
* Add test for readonly notebook
* Disable tooltip on join picker when read-only
* Disable join picker popover when read-only
* Hide back button on date picker
* Simplify backbutton props
* Do not allow going back in readOnly ExpressionWidget
* Allow readOnly AggregationPicker
* Allow opening aggregation popover when it is an expression
* Render aggregation popover when it is a custom expression
* Fix types
* Remove renderPopoverWhenReadOnly flag
* Fix cursor logic on SortStep
* Do not render pointer cursor when step does not have popover
* Render custom expression popover for non-custom aggregations in read-only mode
== Goal ==
In case the Metabase instance does not have the Metabot AI add-on, we
want to show a page in the admin section to purchase it.
== How to test ==
1. Pick a valid token from staging with the `offer-metabase-ai` feature, but without the `metabot-v3` feature (`metabase-ai` add-on)
2. Pick the token's instance's Metabase API key
3. Pick an email address that matches a store user of the token's organization
4. Spin up a Metabase instance with that API key, token and user
5. Navigate to the admin section, AI tab
6. See the new page to purchase the add-on and click the button
Closes: https://linear.app/metabase/issue/CLO-4370/button-in-metabase-admin-section-to-purchase-metabot-ai-add-on
* UXW-503 Triggers error state when underlying card has been deleted
* UXW-503 Makes CardEmbed error state selectable, bordered
* UXW-503 Updates error state per design feedback (less of a jump between loading and error state, fewer different error visuals)
* UXW-503 Adds e2e test
* UXW-503 Removes useMemo per PR feedback
* UXW-503 Simplifies an e2e test, makes it less prone to flakiness
* Add gate table tombstone cleanup to index cleanup job
* add partial index on gated_at for just the tombstone rows
* fix up index creation syntax
* WIP - adding check that indexer ran within last 24 hours
* fix tests
* fix whitespace, adjust log/docstring
* kondo
* fix build, maybe broken from bad rebase?
* fix: handle more header names in translation csv
Handle the locale being specified as 'locale', 'locale code', 'language'
also make the check for header a little looser so we can report errors
in the header row more accurately.
* fixup! have sample CSV match copy
* Call createUser action when inviting new user on the /setup page
* Add e2e test case
* Add invite_sent event / Check user went to next step in e2e test
* Let the backend handle the invite_sent snowplow event
* Change approach to set source value
* Fix unit tests
* Add unwrap() to createUser dispatch
* Remove unused invite param from the /setup endpoint
* Remove unneeded frontend code
* Fix kondo issues
* track search_engine
* track request id
* record request id on search_query event
* fix bug with search_click position not factoring in pagination
* report offset on search_query
* report on the entity model
* move changes into new schema file and update referenced schema
* add back view_more event
* track hashed + salted search term on search_click + search_query events
* refactor trackSearchClick to accept object param instead of long list of args
* report search term for internal purposes only
* pr feedback
* add test coverage for new search reporting attributes
* maybe fix e2e in ci
* fix failing e2e
* Localize rows of initial tags and initial jobs in db on setup or startup
* Also translate job descriptions.
* Use fewer hits to the db on startup
* Use site locale
* kick ci
* Remove extraneous arg
* Remove extraneous strs
* Different translation model (on select)
* setup no longer needed from transforms
* Fix
* Test transform tag translation
* Test transform job initial translations
* Fix runtime crash on field visibility change with preview open
* Update Loki Snapshots
---------
Co-authored-by: Metabase Automation <github-automation@metabase.com>
* Query builder breadcrumbs can link back to archived models
* Update Loki Snapshots
---------
Co-authored-by: Metabase Automation <github-automation@metabase.com>
Closes#62099
When a Field has a coercion strategy (say, `UnixSeconds->DateTime`) set in the `Table Metadata`,
and then that field is **joined**, it will get double-coerced. The first time in the inner query for the join,
the second at the top level `SELECT`.
Since after the first coercion it will be a `TIMESTAMP` instead of and `INTEGER`, the second coercion is a SQL error.
The solution is to add `:qp/ignore-coercion true` in the `resolve-joins` middleware, when the `:fields` of the outer
query are being populated with joined fields. That prevents the QP from re-applying type coercions and date truncations.
* Add user-recency scorer to the semantic search backend
* Only query the recent_views table for models it knows about
* Replace group-by with m/index-by in user-recency scorer
* Cleanup comments
* Improve prometheus metrics for user-recency and add tests
Permission filtering is already recorded in :metabase-search/permission-filtering-ms so no need for a new metric that
records it + collection filtering. Instead, add a new metric :metabase-search/semantic-collection-filter-ms so there
is no overlap.
Also remove embedding label from the collection filter and appdb scorer metrics, as these don't currently depend on
the embedding model in any meaningful way.
* Rename :metabase-search/permission-filtering-ms to :metabase-search/semantic-permission-filter-ms
To match the naming convention of other metrics exposed by the semantic search backend.
Fixes a regression where making sure the autocommit was correctly set to
enable reading streaming result sets disallowed ddl and other
non-readonly changes in postgresql due to the interaction of the
'readOnly', 'readOnlyMode', and 'autoCommit' jdbc driver settings.
* UXW-502 Makes sql error scrollable. Also makes it vertically centered when not scrollable.
* UXW-506 Hides some UI from Download as PDF
* UXW-506 Prevents metabot blocks from getting split across pages in generated PDF
* UXW-506 Adds print-only Metabot icon to Metabot blocks in documents
solves https://github.com/metabase/metabase/security/code-scanning/493
but this is actually not an issue at all. When we build the driver jars
we remove deps provided by metabase itself. Metabase itself relies on
```
org.apache.commons/commons-lang3 {:mvn/version "3.18.0"}
```
```clojure
(defn- remove-provided-libs [basis driver edition]
(let [provided-lib->provider (into {}
(filter (fn [[lib]]
(get-in basis [:libs lib])))
(provided-libs driver edition))]
;; log which libs we're including and excluding.
(doseq [lib (sort (keys (:libs basis)))]
(u/announce (if-let [provider (get provided-lib->provider lib)]
(format "SKIP %%s (provided by %s)" provider)
"INCLUDE %s")
(colorize/yellow lib)))
;; now remove the provide libs from `:classpath`, `:classpath-roots`, and `:libs`
(let [provided-libs-set (into #{} (keys provided-lib->provider))
provided-paths-set (into #{} (mapcat #(get-in basis [:libs % :paths])) provided-libs-set)]
(-> basis
(update :classpath-roots #(vec (remove provided-paths-set %)))
(update :libs #(into {} (remove (fn [[lib]] (provided-libs-set lib))) %))
(update :classpath #(into {} (remove (fn [[path]] (provided-paths-set path))) %))))))
```
That output looks like:
```
Write :clickhouse :ee uberjar -> /home/runner/work/metabase/metabase/resources/modules/clickhouse.metabase-driver.jar
SKIP org.clojure/clojure (provided by metabase-core)
SKIP org.clojure/core.specs.alpha (provided by metabase-core)
SKIP org.clojure/spec.alpha (provided by metabase-core)
INCLUDE com.clickhouse/clickhouse-client
INCLUDE com.clickhouse/clickhouse-data
INCLUDE com.clickhouse/clickhouse-http-client
INCLUDE com.clickhouse/clickhouse-jdbc
INCLUDE com.clickhouse/client-v2
INCLUDE com.clickhouse/jdbc-v2
SKIP com.fasterxml.jackson.core/jackson-core (provided by metabase-core)
SKIP com.google.errorprone/error_prone_annotations (provided by metabase-core)
SKIP com.google.guava/failureaccess (provided by metabase-core)
SKIP com.google.guava/guava (provided by metabase-core)
SKIP com.google.guava/listenablefuture (provided by metabase-core)
SKIP com.google.j2objc/j2objc-annotations (provided by metabase-core)
SKIP commons-codec/commons-codec (provided by metabase-core)
SKIP commons-io/commons-io (provided by metabase-core)
SKIP org.apache.commons/commons-compress (provided by metabase-core)
SKIP org.apache.commons/commons-lang3 (provided by metabase-core) <------------------
SKIP org.apache.httpcomponents.client5/httpclient5 (provided by metabase-core)
SKIP org.apache.httpcomponents.core5/httpcore5 (provided by metabase-core)
SKIP org.apache.httpcomponents.core5/httpcore5-h2 (provided by metabase-core)
SKIP org.clojure/clojure (provided by metabase-core)
SKIP org.clojure/core.specs.alpha (provided by metabase-core)
SKIP org.clojure/spec.alpha (provided by metabase-core)
SKIP org.jspecify/jspecify (provided by metabase-core)
INCLUDE org.lz4/lz4-java
SKIP org.ow2.asm/asm (provided by metabase-core)
INCLUDE org.roaringbitmap/RoaringBitmap
INCLUDE org.roaringbitmap/shims
SKIP org.slf4j/slf4j-api (provided by metabase-core)
```
But this is included here in the hopes that it appeases snyk
* Correctly invalidate transform tags
* Remove Undo button from toast when undoing a tag
* Avoid wrapping in certain columns for transform lists
* Add test for tag removal invalidation
* Fix unit tests
* Use helpers in test
* Prove #27521 is fixed
* Fix the test from #60583
* Code cleanup
* I have fixed#39033
* Enable e2e test
* Add ref for #62383
* Ok so I guess I have fixed#47734 now too
* Test update 🔧
* Update the test again
* Test fix 🔧
* Test fix 🔧
* QP limit middleware => Lib
* Preprocess returns MBQL 5
* Split off changes to limit middleware
* Fix Kondo warnings
* Revert changes from Limit PR
* Code cleanup
* Preprocess returns MBQL 5
* Kondo cleanup
* Rework some tests in `wrap-value-literals-test` to use mock metadata providers
* Update more tests to use mock metadata providers
* Use even less with-temp
* Remove with-temp in QP middleware viz settings tests
* Rework metrics-test to never use with-temp
* Remove more with-temps
* Update case-test to use mock metadata providers
* Even more with-temps
* Update even more tests
* Fix two more
* Nice
* Revert stricter schema for query-hash
* Finish rewriting tests to use Mock MPs instead of with-temp or add ignores
* cljfmt
* Test fix 🔧
* Fix permissions
* Split off malli improvements into #62523
* PR cleanup
* PR feedback
* PR feedback
`{:status 500}` is rendered as a response with status 200 and a body of
`{"status": 500}`, wheras `{:status 500 :body ""}` is rendered as a
response with status 500 with empty body.
Fixes: b7253af4f2
* chore: don't run migrations on reindex!, just ensure index is initialised
* feat: support :force-reset?, to support `/re-init` endpoint
* fix test
* chore: check for index-created? in test
* undo
* feat: include index-created-at
* feat: support lazy gating + stale index behavior
* fix tests
* fix
* fix test
* return new index from record-new-index-table!
* chore: don't pipe opts to create-index-table
* feat: randomize index name
* force new index
* shorter name
* suffix as part of qualify
* re-init test
* rename find-best-index to find-compatible-index
* introduce create-new-index-spec, split out of find-compatible-index
* pick last index
* docstring
* add test
* split into determine-target-index/switch-to-index
* remove index-created? opt
* ensure table always exists
* back out including id/index_created_at in index
* move model-table-suffix out of qualify-index
* fix kondo warnings
* Add PostgreSQL provider icons to db form
fix types and opacity
# Conflicts:
# frontend/src/metabase/databases/components/DatabaseDetailField/DatabaseDetailField.tsx
* Store provider name field in form context and send it to the db create endpoint
* store provider name in the app db
* Use the non-deprecated form field for DB provider field
* Allow setting null value to provider name field
* Fix db tests
fix test
fix test
* Add db submit integration test
fix type error
* Rename mock file name
* Simplify postgesql json form mock
* Set provider_name in the same db update as rest of the fields
* Use select-keys-when for DB put update handler
* wip
* fix unit test and fix utm tags on link
* update docs link
* Update frontend/src/metabase/home/components/EmbedHomepage/tests/EmbedHomepage.unit.spec.tsx
Co-authored-by: Phoomparin Mano <poom@metabase.com>
* Update frontend/src/metabase/home/components/EmbedHomepage/EmbedJsContent.tsx
Co-authored-by: Phoomparin Mano <poom@metabase.com>
* address feedback
* swap static and sdk as per feedback in the pr
---------
Co-authored-by: Phoomparin Mano <poom@metabase.com>
* Fix duplicated table column viz settings
* Add BE logic
* Fix
* Review fixes
* Review fixes
* fixup! use linear time deduplication
---------
Co-authored-by: Edward Paget <ed.paget@gmail.com>
* Updates page title to be document's name
* Memoizes SetTitle so it doesn't perform a bunch of calculations on every render when the title hasn't changed
* fix: adjust schemas for inactive tables too
When toggling the multi-catalog setting for a database, we rename
existing schemas to match. For example, if you turn multi-catalog on and
have the existing schema `foo`, we'll rename it to `mycatalog.foo`.
Similarly, if you have the existing schema `mycatalog.foo` and turn
multi-catalog *off*, we'll rename it to `foo`.
We were ignoring inactive schemas here, which meant that if you turned
on multi-catalog and some of your tables became inactive (quite possibly
because you didn't update the schema filter to match, a major footgun
for the user), we would:
- update the tables going one way (`foo` to `mycatalog.foo`) and
deactivate for not matching the schema filter, then
- never update the tables again, because now they're inactive
Instead we should update the schemas for inactive tables as well.
Related to [UXW-375: Databricks multiple catalogs - Table
"database.catalog.schema.table" is
inactive](https://linear.app/metabase/issue/UXW-375/databricks-multiple-catalogs-table-databasecatalogschematable-is) / https://github.com/metabase/metabase/issues/61492
I don't think this is a complete fix though - we may need to rethink the
feature, and possibly just not allow you to change the setting for
existing databases.
* Add a test
* Allow rendering databases as disabled in DataSelector
* Allow rendering databases as disabled in DataSourceSelectors
* Allow rendering databases as disabled in NativeQueryEditor
* Add features for transforms/* to type
* Disable unsupported source tables in the native queyr transform builder
* Allow disabling databases in DataPicker
* Allow passing shouldDisableItem from Notebook to DataStep
* Disable unsupported databases in datapicker for transforms
* Remove type error in filter
* Pass shouldDisableItem to more pickers
* Add missing database to item
* Filter out unsupported questions too
* Remove database property again from DataPickerItem
* Fix item type
* Rename db_id to database_id
* Pass database_id in collection picker
* Disable unsupported tables, metrics and dashboards
* Hide recent items that have been disabled
* Padd shouldDisableItem to more sub-pickers
* Add database metadata to table
* Add shouldDisableItem to TablePickers
* Rename helper
* Simplify types
* Add hook to avoid using metadata
* Disable questions based on unsupported databases in question picker
* Allow disambiguating RecentItem from TableItem
* Disallow transforms on audit databases
* Add tests for unsupported databases
* Add unit tests for disabling items
* Disable dashboards in Question picker
* Use plain ORDERS_ID
* Move database support helper to utils and pull data fetching out of hook
* Pull out helper into separate definiton
* Add test about creating transforms from metrics
* Remove transforms/view and transforms/materialized-view from api types
* Use Loader from metabase/ui
* Fix types
* fix: handle pivot-grouping as a BigDecimal
Some database drivers (Oracle) seem to convert the pivot-grouping
integer into a BigDecimal; this attempts to safely convert it back to an
integer we can perform bitwise operations on, using .longValueExtract
which will throw an exception if the number is not losslessly
convertible to a long.
* fixup! throw if conversion fails
In `enforced-sandboxes-for-tables` we were only checking the
permission in the `data_permissions` table, without special-casing
administrators like we do elsewhere.
So when checking to see if an administrator in `All Users` (sandboxed)
and `Administrators` had an enforced sandbox for a table, we were
checking to see whether there were rows in `data_permissions` providing
`Administrators` with `:unrestricted` data access to the table - which
will never be the case, because Administrators is special and never gets
any rows there.
Instead, special-case the super user case like we do elsewhere. Don't
bother getting the groups they're a part of - we know the set of tables
they're sandboxed on is empty.
* fix: handle whitespaces in the connection string, decode username and password
* Trim the connection URI at first step
* Add safe uri decode, edge case test and remove redundant trim
* wait up to 10 seconds for worker to stop
in CI getting lots of flakes. Here's reproduction locally:
```
queue-test=> (dotimes [_ 20] (listener-handler-test))
FAIL in (listener-handler-test) (queue_test.clj:178) ;; 1
Standard behavior with a handler
expected: (not (thread-name-running? thread-name))
actual: (not (not true))
FAIL in (listener-handler-test) (queue_test.clj:178) ;; 2
Standard behavior with a handler
expected: (not (thread-name-running? thread-name))
actual: (not (not true))
,,,
FAIL in (listener-handler-test) (queue_test.clj:178) ;; 18
Standard behavior with a handler
expected: (not (thread-name-running? thread-name))
actual: (not (not true))
nil
```
and after:
```
queue-test=> (dotimes [_ 20] (listener-handler-test))
nil
```
* eastwood doesn't like new qualified method stuff
* Attempt to fix flaky tests
emails can end up in here from other tests as well. So this takes two
tracks:
first, i want to just check all emails in the box for this
user. Sometimes there are multiple emails in here and the one we want
isn't first. and second, better error messages. This will show us all
email message subjects. Hopefully that lets us identify what else might
be going wrong.
Next step if this is still failing it to attach which test sent the
message using clojure.test/*testing-vars*.
Adds a simple function and a macro to `metabase.util.log` to allow
propagation of ThreadContext across thread boundaries.
`(with-context-meta some-map)` adds the `mb-` prefixed items in the
`ThreadContext` to the metadata of that map.
`(with-restored-context-from-meta some-map)` takes a map presumably
resulting from calling `with-context-meta`, and sets the current
`ThreadContext` accordingly.
I changed the dispatch-fn and the worker consumption functions in
`metabase.notification.send` to use these to propagate the ThreadContext
to the worker thread.
Sending a notification like
```
(log/with-context {:aoooooook "hey"} (dispatch! {:flippity "ofosidjffoo"}))
```
now results in:
```
2025-08-18 14:42:45,716 ERROR notification.send :: Error in notification worker {mb-aoooooook=hey}
clojure.lang.ExceptionInfo: Invalid input: {:payload_type ["invalid dispatch value, got: {:flippity \"ofosidjffoo\"}"]}
```
* Improve log context for exceptions
This adds a feature to our log context that I've used and loved in the
past.
For illustration, let's say I do this:
```
(defn my-fn []
(log/with-context {:user 123 :database 5}
(/ 1 0)))
(try (my-fn)
(catch Exception e (log/error e)))
```
With the current code, we'll run `my-fn`, throw an exception,
then *exit* the `log/with-context` block. When we eventually log the
exception, the context will be empty.
This is annoying, because when the exception was thrown, we had context
that a developer thought would be useful. But just when it would
be *most* useful to have that information - an uncaught exception we
didn't think about bubbles up to the HTTP request logging middleware or
whatever - it's thrown away.
This change essentially wraps `with-context` calls with a `try/catch`
that catches any exceptions and re-throws them, annotated with the
original context.
This way, when we actually log the exception, we can swap back the
original context. In the above example, `{:user 123 :database 5}` would
be logged with the exception.
* Move log context ex-data to special key
When the exception middleware catches an exception it does some fancy
logic about whether to just let the body be the exception message (like
"Not found") or a blob of data with the message, stacktrace, data, etc.
Including the log context in the ex-data itself caused this to get
confused (because some additional data was present in the `ex-data`) and
spit out the blob where previously we'd just presented a user-friendly
error message.
To avoid doing this accidentally in the future, I just stuck all the log
context data under a special key, `:metabase.util.log/context`, in the
ex-data, which we can explicitly ignore when deciding whether to include
the whole exception in the error response.
* Fix failing test
* Fix conversion of joins with a native source query to legacy
* Don't do nasty stuff to parameters when converting from legacy
* Move parameters to correct location when converting joins
* Fix conversion of joins with filters
* Splice join parameters together
* QP params => Lib
* Fix Kondo warnings
* Simplify PR [ci skip]
* Beautiful QP middleware
* Beautiful QP middleware.
* Add notez
* Test fixes 🔧
* JVM metadata provider fix 🔧
* Kondo & cljfmt
* Test fixes 🔧
* kondo
* Delete the Lib-style rewrites of the parameters code, I decided I didn't like them
* Fix the schema
* Revert unneeded changes
* Fix busted tests 🔧
* Fix two more busted tests.
* Fix even more broken tests
* Fix broken tests 🔧
* Improved normalization for :value clauses
* Test fix 🔧
* Test fix 🔧
* Fix resolution of field name refs in first stage
* Test fixes 🔧
* Another round of test fixes 🔧
* Fix :expression ref parameters
* Maybe fix flakes
* Test fixes 🔧
* cljfmt/kondo
* Reduce coupling between semantic.index and semantic.scoring
Previously ->col-expr was defined in the scoring namespace and had to "know" about hybrid search query table aliases, which is gross.
Instead, define a hybrid-coalesce func in the semantic.index namespace and pass it into the semantic-scorers function. Also use it in hybrid-search-query, to ensure hybrid column references are constructed in the same way in both places.
* Add semantic-scorers->col-expr-is-respected-test
* Remove ->col-expr and just re-project cols from hybrid query with well-known names
* Add an API endpoint to delete collections
This permanently deletes a collection and anything contained inside it.
One "exception" that's not really an exception: say you have A > B > C > D,
you trash C, then trash A.
Now you have:
```
Trash > A > B
Trash > C > D
```
(because C was trashed directly).
But internally, we still represent D's location as `/A/B/C/`.
Now you permanently delete A. We need to make sure D's and C's locations
get fixed. Therefore, before deleting A, we go through all
the *directly*-archived children of A, and move *them* to A's parent.
Moving a collection also moves its children, so we'll move the whole
`C > D` structure out of `B` into the root collction.
Then we'll delete `A > B`, leaving `C > D` behind.
* Don't get the parent for no reason
the children-location of my parent is... my location. :doh:
* Ensure we cleanup the index metadata tables as well as the index table in with-index!
Otherwise, we wind up with many "duplicate" docs in the gate table which results in duplicates in the index. These are
not true duplicates; they have fresh model ids from the with-temp'ed mock docs we create, but they have identical
content, so they have identical embeddings and tsvectors. Those documents then wind up with identical text and
semantic scores when querying the index, and since we ORDER BY the scores (with no tie breaker secondary ordering),
this creates non-determinism in the semantic and keyword rankings, making test assertions wonky.
* Tighten up test assertions in native-query-test
These were relaxed to top-three? in #62101, but since the query results are no longer non-deterministic, it should be
safe to switch back to requiring the match to be the first result.
* feat(mage): add port option to mage nrepl
Adds an option to the mage nrepl command to customize the port mage's
nrepl starts on. This is mostly so I don't need to modify my clojure-mcp
configuration to point to a different port when working on mage stuff.
* feat(mage): add port option to mage nrepl
Adds an option to the mage nrepl command to customize the port mage's
nrepl starts on. This is mostly so I don't need to modify my clojure-mcp
configuration to point to a different port when working on mage stuff.
* feat(mage): print better argument usage help
* make semantic search resilient to errors on fallback engine
* sort active-engines by default-engine first
* more logging around appdb init
* add test
* update the dashboard form to use mantine components
* committing forgotten files
* e2e adjustments and fixes
* test adjustments
* remove shameful hardcoded font sizes
* fix font size on switches
* adjusting unit test
* Fix OSS/EE artifact conflict in uberjar workflow
Problem: When fetching both OSS and EE artifacts in check-existing-artifacts job,
unzip was prompting for confirmation to replace COMMIT-ID file, causing the
workflow to fail with 'EOF or read error, treating as [N]one' error.
Solution: Use different extract paths for OSS and EE artifacts to avoid conflicts:
- EE artifact extracts to 'ee-artifact/' directory
- OSS artifact extracts to 'oss-artifact/' directory
This allows both artifacts to coexist without file conflicts, fixing the
artifact recycling issue for both OSS and EE versions.
🤖 Generated with [Claude Code](https://claude.ai/code)
Co-Authored-By: Claude <noreply@anthropic.com>
* fix(ci): resolve OSS/EE artifact conflicts by extracting to separate paths
Fixes COMMIT-ID file conflicts when fetching both OSS and EE uberjar artifacts
by extracting each artifact directly to its designated directory path instead
of the current directory. Simplifies fetch-artifact action logic while
maintaining full backwards compatibility.
Changes:
- Extract artifacts directly to specified extract-path
- Eliminate file conflicts between OSS and EE artifacts
- Simplify processing logic by unifying unzip and rename steps
- Maintain backwards compatibility for all existing workflow usages
🤖 Generated with [Claude Code](https://claude.ai/code)
Co-Authored-By: Claude <noreply@anthropic.com>
* fix(action): correct error message for missing artifacts to use input name
* fix(ci): optimize uberjar concurrency to prevent duplicate builds per commit
Changes concurrency grouping from branch-based to commit-based to allow
multiple workflows to call uberjar for the same commit while preventing
duplicate builds. The existing artifact checking logic handles deduplication.
- Group by commit SHA instead of branch ref
- Disable cancel-in-progress to avoid wasting build work
- Allow concurrent calls from different workflows (pr-env, perf-test, etc.)
- First caller builds, subsequent callers reuse existing artifacts
🤖 Generated with [Claude Code](https://claude.ai/code)
Co-Authored-By: Claude <noreply@anthropic.com>
* fix(action): use temp directory to avoid file conflicts during artifact extraction
The fetch-artifact action was failing when multiple artifacts were extracted
to the same directory due to file conflicts (e.g., COMMIT-ID files). The unzip
command would prompt to replace existing files, causing CI failures.
This change extracts artifacts to unique temporary directories first, then
moves only the JAR file to the final location, eliminating conflicts while
maintaining full backwards compatibility.
🤖 Generated with [Claude Code](https://claude.ai/code)
Co-Authored-By: Claude <noreply@anthropic.com>
* fix(workflow): remove source_changed input and simplify build conditions
* fix(action): remove unnecessary blank line in fetch-artifact action
---------
Co-authored-by: Claude <noreply@anthropic.com>
* Fix regression - small cta for large upsell banner and big card
* Extract upsell styles to per component css modules
* Regression - bring back margins to cta in large upsellbigcard
* Add upsell components to loki visual tests
* Fix too big margin in upsell large variant
* Align storybook story with the real usage
* Update Loki Snapshots
* Allow custom button styling on UpsellCard, move custom styles card instance
---------
Co-authored-by: Metabase Automation <github-automation@metabase.com>
* Update writing-sql.md
Adds a section below "How Metabase executes SQL queries" noting that we don't officially support multi-statement queries or DDL (and recommend against use of procedures and functions).
* Apply suggestions from code review
Co-authored-by: Jeff Bruemmer <jeff.bruemmer@gmail.com>
---------
Co-authored-by: Jeff Bruemmer <jeff.bruemmer@gmail.com>
* Make Lib schemas better about catching MBQL-only keys in native stages or joins
* Replacing a field ref in join :fields should set the correct join alias
* Add view_count scorer to semantic search backend
* Get pgvector db from global env in semantic.scoring/view-count-percentiles*
Get the db data-source directly rather than passing it in as an argument to this function to side-step potential
issues with using the db as a cache key for the memoized version.
https://github.com/metabase/metabase/pull/62086#discussion_r2272790618
Adds an option to the mage nrepl command to customize the port mage's
nrepl starts on. This is mostly so I don't need to modify my clojure-mcp
configuration to point to a different port when working on mage stuff.
* remove references to static questions
* Ditch overall sidebar for now (#61523)
* Remove list of cards sidebar
* Some type fixes
* Review fixes
* feat: associate in_report cards w/ report_document
* feat: associate cards with reports in create
* better markdown serde
* switch to prosemirror ast
* chore: add enterprise card test
* remove snapshots
* less redux
* send used card ids
* drop flushSync error
* remove replace question, drop refresh question
* fix backend not starting
* saving new report should not show LeaveRouteConfirmModal
* polish
* Insert into reports from metabot (#61642)
* insert from metabot
* Metabot plugin and hook
* make stuff nicer
* cleanup
* no more snapshots
* run build and docker uberjar on push
* remove trigger for uberjar
* Print Report
* cleanup
* always create a card clone when embedding
* always clone non in_report cards
* disable metabot sidebar on mob+b
* stop header bouncing
* jumping layout when loading
* why scroll to bottom when opened
* gross emojis
* cleaning up emoji ui a bit
* icons
* fe: reports -> documents
* update metabot question creation parsing
* keep report cards and collections in sync (#61686)
* do not leak prosemirror styles into charts
* chore: rename everything to document (#61743)
strip out snapshotting code
* fix FE usage of documents api
* fix embedding dashboard-only cards to documents
* uxw-366: Navigation back to the report from a drill through can be difficult - can we add the back button like for dashboards
* mention all entities
* split mention vs command, allow mentioning other entities
* paste smartlinks
* missing import
* fix links to tables
* fix layout
* auto focus on title on new documents
* UXW-404 Improves image formatting
* Tweaks metabot block styling, wires up "stop generating" functionality
* Fix issues with multiple in-flight requests
* Adds subtle animation to "Working on it..." message
* Document Metadata
* Fix some CI issues
* Persist cards when saving document (#61818)
* chore: remove document version model
* persist cards only when saving documents
* chore: remove in_document type
* feat: return document creator
* feat: save cards with parameters pass in as cards
in POST and PUT requests to documents
* chore: exclude doc cards from where shouldn't be
search, recents, collection items
* chore: fixup search and card tests
* contract
* feat: update document ast with new card ids
* fix: report -> document in collection api
* fix: ws
* feat: clone cards not associated with document
* chore: add tests for document permissions
* fix contract
* fetch root collection
* update message
* must be working now
* fix sidebar
---------
Co-authored-by: Edward Paget <ed.paget@gmail.com>
* wip but working sql query editor in documents
* feat: revision history for documents
* feat: include revision info in collection tree res
* document entity
* Fix issue where temporary implementation of metabot block can't create questions
* feat: allow documents to be archived
* Document Collection Item Caching
* Fix issue where submitting empty metabot prompt (by hitting "Run" or cmd+enter) makes it spin forever
* fix UXW-412 again
* feat: allow documents to be bookmarked
* fix: hide documents in collection items if not EE
* fix: add perms info to documents in collections
* fix: always add documents to bookmark lookup
* feat: add document delete endpoint
* fix: require non-blank string for name
* App Bar Breadcrumbs for documents
* fix: tests and lints
* fix: ordering of migrations for mysql
* fix: for OSS tests
* fix: tests and kondos
* fix: module sort
* New metabot documents API
* UXW-400: formatting floating menu
* UXW-416: replace question should replace only 1 question
* swap icon
* l10n
* fix: bad var reference in tests
* undo moves should work once archived is boolean
* Show tooltip with precise time in document metadata
* Updates MetabotEmbed to work with upcoming BE changes
* Updates MetabotEmbed to always use draft cards
* Updates MetabotEmbed "stop generating" functionality to work with regular (non-streaming) api call
* Updates MetabotEmbed to use most up-to-date api contract
* Stubs out the necessary updates to make MetabotEmbed work with an UnsavedCard from the BE rather than a full Card
* Updates MetabotEmbed to use most up-to-date api contract
* fixed drills & query editing for metabot cards
* fix: foreign key indexes
* Replaces hard-coded metabot-generated card title with title from the /generate-content response (or translates a default title if none provided)
* fix: add entity id column to documents
* add title to untitled docs
* UXW-453 Fix 404 when clicking draft card title
* Handle URL slugs, remove version code from document page, handle 404 and 403
* UXW-459 Fix issue where creating a new document when starting from another document overwrites the existing document (and shows previous document's metadata in new document)
* hide object detail row in document tables
* UXW-459 Fix issues with LeaveRouteConfirmModal and new documents
* UXW-461 Prevents jump when Save button appears/disappears. Also adds subtle fade-in transition when it appears.
* UXW-461 Save button always shows up for new documents (and user has necessary permissions)
* improved mention and command
* clean
* fix replace question
* do not show bubble menu on disallowed nodes
* MetabotEmbed polish:
* Prevents prompt from highlighting when you hit "Run" (potentially bringing up the editor bubble menu).
* Prevents useEffect hook from running way more than it needs to.
* Adds margin below MetabotEmbed.
* Hitting escape with nothing in the textarea will now remove the MetabotEmbed block.
* Makes layout less janky when Metabot responds with a long error message.
* Localizes "Created with Metabot" text
* drop emojis
* no emojis and markdown downloading
* fix metadata fetching
* initial e2e tests
* fix: set null on collection cascade for documents
mirrors behavior for dashboards and cards
* fix: remove documents token hack
* feat: documents in search
Also supports view_counts and last_viewed_at for documents
* fix: emit documentment read on api get
* fix: in_place search fixes and test fixes
* fix: migration order
* fix: set type to :question dashboard_id to nil
* fix: more in-place search fixes for documents
* fix: final in-place search test fixes
* fix: handle inadvertent documents in oss tests
* mention + command feedback
* placeholder
* fix command search
* Fix issue where charts sometimes don't get added via metabot when inserted in the middle of a document
* fix command
* feat: support pinning documents
* fixup! pinning reconcilation with documents
* feat: support serdes on documents (#62032)
* fix: kondo warning on unused var
* updated styles
* bump padding
* Markdown Loki Test
* UXW-462 Adds database picker in metabot block
* UXW-462 Fix issue where sometimes custom tiptap node tries to re-render too quickly resulting in an error
* refactor
* feedback on the command
* UXW-462 Fix issue where you can write markdown in metabot blocks
* improve state management UXW-463 UXW-450
* Updates "Created by Metabot" line based on design/user feedback
* feat: add documents to recents
* fix: whitespace and round-trip serdes test
* fix: whitespace exclusion path
* fix: token features in recent views test
* Able to generate sql charts
* fix: casing in tests
* Changes how mention text is referenced in the documents metabot prompt serializer. It uses a cache in the documents slice now. It was using a node attribute before but that was hacky and it messed with the "is dirty" calculation.
* more e2e tests
* Fix issue where metabot nodes sometimes yield a false positive in hasUnsavedChanges because they updated their own attributes
* Update menu items (#62072)
* Update commands and remove descriptions
* Removes unused code
---------
Co-authored-by: Brad Anderson <brad@metabase.com>
* feat: return can_restore, can_delete for docs
* Updates Document type with newly added properties
* Fix issue where "Delete permanently" errors out instead of deleting
* Move, Archive, and Bookmark in documents page
* bury shame
* UXW-472 Adds ArchivedEntityBanner for documents
* fix: remove extraneous precondition migration
* UXW-451 Adds some light dataset error handling
* Reverts probably-accidental change
* feat: check for name length
* Reports fix fe ci (#62046)
* Fix some CI issues
* Fix some CI issues
* Fix typecheck issues
* Fix lint
* Fix unit tests
* Fix e2e tests
* fix pivot query editing in documents
* better command popover positioning
* UXW-473 fix layout
* fix z-index
* only include tables you can use for native query builder
* Fix typecheck
* Light documents metabot cleanup
* Fix issue where browsers spell check areas that shouldn't be spellchecked (e.g. "Ask Metabot ...")
* Removes package-lock.json that was accidentally added in 63a94c8788 and should've been removed in 68366d7d37
* cleanup
* attempting to get a card embed in storybook
* Fix documents stories
* fix: bookmarks fail if token exists but ee doesn't
* fix: bookmark inside a collection error
* Removes now-unused extensions
* chore: tests for metabot tools
* Add token check on the FE
* fix: remove unused ns from test
* Review fixes
* Revert token check
* Add token check on the FE
* fix: correctly serialize token status to FE
* Fix e2e tests (#62113)
* Fix e2e tests
* Fix e2e
* fix: use adhoc metadata if card id is negative
* chore: add tests for isSavedCard guard
* UXW-489: replace shows entity picker, UXW-507: don't highlight chart after insertion
* Fix typecheck
* fix: token feature tests
* fix: add docs to stats ping
* Fix typecheck
* Fix e2e
* Reverts now-unnecessary changes
* Reverts more now-unnecessary changes
* Address cubic PR feedback (type guards)
* Fix issue where landscape-orientation images larger than the max-width get stretched
* Fix e2e
* PR feedback (code cleanliness)
* PR feedback (simplify custom node definition)
* adding loki references
* delete loki reference files
* adding missing loki reference files
* auto-fix generated sql (#62167)
---------
Co-authored-by: Edward Paget <edpaget@users.noreply.github.com>
* PR feedback (unit test + remove unused classes)
* addressing some minor feedback
* chore: add table utils tests
* review
* Cleans up types
* Removes /documents/redux-utils. Uses plain useDispatch/useSelector hooks instead.
* review
* cleanup documents selectors
* remove unnecessary type cast in visualizationUtils.ts
* remove versioning mention in comments
* use _.isEqual instead of json serialization and string compare
* Code review fixes
* remove inline styles from FormatButton
* fix bubble menu does not immediately show what formatting is applied
* remove inline styles
* drop unused suggestion styles
* consistent hook names
* drop unused dep
* remove any from DocumentPage
* clean up change callbacks of the editor component
* drop unused onClode prop in EmbedQuestionSettingsSidebar
* provide correct reportTimezone to ModifyQuestionModal
* make runQuestionQuery optional
* NativeQueryModal cleanup
* clean up NativeQueryModal
* drop unused code in useDocumentState
* unnecessary casting
* reset error state
* drop unused prop
* bubble menu disallowed nodes
* dedupe similar hooks
* review
* fix: kondos
* Update e2e/support/helpers/api/createDocument.ts
Co-authored-by: cubic-dev-ai[bot] <191113872+cubic-dev-ai[bot]@users.noreply.github.com>
* fix: PR comments
* fix: remove schema ns
* Potential fix for e2e test failing on CI only
---------
Co-authored-by: Nick Fitzpatrick <nickfitz.582@gmail.com>
Co-authored-by: Denis Berezin <denis.berezin@metabase.com>
Co-authored-by: Edward Paget <ed.paget@gmail.com>
Co-authored-by: Ryan Laurie <30528226+iethree@users.noreply.github.com>
Co-authored-by: Luiz Arakaki <luiz.arakaki@metabase.com>
Co-authored-by: Ryan Laurie <iethree@gmail.com>
Co-authored-by: Edward Paget <edpaget@users.noreply.github.com>
Co-authored-by: Brad Anderson <brad@metabase.com>
Co-authored-by: Nathan Voxland <nathan@metabase.com>
Co-authored-by: eliotmetabase <eliot@metabase.com>
Co-authored-by: cubic-dev-ai[bot] <191113872+cubic-dev-ai[bot]@users.noreply.github.com>
* Remove feature-visualizer and nm-semantic-search-in-metabase from the Build Uberjar workflow
The feature-visualizer branch was removed in 3e605c6d87, but came back in (presumably due to bad merge conflict
resolution) from the semantic search branch.
* Fix typo in gate test: documen -> document
* Make cause third arg to ex-info in semantic-search /status endpoint
* Create HeaderLinkGroup with new links and add to ExistingDatabaseHeader.tsx
* Don't show database added popup when database is created
* Ensure 'Browse data' button link does nothing when disabled
* Fix e2e tests
* Fix e2e tests part 2
* Adjust 'Browse data' link to be an actual button when disabled
* Fix padding (keep button padding & adjust flex gap)
* copy over relevant code from semantic search service
* copy docker compose, add defenterprise integration
* WIP getting the integration working
* fix up reindex! call
* fix whitespace
* kick ci
* remove ^:const to try to fix compilation errorin tests
* Adds mi/can-read? checks to search results (#61060)
* Add a couple of helpful comments for manual testing
* Add ability to change db-url at the repl
* Add can-read? checks for search results
* Semantic search engine should not be used if the db is not setup
* sort kondo modules
* fix up syntax in metabase.search.impl and whitespace
* remove a blank line
* fix remaining clj-kondo issues
* Add semantic search test workflow (#61065)
* Add new workflow for semantic search
* Add a quick example test to see if things work
* Have the new tests only run if db-url is defined
* Sparks of joy
* Add a delay to appease the kondo gods
Not sure this rule should apply to once-fixture, but for now lets fix
it.
* Semantic search filtering, basic querying tests, connection pooling, etc (#61098)
* fix results postprocessing, add verified field to search index
* scaffold a basic query test
* index filtering tests
* quicky and dirty implementation of the requested ui changes
* Add basic connection pooling + tests
* Fix test indentation
* add test for verified content, refactor tests a bit
* address comments
---------
Co-authored-by: Sloan Sparger <sloansparger@gmail.com>
* Add nm-semantic-search-in-metabase to uberjar workflow and remove metabot-v3-main (#61142)
In preparation to deploy the nm-semantic-search-in-metabase branch to stats.
* Use EE feature for semantic search defenterprises (#61139)
* switch defenterprise features from :none to :semantic-search
* update tests
* Register semantic-search premium feature
* FE - default to app db search, enable semantic in command palette and search page, add disable semantic search on search page
* Test collection based permission checks apply to search results (#61152)
* Fix t2-instance derivation from search docs
Lookup in database as the search document model is lossy
* Test collection based permission checks apply to search results
* Swappable embedding models + semantic search kill switch (#61176)
* first stab at swappable embedding models + setting to disable semantic search globally
* fixups
* fixups
* fix up error in tests
* add semantic-search to stats ping to quiet test
* quiet another test
* fix whitespace
* fix kondo warnings
* fix up and simplify embedding tests
* fix core init test
* fix more tests
* tweak setting defaults + comments, remove debugging code
* fix compilation
* change oss version of supported? to return false instead of throwing
* fix lint issues
* Batched index updates for semantic search (#61117)
* kick ci
* Add comment to trigger reindex job
* Fix docstring typo
* Batch index updates for semantic search
* Wrap {create,drop}-index-table! in mt/as-admin in with-temp-index-table!
* Move binding for *vector-dimensions* into with-mock-embeddings
* Create metabase-enterprise.semantic-search.test-util and move once-fixture there
* Move other test helpes and macros into the semantic.test-util ns
* Factor out with-indexable-documents! from with-index!
* Move binding of search.ingestion/*force-sync* into with-temp-index-table!
Questionable, but realistically can't see tests wanting to use a temp index table without forcing sync reindex.
* Add table-exists-in-db?! to semantic.test-util
* Add some basic tests for funcs in the semantic.index namespace
* Add tests for batched index updates
* Disable flaky tests
* Fix up issues post-rebase
---------
Co-authored-by: Noah Moss <noahbmoss@gmail.com>
* Fix accidental blocking search index init (#61200)
* kick ci
* Fix accidental eager index initialisation
---------
Co-authored-by: Noah Moss <noahbmoss@gmail.com>
* Batched requests to OpenAI for fetching vector embeddings (#61217)
* Batch calls to openai according to token counts
* refactor
* mock the new batch embedding function
* mock both batched and unbatched embedding functions in tests
* a little code consolidation
* address comments
* Init db connection in reindex! if needed
* Add more DB existance checks for safety
* fix fe tests
* Lower batch size, improve logging, and make it configurable via env var (#61274)
* Lower batch size, improve logging, and make it configurable via env var
* fix whitespace
* Remove transaction from batched semantic search index updates (#61229)
* Drop uneccessary ! from table-exists-in-db?!
* Don't wrap batched index updates in a transaction
https://github.com/metabase/metabase/pull/61117#discussion_r2214828302
* Filter indexed entities from semantic search index
* Create HNSW index on semantic search embeddings column (#61238)
* Create HNSW index on semantic search embeddings column
* Add test for HNSW index creation
* Ensure identifiers are quoted when formatting honey sql in semantic search index
The nano-ids used in temp table names when testing (and various other places) might contain uppercase chars or hyphens which need to be quoted.
* Fix batch-delete! log message
* alter create-index-table! to not wipe the index if it already exists, and use the upsert API during index population (stop-gap changes to try to get the staging instance working)
* fix document filtering
* Only create HNSW embeddings index :if-not-exists (#61305)
Since init! no longer wipes the index table if it already exists, we also need to only create the embeddings HNSW
index if it does not exist.
* Better batching for semantic search (#61299)
* streaming embedding results to the index-updating code
* rework
* suppress kondo
* Parameterization refactor (#61328)
Adds the `index` and `embedding-model` parameters to functions in `index` and `embedding` respectively. This allows a degree more flexibility and less coupling to global variables which restrict how indexing functions can be used.
The goal is this PR supports changes regarding active/pending and the embedding model being something you can play with at runtime.
* Add embedding-model parameter
* Restore previous top-level transaction behaviour on init/reindex
* Reduced redundancy
* Embedding test fixes
* Fixup comments
* Fix rebase goofs
* Fix tests again
* Joy
* Add `index` parameter, composed of table-name and embedding model
* Keep index name in the index structure
* English is a hard language
* Remove `:redef` we no longer want
* Another merge goof fix
* Fix bug in associating batched embeddings with original documents (#61347)
* Fix bug in associating batched embeddings with original documents
* tweak
* add log
* Quick follow-up optimization to prior PR: don't iterate over all of filtered-documents in each batch
* [Metabot] Hybrid search w/ RRF (#61353)
* rrf based hybrid search implemented
* keyword results weighted more heavily by name, corrected some issues with the rrf calc
* fix final result ranking
* move tsvector-related util functions into search.util and expose in the search interface to fix kondo errors
* Taking a stab at fixing SQL injection risks following existing postgres tsvector querying pattern
* Move postgres tsvector util tests
* gracefully handle nil names
* switch back to raw embedding literals but validate their contents
* add back perm checks (whoops)
* add back str/replace for ts-search-expr, bump up cosine distance cutoff
* tweak cutoff again, fix create index test
* remove str/replace usage bc it seems unneeded
* fix rrf calc
---------
Co-authored-by: Noah Moss <noahbmoss@gmail.com>
* Remove indexed-entities from the semantic search model-set and documents reducible (#61304)
* Remove indexed-entities from the semantic search model-set
* Add comment about filter-read-permitted and indexed-entity docs
* Add prometheus metrics for semantic search backend (#61309)
* Add :metabase-search/semantic-index-size prometheus metric
And update it on upsert-index! / delete-from-index!
This metric mirrors the corresponding :metabase-search/appdb-index-size
* Add :metabase-search/semantic-embedding-tokens prometheus metric
* Make log levels / messages more consistent
* Remove vestigial populate! reference
* Record actual token usage as reported by OpenAI API response
* BOT-255 Index metadata system (#61379)
* Add index-metadata system
Adds the necessary indirection to support more advanced index table
management.
- Allows multiple indexes to exist
- Indexes are tracked by a new index_metadata table
- A single active pointer determines which index to use for the current
search engine backend
- It is assumed table == logical index and there is no other
partitioning, hence the unique constraint
- Some small simplifications to index.clj (drop index-name, assume names
are prefixed with the table name)
- The top level orchestration code happens in a new pgvector-api
namespace to avoid bloating the enterprise hooks (which assume a global
environment), and allow the api level to be properly parameterized.
* Kondo
* Appease kondo with require for next.jdbc in comment block
---------
Co-authored-by: Mike Appleby <86076+appleby@users.noreply.github.com>
* Enable fts on native query contents in the semantic search backend (#61576)
* Enable fts on native query contents in the semantic search backend
Update the semantic search backend to handle fts of native query contents in exactly the same way as the appdb
backend, namely: all keyword search content gets indexed into two separate columns, one with and one without native
query contents. At runtime, we decide which column to query against based on the user's search setting.
For now, this only modifies the keyword search side of the hybrid search result. The semantic search on the vector
embeddings still does not include native query contents, but any keyword matches can still get blended into the result
set by the RRF ranking.
* Ensure index names do not exceed postgres max of 63 characters
* Allow to toggle NativeQuery independent of DisableSemanticSearch in SearchSidebar.tsx
* Cleanup whitespace
* Add native-query-test
* Add :native_query to semnatic.tu/mock-documents
* Add upsert-index!-tsvectors-test
* Remove unecessary :raw wrapper from column names
* Remove unused =? alias
* [Semantic Search] Embedding generation support for OpenAI compatible APIs (#61641)
* make open base url configurable, add arctic-embed support, sanitize index table name for models with slashes + commas + periods
* make table names slightly shorter
* Update enterprise/backend/src/metabase_enterprise/llm/settings.clj
Co-authored-by: appleby <86076+appleby@users.noreply.github.com>
* Update enterprise/backend/src/metabase_enterprise/semantic_search/index.clj
Co-authored-by: appleby <86076+appleby@users.noreply.github.com>
* Update enterprise/backend/src/metabase_enterprise/semantic_search/index.clj
Co-authored-by: appleby <86076+appleby@users.noreply.github.com>
* split off provider name to make index shorter
* Fix tests failing due to removing text-embedding-3-large model
* Abbreviate model names to ensure we do not exceed postgres identifier name limit
---------
Co-authored-by: appleby <86076+appleby@users.noreply.github.com>
* Ensure that semantic search entrypoints are safe to call without init! (#61438)
* Ensure that semantic search entrypoints are safe to call without init!
* Move more helpers into semantic.test-util namespace
* Add tests for calling semantic search entrypoints prior to calling init!
* Adjust fix and tests to work better with new index-metadata apis
And also log a warning and return nil if anyone calls into update-index! or delete-from-index! prior to calling init!
* set default for ee-openai-api-base-url setting
* BOT-258 semantic indexed entities (#61527)
* kick ci
* Semantic search filtering, basic querying tests, connection pooling, etc (#61098)
* fix results postprocessing, add verified field to search index
* scaffold a basic query test
* index filtering tests
* quicky and dirty implementation of the requested ui changes
* Add basic connection pooling + tests
* Fix test indentation
* add test for verified content, refactor tests a bit
* address comments
---------
Co-authored-by: Sloan Sparger <sloansparger@gmail.com>
* FE - default to app db search, enable semantic in command palette and search page, add disable semantic search on search page
* Init permission support for indexed entities
* fix kondo
* Fix rebase errors
* Fix rebase errors
* pr feedback
* unit test index entity perm filter
---------
Co-authored-by: Noah Moss <noahbmoss@gmail.com>
Co-authored-by: Noah Moss <32746338+noahmoss@users.noreply.github.com>
Co-authored-by: Sloan Sparger <sloansparger@gmail.com>
* BOT-259 Semantic search filter parity (#61396)
* kick ci
* Semantic search filtering, basic querying tests, connection pooling, etc (#61098)
* fix results postprocessing, add verified field to search index
* scaffold a basic query test
* index filtering tests
* quicky and dirty implementation of the requested ui changes
* Add basic connection pooling + tests
* Fix test indentation
* add test for verified content, refactor tests a bit
* address comments
---------
Co-authored-by: Sloan Sparger <sloansparger@gmail.com>
* FE - default to app db search, enable semantic in command palette and search page, add disable semantic search on search page
* Remaining search filter support minus has_temporal_dimensions
* ignore public API warning about metabase.search.filter
* Fix collection filter impl
* fix kondo
* [no ci] remove comments
* More efficient querying
* Fix rebase errors
* fix merge
---------
Co-authored-by: Noah Moss <noahbmoss@gmail.com>
Co-authored-by: Noah Moss <32746338+noahmoss@users.noreply.github.com>
Co-authored-by: Sloan Sparger <sloansparger@gmail.com>
* try something else
* Remove semantic HNSW index breaking where clause (#61765)
* Adjust semantic search query to favor HNSW index over seq scans (#61768)
* Adjust semantic search query to favor HNSW index over seq scans
* remove limit and SET random_page_cost
---------
Co-authored-by: Sloan Sparger <sloansparger@gmail.com>
* [SEMANTIC] base64 embeddings and better logging (#61809)
* Improved logging; base64 embedding
* trim double
* Use parse-double over parseDouble
* fix search.engine/init! return (#61825)
* fix: return model frequencies from search.engine/init!
* fix: document return type of init!, like per delete! and update!
* fix: calc for batches
* semantic search instrumentation (#61790)
* intrument semantic search
* test for analytics
* typo
* fix: id is numeric
* move test next to metrics one
* Support switching between :semantic and :appdb engines (#61826)
* add test
* lift util
* feat: add full api test
* fix: appdb available when search-engine is semantic
* Fallback to appdb search if semantic search doesn't return enough results (#61815)
* Fallback to appdb or in-place search if not enough results are returned from semantic search
* Dedupe combined results
* Bump up hybrid search query limit + make it configurable
* tweak setting doc, fix reflection warning in test code
* clean up tests
* move fallback implementation into the EE namespace, add a new setting for min results threshold
* fix lint issues
* ensure tests set semantic-search feature
* Do not realise document reducibles (#61860)
* wip
* batched streaming upsert-index!
* fix: actually dedupe searchable_text
* Revert "try something else"
This reverts commit eec29881c5.
* can still inner seq
* push indexed-entities last
* fix: semsearch delete
* lift delete-from-index-batch-sql
* test
* dedupe test
* style
* comment
* indexed entities last in search
* Semantic search test fixes (#61884)
* Fix api-engine-switching-test
* Test - switch test fixture to :each to see if it removes a race condition
* Revert "Test - switch test fixture to :each to see if it removes a race condition"
This reverts commit 33386c602b.
* another test - ensure init-delay is derefed before getting datasource
* remove compose-fixtures
* Revert "remove compose-fixtures"
This reverts commit 16c882a599.
* another test
* test
* add back some of core-test
* comment out all code but leave in ns
* add back fixtures
* add back one test
* replace with stub test
* remove compose-fixtures
* fix tests with a try/finally in db_test.clj
* oops, add back :once
* also revert back fixture order
* more cleanup
* use with-index! in appdb-available-with-semantic-test rather than trying to generate real embeddings
* Fix db test
* Only fetch embeddings when needed (#61918)
* chore: use appropriate builder-fn instead of unqualify-keys
* feat: cache on already seen embeddings
* add test
* distinct-on content
* better logging
* reuse test impl
* note
* repl utils
* we have if-not-exists semantics, so no point in not invoking it on init.
We need to invoke it else the new index won't get picked up.
This will change anyway once we have proper migrations with #BOT-278
* fix tests
* Incorporate metadata into semantic search result ranking (#61906)
* Move common code into search.scoring and add search.engine/scorers for backend-specific scorers
* Refactor to remove search.engine/scorers
* Add new index columns required by scoring expressions
* Add metadata score to semantic search backend
* Give up on reusing scorer expressions between appdb and semantic search
* Add :all-scores stats to semantic search query results
* Fix failing pgvector query test
* Adjust rrf weight to fix failing query tests
* Add premium scorers for verified and official_collection
* Get rid of single-use test helper dog-training-native-query-json
* Add scoring tests
* Fix for permissions issue in test
* Remove extra whitespace in bindings
* [Semantic Search] Add AI Service as an embeddings providers (#61850)
* wip
* clean up and better abbrev logic
* fix provider references
* remove magic int
* do not munge model name if not needed
* add / fix tests
* make mock index have a shorter name
* shorten nanoid
* math was not mathing
* fix kondo error
* death by lints
* [BOT-260] sem search consistency, gating and indexer loops (#61602)
* Add document gating
Adds a document gating mechanism and async indexing job.
- Allows indexing to recover if it encounters a transient error.
- Avoids data loss during indexing due to process death
- Provides an ordering which is more closely related to logical record
validity times rather than write time into the index.
* Fix some test docstrings
* Fix some loop tests
* Schedule indexer only if the instance has the :semantic-search feature
* Avoid redundant replay of the very last gate entry when the quartz job fires
* Add a quick candidate-filter byte equality test
* Refine loop interrupt/error test (get rid of old gunk)
* Use shared `tu/spy` instead of copy-pasta
* Replace candidate filter with simple set of last seen
Use TEXT for hash for eq convenience.
* Avoid future foot guns with a (letfn) refactor of indexing-step
Avoids issues that might later be introduced by the 'accidental' closure of (actual) variables that might later be changed during this side-effecting operation. In particular the move-to-next-watermark local function.
^:once was suggested but a hoisted letfn that can only close over args (which keeps the mutable indexing-state at the forefront of ones mind) keeps us honest.
* Check if engine enabled and supported during indexer task
Previously was only checking if the db was configured.
* Be more careful about Instant and timestamp representation.
Preferred is sql.Timestamp for now due to the default setup of postgres / JDBC.
* Fix up some asserts that still reflected byte hashes
* Add a quick e2e index everything test.
Find horrific pagination bug. Fix that.
We did not account for having enough ties that our buffer was full. This was likely in tests even with the high precision of the pg clock because:
- we coerced stuff to Instant which lost precision
- we generally did not account for tie breaks in pagination (bad form)
So we are paginating properly using the secondary id as a tie break. Within the confidence time only, as otherwise this would be dangerous (it is right to stall if you see too much until the lag time has elapsed). This stops stalls if lots of documents share a buffer.
Will follow up with another test that forces time collisions to assert this tie break behaviour independently.
* Test fixes
* Joy
* Add indexer docstring
* Thread pool for semantic search index batch updates (#61947)
* parallel batch processing for index updates
* tweak setting docs
* switch back to transducing version
* Revert "switch back to transducing version"
This reverts commit 4a99c3c6869e28f56d794d8057f006700cfc8432.
* fix setting name
* revamp approach - use a ThreadPoolExecutor with a custom rejection policy
* fix tests
* add concurrency test
* Fix up tests by adding a serial mode that disables thread pool usage
* Fix final test -- ensure we spy on & call only on the 4-arity version of upsert-documents
* Switch to custom retry handler
* Still wrap executor in a delay for lazy initialization
* Fix upt ests
* Fix final tests
* [Semantic Search] Remove disable semantic search filter (#61973)
* remove disable semantic search filter
* fix failing test
* Collapse indexed-entity ids in semantic search results (#61952)
* Collapse indexed-entity ids in semantic search results
The indexed-entity results have composite ids like model-index-id:model-pk. We need to extract the model-pk and attach
that as the :id on the results. Otherwise, the links generated by the frontend are invalid like
/model/12345-blah/1234:62681
* Add indexed-entity-id->model-{index-id,pk} to search.util
And use indexed-entity-id->model-index-id in filter-can-read-indexed-entity
* Fix test failure due to overly long index name
* Use dimensions param in OpenAI api calls (#62010)
* Use dimensions param in OpenAI api calls
* Use starts-with?
* Improved fallback logic & prometheus metrics for semantic search (#62024)
* Add prometheus metrics to track appdb search fallback logic
* undo broken function call
* Update logic to not fallback if no results were filtered out
* fix more tests
* Tweaks
* Try again to fix tests
* Address comments
* update docstring
* Fix non-batched get-embedding for openai and ai-service providers (#62050)
* Fix non-batched get-embedding for openai and ai-service providers
We were passing a single text string rather than a seq of texts, resulting in an error when we tried to get the token
count.
ERROR semantic-search.embedding :: Failed to generate OpenAI embedding for text of length: 8
java.lang.ClassCastException: class java.lang.Character cannot be cast to class java.lang.String
* Add test for extract-base64-response-embeddings
* Add test-get-embedding
* Semantic search indexing status endpoint (#62082)
* add search-items-count function
* Add endpoint for reporting indexed count + total estimate
* try to fix test
* Temp commit to debug CI failure
* Add honeysql classpath to debug output
* Revert "Add honeysql classpath to debug output"
This reverts commit 4395f420f7.
* Revert "Temp commit to debug CI failure"
This reverts commit 36fbf047d6.
* Fix honeysql create-index :raw forms (#62144)
Something changed with honeysql 2.7.1340 and :raw forms in the tail of a create-index now get unwrapped and need to be
nested in another vector.
For example, in earliers versions
(sql/format
(sql.helpers/create-index
[:foo-idx :if-not-exists]
[:foo-table :using-hnsw [:raw "embedding vector_cosine_ops"]]))
produced an sql string like
CREATE INDEX IF NOT EXISTS foo_idx ON foo_table USING HNSW (embedding vector_cosine_ops)
but on 2.7.1340 you get
CREATE INDEX IF NOT EXISTS foo_idx ON foo_table USING HNSW (raw EMBEDDING VECTOR_COSINE_OPS)
Unclear if this is a honeysql bug or we're doing something unsupported, but fix it with a workaround for now by
wrapping the :raw in an extra vector like [[:raw "foo"]], in order to get CI green.
* Move reindex/index to gate the documents (#62101)
* switch init/reindex to gated updates
* nil stats
* sync index in test
* non deterministic search result
* Semantic search migration (#62043)
* Migration
* Rebase follow-up
* [Semantic Search] Add admin setting to enable semantic search (#61622)
* add basic setting switch for enabling semantic search, upsell component, logic for falling back to default search in settings, and basic e2e test
* fix type issue
* Fix serialization baseline test
* finished
* small fixes
* do not animate progress bar when progress === 100
* impl PR feedback
---------
Co-authored-by: Tyler Plude <tsplude@gmail.com>
* revert e2e diffs (#62173)
* Re-order semantic search index columns to test schema migration (#62172)
* Re-order semantic search index columns
Mostly to test schema migration, but this also makes the table contents easier to scan in PgAdmin / psql imho by
putting ids next to ids, timestamps next to timestamps, etc.
* migration 1
* cleanup
---------
Co-authored-by: lbrdnk <lbrdnk@users.noreply.github.com>
* fix reflection warnings in tests (#62151)
* fix reflection warnings in test
* just use bound-fn*
---------
Co-authored-by: Dan Stone <wotbrew@gmail.com>
Co-authored-by: Sloan Sparger <sloansparger@gmail.com>
Co-authored-by: appleby <86076+appleby@users.noreply.github.com>
Co-authored-by: Sloan Sparger <sloansparger@users.noreply.github.com>
Co-authored-by: Tyler Plude <tsplude@gmail.com>
Co-authored-by: Ryan Laurie <iethree@gmail.com>
Co-authored-by: Nicola Mometto <nicola.mometto@metabase.com>
Co-authored-by: Mihael Konjević <konjevic@gmail.com>
Co-authored-by: lbrdnk <lbrdnk@users.noreply.github.com>
* API Keys module cleanup
* Test fixes 🔧
* Fix Kondo errors
* Unmark these tests as `^:parallel` for now because I do not have the energy for dealing with MySQL
* I don't have time to fight MySQL
* Test fix 🔧
* add-alias-info => MLv2
* Restore deleted test
* WIP
* WIP [ci skip]
* Ensure we're actually using a cached metadata provider in tests
* Remove profiling code
* Restore accidentally-deleted `native-query-snippet` fn
* Fix accidental edits
* Make `visible-columns` stage-only
* More minor caching improvements
* cljfmt & kondo
* More minor caching improvements
* Fix#35252
* cljfmt & kondo
* Another round of test fixes and lint fixes 🔧
* Test fixes 🔧
* Test fix 🔧
* PR feedback
* Test & Lint fixes 🔧
* Test fix 🔧
* Kondo/cljfmt
* Appease Eastwood
* cljfmt run against everything
* Revert changes to deps.edn
* Did I fixx??
* cljfmt/kondo
* Fix Oracle test failure
* Fix nested fields (MongoDB)
* cljfmt
* Nested column name fixes
* Fix Mongo for reals
* Revert BigQuery breaking stuff
* Fix equality for binned columns
* Test fix 🔧
* Remove dead :ident keys
* Fix is_verified field refs
* Fix is_verified field refs
* Some sanity checking
* Fix typo
* Fixx
* Ok, don't be super fussy about bad code using :type/Text as a parameter
* Fix validation of legacy expression refs inside stage parameters
* Fix aggregation names not being deduplicated
* :as-alias
* replace saigon with bangkok
* use ho chi minh as equivalent replacement
* use bangkok for sqlserver
* update us/pacific to america/los_angeles
* Test debugging
* Fix debugging
* Test fixes 🔧
* Fix busted hardcoded e2e query and add disallowed-keys helper
* cljfmt
* Update deprecation notices 56 => 57
* Kick the can down the road
* Fix another busted test 🔧
* Improve field resolution even more 😒
* cljfmt
* Fix sandboxing with bad field refs.
* Ok I give up I guess I just accidentally fixed sandboxing with custom columns
* cljfmt
* I fixed it
* Appease the linter
* Test fixes 🔧
* OOOPS
* Fix test
* prettier
---------
Co-authored-by: Riley Thompson <rileythompson99@gmail.com>
* UI for metabot feedback
* copy and spacing tweaks (#62049)
* switch to download based approach
* test
---------
Co-authored-by: Maz Ameli <maz@metabase.com>
* BOT-285: ensure field statistics are populated
* chore: test
* extend field-values treatment to dummy tools
* no more need for ensure-field-values!
* chore: use get-or-create
* style
* remove unused require
* fix
* nil safety
* extract
* wip
* wip2
* fix and wip
* wip
* fiiiix
* use the component name inside the iframe
* fix tests
* use custom elements attributes in the preview
* clean up
* clean up
* merge question with new question in the iframe code
* remove SdkIframeEmbedTagSettings
* styles -> S
* restore defaultTheme functionality that was merged on master
* fix test, now the iframe is not reset and we don't need to select the table again
* Fix containerize job to only run when uberjar succeeds
The containerize job now properly checks that the uberjar job succeeded
before attempting to containerize. This prevents containerization from
being triggered on documentation-only changes where the uberjar job
is skipped.
Added: needs.uberjar.result == 'success' condition
* Set default output name for fetched artifacts to "metabase.jar"
- Add :database-feature to SettingDefinition schema to specify required driver features
- Update validate-settable-for-db\! to check database driver feature support
- Add validation to ensure :database-feature is only used with database-local settings
- Fix validation logic to throw when driver does NOT support required feature
- Add comprehensive tests for database feature validation
- Update defsetting macro documentation with :database-feature option
This allows database-local settings like database-enable-actions and
database-enable-table-editing to require specific driver features
(e.g. :actions, :actions/data-editing) before they can be enabled.
* Fix uberjar containerization and implement flexible fetch-artifact action
Resolves issues where uberjars weren't getting containerized when adding the
build-docker-uberjar label to PRs or when manually dispatching workflows.
Major changes:
- Create containerize-uberjar.yml workflow for centralized containerization logic
- Create uberjar-containerize-labeled.yml for PR label-triggered builds
- Extract containerization from uberjar.yml to avoid 3-level workflow nesting
- Add containerization to run-tests.yml for master branch pushes
- Implement flexible extract-path parameter in fetch-artifact action
- Replace preserve-structure boolean with string-based path specification
- Update all workflows to use appropriate extract-path values
The fetch-artifact action now supports:
- extract-path: "." (current directory with custom output-name)
- extract-path: "target/uberjar" (maintains original structure)
- extract-path: "custom/path" (any directory structure)
Containerization now works in all scenarios:
- Master push/merge → run-tests.yml → containerize-uberjar.yml
- PR with build-docker-uberjar label → uberjar-containerize-labeled.yml
- Manual dispatch → containerize-uberjar.yml
🤖 Generated with [Claude Code](https://claude.ai/code)
Co-Authored-By: Claude <noreply@anthropic.com>
* Fix fetch-artifact action to avoid moving file to itself
Prevents "mv: source and destination are the same file" error when
extract-path matches the existing jar location (e.g. target/uberjar).
🤖 Generated with [Claude Code](https://claude.ai/code)
Co-Authored-By: Claude <noreply@anthropic.com>
* Refactor fetch-artifact action to be more DRY
- Consolidate JAR path logic into a single function
- Export JAR_PATH to GITHUB_ENV for use in subsequent steps
- Remove repetitive path determination code
- Reduce lines of code from 113 to 108
🤖 Generated with [Claude Code](https://claude.ai/code)
Co-Authored-By: Claude <noreply@anthropic.com>
* Add Docker Hub link and pull command to PR comment
Enhanced the PR comment after containerization to include:
- Docker tag name (branch with slashes replaced by dashes)
- Direct link to Docker Hub repository tag page
- Ready-to-use docker pull command
This makes it easier for developers to find and pull the built image.
🤖 Generated with [Claude Code](https://claude.ai/code)
Co-Authored-By: Claude <noreply@anthropic.com>
* Add get-docker-info action and update workflows to utilize it for Docker image retrieval
* Refactor post-comment job to streamline Docker info retrieval and PR comment posting in containerize workflow
* Remove unnecessary blank lines in fetch-artifact and get-docker-info actions for cleaner code
* Update containerize job conditions to trigger on pull request labels for Docker builds
* revert using fetch-artifact for release artifacts
* add migration
* add table before update to correctly set deactivated_at
* no need for schema
* archive tables
* tests
* note
* include active to index
* guard against nil changes
* doc
* guard against archival
* note
* prper count
* serdes/test
* extra safety guard
* more accurate logging
* use epocSecond
* more error validation
* change suffix
* add rollback mig
* move migrations to v56
* safer
* test(embed-js): migrate tests to use custom element api
* use CreateApiKeyResponse
* Update e2e/test/scenarios/embedding/sdk-iframe-embedding/sdk-iframe-embedding.cy.spec.ts
Co-authored-by: Phoomparin Mano <poom@metabase.com>
---------
Co-authored-by: Phoomparin Mano <poom@metabase.com>
Cards created before v0.55 lack the `:card_schema` field in their revision history. When loading these old revisions, the system throws "Cannot SELECT a Card without including :card_schema" because the Card model's `after-select` expects this field to always be present.
Existing code adds the legacy default `card_schema` value (20) in `revert-to-revision!`. This PR adds the value (20) to old revisions in `revision.clj`'s `after-select`.
This allows users to view revision history and revert cards that were created before the v0.55 upgrade.
* Updates the info and info_filled icons
* use currentcolor fill instead
* remove fixed width and height and rely on viewBox
* tweak svg paths slightly
* fix y position
---------
Co-authored-by: Phoomparin Mano <phoomparin@gmail.com>
Co-authored-by: Maz Ameli <maz@metabase.com>
* Ensure we're actually using a cached metadata provider in tests
* Remove profiling code
* Restore accidentally-deleted `native-query-snippet` fn
* Fix accidental edits
* Fix metadata calculation for joins that contain joins
* All Lib tests are passing
* The PR that fixes everything [WIP] [ci skip]
* Test fix 🔧
* Sort namespaces
* Docstring formatting
* [Lib tests] 100 failures, 33 errors.
* [Lib] 98 failures, 24 errors.
* [Lib] 76 failures, 17 errors.
* [Lib] 49 failures, 4 errors
* [Lib] 46 failures, 2 errors.
* [Lib] 15 failures, 0 errors.
* [Lib] 13 failures, 0 errors.
* All Lib tests passing
* [QP tests] 2 failures, 1 error.
* Revert :encode/cache-key
* Minor caching performance improvements
* Fix nest breakouts/cumulative aggregations
* Add some more stuff to lib.core
* Test fixes/cljfmt/kondo
* Test fix 🔧
* Test fixes and fix some busted tests
* OMG CACHE SO MUCH FASTER!!!!!
* Fix MongoDB
* Fix tests and support mock segments
* Code cleanup and test fixes 🔧
* Test fix 🔧
* Fix set versus map
* PR feedback
* cljfmt
* More PR feedback and tighten up the field resolution code a bit
* Some schema cleanup
* PR feedback
* PR feedback
* Add TODO
* PR feedback
* Remove implementation of join `visible-columns` which is the same as :default
* Remove unused Card visible columns
* Restore the Card implementation of visible columns in THIS PR.
Missed this from the big remerge, where login attributes were being
reset to an empty value on jwt login. Also fixed not including the
jwt-group-sync on the initial formik state.
* Added to_session endpoint
* Added to_session endpoint
* Added to_session endpoint
* Added to_session endpoint
* Added to_session endpoint
* Some small cleanup
- remove unnecessary try/catch clauses
- rename/rework unnecessary public wrapper fn
- make `jwt` required for the endpoint
- only return session token, not any JWT data
* Test fixes
---------
Co-authored-by: John Swanson <john.swanson@metabase.com>
* wip: add connection string form prefill feature;
add tests
* Refactor deprecated export format
* Skip jdbc:prefix when parsing
* Add snowflake support
* Add success message
* Add a feedback message when pasting a connection string
* Refactor field mapping
* wip use regexes to parse connection strings
* make port optional
* Add snowflake
* Add bigquery and clickhouse
* Add Athena support
* Add Amazon Redshift
* Improve clickhouse
* Add failure message
* Add databricks and druid
* Add MySQL
* Add presto
* Adapt snowflake, create objectToString helper
* Add Apache Spark
* Add SQL server
* Add Oracle
* Add Starburst
* Add vertica
* Reset connection string input on change of key
* Map only when select db and protocol matches
* Fix and refactor semicolon params parsing
* Add tracking to database string uri parsing
* create mapping config with placeholders
use placeholders also in tests
* Create engineKeys types
* Create Cypress paste command using synthetic react events
* Use fade transition for feedback animation
* Add e2e tests for connection strings form filling
* fill in display name from connection string
* Add succes failure text test
* Validate form after filling in fields
* Use more standard fallback for paste
* Remove unused regex parsing implementation
* use mass setFields for fomik setting values
* Add test case for clearing existing values, including engine select field
* Fix form fields being overwritten when pasting connection string
* Add onPaste behavior and use simpler paste mechanism in Cypress
* Fix typo
* Extract and use engineKey type and isEngineKey guard
* Make connection string field controlled by formik
Fix submit values
Fix type error
* Remove unneeded prop justify
* Run regex just for currently selected db
* Fix naming of the test file
* Add e2e db connection test
* Fix missing path in th rspack emedding setup
* Explain why connection string is not sent to BE
* use pojo instead of map for db engines config
* Revert to * export to maintain consistency
* fix connection string test
* create a more meaningful type name
* Add result_metadata perms check to qp
* Check result metadata view data on card put
* Throw the exception
* Test
* Add perm check for ad-hoc referencing cards
* Docstrings
* Use seq
* Order of checks
* Apply suggestions from code review
Co-authored-by: metamben <103100869+metamben@users.noreply.github.com>
* Simplify into
---------
Co-authored-by: metamben <103100869+metamben@users.noreply.github.com>
* add table link to metadata page
* Table
* Add a e2e test
* Add a e2e test
---------
Co-authored-by: Alexander Polyankin <alexander.polyankin@metabase.com>
When streaming data for a query, we were only including columns if a)
the viz settings for the column were completely empty, or if b) the viz
settings for the columns included specifically enabling that column.
The problem occurs most clearly for things like native SQL queries.
If you construct a query like
```
SELECT id FROM venues;
```
and then set the column viz settings to *something* non-empty (most easy
way: just hide the `id` column and then show it again), and then save
the card, then change the query to:
```
SELECT id, name FROM venues;
```
then if you download the CSV for this card (even after saving it!)
you'll get a result that only contains `id` and not `name`.
A few tests needed to change here because they were implicitly assuming
the old behavior, but I don't think that's entirely intentional. This
behavior was different than the behavior in the Metabase application
proper, where UI results would show all the columns (in the above
example, they'd immediately show `id` *and* `name`).
* jwt edits
* Optimized 2 images, saved 188.8 KB (46.0% smaller)
* why are y'all 'waiting for the status to be reported'
---------
Co-authored-by: Metabase Automation <github-automation@metabase.com>
* Create Upsell CTA component
* Replace pro labe with gem in embed upsell
* Create and use UpsellCta in Embedding Upsell
* Rebuiltd the logic for showing links and upsells in embedding modal
* Add UpsellCta tests
* Remove unneeded hook
Fix test and URL rendering for hosted
* Remove unneeded hook useInteractiveEmbeddingCta
* Fix test not running in target build (OSS)
* Fix embedded modal ee end premium cloud tests
* Add missing translation tag
* Use single upsellcta styling
* feat: promote uberjar as the single uberjar creation workflow
* chore: remove useless reuse-existing-artifacts
* refactor: streamline PR environment workflow by removing wait_for_uberjar input and simplifying uberjar retrieval
* fix: ensure default editions are used if container info retrieval fails
* chore: don't build uberjar outside of run-tests
* fix: update uberjar artifact name to use the current SHA instead of PR head SHA
* fix: update uberjar artifact names to use PR head SHA when available
* fix: update uberjar workflow to simplify input parameters and improve conditional checks
* fix: add concurrency settings to uberjar workflow
* fix: update documentation condition in run-tests workflow
* fix: add default node version
* fix: update conditions for container info retrieval and containerization steps
* fix: update default node version to 22 in uberjar workflow
* Update .github/workflows/uberjar.yml
* Update .github/workflows/uberjar.yml
---------
Co-authored-by: Uladzimir Havenchyk <125459446+uladzimirdev@users.noreply.github.com>
was blocking on startup
stacktrace below is representative. has a few other things like reveal
but shows that we get into a huge loop over all cards and you end up
waiting for the instance to be ready for a long time
```
at metabase.lib.card$fn__77290$fn__77297.invoke(card.cljc:71)
at metabase.lib.card$fn__77302$__GT_card_metadata_columns77301__77303$fn__77304.invoke(card.cljc:141)
at clojure.core$mapv$fn__8569.invoke(core.clj:7059)
at clojure.lang.PersistentVector.reduce(PersistentVector.java:418)
at clojure.core$reduce.invokeStatic(core.clj:6964)
at clojure.core$mapv.invokeStatic(core.clj:7050)
at clojure.core$mapv.invoke(core.clj:7050)
at metabase.lib.card$fn__77302$__GT_card_metadata_columns77301__77303.invoke(card.cljc:141)
at metabase.lib.card$fn__77302$fn__77308.invoke(card.cljc:120)
at metabase.lib.card$fn__77316$card_cols_STAR_77315__77317.invoke(card.cljc:163)
at metabase.lib.card$fn__77316$fn__77322.invoke(card.cljc:157)
at metabase.lib.card$fn__77350$card_metadata_columns77349__77351.invoke(card.cljc:223)
at metabase.lib.card$fn__77350$fn__77354.invoke(card.cljc:217)
at metabase.lib.card$eval77367$_AMPERSAND_f__77368.invoke(card.cljc:257)
at metabase.lib.card$eval77367$fn__77372.invoke(card.cljc:242)
at clojure.lang.MultiFn.invoke(MultiFn.java:244)
at metabase.lib.metadata.calculation$fn__74776$returned_columns74775__74777$fn__74778.invoke(calculation.cljc:524)
at metabase.lib.metadata.cache$fn__74548$do_with_cached_value74547__74549.invoke(cache.cljc:104)
at metabase.lib.metadata.cache$fn__74548$fn__74555.invoke(cache.cljc:92)
at metabase.lib.metadata.calculation$fn__74776$returned_columns74775__74777.invoke(calculation.cljc:523)
at metabase.lib.metadata.calculation$fn__74776$fn__74781.invoke(calculation.cljc:503)
at metabase.lib.metadata.calculation$fn__74776$returned_columns74775__74777.invoke(calculation.cljc:516)
at metabase.lib.metadata.calculation$fn__74776$fn__74781.invoke(calculation.cljc:503)
at metabase.lib.field.resolution$fn__80775$resolve_in_card_returned_columns80774__80776.invoke(resolution.cljc:352)
at metabase.lib.field.resolution$fn__80775$fn__80784.invoke(resolution.cljc:346)
at metabase.lib.field.resolution$fn__80788$resolve_in_card_or_stage_metadata80787__80789.invoke(resolution.cljc:365)
at metabase.lib.field.resolution$fn__80788$fn__80801.invoke(resolution.cljc:358)
at metabase.lib.field.resolution$resolve_from_previous_stage_or_source_STAR_.invokeStatic(resolution.cljc:410)
at metabase.lib.field.resolution$resolve_from_previous_stage_or_source_STAR_.invoke(resolution.cljc:390)
at metabase.lib.field.resolution$fn__80844$resolve_from_previous_stage_or_source80843__80845.invoke(resolution.cljc:440)
at metabase.lib.field.resolution$fn__80844$fn__80861.invoke(resolution.cljc:434)
at metabase.lib.field.resolution$fn__80865$resolve_field_ref80864__80867.invoke(resolution.cljc:471)
at metabase.lib.field.resolution$fn__80865$fn__80878.invoke(resolution.cljc:457)
at metabase.lib.field$eval80922$_AMPERSAND_f__80923.invoke(field.cljc:62)
at metabase.lib.field$eval80922$fn__80925.invoke(field.cljc:60)
at clojure.lang.MultiFn.invoke(MultiFn.java:239)
at metabase.lib.metadata.calculation$fn__74679$metadata74678__74680$fn__74681.invoke(calculation.cljc:287)
at metabase.lib.metadata.cache$fn__74548$do_with_cached_value74547__74549.invoke(cache.cljc:104)
at metabase.lib.metadata.cache$fn__74548$fn__74555.invoke(cache.cljc:92)
at metabase.lib.metadata.calculation$fn__74679$metadata74678__74680.invoke(calculation.cljc:286)
at metabase.lib.metadata.calculation$fn__74679$fn__74684.invoke(calculation.cljc:275)
at metabase.lib.breakout$fn__80328$breakouts_metadata80327__80329$fn__80331.invoke(breakout.cljc:43)
at clojure.core$mapv$fn__8569.invoke(core.clj:7059)
at clojure.lang.PersistentVector.reduce(PersistentVector.java:418)
at clojure.core$reduce.invokeStatic(core.clj:6964)
at clojure.core$mapv.invokeStatic(core.clj:7050)
at clojure.core$mapv.invoke(core.clj:7050)
at metabase.lib.breakout$fn__80328$breakouts_metadata80327__80329.invoke(breakout.cljc:42)
at metabase.lib.breakout$fn__80328$fn__80334.invoke(breakout.cljc:35)
at metabase.lib.stage$fn__81784$breakouts_columns81783__81785.invoke(stage.cljc:59)
at metabase.lib.stage$fn__81784$fn__81787.invoke(stage.cljc:55)
at metabase.lib.stage$fn__81843$summary_columns81842__81844.invoke(stage.cljc:92)
at metabase.lib.stage$fn__81843$fn__81846.invoke(stage.cljc:86)
at metabase.lib.stage$eval81986$_AMPERSAND_f__81988.invoke(stage.cljc:290)
at metabase.lib.stage$eval81986$fn__81996.invoke(stage.cljc:283)
at clojure.lang.MultiFn.invoke(MultiFn.java:244)
at metabase.lib.metadata.calculation$fn__74776$returned_columns74775__74777$fn__74778.invoke(calculation.cljc:524)
at metabase.lib.metadata.cache$fn__74548$do_with_cached_value74547__74549.invoke(cache.cljc:104)
at metabase.lib.metadata.cache$fn__74548$fn__74555.invoke(cache.cljc:92)
at metabase.lib.metadata.calculation$fn__74776$returned_columns74775__74777.invoke(calculation.cljc:523)
at metabase.lib.metadata.calculation$fn__74776$fn__74781.invoke(calculation.cljc:503)
at metabase.lib.metadata.calculation$fn__74776$returned_columns74775__74777.invoke(calculation.cljc:516)
at metabase.lib.metadata.calculation$fn__74776$fn__74781.invoke(calculation.cljc:503)
at metabase.lib.metadata.calculation$fn__74776$returned_columns74775__74777.invoke(calculation.cljc:513)
at metabase.lib.metadata.calculation$fn__74776$fn__74781.invoke(calculation.cljc:503)
at metabase.lib.metadata.calculation$fn__74776$returned_columns74775__74777.invoke(calculation.cljc:510)
at metabase.lib.metadata.calculation$fn__74776$fn__74781.invoke(calculation.cljc:503)
at metabase.queries.models.card$dataset_query__GT_dimensions.invokeStatic(card.clj:1270)
at metabase.queries.models.card$dataset_query__GT_dimensions.invoke(card.clj:1262)
at metabase.queries.models.card$extract_non_temporal_dimension_ids.invokeStatic(card.clj:1277)
at metabase.queries.models.card$extract_non_temporal_dimension_ids.invoke(card.clj:1274)
at metabase.search.ingestion$execute_function_attr.invokeStatic(ingestion.clj:68)
at metabase.search.ingestion$execute_function_attr.invoke(ingestion.clj:63)
at metabase.search.ingestion$execute_all_function_attrs$fn__131886.invoke(ingestion.clj:80)
at clojure.lang.PersistentHashMap$NodeSeq.kvreduce(PersistentHashMap.java:1309)
at clojure.lang.PersistentHashMap$BitmapIndexedNode.kvreduce(PersistentHashMap.java:804)
at clojure.lang.PersistentHashMap$NodeSeq.kvreduce(PersistentHashMap.java:1314)
at clojure.lang.PersistentHashMap$BitmapIndexedNode.kvreduce(PersistentHashMap.java:804)
at clojure.lang.PersistentHashMap.kvreduce(PersistentHashMap.java:238)
at clojure.core$fn__8559.invokeStatic(core.clj:6987)
at clojure.core$fn__8559.invoke(core.clj:6967)
at clojure.core.protocols$fn__8287$G__8282__8296.invoke(protocols.clj:174)
at clojure.core$reduce_kv.invokeStatic(core.clj:6998)
at clojure.core$reduce_kv.invoke(core.clj:6989)
at metabase.search.ingestion$execute_all_function_attrs.invokeStatic(ingestion.clj:76)
at metabase.search.ingestion$execute_all_function_attrs.invoke(ingestion.clj:73)
at metabase.search.ingestion$__GT_document.invokeStatic(ingestion.clj:87)
at metabase.search.ingestion$__GT_document.invoke(ingestion.clj:85)
at clojure.core$map$fn__5952$fn__5953.invoke(core.clj:2759)
at medley.core$distinct_by$fn__15232$fn__15233.invoke(core.cljc:290)
at clojure.core$map$fn__5952$fn__5953.invoke(core.clj:2759)
at clojure.core$completing$fn__8562.invoke(core.clj:7010)
at clojure.core$map$fn__5952$fn__5953.invoke(core.clj:2759)
at toucan2.pipeline$with_init$fn__51829.invoke(pipeline.clj:327)
at clojure.core$completing$fn__8562.invoke(core.clj:7010)
at toucan2.jdbc.result_set$reduce_result_set.invokeStatic(result_set.clj:156)
at toucan2.jdbc.result_set$reduce_result_set.invoke(result_set.clj:125)
at toucan2.jdbc.query$reduce_jdbc_query.invokeStatic(query.clj:51)
at toucan2.jdbc.query$reduce_jdbc_query.invoke(query.clj:22)
at toucan2.jdbc.pipeline$transduce_execute_with_connection_primary_method_java_sql_Connection_default_default.invokeStatic(pipeline.clj:19)
at toucan2.jdbc.pipeline$transduce_execute_with_connection_primary_method_java_sql_Connection_default_default.invoke(pipeline.clj:9)
at methodical.impl.combo.common$partial_STAR_$fn__22829.invoke(common.clj:15)
at methodical.impl.combo.threaded$combine_methods_thread_last$fn__23165$combined_method_thread_last__23166.invoke(threaded.clj:64)
at methodical.util.FnWithMeta.invoke(util.clj:46)
at methodical.impl.standard$invoke_multifn.invokeStatic(standard.clj:65)
at methodical.impl.standard$invoke_multifn.invoke(standard.clj:47)
at methodical.impl.standard.StandardMultiFn.invoke(standard.clj:216)
at toucan2.pipeline$transduce_execute$with_connection_STAR___51727.invoke(pipeline.clj:78)
at toucan2.connection$bind_current_connectable_fn$fn__51305.invoke(connection.clj:104)
at toucan2.connection$bind_current_connectable_fn$fn__51305.invoke(connection.clj:104)
at toucan2.jdbc.connection$do_with_connection_primary_method_java_sql_Connection.invokeStatic(connection.clj:13)
at toucan2.jdbc.connection$do_with_connection_primary_method_java_sql_Connection.invoke(connection.clj:11)
at methodical.impl.combo.common$partial_STAR_$fn__22829.invoke(common.clj:12)
at methodical.util.FnWithMeta.invoke(util.clj:46)
at toucan2.connection$do_with_connection_around_method_toucan2_connection_default.invokeStatic(connection.clj:118)
at toucan2.connection$do_with_connection_around_method_toucan2_connection_default.invoke(connection.clj:106)
at methodical.impl.combo.common$partial_STAR_$fn__22829.invoke(common.clj:12)
at methodical.util.FnWithMeta.invoke(util.clj:46)
at methodical.impl.standard$invoke_multifn.invokeStatic(standard.clj:55)
at methodical.impl.standard$invoke_multifn.invoke(standard.clj:47)
at methodical.impl.standard.StandardMultiFn.invoke(standard.clj:210)
at toucan2.connection$do_with_connection_primary_method_.invokeStatic(connection.clj:204)
at toucan2.connection$do_with_connection_primary_method_.invoke(connection.clj:194)
at methodical.impl.combo.common$partial_STAR_$fn__22829.invoke(common.clj:12)
at methodical.util.FnWithMeta.invoke(util.clj:46)
at toucan2.connection$do_with_connection_around_method_toucan2_connection_default.invokeStatic(connection.clj:118)
at toucan2.connection$do_with_connection_around_method_toucan2_connection_default.invoke(connection.clj:106)
at methodical.impl.combo.common$partial_STAR_$fn__22829.invoke(common.clj:12)
at methodical.util.FnWithMeta.invoke(util.clj:46)
at methodical.impl.standard$invoke_multifn.invokeStatic(standard.clj:55)
at methodical.impl.standard$invoke_multifn.invoke(standard.clj:47)
at methodical.impl.standard.StandardMultiFn.invoke(standard.clj:210)
at toucan2.pipeline$transduce_execute.invokeStatic(pipeline.clj:77)
at toucan2.pipeline$transduce_execute.invoke(pipeline.clj:64)
at clojure.lang.Var.invoke(Var.java:401)
at toucan2.pipeline$transduce_compiled_query.invokeStatic(pipeline.clj:244)
at toucan2.pipeline$transduce_compiled_query.invoke(pipeline.clj:240)
at toucan2.pipeline$transduce_built_query.invokeStatic(pipeline.clj:252)
at toucan2.pipeline$transduce_built_query.invoke(pipeline.clj:246)
at toucan2.pipeline$transduce_query_primary_method_default.invokeStatic(pipeline.clj:272)
at toucan2.pipeline$transduce_query_primary_method_default.invoke(pipeline.clj:269)
at methodical.impl.combo.common$partial_STAR_$fn__22829.invoke(common.clj:15)
at methodical.impl.combo.threaded$combine_methods_thread_last$fn__23165$combined_method_thread_last__23166.invoke(threaded.clj:64)
at methodical.util.FnWithMeta.invoke(util.clj:46)
at methodical.impl.standard$invoke_multifn.invokeStatic(standard.clj:65)
at methodical.impl.standard$invoke_multifn.invoke(standard.clj:47)
at methodical.impl.standard.StandardMultiFn.invoke(standard.clj:216)
at toucan2.pipeline$transduce_query_STAR_.invokeStatic(pipeline.clj:278)
at toucan2.pipeline$transduce_query_STAR_.invoke(pipeline.clj:274)
at toucan2.pipeline$transduce_with_model.invokeStatic(pipeline.clj:293)
at toucan2.pipeline$transduce_with_model.invoke(pipeline.clj:280)
at toucan2.pipeline$transduce_parsed.invokeStatic(pipeline.clj:309)
at toucan2.pipeline$transduce_parsed.invoke(pipeline.clj:295)
at clojure.lang.AFn.applyToHelper(AFn.java:160)
at clojure.lang.AFn.applyTo(AFn.java:144)
at clojure.core$apply.invokeStatic(core.clj:669)
at clojure.core$apply.invoke(core.clj:662)
at toucan2.pipeline$reducible_fn$reify__51862.reduce(pipeline.clj:404)
at clojure.core$transduce.invokeStatic(core.clj:7025)
at clojure.core.Eduction.reduce(core.clj:7874)
at clojure.core$reduce.invokeStatic(core.clj:6964)
at clojure.core$reduce.invoke(core.clj:6947)
at metabase.util$rconcat$reify__16182.coll_reduce(util.cljc:1188)
at clojure.core$transduce.invokeStatic(core.clj:7026)
at clojure.core.Eduction.reduce(core.clj:7874)
at clojure.core$transduce.invokeStatic(core.clj:7025)
at clojure.core$transduce.invokeStatic(core.clj:7021)
at clojure.core$transduce.invoke(core.clj:7012)
at metabase.search.appdb.index$index_docs_BANG_.invokeStatic(index.clj:279)
at metabase.search.appdb.index$index_docs_BANG_.invoke(index.clj:275)
at metabase.search.appdb.core$populate_index_BANG_.invokeStatic(core.clj:169)
at metabase.search.appdb.core$populate_index_BANG_.invoke(core.clj:168)
at metabase.search.appdb.core$eval132669$fn__132671.invoke(core.clj:182)
at clojure.lang.MultiFn.invoke(MultiFn.java:234)
at metabase.search.core$init_index_BANG_$iter__133794__133798$fn__133799.invoke(core.clj:91)
at clojure.lang.LazySeq.force(LazySeq.java:50)
at clojure.lang.LazySeq.realize(LazySeq.java:89)
at clojure.lang.LazySeq.seq(LazySeq.java:106)
at clojure.lang.RT.seq(RT.java:555)
at clojure.core$seq__5488.invokeStatic(core.clj:139)
at clojure.core.protocols$seq_reduce.invokeStatic(protocols.clj:24)
at clojure.core.protocols$fn__8266.invokeStatic(protocols.clj:74)
at clojure.core.protocols$fn__8266.invoke(protocols.clj:74)
at clojure.core.protocols$fn__8207$G__8202__8220.invoke(protocols.clj:13)
at clojure.core$reduce.invokeStatic(core.clj:6965)
at clojure.core$reduce.invoke(core.clj:6947)
at metabase.search.core$init_index_BANG_.invokeStatic(core.clj:88)
at metabase.search.core$init_index_BANG_.doInvoke(core.clj:80)
at clojure.lang.RestFn.invoke(RestFn.java:411)
at metabase.search.task.search_index$init_BANG_$fn__173098.invoke(search_index.clj:38)
at metabase.app_db.cluster_lock$do_with_cluster_lock_STAR_$with_connection_STAR___126394$with_transaction_STAR___126395.invoke(cluster_lock.clj:61)
at toucan2.connection$bind_current_connectable_fn$fn__51305.invoke(connection.clj:104)
at metabase.app_db.connection$do_transaction$thunk__61025.invoke(connection.clj:145)
at metabase.app_db.connection$do_transaction.invokeStatic(connection.clj:164)
at metabase.app_db.connection$do_transaction.invoke(connection.clj:141)
at metabase.app_db.connection$do_with_transaction_primary_method_java_sql_Connection.invokeStatic(connection.clj:201)
at metabase.app_db.connection$do_with_transaction_primary_method_java_sql_Connection.invoke(connection.clj:174)
at methodical.impl.combo.common$partial_STAR_$fn__22829.invoke(common.clj:13)
at methodical.util.FnWithMeta.invoke(util.clj:46)
at toucan2.connection$do_with_transaction_around_method_toucan2_connection_default.invokeStatic(connection.clj:249)
at toucan2.connection$do_with_transaction_around_method_toucan2_connection_default.invoke(connection.clj:245)
at methodical.impl.combo.common$partial_STAR_$fn__22829.invoke(common.clj:13)
at methodical.util.FnWithMeta.invoke(util.clj:46)
at methodical.impl.standard$invoke_multifn.invokeStatic(standard.clj:58)
at methodical.impl.standard$invoke_multifn.invoke(standard.clj:47)
at methodical.impl.standard.StandardMultiFn.invoke(standard.clj:212)
at metabase.app_db.cluster_lock$do_with_cluster_lock_STAR_$with_connection_STAR___126394.invoke(cluster_lock.clj:53)
at toucan2.connection$bind_current_connectable_fn$fn__51305.invoke(connection.clj:104)
at toucan2.connection$bind_current_connectable_fn$fn__51305.invoke(connection.clj:104)
at toucan2.connection$bind_current_connectable_fn$fn__51305.invoke(connection.clj:104)
at toucan2.jdbc.connection$do_with_connection_primary_method_javax_sql_DataSource.invokeStatic(connection.clj:18)
at toucan2.jdbc.connection$do_with_connection_primary_method_javax_sql_DataSource.invoke(connection.clj:15)
at methodical.impl.combo.common$partial_STAR_$fn__22829.invoke(common.clj:12)
at methodical.util.FnWithMeta.invoke(util.clj:46)
at toucan2.connection$do_with_connection_around_method_toucan2_connection_default.invokeStatic(connection.clj:118)
at toucan2.connection$do_with_connection_around_method_toucan2_connection_default.invoke(connection.clj:106)
at methodical.impl.combo.common$partial_STAR_$fn__22829.invoke(common.clj:12)
at methodical.util.FnWithMeta.invoke(util.clj:46)
at methodical.impl.standard$invoke_multifn.invokeStatic(standard.clj:55)
at methodical.impl.standard$invoke_multifn.invoke(standard.clj:47)
at methodical.impl.standard.StandardMultiFn.invoke(standard.clj:210)
at metabase.app_db.connection$do_with_connection_primary_method_default.invokeStatic(connection.clj:132)
at metabase.app_db.connection$do_with_connection_primary_method_default.invoke(connection.clj:130)
at methodical.impl.combo.common$partial_STAR_$fn__22829.invoke(common.clj:12)
at methodical.util.FnWithMeta.invoke(util.clj:46)
at toucan2.connection$do_with_connection_around_method_toucan2_connection_default.invokeStatic(connection.clj:118)
at toucan2.connection$do_with_connection_around_method_toucan2_connection_default.invoke(connection.clj:106)
at methodical.impl.combo.common$partial_STAR_$fn__22829.invoke(common.clj:12)
at methodical.util.FnWithMeta.invoke(util.clj:46)
at methodical.impl.standard$invoke_multifn.invokeStatic(standard.clj:55)
at methodical.impl.standard$invoke_multifn.invoke(standard.clj:47)
at methodical.impl.standard.StandardMultiFn.invoke(standard.clj:210)
at toucan2.connection$do_with_connection_primary_method_.invokeStatic(connection.clj:204)
at toucan2.connection$do_with_connection_primary_method_.invoke(connection.clj:194)
at methodical.impl.combo.common$partial_STAR_$fn__22829.invoke(common.clj:12)
at methodical.util.FnWithMeta.invoke(util.clj:46)
at toucan2.connection$do_with_connection_around_method_toucan2_connection_default.invokeStatic(connection.clj:118)
at toucan2.connection$do_with_connection_around_method_toucan2_connection_default.invoke(connection.clj:106)
at methodical.impl.combo.common$partial_STAR_$fn__22829.invoke(common.clj:12)
at methodical.util.FnWithMeta.invoke(util.clj:46)
at methodical.impl.standard$invoke_multifn.invokeStatic(standard.clj:55)
at methodical.impl.standard$invoke_multifn.invoke(standard.clj:47)
at methodical.impl.standard.StandardMultiFn.invoke(standard.clj:210)
at metabase.app_db.cluster_lock$do_with_cluster_lock_STAR_.invokeStatic(cluster_lock.clj:53)
at metabase.app_db.cluster_lock$do_with_cluster_lock_STAR_.invoke(cluster_lock.clj:51)
at metabase.app_db.cluster_lock$fn__126406$do_with_cluster_lock126405__126407$do_with_cluster_lock_STAR__STAR___126409.invoke(cluster_lock.clj:78)
at clojure.lang.AFn.applyToHelper(AFn.java:152)
at clojure.lang.AFn.applyTo(AFn.java:144)
at clojure.core$apply.invokeStatic(core.clj:667)
at clojure.core$apply.invoke(core.clj:662)
at metabase.util.retry$decorate$fn$reify__126370.call(retry.clj:110)
at io.github.resilience4j.retry.Retry.lambda$decorateCallable$5(Retry.java:306)
at metabase.util.retry$decorate$fn__126369.doInvoke(retry.clj:111)
at clojure.lang.RestFn.invoke(RestFn.java:400)
at metabase.util.retry$fn__126375$make126374__126376$fn__126377.invoke(retry.clj:120)
at metabase.app_db.cluster_lock$fn__126406$do_with_cluster_lock126405__126407.invoke(cluster_lock.clj:82)
at metabase.app_db.cluster_lock$fn__126406$fn__126412.invoke(cluster_lock.clj:63)
at metabase.app_db.cluster_lock$fn__126406$do_with_cluster_lock126405__126407.invoke(cluster_lock.clj:75)
at metabase.app_db.cluster_lock$fn__126406$fn__126412.invoke(cluster_lock.clj:63)
at metabase.search.task.search_index$init_BANG_.invokeStatic(search_index.clj:37)
at metabase.search.task.search_index$init_BANG_.invoke(search_index.clj:33)
at metabase.search.task.search_index$eval173156$fn__173157.invoke(search_index.clj:46)
at metabase.startup.core$run_startup_logic_BANG_$fn__173079.invoke(core.clj:30)
at metabase.startup.core$run_startup_logic_BANG_.invokeStatic(core.clj:28)
at metabase.startup.core$run_startup_logic_BANG_.invoke(core.clj:24)
at metabase.core.core$init_BANG__STAR_.invokeStatic(core.clj:181)
at metabase.core.core$init_BANG__STAR_.invoke(core.clj:117)
at metabase.core.core$init_BANG_.invokeStatic(core.clj:194)
at metabase.core.core$init_BANG_.invoke(core.clj:189)
at dev$init_BANG_.invokeStatic(dev.clj:143)
at dev$init_BANG_.invoke(dev.clj:139)
at dev$start_BANG_.invokeStatic(dev.clj:180)
at dev$start_BANG_.invoke(dev.clj:176)
at dev$eval206099.invokeStatic(NO_SOURCE_FILE:1)
at dev$eval206099.invoke(NO_SOURCE_FILE:1)
at clojure.lang.Compiler.eval(Compiler.java:7739)
at clojure.lang.Compiler.eval(Compiler.java:7729)
at clojure.lang.Compiler.eval(Compiler.java:7694)
at clojure.core$eval.invokeStatic(core.clj:3232)
at clojure.core$eval.invoke(core.clj:3228)
at vlaaad.reveal.repl$wrap_eval$fn__37945.invoke(repl.clj:90)
at clojure.main$repl$read_eval_print__9248$fn__9251.invoke(main.clj:437)
at clojure.main$repl$read_eval_print__9248.invoke(main.clj:437)
at clojure.main$repl$fn__9257.invoke(main.clj:459)
at clojure.main$repl.invokeStatic(main.clj:459)
at clojure.main$repl.doInvoke(main.clj:368)
at clojure.lang.RestFn.applyTo(RestFn.java:140)
at clojure.core$apply.invokeStatic(core.clj:667)
at clojure.core$apply.invoke(core.clj:662)
```
Previously we were treating all currency formatting as if we were in a
table, and only adding the currency symbol if `currency-in-header` were
explicitly set to `false` for the column.
When formatting a scalar value, though, we want to always add the
currency symbol inline (since there is no header in this case).
I found that `metabase.channel.render.body/format-cell` and
`metabase.formatter.core/format-number` were actually only used to
format *scalar* cells and *scalar* numbers (as opposed to cells in a
table), so I renamed them for a bit more clarity and added a new
optional argument, `scalar?`, to
`metabase.formatter.core/number-formatter`. When passed (and truthy) the
currency symbol will be splatted into the formatted number even if
`:currency-in-header` is not `false`.
* minor edits
* Update docs/permissions/row-and-column-security.md
Co-authored-by: Alex Yarosh <alexandra@metabase.com>
---------
Co-authored-by: Alex Yarosh <alexandra@metabase.com>
instead just pass `(not-empty q)` to the search subsystem, which will
convert it to `nil` if it's empty and pass it unchanged if it's not.
I still don't think this is probably the ideal user experience when you
navigate to `/search` but it's not an error at least.
* UXW-382 Updates "Public link" menu item for admins when public links aren't enabled so it doesn't look obviously broken
* UXW-382 Updates tests
* UXW-382 Updates "enable public links" menu item to be disabled per PR feedback
* UXW-382 Swaps core styles for mantine props per PR feedback
* fix: issues generating valid urls for map tiles
Fixes to issues generating map tiles, one is on x-ray dashboards we need
to check that we're not in a real dashboard and send an ad hoc tile
request.
Second if the field ref sent in the request has options we need to
remove the 'base-type' option because the type will be encoded as
something like test%2FFloat and the way Jetty decodes URLs before
parsing turns it into a '/' in a position a path separator cannot be:
https://github.com/jetty/jetty.project/issues/7713
* fixup! typeguard dashbaord id
* Update maps.cy.spec.js
* fixup! better handle null field opts
* Re-implementation of `metabase.driver.common.parameters.*` that uses Lib
* Fix Kondo warnings
* Move tests that have nothing to do with drivers to QP
* Test fixes 🔧
* Test fixes 🔧
* Try simplifying the PR
* db routing for redshift
* simplify test
* dataset name multimethod
* small change
* use delayed defs
* add redshift ci routing env var
* small fix
* update dataset definitions
* change back to original data
* feat: driver/do-with-resilient-connection
* feat: :keep-open? option to do-with-connection-with-options
* feat: attempt describe-database with resilient connection
* feat: execute jdbc query with retry conn
* test
* avoid recursive issues
* simplify
* changelog
* type hint
* force local conn, so the connection being closed won't close outer unrealized result sets
* working test
* more test
* opts
* style
* add test for connection options
* notes
* fix test
* docstring
* Update src/metabase/driver/sql_jdbc/execute.clj
Co-authored-by: lbrdnk <lbrdnk@users.noreply.github.com>
* logs
* better logs
* try to close conn when it's no longer valid
* pull out logic to is-conn-open
* is-conn-open? test
* clarify
* Update docs/developers-guide/driver-changelog.md
Co-authored-by: lbrdnk <lbrdnk@users.noreply.github.com>
* update version
* chore: move tests to right namespace
* retry have select privilege on error
* add test
* improve test
* remove let
* separate connection for each test
* clean up test
* try druid jdbc ci fix
* close connection during test
* use do with resilient connection in test for druid
* formatting
* always set role if supported
* Fix clickhouse and sqlserver
* impersonation test
* use normal-driver-select
* address comments
* kondo
* remove debug
---------
Co-authored-by: Nicola Mometto <nicola.mometto@metabase.com>
Co-authored-by: lbrdnk <lbrdnk@users.noreply.github.com>
* UXW-28 Adds confirmation modal when deleting a tab that contains dashboard-only cards
* UXW-28 Makes confirmation modal smarter; uses short version if all questions on a tab are dashboard questions, lists affected questions if it's a mix
* UXW-28 Adds tests
* feat: driver/do-with-resilient-connection
* feat: :keep-open? option to do-with-connection-with-options
* feat: attempt describe-database with resilient connection
* feat: execute jdbc query with retry conn
* test
* avoid recursive issues
* simplify
* changelog
* type hint
* force local conn, so the connection being closed won't close outer unrealized result sets
* working test
* more test
* opts
* style
* add test for connection options
* notes
* fix test
* docstring
* Update src/metabase/driver/sql_jdbc/execute.clj
Co-authored-by: lbrdnk <lbrdnk@users.noreply.github.com>
* logs
* better logs
* try to close conn when it's no longer valid
* pull out logic to is-conn-open
* is-conn-open? test
* clarify
* Update docs/developers-guide/driver-changelog.md
Co-authored-by: lbrdnk <lbrdnk@users.noreply.github.com>
* update version
* chore: move tests to right namespace
* impersonation test
* use normal-driver-select
* trigger CI
---------
Co-authored-by: lbrdnk <lbrdnk@users.noreply.github.com>
* feat(simple-embed): parse attributes with json5 to be more permissive with quotes
* Update package.json
Co-authored-by: cubic-dev-ai[bot] <191113872+cubic-dev-ai[bot]@users.noreply.github.com>
* fix tests
* Apply suggestion from @heypoom
Co-authored-by: Phoomparin Mano <poom@metabase.com>
---------
Co-authored-by: cubic-dev-ai[bot] <191113872+cubic-dev-ai[bot]@users.noreply.github.com>
Co-authored-by: Phoomparin Mano <poom@metabase.com>
* Remove debugging stuff
* 14 failures, 1 error.
* 10 failures, 1 error.
* 10 failures, 0 errors.
* Better log messages
* Faster field ref resolution
* Test fixes 🔧
* Some test fixes 🔧
* Remove resolve-joined-fields middleware
* Fix lib.underlying calculation of fields with join aliases
* Fix busted pivot tests
* Remove unused fn
* More resolution fixes
* drill-thru use vars so we can get updated fns in the REPL
* Consolidate the lib metadata => legacy metadata functions
* Add a pile of additional tests
* Migrate the `resolve-joined-fields` middleware (#39884)
* Support automatically removing duplicate refs in `:fields` and `:breakouts` during normalization
* More cleanup and extra tests 🔧
* Add ::mbql.s/OrderBys
* Mess with logging in resolution even more
* Order bys should be distinct ignoring keys like :base-type or namespaced keys
* I have fixed#60587
* One last round of test fixes??
* Test fix
* Code cleanup
* Fix test failures 🔧
* MongoDB test fix
* Fix Oracle test 🔧
* MongoDB test fix 🔧
* cljfmt
* e2e test fix?
* Test fix 🔧
* Less noisy logging
* Improve field resolution for implicit joins; PR feedback (part 1)
* PR feedback (part 2) and extra dox
* Run tests if files in lib/drivers/qp are changes
check src/metabase/lib, src/metabase/query_processor, modules/drivers
we key off of team label but this is hopefully close enough to what we
really want
* run an action linter
```
❯ actionlint /Users/dan/projects/work/metabase/.github/workflows/drivers.yml
.github/workflows/drivers.yml:440:71: property "exclude-tag" is not defined in object type {name: string; test-args: string} [expression]
.github/workflows/drivers.yml:537:15: description is required in metadata of "Wait for localhost port to be ready" action at "/Users/dan/projects/work/metabase/.github/actions/await-port/action.yml" [action]
|
537 | - uses: ./.github/actions/await-port
| ^~~~~~~~~~~~~~~~~~~~~~~~~~~~
.github/workflows/drivers.yml:587:13: description is required in metadata of "Test database driver" action at "/Users/dan/projects/work/metabase/.github/actions/test-driver/action.yml" [action]
|
587 | uses: ./.github/actions/test-driver
| ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~
.github/workflows/drivers.yml:743:71: property "exclude-tag" is not defined in object type {name: string; test-args: string} [expression]
```
acitonlint seems pretty nice
not fixing the exclude-tag since that seems to work just fine
* fix syntax
* fetch base branch for file checks
to see if we changed files, need the base
* indentation is fun. super fast iteration
* resequence if before run
* run third tier on devex team label
* empty to poke CI
* make prepare-statement private
probably good for this to be private either way, but this is done now to
kick CI and run tests that aren't running in CI
* close single quote
We're using a setting as a half-baked way to track whether we've ever
created a non-table chart. We do this regardless of whether you're an
admin or not, but the endpoint responds with a 403 if you're not an
admin.
Honestly motivation here was fixing [UXW-152: 403 thrown for non-admins
when viewing question re:
non-table-chart-generated](https://linear.app/metabase/issue/UXW-152/403-thrown-for-non-admins-when-viewing-question-re-non-table-chart)
but it feels a little silly. But... it's not *completely* unreasonable.
* dev: add the ability to run native ddl as part of dataset definitions
* Fixing tests
* Fix kondo
* Fix tests and kondo
* Fix clickhouse tests
* Fix another clickhouse test
* Fix bigquery test
* Fix bigquery test
* Trigger Build
* fix: filter-out non-string attributes
We expected login_attributes to only be strings. When they are created
through SAML or JWT logins we don't actually enforce this, so this
filters out non-string attributes printing a warning message if they are
passed for these login types.
* feat: auto assign jwt users to groups by name
When group sync is turned on and there are no group mappings defined for
the jwt integration.
* fixup! remove tenant user group
* fixup! correct bad merge
* fixup! remove some tenant_ids in tests
* fixup! more test fixes
* feat: make jwt group attribute admin configurable
* feat: update group mapping text for jwt behavior
* feat(new-embed): log an error when the parsing of a json attribute fails
* Update enterprise/frontend/src/metabase-enterprise/embedding_iframe_sdk/webcomponents.ts
Co-authored-by: Phoomparin Mano <poom@metabase.com>
---------
Co-authored-by: Phoomparin Mano <poom@metabase.com>
* fix: allow scalar user-attributes
Only drop collections which cannot be converted to strings when
generating database queries.
* fixup! remove unused namespace
* fixup! use keep to retain false booleans
* fix: filter-out non-string attributes
We expected login_attributes to only be strings. When they are created
through SAML or JWT logins we don't actually enforce this, so this
filters out non-string attributes printing a warning message if they are
passed for these login types.
* feat: auto assign jwt users to groups by name
When group sync is turned on and there are no group mappings defined for
the jwt integration.
* fixup! remove tenant user group
* fixup! correct bad merge
* fixup! remove some tenant_ids in tests
* fixup! more test fixes
* feat: make jwt group attribute admin configurable
The workflow in the `metabase-ee-extra` repository could run before the
container image tags were created by the workflow in the `metabase`
repository, causing the former to fail. Fix by making the image push
an explicit dependency of the ee-extra trigger.
This works by just totally removing the permission grants created by
sandboxes. Instead we use the `gtapped-table` flag added by the
sandboxing middleware to add just the gtapped-table itself to set of
necessary table create-query permissions and exclude any joined tables
from being added to this set.
If those tables are included elsewhere in the query (not the part
created by the sandbox), they will be added to the set of tables to
check normally and the query will be blocked if the user doesn't have
query-builder perms.
Child collections are automatically created with the same namespace as
their parent collection. Since we enforce a validation that this must be
true it seems simpler for api clients to have this value set for them
rather than requiring them to supply the namespace attribute (though it
still remains optional to pass one.)
* fix: filter-out non-string attributes
We expected login_attributes to only be strings. When they are created
through SAML or JWT logins we don't actually enforce this, so this
filters out non-string attributes printing a warning message if they are
passed for these login types.
* feat: auto assign jwt users to groups by name
When group sync is turned on and there are no group mappings defined for
the jwt integration.
* fixup! remove tenant user group
* fixup! correct bad merge
* fixup! remove some tenant_ids in tests
* fixup! more test fixes
* Update common.clj
* wip
* experiment of callback to see if it works on vue with .callback={}
* basic e2e test for dashboard
* more tests and cleanup
* remove my-app playground
* rm unused file
* use custom elements for the preview
* support for async and defer, defaulting to defer
* clean up custom elements code
* rename drills props
* Revert "experiment of callback to see if it works on vue with .callback={}"
This reverts commit 89a38dca47.
* fix theme not being restored from persisted settings
* fix: load script for preview from instance url, make iframe loaded data
attribute work
* change the snippet to use new defer syntax
* support for exploration with question-id=new
* cleanup
* fix types
* Revert "fix types"
This reverts commit bbc8eb8b59.
* Revert "support for exploration with question-id=new"
This reverts commit 4c3f041629.
* reimplement support for exploration
* fix parameters conditional rendering
* fix linting issues
* fix embed code snippet
* fix(new-embed): allow multiple questions/dashboards on the same page
* test for multiple questions
* add note about theme
* feat(sdk): expose the title prop in StaticQuestion (#61001)
* attempot to make tests more stable
* fix types
* maybe fix tests?
* test for ci
* put back support for is-save-enabled
* H.getIframeBody -> H.getSimpleEmbedIframeContent as it should be less flaky
* restore logic
* fix exploration not working in test
* moved comment to the correct place
* potential fix for handleMessage, ignore messages from other iframes
* use question for exploration templates
* fix type errors
* skip flaky "persists default and hidden parameters"
* fix template: exploration in embed flow
* always enable embedding-simple on embed flow tests
* refactor the custom elements tests to use the generic helpers
* fix tests :fingercrossed:
* default title to true
* update helper to grab the first 37.65
* default drills to true
* we're now showing the title by default
---------
Co-authored-by: Phoomparin Mano <phoomparin@gmail.com>
* Do not add new extra empty stages when converting a join from legacy
* Update gitignore
* Simplified fix for #60444
* Fix QUE-1607
* Test fix 🔧
* Fix QUE-1621
* Additional tests.
* Test fix 🔧
* Remove dead code -- schemas for idents in legacy MBQL schema
* Test fix 🔧
* Fix QUE-1623
* Test fix 🔧
* Fix failing tests
* Test fixes 🔧
* Make sure join metadata goes in the right place
* Additional tests
* Make sure join metadata goes in the right place
* Less verbose logging
* Test fixes 🔧
* Some extra tests 🔧
* Another test fix 🔧
* Remove unused export
* Cleanup
* Code cleanup
* Update tests 🔧
* test fixes
* Improve/fix metadata calculation
* Test fixes 🔧
* Update tests
* Test fix 🔧
* Test fixes 🔧
* Test fix 🔧
* Test fixes 🔧
* PR feedback
* Test fix 🔧
* allow tenant attributes CRUD (#60347)
wip merge user and tenant attributes
WIP use new api
Split Mapping Editor
types fixed
so much unit testing
handle null cases
update tests
update a couple more e2e tests
less silly test
make things nicer
invalidate user cache on tenant update
* feat: separate jwt-set attributes from manual-set
* fixup! typescript stuff
* feat: restore structrue-attributes merging
Co-Authored-by: Dan Sutton <dan@metabase.com>
* feat: use merged attributes in routing/sandbox/etc
Co-Authored-By: John Swanson <john.swanson@metabase.com>
* fixup! move group manager test to ee
* fixup! kondo
* fixup! modal spec
* fixup! cypress tests
* fixup! migration timestamp
* fixup! one more cherry pick oopsie
* fixup! sandboxing cypress helper fixes
* fixup! more cypress updates
* fixup! remove eslint directive
* fixup! remove 2-arity router-dbor-id->destination-db-id
All the places it was called with 2-arity were explicitly passing in
@api/current-user anyway
---------
Co-authored-by: Ryan Laurie <30528226+iethree@users.noreply.github.com>
Co-authored-by: Dan Sutton <dan@metabase.com>
Co-authored-by: John Swanson <john.swanson@metabase.com>
* fix: handle headerless translations csv
* fixup! PR feedback
Also fixes an issue with error line number reporting without headers
* feat: allow csv headers to be in any order
* fixup! error reporting test
* fix: remove categories on push
* fix: run each project individually
* fix: run jobs in parallel with pom caching
* fixup! dry it up
* Update snyk.yml
* chore: test deeply nested cards with mixed perms
* chore: test custom expressions and aggregations
* chore: add more tests for blocks and joins
* fixup! whitespace
* fixup! remove confusing comments
Fixes#61253.
Here's the breakdown of the issue, since it's subtle:
- These objects can contain some vanilla JS objects which in embedding
contexts can be `Object.preventExtensions()`'d.
- Eg. the `:visualization-settings` on a card or metric.
- `lib.metadata.cache` uses these objects as part of the cache key.
- Those become the keys of a CLJS map, which uses `cljs.core/hash`
- JS objects and arrays have pointer equality (`identical?` or `===`)
semantics.
- The default `cljs.core.IHash/-hash` uses `goog.getUid(obj)`.
- That subtly mutates objects, adding a non-`enumerable`
`closure_uid_123456789` property with an arbitrary sequence number.
- That's adding a property, which is a violation if
`Object.preventExtensions()` (or `Object.freeze()`) has been used.
- That silently does nothing by default, but throws in strict mode.
This PR modifies the cache key logic (which already has special handling
for `:lib/metadata`) to replace `:lib/type :metadata/card` and
`:metadata/metric` maps with a stub like `[:metric/card 123]`.
* drop unused redux functions
* drop time parsing in event cards
* replace moment types with dayjs
* use dayjs for tz guessing
* drop moment from a hook
* drop unused time functions
* Do not add new extra empty stages when converting a join from legacy
* Update gitignore
* Fix QUE-1607
* Test fix 🔧
* Fix QUE-1621
* Additional tests.
* Test fix 🔧
* Remove dead code -- schemas for idents in legacy MBQL schema
* Test fix 🔧
* Fix QUE-1623
* Test fix 🔧
The test gives a bit of false confidence, as it passed before we added the forced re-indexing at the end of the import.
We probably want a way to keep the transaction open until after the ingestor has processed the messages from the toucan hooks... but that sounds quite invasive and brittle to pull off.
If a locale is not defined for metabase we should not allow
content-translation dictionaries to uploaded with it. Since these
locales can't be used anyway it results in a confusing user experience.
* Populate search index col of non temporal dimension ids using clojure fn
* Add non_temporal_dim_ids filter param to search request
* fix kondo
* Add stub impl of new filter to in-place
* Add metadata provider cache
* Adding has-temporal-dim filter; fix tests
* fix lint
* Fix merge
* Hookup frontend to proper BE search filters; remove deprecated impl of has-temporal-dimensions
* Fix kondo
* Fix lingering refs to has-temporal-dimensions
* Fix lingering refs to has-temporal-dimensions
* fix test
* Fix test
* Remove FE compatibility code that was ported to BE; fix typing of partition helper
* Add helper for parsing non-temporal-dim-ids endpoint param; fix the conditional in FE
* remove debug let
* Add pr ref to comment
* Include dimension filter params in request only if cartesian
* remove specs that depended on checkDimensionCompatibilityForCartesianCharts
* PR feedback
* use existing dataset query transform
* Optimized 1 images, saved 76.0 KB (48.2% smaller)
---------
Co-authored-by: Metabase Automation <github-automation@metabase.com>
* Prg-81-implement-logic-for-opening-store-and-receiving-messages (#60702)
* Add new self hosted upsell flow path
* Add upsell flow hook
* Add company name to params passed to store
* Update url parameters to snake case
* Remove redundant ampersand
* Clean up useEffect dependencies
* Add store origin to token activation message
* Improve put token mock
* Snake case return_url param
* Update messages' types
* Add source check on event
* Use dynamic store value (#60865)
* Create selector for store url
* Use dynamic store value in use upsell flow
* Improve store url selector
* Create useStoreUrl hook and deprecate getStoreUrl selector
* Increase the timeout of toast messages (#60966)
* increase the timeout of toast messages
* Send notification even if reference to store window was lost
* Add metabase window name (#60964)
* Prg-91-migrate-upsell-components-to-support-ee-upsell-flow (#60850)
* Enhance upsell functionality by adding onClick support to UpsellBanner and integrating useUpsellFlow in admin settings
Fix function name
* Use match pattern in UpsellBanner
* Add Upsell baner with click stories and tests
* Add upsell flow to dev instances banner
* Integrate upsell flow into UpsellBanner components
* Fix unrelated tests
* Improve the test assertion
* Prg-95-migrate-upsellcard-to-the-new-upsell-flow (#60989)
* Add onClock to UpsellCard component
* Migrate UpsellCard components to use self hosted upsell flow
* Disable new flow in dev instances upsell (#61048)
* Allow overriding store-url in dev env (#60967)
* Fix upsell flow with undefined store variable (#61056)
* Show failure toast even if connection with store was lost
* Add store-url to e2e default db setup
* Allow undefined store-url value in use upsell flow
* Fix failing unit tests
* Prg 96 migrate upsellbigcard to the new upsell flow (#61063)
* Migrace BigCard to use onClick
* Migrate BigCard upsells
* Add onClick to UpsellPill (#61076)
* Prg-104-run-the-new-upsell-flow-only-for-self-hosted-mb-instances (#61122)
* Disable new upsell flow on hosted instances
* Fix wrong match on UpsellBigCard
* Use getIsHosted selector
add test
* Fix e2e test - selecting upsell button
* Fix extra toast triggered with new upsells
* Add upsell flow to embedding ugprade link, remove binary notice from EE self serve (#61207)
* Fix typos
* Use existing link styles
* Fix tags invalidation for bulk tables update
* Add MassTableVisibilityToggle
* Render mass toggle for schemas
* Add the button for dbs
- fix not-currently-selected branches in the tree not updating
* Fix types
* Align icon
* Align chevron icon
* Always show the toggle button for hidden tables
* Change color of hidden tables
* Only use ids of relevant tables
* Add tests
* Add test
* Take all visibility options into account
* Rename MassTableVisibilityToggle to BulkTableVisibilityToggle
* Fix color
* Render table picker items as anchors
* Remove measuring elements as elements have constant size
* Fix types and tests
* Add test
* [Lib] Finish removing idents
* Fix Kondo warnings
* cljfmt
* Another pass at fixing Kondo warnings
* Test fix 🔧
* Test fix 🔧
* Test fixes 🔧
* A few more fixes 🔧
* Another pass
* Test fix 🔧
* Test fixes 🔧
* Test fix 🔧
* Test fixes 🔧
Suspecting that instances OOM due to loading these into
memory. Increasing context and logging to help diagnose this:
When running tests:
```
clj -X:dev:test :only '["test/metabase/notification" "test/metabase/pulse"]'
...
DEBUG metabase.notification.payload.temp-storage - stored data in temp file {:length 57167}
[main] DEBUG metabase.notification.payload.temp-storage - stored data in temp file {:length 718167}
[main] INFO metabase.notification.payload.temp-storage - reading temp storage file of size: 55.8 kb
[main] INFO metabase.notification.payload.temp-storage - reading file with preamble: {:dashboard_tab_id nil, :card_id 198, :dashboard_id 77}
[main] INFO metabase.notification.payload.temp-storage - reading temp storage file of size: 55.8 kb
[main] INFO metabase.notification.payload.temp-storage - reading file with preamble: {:dashboard_tab_id nil, :card_id 198, :dashboard_id 77}
[main] INFO metabase.notification.payload.temp-storage - reading temp storage file of size: 701.3 kb
[main] INFO metabase.notification.payload.temp-storage - reading file with preamble: {:dashboard_tab_id nil, :card_id 199, :dashboard_id 77}
[main] INFO metabase.notification.payload.temp-storage - reading temp storage file of size: 701.3 kb
[main] INFO metabase.notification.payload.temp-storage - reading file with preamble: {:dashboard_tab_id nil, :card_id 199, :dashboard_id 77}
```
Adding an optional preamble that is just `pr-str`'d and logging a human
readable size of the file that is saved
* add-dimension-projections middleware => Lib (#39877)
* Ban mt/with-column-remappings inside QP tests
* Update pivot QP code to not convert to legacy
* Rename middleware
* Use the remap metadata provider in sandboxing tests
* Overhaul add-implicit-joins middleware
* Restore the wacko metadata updating code
* Ok, joins ARE allowed to have :fields.
* Fix JS tests 🔧
* Fix QUE-1563
* Test fixes 🔧
* Fix 100 more broken things
* Test fixes
* Use same distinct-ref logic as lib
* More test fixes 🔧
* Make add-alias-info better at matching things
* Consolidate the legacy result metadata schemas
* cljfmt
* Use `:decode/normalize` in the REST API for boring things like metadata
* Another round of test fixes 🔧
* Restore kondo ignore
* Another round of test fixes 🔧
* cljfmt
* PR feedback 🔧
* chore: remove wait in favour of waiting for modal to disappear
* feat: wait for query to be completed to possibly avoid DB race condition
* feat: wait for the table name to be shown in the ui before clicking it
* chore: removed un-needed waits
* Update e2e/test/scenarios/admin/admin-reproductions.cy.spec.js
Co-authored-by: Ryan Laurie <30528226+iethree@users.noreply.github.com>
---------
Co-authored-by: Ryan Laurie <30528226+iethree@users.noreply.github.com>
* Move expression widget into subfolder
* Import expression components from top-level
* Move expression helpers to metabase/querying/expressions
* Fix fuzzing paths
* Fix comment
* Add missing exports from expressions
* Remove unused exports
* Sort exports
* Add helper to get supported clauses
* Rename complete to suggestions for consistency
* Move tokenAtPos to position module
* Move assertion helpers to utils
* Split up pratt/types
* Use token text instead of slicing source
* Remove superflouous from and to on token
* Remove superflouous from, to and len on token
* Remove content helper
* Remove superflous fields on Token
* Rename compiler to compile-expression
* Fix imports
* Fix exports
* Remove eslint directives
* Move FK_SYMBOL to formatting/constants
* Use () => undefined instead of jest.fn
* Fix tags invalidation for bulk tables update
* Add MassTableVisibilityToggle
* Render mass toggle for schemas
* Add the button for dbs
- fix not-currently-selected branches in the tree not updating
* Fix types
* Align icon
* Align chevron icon
* Always show the toggle button for hidden tables
* Change color of hidden tables
* Only use ids of relevant tables
* Add tests
* Add test
* Take all visibility options into account
* Rename MassTableVisibilityToggle to BulkTableVisibilityToggle
* Allow supression of case and coalesce arg typechecks
* Add test for ignored type-check
* Configure *suppress-expression-type-check?* in tests
* Fix formatting
As is, field filters and time groupings assume that they can just refer to the field normally. In queries where aliases are
used/required, this doesn't work so well. Letting users specify the alias to use in place of the normal field
reference handles this case.
---------
Co-authored-by: Alexander Polyankin <alexander.polyankin@metabase.com>
* improve loading experience logic and fix bug where there are tool calls without messages causing messages to get cramed together with no space
* add a unit test
* Add `extraAction` prop to `undo` object
* Make `setEditingParameter` switch tab if needed
* Put a dashboard filter in the viewport
* Fix toast text appearance
* Hide "Show filter" button when moving to top nav
* Add e2e test
* Fix tests
* Ensure postgres downloads don't OOM
Fixes https://github.com/metabase/metabase/issues/60733
For performance reasons we set autocommit to true. this has an
unfortunate consequence that the postgres driver does not honor batch
sizes. We go through a lot of trouble to hook the streaming results of a
query to the output stream of the download, but the postgres jdbc driver
will realize all rows in memory.
A fast way forward is to set autocommit to false only when
`write?` (previous condition) or when executing a postgres download, as
reported in https://github.com/metabase/metabase/issues/60733.
How to test:
this is difficult. In CI we use
```clojure
:ci
{:jvm-opts ["-Xmx12g"
"-Xms12g"
```
So hitting a query that would deterministically OOM the instance
requires a lot more muscle than we realistically want to use.
I'm running the following tests locally:
```
:small-ram
{:jvm-opts ["-Xmx400m"
"-Xms400m"
"-Dci=TRUE"]}
```
```clojure
(defn- readable [bytes]
(cond (nil? bytes) "0 bytes"
(zero? bytes) "0 bytes"
:else
(let [units ["b" "kb" "mb" "gb" "tb"]
magnitude 1024.0]
(loop [[unit & remaining] units
current (double bytes)]
(if (and (seq remaining) (> current magnitude))
(recur remaining (/ current magnitude))
(format "%.1f %s" current unit))))))
(defn- consume
[^java.io.InputStream is]
(let [arr (byte-array 8024)]
(loop [c 0, gas 5000000]
(let [r (java.io.InputStream/.read is arr 0 (alength arr))]
(cond (zero? gas)
(throw (ex-info "Ran out of gas" {}))
(neg? r) c
:else (recur (+ c r) (dec gas)))))))
(deftest large-csv-test
(mt/test-driver :postgres
(testing "Postgres can download csvs without holding the entire results in memory #(#60733)"
(let [large-query "SELECT repeat('x', 2048) FROM generate_series(1, 1000000)"
result (mt/user-real-request :crowberto :post 200 "dataset/csv"
{:request-options {:as :stream}}
{:query (mt/native-query
{:query large-query})
:format_rows true})
size (consume result)]
(is (> size 2000000000)
(format "Only consumed %s but expected ~2gb" (readable size)))))))
```
and then
```
❯ time source pg.env && DRIVERS=postgres clojure -X:dev:test:small-ram :only metabase.driver.postgres-test/large-csv-test
Running QP tests against these drivers: #{:postgres}
LONG TEST in metabase.driver.postgres-test/large-csv-test
Test took 23.358 seconds seconds to run
Ran 1 tests in 32.825 seconds
2 assertions, 0 failures, 0 errors.
{:test 1, :pass 2, :fail 0, :error 0, :type :summary, :duration 32825.099875, :single-threaded 1}
Ran 0 tests in parallel, 1 single-threaded.
Finding and running tests took 40.7 s.
All tests passed.
Running after-run hooks...
```
Why not add these tests now?
We probably should write a full memory regression suite. This is a great
candidate for a first one. But it seems silly to run a new CI job with
just this one job. Push back if you think i'm wrong.
Another area of pushback:
I took the lazy way when determining if a driver should have autocommit
set to false or not: i hardcoded it.
```clojure
(and (-> options :download?) (= driver :postgres))
```
The alternative is some method of a new driver multimethod or
driver-supports?. The former feels a bit heavy-weight, the latter feels
not quite right since this is almost a non-support and all other jdbc
drivers do support it, presumably?
I'm quite open to pushback here and invite comments if you see a better way.
* Can't use an if for write?
You'd think you can use an if here but the previous code used a when and
is silent on write?. Lots of tests fail relating to datasets in test
setup and uploads. Don't want to debug that in this bug fix context so
leaving a todo
* remove old arity metadata. thanks eastwood!
* Don't set autoCommit for all postgres and descendant drivers
```clojure
(ns ^:mb/memory metabase.memory.csv-test
(:require
[clojure.test :refer :all]
[metabase.test :as mt]
[metabase.driver :as driver]))
(set! *warn-on-reflection* true)
(defn- readable [bytes]
(cond (nil? bytes) "0 bytes"
(zero? bytes) "0 bytes"
:else
(let [units ["b" "kb" "mb" "gb" "tb"]
magnitude 1024.0]
(loop [[unit & remaining] units
current (double bytes)]
(if (and (seq remaining) (> current magnitude))
(recur remaining (/ current magnitude))
(format "%.1f %s" current unit))))))
(defn- consume
[^java.io.InputStream is]
(let [arr (byte-array 8024)]
(loop [c 0, gas 5000000]
(let [r (java.io.InputStream/.read is arr 0 (alength arr))]
(cond (zero? gas)
(throw (ex-info "Ran out of gas" {}))
(neg? r) c
:else (recur (+ c r) (dec gas)))))))
(def driver->query
{:sqlite "WITH RECURSIVE generate_series(value) AS (
SELECT 1
UNION ALL
SELECT value + 1 FROM generate_series WHERE value + 1 <= 1000000
)
SELECT replace(hex(zeroblob(1024)), '00', 'xx') FROM generate_series;"
:postgres "SELECT repeat('x', 2048) FROM generate_series(1, 1000000)"
:clickhouse "SELECT repeat('x', 2048)
FROM system.numbers
LIMIT 1000000;"
:redshift "WITH series AS (
SELECT ROW_NUMBER() OVER () as i
FROM stl_scan s1, stl_scan s2
LIMIT 1000000
)
SELECT REPEAT('x', 2048) FROM series;"})
(deftest large-csv-test
(mt/test-drivers #{:postgres :sqlite :clickhouse :redshift}
(testing "Postgres can download csvs without holding the entire results in memory #(#60733)"
(let [large-query (or (driver->query driver/*driver*)
(throw (ex-info "Driver doesn't implement big query"
{:driver driver/*driver*
:available (keys driver->query)})))
result (mt/user-real-request :crowberto :post 200 "dataset/csv"
{:request-options {:as :stream}}
{:query (mt/native-query
{:query large-query})
:format_rows true})
size (consume result)]
(is (> size 2000000000)
(format "Only consumed %s but expected ~2gb" (readable size)))))))
(comment
(mt/set-test-drivers! #{:postgres})
(mt/set-test-drivers! #{:sqlite})
(mt/set-test-drivers! #{:clickhouse})
(mt/set-test-drivers! #{:redshift})
(mt/test-drivers #{:redshift}
(mt/db))
)
```
Verify this with
```
:small-ram
{:jvm-opts ["-Xmx400m"
"-Xms400m"
"-Dci=TRUE"]}
```
and then
```
;; driver.env:
;; local clickhouse
export MB_CLICKHOUSE_TEST_password="password"
export MB_CLICKHOUSE_TEST_username="default"
;; local postgres
export MB_POSTGRESQL_TEST_USER=dan
export MB_REDSHIFT_TEST_PASSWORD=<value>
export MB_REDSHIFT_TEST_USER=<value>
export MB_REDSHIFT_TEST_DB=<value>
export MB_REDSHIFT_TEST_HOST=<value>
```
and then running:
```
❯ source driver.env && DRIVERS=redshift clojure -X:dev:test:drivers:drivers-dev:ee:ee-dev:small-ram :only metabase.memory.csv-test/large-csv-test
...
Running tests in metabase.memory.csv-test/large-csv-test
Finding tests took 8.3 s.
Running 1 tests
Running before-run hooks...
Running QP tests against these drivers: #{:redshift}
..
LONG TEST in metabase.memory.csv-test/large-csv-test
Test took 188.320 seconds seconds to run
Ran 1 tests in 188.338 seconds
2 assertions, 0 failures, 0 errors.
{:test 1, :pass 2, :fail 0, :error 0, :type :summary, :duration 188337.678584, :single-threaded 1}
Ran 0 tests in parallel, 1 single-threaded.
Finding and running tests took 3.3 mins.
All tests passed.
Running after-run hooks...
```
redshift takes a while.
contrast with
```
❯ source driver.env && DRIVERS=clickhouse,postgres clojure -X:dev:test:drivers:drivers-dev:ee:ee-dev:small-ram :only metabase.memory.csv-test/large-csv-test
...
Running QP tests against these drivers: #{:postgres :clickhouse}
...
LONG TEST in metabase.memory.csv-test/large-csv-test
Test took 46.638 seconds seconds to run
Ran 1 tests in 46.655 seconds
4 assertions, 0 failures, 0 errors.
{:test 1, :pass 4, :fail 0, :error 0, :type :summary, :duration 46655.436333, :single-threaded 1}
Ran 0 tests in parallel, 1 single-threaded.
Finding and running tests took 56.5 s.
All tests passed.
Running after-run hooks...
```
both running locally happen in ~1 minute compared to 3 minutes for redshift
* Remove commented out old arity
this funciton used to have two arities for clarity. But it wasn't
another entrypoint, it was just trying to keep things a bit cleaner. But
the result is that at the point where we used the string sql query we
didn't have the `info` map in lexical scope. Fixing this across arities
would have pushed us into 7 positional args, so you might want to switch
to a map, but honestly, it's just the same function.
* Add ResponsiveButton component
* Prevent focus state being cut off
* Make the buttons responsive in the table section
* Make the buttons responsive in field section
* Extract useResponsiveButtons
* Add comments
* Extract useResponsiveButtons in FieldSection
* Add aria-label to the button
* Add tests
* Test tooltips
* Fix indent
* Add "move filter" menu
* Add undo
* Handle single tab dashboards
* Update text color for checked items
* Fix collapsed menu appearance
* Add tests
* Fix error when a tab doesn't have any dashcards
* Fix "Top of page" option selected color
* Use "card.title" viz setting
* Truncate long card titles
* Update "Disconnect from cards" button appearance
* Fix type error
* Make the select searchable, touch up on appearance
* Fix e2e test
* Add `moveDashboardFilter` helper
* Fix e2e test name
* Remove subtitles
* Update icon color
* Update icon color
* SEM-503 Add blue border to selected field
* SEM-504 Border radius of table/field name and description boxes should match that of setting cards
* SEM-509 Improve microcopy for table and field editing instructions
* SEM-510 Field descriptions could use "paragraph" text style with greater line height
* save
* revert redshift changes
* kondo
* add ci routing var
* redshift rough work
* cleanup bigquery changes
* remove bigquery specific redefs and bindings
* add separate var for dataset
* custom routing info per driver
* update changelog
* cljfmt
* fix fe lint
* use deferred-tru
* dissoc db_routing_info for tests
* fix other tests
* fix kondo and lint
* use yaml
* fix whitespace
* change log
* kondo and eastwood
* use markdown for routing info
* remove unneeded driver method
* cleanup
* fix tests
* fix remaining test
* add fe test
* add extra info in register-lazy-loaded-driver
* address pr comment
* improve doc string
* add text-secondary color to database info markdown
* update markdown
* alphabetical order
* move db routing info from db to session properties
* FE adjustments
* dont parse extra info
---------
Co-authored-by: Nick Fitzpatrick <nickfitz.582@gmail.com>
* poc
* timing logs
* Only render compatible datasets
* Build uberjar to be able to test in cloud with more data
* search index support for display
* search index support for exclude-display and has-temporal-dimensions
* fix weird filter vs index column bug
* fix temporal filter
* required ids search index filter
* fix build by removing :display from search render-terms
* adding h2 compatibility
* fixing alignment
* whitespace
* all clicks are add clicks now
* reverted labels change
* whitespace
* whitespace
* mutation -> query, load compatibles at startup
* using viz icons in dataset list
* remove exclude_display
* consolidate search endpoint
* rename display param to display_type
* allignment
* fix filter tests
* test fixes
* Closes VIZ-1185: revert "using viz icons in dataset list"
This reverts commit 7f4d177e977643f90250a5add2f946d6f35ad2a0.
* closes VIZ-1184 copy for "no results"
* closes VIZ-1183 spacing between input and results
* removed log statement
* fixed type
* fixed basics.cy.spec.ts
* fixed dashboard-questions.cy.spec.js
* fixed sharing-reproductions.cy.spec.js
* fixed snowplow-tracking.cy.spec.ts
* fixed funnels.cy.spec.ts
* fixed others
* fix import
* return new fields in results for in-place engine
* test fix
* test fix attempt
* rm backup file
* fixed more e2e tests
* fixup
* more stabilization of e2e tests
* tweak whitespace
* first attempt to fix failing postgres tests
* try a simpler approach
* closes VIZ-1227 search not triggered when no datasource is selected
* new attempt to parameterize the define-spec macro with app-db
* removing non-temporal-dimension-ids filter
* fix cljkondo
* fix test
* fixed e2e (snowplow)
* fixed e2e (funnels)
* fixed e2e (basics)
* better loading state, better typing
* reverted type change
* uncomment e2e cartesian; remove unused search param; fix bg color
* Fix query to resolve to false instead of nil
---------
Co-authored-by: Eliot Daigneault-Mestre <eliot@metabase.com>
Co-authored-by: Noah Moss <noahbmoss@gmail.com>
Co-authored-by: Sébastien <sebastien@metabase.com>
* Stop offering expressions and aggregation being edited as columns
Fixes#61010
* Add repro for metabase#61010
* Add test for custom columns too
* Update expected error message in e2e test
* Detect cycles in resolver too
* Revert "Detect cycles in resolver too"
* Add e2e test for referencing custom expressions that are just wrappers
---------
Co-authored-by: Romeo Van Snick <romeo@romeovansnick.be>
Fixes#44077. Fixes#35840.
Previously this used some custom legacy MBQL manipulation. It was
incorrect and led to a handful of bugs. Now it's a straightforward
edit of the query powered by MBQL lib.
* clear secret file memoization on db update
* add rsa env vars
* improve test
* add more narrow test
* improve test
* refactor snowflake private key test to create and drop user per test
* use accountadmin role
* try granting sysadmin
* try using accountadmin role
* use correct details
* throw exceptions for unimplemented methods
* clean up test code
* use different login and user names for sqlserver impersonation setup
* Require role
* fix sqlserver test
* move feature supported def
* remove outdated test
* feat: swap span for flex in Trigger component with chevron
* chore: use span in flex component
* feat: add small left padding to native editor action buttons
* chore: fix typo on data-testid file
* chore: force viewport to be big enough in cy test
* feat: force the test to small window and execute the drag vertically
* "I'll do this later" doesn't exit the flow (#59993)
* parallelize requests and make them use rtk query (#59563)
* parallelize requests and make them use rtk query
* fix multiple "Automatically Generated Dashboards" collections created +
update types
* add test
* EMB-459: remove 3 tables limit (#59997)
* Tweak welcome step content display (#60083)
* tweak welcome step content display
* update test after copy update
---------
Co-authored-by: Nicolò Pretto <info@npretto.com>
* remove useForceLocaleRefresh as the context is now enough (#60145)
* adds `embedding_setup_click` event (#60315)
* embedding setup: remove last fetch call and speed up e2e (#60392)
* close EMB-487, remove last remaining fetch call
* closes EMB-534, speed up e2e
* createdDashboard -> createdDashboards
* embedding setup: use cloud token in e2e and hide appbar and database status (#60335)
* EMB-533: make the e2e test use the CLOUD pro token
* EMB-532: hide the database sync and the app bars in setup/embedding
* Update e2e/test/scenarios/onboarding/setup/setup-embedding.cy.spec.ts
* fix typo
---------
Co-authored-by: Kyle Doherty <5248953+kdoh@users.noreply.github.com>
* Empty
* No need for intersection in test
* Correct modestring casing
* Test datetime(number, mode) for s, ms, us, ns
* Allow uppercase and hyphens but normalize them.
* Don't allow hyphens in frontend.
* Implement unix casts for mongo
* Convert floats to long
* Use hyphenated keywords internally
* expand docstring
* Try other implementation of nanoseconds for BigQuery
* Handle sqlserver unix timestamps with multimethod
* More flexible date type handling
* Use integer division with bigquery
* datetime(number) frontend tests
* Undo change to package.json
* Empty
* Wrap code-like stuff in ``
* Add expressions/today feature
* Add legacy + lib for today()
* Basic test for today() (WIP)
* Add expressions/today feature to sql + implementation
* Finalize today() test
* Implement today() on mongo
* Better comment
* use m/find-first
* Frontend implementation
* Add today() fe test
* Improve fe test for today()
* Add a line to changelog
* Add line to changelog about expressions/today
* Handle different types better in lib schema
* Add special case for mode in formatter
* Add mode to expression options
* Add special case for mode in parser
* Error message
* Schema with options
* Add options map to legacy schema for datetime
* Remove comment
* Add unit test for mode option in compiler
* Add test for formatting the mode option in datetime
* Better round-tripping for :datetime function
* Add :mode to allowed keys for options marshalling
* Fix expression to add mode to options map
* Test datetime() legacy/lib roundtripping
* Fix normalization
* Remove println
* Mongo datetime() with options map
* Move test to custom-column spec
* Remove duplicate check
---------
Co-authored-by: Romeo Van Snick <romeo@romeovansnick.be>
* In main nav, don't show 'Other users' personal collections' if there are no other users
---------
Co-authored-by: Ryan Laurie <iethree@gmail.com>
Co-authored-by: Fraser <fraser@metabase.com>
This PR fixes a bug where dashboard filters linked to multiple fields would show entity keys instead of remapped entity names when viewed by users with grouped (limited) permissions.
The solution in this PR is to add support for explicit remapping field specification.
Also resolved a separate permissions caching bug in data_permissions.clj where the use-cache? logic was inverted.
Co-authored-by: metamben <103100869+metamben@users.noreply.github.com>
* Empty
* No need for intersection in test
* Correct modestring casing
* Test datetime(number, mode) for s, ms, us, ns
* Allow uppercase and hyphens but normalize them.
* Don't allow hyphens in frontend.
* Implement unix casts for mongo
* Convert floats to long
* Use hyphenated keywords internally
* expand docstring
* Try other implementation of nanoseconds for BigQuery
* Handle sqlserver unix timestamps with multimethod
* More flexible date type handling
* Use integer division with bigquery
* datetime(number) frontend tests
* Undo change to package.json
* Wrap code-like stuff in ``
* Better comment
* use m/find-first
* Add a line to changelog
* Handle different types better in lib schema
* Add special case for mode in formatter
* Add mode to expression options
* Add special case for mode in parser
* Error message
* Schema with options
* Add options map to legacy schema for datetime
* Remove comment
* Add unit test for mode option in compiler
* Add test for formatting the mode option in datetime
* Better round-tripping for :datetime function
* Add :mode to allowed keys for options marshalling
* Fix expression to add mode to options map
* Test datetime() legacy/lib roundtripping
* Fix normalization
* Remove println
* Mongo datetime() with options map
* Remove duplicate check
* Add simple bytes to compiler
---------
Co-authored-by: Romeo Van Snick <romeo@romeovansnick.be>
* Add aggregation columns
* Pass expressionIndex to Expression editor in AggregationPicker
* Add expression-parts-method for aggregation references
* Add support in the query processor
* Add more tests
* Hack FE to call aggregablecolumns
* Add missing expressionMode option
* Add columnsForExpressionMode helper
* Update error message from resolver
* Add query with aggregation for tests
* Add tests for resolving aggregations
* Fix tests
* Fix CLJS tests
* Fix more tests
* Add missing docstring
* Add more unit tests
* Add unit test for aggregation completions
* Fix error message in test
* Add tests for second stage resolver
* Add tests for referencing aggregations
* Add test for resolving aggregation in next stage
* Disambiguate test name
* Fix column names returned by QP for aliased aggregations
* Fix merge issues
* Fix final test
* Address review comments
* Move aggregation tests
* Add test for detecting cycles in aggregations
* Add test about disallowing duplicate aggregation names
* Add sort helper
* Add tests for using nested aggregations as question source
* Add e2e test for nested aggregations in follow up stage
* Add tests for using nested aggregations in follow up stage with breakout
* Allow duplicate aggregation names
* Add test for changing aggregation order in underlying question
* Remove nonsensical test
* Fix test descriptions
* Fix test
---------
Co-authored-by: Romeo Van Snick <romeo@romeovansnick.be>
* Add BE repro test for #58314
* Add repro test for #35252
* Add repro test for #39033
* Add repro test for #27521
* Add repro test for #47734
* Add repro test for QUE-1378
* Add repro test for QUE-1374
* Add repro test for #58314
* Add repro test for #44767Closes#44767
* Add repro test for #60498
* Add repro test for #60444
* Make test for #60444 more general
* run embedding-sdk.yml from run-tests.yml
* remove e2e-tests-skipped-stub
* try to put back some structure
* fix name of build
* / -> › because github doesn't uses only the part after the last slash
* use > instead of chevron to make linter happy
* breaking changeS
* remove code to trigger ci
* Fix flaky test by adding aria-hidden on elements rendered only to perform measurements
* Revert "Fix flaky test by adding aria-hidden on elements rendered only to perform measurements"
This reverts commit 5575d6b43b.
* Fix flaky test by adding aria-hidden on elements rendered only to perform measurements
* Use hidden instead of aria-hidden
* Filter out invisible elements
* Revert adding hidden attribute
* moving pickers into same folder, consolidate getInitialContainer hooks
* getCollectionIdPath consolidation
* getStateFromIdPath
* comitting sins to get tests passing
Fixes getting source tables in multistage aggregations. The check for
when we were in the scope of the parent card was in the wrong place in
the match which resulted in us failing to make a source-query block
immediately inside a stage with a source card.
Two changes required to allow this:
- first, for the setting itself, allow setting it if the database is
writable OR an attached DWH.
- second, a similar change to make sure `syncable_schemas` is fetchable
for the attached DWH.
* SEM-239 Data model - Layout boilerplate (#57174)
* Add route and link
* Convert DataModelApp to TypeScript
* Use DataModelApp in DataModel
* Keep all routes
* Extract section components
* Add empty states
* Use ts-pattern
* Drop ts-pattern
* Disable preview
* Rename
* Add border
* Move page components to their own dirs
* SEM-240 Data model - Table section (#57248)
* Add route and link
* Convert DataModelApp to TypeScript
* Use DataModelApp in DataModel
* Keep all routes
* Extract section components
* Add empty states
* Use ts-pattern
* Drop ts-pattern
* Disable preview
* Rename
* Add border
* Move page components to their own dirs
* Add table fetching
* Add NameDescriptionInput boilerplate
* Update table on blur
* Fix focus state and prevent empty names
* Add Hide this table switch
- Label from https://www.loom.com/share/45ce00aaac3f4141adf3f52dec79048f?highlightComment=80327135&t=271
* Style the Switch
* Move FieldOrderPicker
* Move SortableField
* Move NameDescriptionInput
* Make NameDescriptionInput more generic
* Use FieldOrderPicker
* Rename
* Move label into TableVisibilityInput
* Add missing return
* Add FieldOrder
* Get rid of entity framework workarounds
* Add TODO comment
* Make header sticky
* Sort
* Add field scanning section
* Discard + rescan
* Remove todo
* Remove TableVisibilityInput
* Rename FieldOrder to SortableFieldList and move it to metadata/components
* Implement getFieldIcon boilerplate
* Do not use metabase-lib/v1
* Extract tableFieldToColumnMetadata
* Fix focus state being cut off
* Extract RescanFieldsButton, add error and success states
* Extract DiscardFieldValuesButton
* Rename
* Move to metadata/components
* Style name/description input
* Add missing ttag
* Add NameDescriptionInput tests
* Add more tests
* Organize tests
* Add tests
* Add DiscardTableFieldValuesButton tests
* Add test
* Add RescanTableFieldsButton tests
* Lint colors
* Add tests for getItems
* Add tests for getFieldIcon
* Update comment
* Update test names
* Add missing effect
* Add missing effect
* Fix flaky unit tests
* Remove getFieldIcon
* Extract TextInputBlurChange
* Update comment
* SEM-241 Data model - Field - Data section (#57420)
* Add route and link
* Convert DataModelApp to TypeScript
* Use DataModelApp in DataModel
* Keep all routes
* Extract section components
* Add empty states
* Use ts-pattern
* Drop ts-pattern
* Disable preview
* Rename
* Add border
* Move page components to their own dirs
* Add table fetching
* Add NameDescriptionInput boilerplate
* Update table on blur
* Fix focus state and prevent empty names
* Add Hide this table switch
- Label from https://www.loom.com/share/45ce00aaac3f4141adf3f52dec79048f?highlightComment=80327135&t=271
* Style the Switch
* Move FieldOrderPicker
* Move SortableField
* Move NameDescriptionInput
* Make NameDescriptionInput more generic
* Use FieldOrderPicker
* Rename
* Move label into TableVisibilityInput
* Add missing return
* Add FieldOrder
* Get rid of entity framework workarounds
* Add TODO comment
* Make header sticky
* Sort
* Add field scanning section
* Discard + rescan
* Remove todo
* Remove TableVisibilityInput
* Rename FieldOrder to SortableFieldList and move it to metadata/components
* Implement getFieldIcon boilerplate
* Do not use metabase-lib/v1
* Extract tableFieldToColumnMetadata
* Fix focus state being cut off
* Extract RescanFieldsButton, add error and success states
* Extract DiscardFieldValuesButton
* Rename
* Move to metadata/components
* Style name/description input
* Add missing ttag
* Add NameDescriptionInput tests
* Add more tests
* Organize tests
* Add tests
* Add DiscardTableFieldValuesButton tests
* Add test
* Add RescanTableFieldsButton tests
* Lint colors
* Add tests for getItems
* Add tests for getFieldIcon
* Update comment
* Update test names
* Add missing effect
* Add missing effect
* Navigation onClick
* Fix flaky unit tests
* Use a link
* Explain cast
* Improve url generation
* Style active link
* Load field, display name
* Sticky header
* Add SectionPill
* Add field name and data type
* Add cast toggle
* Add CoercionStrategyPicker
* Remove getFieldIcon
* Extract TextInputBlurChange
* Update comment
* Casting toggle state sync
* CoercionStrategyPicker error state
* Extract FieldDataSection
* Reuse existing TextInputBlurChange
* Further cleanup
* Use key for FieldSection
* Lift data fetching
* Add decoration
* Use key for table as well
* Add unit tests
* Add unit tests
* Add unit tests
* Add comments
* Simplify color
* Use type instead of interface
* SEM-297 Drop entity framework usage in `SemanticTypePicker` (#57531)
* Add route and link
* Convert DataModelApp to TypeScript
* Use DataModelApp in DataModel
* Keep all routes
* Extract section components
* Add empty states
* Use ts-pattern
* Drop ts-pattern
* Disable preview
* Rename
* Add border
* Move page components to their own dirs
* Add table fetching
* Add NameDescriptionInput boilerplate
* Update table on blur
* Fix focus state and prevent empty names
* Add Hide this table switch
- Label from https://www.loom.com/share/45ce00aaac3f4141adf3f52dec79048f?highlightComment=80327135&t=271
* Style the Switch
* Move FieldOrderPicker
* Move SortableField
* Move NameDescriptionInput
* Make NameDescriptionInput more generic
* Use FieldOrderPicker
* Rename
* Move label into TableVisibilityInput
* Add missing return
* Add FieldOrder
* Get rid of entity framework workarounds
* Add TODO comment
* Make header sticky
* Sort
* Add field scanning section
* Discard + rescan
* Remove todo
* Remove TableVisibilityInput
* Rename FieldOrder to SortableFieldList and move it to metadata/components
* Implement getFieldIcon boilerplate
* Do not use metabase-lib/v1
* Extract tableFieldToColumnMetadata
* Fix focus state being cut off
* Extract RescanFieldsButton, add error and success states
* Extract DiscardFieldValuesButton
* Rename
* Move to metadata/components
* Style name/description input
* Add missing ttag
* Add NameDescriptionInput tests
* Add more tests
* Organize tests
* Add tests
* Add DiscardTableFieldValuesButton tests
* Add test
* Add RescanTableFieldsButton tests
* Lint colors
* Add tests for getItems
* Add tests for getFieldIcon
* Update comment
* Update test names
* Add missing effect
* Add missing effect
* Navigation onClick
* Fix flaky unit tests
* Use a link
* Explain cast
* Improve url generation
* Style active link
* Load field, display name
* Sticky header
* Add SectionPill
* Add field name and data type
* Add cast toggle
* Add CoercionStrategyPicker
* Remove getFieldIcon
* Extract TextInputBlurChange
* Update comment
* Casting toggle state sync
* CoercionStrategyPicker error state
* Extract FieldDataSection
* Reuse existing TextInputBlurChange
* Further cleanup
* Use key for FieldSection
* Lift data fetching
* Add decoration
* Use key for table as well
* Add unit tests
* Add unit tests
* Add unit tests
* Add comments
* Drop entity framework in FkTargetPicker
* Make table names show up
* Clean up
* Fix types
* Fix table names
* Add tests
* Add tests
* Add tests
* Restrict imports
* Improve config
* Restrict entity framework imports
* Simplify color
* Use type instead of interface
* Lint
* SEM-267 Data model - Field - Metadata section (#57527)
* Add route and link
* Convert DataModelApp to TypeScript
* Use DataModelApp in DataModel
* Keep all routes
* Extract section components
* Add empty states
* Use ts-pattern
* Drop ts-pattern
* Disable preview
* Rename
* Add border
* Move page components to their own dirs
* Add table fetching
* Add NameDescriptionInput boilerplate
* Update table on blur
* Fix focus state and prevent empty names
* Add Hide this table switch
- Label from https://www.loom.com/share/45ce00aaac3f4141adf3f52dec79048f?highlightComment=80327135&t=271
* Style the Switch
* Move FieldOrderPicker
* Move SortableField
* Move NameDescriptionInput
* Make NameDescriptionInput more generic
* Use FieldOrderPicker
* Rename
* Move label into TableVisibilityInput
* Add missing return
* Add FieldOrder
* Get rid of entity framework workarounds
* Add TODO comment
* Make header sticky
* Sort
* Add field scanning section
* Discard + rescan
* Remove todo
* Remove TableVisibilityInput
* Rename FieldOrder to SortableFieldList and move it to metadata/components
* Implement getFieldIcon boilerplate
* Do not use metabase-lib/v1
* Extract tableFieldToColumnMetadata
* Fix focus state being cut off
* Extract RescanFieldsButton, add error and success states
* Extract DiscardFieldValuesButton
* Rename
* Move to metadata/components
* Style name/description input
* Add missing ttag
* Add NameDescriptionInput tests
* Add more tests
* Organize tests
* Add tests
* Add DiscardTableFieldValuesButton tests
* Add test
* Add RescanTableFieldsButton tests
* Lint colors
* Add tests for getItems
* Add tests for getFieldIcon
* Update comment
* Update test names
* Add missing effect
* Add missing effect
* Navigation onClick
* Fix flaky unit tests
* Use a link
* Explain cast
* Improve url generation
* Style active link
* Load field, display name
* Sticky header
* Add SectionPill
* Add field name and data type
* Add cast toggle
* Add CoercionStrategyPicker
* Remove getFieldIcon
* Extract TextInputBlurChange
* Update comment
* Casting toggle state sync
* CoercionStrategyPicker error state
* Extract FieldDataSection
* Reuse existing TextInputBlurChange
* Further cleanup
* Use key for FieldSection
* Lift data fetching
* Add decoration
* Use key for table as well
* Add unit tests
* Add unit tests
* Add unit tests
* Add comments
* Add Metadata section with display name input
* Pass table through components
* Drop entity framework in FkTargetPicker
* Revert "Pass table through components"
This reverts commit 2eb3e3a256.
* Make table names show up
* Clean up
* Fix types
* Fix table names
* Add tests
* Add tests
* Add tests
* Restrict imports
* Improve config
* Restrict entity framework imports
* Simplify color
* Use type instead of interface
* Use SemanticTypeAndTargetPicker in FieldSection
* Sort
* Lint
* Typing
* Do not use v1/metadata wrapper
* Add TextareaBlurChange
* Add description input
* Update comments
* Update icon
* Add plugin query props
* Remove redundant prop
* Remove redundant _.omit
* Fix textarea crashing due to min height on input
* Set minRows
* Fix styling
* Remove PLUGIN_FEATURE_LEVEL_PERMISSIONS plugin usage from DatasetFieldMetadataFkTargetPicker
- because that's not how it works in master
* Drop -v2 suffix for new routes, use -v1 suffix for old routes (#57637)
* SEM-242 Data model - Field - Behavior section (#57638)
* Drop -v2 suffix for new routes, use -v1 suffix for old routes
* Move FieldVisibilityPicker
* Make FieldVisibilityPicker work with api Field
* Use FieldVisibilityType
* Fix gap
* Update tests
* Add label and description
* Add FieldValuesTypePicker
* Add canUnfoldJson and hasDatabaseFeature
* Rename
* Add UnfoldJsonPicker
* Fix typing
* Add RescanFieldButton
* Add DiscardFieldValuesButton
* Style the connector
* Fix scrolling
* Update spacing
* Add tests for FieldValuesTypePicker
* Add tests
* Add UnfoldJsonPicker tests
* Add tests
* Add tests
* Clean up timeouts on unmount
* Improve typing
* SEM-269 Data model - Field - Formatting section (#57682)
* Drop -v2 suffix for new routes, use -v1 suffix for old routes
* Move FieldVisibilityPicker
* Make FieldVisibilityPicker work with api Field
* Use FieldVisibilityType
* Fix gap
* Update tests
* Add label and description
* Add FieldValuesTypePicker
* Add canUnfoldJson and hasDatabaseFeature
* Rename
* Add UnfoldJsonPicker
* Fix typing
* Add RescanFieldButton
* Add DiscardFieldValuesButton
* Style the connector
* Fix scrolling
* Update spacing
* Add tests for FieldValuesTypePicker
* Add tests
* Add UnfoldJsonPicker tests
* Add tests
* Add tests
* Add FormattingSection
* Sort
* Make formatting form full width
* Use style prop instead of maxWidth
* Clean up timeouts on unmount
* Improve typing
* Metadata preview boilerplate (#58015)
* Render preview
* Add preview type selector
* Fix flex layout
* Lift preview type to top component to keep the state
* Move helpers to utils
* Add unit tests for preview picker
* Add feedback to metadata editor when saving config (#58038)
* Add feedback to all table and field settings
* Add missing prop
* Metadata table and detail preview (#57887)
* Handle undefined metadata
* Render Table preview
* Add custom error component
* Add helper for error message
* Add detail preview
* Make comment more clear
* Fix type
* Fix missing props
* Metadata table picker (#57648)
* Set up TablePicker component
* Fix type in api
* Add initial TablePicker implementation
* Add schema icon
* Update url for datapicker
* Move data loading to utils
* Add loading node types
* Derive icon from type
* Try building the treepicker state asynchronously
* Build custom tree
* Add loading state
* Push url state in picker
* Sync open state with url
* Use helper to manage expanded state
* Split Node into separate file
* Expose prefetch hook
* Prefetch data to avoid showing loading states
* Add search to table picker
* Move search-related stuff into Search module
* Use Link for navigation
* Add renderLoading helper
* Remove unused prop
* Make search input sticky
* Add separate sidebar for field picker
* Render single schema but open it by default
* Translate errors
* Defer search value
* Export lazy loaders for database info
* Use top-level data fetching
* Enable virtualization
* Move loader utils into utils
* Remove Node implementation
* Simplify styles
* Pull out some logic
* Reuse results in search
* Reuse tree helpers when handling search results
* Avoid passing down isExpanded
* Memoize
* Fix initial expanded state
* Automatically expand single-schema databases
* Pass tableId to DataPicker
* Add styling for active item
* Add getUrl helper for tree nodes
* Move types to separate file
* Simplify item
* Rename Item to Results
* Simplify
* Allow wrapping everywhere
* Add loading state
* Render tracks
* Remove unused colors
* Scroll active item into view
* Fix paths
* Add missing keys
* Handle browser navigation in table picker
* Handle browser navigation in table picker
* Add more documentation
* Debounce search
* Render track even when no parent was found
* Allow rendering urls to schema's with no name
* Add value to loading nodes
* Add delay to track too
* Add explicit level to FlatItem
* Add explicit parent refence to FlatItem
* Hide nameless schemas in the tree
* Refactor
* Move key to top-level item
* Make TablePicker not depend on router
* Disable smooth scroll
* Add unit tests for TablePicker
* Unnest unnamed schemas directly
* Also fetch tables when only one schema exists
* Remeasure items when they change
* Use more precise typeing for items
* Allow clicking table nodes in search results
* Remove unused classes
* Remove unused useDispatch hook
* Remove delay on loading items
* Add custom empty state
* Typo
* Add keyboard navigation to table picker
* Use test helpers in all tests
* Add keyboard navigation to search results too
* Add custom focus state for search items
* Add unit tests for search
* Jump to parent when pressing left arrow
* Add more cases to keyboard test
* Add hover state to table picker
* Use table display name in table picker
* Flatten single schema in to database
* Rename onChange_ to handleChange
* Use folder for schema
* Make the hover and active colors the same
* Rename Data tab to Data Model
* Remove title
* Fix layout
* Field description editor (#58280)
* Use NameDescriptionInput for the field section too
* Use TextArea for description editor
* Post-merge fixes
* Metadata editing UI tweaks (#58297)
* Add background to fields
* Tweak colors for disabled inputs
* Remove icons in SectionPill
* Remove icon from one more SectionPill
* Remove lint
* Move segments to table picker (#58422)
* Add link to segments in the datamodel sidebar
* Add local state to RouterTablePicker
* Nest segments under DataModel
* Add wrapper classes to revision routes
* Fix spacing for empty state
* Add active state to segments link
* Remove DataModelApp in DataModel routes
* Fix overflow in segment form
* Merge translation strings
* Add background color to segments components
* SEM-363 Data model - Field and table name/description inputs should be sticky and have icons (#58703)
* Remove hover/focus effect and break components down into files
* Update comment
* Consolidate toasts
* Remove unused styles
* Remove redundant card
* Extract SegmentsLink
* Make name/description input sticky
* Add icons
* Remove leftovers
* Fix TS error
* Move visibility toggles to the table picker (#58284)
* Fetch hidden tables
* Fetch hidden schemas
* Add visibility toggles to the table picker
* Remove table visibility toggle in table section
* Simplify visibility toggler
* Move visibility toggle to separate file
* Add unit tests for visibility toggle
* Show table as hidden whenever there is any visibility
* Use ActionIcon
* Break up onClick handler
* Use plan Icon
* Remove data from nodes that don't need it
* Add tooltips to table visibility togglers
* Wrap icons in ActionIcon
---------
Co-authored-by: Kamil Mielnik <kamil@kamilmielnik.com>
* Cleanup
* SEM-361 Data model - Field metadata/formatting settings do not affect the preview (#58841)
* Fix SEM-361
* Fix types
* SEM-311 Table preview - no impact when changing time formatting for seconds and ms (#58927)
* Avoid using a breakout for date-time columns
* Use the same logic for every type of field
* SEM-246 Data model - Filtering preview (#58854)
* Fix SEM-361
* Fix types
* Add FilteringPreview boilerplate
* Use FilterPickerBody
* Another attempt
* Make it work for all types of filters
* Memoize query
* Clear filter input on field change
* Account for prop changes
* Fix TS error
* SEM-268 Data model - Field - Behavior section - Display values (#57684)
* Drop -v2 suffix for new routes, use -v1 suffix for old routes
* Move FieldVisibilityPicker
* Make FieldVisibilityPicker work with api Field
* Use FieldVisibilityType
* Fix gap
* Update tests
* Add label and description
* Add FieldValuesTypePicker
* Add canUnfoldJson and hasDatabaseFeature
* Rename
* Add UnfoldJsonPicker
* Fix typing
* Add RescanFieldButton
* Add DiscardFieldValuesButton
* Style the connector
* Fix scrolling
* Update spacing
* Add tests for FieldValuesTypePicker
* Add tests
* Add UnfoldJsonPicker tests
* Add tests
* Add tests
* Add placeholder
* Add RemappingPicker boilerplate
* Clean up timeouts on unmount
* Improve typing
* Add onChange logic boilerplate
* Extract DisplayValuesPicker
* Remove redundant code
* Format code
* Add fk input boilerplate
* Disable options instead of hiding them
* Rename
* Fetch required data
* Use full data
* Implement fk remapping onChange
* Implement naming tip
* Fix state
* Extract utils
* Fix schema step errors
* Add remappings boilerplate
* Fetch remappings
* Add more modal boilerplate
* Add modal props
* Handle modal state
* Add explicit edit button
* Implement modal form
* Limit amount of Boxes
* Bring all safety checks back in
* Refactor CustomMappingModal
* Style CustomMappingModal
* Extract getHasEmptyValues
* Remove todo
* Prevent flicker
* Fix initial state of fk picker
* Use is touched
* Add CustomMappingModal unit tests
* Add DisplayValuesPicker unit tests
* Move DisplayValuesPicker
* Remove redundant tags
* Fix uncontrolled/controller input react warning
* Add explanatory tooltips for disabled options
* Fix 1px of background leaking into view
* Add missing memoization
* Reset state when cancelling
* Sync single table via UI (#58342)
* add table-manual-sync event
* add table/sync_schema endpoint
* add table sync test
* Add api helper for the sync table schema endpoint
* Add SyncTableSchemaButton component
* Add SyncTableSchemaButton to table section
* Add unit tests for SyncTableSchemaButton
* Also invalidate field-values and parameter-values
* Simplify SyncTableSchemaButton hooks
* Move useTemporaryState into a separate hook
* Apply same pattern in all buttons
* address PR review comments: use `get-database`, `with-database-routing-off`
* Pass argument to useTemporaryState
* update test
---------
Co-authored-by: Nicola Mometto <nicola.mometto@metabase.com>
* Fix SEM-370 (#59051)
* Post-merge conflict fix
* SEM-329 Data model - Table section - Update fields sorting interaction (#59316)
* Add support for size="sm" in SegmentedControl
* Add field sorting interaction boilerplate
* Update FieldOrderPicker to use SegmentedControl
* Extract Label component
* Update tooltips
* Update sortable field
* Rename SortableField to SortableFieldItem
* Render field list
* Align styling
* Use rem, remove Link from SortableFieldItem
* Styling
* Use field prop
* Background styling
* Fix TS error
* Editable name and description
* Watch out for error
* Allow overriding default props in EditableText
* Fix a11y
* A11y
* Prevent overflow
* Styling
* Cleanup
* Update comment
* Fixes
* Update label and hide it in sorting mode
* Update spacing
* Avoid CLS when entering sorting mode
* Pixel-perfect
* Use ellipsis on overflow
* Fix colors
* Fix name input size
* Fix description patching
* Style selected state
* Do not use EditableText for description input, add resetOnEsc prop
* Use flushSync instead of setTimeout
* Try to prevent event propagation
* Fix interactivity in inputs
* SEM-376 Data model - Resizable columns layout (#59100)
* Format code
* Prevent updates when nothing changes
* Use new colors
* Make nav resizable
* Fix nav width
* Remove unused styles
* Prevent CLS in SortableList
* Add comment
* Get rid of second page component
* Make table column resizable
* Add padding and min width for the empty state
* Optimize
* Extract ResizableColumn
* Toggle preview
* Add preview and field resizing
* Extract getTableMetadataQuery
* Preview positioning and shadow
* Show scrollbar on preview overflow
* Fix resizing bounds
* Remove redundant props
* Handle Escape key
* Add comment
* Update default constraints
* Make preview non-resizable
* Fix valid Escape press detection
* Fix types
* Add support for size="sm" in SegmentedControl
* Add field sorting interaction boilerplate
* Update FieldOrderPicker to use SegmentedControl
* Extract Label component
* Update tooltips
* Update sortable field
* Rename SortableField to SortableFieldItem
* Render field list
* Align styling
* Use rem, remove Link from SortableFieldItem
* Styling
* Use field prop
* Background styling
* Fix TS error
* Editable name and description
* Watch out for error
* Allow overriding default props in EditableText
* Fix a11y
* A11y
* Prevent overflow
* Styling
* Cleanup
* Update comment
* Fixes
* Update label and hide it in sorting mode
* Update spacing
* Avoid CLS when entering sorting mode
* Auto-grow table column when possible
* Auto-magic layout
* Fix sign
* Optimize
* Optimize
* Pixel-perfect
* Use ellipsis on overflow
* Fix colors
* Fix name input size
* Fix description patching
* Style selected state
* Do not use EditableText for description input, add resetOnEsc prop
* Use flushSync instead of setTimeout
* Try to prevent event propagation
* Fix interactivity in inputs
* Fix interactivity in inputs
* Move memoization closer to component definition
* SEM-327 Data model - Move the re-scan / discard options away from the bottom of the fields list (#59474)
* Format code
* Prevent updates when nothing changes
* Use new colors
* Make nav resizable
* Fix nav width
* Remove unused styles
* Prevent CLS in SortableList
* Add comment
* Get rid of second page component
* Make table column resizable
* Add padding and min width for the empty state
* Optimize
* Extract ResizableColumn
* Toggle preview
* Add preview and field resizing
* Extract getTableMetadataQuery
* Preview positioning and shadow
* Show scrollbar on preview overflow
* Fix resizing bounds
* Remove redundant props
* Handle Escape key
* Add comment
* Update default constraints
* Make preview non-resizable
* Fix valid Escape press detection
* Fix types
* Add support for size="sm" in SegmentedControl
* Add field sorting interaction boilerplate
* Update FieldOrderPicker to use SegmentedControl
* Extract Label component
* Update tooltips
* Update sortable field
* Rename SortableField to SortableFieldItem
* Render field list
* Align styling
* Use rem, remove Link from SortableFieldItem
* Styling
* Use field prop
* Background styling
* Fix TS error
* Editable name and description
* Watch out for error
* Allow overriding default props in EditableText
* Fix a11y
* A11y
* Prevent overflow
* Styling
* Cleanup
* Update comment
* Fixes
* Update label and hide it in sorting mode
* Update spacing
* Avoid CLS when entering sorting mode
* Auto-grow table column when possible
* Auto-magic layout
* Fix sign
* Optimize
* Optimize
* Pixel-perfect
* SyncOptionsModal
* Move
* Field values modal
* Responsiveness
* Use ellipsis on overflow
* Fix colors
* Fix name input size
* Fix description patching
* Style selected state
* Do not use EditableText for description input, add resetOnEsc prop
* Use flushSync instead of setTimeout
* Try to prevent event propagation
* Fix interactivity in inputs
* Fix interactivity in inputs
* Move memoization closer to component definition
* Tmp
* Revert "Tmp"
This reverts commit 6e561c0de7.
* SEM-424 Tests parity - datamodel.cy.spec.js (#59548)
* Format code
* Prevent updates when nothing changes
* Use new colors
* Make nav resizable
* Fix nav width
* Remove unused styles
* Prevent CLS in SortableList
* Add comment
* Get rid of second page component
* Make table column resizable
* Add padding and min width for the empty state
* Optimize
* Extract ResizableColumn
* Toggle preview
* Add preview and field resizing
* Extract getTableMetadataQuery
* Preview positioning and shadow
* Show scrollbar on preview overflow
* Fix resizing bounds
* Remove redundant props
* Handle Escape key
* Add comment
* Update default constraints
* Make preview non-resizable
* Fix valid Escape press detection
* Fix types
* Add support for size="sm" in SegmentedControl
* Add field sorting interaction boilerplate
* Update FieldOrderPicker to use SegmentedControl
* Extract Label component
* Update tooltips
* Update sortable field
* Rename SortableField to SortableFieldItem
* Render field list
* Align styling
* Use rem, remove Link from SortableFieldItem
* Styling
* Use field prop
* Background styling
* Fix TS error
* Editable name and description
* Watch out for error
* Allow overriding default props in EditableText
* Fix a11y
* A11y
* Prevent overflow
* Styling
* Cleanup
* Update comment
* Fixes
* Update label and hide it in sorting mode
* Update spacing
* Avoid CLS when entering sorting mode
* Auto-grow table column when possible
* Auto-magic layout
* Fix sign
* Optimize
* Optimize
* Pixel-perfect
* SyncOptionsModal
* Move
* Field values modal
* Responsiveness
* Use ellipsis on overflow
* Fix colors
* Fix name input size
* Fix description patching
* Style selected state
* Do not use EditableText for description input, add resetOnEsc prop
* Port test
* Fix typing
* Port test
* Port tests
* Format code, add data-testid
* Port more tests
* Refactor withDatabase to TypeScript
* Await
* Fix prop handling
* Port test
* Memoize textarea for performance
* Port test
* Port tests
* Remove unlawful getFieldDisplayName usages
* ESLint
* Add JSDoc
* Port tests
* Port tests
* Format code
* Use flushSync instead of setTimeout
* Try to prevent event propagation
* Fix interactivity in inputs
* Fix interactivity in inputs
* Move memoization closer to component definition
* Tmp
* Revert "Tmp"
This reverts commit 6e561c0de7.
* Fix circular dependency
* Post-merge fix
* SEM-425 Tests parity - editor.cy.spec.js (#59838)
* Add DataModel helpers
* Update test
* Add waits
* Use schemaId in test helper
* Port tests
* Fix ids
* Add more helpers
* Use helpers
* Auto-open coercion picker, humanize coercion picker labels
* Update test
* Fix typing
* Fix data fetching
* Update tests
* Refactor
* Update tests
* Update helpers
* Simplify assertion
* SEM-426 Tests parity - reproductions.cy.spec.ts (#59970)
* Add DataModel helpers
* Update test
* Add waits
* Use schemaId in test helper
* Port tests
* Fix ids
* Add more helpers
* Use helpers
* Auto-open coercion picker, humanize coercion picker labels
* Update test
* Fix typing
* Fix data fetching
* Update tests
* Refactor
* Update tests
* Update helpers
* Update reproductions.cy.spec.ts
* Fix test
* Simplify assertion
* SEM-427 Tests parity - remaining e2e suites (#59980)
* Add DataModel helpers
* Update test
* Add waits
* Use schemaId in test helper
* Port tests
* Fix ids
* Add more helpers
* Use helpers
* Auto-open coercion picker, humanize coercion picker labels
* Update test
* Fix typing
* Fix data fetching
* Update tests
* Refactor
* Update tests
* Update helpers
* Update reproductions.cy.spec.ts
* Fix test
* Update test
* Update test
* Update test
* Update tests
* Update test
* Update test
* Update test
* Update test
* Update tests
* Update tests
* Typo
* Update unit tests
* Remove encodeURIComponent
* Make it clear what is schemaName and schemaId
* Rename schemaId to schemaName in table picker
* Update unit tests
* Update unit tests
* Skip soon-to-be-ported unit tests
* Attempt to fix a test
* Update test
* Update unit test
* Update test
* Revert refetching
* Data model - Simple flex layout (#59995)
* Add DataModel helpers
* Update test
* Add waits
* Use schemaId in test helper
* Port tests
* Fix ids
* Add more helpers
* Use helpers
* Auto-open coercion picker, humanize coercion picker labels
* Update test
* Fix typing
* Fix data fetching
* Update tests
* Refactor
* Update tests
* Update helpers
* Update reproductions.cy.spec.ts
* Fix test
* Update test
* Update test
* Update test
* Update tests
* Update test
* Update test
* Update test
* Update test
* Update tests
* Update tests
* Typo
* Update unit tests
* Remove encodeURIComponent
* Make it clear what is schemaName and schemaId
* Rename schemaId to schemaName in table picker
* Update unit tests
* Update unit tests
* Skip soon-to-be-ported unit tests
* Attempt to fix a test
* Update test
* Update unit test
* Update test
* Revert refetching
* Try simple flex layout
* Update proportions
* Remove redundant elements
* Clean up
* Use ellipsis instead of wrapping
* Pixel perfect
* Tweak constants
* SEM-448 Tests parity - unit tests (#60042)
* Add DataModel helpers
* Update test
* Add waits
* Use schemaId in test helper
* Port tests
* Fix ids
* Add more helpers
* Use helpers
* Auto-open coercion picker, humanize coercion picker labels
* Update test
* Fix typing
* Fix data fetching
* Update tests
* Refactor
* Update tests
* Update helpers
* Update reproductions.cy.spec.ts
* Fix test
* Update test
* Update test
* Update test
* Update tests
* Update test
* Update test
* Update test
* Update test
* Update tests
* Update tests
* Typo
* Update unit tests
* Remove encodeURIComponent
* Make it clear what is schemaName and schemaId
* Rename schemaId to schemaName in table picker
* Update unit tests
* Update unit tests
* Skip soon-to-be-ported unit tests
* Attempt to fix a test
* Update test
* Update unit test
* Update test
* Revert refetching
* Try simple flex layout
* Update proportions
* Remove redundant elements
* Clean up
* Use ellipsis instead of wrapping
* Pixel perfect
* Tweak constants
* Add data model unit tests boilerplate
* Optimistically remove TODOs
* Use router
* Fix element rendering in tests
* Add tests + fixes
* Port tests
* Port a test, fix a util
* Port test
* Port tests
* Port tests
* Remove old tests
* Port tests
* Remove old tests
* Port tests
* Remove old tests
* Fix disabling tables in search results
* Fix flakes
* Update tests
* Remove redundant check
* SEM-434 Missing metadata in FieldDataSelector (#60262)
* Manually override tables prop and add missing properties to Field objects
- Fixes SEM-434
* Format code
* Use field.display_name
* Memoize tables
* SEM-462 Small tweaks (#60187)
* SEM-436 Make field description input auto-grow indefinitely
* SEM-438 Update icons for tables and fields
* SEM-439 Remove the first column with dashes from the field values preview
* SEM-441 Align color scheme for left navbar and right-side panes
* SEM-440 Have a loader for the Detail view preview
* SEM-437 Sorting mode tweaks
* SEM-446 Chips as section headers feel off
* SEM-445 Revisit the read-only version of field ID and DB name
* Make shadows exactly like in figma
* SEM-457 Top-align text with icon for tables with long names
* Remove redundant selector
* Pixel perfect padding
* Fix 'draggable' prop not kicking in
* Fix tests
* Fix flakes
* Update tests
* Add spinner when toggling table visibility and using predefined sorting order
* Fix events
* Use prop instead of CSS
* Revert "Use prop instead of CSS"
This reverts commit 74f76d1b99.
* Add comments
* SEM-460 Show prefix for JSON-unfolded columns (#60328)
* SEM-436 Make field description input auto-grow indefinitely
* SEM-438 Update icons for tables and fields
* SEM-439 Remove the first column with dashes from the field values preview
* SEM-441 Align color scheme for left navbar and right-side panes
* SEM-440 Have a loader for the Detail view preview
* SEM-437 Sorting mode tweaks
* SEM-446 Chips as section headers feel off
* SEM-445 Revisit the read-only version of field ID and DB name
* Make shadows exactly like in figma
* SEM-457 Top-align text with icon for tables with long names
* Remove redundant selector
* Pixel perfect padding
* Fix 'draggable' prop not kicking in
* Fix tests
* Fix flakes
* Update tests
* Add spinner when toggling table visibility and using predefined sorting order
* Show prefix in FieldItem
* Show prefix in SortableFieldItem
* Fix events
* Handle ellipsis
* Show prefix in NameDescriptionInput
* Update tests
* Update types
* Use prop instead of CSS
* Improve responsiveness
* Revert "Use prop instead of CSS"
This reverts commit 74f76d1b99.
* Add comments
* SEM-420 Organize e2e tests (#60379)
* SEM-436 Make field description input auto-grow indefinitely
* SEM-438 Update icons for tables and fields
* SEM-439 Remove the first column with dashes from the field values preview
* SEM-441 Align color scheme for left navbar and right-side panes
* SEM-440 Have a loader for the Detail view preview
* SEM-437 Sorting mode tweaks
* SEM-446 Chips as section headers feel off
* SEM-445 Revisit the read-only version of field ID and DB name
* Make shadows exactly like in figma
* SEM-457 Top-align text with icon for tables with long names
* Remove redundant selector
* Pixel perfect padding
* Fix 'draggable' prop not kicking in
* Fix tests
* Fix flakes
* Update tests
* Add spinner when toggling table visibility and using predefined sorting order
* Show prefix in FieldItem
* Show prefix in SortableFieldItem
* Fix events
* Handle ellipsis
* Show prefix in NameDescriptionInput
* Update tests
* Update types
* Use prop instead of CSS
* Improve responsiveness
* Revert "Use prop instead of CSS"
This reverts commit 74f76d1b99.
* Move segments to a separate spec
* Organize tests
* Add comments
* SEM-420 Organize e2e tests (part 2) (#60388)
* SEM-436 Make field description input auto-grow indefinitely
* SEM-438 Update icons for tables and fields
* SEM-439 Remove the first column with dashes from the field values preview
* SEM-441 Align color scheme for left navbar and right-side panes
* SEM-440 Have a loader for the Detail view preview
* SEM-437 Sorting mode tweaks
* SEM-446 Chips as section headers feel off
* SEM-445 Revisit the read-only version of field ID and DB name
* Make shadows exactly like in figma
* SEM-457 Top-align text with icon for tables with long names
* Remove redundant selector
* Pixel perfect padding
* Fix 'draggable' prop not kicking in
* Fix tests
* Fix flakes
* Update tests
* Add spinner when toggling table visibility and using predefined sorting order
* Show prefix in FieldItem
* Show prefix in SortableFieldItem
* Fix events
* Handle ellipsis
* Show prefix in NameDescriptionInput
* Update tests
* Update types
* Use prop instead of CSS
* Improve responsiveness
* Revert "Use prop instead of CSS"
This reverts commit 74f76d1b99.
* Move segments to a separate spec
* Organize tests
* Move tests
* Move tests
* Add comments
* Merge tests
* Move tests
* Use modern selectors
* Move tests
* Move tests
* Reorder tests
* SEM-420 Test new features (e2e) (#60439)
* Add new tests, simplify toasts, remove long prefix from utils
* Test toasts
* Reuse utils
* Add table picker test for no schemas
* Fix schemaless tables not shown in search
* Add sync options test
* Fix & test prefixes
* Add logs
* Format code
* Coercion strategy test
* Add tests for visibility
* Update assertions
* Add test for tooltips
* Field values tests
* Add tests for filtering
* Add preview assertion
* Refactor preview verification
* Fix showing 403 error in preview
* Add preview tests
* Add preview assertions
* Add preview assertions
* Add preview assertions
* Add preview tests
* Move modals and state handling, add tests for Esc key
* Update unit tests
* Use useDisclosure
* Add extra error status check
* Revert unnecessary changes in cross-version tests
* SEM-419 Increase test coverage (unit tests) (#60491)
* Add new tests, simplify toasts, remove long prefix from utils
* Test toasts
* Reuse utils
* Add table picker test for no schemas
* Fix schemaless tables not shown in search
* Add sync options test
* Fix & test prefixes
* Add logs
* Format code
* Coercion strategy test
* Add tests for visibility
* Update assertions
* Add test for tooltips
* Field values tests
* Add tests for filtering
* Add preview assertion
* Refactor preview verification
* Fix showing 403 error in preview
* Add preview tests
* Add preview assertions
* Add preview assertions
* Add preview assertions
* Add preview tests
* Move modals and state handling, add tests for Esc key
* Update unit tests
* Add FieldOrderPicker unit tests
* Add CurrencyPicker unit tests
* Add FkTargetPicker unit tests
* Add more tests
* Add SemanticTypeAndTargetPicker unit tests
* Remove redundant code
* Move RemappingPicker
* Use useDisclosure
* Add extra error status check
* SEM-247 Clean up old Table Metadata page code (#60557)
* Remove old metadata page
* Move humanizeCoercionStrategy
* Remove unused functions
* Fix SEM-484 (#60600)
* Add tests
* Center field error
* Do not navigate if there's an explicit param
* Show field error
* Fix schemas loading, add tests
* Revert unnecessary change
* Fix data fetching
* Improve comments
* Remove invalid test for schemas
* Optimize conditions
* When table is not found, show the empty state as if table was not selected yet
---------
Co-authored-by: Romeo Van Snick <romeo@romeovansnick.be>
Co-authored-by: Nicola Mometto <nicola.mometto@metabase.com>
* Use displayName for error
* Fix error position for node
* Move padding to cm-content to avoid whitespace above gutter
* Add border radius to editor to avoid overflowing container
* Fix displayname in unit test
* Fix type for position
* Add test for error position
* Add unit test for position helper
* Add warning_triangle_filled icon
* Use warning_triangle_filled icon for error toasts in buttons
* Show error toasts consistently
* Fix error handling in table preview
* Fix error handling in object detail preview
* Update message
* Add toasts for sorting
* Add test for errors
* Add missing tests
* Update test
* Add iconColor prop to Undo and use it instead of hardcoding color in warning_triangle_filled icon
* Ensure that we keep the selected table even when collapsing the tree
- Fixes SEM-435
* Update tests
* Always highlight selected table
* Use aria-selected
* Add validation for case/if clause types
* Add e2e repro for metabase#57674
* Avoid passing nil default into expressions
* Check for type compatibility, not equality
* Allow unknoen types
* Make e2e test more refined
* Fix e2e test
* Fix clj test
* Fix e2e test
* Handle set arguments to compatible?
* Fix docstring
* Use assignable? instead of most-common-ancestor
* Allow Date/DateTime/... to mix
* Use best-return-type to determine if all types are compatible
* Handle falsy default
* Add missing map
* Update comment to clarify compatibility omission
* Unwrap schema
* Add test for malformed case expressions
* Add type check to coalaesce too
* Add check to coalesce too
* Add tests for :default false
* Add tests for temporal types
* Move combination tests to BE test
* Use separate deftests for valid and invalid case and coalesce tests
* Fold repetitive parts into are expression
* Formatting of test
Co-authored-by: Cam Saul <1455846+camsaul@users.noreply.github.com>
* Formatting of test
Co-authored-by: Cam Saul <1455846+camsaul@users.noreply.github.com>
* Formatting of test
Co-authored-by: Cam Saul <1455846+camsaul@users.noreply.github.com>
* Split up more tests
* Formatting in tests
---------
Co-authored-by: Cam Saul <1455846+camsaul@users.noreply.github.com>
* Add option to use instance settings for number formatting
* Format row count in datagrid Footer with instance settings
* Format row count in datagrid Footer using instance settings
* Format row in question row count using instance settings
* Add unit test for number formatting with instance settings
* Fix unit tests
* Fix e2e tests
* Update Loki Snapshots
* Add useNumberFormatter hook
* Use useNumberFormatter hook
* Add test for useNumberFormatter
* Fix types
* Update Loki Snapshots
* Move useNumberFormatter to metase/common/hooks
* Add redux to DataGrid stories
* Add missing loki references
---------
Co-authored-by: Metabase Automation <github-automation@metabase.com>
We check for query cycles when saving cards, but the checker only understands mbql queries. If we have a native query
that depends on another query, the cycle checker won't go through the native query to see if it is causing a cycle.
Now, the checker understands native queries and can find cycles involving them.
Co-authored-by: Alexander Polyankin <alexander.polyankin@metabase.com>
Co-authored-by: Metabase Automation <github-automation@metabase.com>
* [Lib] Stop passing `:unique-name-fn` as an option
* Add extra test
* Fix column order
* Test fixes and PR feedback
* Fix output schema
* Test fixes!!!
* Test fix?? 🔧
* Fix file order
* Test fix
* Revert accidental e2e changes
* typo
* infer only one column per table to be entity name
* tests
* fix: test
* note
* fix: don't lose other classification results!
* regr test
* use count
* more tests
* fix: only consider active fields!
* also consider visibility_type
* perf: select only if needed
* style: reduce without atom
* style: cleaner
* SEM-239 Data model - Layout boilerplate (#57174)
* Add route and link
* Convert DataModelApp to TypeScript
* Use DataModelApp in DataModel
* Keep all routes
* Extract section components
* Add empty states
* Use ts-pattern
* Drop ts-pattern
* Disable preview
* Rename
* Add border
* Move page components to their own dirs
* SEM-240 Data model - Table section (#57248)
* Add route and link
* Convert DataModelApp to TypeScript
* Use DataModelApp in DataModel
* Keep all routes
* Extract section components
* Add empty states
* Use ts-pattern
* Drop ts-pattern
* Disable preview
* Rename
* Add border
* Move page components to their own dirs
* Add table fetching
* Add NameDescriptionInput boilerplate
* Update table on blur
* Fix focus state and prevent empty names
* Add Hide this table switch
- Label from https://www.loom.com/share/45ce00aaac3f4141adf3f52dec79048f?highlightComment=80327135&t=271
* Style the Switch
* Move FieldOrderPicker
* Move SortableField
* Move NameDescriptionInput
* Make NameDescriptionInput more generic
* Use FieldOrderPicker
* Rename
* Move label into TableVisibilityInput
* Add missing return
* Add FieldOrder
* Get rid of entity framework workarounds
* Add TODO comment
* Make header sticky
* Sort
* Add field scanning section
* Discard + rescan
* Remove todo
* Remove TableVisibilityInput
* Rename FieldOrder to SortableFieldList and move it to metadata/components
* Implement getFieldIcon boilerplate
* Do not use metabase-lib/v1
* Extract tableFieldToColumnMetadata
* Fix focus state being cut off
* Extract RescanFieldsButton, add error and success states
* Extract DiscardFieldValuesButton
* Rename
* Move to metadata/components
* Style name/description input
* Add missing ttag
* Add NameDescriptionInput tests
* Add more tests
* Organize tests
* Add tests
* Add DiscardTableFieldValuesButton tests
* Add test
* Add RescanTableFieldsButton tests
* Lint colors
* Add tests for getItems
* Add tests for getFieldIcon
* Update comment
* Update test names
* Add missing effect
* Add missing effect
* Fix flaky unit tests
* Remove getFieldIcon
* Extract TextInputBlurChange
* Update comment
* SEM-241 Data model - Field - Data section (#57420)
* Add route and link
* Convert DataModelApp to TypeScript
* Use DataModelApp in DataModel
* Keep all routes
* Extract section components
* Add empty states
* Use ts-pattern
* Drop ts-pattern
* Disable preview
* Rename
* Add border
* Move page components to their own dirs
* Add table fetching
* Add NameDescriptionInput boilerplate
* Update table on blur
* Fix focus state and prevent empty names
* Add Hide this table switch
- Label from https://www.loom.com/share/45ce00aaac3f4141adf3f52dec79048f?highlightComment=80327135&t=271
* Style the Switch
* Move FieldOrderPicker
* Move SortableField
* Move NameDescriptionInput
* Make NameDescriptionInput more generic
* Use FieldOrderPicker
* Rename
* Move label into TableVisibilityInput
* Add missing return
* Add FieldOrder
* Get rid of entity framework workarounds
* Add TODO comment
* Make header sticky
* Sort
* Add field scanning section
* Discard + rescan
* Remove todo
* Remove TableVisibilityInput
* Rename FieldOrder to SortableFieldList and move it to metadata/components
* Implement getFieldIcon boilerplate
* Do not use metabase-lib/v1
* Extract tableFieldToColumnMetadata
* Fix focus state being cut off
* Extract RescanFieldsButton, add error and success states
* Extract DiscardFieldValuesButton
* Rename
* Move to metadata/components
* Style name/description input
* Add missing ttag
* Add NameDescriptionInput tests
* Add more tests
* Organize tests
* Add tests
* Add DiscardTableFieldValuesButton tests
* Add test
* Add RescanTableFieldsButton tests
* Lint colors
* Add tests for getItems
* Add tests for getFieldIcon
* Update comment
* Update test names
* Add missing effect
* Add missing effect
* Navigation onClick
* Fix flaky unit tests
* Use a link
* Explain cast
* Improve url generation
* Style active link
* Load field, display name
* Sticky header
* Add SectionPill
* Add field name and data type
* Add cast toggle
* Add CoercionStrategyPicker
* Remove getFieldIcon
* Extract TextInputBlurChange
* Update comment
* Casting toggle state sync
* CoercionStrategyPicker error state
* Extract FieldDataSection
* Reuse existing TextInputBlurChange
* Further cleanup
* Use key for FieldSection
* Lift data fetching
* Add decoration
* Use key for table as well
* Add unit tests
* Add unit tests
* Add unit tests
* Add comments
* Simplify color
* Use type instead of interface
* SEM-297 Drop entity framework usage in `SemanticTypePicker` (#57531)
* Add route and link
* Convert DataModelApp to TypeScript
* Use DataModelApp in DataModel
* Keep all routes
* Extract section components
* Add empty states
* Use ts-pattern
* Drop ts-pattern
* Disable preview
* Rename
* Add border
* Move page components to their own dirs
* Add table fetching
* Add NameDescriptionInput boilerplate
* Update table on blur
* Fix focus state and prevent empty names
* Add Hide this table switch
- Label from https://www.loom.com/share/45ce00aaac3f4141adf3f52dec79048f?highlightComment=80327135&t=271
* Style the Switch
* Move FieldOrderPicker
* Move SortableField
* Move NameDescriptionInput
* Make NameDescriptionInput more generic
* Use FieldOrderPicker
* Rename
* Move label into TableVisibilityInput
* Add missing return
* Add FieldOrder
* Get rid of entity framework workarounds
* Add TODO comment
* Make header sticky
* Sort
* Add field scanning section
* Discard + rescan
* Remove todo
* Remove TableVisibilityInput
* Rename FieldOrder to SortableFieldList and move it to metadata/components
* Implement getFieldIcon boilerplate
* Do not use metabase-lib/v1
* Extract tableFieldToColumnMetadata
* Fix focus state being cut off
* Extract RescanFieldsButton, add error and success states
* Extract DiscardFieldValuesButton
* Rename
* Move to metadata/components
* Style name/description input
* Add missing ttag
* Add NameDescriptionInput tests
* Add more tests
* Organize tests
* Add tests
* Add DiscardTableFieldValuesButton tests
* Add test
* Add RescanTableFieldsButton tests
* Lint colors
* Add tests for getItems
* Add tests for getFieldIcon
* Update comment
* Update test names
* Add missing effect
* Add missing effect
* Navigation onClick
* Fix flaky unit tests
* Use a link
* Explain cast
* Improve url generation
* Style active link
* Load field, display name
* Sticky header
* Add SectionPill
* Add field name and data type
* Add cast toggle
* Add CoercionStrategyPicker
* Remove getFieldIcon
* Extract TextInputBlurChange
* Update comment
* Casting toggle state sync
* CoercionStrategyPicker error state
* Extract FieldDataSection
* Reuse existing TextInputBlurChange
* Further cleanup
* Use key for FieldSection
* Lift data fetching
* Add decoration
* Use key for table as well
* Add unit tests
* Add unit tests
* Add unit tests
* Add comments
* Drop entity framework in FkTargetPicker
* Make table names show up
* Clean up
* Fix types
* Fix table names
* Add tests
* Add tests
* Add tests
* Restrict imports
* Improve config
* Restrict entity framework imports
* Simplify color
* Use type instead of interface
* Lint
* SEM-267 Data model - Field - Metadata section (#57527)
* Add route and link
* Convert DataModelApp to TypeScript
* Use DataModelApp in DataModel
* Keep all routes
* Extract section components
* Add empty states
* Use ts-pattern
* Drop ts-pattern
* Disable preview
* Rename
* Add border
* Move page components to their own dirs
* Add table fetching
* Add NameDescriptionInput boilerplate
* Update table on blur
* Fix focus state and prevent empty names
* Add Hide this table switch
- Label from https://www.loom.com/share/45ce00aaac3f4141adf3f52dec79048f?highlightComment=80327135&t=271
* Style the Switch
* Move FieldOrderPicker
* Move SortableField
* Move NameDescriptionInput
* Make NameDescriptionInput more generic
* Use FieldOrderPicker
* Rename
* Move label into TableVisibilityInput
* Add missing return
* Add FieldOrder
* Get rid of entity framework workarounds
* Add TODO comment
* Make header sticky
* Sort
* Add field scanning section
* Discard + rescan
* Remove todo
* Remove TableVisibilityInput
* Rename FieldOrder to SortableFieldList and move it to metadata/components
* Implement getFieldIcon boilerplate
* Do not use metabase-lib/v1
* Extract tableFieldToColumnMetadata
* Fix focus state being cut off
* Extract RescanFieldsButton, add error and success states
* Extract DiscardFieldValuesButton
* Rename
* Move to metadata/components
* Style name/description input
* Add missing ttag
* Add NameDescriptionInput tests
* Add more tests
* Organize tests
* Add tests
* Add DiscardTableFieldValuesButton tests
* Add test
* Add RescanTableFieldsButton tests
* Lint colors
* Add tests for getItems
* Add tests for getFieldIcon
* Update comment
* Update test names
* Add missing effect
* Add missing effect
* Navigation onClick
* Fix flaky unit tests
* Use a link
* Explain cast
* Improve url generation
* Style active link
* Load field, display name
* Sticky header
* Add SectionPill
* Add field name and data type
* Add cast toggle
* Add CoercionStrategyPicker
* Remove getFieldIcon
* Extract TextInputBlurChange
* Update comment
* Casting toggle state sync
* CoercionStrategyPicker error state
* Extract FieldDataSection
* Reuse existing TextInputBlurChange
* Further cleanup
* Use key for FieldSection
* Lift data fetching
* Add decoration
* Use key for table as well
* Add unit tests
* Add unit tests
* Add unit tests
* Add comments
* Add Metadata section with display name input
* Pass table through components
* Drop entity framework in FkTargetPicker
* Revert "Pass table through components"
This reverts commit 2eb3e3a256.
* Make table names show up
* Clean up
* Fix types
* Fix table names
* Add tests
* Add tests
* Add tests
* Restrict imports
* Improve config
* Restrict entity framework imports
* Simplify color
* Use type instead of interface
* Use SemanticTypeAndTargetPicker in FieldSection
* Sort
* Lint
* Typing
* Do not use v1/metadata wrapper
* Add TextareaBlurChange
* Add description input
* Update comments
* Update icon
* Add plugin query props
* Remove redundant prop
* Remove redundant _.omit
* Fix textarea crashing due to min height on input
* Set minRows
* Fix styling
* Remove PLUGIN_FEATURE_LEVEL_PERMISSIONS plugin usage from DatasetFieldMetadataFkTargetPicker
- because that's not how it works in master
* Drop -v2 suffix for new routes, use -v1 suffix for old routes (#57637)
* SEM-242 Data model - Field - Behavior section (#57638)
* Drop -v2 suffix for new routes, use -v1 suffix for old routes
* Move FieldVisibilityPicker
* Make FieldVisibilityPicker work with api Field
* Use FieldVisibilityType
* Fix gap
* Update tests
* Add label and description
* Add FieldValuesTypePicker
* Add canUnfoldJson and hasDatabaseFeature
* Rename
* Add UnfoldJsonPicker
* Fix typing
* Add RescanFieldButton
* Add DiscardFieldValuesButton
* Style the connector
* Fix scrolling
* Update spacing
* Add tests for FieldValuesTypePicker
* Add tests
* Add UnfoldJsonPicker tests
* Add tests
* Add tests
* Clean up timeouts on unmount
* Improve typing
* SEM-269 Data model - Field - Formatting section (#57682)
* Drop -v2 suffix for new routes, use -v1 suffix for old routes
* Move FieldVisibilityPicker
* Make FieldVisibilityPicker work with api Field
* Use FieldVisibilityType
* Fix gap
* Update tests
* Add label and description
* Add FieldValuesTypePicker
* Add canUnfoldJson and hasDatabaseFeature
* Rename
* Add UnfoldJsonPicker
* Fix typing
* Add RescanFieldButton
* Add DiscardFieldValuesButton
* Style the connector
* Fix scrolling
* Update spacing
* Add tests for FieldValuesTypePicker
* Add tests
* Add UnfoldJsonPicker tests
* Add tests
* Add tests
* Add FormattingSection
* Sort
* Make formatting form full width
* Use style prop instead of maxWidth
* Clean up timeouts on unmount
* Improve typing
* Metadata preview boilerplate (#58015)
* Render preview
* Add preview type selector
* Fix flex layout
* Lift preview type to top component to keep the state
* Move helpers to utils
* Add unit tests for preview picker
* Add feedback to metadata editor when saving config (#58038)
* Add feedback to all table and field settings
* Add missing prop
* Metadata table and detail preview (#57887)
* Handle undefined metadata
* Render Table preview
* Add custom error component
* Add helper for error message
* Add detail preview
* Make comment more clear
* Fix type
* Fix missing props
* Metadata table picker (#57648)
* Set up TablePicker component
* Fix type in api
* Add initial TablePicker implementation
* Add schema icon
* Update url for datapicker
* Move data loading to utils
* Add loading node types
* Derive icon from type
* Try building the treepicker state asynchronously
* Build custom tree
* Add loading state
* Push url state in picker
* Sync open state with url
* Use helper to manage expanded state
* Split Node into separate file
* Expose prefetch hook
* Prefetch data to avoid showing loading states
* Add search to table picker
* Move search-related stuff into Search module
* Use Link for navigation
* Add renderLoading helper
* Remove unused prop
* Make search input sticky
* Add separate sidebar for field picker
* Render single schema but open it by default
* Translate errors
* Defer search value
* Export lazy loaders for database info
* Use top-level data fetching
* Enable virtualization
* Move loader utils into utils
* Remove Node implementation
* Simplify styles
* Pull out some logic
* Reuse results in search
* Reuse tree helpers when handling search results
* Avoid passing down isExpanded
* Memoize
* Fix initial expanded state
* Automatically expand single-schema databases
* Pass tableId to DataPicker
* Add styling for active item
* Add getUrl helper for tree nodes
* Move types to separate file
* Simplify item
* Rename Item to Results
* Simplify
* Allow wrapping everywhere
* Add loading state
* Render tracks
* Remove unused colors
* Scroll active item into view
* Fix paths
* Add missing keys
* Handle browser navigation in table picker
* Handle browser navigation in table picker
* Add more documentation
* Debounce search
* Render track even when no parent was found
* Allow rendering urls to schema's with no name
* Add value to loading nodes
* Add delay to track too
* Add explicit level to FlatItem
* Add explicit parent refence to FlatItem
* Hide nameless schemas in the tree
* Refactor
* Move key to top-level item
* Make TablePicker not depend on router
* Disable smooth scroll
* Add unit tests for TablePicker
* Unnest unnamed schemas directly
* Also fetch tables when only one schema exists
* Remeasure items when they change
* Use more precise typeing for items
* Allow clicking table nodes in search results
* Remove unused classes
* Remove unused useDispatch hook
* Remove delay on loading items
* Add custom empty state
* Typo
* Add keyboard navigation to table picker
* Use test helpers in all tests
* Add keyboard navigation to search results too
* Add custom focus state for search items
* Add unit tests for search
* Jump to parent when pressing left arrow
* Add more cases to keyboard test
* Add hover state to table picker
* Use table display name in table picker
* Flatten single schema in to database
* Rename onChange_ to handleChange
* Use folder for schema
* Make the hover and active colors the same
* Rename Data tab to Data Model
* Remove title
* Fix layout
* Field description editor (#58280)
* Use NameDescriptionInput for the field section too
* Use TextArea for description editor
* Post-merge fixes
* Metadata editing UI tweaks (#58297)
* Add background to fields
* Tweak colors for disabled inputs
* Remove icons in SectionPill
* Remove icon from one more SectionPill
* Remove lint
* Move segments to table picker (#58422)
* Add link to segments in the datamodel sidebar
* Add local state to RouterTablePicker
* Nest segments under DataModel
* Add wrapper classes to revision routes
* Fix spacing for empty state
* Add active state to segments link
* Remove DataModelApp in DataModel routes
* Fix overflow in segment form
* Merge translation strings
* Add background color to segments components
* SEM-363 Data model - Field and table name/description inputs should be sticky and have icons (#58703)
* Remove hover/focus effect and break components down into files
* Update comment
* Consolidate toasts
* Remove unused styles
* Remove redundant card
* Extract SegmentsLink
* Make name/description input sticky
* Add icons
* Remove leftovers
* Fix TS error
* Move visibility toggles to the table picker (#58284)
* Fetch hidden tables
* Fetch hidden schemas
* Add visibility toggles to the table picker
* Remove table visibility toggle in table section
* Simplify visibility toggler
* Move visibility toggle to separate file
* Add unit tests for visibility toggle
* Show table as hidden whenever there is any visibility
* Use ActionIcon
* Break up onClick handler
* Use plan Icon
* Remove data from nodes that don't need it
* Add tooltips to table visibility togglers
* Wrap icons in ActionIcon
---------
Co-authored-by: Kamil Mielnik <kamil@kamilmielnik.com>
* Cleanup
* SEM-361 Data model - Field metadata/formatting settings do not affect the preview (#58841)
* Fix SEM-361
* Fix types
* SEM-311 Table preview - no impact when changing time formatting for seconds and ms (#58927)
* Avoid using a breakout for date-time columns
* Use the same logic for every type of field
* SEM-246 Data model - Filtering preview (#58854)
* Fix SEM-361
* Fix types
* Add FilteringPreview boilerplate
* Use FilterPickerBody
* Another attempt
* Make it work for all types of filters
* Memoize query
* Clear filter input on field change
* Account for prop changes
* Fix TS error
* SEM-268 Data model - Field - Behavior section - Display values (#57684)
* Drop -v2 suffix for new routes, use -v1 suffix for old routes
* Move FieldVisibilityPicker
* Make FieldVisibilityPicker work with api Field
* Use FieldVisibilityType
* Fix gap
* Update tests
* Add label and description
* Add FieldValuesTypePicker
* Add canUnfoldJson and hasDatabaseFeature
* Rename
* Add UnfoldJsonPicker
* Fix typing
* Add RescanFieldButton
* Add DiscardFieldValuesButton
* Style the connector
* Fix scrolling
* Update spacing
* Add tests for FieldValuesTypePicker
* Add tests
* Add UnfoldJsonPicker tests
* Add tests
* Add tests
* Add placeholder
* Add RemappingPicker boilerplate
* Clean up timeouts on unmount
* Improve typing
* Add onChange logic boilerplate
* Extract DisplayValuesPicker
* Remove redundant code
* Format code
* Add fk input boilerplate
* Disable options instead of hiding them
* Rename
* Fetch required data
* Use full data
* Implement fk remapping onChange
* Implement naming tip
* Fix state
* Extract utils
* Fix schema step errors
* Add remappings boilerplate
* Fetch remappings
* Add more modal boilerplate
* Add modal props
* Handle modal state
* Add explicit edit button
* Implement modal form
* Limit amount of Boxes
* Bring all safety checks back in
* Refactor CustomMappingModal
* Style CustomMappingModal
* Extract getHasEmptyValues
* Remove todo
* Prevent flicker
* Fix initial state of fk picker
* Use is touched
* Add CustomMappingModal unit tests
* Add DisplayValuesPicker unit tests
* Move DisplayValuesPicker
* Remove redundant tags
* Fix uncontrolled/controller input react warning
* Add explanatory tooltips for disabled options
* Fix 1px of background leaking into view
* Add missing memoization
* Reset state when cancelling
* Sync single table via UI (#58342)
* add table-manual-sync event
* add table/sync_schema endpoint
* add table sync test
* Add api helper for the sync table schema endpoint
* Add SyncTableSchemaButton component
* Add SyncTableSchemaButton to table section
* Add unit tests for SyncTableSchemaButton
* Also invalidate field-values and parameter-values
* Simplify SyncTableSchemaButton hooks
* Move useTemporaryState into a separate hook
* Apply same pattern in all buttons
* address PR review comments: use `get-database`, `with-database-routing-off`
* Pass argument to useTemporaryState
* update test
---------
Co-authored-by: Nicola Mometto <nicola.mometto@metabase.com>
* Fix SEM-370 (#59051)
* Post-merge conflict fix
* SEM-329 Data model - Table section - Update fields sorting interaction (#59316)
* Add support for size="sm" in SegmentedControl
* Add field sorting interaction boilerplate
* Update FieldOrderPicker to use SegmentedControl
* Extract Label component
* Update tooltips
* Update sortable field
* Rename SortableField to SortableFieldItem
* Render field list
* Align styling
* Use rem, remove Link from SortableFieldItem
* Styling
* Use field prop
* Background styling
* Fix TS error
* Editable name and description
* Watch out for error
* Allow overriding default props in EditableText
* Fix a11y
* A11y
* Prevent overflow
* Styling
* Cleanup
* Update comment
* Fixes
* Update label and hide it in sorting mode
* Update spacing
* Avoid CLS when entering sorting mode
* Pixel-perfect
* Use ellipsis on overflow
* Fix colors
* Fix name input size
* Fix description patching
* Style selected state
* Do not use EditableText for description input, add resetOnEsc prop
* Use flushSync instead of setTimeout
* Try to prevent event propagation
* Fix interactivity in inputs
* SEM-376 Data model - Resizable columns layout (#59100)
* Format code
* Prevent updates when nothing changes
* Use new colors
* Make nav resizable
* Fix nav width
* Remove unused styles
* Prevent CLS in SortableList
* Add comment
* Get rid of second page component
* Make table column resizable
* Add padding and min width for the empty state
* Optimize
* Extract ResizableColumn
* Toggle preview
* Add preview and field resizing
* Extract getTableMetadataQuery
* Preview positioning and shadow
* Show scrollbar on preview overflow
* Fix resizing bounds
* Remove redundant props
* Handle Escape key
* Add comment
* Update default constraints
* Make preview non-resizable
* Fix valid Escape press detection
* Fix types
* Add support for size="sm" in SegmentedControl
* Add field sorting interaction boilerplate
* Update FieldOrderPicker to use SegmentedControl
* Extract Label component
* Update tooltips
* Update sortable field
* Rename SortableField to SortableFieldItem
* Render field list
* Align styling
* Use rem, remove Link from SortableFieldItem
* Styling
* Use field prop
* Background styling
* Fix TS error
* Editable name and description
* Watch out for error
* Allow overriding default props in EditableText
* Fix a11y
* A11y
* Prevent overflow
* Styling
* Cleanup
* Update comment
* Fixes
* Update label and hide it in sorting mode
* Update spacing
* Avoid CLS when entering sorting mode
* Auto-grow table column when possible
* Auto-magic layout
* Fix sign
* Optimize
* Optimize
* Pixel-perfect
* Use ellipsis on overflow
* Fix colors
* Fix name input size
* Fix description patching
* Style selected state
* Do not use EditableText for description input, add resetOnEsc prop
* Use flushSync instead of setTimeout
* Try to prevent event propagation
* Fix interactivity in inputs
* Fix interactivity in inputs
* Move memoization closer to component definition
* SEM-327 Data model - Move the re-scan / discard options away from the bottom of the fields list (#59474)
* Format code
* Prevent updates when nothing changes
* Use new colors
* Make nav resizable
* Fix nav width
* Remove unused styles
* Prevent CLS in SortableList
* Add comment
* Get rid of second page component
* Make table column resizable
* Add padding and min width for the empty state
* Optimize
* Extract ResizableColumn
* Toggle preview
* Add preview and field resizing
* Extract getTableMetadataQuery
* Preview positioning and shadow
* Show scrollbar on preview overflow
* Fix resizing bounds
* Remove redundant props
* Handle Escape key
* Add comment
* Update default constraints
* Make preview non-resizable
* Fix valid Escape press detection
* Fix types
* Add support for size="sm" in SegmentedControl
* Add field sorting interaction boilerplate
* Update FieldOrderPicker to use SegmentedControl
* Extract Label component
* Update tooltips
* Update sortable field
* Rename SortableField to SortableFieldItem
* Render field list
* Align styling
* Use rem, remove Link from SortableFieldItem
* Styling
* Use field prop
* Background styling
* Fix TS error
* Editable name and description
* Watch out for error
* Allow overriding default props in EditableText
* Fix a11y
* A11y
* Prevent overflow
* Styling
* Cleanup
* Update comment
* Fixes
* Update label and hide it in sorting mode
* Update spacing
* Avoid CLS when entering sorting mode
* Auto-grow table column when possible
* Auto-magic layout
* Fix sign
* Optimize
* Optimize
* Pixel-perfect
* SyncOptionsModal
* Move
* Field values modal
* Responsiveness
* Use ellipsis on overflow
* Fix colors
* Fix name input size
* Fix description patching
* Style selected state
* Do not use EditableText for description input, add resetOnEsc prop
* Use flushSync instead of setTimeout
* Try to prevent event propagation
* Fix interactivity in inputs
* Fix interactivity in inputs
* Move memoization closer to component definition
* Tmp
* Revert "Tmp"
This reverts commit 6e561c0de7.
* SEM-424 Tests parity - datamodel.cy.spec.js (#59548)
* Format code
* Prevent updates when nothing changes
* Use new colors
* Make nav resizable
* Fix nav width
* Remove unused styles
* Prevent CLS in SortableList
* Add comment
* Get rid of second page component
* Make table column resizable
* Add padding and min width for the empty state
* Optimize
* Extract ResizableColumn
* Toggle preview
* Add preview and field resizing
* Extract getTableMetadataQuery
* Preview positioning and shadow
* Show scrollbar on preview overflow
* Fix resizing bounds
* Remove redundant props
* Handle Escape key
* Add comment
* Update default constraints
* Make preview non-resizable
* Fix valid Escape press detection
* Fix types
* Add support for size="sm" in SegmentedControl
* Add field sorting interaction boilerplate
* Update FieldOrderPicker to use SegmentedControl
* Extract Label component
* Update tooltips
* Update sortable field
* Rename SortableField to SortableFieldItem
* Render field list
* Align styling
* Use rem, remove Link from SortableFieldItem
* Styling
* Use field prop
* Background styling
* Fix TS error
* Editable name and description
* Watch out for error
* Allow overriding default props in EditableText
* Fix a11y
* A11y
* Prevent overflow
* Styling
* Cleanup
* Update comment
* Fixes
* Update label and hide it in sorting mode
* Update spacing
* Avoid CLS when entering sorting mode
* Auto-grow table column when possible
* Auto-magic layout
* Fix sign
* Optimize
* Optimize
* Pixel-perfect
* SyncOptionsModal
* Move
* Field values modal
* Responsiveness
* Use ellipsis on overflow
* Fix colors
* Fix name input size
* Fix description patching
* Style selected state
* Do not use EditableText for description input, add resetOnEsc prop
* Port test
* Fix typing
* Port test
* Port tests
* Format code, add data-testid
* Port more tests
* Refactor withDatabase to TypeScript
* Await
* Fix prop handling
* Port test
* Memoize textarea for performance
* Port test
* Port tests
* Remove unlawful getFieldDisplayName usages
* ESLint
* Add JSDoc
* Port tests
* Port tests
* Format code
* Use flushSync instead of setTimeout
* Try to prevent event propagation
* Fix interactivity in inputs
* Fix interactivity in inputs
* Move memoization closer to component definition
* Tmp
* Revert "Tmp"
This reverts commit 6e561c0de7.
* Fix circular dependency
* Post-merge fix
* SEM-425 Tests parity - editor.cy.spec.js (#59838)
* Add DataModel helpers
* Update test
* Add waits
* Use schemaId in test helper
* Port tests
* Fix ids
* Add more helpers
* Use helpers
* Auto-open coercion picker, humanize coercion picker labels
* Update test
* Fix typing
* Fix data fetching
* Update tests
* Refactor
* Update tests
* Update helpers
* Simplify assertion
* SEM-426 Tests parity - reproductions.cy.spec.ts (#59970)
* Add DataModel helpers
* Update test
* Add waits
* Use schemaId in test helper
* Port tests
* Fix ids
* Add more helpers
* Use helpers
* Auto-open coercion picker, humanize coercion picker labels
* Update test
* Fix typing
* Fix data fetching
* Update tests
* Refactor
* Update tests
* Update helpers
* Update reproductions.cy.spec.ts
* Fix test
* Simplify assertion
* SEM-427 Tests parity - remaining e2e suites (#59980)
* Add DataModel helpers
* Update test
* Add waits
* Use schemaId in test helper
* Port tests
* Fix ids
* Add more helpers
* Use helpers
* Auto-open coercion picker, humanize coercion picker labels
* Update test
* Fix typing
* Fix data fetching
* Update tests
* Refactor
* Update tests
* Update helpers
* Update reproductions.cy.spec.ts
* Fix test
* Update test
* Update test
* Update test
* Update tests
* Update test
* Update test
* Update test
* Update test
* Update tests
* Update tests
* Typo
* Update unit tests
* Remove encodeURIComponent
* Make it clear what is schemaName and schemaId
* Rename schemaId to schemaName in table picker
* Update unit tests
* Update unit tests
* Skip soon-to-be-ported unit tests
* Attempt to fix a test
* Update test
* Update unit test
* Update test
* Revert refetching
* Data model - Simple flex layout (#59995)
* Add DataModel helpers
* Update test
* Add waits
* Use schemaId in test helper
* Port tests
* Fix ids
* Add more helpers
* Use helpers
* Auto-open coercion picker, humanize coercion picker labels
* Update test
* Fix typing
* Fix data fetching
* Update tests
* Refactor
* Update tests
* Update helpers
* Update reproductions.cy.spec.ts
* Fix test
* Update test
* Update test
* Update test
* Update tests
* Update test
* Update test
* Update test
* Update test
* Update tests
* Update tests
* Typo
* Update unit tests
* Remove encodeURIComponent
* Make it clear what is schemaName and schemaId
* Rename schemaId to schemaName in table picker
* Update unit tests
* Update unit tests
* Skip soon-to-be-ported unit tests
* Attempt to fix a test
* Update test
* Update unit test
* Update test
* Revert refetching
* Try simple flex layout
* Update proportions
* Remove redundant elements
* Clean up
* Use ellipsis instead of wrapping
* Pixel perfect
* Tweak constants
* SEM-448 Tests parity - unit tests (#60042)
* Add DataModel helpers
* Update test
* Add waits
* Use schemaId in test helper
* Port tests
* Fix ids
* Add more helpers
* Use helpers
* Auto-open coercion picker, humanize coercion picker labels
* Update test
* Fix typing
* Fix data fetching
* Update tests
* Refactor
* Update tests
* Update helpers
* Update reproductions.cy.spec.ts
* Fix test
* Update test
* Update test
* Update test
* Update tests
* Update test
* Update test
* Update test
* Update test
* Update tests
* Update tests
* Typo
* Update unit tests
* Remove encodeURIComponent
* Make it clear what is schemaName and schemaId
* Rename schemaId to schemaName in table picker
* Update unit tests
* Update unit tests
* Skip soon-to-be-ported unit tests
* Attempt to fix a test
* Update test
* Update unit test
* Update test
* Revert refetching
* Try simple flex layout
* Update proportions
* Remove redundant elements
* Clean up
* Use ellipsis instead of wrapping
* Pixel perfect
* Tweak constants
* Add data model unit tests boilerplate
* Optimistically remove TODOs
* Use router
* Fix element rendering in tests
* Add tests + fixes
* Port tests
* Port a test, fix a util
* Port test
* Port tests
* Port tests
* Remove old tests
* Port tests
* Remove old tests
* Port tests
* Remove old tests
* Fix disabling tables in search results
* Fix flakes
* Update tests
* Remove redundant check
* SEM-434 Missing metadata in FieldDataSelector (#60262)
* Manually override tables prop and add missing properties to Field objects
- Fixes SEM-434
* Format code
* Use field.display_name
* Memoize tables
* SEM-462 Small tweaks (#60187)
* SEM-436 Make field description input auto-grow indefinitely
* SEM-438 Update icons for tables and fields
* SEM-439 Remove the first column with dashes from the field values preview
* SEM-441 Align color scheme for left navbar and right-side panes
* SEM-440 Have a loader for the Detail view preview
* SEM-437 Sorting mode tweaks
* SEM-446 Chips as section headers feel off
* SEM-445 Revisit the read-only version of field ID and DB name
* Make shadows exactly like in figma
* SEM-457 Top-align text with icon for tables with long names
* Remove redundant selector
* Pixel perfect padding
* Fix 'draggable' prop not kicking in
* Fix tests
* Fix flakes
* Update tests
* Add spinner when toggling table visibility and using predefined sorting order
* Fix events
* Use prop instead of CSS
* Revert "Use prop instead of CSS"
This reverts commit 74f76d1b99.
* Add comments
* SEM-460 Show prefix for JSON-unfolded columns (#60328)
* SEM-436 Make field description input auto-grow indefinitely
* SEM-438 Update icons for tables and fields
* SEM-439 Remove the first column with dashes from the field values preview
* SEM-441 Align color scheme for left navbar and right-side panes
* SEM-440 Have a loader for the Detail view preview
* SEM-437 Sorting mode tweaks
* SEM-446 Chips as section headers feel off
* SEM-445 Revisit the read-only version of field ID and DB name
* Make shadows exactly like in figma
* SEM-457 Top-align text with icon for tables with long names
* Remove redundant selector
* Pixel perfect padding
* Fix 'draggable' prop not kicking in
* Fix tests
* Fix flakes
* Update tests
* Add spinner when toggling table visibility and using predefined sorting order
* Show prefix in FieldItem
* Show prefix in SortableFieldItem
* Fix events
* Handle ellipsis
* Show prefix in NameDescriptionInput
* Update tests
* Update types
* Use prop instead of CSS
* Improve responsiveness
* Revert "Use prop instead of CSS"
This reverts commit 74f76d1b99.
* Add comments
* SEM-420 Organize e2e tests (#60379)
* SEM-436 Make field description input auto-grow indefinitely
* SEM-438 Update icons for tables and fields
* SEM-439 Remove the first column with dashes from the field values preview
* SEM-441 Align color scheme for left navbar and right-side panes
* SEM-440 Have a loader for the Detail view preview
* SEM-437 Sorting mode tweaks
* SEM-446 Chips as section headers feel off
* SEM-445 Revisit the read-only version of field ID and DB name
* Make shadows exactly like in figma
* SEM-457 Top-align text with icon for tables with long names
* Remove redundant selector
* Pixel perfect padding
* Fix 'draggable' prop not kicking in
* Fix tests
* Fix flakes
* Update tests
* Add spinner when toggling table visibility and using predefined sorting order
* Show prefix in FieldItem
* Show prefix in SortableFieldItem
* Fix events
* Handle ellipsis
* Show prefix in NameDescriptionInput
* Update tests
* Update types
* Use prop instead of CSS
* Improve responsiveness
* Revert "Use prop instead of CSS"
This reverts commit 74f76d1b99.
* Move segments to a separate spec
* Organize tests
* Add comments
* SEM-420 Organize e2e tests (part 2) (#60388)
* SEM-436 Make field description input auto-grow indefinitely
* SEM-438 Update icons for tables and fields
* SEM-439 Remove the first column with dashes from the field values preview
* SEM-441 Align color scheme for left navbar and right-side panes
* SEM-440 Have a loader for the Detail view preview
* SEM-437 Sorting mode tweaks
* SEM-446 Chips as section headers feel off
* SEM-445 Revisit the read-only version of field ID and DB name
* Make shadows exactly like in figma
* SEM-457 Top-align text with icon for tables with long names
* Remove redundant selector
* Pixel perfect padding
* Fix 'draggable' prop not kicking in
* Fix tests
* Fix flakes
* Update tests
* Add spinner when toggling table visibility and using predefined sorting order
* Show prefix in FieldItem
* Show prefix in SortableFieldItem
* Fix events
* Handle ellipsis
* Show prefix in NameDescriptionInput
* Update tests
* Update types
* Use prop instead of CSS
* Improve responsiveness
* Revert "Use prop instead of CSS"
This reverts commit 74f76d1b99.
* Move segments to a separate spec
* Organize tests
* Move tests
* Move tests
* Add comments
* Merge tests
* Move tests
* Use modern selectors
* Move tests
* Move tests
* Reorder tests
* SEM-420 Test new features (e2e) (#60439)
* Add new tests, simplify toasts, remove long prefix from utils
* Test toasts
* Reuse utils
* Add table picker test for no schemas
* Fix schemaless tables not shown in search
* Add sync options test
* Fix & test prefixes
* Add logs
* Format code
* Coercion strategy test
* Add tests for visibility
* Update assertions
* Add test for tooltips
* Field values tests
* Add tests for filtering
* Add preview assertion
* Refactor preview verification
* Fix showing 403 error in preview
* Add preview tests
* Add preview assertions
* Add preview assertions
* Add preview assertions
* Add preview tests
* Move modals and state handling, add tests for Esc key
* Update unit tests
* Use useDisclosure
* Add extra error status check
* Revert unnecessary changes in cross-version tests
* SEM-419 Increase test coverage (unit tests) (#60491)
* Add new tests, simplify toasts, remove long prefix from utils
* Test toasts
* Reuse utils
* Add table picker test for no schemas
* Fix schemaless tables not shown in search
* Add sync options test
* Fix & test prefixes
* Add logs
* Format code
* Coercion strategy test
* Add tests for visibility
* Update assertions
* Add test for tooltips
* Field values tests
* Add tests for filtering
* Add preview assertion
* Refactor preview verification
* Fix showing 403 error in preview
* Add preview tests
* Add preview assertions
* Add preview assertions
* Add preview assertions
* Add preview tests
* Move modals and state handling, add tests for Esc key
* Update unit tests
* Add FieldOrderPicker unit tests
* Add CurrencyPicker unit tests
* Add FkTargetPicker unit tests
* Add more tests
* Add SemanticTypeAndTargetPicker unit tests
* Remove redundant code
* Move RemappingPicker
* Use useDisclosure
* Add extra error status check
* SEM-247 Clean up old Table Metadata page code (#60557)
* Remove old metadata page
* Move humanizeCoercionStrategy
* Remove unused functions
* Fix SEM-484 (#60600)
* Revert unnecessary change
---------
Co-authored-by: Romeo Van Snick <romeo@romeovansnick.be>
Co-authored-by: Nicola Mometto <nicola.mometto@metabase.com>
* Closes VIZ-1224 - Make dashcards use chart-specific skeletons (#60090)
* Updates Dashboard cards' loading state to use already-made chart-specific loading skeletons
* Adds SankeySkeleton
* Tweaks MapSkeleton so its fill color can be overridden
* Fixes double name issue in loading skeletons in pinned questions on collections page
* Closes VIZ-1230 - Trigger spinner state when card takes longer to load (#60359)
* Closes VIZ-1218 reset button on data sources
* resets to default only for single datasources cards
* e2e
* e2e
* removed TODO
* simpler
* disable reset btn when already pristine
* revert unnecessary changes to addDataSource
* QUE-1370: Convert AccordionList to TypeScript (#59463)
* Convert AccordionList components to typescript
* Fix types for Select
* Fix types for FilterColumnPicker
* Fix types for QueryColumnPicker
* Fix types for FormSelect
* Fix types for DataSelector
* Fix types for data-picker
* Fix types for AggregationPicker
* Move types into types module
* Simplify searchProp
* Use default export
* Fix info icons always showing
* Remove unused --accordion-list-width
* Rename T for clarity
* Parametrise Section type too
* Fix default label renderer
* Fix missing props
* Avoid calling in on non-object items
* Simplify maxHeight prop
* Move fallbacks into AccordionListCell
* Move shared props to shared type
* Sort props
* Remove styled component for AccordionList
* Rename _list
* Fix itemIsSelected signature
* Simplify callback
* Fix searchFuzzy prop default
* Fix schema picker callback
* Fix aria-selected for tests
* Omit filter description
* Fix loading state
* Remove unused searchFuzzy prop
* Remove unused searchCaseInsensitive prop
* Remove unused hideEmptySectionsInSearch prop
* Reduce this.props usage
* Do not pass aria-expanded to non-expandable items
* Rename AccordionList.unit.spec to tsx
* Remove get and use type guards
* Remove unused cast
* Rename ListItem to Item
* Avoid silly x variable name
* Fix incorrect base type for Item
* Define Item type
* QUE-1359: Add highlight when item has cursor (#59622)
* Add highlight when item has cursor
* Add repro
* Use _.times
* Test using computed style
* Use Cypress._
* QUE-1360: scroll selected item into view (#59617)
* Scroll items into view when selected
* Add e2e test for scrolling items into view
* QUE-1361: Allow using table in search and add fuzzy search (#59693)
* Flatten getRowsCached into getRows
* Use method directly in checkSectionHasItemsMatchingSearch
* Build search index when props change
* Memoize the fuse index
* Add fuzzySearch props to AccordionList
* Enable fuzzy search in Filter and Aggregation picker
* Add unit tests for fuzzy search
* Add fallback for searchProp
* Remove unused variable
* Use shallow equality for memoization
* Remove this.state
* Add tableName and tableDisplayName to filter picker search props
* Strictly type searchProp
* Fix type of default search prop
* Do not include results with bad score
* Search by custom property
* Avoid O(n²) search
* Lower the score
* Fix types in test
* Memoize search
* Pull out isSearching
* Sort results by score
* Export SearchProp types
* Coerce default search props
* Use SearchProp types in Select
* Move itemScore and sectionScore to utils
* Remove search state from state
* Simplify initial cursor method
* Fix cursor logic for sorted items
* Add comment on why a custom memoize function is needed
* Fix unit tests
* Add forwardRef back to AccordionListCell
* Make ref dependency explicit
* Fix broken merge
* Correctly move cursor through rows
* Move searchFilter into getRows
* Fix missing rows
* Lower the search threshold
* Always show section when it has a type
* Move searchThreshold to constant
* Use includes
* Remove unused type fields
* Rename index.js to index.ts
* Refactor search
* Fix types
* Simplify ItemScores
* Allow customizing threshold in strategy
* Enable fuzzy search for query column picker
* Fix substring search
* QUE-1363: Custom expression fallback (#59992)
* Add clausesForMode helper
* Add custom expressions to filter picker
* Do not require helpText for snippets
* Remove helpText arguments from expression completions
* Export expressionClauseSnippet helper
* Allow opening expression editor with custom clause snippet
* Pass expression clause to custom expression editor
* Fix types
* Add onSearchTextChange to Accordionlist
* Select expression clause when typing (
* Only show expression clauses while searching
* Always sort expression clauses last
* Remove reportTimezone arg where no longer needed
* Add expression functions to aggregation picker
* Import underscore as _
* Rename filter
* QUE-1414: Fuzzy search tests (#60257)
* Add test for fuzzy matching in filter popover
* Add tests for picking custom expression in the filter picker
* Add test for selecting expression in aggregation picker
* Add test for selecting items in AccordionList while searching
* Verify that typing non-existing clause does nothing
* Typo
Co-authored-by: Kamil Mielnik <kamil@kamilmielnik.com>
---------
Co-authored-by: Kamil Mielnik <kamil@kamilmielnik.com>
* QUE-1468: Sticky search in AccordionList (#60190)
* Make the search bar floating
* Remove overflow override
* Correctly set width
* Pass width as direct prop
* Use sticky modifier
* Make search sticky in virtualized list
* Add scroll padding for search bar
* Do not scroll searchbar into view
* Use scrollIntoView vs scrollTo
* Pass maxHeight to scroll container directly
* Remove scrollTo in tests
* Use correct scrolling container
* Remove duplicate sticky style
* Unwrap accordionpicker
* QUE-1413: Colum completion prefix (#59921)
* Allow searching by table name in custom expression completions
* More strictly filter results
* Fix unit tests
* Also disable filter on functions
* Fix unit tests
* Add lower weight to displayLabelWithTable
* Fix e2e tests
* Fix e2e test
* Use display value from longest match
* Copy indices to avoid readonly error
* Fix e2e test
* Preserve option identity
* Use first match to render
* Undo e2e test changes
* Fix e2e test
* Allow matching columns names via long display name too
* Use displayLabel
* Increase score threshold
* Fix unit tests
* Remove missing key error
* Memoize fuzzyMatcher
* Fix e2e test
* QUE-1518: join picker custom search (#60493)
* Allow searching by table name in QueryColumnPicker
* Fix scroll on QueryColumnPicker
* Add custom expressions to join column picker
* Allow selecting expression with ( in QueryColumnPicker
---------
Co-authored-by: Kamil Mielnik <kamil@kamilmielnik.com>
* fix: optionally add can_run_adhoc_query in items
Allow the can_run_adhoc_query parameter to be optionally included in the
collection items response, so that targets that are not valid for ad hoc
queries can be removed from the picker in the UI.
* split up large test
* Do not show error when snippet is active
* Add test for hiding errors while snippet is visible
* Also suppress error underline when snippet is active
* vertical cartesian well -> ok
* put the logic in a hook and handled hortizontal well
* added unit tests and handled other types of wells
* reorg and style
* css tweak
* Closes VIZ-1244 Wells light up upon dragging on them, even if the column won't map
* no outline
* Closes VIZ-1216 bubble size well style (#60380)
* fixes wrong drag overlay
* much better fix for draggig overlay
* tpyo
* removed useless cast
* removed comment
* refactored test
* Closes VIZ-1208: smarter detection of selected columns
* no breakout when more than one metric
* no breakdown when more than 1 metric if new col is not date
* combo _is_ cartesian
* [m.lib.fe-util] Introduce match-lite to reduce CLJS bundle size
* [m.lib.filter] Replace match-one with match-lite
* [m.lib.remove-replace] Replace match-one with match-lite
* Replace match-one with match-lite on all CLJC files
* Introduce the option to add data from the sidebar via "Add data" modal (#58408)
* Add CSV panel to the "Add data" modal (#59008)
* Add Metabase storage upsell to upload CSV panel (#59901)
* Add unit test coverage for the missing collection utils (#59902)
* Consolidate analytics around databases, CSV uploads and Google Sheets (#59996)
* Add Google Sheets management tab to the _Add data_ modal (#59905)
* remove stray `t2/debug`
* fix flaky notification send test
another thread was responsible for popping off the queue so there was a
race condition here.
* fix flaky test for not calling DB
i feel bad about this but this seems to be a reasonable solution given
the situation.
Any endpoint that gets some setting inside it might emit an extra SQL
query or two. So we can't reliably check to see whether hitting a given
endpoint hits the DB.
What we can do is hit it a few times and make sure one of those times
didn't hit the DB. Which is silly... but yeah.
Tests fail in CI which try to set timezones in mysql to "alias" types:
'US/Pacific', etc
```
❯ grep "Failed to set timezone" test-log_formatted.json | jq -r '.message'
Failed to set timezone 'US/Pacific' for :mysql database
Failed to set timezone 'US/Pacific' for :mysql database
Failed to set timezone 'US/Pacific' for :mysql database
Failed to set timezone 'US/Eastern' for :mysql database
Failed to set timezone 'US/Hawaii' for :mysql database
Failed to set timezone 'US/Pacific' for :mysql database
Failed to set timezone 'US/Pacific' for :mysql database
Failed to set timezone 'US/Eastern' for :mysql database
```
on mysql image from circle CI:
```
mysql> select now();
+---------------------+
| now() |
+---------------------+
| 2025-07-03 02:00:48 |
+---------------------+
1 row in set (0.01 sec)
mysql> SET @@session.time_zone = "US/Pacific";
ERROR 1298 (HY000): Unknown or incorrect time zone: 'US/Pacific'
mysql> select now();
+---------------------+
| now() |
+---------------------+
| 2025-07-03 02:10:25 |
+---------------------+
1 row in set (0.01 sec)
```
on mysql's official image for 8.0:
```
mysql> select now();
+---------------------+
| now() |
+---------------------+
| 2025-07-03 13:51:36 |
+---------------------+
1 row in set (0.00 sec)
mysql> SET @@session.time_zone = "US/Pacific";
Query OK, 0 rows affected (0.00 sec)
mysql> select now();
+---------------------+
| now() |
+---------------------+
| 2025-07-03 06:51:41 |
+---------------------+
1 row in set (0.01 sec)
```
This happened because circle ci bumped the base image to ubuntu 22 in
511bd63787
and this has different timezone information.
Here's timezone data in their 8.0.32 image
```
ls -l /usr/share/zoneinfo/US/Pacific
lrwxrwxrwx 1 root root 22 Feb 14 21:25 /usr/share/zoneinfo/US/Pacific -> ../America/Los_Angeles
```
and in 8.0.42 (alias to latest 8.0 at time of writing)
```
ls -l /usr/share/zoneinfo/US/Pacific
/usr/bin/ls: cannot access '/usr/share/zoneinfo/US/Pacific': No such file or directory
```
Mysql's official images still have these legacy timezone aliases
The mu/fn macro produces instrumented code, whose tail catches java.lang.Exception to clean up their stacktraces. This is obviously not possible in clojurescript. This adjusts the code to detect whether or not it's compiling for Clojurescript or Clojure and tweak the expansion appropriately.
* Typo in name
* Test an implement iso and simple mode strings
* Implement simple coercion strategy for sqlite
* Implement simple coercion for clickhouse
* Implement simple coercion for vertica
* Try wrapping strings with h2x/literal
* Fix athena
* Fix Vertica concatenation
* Try other clickhouse fix
* Let's try to fix hive-like
* Typo in clickhouse!
* Fix vertica
* Smarter parsing.
* Oops extra paren
* use parseDateTime() for clickhouse
* Add snowflake.
* Fix insane Snowflake test.
* Add databricks sqlite and vertica to test
* Tests, yo
* We need all sql+mongo to support bytes
* Remove redundant limit
* Implement bytes->temporal for hive-like
* Add more dbs to bytes->temporal test
* WIP
* Update frontend schema
* Remove printlns because it's poluting the test logs
* Presto uses LocalDateTime
* Starburst uses LocalDateTime
* Use zoned tiem for sparksql
* Snowflake binary dataset type
* Sqlite BLOB datatype
* SQL Server binary type
* Add vertica binary type
* SparkSQL binary type
* BigQuery binary type
* Capitalize clickhouse binary type
* Try a different type for SparkSQL
* Implement temporal bytes coercion for sqlite
* Implement temporal bytes coercion in sql server
* Add Sqlite binary dates expectations
* Change date type for sparksql expectation for simple datestring
* Implement temporal bytes coercion in vertica
* Implement temporal bytes coercion for Oracle
* MongoDB temporal bytes coercion
* Vertica uses cast as VARCHAR, not to_char()
* Oracle binary date expected return type
* Cut off multiple databases that can't convert temporal bytes to datetimes
* kick ci
* Allow all databases except athena
* Save some code for later
* Athena temporal string coercion (simple)
* Better comment
* Clean up tests
* Use more dispatch values and implement bytes tests
* Implement isobytes coercion strategy
* Test that drivers without binary temporal coercion throw exceptions
* Fix expectation for h2 and postgres
* Implement iso bytes in mongo
* Implement iso bytes in Oracle
* Implement binary iso coercion for hive-like
* Update expected return from h2 and postgres
* Implement binary iso coercion for Starburst
* Implement redshift and presto binary iso coercion
* Remove hive-like implementation for bytes coercion
* Rename bytes-iso and bytes-simple to isobytes and simplebytes
* Implement mode dispatch for datetime
* Fix Oracle typo
* Implement sqlite binary coercion
* Update frontend text.
* Update enum
* Normalize strings to keywords in legacy
* Clean up
* Clean up databricks bytes ingestion.
* Clean up more
* Improve syntax for valid modes check
Co-authored-by: Romeo Van Snick <romeo@romeovansnick.be>
* Improve text
* Add fe tests for mode strings
* Complete coercion strategy code
* Avoid directly referencing driver names
* REmove unused mode string + correct validation logic
* Better feature keywords
* Frontend test for binary cast
* Appease the Kondo
* Improve fe test
* Build new ad hoc question with native question as source
* Fix types
* Fix test description
---------
Co-authored-by: Romeo Van Snick <romeo@romeovansnick.be>
* Remove refresh, fullscreen, and by extension, the nightmode toggle from SDK
* remove SDK_DASHBOARD_VIEW_ACTIONS
* Refactor tests and stories to be a little easier to debug something like this, and use explicit keys for the dashboard actions
* Fix up a thing
* Fix unit tests
---------
Co-authored-by: Mahatthana (Kelvin) Nomsawadi <me@bboykelvin.dev>
* wip
* save progress
* TEMP COMMIT
* go another direction
* add tests, remove debugging colors
* clean up
* add a few more unit tests
* try rounding down
* fix scrollbar flickering for some users, cancel pending animation frames on unmount
* implement updated styling, refactor out error messages from message state to better control error formatting, refactor out thinking and message components from MetabotChat
* finish things up
* unit tests for tool call logic
* add missing test around message clearing after error
* fix copy in e2e test
* Create UpsellDevInstances for Embeddings pages
* Add dev instance upsell to general settings
* Add dev instancles banner
* Hide upsell in in dev mode
* Add StorePaths type
* Update links for dev instances banner
* Update store link for upsell dev instance
* Refactor DevInstanceBanner to use translation function directly for text and external links
* Add unit tests for DevInstanceBanner component
* Refactor imports for UpsellDevInstances across embedding settings components
* Refactor DevInstanceBanner to use a reusable BannerBody
* Replace ExternalLink with Anchor in DevInstanceBanner
* Add jsdoc to getStoreUrl function
* Add target blank to store url
* links and link fixes
* Update docs/dashboards/subscriptions.md
Co-authored-by: Alex Yarosh <alexandra@metabase.com>
---------
Co-authored-by: Alex Yarosh <alexandra@metabase.com>
* update deps
* initial date migration
* default portal reuse target node to false
* remove timezone from MonthYearPicker value
* date picker adjustment for timezone
* unit tests
* fix Quarter Picker Types
* re-apply patch, fix up unit tests
* last adjustments?
* `annotate` middleware => MLv2
* Test fixes
* Fix#42307
* Add `lib.join/rename-join`
* Continue to iterate on `annotate` rewrite
* Squash #58712
* Less test noise
* Revert "Squash #58712"
This reverts commit 9c8d170aa4.
* Only one test failure to go!
* Add more documentation; code cleanup
* Improved column looping code.
* Test fix 🔧
* Code cleanup
* Use existing Join renaming code.
* FIX QUE-1330
* Fix#30397 / QUE-147
* Kondo cleanup
* Test fixes.
* 13384 assertions, 81 failures, 47 errors.
* 13373 assertions, 48 failures, 28 errors.
* 13400 assertions, 48 failures, 15 errors.
* 13391 assertions, 48 failures, 12 errors.
* 13401 assertions, 22 failures, 11 errors.
* 13404 assertions, 20 failures, 6 errors.
* 13407 assertions, 13 failures, 6 errors.
* 13414 assertions, 11 failures, 8 errors.
* 13412 assertions, 10 failures, 8 errors.
* Minor code cleanup
* Minor code cleanup
* 13412 assertions, 10 failures, 5 errors.
* Fix Kondo errors
* 13411 assertions, 10 failures, 5 errors.
* 13404 assertions, 7 failures, 5 errors.
* 13411 assertions, 3 failures, 5 errors.
* 13399 assertions, 2 failures, 5 errors.
* 13396 assertions, 2 failures, 4 errors.
* 13424 assertions, 2 failures, 2 errors.
* Test cleanup
* Fix deps graph
* Update lib module config
* Cleanup
* Kondo cleanup
* Minimize diff
* 13421 assertions, 3 failures, 2 errors.
* Nice refactor of big hairball test
* More beautiful refactoring
* 13474 assertions, 0 failures, 1 errors.
* Fix linter error
* Remove unused `:aggregation_index` column
* Test fixes
* Test fixes 🔧
* 14547 assertions, 1 failure, 1 error.
* Fix calling the zero arity of unique-name-generator twice
* Handful of test fixes 🔧
* Fix :converted-timezone for :convert-timezone with 3 args instead of 4
* Fix metabase.queries.api.card-test/renamed-column-names-are-applied-to-json-test
* Split up mega test
* Split up another large test
* Fix error message
* Resolve#39887. 14536 assertions, 1 failure, 0 errors.
* WOW! Is everything green now??
* Test fix 🔧
* Update issue IDs
* Update more issue IDs
* PR feedback
* A test fix 🔧
* Another test fix 🔧
* More test fixes 🔧
* Formatting
* Test fixes 🔧
* Test fixes 🔧
* More test fixes 🔧
* Less nonsense
* 6618 assertions, 115 failures, 16 errors. [Lib]
* 6637 assertions, 3 failures, 16 errors.
* 8208 assertions, 3 failures, 0 errors. [Lib]
* Almost ready??
* [Lib] 8216 assertions, 0 failures, 0 errors.
* [QP] 5330 assertions, 21 failures, 31 errors.
* [QP] 5344 assertions, 8 failures, 2 errors
* Fix QUE-1380
* Fix QUE-1410
* [QP] 5344 assertions, 8 failures, 1 errors.
* 5345 assertions, 2 failures, 1 error.
* Proper deduplication in field resolution; also, add some normalization tests
* 13563 assertions, 1 failure, 0 errors.
* 13579 assertions, 0 failures, 0 errors.
* Kondo & cljfmt
* Kondo
* Test fixes 🔧
* Disallow duplicate expression names (QUE-1412)
* Test fixes 🔧
* Test fix
* Fix QUE-1418
* Fix error messages about inline-value with MongoDB
* Fix Redshift test failure
* Fix Athena test
* Fix BigQuery tests
* Fix dumb test failure
* Card source-model-cols should work for Cards without a query.
* Fix MongoDB
* Fix Druid
* Fix some busted code formatting.
* Fix column names
* Fix column names
* Do type inference for native queries with MBQL stages if the driver returns all :type/*
* Fix column names
* Fix column names
* Fix column names
* Druid test fixes 🔧
* More test fixes 🔧
* Update more FE tests
* MBQL 5 map normalization should not normalize nil values
* Support new `:decode/mock` key for mock metadata provider
* Split up unrelated large test
* Don't add :was-binned false to refs
* Clean up the display name calculation logic for column metadata
* Update binning middleware to use 'better' column resolution logic in lib.field.resolution
* Don't calculate different display names in QP results metadata
* Properly convert join :source-metadata
* Update expected display name
* Revert making test too complicated
* Simplify the remove-inactive-field-refs middleware
* Misc code cleanup and test fixes 🔧
* Appease whitespace linter
* Add test to make sure Lib result metadata merges in :info :metadata/model-metadata
* Literally an insane number of new tests
* Add `:lib/original-display-name` to column metadata schema
* Improve error message for fix-bad-references when you use Field ID ref with native source query
* Improved field resolution logic
* Card should NOT add :lib/original-display-name all willy-nilly
* Add `:lib/source-display-name` and `:lib/ref-display-name`
* FINALLY fix display name logic??
* Revert making things private (thanks for lying to me LSP)
* Test fixes 🔧
* Remove `:lib/hide-bin-bucket?`
* Test fix
* Update notebook helpers
* Always use LONG display names for columns from the previous stage.
* Fix metabase.query-processor-test.nested-queries-test/space-names-question-test
* Make sure we use correct display names for implicitly joined columns
* Make sure BINNING gets propagated with `:lib/original-binning`.
* Make sure we correctly calculate if a column is inherited or not.
* Make sure binning info is returned in QP results and other misc fixes
* Do NOT return join alias info for columns that come from implicit joins.
* Update tests
* Some test fixes
* Don't generate wacky default join aliases
* Test fixes 🔧
* Fix more tests
* Prettier
* Fixes
* Fixes
* Fixes
* Fixes
* Fixes
* Fixes
* Fixes
* Fixes
* Fixes
* Fixes
* Fixes
* Fixes
* Fixes
* Fixes
* Fixes
* Fixes
* Fixes
* Fixes
* Use 'Unknown Field' as display name for unknown fields
* Fix incorrect use of expression refs
* Mark selected columns correctly
* cljfmt
* Update/fix backend tests again
* More fix selection fixes 🔧
* Fix equality matching for join columns
* Update tests
* Fix Kondo
* Fix line-bar-tooltips.cy.spec.js
* Add transducer version of u/index-by
* Some PR feedback
* Fix joins-reproductions.cy.spec.js 🔧
* Kondo cleanup
* Fix multiple-column-breakouts.cy.spec.ts
* Update tests
* Update tests not to expect :ident
* Update test not to expect duplicate columns
* PR feedback.
* More test fixes.
* IDK, maybe this is the last round of fixes. 🔧
* Support normalizing parameter dimensions
* Test fixes 🔧
* Fix CLJS code
* One last fix for column equality
* Update a couple of e2e tests
* Another test update
* Fix the test
* remove duplicated spec
* Even more tests (1 million tests)
* cljfmt
* Add test to make sure breakoutable-columns does not include remaps
* Fix Kondo
* skip failed tests
---------
Co-authored-by: Alexander Polyankin <alexander.polyankin@metabase.com>
* Enable the :expression-literals feature for the mongo driver
* Wrap :value expressions in $literal when compiling for mongo
* Add mongo compile-cond and compile-filter impls for :expression, :field, and :value
* Better codegen of compile-filter and compile-cond for :expression, :field, and :value on mongo
* Review feedback: hash-map -> array-map
* dev: bigquery static datasets
* Check every two days
* Fix var
* Fixes for sql params and test fixes
* Fix tests
* Fix tests
* Add logging for test debug
* Fix time range
* Fix default dataset check
* Fix dataset name replace
* Ignore exceptions on tracking in case of could not serialize error
* Fix kondo
* More kondo fixes
* Update translation sources 2025-06-26
* Update translations
* comment out locale with wrong format
* let's try to just rename it?
* et
* dont import estonian (yet)
---------
Co-authored-by: github-actions <github-actions@github.com>
Co-authored-by: Nicolò Pretto <info@npretto.com>
Co-authored-by: Ryan Laurie <iethree@gmail.com>
* Adjust License Token setup step
# Conflicts:
# frontend/src/metabase/setup/components/LicenseTokenStep/LicenseTokenForm.tsx
* Add License Token Step tests
* Accessbility improvements
* Use button for skip
* Fix setup flow tests
* Fix e2e setup tests
* Use existing store url and unstyled button
* fix selector usage
* Make the token error test more specific
* Simplify the full width divider styling
* Simplify Popover styles
* Fix the store link
* Use plain Button for skip
* Follow the setup pattern in tests
* Improve copy text
* Provide translation context
* Use HoverCard component instead of Popover
* Don't display license token hover card by default
* fix: collection archive/move w/out parent perms
Allow users to archive and move collections even when they do not have
read-write permissions on the parent collection.
* fixup! docstring feedback
Fixes#59778
Previously this was always rounded to `:day` for a single absolute
datetime filter. (Absolute date ranges already respected the times if
provided.)
* inc perms revision number on collection creation
also delete a misleading comment about data permissions.
The `before` and `after` fields here will be blanked out. This is
unfortunate, but this is a strictly better state of affairs from before,
when the change to permissions would not be recorded at all. At least
now we will record that it happened, and concurrent changes to the
permissions graph will fail instead of succeeding with odd/unexpected
behavior.
We'll add a remark to the revision to explain why it was created.
Co-Authored-By: Ed Paget <ed@metabase.com>
* Add tests for new behavior
Co-Authored-By: John Swanson <john.swanson@metabase.com>
---------
Co-authored-by: John Swanson <john.swanson@metabase.com>
* Improve chart context
* Send query
* Send display type
* Refactor existing code to only provide query once and not send display type on the root
* Only fetch metabot context if plugin is enabled
The query processor would explicitly return `nil` when a query was canceled, but then that `nil` would cause an `AssertionError` to be thrown. This is no longer the case.
Discussion metaboat.slack.com/archives/C0669P4AF9N/p1750783492043299
With this change we only run driver tests if:
on master or release-* branches
PRs labeled with .Team/Drivers, .Team/Querying, or .Team/SemanticLayer
Apparently, it is possible to get collection: null in a card's dataset query. It is unclear why this happens, but
this change ensures that nothing will break when it does happen.
In the past, we have run CI, and local testing with two enterprise tokens:
- all features
- no features
And this really doesn't reflect most real-world scenarios very well. We should be testing with tokens that have actual sets of features that are used in the wild, namely:
- starter cloud
- pro cloud
- pro self-hosted
Some bleeding-edge features that have not yet been implemented into our commercial plans will need to be tested with an all features token still.
This commit updates our testing helpers to use a paradigm that more accurately reflects real-world usage scenarios. Instead of setting specific features for a test, you activate a specific token.
* add BE code
* port over most of code from branch metabot/streaming w/o tests + making changes for supporting both normal requests and streamed ones
* improve test coverage, bug fixes, add support for analyze chart, error handling changes
* small bit of clean up + add unit tests for streaming
* fix some subtle bugs around request cancellation and stream processing
* Remove debugging logs
* fix be lints and things
* Improve ai service response streaming code
- Wrap initialization of response-lines reader with with-open to automatically close it after the response is done
- Remove Thread/sleep code
* Update modules config to register usage of the server module
* Remove request body destructure in the agent-streaming endpoint
* Fix reflection error
* Expand the docstring for the streaming-request function
* sort namespaces
---------
Co-authored-by: Mihael Konjevic <konjevic@gmail.com>
* Add migration to add metabase_field_user_settings table
* Add FieldUserSettings model, handle updates to Field transparently
* Add test
* TODO
* we can't user after-update, as it doesn't reflect the proper model
write through
* respect user schema during fk sync
* test via put endpoint
* comments
* fix
* FieldUserSetting serialisation test
* fk sync test
* simplify
* PoC with realizing set of fks
* add test case
* wip: using tmp table
* unique tbl
* update test
* fix for h2
* table-by-table
* style
* fix test
* stronger test
* fix for mysql
* Add migration to add metabase_field_user_settings table
* Add FieldUserSettings model, handle updates to Field transparently
* Add test
* TODO
* we can't user after-update, as it doesn't reflect the proper model
write through
* respect user schema during fk sync
* test via put endpoint
* comments
* fix
* FieldUserSetting serialisation test
* fk sync test
* simplify
* fix: clean user-set coercion-strategy/semantic-type on base-type update
* add extra test
* VIZ-1114 Updates cartesian graph lines so they're subtler in fullscreen night mode in a way that also allows them to be defined separately from the border color (if desired) in the embedding sdk
* Update Loki Snapshots
---------
Co-authored-by: Metabase Automation <github-automation@metabase.com>
? is used as a jdbc parameter placeholder and is also an operator in postgres. If you want to use it as an operator,
you often need to escape it with ??. This gives a better error message when that happens.
* Add blue dot attention notice to events icon links
* Fix Collection Header tests
* Adjust offset of inidicators to bring them closer to icons
* Improve test by with better API usage
Fixes#59394.
If the card we're trying to use for parameters values has an explicit `:fields` clause, that was surviving into
the final query, which causes us to generate bad SQL and throw a 500. This discards the `:fields`, which aren't
relevant when we're overwriting the query's last stage with a single breakout.
* This PR addresses design feedback from @kdoh
* Show toasts on dictionary upload/download success and failure
* Move localizing user-generated content to its own settings section
* tests
---------
Co-authored-by: Nick Fitzpatrick <nickfitz.582@gmail.com>
* Lift onFormatQuery to NativeQueryEditor
* Pass onFormatQuery to codemirror editor
* Lift canFormatQuery to NativeEditor
* Add format expression keymap to custom expression editor
* Add shortcut documentation
* Add helper to get NativeEditor value
* Add test for formatting using shortcut in native editor
* Add test for formatting in the custom expression editor via shortcut
* Do not allow formatting invalid expressions by shortcut
* Add test for invalid expression
* Wait for format button to be visible
* REmove unused var
* Perform the realPress on the cm editor
* Change order of keyboard event handling so that Ctrl+F handler does not kick in when pressing Ctrl+Shift+F
* Replace search keymap for native editor
* Add CodeMirror wrapper
* Use wrapped CodeMirror in NativeQueryEditor
* Use wrapped CodeMirror in custom expression editor
* Use wrapped CodeMirror in CodeEditor
* Export more from CodeMirror module
* Use highest precedence for shortcuts
* Add support for highlighted ranges to common CodeMirror editor
* Use highlightRanges prop
* Simplify code editor
* Use highlightRanges prop in Native editor
* Add highlight class
* Move shared codemirror style to CodeMirror module
* Move tag out of :global block
* Add missing import
* Fix background color
* Simplify highlights deps
* Move autofocus, autocorrect and tabIndex plugin to shared CodeMirror editor
* Disable autocorrect in SQL editor
* Memoize basicSetup
* Add snippet shortcuts
* Remove unnecessary lowercase
* Use type guard
* Addd types file
* Rename util to utils
* Destructure extension from props
---------
Co-authored-by: Kamil Mielnik <kamil@kamilmielnik.com>
This fixes a couple small typos and makes a few other changes while I was at it. I made the little math expressions consistent, and changed the text under "How…" since it only talks about count at first.
* test db routing on dbs which support it
* remoev an extra newline
* remove hardcoded clickhouse addition
* Remove database routing support from a few more drivers
Some of these are artifacts of no data-loading support in tests and they
can be reenabled when that works:
- databricks: multi-catalog so similar to clickhouse
- presto: data loading
- spark: multi-catalog so similar to clickhouse
- vertica: dataloading
* Turn off in athena, bigquery, oracle, redshift
Change dataset name since "table" is reserved in bigquery
- athena: data loading issue. Can't diagnose yet if this is similar to
other cloud databases.
- bigquery: includes database in query in tests. Seems not in real life.
```sql
SELECT `v4_router_data__transient_1750437202857.t`.`id` AS `id`,
`v4_router_data__transient_1750437202857.t`.`f` AS `f`
FROM `v4_router_data__transient_1750437202857.t`
LIMIT 1048575
```
- oracle: unknown. i suspect a data loading issue
- redshift: includes database in query. But i think
this might be a testing artifact: we aren't creating a database each,
but a schema each and faking it. Which violates the expectation of
db-routing. So this is a testing artifact
```sql
SELECT
"schema_173"."airports_airport"."id" AS "id",
"schema_173"."airports_airport"."name" AS "name",
"schema_173"."airports_airport"."code" AS "code",
"schema_173"."airports_airport"."latitude" AS "latitude",
"schema_173"."airports_airport"."longitude" AS "longitude",
"schema_173"."airports_airport"."municipality-id" AS "municipality-id"
FROM
"schema_173"."airports_airport"
LIMIT
1048575
```
* disable snowflake
includes databases in its queries:
```clojure
(defn- query-db-name []
;; the store is always initialized when running QP queries; for some stuff like the test extensions DDL statements
;; it won't be, *but* they should already be qualified by database name anyway
(when (driver-api/initialized?)
(db-name (driver-api/database (driver-api/metadata-provider)))))
;; unless we're currently using a table alias, we need to prepend Table and Field identifiers with the DB name for the
;; query
;;
;; Table & Field identifiers (usually) need to be qualified with the current database name; this needs to be part of the
;; table e.g.
;;
;; "table"."field" -> "database"."table"."field"
;; This takes care of Table identifiers. We handle Field identifiers in the [[sql.qp/->honeysql]] method for `[:sql
;; :field]` below.
(defn- qualify-identifier [[_identifier identifier-type components, :as identifier]]
{:pre [(h2x/identifier? identifier)]}
(apply h2x/identifier identifier-type #_(query-db-name) components))
```
* actually disable for snowflake
* VIZ-1099 Increases font-size of dash card titles
* VIZ-1099 Adds additional padding above card titles per design feedback
* VIZ-1107 Updates dashboard tabs to more closely match our standard tabs component
* VIZ-1107 Makes contrast less jarring for dashboard tabs in night mode while preserving embedding sdk customizability
* Update Loki Snapshots
---------
Co-authored-by: Metabase Automation <github-automation@metabase.com>
* feat: Add debugger API for card permissions
* fixup! initial set of PR feedback fixes
* fixup! add more information to data
* fixup! simplify queries
better arg names and docstring
* fixup! whitespace lint
Clickhouse's databases are more like schemas, and Database Routing is
not working correctly.
For now, let's disable Database Routing for Clickhouse. In a followup PR
I'll write tests for other drivers to make sure DB routing is working
for them, if not we can disable them as well.
Updates the secrets references to the
- "all features" token
- "no features" token
They are now using newly generated tokens. Respectively:
- `MB_ALL_FEATURES_TOKEN`
- `MB_STARTER_CLOUD_TOKEN`
* chore: all permissions module access through core
* fixup! break circular dep
* fixup! some other broken kondos
* fixup! revert collections changes
* fixup! remove dynamic var from public api
This macro does not do anything any longer the dynamic var it set is
only readed through the `update-card!` path and it was only used in a
path that called `t2/update!` instead.
Fixes#59441.
Previously this was matching against the `returned-columns` of the
previous stage, but that's incorrect. Since we're trying to match stage
`k` refs we must match against stage `k` columns, not stage `k-1`.
* Code cleanup
* Moves query analyzer to metabot-v3 module
* Consolidates SQL query analysis functionality under the metabot-v3 module to reduce cross-module dependencies
* Properly hydrate and pass schema for tables used in the native query for Metabot
* Adds tests
* Do not rely on is_native flag for fetching database schema in Metabot context
* update people nav
* type fix
* use modern upsell
* fix e2e test, cleanup
* wider pages
* update tests
* update unit tests
* update titles and nav
* limit description width
* little updates
* Localize the trash name on memoized collection
* just defer translation on the `trash-collection`
When we're getting it explicitly rather than for all collections - for
the sake of speed here, since this was causing some issues with
serialization.
* parallelize
* docstring
* VIZ-162 Updates percent calculation for stacked bar charts’ tooltips for bars with a mix of positive and negative values. Positive values are a percentage of the positive total and negative values are a percentage of the negative total.
* VIZ-162 Updates e2e tests to verify new stacked bar chart behavior
shorten filename to not overflow byte limit in some filesystems
in particular ext4 limits file name by 255 *bytes*, not chars, which is doable by using 3-byte chars
fixes#59233
* set embedding-homepage visible as soon as we can
* cleanup test
* reorder because it makes more sense
* Update frontend/src/metabase/setup/components/EmbeddingSetup/steps/UserCreationStep.tsx
Co-authored-by: Phoomparin Mano <poom@metabase.com>
---------
Co-authored-by: Phoomparin Mano <poom@metabase.com>
* Fix can't download static SQL question\
when a parameter only expects a single value
* Ignore `?locale` search param in result downloads
* Finish the comment
* Fix public action tests failing
* Address review: Call side-effect explicitly
* Preserve same behavior
* Flip boolean to make it easier to read
* Improve boolean condition
* revert the use of `isEmbeddingSdk` env until #59602 is fixed
* Add missing condition to retain the same behavior
* Fix more failing tests duh!
* Fix should sync logic for real this time.
* Update upgrading-metabase.md
Adding a note about upgrading between major versions
* Update upgrading-metabase.md
* Apply suggestions from code review
Co-authored-by: Jeff Bruemmer <jeff.bruemmer@gmail.com>
---------
Co-authored-by: Jeff Bruemmer <jeff.bruemmer@gmail.com>
* Add embedding setup flow with multi-step wizard. Add EmbeddingSetup component with welcome, data connection, processing, and final steps. Create setupEmbeddingSettings action for configuring embedding and JWT. Add CodeSnippet component for displaying code examples. Set up proper routing for the embedding setup flow.
* Update embedding setup flow to actually create the database and handle errors. Update DataConnectionStep to display error messages.
* Fix linter errors in EmbeddingSetup component and setupEmbeddingSettings action
* Setup embedding flow: Remove SchemaSelectionStep, fix imports and types
* Fix linter errors and finalize embedding setup flow with navigation and original content restored
* Add EmbeddingSetupWrapper to provide context for embedding setup flow
* Fix table fetching in TableSelectionStep to use metadata endpoint
* Implement model and dashboard creation in ProcessingStep using /api/card endpoint
* Fix useEffect and useCallback dependencies and import order in TableSelectionStep
* working flow
* small tweaks
* sidebar with steps
* update snippet to use iframe embed
* better "Need assistance" positioning
* fix and refactor database and tables steps
* style final step
* bottom part of /final
* done step and idk i'm tired
* remove chevron and add syntax highlight as per feedback on the loom
* fix user not being loaded
* welcome step and final step
* remove dead code
* add query builder
* more stuff
* add comment about keepMounted
* set the proper settings during setup, fixes EMB-462 and EMB-461
* hide DatabaseStatus, fixes EMB-458
* "do later" -> link to /, closes EMB-456
* attempt at setting a max width, maybe closes EMB-454
* fix icons
* fix scrollbar on tall pages
* to refactor: wip link between setup and onboarding
* fix jumpy layout while moving across pages
* fix DatabaseStatus showing up after landing to the homepage
* Revert "to refactor: wip link between setup and onboarding"
This reverts commit b4c19da37d.
* wip user step
* revert changes to database status, we don't need those changes now that we're not in the routes that use the App component
* prevent going to setup/embedding if setup is already done
* table search
* restore original position of metabot to keep diff cleaner
* category + icon on search input
* restore routes structures
* add widht and height, fixes EMB-480
* remove "Need assistance" link, closes EMB-479
* change content in DoneStep, closes EMB-478
* removed /embed from the url
* empty set for search + layout adjustements, closes EMB-481
* removing things i commited by mistake
* locale selector, closes EMB-466
* cleanup unused code
* fix loading not being centered in table selection loading
* fix processingStatus i18n issue
* Remove undefined CSS custom property
* Revert "fix loading not being centered in table selection loading"
This reverts commit 8fda19338a.
* re-do the centering
* close EMB-473, refactor to be only one page
* move CodeSnippet in the file where it's used, it's not a generic
component for now
* Add a basic embedding onboarding E2E test (#59323)
* Add a basic E2E test
* Add missing import
* Finish assertions
* Wait for DB sync
* Hide DB sync status, so Cypress could click a button
* Fix test on CI
* Fix E2E after upstream change
* Some adjustment to make CI passes
* Wait for loader to disappear to avoid failing CI
* Simulate being on cloud
* Addresses review feedback
* fix sidebar steps, titles and visibility
* fix another wrong string
* Update frontend/src/metabase/setup/components/EmbeddingSetup/EmbeddingSetupSidebar.tsx
Co-authored-by: Mahatthana (Kelvin) Nomsawadi <me@bboykelvin.dev>
* address pr feedbback
* fix error ui in table selection
* add useForceLocaleRefresh to table step
* update label on test too
* more label updates
* Properly refetch database API
* Add DB sync retries assertion
* fix the retry of the database sync
---------
Co-authored-by: Kyle Doherty <5248953+kdoh@users.noreply.github.com>
Co-authored-by: Mahatthana (Kelvin) Nomsawadi <me@bboykelvin.dev>
* add regression tests
* Add issue references
* make double comment test driver agnostic
* make tests more agnostic
* format
* make another test agnostic
* add prometheus metrics
* add driver query metrics test
* test against failed count
* move clickhouse native tests back, remove parallel from metrics query test
* fix kondo
* add todo attribution
* disable 0.8.4 broken tests, re-enable 0.8.6 broken tests
* Manually close popover on escape
* Add repro for metabase#55673
* Revert "Manually close popover on escape"
This reverts commit cce85f0882.
* Trap focus in click actions popover
* Remove unneeded closeOnEscape
* Add missing :ident to :model/products-and-reviews mock card
* Add tests for implicitly-joinable visible-columns
* Add more context and cross-reference the comment about implicit-self-join-test
* Put keyword first when looking up entries in lib.tu/mock-cards
* process-query supports temporal-unit template tags in native queries (#57295)
The query processor understands temporal-unit template tags and can parse and compile
`{{mb.time_grouping('param', 'table.column')}}` appropriately. This does not include lib or frontend support and this
feature is not yet usable outside of manually making a query and passing it to qp/process-query.
* Update metabase.lib.native to understand mb.time_grouping parameters (#57506)
This adds lib support for time_grouping parameters. To do that, it pulls out the parsing logic from the driver code and pulls it into lib, where both the driver and lib can use it. The frontend still doesn't support these parameters, so while you can technically type in and run a query with a time_grouping parameter, the frontend behaves oddly if you try.
* implement frontend for time grouping in native queries (#57849)
* Update metabase.lib.native to understand mb.time_grouping parameters
* move parse-param tests
* keep picky behavior of old tagfinding
* fix ns alias
* fix ns declaration for cljs
* add lib tests for finding function tags
* remove resurrected file
* fix rebased code
* fix linting
* update doc string
* fix modules
* implement frontend for time grouping in native queries
* fix modules
* fix validation bug
* update sidebar
* fix typecheck
* fix styles
* use correct icon
---------
Co-authored-by: William Zimrin <william@metabase.com>
* implement dashboard mapping to time grouping parameters (#57931)
* Update metabase.lib.native to understand mb.time_grouping parameters
* move parse-param tests
* keep picky behavior of old tagfinding
* fix ns alias
* fix ns declaration for cljs
* add lib tests for finding function tags
* remove resurrected file
* fix rebased code
* fix linting
* update doc string
* fix modules
* implement frontend for time grouping in native queries
* fix modules
* fix validation bug
* update sidebar
* fix typecheck
* fix styles
* implement dashboard mapping to time grouping parameters
* filter columns properly
* fixup
* fixup
* temporal-unit tags use variable targets
* fix backend tests
* use variable
* cleanup
* add handling for required params
* update copy and fix required toggle usage
---------
Co-authored-by: William Zimrin <william@metabase.com>
* tests for native query time grouping (#58091)
* Update metabase.lib.native to understand mb.time_grouping parameters
* move parse-param tests
* keep picky behavior of old tagfinding
* fix ns alias
* fix ns declaration for cljs
* add lib tests for finding function tags
* remove resurrected file
* fix rebased code
* fix linting
* update doc string
* fix modules
* implement frontend for time grouping in native queries
* fix modules
* fix validation bug
* update sidebar
* fix typecheck
* fix styles
* implement dashboard mapping to time grouping parameters
* filter columns properly
* fixup
* fixup
* temporal-unit tags use variable targets
* fix backend tests
* use variable
* cleanup
* add handling for required params
* update copy and fix required toggle usage
* reset default value if time grouping option is excluded from the list of available options
* add tests
* drop not planned test
---------
Co-authored-by: William Zimrin <william@metabase.com>
* add tests for mapping of native query to temporal unit parameter (#58161)
* rework tag editor sidebar and related files to typescript (#58283)
* revert default required logic (#58568)
* revert default required logic
* fix test
* add native query validation of time grouping syntax (#58426)
* add basic fe validation for native queries
* fixup
* add cljs function for validating native queries
* make validate function take a full query and work in cljs
* add backend/cljc tests for validate-native-query
* improved error messages for validate-native-query tests
* use a new dianostics method from lib
* fix clj linting
* update sidebar styling
* fix initial state
* adjust error styling
* fix tests and drop not valid anymore code
* fix linting
* favor first template tag found and update tags when type changes
* validate that temporal unit function parameters aren't empty
* add e2e tests
* convert day/week/month/quarter/year groupings to dates
* fix empty param
* only convert days to dates
* finally actually make date conversion work correctly?
* address review
* adjust tests to the latest changes
---------
Co-authored-by: William Zimrin <william@metabase.com>
* rename test
* update native query time grouping tests to be more comprehensive (#58814)
* Fix time grouping tests (#58878)
* validate time unit parameters and fix bug (#58993)
* Fix and test nil values for native time groupings (#59062)
* add e2e tests for optional usage of time grouping (#59045)
* add e2e tests for optional usage of time grouping
* cleanup
* fixup
* update test
* fix class reference
---------
Co-authored-by: William <william@metabase.com>
* fix: improve effective children performance
on the /items endpoint by reusing the permission CTE
Co-Authored-by: John Swanson <john.swanson@metabase.com>
* fix: remove n+1 hydration call for card children
* fix: remove case/whens in sort statements
* fix: scope search results for modal to collections
Co-Authored-By: Nathan Voxland <nathan@metabase.com>
* fix: function to convert to collection items
* fixup! PR suggestion
Co-Authored-By: Ryan Laurie <ryan@metabase.com>
* fixup! add row-lock fixture for mariadb flake
* add debug for failing test
* fixup! make sure api/bit->boolean is called first
---------
Co-authored-by: John Swanson <john.swanson@metabase.com>
Co-authored-by: Nathan Voxland <nathan@metabase.com>
Co-authored-by: Ryan Laurie <ryan@metabase.com>
* Support type deduction for `case` and `coalesce`
* Revert accidental changes
* Fix CLJS tests
* More misc QP/Lib cleanup
* Test fixes 🔧
* Test fixes 🔧
* Add `:type/HasDate` and `:type/HasTime` to the type hierarchy
* Additional fix 🔧
* Improved type calculation logic
* Fix QUE-1358
* Test fixes 🔧
* Revert "Address review comments from #58408 (#58785)"
This reverts commit ed6e5591e3.
* Revert "Introduce the option to add data from the sidebar via "Add data" modal (#58408)"
This reverts commit de780f53cd.
* Reflect changes from #58405
Fixes#59273.
Fixes#48306.
Swapping two clauses can change their names, eg. two sum aggregations
`sum` and `sum_2`. Since refs in later stages are based on those names,
the refs need updating to track the same column, rather than change
targets.
`lib.remove-replace` already has logic for this when replacing a clause,
it was readily reused here. I added tests for swapping aggregations and
multiple breakouts of the same column.
Note that viz settings (also based on the names) are not updated as part
of this PR. There are separate bugs open for that issue both when
removing a clause and reordering them.
* a bit lazier table->unfold-json-fields
* remove need for dynamic var
* more optimised
* don't leak json parsers
* code style
* better perf
* short circuit earlier in case of a lot of nested fields
* add to changelog
* rename
* Add icon color support to the banner componet and story
* Add token missing banner rules
* Add token missing banner rules
* Add useLicenseTokenMissingBanner tests
* Dismiss license banner in e2e tests by default
* Improve flag naming in LicenseTokenMissingBanner
* Add banner story - non-closable
* Use regular double quotes for simple string
* Remove duplicated CSS property
* Improve docstring for license token missing banner setting
* Fix wrong property name in the test
* Remove empty align property
* Use existing setupUpdateSettingEndpoint mock
* Extract constants out of the function
* Use ready mock helpers in tests
* Refactor mock setup function to conform to existing standards
* Share the dismissal date between all admins
* Use admin settings APIs to interact with dissmisal state
* Improve banner dismissal test
* Use regular settings API to access license token banner dismissal fetch
* fix: set-table/db-permissions! in single txn
Update set-table-permissions! and set-database-permissions! to build
lists of all necessary changes based on the requested permissions
updates and then execute those changes in a single transaction to avoid
possible deadlocks in the recursive calls.
* fixup! incorrect doc string
We can have tables in migrated instances that have two copies of the
audit db tables one with a 'public' schema and one with a 'PUBLIC'
schema. This handles that case.
* FE wip
* more FE progress
* last big pass at UI
* url page param sync, show model name, move new code to new file
* remove pagination in url and add link to new metabot convo
* unit testing
* finish tests
* move prompt result fetching outside of metabot agent hook and into chat sidebar
* fix constant reference
* fix
* Add metabot prompts and wire up AI service
* Return prompt suggestions for cards visible by the user
* pr feedback
* Test metabot-scope
* Add BE tests
* Fix tests and kondo issues
* Fix tests and kondo issues
* Fix tests more
* Fix serialization
* Fix serdes more
* Use MySQL compatible delete syntax
* have FE use new regenerate endpoint instead of delete
* change mocked status code
* fix flaking unit test and fix failing MetabotAdminPage spec from wrong mocked pagination value
* Add prompt regeneration endpoint
* Document prompt regeneration endpoint
* fix test flake
* doh... fix the right test this time
* Use the collection configured int the app-db in answer-sources
---------
Co-authored-by: metamben <103100869+metamben@users.noreply.github.com>
In postgres, json fields are hidden by default, and we just show their children instead. However, mongo fields and
bigquery struct fields work differently -- we default to just showing the parent, even though the ui claims that all
fields are selected. This makes mongo and bigquery work like postgres.
In addition, this fixes a presumably-longstanding bug where mongo had incorrect column names when selecting both a
parent its child. Basically, you can't ask mongo for both a parent and its child. The returned data is just json, so
asking for the parent field includes all of its children. Instead, mongo just errors when you try to make a request
like that. As a workaround, the mongo driver was just requesting and returning the child fields. This meant that it
was returning a different number of fields than expected, which confused annotate.
Now, the mongo query processor only selects for parent fields, because as mentioned, parent fields include their
children. However, it also keeps track of all the fields we asked for in :projections, so when we get data back, we
know to pull all of those fields out of the returned data.
* fix: check for row existence in parameter upsert
Do an exists check instead of relying on the result of `update!` being
zero. `update!` can return zero either when a row does not exist or when
there are no applicable changes in the request.
* fixup! use medley/mapply instead
* test logs as artifact
* better test logs
* debugging improvements
* fix `with-context` query to work with non-literals
```
(with-context {:foo "bar"}
...)
```
worked,
```
(with-context (merge {:foo "bar"} {:bin "baz"})
...)
```
did not, now it does!
* add compiled query args to context
* kondo
* compact logs
* use an `:after` method
* use defonce to avoid repeated `test-var` overwrite
This reverts commit 9dcbcba3f6.
snowflake is now fixed from being flaky and is now required again
the fix was in checking old dataset names to prune. Shared databases
like cloud databases don't get destroyed when docker terminates, so we
have to remove old databases. The way we did it in the past would error
if more than 10,000 rows were returned. The fix is the following from
https://github.com/metabase/metabase/pull/58818 :
```clojure
(defn- old-dataset-names
"Return a collection of all dataset names that are old -- prefixed with a date two days ago or older?"
[]
;; modified the query from the snowflake driver:
;; fa58e34968/src/main/java/net/snowflake/client/jdbc/SnowflakeDatabaseMetaData.java (L1644C21-L1644C90)
;; We run into issues where this returns more than 10,000 items and we get an error: "The result set size exceeded
;; the max number of rows(10000) supported for SHOW statements. Use LIMIT option to limit result set"
(let [query "show /* JDBC:DatabaseMetaData.getCatalogs() with limit */ databases in account limit 10000"]
(into []
(comp (map :name)
(filter sql.tu.unique-prefix/old-dataset-name?))
(jdbc/reducible-query (no-db-connection-spec) [query] {:raw?
true}))))
```
Some mysql -> postgresql migrated users still had adjust to source
failing because they had two tables with downcased names but one with a
"public" schema column and the other with a null schema column. This
handles that case by only selecting tables if there is not already a
table with a "public" schema column.
* VIZ-192 Updates goal tooltip to use same formatting as y-axis (but ignores compact/auto formatting to ensure the goal value is represented accurately)
* VIZ-192 Adds support for formatting percentage-based goals (e.g. bar charts with "Stack - 100%" selected)
* VIZ-192 Updates getGoalLineHoverData to accept a formatGoal param rather than a chartModel per PR feedback
* VIZ-192 Adds e2e tests for goal value formatting
* VIZ-181 Adds boundaryGap to bar chart y-axis if it has negative values to ensure bar labels don’t overlap axis labels
* VIZ-181 Minor performance consideration; `ensureRoomForLabels` only checks `dataSeriesOptions` since "bar" series won't show up in other seriesOptions
* VIZ-181 Tweaks ensureRoomForLabels calculation
* VIZ-181 Adds unit tests for ensureRoomForLabels
* Don't open the database picker when there is only one database
* Allow null for database
* Add repro for metabase#57644
* Remove duplicate it
* Use == to check for undefined
* Fix test description
* Update e2e/test/scenarios/native/native-reproductions.cy.spec.ts
Wording
Co-authored-by: Kamil Mielnik <kamil@kamilmielnik.com>
---------
Co-authored-by: Kamil Mielnik <kamil@kamilmielnik.com>
* Add check for correct clauses for expression mode
* Add support helper to expressions/mode
* Do not show help text for unsupported functions
* Add repro for metabase#58230
* Add e2e test for not showing helptext for unsopported functions
* Allow value operator
* Pass expressionMode where needed
* Tranlate mode name in error message
* Avoid throwing in checkExpressionModeSupportsClause
* Add repro for metabase#57388
* Fix update button helper
* Set to default when value is empty
* Update repro
* Make unit test more robust
* Do not pass default value to default value picker
* Fix unit test
* fix info formatting
* ensure exception is not swallowed
* Ensure we use a different connection per [[have-select-privilege?]] call.
This is to guard against databases (like Oracle) closing a connection when attempting
to execute a priviledged select, causing subsequent calls to fail.
* adjust usage of snowflake driver
* adjust usage of postgres driver
* update tests
* more efficient: move filtering within db-tables
* add test
* add changes to driver changelog
* route through driver rather than jdbc, so athena doesn't fail
* ensure we're testing only the drivers using default jdbc impl
* expose wrapper, rename
* PRG-53 add community translations notice
Enhance localization features by adding community translation links and descriptions across user profile, localization settings, and setup components.
* PRG-53 Refactor localization components to use CommunityLocalizationNotice
Replaced direct translation links and descriptions in UserProfileForm, LocalizationSettingsPage, and LanguageStep with the new CommunityLocalizationNotice component for improved code reuse and consistency.
Signed-off-by: Jakub Chodorowicz <jakub@chodorowicz.com>
* PRG-53 Use external link in the notice and add tests
* PRG-53 Fix e2e i18n test
* PRG-53 Prefer stack over brs, add a11y label, refactor export
* PRG-53 Improve cy selector to a semantic one
* PRG-53 Use hardcoded Metabase name in the setup flow
* PRG-53 use color name in color property
* PRG-53 use show-metabase-link setting
* PRG-53 Use Metabase original name for admin screens
* PRG-53 Add storybook stories
* PRG-53 Don't mention custom name in the translation when white labelling
* PRG-53 use static for crowdin link
* PRG-53 remove CommunityLocalizationNotice story
* PRG-53 Fix e2e i18n test
---------
Signed-off-by: Jakub Chodorowicz <jakub@chodorowicz.com>
* fix: lookup cloud migration part keys
Mapping over multipart-urls map entries wasn't ordered or deterministic, but it happened to be for less than 8 elements.
Changed to a lookup on advice from @dpsutton.
Fix https://metaboat.slack.com/archives/C010ZSXQY87/p1748611922362219
* test: update cloud migration settings test vars
* fix: add early multipart errors
* Update Static and Interactive Embedding Settings
* update sdk settings
* remove useMergeSetting
* remove more dead code
* remove more unnecessary stuff
* update unit tests
* more test updates
* update e2e tests
* Move embedding unit tests to local components
* address review comments
* Fix a flaky test for max-refingerprint-field-count
The code here goes through tables and refingerprints them (with all
their fields). After each table it checks the
`max-refingerprint-field-count` and stops if we've hit it.
The test gave some wiggle-room - we'd allow 10 extra fields. But in a
fresh run of my tests, the biggest table has... 13 fields. So if the
last table we hit before hitting the limit was *that* table, we exceed
our wiggle-room and the test fails.
Instead, the test now actually looks to see what's the biggest table we
have (in terms of number of fields), and asserts that the number of
fields refingerprinted is, at the most, the max plus that number of
fields.
* fix two more flaky tests
one is clearly a symptom of another test being super naughty but I can't
figure out which one. It's marked `parallel` and doing
```
(mt/with-temp [:model/Card {id :id} ...]
(do-something-with (t2/select-one :model/Card id)))
```
which is failing because `(t2/select-one :model/Card id)` is returning
nil. We can just pass the card that was actually returned by
`mt/with-temp` here and that should be more resilient to horrible tests
being horrible.
the other is checking that we're caching something by checking the call
count to toucan on the first and subsequent call to a function. But it
asserts that the first call count is exactly 6, which isn't relevant.
If I start a fresh REPL I get more than 6 the first time. It doesn't
matter! Just assert that we don't get MORE than the first time on the
second run!
* fix flaky tests freal
* fix: use t2/query to skip card callbacks
* fix: retry on failed pk insert
* fix: log when we have the cluster lock
* fix: try catching all sqlexceptions
* fix: add a fixture to pre-create the cluster_lock
* fixup! remove with-connection from update-used-cards
---------
Co-authored-by: Edward Paget <ed.paget@gmail.com>
* Add testing context around before-and-after gen tests
To ensure you can see the before and after queries as well as the steps that produced them when a test fails.
* Fix before-and-after :join in generative tests
Pass stage-number to check for aggregations and breakouts on the stage where the join was added rather than the final
stage.
* Add an option to force multi-staged data picker
* Move type to a proper location
* Add `question` to `entity_types`
* Add ability to select saved questions back to multi-staged data picker
* Auto select Questions or Models when there is only a single option
* Prevent accidental types
* Remove unreachable code
* Default the join step after selecting a question to the question step
* Fix type errors
* Improve typing + fix unit tests
* Remove unnecessary await
* Add unit tests
* Update the default simple data picker value
* Fix the picker name typo
* Ignore `question` in Simple data picker
* Add E2E tests
* Move a constant out of a type file
* Fix types
* Address review. Make code more sane
* Address review. Improve variable name clarity.
---------
Co-authored-by: Oisin Coveney <oisin@metabase.com>
* add tests for SEM-3
* let fingerprinters respect ->Number coercers
* explicitely assert for supported coercions
* add test on fingerprinted supported coercions
This was failing because we call
```
(session/create-session! :sso user (request/device-info request))
```
with the result of this function, and `session/create-session!` requires
a user with at least `:id` and `:last_login`. We didn't have a
`:last_login` in this case so `create-session!` was barfing.
we didn't have any tests that actually logged in as an existing user.
* update sync error handling to take into account `do-not-retry-exceptions?` as per its docstring
* add test
* introduce `with-returning-throwable`, use consistently
* update test
* add remember me to google sign in
* UI adjustments
* assert that remember me checkbox is rendered with google button
* backend test that cooke expiration is set
* rollback model changes in BE test
* create user model and use with-model-cleanup
---------
Co-authored-by: Edward Paget <ed.paget@gmail.com>
* Define the number of engines in a constant
* Improve callback naming
* Use t-shirt sizing
* Remove superfluous e2e test
We're already covering this scenario using Jest.
* Update prop naming
* Simplify CSS
* Change tab as well when url changes
* Do not get confused about editor height
* Add repro for metabase#55486
* Scope loading query
* Test forward button too
* Limit query
* Remove loading check altogether
* Wait for queries to complete after saving changes
* Wait for `dc.series` queries in `visitDashboard`
* Wait until visualizer loads after it's open
* Fix flaky tests
* Remove flaky tag
* Revert "Wait for `dc.series` queries in `visitDashboard`"
This reverts commit 3f6a606442.
Error we were getting:
```
1. Unhandled net.snowflake.client.jdbc.SnowflakeSQLException
The result set size exceeded the max number of rows(10000)
supported for SHOW statements. Use LIMIT option to limit result set
to a smaller number.
SnowflakeUtil.java: 170 net.snowflake.client.jdbc.SnowflakeUtil/checkErrorAndThrowExceptionSub
SnowflakeUtil.java: 102 net.snowflake.client.jdbc.SnowflakeUtil/checkErrorAndThrowException
StmtUtil.java: 488 net.snowflake.client.core.StmtUtil/pollForOutput
StmtUtil.java: 394 net.snowflake.client.core.StmtUtil/execute
SFStatement.java: 502 net.snowflake.client.core.SFStatement/executeHelper
SFStatement.java: 211 net.snowflake.client.core.SFStatement/executeQueryInternal
SFStatement.java: 145 net.snowflake.client.core.SFStatement/executeQuery
SFStatement.java: 779 net.snowflake.client.core.SFStatement/execute
SFStatement.java: 697 net.snowflake.client.core.SFStatement/execute
SnowflakeStatementV1.java: 293 net.snowflake.client.jdbc.SnowflakeStatementV1/executeQueryInternal
SnowflakeStatementV1.java: 149 net.snowflake.client.jdbc.SnowflakeStatementV1/executeQuery
SnowflakeDatabaseMetaData.java: 1648 net.snowflake.client.jdbc.SnowflakeDatabaseMetaData/getCatalogs
REPL: 107 metabase.test.data.snowflake/old-dataset-names/fn
```
We do this in `old-dataset-names` by just calling `(.getCatalogs
metadata)` which runs
fa58e34968/src/main/java/net/snowflake/client/jdbc/SnowflakeDatabaseMetaData.java (L1644C21-L1644C90)
or
```sql
String showDB = "show /* JDBC:DatabaseMetaData.getCatalogs() */ databases in account";
```
I've modified that to have a limit and then run that
`jdbc/reducible-query` to get the names
* Fix zoom drill for clicks on a joined PK column
* Add lib.drill-thru.common.primary-key? and foreign-key? helpers
* Fix zoom and quick-filter drill for clicks on a joined PK column
* Remove unused imports
* Move source fields check into lib.drill-thru.common and revert changes in lib.metadata.calculation
* Remove trailing whitespace from docstring
* Lookup primary key in visible-columns ignoring joins rather than source-table fields
* Fix docstring
* Fixing mapped user facing strings for relative date params in subscriptions and alerts
* Adding frontend fix + spec
* Changing 'is' to '->' in UI
* pr feedback
* e2e test fix
* use badges in oss sidebar; update pulse card rendering; test fixes
* test fix
* test fix
* pr feedback
Currently, if a user is set to `is_active: false`, we still allow them
to log in via JWT or SAML. When they do, we also don't update the value
of `is_active`, which could potentially lead to inaccurate user counts.
After this change, the behavior when discovering an existing user upon
login is:
- if `is_active` is `false` and JWT/SAML user provisioning is on, we'll
log the user in and set `is_active` to `true`.
- if `is_active` is `false` and JWT/SAML user provisioning is off, we'll
give the user an error and not change the existing user.
- if `is_active` is `true`, we'll log the user in like normal.
* Remove binning from pinned items
* Add reproduction for metabase#57697
* Revert "Remove binning from pinned items"
This reverts commit aa2d4abbb4472b835dee901ce7ac36c4bc43cd5e.
* Remove binning from column when calculating displayname
* Add models; liquibase migration; ...
and other model incantations
* Add fk constraint after indexes
* Update model tests for column names
* serdes fixes
* Replace remaining missing metabot_model_entity_id
* Update generated baselines
* fix serialization tests
* initial implementation
* make the api match
* wip
* align types with API
* improved routing
* empty state
* cleanup and tests
* pivot to collection picking
* no personal collection
* update tests
* remove defunct table changes
* fix conflicts
* add copy for embedded metabot
* linter appeasement program
* linter appeasement program
* linter appeasement program
* update api
* no thank you migration
* un-un-un revert some stuff
* address review comments
---------
Co-authored-by: Edward Paget <ed.paget@gmail.com>
* Add lib.util/find-clause-by-uuid
* Lookup clause by uuid to generate the :long display-name for a :metadata/column
* Fix display-name expectations in targets.unit.spec.ts
* Fix display-name expectations in click-behavior.cy.spec.js
* Fix display-name expectations in filter-sources.cy.spec.js
* Review feedback: ensure last-stage? throw when given an invalid stage-number
* Review feedback: use lib.tu/venues-query
* Review feedback: explict true? and false? checks for first-stage? and last-stage?
* WIP
* Adjust process-multiple-queries to also support single pivot queries
* cleanup
* Seemingly working backend support for only running necessary subqueries
* pivot unit test adjustments
* Test fixes
* Rerun pivot query when settings change
* wip fix for pivots as email attachments in subscriptions
* alternate single query qp support; unit test fix
* fix ts
* fix
* fix frontend spec
* pr feedback
* adding api test coverage for pivots with totals disabled
* fix cljkondo
* typo
---------
Co-authored-by: Noah Moss <noahbmoss@gmail.com>
* keep backend pivot exports uncollapsed
* fix kondo errors
* update tests - only run collapsing tests on cljs, add a new test for clj that rows are not collapsed
* fix more kondo warnings
* cleanup
* feat: api and models for managing metabot entities
* Add stubs
* Add models; liquibase migration; ...
and other model incantations
* tests for api
* Update h2 copy
* rename model to dataset
* Add fk constraint after indexes
* Wrap tests with with premium feature
* Add seed migration; drop unused index
* Update API to wrap in items instead of data
* Update model tests for column names
* Pass remaining serdes tests
* Reorder migrations
* serdes fixes
* correct api to remove metabot_model_entity_id
Handle root collection
* fix mistake in merge
* Replace remaining missing metabot_model_entity_id
* Update generated baselines
* fix rollback sql syntax
* update api to support collections
* update serdes for collections in metabot entities
* fix precondition for loading metabots
* fix cleanup metabot table before list endpoint test
* fix whitespace
* fix serialization tests
* fixup! change migration version to 56
* fixup! remove remaining no-metabot tags
* fixup! super nit
* fixup! move api into new namespace
make /metabot API routes singular
* fixup! better combine routes
* cljs: Make currency definitions more compact
Useful for reducing CLJS bundle size.
* cljs: Use unquoted functions in metabase.lib.drill-thru
* cljs: Make var re-exporting leaner in CLJS
* prevent Metabot from making a request for suggested prompts if no user is logged in to prevent auth redirect
* this fix is better
* add code comment
* reference PR in code comment
* revert unneeded change
* Fix sharing e2e test
* Simplify `shouldSplitVisualizerSeries` check
* Handle empty/erroring datasets
* Use card names for series names
* Port e2e tests
* Don't use entity IDs
* Unskip test
* Include metrics and models in data importer search
* Return result_metadata for metrics and datasets
* Fix drillthrough test
* Update comment
* Bring back removed metric tests
* Remove reundant test
* include result_metadata in recents api request
* Fix e2e test
* Fix typo
Co-authored-by: Sébastien <sebastien@metabase.com>
* Fix e2e test
* fix be test
* Fix e2e tests
* Update src/metabase/activity_feed/models/recent_views.clj
Co-authored-by: Noah Moss <32746338+noahmoss@users.noreply.github.com>
---------
Co-authored-by: Tyler Plude <tsplude@gmail.com>
Co-authored-by: Sébastien <sebastien@metabase.com>
Co-authored-by: Noah Moss <32746338+noahmoss@users.noreply.github.com>
* Mega App DB memory usage improvements
* Add a test
* Minor comment formatting
* Fix test name
* This test can be ^:parallel
* Actually no it can not
* Fix test name
* Don't create mega queries when deleting databases (#58491)
* Improved fix
* Don't require new migrations to have preconditions (this is bad)
* Remove test
* Revert migrations
* Manually set Field.parent_id = NULL for all Fields in a Database when deleting a Database
* Tweak query for MySQL
* Yet another fix 🔧
* `IS NULL` => `IS NOT NULL`
* Give up on trying to find a nice fix
* MySQL fix
* MySQL fix (for real)
* Fix SDK cross version tests getting version number from other prerelease labels
* Update code from #58638
* Remove unnecessary console.log
* Simplify code even moarrrrrrr
* fix: set download-results to no on table block
When a view-data permission for a table is set to blocked set the
create-queries and download-results permissions to no. Also includes a
migration script read to backport to v52 to update databases to be
consistent with this change
* fixup! remove precondition
* fixup! join in update to work on mysql
* fixup! separate migrations on mysql/postgres
* fixup! test order
* Setup sdk-based embedding iframe route (#56915)
Co-authored-by: Mahatthana (Kelvin) Nomsawadi <me@bboykelvin.dev>
* restore routes.clj to fix merge conflicts
* Tests for new iframe embedding's script tag and route (#57269)
Co-authored-by: Mahatthana (Kelvin) Nomsawadi <me@bboykelvin.dev>
* remove the safe-to-remove warning on rspack css ordering
* Rename embed.v1.js to embed.js for new iframe embedding (#58415)
* Revert "remove the safe-to-remove warning on rspack css ordering"
This reverts commit 4feae48604.
* Remove redundant sdk-ee-plugins import from the SDK in new iframe embedding (#58416)
* Ignore CSS import order warnings from new iframe embedding (#58417)
* Follow-up validation improvements for new iframe embedding (#58516)
* Explicitly disallow using API keys in production for new iframe embedding (#58520)
* fix wrong merge conflict - do not destructure H.cy in component tests
---------
Co-authored-by: Mahatthana (Kelvin) Nomsawadi <me@bboykelvin.dev>
* Fix static question embed download when parameters have multiple values
* Simplify type
* Don't wanna deal with outdated query builder types 🤷
* Use a proper value when asserting
Before, nest-query relied on the preprocessor (specifically, the add-implicit-clauses middleware) to add all possible
fields to the newly created inner query (and then removes unnecessary fields later). However, add-implicit-fields
ignores fields with a parent id. The main user of parent-id is mongo, and mongo doesn't use nest-query, so this
usually isn't an issue. However, bigquery also uses parent-id, and bigquery is a sql driver that uses nest-query. As
a result, nest-query wasn't adding bigquery struct member fields to the inner query, which caused some queries to
break. Now, nest-query explicitly finds all non-sensitive/retired fields (including fields with parent ids) and
ensures that they are in the inner query.
* used `entityTypeFilter` for new embedding data pickers
* Rename `entityTypeFilter` to be consistent with `entity_types`
* Migrate `entity_types` to a new reducer
* Add E2E test
* remove unused prop
* Fix interactive embedding's `entity_types`
* Fix unit tests
* Avoid using quadratic search in FkTargetPicker
* Reduce further parsing and stringification
* Avoid calling JSON.parse in FkTargetPicker
* Assert fieldId
* Rename getters to be more clear
* use user locale setting if available for lang prop in <html>
* add a test
* fix my broken test
---------
Co-authored-by: John Swanson <john.swanson@metabase.com>
* use mantine menu and reorganize options
* remove comments
* fix unit tests
* e2e test fixes, replace "a" with ForwardRefLink to fix sidebar not getting closed
* fix external links
---------
Co-authored-by: Sloan Sparger <sloansparger@gmail.com>
* Add prompt to the query
* Add prompt to the query
* Add prompt to the query
* Add prompt to the query
* Add prompt to the query
* Add prompt to the query
* Add prompt to the query
* Tests
* Tests
* Tests
* Tests
* Tests
* fix: audit db not working after import form h2
Removes conditions from the `adjust-audit-db-to-source!` function so we
make sure we always restoring the audit db tables to their original form
regardless of what type of app-db is used.
* fixup! test setting up db triggers
* fixup! cljfmt
run breaking changes tests on multiple runners
TO BE REVERTED BEFORE MERGING
force production fe build
build-release as setting node_env fucks up everything
exclude @OSS tests
Revert "TO BE REVERTED BEFORE MERGING"
This reverts commit c907a2f492.
* Add additional check on rendering native preview panel
* adding e2e test
* adjusting e2e test
* do not show native panel when no data is selected
* move e2e test to instance analytics suite
* simplify embedding data picker code
* Show the data picker step based on starting query source
* Set starting model collection ID when joining a model
* Improve initial collection loading UX
* Fix failed unit tests and types
* Fix failed E2E tests
* remove unused var
* Fix sometimes the starting collection is incorrect
* Add an E2E test
* Add a comment to clarify
* Clarify a new prop
* VIZ-329 Updates ChartSettingInput and ChartSettingInputNumeric to apply changes when the user pauses typing in addition to on blur
* VIZ-329 Removes now-redundant blur handler in ChartSettingInputNumeric and updates unit tests
* Revert "fix serialization for visualizer entities (#57467)"
This reverts commit d56e791077.
* drop columns from test data
* fix types, tests
* Revert "[VISUALIZER] Fixing dashboard duplication for visualizer cards (#58106)"
This reverts commit 6cf7606754.
* keep preferred true?
* serdes transformation support for visualizer settings
* Update entity ids in visualizer dashcards on DB read to be card ids
* unit test for the serdes transformations
* cljkondo
* fix on-read to only run if viz settings exists
---------
Co-authored-by: Tyler Plude <tsplude@gmail.com>
Co-authored-by: Noah Moss <noahbmoss@gmail.com>
* Empty
* Add schema for datetime()
* Add datetime() function; works on iso strings
* Implement datetime() for sql to test all drivers
* Remove Z from datetime
* Change the format of the datetime string in tests
* Be more lenient on types
* have multiple formats for the test
* Implement datetime() for mongo
* Use ISO string in test
* Datetime for string with no mode
* Add changelog entry for :expressions/datetime
* Add docs for datetime
* Add e2e tests for datetime
* Update e2e test and datetime text
* Fix :expressions/datetime feature
* Remove reference to iso
* Remove unused schema
* Fix frontend test
* Use multiple arities for datetime()
* Replace T with ' ' in databases that don't like T in datetime timestamps
* Try removing visibility requirement in test
* Fix brittle test
* Test dates with space and with T
---------
Co-authored-by: Romeo Van Snick <romeo@romeovansnick.be>
* fix: explicitly mark when fields are gtapped
* add debug
* fixup! tests setup to set advanced feature
* fixup! strip any user-added gtapped-tables keys
* fix a bad import
* Refactor `browse` navigation section
- Removes the dependency on the `CollapseSection` custom component
- Prepares the section for the "Add Data" project
Co-authored-by: Sloan Sparger <sloansparger@users.noreply.github.com>
* added an ... submenu to open questions
* fix
* menu is now in the title
* addressed feedback
* reverted submenu change
* added title menu items to funnel
* added e2e test
* added chevron
* unit test fix
* remove auto-close logic
* add metabot icon to searchbar, clicking icon opens metabot instead of search
* make chat ui appear in the sidebar
* add header section to the chat sidebar
* progress on conversation ui
* few tiny additions
* fetch mock suggested prompts from an endpoint, animate thinking text, fix search button touch target size, do no clear chat history when closing the chat sidebar
* messages rendered as markdown
* restyles
* more restyles
* fix input autofocus
* impl empty state, remove more textarea code
* hide copy button actions
* handle user aborted requests better
* design iteration
* latest design feedback, better auto-scrolling, better markdown rendering
* fix types
* increase unit test coverage
* impl long convo calc, add unit tests, fix focus issue if clicking the metabot option from the command palette
* move code to enterprise folder, fix some e2e, adjust header alignment, fix some types
* small clean ups, fix failing tests and erroring types
* add unit test and fix selector for getting latest agent messages by type
* review feedback, turn on an e2e test again
* try getting metabot e2e working
* fix embedding sdk
* impl pr feedback
* Make default toast color darker
* Avoid remounting the UndoList on overy toast change
* Tweak the undo animation to come in from the left
* Move more undo state into UndoListingOverlay
* Reverse toast order
* Manually manage undo target
* Remove unused props
* Store pause state in undo itself
* Use react-transition-group to render toast notifications
* Measure height so we can animate toast position
* Use css transforms only and avoid animating bottom
* Mock react-transition-group in test
* Unmount elements on exit
* Wait for animation in test
* Make toast ref optional
* Clean up transition
* Fix e2e test
* Disable transition group in cypress
* Add check_filled icon
* Default to check_filled
* Add text-secondary-inverse
* Use text-secondary-inverse color for icon on tooltip
* Remove min-width from toast
* Control the animation state of the todo's after moving the portal
* Fix icon in unit tests
* Simplify undo hooks
* Avoid overwriting state
* Add missing import
* Allow transitioning new todos even when not moving the portal
* Fix e2e test
* Update Loki Snapshots
---------
Co-authored-by: Metabase Automation <github-automation@metabase.com>
* fix typo
* don't error out during fields-metadata fallback if table is known to not exist
* table known to not exist impl for drivers
* add test
* add debug log
* add to changelog
* address review comments
This commit adds the conditional branding to dashboard email subscriptions and to question alerts. It does so by altering the handlebars templates we use for each, respectively. It follows the same logic as the rest of the PRs from this project. If a token has a whitelabel feature flag (Pro/Enterprise), the branding is not shown. It should be shown for OSS and for Starter plans.
* Add branding content to dashboard subscriptions
* Add branding content to alerts
* docstring typo fixes
* [SEM-8] Exclude all numeric values from Category semantic type consideration
* add unit test for SEM-8
* more typo fixes
* adjust existing tests for new sync behavior
* extend category classifier restriction to any non-text(-like)
* unit test for bolean classification
* further adjust existing tests for new sync behavior
* missed one test
* Fix e2e test by filtering
* fix e2e test by setting field semantic-type before hand
* don't classify int/bool values to category by name either
* rename tests
* extend tests
* style
* remove unused var
---------
Co-authored-by: Romeo Van Snick <romeo@romeovansnick.be>
* Fix the sorting of custom column from a saved card in a new question
* Remove the expression_name that is not actually used.
* Use col.name instead of expression_name in sanatizeResultData
* Remove additional :expression_name references in tests
* Remove comment that was attached to now-removed :expression_name
* Remove expression_name from static-viz components
* Remove expression_name from instance_analytics export
* Add comment to now-deprecated expression_name field in DatasetColumn
* Add e2e test for sorting on nested custom column
---------
Co-authored-by: Lifeng Lv <lvlifen9@gmail.com>
* unlimit pivot exports
* fixups, fix default for collapsed rows setting
* address import issue
* simple unit test
* simplify test a bit
---------
Co-authored-by: Tyler Plude <tsplude@gmail.com>
Running `yarn dev-ee` was broken on the latest master. (introduced in #58054)
The compiler was complaining about the `\` in the docstring because it was interpreting it as an escape character followed by a newline.
We're greatly simplifying and speeding up a Metabase setup
when the use case is "embedding". This currently applies only to
hosted instances. The FE in the main app will expect the query param
`use_case=embedding` to exist. If it does, we're showing the simplified
setup flow.
* Empty
* kick ci
* Allow temporal type in addition to string for `date()`
* Add date(datetime) test
* Use cast for date for postgres
* Extend date() on mysql
* Extend date() on bigquery
* Extend date() at snowflake
* Extend date() to datetime on clickhouse
* Generalize `date()` to :sql
* Update changelog with `date(datetime)`
* Empty
* DateTime->Date Coercion + test
* Slight cleanup
* Whitespace
* Cleaner implementation and test
* Test timestamp as well as timestamptz
* Correct implementation to get subtypes of DateTime
* Update helper text string for date
* Add e2e tests for date
* Also check to correct coercion of dates
* Add customization points to enable clickhouse implementation.
* Remove changelog-test/ignore true from ->date
* Fix defn ordering
* Fix expected type for datetime coercions
* Fix tests for sqlite
* Implement coercion for Mongo
* Implement specialization for oracle
* Add :expressions/date feature to all sql dbs
* Add :expressions/date feature to mongo
* Mongo: add text() and date()
* Better check
* Better check
* kick ci
* Add text() to all sql dbs
* Empty
* kick ci
* kick ci
* Format code
* Format code
* Add :text implementation to sql driver
* More robust tests across drivers
* Correct changelog text
* Ors on one line
* Or on same line
* kick ci
* Add `text-dbtype` multi-method
* Add athena implementation of text-dbtype
* Add databricks implementation of text-dbtype
* Remove date expression in tests that breaks some sql dbs
* Improve cast test for dates on other dbs
* Remove duplicate test
* Remove test that won't work for many dbs
* Use VARCHAR for oracle
* Use STRING for SparkSQL
* Use VARCHAR on Vertica
* Remove test that converts date to text before parsing it
* Add mongo exception
* Only run this test for SQL databases (requires native sql query)
* Add SQL Server text type
* Add Starburst text type
* Try varchar2(100)
* Simpler implementation
* Don't support date() for Oracle
* Try selecting from people to get oracle to work.
* Compile nested query for `text()` test
* Don't test empty string because Oracle treats it as NULL.
* Appease the kondo
* Implement Presto cast to text
* Implement cast-to-text in terms of cast (for presto)
* Correct test name
---------
Co-authored-by: Romeo Van Snick <romeo@romeovansnick.be>
We're using the `computed.filters` value in the handlebars email template.
Previously, if `parameters` were an empty list `[]` we were returning a truthy value,
and handlebars was rendering an empty row in an email.
@ngoc figured out that we need to wrap parameters with `seq` in order to have a
a proper `nil` assigned to `:parameters`.
* Cast boolean expressions in :fields clauses to type bit on sqlserver
* Compile boolean literals :inline on sqlserver
* Add some tests for non-booleans to boolean_to_comparison_test.clj
* Add tests for boolean-expression-clause?
* Mention sql.qp/*inner-query* in boolean-expression-clause? docstring
* Split separate cases into separate tests
* Add cast in sql.qp/as if field options contain ::sql.qp/add-cast
Rather than duplicating logic for :select-top in apply-top-level-clause [:sqlserver :fields], just decorate boolean
:fields clauses with ::sql.qp/add-cast and let the parent method handle it. In this way, the :sqlserver method can
pre-process the MBQL :fields clauses rather than post-processing the raw honeysql.
* Add tests for {sum,count,distinct}-where with literal booleans
* Fix typo in literal expressions tests: filters -> filter
* Fix boolean->comparison for boolean expressions coming from nested queries
* Add sql.u/->honeysql-parent-method util
* Remove ::boolean-is-comparison abstract driver
* Lift ->honeysql call out of boolean-to-comparison
* Pass in boolean-types to boolean->comparison from driver
* Update comments and docstrings
* Require :left-join feature in nested-and-joined-literal-boolean-expressions-in-conditions-test
* Revert whitespace changes
* Implement sql.qp/apply-top-level-clause :filter in terms of parent method
* Fix typo in docstring
* Remove unused sql.helpers import
* Add comment about Oracle 23+ support to boolean_to_comparison.clj
* Rename local has-types? to has-some-types? in boolean-field-clause?
* [Backend modularization] `parameters` module (first pass)
* Make sure DB is initialized
* Oops revert the change I made to demo something during BE Guild Sync
* feat: generic data permissions visibility queries
Creates a new namespace with a generically applicable set of sql queries
for getting only models controlled by data permissions that a given user
has access to.
It also creates a new model.interface defmulti `visible-filter-clause`
that allows us to generate a filter clause for a given model to filter a
query on a supplied expression to only those models allowed for the
supplied user. There is a default implementation that is always true.
the `visible-filter-clause` method is only implemented for the Table
model so far since that is used in a search.
This also fixes a bug in the previous implementation of the sql
filtering for Table models that did not correctly handle the logic in
`data-perms/coalesce` for `:perms/view-data` where `:blocked` is ranked
ahead of `:legacy-no-self-service` for the purposes of selecting the
applicable user permission but is ranked below when testing permission levels.
Co-Authored-By: John Swanson <john.swanson@metabase.com>
* fixup! default for visible-fliter-clause to false
* fixup! add cast back for search permissions
* fixup! correct db type test
* fixup! more verbose table names in perms test
---------
Co-authored-by: John Swanson <john.swanson@metabase.com>
Closes#41265.
The schema for is-empty and not-empty was updated in #56855 to allow StringExpressionArg as well as Field. This
updates desugar-is-empty-and-not-empty to also understand non-field args.
* fix: move quartz metric listeners to analytics
To make sure the task module does not need to depend on the analytics module
* fixup! remove analytics from task uses
* fixup! export listener adder methods
* fixup! missing docstring
* fixup! put the comment back in the right spot
* Update upgrading-metabase.md
Add changes that were needed for a long time :)
* suggested edits (#57913)
---------
Co-authored-by: Jeff Bruemmer <jeff.bruemmer@gmail.com>
Remove `expectGoodSnowplowEvents` entirely - it was an antipattern.
Replace it with a good set of snowplow helper functions that actually
accomplish what we want:
- `expectSnowplowEvent`, which matches on the *entire* event rather than
assuming the event is an unstructured event (most of ours are, but some
are not).
- `expectUnstructuredSnowplowEvent`, which behaves the same as
`expectGoodSnowplowEvent` used to, except I removed an unused behavior
where it took a function instead of a partial-match object.
- `assertNoUnstructuredSnowplowEvent`, which asserts the inverse of
`expectGoodSnowplowEvent` - there are zero matching snowplow events.
I removed `Good` from all these names as I think that's implied - if you
want to make an assertion that a bad event is being received you can
write your own helper :)
* Only send id and value when downloading csv
* Add repro for metabase#51813
* Merge in template tag parameters to parameters when running cards
* Fix erroneous test
* Do not pass all parameter data for cards
* Remove commented out code
* Only pass sparse parameters on public dashboards or cards
* Remove unused import
* Fix lint
---------
Co-authored-by: Tamás Benkő <tamas@metabase.com>
* [Backend modularization] `collections` module (DEV-444)
* Fix wrong namespace
* Fix new metabot v3 code
* Some test fixes. 🔧
* Move test for EE-specific behavior to EE
* Make `/api/collection` tests work correctly from the REPL
* Try to debug failing test
* Fix sanity checc
* Fix Kondo warnings
* Test fix 🔧
* Attempt to debug test failures
* More test failure debugging
* Cleanup
* More test failure debugging
* More test debugging
* Fix private var
* Test fix? 🔧
* adding get-visualization-columns in clojure land
* remove inline defs
* add util unit tests
* update comment
* linter fixes
* Fix render visualizer utils to work with entity ids, add test
* Fix body test
* Remove unused endpoints
* Resolve implicit FK Field -> PK Field -> [Name] Field remapping
This affects the field and dashboard APIs.
* New endpoints
* Remove unused code
* Always return value, possibly without label
* Stop requiring FK entry in FK -> PK -> Name resolution
* Relax FK requirement in remapped value resolution
* Remove remaining /field/ endpoints
---------
Co-authored-by: metamben <103100869+metamben@users.noreply.github.com>
* use existing col settings helper
* forward declaration of viz-settings-for-col to prevent compilation issues
* fixing typo in parens
* fix unit tests
* Fix clj kondo warnings
* fix: wrap un-translated strings in t``
* fix more untranslated strings
* translate some aria-labels
* use interpolation instead of having to provide context
* address feedback and add context
* more t calls added
* revert changes to slider component as it's not used and may get removed
* remove file that i re-created when rebasing
* more context
* more context
* Update frontend/src/metabase/query_builder/components/MoveQuestionModal/MoveQuestionModal.tsx
Co-authored-by: Raphael Krut-Landau <raphael.kl@gmail.com>
* Update frontend/src/metabase/query_builder/components/expressions/Editor/Listbox/Listbox.tsx
Co-authored-by: Raphael Krut-Landau <raphael.kl@gmail.com>
* address feedback
* add context to what is SSL short for
* remove whitespace that could possibly create issues
* add missing import for c
---------
Co-authored-by: Raphael Krut-Landau <raphael.kl@gmail.com>
* Empty
* kick ci
* Allow temporal type in addition to string for `date()`
* Add date(datetime) test
* Use cast for date for postgres
* Extend date() on mysql
* Extend date() on bigquery
* Extend date() at snowflake
* Extend date() to datetime on clickhouse
* Generalize `date()` to :sql
* Update changelog with `date(datetime)`
* Empty
* DateTime->Date Coercion + test
* Slight cleanup
* Whitespace
* Cleaner implementation and test
* Test timestamp as well as timestamptz
* Correct implementation to get subtypes of DateTime
* Update helper text string for date
* Add e2e tests for date
* Also check to correct coercion of dates
* Add customization points to enable clickhouse implementation.
* Remove changelog-test/ignore true from ->date
* Fix defn ordering
* Fix expected type for datetime coercions
* Fix tests for sqlite
* Implement coercion for Mongo
* Implement specialization for oracle
* Better check
* kick ci
* kick ci
* Format code
* Format code
* Correct changelog text
* Ors on one line
* Or on same line
* Remove zh from the list of supported translations (#57672)
We don't support the `zh` (Traditional Chinese) locale anymore – only `zh-CN` (Simplified Chinese), and `zh-HK` (Hong Kong), and `zh-TW` (Taiwan). Here's a [Slack discussion](https://metaboat.slack.com/archives/C6UBFU9GW/p1741983889165599).
So let's remove `zh` from the list of supported translations.
For some reason the list wasn't sorted alphabetically by language name - this PR fixes that too.
* [Backend modularization] Mega Renaming PR 1 (#57561)
* auth-provider => auth-provider.core
* enterprise/auth-provider
* `metabase.events.core`
* Move `sql-parsing-enabled` Setting into `query-analysis` (DEV-449)
* `metabase.events.core` (DEV-455)
* `metabase-enterprise.stats` => `metabase-enterprise.analytics.stats` (DEV-475)
* `enterprise/stale` cleanup (DEV-474)
* `enterprise/airgap` => `enterprise/premium-features`
* Appease Kondo
* Remove unused
* Mark setup tests flaky (#57710)
* mark setup tests flaky
* add flake
---------
Co-authored-by: Sloan Sparger <sloansparger@gmail.com>
* Fix parameter types! (#57677)
* [Notification] reduce module apis (#57678)
* fix audit log to record external emails when sending notifications (#57679)
* Remove "Model" and "Action" from the "New" button menu (#57647)
* feat: add a link to field filter docs from dashboard sidebar with native question (#57687)
* feat: add a link to field filter docs from dashboard sidebar with native question
* oops
* add missing translation
* Metabot v3 PoC (#52436)
Co-authored-by: Tyler Plude <tsplude@gmail.com>
Co-authored-by: Phoomparin Mano <poom@metabase.com>
Co-authored-by: Mihael Konjević <konjevic@gmail.com>
Co-authored-by: John Swanson <john.swanson@metabase.com>
Co-authored-by: Aleksandr Lesnenko <alxnddr@gmail.com>
Co-authored-by: Alexander Polyankin <alexander.polyankin@metabase.com>
Co-authored-by: Thomas Schmidt <thomas@metabase.com>
Co-authored-by: Romeo Van Snick <romeo@romeovansnick.be>
Co-authored-by: Ngoc Khuat <qn.khuat@gmail.com>
Co-authored-by: Mahatthana (Kelvin) Nomsawadi <me@bboykelvin.dev>
Co-authored-by: github-automation-metabase <166700802+github-automation-metabase@users.noreply.github.com>
Co-authored-by: Nicolò Pretto <info@npretto.com>
Co-authored-by: Aleksandr Lesnenko <alxnddr@users.noreply.github.com>
Co-authored-by: Phoomparin Mano <phoomparin@gmail.com>
* Patch the metabot-v3 merge (#57731)
* Remove unused props from the "New" menu
* Remove Metabot code related to the "New" menu
* Fix flaky e2e test in the onboarding `setup` spec (#57735)
* remove unused slider component (#57745)
* [Admin] Modernize components for People + Groups pages (#57365)
* update components in frontend/src/metabase/admin/people/components to use Mantine components and remove deprecated component usage
* update people and group pages/modals to use more modern components
* use relative sizes
* fix user success modal not closing on "Done" confirm button click
* remove CS from PeopleListRow
* clean up admin/people/colors.ts
* lint fix
* people list footer fix
* pr feedback
* [Admin] Convert people / group pages to RTKQuery (#57490)
* update components in frontend/src/metabase/admin/people/components to use Mantine components and remove deprecated component usage
* update people and group pages/modals to use more modern components
* use relative sizes
* fix user success modal not closing on "Done" confirm button click
* remove CS from PeopleListRow
* clean up admin/people/colors.ts
* lint fix
* people list footer fix
* pr feedback
* wip
* fix issues, people page updates, some clean up
* add unsaved files
* type fix, fetch user list from rktquery for PeopleList
* remove entity usage from unsubscribe modal, lint fix
* remove all remaining usages of user entity
* remove user entity
* test fixes, remove unused user schema
* bit of clean up
* qa pass - invalidate groups cache if user is deactivated/reactivated
* address pr feedback
* Bump org.apache.poi/poi-ooxml-full from 5.4.0 to 5.4.1 (#56357)
Inspect dependency changes here:
Co-authored-by: Metabase bot <metabase-bot@metabase.com>
* Add analytics for new entity creation being initiated (#57673)
This commit adds analytics to the:
1. New button being clicked
- In the top app bar
- From the empty collection
2. New menu individual items being clicked:
- Dashboard
- ~Question~ (not possible at the moment)
- ~Native query~ (not possible at the moment)
3. An entity creation being initiated
- Model from the /browse/models page
- Metric from the /browse/metrics page
- Collection from the collection header
- Collection from the main nav sidebar
* Bump io.github.eerohele/pp from 9598f4c72b528480a85ea41651c45b08d1717428 to 4998a1fdd9d3713d7034d04509e9212204773bf2 (#57577)
Inspect dependency changes here: 9598f4c72b...4998a1fdd9
Co-authored-by: Metabase bot <metabase-bot@metabase.com>
* Bump commons-io/commons-io from 2.18.0 to 2.19.0 (#56654)
Inspect dependency changes here:
Co-authored-by: Metabase bot <metabase-bot@metabase.com>
* Shutdown index sync while its results are unused (#56885)
* Shutdown index sync while its results are unused
* Update tests
* Update mongo tests
* Bump com.google.guava/guava from 33.4.6-jre to 33.4.8-jre (#56952)
Inspect dependency changes here:
Co-authored-by: Metabase bot <metabase-bot@metabase.com>
* Bump com.amazonaws/aws-java-sdk-core from 1.12.782 to 1.12.783 (#57586)
Inspect dependency changes here:
Co-authored-by: Metabase bot <metabase-bot@metabase.com>
* Bump io.github.tonsky/clj-reload from 0.9.4 to 0.9.5 (#57578)
Inspect dependency changes here: https://github.com/tonsky/clj-reload/blob/0.9.5/CHANGELOG.md
Co-authored-by: Metabase bot <metabase-bot@metabase.com>
Co-authored-by: Cam Saul <1455846+camsaul@users.noreply.github.com>
* Bump babashka/fs from 0.5.24 to 0.5.25 (#57575)
Inspect dependency changes here: https://github.com/babashka/fs/blob/v0.5.25/CHANGELOG.md
Co-authored-by: Metabase bot <metabase-bot@metabase.com>
Co-authored-by: Cam Saul <1455846+camsaul@users.noreply.github.com>
* Bump org.apache.calcite.avatica/avatica from 1.25.0 to 1.26.0 (#54171)
Inspect dependency changes here: https://github.com/apache/calcite-avatica/compare/avatica-1.25.0-rc0...avatica-1.26.0-rc0
Co-authored-by: Metabase bot <metabase-bot@metabase.com>
Co-authored-by: Cam Saul <1455846+camsaul@users.noreply.github.com>
* fix: do not render gauge if grouping col is date (#56715)
* fix: toolbar button color not changing on hover (#56694)
* Bump refactor-nrepl/refactor-nrepl from 3.10.0 to 3.11.0 (#57584)
Inspect dependency changes here: https://github.com/clojure-emacs/refactor-nrepl/blob/v3.11.0/CHANGELOG.md
Co-authored-by: Metabase bot <metabase-bot@metabase.com>
Co-authored-by: Cam Saul <1455846+camsaul@users.noreply.github.com>
* Bump org.jsoup/jsoup from 1.19.1 to 1.20.1 (#57583)
Inspect dependency changes here: https://github.com/jhy/jsoup/blob/jsoup-1.20.1/CHANGES.md
Co-authored-by: Metabase bot <metabase-bot@metabase.com>
Co-authored-by: Cam Saul <1455846+camsaul@users.noreply.github.com>
* [Backend modularization] `view-log` module (DEV-427) (#57628)
* [Backend modularization] Move anonymous-stats-related stuff into `analytics` module (DEV-437) (#57631)
* Bump org.eclipse.jetty.ee9.websocket/jetty-ee9-websocket-jetty-server from 12.0.19 to 12.0.20 (#57579)
* Bump org.eclipse.jetty.ee9.websocket/jetty-ee9-websocket-jetty-server from 12.0.19 to 12.0.20
Inspect dependency changes here:
* Update deps.edn
---------
Co-authored-by: Metabase bot <metabase-bot@metabase.com>
Co-authored-by: Edward Paget <edpaget@users.noreply.github.com>
Co-authored-by: Cam Saul <1455846+camsaul@users.noreply.github.com>
* Bump com.taoensso/nippy from 3.4.2 to 3.5.0 (#56953)
Inspect dependency changes here: https://github.com/taoensso/nippy/blob/v3.5.0/CHANGELOG.md
Co-authored-by: Metabase bot <metabase-bot@metabase.com>
* Bump com.ibm.icu/icu4j from 76.1 to 77.1 (#55241)
Inspect dependency changes here:
Co-authored-by: Metabase bot <metabase-bot@metabase.com>
Co-authored-by: Cam Saul <1455846+camsaul@users.noreply.github.com>
* Bump thheller/shadow-cljs from 2.28.21 to 2.28.23 (#56657)
Inspect dependency changes here:
Co-authored-by: Metabase bot <metabase-bot@metabase.com>
Co-authored-by: Cam Saul <1455846+camsaul@users.noreply.github.com>
* `date(datetime)` integration branch (#57448)
* Empty
* kick ci
* Allow temporal type in addition to string for `date()`
* Add date(datetime) test
* Use cast for date for postgres
* Extend date() on mysql
* Extend date() on bigquery
* Extend date() at snowflake
* Extend date() to datetime on clickhouse
* Generalize `date()` to :sql
* Update changelog with `date(datetime)`
* Update helper text string for date
* Add e2e tests for date
* Also check to correct coercion of dates
* Better check
* kick ci
* Format code
---------
Co-authored-by: Romeo Van Snick <romeo@romeovansnick.be>
* Bump org.apache.poi/poi from 5.4.0 to 5.4.1 (#56355)
* Bump org.apache.poi/poi from 5.4.0 to 5.4.1
Inspect dependency changes here:
* Update deps.edn
---------
Co-authored-by: Metabase bot <metabase-bot@metabase.com>
Co-authored-by: Cam Saul <1455846+camsaul@users.noreply.github.com>
* Bump cider/cider-nrepl from 0.55.4 to 0.55.7 (#57576)
Inspect dependency changes here: https://github.com/clojure-emacs/cider-nrepl/blob/v0.55.7/CHANGELOG.md
Co-authored-by: Metabase bot <metabase-bot@metabase.com>
Co-authored-by: Cam Saul <1455846+camsaul@users.noreply.github.com>
* Bump org.postgresql/postgresql from 42.7.4 to 42.7.5 (#57314)
Inspect dependency changes here: https://github.com/pgjdbc/pgjdbc/blob/REL42.7.5/CHANGELOG.md
Co-authored-by: Metabase bot <metabase-bot@metabase.com>
Co-authored-by: Cam Saul <1455846+camsaul@users.noreply.github.com>
* Bump clj-kondo/clj-kondo from 2025.02.20 to 2025.04.07 (#56650)
* Bump clj-kondo/clj-kondo from 2025.02.20 to 2025.04.07
Inspect dependency changes here: https://github.com/clj-kondo/clj-kondo/blob/v2025.04.07/CHANGELOG.md
* Add kondo ignore to tests for new :missing-protocol-method linter
---------
Co-authored-by: Metabase bot <metabase-bot@metabase.com>
Co-authored-by: Mike Appleby <86076+appleby@users.noreply.github.com>
Co-authored-by: Cam Saul <1455846+camsaul@users.noreply.github.com>
* Bump thheller/shadow-cljs from 2.28.21 to 3.0.4 (#57585)
Inspect dependency changes here:
Co-authored-by: Metabase bot <metabase-bot@metabase.com>
Co-authored-by: Cam Saul <1455846+camsaul@users.noreply.github.com>
* Remove unused code in embedding data picker (#57700)
* Remove unused exported components
* Remove references to metrics
* Remove references to saved questions
* Remove irrelevant tests
* Remove unnecessary prop
* Fix more unit tests
* Remove more unnecessary code
* Fix failed unit tests
* Remove unnecessary prop
* speed up notification tests (#57763)
* Update logger presets (#57667)
* Update presets
* Update presets
* Update test
* Apply suggestions from code review
Co-authored-by: appleby <86076+appleby@users.noreply.github.com>
---------
Co-authored-by: appleby <86076+appleby@users.noreply.github.com>
* better snowplow errors when expecting the wrong number of good events (#57730)
* add ?. to make it more resilient
* add more logssss
* cy.log
* some nicer error message
* fix eventType
* removes leftover lines
* typos on comments
* Update e2e/support/helpers/e2e-snowplow-helpers.js
Co-authored-by: Phoomparin Mano <poom@metabase.com>
---------
Co-authored-by: Phoomparin Mano <poom@metabase.com>
---------
Co-authored-by: Romeo Van Snick <romeo@romeovansnick.be>
Co-authored-by: Raphael Krut-Landau <raphael.kl@gmail.com>
Co-authored-by: Cam Saul <1455846+camsaul@users.noreply.github.com>
Co-authored-by: Ryan Laurie <30528226+iethree@users.noreply.github.com>
Co-authored-by: Sloan Sparger <sloansparger@gmail.com>
Co-authored-by: Alexander Polyankin <alexander.polyankin@metabase.com>
Co-authored-by: Ngoc Khuat <qn.khuat@gmail.com>
Co-authored-by: Nemanja Glumac <31325167+nemanjaglumac@users.noreply.github.com>
Co-authored-by: Uladzimir Havenchyk <125459446+uladzimirdev@users.noreply.github.com>
Co-authored-by: metamben <103100869+metamben@users.noreply.github.com>
Co-authored-by: Tyler Plude <tsplude@gmail.com>
Co-authored-by: Phoomparin Mano <poom@metabase.com>
Co-authored-by: Mihael Konjević <konjevic@gmail.com>
Co-authored-by: John Swanson <john.swanson@metabase.com>
Co-authored-by: Aleksandr Lesnenko <alxnddr@gmail.com>
Co-authored-by: Thomas Schmidt <thomas@metabase.com>
Co-authored-by: Mahatthana (Kelvin) Nomsawadi <me@bboykelvin.dev>
Co-authored-by: github-automation-metabase <166700802+github-automation-metabase@users.noreply.github.com>
Co-authored-by: Nicolò Pretto <info@npretto.com>
Co-authored-by: Aleksandr Lesnenko <alxnddr@users.noreply.github.com>
Co-authored-by: Phoomparin Mano <phoomparin@gmail.com>
Co-authored-by: Sloan Sparger <sloansparger@users.noreply.github.com>
Co-authored-by: Metabase bot <metabase-bot@metabase.com>
Co-authored-by: lbrdnk <lbrdnk@users.noreply.github.com>
Co-authored-by: Adithya Vardhan <imadithyavardhan@gmail.com>
Co-authored-by: Edward Paget <edpaget@users.noreply.github.com>
Co-authored-by: Mike Appleby <86076+appleby@users.noreply.github.com>
* add ?. to make it more resilient
* add more logssss
* cy.log
* some nicer error message
* fix eventType
* removes leftover lines
* typos on comments
* Update e2e/support/helpers/e2e-snowplow-helpers.js
Co-authored-by: Phoomparin Mano <poom@metabase.com>
---------
Co-authored-by: Phoomparin Mano <poom@metabase.com>
* Empty
* kick ci
* Allow temporal type in addition to string for `date()`
* Add date(datetime) test
* Use cast for date for postgres
* Extend date() on mysql
* Extend date() on bigquery
* Extend date() at snowflake
* Extend date() to datetime on clickhouse
* Generalize `date()` to :sql
* Update changelog with `date(datetime)`
* Update helper text string for date
* Add e2e tests for date
* Also check to correct coercion of dates
* Better check
* kick ci
* Format code
---------
Co-authored-by: Romeo Van Snick <romeo@romeovansnick.be>
This commit adds analytics to the:
1. New button being clicked
- In the top app bar
- From the empty collection
2. New menu individual items being clicked:
- Dashboard
- ~Question~ (not possible at the moment)
- ~Native query~ (not possible at the moment)
3. An entity creation being initiated
- Model from the /browse/models page
- Metric from the /browse/metrics page
- Collection from the collection header
- Collection from the main nav sidebar
* update components in frontend/src/metabase/admin/people/components to use Mantine components and remove deprecated component usage
* update people and group pages/modals to use more modern components
* use relative sizes
* fix user success modal not closing on "Done" confirm button click
* remove CS from PeopleListRow
* clean up admin/people/colors.ts
* lint fix
* people list footer fix
* pr feedback
* wip
* fix issues, people page updates, some clean up
* add unsaved files
* type fix, fetch user list from rktquery for PeopleList
* remove entity usage from unsubscribe modal, lint fix
* remove all remaining usages of user entity
* remove user entity
* test fixes, remove unused user schema
* bit of clean up
* qa pass - invalidate groups cache if user is deactivated/reactivated
* address pr feedback
* update components in frontend/src/metabase/admin/people/components to use Mantine components and remove deprecated component usage
* update people and group pages/modals to use more modern components
* use relative sizes
* fix user success modal not closing on "Done" confirm button click
* remove CS from PeopleListRow
* clean up admin/people/colors.ts
* lint fix
* people list footer fix
* pr feedback
We don't support the `zh` (Traditional Chinese) locale anymore – only `zh-CN` (Simplified Chinese), and `zh-HK` (Hong Kong), and `zh-TW` (Taiwan). Here's a [Slack discussion](https://metaboat.slack.com/archives/C6UBFU9GW/p1741983889165599).
So let's remove `zh` from the list of supported translations.
For some reason the list wasn't sorted alphabetically by language name - this PR fixes that too.
* convert UnsubscribeUserModal and UnsubscribeUserForm components
* convert UserActivationModal to TS
* convert UserPasswordResetModal to TS
* convert UserSuccessModal to TS
* convert PeopleList and PeopleListingApp components to TS
* convert PeopleListRow to TS
* convert use-people-query to TS
* fix deprecated usages of google_auth and ldap_auth with sso_source field
* fix lint
* convert GroupsListing, GroupsListingApp, and AddRow components to TS
* convert lib/groups, AdminContentTable, and AddMemberRow to TS
* convert GroupDetailApp and GroupDetail to TS
* fix import
* convert GroupSummary to TS, make adjustments to other converted files to make types happy
* remove people/components/SearchInput and convert usage to mantine component to avoid TS migration
* remove styled component usage from people and groups pages and components
* adjustment to people listing app
* refactor: rename test file to tsx to have hopefully better diff in the
next commit
* fix EMB-360: Make Mantine components use the locale from LocaleProvider
* close EMB-373: make useLocale return the locale from LocaleProvider if
present
* Use Mantine dropdown for snippet menu
* Fix search icon in snippet header too
* Remove default ICON_SIZE
* Fix MIN_SNIPPETS_FOR_SEARCH
* Add createSnippet helper
* Add repro for metabase#57441
* Add test for snippet search
* Move snippet helper to api
* Use actual ref for input
* Simplify header search rendering
* Use SidebarHeader for snippet sidebar
* Use Flex for most components
* Simplify padding
* Use class component for archived snippets
* Use class for header button
* Use direct properties
* Rely less on custom css
* Fix header padding
* Fix e2e tests
* Remove unused eslint-disable
* Add type annotation
Co-authored-by: Kamil Mielnik <kamil@kamilmielnik.com>
---------
Co-authored-by: Kamil Mielnik <kamil@kamilmielnik.com>
* Move SimpleDataPicker to a new directory preparing for new components
* Use the embedding data picker from a new folder
This requires turning the component into a plugin, which we'll do in the
next commit.
* Move plugin property to a more appropriate place
* Implement embedding multi-stage data picker as an EE plugin
* Add DataSourceSelectorProps
* Remove reference to query builder data picker
* Fix failing unit tests
* Move embedding data picker components to a proper location
* Update Appearance Settings
* unit test additions + updates
* update e2e tests
* test updates
* unit test fix
* fix lint errors
* obey the linter
* add test coverage
* add more tests
* support env var override ui
* remove dead code
* nicer routing
* useDisclosure
* stabilize test
* adding watermark for dynamic viz
* starting to put together e2e tests for watermarks
* adding unit test
* adding loki tests
* adding more e2e tests
* changing test id
* adding e2e tests on chart interactivity
* fixing type error
* Split up config into categories
* Break out clauses into groups
* Add duplicate name check
* Allow setting shared options
* Add category to mbql clause config
* Rename groups
* Use enum for category
* Build examples using helper functions
* Build ExpressionParts directly in helper-text
* Flatten examples
* Wrap example in function
* Enable late-binding for args
* Unwrap example
* Remove get name and get description wrappers
* Remove getHelpDocsUrl helper
* Simplify example fn
* Add missing export
* Remove getFunctionByStructure
* Simplify help text helpers
* Rename structure to displayName
* Make args required
* Manually type clause definition type
* Merge HELPER_TEXT_STRINGS into config
* Move help text helpers to separate file
* Use data from MBQL_CLAUSES instead of HELPER_TEXT_STRINGS
* Allow defining stuff in config
* Move help text into config
* Remove fallback to HELPER_TEXT_STRINGS
* Remove helper-text-strings
* Allow more stuff to be optional
* Build clause in getClauseDefinition
* Avoid self-referential types
* Use getClauseDefinition in completer
* Handle new args definition
* Handle null instead of undefined
* Enforce type on all argument definitions
* Enforce name on all argument definitions
* Get clause defns
* Fix types
* Simplify tests
* Remove unnecessary guard
* Avoid creating unnecessary functions
* Move defineClauses to separate module
* Remove any in config
* Rename helper-text-strings test to help-text
* Implement Tenant Groups
- normal users cannot be added to a tenant group
- tenant users cannot be added to a normal group
- tenant groups cannot receive write access to any collection
- tenant groups cannot receive any access to the audit collection
- tenant groups cannot receive any application permissions
* Add a setting to turn tenants off and on
When the setting is turned on:
- create the All External Users group, with
`magic_group_type=all-exsternal-users` and `is_tenant_group=true`
When the setting is turned off:
- make sure there are no remaining tenants or tenant users
- delete the All External Users group
* Remove the need for memoized `tenant?` functions
* Fix tests
* allowe using with-temp with perms group membership
* fix more tests
* remove unnecessary bindings
* Put setting behind a feature flag
* Don't read setting from env
closes adm-648
closes adm-657
closes adm-649
Resolves PRG-25
This PR adds a new button to the collection header menu. The button simply opens a new collection modal (lets you create a new collection). There are two important things to note about this change:
1. This button shows only when a user has a curate permissions for the current collection
2. This change required adding a new icon to our design system, which was approved by Maz.
Resolves PRG-20
This PR replaces the old `...` menu next to the "Collections" sidebar header. As a reminder, this menu was previously triggering a popup with two items:
1. New collection
2. Other users' personal collections (in case the user is an admin)
We're now improving the UX and replacing this menu with two distinct items:
1. The `+` icon that creates a new collection directly
2. The new individual sidebar link that leads to other users' personal collections (again, only if admin)
* update title sizes to match figma - add multi line title stories for lineheight examples
* little more line height in modal titles
* little fixes
* more little fixes
* reduce smartscalar title size
* Update Loki Snapshots
---------
Co-authored-by: Ryan Laurie <iethree@gmail.com>
Co-authored-by: Metabase Automation <github-automation@metabase.com>
* fix: add table-level sql based permission query
Some places in our code we may be returning fewer tables than a user has
access to because we load a limited number of models from the database
then apply permissions filters to those. This provides a subquery based
check for table permissions that we can use in other queries to do the
same kind of test for table access that we would do on the clojure-side.
Co-Authored-By: Nathan Voxland <nathan@metabase.com>
* fixup! restore check-permissions-for-model impl
* Update src/metabase/models/table.clj
Co-authored-by: Noah Moss <32746338+noahmoss@users.noreply.github.com>
* fixup! caluse -> clause spelling
* fixup! reformat malli annotations; add docstring
* fixup! add Literal type to honeysql helpers
* fixup! restore accidentally deleted clause
---------
Co-authored-by: Nathan Voxland <nathan@metabase.com>
Co-authored-by: Noah Moss <32746338+noahmoss@users.noreply.github.com>
* fix: filter button labels
* fix: part of EMB-335, uiName: t => getUiName: () => t
* fix linting issue
* fixed unit test mock missing getUiName
frontend/src/metabase/visualizations/components/Visualization/Visualization.unit.spec.js
missing
* make application-permissions.cy.spec.js pass
* cursor says this could fix the issue
* harder vibe coding
* remove handling of made up by llm elements that do not exist
---------
Co-authored-by: Aleksandr Lesnenko <alxnddr@gmail.com>
* Rename ALL_NODES to NODE_TYPE
* Remove unnecessary name override
* Remove unused EXPRESSION_OPERATOR_WITHOUT_ORDER_PRIORITY
* Use precedence from pratt compiler
* Move EDITOR_FK_SYMBOLS to identifier module
* Lift const descriptor
* Remove unused characters config
* Remove identifierAlwaysQuoted from EDITOR_QUOTES config and make the behaviour the default
* Move identifierDefaultQuote out of config and hard-code the behaviour
* Flatten default quotes config and move default into string module
* Remove wrong comment
* Use parseOperatorType to find operator type
* Remove separate operator groups
* Remove unused FUNCTIONS variable
* Remove barely used getExpressionName
* Use Boolean token to parse booleans
* Remove True and False from operators
* Rename Star and Slash to Multiply and Divide
* Remove FIELD_MARKERS
* Simplify config definition
* Inline getDescriptionForNow
* Move OPERATOR check to formatter
* Use normal quotes in config
* Split up MBQL_CLAUSES to avoid duplicating names
* Split clause helpers into separate files
* Fix sort order in completion tests
* Move operators to a separate group in config
* Use EXPRESSION_OPERATORS in formatter
* Export using star
* Rename OPERATOR to PUNCTUATOR
* Rename parseOperatorType to parsePunctuator
* Rename parseOperatorType to parsePunctuator
* Add helper for potential function prefixes
* Remove unused OPERATORS
* Remove superflouous helper
* Inline formatCallExpression
* Use Lib.ExpressionOperator where possible
* Remove impossible case for expression
* Inline isLogicOperator
* Rename variable for clarity
* Deduplicate helpers
* Move formatter helpers into utils
* Simplify helper
* Fix import order
* Remove reliance on PUNCTUATOR type
* Fix variable name
* Simplify punctuator definition
* Clean up lexifier
* Flatten lexifier results
* Remove errors for parser results
* Flatten parser result
* Simplify parser tests
* Remove cast
* Remove unused import
* Translate all error messages in formatter
* Simplify type
* Use sentence case name for type
When making sure stuff can load even with a loop, the circuit breaker was removed, leading to endless loops. That
would happen if the data causing the loop wasn't stripped properly. Right now if stripping things once did not help
and the entity is in the pipeline again, the process will break (with hopefully enough data to detect why).
This also changes the logic from exception to just recursive calls. Plus it's back at skipping all dashcards in a
Dashboard instead of just skipping visualization logic, which should make for a way faster processing (less deep for
sure). Plus it tries to improve logging a bit so it's easier to understand what's going on in complex cases.
* Remove Legacy Metrics code from the backend
* Code cleanup
* Test fix 🔧
* Test fix 🔧
* PR feedback
* Extra test
* Revert some X-Rays changes
* Remove commented-out test
* Test fixes 🔧
* feat: add prometheus metric for quartz task states
Adds a new quartz_task_state metric that will report to prometheus the
states of all quartz jobs running on a metabase instance using a guage.
Each task state is reported as a label and we also report a state for
tasks that are executing.
All hosts on a metabase cluster _should_ report the same value for these
metrics if they are scraped at the same time, and we need to consider if
the additional DB load is okay since this does query the database during
a prometheus scrape to report this metric.
* fixup! add job-name tag and histogram of job time
* try to fix cypress download format flakes
* handle unset last_table_download_format
* more test tweaks
* make tests shorter, refactor, and add wait periods
* rearrange more code
* remove the waits, quarantine tests as flaky
* move cy.intercept to right before reload
* skip tests again
* unravel .then calls
* feat: Implement multi-catalog for databricks
* Fix schema for nil
* Add tests for joins on multi-catalog
* Fix multicatalog driver name
* Remove require
* Alphabetize drivers
* Add changelog
* Fix tests
* Fix tests
* Do not sync system catalogs
* Address review comments
* Update connection properties
* Update example to make clear its a schema
* [SEM-219] Humanize error message even more
To provide the customized error message, the type check built into the defendpoint framework has to be disabled, which
results in less specific API documentation.
(Malli has an experimental feature we currently don't use for looking up humanized errors from parent schemas, that
would let us configure the error message AND keep the input type information.)
* Test comparing case vs filtered ags
* Add test and update case definition
* Add logging to check CI, mark as driver ns
* Remove metadata to see which driver tests run
* add failing op for some drivers
* Use case to filter metric ag
* Filter in case
* Existing tests update
* WIP Respect expression deduplication
* Move transformation right after metric fetch
* Update test
* Aggregation maps to be used in tests
* Simplify metric transform
* New tests
* Cleanup
* Cleanup
* Cleanup
* Fix equality
* Cleanup
* Cleanup equality
* Simplify test
* Tolerate float deviation
* Feature dependent aggregations
* Update test
* Fix value formatting
* Add var to std dev ags
* Update the filtering logic
* Speculate
* Adjust tests
* Update test
* Kondo
* Adjust test for starburst
* Remove debug def
* Apply suggestions from code review
Co-authored-by: metamben <103100869+metamben@users.noreply.github.com>
* Adjust naming
* Comment
* fresh-uuids into definition
* Add shortcut for empty condition
* Adjust meta and case default
* Remove now redundant condition
* Docstring
* Indentation
* Remove dev leftover
* Remove dev leftover
* Update tests and distinct-where
* Nullary aggregations map
* Distinct-where feature
* Fix formatting
* Adjust e2e test
---------
Co-authored-by: metamben <103100869+metamben@users.noreply.github.com>
* convert UnsubscribeUserModal and UnsubscribeUserForm components
* convert UserActivationModal to TS
* convert UserPasswordResetModal to TS
* convert UserSuccessModal to TS
* convert PeopleList and PeopleListingApp components to TS
* convert PeopleListRow to TS
* convert use-people-query to TS
* fix deprecated usages of google_auth and ldap_auth with sso_source field
* fix lint
* convert GroupsListing, GroupsListingApp, and AddRow components to TS
* convert lib/groups, AdminContentTable, and AddMemberRow to TS
* convert GroupDetailApp and GroupDetail to TS
* fix import
* convert GroupSummary to TS, make adjustments to other converted files to make types happy
* remove people/components/SearchInput and convert usage to mantine component to avoid TS migration
* Add float cases to integer cast testing
* Update types of integer mbql expression
* Better messages for tests
* Add rounding test and specialize for redshift
* Don't use extreme values for redshift
* Add e2e tests for float -> integer casts
* Update helper text for integer
* Change test message.
Co-authored-by: appleby <86076+appleby@users.noreply.github.com>
* Helper function to convert to integer
* Appease the kondo
* More accurate comment
* Update arithmetic.cljc
* Refactor to use better ->integer implementation for SQL dbs
* kick ci
* Missed an argument
* Round properly for inline constants
* Add String->Integer and Float->Integer coercion strategies
* Add MongoDB string->integer and float->integer coercions
* Fix types in coercion test
* Also cast in mongodb coercion
* Use NUMBER(p) type in Oracle
* Spark SQL needs explicit round()
* SQLite needs explicit round()
* SQL Server needs explicit round()
* Clickhouse cast to float first
* SQLite needs to cast to float first so it can round
* SQL Server needs to cast to float so it can round
* Mongo needs to cast to float before rounding
* Spark SQL needs to cast to float so it can round
* Revert converting multimethod ->integer to defn
* Remove cast-integer multi-method and restore ->integer
* Update `integer-dbtype` in changelog
* Fix oracle integer type
* Fix spark sql casting
* Databricks driver needs to round
* kick ci
* Use ->integer in coercion
* Helper function for cast to integer with round
* kick ci
* Remove unnecessary ->float implementation
* Fix cljfmt precommmit hook
* Fix clickhouse returning bigdecimal
* Handle raw values in untyped-inline-value
* This came in with the merge
* Allow integer(float) and integer(string)
* Remove stray println
* Work for known types only
* Test for converting literals at compile time
* Update mysql
* Fix timezone test failing on null week-of-year
* Update fe test for float coercion
* Doh! Typo.
---------
Co-authored-by: Romeo Van Snick <romeo@romeovansnick.be>
Co-authored-by: appleby <86076+appleby@users.noreply.github.com>
* add viz setting fe
* ClJS changes
* Fix up some BE tests
* fix last test
* spec, hint
* specs
* review
---------
Co-authored-by: Noah Moss <noahbmoss@gmail.com>
Avoid the possiblity of deadlocks during cached query processing by no
taking the connection using with-connection and instead just selecting
the cached result using toucan.
I _think_ this connection holding was introduced under the assumption
that we could 'stream' a binary column from postgresql using the
.getBinaryStreaming jdbc method. However postgres requires binary
columns to be full read and written and so we are unable to stream
results from that column, so this operation ends up being the same as
just querying the column normally.
Because we are not streaming the result in it seems safe to just yeild
the connection back to pool so it can shared with other requests.
* initial implementation
* e2e test
* tweaks
* fix e2e test + refactor
* reduce unneeded comments
* clean ns
* fix unit tests
* fix e2e test
* disable userKV when no user is logged in
* fix up a number of unit tests
* refactor out BaseQuestionDownloadPopoverProps with two wrapper components to avoid using the user kv hook in public/embedded contexts
* fix up types
* fix up e2e test
* use ?. syntax
* hide email domains for users without settings access
* fix test
* fix up e2e test
* update BE setting visibility
* Revert "update BE setting visibility"
This reverts commit b3728ad21d.
* remove commented-out line
* fix: scheduler state management in cloud migration
When we create the h2 database to dump a metabase instance to during
cloud migration we need to stop and start the quartz scheduler instance
running on the instance during the migration.
Some liquibase migrations handling quartz job changes expect to control
the scehduler and start/stop it while those migrations run. Recent
changes to guard against this behaviour in testing case these
migrations to throw an exception when they run during the cloud
migration task.
This also fixes an older bug that the quartz scheduler would remain off
on the instance being migrated after the cloud migration task finishes.
* fixup! stop scheduler after all calls to migrate!
* fixup! formatting
* fix: wrong syntax on `if:` was making 'cross-version-for-breaking-changes'
always run
* Update .github/workflows/embedding-sdk.yml
Co-authored-by: Phoomparin Mano <poom@metabase.com>
---------
Co-authored-by: Phoomparin Mano <poom@metabase.com>
* Rename OPERATORS in tokenizer to FIELD_PUNCTUATORS to avoid confusion with expression operators
* Rename Reference to Field in tokenizer for consistency
* Allow specifying the string delimiter to unquote
* Reduce state in tokenizer
* Allow the tokenizer to parse field tokens that are missing an opening bracket
* Add more comments and rename variable for clarity
* Implement proper escaping in field tokenizer
* Remove duplicate OPEN_BRACKET from punctuators
* Simplify lexifier now that tokenizer returns the correct token for fields
* Reorder loop to reduce duplication
* Simplify lexifier now that tokenizer returns the correct token for fields
* Add unit tests for unquoting with specific character
* Make invalid token error message more precise
* Punctuation
* Typo
Co-authored-by: Kamil Mielnik <kamil@kamilmielnik.com>
---------
Co-authored-by: Kamil Mielnik <kamil@kamilmielnik.com>
Resolves PRG-1 by introducing the branding content to the dashboard PDF exports
Resolves PRG-7 by making the branding content clickable
Additionally resolves PRG-14
* convert UnsubscribeUserModal and UnsubscribeUserForm components
* convert UserActivationModal to TS
* convert UserPasswordResetModal to TS
* convert UserSuccessModal to TS
* convert PeopleList and PeopleListingApp components to TS
* convert PeopleListRow to TS
* convert use-people-query to TS
* fix deprecated usages of google_auth and ldap_auth with sso_source field
* fix lint
* float() tests + tier 1 (postgres)
* Add more parsing test.
* Better float comparison
* Add expression/float feature
* Add float function to config
* Add float help text
* Add e2e test for float()
* Add `:expressions/float` to list of feature flags
* Only allow casting text to float (not number)
* Better test message
* Refactor for nicer code
* `float()` for snowflake (#56927)
* Implement float() for snowflake
Also add text and date feature keys because somehow they were left off.
* Don't use Long/MAX_VALUE in tests because it breaks some dbs.
* Fix copy-paste error
* kick ci
* Implement float() for mysql (#56919)
Also, add :expressions/date and :expressions/text features that were missing somehow.
* Cast and split custom expression functions for Clickhouse (#56924)
* Implement integer() for clickhouse
ClickHouse is recently tier 2!
* Implement float() for clickhouse
* Implement text() for clickhouse
* Add date() to clickhouse
* Implement splitPart() for clickhouse
* Cast to Date32 after parsing date
* Use a nullable type
* Use nullable types for casts
* Try parseDateTime instead to avoid null
* Use case sensitive function names
* Fix argument order of splitByString()
* Don't use Long/MAX_VALUE in Clickhouse
* Clickhouse needs a lot of type hints!
* Cast position to int
* Better tests for split part
* Negative indexes should return empty string
* kick ci
* Convert index to long (for ClickHouse)
* `float()` implementation for BigQuery (#56920)
* Add float() support for bigquery.
Also add features for date and text that mistakenly got left out.
* Correct argument order for parse_date
* Refactor ->float
* Remove cast-float implementations
* Reinstate Clickhouses' cast to float function.
* Remove specialized :float functions for sql drivers
Replace them with a single :sql implementation
* kick ci
* More refactoring to get rid of questionable recursion
* Fix three tests
* Add null case for literal ->float compilation
* Add type info
* Revert making typed? public (don't need it no more)
* Revert converting multimethod ->float to defn
Also, use a new function called `coerce-float` that does the work.
* Try clickhouse using cast() instead of toFloat64()
* Revert presto jdbc ->float implementation
* Use keyword type name for float-dbtype
* Revert removing ->float method from bigquery clickhouse and starburst
* Add `float-dbtype`, `cast-float`, and `:expressions/float` to changelog
* Remove cast-float multimethod and redundant ->float methods
* REmove mt/dataset test-data from split-part tests.
Is it really necessary?
* Revert qpt changes
* Properly handle NULL in clickhouse cast to float
* Fix presto and starburst
With an improved safe-denominator function
* Change to double as default sql type for casting
* Fix test for casting to double
* Use ->float instead of coerce-float for coercion
* Use DOUBLE PRECISION for Vertica driver
* Use BINARY_DOUBLE for oracle
* Use float for sqlserver
---------
Co-authored-by: Romeo Van Snick <romeo@romeovansnick.be>
and use it as the source of truth for whether a group is the "All Users"
or "Administrators" group, instead of just the name.
This is a unique field, but it's nullable. H2, MariaDB, MySQL, and
Postgres all treat `null` values as unequal, so it's fine to have many
permissions groups without a `magic_group_type`, but any row *with* a
`magic_group_type` must be distinct from the others. In other words, we
can have lots of non-magic groups and exactly one magic group of each
type.
For now, we'll just have the same Administrators and All Users groups
and the names won't change. At some point, when tenants are enabled,
we'll want to rename "All Users" to "All Internal Users" and create a
separate "All External Users" group.
Fixes QUE-989, QUE-905.
Stop backfilling idents in `result_metadata` on reading cards. That's costly and recursive, and was the source
of the perf issues on stats.
We're planning instead to support optional, best-effort idents for a longer period, and let the background job do
the work.
Fixes QUE-904.
With (1) a fresh column like an aggregation, and (2) when the source was a model:
`annotate` was wrapping the column's ident as if it came from the source model, but that's incorrect.
This adjusts the order of operations so that only columns inherited from the source get their idents
wrapped with the source model.
* Updating text wrap and minibar logic to gracefully handle structure of column widths; updated fallback column width when no width is present in viz settings
* removing inline defs
* updating shortcuts, adding shortcut in profile menu, add hotkeys to close shortcut modal and sidesheets
* updating tests
* no one will know
* updating to use mantine hooks
* teach claude how to write backend code
* Add solid instructions for calling into the repl
* teach it how to behave when there's no repl
* remove mcp stuff for now
* add lint pr quick command
* fix: Add missing docstrings in mage/be_dev.clj
Added docstrings to parse-int, nrepl-port, and nrepl-eval functions to fix clj-kondo linting warnings.
* flesh out Lint PR quick command
* feat: enable equals-true Kondo linter and fix warnings
This change enables the `:equals-true` warning in the clj-kondo linter configuration,
which identifies code patterns where `(= true x)` is used instead of the more idiomatic
`(true? x)` form in Clojure.
Key changes:
- Enable `:equals-true` linter with warning level in .clj-kondo/config.edn
- Fix all instances of `(= true x)` throughout the codebase, replacing with `(true? x)`
- Add a new utility to check balanced parentheses in Clojure forms
- Add a new mage command `-balance-parens-check` to validate balanced parentheses
- Update CLAUDE.md with documentation about the balanced parentheses utility
This change improves code quality by promoting idiomatic Clojure usage and adds
a helpful tool for ensuring proper Clojure syntax when editing code.
* fix: update code to resolve equals-true Kondo linter warnings
This commit addresses the newly enabled equals-true linter warnings by:
- Adding #_{:clj-kondo/ignore [:equals-true]} to are forms that legitimately need to test both true/false values
- Replacing (= true x) with (true? x) and (= false x) with (false? x) in test assertions where appropriate
🤖 Generated with [Claude Code](https://claude.ai/code)
Co-Authored-By: Claude <noreply@anthropic.com>
* fix more, refine claude instructions
* fix more = true -> true? linter errors automatically
* fix remaining (= true x) -> (true? x) linter warnings by adding clj-kondo ignores
We need the ignores for 'are' macro clj-kondo tests where (= expected (fn x)) is the idiomatic pattern.
🤖 Generated with [Claude Code](https://claude.ai/code)
Co-Authored-By: Claude <noreply@anthropic.com>
* move the linter to the things-we-use section
* update claude.md from github copliot's review
* balanced-parens? -> is-readable-code?
- Returns explicit errors, usually with line numbers.
* update claude.md (mostly formatting)
* Fix kondo-updated when there are no matching files
* Update CLAUDE.md to replace -balance-parens-check with -code-is-readable
* Refactor mage -eval to mage -repl with improved usability and reliability
The implementation uses eval and read-string strategically to control when and where code is evaluated:
1. The code from the command line must first be transported as a string to the running REPL server
2. read-string allows parsing the code without executing it
3. eval then executes the code in the correct namespace context
4. This creates a "REPL inside a REPL" where namespace switching and evaluation happen in a controlled environment
5. This approach prevents premature evaluation and ensures code runs in the intended scope
* -check-readable task naming
* cleanup -check-readable
* fix output + fix user.clj
* shuffle code around
* Update CLAUDE.md
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
---------
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
* Use Resolver type from resolver module
* Move getKindForType to resolver
* Inline compileDimension
* Simplify resolver and make data flow requirements more obvious
* Avoid fetching unused dimensions
* Cache metadata displayInfo while resolving
* Move compiler-based resolver tests to compiler
* Rename variables to be more generic
* Refactor resolver
* Merge ExpressionType and MBQLClauseReturnType
* Add tests for resolver
* Be more precise in the resolver error messages
* Be more specific about expected field type in errors
* Add tests for unknown dimensions
* Rename any_ to any
* Add tests for mixed case matching
* Allow matching column names case-insensitively
* Replace ResolveError with CompilerError in resolver
* Use expressionMode in tests
* Fix types
* Remove cast in test
* Add more comments
* Add e2e tests for field matching
* Fix lint
* Rename Field to column in resolver error messages
* Remove TODO's and replace them with comments
* Fix error message in e2e test
Resolves PRG-2
This commit adds Metabase branding to the PNG downloads
for instances that **do not** have the whitelabel feature flag.
Even though it is possible to download the dark mode PNG,
this commit does not fully resolve PRG-6 because it doesn't
turn the text white.
* Check all mismatched sibling tokens, not just missing comma's
* Rename siblings check to checkMismatchedSiblings
* Merge imports
* Also catch call expressions
result_metadata for a query on a model returns the underlying columns, not the model's columns. This was causing
drills to break in certain circumstances, because the query processor was trying to cast a particular value twice.
Making drills default to using the column from lib/returned-columns fixes the issue.
* update copy w/ description, update db routing docs link, impl prefill for new destination dbs, fix bug where editing db details would not let you tab through the form
* fixes
* make no user attributes error message link to people page, db routing section expanded by default if feature enabled
* pull out fix for edit db form not being tab-able
* fix failing tests
* copy feedback
* wip
* More robust checking for timeout exceptions
Overview:
We enumerate items in the database and then we want to see which ones
the user can actually run queries against. To do this we run a query
like `select false from TABLE where 1 != 1 limit 0`. The hope is that
the query analyzer realizes it only needs to do permission checks and
doesn't try to satisfy these conditions (either because limit says no
rows, or 1 != 1 is easily proven to filter all rows). So we _usually_
get very quick feedback if we can select from the table or not.
Snowflake seems to not do this on views. If a view takes a long time to
query, snowflake seems to take a long time to query. The original fix
was to add a 15 second timeout to the query. The expectation here is
that Snowflake can quickly fail if there are no permissions for the
table or view, but might take a long time to run the probe query. But we
can interpret a timeout as a successful permission check.
The original way we checked for timeouts was a very naive check for
`java.sql.SQLTimeoutException`. It seems basically no one uses this when
you do:
```clojure
(with-open [stmt (...)]
(doto stmt
(.setQueryTimeout 15)
(.execute)))
```
For instance, here's some logs from Snowflake:
```
WARN net.snowflake.client.core.StmtUtil Cancelling query 0e8987ee-2d22-40f9-b30d-0129d38ce976 with reason TIMEOUT
INFO metabase.driver.sql-jdbc.sync.describe-database <schema>:<table-name>: Assuming no SELECT privileges: caught exception,net.snowflake.client.jdbc.SnowflakeSQLException: SQL execution canceled, at ...
```
Note It's a regular `net.snowflake.client.jdbc.SnowflakeSQLException`.
Here's a true permissions failure for comparison:
```
sync.describe-database <schema>:<table>: Assuming no SELECT privileges: caught exception,net.snowflake.client.jdbc.SnowflakeSQLException: SQL compilation error:,Failure during expansion of view 'SOME_VIEW': SQL compilation error:,Object 'DB.SCHEMA.TABLE' does not exist or not authorized.,
```
So we use some regex checks against the messages to try to detect when
we have a permissions failure (cannot select) vs a timeout of the
probe (presumably can select).
And a reminder of the balances here: it's better to show someone a table
that they get an error if they try to query than to hide a table that
they can query. So this change should bias towards reporting can-select
in cases where it might not be clear.
* Only run tests when classes are present
if you don't have :drivers on classpath it would fail
* ensure boolean is returned
tests assert false? on a nil
* CI fails based on the indentation of comments
* empty commit for CI
* Move postgres test into postgres test namespace
* Replace ex-msg matching with sqlerrorcode/sqlstate
Use jdbc error codes and sql state matching to figure out if a
SQLException is caused by user action canceling the request (essentially
how setQueryTimeout works). This adds a multimethod dispatching off the
driver keyword to match up either the SQLState code or the SQLErrorCode
with the appropriate value for a given database driver. Some of these
were found by looking how QueryTimeoutException is implemented in
Hibernate and others were found from JDBC driver source code or documentation.
* fixup! handle multiple codes for mariadb
* fixup! add partial query timeout test
* fixup! split apart sql timeout tests
* fixup! use correct sqlstate for snowflake timeout
* fixup! remove split out tests
* fixup! remove when-class
* fixup! snowflake sql state
* fixup! move query-canceled? into driver code
Removes the jdbc-exceptions namespace and instead makes query-canceled a
driver multimethod that extracts and dispatches on the SQLException
class within the sql-jdbc driver.
Non-jdbc drivers can implement this method, but it is only used for
handling errors in the cluster lock using the appdb and for the sql-jdbc
describe-database sync task.
Co-Authored-By: Nathan Voxland <nathan@metabase.com>
* fixup! update driver changelog add default method
* fixup! merge mysql/mariadb methods
* fixup! add added metadata
* fixup! make test throw instead of is false
---------
Co-authored-by: Edward Paget <ed.paget@gmail.com>
Co-authored-by: Edward Paget <edpaget@users.noreply.github.com>
Co-authored-by: Nathan Voxland <nathan@metabase.com>
* Fix deleting a DB with database routing on
* Fix database sync with a user attribute set
When an admin has the relevant user attribute set and tries to sync the
database schema, we use `(with-database-routing-off ...)` to explicitly
turn off database routing for the purposes of sync. However, there are
two problems here:
First, this is just informational for the purposes of error checking, it
didn't actually turn off database routing, it just made us throw an
exception if a driver touched a mirror database (in dev).
Second, during query processing, when we're actually doing the database
routing "swap" to replace the router with the mirror DB, we were doing
`with-database-routing-on`. This essentially means that if you do
```
(with-db-routing-off
(qp/process-query query rff))
```
you *don't* actually turn off DB routing. 🤦
Both underlying issues get fixed by the same change: if database-routing
is OFF, don't DO any database routing in the QP middleware. This seems
more reasonable to me anyway.
* CR resp
* add ttag/no-module-declaration rule + eslint ignore comments
* change the reference to github issue instead of linear
* add new ignores after rebase
* remove formatting changes from eslint files
* fix eslint rule mising from a file
* convert one field to getter as that got added after I opened the codemod
pr and it's failing the check here
* more //eslint-disable-next-line
* Fix 66k statement params in sync-table-privileges and sync-fks
* Test for privileges
* Test privileges
* Update and test set-initial-table-sync-complete-for-db!
* Move :expression-literals aggregation tests into expressions_test.clj
These tests all use :basic-aggregations not :expression-aggregations, so move them into expressions_test.clj to
consolidate the :expression-literals tests in one place.
* Add clause with top-level boolean literal in case-with-literal-expression-test
* Prototype sharing CSV pivot export code with Excel instead of Excel native pivots
* Cleanup some code, fix reflection warnings
* clean ns
* halfway to getting native excel formatting working
* native formatting working, but no auto-sizing
* fix up formatting, get auto-sizing working (+ refactor)
* Fix BE tests
* remove an old test case no longer needed
* lint
* remove final broken tests which is no longer needed
* lint, and clean up/refactor CSV code a bit
* clean up/refactor XLSX code
* remove debug symbol
* fix whitespace
* address feedback
* Add missing t to error messages
* Pull out lexification from diagnoseSyntaxChecks
* Add hooks to compiler
* Define diagnose and diagnoseAndCompile using compiler hooks
* Add maybe helper to throw the first error
* Use maybe helper to make sure parsing and lexifying throws
* Remove throwOnError argument to parser
* Add error hook
* Use error hook to make lexifier and parser throw
* Pass token and source in object
* Return position information for mismatching parenthesis
* Add length to Token
* Handle unknown string quotes
* Simplify lexifier
* Move malformed number check to diagnostics
* Move malformed string check to diagnostics
* Move malformed field name check to diagnostics
* Move bad token check to diagnostics
* Stop returning errors from lexifier
* Merge nodes when unopened bracket occurs
* Add error helper for diagnostics
* Use error helper for diagnostics
* Remove unused helpers
* Remove unused ParseError
* Remove errors fuzz test
* Add default hooks to parser
* Remove maybe helper
* Split up checks into syntax and expression folders
* Format no-argument expression clauses with empty ()
* Suggest no-arg clauses with empty ()
* Remove fallback for non-arg functions in resolver
* Fix unit tests
* Fix e2e test
* Add test for no-arg suggestion completions
* Update examples in docs
* Render parens for no-arg clauses in helptext header
* Make function browser insert parens for no-arg functions
* Do not show help text when a function call is not made
* Add repro for metabase#55300
* Fix function browser e2e test
* Remove unnecessary branch
* Remove fallback for non-arg functions in resolver
* Add () to now in one more location in docs
* Fix unit tests
Fixes#56434.
A join clause's `:fields` might contain refs to columns that no longer exist on the joined table/card.
Before this change, a malformed column was being passed to `with-join-alias`, causing it to throw.
Now the bad fields are dropped from `returned-columns`, and if they all get dropped then it is treated as `:all`.
Resolves PRG-10.
This commit keeps the DOM node that we use for downloads
deterministic, while ensuring that we always target the correct
chart in the dashboard.
The same question can be added to the dashboard multiple times
so the only unique identifier we can use is the dashcard id.
But this brings us to the weird situation that the node we're sending
to the canvas to render before downloading is different when we're
downloading the chart from a dashboard vs when we're downloading
it directly from the question page.
This pseudo DOM code shows what we did before:
Dashboard:
```
dashcard-id <--- we were grabbing this
visualization-root
card-id
```
Question:
```
div
visualization-root
card-id <-- we were grabbing this
```
Compare that to what we're doing now
Dashboard:
```
dashcard-id
visualization-root
card-id <--- we are now grabbing this*
```
* but only when it is under the specific dashcard-id
Question:
```
div
visualization-root
card-id <-- we continue grabbing this like before
```
* Fix literal-expressions-inside-nested-and-filtered-aggregations-test
* Use mt/formatted-rows in order-by-integer-literal-expression-test
* Support selecting an empty string literal in select-clause-deduplicate-aliases
For drivers like Oracle that inherit from ::sql.qp.empty-string-is-null/empty-string-is-null, if a user adds an empty
string as a custom literal expression, that literal "" value will get compiled to nil, so ensure that
select-clause-deduplicate-aliases can handle such nil values in the :fields list.
Basically, we want to ensure that a query like
{:expressions {"empty": [:value ""]},
:fields [[:expression "empty"]]
:limit 1}
gets compiled to
SELECT NULL [AS] "EMPTY" ...
* Add sql.qp.expression-literals/boolean->comparison abstract driver
This is similar in spirit to the existing sql.qp.empty-string-is-null driver and can be used for drivers like oracle
and sqlserver that require booleans to be converted to comparison expressions in logical operators like :and, :or, and
:not, and in the top-level of a :filter clause.
* Enable :expression-literals for sqlserver
* Enable :expression-literals for oracle
* Review feedback: not -> nil?
Co-authored-by: metamben <103100869+metamben@users.noreply.github.com>
* Implement ->honeysql :sum-where and :distinct-where in terms of :case for :sql driver
Change the ->honeysql methods for :sum-where and :distinct-where to build an MBQL :case form and compile that, rather
than building a honeysql :case form directly. In this way, drivers that need special handling for :case will get
:sum-where and :distinct-where for free. Note that :count-where was already implemented this way. See, e.g. the
:metabase.driver.sql.query-processor.expression-literals/boolean->comparison abstract driver which converts booleans
in the case conditions to equivalent := expressions.
* Fix typo in test: case clause -> expression
* Rename boolean->comparison abstract driver to boolean-is-comparison
Rename
::sql.qp.expression-literals/boolean->comparison to
::sql.qp.boolean-is-comparison/boolean-is-comparison
This makes it clear that the namespace should only contain the abstract driver, and is also more consistent with the
existing driver in sql.qp.empty-string-is-null.
---------
Co-authored-by: metamben <103100869+metamben@users.noreply.github.com>
the view_count and last_view_at updates of report cards can deadlock
against each other, so introduce a shared keyword they can use to each
take a cluster lock when updating these values to prevent the deadlocks.
* Adjust missing base type logging for jdbc drivers
* Adjust base type logging for Athena
* Add schema and table to missing base type logging
* Comments
* Update test
* Drop only preceding nils
* Use Mantine instead of styled components
* Fix centering
* Add task and status params
* Add TaskStatusPicker
* Make picker clearable
* Add task picker
* Put filters and pagination together
* Translate task status
* Break down logic
* Move loading and error state
* Always render something
* Make task picker searchable
* Add empty state to the table
* Fix clear button being transparent
* Add useUrlState
* Use useUrlState in TasksApp
* Move UrlState to its own file
* Sort
* Rename file
* Rename
* Clean invalid query params on mount
* Prettier
* Use Action
* Ignore invalid pages
* Reuse variable
* Fix tests
* Add test
* Add label to clear button
* Remove redundant condition
* Add assertions
* Improve typing
* Be explicit about empty string handling
* Add sorting params to ListTasksRequest
* Add sorting params to url state
* Render icon for the first column
* Add SortableColumnHeader
* Extract TasksTable
* Move use-url-state to its own directory
* Add unit tests for useUrlState
* Add test
* Refactor
* Add more tests
* Fix missing import
* Add task endpoints mocks
* Add TasksApp tests boilerplate
* Add pagination test
* Add tests for filters
* Add more tests
* Tmp
* Make SortColumn generic
* Use SortingOptions
* Remove unused type
* Make BaseItemsTableProps non-generic
* Integrate sortingOptions
* Prevent CLS
* Prevent table content from disappearing on sort or filter
* Improve typing
* Use isLoading
* Get fresh data
* Update comment
* Reset page on sorting change
* Update tests
* Add tests for page reset
* Update tests
* Mark test as flaky
* Reuse SortingOptions
* Extract getFirstParamValue
* add tests for mb-wrapper leaks
* revert me - temporarily check diffs
* Revert "revert me - temporarily check diffs"
This reverts commit f474fe7b25.
* only apply mb-wrapper when in the sdk
* use cy.log to show e2e messages
---------
Co-authored-by: Mahatthana (Kelvin) Nomsawadi <me@bboykelvin.dev>
* Remove legacy expression in the return type from compiler
* Move test cases to formatter tests
* Move op and opt helpers to test utils
* Move value helper to test utils
* Rewrite compiler tests to use expression parts
* Remove unused test helpers
* Use actual invalid metadata instead of broken references
* Remove Lib.legacyExpressionForExpressionClause and Lib.expressionClauseForLegacyExpression
* Remove unused mock in fuzz test
* Remove legacy-expression-for-expression-clause and expression-clause-for-legacy-expression helpers
* Merge __support__ and test folders
* Reuse query from shared testing data
* Remove legacy-expression-for-expression-clause and expression-clause-for-legacy-expression tests
* Fix import
* Rename util to utils
* float() tests + tier 1 (postgres)
* Add more parsing test.
* Better float comparison
* Add expression/float feature
* Add float function to config
* Add float help text
* Add e2e test for float()
* Add `:expressions/float` to list of feature flags
* Only allow casting text to float (not number)
* Better test message
* Refactor for nicer code
---------
Co-authored-by: Romeo Van Snick <romeo@romeovansnick.be>
To avoid deadlocks on postgresql appdbs caused by overlapping ids in the
in clause of these updates.
I also had to change the `call-count` implementation for one of the
tests to spy on t2/query instead. The call-count pipeline middleware
will register either 2 or 4 depending on if we need to insert rows into
the lock table or not.
Fixes QUE-755
Since killing connections this way is the mechanism we use to terminate long running JDBC driver queries, this timeout
shouldn't differ from the db-query-timeout-minutes setting.
* Resolve dimensions while compiling
* Use resolver in pratt compiler instead of resolver pass
* Remove legacy resolver
* Add missing imports
* Add startRule to compiler in test
* Use helper to calculate argtype
* Provide a default implementation of argType
* Return expressionParts from compiler
* Convert resolver tests to use compiler
* Remove superflouous comment
* Convert MAP_TYPE to a function
* Rename field-resolver to resolver
* Simplify resolver check
* Move resolver-related code into resolver module
* pinned questions show moderated status, handle their own caption title
* unit tests, description
* fixing grid spacing
* e2e test adjustments
* fix for verified skeletons
* update quetion embed test, dashboard embed test
Fixes [QUE-701](https://linear.app/metabase/issue/QUE-701), [QUE-778](https://linear.app/metabase/issue/QUE-778), [QUE-777](https://linear.app/metabase/issue/QUE-777), [QUE-776](https://linear.app/metabase/issue/QUE-776), [QUE-703](https://linear.app/metabase/issue/QUE-703), [QUE-704](https://linear.app/metabase/issue/QUE-704).
### Description
Directly implements `:ident` calculation for legacy MBQL in the QP!
Also changes the spelling of compound idents to eg. `join[JoinClauseIdent]__InnerColumnIdent`, `native[CardEntityId]__COLUMN_NAME`, etc. The advantage of this is that NanoIDs cannot contain `[]`, whereas they do contain `_`s. Then we can make sure there's no missing `entity_id`s (a common issue) by looking for `native[]`. `native____` might have happened by accident is the NanoID started with two underscores.
This is an unfortunately large PR, but I don't think much can be done to help that.
The output of `annotate` becomes `Card.result_metadata`, which is a key **input** to
`annotate`. So we have to tackle legacy metadata end to end to produce a healthy PR.
(A prior attempt to use the lib and match up the results turned into a cluster of hacks,
and required half of these changes anyway, so this direct approach seems more sound.)
Most of this logic lives in the `annotate` post-processing middleware, with
some support from `add-source-metadata` pre-processing.
#### Idents
This generates correct idents for proper fields, expressions,
aggregations, breakouts, explicit and implicit joins, native queries and
models (native and MBQL).
#### Safety checks and Guarantees
Also adds and strengthens several safety checks in the library and
elsewhere. The guarantees I hope we can count on at this point are:
- Cards are saved with correct `:ident`s in their `:result_metadata`.
- Cards with old `:result_metadata` get (correct) `:ident`s added on read.
- Lib can now rely on getting correct idents for all input cards!
- Therefore the lib (BE + FE) can compute the ident for **every column**.
#### Placeholder idents
For native queries, we can only get `result_metadata` from executing them.
The usual flow is:
- Write the query
- Send it as an ad-hoc `/api/dataset` request
- Get back `result_metadata`
- Save it, passing the `result_metadata` back to the BE.
We need a card `entity_id` to create idents for a native query, but an ad-hoc
query doesn't have a card yet! The BE now generates a placeholder `entity_id`,
which gets replaced on save with the new card's actual `entity_id`. These
placeholders contain (1) illegal NanoID characters, and (2) a NanoID, which
means they cannot be mistaken for non-placeholder NanoID, and that they are
still globally unique, so there's no caching concerns.
* Move diagnostics to separate dir
* Use correct type for startRule arg
* Split up diagnostics into sepearate files per check
* Use throw for checks that don't yet
* Pull out syntax checks into separate function
* Split up lib.diagnoseExpression into helper
* Return expressionParts from compiler
* Add vist helper that walks Lib.ExpressionParts
* Convert check-known-functions to use Lib.ExpressionParts
* Convert check-supported-functions to use Lib.ExpressionParts
* Convert check-comparison-operator-args to use Lib.ExpressionParts
* Convert check-arg-count to use Lib.ExpressionParts
* Convert check-case-or-if-arg-count to use Lib.ExpressionParts
* Convert check-arg-validators to use Lib.ExpressionParts
* Remove legacy visitor
* Also pass expression clause to diagnostics
* Use pMBQL in diagnose-expression
* Stop passing expression to diagnostic checkers
* Rename visit to visitor
* replace the export button in dashboard header
* use the same download button style for both
* add styles for compact download button
* change the compact export button style
* add download button logic
* update data-testid
* update dashboard tests in public-resource-downloads
* use getDashboardCard helper instead
* move question result download to header
* remove PDF export localized test
* add PublicOrEmbeddedDashboardView loki diffs
* use text-primary instead of text-dark for download icons
* simplify downloadAndAssert embed logic
* simplify classnames merging
* revert ExportPdfMenuItem
* cleanup TitleAndDescriptionContainer
* revert LegendCaption
* show the header if headerButtons is not null
* standalone question download results only shows up on card hover
* always show header if there is a header button
* add floating headers when title is hidden
* add comment on floating header buttons
* use accessible label selectors instead of data-testid
* replace question-results-download-button testid with accessible label
* cleanup downloadAndAssert logic
* update tests
* support shared selector
* update unit test for PublicOrEmbeddedDashboardPage.enterprise
* update unit test for PublicOrEmbeddedDashboardPage.premium
* update unit test for PublicOrEmbedded
* update localization tests
* localize aria-label and re-add label tests
* make aria-label "Download as PDF" for all instances
* fix question tests on locale hash params
* fix tests on disabling downloads
* add realHover case to downloadAndAssert
* update testid in LegendDescriptionIcon
* update public-resource-downloads cypress test
* use visibility checks in tests
* restore ellipsis button size
* restore hover parent state to standalone question
* update localization tests that referenced PDF export button
* remove eslint-disable directives
* update ChartSkeleton
* update ChartSkeleton
* lift spacing to LegendCaption
* use lighter colors for dashcard downloads
* add new loki snapshots
* use the auto refresh text for locale tests as pdf export text is gone
* use the lighter color for all download icons
* add new loki snapshots
* update test assertions for locale tests
* fix(sdk): use instance locale if no locale is passed
* update docs
* we do support country codes, we have tests for it
* Update e2e/test-component/scenarios/embedding-sdk/locale-on-provider.cy.spec.tsx
Co-authored-by: Phoomparin Mano <poom@metabase.com>
* remove manual fallback to "en", site-locale is never undefined as it has
a default
---------
Co-authored-by: Phoomparin Mano <poom@metabase.com>
Avoids hitting the postgresql query parameter limit when there are more
than MAX_UINT16 tables in a database, also should be more efficient to
fetch this in a single query.
* Use Mantine instead of styled components
* Fix centering
* Add task and status params
* Add TaskStatusPicker
* Make picker clearable
* Add task picker
* Put filters and pagination together
* Translate task status
* Break down logic
* Move loading and error state
* Always render something
* Make task picker searchable
* Add empty state to the table
* Fix clear button being transparent
* Add useUrlState
* Use useUrlState in TasksApp
* Move UrlState to its own file
* Sort
* Rename file
* Rename
* Clean invalid query params on mount
* Prettier
* Use Action
* Ignore invalid pages
* Reuse variable
* Fix tests
* Add test
* Add label to clear button
* Remove redundant condition
* Add assertions
* Improve typing
* Be explicit about empty string handling
* Add url state to logs
* Add download button
* Disable the button instead of hiding it
* Add query input
* Add clear button
* Improve copy
* Typo
* Fix unit tests setup
* Improve empty state message
* Support location in setup
* Add a test for empty state message when filtering
* Add a test
* Add download button assertions
* Add e2e tests
* Simplify assertion
* Remove redundant element
* Add debounce
* Fix timezone issue
* Extract TasksTable
* Move use-url-state to its own directory
* Add unit tests for useUrlState
* Add test
* Refactor
* Add more tests
* Fix missing import
* Add task endpoints mocks
* Add TasksApp tests boilerplate
* Add pagination test
* Add tests for filters
* Add more tests
* Update tests
* Update tests
* Add more assertions
* Use Mantine instead of styled components
* Fix centering
* Add task and status params
* Add TaskStatusPicker
* Make picker clearable
* Add task picker
* Put filters and pagination together
* Translate task status
* Break down logic
* Move loading and error state
* Always render something
* Make task picker searchable
* Add empty state to the table
* Fix clear button being transparent
* Add useUrlState
* Use useUrlState in TasksApp
* Move UrlState to its own file
* Sort
* Rename file
* Rename
* Clean invalid query params on mount
* Prettier
* Use Action
* Ignore invalid pages
* Reuse variable
* Fix tests
* Add test
* Add label to clear button
* Remove redundant condition
* Add assertions
* Improve typing
* Be explicit about empty string handling
* Extract TasksTable
* Move use-url-state to its own directory
* Add unit tests for useUrlState
* Add test
* Refactor
* Add more tests
* Fix missing import
* Add task endpoints mocks
* Add TasksApp tests boilerplate
* Add pagination test
* Add tests for filters
* Add more tests
* Convert pratt compiler to use ExpressionParts
* Simplify compile by using compileNode helper
* Add todo's about casts
* Simplify error creation
* Make assertions more consistent and translate error messages
* Remove any check
* Simplify compiler by moving stuff into infix/unary helpers
* Remove unused assertions
* Remove casts on constant operator names
* Add assertions for operators
* Wrap top-level literals in a value clause
* Handle bigints in pratt compiler
* Accept ExpressionParts in Lib.expressionClause
* Remove null from ExpressionArgs
* Add field and metric to expression operator
* Simplify pratt compiler types
* Add more tests for equivalent clauses
* Return metadata objects directly from field-resolver
* Add more operators to type
* Create type for field type
* Remove resolverPass
* Convert resolver to Lib.ExpressionParts
* Convert compiler to expressionParts
* Fix syntax error
* Fix types
* Remove unused passes
* Fix resolver tests
* Add types to top-level literals
* WIP: convert untagged expression-parts from #js
* Manually unwrap field, segment and metric metadata
* Unwrap options in pratt compiler
* Remove empty options
* Handle normalized expression when diagnosing
* Allow customizing field resolver
* Fix fuzz tests
* Add value to ExpressionOperator
* Remove unused TODO's
* Use constants in test so types match
* Wrap clauses in defineClause and extract list of defined clauses
* Forbid setting name on clause config
* Use defined clause name as ExpressionOperator
* Use getClauseDefinition helper to get clause definitions
* Ignore type in filters
* Move expression-parts transformation to lib.js
* Add type-of method implementations for field, metrics and segments
* Trigger change when fast setting custom expression
* Remove superflouous casts
* Simplify resolver return type
* Use reference for top-level fields
* Be more specific about types in compiler
* Convert operator to string
* Add unit test for specific expressions
* Fix literals icons
* Remove unused matchers
* Remove unused option from compiler
* Negate number literals directly in compiler
* Add expression-clause implementation that takes one argument
* Use single-arg version of Lib.expressionParts
* Remove TODO
* Use helper to compile top level literal
* Add effective-type to top-level literals
* Remove unused MBQLCLauseMap type
* Remove unused import
* Add two-argument version of expressionClause
* Wait for button to be visible
* Add operator to invalid operator assertion
* Remove incorrect operator check
* Add more specific type to clause name
* Always throw if function name is not found
* Add overloads to getDefinedClause for known function names
* Document the allowFastSet typing hack
* Lift operator check
* Convert nil to {} in body
Co-authored-by: metamben <103100869+metamben@users.noreply.github.com>
* Make expression-parts js conversion more clear
* Pass through lib.metadata.calculation/type-of for the type-of implementations
* Always throw assertion errors
* Use helper to check query invariants
* Remove lib.schema.expression/type-of methods
* Add test for normalization of uppercase keyword arguments
* Allow some normalizers to ignore case
* Use threading for lowercase converter
Co-authored-by: metamben <103100869+metamben@users.noreply.github.com>
* Structure test better
Co-authored-by: metamben <103100869+metamben@users.noreply.github.com>
* Remove unused var
---------
Co-authored-by: metamben <103100869+metamben@users.noreply.github.com>
* Log metric data for failures in qp.middleware.metrics/adjust
When qp.middleware.metrics/adjust fails to expand a metric, log the metric's name and dataset_query, not just the
metric id.
Closes QUE-862
* Update test to assert that metric data is present in ExceptionInfo
* wip script to build the frankenjar
* build/run/test functions that seem to be working
* preparing for ci
* wip ci workflow
* need to run the job "manually" at least one to be able to dispatch it
from the gh cli
* add workflow_call
* bash instead of sh to solve issue of illegal option -o pipefail?
* use correct version of node
* prepare backend
* more envs to ci
* make jar run on the background in ci
* trying to run a list of specs in ci, tagging with tags probably won't
work as tags potentially need to be there on "older" versions of the
test
* fewer tests so it maybe doesn't timeout
* run on prs targerting the pr branch and the target base one
:wq
* clean up
* fix: without --env it wasn't really using the grepTags param:wq
* refactor: check for the actual files instead of doing ls*
* add back some more test files
* add log for ci
* remove colors as they're not shown in ci
* test: wait for data to load in a flaky test
* test: wait for data to load in a flaky test
* make sure we're building the EE jar
* skip licnese step as we don't need it
* move script out of embedding-sdk folder
* make curl -s to spam less in ci
* refactor: move jops to keep the result job last
---------
Co-authored-by: Uladzimir Havenchyk <uladzimir.dev@gmail.com>
* Correct output for dashboard filter variables
* Whitespace
* Add node script for testing clojurescript
* Enhance format-unit and add parse-unit.
parse-unit is new.
format-unit now works with moment (cljs) or java.time (clj) values, not just strings and numbers. I also added two new
units (:day-of-week-abbrev and :month-of-year-full).
* Refactor to use time/parse-unit and time/format-unit
this happens because we're not hashing `now` in this case, but what is coming from DB; which is OffsetDateTime and can
have slightly different precision, leading to different hash.
* Replace confirm content in use-confirmation hook
* Ensure modal closes after confirmation
* Update to mantine popover
* Toggle closeOnClickOutside when modal is open/closed
Fixes#51175
BigQuery sometimes returns timestamp values as floating point values (as strings, of course). We used to pass the
parsed double value to java-time.api/instant, but that seems to have microsecond precision. In this change we use
Java's built-in Instant construction function to keep the full precision.
* Add repro for #56596
* Allow escaping backslash
* Use unquoteString everywhere to parse strings and references
* Do string quoting and unquoting more consistently
* Add repro for #56596
* Make string parsing more straightforward and consistent
* Use quoteString to find errors with quoting
* Type string helpers more strictly
* Add fuzz tests for string quoting
* Move fuzz helper to test folder
* Make string fuzzing repeatable
* Add sanity check
* Fuzz all three quoting modalities
* Add [-quote tests
* Rename quote to delimiter
* Fix path for lexifier fuzz testing
* Fix name for compiler fuzz testing
* Use constant to store delimiter pairs
* Fix option name in formatter
* Add more cases around half-quoted strings
* Rename quote to delimiter in comment
Co-authored-by: Kamil Mielnik <kamil@kamilmielnik.com>
---------
Co-authored-by: Kamil Mielnik <kamil@kamilmielnik.com>
* Add isExpressionParts helper to check
* Export Lib.isColumnMetadata
* Add Lib.isColumnMetadata to check
* Mark is-column-metadata as Single use instead of smelly
* Add is-metric-metadata helper
* Add is-segment-metadata helper
* Add isSegmentMetadata and isMetricMetadata to checks
* Use expressionParts and expressionClause in formatter
* Avoid stringifying the ast
* Allow passing literals to filter-clause
* Simplify test case for now
* Handle [:value x] in formatter
* Unnest expression-parts
* Add metric-clause? and segment-clause? helpers
* Handle metric and segment clauses in expression-parts
* Flatten case/if statement args in expression-parts
* Remove special case for case/if in formatter
* Support keywords in expression-parts
* Fix duplicate negation test case
* Prefer desctructuring over get
Co-authored-by: Braden Shepherdson <braden@metabase.com>
* Use set to match case and if
Co-authored-by: Braden Shepherdson <braden@metabase.com>
* Fix typo
* Use conditional threading in maybe-expand-*-expression
* Use destructuring to match case/if clauses
* Use cond-> instead of maybe-expand- helpers
* Fix literal in formatter tests
* Rename is-*-metadata to *-metadata?
* Swap test case order
* Use multi-method for expression-parts
* Inline recurse fn
* Accept ExpressionClause in the formatter
* Pass expression clause in HighlightExpression
* Build expressionParts for examples manually
* Be more strict in formatter typing
* Be more strict in formatter typing
* Fix FunctionBrowser example
* Add test case for 1 + 1
* Pass expression clause to format in Editor
* Add docstring to expression-parts-method
* Remove superflouous check
* Use threading macro in column-metadata-from-ref
* Unwrap case/if into pairs and a fallback argument
* Remove reference to unknown function
* Add stage-index-less version of expression-parts
* Add MetricMetadata and SegmentMetadata to ExpressionArg
* Fix case/if expansion not to have nil argument when fallback is nil
* Add test for case and if
* Fix case/if defop
* Add more test cases to case and if test
* Add ExpressionArg type
* Allow passing Expressionable to formatter
* Add more tests for case and if
* Move case and if conversions to fe_util
* Deduplicate case-or-if matcher
* Document case-or-if helpers more
* Be more precise in the type of ExpressionParts args
* Fix args for expression-clause in exclude-date-filter
* Remove faulty test
* Add test for nested parts
* Move fix-expression into ->op-arg
* Move ensure-uuid into fix-expression
* Wrap operator in keyword
* Correctly recurse through expression-parts
* Add test for nested case parts round-trip
* Add source-field to test data
* Comment out broken test case
* Add source-field to ReferenceOptions type
* Remove unused imports
* Allow more arguments to expression-parts
* Use expression-parts in filter test
* Handle bigint in formatter
* Add missing import
* Remove unused import
* Normalize mbql
* Allow lib.expression/expression as arg to expression-clause
* Handle expression refs
* Add tests for unknown fields, metrics and segments
* Add metrics to formatting tests
* Handle non-existent metrics and segments
* Add workaround for unknown fields
* Use cond-> and destructuring for unknown field
Co-authored-by: Braden Shepherdson <braden@metabase.com>
* Remove superflouous call to lib.field/extend-column-metadata-from-ref
* Inline segment-metadata-from-ref and metric-metadata-from-ref
* Collapse all identity expression-parts-methods in a doseq
* Call expression-parts instead of expression-parts-method
* Use multi-method to handle case and if
* Pull out temporal expression expansion into separate multimethod
* Move check closer to where it is used
* Remove constant conditional
* Use threading macro in expression-parts
* Move nested case/if test out of deftest
* Fix formatting
* Fix metadata method when metric is not found
* Add display-name to unknown field stub
* Use lib.metadata helpers directly for field, metric and segments in expression-parts
* Fix e2e test
* Handle segments in filter parts directly
* Fix e2e test
* Add TODO
* Use doseq in literals test
* Add test for bigint value clause in expression-parts
* Remove TODO
* Use id from sample database
* Use vec
Co-authored-by: Braden Shepherdson <braden@metabase.com>
* Fix types
* Use simpler assertion helper
* Move metric to aggregations tests
* Do not wrap value clauses in parentheses
* Return metric metadata from expression-parts
* Emit expression reference in the case of expressions
* Strip expression-name from columns metadata in expression-parts
* Add e2e test for nested references
* Return stub metric metadata if metric is not found
* Use expression-name from the reference
* Add tests for field reference in expression-parts
* Add tests for segment reference in expression-parts
* Add tests for metric reference in expression-parts
* Add tests for expression reference in expression-parts
---------
Co-authored-by: Braden Shepherdson <braden@metabase.com>
* Support drivers that need special handling for literal string expressions
* Enable the :expression-literals feature for vertica
* Enable the :expression-literals feature for redshift
* Review feedback: lift comment about inlined literals from h2 to sql driver
- if you have bb on your path, then this would never be hit.
Testing by uninstalling bb will not uncover this issue if you have
repo/path/metabase/bin on your PATH!
* Rename True and False expressions in literal expression tests
Speculative fix for sqllite failing the tests.
* Update literal expression tests to work with drivers that treat empty string as null
* Add empty-string-literal-expression-test and remove empty string form standard-literal-expression-defs
* Fix logical merge conflicts in case-with-literal-expression-test
* Enable the :expression-literals feature for sqlite and starburst
* Review feedback: drop format-nil->empty-str
As pointed out by metamben in code review `(str nil)` is already the empty string. In fact, the culprit here was a
missing `format-nil-values?` arg to `format-rows-by`.
* Add route prop to disable command palette, Disable in setup routes
* increase wait timeout
* refresh user on setup
* Lets try a unit test instead
* removing refresh user from UserStep
We need to ensure that we consistently have entity-ids in metadata (and potentially elsewhere) even before we've finished backfilling them in. To do this, we check for entity-ids in the field/table/database/card after-select and run calculation/deduplication logic to get them when necessary. We then cache the results (because these calculations can be slow), and a separate job then drains the cache and writes the results back to the db.
* Migrations for database sandboxing (#53125)
Two changes here:
- add a self-referential foreign key,
`metabase_database.router_database_id`, to allow marking a mirror
database's router database. Databases with this set are mirror
databases, databases without it are primary databases.
- add a `db_sandbox` table with only three columns:
- `id`, primary ID,
- `db_id`, the `metabase_database.id` this sandbox is for, and
- `user_attribute`, the user attribute used to map users to a
particular mirror database.
* CRUD API for database routing (#54102)
* Migrations for database sandboxing
Two changes here:
- add a self-referential foreign key,
`metabase_database.primary_database_id`, to allow marking a mirror
database's primary database. Databases with this set are mirror
databases, databases without it are primary databases.
- add a `db_sandbox` table with only three columns:
- `id`, primary ID,
- `db_id`, the `metabase_database.id` this sandbox is for, and
- `user_attribute`, the user attribute used to map users to a
particular mirror database.
* Bulk endpoint for creating mirror dbs
The error case is a bit wonky right now, it just returns the errors for
the details that failed... without the details or name.
Co-authored-by: Ed Paget <ed@metabase.com>
* Add endpoint for marking a database as a RouterDB
Co-authored-by: Ed Paget <ed@metabase.com>
* Rebase fixes
* Exclude mirrordbs from regular db endpoints
* kondo fixes
* mark `DatabaseRouter` as not serialized
* serdes fix
* copy fix
* delete and update routers
Also reorganized things to be a bit less cattywompus.
Now we have:
- `POST api/ee/database-routing/mirror-database` to create a new mirror database
- `POST api/ee/database-routing/router` to create a new router using an
existing database
- `PUT api/ee/database-routing/router/:id` to update a router to use a
new user attribute
- `DELETE api/ee/database-routing/router/:id` to delete a router and all
of the associated mirror databases.
* GET endpoint for mirrors of a particular router db
* Don't test database conn for mirror dbs
---------
Co-authored-by: Ed Paget <ed@metabase.com>
* Modify the API - don't care about database routers (#55036)
* Modify the API - don't care about database routers
Add a `router_user_attribute` to the `GET /api/database/:id` API
endpoint.
Modifies the database routing API to address only by the database ID.
There's now just two endpoints:
- POST /ee/database-routing/mirror-database to create a mirror database
- PUT /ee/database-routing/router-database/:id to change the
`user_attribute` used for routing an existing database. Can be set to
`null` to turn of database routing for the database.
* fix test
* kondo
* DB Routing: Sloan's API improvements (#55438)
* router_user_attribute on `/database/`
* unique database names for mirror dbs
* Don't delete mirror databases when deleting router
* query param to check conn details
* no making a mirror a router
* Require `:manage-database` perms to get dest dbs
(rather than superuser.)
* set `visible-if`, NA on dest dbs
* Add conditions when making db router
* fix db migrations
* fix tests
* kondo
* reorder migrations for mysql/mariadb
* Implement database routing (#55053)
* Swap DB for DB routing
Caching should work, field values does not currently.
* Rename the metadata replacement dynamic var
for clarity. Since we're using it outside of tests now.
* Refactor Advanced Field Values
Previously we had 3 types of Advanced field values:
- sandbox,
- impersonation,
- and linked-filter
Each of these three had their own function, `hash-key-for-*`, and the
caller of this function was responsible for generating the correct hash
key for the field provided.
For example, `hash-key-for-advanced-field-values` took a `fv-type` and
called either `hash-key-for-linked-filters` or `hash-key-for-sandbox` or
`hash-key-for-impersonation` depending on the type.
The big problem here is that the hash keys aren't composable. You can
see this being an issue in `hash-key-for-linked-filters`, where the
enterprise version needs to also check whether the field is sandboxed,
and if so, include sandboxing data in the generated hash as well as the
linked filter data.
If we add more orthogonal things to the hash (like database routing) we
would end up with a rat's nest where each hashing function partially
implements a bunch of other hashing functions to make things work. E.g.
`hash-key-for-database-routing` might check if the field is sandboxed,
and if so include data about the sandbox, but also
`hash-key-for-linked-filters` should check if the field is
sandboxed *or* impersonated *or* subject to db-routing and include data
about those, and ...
Instead, I've replaced all these functions with `hash-input-for-*`
functions, which return self-contained maps that reflect
their *internal* knowledge of what should/should not be cached.
For example, the `hash-input-for-sandbox` function returns either
`nil` (if the field is not sandboxed for the current user) or
`{:sandbox-attributes {<the user attributes used for the sandbox>}}` (if
it is).
`hash-input-for-field-values` simply merges all of these together into
one lump.
To tell whether we're generating a `full` or `advanced` field value, we
can just check to see whether the hash input has any data other than the
field ID.
The one downside here is that we'll no longer be able to look at the DB
and distinguish between different types of FieldValues. As far as I can
tell, the code itself does not ever do this, so I think this is an
acceptable tradeoff.
* database routing for field vals
* move things around a bit
* fix tests
* Lint
* WIP: database routing
* fix test
* more tests, caching tests
* kondo
* stray #p
* Exclude mirror databases from search
* Exclude mirror databases from sync
* Do not create permissions for mirror databases
* fix more tests
* lint
* Add another e2e test for field values
* Don't grab database from query in h2 driver
* Fix my rebase error
* [DB Routing] Implement UI (#55132)
* save
* wip
* wip
* lots of progress
* iteration
* more progress, add delete mirror modal
* type clean up, bug fixes, code organization
* address lots of TODOs
* prevent configuring uploads to routed dbs, prevent db routing if db is upload destination db, replace all usages of routed database and mirror database in favor of destination database
* clean up url usage, add health check to destination database lists
* support hidden fields for database details
* fix types and TODOs
* render warning that model features can not be enabled when db routing is enabled, add unit tests for DestinationDatabasesList
* add e2e coverage for db routing CRUD
* move db routing e2e to a new file, make tests nicer, add test coverage for not showing destination databases outside of database settings
* test that db routing can not be enabled if other conflicting features are enabled
* finish e2e tests as much as possible, add feature token to session properties payload
* move more code into enterprise folder, add redirects and permissions checks for destination db routes, handle no user attributes on instance state, fix failing unit tests, make alerts tooltips when features cant be enabled based on other features being enabled
* test fixes, impl error states kyle provided
* split up tests by ee and oss
* error state updates, fix failing e2e tests
* fix failing oss e2e test, beef up db management perms test
* PR feedback
* impl more feedback
* Fix BE tests
* another be test
* pr feedback
* fix one of the failing e2e tests
* include_mirror_databases -> router_database_id
* skip test
---------
Co-authored-by: John Swanson <john.swanson@metabase.com>
* fix deletion of router dbs with dest dbs (#55831)
* Don't require superuser perms to get dest db (#55754)
* [DB Routing] Database pages tech debt - RTKQuery + TS migrations (#55744)
* convert DatabaseList to typescript
* make DatabaseEditApp use rtkquery, remove old initialize approach, remove related redux code, port code to use api db rather than metabase lib Database class
* remove dismissDatabaseSyncSpinner redux code in favor of rtkquery
* fix some mistakes causes tests to fail
* port DatabaseList to typescript
* port adding sample database to rtkquery
* misc fixes
* fix failing unit test
* convert admin/databases/database.js and admin/databases/selectors.js to TS
* editParamsForUserControlledScheduling.js to editParamsForUserControlledScheduling.ts
* remove all usages of metabase lib Database class from Database pages
* fix type errors
* fix failing e2e
* prettier
* [DB Routing] Port Admin DB page components to Mantine (#55811)
* port components to mantine for admin db pages
* fix failing tests
* remove unused comment
* fix type issue
* Require opt in/out of database routing (#55841)
I was considering possible errors here and I think this will work well
as a safety harness.
In dev/testing (non-prod) environments, let's add an assertion to the
JDBC driver that if we're about to connect to a database. Either:
- we're *explicitly* trying to only connect to Router Databases. For
example, sync should *never* connect to mirror databases or go through
Database Routing. OR
- we've enabled Database Routing, and we *don't* want to access the
Router Database (unless of course DB routing has been configured to send
us there)
Another way of putting this is that there are two obvious ways for this
feature to go sideways: we accidentally sync a mirror database or do
something else that shouldn't work for mirror databases, OR we
accidentally skip the Database Routing process and connect to the router
database when we shouldn't.
Of course, we can't guarantee that all drivers will call
`(database-routing/check-allowed-access!)` before connecting! But we can
just call it from the JDBC router and that should be sufficient to catch
any issues in development in the vast vast majority of cases.
* [DB Routing] UI feedback part 1 (#55893)
* make dest db validate on creation, invalidate cache of user attributes after user is updated
* add test coverage for updates and uncomment e2e tests parts that were previously failing
* fix type issue
* Fix audit/query logging for DB routing (#56103)
[ADM-585: Handle audit logging and query logging](https://linear.app/metabase/issue/ADM-585/handle-audit-logging-and-query-logging)
* exclude destination dbs from serialization (#56076)
* Decrease the pool size for mirror DBs (#56104)
DB metadata seems like an odd thing - it's the Metabase Database
converted to a snake-hating map, so `router_database_id` becomes
`router-database-id`.
We can't write a function of `:model/Database` that decides whether
we're looking at a mirror database (because we're not looking at a
`:model/Database` here). So we're reduced to just checking the key.
Feels weird, but I think the cleanest way available here?
* Fix sandbox on a routed DB with question as filter (#56393)
This only affected turning on sandboxing, and was not an error for
sandboxing itself.
When we configure sandboxing, we want to make sure the column types for
the query we're using match the columns in the sandboxed table.
As part of this, we ran the query anonymously with
`(request/with-current-user nil ...)`. With Database Routing, however,
we added a new constraint: you can't run any queries against a database
with routing configured anonymously.
Instead just run the query as a superuser. This is effectively the same,
and it's consistent with what we do when actually *enforcing* the row
level restrictions anyway: we preprocess the source query as an admin
and then splice it into the user's query.
The one thing that gives me pause: I'm not really sure why we would have
done `(request/with-current-user nil ...)` in the first place here.
* Add database-router usage e2e tests (#56255)
* Improve error messages and UI for config (#56392)
* Improve error messages and UI for config
- improve error messages when a dest db can't be found
- improve the admin UI for configuring DB routing: https://metaboat.slack.com/archives/C07L6VCCYQL/p1744044752380479
* changes suggested by Jeff
* fix error msg tests
* disable linting for DB Routing description
* fix getting a destination db
db `can-read/write?` use router DB if necessary, since destination
databases don't actually have permissions saved.
* i18n for errors
* spelling 😁
* fix tests
---------
Co-authored-by: Nick Fitzpatrick <nickfitz.582@gmail.com>
Co-authored-by: Ryan Laurie <iethree@gmail.com>
* Fix options for Destination DBs (#56450)
* Fix options for Destination DBs
This may still require a bit of frontend work.
* Set destination-database=true for destination db modal
---------
Co-authored-by: Fraser McIntosh <fraser@metabase.com>
* Fix impersonation with Database Routing (#56503)
* Fix impersonation with Database Routing
Database routing was not working with impersonation, because by the time
the driver calls `set-role-if-supported!` the router database (which is
configured for impersonation) has already been swapped out for the
mirror database (which is not).
To fix this, have a new impersonation middleware set a dynamic var,
`*impersonation-role*`, which is preferred by `set-role-if-supported!`
if set. This way, in queries, `set-role-if-supported!` will use the
pre-calculated role (which seems better than computing it twice anyway).
Co-authored-by: Ed Paget <ed@metabase.com>
* Add impersonation tests
* Add sandbox question test
---------
Co-authored-by: Ed Paget <ed@metabase.com>
Co-authored-by: Fraser McIntosh <fraser@metabase.com>
* keep metadata store on metadata provider swap (#56502)
* keep metadata store on metadata provider swap
Before, we would blow away the existing QP store entirely when using
`*DANGER-allow-replacing-metadata-provider* to swap out the existing
metadata provider.
Instead we can only initialize the QP store when it was truly
uninitialized, and replace *only* the metadata provider when we're
trying to do that.
This is just in case something in QP post-processing needs data from the
original store, e.g. something that was stored with
`qp.store/store-miscellaneous-value!`.
Co-authored-by: Ed Paget <ed@metabase.com>
* only allow replacing md provider once
---------
Co-authored-by: Ed Paget <ed@metabase.com>
---------
Co-authored-by: Ed Paget <ed@metabase.com>
Co-authored-by: Sloan Sparger <sloansparger@users.noreply.github.com>
Co-authored-by: Fraser <fraser@metabase.com>
Co-authored-by: Nick Fitzpatrick <nickfitz.582@gmail.com>
Co-authored-by: Ryan Laurie <iethree@gmail.com>
* feat: upgrade ring-jetty to 1.14.1
Moves us onto Jetty 12 which has some features (pthreading, QoSHanlder)
that we may want to take advantage of.
* fixup! customize jetty metrics
* fixup! license info for jetty ee9
* fixup! make a var private
* fixup! don't use deprecated method
* fixup! create custom statistics handler
Replicate the Jetty 11 statistics handler that is able to report on
asynchronously handled requests separately from synchronously handled
ones. Also ports it to directly report statistics to our prometheus
rather than needing to build a connector between the stats handler
object and our metrics reporting
Co-Authored-By: John Swanson <john.swanson@metabase.com>
* fixup! fix race condition in stats test
* fixup! missing private annotation
* fixup! use normal jetty release with logging fix
---------
Co-authored-by: John Swanson <john.swanson@metabase.com>
* call e2e-component-tests-embedding-sdk-cross-version from
embedding-sdk.yml
* add e2e-component-tests-embedding-sdk-cross-version to the tests-results
so it needs to pass (or be skipped)
* Reset all errored triggers to `WAITING` on startup
Co-authored-by: Ed Paget <ed@metabase.com>
* Add explanatory comments
---------
Co-authored-by: Ed Paget <ed@metabase.com>
Metabase jars would crash when running on JVM 24 because the getReourse
method on the ClassLoader object was changed to return null if there is
no file at the provided path. This adds a ".keep-me" file to the
resources directory that Metabase can use to check if it is running in a
jar.
* fix: resolve snowflake sync cost issue
Using information_schema queries will cause snowflake to wake an
instance and thus incur costs. Using `show` queries will not do so.
In addition this adds a new multi-method for manipulating the results of
`describe-fields-sql`.
* Fix database-position and sorting
* Fix kondo
* Add order-by to mysql describe-fields
* Sort deps
* Use identity not map identity
* Fix databricks
* Address PR comments
* scaffold separate download options for dashboard static embeds
* parse the display options to hash options
* add toggles to the embed preview
* workaround for duplicate types
* cleanup the dashboard workaround
* cleanup the dashcard workaround
* downloads parameter should not show up on OSS
* replace the download button tests
* update existing tests
* fix test for Download buttons
* remove unnecessary fallback
* add a new snowplow schema
* update unit tests
* fix snowplow test assertions
* add snowplow tests
* use results instead of dashcard for consistency
* make the JSON schema backwards-compatible
* split download settings into two components
* use downloadsEnabled.results instead
* snowplow type must be results, not dashcard
* revert StaticEmbedSetupPane changes as it is unused
* create a new field for snowplow tracking
* Enable expression literals on drivers that pass tests
Closes QUE-780
* Update .devcontainer/Dockerfile for java 21 and node 22
This Dockerfile is apparently used by the "metabase assembler", which in turn is used by
.github/workflows/presto-kerberos-integration-test.yml to run the Presto Kerberos tests.
* Add case-with-literal-expression-test
* Review feedback: remove extra whitespace
Co-authored-by: metamben <103100869+metamben@users.noreply.github.com>
---------
Co-authored-by: metamben <103100869+metamben@users.noreply.github.com>
* feat: make cluster locking reusable
Builds a more generic interface for cluster locking using our existing
retry mechanism and jdbc level timeouts that will work with any database
rather than just postgresql.
* fixup! handle alt error text on mysql/mariadb
* fixup! don't lock on h2
* fixup! move malli annotations to retry
* Store all native editor selected ranges in redux
* Pass all selected ranges to redux
* Add repro for metabase#54799
* Fix unit tests
* Fix typo
Co-authored-by: Kamil Mielnik <kamil@kamilmielnik.com>
* Do not intersperse space in when running selected text
---------
Co-authored-by: Kamil Mielnik <kamil@kamilmielnik.com>
This test was flaking if it was run close to the boundary of a UTC day
where the models could be created one day and then the clock ticked over
to a new day before the query ran.
* Don't update card metadata if it has not changed
* Store stored-metadata as a different field on info
* Fixed failing tests
* Update src/metabase/query_processor/middleware/results_metadata.clj
Co-authored-by: Edward Paget <edpaget@users.noreply.github.com>
* Addressed review comments
* Update src/metabase/query_processor/middleware/results_metadata.clj
Co-authored-by: Braden Shepherdson <braden@metabase.com>
* Addressed review comments
* Update src/metabase/query_processor/middleware/results_metadata.clj
Co-authored-by: Braden Shepherdson <braden@metabase.com>
* Addressed review comments
* fixup! exclude info map from legacy normalization
* fixup! test that metadata equality skips update
* fixup! make adding the info map conditional
* fixup! use qp.store instead of adding to info map
* fixup! remove #p
* fixup! remove cyclic dep
* fixup! move load and store fns to same ns
* fixup! test name
---------
Co-authored-by: Edward Paget <edpaget@users.noreply.github.com>
Co-authored-by: Braden Shepherdson <braden@metabase.com>
Co-authored-by: Edward Paget <ed.paget@gmail.com>
The new `:card_schema` field on `:model/Card` was wrongly excluded from Revisions.
This PR includes them, and adds a default value for old Revisions which predate `:card_schema`.
* fix: snowflake can-connect with ssh without touching other drivers
* Adjust host-and-port for can-connect without calling incorporate-ssh-tunnel-details
We were inadvertently logging db-details in the rarely used
`maybe-test-and-migrate-details`. This changes to more of a data
diff where we can still get important information on how it changed.
* Rename checkCompiledExpression to diagnoseExpression
* Add tests for arg count validation
* Add arg validator tests
* Add test for unsupported functions to diagnostics
* Add isCallExpression matcher
* Add visit helper
* Move unsupported function check to diagnostics
* Move arg validator to diagnostics
* Move arg count validation to diagnostics
* Remove EQUIVALENT_CLAUSES since we have them in config.js
* Remove database argument where no longer needed
* Add COMPARISON_OPERATORS to config
* Add LOGICAL_OPS to config
* Add NUMBER_OPERATORS to config
* Add FIELD_MARKERS to config
* Move comparison validator to diagnostics
* Simplify resolver
* Move checkers before Lib.diagnoseExpression
* Fix diagnoseExpression return type
* Move case or if operator argument check to diagnostics
* Remove unused tokenName
* Make expression types more specific
* Convert resolver to typescript
* Add test for multi-arg infix operators
* Add checker for unknown functions
* Add more tests for coalesce typing
* Add more tests for comparison operator resolving
* Add test for number operators
* Add test for logic operators
* Remove branches that are handled in the config already
* Allow mbql clause config to define the arg types
* Allow case/if to define argType too
* Move getNode and getToken to utils
* Make resolver fn required
* Validate field names in resolver
* Move trim into getMBQLName
* Fix top-level literals
* Handle expression refs
* Allow number operators to be multiple
* Add more tests for custom arg validators
* Test operator resolvers more
* Add more tests for multi arg functions with options
* Use better formatting in test
* Be more generic in getNode and getToken
* Use correct location for errors
* Make it work
* Use Mantine
* Update tests
* Display db type instead of mbql type
* Show db type instead of semantic type
* Revert "Show db type instead of semantic type"
This reverts commit 5a51c5abca.
* Update tests
* Prevent CLS
* Revert change
* Revert "Prevent CLS"
This reverts commit f74b6e80af.
This commit introduces empty states for visualizations
in order to help less experienced users create their first chart.
These new empty states come in two flavors:
1. With a "Summarize" button CTA that will open the summarize sidebar
2. With a link to our helpful documentation (for more complex charts)
We have been previously relying on the `placeholderSeries` dummy data to display a greyed out chart preview when we cannot render a chart. That functionality is now replaced with a new property called `hasEmptyState`.
`checkRenderable()` util will try to render a chart based on the series and settings. If it is unable to do so, it will throw a `ChartSettingsError`. This, in combination with hasEmptyState and a few other properties, is a cue to render a new, empty state for that visualization.
* Initial BE changes for custom expression literals (#55206)
* Empty
* Add `value` support in the expression editor (#55203)
* Literals in filters tests (#55283)
* E2E tests for literals in custom columns (#55288)
* BE changes for :value wrapped expression literals
* Extend BE test with more literal values
* Add float value to qp test for expression literals
* Add Float and Boolean cases to literal-expression-test
* Remove ^:internal from value clause
:value clauses are now generated by the frontend when creating a top-level expression literal and preserved when
converting legacy <-> pMBQL.
* Add some lib.convert tests for literal custom expressions
* Add legacy schema tests for literal custom expressions
* Add normalize / canonicalize tests for literal custom expressions
* Remove vestigial "date" expression from fields clause in query-with-literal-custom-expression-test
* Fix literal-expression-test for cljs
CLJS parses 1.00 as 1 and hence :type/Integer not :type/Float
* Fix js_test expression-clause-<->-legacy-expression-test for wrapped literals
* Move QP tests into expressions_test.clj and add more tests
* Remove commented out :value and update comment in stage-expressions->legacy-MBQL
* Restrict QP literal expressison tests to supported drivers
* Update boolish->bool to accept BigDecimal inputs
Apparently some DBs return BigDecimal results for boolean expressions.
* Temporarily remove literal-expression-case-test
Needs more debugging
* Move aggregations tests into expression_aggregations_test.clj
* Add some basic filter tests to QP expressions_test.clj
* E2E tests for literal custom columns in other clauses and fixes (#55295)
* E2E tests for literal custom columns in other clauses and fixes (#55295)
* Move changes from the FE branch
* Move changes from the FE branch
* Move changes from the FE branch
* Fix filter schema and false = false filter (#55417)
* Fix date/string literals effective type (#55427)
* Fix date/string literals effective type
* Fix date/string literals effective type
* Fix date/string literals effective type
* Fix date/string literals effective type
* Fix date/string literals effective type
* Fix date/string literals effective type
* Fix date/string literals effective type
* Fix date/string literals effective type
* Fix malformed order-by clauses
* Remove tests for offset expressions as non-aggregation expressions
These tests were initially added in #42302, but we disabled offset in custom columns in #43224.
Remove these tests so we can re-enabled nest-expressions when expressions appear in an order-by clause.
https://metaboat.slack.com/archives/C0645JP1W81/p1742317450874479
* Add order-by to the list of places to check in should-nest-expressions?
This is to support order-by of custom expressions, in particular literal custom expressions.
https://metaboat.slack.com/archives/C0645JP1W81/p1742317450874479
* Rename round-trip-expression-literal-test and move it next to existing literal tests in the same file
* Uncomment filter-literal-boolean-expression-with-no-operator-test
The schema was relaxed in bd82763beb, so now the test passes.
* Uncomment filter-literal-expression-with-and-or-test
* Add round-trip-filter-expression-test
* Add more cases to legacy schema filter-test
* Remove unused import
* Review feedback: set/difference -> disj
* Add :expression-literals driver feature (#55662)
* Add the :expression-literals database feature
And enabled it for postgres, mysql, bigquery, and snowflake.
These are the tier 1 and tier 2 DBs, minus redshift which is currently failing some tests.
* Add lib.metadata/database-supports? helper
* Check if :expression-literals feature is enabled in diagnose-expression
* Use standard-literal-expression-values in order-by test expectation
* Add comment about compiler warning to boolish->bool
* Add breakout-by-literal-expression-test
* Add expression-literals-test to nested-queries-test
* Add compound expressions involving literal custom columns to e2e filters tests
* Use lib.schema.common/is-clause? to detect :value expr in diagnose-expression
* Review feedback: fix order-by-integer-literal-expression-test
* Review feedback: conj -> vector
---------
Co-authored-by: Alexander Polyankin <alexander.polyankin@metabase.com>
* Add explicit CAST when compiling string literal expressions in H2 driver (#55920)
* Add explicit CAST when compiling string literal expressions in H2 driver
Closes QUE-726
* Add e2e test for nested question with literal custom columns
* Add :expression-literals feature to various mock providers used in tests
* Update diagnose-expression tests
* Ensure legacy :value options are normalized in mbql.normalize
Values coming from the FE in expression-clause-for-legacy-expression call normalize and expect to have :value options
converted from strings to keywords.
* Add base_type to value literals in adjustTopLevelLiteral
* Consistent spelling of base_type
* Use lib.metadata/database-supports? to check for database :features (#56114)
* Use lib.metadata/database-supports? to check for database :features
Closes QUE-836
* Convert a couple more tests to use meta/updated-metadata-provider
* Add e2e test to ensure that sandboxing on literal custom column fails closed (#56190)
* Add e2e test to ensure that sandboxing on literal custom column fails closed
Closes QUE-844
* Review feedback: dedup CustomColumnType definition
---------
Co-authored-by: Alexander Polyankin <alexander.polyankin@metabase.com>
* scrape releases page to find latest major version
e.g.:
```
$./bin/mage jar-download 50
Warning: JARS not found in env.
Looking for latest version of 50 on release page 1
Downloading from: https://downloads.metabase.com/enterprise/v1.50.35/metabase.jar
to: /Users/bcm/dv/mb/metabase/jars/metabase_1.50.35.jar ...
Downloaded 1.50.35.jar to /Users/bcm/dv/mb/metabase/jars
```
* better feedback
* Make segment name clickable
* Upadte e2e test
* Convert SegmentItem to TS
* Modernize SegmentItem
* Add cells
* Add TableBreadcrumbs component
* Use TableBreadcrumbs in SegmentEditor
* Use real data
* Improve data handling
* Handle ellispis
* Add tests
* Add test for no schema
* Rename
* Prettier
* Update icon color
* Fix icon shrinking
* Use TableBreadcrumbs in SegmentItem
* Fix icon shrinking
* Update max width
* Update tests
* Sort
* Remove css file
* Add TableBreadcrumbs component
* Use TableBreadcrumbs in SegmentEditor
* Use real data
* Improve data handling
* Handle ellispis
* Add tests
* Add test for no schema
* Rename
* Prettier
* Update icon color
* Fix icon shrinking
* Remove css file
* scaffold setting up individual downloads
* use downloadsEnabled as boolean for result downloads
* allow SDK to use a single withDownloads props for compatibility
* sdk static dashboard should continue to support single withDownloads prop
* update downloads prop to be string compatible
* fix the areDownloadsEnabled matcher
* refactor download plugin matcher
* do not modify EmbedDisplayParams for now
* fix misleading types
* remove ConnectedDashboard default
* undo LookAndFeelSettings
* revert embed flow and analytics changes
* add test cases for public resource downloads
* add test cases for public question download
* do not support PDF downloads on questions
* remove redundant showDashboardCardActions
* create types for download results
* rename cardResult to dashcard
* replace cardResult with dashcard
* add downloadsEnabled to storybook
* update unit test for areDownloadsEnabled
* update unit test for no-token case
* update unit test for oss case
* try to disable downloads in storybook
* update story defaults
* make downloadsEnabled a component default
* add default to downloadsEnabled in PublicOrEmbeddedDashboardView for loki
* revert changes as loki diffs seems good now
* fix loki diff issue in stories
* Empty
* Date generalized tests (#55953)
* Centralize and generalize tests for `integer()`
Also make room for other casting functions to be similarly tested
* Remove unsued imports
* Move postgres date cast test into cast_test.clj
* Rename :cast to :expressions/date
* Remove old imports
* Remove unused features
* Remove more!
* Fix test files
* Centralize and generalize tests for `integer()` (#55950)
* Centralize and generalize tests for `integer()`
Also make room for other casting functions to be similarly tested
* Remove unsued imports
* Example tests for integer()
* Remove empty line
* Use field type
* Appease the Kondo
* Check for Number instead of Integer
* Cast to big-integer before comparing (for snowflake)
* Remove split-part-test. It doesn't belong in this branch.
* Rename :cast to :expressions/integer
* Fix parens after merge
* Unified text() tests
* Appease the kondo
* Appease the whitespace demons
* Oops, wasn't filtering for drivers
* Use literal string for redshift
* Add a repro for #55951
* Fix typos
* Refactor repro
* Fix#55951
* Remove unused components
* Use 'loading'
* Format code
* Fix test setup
* Rename isLoading back to isSearchLoading since it indicates Search.loadList usage
* Use 1 /search API call instead of 2
* Get rid of rawData
* Update unit tests
* fix: deadlock in last used at
This prevents bulk updates to last used at from deadlocking by creating
a new table in metabase's database that can be used by instances to take
cluster wide locks. It adds an initial implementation of a customizeable
db-based locking mechanism.
For the last_used_at update, a metabase instance will first set a
session variable to control locking timesout then take a cluster lock
using SELECT * FROM metabase_cluster_lock WHERE lock_name = 'name' then
execute the update_at.
If another table is holding this lock metabase will wait .5s for the
lock to be released then timeout its SQL transaction, sleep the Thread
for 1s then retry taking the lock. This makes sure we're not holding DB
connections waiting for locks.
* fixup! kondo
* Update resources/migrations/001_update_migrations.yaml
Co-authored-by: Noah Moss <32746338+noahmoss@users.noreply.github.com>
* Update src/metabase/query_processor/middleware/update_used_cards.clj
Co-authored-by: Noah Moss <32746338+noahmoss@users.noreply.github.com>
* fixup! finish method rename
---------
Co-authored-by: Noah Moss <32746338+noahmoss@users.noreply.github.com>
* Fix formatter support for when both field ids and col names are used in viz settings
* Apply fix in qp common as well, which date formatter consumes
* Addressing PR feedback and breaking up large test
* Fixing or->merge reversion; adding datetime formatter tests
* Improving viz setting merges based on types of field_ref and column setting keys
* Addressing PR feedback
* Unified tests for `splitPart()`
* Remove postgres specific split-part-test
* Remove Redshift split-part-test
* Remove snowflake split-part-test
* Remove mysql split-part-test
* Remove bigquery-specific split-part test
* Fix MySQL implementation of splitPart()
* Only allow string literals for 2nd and 3rd args of replace()
* Only allow non-empty string for delimiter of splitPart()
* Better non-blank string
* Make position in splitPart() positive literal or numeric expression
* WIP new examples tests + corner cases
* Oops, go back to non-empty string
* Improve tests
* Improve test message
* Empty string on non-positive position in splitPart() for mysql and postgre
* Fix Redshift error
* Snowflake: return empty string with non-positive position
* Also test zero position
* Correct misplaced paren
* Fix `splitPart()` for snowflake
* Nicer comment on weird mysql implementation
* Rename positive-integer schema
* Clearer parameter comments
* dont skip mage tests when BE job is skipped
* checkout metabase for mage tests
* make mage tests its own thing
* add mage test workflow
* if it aint broke dont fix it
* remove dupe mage test from backend workflow
* fix: correct setting name check for static stats
also fix namespace for the defenterprise in token check to load the
embedding stats module
* fixup! use get-raw-value
* fixup! use-get-value-of-type
Fixes [QUE-838](https://linear.app/metabase/issue/QUE-838/linked-filters-are-not-working-on-public-dashboards).
Query params were getting parsed incorrectly to a map like `{:17fdc812 ...}`
with a keyword rather than a string. These failed to match any of the
dashboard's parameters (which are keyed by strings like `"17fdc812"`)
and returned no values (but did not throw any errors in prod).
Before, when you had an outer query referring to something in a nested field from the inner query, the query processor
would assume the field is still nested and generate invalid sql. Now, it will just refer to the alias.
* Move the lexifier to it's own file
* Remove duplicate type union member
* Remove unused typeinferencer
* Add ParseError type to errors
* Lexify straight in to pratt.Token
* Remove unused TOKEN type
* Use pratt.Token type everywhere
* Fix type
* Remove unused __support__/expressions helper file
* Ignore END_OF_INPUT when matching suggestions
Fixes#55929. (Requires a new export for 54 that also gets backported.)
Adds a new setting `install-analytics-database` that skips loading the
database at all, if set to false. That allows the dev instance that
exports the analytics content to have no real audit DB, and therefore to
generate the right `entity_id` hashes for the audit export!
Existing audit DBs are fine; the checksum will see these new entity_ids
in the export and overwrite any existing content.
* Implement splitPart() for BigQuery
* Always return string instead of null
* Better test failure message
It's running fine on my machine ;)
* Use coalesce instead
* fix: Ensure that local timestamps params work
Fixes#53371
By filtering a timestamp or date column on an OffsetDateTime it results
in an index being missed since the lhs needs to be upcast by the db engine
to a `timestamp with timezone`.
This ensures we only pass OffsetDateTime to `timestamp with timezone`
columns.
Context about decision here:
https://metaboat.slack.com/archives/C0645JP1W81/p1743111964032089
* Handle sending a datetime range rather than just a date
* Feature gate to dynamic dataset loading
* Fix kondo
* Remove comment
* Rename caseInsensitive to caseSensitivity
* Use literal string for case-insensitive template
* Update argument description to be more clear
* Support inline expression highlights
* Render help text description with Markdown
* Add more markdown to help text strings
* Use markdown code blocks to mark code in help text descriptions
* Highlight argument references in help text descriptions
* Fix arg alignment
* Fix HelpText unit tests
* Fix FunctionBrowser tests
* Render FunctionBrowser descriptions using Markdown too
* Mark true
* Fix e2e tests
* Fix e2e tests
* Remove commented-out code
Co-authored-by: Uladzimir Havenchyk <125459446+uladzimirdev@users.noreply.github.com>
---------
Co-authored-by: Uladzimir Havenchyk <125459446+uladzimirdev@users.noreply.github.com>
* Auto-categorize truncation units as datetime columns in insights code
* detect datetime units on the FE as well when rendering trend visualization
* prettier
* add unit test
* typo
* fix comment alignment
* Initial support for text wrapping in static viz
* Add text-align support
* Add row index support
* Fixing broken unit tests
* Adding new unit test coverage for Tanstack viz settings
* Fixing typo in whitespace; removing debug def
* Removing defs & outdated comment
* Addressing PR feedback
* Adding column width support to minibar columns; improving doc strings
* Adding minibar column width support; improving doc strings; moving consts to style namespace
* Fixing broken unit test after font size change
* Remove trailing whitespace
# Support Environment Variables for API Keys
### Overview
This PR adds support for using environment variables to define API keys in the configuration file, enhancing security and flexibility for deployment environments.
### Changes
- Modified API key validation to support environment variable substitution in the key field
- Added runtime validation for API keys from environment variables to ensure proper formatting
- Refactored tests to properly validate environment variable integration
- All API keys (including those from environment variables) still must follow the required format:
- Must be between 11-254 characters
- Must start with mb_
- Must match the pattern mb_[A-Za-z0-9+/=]+
### Usage
```
api-keys:
- name: "ENV API Key"
key: "{{env API_KEY_FROM_ENV}}"
creator: "admin@example.com"
group: "admin"
```
### Testing
- Creation of API keys from environment variables
- Validation of API key format from environment variables
- Error handling for improperly formatted environment variable values
# Skip keys if they exist
To guarantee that it is possible to keep the same yaml file on each instance initialization, if there is a key in the instance and the config file has a key with the same name, the creation is skipped.
* Minibar chart support in table static viz
* Fixing unit test; adding new minibar unit test
* Fixing clj kondo warnings
* Addressing PR feedback
* Removing dead minibar code
* Removing old minibar unit tests
* Empty
* Implement and test snowflake integer()
* Use card-with-source-metadata-for-query in snowflake tests
* Use lib/functions instead of lib/expression-clause in snowflake
* integer() for BigQuery
* Fix BigQuery tests for integer()
* Change to backticks
* kick ci
* Fix for BigQuery
* Capitalize name of field
* Use STRING type for casting in BigQuery
* Sometimes we get an integer back, and that's okay
* Use mt/card-with-source-metadata-for-query in bigquery tests
* use lib/functions instead of lib/expression-clause in bigquery
* feat: add remaining token check stats and settings
Adds the remaining five data points we want to report during the token
check request. Adds a new module to internal stats that does the
embedding counts and setting checks.
* fixup! log stats
* fixup! move enterprise settings into defenterprise
* fixup! remove row count
* fix: only remove vis_settings from dashboardcard
Fixes loading serialized dashboard cards where we would previously
delete and recreate them on each load.
In order to handle circular dependencies caused by links to other
dashboards in a dashboardcard's visualization_settings we were removing
the dashboardcards entirely from the dashboard model to break the cycle.
Later when we traversed back through the dependency path we would load
the original dashboard a second time with the restored dashboardcards.
However, on the load of the cycle breaking change we could entirely drop
dashcards belonging to the dashboard because the serialization code was
treating our cycle breaking changes as an edit to the dashboard to drop those
cards. This meant when we got back to the the second load of the
dashboard we would create the dashcards fresh instead of updating
existing instances.
The fix here is that we only remove the values from the dasboard card
model that can cause the cycle -- in this case just the
visualization_settings column. This seemed the most straightforward fix,
but there are other possibilities here.
One is that we could _just_ preserve the entity_id column in dashcards
which would eliminate the possibility of any other cycles here.
Another option would be to change how we handle nested models in serialization.
There is not really a foreign key dependency here between dashboard A
and dashboard B linked by the dashboard card. If we instead just created
those dashboards first _then_ went back and created their dashcards we
don't have any problems at all. We could move nested models up to the
same level of handling in serialization that we do for models, which
would let us create the dashboards first then create any dashboard cards
in a subsequent step.
* fixup! update docstring
Closes QUE-794 and QUE-705.
This will enable us to *know*, rather than *infer*, what version of
`report_card` we're looking at.
One example: in the near future, I'm going to add some values to
`result_metadata` on cards. If we later discovered a bug in that logic
and could no longer trust those fields written to AppDB in
`result_metadata`, the `card_schema` will tell us with certainty
whether the card was saved with the broken logic (`:card_schema N`)
or the new logic (`:card_schema (+ N k)`).
The upgrade steps could drop the invalid fields and, if desired, infer
the correct ones.
Note that there's nothing here about writing back! This is about
cleaning up old cards during `after-select`.
There are no upgrades yet; the current version is still the default.
```
2025-03-27 15:25:21,322 INFO sync.describe-database :: Checking "PUBLIC"."LARGE_NUMBERS"
2025-03-27 15:25:21,757 INFO sync.describe-database :: "PUBLIC"."LARGE_NUMBERS": SELECT privileges confirmed
2025-03-27 15:25:21,759 INFO sync.describe-database :: Checking "PUBLIC"."SINGLE_ROW_TABLE"
2025-03-27 15:25:21,975 INFO sync.describe-database :: "PUBLIC"."SINGLE_ROW_TABLE": SELECT privileges confirmed
2025-03-27 15:25:21,976 INFO sync.describe-database :: Checking "PUBLIC"."SLOW_QUERY_VIEW"
2025-03-27 15:25:22,219 INFO sync.describe-database :: "PUBLIC"."SLOW_QUERY_VIEW": SELECT privileges confirmed
```
This will list all tables it enumerates and then which ones it
removes. There are also two error handling states:
```
2025-03-27 15:29:25,418 INFO sync.describe-database :: "schema"."some_table": SELECT privileges confirmed
2025-03-27 15:29:25,419 INFO sync.describe-database :: "schema"."some_table": Assuming SELECT privileges: caught timeout exception
2025-03-27 15:29:25,419 INFO sync.describe-database :: "schema"."some_table": Assuming no SELECT privileges: caught exception
java.lang.Exception: Some exception
at metabase.driver.sql_jdbc.sync.describe_database$eval194718.invokeStatic(NO_SOURCE_FILE:33)
at metabase.driver.sql_jdbc.sync.describe_database$eval194718.invoke(NO_SOURCE_FILE:33)
[more stacktrace from the exception]
```
This is essentially just changing the log level. We can and perhaps
should change these back to debug or trace in the future but this is
useful for now.
* WIP Implement `integer()` for all :sql descendants
* Add test-data to tests
* Just do MySQL
* Undo change to query_processor_test
* Remove unused imports
* Remove extra blank line in query_processor
* Convert postgres tests to mysql
* Appease MySQL syntax
* Remove old tests for mysql integer()
* Remove unsupported postgres tests
* Fix kondo
* kick ci
* Fix up for MariaDB
* Fix queries in test
* Try bigger numbers for MariaDB
* Remove ability to cast integer to integer
* Check that boolean = 1 in mysql
* Use mt/card-with-source-metadata-for-query
* Use lib/functions instead of lib/expression-clause
* Use lib/functions instead of lib/expression-clause in postgres
* Use card-with-source-metadata-for-query for postgres tests
* Update some function names to match modern casing
* Fix test cases
* Update casing in docs for isEmpty, notEmpty, lTrim, rTrim, isNull, and notNull
* Fix isNull and isEmpty casing in tests
* Fix case in unit test
* Fix casing in e2e tests
* Fix couldn't find Slack channel
When the first `conversations.list` doesn't return the Slack channel we
want, we need to continue to paginate until we found it.
* [debug] for testing
* Revert "[debug] for testing"
This reverts commit 67dd2c0405.
* Simplify logic
Co-Authored-By: Timofey Kachalov <timofei@metabase.com>
---------
Co-authored-by: Timofey Kachalov <timofei@metabase.com>
* rename and reorganize
# Conflicts:
# enterprise/frontend/src/metabase-enterprise/google_drive/GdriveConnectionModal.tsx
# enterprise/frontend/src/metabase-enterprise/google_drive/GdriveSyncStatus.tsx
* resolve conflicts
* only show sync status to admin who created it
* Add connection info to disconnect modal (#55588)
* rename and reorganize
# Conflicts:
# enterprise/frontend/src/metabase-enterprise/google_drive/GdriveConnectionModal.tsx
# enterprise/frontend/src/metabase-enterprise/google_drive/GdriveSyncStatus.tsx
* resolve conflicts
* update disconnect modal
* better spacing
Improve the performance of the collection permission query by using a
semi-join instead of an inner join to the permission tables. This
ensures each collection only needs to be tested once for a route that
gives a user permission to access it rather than multiple times.
This reduces the query time for users that are members of lots of
collections since we dont need postgresql to create large join tables or
return the same collection id more than once from this query.
* test: make dashboard flaky test more robust and stable
* revert flaky
* add additional check
* fix that works locally
* revert prev version
* add missing line
* Add baseType prop to SemanticTypePicker
* Update unit tests
* Fix legacy v1/ Field['base_type'] typing, to make it same as Field['base_type'] from api/
* Use isa
* Add a hack for numerical types
* Fix typo
* Pass field
* Reuse type
* Use effective type
* First pass on the tech doc
* Handle booleans
* Use isa
* Refactor
* Update test
* Update test
* Assume no type can have 2 level-one ancestors
* Add comments
* Extract getCompatibleSemanticTypes
* Refactor isDeprecated
* Refactor
* Add fieldId param
* Add test for booleans
* Make initialValue required
* Do not assume a field cannot have multiple level-one ancestors
- these types have 2 level-one ancestors: type/JSON, type/XML, type/DruidJSON, type/OracleCLOB
* Extract FIELDS
* Add test
* Test Entity Name
* Remove '
* Add test
* Update test
* Add test for base_type fallback
* Update comment
* Refactor tests
* Add Field containing JSON tests
* Refactor
* Add a comment
* Improve naming
* Improve naming
* Use SemanticTypePicker in DatasetFieldMetadataSidebar
* Use Field from types/api
* Extract DatasetFieldMetadataSemanticTypePicker
* Fix typing
* Pass label, tabIndex, and onKeyDown
* Remove old picker
* Use SemanticTypePicker in FieldTypeDetail
* Remove unused prop
* Remove redundant margin
* Fix#55617
* Use SemanticTypePicker in Field
* Remove old SemanticTypePicker
* Reuse FkTargetPicker
* Introduce new top-level frontend directory: frontend/src/metabase/metadata
- Move semantic type picker + subcomponents in there
* Add readme
* Get rid of global CSS classes
* Remove old FKTargetPicker component
* Pass databaseId to components
* Don't check for text-like fields
* Merge branch 'SEM-97-limit-semantic-types' into SEM-176-new-semantic-type-picker-everywhere
* Update test
* Update tests
* Sort options by label
* Update test
* Fix docs links
* Change reproductions suite to TypeScript
* Repro #55618
* Repro #55617
* Make sure combobox flips if there is no space underneath
* fix: add index to field_usage query_execution_id
Supports faster deletes on the query_execution table. Without this index
deleting rows from that table leads to a seq scan on the field usage
table for each row deleted to handle the ON DELETE CASCADE trigger.
* fixup! drop empty preconditions
* Add baseType prop to SemanticTypePicker
* Update unit tests
* Fix legacy v1/ Field['base_type'] typing, to make it same as Field['base_type'] from api/
* Use isa
* Add a hack for numerical types
* Fix typo
* Pass field
* Reuse type
* Use effective type
* First pass on the tech doc
* Handle booleans
* Use isa
* Refactor
* Update test
* Update test
* Assume no type can have 2 level-one ancestors
* Add comments
* Extract getCompatibleSemanticTypes
* Refactor isDeprecated
* Refactor
* Add fieldId param
* Add test for booleans
* Make initialValue required
* Do not assume a field cannot have multiple level-one ancestors
- these types have 2 level-one ancestors: type/JSON, type/XML, type/DruidJSON, type/OracleCLOB
* Extract FIELDS
* Add test
* Test Entity Name
* Remove '
* Add test
* Update test
* Add test for base_type fallback
* Update comment
* Refactor tests
* Add Field containing JSON tests
* Refactor
* Add a comment
* Improve naming
* Improve naming
* Use Field from types/api
* Fix typing
* Don't check for text-like fields
Previously we looked at 1000 users and got the set of unique user
attribute names from them. So if 1000 users each had a `name` and user
1001 had `some-other-attribute`, we would give you back `name` as the
complete set of available attributes.
This modifies the magic number (to 5000 from 1000) and the logic. We
select the first 5000 unique *attributes*. So if 1000 users each have a
`name`, that only counts as 1, and we'll continue looking for other
unique attributes before returning to the client.
Implemented validation that ensures API keys when created with config.yml:
Start with the mb_ prefix
Have a minimum length of 12 characters and max 254
Prefix (first 7 characters) must be unique
String must be Base64
* Use field refs instead of field ids/names for tiles queries
* Fix up embedded + public endpoints
* use field_refs
---------
Co-authored-by: Aleksandr Lesnenko <alxnddr@gmail.com>
* resetting qb state before running card on back navigation
* try something else
* adding e2e test for notebook navigation
* wait for fk call in test
* increase timeout
* e2e adjustment
* Control visible entities in multi-stage data picker (#54845)
* Add props to DataSelector, and keeps the same behaviors.
* Read entity_types from URL search params
* Fix type errors
* Fix SDK not rendering multi-stage data picker
* Add redux slice tests
* Add E2E tests
* WIP add integration tests
* Add all integration tests
* Put tests file in a proper folder
So the SDK build process doesn't check test file types
* Cover more E2E scenarios
* Cover the new code with a test
* Add more assertions to make sure more paths are tested
* Fix imports
* Make functions easier to understand
* Fix test case name typo
Co-authored-by: Nicolò Pretto <info@npretto.com>
* Clarify embed state type
* Address review: Make code more concise
Co-authored-by: Romeo Van Snick <romeo@romeovansnick.be>
---------
Co-authored-by: Nicolò Pretto <info@npretto.com>
Co-authored-by: Romeo Van Snick <romeo@romeovansnick.be>
* Control visible entities in simple data picker (#55329)
* Filter simple data picker by "entity_types"
* Add a happy path for multi-stage data picker E2E test
* Fix loading a wrong entity
* Control visible entities on new item menu dropdown (#55331)
* Control visible entities on new item menu dropdown
* Hide action following product discussion
* Make a new menu item more accessible
* Add tests
* Control visible entities in Browse sidebar (#55332)
* Control visible entities in Browse sidebar
* Add relevant tests
* Fix test case typos
Co-authored-by: Nicolò Pretto <info@npretto.com>
* Add comment to clarify magic value
Co-authored-by: Nicolò Pretto <info@npretto.com>
---------
Co-authored-by: Nicolò Pretto <info@npretto.com>
---------
Co-authored-by: Nicolò Pretto <info@npretto.com>
Co-authored-by: Romeo Van Snick <romeo@romeovansnick.be>
* Empty
* Add split-part backend (#55425)
* feat: prepare fe part of split-part (#55483)
* Add split-part backend
* feat: prepare fe part of split-part
* rename function and add test
* use postgres for testing
* add test for an error
* add db flag
* remove docs page
* update copy
---------
Co-authored-by: Eric Normand <ericwnormand@gmail.com>
---------
Co-authored-by: Uladzimir Havenchyk <125459446+uladzimirdev@users.noreply.github.com>
Co-authored-by: Uladzimir Havenchyk <uladzimir.dev@gmail.com>
* feat: prepare fe part of url-pathname
* fix dropdown styling
* fixup example
* rename function to path and update example
* Backend for url-pathname
* Rename to path()
* Backend tests for `path()`
* Frontend and Backend for url-pathname (#55353)
* feat: prepare fe part of url-pathname
* fix dropdown styling
* fixup example
* rename function to path and update example
* Backend for url-pathname
* Rename to path()
* Backend tests for `path()`
---------
Co-authored-by: Uladzimir Havenchyk <uladzimir.dev@gmail.com>
* Fix kondo
* Fix kondo
* feat: prepare fe part of url-pathname (#55291)
* fix description
* fix styling
* cleanup
* fix rebase
* fix rebase
* add tests
* add tests
* update test
* fix test
* address review
* prevent url translations
* make tests less flaky
* update copy
* add category
* remove docs page and add feature flag
---------
Co-authored-by: Uladzimir Havenchyk <uladzimir.dev@gmail.com>
Co-authored-by: Uladzimir Havenchyk <125459446+uladzimirdev@users.noreply.github.com>
* Convert typeinferencer to typescript
* Split up diagnostics
* Split of lib check from diagnostics function
* Move getDatabase to util file
* Move isErrorWithMessage to utils
* Add getExpressionMode helper
* Simplify branches
* Move resolveMBQLField to separate module
* Simplify expressions import
* Remove unused options
* Fix type of mock env
* Move compiler to separate file
* Remove unused arguments to compiler
* Move passes to separate file
* Return expression from diagnose
* Remove unused processSource helper
* Fix types in passes
* Make resolving optional in compiler
* Add fuzz testing for compiler
* Mock legacy mbql to clause in fuzz test
* Remove need for compare
* Use default parameters
* Remove compare and compareSeed
* Remove unused recursive-parser
* Remove fuzz testing for recursive parser
* Remove unused arguments
* Colocate helper with the tests that use it
* Use compileExpression in formatter tests
* Use compileExpression in typeinferencer tests
* Order compiler passes
* Move StartRule and ClauseType to metabase-lib/v1/expressions
* Remove useShorthands pass
* Fix token escaping in pratt compiler #54751
* Remove POPULAR_ functions and aggregations
* Move compiler passes out of pratt compiler
* Make getMBQLName optional
* Add missing adjustBooleans pass
* Type startRule in compiler
* Move resolverPass to field-resolver
* Add isCompilerPass helper
* Allow use of short-circuiting for disabling passes
* Only render friendly errors
* Mark ExpressionError and ResolverError as user-friendly
* Add check for missing comma in argument list
* Simplify diagnostic checks
* Always return error and expression
* Make compileExpression and diagnoseAndCompile generic to StartRule
* Remove diagnoseAndCompileExpression helper
* Remove unused completer utils
* Move enclosingFunction to expressions module
* Fix typeinferencer tests
* Fix e2e test
* Detect cycles
* Inline passes
* Flatten compiler
* Remove prevError
* Remove default from diagnoseAndCompile type definition
* Use try/catch to simplify diagnoseAndCompile
* Wrap diagnostic errors in their own type
* Remove unnecessary wrapping of diagnostic error
* Move errors to top-level expression module
* Remove duplicate error types in expressions
* Fix error format in tests
* Add comments to passes
* Add tests for passes
* Fix lint
* Fix type imports
* Flesh out references in test data with base-type
* Use fleshed out field reference in all test cases
* Add boolean expression to test data
* Port recursive-parser tests for compiler
* Fix support for unary +
* Add cast support to test database
* Move cast calls to separate test
* Add test for escaping identifiers
* Unskip test
* Fix compiler tests to support top-level literal
* Typo
* Use normal quotes
* Fix typeinferencer for value clause
* Use all passes by default
* Fix pratt compiler tests
* Unskip garbage test
* Use dedent from ts-dedent
* Remove ClauseType helper for expression and make everything use Lib.ExpressionClause
* Coerce clause to be expression clause in filter and aggregation pickers
* Fix diagnostic test
* Wrap diagnostic error in a DiagnosticError
* Use instanceof to check type of error
* Do not assume all diagnostic errors are human-friendly
* Update translation sources 2025-03-17
* Update translations
* don't add lithuanian yet
* language is always locale
* Fix the `ttag` i18n calls for pluralized strings
Pluralized strings *without* the number interpolated into them (eg.
"Month" and "Months") were not getting found by `i18n/trun` in CLJS,
which made them fall back to English.
This fixed the `ttag/msgid` call to be correctly formed in both cases.
* update some localized strings
---------
Co-authored-by: github-actions <github-actions@github.com>
Co-authored-by: Ryan Laurie <iethree@gmail.com>
Co-authored-by: Bryan Maass <bryan.maass@gmail.com>
Co-authored-by: Braden Shepherdson <braden@metabase.com>
We are running a probe query on snowlfake to see if the current db user
can see views
```
select true from X where 1 <> 1 limit 0
```
Something like this.
But on large views, it seems snowflake does not quickly determine this
has no rows and it can take a while.
This leads to the connection being closed. Then in the error handler we
attempt to rollback. But if the connection is closed _this_ throws and
we have now thrown from the error handler and we break sync.
This does two things
1. set a timeout on sql statements. This does 10 seconds.
2. If a timeout occurs, assume that they _can_ see the object
The idea is that timeouts might happen but it's trying to run a query
that it would quickly reject if the permissions were lacking. So
timeouts mean you can see the data, it's just taking a while
* feat: add embedding stats to token request
Adds the embedding_users, internal_users, domains, and query_execution_*
stats to the token check request in order to track embedding usage.
This creates a new internal_stats module to share code between the
anonymously reported stats the stats for the token check request to
avoid some circular depedencies. This also seperates creation of a stat
and how it is reported, so we could move over other stats calculated for
the phone home metric.
* temp! add a logging statement to figure out why CI is different
* fixup! exclude api-key email
* fixup! exclude api-key/internal type users
* fix localhost:* in embedding app origins sdk
We need to call the setting's getter directly, and not use
`setting/get-value-of-type`. The setter it adds localhost:* which is required.
* add test
* test fix
* update module config
* improve test feedback
* fix ci
* use whitelabel colors in the static viz
* Fix backend whitelabel test for app colors
* update CI
---------
Co-authored-by: Tyler Plude <tsplude@gmail.com>
* Fix cond fmt of null vs empty str in table static viz
* Adding unit test
* Add TextWrapper support in json exports
* Updating unit tests with TextWrapper
* Empty
* Add `value` support in the expression editor (#55203)
* Literals in filters tests (#55283)
* E2E tests for literals in custom columns (#55288)
* E2E tests for literal custom columns in other clauses and fixes (#55295)
* Move changes from the FE branch
* Move changes from the FE branch
* Move changes from the FE branch
* Fix filter schema and false = false filter (#55417)
* Fix date/string literals effective type (#55427)
* Fix date/string literals effective type
* Fix date/string literals effective type
* Fix date/string literals effective type
* Fix date/string literals effective type
* Fix date/string literals effective type
* Fix date/string literals effective type
* Fix date/string literals effective type
* Fix date/string literals effective type
* Forbid top-level constants until the BE is fixed (#55459)
* Use matches
* Make test flaky
---------
Co-authored-by: Mike Appleby <86076+appleby@users.noreply.github.com>
* ci(sdk): adds embedding-sdk-tests-result
* ci: should run the sdk tests as part of the embedding-sdk workflow
* let's try without concurrency
* rename "frontend" -> "embedding SDK"
* Integration
* Support for `text` MBQL function on the FE (#55274)
* Cast text backend on postgres (#55292)
* Add `text()` custom expression that casts to text
* Add :cast to list of all possible features
* Complete testing plan
* Fix kondo warnings
* Fix test
* Empty
* Add e2e tests (#55361)
* Fix failed tests for text() function (#55374)
* Add UI for integer() function, with e2e tests (#55366)
* Set category
* Set category
* Set category
* Set category
* Fix tests
* Cast integer backend (#55381)
* integer() custom expression with simple tests
(more tests coming)
* MOAR tests
* EVEN MOAR tests
* Change type to BigInteger
---------
Co-authored-by: Eric Normand <ericwnormand@gmail.com>
This adds coercion support for serialization and adds a very limited partial schema for a Dashboard, in hopes that
this grows to be a full schema for our data models at some point.
fixes#54941
* sync translations with crowdin
* remove poeditor integration
* fix run name
* update to new project
* sync
* path
* move to config
* not master lol
* kick ci
* update crowdin config
* remove croatian
* roflcopter
* token
* maybe fetch harder
* test code - revert this
* test
* more tokens
* just use the cli
* line breaks
* le facepalm
* no verify
* token swap
* auto approve
* revert
* remove zh e2e test
* remove test code
* don't backport translations
* don't backport translations
* Add FunctionBrowser component
* Move header into CombineColumns for consistency
* Render the function browser
* Allow wrapping shortcuts
* Render toolbar buttons on top again
* Add missing Lib import
* Fix e2e test
* Use ellipsis in placeholder
* Add test-id to function browser
* Add e2e tests for function browser
* Add category to help text configs
* Only render supported functions in function browser
* Convert to helptext in getFilteredClauses
* Group functions by category
* More e2e tests
* Add empty state to function browser search
* Add more breakpoints for expression editor layout
* Move function browser item into its own component
* Increase width of function browser
* Add case to functions
* Remove unnecessary maw prop
* Sort functions
* Add tests for FunctionBrowser
* Remove nested component definitions
* Use standard breakpoints
* Add spacing between toolbar buttons
* Fix functions test
* Add category to distinct-if
* Fix tests that were broken by sorting
* Fix unit test examples
* Add more unit tests for filtering
* Do not enter parentheses when there are no arguments to the clause
* Sort by displayName
* Sort by structure
* Add e2e about clauses with no args
* Fix sorting test
* Fix font sizes for function browser
* Fix function browser spacing
* Allow replacing selection with function browser
* Context Driven Indicator Menu
* add unit tests
* unit test adjustments
* Ho
C time
* hasSeenAll
* cleanup, passing tests
* adjusting some tests, PR feedback
* more small adjustments. Starting to add docs
* hopfully final adjustments
* adjusting storybook path
Closes QUE-764.
Adds support for returning "remapped" columns, which have not been
supported in the library previously.
- Refactor `lib.field/remapped-field` to `lib.metadata`
- Add `lib.metadata.calculation/remapped-columns`
Co-authored-by: Alexander Polyankin <alexander.polyankin@metabase.com>
* fix: improve startup by not scheduling all dbs
Improve long startup times by not scheduling tasks for all dbs on
startup. Due to the lack of bulk operations in quartz if an installation
has 500 dbs we need to load and scan through all associated db scanning
triggers 500 times for all dbs which means loading about 500k records in
1000 operations.
This is generally pretty fast on a per operation basis (about .5s
in local development), but on some postgresql installations this can
blow up to a few seconds per db, which makes the whole operation take 40
minutes plus. Additionally there maybe both lock contention and resource
contention between the quartz scheduler and the main app thread that
draws out this time.
Instead we want to init the schedule and put it into standby mode -- where
we can schedule tasks but are not executing them -- before we process
the configuration file. That we can schedule all necessary tasks as part
of loading the database from the config file and not need to then
iterate all database to schedule their tasks later in the startup.
Long term, it would be good to support rescheduling tasks using a bulk
operation in quartz (there are TODOs in their code base for supporting
this) rather than needing to update each trigger individually.
* fixup! doc string for check-health! function
* fix some mage installer logic
* make `cd bin && ./mage` work
get the absolute path, + cd to mb repo root in subshell
---------
Co-authored-by: Bryan Maass <bryan.maass@gmail.com>
* do not setup sandboxing if using a sample database
* do not ask for tenancy columns if using a sample database
* alter copywriting when in sample database mode
* improve copywriting for scenarios with a sample database
Co-authored-by: Jeff Bruemmer <jeff.bruemmer@gmail.com>
* fix misleading comments
Co-authored-by: Mahatthana (Kelvin) Nomsawadi <me@bboykelvin.dev>
---------
Co-authored-by: Jeff Bruemmer <jeff.bruemmer@gmail.com>
Co-authored-by: Mahatthana (Kelvin) Nomsawadi <me@bboykelvin.dev>
The backfill job originally retried up to 20 times and then gave up. However, some instances apparently have ~150
duplicates or more, so 20 retries is insufficient. In addition, 2 db calls per conflict is sort of a problem when the
conflict count is O(duplicates^2). This bumps up the retry limit and reduces the db calls to 1 per 50 conflicts.
* Disallow navigating from model/{slug}/query and model/{slug}/metadata to model/{slug} when there are unsaved changes
* Reset state
* Simplify reset
* Use appropriate redux action when programmatically invoking navigation
* Disallow navigating from metric/{slug}/query and metric/{slug}/metadata to metric/{slug} when there are unsaved changes
* Remove mentions on inexisting /metric/metadata and /metric/{slug}/metadata routes
* Rename test
* Add repro for #32037 (models)
* Improve assertions
* Fix state cleanup
* Fix flaky test
* Deal with race conditions
* Add repro for metrics
* Correctly reset ui state when cancelling editing a model/metric
* Add repro for editing model's query
* Remove log
* Add a comment
* Revert "Skip tests that deal with the click outside and escape behaviour of the custom expression editor"
This reverts commit 2c05bc08c2.
* Revert "Remove CloseModal component"
This reverts commit ad1795e05a.
* Revert "Move usePreventClosePopover into utils"
This reverts commit de31b21282.
* Add PreventPopoverExit util
* Remove use of non-existant hook
* Prevent popover from closing on outside click or escape
* Rename argument to avoid confusion with double negatives
* Prevent custom expression editor from closing in aggregation sidebar
* Allow exiting the component when navigating away from custom expression editor
* Add a comment explaining why the useClickOutside does not work
* Add comment on how CloseModal works
* Remov empty file
* Add comment on what children should be
* Allow overriding cancel button text on ConfirmModal
* Allow passing more props to ConfirmModal
* Fix primary button on ConfirmModal
* Use ConfirmModal for custom expression editor
* Fix e2e test
* Remove second message in closemodal
* Update loki
If you run an uberjar with `MB_RUN_MODE=dev`, you get an exception
because you try to run the `watch-directory!` bit (which only works for
actual files) in an uberjar environment.
Let's just print a warning but continue along.
* Rename arg to caseInsensitive
* Add matcher for internal [:dimension name]
* Add formatting support for internal dimension clause
* Use raw MBQL in examples
* Format examples at certain width
* Remove unused suggest code
* Fix unit test for help text
* Add unit test for rendering examples on multiple lines in HelpText
* Improve spacing
* Make sure that all help texts can be formatted properly
* Add custom formatExample helper to format example expressions
* Use useAsync to format example
* fixup! Add custom formatExample helper to format example expressions
* Add HighlightExpression component
* Export all modules
* Fix `row` type to be `RowValue[]`
* Use richer `clicked` object in object detail view
* Fix column settings not available in detail view
* Add e2e test
* Rename Select to LegacySelect
* Convert frontend/src/metabase/lib/core.js to TypeScript
* Format code
* Extract SemanticTypePicker
* Fix type
* Fix styling
* Fix nulls
* Add nothingFoundMessage
* Extract DefaultSelectItem
* Add CurrencyPicker
* Optimize
* Add checked icon to CurrencyPicker
* Improve overflow in SelectItem
* Add FkTargetPicker
* Style font-weights to be the same as in other inputs around
* Fix colors
* Add a comment
* Fix imports
* Style dropdowns
* Make currency input full width
* Add Combobox component
* Show currency symbol next to value
* Modernize field settings button
* Modernize FieldVisibilityPicker
* Update tests
* Update test
* Update and merge tests
* Update tests
* Remove bold, fix padding and move components to their own dirs
* Update unit tests
* Update e2e tests
* Update tests
* Reduce flakiness
* Add comment
* Repro #53595
* Replace CSS modules overrides with props
* Fix symbol styling for selected option in CurrencyPicker
* POC - moving two pivot functions into CLJC
* Debug logs in frontend pivot impl
* POC continued
* setting isCollapsed consistently
* Working replication of valuesByKey map construction
* sorting logic
* hook up new code to the FE impl
* Fixing small potential bug in valuesByKey
* Fixing bug in clojure vs javascript string key for valuesByKey
* fix valueColumns key name
* Some performance improvements + cleanup
* A couple more attempts at perf improvements
* Fix bugs
* port format-values-in-tree
* WIP add-subtotals
* @tsp - Improved build-values-by-key perf; some debug perf lines in frontend land
* @tsp - in progress conversion of add-subtotals
* @tsp - working converted Pivot.add_subtotals
* @tsp - fixing bug in add-subtotal mapcat recursion
* @tsp - add-subtotal restoring map to mapcat (oops)
* @tsp - adding working Pivot.add_value_column_nodes to replace addValueColumnNodes
* @tsp - removing debug logs & perf timing logs
* Appease whitespace linter
* More formatting code ported, cleanup
* Fix bugs, cleanup
* Fix some tests
* Start hooking up the BE, fix some bugs
* Fix lint/formatting errors
* Fix comparator + i18n
* Fix another module issue
* Test using js-specific i18n function
* Revert "Test using js-specific i18n function"
This reverts commit 9a8604b583885f1c35f48507112e3a142ca0d3a0.
* Bump up timeout on cypress assertion
* WIP - migrate row section getter to cljc
* fix tests
* Fix e2e test
* Switch exports over to fully using CLJC code paths, delete now-unused code
* Delete outdated postprocess tests
* Bug fix
* Fix build
* cleanup, consolidation
* fix function call
* more cleanup
* empty test ns
* misc cleanup/docstrings
* WIP tests
* more tests and refactor
* small fix
* fix getting column title for value columns
* Fix some tests
* Fix another test
* Revert a couple changes
* Fix some more tests
* Fix more tests, fix default values for pivot viz settings
* Ensure headers respect format-rows? settings in exports, and fix more
tests
* Fix lint issues
* fix another test
* Fix a test, a little cleanup
* Try to fix another test by setting format-rows middleware to false for subscription attachments
* Revert "Try to fix another test by setting format-rows middleware to false for subscription attachments"
This reverts commit d860522a6d17c27e67afabccf119039508c96cca.
* Update formatting logic to fix another test
* Fix conditional formatting
* Fix last BE test
* remove test not needed anymore
* Fix drill through + more cleanup
* clean ns
* remove old code from .clj-kondo/config.edn
* add a comment + tweak keywords
* perf improvements and refactoring the CSV export path
* Performance improvements
* Fix bad rebase
* more perf improvements
* fix test
* lint
* oops, fix test
* more optimizations, memoize backend formatting
* fix range-based conditional formatting
* remove unused function
* add some new backend unit tests
* fix FE type checks - caused by an experiment that I didn't revert fully
* address PR review feedback
* fix up tests
---------
Co-authored-by: Tyler Plude <tsplude@gmail.com>
We are adding entity_ids to dbs, tables, and fields, but existing rows don't have entity_ids yet, and newly created dbs/tables/fields don't have entity_ids when created. This job adds those missing ids.
Ideally, we'd copy them too, but we need to decide if we should duplicate them or not (do a deep or shallow copy) is a
too hard to make immediately, and this at least will unblock people being unable to duplicate dashboard with actions
defined.
somewhat improves on #38224
* setup discouraged-tag linting for #p
- this is fine to use in dev, but shouldn't get comitted
so this will fail kondo linting.
* sample #p commit (check linting)
* Revert "sample #p commit (check linting)"
This reverts commit 171130c668.
I added it to double check the linter will complain correctly
API Key Configuration via config.yml
Overview
This PR adds support for configuring API keys through the config.yml file in Metabase Enterprise. This enhancement allows administrators to define and manage API keys declaratively through configuration, making it easier to set up and maintain API access across different environments.
Features
Adds support for defining API keys in config.yml under the api-keys section
API keys can be configured with:
name (required)
group ("admin" or "all-users")
creator (email of the user to assign it, user must exist in the instance or be created in the config file, must be superuser)
Automatically creates API keys on Metabase startup if they don't already exist
Skips creation of API keys that already exist (based on name)
Validates configuration including:
Required fields
Valid group values
Creator permissions
Example Configuration
version: 1
config:
api-keys:
- name: "My Admin API Key"
group: admin
description: "API key for admin operations"
creator: admin@metabase.com
- name: "My All Users API Key"
group: all-users
description: "API key for general operations"
creator: admin@metabase.com
Security Considerations
API keys are only created by admin users
Keys are properly hashed before storage
Existing keys are preserved to maintain continuity
Group validation ensures keys have appropriate permission levels
Use Cases
Automated deployment and configuration
Infrastructure as code
Consistent API key setup across environments
DevOps automation
Testing
Unit tests cover key creation, validation, and error cases
Tests for configuration parsing and application
Validation of group permissions
Error handling for invalid configurations
This enhancement is part of Metabase Enterprise's advanced configuration capabilities, providing more flexibility in managing API access through configuration files.
* Use timezoned timestamp for export filenames
* clean ns
* tweak docstring
* add unit test + make namespace aliases consistent
* use new function in xlsx exports
* feat: add malli schema, test converage to ldap api
Restores the malli schema that was causing e2e tests to fail on the
first attempt and updates those e2e tests to look for the new error
message returned when the api request is rejected due to schema
validation issues.
Expand test cases around ldap connection testing errors so those
messages can be caught on future api changes.
* fixup! error capitalization
* Add reproduction for #54638
* Add reproduction for #54722
* Add reproduction for #31964
* Correct user of metabase issue in test name
Co-authored-by: Kamil Mielnik <kamil@kamilmielnik.com>
* Correct use of issue name in test title
Co-authored-by: Kamil Mielnik <kamil@kamilmielnik.com>
* Use addCustomColumn helper
Co-authored-by: Kamil Mielnik <kamil@kamilmielnik.com>
* Use helpers to open filter and summarize popovers
* Check for textbox role instead on cm-content class
* Just use enter
---------
Co-authored-by: Kamil Mielnik <kamil@kamilmielnik.com>
I enforced this by checking the return values of `visible-columns` and
`returned-columns`. I want to check that in (for dev and test only) but
it fails tests currently.
The trouble is that cards don't have `:ident` in their `result_metadata`.
That's coming later, so the enforcement flag is disabled for now.
* Truncate mrkdwn links
They could exceed block limits
* Truncate header block text
* Remove incorrect (testing) block in escaping test
* Make length consts under test public
* Add link truncation test
* Add header truncation tests
* Use lezer parser for custom expressions (#53540)
* Add @lezer/generator
* Add missing value on Token
* Add script to generate lezer grammar parser
* Add @lezer/lr
* Add more tests to tokenizer
* Add lezer grammar for custom expressions
* Generate lezer parser for custom expressions
* Convert TOKEN and OPERATOR to enums
* Make Token type more specific
* Add isReference to identifier token
* Typo
* Add more tests for tokenizer
* Add tokenize alternative that uses lezer grammar
* Remove old tokenizer
* Update location of grammer in generate rule
* Add test cases for tokens that share a prefix
* Merge tokens when necessary
* Only add character error when text is 1 character long
* Reduce the need to manually merge nodes
* Stop the iteration when node is found
* Update tests to be more specific
* Simplify tokenize
* Uncommit edge cases
* Generate parser as js
* Move previous tokenizer tests into expressions folder
* Add typing to unit test
* Allow escaping [] in reference
* Add test for escaping closing bracket
* Handle invalid parses manually
* Add support for signed expressions
* Document parser more
* Add custom tokenizer for Reference
* Make tests more explicit
* Fix e2e test
* Merge old test suite into tokenizer test suite
* Move @lezer/generator to devDependencies
* Custom expression suggestions (#53679)
* Add new suggesters for custom expressions
* Add test for joined fields
* Move complete into separate folder
* Split up completions into separate files
* Split up completion tests too
* Copy more tests from legacy completer
* Fix types
* Lezer highlight for custom expressions (#53651)
* Rename generated parser to lezer
* Add lezer highlighting rules to parser
* Add HighlightExpression component
* Use HighlightExpression in expression HelpText
* Add type annotation to highlighter
* Add test for highlight
* Simplify unit test
* Move syntax highlighting to shared module
* Rename util.tsx to util.ts
* Add monospace font variable
* Use monospace font from theme for syntax block
* Use relative font size in HighlightExpression
* Use mantine var for border-radius
* Add test id to example
* Fix tests that rely on example content
* Move highlighting code to metabase/ui/syntax
* Replace ACE with codemirror in custom expression editor (#53746)
* Rename generated parser to lezer
* Add lezer highlighting rules to parser
* Add HighlightExpression component
* Use HighlightExpression in expression HelpText
* Add type annotation to highlighter
* Add test for highlight
* Simplify unit test
* Move syntax highlighting to shared module
* Rename util.tsx to util.ts
* Add monospace font variable
* Use monospace font from theme for syntax block
* Use relative font size in HighlightExpression
* Use mantine var for border-radius
* Add test id to example
* Fix tests that rely on example content
* Move highlighting code to metabase/ui/syntax
* Replace custom expression editor with CodeMirror editor
* Add test mock for CodeMirror
* Colocate custom expression tests (#53812)
* Remove unused DISPLAY_QUOTES
* Colocate custom expressions diagnostic tests
* Colocate custom expressions process tests
* Colocate custom expressions recursive-parser tests
* Colocate custom expressions typeinferencer tests
* Colocate custom expressions completer tests
* Colocate custom expressions format tests
* Colocate custom expressions resolver tests
* Colocate custom expressions index tests
* Colocate custom expressions pratt compiler tests
* Colocate fuzz tests with custom expressions code
* Update paths for fuzz tests
* Fix types in fuzz test
* Fix types in process test
* Remove unused var
* Remove unused var in generator
* Merge types for custom expressions (#53902)
* Add missing expression types to metabase-types
* Add more explicit types to matchers
* Use Expression type from metabase-types everywhere instead of custom type
* Move matchers to separate file
* Further use Expression types for matchers
* Add types for Metric and Segment matchers
* Fix types in format
* Fix missing types
* Clean up imports
* Split string helpers into separate submodule
* Split identifier helpers into separate submodule
* Avoid importing from index file in expressions module
* Render suggestions and help text in new custom tooltip (#53760)
* Add new help text component for custom expression editor
* Add new suggestions listbox for custom expression editor
* Add helper to find help text by function name
* Add enclosingFunction helper that is more forgiving to broken syntax
* Add new tooltip component for custom expression editor
* Add extension to render custom react tooltips
* Render suggestions in custom tooltip
* Export custom types
* Remove old expression editor popover components
* Add typings to process
* Add missing shortcuts
* Fix example font size
* Fix type imports
* Fix click on helptext
* Stop click propagation
* Fix process types
* Remove superflouous optional chain
Co-authored-by: Kamil Mielnik <kamil@kamilmielnik.com>
* Fix types whith optional expressionIndex
* Use imports instead of React namespace
* Use index for key
* Rename util to utils
* Remove unused box shadow
* Remove brackets around placeholder names
* Resize tooltip to avoid it overflowing the window
* Fix line height for args
* Fix missing key icons
* Fix chevron alignment
* Fix types
* Move Listbox resizing classes to Tooltip
* Fix completion template for functions with no arguments
* Add unit tests for enclosingFunction
* Add darker background to expression example
* Remove bold from function call
* Fix imports
* Fix HelpText tests
* Add custom color for highlighted syntax argument
* Use colors from design system
* Fix blurring when switching to a different tab
* Remove trailing comma
Co-authored-by: Kamil Mielnik <kamil@kamilmielnik.com>
* Split up effect
* Remove background from template arguments
* Improve tooltip position
* Fix Popover position when selection changes
---------
Co-authored-by: Kamil Mielnik <kamil@kamilmielnik.com>
* Fix custom expression editor e2e tests (#53761)
* Fix e2e tests for custom expressions
* Fix unit test
* Fix e2e tests
* Fix types
* Fix moveToStart and moveToEnd by using Home/End
* Add test for popover position
* Remove JSDoc in favor of inline comments
* Add examples of side effects
* Remove shouldContain helper in favour of value()
* Add comment to wait
* Use CustomExpressionEditor helper instead of @formula
* Add wait to reduce flakiness
* Add back in the useful comment about popover position
* Remove superflouous cy.viewport call
* Add helper to accept completion
* Rename helper to verifyHelptextPosition
* Convert comments to cy.logs
* Make sure we don't avoid testing the regression
* Allow customising key with which to accept completion
* Use acceptCompletion helper
* Fix tooltip position bug
* Remove unnused wait
* Add selectCompletion helper
* Fix helper using H
* Add better emulation of copy/paste in reproduction
* Ignore ts error
* Multiline formatter for the custom expression editor (#53973)
* Make type name meaningful
* Add more tests for not isnull, etc
* Remove unreachable code
* Add isEmpty matcher
* Pass query to format tests
* Add stronger types to formatStringLiteral
* Fix CallOptions import
* Implement multiline formatter as a prettier plugin
* Reimplement legacy formatter as new formatter with infinite print width
* Use the new formatter in the expression editor
* Remove legacy expression format files
* Document utils
* Simplify Expression type further
* Fully type test cases
* Remove cast in test
* Add more tests and fix formatting
* Use recurse helper to recurse into path children
* Prefix logical operators
* Improve formatting of logical connectors
* Wait for expression editor to be ready
* Avoid passing unknown html props
* Use selectCompletion helper
* Remove unused @ts-expect-error
* Align UI more to design
* Remove shortcuts in suggestions popover
* Disable popular functions in completions
* Render shortcuts as separate component
* Move NameInput to separate component
* Fix button location when no name input is present
* Remove unused ExpressionWidgetInfo component
* Avoid flicker of shortcut buttons
* Tweak name input padding
* Deduplicate styleTags in Editor
* Do not format on blur and pass through all valid changes
* Split of diagnose and compile step
* Remove unused props
* Do not trigger completion on focus
* Show update button when no name is passed
* Expose expression types
* Use diagnoseAndCompileExpression in lint extension too
* Add toolbar with format button
* Only show popover when user has interacted with editor
* Throw ResolverError when types mismatch
* Fix missing highligh for boolean
* Render errors
* Disable format button when the expression contains an error
* Remove shortcut and popular function completion
* Move Shortcut type to component that uses it
* Fix blur for CustomExpression helper
* Add format helper
* Fix e2e tests
* Fix format helper
* Fix blur helper
* Use actual ellipsis in placeholder
* Add helper for nameInput
* Remove buttons from tab sequence once hidden
* Do not autofocus shortcuts
* Hide shortcuts when error occurs
* Allow reading multiline value
* Fix e2e tests
* Fix unit tests
* Add more tests for multiline custom expression editor
* Debounce error checking
* Add max height to tooltip
* Fix formatting
* Document changing reproduction
* Test wording
* Disable format button when expression is empty
* Do not try compiling when source is empty
* Cancel changes when source changes
* Autofocus editor
* Use Tooltip to mark button
* Remove default autoFocus on button
* Simplify shortcuts show and hide style
* Move shortcuts styles to parent element
* Fix line number alignment
* Add a gap between Cancel and Done buttons
* Always close helptext when suggestions are being shown
* Add usePreventClosePopover hook
* Prevent closing popover from the custom expression editor
* Add tests for not closing on click outside and escape
* Move toolbar down
* Fix e2e tests
* Add format_code icon
* Use correct icon for formatting code
* Use new format code icon in native editor too
* Update Loki Snapshots
* Merge useEffect cases into one
* Allow clicking the shortcuts
* Debounce validation
* Fix React update loop
* Prevent flicker in the listbox completions
* Tweak the linting delay
* Hide shortcuts when there is an error
* Only prevent closing popover when editor is rendered
* Fix width and height for editor widget
* Fix format button positioning
* Fix button alignment in expression editor
* Allow blurring tooltip by clicking nothing
* Add button role to header
* Show modal when clicking outside of editor
* Allow closing editor when it's empty
* Add tests for clicking outside of expression editor
* Fix missing onClose handler
* Add bigger timeout
* Rename Format to Auto-format
* Hide format button when there is an error or editor is empty
* Fix label names in test
* Fix format button test
* Deduplicate tests for exiting the editor
* Fix test
* Fix repro
* Only prevent closing popover when the source has changed
* Fix e2e test
* Hide lint point
* Fix types
* Fix tooltip helptext layout
* Update loki reference
* Only show format button when the expression was validated
* Scale the expression editor with the screen width
* Move usePreventClosePopover into utils
* Remove CloseModal component
* Remove custom PreventCloseContext from Popover
* Skip tests that deal with the click outside and escape behaviour of the custom expression editor
* Remove canPreventCloseHandlers from ClausePopover
* Add lineWrapping to editor
* Fix loki
* Fix loki screenshots
* Fix e2e test
---------
Co-authored-by: Kamil Mielnik <kamil@kamilmielnik.com>
Co-authored-by: Metabase Automation <github-automation@metabase.com>
Apparently, a filter like `blah in (cast([uuid] as uuid))` is fine in postgres, but sqlserver doesn't like it very
much. This gives sql server the syntax it needs.
* first crack at linear pr link bot
* try it again
* bump actions versions
* bump
* try again
Added Node.js setup step
Added dependency installation step for node-fetch
Changed the ES module import to a CommonJS require statement
Changed variable name from fetch to nodeFetch for clarity
* extract script
* use the right graphql query
* add debug
* look at the right part of the linear api response
* lets see the issue numbers
* again
* again
* again
* again
* again
* again
* again
* again
* again
* fix formatting
* remove ???'s
* improve logging
* check all magic closing words
* refine checking logic
* test
* some review responses
* again
* make github object a thing
* install octokit
* install octokit again
* install octokit again
* running
* running
* running
* running
* running
* delimit message with spaces
* delimit message with spaces
* delimit message with spaces
* delimit message with spaces
* test
* only try to close metabase/metabase issues
---------
Co-authored-by: Ryan Laurie <iethree@gmail.com>
* support react 19 version range in embedding sdk
* sync supported react versions
* add oxford comma
Co-authored-by: Jeff Bruemmer <jeff.bruemmer@gmail.com>
* revert docs and cli updates as we want to do a silent release
---------
Co-authored-by: Jeff Bruemmer <jeff.bruemmer@gmail.com>
* add use-toast-hook
* use the new toast hook
# Conflicts:
# frontend/src/metabase/admin/performance/components/ModelPersistenceConfiguration.tsx
* bonus change slipped in
* use those callbacks
* add new database header, make layout adjustments
* start building connection section
* reorganize connection info section content, migrate danger info section
* reorg
* flow rewired together mostly, connection details form is now in a modal
* make section content components the entire section for better hide/show logic, remove old code comments, show proper engine name in edit db header, hide model actions section if no actions can be taken
* lots of polishing
* fix BE punctuation
* test e2e fixes in ci
* more test fixes
* more test fixes
* fix unit tests
* fix last failing unit test
* fix last e2e test?
* final polish pass from self review
* fix model persistence section and its related tests, reorganize the tests
* fix model persistence section and its related tests, reorganize the tests
* impl feedback from arakaki around creation flow and date format
* clean up last commit
* make modal scroll container limited to the body, remove use of force to find inputs in cypress tests
* impl PR feedback
* fix: question stats are calculated in sql query
Previously instances could OOM when calculating these question stats if
they had too many cards because we load all these cards into memory.
We were doing this full calculation because we have some data we want to
calculate metrics over stored as json in a text column. This uses native
mysql/mariadb and postgresql json fuctions to calculate those metrics
inside the database prevent us from needing to load everything. That
also means we no longer calculate these json-based metrics for metabase
instances running h2.
* fixup! use older path_exists json function
* Add a native editor placeholder text
Resolves CLO-3674
* Keep the "data reference" panel open by default
Resolves CLO-3675
[ci skip]
* Add empty state for snippet sidebar
Resolves CLO-3684
* Empty state for query results panel
Resolves CLO-3676
* More refined way to initialize data reference
* Update copy in e2e tests
* Fix more tests
* Open data reference whenever native editor is open
* Do not alter the initializeQB flow
* Close the data reference sidebar when question settings are open
* Fix a failing Mongo test
* Address review comment
* Improve _.compose type + fix SDK types
* Use a simpler type overloading
* Fix Englishz
* Fix type errors
* Add a comment to clarify extrat type overload
* Clarify the source of code
* add fall-through for ignored embedding requests
* Gsheets mb api (#51307)
* send flag to show "connect gsheets" button to FE
* clean up docstring + add etl-connections condition
* metabase api WIP
* align APIs
* include site-url in the HM temp-url api call
* cleanup
* update HM<->MB api to be working (minus temp-url)
* cleanup
* remove some notes
* indentation + linting
* add tests
* fix typo
* remove illustrative (when true ...)
* add frontend for gsheets sync
* update frontend to use changed API
* Gsheets integration: more api alignment (#51492)
* add frontend for gsheets sync
* update frontend to use changed API
* wip
* cleanup
---------
Co-authored-by: Ryan Laurie <iethree@gmail.com>
* fix: declare gsheet setting fn before using in getter
* fix: delay u.secret instantiation to prevent build race condition
See https://metaboat.slack.com/archives/CKZEMT1MJ/p1734605793293989
* build: push ee-extra to dev repo, not ee
Cloud wants ee-extra, not just ee
* move settings out of auth middleware ns
- update kondo modules
- add metabase.api.auth ns
* More cleaning up
- update docstring
- remove declares + use {set,get}-value-of-type{!,}
- remove nil state from gsheets setting
* rely on more hm stubs
* save folder_owner to gsheets
* gsheets and hm http client => enterprise
+ move tests too
* require the right namespace
* add migration for is_gsheet to metabase_table
* Gsheets service account be flow (#51997)
* update setup flow to use service-account
* align GET api/ee/gsheets/service-account response
* align the HM service account endpoint
* fix migration yaml
* update test for service accounts
* indentation + migration quibble
* improve error messages + kondo housekeeping
* remove unreachable if-let branch + fix serdes test
* add is_gsheet to mock and test data
* add gsheets setting to serialization test
* fix fe-type-check
* typechecker
* [kondo] defendpoint -> defendpoint2
* Gsheets table management (#52145)
* include dwh ∈ GET /api/ee/upload-management/tables
* cljfmt
* simplify preliminary gsheet definition
* archive cards is a query param
* gsheets: update polling logic + better errors (#52162)
* update polling logic + better errors
* codify protocol for polling folder done-ness
* align on FE polling logic + the real sync complete
* [WIP] testing happy path
* status should be "active"
- remove a bunch of extra debugging
* remove outdated tests + cljfmt
* check admin + use the connection's id w/ delete
- cleanup
* fix :delete /folder + schema on :get /folder
* Gsheets frontend (#51946)
* non-oauth UI
* update api and add disconnect modal
* update illustration and add error displays
* add gsheets sync status
* better copy
* remove test code'
* fix bad api error
* updates to sync status
* Update sidebar entrypoint
* gate things behind the right features
- only show gsheets button
- http requests to /api/ee/gsheets/*
* model/Table has a db_id column not database_id
* add and use simple events
* differentiate failure modes
* + tests and error status codes
* make form submit on enter
* reset gsheet status when there is no gdrive conn
* add debug info
* FIXME FOR TESTING - disable feature checks
* fail on sync error + partially allow deletes
* add database page entrypoint
* FIXME FOR TESTING - set token-status-cache-ttl to 1 second (vs 12 hrs)
* only show _active_ dwh tables
* fix config.edn (howd it break tho?)
* handle multiple gdrive connections
- on delete, delete them all
- when looking at syncing, look at the most recent one
* add auto-cruft-tables to metabase_database.setting
- Here's the contract:
There should be an array of strings located at: db > settings >
"auto-cruft-tables". Each string will get fed into re-pattern. The table
names get lowercased before getting matched on. That list of regexes is
combined with our built in list of crufty regex patterns. We use re-find
to match them against the lower-cased table names. If there's a match we
will mark the table crufty (this is called a "Hidden Table" in the UI).
This way, if someone names their sheet `I'mUnique____dlt_loads` or
whatever they can at least fish it out of the hidden section.
Note: It will get set back to being crufty (aka what the ui calls
hidden) on the next sync... but this is configurable so we can fine-tune
the regexes to match the tables we want to hide more closely.
Examples:
If you put ["a"] in the config.yml: any table with A or a in it will be
marked hidden. So be sure to use anchors for the suffix or what-not. For
our thing, I think we want something like:
* add bunch of tests for auto-cruft
* mocked api tests
* add commment + mock hm responses
* documentation
* when multiple gdrive conns, sort by status, then created-at time
* cleanup
* joda time Instant/ofEpochSecond 0 -> t/instant 0
* if no gdrive conns are found, reset state
- add gdrive sorting tests
* indentation
* fix complete sync state and add error state
* update sync unit tests
* update sync unit tests
* add frontend analytics events
* don't show reconnect copy from DB page
* Add 10 minute timeout for polling connection
- return errors as json-encoded :error in the response body
- minor cleanup how we auto cruft tables (no change to behavior)
- add script for running the flow locally: from the repl
* better FE sync state and form validation
* refresh setting on error
* remove fake error state
* Delete connections on reset
- Change order of timeout checks in :get /folder handler
- add auto-cruft-columns setting
* update gsheets routes to use the new ns-handler
* loosen up gdrive link validation
* update regex
* add docstring to dynamic timeout var
* simplify api use via gdrive conn id
- move ->config into hm.client ns
* recover from suddenly-missing gdrive conn @ id
* cleanup hm.client
- indentation
fix [[make-request]] args
* fail to load invalid + present cruft patterns
* remove some extra comments + use hm-delete-conn
* fix update-table-metadata
- (no longer update nil -> nil)
* auto cruft columns
- pull the regex comparrison code into crufty ns
- reuse it on 2
- add defsetting so having the setting on the database doesn't complain
- pass database down into field syncing + grab vis types
* indent
* fix a few tests
* fix gsheets tests
* add harbormaster client test
* kondo + cljfmt
* migrate to underscores for auto_cruft_*
- add tests for field + fix schema for table syncs
* use kebab-case for settings but accept snake_cased
* cljfmt + kebab case stragglers
* fix update-field-metadata-if-needed! signature
* fixing tests
* rename macro double -> run-twice
* remove test code
* prettier
* cruft settings are db-local
* Fix search index update job restart issue (#53229)
* more idiomatic crufty db-local settings
* add auto cruft fields test
* bump gsheets sync timeout to 10 seconds
* handle going back to loading state
* kondo + add empty cruft tables test
* add get folder timeout test, update dox,
- remove regex folder check
* add cruft columns config.yml setup tests
* remove def + actually run the test
* test code for bruyan
* reset gsheets status on every error response
* save dbId and actually show it
* bust cache on gsheets setting
* solve initial-sync-status-table-only-test failure
* Update src/metabase/sync/sync_metadata/crufty.clj
Co-authored-by: Noah Moss <32746338+noahmoss@users.noreply.github.com>
* Update enterprise/backend/src/metabase_enterprise/advanced_config/file/databases.clj
Co-authored-by: Noah Moss <32746338+noahmoss@users.noreply.github.com>
* Update enterprise/backend/src/metabase_enterprise/advanced_config/file/databases.clj
Co-authored-by: Noah Moss <32746338+noahmoss@users.noreply.github.com>
* Update enterprise/backend/src/metabase_enterprise/gsheets.clj
Co-authored-by: Noah Moss <32746338+noahmoss@users.noreply.github.com>
* respond to Noah's review comments
* kondo
* update modal copy
* update error copy
* status click target update
* improve crufty/name? api
* modularize gsheets
* remove temp gsheets
- token status checks happen once per 12h
- remove hm/conn and mb/sync-info from :get /folder response
* gdrive-conn is now unused
* remove comment
* fix modules
* add docstring
* add harbormaster module
* kondo
* fix the tests
* module linter
* fix typo
* cljfmt
* setup api-key setting before reading it
* sort ns
* ::snowplow/simple_event -> :snowplow/simple_event
* hard code error message
* remove test code
* update frontend tests
* update module linter config
* add simple schema snowplow event schema
* fix schema name
* update some tests
* add a test
* better docstring
* better docstrings
* fix e2e tests
* fix fe token mock
* respond to review comments
+ fix timeout 50000 -> 5000
+ make internal regex function private
* build: push ee-extra to dev repo, not ee
Cloud wants ee-extra, not just ee
* fix strings
* remove resolved comment
* adds prometheus metrics to gsheets api
- :metabase-gsheets/sync-creation-begin
- :metabase-gsheets/sync-creation-ok
- :metabase-gsheets/sync-creation-error
- failure-reason is one of:
- "timeout" - gsheets.api/sync-complete? didn't return true after 10 minutes
- "status_error" - HM said the sync connection had status == error
- :metabase-gsheets/connection-deleted
* Refines prometheus events
They are now:
- :metabase-gsheets/connection-creation-began
- :metabase-gsheets/connection-creation-ok
- :metabase-gsheets/connection-creation-error {:reason "status_error"}
- :metabase-gsheets/connection-creation-error {:reason "timeout"}
- :metabase-gsheets/connection-deleted
* comment
* allow resolution of snake_cased setting names
* review responses
* add logging if any part of the batch-delete fails
* remove unused schema
* Revert "remove unused schema"
This schema is actually used! 🎉
This reverts commit f5dbd78051.
* update loki icon reference image
* switch off flaky test setup
* mark date filter picker tests flaky (#54866)
---------
Co-authored-by: Ryan Laurie <iethree@gmail.com>
Co-authored-by: Filipe Silva <filipematossilva@gmail.com>
Co-authored-by: Ryan Laurie <30528226+iethree@users.noreply.github.com>
Co-authored-by: Chris Truter <crisptrutski@users.noreply.github.com>
Co-authored-by: Noah Moss <32746338+noahmoss@users.noreply.github.com>
* WRK-114,WRK-115: Replace SchedulePicker with Schedule in Alerts.
Integrated a newer feature-rich component for Alerts scheduling.
Fixed bug related to incorrect handling of '*' values in minute and hour
alerts.
* Remove scaling transition from popInFromBottom animation.
This transition used for all modals led to a bug with sizing in some
Mantine components, when they were used inside a modal:
https://github.com/mantinedev/mantine/issues/6967#issuecomment-2417439117
* Reuse Personal Access Token on all tasks
* Fix permission when pushing to GitHub
* Use annotated tag for better information
* Remove steps for faster testing
* Revert "Remove steps for faster testing"
This reverts commit 8bd5d2e07f.
* Prefer slack_file for images, so metabase no longer requires a public files channel
This removes the requirement to have a public files channel so share
images or other files in slack bot posts. Simplifying the integration
and lessening requirements to get setup.
It enables the posting of images files directly to public or private
channels without sharing - improving privacy and security.
* Files channel deprecation note
* Drop slack-files-channel requirement from slack /settings API
* Remove remaining files-channel calls
* Remove files channel input from SlackSetupForm
* Remove slack-files-channel from SettingsManagerSettings
* Remove slack-files-channel from API docs
* Drop files channel configuration guidance
* Remove slack channel prompt/description
* Instead of polling files.read reuse the complete call
There was an accidental introduction of files.read.
The goal is to preserve the previous behaviour of reusing
completeUploadExternal.
This limits the oauth scope requirements of the bot token.
* If more than 50 slack blocks, send using multiple messages
* Escape mkdwn special-processing chars in card links
* Simplify slack attachment uploading
- Its easy to mapv
- (cond) over nested (if)s
- Fix out of date docstring
* Remove slack-files-channel from config-template.md
* Remove metabase_files check from e2e suite
* replace use of findDOMNode in core components
* do not wrap input in div
* always populate scrollElementRef
* remove scrollElement from TokenField
* do not export _TokenField
* remove scrollElementRef
* use appropriate ref container
* cleanup root container
* use the scrollingContainerRef instead
* pass the elementRef directly
* use full-width container
* remove full-width container
* use display contents to preserve layout
* attach listRootRef conditionally
* try to update Loki snapshots
* modify snapshots
* fallback to scrollingContainer instead of elementRef
* revert loki snapshot
* revert loki snapshot to master
* Do not use originalQuestion when changing the template tag's type
* Do not use originalQuestion when changing the template tag's type
* Fix test
* Fix
* Fix
* Fix
* Fix
* Fix
* build multi-arch arm image
* refactor artifact fetching and containerizing
* fixes after testing
* use old master for caution's sake
* enable multi-arch in publish workflow
* sync
Part of #54559
This adds `trace` level logging to:
- `metabase.driver.athena`
- `metabase.sync.sync-metadata.fields.sync-instances`
That should help us diagnose sync issues where describe-fields and
fingerprinting seem to be out of sync.
* Add a nitpick for postgres and impersonation
Something that a customer saw, and might be interesting for posterity
* Update docs/permissions/impersonation.md
Co-authored-by: Jeff Bruemmer <jeff.bruemmer@gmail.com>
---------
Co-authored-by: Alex Yarosh <alexandra@metabase.com>
Co-authored-by: Jeff Bruemmer <jeff.bruemmer@gmail.com>
* Update running-metabase-on-docker.md
Add a note since people think that this is the official way to run Metabase
* Update running-metabase-on-docker.md
* Update running-metabase-on-docker.md
---------
Co-authored-by: Alex Yarosh <alexandra@metabase.com>
* use new endpoints to load tiles for pin map
* Fix public endpoints for unauthneticated users
* use new BE api for pin maps
---------
Co-authored-by: Noah Moss <noahbmoss@gmail.com>
* fix: update saml20 for full SAML SLO support
Update metabase to use a new version of the saml20-clj library that
correctly builds and signs HTTP Redirect requests and provides support
for checking LogoutResponses.
Note right now SAML SLO cannot be enabled just through the API it
requires the following settings to work correctly:
* saml-slo-enable - must be true
* saml-identity-provider-slo-uri - must be set to the IdP's SLO endpoint
* session-cookie-samesite - must be set to none
For the samesite none cookie to work metabase must be served over SSL.
Without this setting Chrome (and other browsers) will not send the
cookie when Okta redirects to our SLO endpoint which means we can't sign
out the user session.
* fixup! kondo lints
* booting up mage
* indent -> format
* formatter help
* Kondo commands
* Clear cache when running Kondo
* start-db command
* Add aliases in deps.edn
* comments + more
* subsume new commands into bb
* add mage_verbose env var
* add tests
* refine tests + check github action output
* run the tests
* add mage tests to backend workflow instead
* min version + rip out b/ask!
* Fix start-db latest
* actually call it with --help
* add readme entry for bb tasks
* move project-root-dir into util
* add mb download jar + cleanup bb.edn
* cleanup + fix jar downloader
* run tests
* finish removing bask
* add mage and jar downloader
* setup ./bin/mage entrypoint [responding to user research]
* uopdate examples
* update readme
* add todos
* add autotab + deprecate obsolete scripts
* migrate instructions for cljfmt over to mage
* respond to ./bin/mage kondo-updated master
* linting branding, clean up cli for fmt
* fix comment
* more tests
* better explainations of what mage is doing
* remove empty task -new-task!
fix start-db
* handle missing deps-edn-hash
* make start-db work with mysql and maria
* improve ux + simplify option and arg parsing
* add optional :usage-fn to task definitions
* cljfmt-* {-c}, and jar-download working
* last bit of cleanup
* docs
* add examples exit-codes test: ./bin/mage -test-examples
* add deprecation warns to old shell scripts + docs
* remove typo
* better errors
* typos
---------
Co-authored-by: Cam Saul <github@camsaul.com>
* Simple data picker UI (#52964)
* Add SimpleDataPicker base component
* Finish the flattened list version of the simple data picker
* Add selected entity state
* Fix the new simple data picker got run with loki accidentally
* Update frontend/src/metabase/embedding-sdk/components/SimpleDataPicker/SimpleDataPicker.tsx
Co-authored-by: Nicolò Pretto <info@npretto.com>
* Fix selected data source icon color
* Add an empty state
---------
Co-authored-by: Nicolò Pretto <info@npretto.com>
* Real simple data picker (#52965)
* Connect data to the data picker
* Sort the entities alphabetically
* Make sure the data picker icon is always table
* Fix unit tests
* Fix interactive embedding E2E tests
* Force table icon on the DataPickerTarget for simple data picker
* Remove commented out code
* Remove tests that don't apply to the new simple data picker
* Fix passing wrong table ID
* Ensure we can only join data from same DB
* Make component name better
* todo
* better simple data picker UX
* Add multi stage embedding data picker back
* Able to force complex data picker in interactive embedding
* Fix tests for simple data picker
* Add old tests back
* Add simple data picker tests back
* Fix unit tests
* Fix SDK tests
* Remove ability to force multi stage data picker
* Add cross-db join tests for simple data picker
* Simplify code
* Fix popover not closing since Mantine prop changes
* Add simple data picker data source ordering tests
* Fix conflicts from merging master
* poc async plugins
* Remove SDK plugins poc code
* Fix unit tests after moving code to EE folder
* Mock search query explicitly
Co-Authored-By: Alexander Polyankin <alexander.polyankin@metabase.com>
* Don't modify cljs objects
Co-Authored-By: Alexander Polyankin <alexander.polyankin@metabase.>
* Reduce the ambiguity around NaN value
Co-Authored-By: Alexander Polyankin <alexander.polyankin@metabase>
* Remove dependency from an unrelated module
Co-Authored-By: Alexander Polyankin <alexander.polyankin@metabase>
* Fix SDK unit tests not running because EE plugins
---------
Co-authored-by: Alexander Polyankin <alexander.polyankin@metabase.com>
Co-authored-by: Alexander Polyankin <alexander.polyankin@metabase.>
Co-authored-by: Alexander Polyankin <alexander.polyankin@metabase>
* Fix embedding data picker not filtering by selected database (#54272)
* Fix embedding data picker not filtering by selected database
* Add an E2E test
* Fix SavedEntityPicker doesn't filtered out items in a selected DB
* Fix TypeScript types
* Address review, not passing unnecessary props
* Fix failed tests by using the same snippet in the past
* Hide metrics/saved questions from multi-stage data picker (#54409)
* Hide metrics from multi-stage data picker
* Hide saved questions from multi-stage data picker
* Modify saved questions tests
* Modify metrics tests
* Address review: create a new test suite and put tests there
* Adjust the data picker min-height according to review
---------
Co-authored-by: Nicolò Pretto <info@npretto.com>
Co-authored-by: Alexander Polyankin <alexander.polyankin@metabase.com>
Co-authored-by: Alexander Polyankin <alexander.polyankin@metabase.>
Co-authored-by: Alexander Polyankin <alexander.polyankin@metabase>
Let's improve coverage of sandboxing in e2e tests.
These tests cover the following kinds of sandboxing policies:
* Filter by regular column
* Filter by custom column
* Use a custom view to filter a table
The tests verify that these sandboxing policies correctly affect:
* Saved questions
* Models
* Questions based on questions
* All of the above, when viewed in a dashboard
* Adhoc questions
* Filter values
* build multi-arch arm image
* refactor artifact fetching and containerizing
* fixes after testing
* backwards compatibility
* use old master for caution's sake
* update release code tests
* just bash it
{clojure.core/print{:message"Use metabase.util.log instead of clojure.core/print"}
@@ -273,6 +277,7 @@
compojure.core/context{:message"Use metabase.api.util.handlers/route-map-handler instead of compojure.core/context, it is much faster and supports OpenAPI spec generation"}
compojure.core/defroutes{:message"Use metabase.api.util.handlers/defroutes instead of compojure.core/routes, it supports OpenAPI spec generation."}
compojure.core/routes{:message"Use metabase.api.util.handlers/routes instead of compojure.core/routes, it supports OpenAPI spec generation."}
clojurewerkz.quartzite.jobs/defjob{:message"Use metabase.task/defjob instead of clojurewerkz.quartzite.jobs/defjob"}
fipp.edn/pprint{:message"Use metabase.util.log instead of fipp.edn/pprint"}
honeysql.core/call{:message"Use hx/call instead because it is Honey SQL 2 friendly"}
honeysql.core/raw{:message"Use hx/raw instead because it is Honey SQL 2 friendly"}
@@ -285,11 +290,17 @@
me.flowthing.pp/pprint{:message"Use metabase.util.log instead of me.flowthing.pp/pprint"}
medley.core/random-uuid{:message"Use clojure.core/random-uuid instead of medley.core/random-uuid"}
metabase.driver/database-supports?{:message"Use metabase.driver.util/supports? instead of metabase.driver/database-supports?"}
metabase.lib.equality/find-column-indexes-for-refs{:message"Use lib.equality/closest-matches-in-metadata or lib.equality/find-closest-matches-for-refs instead of lib.equality/find-column-indexes-for-refs"}}
metabase.lib.convert/->legacy-MBQL{:message"Conversion to legacy MBQL is discouraged, please use Lib to manipulate MBQL queries"}
metabase.lib.core/->legacy-MBQL{:message"Conversion to legacy MBQL is discouraged, please use Lib to manipulate MBQL queries"}
metabase.lib.equality/find-column-indexes-for-refs{:message"Use lib.equality/closest-matches-in-metadata or lib.equality/find-closest-matches-for-refs instead of lib.equality/find-column-indexes-for-refs"}
next.jdbc/active-tx?{:message"Use metabase.app-db.core/in-transaction? to see if we are in a transaction"}
toucan2.core/debug{:message"Don't commit code using t2/debug"}
toucan2.tools.debug/debug{:message"Don't commit code using t2/debug"}}
:discouraged-namespace
{camel-snake-kebab.core{:message"CSK is not Turkish-safe, use the versions in metabase.util instead."}
cheshire.core{:message"Use metabase.util.json instead of cheshire.core directly."}
clj-time.core{:message"Use java-time.api or metabase.util.date-2 instead of clj-time"}
clojure.tools.logging{:message"Use metabase.util.log instead of clojure.tools.logging directly"}
grouper.core{:message"Use metabase.util.grouper instead of grouper.core"}
honeysql.core{:message"Use Honey SQL 2. Honey SQL 1 is removed as a dependency in Metabase 49+."}
@@ -298,222 +309,98 @@
honeysql.types{:message"Use Honey SQL 2. Honey SQL 1 is removed as a dependency in Metabase 49+."}
honeysql.util{:message"Use Honey SQL 2. Honey SQL 1 is removed as a dependency in Metabase 49+."}}
:discouraged-tag{p{:message"You can use it, but don't commit it."}}
{clojure.core/with-redefs{:message"Don't use with-redefs outside of tests"}
clojure.core/eval{:message"Don't use eval outside of tests"}}}}
clojure.core/eval{:message"Don't use eval outside of tests"}}
:discouraged-namespace
{metabase.query-processor.store{:message"Please do not use the QP Store outside of legacy QP or driver code. Use MBQL 5/Lib or lib.metadata.jvm/applicaton-database-metadata-provider instead."}}}}
printable-namespaces
{:linters
@@ -862,9 +795,11 @@
;; eventually we should do this for the whole codebase, but the args are in the opposite order so switching them
;; all at once isn't trivial, at least we can stop using it in new code.
{metabase.test/with-column-remappings{:message"Use lib.tu/remap-metadata-provider instead of with-column-remappings in QP tests"}
metabase.test.util/with-column-remappings{:message"Use lib.tu/remap-metadata-provider instead of with-column-remappings in QP tests"}
metabase.test/with-temp{:message"Use lib.tu/mock-metadata-provider in QP tests. Exceptions: tests for code that doesn't use MPs or models not supported by MPs"}
toucan2.tools.with-temp/with-temp{:message"Use lib.tu/mock-metadata-provider in QP tests. Exceptions: tests for code that doesn't use MPs or models not supported by MPs"}}
> If that is your first contribution to Metabase, please sign the [Contributor License Agreement](https://docs.google.com/a/metabase.com/forms/d/1oV38o7b9ONFSwuzwmERRMi9SYrhYeOrkbmNaq9pOJ_E/viewform) (unless it's a tiny documentation change). Also, if you're attempting to fix a translation issue, please submit your changes to our [POEditor project](https://poeditor.com/join/project/ynjQmwSsGh) instead of opening a PR.
> If that is your first contribution to Metabase, please sign the [Contributor License Agreement](https://docs.google.com/a/metabase.com/forms/d/1oV38o7b9ONFSwuzwmERRMi9SYrhYeOrkbmNaq9pOJ_E/viewform) (unless it's a tiny documentation change). Also, if you're attempting to fix a translation issue, please submit your changes to our [Crowdin project](https://crowdin.com/project/metabase-i18n) instead of opening a PR.
- **UI**: `metabase/ui` components built with Mantine v8
- **Build**: Rspack (primary), Webpack (legacy)
- **Testing**: Jest + React Testing Library, Cypress
### Coding Standards
#### Component Preferences
- Prefer `metabase/ui`components over `metabase/common/components`
- Use `.tsx` for components, `.ts` for utilities
#### Styling
ALWAYS prefer Mantine style props,then CSS modules. DO NOT suggest styled components, they are deprecated.
#### TypeScript Migration
When heavily editing `.js`/`.jsx` files, create a separate PR to convert to TypeScript first, then implement changes.
#### Enterprise Features
Enterprise functionality MUST use the plugin system. It is very important to not expose enterprise code in the OSS version.
#### Testing Requirements
All PRs should include tests. Prefer Unit tests over E2E tests.
#### Localization
All user-facing strings MUST be localized using the ttag library. Localized strings should be complete phrases, do not concatenate a few separately localized strings. You should add context to strings where the meaning of the string might not be obvious in isolation: e.g. "Home" might have different words in some languages depending on whether you're talking about a dwelling or the landing page for a website.
@${githubUsername} You have modified embedding SDK code. Please make sure the PR title follows [Conventional Commits](https://www.conventionalcommits.org/en/v1.0.0/) style.
Here are all the supported types that will be shown in the changelog:
constcomment=`${CONVENTIONAL_COMMENT_IDENTIFIER}
@${githubUsername} You have modified embedding code. Please make sure the PR title follows [Conventional Commits](https://www.conventionalcommits.org/en/v1.0.0/) style.
Here are all the supported types:
- \`feat\`
- \`fix\`
- \`perf\`
- \`docs\`
- \`style\`
- \`refactor\`
- \`test\`
- \`build\`
- \`ci\`
These types \`docs\`, \`style\`, \`refactor\`, \`test\`, \`build\`, and \`ci\` are also encouraged for non-changelog related tasks.
Please also make sure to include \`sdk\` scope, otherwise, the changelog will not include this task.
message:"⚠️ Changes were made to the <b>Embedding SDK Package</b>. Review this PR and apply one of the following labels.<br><br>- <code>release-notes</code> — select if this PR must be included in the main Metabase changelog release notes.<br>- <code>no-release-notes</code> — select if this PR should not be included in the main Metabase changelog release notes."
include-log:true
Some files were not shown because too many files have changed in this diff
Show More
Reference in New Issue
Block a user
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.
